# Forma·Ka
### Every training program in Québec. One place.
**[www.forma-ka.com](https://www.forma-ka.com)**
*Independent aggregator of training programs — online courses, university and
college courses, seminars, workshops, certifications and bootcamps — every
program with its full standardized details and a direct link to the original
page. Always up to date, automatically.*
**Author : Simon-Pierre Boucher — [contact@spboucher.ai](mailto:contact@spboucher.ai)**
---
## Screenshots
| Home — search, filters, live ticker | Program page — details first |
|---|---|
|  |  |
Stats page — live portrait of Québec's training offer
## Why Forma-Ka?
Looking for training in Québec means opening dozens of websites — universities,
CEGEPs, training firms, event organizers — each with its own navigation and
format. **Forma-Ka flips the problem**: a dedicated connector per institution
visits each site, normalizes every program into a single schema, and detects
changes continuously.
> Training sites don't offer webhooks. Forma-Ka reproduces the equivalent:
> **periodic sync + content hashing** → additions, updates and removals detected
> automatically. A program that disappears from the source site disappears from
> Forma-Ka (after a 2-sync grace period).
**Philosophy**: unlike a product aggregator, **price is optional** (university
courses don't display one) — what matters are the **details** of each program:
full description, learning objectives, course outline, prerequisites, target
audience, duration, credits/CEUs, delivery mode, offered dates.
## Architecture in 30 seconds
```mermaid
flowchart LR
subgraph Sources["18 training institutions"]
S1["ÉTS Formation · TÉLUQ · ULaval
McGill · HEC · UQAM · Technologia
AFI · Cégep à distance · Les Affaires
… one connector per site"]
end
subgraph FormaKa["Forma-Ka"]
C["Connectors
1 adapter / site"] --> N["Normalization
single Formation schema"]
N --> D[("SQLite
hash + diff")]
D --> A["FastAPI
/api/formations · /api/facets"]
A --> F["React 18 + Vite
mobile PWA · light theme"]
end
W["⏱ Periodic watcher
(PM2)"] -.-> C
S1 --> C
F --> U["🔑 Learner"]
```
| Layer | Role | Files |
|---|---|---|
| **Connectors** | 1 Python module per institution: server-rendered HTML, internal JSON APIs (Shopify products.json, WordPress REST, Gatsby page-data, Destiny One…), schema.org JSON-LD (Course/Event), **Scrapfly** (robust anti-bot) or **Firecrawl** (JS rendering) for hard sites | `formaka/connectors/*.py` |
| **Schema** | Standardized `Formation`: type, category, mode, description, objectives, outline, prerequisites, duration, credits/CEUs, sessions, price *(optional)* | `formaka/schema.py` |
| **Diff engine** | Content-hash upsert — new / changed / gone (grace period), connector drift detection | `formaka/db.py` |
| **API** | Filters: type / category / mode / city / language / level / free / price / source / search, facets, stats, sync trigger | `formaka/web.py` |
| **Frontend** | "Editorial sharp" design: Space Grotesk, offset shadows, amber accent, live ticker, mobile bottom sheet, 12-per-page pagination, installable PWA | `frontend/` |
## The three fetch backends
Each connector picks one (or chains them via the automatic `fetch_html()` fallback):
1. **direct requests** — server-rendered sites (fast, free);
2. **Scrapfly** (`SCRAPFLY_API_KEY`) — robust backend: anti-bot bypass (`asp`),
JavaScript rendering (`render_js`), Canadian geolocation;
3. **Firecrawl** (`FIRECRAWL_API_KEY`) — fallback JS rendering, html/markdown formats.
Detail pages are cached in the database (`detail_cache`) with a weekly key:
each page is revisited only when new, changed, or when the ISO week rolls over.
## Aggregated sources
| Institution | Offer | Programs |
|---|---|---|
| Université Laval — Distance | University courses (distance/hybrid) | ~1,760 |
| Technologia | Professional training — IT, AI, management | ~590 |
| Université TÉLUQ | University courses, 100 % online | ~510 |
| AFI by Edgenda | Professional training — IT, leadership | ~360 |
| ÉTS Formation | Continuing education + CEUs | ~300 |
| Versalys | Office tools, IT, languages | ~230 |
| McGill School of Continuing Studies | Continuing education (English) | ~200 |
| Isarta Formations | Marketing, communications, HR | ~190 |
| CRHA — Espace Formation | HR training and events | ~175 |
| Événements Les Affaires | Business conferences and webinars | ~135 |
| Cégep à distance | College courses at a distance | ~120 |
| HEC Montréal — École des dirigeant(e)s | Executive seminars and certifications | ~90 |
| ITHQ | Wine/food workshops + hospitality training | ~80 |
| École des entrepreneurs du Québec | Entrepreneur training (mostly free) | ~45 |
| Institut de leadership | Leadership certifications and programs | ~35 |
| AlphaNumérique | Free digital-literacy courses | ~30 |
| Formation continue UQAM | Continuing education + CEUs | ~50 |
| Le Wagon Montréal | Web dev / data / AI bootcamps | 4 |
## Quick start
```bash
git clone https://git.spboucher.ai/forma-ka.git && cd forma-ka
# Backend
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
# Frontend
cd frontend && npm install && npm run build && cd ..
# Scraping backend keys (JavaScript / anti-bot sites)
cat > .env < http://localhost:8080
.venv/bin/python run.py watch 360 # sync loop (default: every 6 h)
```
## Adding a connector
1. Create `formaka/connectors/.py`: a class inheriting from
`BaseConnector`, define `source_id` and implement `fetch() -> list[Formation]`.
The registry is **auto-discovering** — nothing else to edit.
2. Add the matching entry to `data/sources.json`.
3. Test: `.venv/bin/python run.py sync `.
```python
class MySchoolConnector(BaseConnector):
source_id = "my_school"
def fetch(self) -> list[Formation]:
html = self.fetch_html(LIST_URL) # direct -> Scrapfly -> Firecrawl
...
return [Formation(source=self.source_id, external_id=..., url=...,
title=..., description=..., objectives=[...], ...)]
```
## API
| Endpoint | Description |
|---|---|
| `GET /api/formations` | Filterable list (`training_type`, `category`, `mode`, `city`, `language`, `level`, `source`, `free`, `price_max`, `starts_after`, `q`, `sort`, `limit`, `offset`) |
| `GET /api/formations/{uid}` | Full program page + price history + similar programs |
| `GET /api/facets` | Distinct values for building filters |
| `GET /api/sources` | Institution registry + sync state |
| `GET /api/stats` | Global portrait + sync log |
| `POST /api/sync` | Trigger a background sync |
## Tests
```bash
.venv/bin/python -m pytest tests/ -q
```
---
© 2026 **Simon-Pierre Boucher** — [contact@spboucher.ai](mailto:contact@spboucher.ai)