SPB Git
spboucher.ai

spb/qc26

Public

QC26 2.0.0 — au-delà des sondages + refonte visuelle

Moteur : signaux (Polymarket, Wikipédia, presse Serper+LLM, GDELT, questions de leadership, partielles DGEQ),
ensemble (prior fondamentaux, momentum borné, tempérage par repondération IPF vers les marchés, ajustement
partielles), statistiques pondérées, summary.components (sondages seuls / sans marchés / ensemble), API
/api/signals*, /api/forecast/components, jobs planifiés, admin signals/refresh, changelog 2.0.0.
Correctifs : sortants depuis la liste DGEQ des candidatures (5 circonscriptions de partielles erronées),
chemins d'archive relocalisables, horodatages UTC explicites (TZDateTime).
Web : design system clair/sombre (Fraunces, Inter, JetBrains Mono, accent sarcelle), couleurs de partis en
variables CSS, héros éditorial avec gaufre des 127 sièges, comparaison sondages/marchés/ensemble, page
/signaux, tendance interactive, refonte nav/mobile/footer, méthodologie v2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Sep 7, 2026) parent 570fe76 Browse files

70 changed files +3,279 −634

70 changed files
Load all 70 files Unified Split
modified CLAUDE.md +7 −2
@@ -12,11 +12,16 @@ Spécification produit complète : voir la mission QC26 (design « Editorial Dat
12 12 ## Où sont les choses
13 13 - Moteur : `engine/` (FastAPI :8201). Base SQLite `engine/data/qc26.db` (WAL). Archives `engine/data/archive/{polls,documents,live}` par SHA-256. Copies officielles DGEQ `engine/data/official/`.
14 14 - Web : `web/` (Next 16, :8200), rewrite `/api/*` → moteur. Composants du design system dans `web/src/components/`.
15 −- Modèle : `engine/qc26/model/` (aggregate = poids/effets maison/Kalman ; ridings = swing proportionnel + erreurs ; run = orchestration, diff, impacts). Changer un paramètre = bump `model_version` + entrée `CHANGELOG` dans `api/public.py`.
15 +- Modèle : `engine/qc26/model/` (aggregate = poids/effets maison/Kalman ; ridings = swing proportionnel + erreurs ; ensemble = v2 : prior fondamentaux, momentum borné, tempérage par les marchés via repondération IPF des simulations, ajustement partielles ; run = orchestration, stats pondérées, `summary.components`, diff). Changer un paramètre = bump `model_version` + entrée `CHANGELOG` dans `api/public.py`.
16 +- Signaux (v2) : `engine/qc26/signals/` — markets (Polymarket Gamma/CLOB), attention (Wikimedia pageviews), media (Serper news + classement LLM ; GDELT best-effort, 1 req/6,5 s), fundamentals (questions de leadership des sondages → satisfaction, meilleur PM, favorabilité…), byelections (DGEQ via Firecrawl, code URL = code DGEQ carte 2017). Tables `signals`, `markets`, `market_prices`, `media_items`, `by_elections`. API `/api/signals*`, `/api/forecast/components`, admin `POST /api/admin/signals/refresh?what=all`. Page `/signaux`.
17 +- Design : tokens clair/sombre dans `web/src/app/globals.css` (`data-theme`, script inline `THEME_INIT_SCRIPT`), Fraunces (display) + Inter + JetBrains Mono ; couleurs de partis via `partyVar()/partyAlpha()` (variables CSS `--p-*`, éclaircies en sombre) — `partyColor()` (hex) réservé aux images OG.
16 18 - Soirée électorale : `engine/qc26/live/` (feed officiel, simulation 2022, nowcast). `POST /api/admin/live/simulation/start` rejoue 2022 en accéléré pour tester la chaîne complète.
17 19
18 20 ## Gotchas
19 −- SQLite : ne jamais tenir une transaction ouverte pendant un appel LLM (commits explicites dans les connecteurs) ; timeout 90 s.
21 +- SQLite : ne jamais tenir une transaction ouverte pendant un appel LLM (commits explicites dans les connecteurs) ; timeout 90 s. Les `DateTime` passent par `TZDateTime` (UTC explicite à la relecture) : toujours `.isoformat()` sur des objets tz-aware.
22 +- Chemins d'archive stockés absolus : lire via `util.resolve_archive_path()` (la base a été créée sur le laptop puis déplacée).
23 +- Sortant·e = drapeau `depute_sortant` de la liste DGEQ des candidatures, puis gagnant de partielle, puis gagnant 2022 transposé (jamais l'inverse).
24 +- Fraunces via `next/font/google` : `axes` seulement sans `weight` (police variable).
20 25 - DGEQ : « PCQ » = Parti communiste, le Parti conservateur est « PCOQ » ; CSV en cp1252 avec tiret 0x96 (`util.fix_dgeq_dash`).
21 26 - Wikipédia n'est qu'un index d'amorçage (tier 5) : les lignes `secondary-index` sont remplacées par le PDF de la firme dès qu'il est archivé ; les URL médias (La Presse, L'actualité, X…) ne sont pas des documents primaires.
22 27 - La règle « pas de candidature → part résiduelle » ne s'active que pour un parti dont la liste est ≥ 90 % complète.
modified README.md +5 −0
@@ -50,3 +50,8 @@ Le premier démarrage sur base vide télécharge les données officielles, amorc
50 50 ## Production
51 51
52 52 Déployé par `mld` (passerelle M1M32) : `mld stage <dir> qc26` → `mld deploy qc26`. Deux processus PM2 (`qc26-engine` :8201, `qc26-web` :8200) + `qc26-ngrok` (www.qc26.ca → 8200). Le web réécrit `/api/*` vers le moteur.
53 +
54 +
55 +## Modèle 2.0 (2026-09-07) — au-delà des sondages
56 +
57 +Quatre familles de signaux indépendants, archivés avec leur provenance et publiés séparément (`/signaux`, `summary.components`) : marchés de prédiction Polymarket (repondération des simulations, poids 30 %), fondamentaux (satisfaction du gouvernement → prior sur le parti sortant, poids décroissant vers 0 le jour du vote), momentum borné (prime au chef, attention Wikipédia, ton de presse classé par LLM), élections partielles depuis 2022 (excès de swing local). La vue « sondages seuls » reste publiée. Refonte visuelle « Editorial Data Intelligence » : thème clair/sombre, Fraunces + Inter + JetBrains Mono, gaufre des 127 sièges, tendance interactive.
modified deploy/qc26.manifest.json +125 −123
@@ -1,127 +1,129 @@
1 1 {
2 − "app": "qc26",
3 − "label": "QC26 — L'élection québécoise 2026, en données",
4 − "domain": "www.qc26.ca",
5 − "port": 8200,
6 − "health_path": "/healthz",
7 − "dir": "~/apps/qc26",
8 − "extra_paths": [],
9 − "sync_excludes": [
10 − "engine/.venv/",
11 − "__pycache__/",
12 − ".pytest_cache/",
13 − ".ruff_cache/",
14 − "*.egg-info/",
15 − ".git/",
16 − "web/node_modules/",
17 − "web/.next/",
18 − "engine/logs/",
19 − "logs/",
20 − ".DS_Store",
21 − "engine/data/qc26.db",
22 − "engine/data/qc26.db-shm",
23 − "engine/data/qc26.db-wal",
24 − "engine/data/archive/",
25 − "engine/data/cache/"
2 + "app": "qc26",
3 + "label": "QC26 — L'élection québécoise 2026, en données",
4 + "domain": "www.qc26.ca",
5 + "port": 8200,
6 + "health_path": "/healthz",
7 + "dir": "~/apps/qc26",
8 + "extra_paths": [],
9 + "sync_excludes": [
10 + "engine/.venv/",
11 + "__pycache__/",
12 + ".pytest_cache/",
13 + ".ruff_cache/",
14 + "*.egg-info/",
15 + ".git/",
16 + "web/node_modules/",
17 + "web/.next/",
18 + "engine/logs/",
19 + "logs/",
20 + ".DS_Store",
21 + "engine/data/qc26.db",
22 + "engine/data/qc26.db-shm",
23 + "engine/data/qc26.db-wal",
24 + "engine/data/archive/",
25 + "engine/data/cache/"
26 + ],
27 + "requires": {
28 + "runtimes": [
29 + "pm2",
30 + "ngrok",
31 + "node",
32 + "pnpm",
33 + "uv",
34 + "uv-python@3.12"
26 35 ],
27 − "requires": {
28 − "runtimes": [
29 − "pm2",
30 − "ngrok",
31 − "node",
32 − "pnpm",
33 − "uv",
34 − "uv-python@3.12"
35 − ],
36 − "ram_gb": 4,
37 − "ports": [
38 − 8200,
39 − 8201
40 − ]
41 − },
42 − "ram_mb_observed": 1200,
43 − "size_mb": 120,
44 − "placement": {
45 − "pin": null,
46 − "prefer": "M3U96a",
47 − "avoid": [
48 − "M3U96b",
49 − "M1M32"
50 − ],
51 − "reason": "Monte Carlo numpy + build Next.js : cœurs et RAM ; uv-py3.12 + pnpm présents ; M3U96b réservé hfmarketdata"
52 − },
53 − "processes": [
54 − {
55 − "name": "qc26-engine",
56 − "manager": "pm2",
57 − "script": "{{HOME}}/apps/qc26/engine/.venv/bin/python",
58 − "args": [
59 − "-m",
60 − "uvicorn",
61 − "qc26.main:app",
62 − "--host",
63 − "127.0.0.1",
64 − "--port",
65 − "8201",
66 − "--no-access-log"
67 − ],
68 − "interpreter": null,
69 − "cwd": "{{HOME}}/apps/qc26/engine",
70 − "env": {
71 − "QC26_ENVIRONMENT": "production",
72 − "QC26_PORT": "8201",
73 − "QC26_PUBLIC_URL": "https://www.qc26.ca",
74 − "QC26_ADMIN_TOKEN": "qc26-admin-2026-Kp8vT3nQ7zLw",
75 − "QC26_FIRECRAWL_API_KEY": "fc-8cc44cb683044aee82e8cbad6505a01a",
76 − "QC26_ANTHROPIC_API_KEY": "__SET_AT_DEPLOY__",
77 − "QC26_LLM_MODEL": "claude-opus-5",
78 − "QC26_N_SIMULATIONS": "20000",
79 − "PYTHONUNBUFFERED": "1"
80 − },
81 − "cron_restart": null,
82 − "autorestart": true,
83 − "max_memory_restart": "3G"
84 − },
85 − {
86 − "name": "qc26-web",
87 − "manager": "pm2",
88 − "script": "/opt/homebrew/bin/node",
89 − "args": [
90 − "node_modules/next/dist/bin/next",
91 − "start",
92 − "-p",
93 − "8200",
94 − "-H",
95 − "0.0.0.0"
96 − ],
97 − "interpreter": null,
98 − "cwd": "{{HOME}}/apps/qc26/web",
99 − "env": {
100 − "NODE_ENV": "production",
101 − "QC26_ENGINE_URL": "http://127.0.0.1:8201",
102 − "NEXT_PUBLIC_SITE_URL": "https://www.qc26.ca",
103 − "NEXT_TELEMETRY_DISABLED": "1"
104 − },
105 − "cron_restart": null,
106 − "autorestart": true,
107 − "max_memory_restart": "1500M"
108 − }
36 + "ram_gb": 4,
37 + "ports": [
38 + 8200,
39 + 8201
40 + ]
41 + },
42 + "ram_mb_observed": 1200,
43 + "size_mb": 120,
44 + "placement": {
45 + "pin": null,
46 + "prefer": "M3U96a",
47 + "avoid": [
48 + "M3U96b",
49 + "M1M32"
109 50 ],
110 − "ngrok": {
111 − "name": "qc26-ngrok",
112 − "url": "www.qc26.ca",
113 − "port": 8200
51 + "reason": "Monte Carlo numpy + build Next.js : cœurs et RAM ; uv-py3.12 + pnpm présents ; M3U96b réservé hfmarketdata"
52 + },
53 + "processes": [
54 + {
55 + "name": "qc26-engine",
56 + "manager": "pm2",
57 + "script": "{{HOME}}/apps/qc26/engine/.venv/bin/python",
58 + "args": [
59 + "-m",
60 + "uvicorn",
61 + "qc26.main:app",
62 + "--host",
63 + "127.0.0.1",
64 + "--port",
65 + "8201",
66 + "--no-access-log"
67 + ],
68 + "interpreter": null,
69 + "cwd": "{{HOME}}/apps/qc26/engine",
70 + "env": {
71 + "QC26_ENVIRONMENT": "production",
72 + "QC26_PORT": "8201",
73 + "QC26_PUBLIC_URL": "https://www.qc26.ca",
74 + "QC26_ADMIN_TOKEN": "qc26-admin-2026-Kp8vT3nQ7zLw",
75 + "QC26_FIRECRAWL_API_KEY": "fc-8cc44cb683044aee82e8cbad6505a01a",
76 + "QC26_ANTHROPIC_API_KEY": "__SET_AT_DEPLOY__",
77 + "QC26_LLM_MODEL": "claude-opus-5",
78 + "QC26_N_SIMULATIONS": "20000",
79 + "PYTHONUNBUFFERED": "1",
80 + "QC26_SERPER_API_KEY": "__SET_AT_DEPLOY__",
81 + "QC26_TAVILY_API_KEY": "__SET_AT_DEPLOY__"
82 + },
83 + "cron_restart": null,
84 + "autorestart": true,
85 + "max_memory_restart": "3G"
114 86 },
115 − "launchd": [],
116 − "env_overrides": {},
117 − "hooks": {
118 − "post_sync": [
119 − "export PATH=\"$HOME/.local/bin:/opt/homebrew/bin:$PATH\"; cd engine && (test -x .venv/bin/python || uv venv --python 3.12 .venv) && uv pip install -q --python .venv/bin/python -e . && echo ' engine deps ok'",
120 − "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && pnpm install --frozen-lockfile --silent && echo ' web deps ok'",
121 − "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && QC26_ENGINE_URL=http://127.0.0.1:8201 NEXT_PUBLIC_SITE_URL=https://www.qc26.ca NEXT_TELEMETRY_DISABLED=1 pnpm build 2>&1 | tail -3 && echo ' web build ok'",
122 − "mkdir -p engine/data/archive/polls engine/data/archive/documents engine/data/archive/live engine/data/cache engine/logs && echo ' data ok'"
123 − ],
124 − "post_start": []
125 − },
126 − "notes": "Après le premier déploiement (2026-09-06), la base et les archives du nœud sont la source de vérité : exclues des syncs suivants."
127 −}
\ No newline at end of file
87 + {
88 + "name": "qc26-web",
89 + "manager": "pm2",
90 + "script": "/opt/homebrew/bin/node",
91 + "args": [
92 + "node_modules/next/dist/bin/next",
93 + "start",
94 + "-p",
95 + "8200",
96 + "-H",
97 + "0.0.0.0"
98 + ],
99 + "interpreter": null,
100 + "cwd": "{{HOME}}/apps/qc26/web",
101 + "env": {
102 + "NODE_ENV": "production",
103 + "QC26_ENGINE_URL": "http://127.0.0.1:8201",
104 + "NEXT_PUBLIC_SITE_URL": "https://www.qc26.ca",
105 + "NEXT_TELEMETRY_DISABLED": "1"
106 + },
107 + "cron_restart": null,
108 + "autorestart": true,
109 + "max_memory_restart": "1500M"
110 + }
111 + ],
112 + "ngrok": {
113 + "name": "qc26-ngrok",
114 + "url": "www.qc26.ca",
115 + "port": 8200
116 + },
117 + "launchd": [],
118 + "env_overrides": {},
119 + "hooks": {
120 + "post_sync": [
121 + "export PATH=\"$HOME/.local/bin:/opt/homebrew/bin:$PATH\"; cd engine && (test -x .venv/bin/python || uv venv --python 3.12 .venv) && uv pip install -q --python .venv/bin/python -e . && echo ' engine deps ok'",
122 + "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && pnpm install --frozen-lockfile --silent && echo ' web deps ok'",
123 + "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && QC26_ENGINE_URL=http://127.0.0.1:8201 NEXT_PUBLIC_SITE_URL=https://www.qc26.ca NEXT_TELEMETRY_DISABLED=1 pnpm build 2>&1 | tail -3 && echo ' web build ok'",
124 + "mkdir -p engine/data/archive/polls engine/data/archive/documents engine/data/archive/live engine/data/cache engine/logs && echo ' data ok'"
125 + ],
126 + "post_start": []
127 + },
128 + "notes": "Après le premier déploiement (2026-09-06), la base et les archives du nœud sont la source de vérité : exclues des syncs suivants. v2.0 (2026-09-07) : clés Serper/Tavily requises pour la revue de presse ; les tables signaux sont créées au démarrage (create_all)."
129 +}
modified engine/qc26/api/admin.py +26 −0
@@ -110,6 +110,32 @@ def compass_positions(only_missing: bool = True, s: Session = Depends(db)):
110 110 return positions.compute_positions(s, only_missing=only_missing)
111 111
112 112
113 +@router.post("/signals/refresh", dependencies=[Depends(require_admin)])
114 +def signals_refresh(what: str = "all", run: bool = True, s: Session = Depends(db)):
115 + """Rafraîchit les signaux (markets | attention | fundamentals | byelections | news | gdelt | all) puis recalcule."""
116 + from ..ingest import official
117 + from ..signals import attention, byelections, fundamentals, markets, media
118 + out = {}
119 + parts = [x.strip() for x in what.split(",")] if what != "all" else ["markets", "attention", "fundamentals", "byelections", "news"]
120 + if "markets" in parts:
121 + out["markets"] = markets.sync_markets(s); s.commit()
122 + if "attention" in parts:
123 + out["attention"] = attention.sync_attention(s); s.commit()
124 + if "fundamentals" in parts:
125 + out["fundamentals"] = fundamentals.extract_from_polls(s); s.commit()
126 + if "byelections" in parts:
127 + out["byelections"] = byelections.sync_by_elections(s); official.apply_by_election_incumbents(s); s.commit()
128 + if "news" in parts:
129 + out["news"] = media.sync_news(s, classify=True); s.commit()
130 + if "gdelt" in parts:
131 + out["gdelt"] = media.sync_gdelt(s); s.commit()
132 + if run:
133 + from ..model.run import run_forecast
134 + r = run_forecast(s, trigger="admin:signals")
135 + out["run"] = r.id
136 + return out
137 +
138 +
113 139 @router.get("/review", dependencies=[Depends(require_admin)])
114 140 def review_queue(s: Session = Depends(db)):
115 141 polls = s.scalars(select(Poll).where(Poll.validated == False).order_by(desc(Poll.fieldwork_end))).all() # noqa: E712
modified engine/qc26/api/public.py +74 −4
@@ -389,7 +389,11 @@ def poll_document_file(sha: str, s: Session = Depends(db)):
389 389 doc = s.scalar(select(PollDocument).where(PollDocument.sha256 == sha))
390 390 if not doc:
391 391 raise HTTPException(404)
392 − return FileResponse(doc.path, media_type=doc.content_type or "application/octet-stream", filename=f"qc26-archive-{sha[:12]}{'.pdf' if doc.path.endswith('.pdf') else '.html'}")
392 + from ..util import resolve_archive_path
393 + fp = resolve_archive_path(doc.path)
394 + if not fp or not fp.exists():
395 + raise HTTPException(404, "fichier d'archive introuvable")
396 + return FileResponse(str(fp), media_type=doc.content_type or "application/octet-stream", filename=f"qc26-archive-{sha[:12]}{'.pdf' if doc.path.endswith('.pdf') else '.html'}")
393 397
394 398
395 399 @router.get("/documents/{doc_id}")
@@ -413,7 +417,11 @@ def document_file(doc_id: int, vid: int, s: Session = Depends(db)):
413 417 v = s.get(DocumentVersion, vid)
414 418 if not v or v.document_id != doc_id:
415 419 raise HTTPException(404)
416 − return FileResponse(v.path, media_type=v.content_type or "application/octet-stream")
420 + from ..util import resolve_archive_path
421 + fp = resolve_archive_path(v.path)
422 + if not fp or not fp.exists():
423 + raise HTTPException(404, "fichier d'archive introuvable")
424 + return FileResponse(str(fp), media_type=v.content_type or "application/octet-stream")
417 425
418 426
419 427 @router.get("/proposals")
@@ -623,7 +631,8 @@ def replay(t: str | None = None, s: Session = Depends(db)):
623 631 if not target:
624 632 raise HTTPException(404)
625 633 from pathlib import Path
626 − data = json.loads(Path(target.path).read_text())
634 + from ..util import resolve_archive_path
635 + data = json.loads((resolve_archive_path(target.path) or Path(target.path)).read_text())
627 636 ridings_by_name = {}
628 637 from ..util import fix_dgeq_dash, norm_name
629 638 for r in s.scalars(select(Riding)).all():
@@ -678,7 +687,8 @@ def status(s: Session = Depends(db)):
678 687 return {"components": {"pollWatcher": "operational" if poll_srcs and all(x.status == "operational" for x in poll_srcs) else ("degraded" if poll_srcs else "ready"),
679 688 "forecastEngine": "operational" if run and run.status == "ok" else "degraded",
680 689 "electionFeed": next((x.status for x in sources if x.id == "eq-live"), "ready"),
681 − "documentWatcher": comp("party-docs"), "officialData": "operational" if all(x.status == "operational" for x in sources if x.kind == "official") else "degraded"},
690 + "documentWatcher": comp("party-docs"), "officialData": "operational" if all(x.status == "operational" for x in sources if x.kind == "official") else "degraded",
691 + "signals": ("operational" if [x for x in sources if x.kind == "signal"] and all(x.status in ("operational", "ready") for x in sources if x.kind == "signal") else ("degraded" if [x for x in sources if x.kind == "signal"] else "ready"))},
682 692 "sources": [{"id": x.id, "name": x.name, "url": x.url, "tier": x.tier, "kind": x.kind, "status": x.status, "lastCheckedAt": x.last_checked_at.isoformat() if x.last_checked_at else None,
683 693 "lastOkAt": x.last_ok_at.isoformat() if x.last_ok_at else None, "detail": x.detail} for x in sources],
684 694 "jobs": [{"id": j.id, "kind": j.kind, "status": j.status, "startedAt": j.started_at.isoformat(), "finishedAt": j.finished_at.isoformat() if j.finished_at else None,
@@ -688,6 +698,13 @@ def status(s: Session = Depends(db)):
688 698
689 699
690 700 CHANGELOG = [
701 + {"version": "2.0.0", "date": "2026-09-07", "title": "Au-delà des sondages : ensemble à quatre composantes",
702 + "notes": ["Marchés de prédiction (Polymarket, ~800 k$ de volume) : marchés « plus de sièges », « majorité PQ », tranches de sièges (PQ, CAQ, PLQ), 2e/3e place, historique quotidien archivé. Les 20 000 simulations sont repondérées (ajustement proportionnel itératif) pour que les probabilités de l'ensemble valent 70 % modèle + 30 % marché (15 % pour les tranches, moins liquides). La vue « sondages seuls » reste publiée.",
703 + "Fondamentaux : la satisfaction à l'égard du gouvernement (questions de leadership des rapports de sondage, pondérées récence) fixe un prior sur la part du parti sortant (≈ 6 + 0,6 × satisfaction, σ 5,5 pt) mélangé aux sondages avec un poids qui décroît de 20 % à 120 jours du vote jusqu'à 0 le jour J.",
704 + "Momentum borné (± 1 pt, somme nulle) : prime au chef (meilleur premier ministre vs intention de vote, κ 0,15), attention publique (pages vues Wikipédia des partis et chef·fes, part 7 j vs 28 j, κ 0,08), ton médiatique (titres de presse classés par LLM, κ 0,5, atténué sous 10 articles).",
705 + "Élections partielles depuis 2022 (Saint-Henri–Sainte-Anne, Jean-Talon, Terrebonne, Arthabaska, Chicoutimi — résultats officiels DGEQ archivés) : l'excès de swing local observé, net du swing national du jour, corrige le prior de la circonscription (κ 0,35).",
706 + "Correction : le·la député·e sortant·e vient désormais de la liste officielle des candidatures (drapeau DGEQ) puis du résultat des partielles, et non plus du seul gagnant 2022 transposé (Jean-Talon, Terrebonne, Arthabaska–L'Érable, Chicoutimi, Saint-Henri–Sainte-Anne étaient erronés).",
707 + "Publication de chaque composante (summary.components) : sondages seuls, modèle sans marchés, prior fondamentaux, décomposition du momentum, cibles et probabilités atteintes du tempérage, taille d'échantillon effective."]},
691 708 {"version": "1.1.0", "date": "2026-09-06", "title": "Recensement 2021 et swing par segments",
692 709 "notes": ["Recensement de la population 2021 (Statistique Canada, profil des aires de diffusion agrégées + limites) agrégé par intersection surfacique aux 127 circonscriptions : langue maternelle, première langue officielle parlée, immigration, minorités visibles, âge, revenu, tenure, scolarité, chômage.",
693 710 "Excès de swing par langue : ventilations francophones / non-francophones des sondages (90 jours, pondérées récence × √n) comparées à une référence 2022 par régression écologique des résultats sur la part francophone ; appliqué à chaque circonscription selon sa composition (κ = 0,6).",
@@ -704,6 +721,59 @@ CHANGELOG = [
704 721 ]
705 722
706 723
724 +# ---------------------------------------------------------------- signaux (v2)
725 +@router.get("/signals")
726 +def signals_all(s: Session = Depends(db)):
727 + from ..signals import attention, byelections, fundamentals, markets, media
728 + run = _latest_run(s)
729 + comp = (run.summary or {}).get("components") if run else None
730 + return {"asOf": datetime.now(timezone.utc).isoformat(), "runId": run.id if run else None, "components": comp,
731 + "markets": {**markets.implied(s), "history": markets.history(s)["series"]},
732 + "attention": attention.attention_summary(s), "media": media.media_summary(s),
733 + "fundamentals": fundamentals.fundamentals_summary(s), "byElections": byelections.byelection_summary(s)}
734 +
735 +
736 +@router.get("/signals/markets")
737 +def signals_markets(days: int = 120, s: Session = Depends(db)):
738 + from ..signals import markets
739 + return {**markets.implied(s), "history": markets.history(s, days)["series"]}
740 +
741 +
742 +@router.get("/signals/attention")
743 +def signals_attention(s: Session = Depends(db)):
744 + from ..signals import attention
745 + return attention.attention_summary(s)
746 +
747 +
748 +@router.get("/signals/media")
749 +def signals_media(window: int = 7, limit: int = 60, s: Session = Depends(db)):
750 + from ..signals import media
751 + d = media.media_summary(s, window=window)
752 + d["latest"] = d["latest"][:limit]
753 + return d
754 +
755 +
756 +@router.get("/signals/fundamentals")
757 +def signals_fundamentals(s: Session = Depends(db)):
758 + from ..signals import fundamentals
759 + return fundamentals.fundamentals_summary(s)
760 +
761 +
762 +@router.get("/signals/byelections")
763 +def signals_byelections(s: Session = Depends(db)):
764 + from ..signals import byelections
765 + return byelections.byelection_summary(s)
766 +
767 +
768 +@router.get("/forecast/components")
769 +def forecast_components(s: Session = Depends(db)):
770 + run = _latest_run(s)
771 + if not run:
772 + raise HTTPException(404)
773 + return {"runId": run.id, "modelVersion": run.model_version, "completedAt": run.completed_at.isoformat() if run.completed_at else None,
774 + "components": (run.summary or {}).get("components"), "parties": run.summary.get("parties"), "pMinority": run.summary.get("p_minority")}
775 +
776 +
707 777 @router.get("/changelog")
708 778 def changelog(s: Session = Depends(db)):
709 779 runs = s.scalars(select(ForecastRun).where(ForecastRun.kind == "forecast", ForecastRun.status == "ok").order_by(desc(ForecastRun.id)).limit(30)).all()
modified engine/qc26/config.py +32 −2
@@ -52,7 +52,7 @@ class Settings(BaseSettings):
52 52 # ils dérivent de la liste officielle des circonscriptions (127 en 2026 → 64).
53 53
54 54 # --- Modèle ---
55 − model_version: str = "1.1.0"
55 + model_version: str = "2.0.0"
56 56 n_simulations: int = 20000
57 57 random_seed: int | None = None
58 58 recency_halflife_days: float = 14.0 # hors campagne
@@ -68,9 +68,39 @@ class Settings(BaseSettings):
68 68 riding_error_sd: float = 0.20 # logit
69 69 incumbent_retirement_penalty: float = 0.06 # logit (~1,3 pp)
70 70
71 + # --- Ensemble v2 : au-delà des sondages ---
72 + # Fondamentaux (satisfaction du gouvernement → part du parti sortant) : poids maximal loin du
73 + # scrutin, décroissant linéairement vers 0 le jour du vote (horizon en jours).
74 + fundamentals_weight_max: float = 0.20
75 + fundamentals_horizon_days: int = 120
76 + fundamentals_alpha: float = 6.0 # part sortant ≈ α + β·satisfaction (%), calibration indicative QC 2003-2022
77 + fundamentals_beta: float = 0.60
78 + fundamentals_sd_pp: float = 5.5
79 + # Momentum : leadership (meilleur PM vs vote), attention (Wikipédia), ton médiatique → nudge borné (pp)
80 + momentum_leader_kappa: float = 0.15
81 + momentum_attention_kappa: float = 0.08 # pp de vote par pp de variation de la part d'attention (7 j vs 28 j)
82 + momentum_media_kappa: float = 0.5 # pp de vote par unité de ton médiatique (−1…+1), atténué sous 10 articles
83 + momentum_max_pp: float = 1.0
84 + # Marchés de prédiction : tempérage des simulations (pondération itérative) vers les prix implicites
85 + markets_weight: float = 0.30
86 + markets_min_volume: float = 5000.0
87 + markets_max_age_hours: int = 72
88 + # Partielles : ajustement des priors de circonscription du swing observé (rétréci)
89 + byelection_kappa: float = 0.35
90 +
71 91 # --- Sources / connecteurs ---
72 92 firecrawl_api_key: str = ""
73 93 anthropic_api_key: str = ""
94 + serper_api_key: str = ""
95 + tavily_api_key: str = ""
96 + polymarket_gamma: str = "https://gamma-api.polymarket.com"
97 + polymarket_clob: str = "https://clob.polymarket.com"
98 + polymarket_event_slugs: str = ("quebec-general-election-winner,of-seats-won-by-pq-in-the-2026-quebec-general-election,"
99 + "of-seats-won-by-caq-in-the-2026-quebec-general-election,of-seats-won-by-plq-in-the-2026-quebec-general-election,"
100 + "will-the-pq-win-a-majority-in-the-2026-quebec-general-election,quebec-general-election-second-place-20260825152158,"
101 + "quebec-general-election-third-place-20260825152158")
102 + signals_markets_minutes: int = 30
103 + signals_media_hours: int = 3
74 104 llm_model: str = "claude-opus-5"
75 105 llm_enabled: bool = True
76 106 user_agent: str = (
@@ -111,5 +141,5 @@ class Settings(BaseSettings):
111 141
112 142 settings = Settings()
113 143 for _d in (settings.archive_dir / "polls", settings.archive_dir / "documents",
114 − settings.archive_dir / "live", settings.cache_dir):
144 + settings.archive_dir / "live", settings.archive_dir / "partielles", settings.cache_dir):
115 145 _d.mkdir(parents=True, exist_ok=True)
modified engine/qc26/connectors/parties/documents.py +8 −5
@@ -21,7 +21,7 @@ from sqlalchemy.orm import Session
21 21 from ... import llm
22 22 from ...config import settings
23 23 from ...db import (DocumentChunk, DocumentVersion, IngestionJob, Party, PartyDocument, PolicyProposal, Source, audit, utcnow)
24 −from ...util import fetch_bytes, html_to_text, http, parse_fr_date, pdf_pages_text, sha256_bytes
24 +from ...util import fetch_bytes, html_to_text, http, parse_fr_date, pdf_pages_text, resolve_archive_path, sha256_bytes
25 25 from ..polls.registry import firecrawl_map, firecrawl_scrape
26 26
27 27 log = logging.getLogger("qc26.parties.docs")
@@ -232,8 +232,9 @@ def fetch_version(s: Session, doc: PartyDocument) -> DocumentVersion | None:
232 232
233 233 def diff_versions(old: DocumentVersion, new: DocumentVersion) -> dict:
234 234 import difflib
235 − a = Path(old.text_path).read_text().splitlines() if old.text_path and Path(old.text_path).exists() else []
236 − b = Path(new.text_path).read_text().splitlines() if new.text_path and Path(new.text_path).exists() else []
235 + _pa, _pb = resolve_archive_path(old.text_path), resolve_archive_path(new.text_path)
236 + a = _pa.read_text().splitlines() if _pa and _pa.exists() else []
237 + b = _pb.read_text().splitlines() if _pb and _pb.exists() else []
237 238 sm = difflib.SequenceMatcher(None, a, b, autojunk=False)
238 239 added, removed = [], []
239 240 for tag, i1, i2, j1, j2 in sm.get_opcodes():
@@ -276,7 +277,8 @@ Si l'extrait ne contient aucune proposition (page de navigation, formulaire, bio
276 277
277 278
278 279 def analyze_version(s: Session, doc: PartyDocument, v: DocumentVersion, max_chunks: int = 12) -> int:
279 − text = Path(v.text_path).read_text() if v.text_path else ""
280 + _tp = resolve_archive_path(v.text_path)
281 + text = _tp.read_text() if _tp and _tp.exists() else ""
280 282 if len(text.strip()) < 300:
281 283 doc.status = "ignored"
282 284 v.analyzed_at = utcnow()
@@ -365,7 +367,8 @@ def sync_party(s: Session, party_id: str, max_docs: int = 80, analyze: bool = Tr
365 367 s.commit()
366 368 if not doc.title or doc.title == url.rsplit("/", 1)[-1][:200]:
367 369 # titre depuis le texte (première ligne significative)
368 − txt = Path(v.text_path).read_text()[:2000] if v.text_path else ""
370 + _tp2 = resolve_archive_path(v.text_path)
371 + txt = _tp2.read_text()[:2000] if _tp2 and _tp2.exists() else ""
369 372 first = next((l.strip() for l in txt.splitlines() if len(l.strip()) > 12), None)
370 373 if first:
371 374 doc.title = first[:200]
modified engine/qc26/connectors/polls/base.py +4 −2
@@ -20,7 +20,7 @@ from ... import llm
20 20 from ...config import settings
21 21 from ...db import Poll, PollDocument, PollResult, PollSubsample, Pollster, audit, utcnow
22 22 from ...parties import PARTY_IDS, party_id_from_label
23 −from ...util import fetch_bytes, html_to_text, pdf_pages_text, pdf_tables_text, sha256_bytes
23 +from ...util import fetch_bytes, html_to_text, pdf_pages_text, pdf_tables_text, resolve_archive_path, sha256_bytes
24 24
25 25 log = logging.getLogger("qc26.polls.connector")
26 26
@@ -95,7 +95,9 @@ class PollConnector:
95 95 return doc
96 96
97 97 def extract_text(self, doc: PollDocument) -> tuple[list[str], list[tuple[int, str]]]:
98 − path = Path(doc.path)
98 + path = resolve_archive_path(doc.path) or Path(doc.path)
99 + if not path.exists():
100 + raise FileNotFoundError(f"archive absente : {doc.path}")
99 101 if path.suffix == ".pdf":
100 102 pages = pdf_pages_text(path)
101 103 tables = pdf_tables_text(path)
modified engine/qc26/db.py +135 −26
@@ -11,7 +11,7 @@ from contextlib import contextmanager
11 11 from datetime import date, datetime, timezone
12 12 from typing import Any, Iterator
13 13
14 −from sqlalchemy import (JSON, Boolean, Date, DateTime, Float, ForeignKey, Integer, String, Text,
14 +from sqlalchemy import (JSON, Boolean, Date, DateTime, Float, ForeignKey, Integer, String, Text, TypeDecorator,
15 15 UniqueConstraint, create_engine, event, text)
16 16 from sqlalchemy.orm import DeclarativeBase, Mapped, Session, mapped_column, relationship, sessionmaker
17 17
@@ -22,6 +22,23 @@ def utcnow() -> datetime:
22 22 return datetime.now(timezone.utc)
23 23
24 24
25 +class TZDateTime(TypeDecorator):
26 + """SQLite ne conserve pas le fuseau : on stocke en UTC naïf et on relit en UTC explicite, pour que
27 + `.isoformat()` porte toujours « +00:00 » (sinon le navigateur interprète l'heure comme locale)."""
28 + impl = DateTime(timezone=True)
29 + cache_ok = True
30 +
31 + def process_bind_param(self, value, dialect):
32 + if value is not None and value.tzinfo is not None:
33 + return value.astimezone(timezone.utc).replace(tzinfo=None)
34 + return value
35 +
36 + def process_result_value(self, value, dialect):
37 + if value is not None and value.tzinfo is None:
38 + return value.replace(tzinfo=timezone.utc)
39 + return value
40 +
41 +
25 42 class Base(DeclarativeBase):
26 43 pass
27 44
@@ -47,7 +64,7 @@ class Party(Base):
47 64 vote_2022: Mapped[float | None] = mapped_column(Float)
48 65 order: Mapped[int] = mapped_column(Integer, default=0)
49 66 color_note: Mapped[str | None] = mapped_column(Text)
50 − updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow)
67 + updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
51 68
52 69
53 70 class Region(Base):
@@ -115,7 +132,7 @@ class RidingDemographics(Base):
115 132 ada_count: Mapped[int] = mapped_column(Integer, default=0)
116 133 coverage: Mapped[float | None] = mapped_column(Float) # part de la population de la circ. couverte par des ADA
117 134 source: Mapped[str] = mapped_column(String(300))
118 − computed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
135 + computed_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
119 136
120 137
121 138 class HistoricalResult(Base):
@@ -149,8 +166,8 @@ class Candidate(Base):
149 166 incumbent: Mapped[bool] = mapped_column(Boolean, default=False)
150 167 status: Mapped[str] = mapped_column(String(8)) # CPA (confirmé parti) / CIA (indépendant) …
151 168 state: Mapped[str] = mapped_column(String(8))
152 − dgeq_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
153 − fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
169 + dgeq_updated_at: Mapped[datetime | None] = mapped_column(TZDateTime)
170 + fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
154 171
155 172
156 173 # --------------------------------------------------------------------- sondages
@@ -175,7 +192,7 @@ class PollDocument(Base):
175 192 content_type: Mapped[str | None] = mapped_column(String(80))
176 193 bytes: Mapped[int] = mapped_column(Integer)
177 194 pages: Mapped[int | None] = mapped_column(Integer)
178 − downloaded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
195 + downloaded_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
179 196 text_path: Mapped[str | None] = mapped_column(String(300))
180 197 http_last_modified: Mapped[str | None] = mapped_column(String(80))
181 198 pollster_id: Mapped[str | None] = mapped_column(String(32))
@@ -204,8 +221,8 @@ class Poll(Base):
204 221 validation_notes: Mapped[dict | list | None] = mapped_column(JSON)
205 222 undecided: Mapped[float | None] = mapped_column(Float)
206 223 notes: Mapped[dict | list | None] = mapped_column(JSON)
207 − ingested_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
208 − updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow)
224 + ingested_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
225 + updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
209 226 excluded: Mapped[bool] = mapped_column(Boolean, default=False)
210 227 exclusion_reason: Mapped[str | None] = mapped_column(String(200))
211 228 results: Mapped[list["PollResult"]] = relationship(cascade="all, delete-orphan", lazy="selectin")
@@ -250,8 +267,8 @@ class PartyDocument(Base):
250 267 url: Mapped[str] = mapped_column(String(600), unique=True)
251 268 doc_type: Mapped[str] = mapped_column(String(32)) # plateforme / cadre_financier / communique / page / engagement
252 269 published_date: Mapped[date | None] = mapped_column(Date)
253 − discovered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
254 − last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
270 + discovered_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
271 + last_checked_at: Mapped[datetime | None] = mapped_column(TZDateTime)
255 272 current_version_id: Mapped[int | None] = mapped_column(Integer)
256 273 proposals_count: Mapped[int] = mapped_column(Integer, default=0)
257 274 status: Mapped[str] = mapped_column(String(16), default="new") # new / analyzed / failed / ignored
@@ -268,8 +285,8 @@ class DocumentVersion(Base):
268 285 content_type: Mapped[str | None] = mapped_column(String(80))
269 286 bytes: Mapped[int] = mapped_column(Integer)
270 287 pages: Mapped[int | None] = mapped_column(Integer)
271 − fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
272 − analyzed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
288 + fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
289 + analyzed_at: Mapped[datetime | None] = mapped_column(TZDateTime)
273 290 diff_summary: Mapped[dict | None] = mapped_column(JSON)
274 291
275 292
@@ -298,7 +315,7 @@ class PolicyProposal(Base):
298 315 quote: Mapped[str] = mapped_column(Text)
299 316 confidence: Mapped[float] = mapped_column(Float)
300 317 status: Mapped[str] = mapped_column(String(16), default="accepted") # accepted / review / rejected
301 − extracted_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
318 + extracted_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
302 319
303 320
304 321 class CompassQuestion(Base):
@@ -324,7 +341,7 @@ class PartyPosition(Base):
324 341 proposal_ids: Mapped[list | None] = mapped_column(JSON)
325 342 rationale: Mapped[str | None] = mapped_column(Text)
326 343 status: Mapped[str] = mapped_column(String(16), default="pending") # pending / extracted / review / manual
327 − updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow)
344 + updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
328 345
329 346
330 347 # -------------------------------------------------------------------- forecast
@@ -334,8 +351,8 @@ class ForecastRun(Base):
334 351 model_version: Mapped[str] = mapped_column(String(16))
335 352 kind: Mapped[str] = mapped_column(String(16), default="forecast") # forecast / nowcast
336 353 data_cutoff: Mapped[date] = mapped_column(Date)
337 − started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
338 − completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
354 + started_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
355 + completed_at: Mapped[datetime | None] = mapped_column(TZDateTime)
339 356 poll_count: Mapped[int] = mapped_column(Integer, default=0)
340 357 n_simulations: Mapped[int] = mapped_column(Integer, default=0)
341 358 seed: Mapped[int | None] = mapped_column(Integer)
@@ -427,7 +444,7 @@ class PollWeight(Base):
427 444 class LiveSnapshot(Base):
428 445 __tablename__ = "live_snapshots"
429 446 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
430 − fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True)
447 + fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
431 448 source_updated_at: Mapped[str | None] = mapped_column(String(40))
432 449 sha256: Mapped[str] = mapped_column(String(64), index=True)
433 450 path: Mapped[str] = mapped_column(String(300))
@@ -445,7 +462,7 @@ class LiveSnapshot(Base):
445 462 class LiveRidingState(Base):
446 463 __tablename__ = "live_riding_states"
447 464 riding_code: Mapped[int] = mapped_column(Integer, primary_key=True)
448 − updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
465 + updated_at: Mapped[datetime | None] = mapped_column(TZDateTime)
449 466 source_updated_at: Mapped[str | None] = mapped_column(String(40))
450 467 status: Mapped[str] = mapped_column(String(16), default="NOT_STARTED") # NOT_STARTED/REPORTING/LEADING/PROJECTED/ELECTED/FINAL
451 468 bureaux_completes: Mapped[int] = mapped_column(Integer, default=0)
@@ -466,7 +483,7 @@ class LiveRidingState(Base):
466 483 class ElectionEvent(Base):
467 484 __tablename__ = "election_events"
468 485 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
469 − ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True)
486 + ts: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
470 487 kind: Mapped[str] = mapped_column(String(32), index=True) # lead_change / riding_complete / seats_update / reporting / first_results / final
471 488 riding_code: Mapped[int | None] = mapped_column(Integer)
472 489 party_id: Mapped[str | None] = mapped_column(String(8))
@@ -478,12 +495,104 @@ class ElectionEvent(Base):
478 495 class NowcastRun(Base):
479 496 __tablename__ = "nowcast_runs"
480 497 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
481 − ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True)
498 + ts: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
482 499 snapshot_id: Mapped[int | None] = mapped_column(Integer)
483 500 ridings_reporting: Mapped[int] = mapped_column(Integer)
484 501 summary: Mapped[dict] = mapped_column(JSON) # par parti : élus, en avance, total, p_majority, seats mean/ci
485 502
486 503
504 +# ------------------------------------------------------- signaux alternatifs (v2)
505 +class Signal(Base):
506 + """Série temporelle d'un signal « au-delà des sondages » : marchés de prédiction, attention
507 + (Wikipédia), médias (volume / ton), fondamentaux tirés des questions de leadership, partielles.
508 + Une ligne = une observation datée, avec sa provenance (URL) et ses métadonnées."""
509 + __tablename__ = "signals"
510 + __table_args__ = (UniqueConstraint("source", "key", "party_id", "date"),)
511 + id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
512 + source: Mapped[str] = mapped_column(String(32), index=True) # polymarket / wikipedia / gdelt / serper-news / polls-leadership / dgeq-partielles
513 + key: Mapped[str] = mapped_column(String(64), index=True) # ex. p_most_seats / pageviews / tone / best_pm / satisfaction
514 + party_id: Mapped[str | None] = mapped_column(String(8), index=True)
515 + date: Mapped[date] = mapped_column(Date, index=True)
516 + value: Mapped[float] = mapped_column(Float)
517 + n: Mapped[int | None] = mapped_column(Integer)
518 + meta: Mapped[dict | list | None] = mapped_column(JSON)
519 + url: Mapped[str | None] = mapped_column(String(600))
520 + fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
521 +
522 +
523 +class Market(Base):
524 + """Marché de prédiction (Polymarket) rattaché à l'élection : question, type, parti, tranche."""
525 + __tablename__ = "markets"
526 + id: Mapped[str] = mapped_column(String(32), primary_key=True) # id Polymarket
527 + source: Mapped[str] = mapped_column(String(24), default="polymarket")
528 + event_slug: Mapped[str] = mapped_column(String(160), index=True)
529 + event_title: Mapped[str | None] = mapped_column(String(200))
530 + question: Mapped[str] = mapped_column(String(300))
531 + kind: Mapped[str] = mapped_column(String(24), index=True) # most_seats / majority / seats_band / place
532 + party_id: Mapped[str | None] = mapped_column(String(8), index=True)
533 + band_lo: Mapped[int | None] = mapped_column(Integer)
534 + band_hi: Mapped[int | None] = mapped_column(Integer)
535 + place: Mapped[int | None] = mapped_column(Integer)
536 + condition_id: Mapped[str | None] = mapped_column(String(80))
537 + token_id: Mapped[str | None] = mapped_column(String(100))
538 + volume: Mapped[float | None] = mapped_column(Float)
539 + liquidity: Mapped[float | None] = mapped_column(Float)
540 + end_date: Mapped[str | None] = mapped_column(String(40))
541 + url: Mapped[str | None] = mapped_column(String(300))
542 + last_price: Mapped[float | None] = mapped_column(Float)
543 + best_bid: Mapped[float | None] = mapped_column(Float)
544 + best_ask: Mapped[float | None] = mapped_column(Float)
545 + active: Mapped[bool] = mapped_column(Boolean, default=True)
546 + updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
547 +
548 +
549 +class MarketPrice(Base):
550 + __tablename__ = "market_prices"
551 + __table_args__ = (UniqueConstraint("market_id", "ts"),)
552 + id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
553 + market_id: Mapped[str] = mapped_column(ForeignKey("markets.id"), index=True)
554 + ts: Mapped[datetime] = mapped_column(TZDateTime, index=True)
555 + price: Mapped[float] = mapped_column(Float) # prix « Oui » (0–1)
556 +
557 +
558 +class MediaItem(Base):
559 + """Article de presse repéré (titre, média, date) et classé par le LLM : partis mentionnés et ton."""
560 + __tablename__ = "media_items"
561 + id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
562 + url: Mapped[str] = mapped_column(String(600), unique=True)
563 + title: Mapped[str] = mapped_column(String(400))
564 + domain: Mapped[str | None] = mapped_column(String(120), index=True)
565 + snippet: Mapped[str | None] = mapped_column(Text)
566 + published_at: Mapped[datetime | None] = mapped_column(TZDateTime, index=True)
567 + fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
568 + query: Mapped[str | None] = mapped_column(String(200))
569 + source: Mapped[str] = mapped_column(String(24), default="serper-news")
570 + parties: Mapped[dict | None] = mapped_column(JSON) # {party_id: tone −1…+1}
571 + topic: Mapped[str | None] = mapped_column(String(48))
572 + summary: Mapped[str | None] = mapped_column(String(300))
573 + confidence: Mapped[float | None] = mapped_column(Float)
574 + classified: Mapped[bool] = mapped_column(Boolean, default=False, index=True)
575 +
576 +
577 +class ByElection(Base):
578 + """Élection partielle provinciale depuis 2022 (résultats officiels DGEQ) comparée au résultat
579 + de 2022 dans la même circonscription : mesure directe du swing en conditions réelles."""
580 + __tablename__ = "by_elections"
581 + id: Mapped[str] = mapped_column(String(32), primary_key=True) # ex. 2025-03-17-542
582 + date: Mapped[date] = mapped_column(Date, index=True)
583 + dgeq_code: Mapped[int] = mapped_column(Integer)
584 + riding_name: Mapped[str | None] = mapped_column(String(80))
585 + url: Mapped[str] = mapped_column(String(300))
586 + sha256: Mapped[str | None] = mapped_column(String(64))
587 + path: Mapped[str | None] = mapped_column(String(300))
588 + results: Mapped[list | None] = mapped_column(JSON) # [{candidate, party_label, party_id, votes, share}]
589 + previous: Mapped[dict | None] = mapped_column(JSON) # {party_id: share 2022} (résultat général précédent, mêmes limites)
590 + turnout: Mapped[float | None] = mapped_column(Float)
591 + winner_party_id: Mapped[str | None] = mapped_column(String(8))
592 + status: Mapped[str] = mapped_column(String(16), default="new")
593 + fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
594 +
595 +
487 596 # ------------------------------------------------------------------ exploitation
488 597 class Source(Base):
489 598 __tablename__ = "sources"
@@ -492,8 +601,8 @@ class Source(Base):
492 601 url: Mapped[str] = mapped_column(String(600))
493 602 tier: Mapped[int] = mapped_column(Integer, default=1) # 1 officiel, 2 firme, 3 média, 5 index
494 603 kind: Mapped[str] = mapped_column(String(32))
495 − last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
496 − last_ok_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
604 + last_checked_at: Mapped[datetime | None] = mapped_column(TZDateTime)
605 + last_ok_at: Mapped[datetime | None] = mapped_column(TZDateTime)
497 606 status: Mapped[str] = mapped_column(String(16), default="unknown") # operational / degraded / down / ready
498 607 detail: Mapped[str | None] = mapped_column(String(300))
499 608
@@ -502,8 +611,8 @@ class IngestionJob(Base):
502 611 __tablename__ = "ingestion_jobs"
503 612 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
504 613 kind: Mapped[str] = mapped_column(String(48), index=True)
505 − started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow)
506 − finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
614 + started_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
615 + finished_at: Mapped[datetime | None] = mapped_column(TZDateTime)
507 616 status: Mapped[str] = mapped_column(String(16), default="running")
508 617 items: Mapped[int] = mapped_column(Integer, default=0)
509 618 message: Mapped[str | None] = mapped_column(Text)
@@ -513,7 +622,7 @@ class IngestionJob(Base):
513 622 class AuditLog(Base):
514 623 __tablename__ = "audit_logs"
515 624 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
516 − ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True)
625 + ts: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
517 626 actor: Mapped[str] = mapped_column(String(64)) # system:<job> / admin
518 627 entity: Mapped[str] = mapped_column(String(48))
519 628 entity_id: Mapped[str] = mapped_column(String(80))
@@ -537,7 +646,7 @@ class Setting(Base):
537 646 __tablename__ = "settings"
538 647 key: Mapped[str] = mapped_column(String(64), primary_key=True)
539 648 value: Mapped[dict | list | str | None] = mapped_column(JSON)
540 − updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow)
649 + updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
541 650
542 651
543 652 # -------------------------------------------------------------------- moteur
modified engine/qc26/ingest/official.py +31 −0
@@ -224,16 +224,47 @@ def sync_candidates(s: Session, refresh: bool = True) -> int:
224 224 counts[pid] += 1
225 225 if c.get("depute_sortant") == "O":
226 226 incumbents_running[code] = True
227 + # la liste officielle des candidatures est la vérité sur le·la député·e sortant·e :
228 + # elle corrige le gagnant 2022 transposé quand une partielle a changé la donne (Jean-Talon, Terrebonne, Arthabaska, Chicoutimi…)
229 + rd = ridings[code]
230 + if pid and (rd.incumbent_party_id != pid or not rd.incumbent_name):
231 + rd.incumbent_party_id = pid
232 + rd.incumbent_name = f"{c.get('prenom_bulletin_vote') or ''} {c.get('nom_bulletin_vote') or ''}".strip()
227 233 for pid, n in counts.items():
228 234 p = s.get(Party, pid)
229 235 if p:
230 236 p.candidates_count = n
231 237 for code, rd in ridings.items():
232 238 rd.incumbent_running = incumbents_running.get(code, False)
239 + apply_by_election_incumbents(s, ridings)
233 240 _mark_source(s, "eq-candidates", "Élections Québec — candidatures officielles", settings.eq_candidates_url, True)
234 241 return len(data)
235 242
236 243
244 +def apply_by_election_incumbents(s: Session, ridings: dict | None = None) -> int:
245 + """Sortant·e des circonscriptions ayant connu une partielle depuis 2022 : le·la gagnant·e de la partielle
246 + (résultat officiel DGEQ archivé), sauf si la liste des candidatures désigne déjà un·e sortant·e."""
247 + from ..db import ByElection, Candidate
248 + ridings = ridings or {r.code: r for r in s.scalars(select(Riding)).all()}
249 + by_name = {norm_name(r.name): r for r in ridings.values()}
250 + n = 0
251 + for b in s.scalars(select(ByElection).where(ByElection.status == "ok").order_by(ByElection.date)).all():
252 + if not b.riding_name or not b.results:
253 + continue
254 + key = norm_name(b.riding_name)
255 + rd = by_name.get(key) or next((r for k, r in by_name.items() if k.startswith(key)), None)
256 + if not rd:
257 + continue
258 + has_official = s.scalar(select(Candidate).where(Candidate.riding_code == rd.code, Candidate.incumbent == True).limit(1)) # noqa: E712
259 + if has_official:
260 + continue
261 + win = max(b.results, key=lambda r: r["votes"])
262 + if win.get("party_id") and rd.incumbent_party_id != win["party_id"]:
263 + rd.incumbent_party_id, rd.incumbent_name = win["party_id"], win.get("candidate")
264 + n += 1
265 + return n
266 +
267 +
237 268 # ------------------------------------------------------------------ résultats
238 269 def _load_results_json(path_dir: Path, url_dir: str, refresh: bool) -> dict:
239 270 raw = _download_or_local(f"{url_dir}/resultats.json", path_dir / "resultats.json", refresh)
modified engine/qc26/main.py +7 −0
@@ -44,6 +44,13 @@ def bootstrap() -> None:
44 44 run_forecast(s, trigger="reference_polls_2022")
45 45 except Exception as e: # noqa: BLE001
46 46 log.warning("sondages de référence 2022 : %s", e)
47 + try:
48 + from .db import Signal
49 + from .signals import fundamentals
50 + if not s.scalar(select(Signal).where(Signal.source == "polls-leadership").limit(1)):
51 + fundamentals.extract_from_polls(s) # dérivé des sondages déjà en base, sans réseau
52 + except Exception as e: # noqa: BLE001
53 + log.warning("fondamentaux : %s", e)
47 54 if not s.scalar(select(ForecastRun).where(ForecastRun.status == "ok").limit(1)):
48 55 from .model.run import run_forecast
49 56 try:
added engine/qc26/model/ensemble.py +258 −0
@@ -0,0 +1,258 @@
1 +"""Ensemble QC26 (modèle 2.0) — au-delà des sondages.
2 +
3 +Quatre composantes, toutes optionnelles (désactivées si le signal est absent de la base), toutes
4 +paramétrées dans `config.py` et toutes publiées séparément dans le run (« Sondages seuls »
5 +vs « Ensemble QC26 ») :
6 +
7 +1. **Fondamentaux** — prior structurel sur la part du parti sortant : part ≈ α + β·satisfaction
8 + (satisfaction à l'égard du gouvernement, pondérée récence, tirée des rapports de sondage). Mélangé
9 + à l'état latent des sondages dans l'espace logit avec un poids w_f qui décroît linéairement des
10 + `fundamentals_horizon_days` jours précédant le scrutin jusqu'à 0 le jour du vote.
11 +2. **Momentum** — nudge additif borné (± `momentum_max_pp`) sur la moyenne du vote au jour du vote :
12 + prime au chef (meilleur PM vs intention de vote), attention (Wikipédia, 7 j vs 28 j), ton médiatique
13 + (titres classés). Recentré à somme nulle entre les cinq partis.
14 +3. **Marchés de prédiction** — tempérage : les simulations Monte Carlo sont **repondérées**
15 + (ajustement proportionnel itératif) pour que les probabilités « plus de sièges », « majorité » et les
16 + tranches de sièges implicites de l'ensemble valent (1 − w_m)·modèle + w_m·marché. Toute la
17 + distribution (sièges, circonscriptions) reste cohérente : ce sont les mêmes simulations, pesées.
18 +4. **Partielles** — excès de swing local observé lors d'une partielle (net du swing national à la même
19 + date) appliqué, rétréci (κ), au prior de la circonscription concernée.
20 +"""
21 +from __future__ import annotations
22 +
23 +import math
24 +from dataclasses import dataclass, field
25 +from datetime import date
26 +
27 +import numpy as np
28 +
29 +from ..config import settings
30 +from ..parties import PARTY_IDS
31 +from .aggregate import PARTIES_ALL
32 +
33 +
34 +def _logit(p: float) -> float:
35 + p = min(max(p, 0.002), 0.998)
36 + return math.log(p / (1 - p))
37 +
38 +
39 +def _clip(x: float, lo: float, hi: float) -> float:
40 + return max(lo, min(hi, x))
41 +
42 +
43 +# --------------------------------------------------------------- pondérations
44 +def wmean(x: np.ndarray, w: np.ndarray) -> float:
45 + return float(np.average(x, weights=w))
46 +
47 +
48 +def wquantile(x: np.ndarray, w: np.ndarray, q: float) -> float:
49 + idx = np.argsort(x)
50 + xs, ws = x[idx], w[idx]
51 + cw = np.cumsum(ws)
52 + cw = cw / cw[-1]
53 + return float(xs[min(len(xs) - 1, int(np.searchsorted(cw, q, side="left")))])
54 +
55 +
56 +def whist(x: np.ndarray, w: np.ndarray, minlength: int) -> list[float]:
57 + h = np.bincount(x, weights=w, minlength=minlength)
58 + return [round(float(v), 2) for v in h]
59 +
60 +
61 +def ess(w: np.ndarray) -> float:
62 + return float(w.sum() ** 2 / (w ** 2).sum())
63 +
64 +
65 +# ------------------------------------------------------------- fondamentaux
66 +def time_weight(days_left: int) -> float:
67 + if settings.fundamentals_horizon_days <= 0:
68 + return 0.0
69 + return settings.fundamentals_weight_max * _clip(days_left / settings.fundamentals_horizon_days, 0.0, 1.0)
70 +
71 +
72 +def fundamentals_blend(mu_logit: np.ndarray, var_logit: np.ndarray, last_pct: np.ndarray, fund: dict | None, days_left: int, incumbent: str = "caq") -> tuple[np.ndarray, np.ndarray, dict]:
73 + """Mélange logit de l'état latent (sondages) et du prior fondamentaux. Retourne (mu, var, détail)."""
74 + detail = {"enabled": False, "weight": 0.0}
75 + if not fund or not fund.get("enabled") or fund.get("satisfaction") is None or incumbent not in PARTIES_ALL:
76 + return mu_logit, var_logit, detail
77 + w = time_weight(days_left)
78 + sat = float(fund["satisfaction"])
79 + inc_prior = _clip(settings.fundamentals_alpha + settings.fundamentals_beta * sat, 5.0, 60.0)
80 + j_inc = PARTIES_ALL.index(incumbent)
81 + polls = last_pct.copy() # % somme 100
82 + # les autres partis se partagent le reste au prorata des sondages (le prior n'informe que le sortant)
83 + rest_polls = polls.sum() - polls[j_inc]
84 + rest_prior = 100.0 - inc_prior
85 + prior = polls * (rest_prior / rest_polls) if rest_polls > 0 else polls.copy()
86 + prior[j_inc] = inc_prior
87 + prior_logit = np.array([_logit(p / 100.0) for p in prior])
88 + sd_pp = settings.fundamentals_sd_pp
89 + prior_var_logit = np.array([(sd_pp / 100.0) ** 2 / max((p / 100.0) * (1 - p / 100.0), 1e-3) ** 2 for p in prior])
90 + mu = (1 - w) * mu_logit + w * prior_logit
91 + var = var_logit + (w ** 2) * prior_var_logit
92 + # traduction en points de % (pour l'affichage) : moyenne après mélange vs avant
93 + def to_pct(z: np.ndarray) -> np.ndarray:
94 + p = 1 / (1 + np.exp(-z))
95 + return 100.0 * p / p.sum()
96 + before, after = to_pct(mu_logit), to_pct(mu)
97 + detail = {"enabled": True, "weight": round(w, 3), "daysLeft": days_left, "satisfaction": round(sat, 1), "satisfactionPolls": fund.get("satisfactionPolls"),
98 + "incumbent": incumbent, "incumbentPrior": round(inc_prior, 1), "prior": {p: round(float(v), 1) for p, v in zip(PARTIES_ALL, prior)},
99 + "shiftPp": {p: round(float(a - b), 2) for p, a, b in zip(PARTIES_ALL, after, before)},
100 + "params": {"alpha": settings.fundamentals_alpha, "beta": settings.fundamentals_beta, "sdPp": sd_pp, "weightMax": settings.fundamentals_weight_max, "horizonDays": settings.fundamentals_horizon_days}}
101 + return mu, var, detail
102 +
103 +
104 +# ----------------------------------------------------------------- momentum
105 +def momentum_shift(last_pct: np.ndarray, fund: dict | None, attention: dict | None, media: dict | None) -> tuple[np.ndarray, dict]:
106 + """Nudge additif (pp) par parti, borné et recentré ; détail par composante."""
107 + P = len(PARTIES_ALL)
108 + parts = {"leader": {}, "attention": {}, "media": {}}
109 + shift = np.zeros(P)
110 + for j, pid in enumerate(PARTY_IDS):
111 + v = float(last_pct[j])
112 + comp = 0.0
113 + if fund and fund.get("enabled"):
114 + bp = (fund.get("bestPmShare") or {}).get(pid)
115 + if bp is not None:
116 + x = settings.momentum_leader_kappa * (float(bp) - v)
117 + parts["leader"][pid] = round(x, 2)
118 + comp += x
119 + if attention and attention.get("enabled"):
120 + d = (attention.get("delta") or {}).get(pid)
121 + if d is not None:
122 + x = settings.momentum_attention_kappa * float(d)
123 + parts["attention"][pid] = round(x, 2)
124 + comp += x
125 + if media and media.get("enabled"):
126 + t = (media.get("tone") or {}).get(pid)
127 + n = (media.get("counts") or {}).get(pid, 0)
128 + if t is not None and n:
129 + x = settings.momentum_media_kappa * float(t) * min(1.0, n / 10.0)
130 + parts["media"][pid] = round(x, 2)
131 + comp += x
132 + shift[j] = _clip(comp, -settings.momentum_max_pp, settings.momentum_max_pp)
133 + main = shift[: len(PARTY_IDS)]
134 + if np.any(main):
135 + main = main - main.mean() # somme nulle : le momentum redistribue, il ne crée pas de vote
136 + shift[: len(PARTY_IDS)] = main
137 + detail = {"enabled": bool(np.any(shift)), "shiftPp": {p: round(float(x), 2) for p, x in zip(PARTIES_ALL, shift)}, "parts": parts,
138 + "params": {"leaderKappa": settings.momentum_leader_kappa, "attentionKappa": settings.momentum_attention_kappa, "mediaKappa": settings.momentum_media_kappa, "maxPp": settings.momentum_max_pp}}
139 + return shift, detail
140 +
141 +
142 +def apply_shift(national: np.ndarray, shift: np.ndarray) -> np.ndarray:
143 + """Décale les tirages nationaux (S, P) de `shift` pp, puis renormalise (plancher 0,2 %)."""
144 + if not np.any(shift):
145 + return national
146 + out = np.clip(national + shift[None, :], 0.2, None)
147 + return 100.0 * out / out.sum(axis=1, keepdims=True)
148 +
149 +
150 +# ------------------------------------------------------------------ marchés
151 +@dataclass
152 +class Tempering:
153 + weights: np.ndarray
154 + detail: dict = field(default_factory=dict)
155 +
156 +
157 +def market_tempering(seats: np.ndarray, majority: int, markets: dict | None, w_m: float | None = None) -> Tempering:
158 + """Repondération IPF des simulations vers (1−w)·modèle + w·marché sur : gagnant (plus de sièges),
159 + majorité (parti concerné), tranches de sièges (poids réduit de moitié : marchés moins liquides)."""
160 + S, P = seats.shape
161 + w = np.ones(S)
162 + w_m = settings.markets_weight if w_m is None else w_m
163 + detail = {"enabled": False, "weight": w_m}
164 + if not markets or w_m <= 0:
165 + return Tempering(w, detail)
166 + most = seats.argmax(axis=1)
167 + constraints: list[tuple[np.ndarray, dict[int, float], str]] = []
168 + # 1) plus de sièges
169 + ms = markets.get("mostSeats") or {}
170 + if ms:
171 + model = {j: float((most == j).mean()) for j in range(P)}
172 + target = {}
173 + for j, pid in enumerate(PARTIES_ALL):
174 + mk = ms.get(pid)
175 + target[j] = (1 - w_m) * model[j] + w_m * float(mk) if mk is not None else model[j]
176 + tot = sum(target.values()) or 1
177 + target = {j: v / tot for j, v in target.items()}
178 + constraints.append((most, target, "mostSeats"))
179 + # 2) majorité
180 + for pid, pm in (markets.get("majority") or {}).items():
181 + if pid in PARTIES_ALL:
182 + j = PARTIES_ALL.index(pid)
183 + ind = (seats[:, j] >= majority).astype(int)
184 + model_p = float(ind.mean())
185 + t = (1 - w_m) * model_p + w_m * float(pm)
186 + constraints.append((ind, {1: t, 0: 1 - t}, f"majority:{pid}"))
187 + # 3) tranches de sièges (poids réduit)
188 + wb = w_m / 2
189 + for pid, sb in (markets.get("seatBands") or {}).items():
190 + if pid not in PARTIES_ALL or not sb.get("bands"):
191 + continue
192 + j = PARTIES_ALL.index(pid)
193 + bands = sb["bands"]
194 + cat = np.full(S, -1)
195 + for k, b in enumerate(bands):
196 + lo, hi = b["lo"], b["hi"] if b["hi"] is not None else 10**6
197 + cat[(seats[:, j] >= lo) & (seats[:, j] <= hi)] = k
198 + if (cat < 0).any():
199 + continue
200 + model = {k: float((cat == k).mean()) for k in range(len(bands))}
201 + target = {k: (1 - wb) * model[k] + wb * float(bands[k]["p"]) for k in range(len(bands))}
202 + tot = sum(target.values()) or 1
203 + constraints.append((cat, {k: v / tot for k, v in target.items()}, f"seatBands:{pid}"))
204 + if not constraints:
205 + return Tempering(w, detail)
206 + for _ in range(25):
207 + for cat, target, _name in constraints:
208 + cur = {k: float(w[cat == k].sum() / w.sum()) for k in target}
209 + for k, t in target.items():
210 + if cur.get(k, 0) > 1e-9 and t > 0:
211 + w[cat == k] *= t / cur[k]
212 + elif t <= 0:
213 + w[cat == k] *= 0.0
214 + w = np.clip(w, 0.02, 50.0)
215 + w *= S / w.sum()
216 + achieved = {}
217 + for cat, target, name in constraints:
218 + achieved[name] = {str(k): round(float(w[cat == k].sum() / w.sum()), 4) for k in target}
219 + detail = {"enabled": True, "weight": w_m, "bandWeight": wb, "ess": round(ess(w), 0), "constraints": [c[2] for c in constraints], "achieved": achieved,
220 + "marketAsOf": markets.get("asOf"), "marketVolume": markets.get("volume"), "mostSeatsMarket": markets.get("mostSeats"), "majorityMarket": markets.get("majority"),
221 + "seatBandsImpliedMean": {p: v.get("impliedMean") for p, v in (markets.get("seatBands") or {}).items()}}
222 + return Tempering(w, detail)
223 +
224 +
225 +# --------------------------------------------------------------- partielles
226 +def byelection_adjustments(byel: dict | None, ridings: list, trend: list[dict], national_2022: dict[str, float]) -> tuple[np.ndarray, dict]:
227 + """Matrice (R, P) d'ajustements logit : κ × (swing local observé à la partielle − swing national du jour)."""
228 + R, P = len(ridings), len(PARTIES_ALL)
229 + adj = np.zeros((R, P))
230 + detail = {"enabled": False, "ridings": []}
231 + if not byel or not byel.get("enabled") or not trend:
232 + return adj, detail
233 + from ..util import norm_name
234 + by_name = {norm_name(r.name): i for i, r in enumerate(ridings) if getattr(r, "name", None)}
235 + by_date = {t["date"]: t["mean"] for t in trend}
236 + dates = sorted(by_date)
237 + for b in byel.get("byElections", []):
238 + key = norm_name(b.get("riding") or "")
239 + i = by_name.get(key)
240 + if i is None:
241 + i = next((idx for k, idx in by_name.items() if k.startswith(key) and key), None)
242 + if i is None or not b.get("previous"):
243 + continue
244 + d = b["date"]
245 + nat_d = by_date.get(d) or by_date.get(next((x for x in dates if x >= d), dates[-1]))
246 + res = {r["party_id"]: r["share"] for r in b.get("results", []) if r.get("party_id")}
247 + row = {}
248 + for j, pid in enumerate(PARTY_IDS):
249 + if pid not in res or pid not in b["previous"] or pid not in nat_d:
250 + continue
251 + local = _logit(res[pid] / 100) - _logit(b["previous"][pid] / 100)
252 + national = _logit(nat_d[pid] / 100) - _logit(national_2022.get(pid, 1) / 100)
253 + x = _clip(settings.byelection_kappa * (local - national), -1.0, 1.0)
254 + adj[i, j] = x
255 + row[pid] = round(x, 3)
256 + detail["ridings"].append({"code": ridings[i].code, "riding": b.get("riding"), "date": d, "adjLogit": row, "kappa": settings.byelection_kappa})
257 + detail["enabled"] = bool(detail["ridings"])
258 + return adj, detail
modified engine/qc26/model/ridings.py +1 −0
@@ -31,6 +31,7 @@ class RidingInput:
31 31 electors: int = 50000
32 32 pct_french: float | None = None # langue maternelle française (recensement 2021)
33 33 poll_region: str = "reste"
34 + name: str = ""
34 35
35 36
36 37 def simulate_ridings(ridings: list[RidingInput], national_draws: np.ndarray, national_2022: dict[str, float],
modified engine/qc26/model/run.py +131 −50
@@ -1,5 +1,7 @@
1 −"""Exécution d'un run de prévision : sondages → latent → Monte Carlo → sièges →
2 −persistance versionnée → « Qu'est-ce qui a changé ? »."""
1 +"""Exécution d'un run de prévision (modèle 2.0) : sondages → latent → mélange fondamentaux →
2 +Monte Carlo → momentum → sièges → tempérage par les marchés (repondération) → persistance
3 +versionnée → « Qu'est-ce qui a changé ? ». Chaque composante est publiée séparément (summary.components)
4 +et la vue « sondages seuls » (pondération uniforme, sans fondamentaux ni momentum) est conservée."""
3 5 from __future__ import annotations
4 6
5 7 import logging
@@ -17,6 +19,7 @@ from ..db import (Candidate, ForecastHouseEffect, ForecastPartyResult, ForecastR
17 19 from ..parties import PARTY_IDS
18 20 from ..regions import POLL_REGION_LABELS, poll_region_for
19 21 from .aggregate import PARTIES_ALL, Obs, compute_weights, election_day_distribution, house_effects, kalman_latent
22 +from .ensemble import (apply_shift, byelection_adjustments, ess, fundamentals_blend, market_tempering, momentum_shift, whist, wmean, wquantile)
20 23 from .ridings import RidingInput, explain_factors, rating, simulate_ridings
21 24 from .segments import build_segment_model, riding_adjustments
22 25
@@ -57,7 +60,7 @@ def load_ridings(s: Session) -> tuple[list[RidingInput], dict[str, float]]:
57 60 out = [RidingInput(code=r.code, region_id=r.region_id or "inconnue", baseline=baselines.get(r.code, {}),
58 61 incumbent_party=r.incumbent_party_id, incumbent_running=r.incumbent_running, candidates=cands.get(r.code, set()),
59 62 electors=r.electors_2026 or 50000, pct_french=(demo[r.code].pct_french_mt if r.code in demo else None),
60 − poll_region=poll_region_for(r.code))
63 + poll_region=poll_region_for(r.code), name=r.name)
61 64 for r in rows if baselines.get(r.code)]
62 65 national_2022 = {p.id: (p.vote_2022 or 0.0) for p in s.scalars(select(Party)).all()}
63 66 national_2022["aut"] = max(0.5, 100.0 - sum(national_2022.values()))
@@ -70,6 +73,36 @@ def _draw_national(mu: np.ndarray, cov: np.ndarray, n: int, rng: np.random.Gener
70 73 return 100.0 * p / p.sum(axis=1, keepdims=True)
71 74
72 75
76 +def _load_signals(s: Session, cutoff: date) -> dict:
77 + """Signaux disponibles en base (jamais interrogés en ligne pendant un run)."""
78 + out = {"markets": None, "fundamentals": None, "attention": None, "media": None, "byelections": None}
79 + try:
80 + from ..signals import attention, byelections, fundamentals, markets, media
81 + out["markets"] = markets.implied(s)
82 + out["fundamentals"] = fundamentals.fundamentals_summary(s, cutoff)
83 + out["attention"] = attention.attention_summary(s, cutoff)
84 + out["media"] = media.media_summary(s, cutoff)
85 + out["byelections"] = byelections.byelection_summary(s)
86 + except Exception as e: # noqa: BLE001
87 + log.warning("signaux indisponibles : %s", e)
88 + return out
89 +
90 +
91 +def _party_stats(national: np.ndarray, seats: np.ndarray, w: np.ndarray, majority: int, seats_total: int) -> dict:
92 + most = seats.argmax(axis=1)
93 + out = {}
94 + for j, pid in enumerate(PARTIES_ALL):
95 + v, sj = national[:, j], seats[:, j]
96 + out[pid] = {"vote": round(wmean(v, w), 2), "vote_sd": round(float(np.sqrt(np.average((v - wmean(v, w)) ** 2, weights=w))), 2),
97 + "vote80": [round(wquantile(v, w, 0.10), 1), round(wquantile(v, w, 0.90), 1)],
98 + "vote95": [round(wquantile(v, w, 0.025), 1), round(wquantile(v, w, 0.975), 1)],
99 + "seats": int(wquantile(sj.astype(float), w, 0.5)), "seats_mean": round(wmean(sj, w), 1),
100 + "seats80": [int(wquantile(sj.astype(float), w, 0.10)), int(wquantile(sj.astype(float), w, 0.90))],
101 + "seats95": [int(wquantile(sj.astype(float), w, 0.025)), int(wquantile(sj.astype(float), w, 0.975))],
102 + "p_most_seats": round(float(np.average(most == j, weights=w)), 4), "p_majority": round(float(np.average(sj >= majority, weights=w)), 4)}
103 + return out
104 +
105 +
73 106 def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None, publish: bool = True,
74 107 n_sims: int | None = None) -> ForecastRun:
75 108 cutoff = cutoff or date.today()
@@ -84,66 +117,87 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
84 117 obs = load_observations(s, cutoff)
85 118 if len(obs) < 3:
86 119 raise RuntimeError("trop peu de sondages")
120 + days_left = max(0, (settings.election_date - cutoff).days)
87 121 weights = compute_weights(obs, cutoff)
88 122 effects = house_effects(obs, cutoff)
89 123 latent = kalman_latent(obs, cutoff, effects)
90 − mu, cov = election_day_distribution(latent, cutoff, settings.election_date)
91 − national = _draw_national(mu, cov, n_sims, rng) # (S, P) %
124 + signals = _load_signals(s, cutoff)
125 +
126 + # --- 1. vote national : sondages seuls, puis mélange fondamentaux ---
127 + mu_polls, cov_polls = election_day_distribution(latent, cutoff, settings.election_date)
128 + national_polls = _draw_national(mu_polls, cov_polls, n_sims, rng) # (S, P) % — sondages seuls
129 + mu_f, var_f, fund_detail = fundamentals_blend(latent["mean_logit"], latent["var_logit"], latent["last_pct"], signals["fundamentals"], days_left)
130 + if fund_detail["enabled"]:
131 + latent_f = {**latent, "mean_logit": mu_f, "var_logit": var_f}
132 + mu_e, cov_e = election_day_distribution(latent_f, cutoff, settings.election_date)
133 + rng_e = np.random.default_rng(seed) # mêmes aléas → différence attribuable au seul mélange
134 + national = _draw_national(mu_e, cov_e, n_sims, rng_e)
135 + else:
136 + national = national_polls.copy()
137 + # --- 2. momentum (nudge borné, somme nulle) ---
138 + shift, mom_detail = momentum_shift(latent["last_pct"], signals["fundamentals"], signals["attention"], signals["media"])
139 + national = apply_shift(national, shift)
92 140
141 + # --- 3. circonscriptions ---
93 142 ridings, national_2022 = load_ridings(s)
94 143 national_now_mean = {p: float(national[:, j].mean()) for j, p in enumerate(PARTIES_ALL)}
95 144 seg_model, f_r = build_segment_model(s, cutoff, national_now_mean, national_2022)
96 145 seg_adj = riding_adjustments(seg_model, [r.code for r in ridings], f_r)
97 − winners, shares = simulate_ridings(ridings, national, national_2022, rng, seg_adj=seg_adj)
146 + be_adj, be_detail = byelection_adjustments(signals["byelections"], ridings, latent["series"], national_2022)
147 + total_adj = seg_adj + be_adj
148 + rng_r = np.random.default_rng(seed + 1)
149 + winners, shares = simulate_ridings(ridings, national, national_2022, rng_r, seg_adj=total_adj)
98 150 S, R, P = shares.shape
99 151 seats_total = int(get_setting(s, "seats_total", R))
100 152 majority = int(get_setting(s, "majority_threshold", seats_total // 2 + 1))
101 153 seats = np.zeros((S, P), dtype=int)
102 154 for j in range(P):
103 155 seats[:, j] = (winners == j).sum(axis=1)
104 − most = seats.argmax(axis=1)
156 + # sondages seuls (mêmes aléas de circonscription, sans fondamentaux, momentum ni partielles) — pour comparaison
157 + rng_r0 = np.random.default_rng(seed + 1)
158 + winners0, _ = simulate_ridings(ridings, national_polls, national_2022, rng_r0, seg_adj=seg_adj)
159 + seats0 = np.zeros((S, P), dtype=int)
160 + for j in range(P):
161 + seats0[:, j] = (winners0 == j).sum(axis=1)
162 +
163 + # --- 4. tempérage par les marchés : repondération des simulations ---
164 + temp = market_tempering(seats, majority, signals["markets"])
165 + w = temp.weights
166 + uniform = np.ones(S)
105 167
106 − summary_parties = {}
168 + # --- statistiques pondérées ---
169 + most = seats.argmax(axis=1)
170 + summary_parties = _party_stats(national, seats, w, majority, seats_total)
171 + polls_only = _party_stats(national_polls, seats0, uniform, majority, seats_total)
172 + model_only = _party_stats(national, seats, uniform, majority, seats_total) # fondamentaux + momentum, sans marchés
107 173 s.execute(delete(ForecastPartyResult).where(ForecastPartyResult.run_id == run.id))
108 174 for j, pid in enumerate(PARTIES_ALL):
109 − v = national[:, j]
110 − sj = seats[:, j]
111 − hist = np.bincount(sj, minlength=seats_total + 1).tolist()
112 − fpr = ForecastPartyResult(
113 − run_id=run.id, party_id=pid, vote_mean=float(v.mean()), vote_sd=float(v.std()),
114 − vote_lo80=float(np.percentile(v, 10)), vote_hi80=float(np.percentile(v, 90)),
115 − vote_lo95=float(np.percentile(v, 2.5)), vote_hi95=float(np.percentile(v, 97.5)),
116 − seats_mean=float(sj.mean()), seats_median=int(np.median(sj)),
117 − seats_lo80=int(np.percentile(sj, 10)), seats_hi80=int(np.percentile(sj, 90)),
118 − seats_lo95=int(np.percentile(sj, 2.5)), seats_hi95=int(np.percentile(sj, 97.5)),
119 − p_most_seats=float((most == j).mean()), p_majority=float((sj >= majority).mean()),
120 − seat_distribution=hist)
121 − s.add(fpr)
122 − summary_parties[pid] = {"vote": round(fpr.vote_mean, 2), "vote_sd": round(fpr.vote_sd, 2),
123 − "vote80": [round(fpr.vote_lo80, 1), round(fpr.vote_hi80, 1)],
124 − "seats": fpr.seats_median, "seats_mean": round(fpr.seats_mean, 1),
125 − "seats80": [fpr.seats_lo80, fpr.seats_hi80], "seats95": [fpr.seats_lo95, fpr.seats_hi95],
126 − "p_most_seats": round(fpr.p_most_seats, 4), "p_majority": round(fpr.p_majority, 4)}
127 − p_minority = float((seats.max(axis=1) < majority).mean())
175 + v, sj, sp = national[:, j], seats[:, j], summary_parties[pid]
176 + s.add(ForecastPartyResult(
177 + run_id=run.id, party_id=pid, vote_mean=sp["vote"], vote_sd=sp["vote_sd"],
178 + vote_lo80=sp["vote80"][0], vote_hi80=sp["vote80"][1], vote_lo95=sp["vote95"][0], vote_hi95=sp["vote95"][1],
179 + seats_mean=sp["seats_mean"], seats_median=sp["seats"], seats_lo80=sp["seats80"][0], seats_hi80=sp["seats80"][1],
180 + seats_lo95=sp["seats95"][0], seats_hi95=sp["seats95"][1], p_most_seats=sp["p_most_seats"], p_majority=sp["p_majority"],
181 + seat_distribution=whist(sj, w, seats_total + 1)))
182 + p_minority = float(np.average(seats.max(axis=1) < majority, weights=w))
128 183
129 184 # circonscriptions
130 185 s.execute(delete(ForecastRidingResult).where(ForecastRidingResult.run_id == run.id))
131 186 s.execute(delete(ForecastRidingSummary).where(ForecastRidingSummary.run_id == run.id))
132 − national_now = {p: round(float(national[:, j].mean()), 2) for j, p in enumerate(PARTIES_ALL)}
133 − battlegrounds = []
134 − riding_rows = []
187 + national_now = {p: summary_parties[p]["vote"] for p in PARTIES_ALL}
188 + battlegrounds, riding_rows = [], []
189 + be_codes = {x["code"]: x for x in be_detail.get("ridings", [])}
135 190 for i, r in enumerate(ridings):
136 191 sh = shares[:, i, :]
137 − pw = np.array([(winners[:, i] == j).mean() for j in range(P)])
192 + pw = np.array([float(np.average(winners[:, i] == j, weights=w)) for j in range(P)])
138 193 order = np.argsort(-pw)
139 194 fav, second = int(order[0]), int(order[1])
140 − share_mean = {p: float(sh[:, j].mean()) for j, p in enumerate(PARTIES_ALL)}
195 + share_mean = {p: wmean(sh[:, j], w) for j, p in enumerate(PARTIES_ALL)}
141 196 for j, pid in enumerate(PARTIES_ALL):
142 197 if pid == "aut" and share_mean[pid] < 1.5:
143 198 continue
144 199 s.add(ForecastRidingResult(run_id=run.id, riding_code=r.code, party_id=pid, vote_mean=share_mean[pid],
145 − vote_lo80=float(np.percentile(sh[:, j], 10)), vote_hi80=float(np.percentile(sh[:, j], 90)),
146 − p_win=float(pw[j])))
200 + vote_lo80=wquantile(sh[:, j], w, 0.10), vote_hi80=wquantile(sh[:, j], w, 0.90), p_win=float(pw[j])))
147 201 fav_id = PARTIES_ALL[fav]
148 202 margin = share_mean[fav_id] - share_mean[PARTIES_ALL[second]]
149 203 swing = share_mean[fav_id] - r.baseline.get(fav_id, 0.0)
@@ -154,11 +208,21 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
154 208 if r.poll_region in seg_model.current and fav_id in seg_model.current[r.poll_region]:
155 209 seg_detail["region"] = {"label": POLL_REGION_LABELS.get(r.poll_region, r.poll_region), "now": seg_model.current[r.poll_region][fav_id],
156 210 "2022": seg_model.baseline.get(r.poll_region, {}).get(fav_id, 0)}
211 + factors = explain_factors(r, national_now, national_2022, fav_id, share_mean, seg_adj_fav=float(seg_adj[i, fav]), seg_detail=seg_detail)
212 + if r.code in be_codes:
213 + be = be_codes[r.code]
214 + a = be["adjLogit"].get(fav_id, 0.0)
215 + _d = date.fromisoformat(be["date"])
216 + _mois = ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"][_d.month - 1]
217 + factors.insert(2, {"factor": "partielle", "label": f"Élection partielle du {_d.day}{'er' if _d.day == 1 else ''} {_mois} {_d.year} (résultat officiel)",
218 + "detail": f"excès de swing local observé, net du swing national du jour, rétréci (κ {be['kappa']}) : {a:+.2f} logit pour {fav_id.upper()}",
219 + "direction": "+" if a > 0.02 else "-" if a < -0.02 else "±"})
220 + if temp.detail.get("enabled"):
221 + factors.append({"factor": "marches", "label": "Tempérage par les marchés de prédiction (Polymarket)",
222 + "detail": f"simulations repondérées (poids {temp.detail['weight']}) vers les probabilités implicites nationales ; effet indirect sur la circonscription", "direction": "±"})
157 223 summ = ForecastRidingSummary(run_id=run.id, riding_code=r.code, favorite_party_id=fav_id, favorite_p=float(pw[fav]),
158 224 runner_up_party_id=PARTIES_ALL[second], margin_mean=float(margin),
159 − volatility=float(1 - pw[fav]), rating=rating(float(pw[fav])), swing_from_2022=float(swing),
160 − factors=explain_factors(r, national_now, national_2022, fav_id, share_mean,
161 − seg_adj_fav=float(seg_adj[i, fav]), seg_detail=seg_detail))
225 + volatility=float(1 - pw[fav]), rating=rating(float(pw[fav])), swing_from_2022=float(swing), factors=factors)
162 226 s.add(summ)
163 227 riding_rows.append((r.code, fav_id, float(pw[fav]), PARTIES_ALL[second], float(margin)))
164 228 battlegrounds.append((float(1 - pw[fav]), r.code))
@@ -177,7 +241,7 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
177 241 recent_ids = [o.poll_id for o in obs if (cutoff - o.end).days <= 45]
178 242 base_last = {p: float(x) for p, x in zip(PARTIES_ALL, latent["last_pct"])}
179 243 for o in obs:
180 − w = weights[o.poll_id]
244 + wgt = weights[o.poll_id]
181 245 impact = None
182 246 if o.poll_id in recent_ids and len(obs) > 4:
183 247 sub = [x for x in obs if x.poll_id != o.poll_id]
@@ -186,29 +250,38 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
186 250 impact = {p: round(base_last[p] - float(x), 2) for p, x in zip(PARTIES_ALL, lat2["last_pct"])}
187 251 except Exception: # noqa: BLE001
188 252 impact = None
189 − s.add(PollWeight(run_id=run.id, poll_id=o.poll_id, weight=w["weight"], recency_weight=w["recency"],
190 − sample_weight=w["sample"], impact=impact))
253 + s.add(PollWeight(run_id=run.id, poll_id=o.poll_id, weight=wgt["weight"], recency_weight=wgt["recency"],
254 + sample_weight=wgt["sample"], impact=impact))
191 255
192 − # tendance quotidienne (série pour les graphiques) — allégée à 1 point / jour
193 256 trend = latent["series"]
194 −
195 − # résumé + diff avec le run publié précédent
196 257 prev = s.scalar(select(ForecastRun).where(ForecastRun.is_published == True, ForecastRun.kind == "forecast", # noqa: E712
197 258 ForecastRun.id != run.id).order_by(ForecastRun.id.desc()))
198 259 top = sorted(summary_parties.items(), key=lambda kv: -kv[1]["seats_mean"])
260 + components = {
261 + "pollsOnly": {"parties": polls_only, "p_minority": round(float((seats0.max(axis=1) < majority).mean()), 4)},
262 + "modelNoMarkets": {"parties": model_only, "p_minority": round(float((seats.max(axis=1) < majority).mean()), 4)},
263 + "fundamentals": fund_detail, "momentum": mom_detail, "markets": temp.detail, "byelections": be_detail,
264 + "signals": {"attention": {k: v for k, v in (signals["attention"] or {}).items() if k in ("enabled", "asOf", "share7", "share28", "delta")},
265 + "media": {k: v for k, v in (signals["media"] or {}).items() if k in ("enabled", "asOf", "articles", "shareOfVoice", "tone", "counts")},
266 + "fundamentals": {k: v for k, v in (signals["fundamentals"] or {}).items() if k in ("enabled", "asOf", "satisfaction", "satisfactionNet", "bestPmShare", "leaderNetFavorable", "secondChoice", "sovereigntyYes")},
267 + "markets": {k: v for k, v in (signals["markets"] or {}).items() if k in ("asOf", "mostSeats", "majority", "volume")} if signals["markets"] else None,
268 + "byelections": {"n": (signals["byelections"] or {}).get("n", 0), "meanSwing": (signals["byelections"] or {}).get("meanSwing")}},
269 + "ess": round(ess(w), 0), "nSimulations": n_sims,
270 + }
199 271 summary = {
200 272 "seats_total": seats_total, "majority": majority, "parties": summary_parties, "p_minority": round(p_minority, 4),
201 273 "leader": top[0][0], "poll_count": len(obs), "latest_poll": max(o.end for o in obs).isoformat(),
202 − "national_now": national_now, "cutoff": cutoff.isoformat(), "days_to_election": (settings.election_date - cutoff).days,
274 + "national_now": national_now, "cutoff": cutoff.isoformat(), "days_to_election": days_left,
203 275 "battlegrounds": [c for _, c in sorted(battlegrounds, reverse=True)[:15]],
204 276 "segments": seg_model.payload(),
205 277 "demographics_coverage": sum(1 for r in ridings if r.pct_french is not None),
206 278 "ridings": {str(c): {"fav": f, "p": round(p, 3), "second": sec, "margin": round(m, 1)} for c, f, p, sec, m in riding_rows},
279 + "components": components,
207 280 }
208 281 diff = None
209 − if prev and prev.summary:
210 − new_polls = [o.poll_id for o in obs if o.end > date.fromisoformat(prev.summary.get("latest_poll", "1900-01-01")) or
211 − o.poll_id not in {pw.poll_id for pw in s.scalars(select(PollWeight).where(PollWeight.run_id == prev.id)).all()}]
282 + if prev and prev.summary and "parties" in prev.summary:
283 + prev_ids = {pw.poll_id for pw in s.scalars(select(PollWeight).where(PollWeight.run_id == prev.id)).all()}
284 + new_polls = [o.poll_id for o in obs if o.end > date.fromisoformat(prev.summary.get("latest_poll", "1900-01-01")) or o.poll_id not in prev_ids]
212 285 changes = {}
213 286 for pid in PARTIES_ALL:
214 287 a, b = prev.summary["parties"].get(pid), summary_parties[pid]
@@ -224,8 +297,16 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
224 297 causes = []
225 298 if new_polls:
226 299 causes.append({"kind": "new_polls", "detail": new_polls[:6]})
227 − if not new_polls:
228 − causes.append({"kind": "time", "detail": "recalcul sans nouveau sondage (récence, dérive vers le scrutin)"})
300 + prev_mk = ((prev.summary.get("components") or {}).get("markets") or {}).get("mostSeatsMarket")
301 + cur_mk = temp.detail.get("mostSeatsMarket")
302 + if prev_mk and cur_mk:
303 + moved = {p: round(cur_mk.get(p, 0) - prev_mk.get(p, 0), 3) for p in cur_mk if abs(cur_mk.get(p, 0) - prev_mk.get(p, 0)) >= 0.02}
304 + if moved:
305 + causes.append({"kind": "markets", "detail": moved})
306 + if prev.model_version != settings.model_version:
307 + causes.append({"kind": "model", "detail": f"modèle {prev.model_version} → {settings.model_version}"})
308 + if not causes:
309 + causes.append({"kind": "time", "detail": "recalcul sans nouveau sondage (récence, dérive vers le scrutin, signaux mis à jour)"})
229 310 diff = {"previous_run": prev.id, "previous_at": prev.completed_at.isoformat() if prev.completed_at else None,
230 311 "changes": changes, "flips": flips, "causes": causes}
231 312
@@ -239,7 +320,7 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
239 320 run.is_published = True
240 321 set_setting(s, "latest_forecast_run", run.id)
241 322 s.flush()
242 − log.info("run %d : %s — sièges %s", run.id, summary["leader"], {k: v["seats"] for k, v in summary_parties.items()})
323 + log.info("run %d : %s — sièges %s (ESS %s)", run.id, summary["leader"], {k: v["seats"] for k, v in summary_parties.items()}, components["ess"])
243 324 return run
244 325 except Exception as e: # noqa: BLE001
245 326 run.status = "failed"
modified engine/qc26/scheduler.py +52 −0
@@ -92,6 +92,54 @@ def job_documents():
92 92 return {"docs": res, "positions": pos}
93 93
94 94
95 +@_guarded("signals_markets")
96 +def job_signals_markets():
97 + from .signals import markets
98 + with session_scope() as s:
99 + return markets.sync_markets(s, with_history=True)
100 +
101 +
102 +@_guarded("signals_daily")
103 +def job_signals_daily():
104 + """Attention (Wikipédia), fondamentaux (questions de leadership), partielles, GDELT — une fois par jour, puis recalcul."""
105 + from .ingest import official
106 + from .model.run import run_forecast
107 + from .signals import attention, byelections, fundamentals, media
108 + out = {}
109 + with session_scope() as s:
110 + out["attention"] = attention.sync_attention(s)
111 + with session_scope() as s:
112 + out["fundamentals"] = fundamentals.extract_from_polls(s)
113 + with session_scope() as s:
114 + out["byelections"] = byelections.sync_by_elections(s)
115 + official.apply_by_election_incumbents(s)
116 + try:
117 + with session_scope() as s:
118 + out["gdelt"] = media.sync_gdelt(s)
119 + except Exception as e: # noqa: BLE001
120 + out["gdelt"] = {"error": str(e)[:200]}
121 + with session_scope() as s:
122 + run = run_forecast(s, trigger="signals_daily")
123 + out["run"] = run.id
124 + return out
125 +
126 +
127 +@_guarded("signals_media")
128 +def job_signals_media():
129 + from .signals import media
130 + with session_scope() as s:
131 + return media.sync_news(s, classify=True)
132 +
133 +
134 +@_guarded("periodic_forecast")
135 +def job_periodic_forecast():
136 + """Recalcul toutes les 6 h : les marchés bougent en continu, la récence des sondages aussi."""
137 + from .model.run import run_forecast
138 + with session_scope() as s:
139 + run = run_forecast(s, trigger="periodic")
140 + return {"run": run.id}
141 +
142 +
95 143 def _in_live_window() -> bool:
96 144 if settings.election_simulation:
97 145 return True
@@ -126,6 +174,10 @@ def start() -> BackgroundScheduler:
126 174 sch.add_job(job_daily_forecast, "cron", hour=6, minute=15, id="daily_forecast")
127 175 sch.add_job(job_documents, "interval", minutes=settings.doc_watch_minutes, id="documents", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=30))
128 176 sch.add_job(job_live, "interval", seconds=settings.live_poll_seconds, id="live")
177 + sch.add_job(job_signals_markets, "interval", minutes=settings.signals_markets_minutes, id="signals_markets", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=5))
178 + sch.add_job(job_signals_media, "interval", hours=settings.signals_media_hours, id="signals_media", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=12))
179 + sch.add_job(job_signals_daily, "cron", hour=5, minute=40, id="signals_daily")
180 + sch.add_job(job_periodic_forecast, "cron", hour="0,12,18", minute=20, id="periodic_forecast")
129 181 sch.start()
130 182 _sched = sch
131 183 log.info("planificateur démarré")
added engine/qc26/signals/__init__.py +8 −0
@@ -0,0 +1,8 @@
1 +"""Signaux « au-delà des sondages » (modèle 2.0) : marchés de prédiction, attention publique,
2 +couverture médiatique, fondamentaux tirés des questions de leadership, élections partielles.
3 +
4 +Chaque connecteur écrit des observations datées et sourcées dans `signals` (et les tables
5 +dédiées `markets`, `market_prices`, `media_items`, `by_elections`). Le modèle (`model/ensemble.py`)
6 +n'utilise que ce qui est en base : aucun signal n'est inventé, l'absence d'un signal désactive
7 +simplement la composante correspondante.
8 +"""
added engine/qc26/signals/attention.py +113 −0
@@ -0,0 +1,113 @@
1 +"""Attention publique — pages vues Wikipédia (API REST Wikimedia, publique, sans clé).
2 +
3 +Pour chaque parti : page du parti + page de son·sa chef·fe (Wikipédia FR), vues quotidiennes
4 +« user » (robots exclus) depuis le 1er juin 2026. Le signal dérivé est la **part d'attention**
5 +(part des vues du parti dans le total des cinq partis) et son évolution 7 j vs 28 j. C'est un
6 +indicateur de saillance, pas d'intention de vote : il n'entre dans le modèle que comme
7 +composante de momentum bornée.
8 +"""
9 +from __future__ import annotations
10 +
11 +import logging
12 +from datetime import date, datetime, timedelta
13 +from urllib.parse import quote
14 +
15 +import httpx
16 +from sqlalchemy import select
17 +from sqlalchemy.orm import Session
18 +
19 +from ..config import settings
20 +from ..db import Signal, Source, utcnow
21 +from ..parties import PARTY_IDS
22 +from .markets import _upsert_signal
23 +
24 +log = logging.getLogger("qc26.signals.attention")
25 +
26 +# Pages Wikipédia FR (titres exacts) — parti puis chef·fe (registre DGEQ ; pour QS la cheffe
27 +# officielle est peu connue du public : on suit la co-porte-parole parlementaire).
28 +PAGES: dict[str, list[str]] = {
29 + "caq": ["Coalition_avenir_Québec", "Christine_Fréchette"],
30 + "plq": ["Parti_libéral_du_Québec", "Charles_Milliard"],
31 + "pq": ["Parti_québécois", "Paul_St-Pierre_Plamondon"],
32 + "qs": ["Québec_solidaire", "Ruba_Ghazal"],
33 + "pcq": ["Parti_conservateur_du_Québec_(2009)", "Éric_Duhaime"],
34 +}
35 +ELECTION_PAGE = "Élections_générales_québécoises_de_2026"
36 +START = date(2026, 6, 1)
37 +
38 +
39 +def fetch_views(title: str, start: date, end: date) -> dict[date, int]:
40 + url = (f"https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/fr.wikipedia/all-access/user/"
41 + f"{quote(title, safe='')}/daily/{start:%Y%m%d}/{end:%Y%m%d}")
42 + with httpx.Client(timeout=30, headers={"User-Agent": "QC26Bot/1.0 (https://www.qc26.ca; contact@qc26.ca)"}) as c:
43 + r = c.get(url)
44 + if r.status_code == 404:
45 + return {}
46 + r.raise_for_status()
47 + items = r.json().get("items", [])
48 + return {datetime.strptime(i["timestamp"][:8], "%Y%m%d").date(): int(i["views"]) for i in items}
49 +
50 +
51 +def sync_attention(s: Session) -> dict:
52 + src = s.get(Source, "wikipedia-pageviews")
53 + if not src:
54 + src = Source(id="wikipedia-pageviews", name="Wikimedia — pages vues Wikipédia FR (partis, chef·fes)", url="https://wikimedia.org/api/rest_v1/", tier=3, kind="signal")
55 + s.add(src)
56 + src.last_checked_at = utcnow()
57 + end = date.today() - timedelta(days=1)
58 + n, errors = 0, 0
59 + # reprise : on repart 10 jours avant la dernière observation connue
60 + last = s.scalar(select(Signal.date).where(Signal.source == "wikipedia", Signal.key == "pageviews").order_by(Signal.date.desc()))
61 + start = max(START, (last - timedelta(days=10))) if last else START
62 + for pid, titles in PAGES.items():
63 + total: dict[date, int] = {}
64 + detail: dict[str, dict[date, int]] = {}
65 + for t in titles:
66 + try:
67 + v = fetch_views(t, start, end)
68 + detail[t] = v
69 + for d, x in v.items():
70 + total[d] = total.get(d, 0) + x
71 + except Exception as e: # noqa: BLE001
72 + errors += 1
73 + log.warning("pageviews %s : %s", t, e)
74 + for d, x in total.items():
75 + _upsert_signal(s, "wikipedia", "pageviews", pid, d, float(x), url=f"https://fr.wikipedia.org/wiki/{titles[0]}",
76 + meta={"pages": {t: detail[t].get(d) for t in detail}})
77 + n += 1
78 + try:
79 + for d, x in fetch_views(ELECTION_PAGE, start, end).items():
80 + _upsert_signal(s, "wikipedia", "pageviews_election", None, d, float(x), url=f"https://fr.wikipedia.org/wiki/{ELECTION_PAGE}")
81 + except Exception as e: # noqa: BLE001
82 + errors += 1
83 + log.warning("pageviews élection : %s", e)
84 + src.last_ok_at, src.status = utcnow(), "operational" if not errors else "degraded"
85 + src.detail = f"{n} observations parti-jour"
86 + return {"observations": n, "errors": errors, "from": start.isoformat(), "to": end.isoformat()}
87 +
88 +
89 +def attention_summary(s: Session, cutoff: date | None = None) -> dict:
90 + """Part d'attention 7 j, 28 j et variation, par parti ; série quotidienne des parts."""
91 + cutoff = cutoff or date.today()
92 + rows = s.execute(select(Signal.party_id, Signal.date, Signal.value).where(Signal.source == "wikipedia", Signal.key == "pageviews", Signal.date <= cutoff)).all()
93 + if not rows:
94 + return {"enabled": False}
95 + by_day: dict[date, dict[str, float]] = {}
96 + for pid, d, v in rows:
97 + by_day.setdefault(d, {})[pid] = v
98 + days = sorted(by_day)
99 + series = []
100 + for d in days:
101 + tot = sum(by_day[d].values()) or 1
102 + series.append({"date": d.isoformat(), "share": {p: round(100 * by_day[d].get(p, 0) / tot, 2) for p in PARTY_IDS}, "views": {p: by_day[d].get(p, 0) for p in PARTY_IDS}})
103 + last = days[-1]
104 + def window(n: int) -> dict[str, float]:
105 + w = [d for d in days if (last - d).days < n]
106 + tot = {p: sum(by_day[d].get(p, 0) for d in w) for p in PARTY_IDS}
107 + all_ = sum(tot.values()) or 1
108 + return {p: 100 * tot[p] / all_ for p in PARTY_IDS}
109 + s7, s28 = window(7), window(28)
110 + election = s.execute(select(Signal.date, Signal.value).where(Signal.source == "wikipedia", Signal.key == "pageviews_election", Signal.date <= cutoff).order_by(Signal.date)).all()
111 + return {"enabled": True, "asOf": last.isoformat(), "share7": {p: round(v, 2) for p, v in s7.items()}, "share28": {p: round(v, 2) for p, v in s28.items()},
112 + "delta": {p: round(s7[p] - s28[p], 2) for p in PARTY_IDS}, "series": series[-120:],
113 + "election": [{"date": d.isoformat(), "views": v} for d, v in election][-120:], "pages": PAGES}
added engine/qc26/signals/byelections.py +177 −0
Large diff (177 lines) is not loaded by default to keep the page fast.Load diff
added engine/qc26/signals/fundamentals.py +230 −0
Large diff (230 lines) is not loaded by default to keep the page fast.Load diff
added engine/qc26/signals/markets.py +226 −0
Large diff (226 lines) is not loaded by default to keep the page fast.Load diff
added engine/qc26/signals/media.py +267 −0
Large diff (267 lines) is not loaded by default to keep the page fast.Load diff
modified engine/qc26/util.py +18 −0
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/boussole/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/carte/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/circonscription/[slug]/page.tsx +6 −6
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/circonscriptions/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/documents/[id]/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/documents/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/enjeux/[topic]/page.tsx +3 −3
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/enjeux/page.tsx +4 −4
Large diff (26 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/globals.css +166 −46
Large diff (273 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/layout.tsx +21 −15
Large diff (72 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/live/model-vs-reality/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/live/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/methodologie/changelog/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/methodologie/page.tsx +15 −5
Large diff (46 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/page.tsx +54 −29
Large diff (168 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/partis/[id]/page.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/partis/comparateur/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/partis/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/plus/page.tsx +3 −3
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/prevision/page.tsx +58 −20
Large diff (117 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/replay/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
added web/src/app/signaux/page.tsx +69 −0
Large diff (69 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/sitemap.ts +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/sondages/[id]/page.tsx +5 −5
Large diff (34 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/sondages/page.tsx +9 −11
Large diff (42 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/statut/page.tsx +4 −4
Large diff (20 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/ask-box.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/charts.tsx +162 −49
Large diff (323 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/compass.tsx +7 −7
Large diff (45 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/forecast-blocks.tsx +125 −83
Large diff (319 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/forecast-history.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/live.tsx +7 −7
Large diff (50 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/map-section.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/map.tsx +16 −16
Large diff (70 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/party.tsx +3 −3
Large diff (22 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/polls-table.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/replay.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/riding-live.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/riding-map.tsx +2 −2
Large diff (14 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/ridings-grid.tsx +3 −3
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/shell.tsx +50 −38
Large diff (174 lines) is not loaded by default to keep the page fast.Load diff
added web/src/components/signals.tsx +284 −0
Large diff (284 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/status-strip.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
added web/src/components/theme.tsx +35 −0
Large diff (35 lines) is not loaded by default to keep the page fast.Load diff
added web/src/components/trend-chart.tsx +115 −0
Large diff (115 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/ui.tsx +45 −22
Large diff (120 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/lib/api.ts +24 −1
Large diff (42 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/lib/parties.ts +11 −0
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere