SPB Git
spboucher.ai

spb/qc26

Public

QC26 2.0.0 — au-delà des sondages + refonte visuelle

Moteur : signaux (Polymarket, Wikipédia, presse Serper+LLM, GDELT, questions de leadership, partielles DGEQ),
ensemble (prior fondamentaux, momentum borné, tempérage par repondération IPF vers les marchés, ajustement
partielles), statistiques pondérées, summary.components (sondages seuls / sans marchés / ensemble), API
/api/signals*, /api/forecast/components, jobs planifiés, admin signals/refresh, changelog 2.0.0.
Correctifs : sortants depuis la liste DGEQ des candidatures (5 circonscriptions de partielles erronées),
chemins d'archive relocalisables, horodatages UTC explicites (TZDateTime).
Web : design system clair/sombre (Fraunces, Inter, JetBrains Mono, accent sarcelle), couleurs de partis en
variables CSS, héros éditorial avec gaufre des 127 sièges, comparaison sondages/marchés/ensemble, page
/signaux, tendance interactive, refonte nav/mobile/footer, méthodologie v2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Sep 7, 2026) parent 570fe76 Browse files

70 changed files +3,279 −634

70 changed files
Load all 70 files Unified Split
modified CLAUDE.md +7 −2
@@ -12,11 +12,16 @@ Spécification produit complète : voir la mission QC26 (design « Editorial Dat
12 ## Où sont les choses 12 ## Où sont les choses
13 - Moteur : `engine/` (FastAPI :8201). Base SQLite `engine/data/qc26.db` (WAL). Archives `engine/data/archive/{polls,documents,live}` par SHA-256. Copies officielles DGEQ `engine/data/official/`. 13 - Moteur : `engine/` (FastAPI :8201). Base SQLite `engine/data/qc26.db` (WAL). Archives `engine/data/archive/{polls,documents,live}` par SHA-256. Copies officielles DGEQ `engine/data/official/`.
14 - Web : `web/` (Next 16, :8200), rewrite `/api/*` → moteur. Composants du design system dans `web/src/components/`. 14 - Web : `web/` (Next 16, :8200), rewrite `/api/*` → moteur. Composants du design system dans `web/src/components/`.
15−- Modèle : `engine/qc26/model/` (aggregate = poids/effets maison/Kalman ; ridings = swing proportionnel + erreurs ; run = orchestration, diff, impacts). Changer un paramètre = bump `model_version` + entrée `CHANGELOG` dans `api/public.py`. 15+- Modèle : `engine/qc26/model/` (aggregate = poids/effets maison/Kalman ; ridings = swing proportionnel + erreurs ; ensemble = v2 : prior fondamentaux, momentum borné, tempérage par les marchés via repondération IPF des simulations, ajustement partielles ; run = orchestration, stats pondérées, `summary.components`, diff). Changer un paramètre = bump `model_version` + entrée `CHANGELOG` dans `api/public.py`.
16+- Signaux (v2) : `engine/qc26/signals/` — markets (Polymarket Gamma/CLOB), attention (Wikimedia pageviews), media (Serper news + classement LLM ; GDELT best-effort, 1 req/6,5 s), fundamentals (questions de leadership des sondages → satisfaction, meilleur PM, favorabilité…), byelections (DGEQ via Firecrawl, code URL = code DGEQ carte 2017). Tables `signals`, `markets`, `market_prices`, `media_items`, `by_elections`. API `/api/signals*`, `/api/forecast/components`, admin `POST /api/admin/signals/refresh?what=all`. Page `/signaux`.
17+- Design : tokens clair/sombre dans `web/src/app/globals.css` (`data-theme`, script inline `THEME_INIT_SCRIPT`), Fraunces (display) + Inter + JetBrains Mono ; couleurs de partis via `partyVar()/partyAlpha()` (variables CSS `--p-*`, éclaircies en sombre) — `partyColor()` (hex) réservé aux images OG.
16 - Soirée électorale : `engine/qc26/live/` (feed officiel, simulation 2022, nowcast). `POST /api/admin/live/simulation/start` rejoue 2022 en accéléré pour tester la chaîne complète. 18 - Soirée électorale : `engine/qc26/live/` (feed officiel, simulation 2022, nowcast). `POST /api/admin/live/simulation/start` rejoue 2022 en accéléré pour tester la chaîne complète.
17 19
18 ## Gotchas 20 ## Gotchas
19−- SQLite : ne jamais tenir une transaction ouverte pendant un appel LLM (commits explicites dans les connecteurs) ; timeout 90 s. 21+- SQLite : ne jamais tenir une transaction ouverte pendant un appel LLM (commits explicites dans les connecteurs) ; timeout 90 s. Les `DateTime` passent par `TZDateTime` (UTC explicite à la relecture) : toujours `.isoformat()` sur des objets tz-aware.
22+- Chemins d'archive stockés absolus : lire via `util.resolve_archive_path()` (la base a été créée sur le laptop puis déplacée).
23+- Sortant·e = drapeau `depute_sortant` de la liste DGEQ des candidatures, puis gagnant de partielle, puis gagnant 2022 transposé (jamais l'inverse).
24+- Fraunces via `next/font/google` : `axes` seulement sans `weight` (police variable).
20 - DGEQ : « PCQ » = Parti communiste, le Parti conservateur est « PCOQ » ; CSV en cp1252 avec tiret 0x96 (`util.fix_dgeq_dash`). 25 - DGEQ : « PCQ » = Parti communiste, le Parti conservateur est « PCOQ » ; CSV en cp1252 avec tiret 0x96 (`util.fix_dgeq_dash`).
21 - Wikipédia n'est qu'un index d'amorçage (tier 5) : les lignes `secondary-index` sont remplacées par le PDF de la firme dès qu'il est archivé ; les URL médias (La Presse, L'actualité, X…) ne sont pas des documents primaires. 26 - Wikipédia n'est qu'un index d'amorçage (tier 5) : les lignes `secondary-index` sont remplacées par le PDF de la firme dès qu'il est archivé ; les URL médias (La Presse, L'actualité, X…) ne sont pas des documents primaires.
22 - La règle « pas de candidature → part résiduelle » ne s'active que pour un parti dont la liste est ≥ 90 % complète. 27 - La règle « pas de candidature → part résiduelle » ne s'active que pour un parti dont la liste est ≥ 90 % complète.
modified README.md +5 −0
@@ -50,3 +50,8 @@ Le premier démarrage sur base vide télécharge les données officielles, amorc
50 ## Production 50 ## Production
51 51
52 Déployé par `mld` (passerelle M1M32) : `mld stage <dir> qc26` → `mld deploy qc26`. Deux processus PM2 (`qc26-engine` :8201, `qc26-web` :8200) + `qc26-ngrok` (www.qc26.ca → 8200). Le web réécrit `/api/*` vers le moteur. 52 Déployé par `mld` (passerelle M1M32) : `mld stage <dir> qc26` → `mld deploy qc26`. Deux processus PM2 (`qc26-engine` :8201, `qc26-web` :8200) + `qc26-ngrok` (www.qc26.ca → 8200). Le web réécrit `/api/*` vers le moteur.
53+
54+
55+## Modèle 2.0 (2026-09-07) — au-delà des sondages
56+
57+Quatre familles de signaux indépendants, archivés avec leur provenance et publiés séparément (`/signaux`, `summary.components`) : marchés de prédiction Polymarket (repondération des simulations, poids 30 %), fondamentaux (satisfaction du gouvernement → prior sur le parti sortant, poids décroissant vers 0 le jour du vote), momentum borné (prime au chef, attention Wikipédia, ton de presse classé par LLM), élections partielles depuis 2022 (excès de swing local). La vue « sondages seuls » reste publiée. Refonte visuelle « Editorial Data Intelligence » : thème clair/sombre, Fraunces + Inter + JetBrains Mono, gaufre des 127 sièges, tendance interactive.
modified deploy/qc26.manifest.json +125 −123
@@ -1,127 +1,129 @@
1 { 1 {
2− "app": "qc26", 2+ "app": "qc26",
3− "label": "QC26 — L'élection québécoise 2026, en données", 3+ "label": "QC26 — L'élection québécoise 2026, en données",
4− "domain": "www.qc26.ca", 4+ "domain": "www.qc26.ca",
5− "port": 8200, 5+ "port": 8200,
6− "health_path": "/healthz", 6+ "health_path": "/healthz",
7− "dir": "~/apps/qc26", 7+ "dir": "~/apps/qc26",
8− "extra_paths": [], 8+ "extra_paths": [],
9− "sync_excludes": [ 9+ "sync_excludes": [
10− "engine/.venv/", 10+ "engine/.venv/",
11− "__pycache__/", 11+ "__pycache__/",
12− ".pytest_cache/", 12+ ".pytest_cache/",
13− ".ruff_cache/", 13+ ".ruff_cache/",
14− "*.egg-info/", 14+ "*.egg-info/",
15− ".git/", 15+ ".git/",
16− "web/node_modules/", 16+ "web/node_modules/",
17− "web/.next/", 17+ "web/.next/",
18− "engine/logs/", 18+ "engine/logs/",
19− "logs/", 19+ "logs/",
20− ".DS_Store", 20+ ".DS_Store",
21− "engine/data/qc26.db", 21+ "engine/data/qc26.db",
22− "engine/data/qc26.db-shm", 22+ "engine/data/qc26.db-shm",
23− "engine/data/qc26.db-wal", 23+ "engine/data/qc26.db-wal",
24− "engine/data/archive/", 24+ "engine/data/archive/",
25− "engine/data/cache/" 25+ "engine/data/cache/"
26+ ],
27+ "requires": {
28+ "runtimes": [
29+ "pm2",
30+ "ngrok",
31+ "node",
32+ "pnpm",
33+ "uv",
34+ "uv-python@3.12"
26 ], 35 ],
27− "requires": { 36+ "ram_gb": 4,
28− "runtimes": [ 37+ "ports": [
29− "pm2", 38+ 8200,
30− "ngrok", 39+ 8201
31− "node", 40+ ]
32− "pnpm", 41+ },
33− "uv", 42+ "ram_mb_observed": 1200,
34− "uv-python@3.12" 43+ "size_mb": 120,
35− ], 44+ "placement": {
36− "ram_gb": 4, 45+ "pin": null,
37− "ports": [ 46+ "prefer": "M3U96a",
38− 8200, 47+ "avoid": [
39− 8201 48+ "M3U96b",
40− ] 49+ "M1M32"
41− },
42− "ram_mb_observed": 1200,
43− "size_mb": 120,
44− "placement": {
45− "pin": null,
46− "prefer": "M3U96a",
47− "avoid": [
48− "M3U96b",
49− "M1M32"
50− ],
51− "reason": "Monte Carlo numpy + build Next.js : cœurs et RAM ; uv-py3.12 + pnpm présents ; M3U96b réservé hfmarketdata"
52− },
53− "processes": [
54− {
55− "name": "qc26-engine",
56− "manager": "pm2",
57− "script": "{{HOME}}/apps/qc26/engine/.venv/bin/python",
58− "args": [
59− "-m",
60− "uvicorn",
61− "qc26.main:app",
62− "--host",
63− "127.0.0.1",
64− "--port",
65− "8201",
66− "--no-access-log"
67− ],
68− "interpreter": null,
69− "cwd": "{{HOME}}/apps/qc26/engine",
70− "env": {
71− "QC26_ENVIRONMENT": "production",
72− "QC26_PORT": "8201",
73− "QC26_PUBLIC_URL": "https://www.qc26.ca",
74− "QC26_ADMIN_TOKEN": "qc26-admin-2026-Kp8vT3nQ7zLw",
75− "QC26_FIRECRAWL_API_KEY": "fc-8cc44cb683044aee82e8cbad6505a01a",
76− "QC26_ANTHROPIC_API_KEY": "__SET_AT_DEPLOY__",
77− "QC26_LLM_MODEL": "claude-opus-5",
78− "QC26_N_SIMULATIONS": "20000",
79− "PYTHONUNBUFFERED": "1"
80− },
81− "cron_restart": null,
82− "autorestart": true,
83− "max_memory_restart": "3G"
84− },
85− {
86− "name": "qc26-web",
87− "manager": "pm2",
88− "script": "/opt/homebrew/bin/node",
89− "args": [
90− "node_modules/next/dist/bin/next",
91− "start",
92− "-p",
93− "8200",
94− "-H",
95− "0.0.0.0"
96− ],
97− "interpreter": null,
98− "cwd": "{{HOME}}/apps/qc26/web",
99− "env": {
100− "NODE_ENV": "production",
101− "QC26_ENGINE_URL": "http://127.0.0.1:8201",
102− "NEXT_PUBLIC_SITE_URL": "https://www.qc26.ca",
103− "NEXT_TELEMETRY_DISABLED": "1"
104− },
105− "cron_restart": null,
106− "autorestart": true,
107− "max_memory_restart": "1500M"
108− }
109 ], 50 ],
110− "ngrok": { 51+ "reason": "Monte Carlo numpy + build Next.js : cœurs et RAM ; uv-py3.12 + pnpm présents ; M3U96b réservé hfmarketdata"
111− "name": "qc26-ngrok", 52+ },
112− "url": "www.qc26.ca", 53+ "processes": [
113− "port": 8200 54+ {
55+ "name": "qc26-engine",
56+ "manager": "pm2",
57+ "script": "{{HOME}}/apps/qc26/engine/.venv/bin/python",
58+ "args": [
59+ "-m",
60+ "uvicorn",
61+ "qc26.main:app",
62+ "--host",
63+ "127.0.0.1",
64+ "--port",
65+ "8201",
66+ "--no-access-log"
67+ ],
68+ "interpreter": null,
69+ "cwd": "{{HOME}}/apps/qc26/engine",
70+ "env": {
71+ "QC26_ENVIRONMENT": "production",
72+ "QC26_PORT": "8201",
73+ "QC26_PUBLIC_URL": "https://www.qc26.ca",
74+ "QC26_ADMIN_TOKEN": "qc26-admin-2026-Kp8vT3nQ7zLw",
75+ "QC26_FIRECRAWL_API_KEY": "fc-8cc44cb683044aee82e8cbad6505a01a",
76+ "QC26_ANTHROPIC_API_KEY": "__SET_AT_DEPLOY__",
77+ "QC26_LLM_MODEL": "claude-opus-5",
78+ "QC26_N_SIMULATIONS": "20000",
79+ "PYTHONUNBUFFERED": "1",
80+ "QC26_SERPER_API_KEY": "__SET_AT_DEPLOY__",
81+ "QC26_TAVILY_API_KEY": "__SET_AT_DEPLOY__"
82+ },
83+ "cron_restart": null,
84+ "autorestart": true,
85+ "max_memory_restart": "3G"
114 }, 86 },
115− "launchd": [],
116− "env_overrides": {},
117− "hooks": {
118− "post_sync": [
119− "export PATH=\"$HOME/.local/bin:/opt/homebrew/bin:$PATH\"; cd engine && (test -x .venv/bin/python || uv venv --python 3.12 .venv) && uv pip install -q --python .venv/bin/python -e . && echo ' engine deps ok'",
120− "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && pnpm install --frozen-lockfile --silent && echo ' web deps ok'",
121− "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && QC26_ENGINE_URL=http://127.0.0.1:8201 NEXT_PUBLIC_SITE_URL=https://www.qc26.ca NEXT_TELEMETRY_DISABLED=1 pnpm build 2>&1 | tail -3 && echo ' web build ok'",
122− "mkdir -p engine/data/archive/polls engine/data/archive/documents engine/data/archive/live engine/data/cache engine/logs && echo ' data ok'"
123− ],
124− "post_start": []
125− },
126− "notes": "Après le premier déploiement (2026-09-06), la base et les archives du nœud sont la source de vérité : exclues des syncs suivants."
127−}
\ No newline at end of file
87+ {
88+ "name": "qc26-web",
89+ "manager": "pm2",
90+ "script": "/opt/homebrew/bin/node",
91+ "args": [
92+ "node_modules/next/dist/bin/next",
93+ "start",
94+ "-p",
95+ "8200",
96+ "-H",
97+ "0.0.0.0"
98+ ],
99+ "interpreter": null,
100+ "cwd": "{{HOME}}/apps/qc26/web",
101+ "env": {
102+ "NODE_ENV": "production",
103+ "QC26_ENGINE_URL": "http://127.0.0.1:8201",
104+ "NEXT_PUBLIC_SITE_URL": "https://www.qc26.ca",
105+ "NEXT_TELEMETRY_DISABLED": "1"
106+ },
107+ "cron_restart": null,
108+ "autorestart": true,
109+ "max_memory_restart": "1500M"
110+ }
111+ ],
112+ "ngrok": {
113+ "name": "qc26-ngrok",
114+ "url": "www.qc26.ca",
115+ "port": 8200
116+ },
117+ "launchd": [],
118+ "env_overrides": {},
119+ "hooks": {
120+ "post_sync": [
121+ "export PATH=\"$HOME/.local/bin:/opt/homebrew/bin:$PATH\"; cd engine && (test -x .venv/bin/python || uv venv --python 3.12 .venv) && uv pip install -q --python .venv/bin/python -e . && echo ' engine deps ok'",
122+ "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && pnpm install --frozen-lockfile --silent && echo ' web deps ok'",
123+ "export PATH=\"/opt/homebrew/bin:$PATH\"; cd web && QC26_ENGINE_URL=http://127.0.0.1:8201 NEXT_PUBLIC_SITE_URL=https://www.qc26.ca NEXT_TELEMETRY_DISABLED=1 pnpm build 2>&1 | tail -3 && echo ' web build ok'",
124+ "mkdir -p engine/data/archive/polls engine/data/archive/documents engine/data/archive/live engine/data/cache engine/logs && echo ' data ok'"
125+ ],
126+ "post_start": []
127+ },
128+ "notes": "Après le premier déploiement (2026-09-06), la base et les archives du nœud sont la source de vérité : exclues des syncs suivants. v2.0 (2026-09-07) : clés Serper/Tavily requises pour la revue de presse ; les tables signaux sont créées au démarrage (create_all)."
129+}
modified engine/qc26/api/admin.py +26 −0
@@ -110,6 +110,32 @@ def compass_positions(only_missing: bool = True, s: Session = Depends(db)):
110 return positions.compute_positions(s, only_missing=only_missing) 110 return positions.compute_positions(s, only_missing=only_missing)
111 111
112 112
113+@router.post("/signals/refresh", dependencies=[Depends(require_admin)])
114+def signals_refresh(what: str = "all", run: bool = True, s: Session = Depends(db)):
115+ """Rafraîchit les signaux (markets | attention | fundamentals | byelections | news | gdelt | all) puis recalcule."""
116+ from ..ingest import official
117+ from ..signals import attention, byelections, fundamentals, markets, media
118+ out = {}
119+ parts = [x.strip() for x in what.split(",")] if what != "all" else ["markets", "attention", "fundamentals", "byelections", "news"]
120+ if "markets" in parts:
121+ out["markets"] = markets.sync_markets(s); s.commit()
122+ if "attention" in parts:
123+ out["attention"] = attention.sync_attention(s); s.commit()
124+ if "fundamentals" in parts:
125+ out["fundamentals"] = fundamentals.extract_from_polls(s); s.commit()
126+ if "byelections" in parts:
127+ out["byelections"] = byelections.sync_by_elections(s); official.apply_by_election_incumbents(s); s.commit()
128+ if "news" in parts:
129+ out["news"] = media.sync_news(s, classify=True); s.commit()
130+ if "gdelt" in parts:
131+ out["gdelt"] = media.sync_gdelt(s); s.commit()
132+ if run:
133+ from ..model.run import run_forecast
134+ r = run_forecast(s, trigger="admin:signals")
135+ out["run"] = r.id
136+ return out
137+
138+
113 @router.get("/review", dependencies=[Depends(require_admin)]) 139 @router.get("/review", dependencies=[Depends(require_admin)])
114 def review_queue(s: Session = Depends(db)): 140 def review_queue(s: Session = Depends(db)):
115 polls = s.scalars(select(Poll).where(Poll.validated == False).order_by(desc(Poll.fieldwork_end))).all() # noqa: E712 141 polls = s.scalars(select(Poll).where(Poll.validated == False).order_by(desc(Poll.fieldwork_end))).all() # noqa: E712
modified engine/qc26/api/public.py +74 −4
@@ -389,7 +389,11 @@ def poll_document_file(sha: str, s: Session = Depends(db)):
389 doc = s.scalar(select(PollDocument).where(PollDocument.sha256 == sha)) 389 doc = s.scalar(select(PollDocument).where(PollDocument.sha256 == sha))
390 if not doc: 390 if not doc:
391 raise HTTPException(404) 391 raise HTTPException(404)
392− return FileResponse(doc.path, media_type=doc.content_type or "application/octet-stream", filename=f"qc26-archive-{sha[:12]}{'.pdf' if doc.path.endswith('.pdf') else '.html'}") 392+ from ..util import resolve_archive_path
393+ fp = resolve_archive_path(doc.path)
394+ if not fp or not fp.exists():
395+ raise HTTPException(404, "fichier d'archive introuvable")
396+ return FileResponse(str(fp), media_type=doc.content_type or "application/octet-stream", filename=f"qc26-archive-{sha[:12]}{'.pdf' if doc.path.endswith('.pdf') else '.html'}")
393 397
394 398
395 @router.get("/documents/{doc_id}") 399 @router.get("/documents/{doc_id}")
@@ -413,7 +417,11 @@ def document_file(doc_id: int, vid: int, s: Session = Depends(db)):
413 v = s.get(DocumentVersion, vid) 417 v = s.get(DocumentVersion, vid)
414 if not v or v.document_id != doc_id: 418 if not v or v.document_id != doc_id:
415 raise HTTPException(404) 419 raise HTTPException(404)
416− return FileResponse(v.path, media_type=v.content_type or "application/octet-stream") 420+ from ..util import resolve_archive_path
421+ fp = resolve_archive_path(v.path)
422+ if not fp or not fp.exists():
423+ raise HTTPException(404, "fichier d'archive introuvable")
424+ return FileResponse(str(fp), media_type=v.content_type or "application/octet-stream")
417 425
418 426
419 @router.get("/proposals") 427 @router.get("/proposals")
@@ -623,7 +631,8 @@ def replay(t: str | None = None, s: Session = Depends(db)):
623 if not target: 631 if not target:
624 raise HTTPException(404) 632 raise HTTPException(404)
625 from pathlib import Path 633 from pathlib import Path
626− data = json.loads(Path(target.path).read_text()) 634+ from ..util import resolve_archive_path
635+ data = json.loads((resolve_archive_path(target.path) or Path(target.path)).read_text())
627 ridings_by_name = {} 636 ridings_by_name = {}
628 from ..util import fix_dgeq_dash, norm_name 637 from ..util import fix_dgeq_dash, norm_name
629 for r in s.scalars(select(Riding)).all(): 638 for r in s.scalars(select(Riding)).all():
@@ -678,7 +687,8 @@ def status(s: Session = Depends(db)):
678 return {"components": {"pollWatcher": "operational" if poll_srcs and all(x.status == "operational" for x in poll_srcs) else ("degraded" if poll_srcs else "ready"), 687 return {"components": {"pollWatcher": "operational" if poll_srcs and all(x.status == "operational" for x in poll_srcs) else ("degraded" if poll_srcs else "ready"),
679 "forecastEngine": "operational" if run and run.status == "ok" else "degraded", 688 "forecastEngine": "operational" if run and run.status == "ok" else "degraded",
680 "electionFeed": next((x.status for x in sources if x.id == "eq-live"), "ready"), 689 "electionFeed": next((x.status for x in sources if x.id == "eq-live"), "ready"),
681− "documentWatcher": comp("party-docs"), "officialData": "operational" if all(x.status == "operational" for x in sources if x.kind == "official") else "degraded"}, 690+ "documentWatcher": comp("party-docs"), "officialData": "operational" if all(x.status == "operational" for x in sources if x.kind == "official") else "degraded",
691+ "signals": ("operational" if [x for x in sources if x.kind == "signal"] and all(x.status in ("operational", "ready") for x in sources if x.kind == "signal") else ("degraded" if [x for x in sources if x.kind == "signal"] else "ready"))},
682 "sources": [{"id": x.id, "name": x.name, "url": x.url, "tier": x.tier, "kind": x.kind, "status": x.status, "lastCheckedAt": x.last_checked_at.isoformat() if x.last_checked_at else None, 692 "sources": [{"id": x.id, "name": x.name, "url": x.url, "tier": x.tier, "kind": x.kind, "status": x.status, "lastCheckedAt": x.last_checked_at.isoformat() if x.last_checked_at else None,
683 "lastOkAt": x.last_ok_at.isoformat() if x.last_ok_at else None, "detail": x.detail} for x in sources], 693 "lastOkAt": x.last_ok_at.isoformat() if x.last_ok_at else None, "detail": x.detail} for x in sources],
684 "jobs": [{"id": j.id, "kind": j.kind, "status": j.status, "startedAt": j.started_at.isoformat(), "finishedAt": j.finished_at.isoformat() if j.finished_at else None, 694 "jobs": [{"id": j.id, "kind": j.kind, "status": j.status, "startedAt": j.started_at.isoformat(), "finishedAt": j.finished_at.isoformat() if j.finished_at else None,
@@ -688,6 +698,13 @@ def status(s: Session = Depends(db)):
688 698
689 699
690 CHANGELOG = [ 700 CHANGELOG = [
701+ {"version": "2.0.0", "date": "2026-09-07", "title": "Au-delà des sondages : ensemble à quatre composantes",
702+ "notes": ["Marchés de prédiction (Polymarket, ~800 k$ de volume) : marchés « plus de sièges », « majorité PQ », tranches de sièges (PQ, CAQ, PLQ), 2e/3e place, historique quotidien archivé. Les 20 000 simulations sont repondérées (ajustement proportionnel itératif) pour que les probabilités de l'ensemble valent 70 % modèle + 30 % marché (15 % pour les tranches, moins liquides). La vue « sondages seuls » reste publiée.",
703+ "Fondamentaux : la satisfaction à l'égard du gouvernement (questions de leadership des rapports de sondage, pondérées récence) fixe un prior sur la part du parti sortant (≈ 6 + 0,6 × satisfaction, σ 5,5 pt) mélangé aux sondages avec un poids qui décroît de 20 % à 120 jours du vote jusqu'à 0 le jour J.",
704+ "Momentum borné (± 1 pt, somme nulle) : prime au chef (meilleur premier ministre vs intention de vote, κ 0,15), attention publique (pages vues Wikipédia des partis et chef·fes, part 7 j vs 28 j, κ 0,08), ton médiatique (titres de presse classés par LLM, κ 0,5, atténué sous 10 articles).",
705+ "Élections partielles depuis 2022 (Saint-Henri–Sainte-Anne, Jean-Talon, Terrebonne, Arthabaska, Chicoutimi — résultats officiels DGEQ archivés) : l'excès de swing local observé, net du swing national du jour, corrige le prior de la circonscription (κ 0,35).",
706+ "Correction : le·la député·e sortant·e vient désormais de la liste officielle des candidatures (drapeau DGEQ) puis du résultat des partielles, et non plus du seul gagnant 2022 transposé (Jean-Talon, Terrebonne, Arthabaska–L'Érable, Chicoutimi, Saint-Henri–Sainte-Anne étaient erronés).",
707+ "Publication de chaque composante (summary.components) : sondages seuls, modèle sans marchés, prior fondamentaux, décomposition du momentum, cibles et probabilités atteintes du tempérage, taille d'échantillon effective."]},
691 {"version": "1.1.0", "date": "2026-09-06", "title": "Recensement 2021 et swing par segments", 708 {"version": "1.1.0", "date": "2026-09-06", "title": "Recensement 2021 et swing par segments",
692 "notes": ["Recensement de la population 2021 (Statistique Canada, profil des aires de diffusion agrégées + limites) agrégé par intersection surfacique aux 127 circonscriptions : langue maternelle, première langue officielle parlée, immigration, minorités visibles, âge, revenu, tenure, scolarité, chômage.", 709 "notes": ["Recensement de la population 2021 (Statistique Canada, profil des aires de diffusion agrégées + limites) agrégé par intersection surfacique aux 127 circonscriptions : langue maternelle, première langue officielle parlée, immigration, minorités visibles, âge, revenu, tenure, scolarité, chômage.",
693 "Excès de swing par langue : ventilations francophones / non-francophones des sondages (90 jours, pondérées récence × √n) comparées à une référence 2022 par régression écologique des résultats sur la part francophone ; appliqué à chaque circonscription selon sa composition (κ = 0,6).", 710 "Excès de swing par langue : ventilations francophones / non-francophones des sondages (90 jours, pondérées récence × √n) comparées à une référence 2022 par régression écologique des résultats sur la part francophone ; appliqué à chaque circonscription selon sa composition (κ = 0,6).",
@@ -704,6 +721,59 @@ CHANGELOG = [
704 ] 721 ]
705 722
706 723
724+# ---------------------------------------------------------------- signaux (v2)
725+@router.get("/signals")
726+def signals_all(s: Session = Depends(db)):
727+ from ..signals import attention, byelections, fundamentals, markets, media
728+ run = _latest_run(s)
729+ comp = (run.summary or {}).get("components") if run else None
730+ return {"asOf": datetime.now(timezone.utc).isoformat(), "runId": run.id if run else None, "components": comp,
731+ "markets": {**markets.implied(s), "history": markets.history(s)["series"]},
732+ "attention": attention.attention_summary(s), "media": media.media_summary(s),
733+ "fundamentals": fundamentals.fundamentals_summary(s), "byElections": byelections.byelection_summary(s)}
734+
735+
736+@router.get("/signals/markets")
737+def signals_markets(days: int = 120, s: Session = Depends(db)):
738+ from ..signals import markets
739+ return {**markets.implied(s), "history": markets.history(s, days)["series"]}
740+
741+
742+@router.get("/signals/attention")
743+def signals_attention(s: Session = Depends(db)):
744+ from ..signals import attention
745+ return attention.attention_summary(s)
746+
747+
748+@router.get("/signals/media")
749+def signals_media(window: int = 7, limit: int = 60, s: Session = Depends(db)):
750+ from ..signals import media
751+ d = media.media_summary(s, window=window)
752+ d["latest"] = d["latest"][:limit]
753+ return d
754+
755+
756+@router.get("/signals/fundamentals")
757+def signals_fundamentals(s: Session = Depends(db)):
758+ from ..signals import fundamentals
759+ return fundamentals.fundamentals_summary(s)
760+
761+
762+@router.get("/signals/byelections")
763+def signals_byelections(s: Session = Depends(db)):
764+ from ..signals import byelections
765+ return byelections.byelection_summary(s)
766+
767+
768+@router.get("/forecast/components")
769+def forecast_components(s: Session = Depends(db)):
770+ run = _latest_run(s)
771+ if not run:
772+ raise HTTPException(404)
773+ return {"runId": run.id, "modelVersion": run.model_version, "completedAt": run.completed_at.isoformat() if run.completed_at else None,
774+ "components": (run.summary or {}).get("components"), "parties": run.summary.get("parties"), "pMinority": run.summary.get("p_minority")}
775+
776+
707 @router.get("/changelog") 777 @router.get("/changelog")
708 def changelog(s: Session = Depends(db)): 778 def changelog(s: Session = Depends(db)):
709 runs = s.scalars(select(ForecastRun).where(ForecastRun.kind == "forecast", ForecastRun.status == "ok").order_by(desc(ForecastRun.id)).limit(30)).all() 779 runs = s.scalars(select(ForecastRun).where(ForecastRun.kind == "forecast", ForecastRun.status == "ok").order_by(desc(ForecastRun.id)).limit(30)).all()
modified engine/qc26/config.py +32 −2
@@ -52,7 +52,7 @@ class Settings(BaseSettings):
52 # ils dérivent de la liste officielle des circonscriptions (127 en 2026 → 64). 52 # ils dérivent de la liste officielle des circonscriptions (127 en 2026 → 64).
53 53
54 # --- Modèle --- 54 # --- Modèle ---
55− model_version: str = "1.1.0" 55+ model_version: str = "2.0.0"
56 n_simulations: int = 20000 56 n_simulations: int = 20000
57 random_seed: int | None = None 57 random_seed: int | None = None
58 recency_halflife_days: float = 14.0 # hors campagne 58 recency_halflife_days: float = 14.0 # hors campagne
@@ -68,9 +68,39 @@ class Settings(BaseSettings):
68 riding_error_sd: float = 0.20 # logit 68 riding_error_sd: float = 0.20 # logit
69 incumbent_retirement_penalty: float = 0.06 # logit (~1,3 pp) 69 incumbent_retirement_penalty: float = 0.06 # logit (~1,3 pp)
70 70
71+ # --- Ensemble v2 : au-delà des sondages ---
72+ # Fondamentaux (satisfaction du gouvernement → part du parti sortant) : poids maximal loin du
73+ # scrutin, décroissant linéairement vers 0 le jour du vote (horizon en jours).
74+ fundamentals_weight_max: float = 0.20
75+ fundamentals_horizon_days: int = 120
76+ fundamentals_alpha: float = 6.0 # part sortant ≈ α + β·satisfaction (%), calibration indicative QC 2003-2022
77+ fundamentals_beta: float = 0.60
78+ fundamentals_sd_pp: float = 5.5
79+ # Momentum : leadership (meilleur PM vs vote), attention (Wikipédia), ton médiatique → nudge borné (pp)
80+ momentum_leader_kappa: float = 0.15
81+ momentum_attention_kappa: float = 0.08 # pp de vote par pp de variation de la part d'attention (7 j vs 28 j)
82+ momentum_media_kappa: float = 0.5 # pp de vote par unité de ton médiatique (−1…+1), atténué sous 10 articles
83+ momentum_max_pp: float = 1.0
84+ # Marchés de prédiction : tempérage des simulations (pondération itérative) vers les prix implicites
85+ markets_weight: float = 0.30
86+ markets_min_volume: float = 5000.0
87+ markets_max_age_hours: int = 72
88+ # Partielles : ajustement des priors de circonscription du swing observé (rétréci)
89+ byelection_kappa: float = 0.35
90+
71 # --- Sources / connecteurs --- 91 # --- Sources / connecteurs ---
72 firecrawl_api_key: str = "" 92 firecrawl_api_key: str = ""
73 anthropic_api_key: str = "" 93 anthropic_api_key: str = ""
94+ serper_api_key: str = ""
95+ tavily_api_key: str = ""
96+ polymarket_gamma: str = "https://gamma-api.polymarket.com"
97+ polymarket_clob: str = "https://clob.polymarket.com"
98+ polymarket_event_slugs: str = ("quebec-general-election-winner,of-seats-won-by-pq-in-the-2026-quebec-general-election,"
99+ "of-seats-won-by-caq-in-the-2026-quebec-general-election,of-seats-won-by-plq-in-the-2026-quebec-general-election,"
100+ "will-the-pq-win-a-majority-in-the-2026-quebec-general-election,quebec-general-election-second-place-20260825152158,"
101+ "quebec-general-election-third-place-20260825152158")
102+ signals_markets_minutes: int = 30
103+ signals_media_hours: int = 3
74 llm_model: str = "claude-opus-5" 104 llm_model: str = "claude-opus-5"
75 llm_enabled: bool = True 105 llm_enabled: bool = True
76 user_agent: str = ( 106 user_agent: str = (
@@ -111,5 +141,5 @@ class Settings(BaseSettings):
111 141
112 settings = Settings() 142 settings = Settings()
113 for _d in (settings.archive_dir / "polls", settings.archive_dir / "documents", 143 for _d in (settings.archive_dir / "polls", settings.archive_dir / "documents",
114− settings.archive_dir / "live", settings.cache_dir): 144+ settings.archive_dir / "live", settings.archive_dir / "partielles", settings.cache_dir):
115 _d.mkdir(parents=True, exist_ok=True) 145 _d.mkdir(parents=True, exist_ok=True)
modified engine/qc26/connectors/parties/documents.py +8 −5
@@ -21,7 +21,7 @@ from sqlalchemy.orm import Session
21 from ... import llm 21 from ... import llm
22 from ...config import settings 22 from ...config import settings
23 from ...db import (DocumentChunk, DocumentVersion, IngestionJob, Party, PartyDocument, PolicyProposal, Source, audit, utcnow) 23 from ...db import (DocumentChunk, DocumentVersion, IngestionJob, Party, PartyDocument, PolicyProposal, Source, audit, utcnow)
24−from ...util import fetch_bytes, html_to_text, http, parse_fr_date, pdf_pages_text, sha256_bytes 24+from ...util import fetch_bytes, html_to_text, http, parse_fr_date, pdf_pages_text, resolve_archive_path, sha256_bytes
25 from ..polls.registry import firecrawl_map, firecrawl_scrape 25 from ..polls.registry import firecrawl_map, firecrawl_scrape
26 26
27 log = logging.getLogger("qc26.parties.docs") 27 log = logging.getLogger("qc26.parties.docs")
@@ -232,8 +232,9 @@ def fetch_version(s: Session, doc: PartyDocument) -> DocumentVersion | None:
232 232
233 def diff_versions(old: DocumentVersion, new: DocumentVersion) -> dict: 233 def diff_versions(old: DocumentVersion, new: DocumentVersion) -> dict:
234 import difflib 234 import difflib
235− a = Path(old.text_path).read_text().splitlines() if old.text_path and Path(old.text_path).exists() else [] 235+ _pa, _pb = resolve_archive_path(old.text_path), resolve_archive_path(new.text_path)
236− b = Path(new.text_path).read_text().splitlines() if new.text_path and Path(new.text_path).exists() else [] 236+ a = _pa.read_text().splitlines() if _pa and _pa.exists() else []
237+ b = _pb.read_text().splitlines() if _pb and _pb.exists() else []
237 sm = difflib.SequenceMatcher(None, a, b, autojunk=False) 238 sm = difflib.SequenceMatcher(None, a, b, autojunk=False)
238 added, removed = [], [] 239 added, removed = [], []
239 for tag, i1, i2, j1, j2 in sm.get_opcodes(): 240 for tag, i1, i2, j1, j2 in sm.get_opcodes():
@@ -276,7 +277,8 @@ Si l'extrait ne contient aucune proposition (page de navigation, formulaire, bio
276 277
277 278
278 def analyze_version(s: Session, doc: PartyDocument, v: DocumentVersion, max_chunks: int = 12) -> int: 279 def analyze_version(s: Session, doc: PartyDocument, v: DocumentVersion, max_chunks: int = 12) -> int:
279− text = Path(v.text_path).read_text() if v.text_path else "" 280+ _tp = resolve_archive_path(v.text_path)
281+ text = _tp.read_text() if _tp and _tp.exists() else ""
280 if len(text.strip()) < 300: 282 if len(text.strip()) < 300:
281 doc.status = "ignored" 283 doc.status = "ignored"
282 v.analyzed_at = utcnow() 284 v.analyzed_at = utcnow()
@@ -365,7 +367,8 @@ def sync_party(s: Session, party_id: str, max_docs: int = 80, analyze: bool = Tr
365 s.commit() 367 s.commit()
366 if not doc.title or doc.title == url.rsplit("/", 1)[-1][:200]: 368 if not doc.title or doc.title == url.rsplit("/", 1)[-1][:200]:
367 # titre depuis le texte (première ligne significative) 369 # titre depuis le texte (première ligne significative)
368− txt = Path(v.text_path).read_text()[:2000] if v.text_path else "" 370+ _tp2 = resolve_archive_path(v.text_path)
371+ txt = _tp2.read_text()[:2000] if _tp2 and _tp2.exists() else ""
369 first = next((l.strip() for l in txt.splitlines() if len(l.strip()) > 12), None) 372 first = next((l.strip() for l in txt.splitlines() if len(l.strip()) > 12), None)
370 if first: 373 if first:
371 doc.title = first[:200] 374 doc.title = first[:200]
modified engine/qc26/connectors/polls/base.py +4 −2
@@ -20,7 +20,7 @@ from ... import llm
20 from ...config import settings 20 from ...config import settings
21 from ...db import Poll, PollDocument, PollResult, PollSubsample, Pollster, audit, utcnow 21 from ...db import Poll, PollDocument, PollResult, PollSubsample, Pollster, audit, utcnow
22 from ...parties import PARTY_IDS, party_id_from_label 22 from ...parties import PARTY_IDS, party_id_from_label
23−from ...util import fetch_bytes, html_to_text, pdf_pages_text, pdf_tables_text, sha256_bytes 23+from ...util import fetch_bytes, html_to_text, pdf_pages_text, pdf_tables_text, resolve_archive_path, sha256_bytes
24 24
25 log = logging.getLogger("qc26.polls.connector") 25 log = logging.getLogger("qc26.polls.connector")
26 26
@@ -95,7 +95,9 @@ class PollConnector:
95 return doc 95 return doc
96 96
97 def extract_text(self, doc: PollDocument) -> tuple[list[str], list[tuple[int, str]]]: 97 def extract_text(self, doc: PollDocument) -> tuple[list[str], list[tuple[int, str]]]:
98− path = Path(doc.path) 98+ path = resolve_archive_path(doc.path) or Path(doc.path)
99+ if not path.exists():
100+ raise FileNotFoundError(f"archive absente : {doc.path}")
99 if path.suffix == ".pdf": 101 if path.suffix == ".pdf":
100 pages = pdf_pages_text(path) 102 pages = pdf_pages_text(path)
101 tables = pdf_tables_text(path) 103 tables = pdf_tables_text(path)
modified engine/qc26/db.py +135 −26
@@ -11,7 +11,7 @@ from contextlib import contextmanager
11 from datetime import date, datetime, timezone 11 from datetime import date, datetime, timezone
12 from typing import Any, Iterator 12 from typing import Any, Iterator
13 13
14−from sqlalchemy import (JSON, Boolean, Date, DateTime, Float, ForeignKey, Integer, String, Text, 14+from sqlalchemy import (JSON, Boolean, Date, DateTime, Float, ForeignKey, Integer, String, Text, TypeDecorator,
15 UniqueConstraint, create_engine, event, text) 15 UniqueConstraint, create_engine, event, text)
16 from sqlalchemy.orm import DeclarativeBase, Mapped, Session, mapped_column, relationship, sessionmaker 16 from sqlalchemy.orm import DeclarativeBase, Mapped, Session, mapped_column, relationship, sessionmaker
17 17
@@ -22,6 +22,23 @@ def utcnow() -> datetime:
22 return datetime.now(timezone.utc) 22 return datetime.now(timezone.utc)
23 23
24 24
25+class TZDateTime(TypeDecorator):
26+ """SQLite ne conserve pas le fuseau : on stocke en UTC naïf et on relit en UTC explicite, pour que
27+ `.isoformat()` porte toujours « +00:00 » (sinon le navigateur interprète l'heure comme locale)."""
28+ impl = DateTime(timezone=True)
29+ cache_ok = True
30+
31+ def process_bind_param(self, value, dialect):
32+ if value is not None and value.tzinfo is not None:
33+ return value.astimezone(timezone.utc).replace(tzinfo=None)
34+ return value
35+
36+ def process_result_value(self, value, dialect):
37+ if value is not None and value.tzinfo is None:
38+ return value.replace(tzinfo=timezone.utc)
39+ return value
40+
41+
25 class Base(DeclarativeBase): 42 class Base(DeclarativeBase):
26 pass 43 pass
27 44
@@ -47,7 +64,7 @@ class Party(Base):
47 vote_2022: Mapped[float | None] = mapped_column(Float) 64 vote_2022: Mapped[float | None] = mapped_column(Float)
48 order: Mapped[int] = mapped_column(Integer, default=0) 65 order: Mapped[int] = mapped_column(Integer, default=0)
49 color_note: Mapped[str | None] = mapped_column(Text) 66 color_note: Mapped[str | None] = mapped_column(Text)
50− updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow) 67+ updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
51 68
52 69
53 class Region(Base): 70 class Region(Base):
@@ -115,7 +132,7 @@ class RidingDemographics(Base):
115 ada_count: Mapped[int] = mapped_column(Integer, default=0) 132 ada_count: Mapped[int] = mapped_column(Integer, default=0)
116 coverage: Mapped[float | None] = mapped_column(Float) # part de la population de la circ. couverte par des ADA 133 coverage: Mapped[float | None] = mapped_column(Float) # part de la population de la circ. couverte par des ADA
117 source: Mapped[str] = mapped_column(String(300)) 134 source: Mapped[str] = mapped_column(String(300))
118− computed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 135+ computed_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
119 136
120 137
121 class HistoricalResult(Base): 138 class HistoricalResult(Base):
@@ -149,8 +166,8 @@ class Candidate(Base):
149 incumbent: Mapped[bool] = mapped_column(Boolean, default=False) 166 incumbent: Mapped[bool] = mapped_column(Boolean, default=False)
150 status: Mapped[str] = mapped_column(String(8)) # CPA (confirmé parti) / CIA (indépendant) … 167 status: Mapped[str] = mapped_column(String(8)) # CPA (confirmé parti) / CIA (indépendant) …
151 state: Mapped[str] = mapped_column(String(8)) 168 state: Mapped[str] = mapped_column(String(8))
152− dgeq_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 169+ dgeq_updated_at: Mapped[datetime | None] = mapped_column(TZDateTime)
153− fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 170+ fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
154 171
155 172
156 # --------------------------------------------------------------------- sondages 173 # --------------------------------------------------------------------- sondages
@@ -175,7 +192,7 @@ class PollDocument(Base):
175 content_type: Mapped[str | None] = mapped_column(String(80)) 192 content_type: Mapped[str | None] = mapped_column(String(80))
176 bytes: Mapped[int] = mapped_column(Integer) 193 bytes: Mapped[int] = mapped_column(Integer)
177 pages: Mapped[int | None] = mapped_column(Integer) 194 pages: Mapped[int | None] = mapped_column(Integer)
178− downloaded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 195+ downloaded_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
179 text_path: Mapped[str | None] = mapped_column(String(300)) 196 text_path: Mapped[str | None] = mapped_column(String(300))
180 http_last_modified: Mapped[str | None] = mapped_column(String(80)) 197 http_last_modified: Mapped[str | None] = mapped_column(String(80))
181 pollster_id: Mapped[str | None] = mapped_column(String(32)) 198 pollster_id: Mapped[str | None] = mapped_column(String(32))
@@ -204,8 +221,8 @@ class Poll(Base):
204 validation_notes: Mapped[dict | list | None] = mapped_column(JSON) 221 validation_notes: Mapped[dict | list | None] = mapped_column(JSON)
205 undecided: Mapped[float | None] = mapped_column(Float) 222 undecided: Mapped[float | None] = mapped_column(Float)
206 notes: Mapped[dict | list | None] = mapped_column(JSON) 223 notes: Mapped[dict | list | None] = mapped_column(JSON)
207− ingested_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 224+ ingested_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
208− updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow) 225+ updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
209 excluded: Mapped[bool] = mapped_column(Boolean, default=False) 226 excluded: Mapped[bool] = mapped_column(Boolean, default=False)
210 exclusion_reason: Mapped[str | None] = mapped_column(String(200)) 227 exclusion_reason: Mapped[str | None] = mapped_column(String(200))
211 results: Mapped[list["PollResult"]] = relationship(cascade="all, delete-orphan", lazy="selectin") 228 results: Mapped[list["PollResult"]] = relationship(cascade="all, delete-orphan", lazy="selectin")
@@ -250,8 +267,8 @@ class PartyDocument(Base):
250 url: Mapped[str] = mapped_column(String(600), unique=True) 267 url: Mapped[str] = mapped_column(String(600), unique=True)
251 doc_type: Mapped[str] = mapped_column(String(32)) # plateforme / cadre_financier / communique / page / engagement 268 doc_type: Mapped[str] = mapped_column(String(32)) # plateforme / cadre_financier / communique / page / engagement
252 published_date: Mapped[date | None] = mapped_column(Date) 269 published_date: Mapped[date | None] = mapped_column(Date)
253− discovered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 270+ discovered_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
254− last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 271+ last_checked_at: Mapped[datetime | None] = mapped_column(TZDateTime)
255 current_version_id: Mapped[int | None] = mapped_column(Integer) 272 current_version_id: Mapped[int | None] = mapped_column(Integer)
256 proposals_count: Mapped[int] = mapped_column(Integer, default=0) 273 proposals_count: Mapped[int] = mapped_column(Integer, default=0)
257 status: Mapped[str] = mapped_column(String(16), default="new") # new / analyzed / failed / ignored 274 status: Mapped[str] = mapped_column(String(16), default="new") # new / analyzed / failed / ignored
@@ -268,8 +285,8 @@ class DocumentVersion(Base):
268 content_type: Mapped[str | None] = mapped_column(String(80)) 285 content_type: Mapped[str | None] = mapped_column(String(80))
269 bytes: Mapped[int] = mapped_column(Integer) 286 bytes: Mapped[int] = mapped_column(Integer)
270 pages: Mapped[int | None] = mapped_column(Integer) 287 pages: Mapped[int | None] = mapped_column(Integer)
271− fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 288+ fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
272− analyzed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 289+ analyzed_at: Mapped[datetime | None] = mapped_column(TZDateTime)
273 diff_summary: Mapped[dict | None] = mapped_column(JSON) 290 diff_summary: Mapped[dict | None] = mapped_column(JSON)
274 291
275 292
@@ -298,7 +315,7 @@ class PolicyProposal(Base):
298 quote: Mapped[str] = mapped_column(Text) 315 quote: Mapped[str] = mapped_column(Text)
299 confidence: Mapped[float] = mapped_column(Float) 316 confidence: Mapped[float] = mapped_column(Float)
300 status: Mapped[str] = mapped_column(String(16), default="accepted") # accepted / review / rejected 317 status: Mapped[str] = mapped_column(String(16), default="accepted") # accepted / review / rejected
301− extracted_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 318+ extracted_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
302 319
303 320
304 class CompassQuestion(Base): 321 class CompassQuestion(Base):
@@ -324,7 +341,7 @@ class PartyPosition(Base):
324 proposal_ids: Mapped[list | None] = mapped_column(JSON) 341 proposal_ids: Mapped[list | None] = mapped_column(JSON)
325 rationale: Mapped[str | None] = mapped_column(Text) 342 rationale: Mapped[str | None] = mapped_column(Text)
326 status: Mapped[str] = mapped_column(String(16), default="pending") # pending / extracted / review / manual 343 status: Mapped[str] = mapped_column(String(16), default="pending") # pending / extracted / review / manual
327− updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow) 344+ updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
328 345
329 346
330 # -------------------------------------------------------------------- forecast 347 # -------------------------------------------------------------------- forecast
@@ -334,8 +351,8 @@ class ForecastRun(Base):
334 model_version: Mapped[str] = mapped_column(String(16)) 351 model_version: Mapped[str] = mapped_column(String(16))
335 kind: Mapped[str] = mapped_column(String(16), default="forecast") # forecast / nowcast 352 kind: Mapped[str] = mapped_column(String(16), default="forecast") # forecast / nowcast
336 data_cutoff: Mapped[date] = mapped_column(Date) 353 data_cutoff: Mapped[date] = mapped_column(Date)
337− started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 354+ started_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
338− completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 355+ completed_at: Mapped[datetime | None] = mapped_column(TZDateTime)
339 poll_count: Mapped[int] = mapped_column(Integer, default=0) 356 poll_count: Mapped[int] = mapped_column(Integer, default=0)
340 n_simulations: Mapped[int] = mapped_column(Integer, default=0) 357 n_simulations: Mapped[int] = mapped_column(Integer, default=0)
341 seed: Mapped[int | None] = mapped_column(Integer) 358 seed: Mapped[int | None] = mapped_column(Integer)
@@ -427,7 +444,7 @@ class PollWeight(Base):
427 class LiveSnapshot(Base): 444 class LiveSnapshot(Base):
428 __tablename__ = "live_snapshots" 445 __tablename__ = "live_snapshots"
429 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) 446 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
430− fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True) 447+ fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
431 source_updated_at: Mapped[str | None] = mapped_column(String(40)) 448 source_updated_at: Mapped[str | None] = mapped_column(String(40))
432 sha256: Mapped[str] = mapped_column(String(64), index=True) 449 sha256: Mapped[str] = mapped_column(String(64), index=True)
433 path: Mapped[str] = mapped_column(String(300)) 450 path: Mapped[str] = mapped_column(String(300))
@@ -445,7 +462,7 @@ class LiveSnapshot(Base):
445 class LiveRidingState(Base): 462 class LiveRidingState(Base):
446 __tablename__ = "live_riding_states" 463 __tablename__ = "live_riding_states"
447 riding_code: Mapped[int] = mapped_column(Integer, primary_key=True) 464 riding_code: Mapped[int] = mapped_column(Integer, primary_key=True)
448− updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 465+ updated_at: Mapped[datetime | None] = mapped_column(TZDateTime)
449 source_updated_at: Mapped[str | None] = mapped_column(String(40)) 466 source_updated_at: Mapped[str | None] = mapped_column(String(40))
450 status: Mapped[str] = mapped_column(String(16), default="NOT_STARTED") # NOT_STARTED/REPORTING/LEADING/PROJECTED/ELECTED/FINAL 467 status: Mapped[str] = mapped_column(String(16), default="NOT_STARTED") # NOT_STARTED/REPORTING/LEADING/PROJECTED/ELECTED/FINAL
451 bureaux_completes: Mapped[int] = mapped_column(Integer, default=0) 468 bureaux_completes: Mapped[int] = mapped_column(Integer, default=0)
@@ -466,7 +483,7 @@ class LiveRidingState(Base):
466 class ElectionEvent(Base): 483 class ElectionEvent(Base):
467 __tablename__ = "election_events" 484 __tablename__ = "election_events"
468 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) 485 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
469− ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True) 486+ ts: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
470 kind: Mapped[str] = mapped_column(String(32), index=True) # lead_change / riding_complete / seats_update / reporting / first_results / final 487 kind: Mapped[str] = mapped_column(String(32), index=True) # lead_change / riding_complete / seats_update / reporting / first_results / final
471 riding_code: Mapped[int | None] = mapped_column(Integer) 488 riding_code: Mapped[int | None] = mapped_column(Integer)
472 party_id: Mapped[str | None] = mapped_column(String(8)) 489 party_id: Mapped[str | None] = mapped_column(String(8))
@@ -478,12 +495,104 @@ class ElectionEvent(Base):
478 class NowcastRun(Base): 495 class NowcastRun(Base):
479 __tablename__ = "nowcast_runs" 496 __tablename__ = "nowcast_runs"
480 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) 497 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
481− ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True) 498+ ts: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
482 snapshot_id: Mapped[int | None] = mapped_column(Integer) 499 snapshot_id: Mapped[int | None] = mapped_column(Integer)
483 ridings_reporting: Mapped[int] = mapped_column(Integer) 500 ridings_reporting: Mapped[int] = mapped_column(Integer)
484 summary: Mapped[dict] = mapped_column(JSON) # par parti : élus, en avance, total, p_majority, seats mean/ci 501 summary: Mapped[dict] = mapped_column(JSON) # par parti : élus, en avance, total, p_majority, seats mean/ci
485 502
486 503
504+# ------------------------------------------------------- signaux alternatifs (v2)
505+class Signal(Base):
506+ """Série temporelle d'un signal « au-delà des sondages » : marchés de prédiction, attention
507+ (Wikipédia), médias (volume / ton), fondamentaux tirés des questions de leadership, partielles.
508+ Une ligne = une observation datée, avec sa provenance (URL) et ses métadonnées."""
509+ __tablename__ = "signals"
510+ __table_args__ = (UniqueConstraint("source", "key", "party_id", "date"),)
511+ id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
512+ source: Mapped[str] = mapped_column(String(32), index=True) # polymarket / wikipedia / gdelt / serper-news / polls-leadership / dgeq-partielles
513+ key: Mapped[str] = mapped_column(String(64), index=True) # ex. p_most_seats / pageviews / tone / best_pm / satisfaction
514+ party_id: Mapped[str | None] = mapped_column(String(8), index=True)
515+ date: Mapped[date] = mapped_column(Date, index=True)
516+ value: Mapped[float] = mapped_column(Float)
517+ n: Mapped[int | None] = mapped_column(Integer)
518+ meta: Mapped[dict | list | None] = mapped_column(JSON)
519+ url: Mapped[str | None] = mapped_column(String(600))
520+ fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
521+
522+
523+class Market(Base):
524+ """Marché de prédiction (Polymarket) rattaché à l'élection : question, type, parti, tranche."""
525+ __tablename__ = "markets"
526+ id: Mapped[str] = mapped_column(String(32), primary_key=True) # id Polymarket
527+ source: Mapped[str] = mapped_column(String(24), default="polymarket")
528+ event_slug: Mapped[str] = mapped_column(String(160), index=True)
529+ event_title: Mapped[str | None] = mapped_column(String(200))
530+ question: Mapped[str] = mapped_column(String(300))
531+ kind: Mapped[str] = mapped_column(String(24), index=True) # most_seats / majority / seats_band / place
532+ party_id: Mapped[str | None] = mapped_column(String(8), index=True)
533+ band_lo: Mapped[int | None] = mapped_column(Integer)
534+ band_hi: Mapped[int | None] = mapped_column(Integer)
535+ place: Mapped[int | None] = mapped_column(Integer)
536+ condition_id: Mapped[str | None] = mapped_column(String(80))
537+ token_id: Mapped[str | None] = mapped_column(String(100))
538+ volume: Mapped[float | None] = mapped_column(Float)
539+ liquidity: Mapped[float | None] = mapped_column(Float)
540+ end_date: Mapped[str | None] = mapped_column(String(40))
541+ url: Mapped[str | None] = mapped_column(String(300))
542+ last_price: Mapped[float | None] = mapped_column(Float)
543+ best_bid: Mapped[float | None] = mapped_column(Float)
544+ best_ask: Mapped[float | None] = mapped_column(Float)
545+ active: Mapped[bool] = mapped_column(Boolean, default=True)
546+ updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
547+
548+
549+class MarketPrice(Base):
550+ __tablename__ = "market_prices"
551+ __table_args__ = (UniqueConstraint("market_id", "ts"),)
552+ id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
553+ market_id: Mapped[str] = mapped_column(ForeignKey("markets.id"), index=True)
554+ ts: Mapped[datetime] = mapped_column(TZDateTime, index=True)
555+ price: Mapped[float] = mapped_column(Float) # prix « Oui » (0–1)
556+
557+
558+class MediaItem(Base):
559+ """Article de presse repéré (titre, média, date) et classé par le LLM : partis mentionnés et ton."""
560+ __tablename__ = "media_items"
561+ id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
562+ url: Mapped[str] = mapped_column(String(600), unique=True)
563+ title: Mapped[str] = mapped_column(String(400))
564+ domain: Mapped[str | None] = mapped_column(String(120), index=True)
565+ snippet: Mapped[str | None] = mapped_column(Text)
566+ published_at: Mapped[datetime | None] = mapped_column(TZDateTime, index=True)
567+ fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
568+ query: Mapped[str | None] = mapped_column(String(200))
569+ source: Mapped[str] = mapped_column(String(24), default="serper-news")
570+ parties: Mapped[dict | None] = mapped_column(JSON) # {party_id: tone −1…+1}
571+ topic: Mapped[str | None] = mapped_column(String(48))
572+ summary: Mapped[str | None] = mapped_column(String(300))
573+ confidence: Mapped[float | None] = mapped_column(Float)
574+ classified: Mapped[bool] = mapped_column(Boolean, default=False, index=True)
575+
576+
577+class ByElection(Base):
578+ """Élection partielle provinciale depuis 2022 (résultats officiels DGEQ) comparée au résultat
579+ de 2022 dans la même circonscription : mesure directe du swing en conditions réelles."""
580+ __tablename__ = "by_elections"
581+ id: Mapped[str] = mapped_column(String(32), primary_key=True) # ex. 2025-03-17-542
582+ date: Mapped[date] = mapped_column(Date, index=True)
583+ dgeq_code: Mapped[int] = mapped_column(Integer)
584+ riding_name: Mapped[str | None] = mapped_column(String(80))
585+ url: Mapped[str] = mapped_column(String(300))
586+ sha256: Mapped[str | None] = mapped_column(String(64))
587+ path: Mapped[str | None] = mapped_column(String(300))
588+ results: Mapped[list | None] = mapped_column(JSON) # [{candidate, party_label, party_id, votes, share}]
589+ previous: Mapped[dict | None] = mapped_column(JSON) # {party_id: share 2022} (résultat général précédent, mêmes limites)
590+ turnout: Mapped[float | None] = mapped_column(Float)
591+ winner_party_id: Mapped[str | None] = mapped_column(String(8))
592+ status: Mapped[str] = mapped_column(String(16), default="new")
593+ fetched_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
594+
595+
487 # ------------------------------------------------------------------ exploitation 596 # ------------------------------------------------------------------ exploitation
488 class Source(Base): 597 class Source(Base):
489 __tablename__ = "sources" 598 __tablename__ = "sources"
@@ -492,8 +601,8 @@ class Source(Base):
492 url: Mapped[str] = mapped_column(String(600)) 601 url: Mapped[str] = mapped_column(String(600))
493 tier: Mapped[int] = mapped_column(Integer, default=1) # 1 officiel, 2 firme, 3 média, 5 index 602 tier: Mapped[int] = mapped_column(Integer, default=1) # 1 officiel, 2 firme, 3 média, 5 index
494 kind: Mapped[str] = mapped_column(String(32)) 603 kind: Mapped[str] = mapped_column(String(32))
495− last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 604+ last_checked_at: Mapped[datetime | None] = mapped_column(TZDateTime)
496− last_ok_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 605+ last_ok_at: Mapped[datetime | None] = mapped_column(TZDateTime)
497 status: Mapped[str] = mapped_column(String(16), default="unknown") # operational / degraded / down / ready 606 status: Mapped[str] = mapped_column(String(16), default="unknown") # operational / degraded / down / ready
498 detail: Mapped[str | None] = mapped_column(String(300)) 607 detail: Mapped[str | None] = mapped_column(String(300))
499 608
@@ -502,8 +611,8 @@ class IngestionJob(Base):
502 __tablename__ = "ingestion_jobs" 611 __tablename__ = "ingestion_jobs"
503 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) 612 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
504 kind: Mapped[str] = mapped_column(String(48), index=True) 613 kind: Mapped[str] = mapped_column(String(48), index=True)
505− started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) 614+ started_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow)
506− finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) 615+ finished_at: Mapped[datetime | None] = mapped_column(TZDateTime)
507 status: Mapped[str] = mapped_column(String(16), default="running") 616 status: Mapped[str] = mapped_column(String(16), default="running")
508 items: Mapped[int] = mapped_column(Integer, default=0) 617 items: Mapped[int] = mapped_column(Integer, default=0)
509 message: Mapped[str | None] = mapped_column(Text) 618 message: Mapped[str | None] = mapped_column(Text)
@@ -513,7 +622,7 @@ class IngestionJob(Base):
513 class AuditLog(Base): 622 class AuditLog(Base):
514 __tablename__ = "audit_logs" 623 __tablename__ = "audit_logs"
515 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) 624 id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
516− ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True) 625+ ts: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, index=True)
517 actor: Mapped[str] = mapped_column(String(64)) # system:<job> / admin 626 actor: Mapped[str] = mapped_column(String(64)) # system:<job> / admin
518 entity: Mapped[str] = mapped_column(String(48)) 627 entity: Mapped[str] = mapped_column(String(48))
519 entity_id: Mapped[str] = mapped_column(String(80)) 628 entity_id: Mapped[str] = mapped_column(String(80))
@@ -537,7 +646,7 @@ class Setting(Base):
537 __tablename__ = "settings" 646 __tablename__ = "settings"
538 key: Mapped[str] = mapped_column(String(64), primary_key=True) 647 key: Mapped[str] = mapped_column(String(64), primary_key=True)
539 value: Mapped[dict | list | str | None] = mapped_column(JSON) 648 value: Mapped[dict | list | str | None] = mapped_column(JSON)
540− updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, onupdate=utcnow) 649+ updated_at: Mapped[datetime] = mapped_column(TZDateTime, default=utcnow, onupdate=utcnow)
541 650
542 651
543 # -------------------------------------------------------------------- moteur 652 # -------------------------------------------------------------------- moteur
modified engine/qc26/ingest/official.py +31 −0
@@ -224,16 +224,47 @@ def sync_candidates(s: Session, refresh: bool = True) -> int:
224 counts[pid] += 1 224 counts[pid] += 1
225 if c.get("depute_sortant") == "O": 225 if c.get("depute_sortant") == "O":
226 incumbents_running[code] = True 226 incumbents_running[code] = True
227+ # la liste officielle des candidatures est la vérité sur le·la député·e sortant·e :
228+ # elle corrige le gagnant 2022 transposé quand une partielle a changé la donne (Jean-Talon, Terrebonne, Arthabaska, Chicoutimi…)
229+ rd = ridings[code]
230+ if pid and (rd.incumbent_party_id != pid or not rd.incumbent_name):
231+ rd.incumbent_party_id = pid
232+ rd.incumbent_name = f"{c.get('prenom_bulletin_vote') or ''} {c.get('nom_bulletin_vote') or ''}".strip()
227 for pid, n in counts.items(): 233 for pid, n in counts.items():
228 p = s.get(Party, pid) 234 p = s.get(Party, pid)
229 if p: 235 if p:
230 p.candidates_count = n 236 p.candidates_count = n
231 for code, rd in ridings.items(): 237 for code, rd in ridings.items():
232 rd.incumbent_running = incumbents_running.get(code, False) 238 rd.incumbent_running = incumbents_running.get(code, False)
239+ apply_by_election_incumbents(s, ridings)
233 _mark_source(s, "eq-candidates", "Élections Québec — candidatures officielles", settings.eq_candidates_url, True) 240 _mark_source(s, "eq-candidates", "Élections Québec — candidatures officielles", settings.eq_candidates_url, True)
234 return len(data) 241 return len(data)
235 242
236 243
244+def apply_by_election_incumbents(s: Session, ridings: dict | None = None) -> int:
245+ """Sortant·e des circonscriptions ayant connu une partielle depuis 2022 : le·la gagnant·e de la partielle
246+ (résultat officiel DGEQ archivé), sauf si la liste des candidatures désigne déjà un·e sortant·e."""
247+ from ..db import ByElection, Candidate
248+ ridings = ridings or {r.code: r for r in s.scalars(select(Riding)).all()}
249+ by_name = {norm_name(r.name): r for r in ridings.values()}
250+ n = 0
251+ for b in s.scalars(select(ByElection).where(ByElection.status == "ok").order_by(ByElection.date)).all():
252+ if not b.riding_name or not b.results:
253+ continue
254+ key = norm_name(b.riding_name)
255+ rd = by_name.get(key) or next((r for k, r in by_name.items() if k.startswith(key)), None)
256+ if not rd:
257+ continue
258+ has_official = s.scalar(select(Candidate).where(Candidate.riding_code == rd.code, Candidate.incumbent == True).limit(1)) # noqa: E712
259+ if has_official:
260+ continue
261+ win = max(b.results, key=lambda r: r["votes"])
262+ if win.get("party_id") and rd.incumbent_party_id != win["party_id"]:
263+ rd.incumbent_party_id, rd.incumbent_name = win["party_id"], win.get("candidate")
264+ n += 1
265+ return n
266+
267+
237 # ------------------------------------------------------------------ résultats 268 # ------------------------------------------------------------------ résultats
238 def _load_results_json(path_dir: Path, url_dir: str, refresh: bool) -> dict: 269 def _load_results_json(path_dir: Path, url_dir: str, refresh: bool) -> dict:
239 raw = _download_or_local(f"{url_dir}/resultats.json", path_dir / "resultats.json", refresh) 270 raw = _download_or_local(f"{url_dir}/resultats.json", path_dir / "resultats.json", refresh)
modified engine/qc26/main.py +7 −0
@@ -44,6 +44,13 @@ def bootstrap() -> None:
44 run_forecast(s, trigger="reference_polls_2022") 44 run_forecast(s, trigger="reference_polls_2022")
45 except Exception as e: # noqa: BLE001 45 except Exception as e: # noqa: BLE001
46 log.warning("sondages de référence 2022 : %s", e) 46 log.warning("sondages de référence 2022 : %s", e)
47+ try:
48+ from .db import Signal
49+ from .signals import fundamentals
50+ if not s.scalar(select(Signal).where(Signal.source == "polls-leadership").limit(1)):
51+ fundamentals.extract_from_polls(s) # dérivé des sondages déjà en base, sans réseau
52+ except Exception as e: # noqa: BLE001
53+ log.warning("fondamentaux : %s", e)
47 if not s.scalar(select(ForecastRun).where(ForecastRun.status == "ok").limit(1)): 54 if not s.scalar(select(ForecastRun).where(ForecastRun.status == "ok").limit(1)):
48 from .model.run import run_forecast 55 from .model.run import run_forecast
49 try: 56 try:
added engine/qc26/model/ensemble.py +258 −0
@@ -0,0 +1,258 @@
1+"""Ensemble QC26 (modèle 2.0) — au-delà des sondages.
2+
3+Quatre composantes, toutes optionnelles (désactivées si le signal est absent de la base), toutes
4+paramétrées dans `config.py` et toutes publiées séparément dans le run (« Sondages seuls »
5+vs « Ensemble QC26 ») :
6+
7+1. **Fondamentaux** — prior structurel sur la part du parti sortant : part ≈ α + β·satisfaction
8+ (satisfaction à l'égard du gouvernement, pondérée récence, tirée des rapports de sondage). Mélangé
9+ à l'état latent des sondages dans l'espace logit avec un poids w_f qui décroît linéairement des
10+ `fundamentals_horizon_days` jours précédant le scrutin jusqu'à 0 le jour du vote.
11+2. **Momentum** — nudge additif borné (± `momentum_max_pp`) sur la moyenne du vote au jour du vote :
12+ prime au chef (meilleur PM vs intention de vote), attention (Wikipédia, 7 j vs 28 j), ton médiatique
13+ (titres classés). Recentré à somme nulle entre les cinq partis.
14+3. **Marchés de prédiction** — tempérage : les simulations Monte Carlo sont **repondérées**
15+ (ajustement proportionnel itératif) pour que les probabilités « plus de sièges », « majorité » et les
16+ tranches de sièges implicites de l'ensemble valent (1 − w_m)·modèle + w_m·marché. Toute la
17+ distribution (sièges, circonscriptions) reste cohérente : ce sont les mêmes simulations, pesées.
18+4. **Partielles** — excès de swing local observé lors d'une partielle (net du swing national à la même
19+ date) appliqué, rétréci (κ), au prior de la circonscription concernée.
20+"""
21+from __future__ import annotations
22+
23+import math
24+from dataclasses import dataclass, field
25+from datetime import date
26+
27+import numpy as np
28+
29+from ..config import settings
30+from ..parties import PARTY_IDS
31+from .aggregate import PARTIES_ALL
32+
33+
34+def _logit(p: float) -> float:
35+ p = min(max(p, 0.002), 0.998)
36+ return math.log(p / (1 - p))
37+
38+
39+def _clip(x: float, lo: float, hi: float) -> float:
40+ return max(lo, min(hi, x))
41+
42+
43+# --------------------------------------------------------------- pondérations
44+def wmean(x: np.ndarray, w: np.ndarray) -> float:
45+ return float(np.average(x, weights=w))
46+
47+
48+def wquantile(x: np.ndarray, w: np.ndarray, q: float) -> float:
49+ idx = np.argsort(x)
50+ xs, ws = x[idx], w[idx]
51+ cw = np.cumsum(ws)
52+ cw = cw / cw[-1]
53+ return float(xs[min(len(xs) - 1, int(np.searchsorted(cw, q, side="left")))])
54+
55+
56+def whist(x: np.ndarray, w: np.ndarray, minlength: int) -> list[float]:
57+ h = np.bincount(x, weights=w, minlength=minlength)
58+ return [round(float(v), 2) for v in h]
59+
60+
61+def ess(w: np.ndarray) -> float:
62+ return float(w.sum() ** 2 / (w ** 2).sum())
63+
64+
65+# ------------------------------------------------------------- fondamentaux
66+def time_weight(days_left: int) -> float:
67+ if settings.fundamentals_horizon_days <= 0:
68+ return 0.0
69+ return settings.fundamentals_weight_max * _clip(days_left / settings.fundamentals_horizon_days, 0.0, 1.0)
70+
71+
72+def fundamentals_blend(mu_logit: np.ndarray, var_logit: np.ndarray, last_pct: np.ndarray, fund: dict | None, days_left: int, incumbent: str = "caq") -> tuple[np.ndarray, np.ndarray, dict]:
73+ """Mélange logit de l'état latent (sondages) et du prior fondamentaux. Retourne (mu, var, détail)."""
74+ detail = {"enabled": False, "weight": 0.0}
75+ if not fund or not fund.get("enabled") or fund.get("satisfaction") is None or incumbent not in PARTIES_ALL:
76+ return mu_logit, var_logit, detail
77+ w = time_weight(days_left)
78+ sat = float(fund["satisfaction"])
79+ inc_prior = _clip(settings.fundamentals_alpha + settings.fundamentals_beta * sat, 5.0, 60.0)
80+ j_inc = PARTIES_ALL.index(incumbent)
81+ polls = last_pct.copy() # % somme 100
82+ # les autres partis se partagent le reste au prorata des sondages (le prior n'informe que le sortant)
83+ rest_polls = polls.sum() - polls[j_inc]
84+ rest_prior = 100.0 - inc_prior
85+ prior = polls * (rest_prior / rest_polls) if rest_polls > 0 else polls.copy()
86+ prior[j_inc] = inc_prior
87+ prior_logit = np.array([_logit(p / 100.0) for p in prior])
88+ sd_pp = settings.fundamentals_sd_pp
89+ prior_var_logit = np.array([(sd_pp / 100.0) ** 2 / max((p / 100.0) * (1 - p / 100.0), 1e-3) ** 2 for p in prior])
90+ mu = (1 - w) * mu_logit + w * prior_logit
91+ var = var_logit + (w ** 2) * prior_var_logit
92+ # traduction en points de % (pour l'affichage) : moyenne après mélange vs avant
93+ def to_pct(z: np.ndarray) -> np.ndarray:
94+ p = 1 / (1 + np.exp(-z))
95+ return 100.0 * p / p.sum()
96+ before, after = to_pct(mu_logit), to_pct(mu)
97+ detail = {"enabled": True, "weight": round(w, 3), "daysLeft": days_left, "satisfaction": round(sat, 1), "satisfactionPolls": fund.get("satisfactionPolls"),
98+ "incumbent": incumbent, "incumbentPrior": round(inc_prior, 1), "prior": {p: round(float(v), 1) for p, v in zip(PARTIES_ALL, prior)},
99+ "shiftPp": {p: round(float(a - b), 2) for p, a, b in zip(PARTIES_ALL, after, before)},
100+ "params": {"alpha": settings.fundamentals_alpha, "beta": settings.fundamentals_beta, "sdPp": sd_pp, "weightMax": settings.fundamentals_weight_max, "horizonDays": settings.fundamentals_horizon_days}}
101+ return mu, var, detail
102+
103+
104+# ----------------------------------------------------------------- momentum
105+def momentum_shift(last_pct: np.ndarray, fund: dict | None, attention: dict | None, media: dict | None) -> tuple[np.ndarray, dict]:
106+ """Nudge additif (pp) par parti, borné et recentré ; détail par composante."""
107+ P = len(PARTIES_ALL)
108+ parts = {"leader": {}, "attention": {}, "media": {}}
109+ shift = np.zeros(P)
110+ for j, pid in enumerate(PARTY_IDS):
111+ v = float(last_pct[j])
112+ comp = 0.0
113+ if fund and fund.get("enabled"):
114+ bp = (fund.get("bestPmShare") or {}).get(pid)
115+ if bp is not None:
116+ x = settings.momentum_leader_kappa * (float(bp) - v)
117+ parts["leader"][pid] = round(x, 2)
118+ comp += x
119+ if attention and attention.get("enabled"):
120+ d = (attention.get("delta") or {}).get(pid)
121+ if d is not None:
122+ x = settings.momentum_attention_kappa * float(d)
123+ parts["attention"][pid] = round(x, 2)
124+ comp += x
125+ if media and media.get("enabled"):
126+ t = (media.get("tone") or {}).get(pid)
127+ n = (media.get("counts") or {}).get(pid, 0)
128+ if t is not None and n:
129+ x = settings.momentum_media_kappa * float(t) * min(1.0, n / 10.0)
130+ parts["media"][pid] = round(x, 2)
131+ comp += x
132+ shift[j] = _clip(comp, -settings.momentum_max_pp, settings.momentum_max_pp)
133+ main = shift[: len(PARTY_IDS)]
134+ if np.any(main):
135+ main = main - main.mean() # somme nulle : le momentum redistribue, il ne crée pas de vote
136+ shift[: len(PARTY_IDS)] = main
137+ detail = {"enabled": bool(np.any(shift)), "shiftPp": {p: round(float(x), 2) for p, x in zip(PARTIES_ALL, shift)}, "parts": parts,
138+ "params": {"leaderKappa": settings.momentum_leader_kappa, "attentionKappa": settings.momentum_attention_kappa, "mediaKappa": settings.momentum_media_kappa, "maxPp": settings.momentum_max_pp}}
139+ return shift, detail
140+
141+
142+def apply_shift(national: np.ndarray, shift: np.ndarray) -> np.ndarray:
143+ """Décale les tirages nationaux (S, P) de `shift` pp, puis renormalise (plancher 0,2 %)."""
144+ if not np.any(shift):
145+ return national
146+ out = np.clip(national + shift[None, :], 0.2, None)
147+ return 100.0 * out / out.sum(axis=1, keepdims=True)
148+
149+
150+# ------------------------------------------------------------------ marchés
151+@dataclass
152+class Tempering:
153+ weights: np.ndarray
154+ detail: dict = field(default_factory=dict)
155+
156+
157+def market_tempering(seats: np.ndarray, majority: int, markets: dict | None, w_m: float | None = None) -> Tempering:
158+ """Repondération IPF des simulations vers (1−w)·modèle + w·marché sur : gagnant (plus de sièges),
159+ majorité (parti concerné), tranches de sièges (poids réduit de moitié : marchés moins liquides)."""
160+ S, P = seats.shape
161+ w = np.ones(S)
162+ w_m = settings.markets_weight if w_m is None else w_m
163+ detail = {"enabled": False, "weight": w_m}
164+ if not markets or w_m <= 0:
165+ return Tempering(w, detail)
166+ most = seats.argmax(axis=1)
167+ constraints: list[tuple[np.ndarray, dict[int, float], str]] = []
168+ # 1) plus de sièges
169+ ms = markets.get("mostSeats") or {}
170+ if ms:
171+ model = {j: float((most == j).mean()) for j in range(P)}
172+ target = {}
173+ for j, pid in enumerate(PARTIES_ALL):
174+ mk = ms.get(pid)
175+ target[j] = (1 - w_m) * model[j] + w_m * float(mk) if mk is not None else model[j]
176+ tot = sum(target.values()) or 1
177+ target = {j: v / tot for j, v in target.items()}
178+ constraints.append((most, target, "mostSeats"))
179+ # 2) majorité
180+ for pid, pm in (markets.get("majority") or {}).items():
181+ if pid in PARTIES_ALL:
182+ j = PARTIES_ALL.index(pid)
183+ ind = (seats[:, j] >= majority).astype(int)
184+ model_p = float(ind.mean())
185+ t = (1 - w_m) * model_p + w_m * float(pm)
186+ constraints.append((ind, {1: t, 0: 1 - t}, f"majority:{pid}"))
187+ # 3) tranches de sièges (poids réduit)
188+ wb = w_m / 2
189+ for pid, sb in (markets.get("seatBands") or {}).items():
190+ if pid not in PARTIES_ALL or not sb.get("bands"):
191+ continue
192+ j = PARTIES_ALL.index(pid)
193+ bands = sb["bands"]
194+ cat = np.full(S, -1)
195+ for k, b in enumerate(bands):
196+ lo, hi = b["lo"], b["hi"] if b["hi"] is not None else 10**6
197+ cat[(seats[:, j] >= lo) & (seats[:, j] <= hi)] = k
198+ if (cat < 0).any():
199+ continue
200+ model = {k: float((cat == k).mean()) for k in range(len(bands))}
201+ target = {k: (1 - wb) * model[k] + wb * float(bands[k]["p"]) for k in range(len(bands))}
202+ tot = sum(target.values()) or 1
203+ constraints.append((cat, {k: v / tot for k, v in target.items()}, f"seatBands:{pid}"))
204+ if not constraints:
205+ return Tempering(w, detail)
206+ for _ in range(25):
207+ for cat, target, _name in constraints:
208+ cur = {k: float(w[cat == k].sum() / w.sum()) for k in target}
209+ for k, t in target.items():
210+ if cur.get(k, 0) > 1e-9 and t > 0:
211+ w[cat == k] *= t / cur[k]
212+ elif t <= 0:
213+ w[cat == k] *= 0.0
214+ w = np.clip(w, 0.02, 50.0)
215+ w *= S / w.sum()
216+ achieved = {}
217+ for cat, target, name in constraints:
218+ achieved[name] = {str(k): round(float(w[cat == k].sum() / w.sum()), 4) for k in target}
219+ detail = {"enabled": True, "weight": w_m, "bandWeight": wb, "ess": round(ess(w), 0), "constraints": [c[2] for c in constraints], "achieved": achieved,
220+ "marketAsOf": markets.get("asOf"), "marketVolume": markets.get("volume"), "mostSeatsMarket": markets.get("mostSeats"), "majorityMarket": markets.get("majority"),
221+ "seatBandsImpliedMean": {p: v.get("impliedMean") for p, v in (markets.get("seatBands") or {}).items()}}
222+ return Tempering(w, detail)
223+
224+
225+# --------------------------------------------------------------- partielles
226+def byelection_adjustments(byel: dict | None, ridings: list, trend: list[dict], national_2022: dict[str, float]) -> tuple[np.ndarray, dict]:
227+ """Matrice (R, P) d'ajustements logit : κ × (swing local observé à la partielle − swing national du jour)."""
228+ R, P = len(ridings), len(PARTIES_ALL)
229+ adj = np.zeros((R, P))
230+ detail = {"enabled": False, "ridings": []}
231+ if not byel or not byel.get("enabled") or not trend:
232+ return adj, detail
233+ from ..util import norm_name
234+ by_name = {norm_name(r.name): i for i, r in enumerate(ridings) if getattr(r, "name", None)}
235+ by_date = {t["date"]: t["mean"] for t in trend}
236+ dates = sorted(by_date)
237+ for b in byel.get("byElections", []):
238+ key = norm_name(b.get("riding") or "")
239+ i = by_name.get(key)
240+ if i is None:
241+ i = next((idx for k, idx in by_name.items() if k.startswith(key) and key), None)
242+ if i is None or not b.get("previous"):
243+ continue
244+ d = b["date"]
245+ nat_d = by_date.get(d) or by_date.get(next((x for x in dates if x >= d), dates[-1]))
246+ res = {r["party_id"]: r["share"] for r in b.get("results", []) if r.get("party_id")}
247+ row = {}
248+ for j, pid in enumerate(PARTY_IDS):
249+ if pid not in res or pid not in b["previous"] or pid not in nat_d:
250+ continue
251+ local = _logit(res[pid] / 100) - _logit(b["previous"][pid] / 100)
252+ national = _logit(nat_d[pid] / 100) - _logit(national_2022.get(pid, 1) / 100)
253+ x = _clip(settings.byelection_kappa * (local - national), -1.0, 1.0)
254+ adj[i, j] = x
255+ row[pid] = round(x, 3)
256+ detail["ridings"].append({"code": ridings[i].code, "riding": b.get("riding"), "date": d, "adjLogit": row, "kappa": settings.byelection_kappa})
257+ detail["enabled"] = bool(detail["ridings"])
258+ return adj, detail
modified engine/qc26/model/ridings.py +1 −0
@@ -31,6 +31,7 @@ class RidingInput:
31 electors: int = 50000 31 electors: int = 50000
32 pct_french: float | None = None # langue maternelle française (recensement 2021) 32 pct_french: float | None = None # langue maternelle française (recensement 2021)
33 poll_region: str = "reste" 33 poll_region: str = "reste"
34+ name: str = ""
34 35
35 36
36 def simulate_ridings(ridings: list[RidingInput], national_draws: np.ndarray, national_2022: dict[str, float], 37 def simulate_ridings(ridings: list[RidingInput], national_draws: np.ndarray, national_2022: dict[str, float],
modified engine/qc26/model/run.py +131 −50
@@ -1,5 +1,7 @@
1−"""Exécution d'un run de prévision : sondages → latent → Monte Carlo → sièges → 1+"""Exécution d'un run de prévision (modèle 2.0) : sondages → latent → mélange fondamentaux →
2−persistance versionnée → « Qu'est-ce qui a changé ? ».""" 2+Monte Carlo → momentum → sièges → tempérage par les marchés (repondération) → persistance
3+versionnée → « Qu'est-ce qui a changé ? ». Chaque composante est publiée séparément (summary.components)
4+et la vue « sondages seuls » (pondération uniforme, sans fondamentaux ni momentum) est conservée."""
3 from __future__ import annotations 5 from __future__ import annotations
4 6
5 import logging 7 import logging
@@ -17,6 +19,7 @@ from ..db import (Candidate, ForecastHouseEffect, ForecastPartyResult, ForecastR
17 from ..parties import PARTY_IDS 19 from ..parties import PARTY_IDS
18 from ..regions import POLL_REGION_LABELS, poll_region_for 20 from ..regions import POLL_REGION_LABELS, poll_region_for
19 from .aggregate import PARTIES_ALL, Obs, compute_weights, election_day_distribution, house_effects, kalman_latent 21 from .aggregate import PARTIES_ALL, Obs, compute_weights, election_day_distribution, house_effects, kalman_latent
22+from .ensemble import (apply_shift, byelection_adjustments, ess, fundamentals_blend, market_tempering, momentum_shift, whist, wmean, wquantile)
20 from .ridings import RidingInput, explain_factors, rating, simulate_ridings 23 from .ridings import RidingInput, explain_factors, rating, simulate_ridings
21 from .segments import build_segment_model, riding_adjustments 24 from .segments import build_segment_model, riding_adjustments
22 25
@@ -57,7 +60,7 @@ def load_ridings(s: Session) -> tuple[list[RidingInput], dict[str, float]]:
57 out = [RidingInput(code=r.code, region_id=r.region_id or "inconnue", baseline=baselines.get(r.code, {}), 60 out = [RidingInput(code=r.code, region_id=r.region_id or "inconnue", baseline=baselines.get(r.code, {}),
58 incumbent_party=r.incumbent_party_id, incumbent_running=r.incumbent_running, candidates=cands.get(r.code, set()), 61 incumbent_party=r.incumbent_party_id, incumbent_running=r.incumbent_running, candidates=cands.get(r.code, set()),
59 electors=r.electors_2026 or 50000, pct_french=(demo[r.code].pct_french_mt if r.code in demo else None), 62 electors=r.electors_2026 or 50000, pct_french=(demo[r.code].pct_french_mt if r.code in demo else None),
60− poll_region=poll_region_for(r.code)) 63+ poll_region=poll_region_for(r.code), name=r.name)
61 for r in rows if baselines.get(r.code)] 64 for r in rows if baselines.get(r.code)]
62 national_2022 = {p.id: (p.vote_2022 or 0.0) for p in s.scalars(select(Party)).all()} 65 national_2022 = {p.id: (p.vote_2022 or 0.0) for p in s.scalars(select(Party)).all()}
63 national_2022["aut"] = max(0.5, 100.0 - sum(national_2022.values())) 66 national_2022["aut"] = max(0.5, 100.0 - sum(national_2022.values()))
@@ -70,6 +73,36 @@ def _draw_national(mu: np.ndarray, cov: np.ndarray, n: int, rng: np.random.Gener
70 return 100.0 * p / p.sum(axis=1, keepdims=True) 73 return 100.0 * p / p.sum(axis=1, keepdims=True)
71 74
72 75
76+def _load_signals(s: Session, cutoff: date) -> dict:
77+ """Signaux disponibles en base (jamais interrogés en ligne pendant un run)."""
78+ out = {"markets": None, "fundamentals": None, "attention": None, "media": None, "byelections": None}
79+ try:
80+ from ..signals import attention, byelections, fundamentals, markets, media
81+ out["markets"] = markets.implied(s)
82+ out["fundamentals"] = fundamentals.fundamentals_summary(s, cutoff)
83+ out["attention"] = attention.attention_summary(s, cutoff)
84+ out["media"] = media.media_summary(s, cutoff)
85+ out["byelections"] = byelections.byelection_summary(s)
86+ except Exception as e: # noqa: BLE001
87+ log.warning("signaux indisponibles : %s", e)
88+ return out
89+
90+
91+def _party_stats(national: np.ndarray, seats: np.ndarray, w: np.ndarray, majority: int, seats_total: int) -> dict:
92+ most = seats.argmax(axis=1)
93+ out = {}
94+ for j, pid in enumerate(PARTIES_ALL):
95+ v, sj = national[:, j], seats[:, j]
96+ out[pid] = {"vote": round(wmean(v, w), 2), "vote_sd": round(float(np.sqrt(np.average((v - wmean(v, w)) ** 2, weights=w))), 2),
97+ "vote80": [round(wquantile(v, w, 0.10), 1), round(wquantile(v, w, 0.90), 1)],
98+ "vote95": [round(wquantile(v, w, 0.025), 1), round(wquantile(v, w, 0.975), 1)],
99+ "seats": int(wquantile(sj.astype(float), w, 0.5)), "seats_mean": round(wmean(sj, w), 1),
100+ "seats80": [int(wquantile(sj.astype(float), w, 0.10)), int(wquantile(sj.astype(float), w, 0.90))],
101+ "seats95": [int(wquantile(sj.astype(float), w, 0.025)), int(wquantile(sj.astype(float), w, 0.975))],
102+ "p_most_seats": round(float(np.average(most == j, weights=w)), 4), "p_majority": round(float(np.average(sj >= majority, weights=w)), 4)}
103+ return out
104+
105+
73 def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None, publish: bool = True, 106 def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None, publish: bool = True,
74 n_sims: int | None = None) -> ForecastRun: 107 n_sims: int | None = None) -> ForecastRun:
75 cutoff = cutoff or date.today() 108 cutoff = cutoff or date.today()
@@ -84,66 +117,87 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
84 obs = load_observations(s, cutoff) 117 obs = load_observations(s, cutoff)
85 if len(obs) < 3: 118 if len(obs) < 3:
86 raise RuntimeError("trop peu de sondages") 119 raise RuntimeError("trop peu de sondages")
120+ days_left = max(0, (settings.election_date - cutoff).days)
87 weights = compute_weights(obs, cutoff) 121 weights = compute_weights(obs, cutoff)
88 effects = house_effects(obs, cutoff) 122 effects = house_effects(obs, cutoff)
89 latent = kalman_latent(obs, cutoff, effects) 123 latent = kalman_latent(obs, cutoff, effects)
90− mu, cov = election_day_distribution(latent, cutoff, settings.election_date) 124+ signals = _load_signals(s, cutoff)
91− national = _draw_national(mu, cov, n_sims, rng) # (S, P) % 125+
126+ # --- 1. vote national : sondages seuls, puis mélange fondamentaux ---
127+ mu_polls, cov_polls = election_day_distribution(latent, cutoff, settings.election_date)
128+ national_polls = _draw_national(mu_polls, cov_polls, n_sims, rng) # (S, P) % — sondages seuls
129+ mu_f, var_f, fund_detail = fundamentals_blend(latent["mean_logit"], latent["var_logit"], latent["last_pct"], signals["fundamentals"], days_left)
130+ if fund_detail["enabled"]:
131+ latent_f = {**latent, "mean_logit": mu_f, "var_logit": var_f}
132+ mu_e, cov_e = election_day_distribution(latent_f, cutoff, settings.election_date)
133+ rng_e = np.random.default_rng(seed) # mêmes aléas → différence attribuable au seul mélange
134+ national = _draw_national(mu_e, cov_e, n_sims, rng_e)
135+ else:
136+ national = national_polls.copy()
137+ # --- 2. momentum (nudge borné, somme nulle) ---
138+ shift, mom_detail = momentum_shift(latent["last_pct"], signals["fundamentals"], signals["attention"], signals["media"])
139+ national = apply_shift(national, shift)
92 140
141+ # --- 3. circonscriptions ---
93 ridings, national_2022 = load_ridings(s) 142 ridings, national_2022 = load_ridings(s)
94 national_now_mean = {p: float(national[:, j].mean()) for j, p in enumerate(PARTIES_ALL)} 143 national_now_mean = {p: float(national[:, j].mean()) for j, p in enumerate(PARTIES_ALL)}
95 seg_model, f_r = build_segment_model(s, cutoff, national_now_mean, national_2022) 144 seg_model, f_r = build_segment_model(s, cutoff, national_now_mean, national_2022)
96 seg_adj = riding_adjustments(seg_model, [r.code for r in ridings], f_r) 145 seg_adj = riding_adjustments(seg_model, [r.code for r in ridings], f_r)
97− winners, shares = simulate_ridings(ridings, national, national_2022, rng, seg_adj=seg_adj) 146+ be_adj, be_detail = byelection_adjustments(signals["byelections"], ridings, latent["series"], national_2022)
147+ total_adj = seg_adj + be_adj
148+ rng_r = np.random.default_rng(seed + 1)
149+ winners, shares = simulate_ridings(ridings, national, national_2022, rng_r, seg_adj=total_adj)
98 S, R, P = shares.shape 150 S, R, P = shares.shape
99 seats_total = int(get_setting(s, "seats_total", R)) 151 seats_total = int(get_setting(s, "seats_total", R))
100 majority = int(get_setting(s, "majority_threshold", seats_total // 2 + 1)) 152 majority = int(get_setting(s, "majority_threshold", seats_total // 2 + 1))
101 seats = np.zeros((S, P), dtype=int) 153 seats = np.zeros((S, P), dtype=int)
102 for j in range(P): 154 for j in range(P):
103 seats[:, j] = (winners == j).sum(axis=1) 155 seats[:, j] = (winners == j).sum(axis=1)
104− most = seats.argmax(axis=1) 156+ # sondages seuls (mêmes aléas de circonscription, sans fondamentaux, momentum ni partielles) — pour comparaison
157+ rng_r0 = np.random.default_rng(seed + 1)
158+ winners0, _ = simulate_ridings(ridings, national_polls, national_2022, rng_r0, seg_adj=seg_adj)
159+ seats0 = np.zeros((S, P), dtype=int)
160+ for j in range(P):
161+ seats0[:, j] = (winners0 == j).sum(axis=1)
162+
163+ # --- 4. tempérage par les marchés : repondération des simulations ---
164+ temp = market_tempering(seats, majority, signals["markets"])
165+ w = temp.weights
166+ uniform = np.ones(S)
105 167
106− summary_parties = {} 168+ # --- statistiques pondérées ---
169+ most = seats.argmax(axis=1)
170+ summary_parties = _party_stats(national, seats, w, majority, seats_total)
171+ polls_only = _party_stats(national_polls, seats0, uniform, majority, seats_total)
172+ model_only = _party_stats(national, seats, uniform, majority, seats_total) # fondamentaux + momentum, sans marchés
107 s.execute(delete(ForecastPartyResult).where(ForecastPartyResult.run_id == run.id)) 173 s.execute(delete(ForecastPartyResult).where(ForecastPartyResult.run_id == run.id))
108 for j, pid in enumerate(PARTIES_ALL): 174 for j, pid in enumerate(PARTIES_ALL):
109− v = national[:, j] 175+ v, sj, sp = national[:, j], seats[:, j], summary_parties[pid]
110− sj = seats[:, j] 176+ s.add(ForecastPartyResult(
111− hist = np.bincount(sj, minlength=seats_total + 1).tolist() 177+ run_id=run.id, party_id=pid, vote_mean=sp["vote"], vote_sd=sp["vote_sd"],
112− fpr = ForecastPartyResult( 178+ vote_lo80=sp["vote80"][0], vote_hi80=sp["vote80"][1], vote_lo95=sp["vote95"][0], vote_hi95=sp["vote95"][1],
113− run_id=run.id, party_id=pid, vote_mean=float(v.mean()), vote_sd=float(v.std()), 179+ seats_mean=sp["seats_mean"], seats_median=sp["seats"], seats_lo80=sp["seats80"][0], seats_hi80=sp["seats80"][1],
114− vote_lo80=float(np.percentile(v, 10)), vote_hi80=float(np.percentile(v, 90)), 180+ seats_lo95=sp["seats95"][0], seats_hi95=sp["seats95"][1], p_most_seats=sp["p_most_seats"], p_majority=sp["p_majority"],
115− vote_lo95=float(np.percentile(v, 2.5)), vote_hi95=float(np.percentile(v, 97.5)), 181+ seat_distribution=whist(sj, w, seats_total + 1)))
116− seats_mean=float(sj.mean()), seats_median=int(np.median(sj)), 182+ p_minority = float(np.average(seats.max(axis=1) < majority, weights=w))
117− seats_lo80=int(np.percentile(sj, 10)), seats_hi80=int(np.percentile(sj, 90)),
118− seats_lo95=int(np.percentile(sj, 2.5)), seats_hi95=int(np.percentile(sj, 97.5)),
119− p_most_seats=float((most == j).mean()), p_majority=float((sj >= majority).mean()),
120− seat_distribution=hist)
121− s.add(fpr)
122− summary_parties[pid] = {"vote": round(fpr.vote_mean, 2), "vote_sd": round(fpr.vote_sd, 2),
123− "vote80": [round(fpr.vote_lo80, 1), round(fpr.vote_hi80, 1)],
124− "seats": fpr.seats_median, "seats_mean": round(fpr.seats_mean, 1),
125− "seats80": [fpr.seats_lo80, fpr.seats_hi80], "seats95": [fpr.seats_lo95, fpr.seats_hi95],
126− "p_most_seats": round(fpr.p_most_seats, 4), "p_majority": round(fpr.p_majority, 4)}
127− p_minority = float((seats.max(axis=1) < majority).mean())
128 183
129 # circonscriptions 184 # circonscriptions
130 s.execute(delete(ForecastRidingResult).where(ForecastRidingResult.run_id == run.id)) 185 s.execute(delete(ForecastRidingResult).where(ForecastRidingResult.run_id == run.id))
131 s.execute(delete(ForecastRidingSummary).where(ForecastRidingSummary.run_id == run.id)) 186 s.execute(delete(ForecastRidingSummary).where(ForecastRidingSummary.run_id == run.id))
132− national_now = {p: round(float(national[:, j].mean()), 2) for j, p in enumerate(PARTIES_ALL)} 187+ national_now = {p: summary_parties[p]["vote"] for p in PARTIES_ALL}
133− battlegrounds = [] 188+ battlegrounds, riding_rows = [], []
134− riding_rows = [] 189+ be_codes = {x["code"]: x for x in be_detail.get("ridings", [])}
135 for i, r in enumerate(ridings): 190 for i, r in enumerate(ridings):
136 sh = shares[:, i, :] 191 sh = shares[:, i, :]
137− pw = np.array([(winners[:, i] == j).mean() for j in range(P)]) 192+ pw = np.array([float(np.average(winners[:, i] == j, weights=w)) for j in range(P)])
138 order = np.argsort(-pw) 193 order = np.argsort(-pw)
139 fav, second = int(order[0]), int(order[1]) 194 fav, second = int(order[0]), int(order[1])
140− share_mean = {p: float(sh[:, j].mean()) for j, p in enumerate(PARTIES_ALL)} 195+ share_mean = {p: wmean(sh[:, j], w) for j, p in enumerate(PARTIES_ALL)}
141 for j, pid in enumerate(PARTIES_ALL): 196 for j, pid in enumerate(PARTIES_ALL):
142 if pid == "aut" and share_mean[pid] < 1.5: 197 if pid == "aut" and share_mean[pid] < 1.5:
143 continue 198 continue
144 s.add(ForecastRidingResult(run_id=run.id, riding_code=r.code, party_id=pid, vote_mean=share_mean[pid], 199 s.add(ForecastRidingResult(run_id=run.id, riding_code=r.code, party_id=pid, vote_mean=share_mean[pid],
145− vote_lo80=float(np.percentile(sh[:, j], 10)), vote_hi80=float(np.percentile(sh[:, j], 90)), 200+ vote_lo80=wquantile(sh[:, j], w, 0.10), vote_hi80=wquantile(sh[:, j], w, 0.90), p_win=float(pw[j])))
146− p_win=float(pw[j])))
147 fav_id = PARTIES_ALL[fav] 201 fav_id = PARTIES_ALL[fav]
148 margin = share_mean[fav_id] - share_mean[PARTIES_ALL[second]] 202 margin = share_mean[fav_id] - share_mean[PARTIES_ALL[second]]
149 swing = share_mean[fav_id] - r.baseline.get(fav_id, 0.0) 203 swing = share_mean[fav_id] - r.baseline.get(fav_id, 0.0)
@@ -154,11 +208,21 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
154 if r.poll_region in seg_model.current and fav_id in seg_model.current[r.poll_region]: 208 if r.poll_region in seg_model.current and fav_id in seg_model.current[r.poll_region]:
155 seg_detail["region"] = {"label": POLL_REGION_LABELS.get(r.poll_region, r.poll_region), "now": seg_model.current[r.poll_region][fav_id], 209 seg_detail["region"] = {"label": POLL_REGION_LABELS.get(r.poll_region, r.poll_region), "now": seg_model.current[r.poll_region][fav_id],
156 "2022": seg_model.baseline.get(r.poll_region, {}).get(fav_id, 0)} 210 "2022": seg_model.baseline.get(r.poll_region, {}).get(fav_id, 0)}
211+ factors = explain_factors(r, national_now, national_2022, fav_id, share_mean, seg_adj_fav=float(seg_adj[i, fav]), seg_detail=seg_detail)
212+ if r.code in be_codes:
213+ be = be_codes[r.code]
214+ a = be["adjLogit"].get(fav_id, 0.0)
215+ _d = date.fromisoformat(be["date"])
216+ _mois = ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"][_d.month - 1]
217+ factors.insert(2, {"factor": "partielle", "label": f"Élection partielle du {_d.day}{'er' if _d.day == 1 else ''} {_mois} {_d.year} (résultat officiel)",
218+ "detail": f"excès de swing local observé, net du swing national du jour, rétréci (κ {be['kappa']}) : {a:+.2f} logit pour {fav_id.upper()}",
219+ "direction": "+" if a > 0.02 else "-" if a < -0.02 else "±"})
220+ if temp.detail.get("enabled"):
221+ factors.append({"factor": "marches", "label": "Tempérage par les marchés de prédiction (Polymarket)",
222+ "detail": f"simulations repondérées (poids {temp.detail['weight']}) vers les probabilités implicites nationales ; effet indirect sur la circonscription", "direction": "±"})
157 summ = ForecastRidingSummary(run_id=run.id, riding_code=r.code, favorite_party_id=fav_id, favorite_p=float(pw[fav]), 223 summ = ForecastRidingSummary(run_id=run.id, riding_code=r.code, favorite_party_id=fav_id, favorite_p=float(pw[fav]),
158 runner_up_party_id=PARTIES_ALL[second], margin_mean=float(margin), 224 runner_up_party_id=PARTIES_ALL[second], margin_mean=float(margin),
159− volatility=float(1 - pw[fav]), rating=rating(float(pw[fav])), swing_from_2022=float(swing), 225+ volatility=float(1 - pw[fav]), rating=rating(float(pw[fav])), swing_from_2022=float(swing), factors=factors)
160− factors=explain_factors(r, national_now, national_2022, fav_id, share_mean,
161− seg_adj_fav=float(seg_adj[i, fav]), seg_detail=seg_detail))
162 s.add(summ) 226 s.add(summ)
163 riding_rows.append((r.code, fav_id, float(pw[fav]), PARTIES_ALL[second], float(margin))) 227 riding_rows.append((r.code, fav_id, float(pw[fav]), PARTIES_ALL[second], float(margin)))
164 battlegrounds.append((float(1 - pw[fav]), r.code)) 228 battlegrounds.append((float(1 - pw[fav]), r.code))
@@ -177,7 +241,7 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
177 recent_ids = [o.poll_id for o in obs if (cutoff - o.end).days <= 45] 241 recent_ids = [o.poll_id for o in obs if (cutoff - o.end).days <= 45]
178 base_last = {p: float(x) for p, x in zip(PARTIES_ALL, latent["last_pct"])} 242 base_last = {p: float(x) for p, x in zip(PARTIES_ALL, latent["last_pct"])}
179 for o in obs: 243 for o in obs:
180− w = weights[o.poll_id] 244+ wgt = weights[o.poll_id]
181 impact = None 245 impact = None
182 if o.poll_id in recent_ids and len(obs) > 4: 246 if o.poll_id in recent_ids and len(obs) > 4:
183 sub = [x for x in obs if x.poll_id != o.poll_id] 247 sub = [x for x in obs if x.poll_id != o.poll_id]
@@ -186,29 +250,38 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
186 impact = {p: round(base_last[p] - float(x), 2) for p, x in zip(PARTIES_ALL, lat2["last_pct"])} 250 impact = {p: round(base_last[p] - float(x), 2) for p, x in zip(PARTIES_ALL, lat2["last_pct"])}
187 except Exception: # noqa: BLE001 251 except Exception: # noqa: BLE001
188 impact = None 252 impact = None
189− s.add(PollWeight(run_id=run.id, poll_id=o.poll_id, weight=w["weight"], recency_weight=w["recency"], 253+ s.add(PollWeight(run_id=run.id, poll_id=o.poll_id, weight=wgt["weight"], recency_weight=wgt["recency"],
190− sample_weight=w["sample"], impact=impact)) 254+ sample_weight=wgt["sample"], impact=impact))
191 255
192− # tendance quotidienne (série pour les graphiques) — allégée à 1 point / jour
193 trend = latent["series"] 256 trend = latent["series"]
194−
195− # résumé + diff avec le run publié précédent
196 prev = s.scalar(select(ForecastRun).where(ForecastRun.is_published == True, ForecastRun.kind == "forecast", # noqa: E712 257 prev = s.scalar(select(ForecastRun).where(ForecastRun.is_published == True, ForecastRun.kind == "forecast", # noqa: E712
197 ForecastRun.id != run.id).order_by(ForecastRun.id.desc())) 258 ForecastRun.id != run.id).order_by(ForecastRun.id.desc()))
198 top = sorted(summary_parties.items(), key=lambda kv: -kv[1]["seats_mean"]) 259 top = sorted(summary_parties.items(), key=lambda kv: -kv[1]["seats_mean"])
260+ components = {
261+ "pollsOnly": {"parties": polls_only, "p_minority": round(float((seats0.max(axis=1) < majority).mean()), 4)},
262+ "modelNoMarkets": {"parties": model_only, "p_minority": round(float((seats.max(axis=1) < majority).mean()), 4)},
263+ "fundamentals": fund_detail, "momentum": mom_detail, "markets": temp.detail, "byelections": be_detail,
264+ "signals": {"attention": {k: v for k, v in (signals["attention"] or {}).items() if k in ("enabled", "asOf", "share7", "share28", "delta")},
265+ "media": {k: v for k, v in (signals["media"] or {}).items() if k in ("enabled", "asOf", "articles", "shareOfVoice", "tone", "counts")},
266+ "fundamentals": {k: v for k, v in (signals["fundamentals"] or {}).items() if k in ("enabled", "asOf", "satisfaction", "satisfactionNet", "bestPmShare", "leaderNetFavorable", "secondChoice", "sovereigntyYes")},
267+ "markets": {k: v for k, v in (signals["markets"] or {}).items() if k in ("asOf", "mostSeats", "majority", "volume")} if signals["markets"] else None,
268+ "byelections": {"n": (signals["byelections"] or {}).get("n", 0), "meanSwing": (signals["byelections"] or {}).get("meanSwing")}},
269+ "ess": round(ess(w), 0), "nSimulations": n_sims,
270+ }
199 summary = { 271 summary = {
200 "seats_total": seats_total, "majority": majority, "parties": summary_parties, "p_minority": round(p_minority, 4), 272 "seats_total": seats_total, "majority": majority, "parties": summary_parties, "p_minority": round(p_minority, 4),
201 "leader": top[0][0], "poll_count": len(obs), "latest_poll": max(o.end for o in obs).isoformat(), 273 "leader": top[0][0], "poll_count": len(obs), "latest_poll": max(o.end for o in obs).isoformat(),
202− "national_now": national_now, "cutoff": cutoff.isoformat(), "days_to_election": (settings.election_date - cutoff).days, 274+ "national_now": national_now, "cutoff": cutoff.isoformat(), "days_to_election": days_left,
203 "battlegrounds": [c for _, c in sorted(battlegrounds, reverse=True)[:15]], 275 "battlegrounds": [c for _, c in sorted(battlegrounds, reverse=True)[:15]],
204 "segments": seg_model.payload(), 276 "segments": seg_model.payload(),
205 "demographics_coverage": sum(1 for r in ridings if r.pct_french is not None), 277 "demographics_coverage": sum(1 for r in ridings if r.pct_french is not None),
206 "ridings": {str(c): {"fav": f, "p": round(p, 3), "second": sec, "margin": round(m, 1)} for c, f, p, sec, m in riding_rows}, 278 "ridings": {str(c): {"fav": f, "p": round(p, 3), "second": sec, "margin": round(m, 1)} for c, f, p, sec, m in riding_rows},
279+ "components": components,
207 } 280 }
208 diff = None 281 diff = None
209− if prev and prev.summary: 282+ if prev and prev.summary and "parties" in prev.summary:
210− new_polls = [o.poll_id for o in obs if o.end > date.fromisoformat(prev.summary.get("latest_poll", "1900-01-01")) or 283+ prev_ids = {pw.poll_id for pw in s.scalars(select(PollWeight).where(PollWeight.run_id == prev.id)).all()}
211− o.poll_id not in {pw.poll_id for pw in s.scalars(select(PollWeight).where(PollWeight.run_id == prev.id)).all()}] 284+ new_polls = [o.poll_id for o in obs if o.end > date.fromisoformat(prev.summary.get("latest_poll", "1900-01-01")) or o.poll_id not in prev_ids]
212 changes = {} 285 changes = {}
213 for pid in PARTIES_ALL: 286 for pid in PARTIES_ALL:
214 a, b = prev.summary["parties"].get(pid), summary_parties[pid] 287 a, b = prev.summary["parties"].get(pid), summary_parties[pid]
@@ -224,8 +297,16 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
224 causes = [] 297 causes = []
225 if new_polls: 298 if new_polls:
226 causes.append({"kind": "new_polls", "detail": new_polls[:6]}) 299 causes.append({"kind": "new_polls", "detail": new_polls[:6]})
227− if not new_polls: 300+ prev_mk = ((prev.summary.get("components") or {}).get("markets") or {}).get("mostSeatsMarket")
228− causes.append({"kind": "time", "detail": "recalcul sans nouveau sondage (récence, dérive vers le scrutin)"}) 301+ cur_mk = temp.detail.get("mostSeatsMarket")
302+ if prev_mk and cur_mk:
303+ moved = {p: round(cur_mk.get(p, 0) - prev_mk.get(p, 0), 3) for p in cur_mk if abs(cur_mk.get(p, 0) - prev_mk.get(p, 0)) >= 0.02}
304+ if moved:
305+ causes.append({"kind": "markets", "detail": moved})
306+ if prev.model_version != settings.model_version:
307+ causes.append({"kind": "model", "detail": f"modèle {prev.model_version} → {settings.model_version}"})
308+ if not causes:
309+ causes.append({"kind": "time", "detail": "recalcul sans nouveau sondage (récence, dérive vers le scrutin, signaux mis à jour)"})
229 diff = {"previous_run": prev.id, "previous_at": prev.completed_at.isoformat() if prev.completed_at else None, 310 diff = {"previous_run": prev.id, "previous_at": prev.completed_at.isoformat() if prev.completed_at else None,
230 "changes": changes, "flips": flips, "causes": causes} 311 "changes": changes, "flips": flips, "causes": causes}
231 312
@@ -239,7 +320,7 @@ def run_forecast(s: Session, trigger: str = "manual", cutoff: date | None = None
239 run.is_published = True 320 run.is_published = True
240 set_setting(s, "latest_forecast_run", run.id) 321 set_setting(s, "latest_forecast_run", run.id)
241 s.flush() 322 s.flush()
242− log.info("run %d : %s — sièges %s", run.id, summary["leader"], {k: v["seats"] for k, v in summary_parties.items()}) 323+ log.info("run %d : %s — sièges %s (ESS %s)", run.id, summary["leader"], {k: v["seats"] for k, v in summary_parties.items()}, components["ess"])
243 return run 324 return run
244 except Exception as e: # noqa: BLE001 325 except Exception as e: # noqa: BLE001
245 run.status = "failed" 326 run.status = "failed"
modified engine/qc26/scheduler.py +52 −0
@@ -92,6 +92,54 @@ def job_documents():
92 return {"docs": res, "positions": pos} 92 return {"docs": res, "positions": pos}
93 93
94 94
95+@_guarded("signals_markets")
96+def job_signals_markets():
97+ from .signals import markets
98+ with session_scope() as s:
99+ return markets.sync_markets(s, with_history=True)
100+
101+
102+@_guarded("signals_daily")
103+def job_signals_daily():
104+ """Attention (Wikipédia), fondamentaux (questions de leadership), partielles, GDELT — une fois par jour, puis recalcul."""
105+ from .ingest import official
106+ from .model.run import run_forecast
107+ from .signals import attention, byelections, fundamentals, media
108+ out = {}
109+ with session_scope() as s:
110+ out["attention"] = attention.sync_attention(s)
111+ with session_scope() as s:
112+ out["fundamentals"] = fundamentals.extract_from_polls(s)
113+ with session_scope() as s:
114+ out["byelections"] = byelections.sync_by_elections(s)
115+ official.apply_by_election_incumbents(s)
116+ try:
117+ with session_scope() as s:
118+ out["gdelt"] = media.sync_gdelt(s)
119+ except Exception as e: # noqa: BLE001
120+ out["gdelt"] = {"error": str(e)[:200]}
121+ with session_scope() as s:
122+ run = run_forecast(s, trigger="signals_daily")
123+ out["run"] = run.id
124+ return out
125+
126+
127+@_guarded("signals_media")
128+def job_signals_media():
129+ from .signals import media
130+ with session_scope() as s:
131+ return media.sync_news(s, classify=True)
132+
133+
134+@_guarded("periodic_forecast")
135+def job_periodic_forecast():
136+ """Recalcul toutes les 6 h : les marchés bougent en continu, la récence des sondages aussi."""
137+ from .model.run import run_forecast
138+ with session_scope() as s:
139+ run = run_forecast(s, trigger="periodic")
140+ return {"run": run.id}
141+
142+
95 def _in_live_window() -> bool: 143 def _in_live_window() -> bool:
96 if settings.election_simulation: 144 if settings.election_simulation:
97 return True 145 return True
@@ -126,6 +174,10 @@ def start() -> BackgroundScheduler:
126 sch.add_job(job_daily_forecast, "cron", hour=6, minute=15, id="daily_forecast") 174 sch.add_job(job_daily_forecast, "cron", hour=6, minute=15, id="daily_forecast")
127 sch.add_job(job_documents, "interval", minutes=settings.doc_watch_minutes, id="documents", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=30)) 175 sch.add_job(job_documents, "interval", minutes=settings.doc_watch_minutes, id="documents", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=30))
128 sch.add_job(job_live, "interval", seconds=settings.live_poll_seconds, id="live") 176 sch.add_job(job_live, "interval", seconds=settings.live_poll_seconds, id="live")
177+ sch.add_job(job_signals_markets, "interval", minutes=settings.signals_markets_minutes, id="signals_markets", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=5))
178+ sch.add_job(job_signals_media, "interval", hours=settings.signals_media_hours, id="signals_media", next_run_time=datetime.now(timezone.utc) + timedelta(minutes=12))
179+ sch.add_job(job_signals_daily, "cron", hour=5, minute=40, id="signals_daily")
180+ sch.add_job(job_periodic_forecast, "cron", hour="0,12,18", minute=20, id="periodic_forecast")
129 sch.start() 181 sch.start()
130 _sched = sch 182 _sched = sch
131 log.info("planificateur démarré") 183 log.info("planificateur démarré")
added engine/qc26/signals/__init__.py +8 −0
@@ -0,0 +1,8 @@
1+"""Signaux « au-delà des sondages » (modèle 2.0) : marchés de prédiction, attention publique,
2+couverture médiatique, fondamentaux tirés des questions de leadership, élections partielles.
3+
4+Chaque connecteur écrit des observations datées et sourcées dans `signals` (et les tables
5+dédiées `markets`, `market_prices`, `media_items`, `by_elections`). Le modèle (`model/ensemble.py`)
6+n'utilise que ce qui est en base : aucun signal n'est inventé, l'absence d'un signal désactive
7+simplement la composante correspondante.
8+"""
added engine/qc26/signals/attention.py +113 −0
@@ -0,0 +1,113 @@
1+"""Attention publique — pages vues Wikipédia (API REST Wikimedia, publique, sans clé).
2+
3+Pour chaque parti : page du parti + page de son·sa chef·fe (Wikipédia FR), vues quotidiennes
4+« user » (robots exclus) depuis le 1er juin 2026. Le signal dérivé est la **part d'attention**
5+(part des vues du parti dans le total des cinq partis) et son évolution 7 j vs 28 j. C'est un
6+indicateur de saillance, pas d'intention de vote : il n'entre dans le modèle que comme
7+composante de momentum bornée.
8+"""
9+from __future__ import annotations
10+
11+import logging
12+from datetime import date, datetime, timedelta
13+from urllib.parse import quote
14+
15+import httpx
16+from sqlalchemy import select
17+from sqlalchemy.orm import Session
18+
19+from ..config import settings
20+from ..db import Signal, Source, utcnow
21+from ..parties import PARTY_IDS
22+from .markets import _upsert_signal
23+
24+log = logging.getLogger("qc26.signals.attention")
25+
26+# Pages Wikipédia FR (titres exacts) — parti puis chef·fe (registre DGEQ ; pour QS la cheffe
27+# officielle est peu connue du public : on suit la co-porte-parole parlementaire).
28+PAGES: dict[str, list[str]] = {
29+ "caq": ["Coalition_avenir_Québec", "Christine_Fréchette"],
30+ "plq": ["Parti_libéral_du_Québec", "Charles_Milliard"],
31+ "pq": ["Parti_québécois", "Paul_St-Pierre_Plamondon"],
32+ "qs": ["Québec_solidaire", "Ruba_Ghazal"],
33+ "pcq": ["Parti_conservateur_du_Québec_(2009)", "Éric_Duhaime"],
34+}
35+ELECTION_PAGE = "Élections_générales_québécoises_de_2026"
36+START = date(2026, 6, 1)
37+
38+
39+def fetch_views(title: str, start: date, end: date) -> dict[date, int]:
40+ url = (f"https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/fr.wikipedia/all-access/user/"
41+ f"{quote(title, safe='')}/daily/{start:%Y%m%d}/{end:%Y%m%d}")
42+ with httpx.Client(timeout=30, headers={"User-Agent": "QC26Bot/1.0 (https://www.qc26.ca; contact@qc26.ca)"}) as c:
43+ r = c.get(url)
44+ if r.status_code == 404:
45+ return {}
46+ r.raise_for_status()
47+ items = r.json().get("items", [])
48+ return {datetime.strptime(i["timestamp"][:8], "%Y%m%d").date(): int(i["views"]) for i in items}
49+
50+
51+def sync_attention(s: Session) -> dict:
52+ src = s.get(Source, "wikipedia-pageviews")
53+ if not src:
54+ src = Source(id="wikipedia-pageviews", name="Wikimedia — pages vues Wikipédia FR (partis, chef·fes)", url="https://wikimedia.org/api/rest_v1/", tier=3, kind="signal")
55+ s.add(src)
56+ src.last_checked_at = utcnow()
57+ end = date.today() - timedelta(days=1)
58+ n, errors = 0, 0
59+ # reprise : on repart 10 jours avant la dernière observation connue
60+ last = s.scalar(select(Signal.date).where(Signal.source == "wikipedia", Signal.key == "pageviews").order_by(Signal.date.desc()))
61+ start = max(START, (last - timedelta(days=10))) if last else START
62+ for pid, titles in PAGES.items():
63+ total: dict[date, int] = {}
64+ detail: dict[str, dict[date, int]] = {}
65+ for t in titles:
66+ try:
67+ v = fetch_views(t, start, end)
68+ detail[t] = v
69+ for d, x in v.items():
70+ total[d] = total.get(d, 0) + x
71+ except Exception as e: # noqa: BLE001
72+ errors += 1
73+ log.warning("pageviews %s : %s", t, e)
74+ for d, x in total.items():
75+ _upsert_signal(s, "wikipedia", "pageviews", pid, d, float(x), url=f"https://fr.wikipedia.org/wiki/{titles[0]}",
76+ meta={"pages": {t: detail[t].get(d) for t in detail}})
77+ n += 1
78+ try:
79+ for d, x in fetch_views(ELECTION_PAGE, start, end).items():
80+ _upsert_signal(s, "wikipedia", "pageviews_election", None, d, float(x), url=f"https://fr.wikipedia.org/wiki/{ELECTION_PAGE}")
81+ except Exception as e: # noqa: BLE001
82+ errors += 1
83+ log.warning("pageviews élection : %s", e)
84+ src.last_ok_at, src.status = utcnow(), "operational" if not errors else "degraded"
85+ src.detail = f"{n} observations parti-jour"
86+ return {"observations": n, "errors": errors, "from": start.isoformat(), "to": end.isoformat()}
87+
88+
89+def attention_summary(s: Session, cutoff: date | None = None) -> dict:
90+ """Part d'attention 7 j, 28 j et variation, par parti ; série quotidienne des parts."""
91+ cutoff = cutoff or date.today()
92+ rows = s.execute(select(Signal.party_id, Signal.date, Signal.value).where(Signal.source == "wikipedia", Signal.key == "pageviews", Signal.date <= cutoff)).all()
93+ if not rows:
94+ return {"enabled": False}
95+ by_day: dict[date, dict[str, float]] = {}
96+ for pid, d, v in rows:
97+ by_day.setdefault(d, {})[pid] = v
98+ days = sorted(by_day)
99+ series = []
100+ for d in days:
101+ tot = sum(by_day[d].values()) or 1
102+ series.append({"date": d.isoformat(), "share": {p: round(100 * by_day[d].get(p, 0) / tot, 2) for p in PARTY_IDS}, "views": {p: by_day[d].get(p, 0) for p in PARTY_IDS}})
103+ last = days[-1]
104+ def window(n: int) -> dict[str, float]:
105+ w = [d for d in days if (last - d).days < n]
106+ tot = {p: sum(by_day[d].get(p, 0) for d in w) for p in PARTY_IDS}
107+ all_ = sum(tot.values()) or 1
108+ return {p: 100 * tot[p] / all_ for p in PARTY_IDS}
109+ s7, s28 = window(7), window(28)
110+ election = s.execute(select(Signal.date, Signal.value).where(Signal.source == "wikipedia", Signal.key == "pageviews_election", Signal.date <= cutoff).order_by(Signal.date)).all()
111+ return {"enabled": True, "asOf": last.isoformat(), "share7": {p: round(v, 2) for p, v in s7.items()}, "share28": {p: round(v, 2) for p, v in s28.items()},
112+ "delta": {p: round(s7[p] - s28[p], 2) for p in PARTY_IDS}, "series": series[-120:],
113+ "election": [{"date": d.isoformat(), "views": v} for d, v in election][-120:], "pages": PAGES}
added engine/qc26/signals/byelections.py +177 −0
Large diff (177 lines) is not loaded by default to keep the page fast.Load diff
added engine/qc26/signals/fundamentals.py +230 −0
Large diff (230 lines) is not loaded by default to keep the page fast.Load diff
added engine/qc26/signals/markets.py +226 −0
Large diff (226 lines) is not loaded by default to keep the page fast.Load diff
added engine/qc26/signals/media.py +267 −0
Large diff (267 lines) is not loaded by default to keep the page fast.Load diff
modified engine/qc26/util.py +18 −0
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/boussole/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/carte/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/circonscription/[slug]/page.tsx +6 −6
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/circonscriptions/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/documents/[id]/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/documents/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/enjeux/[topic]/page.tsx +3 −3
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/enjeux/page.tsx +4 −4
Large diff (26 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/globals.css +166 −46
Large diff (273 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/layout.tsx +21 −15
Large diff (72 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/live/model-vs-reality/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/live/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/methodologie/changelog/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/methodologie/page.tsx +15 −5
Large diff (46 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/page.tsx +54 −29
Large diff (168 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/partis/[id]/page.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/partis/comparateur/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/partis/page.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/plus/page.tsx +3 −3
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/prevision/page.tsx +58 −20
Large diff (117 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/replay/page.tsx +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
added web/src/app/signaux/page.tsx +69 −0
Large diff (69 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/sitemap.ts +1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/sondages/[id]/page.tsx +5 −5
Large diff (34 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/sondages/page.tsx +9 −11
Large diff (42 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/app/statut/page.tsx +4 −4
Large diff (20 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/ask-box.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/charts.tsx +162 −49
Large diff (323 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/compass.tsx +7 −7
Large diff (45 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/forecast-blocks.tsx +125 −83
Large diff (319 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/forecast-history.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/live.tsx +7 −7
Large diff (50 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/map-section.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/map.tsx +16 −16
Large diff (70 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/party.tsx +3 −3
Large diff (22 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/polls-table.tsx +3 −3
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/replay.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/riding-live.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/riding-map.tsx +2 −2
Large diff (14 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/ridings-grid.tsx +3 −3
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/shell.tsx +50 −38
Large diff (174 lines) is not loaded by default to keep the page fast.Load diff
added web/src/components/signals.tsx +284 −0
Large diff (284 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/status-strip.tsx +2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
added web/src/components/theme.tsx +35 −0
Large diff (35 lines) is not loaded by default to keep the page fast.Load diff
added web/src/components/trend-chart.tsx +115 −0
Large diff (115 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/components/ui.tsx +45 −22
Large diff (120 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/lib/api.ts +24 −1
Large diff (42 lines) is not loaded by default to keep the page fast.Load diff
modified web/src/lib/parties.ts +11 −0
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere