SPB Git forge

spb/trouve-ka

Public ★ Pinned

Trouve-KA — moteur de recherche web indépendant, Québec-first. Crawler distribué, index OpenSearch, ranking bilingue, galerie d'images. En prod : www.trouve-ka.com

40commits 1branches 0releases
9.2 MBsize
maindefault branch
22 days agolast push
Python 51.9% TypeScript 32.7% JavaScript 4.8% CSS 4.1% Shell 2.9% HTML 1.8% SQL 1.5%

Page /stats : tableau de bord analytique du moteur + rapport PDF Groupe-KA

- API : GET /api/stats/dashboard (contrat ka-ui/stats SPEC, cache mémoire
  >= 5 min par période) et GET /api/stats/report (PDF fpdf2, gabarit
  Groupe-KA, modes complet/synthese, graphiques vectoriels).
- Module apps/api/stats.py : KPI (pages indexées, domaines, indexées sur
  la période, rythme/h, erreurs, frontier, recherches, taux de résultats),
  séries cumul/indexées/erreurs/recherches par jour (heure si <= 36 h),
  répartitions (résultats de crawl, top domaines, frontier), heatmap 26
  semaines, tables (top domaines, top requêtes, erreurs par type,
  dernières erreurs), records réels — aucune stat inventée, deltas omis
  sans période de référence. Purge 90 j des journaux de recherche (1x/j).
- Migration 005 + index créés CONCURRENTLY en prod :
  documents(first_indexed_at), crawl_attempts erreurs (partiel),
  search_queries(created_at) — agrégats bornés par index, pas de scan
  complet à chaque appel.
- Web : page /stats (client, kit ka-ui/stats kacharts) avec PdfButton,
  Fraicheur, PeriodSelector, LineChart/BarChart/Donut/Heatmap/DataTable/
  Records ; liens Stats dans le header et le footer ; responsive 360/768/
  1440 vérifié (Playwright, zéro débordement).
- Dépendance fpdf2 ajoutée (pyproject + venv).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed 1 mo ago (Aug 18, 2026) parent 099b6b5

11 changed files +2,388 −1

added apps/api/kapdf.py +558 −0
@@ -0,0 +1,558 @@
1 +# Auteur : Simon-Pierre Boucher — contact@spboucher.ai
2 +# ka-ui/stats/kapdf.py — moteur PDF commun Groupe KA (fpdf2).
3 +# Consomme le JSON du contrat /api/stats/dashboard (voir SPEC.md) et produit
4 +# le rapport estampillé Groupe-KA : couverture, sommaire, KPI, graphiques
5 +# VECTORIELS (courbes/barres/anneaux), tableaux paginés, records, page de fin.
6 +# Usage :
7 +# from kapdf import GroupeKAReport
8 +# pdf_bytes = GroupeKAReport(site={"wordmark":"Lou·Ka","accent":"#ff6a00",
9 +# "domain":"www.lou-ka.com","tagline":"…"}, dashboard=dash_json,
10 +# mode="complet").build()
11 +# Dépendance : pip install fpdf2 (aucune autre)
12 +from __future__ import annotations
13 +
14 +import math
15 +from datetime import datetime
16 +from zoneinfo import ZoneInfo
17 +
18 +from fpdf import FPDF
19 +
20 +INK = (20, 24, 20)
21 +INK2 = (77, 85, 81)
22 +INK3 = (139, 146, 140)
23 +PAPER = (245, 243, 238)
24 +SURFACE2 = (250, 249, 245)
25 +GREEN = (28, 92, 65)
26 +DANGER = (179, 66, 58)
27 +WHITE = (255, 255, 255)
28 +
29 +EMAILS = [
30 + ("contact@groupe-ka.com", "Projets, partenariats & données"),
31 + ("info@groupe-ka.com", "Médias & questions générales"),
32 + ("admin@groupe-ka.com", "Légal, vie privée & Loi 25"),
33 +]
34 +DISCLAIMER = (
35 + "Groupe KA est un agrégateur de contenu : nous ne vendons rien, ne louons "
36 + "rien et ne sommes partie à aucune transaction. Données lues à la source, "
37 + "rien d'inventé, tout est traçable."
38 +)
39 +
40 +
41 +def _hex(c: str) -> tuple[int, int, int]:
42 + c = c.lstrip("#")
43 + return tuple(int(c[i : i + 2], 16) for i in (0, 2, 4)) # type: ignore
44 +
45 +
46 +def _fr(n) -> str:
47 + if isinstance(n, float) and not n.is_integer():
48 + return f"{n:,.2f}".replace(",", " ").replace(".", ",")
49 + return f"{int(n):,}".replace(",", " ")
50 +
51 +
52 +_SUBST = {
53 + "—": "-", "–": "-", "→": "->", "▲": "+", "▼": "-",
54 + "…": "...", "’": "'", "‘": "'", "“": '"', "”": '"',
55 + "œ": "oe", "Œ": "OE", "−": "-", " ": " ", " ": " ",
56 +}
57 +
58 +
59 +def _latin1(s: str) -> str:
60 + for k, v in _SUBST.items():
61 + s = s.replace(k, v)
62 + return s.encode("latin-1", "replace").decode("latin-1")
63 +
64 +
65 +class _PDF(FPDF):
66 + """FPDF avec en-tête/pied Groupe-KA sur chaque page (sauf couverture).
67 + Les polices core sont latin-1 : normalize_text sanitise en amont."""
68 +
69 + def normalize_text(self, text):
70 + return super().normalize_text(_latin1(text))
71 +
72 + def __init__(self, brand: str, accent: tuple, period_label: str):
73 + super().__init__(orientation="P", unit="mm", format="A4")
74 + self.brand = brand
75 + self.accent = accent
76 + self.period_label = period_label
77 + self.cover_mode = False
78 + self.set_margins(18, 20, 18)
79 + self.set_auto_page_break(True, margin=22)
80 +
81 + def header(self):
82 + if self.cover_mode:
83 + return
84 + self.set_font("helvetica", "B", 8.5)
85 + self.set_text_color(*INK)
86 + self.set_xy(18, 9)
87 + self.cell(0, 5, f"Groupe KA · {self.brand}")
88 + self.set_font("helvetica", "", 8)
89 + self.set_text_color(*INK3)
90 + self.set_xy(18, 9)
91 + self.cell(0, 5, "Rapport statistique", align="R")
92 + self.set_draw_color(*INK)
93 + self.set_line_width(0.5)
94 + self.line(18, 15.5, 192, 15.5)
95 + self.set_y(20)
96 +
97 + def footer(self):
98 + if self.cover_mode:
99 + return
100 + self.set_y(-15)
101 + self.set_draw_color(*INK3)
102 + self.set_line_width(0.2)
103 + self.line(18, self.get_y() - 1.5, 192, self.get_y() - 1.5)
104 + self.set_font("helvetica", "", 7.5)
105 + self.set_text_color(*INK3)
106 + year = datetime.now(ZoneInfo("America/Toronto")).year
107 + self.cell(130, 5, f"© Groupe-KA — {year} — groupe-ka.com · {self.period_label}")
108 + self.cell(0, 5, f"p. {self.page_no()}/{{nb}}", align="R")
109 +
110 +
111 +class GroupeKAReport:
112 + def __init__(self, site: dict, dashboard: dict, mode: str = "complet"):
113 + self.site = site
114 + self.d = dashboard
115 + self.mode = mode
116 + self.accent = _hex(site.get("accent", "#d9f26b"))
117 + period = dashboard.get("period", {}) or {}
118 + self.period_label = period.get("label") or "toute la période"
119 + self.pdf = _PDF(site.get("wordmark", ""), self.accent, self.period_label)
120 + self.toc: list[tuple[str, int]] = []
121 +
122 + # ---------- primitives ----------
123 + def _card(self, x, y, w, h, fill=WHITE):
124 + p = self.pdf
125 + p.set_draw_color(*INK)
126 + p.set_line_width(0.45)
127 + p.set_fill_color(*fill)
128 + p.rect(x, y, w, h, style="DF", round_corners=True, corner_radius=2.2)
129 +
130 + def _kicker(self, text):
131 + p = self.pdf
132 + p.set_font("helvetica", "B", 8)
133 + p.set_text_color(*GREEN)
134 + p.set_draw_color(*GREEN)
135 + p.set_line_width(0.6)
136 + y = p.get_y() + 2
137 + p.line(p.l_margin, y, p.l_margin + 7, y)
138 + p.set_xy(p.l_margin + 9, y - 2.5)
139 + p.cell(0, 5, text.upper())
140 + p.ln(8)
141 +
142 + def _section_title(self, title):
143 + if self.pdf.get_y() > 240:
144 + self.pdf.add_page()
145 + self._kicker("Groupe KA · " + self.site.get("wordmark", ""))
146 + self.pdf.set_font("helvetica", "B", 15)
147 + self.pdf.set_text_color(*INK)
148 + self.pdf.set_x(self.pdf.l_margin)
149 + self.pdf.cell(0, 8, title)
150 + self.toc.append((title, self.pdf.page_no()))
151 + self.pdf.ln(11)
152 +
153 + # ---------- pages ----------
154 + def _cover(self):
155 + p = self.pdf
156 + p.cover_mode = True
157 + p.set_auto_page_break(False)
158 + p.add_page()
159 + p.set_fill_color(*PAPER)
160 + p.rect(0, 0, 210, 297, style="F")
161 + p.set_draw_color(*INK)
162 + p.set_line_width(1.0)
163 + p.rect(10, 10, 190, 277)
164 + # kicker
165 + p.set_font("helvetica", "B", 10)
166 + p.set_text_color(*GREEN)
167 + p.set_xy(24, 34)
168 + p.cell(0, 6, "GROUPE KA · RAPPORT STATISTIQUE")
169 + # wordmark : partie gauche + boîte encre/accent
170 + wm = self.site.get("wordmark", "")
171 + left, boxed = (wm.split("·") + [None])[:2] if "·" in wm else (wm, None)
172 + p.set_xy(24, 70)
173 + p.set_font("helvetica", "B", 40)
174 + p.set_text_color(*INK)
175 + p.cell(p.get_string_width(left) + 2, 20, left)
176 + if boxed:
177 + bw = p.get_string_width(boxed) + 12
178 + x = p.get_x() + 2
179 + p.set_fill_color(*INK)
180 + p.rect(x, 68, bw, 22, style="F", round_corners=True, corner_radius=3)
181 + p.set_text_color(*self.accent)
182 + p.set_xy(x + 6, 70)
183 + p.cell(bw - 12, 18, boxed)
184 + p.set_xy(24, 100)
185 + p.set_font("helvetica", "", 13)
186 + p.set_text_color(*INK2)
187 + p.multi_cell(150, 7, f"Rapport statistique — {wm}")
188 + now = datetime.now(ZoneInfo("America/Toronto"))
189 + per = self.d.get("period", {}) or {}
190 + p.set_xy(24, 125)
191 + p.set_font("helvetica", "", 10.5)
192 + rows = [
193 + ("Période couverte", self.period_label + (f" ({per.get('from')} → {per.get('to')})" if per.get("from") else "")),
194 + ("Généré le", now.strftime("%Y-%m-%d à %H:%M") + " (heure de l'Est)"),
195 + ("Plateforme", "https://" + self.site.get("domain", "")),
196 + ("Mode", "Rapport complet" if self.mode == "complet" else "Synthèse"),
197 + ]
198 + y = 128
199 + for k, v in rows:
200 + p.set_xy(24, y)
201 + p.set_text_color(*INK3)
202 + p.cell(40, 6, k)
203 + p.set_text_color(*INK)
204 + p.set_font("helvetica", "B", 10.5)
205 + p.cell(0, 6, str(v))
206 + p.set_font("helvetica", "", 10.5)
207 + y += 8
208 + # bande encre au pied
209 + p.set_fill_color(*INK)
210 + p.rect(10, 262, 190, 25, style="F")
211 + p.set_xy(24, 270)
212 + p.set_font("helvetica", "B", 12)
213 + p.set_text_color(*WHITE)
214 + p.cell(60, 8, "par Groupe ")
215 + p.set_text_color(*self.accent)
216 + p.set_xy(24 + p.get_string_width("par Groupe ") + 1, 270)
217 + p.cell(20, 8, "KA")
218 + p.set_font("helvetica", "B", 10)
219 + p.set_xy(24, 270)
220 + p.set_text_color(*self.accent)
221 + p.cell(162, 8, "groupe-ka.com", align="R")
222 + p.set_auto_page_break(True, margin=22)
223 + p.cover_mode = False
224 +
225 + def _kpis(self):
226 + kpis = self.d.get("kpis") or []
227 + if not kpis:
228 + return
229 + self._section_title("Synthèse des indicateurs")
230 + p = self.pdf
231 + cols, gw, gh, gap = 3, 56, 26, 3
232 + x0, y = p.l_margin, p.get_y()
233 + for i, k in enumerate(kpis[:9]):
234 + x = x0 + (i % cols) * (gw + gap)
235 + if i and i % cols == 0:
236 + y += gh + gap
237 + if y > 250:
238 + p.add_page(); y = p.get_y()
239 + self._card(x, y, gw, gh)
240 + p.set_xy(x + 4, y + 4)
241 + p.set_font("helvetica", "B", 14)
242 + p.set_text_color(*INK)
243 + val = k.get("value")
244 + p.cell(gw - 8, 7, (_fr(val) if isinstance(val, (int, float)) else str(val)) + (" " + k["unit"] if k.get("unit") else ""))
245 + p.set_xy(x + 4, y + 12)
246 + p.set_font("helvetica", "", 7.6)
247 + p.set_text_color(*INK2)
248 + p.multi_cell(gw - 8, 3.6, str(k.get("label", ""))[:70])
249 + if k.get("delta_pct") is not None:
250 + up = (k.get("direction") or ("up" if k["delta_pct"] >= 0 else "down")) == "up"
251 + p.set_xy(x + 4, y + gh - 6.5)
252 + p.set_font("helvetica", "B", 8)
253 + p.set_text_color(*(GREEN if up else DANGER))
254 + arrow = "+" if k["delta_pct"] >= 0 else ""
255 + p.cell(gw - 8, 4, f"{'▲' if up else '▼'} {arrow}{str(k['delta_pct']).replace('.', ',')} % vs période préc.")
256 + p.set_y(y + gh + 8)
257 +
258 + def _line_chart(self, s):
259 + p = self.pdf
260 + pts = s.get("points") or []
261 + if len(pts) < 2:
262 + return
263 + if p.get_y() > 200:
264 + p.add_page()
265 + p.set_font("helvetica", "B", 10)
266 + p.set_text_color(*INK)
267 + p.cell(0, 6, s.get("title", ""))
268 + p.ln(7)
269 + x0, y0, w, h = p.l_margin, p.get_y(), 174, 52
270 + self._card(x0, y0, w, h, fill=WHITE)
271 + cx, cy, cw, ch = x0 + 12, y0 + 6, w - 20, h - 16
272 + vals = [pt["v"] for pt in pts] + [c["v"] for c in (s.get("compare") or [])]
273 + vmax = max(vals) or 1
274 + vmin = min(0, min(vals))
275 + rng = (vmax - vmin) or 1
276 + # grille + graduations
277 + p.set_font("helvetica", "", 6.3)
278 + p.set_text_color(*INK3)
279 + p.set_draw_color(200, 200, 195)
280 + p.set_line_width(0.15)
281 + for g in range(5):
282 + gy = cy + ch - ch * g / 4
283 + p.line(cx, gy, cx + cw, gy)
284 + p.set_xy(x0 + 1, gy - 1.6)
285 + p.cell(10, 3, _fr(vmin + rng * g / 4), align="R")
286 +
287 + def draw(series, color, width, dash=None):
288 + n = len(series)
289 + p.set_draw_color(*color)
290 + p.set_line_width(width)
291 + if dash:
292 + p.set_dash_pattern(dash=1.2, gap=1.2)
293 + last = None
294 + for i, pt in enumerate(series):
295 + px = cx + cw * (i / (n - 1))
296 + py = cy + ch - ch * ((pt["v"] - vmin) / rng)
297 + if last:
298 + p.line(last[0], last[1], px, py)
299 + last = (px, py)
300 + p.set_dash_pattern()
301 +
302 + if s.get("compare"):
303 + draw(s["compare"], INK3, 0.35, dash=True)
304 + draw(pts, self.accent, 0.7)
305 + # libellés d'axe X (premier / milieu / dernier)
306 + p.set_text_color(*INK3)
307 + for frac, idx in ((0, 0), (0.5, len(pts) // 2), (1, -1)):
308 + p.set_xy(cx + cw * frac - 9, cy + ch + 1.5)
309 + p.cell(18, 3, str(pts[idx].get("t", ""))[:10], align="C")
310 + p.set_y(y0 + h + 4)
311 + if s.get("compare"):
312 + p.set_font("helvetica", "", 6.8)
313 + p.set_text_color(*INK3)
314 + p.cell(0, 4, "— période courante (accent) · ---- période comparée")
315 + p.ln(6)
316 + else:
317 + p.ln(2)
318 +
319 + def _bars(self, title, items, unit=""):
320 + p = self.pdf
321 + items = [it for it in (items or []) if isinstance(it.get("value"), (int, float))][:12]
322 + if not items:
323 + return
324 + need = 10 + len(items) * 7
325 + if p.get_y() + need > 265:
326 + p.add_page()
327 + p.set_font("helvetica", "B", 10)
328 + p.set_text_color(*INK)
329 + p.cell(0, 6, title)
330 + p.ln(8)
331 + vmax = max(it["value"] for it in items) or 1
332 + for it in items:
333 + y = p.get_y()
334 + p.set_font("helvetica", "", 7.6)
335 + p.set_text_color(*INK)
336 + p.set_x(p.l_margin)
337 + p.cell(46, 5, str(it["label"])[:34])
338 + bw = 96 * (it["value"] / vmax)
339 + p.set_fill_color(*self.accent)
340 + p.set_draw_color(*INK)
341 + p.set_line_width(0.25)
342 + p.rect(p.l_margin + 48, y + 0.7, max(bw, 0.8), 3.6, style="DF")
343 + p.set_xy(p.l_margin + 148, y)
344 + p.set_font("helvetica", "B", 7.6)
345 + p.cell(26, 5, _fr(it["value"]) + (" " + unit if unit else ""), align="R")
346 + p.ln(6.4)
347 + p.ln(3)
348 +
349 + def _donut(self, b):
350 + # anneau vectoriel simple (arcs) + légende
351 + p = self.pdf
352 + items = [it for it in (b.get("items") or []) if it.get("value")][:8]
353 + total = sum(it["value"] for it in items)
354 + if not items or not total:
355 + return
356 + if p.get_y() > 210:
357 + p.add_page()
358 + p.set_font("helvetica", "B", 10)
359 + p.set_text_color(*INK)
360 + p.cell(0, 6, b.get("title", ""))
361 + p.ln(8)
362 + cx, cy, r = p.l_margin + 26, p.get_y() + 24, 20
363 + shades = [1.0, 0.78, 0.58, 0.42, 0.30, 0.22, 0.15, 0.10]
364 + start = -90.0
365 + for i, it in enumerate(items):
366 + frac = it["value"] / total
367 + f = shades[i % len(shades)]
368 + col = tuple(int(PAPER[j] + (self.accent[j] - PAPER[j]) * f) for j in range(3))
369 + steps = max(2, int(72 * frac))
370 + p.set_fill_color(*col)
371 + p.set_draw_color(*col)
372 + for st in range(steps):
373 + a0 = math.radians(start + 360 * frac * st / steps)
374 + a1 = math.radians(start + 360 * frac * (st + 1) / steps)
375 + p.polygon(
376 + [(cx, cy),
377 + (cx + r * math.cos(a0), cy + r * math.sin(a0)),
378 + (cx + r * math.cos(a1), cy + r * math.sin(a1))],
379 + style="DF",
380 + )
381 + start += 360 * frac
382 + p.set_fill_color(*WHITE)
383 + p.set_draw_color(*INK)
384 + p.set_line_width(0.4)
385 + p.ellipse(cx - 11, cy - 11, 22, 22, style="DF")
386 + p.ellipse(cx - r, cy - r, 2 * r, 2 * r, style="D")
387 + # légende
388 + ly = cy - 22
389 + for i, it in enumerate(items):
390 + f = shades[i % len(shades)]
391 + col = tuple(int(PAPER[j] + (self.accent[j] - PAPER[j]) * f) for j in range(3))
392 + p.set_fill_color(*col)
393 + p.set_draw_color(*INK)
394 + p.rect(p.l_margin + 60, ly + 0.8, 4, 4, style="DF")
395 + p.set_xy(p.l_margin + 66, ly)
396 + p.set_font("helvetica", "", 7.6)
397 + p.set_text_color(*INK)
398 + pct = 100 * it["value"] / total
399 + p.cell(0, 5.6, f"{str(it['label'])[:40]} — {_fr(it['value'])} ({pct:.1f} %)".replace(".", ","))
400 + ly += 5.6
401 + p.set_y(max(cy + r, ly) + 6)
402 +
403 + def _table(self, t):
404 + p = self.pdf
405 + cols = t.get("columns") or []
406 + rows = t.get("rows") or []
407 + if not cols or not rows:
408 + return
409 + self._section_title(t.get("title", "Tableau"))
410 + w = 174 / len(cols)
411 + def head():
412 + p.set_font("helvetica", "B", 7.6)
413 + p.set_fill_color(*INK)
414 + p.set_text_color(*WHITE)
415 + for c in cols:
416 + p.cell(w, 6, " " + str(c)[:30], fill=True)
417 + p.ln(6)
418 + head()
419 + p.set_text_color(*INK)
420 + for i, row in enumerate(rows[:200]):
421 + if p.get_y() > 262:
422 + p.add_page()
423 + head()
424 + p.set_text_color(*INK)
425 + p.set_font("helvetica", "", 7.4)
426 + p.set_fill_color(*(SURFACE2 if i % 2 else WHITE))
427 + for cell in row:
428 + txt = _fr(cell) if isinstance(cell, (int, float)) else str(cell)
429 + p.cell(w, 5.4, " " + txt[:34], fill=True)
430 + p.ln(5.4)
431 + if len(rows) > 200:
432 + p.set_font("helvetica", "", 7)
433 + p.set_text_color(*INK3)
434 + p.cell(0, 5, f"… {len(rows) - 200} lignes supplémentaires non imprimées")
435 + p.ln(6)
436 +
437 + def _records(self):
438 + recs = self.d.get("records") or []
439 + if not recs:
440 + return
441 + self._section_title("Records & faits marquants")
442 + p = self.pdf
443 + for r in recs[:10]:
444 + if p.get_y() > 258:
445 + p.add_page()
446 + y = p.get_y()
447 + self._card(p.l_margin, y, 174, 11, fill=SURFACE2)
448 + p.set_xy(p.l_margin + 4, y + 2)
449 + p.set_font("helvetica", "", 8.6)
450 + p.set_text_color(*INK2)
451 + p.cell(96, 7, str(r.get("label", ""))[:70])
452 + p.set_font("helvetica", "B", 9)
453 + p.set_text_color(*INK)
454 + p.cell(52, 7, str(r.get("value", ""))[:36], align="R")
455 + p.set_font("helvetica", "", 7.6)
456 + p.set_text_color(*INK3)
457 + p.cell(20, 7, str(r.get("date", "") or ""), align="R")
458 + p.set_y(y + 13.5)
459 + p.ln(4)
460 +
461 + def _final_page(self):
462 + p = self.pdf
463 + p.add_page()
464 + self._kicker("Groupe KA · contact")
465 + p.set_font("helvetica", "B", 15)
466 + p.set_text_color(*INK)
467 + p.cell(0, 8, "Coordonnées du Groupe KA")
468 + p.ln(12)
469 + for email, role in EMAILS:
470 + p.set_font("helvetica", "B", 10.5)
471 + p.set_text_color(*INK)
472 + p.cell(0, 6, email)
473 + p.ln(5.5)
474 + p.set_font("helvetica", "", 8.6)
475 + p.set_text_color(*INK3)
476 + p.cell(0, 5, role)
477 + p.ln(8)
478 + p.ln(2)
479 + p.set_font("helvetica", "B", 10)
480 + p.set_text_color(*GREEN)
481 + p.cell(0, 6, "groupe-ka.com — le portail de l'écosystème ·Ka")
482 + p.ln(10)
483 + p.set_draw_color(*self.accent)
484 + p.set_line_width(0.8)
485 + p.line(p.l_margin, p.get_y(), p.l_margin + 30, p.get_y())
486 + p.ln(4)
487 + p.set_font("helvetica", "", 8.6)
488 + p.set_text_color(*INK2)
489 + p.multi_cell(160, 4.6, DISCLAIMER)
490 + p.ln(4)
491 + p.set_font("helvetica", "", 7.6)
492 + p.set_text_color(*INK3)
493 + p.multi_cell(
494 + 160, 4.2,
495 + "Mentions : rapport généré automatiquement à partir des données réelles de la "
496 + "plateforme au moment indiqué en couverture. Conditions d'utilisation, politique "
497 + "de confidentialité et protection des renseignements personnels (Loi 25) : "
498 + "groupe-ka.com/conditions · /confidentialite · /loi-25.",
499 + )
500 +
501 + def _toc_page(self):
502 + # insérée après coup ? fpdf ne réordonne pas : on écrit le sommaire en
503 + # page 2 en réservant la page lors du build (voir build()).
504 + pass
505 +
506 + def build(self) -> bytes:
507 + p = self.pdf
508 + p.alias_nb_pages()
509 + self._cover()
510 + if self.mode == "synthese":
511 + p.add_page()
512 + self._kpis()
513 + self._records()
514 + self._final_page()
515 + else:
516 + p.add_page()
517 + toc_page_no = p.page_no()
518 + p.add_page()
519 + self._kpis()
520 + for s in self.d.get("series") or []:
521 + if s.get("kind") == "bar":
522 + self._bars(s.get("title", ""), [{"label": pt.get("t"), "value": pt.get("v")} for pt in (s.get("points") or [])], s.get("unit", ""))
523 + else:
524 + self._line_chart(s)
525 + for b in self.d.get("breakdowns") or []:
526 + if b.get("kind") == "donut":
527 + self._donut(b)
528 + else:
529 + self._bars(b.get("title", ""), b.get("items"))
530 + geo = self.d.get("geo")
531 + if geo:
532 + self._bars(geo.get("title", "Répartition géographique"), geo.get("items"))
533 + for t in self.d.get("tables") or []:
534 + self._table(t)
535 + self._records()
536 + self._final_page()
537 + # sommaire écrit sur la page réservée (page 2)
538 + last_page = p.page
539 + p.page = toc_page_no
540 + p.set_y(22)
541 + p.set_font("helvetica", "B", 15)
542 + p.set_text_color(*INK)
543 + p.cell(0, 8, "Sommaire")
544 + p.ln(12)
545 + p.set_font("helvetica", "", 9.5)
546 + for title, page_no in self.toc:
547 + p.set_text_color(*INK)
548 + p.cell(140, 6.5, title[:80])
549 + p.set_text_color(*INK3)
550 + p.cell(0, 6.5, str(page_no), align="R")
551 + p.ln(6.5)
552 + p.page = last_page
553 + return bytes(p.output())
554 +
555 +
556 +def filename(platform_id: str, period: str) -> str:
557 + today = datetime.now(ZoneInfo("America/Toronto")).strftime("%Y-%m-%d")
558 + return f"groupe-ka_{platform_id}_stats_{period}_{today}.pdf"
modified apps/api/main.py +47 −1
@@ -18,10 +18,14 @@ import time
18 18 from contextlib import asynccontextmanager
19 19 from typing import Annotated
20 20
21 −from fastapi import Depends, FastAPI, Header, HTTPException, Query
21 +import anyio.to_thread
22 +from fastapi import Depends, FastAPI, Header, HTTPException, Query, Response
22 23 from fastapi.middleware.cors import CORSMiddleware
23 24 from pydantic import BaseModel
24 25
26 +from trouveka.api import stats as stats_mod
27 +from trouveka.api.kapdf import GroupeKAReport
28 +from trouveka.api.kapdf import filename as pdf_filename
25 29 from trouveka.config import get_settings
26 30 from trouveka.database import Database
27 31 from trouveka.logging import get_logger
@@ -291,6 +295,48 @@ async def api_status():
291 295 }
292 296
293 297
298 +PERIOD_PATTERN = "^(auj|7j|30j|3m|6m|12m|annee|tout)$"
299 +
300 +
301 +@app.get("/api/stats/dashboard")
302 +async def api_stats_dashboard(
303 + period: str = Query("30j", pattern=PERIOD_PATTERN),
304 + from_date: str | None = Query(None, alias="from", pattern=r"^\d{4}-\d{2}-\d{2}$"),
305 + to_date: str | None = Query(None, alias="to", pattern=r"^\d{4}-\d{2}-\d{2}$"),
306 +):
307 + """Tableau de bord analytique (contrat ka-ui/stats/SPEC.md, cache >= 5 min)."""
308 + try:
309 + return await stats_mod.dashboard(db, period, from_date, to_date)
310 + except Exception:
311 + log.exception("dashboard stats indisponible")
312 + raise HTTPException(status_code=503, detail="Statistiques temporairement indisponibles")
313 +
314 +
315 +@app.get("/api/stats/report")
316 +async def api_stats_report(
317 + period: str = Query("30j", pattern=PERIOD_PATTERN),
318 + from_date: str | None = Query(None, alias="from", pattern=r"^\d{4}-\d{2}-\d{2}$"),
319 + to_date: str | None = Query(None, alias="to", pattern=r"^\d{4}-\d{2}-\d{2}$"),
320 + mode: str = Query("complet", pattern="^(complet|synthese)$"),
321 +):
322 + """Rapport PDF estampillé Groupe-KA (gabarit kapdf, graphiques vectoriels)."""
323 + try:
324 + dash = await stats_mod.dashboard(db, period, from_date, to_date)
325 + pdf_bytes = await anyio.to_thread.run_sync(
326 + lambda: GroupeKAReport(site=stats_mod.SITE, dashboard=dash, mode=mode).build()
327 + )
328 + except Exception:
329 + log.exception("rapport PDF indisponible")
330 + raise HTTPException(status_code=503, detail="Rapport temporairement indisponible")
331 + return Response(
332 + content=pdf_bytes,
333 + media_type="application/pdf",
334 + headers={
335 + "Content-Disposition": f'attachment; filename="{pdf_filename("trouve-ka", period)}"'
336 + },
337 + )
338 +
339 +
294 340 @app.get("/api/live")
295 341 async def api_live():
296 342 """Dernière page visitée par TrouveKABot — alimente le flux temps réel du footer.
added apps/api/stats.py +452 −0
@@ -0,0 +1,452 @@
1 +# Trouve-KA — module Stats : dashboard analytique + rapport PDF Groupe-KA
2 +# Author: Simon-Pierre Boucher
3 +# Contact: contact@spboucher.ai
4 +
5 +"""Statistiques du moteur (contrat ka-ui/stats/SPEC.md).
6 +
7 +- `dashboard()` agrège Postgres (documents, crawl_attempts, frontier_items,
8 + domains, search_queries) en JSON conforme au contrat /api/stats/dashboard.
9 +- Cache mémoire par période (>= 5 min) : les tables sont volumineuses
10 + (crawl_attempts ~5 M lignes), on ne re-agrège jamais à chaque appel.
11 +- AUCUNE donnée inventée : une section sans données est simplement absente
12 + (le front affiche « Pas encore mesuré »).
13 +- Purge opportuniste des journaux de recherche (> 90 jours), 1×/jour max.
14 +"""
15 +
16 +from __future__ import annotations
17 +
18 +import asyncio
19 +import time
20 +from datetime import date, datetime, timedelta
21 +from zoneinfo import ZoneInfo
22 +
23 +from trouveka.logging import get_logger
24 +
25 +log = get_logger("stats")
26 +
27 +TZ = ZoneInfo("America/Toronto")
28 +
29 +SITE = {
30 + "wordmark": "Trouve·Ka",
31 + "domain": "www.trouve-ka.com",
32 + "accent": "#1c7ed6",
33 + "tagline": "Le moteur de recherche du web québécois",
34 +}
35 +
36 +PERIOD_LABELS = {
37 + "auj": "aujourd'hui",
38 + "7j": "7 jours",
39 + "30j": "30 jours",
40 + "3m": "3 mois",
41 + "6m": "6 mois",
42 + "12m": "12 mois",
43 + "annee": "année en cours",
44 + "tout": "toute la période",
45 +}
46 +
47 +OUTCOME_LABELS = {
48 + "indexed": "Indexée",
49 + "duplicate": "Doublon",
50 + "error": "Erreur",
51 + "not_quebec": "Hors Québec",
52 + "robots_blocked": "Bloquée (robots.txt)",
53 + "redirect": "Redirection",
54 + "unchanged": "Inchangée",
55 +}
56 +
57 +FRONTIER_LABELS = {
58 + "pending": "En attente",
59 + "in_progress": "En cours",
60 + "done": "Complétée",
61 + "failed": "Échouée",
62 + "blocked": "Bloquée",
63 +}
64 +
65 +# Cache mémoire : clé période → (expiration monotonic, payload)
66 +_cache: dict[str, tuple[float, dict]] = {}
67 +_global_cache: dict[str, tuple[float, object]] = {}
68 +_build_lock = asyncio.Lock()
69 +_last_purge = 0.0
70 +
71 +CACHE_TTL = 300 # 5 min par période
72 +CACHE_TTL_TOUT = 900 # « tout » agrège 5 M lignes : 15 min
73 +GLOBAL_TTL = 600 # heatmap / records globaux
74 +DEEP_TTL = 3600 # domaine le plus profond (scan frontier 3,7 M lignes)
75 +
76 +
77 +def _fr_int(n: int | float) -> str:
78 + return f"{int(n):,}".replace(",", " ")
79 +
80 +
81 +def _delta(cur: float | None, prev: float | None) -> float | None:
82 + """Variation % vs période précédente. Pas de période de référence → None."""
83 + if cur is None or prev is None or prev == 0:
84 + return None # pas de période de référence mesurée → pas de faux delta
85 + return round(100.0 * (cur - prev) / prev, 1)
86 +
87 +
88 +def resolve_period(
89 + period: str, from_s: str | None, to_s: str | None, dataset_start: datetime
90 +) -> tuple[datetime, datetime, str, datetime | None, datetime | None]:
91 + """→ (start, end, label, prev_start, prev_end) en heure de l'Est."""
92 + now = datetime.now(TZ)
93 + if from_s and to_s:
94 + start = datetime.strptime(from_s, "%Y-%m-%d").replace(tzinfo=TZ)
95 + end = datetime.strptime(to_s, "%Y-%m-%d").replace(tzinfo=TZ) + timedelta(days=1)
96 + end = min(end, now)
97 + label = f"du {from_s} au {to_s}"
98 + elif period == "auj":
99 + start = now.replace(hour=0, minute=0, second=0, microsecond=0)
100 + end, label = now, PERIOD_LABELS["auj"]
101 + elif period == "annee":
102 + start = now.replace(month=1, day=1, hour=0, minute=0, second=0, microsecond=0)
103 + end, label = now, PERIOD_LABELS["annee"]
104 + elif period == "tout":
105 + start = dataset_start.astimezone(TZ)
106 + end, label = now, PERIOD_LABELS["tout"]
107 + else:
108 + days = {"7j": 7, "30j": 30, "3m": 91, "6m": 182, "12m": 365}.get(period, 30)
109 + start = now - timedelta(days=days)
110 + end, label = now, PERIOD_LABELS.get(period, PERIOD_LABELS["30j"])
111 + if start >= end:
112 + start = end - timedelta(days=1)
113 + if period == "tout" and not (from_s and to_s):
114 + return start, end, label, None, None
115 + span = end - start
116 + return start, end, label, start - span, start
117 +
118 +
119 +async def _dataset_start(db) -> datetime:
120 + """Première trace du crawler (min borné par index) — cache 1 h."""
121 + hit = _global_cache.get("dataset_start")
122 + if hit and hit[0] > time.monotonic():
123 + return hit[1] # type: ignore[return-value]
124 + row = await db.pool.fetchrow("SELECT min(fetched_at) AS t FROM crawl_attempts")
125 + start = row["t"] or datetime.now(TZ) - timedelta(days=1)
126 + _global_cache["dataset_start"] = (time.monotonic() + 3600, start)
127 + return start
128 +
129 +
130 +async def _bucket_counts(
131 + db, table: str, ts_col: str, start: datetime, end: datetime,
132 + bucket: str, extra_where: str = ""
133 +) -> list[dict]:
134 + """Comptes par jour (ou heure) en heure locale, trous remplis de zéros.
135 +
136 + Toujours borné sur la colonne temporelle indexée — jamais de scan complet.
137 + """
138 + unit = "hour" if bucket == "hour" else "day"
139 + rows = await db.pool.fetch(
140 + f"""
141 + SELECT date_trunc('{unit}', {ts_col} AT TIME ZONE 'America/Toronto') AS b,
142 + count(*)::int AS n
143 + FROM {table}
144 + WHERE {ts_col} >= $1 AND {ts_col} < $2 {extra_where}
145 + GROUP BY 1 ORDER BY 1
146 + """,
147 + start, end,
148 + )
149 + got = {r["b"]: r["n"] for r in rows}
150 + points: list[dict] = []
151 + if bucket == "hour":
152 + cur = start.replace(minute=0, second=0, microsecond=0)
153 + while cur < end:
154 + key = cur.astimezone(TZ).replace(tzinfo=None).replace(minute=0, second=0, microsecond=0)
155 + points.append({"t": key.strftime("%Hh"), "v": got.get(key, 0)})
156 + cur += timedelta(hours=1)
157 + else:
158 + cur_d = start.astimezone(TZ).date()
159 + end_d = end.astimezone(TZ).date()
160 + while cur_d <= end_d:
161 + key = datetime(cur_d.year, cur_d.month, cur_d.day)
162 + points.append({"t": cur_d.isoformat(), "v": got.get(key, 0)})
163 + cur_d += timedelta(days=1)
164 + return points
165 +
166 +
167 +async def _heatmap_cells(db) -> list[dict]:
168 + """Indexations/jour sur 182 jours — partagé entre périodes (cache 10 min)."""
169 + hit = _global_cache.get("heatmap")
170 + if hit and hit[0] > time.monotonic():
171 + return hit[1] # type: ignore[return-value]
172 + since = datetime.now(TZ) - timedelta(days=182)
173 + rows = await db.pool.fetch(
174 + """
175 + SELECT (first_indexed_at AT TIME ZONE 'America/Toronto')::date AS d, count(*)::int AS n
176 + FROM documents WHERE first_indexed_at >= $1
177 + GROUP BY 1 ORDER BY 1
178 + """,
179 + since,
180 + )
181 + cells = [{"date": r["d"].isoformat(), "value": r["n"]} for r in rows]
182 + _global_cache["heatmap"] = (time.monotonic() + GLOBAL_TTL, cells)
183 + return cells
184 +
185 +
186 +async def _deepest_domain(db) -> dict | None:
187 + """Domaine atteint à la plus grande profondeur de crawl (scan lourd → 1 h)."""
188 + hit = _global_cache.get("deepest")
189 + if hit and hit[0] > time.monotonic():
190 + return hit[1] # type: ignore[return-value]
191 + row = await db.pool.fetchrow(
192 + """
193 + SELECT d.domain, f.depth
194 + FROM frontier_items f
195 + JOIN urls u ON u.id = f.url_id
196 + JOIN domains d ON d.id = u.domain_id
197 + ORDER BY f.depth DESC LIMIT 1
198 + """
199 + )
200 + val = {"domain": row["domain"], "depth": row["depth"]} if row else None
201 + _global_cache["deepest"] = (time.monotonic() + DEEP_TTL, val)
202 + return val
203 +
204 +
205 +async def _purge_old_queries(db) -> None:
206 + """Rétention 90 jours des journaux de recherche — au plus 1×/jour."""
207 + global _last_purge
208 + if time.monotonic() - _last_purge < 86400 and _last_purge:
209 + return
210 + _last_purge = time.monotonic()
211 + try:
212 + res = await db.pool.execute(
213 + "DELETE FROM search_queries WHERE created_at < now() - interval '90 days'"
214 + )
215 + log.info("purge search_queries (90 j)", extra={"ctx": {"result": res}})
216 + except Exception:
217 + log.exception("purge search_queries échouée")
218 +
219 +
220 +async def dashboard(db, period: str = "30j", from_s: str | None = None, to_s: str | None = None) -> dict:
221 + key = f"{period}|{from_s or ''}|{to_s or ''}"
222 + hit = _cache.get(key)
223 + if hit and hit[0] > time.monotonic():
224 + return hit[1]
225 + async with _build_lock:
226 + hit = _cache.get(key)
227 + if hit and hit[0] > time.monotonic():
228 + return hit[1]
229 + data = await _build(db, period, from_s, to_s)
230 + ttl = CACHE_TTL_TOUT if period == "tout" else CACHE_TTL
231 + _cache[key] = (time.monotonic() + ttl, data)
232 + return data
233 +
234 +
235 +async def _build(db, period: str, from_s: str | None, to_s: str | None) -> dict:
236 + now = datetime.now(TZ)
237 + ds_start = await _dataset_start(db)
238 + start, end, label, prev_start, prev_end = resolve_period(period, from_s, to_s, ds_start)
239 + bucket = "hour" if (end - start) <= timedelta(hours=36) else "day"
240 + hours = max((end - start).total_seconds() / 3600.0, 0.01)
241 +
242 + await _purge_old_queries(db)
243 +
244 + # ---- KPI (toutes les bornes exploitent les index temporels) ----
245 + cur = await db.pool.fetchrow(
246 + """
247 + SELECT
248 + (SELECT count(*)::int FROM documents) AS docs_total,
249 + (SELECT count(*)::int FROM domains WHERE page_count > 0) AS domains_count,
250 + (SELECT count(*)::int FROM documents
251 + WHERE first_indexed_at >= $1 AND first_indexed_at < $2) AS indexed_cur,
252 + (SELECT count(*)::int FROM documents WHERE first_indexed_at < $1) AS docs_before,
253 + (SELECT count(*)::int FROM crawl_attempts
254 + WHERE outcome = 'error' AND fetched_at >= $1 AND fetched_at < $2) AS errors_cur,
255 + (SELECT count(*)::int FROM frontier_items WHERE status = 'pending') AS frontier_pending
256 + """,
257 + start, end,
258 + )
259 + prev = None
260 + if prev_start is not None:
261 + prev = await db.pool.fetchrow(
262 + """
263 + SELECT
264 + (SELECT count(*)::int FROM documents
265 + WHERE first_indexed_at >= $1 AND first_indexed_at < $2) AS indexed_prev,
266 + (SELECT count(*)::int FROM crawl_attempts
267 + WHERE outcome = 'error' AND fetched_at >= $1 AND fetched_at < $2) AS errors_prev,
268 + (SELECT count(*)::int FROM search_queries
269 + WHERE created_at >= $1 AND created_at < $2) AS searches_prev
270 + """,
271 + prev_start, prev_end,
272 + )
273 + sq = await db.pool.fetchrow(
274 + """
275 + SELECT count(*)::int AS n,
276 + round(avg((NOT zero_result)::int) * 100, 1)::float AS found_rate
277 + FROM search_queries WHERE created_at >= $1 AND created_at < $2
278 + """,
279 + start, end,
280 + )
281 +
282 + indexed_cur = cur["indexed_cur"]
283 + indexed_prev = prev["indexed_prev"] if prev else None
284 + errors_prev = prev["errors_prev"] if prev else None
285 + searches_prev = prev["searches_prev"] if prev else None
286 + growth = round(100.0 * indexed_cur / cur["docs_before"], 1) if cur["docs_before"] else None
287 + err_delta = _delta(cur["errors_cur"], errors_prev)
288 +
289 + kpis = [
290 + {"id": "pages", "label": "Pages indexées (total)", "value": cur["docs_total"],
291 + "delta_pct": growth, "direction": "up"},
292 + {"id": "domaines", "label": "Domaines dans l'index", "value": cur["domains_count"]},
293 + {"id": "indexees", "label": f"Pages indexées — {label}", "value": indexed_cur,
294 + "delta_pct": _delta(indexed_cur, indexed_prev)},
295 + {"id": "rythme", "label": "Rythme d'indexation", "value": round(indexed_cur / hours, 1),
296 + "unit": "pages/h", "delta_pct": _delta(indexed_cur, indexed_prev)},
297 + {"id": "erreurs", "label": f"Erreurs de crawl — {label}", "value": cur["errors_cur"],
298 + "delta_pct": err_delta},
299 + {"id": "frontier", "label": "Frontier en attente", "value": cur["frontier_pending"]},
300 + {"id": "recherches", "label": f"Recherches — {label}", "value": sq["n"],
301 + "delta_pct": _delta(sq["n"], searches_prev)},
302 + ]
303 + if sq["n"]:
304 + kpis.append({"id": "taux", "label": "Recherches avec résultats", "value": sq["found_rate"],
305 + "unit": "%"})
306 +
307 + # ---- Séries temporelles ----
308 + per_bucket = await _bucket_counts(db, "documents", "first_indexed_at", start, end, bucket)
309 + err_bucket = await _bucket_counts(
310 + db, "crawl_attempts", "fetched_at", start, end, bucket, "AND outcome = 'error'"
311 + )
312 + unit_label = "par heure" if bucket == "hour" else "par jour"
313 + running = cur["docs_before"]
314 + cum_points = []
315 + for p in per_bucket:
316 + running += p["v"]
317 + cum_points.append({"t": p["t"], "v": running})
318 + series = [
319 + {"id": "cumul", "title": "Pages indexées — cumul", "unit": "pages", "kind": "line",
320 + "points": cum_points},
321 + {"id": "indexees", "title": f"Pages indexées {unit_label}", "unit": "pages", "kind": "line",
322 + "points": per_bucket},
323 + {"id": "erreurs", "title": f"Erreurs de crawl {unit_label}", "unit": "erreurs", "kind": "line",
324 + "points": err_bucket},
325 + ]
326 + if sq["n"]:
327 + sq_bucket = await _bucket_counts(db, "search_queries", "created_at", start, end, bucket)
328 + series.append({"id": "recherches", "title": f"Recherches {unit_label}", "unit": "recherches",
329 + "kind": "line", "points": sq_bucket})
330 +
331 + # ---- Répartitions ----
332 + outcomes = await db.pool.fetch(
333 + """
334 + SELECT outcome, count(*)::int AS n FROM crawl_attempts
335 + WHERE fetched_at >= $1 AND fetched_at < $2
336 + GROUP BY outcome ORDER BY n DESC
337 + """,
338 + start, end,
339 + )
340 + frontier = await db.pool.fetch(
341 + "SELECT status, count(*)::int AS n FROM frontier_items GROUP BY status ORDER BY n DESC"
342 + )
343 + top_domains = await db.pool.fetch(
344 + """
345 + SELECT domain, page_count, url_count, round(quebec_score::numeric, 2)::float AS qs
346 + FROM domains WHERE page_count > 0
347 + ORDER BY page_count DESC LIMIT 100
348 + """
349 + )
350 + breakdowns = [
351 + {"id": "outcomes", "title": f"Résultats de crawl — {label}", "kind": "donut",
352 + "items": [{"label": OUTCOME_LABELS.get(r["outcome"], r["outcome"]), "value": r["n"]}
353 + for r in outcomes]},
354 + {"id": "topdom", "title": "Top domaines par pages indexées (tout l'index)", "kind": "bar",
355 + "items": [{"label": r["domain"], "value": r["page_count"]} for r in top_domains[:12]]},
356 + {"id": "frontier", "title": "File frontier par statut", "kind": "donut",
357 + "items": [{"label": FRONTIER_LABELS.get(r["status"], r["status"]), "value": r["n"]}
358 + for r in frontier]},
359 + ]
360 +
361 + # ---- Tableaux ----
362 + tables = [
363 + {"id": "topdom", "title": "Top domaines (tout l'index)",
364 + "columns": ["Domaine", "Pages indexées", "URLs connues", "Score Québec"],
365 + "rows": [[r["domain"], r["page_count"], r["url_count"], r["qs"]] for r in top_domains]},
366 + ]
367 + if sq["n"]:
368 + top_q = await db.pool.fetch(
369 + """
370 + SELECT lower(query) AS q, count(*)::int AS n,
371 + round(avg(results_total))::int AS avg_res, max(created_at) AS last_at
372 + FROM search_queries WHERE created_at >= $1 AND created_at < $2
373 + GROUP BY 1 ORDER BY n DESC, last_at DESC LIMIT 50
374 + """,
375 + start, end,
376 + )
377 + tables.append({
378 + "id": "topq", "title": f"Top requêtes de recherche — {label}",
379 + "columns": ["Requête", "Recherches", "Résultats moy.", "Dernière fois"],
380 + "rows": [[r["q"], r["n"], r["avg_res"],
381 + r["last_at"].astimezone(TZ).strftime("%Y-%m-%d %H:%M")] for r in top_q],
382 + })
383 + err_types = await db.pool.fetch(
384 + """
385 + SELECT coalesce(error_code, 'inconnu') AS code, count(*)::int AS n, max(fetched_at) AS last_at
386 + FROM crawl_attempts
387 + WHERE outcome = 'error' AND fetched_at >= $1 AND fetched_at < $2
388 + GROUP BY 1 ORDER BY n DESC
389 + """,
390 + start, end,
391 + )
392 + if err_types:
393 + tables.append({
394 + "id": "errtypes", "title": f"Erreurs de crawl par type — {label}",
395 + "columns": ["Type d'erreur", "Occurrences", "Dernière occurrence"],
396 + "rows": [[r["code"], r["n"],
397 + r["last_at"].astimezone(TZ).strftime("%Y-%m-%d %H:%M")] for r in err_types],
398 + })
399 + last_errors = await db.pool.fetch(
400 + """
401 + SELECT a.fetched_at, coalesce(a.error_code, 'inconnu') AS code, u.url
402 + FROM crawl_attempts a JOIN urls u ON u.id = a.url_id
403 + WHERE a.outcome = 'error' AND a.fetched_at >= $1 AND a.fetched_at < $2
404 + ORDER BY a.fetched_at DESC LIMIT 25
405 + """,
406 + start, end,
407 + )
408 + if last_errors:
409 + tables.append({
410 + "id": "lasterr", "title": "Dernières erreurs de crawl",
411 + "columns": ["Quand", "Type", "URL"],
412 + "rows": [[r["fetched_at"].astimezone(TZ).strftime("%Y-%m-%d %H:%M"),
413 + r["code"], r["url"][:120]] for r in last_errors],
414 + })
415 +
416 + # ---- Heatmap & records (réels, jamais inventés) ----
417 + cells = await _heatmap_cells(db)
418 + records = []
419 + if cells:
420 + best = max(cells, key=lambda c: c["value"])
421 + records.append({"label": "Jour record d'indexation",
422 + "value": f"{_fr_int(best['value'])} pages", "date": best["date"]})
423 + if top_domains:
424 + records.append({"label": "Domaine le plus fourni",
425 + "value": f"{top_domains[0]['domain']} — {_fr_int(top_domains[0]['page_count'])} pages"})
426 + deepest = await _deepest_domain(db)
427 + if deepest:
428 + records.append({"label": "Domaine exploré le plus profondément",
429 + "value": f"{deepest['domain']} (profondeur {deepest['depth']})"})
430 + if sq["n"]:
431 + best_sq = await db.pool.fetchrow(
432 + """
433 + SELECT (created_at AT TIME ZONE 'America/Toronto')::date AS d, count(*)::int AS n
434 + FROM search_queries GROUP BY 1 ORDER BY n DESC LIMIT 1
435 + """
436 + )
437 + if best_sq:
438 + records.append({"label": "Jour record de recherches",
439 + "value": f"{_fr_int(best_sq['n'])} recherches",
440 + "date": best_sq["d"].isoformat()})
441 +
442 + return {
443 + "updated": now.isoformat(),
444 + "period": {"from": start.astimezone(TZ).date().isoformat(),
445 + "to": end.astimezone(TZ).date().isoformat(), "label": label},
446 + "kpis": kpis,
447 + "series": series,
448 + "breakdowns": breakdowns,
449 + "heatmap": {"title": "Indexations par jour (26 dernières semaines)", "cells": cells},
450 + "tables": tables,
451 + "records": records,
452 + }
modified apps/web/app/layout.tsx +2 −0
@@ -77,6 +77,7 @@ export const viewport: Viewport = {
77 77
78 78 const NAV_LINKS = [
79 79 { href: "/status", label: "État du moteur" },
80 + { href: "/stats", label: "Stats" },
80 81 { href: "/soumettre", label: "Soumettre un site" },
81 82 { href: "/contact", label: "Contact" },
82 83 ] as const;
@@ -84,6 +85,7 @@ const NAV_LINKS = [
84 85 const LOCAL_FOOTER_LINKS = [
85 86 { href: "/trouveka-bot", label: "À propos du robot" },
86 87 { href: "/status", label: "État du moteur" },
88 + { href: "/stats", label: "Statistiques" },
87 89 { href: "/soumettre", label: "Soumettre un site" },
88 90 { href: "/contact", label: "Contact" },
89 91 ];
added apps/web/app/stats/page.tsx +22 −0
@@ -0,0 +1,22 @@
1 +/**
2 + * Trouve-KA — page /stats : tableau de bord analytique du moteur
3 + * Author: Simon-Pierre Boucher
4 + * Contact: contact@spboucher.ai
5 + */
6 +
7 +import type { Metadata } from "next";
8 +import { StatsPanel } from "./stats-panel";
9 +
10 +export const metadata: Metadata = {
11 + title: "Statistiques",
12 + description:
13 + "Tableau de bord analytique de Trouve-KA : pages et domaines indexés, rythme de crawl, erreurs, file frontier, tendances de recherche — et rapport PDF Groupe-KA.",
14 +};
15 +
16 +export default function StatsPage() {
17 + return (
18 + <main id="contenu" className="mx-auto w-full max-w-6xl flex-1 px-4 py-10 sm:px-6">
19 + <StatsPanel />
20 + </main>
21 + );
22 +}
added apps/web/app/stats/stats-panel.tsx +204 −0
@@ -0,0 +1,204 @@
1 +/**
2 + * Trouve-KA — panneau client /stats (kit ka-ui/stats, contrat SPEC.md)
3 + * Author: Simon-Pierre Boucher
4 + * Contact: contact@spboucher.ai
5 + */
6 +
7 +"use client";
8 +
9 +import { useCallback, useEffect, useState } from "react";
10 +import {
11 + BarChart,
12 + CalendarHeatmap,
13 + DataTable,
14 + Donut,
15 + EmptyBlock,
16 + Fraicheur,
17 + KpiCard,
18 + LineChart,
19 + PdfButton,
20 + PeriodSelector,
21 + RecordCard,
22 + type BreakItem,
23 + type Kpi,
24 + type RecordFact,
25 + type Serie,
26 + type TableSpec,
27 +} from "@/ka/stats/kacharts";
28 +
29 +type Breakdown = { id: string; title: string; kind?: "donut" | "bar"; items: BreakItem[] };
30 +
31 +type Dashboard = {
32 + updated: string;
33 + period: { from: string; to: string; label: string };
34 + kpis: Kpi[];
35 + series: Serie[];
36 + breakdowns: Breakdown[];
37 + heatmap?: { title: string; cells: { date: string; value: number }[] };
38 + tables: TableSpec[];
39 + records: RecordFact[];
40 +};
41 +
42 +export function StatsPanel() {
43 + const [period, setPeriod] = useState("30j");
44 + const [custom, setCustom] = useState<{ from: string; to: string }>({ from: "", to: "" });
45 + const [data, setData] = useState<Dashboard | null>(null);
46 + const [loading, setLoading] = useState(true);
47 + const [error, setError] = useState(false);
48 +
49 + const hasCustom = Boolean(custom.from && custom.to);
50 +
51 + const load = useCallback(async () => {
52 + setLoading(true);
53 + setError(false);
54 + try {
55 + const p = new URLSearchParams({ period });
56 + if (hasCustom) {
57 + p.set("from", custom.from);
58 + p.set("to", custom.to);
59 + }
60 + const res = await fetch(`/api/stats/dashboard?${p}`);
61 + if (!res.ok) throw new Error(`Erreur ${res.status}`);
62 + setData((await res.json()) as Dashboard);
63 + } catch {
64 + setError(true);
65 + } finally {
66 + setLoading(false);
67 + }
68 + }, [period, custom.from, custom.to, hasCustom]);
69 +
70 + useEffect(() => {
71 + void load();
72 + }, [load]);
73 +
74 + const topQueries = data?.tables.find((t) => t.id === "topq");
75 + const otherTables = data?.tables.filter((t) => t.id !== "topq") ?? [];
76 +
77 + return (
78 + <div>
79 + <p className="kicker">Transparence</p>
80 + <div className="mt-3 flex flex-wrap items-start justify-between gap-4">
81 + <div>
82 + <h1 className="text-3xl font-bold tracking-[-0.03em] text-ink">
83 + Statistiques du moteur
84 + </h1>
85 + <p className="mt-2 max-w-xl text-sm text-ink-2">
86 + Le tableau de bord complet de Trouve-KA : indexation, crawl, erreurs et
87 + recherches — calculé sur les données réelles du moteur, rien d&apos;inventé.
88 + </p>
89 + </div>
90 + <div className="flex flex-col items-start gap-3 sm:items-end">
91 + <PdfButton period={period} from={hasCustom ? custom.from : undefined} to={hasCustom ? custom.to : undefined} />
92 + {data && <Fraicheur updated={data.updated} onRefresh={() => void load()} />}
93 + </div>
94 + </div>
95 +
96 + <div className="mt-8">
97 + <PeriodSelector
98 + value={period}
99 + onChange={(p) => {
100 + setCustom({ from: "", to: "" });
101 + setPeriod(p);
102 + }}
103 + custom={custom}
104 + onCustom={(from, to) => setCustom({ from, to })}
105 + />
106 + </div>
107 +
108 + {error && (
109 + <div className="card mt-8" style={{ padding: 20 }}>
110 + <b style={{ fontFamily: "var(--font-display)" }}>Statistiques indisponibles</b>
111 + <p className="klabel" style={{ margin: "6px 0 12px" }}>
112 + Le moteur n&apos;a pas pu répondre. Réessayez dans un instant.
113 + </p>
114 + <button type="button" className="btn btn-primary" onClick={() => void load()}>
115 + Réessayer
116 + </button>
117 + </div>
118 + )}
119 +
120 + {!error && !data && loading && (
121 + <p className="klabel mt-10" role="status">
122 + Chargement des statistiques…
123 + </p>
124 + )}
125 +
126 + {data && (
127 + <div style={{ opacity: loading ? 0.55 : 1, transition: "opacity 0.15s" }}>
128 + {/* 1. Bandeau KPI */}
129 + <section aria-label="Indicateurs clés" className="mt-6 grid grid-cols-1 gap-3 sm:grid-cols-2 lg:grid-cols-4">
130 + {data.kpis.map((k) => (
131 + <KpiCard key={k.id} k={k} />
132 + ))}
133 + </section>
134 +
135 + {/* 3. Courbes d'évolution */}
136 + <section aria-label="Évolution" className="mt-8 grid grid-cols-1 gap-4 xl:grid-cols-2">
137 + {data.series.map((s, i) => (
138 + <div key={s.id} className={i === 0 ? "xl:col-span-2" : undefined}>
139 + {s.kind === "bar" ? (
140 + <BarChart
141 + title={s.title}
142 + items={s.points.map((p) => ({ label: p.t, value: p.v }))}
143 + unit={s.unit}
144 + />
145 + ) : (
146 + <LineChart serie={s} />
147 + )}
148 + </div>
149 + ))}
150 + </section>
151 +
152 + {/* Répartitions */}
153 + <section aria-label="Répartitions" className="mt-8 grid grid-cols-1 gap-4 md:grid-cols-2">
154 + {data.breakdowns.map((b) =>
155 + b.kind === "bar" ? (
156 + <BarChart key={b.id} title={b.title} items={b.items} unit="pages" />
157 + ) : (
158 + <Donut key={b.id} title={b.title} items={b.items} />
159 + ),
160 + )}
161 + </section>
162 +
163 + {/* Calendrier de chaleur */}
164 + {data.heatmap && (
165 + <section aria-label="Calendrier d'activité" className="mt-8">
166 + <CalendarHeatmap title={data.heatmap.title} cells={data.heatmap.cells} />
167 + </section>
168 + )}
169 +
170 + {/* 5. Tableaux détaillés */}
171 + <section aria-label="Tableaux détaillés" className="mt-8 grid grid-cols-1 gap-4">
172 + {topQueries ? (
173 + <DataTable spec={topQueries} />
174 + ) : (
175 + <EmptyBlock title="Top requêtes de recherche" />
176 + )}
177 + {otherTables.map((t) => (
178 + <DataTable key={t.id} spec={t} />
179 + ))}
180 + </section>
181 +
182 + {/* 6. Records & faits marquants */}
183 + {data.records.length > 0 && (
184 + <section aria-label="Records et faits marquants" className="mt-8">
185 + <h2 className="text-lg font-bold tracking-[-0.02em] text-ink">
186 + Records &amp; faits marquants
187 + </h2>
188 + <div className="mt-3 grid grid-cols-1 gap-3 sm:grid-cols-2">
189 + {data.records.map((r) => (
190 + <RecordCard key={r.label} r={r} />
191 + ))}
192 + </div>
193 + </section>
194 + )}
195 +
196 + {/* 7. Fraîcheur (rappel en bas de page) */}
197 + <div className="mt-10">
198 + <Fraicheur updated={data.updated} onRefresh={() => void load()} />
199 + </div>
200 + </div>
201 + )}
202 + </div>
203 + );
204 +}
added apps/web/ka/stats/SPEC.md +123 −0
@@ -0,0 +1,123 @@
1 +# ka-stats — module Stats commun Groupe KA (spec v1)
2 +
3 +Contrat partagé par les 12 plateformes pour leurs pages **/stats** (tableau de
4 +bord analytique) et l'**export PDF** estampillé Groupe-KA. Le visuel suit le
5 +design system ka-ui (tokens.css) avec l'accent de la marque.
6 +
7 +## 1. Page /stats — structure obligatoire (dans cet ordre)
8 +
9 +1. **Bandeau KPI** : 4–6 grandes cartes (`KpiCard`) — valeur, libellé,
10 + variation vs période précédente (▲/▼ + %, vert `--green` / rouge `--danger`).
11 +2. **Sélecteur de période global** (`PeriodSelector`) : `aujourd'hui · 7 j ·
12 + 30 j · 3 m · 6 m · 12 m · année en cours · tout` + plage personnalisée
13 + (2 champs date). Toute la page se recalcule (state → refetch dashboard).
14 +3. **Graphiques** : courbes d'évolution (`LineChart`, survol = infobulle,
15 + légende cliquable pour masquer une série, comparaison N vs N-1 en
16 + pointillé), barres (`BarChart`), anneaux (`Donut`), calendrier de chaleur
17 + (`CalendarHeatmap`) quand pertinent.
18 +4. **Répartition géographique** (par ville/région) quand pertinent — barres
19 + horizontales triées (pas besoin de vraie carte).
20 +5. **Tableaux détaillés** (`DataTable`) : tri par colonne, recherche interne,
21 + pagination (25/pg), débordement horizontal propre sur mobile (.tbl-wrap).
22 +6. **Records & faits marquants** : générés depuis les données (jour record,
23 + plus forte croissance, meilleure entrée…) — cartes compactes.
24 +7. **Fraîcheur** : « Mis à jour le {date heure} » + bouton Rafraîchir.
25 +8. **Bouton PDF** bien visible en haut : « Télécharger le rapport PDF » avec
26 + deux choix (Rapport complet / Synthèse 2 pages). Indicateur de progression
27 + si > 2 s.
28 +
29 +Responsive : KPI empilés < 768 px, graphiques pleine largeur redimensionnés
30 +(SVG viewBox), tableaux en défilement horizontal contenu, tactile ≥ 44 px.
31 +AUCUNE donnée inventée : une stat indisponible = bloc « Pas encore mesuré »
32 +(carte grise propre), jamais un faux chiffre.
33 +
34 +## 2. API — contrat commun
35 +
36 +`GET /api/stats/dashboard?period=7j|30j|3m|6m|12m|annee|tout|auj&from=YYYY-MM-DD&to=YYYY-MM-DD`
37 +
38 +```jsonc
39 +{
40 + "updated": "2026-08-17T21:04:00-04:00",
41 + "period": { "from": "2026-07-18", "to": "2026-08-17", "label": "30 jours" },
42 + "kpis": [ { "id": "total", "label": "Annonces actives", "value": 33744,
43 + "unit": "", "delta_pct": 4.2, "direction": "up" } ],
44 + "series": [ { "id": "vol", "title": "Annonces actives par jour", "unit": "annonces",
45 + "kind": "line", "points": [{ "t": "2026-07-18", "v": 31200 }],
46 + "compare": [{ "t": "2025-07-18", "v": 24100 }] } ],
47 + "breakdowns": [ { "id": "types", "title": "Par type", "kind": "donut",
48 + "items": [{ "label": "4½", "value": 9120 }] } ],
49 + "geo": { "title": "Par région", "items": [{ "label": "Montréal", "value": 15680 }] },
50 + "heatmap": { "title": "Activité", "cells": [{ "date": "2026-08-01", "value": 210 }] },
51 + "tables": [ { "id": "top", "title": "Top villes", "columns": ["Ville", "Annonces", "Δ 30 j"],
52 + "rows": [["Montréal", 15680, "+3,1 %"]] } ],
53 + "records": [ { "label": "Jour record d'ajouts", "value": "412 annonces", "date": "2026-08-09" } ]
54 +}
55 +```
56 +
57 +Champs absents = section masquée. Cache serveur recommandé (≥ 5 min par
58 +période). Les valeurs proviennent des données réelles (DB de la plateforme,
59 +journaux de sync des connecteurs, /api/v1/runs d'API-KA…).
60 +
61 +`GET /api/stats/report?period=…&from=&to=&mode=complet|synthese`
62 +→ `application/pdf`, en-tête `Content-Disposition: attachment; filename=
63 +groupe-ka_<plateforme>_stats_<periode>_<YYYY-MM-DD>.pdf`.
64 +
65 +## 3. PDF — gabarit Groupe-KA (implémentations : `kapdf.py` fpdf2 pour les
66 +apps Python ; les apps Next portent le même gabarit en pdfkit)
67 +
68 +- **Couverture** : cadre encre, kicker « GROUPE KA · RAPPORT STATISTIQUE »,
69 + wordmark de la plateforme (boîte encre + accent), sous-titre, période
70 + couverte, date/heure de génération, bande encre au pied avec
71 + « par Groupe KA — groupe-ka.com ».
72 +- **Sommaire** avec numéros de pages (mode complet).
73 +- **KPI** : grille de cartes (bordure encre, valeur en gros, delta coloré).
74 +- **Graphiques VECTORIELS** (dessinés en primitives, jamais de capture) :
75 + courbes, barres, anneaux — accent de la plateforme, axes/graduations encre.
76 +- **Tableaux** paginés proprement (lignes zébrées `--surface-2`, jamais
77 + coupés en deux à cheval sur une ligne).
78 +- **Records** puis **page de fin** : coordonnées Groupe KA (3 courriels +
79 + rôles d'ecosystem.json, groupe-ka.com), avertissement d'agrégateur,
80 + mentions légales courtes.
81 +- **Chaque page** : en-tête discret (« Groupe KA · {Plateforme} », filet
82 + encre) + pied (« © Groupe-KA — {année} — groupe-ka.com · {période} · p. X/Y »).
83 +- A4 portrait, marges 18 mm, typo : Helvetica (fallback sûr) ou fonts TTF du
84 + DS si présentes. Mode « synthese » = couverture + 1 page KPI/records.
85 +
86 +## 4. Spécifique par plateforme (sections métier attendues)
87 +
88 +- **groupe-ka** : tableau de bord maître — consolidation des 12 (volume total,
89 + croissance), classement des plateformes, bloc résumé par plateforme + lien
90 + vers sa page /stats ; « Rapport écosystème complet » = PDF consolidé.
91 +- **lou-ka** : annonces actives/nouvelles/retirées, loyers moyens/médians par
92 + ville & taille, évolution, répartition par type, top villes.
93 +- **immo-ka** : annonces actives/nouvelles/vendues-retirées, prix moyen/médian
94 + par ville/région/type, délai de présence, top villes, tension du marché.
95 +- **vrai-prix** : couverture du rôle (unités, valeur totale), estimations
96 + servies si journalisées, répartitions par municipalité/type, indices marché.
97 +- **auto-ka** : volume par marque/modèle/année/carburant/boîte, prix moyens et
98 + km moyens par segment, top marques/modèles.
99 +- **fabri-ka** : produits par catégorie/région/boutique, fourchettes de prix,
100 + nouveautés par période, top catégories.
101 +- **food-ka** : produits suivis, relevés de prix, soldes détectés (baisses/
102 + hausses, amplitude), top produits en solde, prix moyens par catégorie.
103 +- **resto-ka** : restos par cuisine/ville/gamme, menus & plats, nouveautés/
104 + fermetures détectées, top établissements.
105 +- **sorti-ka** : événements à venir/passés par catégorie/ville, gratuits vs
106 + payants, heatmap calendrier, top lieux.
107 +- **crea-ka** : créateurs par plateforme/niche/tier, comptes reliés, top
108 + créateurs, croissance du répertoire.
109 +- **trouve-ka** : pages indexées, domaines, rythme de crawl (indexées/h),
110 + erreurs, file frontier, tendances si les requêtes sont journalisées.
111 +- **api-ka** : appels par endpoint/jour/heure, latences moyennes + p95, taux
112 + d'erreur, top endpoints, uptime (données des middlewares de logging + runs).
113 +- **Transverse (tous)** : volume total agrégé + croissance, connecteurs actifs
114 + et éléments ajoutés/mis à jour par période (journaux de sync), complétude/
115 + fraîcheur moyenne des fiches quand mesurable. Trafic web : seulement si des
116 + journaux d'accès existent — sinon état vide propre.
117 +
118 +## 5. Ajouter une métrique / un graphique / une plateforme
119 +
120 +1 métrique = 1 entrée `kpis[]` ou `series[]` côté API (requête SQL agrégée +
121 +cache) — le front la rend automatiquement. 1 plateforme = implémenter les 2
122 +endpoints du contrat + une page /stats montée sur les composants du kit +
123 +`kapdf.py` (ou gabarit pdfkit) branché sur le même JSON de dashboard.
added apps/web/ka/stats/kacharts.tsx +390 −0
@@ -0,0 +1,390 @@
1 +"use client";
2 +// Auteur : Simon-Pierre Boucher — contact@spboucher.ai
3 +// ka-ui/stats/kacharts.tsx — kit de graphiques SVG du module Stats commun
4 +// Groupe KA (zéro dépendance, React 18+). Style : design system ka-ui
5 +// (bordures encre, accent de la plateforme via var(--accent)).
6 +// Composants : KpiCard, PeriodSelector, LineChart (infobulle + légende
7 +// cliquable + comparaison N-1), BarChart, Donut, CalendarHeatmap, DataTable
8 +// (tri/recherche/pagination), RecordCard, PdfButton, EmptyBlock, Fraicheur.
9 +import { useMemo, useState } from "react";
10 +
11 +/* ---------- types (contrat SPEC.md) ---------- */
12 +export type Kpi = {
13 + id: string; label: string; value: number | string; unit?: string;
14 + delta_pct?: number | null; direction?: "up" | "down";
15 +};
16 +export type Point = { t: string; v: number };
17 +export type Serie = {
18 + id: string; title: string; unit?: string; kind?: "line" | "bar";
19 + points: Point[]; compare?: Point[];
20 +};
21 +export type BreakItem = { label: string; value: number };
22 +export type TableSpec = { id: string; title: string; columns: string[]; rows: (string | number)[][] };
23 +export type RecordFact = { label: string; value: string; date?: string };
24 +
25 +export const PERIODS: { id: string; label: string }[] = [
26 + { id: "auj", label: "Aujourd'hui" },
27 + { id: "7j", label: "7 jours" },
28 + { id: "30j", label: "30 jours" },
29 + { id: "3m", label: "3 mois" },
30 + { id: "6m", label: "6 mois" },
31 + { id: "12m", label: "12 mois" },
32 + { id: "annee", label: "Année en cours" },
33 + { id: "tout", label: "Tout" },
34 +];
35 +
36 +export const fmtInt = (n: number) => n.toLocaleString("fr-CA");
37 +export const fmtNum = (n: number) =>
38 + Number.isInteger(n) ? fmtInt(n) : n.toLocaleString("fr-CA", { maximumFractionDigits: 2 });
39 +
40 +/* ---------- KPI ---------- */
41 +export function KpiCard({ k }: { k: Kpi }) {
42 + const up = (k.direction ?? ((k.delta_pct ?? 0) >= 0 ? "up" : "down")) === "up";
43 + return (
44 + <article className="card" style={{ padding: "14px 16px", minWidth: 0 }}>
45 + <p style={{ margin: 0, fontFamily: "var(--font-display)", fontWeight: 700, fontSize: "clamp(22px,2.4vw,30px)", letterSpacing: "-0.02em" }}>
46 + {typeof k.value === "number" ? fmtNum(k.value) : k.value}
47 + {k.unit ? <span style={{ fontSize: "0.6em", color: "var(--ink-2)" }}> {k.unit}</span> : null}
48 + </p>
49 + <p className="klabel" style={{ margin: "6px 0 0" }}>{k.label}</p>
50 + {k.delta_pct !== undefined && k.delta_pct !== null && (
51 + <p style={{ margin: "8px 0 0", fontFamily: "var(--font-mono)", fontSize: 11, fontWeight: 700, color: up ? "var(--green)" : "var(--danger)" }}>
52 + {up ? "▲" : "▼"} {k.delta_pct >= 0 ? "+" : ""}{fmtNum(k.delta_pct)} % <span style={{ color: "var(--ink-3)", fontWeight: 500 }}>vs période préc.</span>
53 + </p>
54 + )}
55 + </article>
56 + );
57 +}
58 +
59 +/* ---------- Sélecteur de période ---------- */
60 +export function PeriodSelector({
61 + value, onChange, custom, onCustom,
62 +}: {
63 + value: string; onChange: (p: string) => void;
64 + custom?: { from: string; to: string }; onCustom?: (from: string, to: string) => void;
65 +}) {
66 + return (
67 + <div style={{ display: "flex", flexWrap: "wrap", gap: 8, alignItems: "center" }}>
68 + {PERIODS.map((p) => (
69 + <button key={p.id} type="button" onClick={() => onChange(p.id)}
70 + className="chip" aria-pressed={value === p.id}
71 + style={{ cursor: "pointer", minHeight: 44, background: value === p.id ? "var(--accent)" : "var(--surface)", color: value === p.id ? "var(--on-accent)" : "var(--ink)" }}>
72 + {p.label}
73 + </button>
74 + ))}
75 + {onCustom && (
76 + <span style={{ display: "inline-flex", gap: 6, alignItems: "center" }}>
77 + <input className="input" type="date" style={{ width: 150 }} value={custom?.from ?? ""} aria-label="Du"
78 + onChange={(e) => onCustom(e.target.value, custom?.to ?? "")} />
79 + <span className="klabel">au</span>
80 + <input className="input" type="date" style={{ width: 150 }} value={custom?.to ?? ""} aria-label="Au"
81 + onChange={(e) => onCustom(custom?.from ?? "", e.target.value)} />
82 + </span>
83 + )}
84 + </div>
85 + );
86 +}
87 +
88 +/* ---------- Courbe ---------- */
89 +export function LineChart({ serie, height = 240 }: { serie: Serie; height?: number }) {
90 + const [hide, setHide] = useState<{ cur: boolean; cmp: boolean }>({ cur: false, cmp: false });
91 + const [hover, setHover] = useState<number | null>(null);
92 + const W = 720, H = height, PL = 54, PR = 10, PT = 14, PB = 26;
93 + const pts = serie.points ?? [];
94 + if (pts.length < 2) return <EmptyBlock title={serie.title} />;
95 + const all = [...(hide.cur ? [] : pts), ...(!hide.cmp && serie.compare ? serie.compare : [])];
96 + const vmax = Math.max(...all.map((p) => p.v), 1);
97 + const vmin = Math.min(0, ...all.map((p) => p.v));
98 + const X = (i: number, n: number) => PL + ((W - PL - PR) * i) / (n - 1);
99 + const Y = (v: number) => PT + (H - PT - PB) * (1 - (v - vmin) / (vmax - vmin || 1));
100 + const path = (s: Point[]) => s.map((p, i) => `${i ? "L" : "M"}${X(i, s.length)},${Y(p.v)}`).join("");
101 + const hi = hover !== null ? Math.min(pts.length - 1, Math.max(0, hover)) : null;
102 + return (
103 + <figure className="card" style={{ margin: 0, padding: 16 }}>
104 + <figcaption style={{ display: "flex", justifyContent: "space-between", flexWrap: "wrap", gap: 8 }}>
105 + <b style={{ fontFamily: "var(--font-display)", fontSize: 15 }}>{serie.title}</b>
106 + <span style={{ display: "flex", gap: 10 }}>
107 + <LegendChip label="Période courante" color="var(--accent)" off={hide.cur} onClick={() => setHide((h) => ({ ...h, cur: !h.cur }))} />
108 + {serie.compare && <LegendChip label="Période comparée" color="var(--ink-3)" dashed off={hide.cmp} onClick={() => setHide((h) => ({ ...h, cmp: !h.cmp }))} />}
109 + </span>
110 + </figcaption>
111 + <svg viewBox={`0 0 ${W} ${H}`} style={{ width: "100%", height: "auto", marginTop: 10, touchAction: "pan-y" }} role="img" aria-label={serie.title}
112 + onMouseMove={(e) => {
113 + const r = (e.currentTarget as SVGSVGElement).getBoundingClientRect();
114 + const fx = ((e.clientX - r.left) / r.width) * W;
115 + setHover(Math.round(((fx - PL) / (W - PL - PR)) * (pts.length - 1)));
116 + }}
117 + onMouseLeave={() => setHover(null)}>
118 + {[0, 1, 2, 3, 4].map((g) => {
119 + const y = PT + ((H - PT - PB) * g) / 4;
120 + const v = vmax - ((vmax - vmin) * g) / 4;
121 + return (
122 + <g key={g}>
123 + <line x1={PL} x2={W - PR} y1={y} y2={y} stroke="var(--line)" strokeWidth={1} />
124 + <text x={PL - 6} y={y + 3} textAnchor="end" fontSize={10} fill="var(--ink-3)" fontFamily="var(--font-mono)">{fmtInt(Math.round(v))}</text>
125 + </g>
126 + );
127 + })}
128 + {[0, Math.floor(pts.length / 2), pts.length - 1].map((i) => (
129 + <text key={i} x={X(i, pts.length)} y={H - 8} textAnchor="middle" fontSize={10} fill="var(--ink-3)" fontFamily="var(--font-mono)">{pts[i].t}</text>
130 + ))}
131 + {!hide.cmp && serie.compare && serie.compare.length > 1 && (
132 + <path d={path(serie.compare)} fill="none" stroke="var(--ink-3)" strokeWidth={1.4} strokeDasharray="4 4" />
133 + )}
134 + {!hide.cur && <path d={path(pts)} fill="none" stroke="var(--accent)" strokeWidth={2.4} />}
135 + {hi !== null && (
136 + <g>
137 + <line x1={X(hi, pts.length)} x2={X(hi, pts.length)} y1={PT} y2={H - PB} stroke="var(--ink)" strokeWidth={1} strokeDasharray="2 3" />
138 + <circle cx={X(hi, pts.length)} cy={Y(pts[hi].v)} r={4} fill="var(--accent)" stroke="var(--ink)" strokeWidth={1.5} />
139 + </g>
140 + )}
141 + </svg>
142 + {hi !== null && (
143 + <p className="chip" style={{ marginTop: 8 }}>
144 + {pts[hi].t} — <b>{fmtNum(pts[hi].v)}{serie.unit ? ` ${serie.unit}` : ""}</b>
145 + {serie.compare?.[hi] && !hide.cmp ? <span style={{ color: "var(--ink-3)" }}> · N-1 : {fmtNum(serie.compare[hi].v)}</span> : null}
146 + </p>
147 + )}
148 + </figure>
149 + );
150 +}
151 +
152 +function LegendChip({ label, color, off, dashed, onClick }: { label: string; color: string; off: boolean; dashed?: boolean; onClick: () => void }) {
153 + return (
154 + <button type="button" onClick={onClick} aria-pressed={!off}
155 + style={{ display: "inline-flex", alignItems: "center", gap: 6, border: 0, background: "none", cursor: "pointer", opacity: off ? 0.4 : 1, fontFamily: "var(--font-mono)", fontSize: 10.5, fontWeight: 700, textTransform: "uppercase", letterSpacing: "0.06em", minHeight: 44 }}>
156 + <span style={{ width: 18, height: 0, borderTop: `3px ${dashed ? "dashed" : "solid"} ${color}` }} />
157 + {label}
158 + </button>
159 + );
160 +}
161 +
162 +/* ---------- Barres horizontales (répartitions, géo) ---------- */
163 +export function BarChart({ title, items, unit }: { title: string; items: BreakItem[]; unit?: string }) {
164 + const rows = (items ?? []).slice(0, 14);
165 + if (!rows.length) return <EmptyBlock title={title} />;
166 + const max = Math.max(...rows.map((r) => r.value), 1);
167 + return (
168 + <figure className="card" style={{ margin: 0, padding: 16 }}>
169 + <figcaption><b style={{ fontFamily: "var(--font-display)", fontSize: 15 }}>{title}</b></figcaption>
170 + <div style={{ marginTop: 12, display: "grid", gap: 9 }}>
171 + {rows.map((r) => (
172 + <div key={r.label} title={`${r.label} — ${fmtNum(r.value)}${unit ? ` ${unit}` : ""}`}>
173 + <div style={{ display: "flex", justifyContent: "space-between", fontSize: 12.5 }}>
174 + <span style={{ overflow: "hidden", textOverflow: "ellipsis", whiteSpace: "nowrap" }}>{r.label}</span>
175 + <b style={{ fontFamily: "var(--font-mono)", fontSize: 11.5 }}>{fmtNum(r.value)}{unit ? ` ${unit}` : ""}</b>
176 + </div>
177 + <div style={{ marginTop: 3, height: 12, background: "rgba(20,24,20,0.06)", borderRadius: "0 3px 3px 0" }}>
178 + <div style={{ height: "100%", width: `${Math.max((r.value / max) * 100, 1)}%`, background: "var(--accent)", border: "1px solid var(--ink)", borderRadius: "0 3px 3px 0", boxSizing: "border-box" }} />
179 + </div>
180 + </div>
181 + ))}
182 + </div>
183 + </figure>
184 + );
185 +}
186 +
187 +/* ---------- Anneau ---------- */
188 +export function Donut({ title, items }: { title: string; items: BreakItem[] }) {
189 + const rows = (items ?? []).filter((i) => i.value > 0).slice(0, 8);
190 + const total = rows.reduce((s, r) => s + r.value, 0);
191 + if (!total) return <EmptyBlock title={title} />;
192 + const R = 74, C = 2 * Math.PI * R;
193 + let acc = 0;
194 + const shades = [1, 0.78, 0.58, 0.42, 0.3, 0.22, 0.15, 0.1];
195 + return (
196 + <figure className="card" style={{ margin: 0, padding: 16 }}>
197 + <figcaption><b style={{ fontFamily: "var(--font-display)", fontSize: 15 }}>{title}</b></figcaption>
198 + <div style={{ display: "flex", flexWrap: "wrap", gap: 18, alignItems: "center", marginTop: 12 }}>
199 + <svg viewBox="0 0 200 200" style={{ width: 180, maxWidth: "100%" }} role="img" aria-label={title}>
200 + {rows.map((r, i) => {
201 + const frac = r.value / total;
202 + const off = acc; acc += frac;
203 + return (
204 + <circle key={r.label} cx={100} cy={100} r={R} fill="none"
205 + stroke="var(--accent)" strokeOpacity={shades[i % shades.length]}
206 + strokeWidth={30} strokeDasharray={`${frac * C} ${C}`} strokeDashoffset={-off * C}
207 + transform="rotate(-90 100 100)">
208 + <title>{`${r.label} — ${fmtNum(r.value)} (${((100 * r.value) / total).toFixed(1)} %)`}</title>
209 + </circle>
210 + );
211 + })}
212 + <circle cx={100} cy={100} r={R} fill="none" stroke="var(--ink)" strokeWidth={1} opacity={0.5} />
213 + </svg>
214 + <ul style={{ listStyle: "none", margin: 0, padding: 0, display: "grid", gap: 6, minWidth: 200, flex: 1 }}>
215 + {rows.map((r, i) => (
216 + <li key={r.label} style={{ display: "flex", alignItems: "center", gap: 8, fontSize: 12.5 }}>
217 + <span style={{ width: 11, height: 11, borderRadius: 3, border: "1px solid var(--ink)", background: "var(--accent)", opacity: shades[i % shades.length] }} />
218 + <span style={{ flex: 1, overflow: "hidden", textOverflow: "ellipsis", whiteSpace: "nowrap" }}>{r.label}</span>
219 + <b style={{ fontFamily: "var(--font-mono)", fontSize: 11 }}>{((100 * r.value) / total).toFixed(1)} %</b>
220 + </li>
221 + ))}
222 + </ul>
223 + </div>
224 + </figure>
225 + );
226 +}
227 +
228 +/* ---------- Calendrier de chaleur ---------- */
229 +export function CalendarHeatmap({ title, cells }: { title: string; cells: { date: string; value: number }[] }) {
230 + if (!cells?.length) return <EmptyBlock title={title} />;
231 + const byDate = new Map(cells.map((c) => [c.date, c.value]));
232 + const dates = cells.map((c) => c.date).sort();
233 + const end = new Date(dates[dates.length - 1] + "T12:00:00");
234 + const max = Math.max(...cells.map((c) => c.value), 1);
235 + const weeks = 26, cols: { date: string; v: number }[][] = [];
236 + const cur = new Date(end);
237 + cur.setDate(cur.getDate() - (weeks * 7 - 1));
238 + for (let w = 0; w < weeks; w++) {
239 + const col: { date: string; v: number }[] = [];
240 + for (let d = 0; d < 7; d++) {
241 + const iso = cur.toISOString().slice(0, 10);
242 + col.push({ date: iso, v: byDate.get(iso) ?? 0 });
243 + cur.setDate(cur.getDate() + 1);
244 + }
245 + cols.push(col);
246 + }
247 + return (
248 + <figure className="card" style={{ margin: 0, padding: 16 }}>
249 + <figcaption><b style={{ fontFamily: "var(--font-display)", fontSize: 15 }}>{title}</b> <span className="klabel">26 dernières semaines</span></figcaption>
250 + <div className="tbl-wrap" style={{ marginTop: 12 }}>
251 + <svg viewBox={`0 0 ${weeks * 14} ${7 * 14}`} style={{ minWidth: 480, width: "100%", height: "auto" }} role="img" aria-label={title}>
252 + {cols.map((col, w) => col.map((c, d) => (
253 + <rect key={c.date} x={w * 14} y={d * 14} width={12} height={12} rx={2.5}
254 + fill={c.v ? "var(--accent)" : "rgba(20,24,20,0.07)"} fillOpacity={c.v ? 0.25 + 0.75 * (c.v / max) : 1}
255 + stroke="rgba(20,24,20,0.15)" strokeWidth={0.5}>
256 + <title>{`${c.date} — ${fmtNum(c.v)}`}</title>
257 + </rect>
258 + )))}
259 + </svg>
260 + </div>
261 + </figure>
262 + );
263 +}
264 +
265 +/* ---------- Tableau : tri, recherche, pagination ---------- */
266 +export function DataTable({ spec, pageSize = 25 }: { spec: TableSpec; pageSize?: number }) {
267 + const [q, setQ] = useState("");
268 + const [sort, setSort] = useState<{ col: number; dir: 1 | -1 } | null>(null);
269 + const [page, setPage] = useState(0);
270 + const rows = useMemo(() => {
271 + let r = spec.rows ?? [];
272 + if (q) r = r.filter((row) => row.some((c) => String(c).toLowerCase().includes(q.toLowerCase())));
273 + if (sort) r = [...r].sort((a, b) => {
274 + const x = a[sort.col], y = b[sort.col];
275 + const nx = typeof x === "number" ? x : parseFloat(String(x).replace(/[^\d.,-]/g, "").replace(",", "."));
276 + const ny = typeof y === "number" ? y : parseFloat(String(y).replace(/[^\d.,-]/g, "").replace(",", "."));
277 + if (!Number.isNaN(nx) && !Number.isNaN(ny)) return (nx - ny) * sort.dir;
278 + return String(x).localeCompare(String(y), "fr") * sort.dir;
279 + });
280 + return r;
281 + }, [spec.rows, q, sort]);
282 + const pages = Math.max(1, Math.ceil(rows.length / pageSize));
283 + const cur = Math.min(page, pages - 1);
284 + return (
285 + <section className="card" style={{ padding: 16 }}>
286 + <div style={{ display: "flex", flexWrap: "wrap", gap: 10, justifyContent: "space-between", alignItems: "center" }}>
287 + <b style={{ fontFamily: "var(--font-display)", fontSize: 15 }}>{spec.title}</b>
288 + <input className="input" style={{ maxWidth: 240 }} placeholder="Rechercher…" value={q}
289 + onChange={(e) => { setQ(e.target.value); setPage(0); }} aria-label={`Rechercher dans ${spec.title}`} />
290 + </div>
291 + <div className="tbl-wrap" style={{ marginTop: 10 }}>
292 + <table style={{ width: "100%", borderCollapse: "collapse", fontSize: 13 }}>
293 + <thead>
294 + <tr>
295 + {spec.columns.map((c, i) => (
296 + <th key={c} onClick={() => setSort((s) => ({ col: i, dir: s?.col === i && s.dir === 1 ? -1 : 1 }))}
297 + style={{ cursor: "pointer", textAlign: "left", padding: "8px 10px", background: "var(--ink)", color: "var(--paper)", fontFamily: "var(--font-mono)", fontSize: 10.5, textTransform: "uppercase", letterSpacing: "0.06em", whiteSpace: "nowrap", userSelect: "none" }}
298 + aria-sort={sort?.col === i ? (sort.dir === 1 ? "ascending" : "descending") : "none"}>
299 + {c} {sort?.col === i ? (sort.dir === 1 ? "▲" : "▼") : "↕"}
300 + </th>
301 + ))}
302 + </tr>
303 + </thead>
304 + <tbody>
305 + {rows.slice(cur * pageSize, (cur + 1) * pageSize).map((row, ri) => (
306 + <tr key={ri} style={{ background: ri % 2 ? "var(--surface-2)" : "var(--surface)" }}>
307 + {row.map((c, ci) => (
308 + <td key={ci} style={{ padding: "7px 10px", borderBottom: "1px solid var(--line)", whiteSpace: "nowrap" }}>
309 + {typeof c === "number" ? fmtNum(c) : c}
310 + </td>
311 + ))}
312 + </tr>
313 + ))}
314 + </tbody>
315 + </table>
316 + </div>
317 + <div style={{ display: "flex", justifyContent: "space-between", alignItems: "center", marginTop: 10, flexWrap: "wrap", gap: 8 }}>
318 + <span className="klabel">{fmtInt(rows.length)} lignes</span>
319 + <span style={{ display: "flex", gap: 6 }}>
320 + <button type="button" className="btn btn-ghost" disabled={cur === 0} onClick={() => setPage(cur - 1)}>←</button>
321 + <span className="chip">{cur + 1} / {pages}</span>
322 + <button type="button" className="btn btn-ghost" disabled={cur >= pages - 1} onClick={() => setPage(cur + 1)}>→</button>
323 + </span>
324 + </div>
325 + </section>
326 + );
327 +}
328 +
329 +/* ---------- Records / faits marquants ---------- */
330 +export function RecordCard({ r }: { r: RecordFact }) {
331 + return (
332 + <article className="card" style={{ padding: "12px 16px", display: "flex", justifyContent: "space-between", gap: 12, alignItems: "baseline", background: "var(--surface-2)" }}>
333 + <span style={{ fontSize: 13, color: "var(--ink-2)" }}>{r.label}</span>
334 + <span style={{ textAlign: "right" }}>
335 + <b style={{ fontFamily: "var(--font-display)", fontSize: 15 }}>{r.value}</b>
336 + {r.date && <span className="klabel" style={{ display: "block" }}>{r.date}</span>}
337 + </span>
338 + </article>
339 + );
340 +}
341 +
342 +/* ---------- Bouton PDF ---------- */
343 +export function PdfButton({ period, from, to, endpoint = "/api/stats/report" }: { period: string; from?: string; to?: string; endpoint?: string }) {
344 + const [busy, setBusy] = useState(false);
345 + const url = (mode: string) => {
346 + const p = new URLSearchParams({ period, mode });
347 + if (from) p.set("from", from);
348 + if (to) p.set("to", to);
349 + return `${endpoint}?${p}`;
350 + };
351 + const dl = (mode: string) => {
352 + setBusy(true);
353 + const a = document.createElement("a");
354 + a.href = url(mode);
355 + a.download = "";
356 + document.body.appendChild(a);
357 + a.click();
358 + a.remove();
359 + setTimeout(() => setBusy(false), 2500);
360 + };
361 + return (
362 + <span style={{ display: "inline-flex", gap: 8, flexWrap: "wrap" }}>
363 + <button type="button" className="btn btn-primary" onClick={() => dl("complet")} disabled={busy}>
364 + {busy ? "Génération…" : "⬇ Télécharger le rapport PDF"}
365 + </button>
366 + <button type="button" className="btn btn-ghost" onClick={() => dl("synthese")} disabled={busy}>
367 + Synthèse (2 p.)
368 + </button>
369 + </span>
370 + );
371 +}
372 +
373 +/* ---------- États ---------- */
374 +export function EmptyBlock({ title }: { title: string }) {
375 + return (
376 + <div className="card" style={{ padding: 20, background: "var(--surface-2)", borderStyle: "dashed" }}>
377 + <b style={{ fontFamily: "var(--font-display)", fontSize: 14 }}>{title}</b>
378 + <p className="klabel" style={{ margin: "6px 0 0" }}>Pas encore mesuré — aucune donnée disponible pour cette période.</p>
379 + </div>
380 + );
381 +}
382 +
383 +export function Fraicheur({ updated, onRefresh }: { updated: string; onRefresh: () => void }) {
384 + return (
385 + <p style={{ display: "flex", gap: 10, alignItems: "center", flexWrap: "wrap", margin: 0 }}>
386 + <span className="klabel">Mis à jour le {new Date(updated).toLocaleString("fr-CA", { dateStyle: "medium", timeStyle: "short" })}</span>
387 + <button type="button" className="btn btn-ghost" onClick={onRefresh}>↻ Rafraîchir</button>
388 + </p>
389 + );
390 +}
added apps/web/ka/stats/kapdf.py +558 −0
@@ -0,0 +1,558 @@
1 +# Auteur : Simon-Pierre Boucher — contact@spboucher.ai
2 +# ka-ui/stats/kapdf.py — moteur PDF commun Groupe KA (fpdf2).
3 +# Consomme le JSON du contrat /api/stats/dashboard (voir SPEC.md) et produit
4 +# le rapport estampillé Groupe-KA : couverture, sommaire, KPI, graphiques
5 +# VECTORIELS (courbes/barres/anneaux), tableaux paginés, records, page de fin.
6 +# Usage :
7 +# from kapdf import GroupeKAReport
8 +# pdf_bytes = GroupeKAReport(site={"wordmark":"Lou·Ka","accent":"#ff6a00",
9 +# "domain":"www.lou-ka.com","tagline":"…"}, dashboard=dash_json,
10 +# mode="complet").build()
11 +# Dépendance : pip install fpdf2 (aucune autre)
12 +from __future__ import annotations
13 +
14 +import math
15 +from datetime import datetime
16 +from zoneinfo import ZoneInfo
17 +
18 +from fpdf import FPDF
19 +
20 +INK = (20, 24, 20)
21 +INK2 = (77, 85, 81)
22 +INK3 = (139, 146, 140)
23 +PAPER = (245, 243, 238)
24 +SURFACE2 = (250, 249, 245)
25 +GREEN = (28, 92, 65)
26 +DANGER = (179, 66, 58)
27 +WHITE = (255, 255, 255)
28 +
29 +EMAILS = [
30 + ("contact@groupe-ka.com", "Projets, partenariats & données"),
31 + ("info@groupe-ka.com", "Médias & questions générales"),
32 + ("admin@groupe-ka.com", "Légal, vie privée & Loi 25"),
33 +]
34 +DISCLAIMER = (
35 + "Groupe KA est un agrégateur de contenu : nous ne vendons rien, ne louons "
36 + "rien et ne sommes partie à aucune transaction. Données lues à la source, "
37 + "rien d'inventé, tout est traçable."
38 +)
39 +
40 +
41 +def _hex(c: str) -> tuple[int, int, int]:
42 + c = c.lstrip("#")
43 + return tuple(int(c[i : i + 2], 16) for i in (0, 2, 4)) # type: ignore
44 +
45 +
46 +def _fr(n) -> str:
47 + if isinstance(n, float) and not n.is_integer():
48 + return f"{n:,.2f}".replace(",", " ").replace(".", ",")
49 + return f"{int(n):,}".replace(",", " ")
50 +
51 +
52 +_SUBST = {
53 + "—": "-", "–": "-", "→": "->", "▲": "+", "▼": "-",
54 + "…": "...", "’": "'", "‘": "'", "“": '"', "”": '"',
55 + "œ": "oe", "Œ": "OE", "−": "-", " ": " ", " ": " ",
56 +}
57 +
58 +
59 +def _latin1(s: str) -> str:
60 + for k, v in _SUBST.items():
61 + s = s.replace(k, v)
62 + return s.encode("latin-1", "replace").decode("latin-1")
63 +
64 +
65 +class _PDF(FPDF):
66 + """FPDF avec en-tête/pied Groupe-KA sur chaque page (sauf couverture).
67 + Les polices core sont latin-1 : normalize_text sanitise en amont."""
68 +
69 + def normalize_text(self, text):
70 + return super().normalize_text(_latin1(text))
71 +
72 + def __init__(self, brand: str, accent: tuple, period_label: str):
73 + super().__init__(orientation="P", unit="mm", format="A4")
74 + self.brand = brand
75 + self.accent = accent
76 + self.period_label = period_label
77 + self.cover_mode = False
78 + self.set_margins(18, 20, 18)
79 + self.set_auto_page_break(True, margin=22)
80 +
81 + def header(self):
82 + if self.cover_mode:
83 + return
84 + self.set_font("helvetica", "B", 8.5)
85 + self.set_text_color(*INK)
86 + self.set_xy(18, 9)
87 + self.cell(0, 5, f"Groupe KA · {self.brand}")
88 + self.set_font("helvetica", "", 8)
89 + self.set_text_color(*INK3)
90 + self.set_xy(18, 9)
91 + self.cell(0, 5, "Rapport statistique", align="R")
92 + self.set_draw_color(*INK)
93 + self.set_line_width(0.5)
94 + self.line(18, 15.5, 192, 15.5)
95 + self.set_y(20)
96 +
97 + def footer(self):
98 + if self.cover_mode:
99 + return
100 + self.set_y(-15)
101 + self.set_draw_color(*INK3)
102 + self.set_line_width(0.2)
103 + self.line(18, self.get_y() - 1.5, 192, self.get_y() - 1.5)
104 + self.set_font("helvetica", "", 7.5)
105 + self.set_text_color(*INK3)
106 + year = datetime.now(ZoneInfo("America/Toronto")).year
107 + self.cell(130, 5, f"© Groupe-KA — {year} — groupe-ka.com · {self.period_label}")
108 + self.cell(0, 5, f"p. {self.page_no()}/{{nb}}", align="R")
109 +
110 +
111 +class GroupeKAReport:
112 + def __init__(self, site: dict, dashboard: dict, mode: str = "complet"):
113 + self.site = site
114 + self.d = dashboard
115 + self.mode = mode
116 + self.accent = _hex(site.get("accent", "#d9f26b"))
117 + period = dashboard.get("period", {}) or {}
118 + self.period_label = period.get("label") or "toute la période"
119 + self.pdf = _PDF(site.get("wordmark", ""), self.accent, self.period_label)
120 + self.toc: list[tuple[str, int]] = []
121 +
122 + # ---------- primitives ----------
123 + def _card(self, x, y, w, h, fill=WHITE):
124 + p = self.pdf
125 + p.set_draw_color(*INK)
126 + p.set_line_width(0.45)
127 + p.set_fill_color(*fill)
128 + p.rect(x, y, w, h, style="DF", round_corners=True, corner_radius=2.2)
129 +
130 + def _kicker(self, text):
131 + p = self.pdf
132 + p.set_font("helvetica", "B", 8)
133 + p.set_text_color(*GREEN)
134 + p.set_draw_color(*GREEN)
135 + p.set_line_width(0.6)
136 + y = p.get_y() + 2
137 + p.line(p.l_margin, y, p.l_margin + 7, y)
138 + p.set_xy(p.l_margin + 9, y - 2.5)
139 + p.cell(0, 5, text.upper())
140 + p.ln(8)
141 +
142 + def _section_title(self, title):
143 + if self.pdf.get_y() > 240:
144 + self.pdf.add_page()
145 + self._kicker("Groupe KA · " + self.site.get("wordmark", ""))
146 + self.pdf.set_font("helvetica", "B", 15)
147 + self.pdf.set_text_color(*INK)
148 + self.pdf.set_x(self.pdf.l_margin)
149 + self.pdf.cell(0, 8, title)
150 + self.toc.append((title, self.pdf.page_no()))
151 + self.pdf.ln(11)
152 +
153 + # ---------- pages ----------
154 + def _cover(self):
155 + p = self.pdf
156 + p.cover_mode = True
157 + p.set_auto_page_break(False)
158 + p.add_page()
159 + p.set_fill_color(*PAPER)
160 + p.rect(0, 0, 210, 297, style="F")
161 + p.set_draw_color(*INK)
162 + p.set_line_width(1.0)
163 + p.rect(10, 10, 190, 277)
164 + # kicker
165 + p.set_font("helvetica", "B", 10)
166 + p.set_text_color(*GREEN)
167 + p.set_xy(24, 34)
168 + p.cell(0, 6, "GROUPE KA · RAPPORT STATISTIQUE")
169 + # wordmark : partie gauche + boîte encre/accent
170 + wm = self.site.get("wordmark", "")
171 + left, boxed = (wm.split("·") + [None])[:2] if "·" in wm else (wm, None)
172 + p.set_xy(24, 70)
173 + p.set_font("helvetica", "B", 40)
174 + p.set_text_color(*INK)
175 + p.cell(p.get_string_width(left) + 2, 20, left)
176 + if boxed:
177 + bw = p.get_string_width(boxed) + 12
178 + x = p.get_x() + 2
179 + p.set_fill_color(*INK)
180 + p.rect(x, 68, bw, 22, style="F", round_corners=True, corner_radius=3)
181 + p.set_text_color(*self.accent)
182 + p.set_xy(x + 6, 70)
183 + p.cell(bw - 12, 18, boxed)
184 + p.set_xy(24, 100)
185 + p.set_font("helvetica", "", 13)
186 + p.set_text_color(*INK2)
187 + p.multi_cell(150, 7, f"Rapport statistique — {wm}")
188 + now = datetime.now(ZoneInfo("America/Toronto"))
189 + per = self.d.get("period", {}) or {}
190 + p.set_xy(24, 125)
191 + p.set_font("helvetica", "", 10.5)
192 + rows = [
193 + ("Période couverte", self.period_label + (f" ({per.get('from')} → {per.get('to')})" if per.get("from") else "")),
194 + ("Généré le", now.strftime("%Y-%m-%d à %H:%M") + " (heure de l'Est)"),
195 + ("Plateforme", "https://" + self.site.get("domain", "")),
196 + ("Mode", "Rapport complet" if self.mode == "complet" else "Synthèse"),
197 + ]
198 + y = 128
199 + for k, v in rows:
200 + p.set_xy(24, y)
201 + p.set_text_color(*INK3)
202 + p.cell(40, 6, k)
203 + p.set_text_color(*INK)
204 + p.set_font("helvetica", "B", 10.5)
205 + p.cell(0, 6, str(v))
206 + p.set_font("helvetica", "", 10.5)
207 + y += 8
208 + # bande encre au pied
209 + p.set_fill_color(*INK)
210 + p.rect(10, 262, 190, 25, style="F")
211 + p.set_xy(24, 270)
212 + p.set_font("helvetica", "B", 12)
213 + p.set_text_color(*WHITE)
214 + p.cell(60, 8, "par Groupe ")
215 + p.set_text_color(*self.accent)
216 + p.set_xy(24 + p.get_string_width("par Groupe ") + 1, 270)
217 + p.cell(20, 8, "KA")
218 + p.set_font("helvetica", "B", 10)
219 + p.set_xy(24, 270)
220 + p.set_text_color(*self.accent)
221 + p.cell(162, 8, "groupe-ka.com", align="R")
222 + p.set_auto_page_break(True, margin=22)
223 + p.cover_mode = False
224 +
225 + def _kpis(self):
226 + kpis = self.d.get("kpis") or []
227 + if not kpis:
228 + return
229 + self._section_title("Synthèse des indicateurs")
230 + p = self.pdf
231 + cols, gw, gh, gap = 3, 56, 26, 3
232 + x0, y = p.l_margin, p.get_y()
233 + for i, k in enumerate(kpis[:9]):
234 + x = x0 + (i % cols) * (gw + gap)
235 + if i and i % cols == 0:
236 + y += gh + gap
237 + if y > 250:
238 + p.add_page(); y = p.get_y()
239 + self._card(x, y, gw, gh)
240 + p.set_xy(x + 4, y + 4)
241 + p.set_font("helvetica", "B", 14)
242 + p.set_text_color(*INK)
243 + val = k.get("value")
244 + p.cell(gw - 8, 7, (_fr(val) if isinstance(val, (int, float)) else str(val)) + (" " + k["unit"] if k.get("unit") else ""))
245 + p.set_xy(x + 4, y + 12)
246 + p.set_font("helvetica", "", 7.6)
247 + p.set_text_color(*INK2)
248 + p.multi_cell(gw - 8, 3.6, str(k.get("label", ""))[:70])
249 + if k.get("delta_pct") is not None:
250 + up = (k.get("direction") or ("up" if k["delta_pct"] >= 0 else "down")) == "up"
251 + p.set_xy(x + 4, y + gh - 6.5)
252 + p.set_font("helvetica", "B", 8)
253 + p.set_text_color(*(GREEN if up else DANGER))
254 + arrow = "+" if k["delta_pct"] >= 0 else ""
255 + p.cell(gw - 8, 4, f"{'▲' if up else '▼'} {arrow}{str(k['delta_pct']).replace('.', ',')} % vs période préc.")
256 + p.set_y(y + gh + 8)
257 +
258 + def _line_chart(self, s):
259 + p = self.pdf
260 + pts = s.get("points") or []
261 + if len(pts) < 2:
262 + return
263 + if p.get_y() > 200:
264 + p.add_page()
265 + p.set_font("helvetica", "B", 10)
266 + p.set_text_color(*INK)
267 + p.cell(0, 6, s.get("title", ""))
268 + p.ln(7)
269 + x0, y0, w, h = p.l_margin, p.get_y(), 174, 52
270 + self._card(x0, y0, w, h, fill=WHITE)
271 + cx, cy, cw, ch = x0 + 12, y0 + 6, w - 20, h - 16
272 + vals = [pt["v"] for pt in pts] + [c["v"] for c in (s.get("compare") or [])]
273 + vmax = max(vals) or 1
274 + vmin = min(0, min(vals))
275 + rng = (vmax - vmin) or 1
276 + # grille + graduations
277 + p.set_font("helvetica", "", 6.3)
278 + p.set_text_color(*INK3)
279 + p.set_draw_color(200, 200, 195)
280 + p.set_line_width(0.15)
281 + for g in range(5):
282 + gy = cy + ch - ch * g / 4
283 + p.line(cx, gy, cx + cw, gy)
284 + p.set_xy(x0 + 1, gy - 1.6)
285 + p.cell(10, 3, _fr(vmin + rng * g / 4), align="R")
286 +
287 + def draw(series, color, width, dash=None):
288 + n = len(series)
289 + p.set_draw_color(*color)
290 + p.set_line_width(width)
291 + if dash:
292 + p.set_dash_pattern(dash=1.2, gap=1.2)
293 + last = None
294 + for i, pt in enumerate(series):
295 + px = cx + cw * (i / (n - 1))
296 + py = cy + ch - ch * ((pt["v"] - vmin) / rng)
297 + if last:
298 + p.line(last[0], last[1], px, py)
299 + last = (px, py)
300 + p.set_dash_pattern()
301 +
302 + if s.get("compare"):
303 + draw(s["compare"], INK3, 0.35, dash=True)
304 + draw(pts, self.accent, 0.7)
305 + # libellés d'axe X (premier / milieu / dernier)
306 + p.set_text_color(*INK3)
307 + for frac, idx in ((0, 0), (0.5, len(pts) // 2), (1, -1)):
308 + p.set_xy(cx + cw * frac - 9, cy + ch + 1.5)
309 + p.cell(18, 3, str(pts[idx].get("t", ""))[:10], align="C")
310 + p.set_y(y0 + h + 4)
311 + if s.get("compare"):
312 + p.set_font("helvetica", "", 6.8)
313 + p.set_text_color(*INK3)
314 + p.cell(0, 4, "— période courante (accent) · ---- période comparée")
315 + p.ln(6)
316 + else:
317 + p.ln(2)
318 +
319 + def _bars(self, title, items, unit=""):
320 + p = self.pdf
321 + items = [it for it in (items or []) if isinstance(it.get("value"), (int, float))][:12]
322 + if not items:
323 + return
324 + need = 10 + len(items) * 7
325 + if p.get_y() + need > 265:
326 + p.add_page()
327 + p.set_font("helvetica", "B", 10)
328 + p.set_text_color(*INK)
329 + p.cell(0, 6, title)
330 + p.ln(8)
331 + vmax = max(it["value"] for it in items) or 1
332 + for it in items:
333 + y = p.get_y()
334 + p.set_font("helvetica", "", 7.6)
335 + p.set_text_color(*INK)
336 + p.set_x(p.l_margin)
337 + p.cell(46, 5, str(it["label"])[:34])
338 + bw = 96 * (it["value"] / vmax)
339 + p.set_fill_color(*self.accent)
340 + p.set_draw_color(*INK)
341 + p.set_line_width(0.25)
342 + p.rect(p.l_margin + 48, y + 0.7, max(bw, 0.8), 3.6, style="DF")
343 + p.set_xy(p.l_margin + 148, y)
344 + p.set_font("helvetica", "B", 7.6)
345 + p.cell(26, 5, _fr(it["value"]) + (" " + unit if unit else ""), align="R")
346 + p.ln(6.4)
347 + p.ln(3)
348 +
349 + def _donut(self, b):
350 + # anneau vectoriel simple (arcs) + légende
351 + p = self.pdf
352 + items = [it for it in (b.get("items") or []) if it.get("value")][:8]
353 + total = sum(it["value"] for it in items)
354 + if not items or not total:
355 + return
356 + if p.get_y() > 210:
357 + p.add_page()
358 + p.set_font("helvetica", "B", 10)
359 + p.set_text_color(*INK)
360 + p.cell(0, 6, b.get("title", ""))
361 + p.ln(8)
362 + cx, cy, r = p.l_margin + 26, p.get_y() + 24, 20
363 + shades = [1.0, 0.78, 0.58, 0.42, 0.30, 0.22, 0.15, 0.10]
364 + start = -90.0
365 + for i, it in enumerate(items):
366 + frac = it["value"] / total
367 + f = shades[i % len(shades)]
368 + col = tuple(int(PAPER[j] + (self.accent[j] - PAPER[j]) * f) for j in range(3))
369 + steps = max(2, int(72 * frac))
370 + p.set_fill_color(*col)
371 + p.set_draw_color(*col)
372 + for st in range(steps):
373 + a0 = math.radians(start + 360 * frac * st / steps)
374 + a1 = math.radians(start + 360 * frac * (st + 1) / steps)
375 + p.polygon(
376 + [(cx, cy),
377 + (cx + r * math.cos(a0), cy + r * math.sin(a0)),
378 + (cx + r * math.cos(a1), cy + r * math.sin(a1))],
379 + style="DF",
380 + )
381 + start += 360 * frac
382 + p.set_fill_color(*WHITE)
383 + p.set_draw_color(*INK)
384 + p.set_line_width(0.4)
385 + p.ellipse(cx - 11, cy - 11, 22, 22, style="DF")
386 + p.ellipse(cx - r, cy - r, 2 * r, 2 * r, style="D")
387 + # légende
388 + ly = cy - 22
389 + for i, it in enumerate(items):
390 + f = shades[i % len(shades)]
391 + col = tuple(int(PAPER[j] + (self.accent[j] - PAPER[j]) * f) for j in range(3))
392 + p.set_fill_color(*col)
393 + p.set_draw_color(*INK)
394 + p.rect(p.l_margin + 60, ly + 0.8, 4, 4, style="DF")
395 + p.set_xy(p.l_margin + 66, ly)
396 + p.set_font("helvetica", "", 7.6)
397 + p.set_text_color(*INK)
398 + pct = 100 * it["value"] / total
399 + p.cell(0, 5.6, f"{str(it['label'])[:40]} — {_fr(it['value'])} ({pct:.1f} %)".replace(".", ","))
400 + ly += 5.6
401 + p.set_y(max(cy + r, ly) + 6)
402 +
403 + def _table(self, t):
404 + p = self.pdf
405 + cols = t.get("columns") or []
406 + rows = t.get("rows") or []
407 + if not cols or not rows:
408 + return
409 + self._section_title(t.get("title", "Tableau"))
410 + w = 174 / len(cols)
411 + def head():
412 + p.set_font("helvetica", "B", 7.6)
413 + p.set_fill_color(*INK)
414 + p.set_text_color(*WHITE)
415 + for c in cols:
416 + p.cell(w, 6, " " + str(c)[:30], fill=True)
417 + p.ln(6)
418 + head()
419 + p.set_text_color(*INK)
420 + for i, row in enumerate(rows[:200]):
421 + if p.get_y() > 262:
422 + p.add_page()
423 + head()
424 + p.set_text_color(*INK)
425 + p.set_font("helvetica", "", 7.4)
426 + p.set_fill_color(*(SURFACE2 if i % 2 else WHITE))
427 + for cell in row:
428 + txt = _fr(cell) if isinstance(cell, (int, float)) else str(cell)
429 + p.cell(w, 5.4, " " + txt[:34], fill=True)
430 + p.ln(5.4)
431 + if len(rows) > 200:
432 + p.set_font("helvetica", "", 7)
433 + p.set_text_color(*INK3)
434 + p.cell(0, 5, f"… {len(rows) - 200} lignes supplémentaires non imprimées")
435 + p.ln(6)
436 +
437 + def _records(self):
438 + recs = self.d.get("records") or []
439 + if not recs:
440 + return
441 + self._section_title("Records & faits marquants")
442 + p = self.pdf
443 + for r in recs[:10]:
444 + if p.get_y() > 258:
445 + p.add_page()
446 + y = p.get_y()
447 + self._card(p.l_margin, y, 174, 11, fill=SURFACE2)
448 + p.set_xy(p.l_margin + 4, y + 2)
449 + p.set_font("helvetica", "", 8.6)
450 + p.set_text_color(*INK2)
451 + p.cell(96, 7, str(r.get("label", ""))[:70])
452 + p.set_font("helvetica", "B", 9)
453 + p.set_text_color(*INK)
454 + p.cell(52, 7, str(r.get("value", ""))[:36], align="R")
455 + p.set_font("helvetica", "", 7.6)
456 + p.set_text_color(*INK3)
457 + p.cell(20, 7, str(r.get("date", "") or ""), align="R")
458 + p.set_y(y + 13.5)
459 + p.ln(4)
460 +
461 + def _final_page(self):
462 + p = self.pdf
463 + p.add_page()
464 + self._kicker("Groupe KA · contact")
465 + p.set_font("helvetica", "B", 15)
466 + p.set_text_color(*INK)
467 + p.cell(0, 8, "Coordonnées du Groupe KA")
468 + p.ln(12)
469 + for email, role in EMAILS:
470 + p.set_font("helvetica", "B", 10.5)
471 + p.set_text_color(*INK)
472 + p.cell(0, 6, email)
473 + p.ln(5.5)
474 + p.set_font("helvetica", "", 8.6)
475 + p.set_text_color(*INK3)
476 + p.cell(0, 5, role)
477 + p.ln(8)
478 + p.ln(2)
479 + p.set_font("helvetica", "B", 10)
480 + p.set_text_color(*GREEN)
481 + p.cell(0, 6, "groupe-ka.com — le portail de l'écosystème ·Ka")
482 + p.ln(10)
483 + p.set_draw_color(*self.accent)
484 + p.set_line_width(0.8)
485 + p.line(p.l_margin, p.get_y(), p.l_margin + 30, p.get_y())
486 + p.ln(4)
487 + p.set_font("helvetica", "", 8.6)
488 + p.set_text_color(*INK2)
489 + p.multi_cell(160, 4.6, DISCLAIMER)
490 + p.ln(4)
491 + p.set_font("helvetica", "", 7.6)
492 + p.set_text_color(*INK3)
493 + p.multi_cell(
494 + 160, 4.2,
495 + "Mentions : rapport généré automatiquement à partir des données réelles de la "
496 + "plateforme au moment indiqué en couverture. Conditions d'utilisation, politique "
497 + "de confidentialité et protection des renseignements personnels (Loi 25) : "
498 + "groupe-ka.com/conditions · /confidentialite · /loi-25.",
499 + )
500 +
501 + def _toc_page(self):
502 + # insérée après coup ? fpdf ne réordonne pas : on écrit le sommaire en
503 + # page 2 en réservant la page lors du build (voir build()).
504 + pass
505 +
506 + def build(self) -> bytes:
507 + p = self.pdf
508 + p.alias_nb_pages()
509 + self._cover()
510 + if self.mode == "synthese":
511 + p.add_page()
512 + self._kpis()
513 + self._records()
514 + self._final_page()
515 + else:
516 + p.add_page()
517 + toc_page_no = p.page_no()
518 + p.add_page()
519 + self._kpis()
520 + for s in self.d.get("series") or []:
521 + if s.get("kind") == "bar":
522 + self._bars(s.get("title", ""), [{"label": pt.get("t"), "value": pt.get("v")} for pt in (s.get("points") or [])], s.get("unit", ""))
523 + else:
524 + self._line_chart(s)
525 + for b in self.d.get("breakdowns") or []:
526 + if b.get("kind") == "donut":
527 + self._donut(b)
528 + else:
529 + self._bars(b.get("title", ""), b.get("items"))
530 + geo = self.d.get("geo")
531 + if geo:
532 + self._bars(geo.get("title", "Répartition géographique"), geo.get("items"))
533 + for t in self.d.get("tables") or []:
534 + self._table(t)
535 + self._records()
536 + self._final_page()
537 + # sommaire écrit sur la page réservée (page 2)
538 + last_page = p.page
539 + p.page = toc_page_no
540 + p.set_y(22)
541 + p.set_font("helvetica", "B", 15)
542 + p.set_text_color(*INK)
543 + p.cell(0, 8, "Sommaire")
544 + p.ln(12)
545 + p.set_font("helvetica", "", 9.5)
546 + for title, page_no in self.toc:
547 + p.set_text_color(*INK)
548 + p.cell(140, 6.5, title[:80])
549 + p.set_text_color(*INK3)
550 + p.cell(0, 6.5, str(page_no), align="R")
551 + p.ln(6.5)
552 + p.page = last_page
553 + return bytes(p.output())
554 +
555 +
556 +def filename(platform_id: str, period: str) -> str:
557 + today = datetime.now(ZoneInfo("America/Toronto")).strftime("%Y-%m-%d")
558 + return f"groupe-ka_{platform_id}_stats_{period}_{today}.pdf"
added infrastructure/migrations/005_stats_indexes.sql +31 −0
@@ -0,0 +1,31 @@
1 +-- Trouve-KA — migration 005 : index pour la page /stats
2 +-- Author: Simon-Pierre Boucher
3 +-- Contact: contact@spboucher.ai
4 +--
5 +-- Le tableau de bord /api/stats/dashboard agrège par plage temporelle :
6 +-- 1) documents.first_indexed_at n'était pas indexé (séries « indexées/jour »,
7 +-- cumul, heatmap) — sur 1 M+ lignes, chaque refresh du cache scannait tout.
8 +-- 2) crawl_attempts : index partiel sur les erreurs (séries d'erreurs,
9 +-- erreurs par type, dernières erreurs) — les erreurs sont une petite
10 +-- fraction des ~5 M tentatives.
11 +-- 3) search_queries.created_at : bornage par période + purge 90 jours.
12 +--
13 +-- NOTE : sur la base de prod ces index ont été créés CONCURRENTLY (hors
14 +-- transaction) pour ne pas bloquer les crawlers; ce fichier reste idempotent
15 +-- pour les installations neuves.
16 +
17 +BEGIN;
18 +
19 +CREATE INDEX IF NOT EXISTS idx_documents_first_indexed
20 + ON documents (first_indexed_at);
21 +
22 +CREATE INDEX IF NOT EXISTS idx_attempts_errors_time
23 + ON crawl_attempts (fetched_at DESC)
24 + WHERE outcome = 'error';
25 +
26 +CREATE INDEX IF NOT EXISTS idx_queries_time
27 + ON search_queries (created_at);
28 +
29 +INSERT INTO schema_migrations (version) VALUES (5) ON CONFLICT DO NOTHING;
30 +
31 +COMMIT;
modified pyproject.toml +1 −0
@@ -22,6 +22,7 @@ dependencies = [
22 22 "pydantic-settings>=2.3",
23 23 "langdetect>=1.0.9",
24 24 "python-dateutil>=2.9",
25 + "fpdf2>=2.7",
25 26 ]
26 27
27 28 [project.optional-dependencies]
28 29