SPB Git
spboucher.ai

spb/api-ka

Public

API-KA — plateforme centrale : collecte quotidienne des 8 services KA, historisation append-only et API publique sur www.api-ka.com

fastapipostgresqlagregationka

Page /stats : tableau de bord analytique de la plateforme + rapport PDF Groupe-KA

- Journalisation légère des requêtes API : table api_requests (ts, méthode,
  endpoint normalisé, statut, durée) alimentée par le middleware de logging
  via un tampon mémoire non bloquant + tâche de fond (flush 5 s, purge 90 j).
- Backfill : scripts/backfill_api_requests.py réimporte l historique réel des
  logs JSON (811 requêtes depuis le 2026-08-16), idempotent.
- GET /api/stats/dashboard : contrat commun Groupe KA (SPEC.md) — KPI avec
  deltas, séries jour/heure (appels, latence, enregistrements collectés),
  top endpoints, anneau par service, heatmap, tableaux, records ; cache 5 min.
- GET /api/stats/report : PDF Groupe-KA (moteur kapdf/fpdf2, wordmark API·Ka,
  accent #3b5bdb), modes complet et synthèse ; correctif kit : pied de page
  qui fuyait sur la couverture.
- Page /stats (vanilla JS + SVG, langage visuel du kit kacharts) : chips de
  période + plage personnalisée, KPI, courbes avec infobulle et légende
  cliquable, barres, anneau, heatmap, tableaux triables (recherche +
  pagination), records, boutons PDF, états vides propres, mobile 360-1440.
- Lien « Stats » dans la nav de index.html et contact.html ; fpdf2 ajouté aux
  requirements ; tests /api/stats (32 tests verts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Aug 18, 2026) parent 9672813 Browse files

15 changed files +3,229 −4

15 changed files
Load all 15 files Unified Split
modified requirements.txt +1 −0
@@ -15,6 +15,7 @@ alembic>=1.13
15 apscheduler>=3.10 15 apscheduler>=3.10
16 httpx>=0.27 16 httpx>=0.27
17 python-dotenv>=1.0 17 python-dotenv>=1.0
18+fpdf2>=2.8
18 19
19 # Développement / tests 20 # Développement / tests
20 pytest>=8.0 21 pytest>=8.0
added scripts/backfill_api_requests.py +93 −0
@@ -0,0 +1,93 @@
1+# ============================================
2+# Projet : API-KA
3+# Fichier : scripts/backfill_api_requests.py
4+# Node : m3u96b
5+# Author : Simon-Pierre Boucher
6+# Contact : contact@spboucher.ai
7+# Date : 2026-08-17
8+# ============================================
9+"""Rattrapage : importe l'historique des requêtes HTTP déjà journalisées dans
10+``logs/apika.log*`` (lignes JSON ``http_request`` du middleware) vers la table
11+``api_requests`` qui alimente la page /stats.
12+
13+Idempotent : seules les lignes STRICTEMENT antérieures au plus ancien
14+enregistrement déjà présent dans ``api_requests`` sont insérées (à la première
15+exécution, tout l'historique des logs est repris ; aux suivantes, rien).
16+
17+Usage : venv/bin/python -m scripts.backfill_api_requests
18+"""
19+
20+from __future__ import annotations
21+
22+import datetime
23+import json
24+
25+from sqlalchemy import func, select
26+
27+from src.api.reqstats import normalize_endpoint
28+from src.config import get_settings
29+from src.database.db import init_db, session_scope
30+from src.database.models import ApiRequest
31+from src.utils.logger import get_logger
32+
33+
34+def main() -> None:
35+ logger = get_logger("apika.backfill_requests")
36+ init_db()
37+ settings = get_settings()
38+
39+ with session_scope() as session:
40+ cutoff = session.execute(select(func.min(ApiRequest.ts))).scalar()
41+ if cutoff is not None and cutoff.tzinfo is None:
42+ cutoff = cutoff.replace(tzinfo=datetime.UTC)
43+ if cutoff is None:
44+ cutoff = datetime.datetime.now(tz=datetime.UTC)
45+
46+ entries: list[dict] = []
47+ parsed = skipped = 0
48+ for log_file in sorted(settings.logs_dir.glob("apika.log*")):
49+ with open(log_file, encoding="utf-8") as fh:
50+ for line in fh:
51+ line = line.strip()
52+ if not line or '"http_request"' not in line:
53+ continue
54+ try:
55+ rec = json.loads(line)
56+ except json.JSONDecodeError:
57+ continue
58+ if rec.get("message") != "http_request":
59+ continue
60+ parsed += 1
61+ try:
62+ ts = datetime.datetime.fromisoformat(rec["timestamp"])
63+ except (KeyError, ValueError):
64+ skipped += 1
65+ continue
66+ if ts.tzinfo is None:
67+ ts = ts.replace(tzinfo=datetime.UTC)
68+ if ts >= cutoff:
69+ skipped += 1
70+ continue
71+ entries.append(
72+ {
73+ "ts": ts,
74+ "method": rec.get("method", "GET"),
75+ "endpoint": normalize_endpoint(rec.get("path", "(autre)")),
76+ "status": int(rec.get("status_code", 0)),
77+ "duration_ms": round(float(rec.get("duration_ms", 0.0)), 2),
78+ }
79+ )
80+
81+ if entries:
82+ with session_scope() as session:
83+ session.bulk_insert_mappings(ApiRequest.__mapper__, entries)
84+
85+ logger.info(
86+ "Backfill api_requests terminé",
87+ extra={"inserted": len(entries), "parsed": parsed, "skipped": skipped},
88+ )
89+ print(f"Backfill : {len(entries)} requêtes insérées ({parsed} lues, {skipped} ignorées)")
90+
91+
92+if __name__ == "__main__":
93+ main()
added src/api/kapdf.py +568 −0
@@ -0,0 +1,568 @@
1+# ============================================
2+# Projet : API-KA
3+# Fichier : src/api/kapdf.py
4+# Node : m3u96b
5+# Author : Simon-Pierre Boucher
6+# Contact : contact@spboucher.ai
7+# Date : 2026-08-17
8+# ============================================
9+# Auteur : Simon-Pierre Boucher — contact@spboucher.ai
10+# ka-ui/stats/kapdf.py — moteur PDF commun Groupe KA (fpdf2).
11+# Consomme le JSON du contrat /api/stats/dashboard (voir SPEC.md) et produit
12+# le rapport estampillé Groupe-KA : couverture, sommaire, KPI, graphiques
13+# VECTORIELS (courbes/barres/anneaux), tableaux paginés, records, page de fin.
14+# Usage :
15+# from kapdf import GroupeKAReport
16+# pdf_bytes = GroupeKAReport(site={"wordmark":"Lou·Ka","accent":"#ff6a00",
17+# "domain":"www.lou-ka.com","tagline":"…"}, dashboard=dash_json,
18+# mode="complet").build()
19+# Dépendance : pip install fpdf2 (aucune autre)
20+from __future__ import annotations
21+
22+import math
23+from datetime import datetime
24+from zoneinfo import ZoneInfo
25+
26+from fpdf import FPDF
27+
28+INK = (20, 24, 20)
29+INK2 = (77, 85, 81)
30+INK3 = (139, 146, 140)
31+PAPER = (245, 243, 238)
32+SURFACE2 = (250, 249, 245)
33+GREEN = (28, 92, 65)
34+DANGER = (179, 66, 58)
35+WHITE = (255, 255, 255)
36+
37+EMAILS = [
38+ ("contact@groupe-ka.com", "Projets, partenariats & données"),
39+ ("info@groupe-ka.com", "Médias & questions générales"),
40+ ("admin@groupe-ka.com", "Légal, vie privée & Loi 25"),
41+]
42+DISCLAIMER = (
43+ "Groupe KA est un agrégateur de contenu : nous ne vendons rien, ne louons "
44+ "rien et ne sommes partie à aucune transaction. Données lues à la source, "
45+ "rien d'inventé, tout est traçable."
46+)
47+
48+
49+def _hex(c: str) -> tuple[int, int, int]:
50+ c = c.lstrip("#")
51+ return tuple(int(c[i : i + 2], 16) for i in (0, 2, 4)) # type: ignore
52+
53+
54+def _fr(n) -> str:
55+ if isinstance(n, float) and not n.is_integer():
56+ return f"{n:,.2f}".replace(",", " ").replace(".", ",")
57+ return f"{int(n):,}".replace(",", " ")
58+
59+
60+_SUBST = {
61+ "—": "-", "–": "-", "→": "->", "▲": "+", "▼": "-",
62+ "…": "...", "’": "'", "‘": "'", "“": '"', "”": '"',
63+ "œ": "oe", "Œ": "OE", "−": "-", " ": " ", " ": " ",
64+}
65+
66+
67+def _latin1(s: str) -> str:
68+ for k, v in _SUBST.items():
69+ s = s.replace(k, v)
70+ return s.encode("latin-1", "replace").decode("latin-1")
71+
72+
73+class _PDF(FPDF):
74+ """FPDF avec en-tête/pied Groupe-KA sur chaque page (sauf couverture).
75+ Les polices core sont latin-1 : normalize_text sanitise en amont."""
76+
77+ def normalize_text(self, text):
78+ return super().normalize_text(_latin1(text))
79+
80+ def __init__(self, brand: str, accent: tuple, period_label: str):
81+ super().__init__(orientation="P", unit="mm", format="A4")
82+ self.brand = brand
83+ self.accent = accent
84+ self.period_label = period_label
85+ self.cover_mode = False
86+ self.set_margins(18, 20, 18)
87+ self.set_auto_page_break(True, margin=22)
88+
89+ def header(self):
90+ if self.cover_mode:
91+ return
92+ self.set_font("helvetica", "B", 8.5)
93+ self.set_text_color(*INK)
94+ self.set_xy(18, 9)
95+ self.cell(0, 5, f"Groupe KA · {self.brand}")
96+ self.set_font("helvetica", "", 8)
97+ self.set_text_color(*INK3)
98+ self.set_xy(18, 9)
99+ self.cell(0, 5, "Rapport statistique", align="R")
100+ self.set_draw_color(*INK)
101+ self.set_line_width(0.5)
102+ self.line(18, 15.5, 192, 15.5)
103+ self.set_y(20)
104+
105+ def footer(self):
106+ # La page 1 est toujours la couverture : fpdf dessine son pied de page
107+ # au add_page() suivant, quand cover_mode est deja retombe a False.
108+ if self.cover_mode or self.page_no() == 1:
109+ return
110+ self.set_y(-15)
111+ self.set_draw_color(*INK3)
112+ self.set_line_width(0.2)
113+ self.line(18, self.get_y() - 1.5, 192, self.get_y() - 1.5)
114+ self.set_font("helvetica", "", 7.5)
115+ self.set_text_color(*INK3)
116+ year = datetime.now(ZoneInfo("America/Toronto")).year
117+ self.cell(130, 5, f"© Groupe-KA — {year} — groupe-ka.com · {self.period_label}")
118+ self.cell(0, 5, f"p. {self.page_no()}/{{nb}}", align="R")
119+
120+
121+class GroupeKAReport:
122+ def __init__(self, site: dict, dashboard: dict, mode: str = "complet"):
123+ self.site = site
124+ self.d = dashboard
125+ self.mode = mode
126+ self.accent = _hex(site.get("accent", "#d9f26b"))
127+ period = dashboard.get("period", {}) or {}
128+ self.period_label = period.get("label") or "toute la période"
129+ self.pdf = _PDF(site.get("wordmark", ""), self.accent, self.period_label)
130+ self.toc: list[tuple[str, int]] = []
131+
132+ # ---------- primitives ----------
133+ def _card(self, x, y, w, h, fill=WHITE):
134+ p = self.pdf
135+ p.set_draw_color(*INK)
136+ p.set_line_width(0.45)
137+ p.set_fill_color(*fill)
138+ p.rect(x, y, w, h, style="DF", round_corners=True, corner_radius=2.2)
139+
140+ def _kicker(self, text):
141+ p = self.pdf
142+ p.set_font("helvetica", "B", 8)
143+ p.set_text_color(*GREEN)
144+ p.set_draw_color(*GREEN)
145+ p.set_line_width(0.6)
146+ y = p.get_y() + 2
147+ p.line(p.l_margin, y, p.l_margin + 7, y)
148+ p.set_xy(p.l_margin + 9, y - 2.5)
149+ p.cell(0, 5, text.upper())
150+ p.ln(8)
151+
152+ def _section_title(self, title):
153+ if self.pdf.get_y() > 240:
154+ self.pdf.add_page()
155+ self._kicker("Groupe KA · " + self.site.get("wordmark", ""))
156+ self.pdf.set_font("helvetica", "B", 15)
157+ self.pdf.set_text_color(*INK)
158+ self.pdf.set_x(self.pdf.l_margin)
159+ self.pdf.cell(0, 8, title)
160+ self.toc.append((title, self.pdf.page_no()))
161+ self.pdf.ln(11)
162+
163+ # ---------- pages ----------
164+ def _cover(self):
165+ p = self.pdf
166+ p.cover_mode = True
167+ p.set_auto_page_break(False)
168+ p.add_page()
169+ p.set_fill_color(*PAPER)
170+ p.rect(0, 0, 210, 297, style="F")
171+ p.set_draw_color(*INK)
172+ p.set_line_width(1.0)
173+ p.rect(10, 10, 190, 277)
174+ # kicker
175+ p.set_font("helvetica", "B", 10)
176+ p.set_text_color(*GREEN)
177+ p.set_xy(24, 34)
178+ p.cell(0, 6, "GROUPE KA · RAPPORT STATISTIQUE")
179+ # wordmark : partie gauche + boîte encre/accent
180+ wm = self.site.get("wordmark", "")
181+ left, boxed = (wm.split("·") + [None])[:2] if "·" in wm else (wm, None)
182+ p.set_xy(24, 70)
183+ p.set_font("helvetica", "B", 40)
184+ p.set_text_color(*INK)
185+ p.cell(p.get_string_width(left) + 2, 20, left)
186+ if boxed:
187+ bw = p.get_string_width(boxed) + 12
188+ x = p.get_x() + 2
189+ p.set_fill_color(*INK)
190+ p.rect(x, 68, bw, 22, style="F", round_corners=True, corner_radius=3)
191+ p.set_text_color(*self.accent)
192+ p.set_xy(x + 6, 70)
193+ p.cell(bw - 12, 18, boxed)
194+ p.set_xy(24, 100)
195+ p.set_font("helvetica", "", 13)
196+ p.set_text_color(*INK2)
197+ p.multi_cell(150, 7, f"Rapport statistique — {wm}")
198+ now = datetime.now(ZoneInfo("America/Toronto"))
199+ per = self.d.get("period", {}) or {}
200+ p.set_xy(24, 125)
201+ p.set_font("helvetica", "", 10.5)
202+ rows = [
203+ ("Période couverte", self.period_label + (f" ({per.get('from')} → {per.get('to')})" if per.get("from") else "")),
204+ ("Généré le", now.strftime("%Y-%m-%d à %H:%M") + " (heure de l'Est)"),
205+ ("Plateforme", "https://" + self.site.get("domain", "")),
206+ ("Mode", "Rapport complet" if self.mode == "complet" else "Synthèse"),
207+ ]
208+ y = 128
209+ for k, v in rows:
210+ p.set_xy(24, y)
211+ p.set_text_color(*INK3)
212+ p.cell(40, 6, k)
213+ p.set_text_color(*INK)
214+ p.set_font("helvetica", "B", 10.5)
215+ p.cell(0, 6, str(v))
216+ p.set_font("helvetica", "", 10.5)
217+ y += 8
218+ # bande encre au pied
219+ p.set_fill_color(*INK)
220+ p.rect(10, 262, 190, 25, style="F")
221+ p.set_xy(24, 270)
222+ p.set_font("helvetica", "B", 12)
223+ p.set_text_color(*WHITE)
224+ p.cell(60, 8, "par Groupe ")
225+ p.set_text_color(*self.accent)
226+ p.set_xy(24 + p.get_string_width("par Groupe ") + 1, 270)
227+ p.cell(20, 8, "KA")
228+ p.set_font("helvetica", "B", 10)
229+ p.set_xy(24, 270)
230+ p.set_text_color(*self.accent)
231+ p.cell(162, 8, "groupe-ka.com", align="R")
232+ p.set_auto_page_break(True, margin=22)
233+ p.cover_mode = False
234+
235+ def _kpis(self):
236+ kpis = self.d.get("kpis") or []
237+ if not kpis:
238+ return
239+ self._section_title("Synthèse des indicateurs")
240+ p = self.pdf
241+ cols, gw, gh, gap = 3, 56, 26, 3
242+ x0, y = p.l_margin, p.get_y()
243+ for i, k in enumerate(kpis[:9]):
244+ x = x0 + (i % cols) * (gw + gap)
245+ if i and i % cols == 0:
246+ y += gh + gap
247+ if y > 250:
248+ p.add_page(); y = p.get_y()
249+ self._card(x, y, gw, gh)
250+ p.set_xy(x + 4, y + 4)
251+ p.set_font("helvetica", "B", 14)
252+ p.set_text_color(*INK)
253+ val = k.get("value")
254+ p.cell(gw - 8, 7, (_fr(val) if isinstance(val, (int, float)) else str(val)) + (" " + k["unit"] if k.get("unit") else ""))
255+ p.set_xy(x + 4, y + 12)
256+ p.set_font("helvetica", "", 7.6)
257+ p.set_text_color(*INK2)
258+ p.multi_cell(gw - 8, 3.6, str(k.get("label", ""))[:70])
259+ if k.get("delta_pct") is not None:
260+ up = (k.get("direction") or ("up" if k["delta_pct"] >= 0 else "down")) == "up"
261+ p.set_xy(x + 4, y + gh - 6.5)
262+ p.set_font("helvetica", "B", 8)
263+ p.set_text_color(*(GREEN if up else DANGER))
264+ arrow = "+" if k["delta_pct"] >= 0 else ""
265+ p.cell(gw - 8, 4, f"{'▲' if up else '▼'} {arrow}{str(k['delta_pct']).replace('.', ',')} % vs période préc.")
266+ p.set_y(y + gh + 8)
267+
268+ def _line_chart(self, s):
269+ p = self.pdf
270+ pts = s.get("points") or []
271+ if len(pts) < 2:
272+ return
273+ if p.get_y() > 200:
274+ p.add_page()
275+ p.set_font("helvetica", "B", 10)
276+ p.set_text_color(*INK)
277+ p.cell(0, 6, s.get("title", ""))
278+ p.ln(7)
279+ x0, y0, w, h = p.l_margin, p.get_y(), 174, 52
280+ self._card(x0, y0, w, h, fill=WHITE)
281+ cx, cy, cw, ch = x0 + 12, y0 + 6, w - 20, h - 16
282+ vals = [pt["v"] for pt in pts] + [c["v"] for c in (s.get("compare") or [])]
283+ vmax = max(vals) or 1
284+ vmin = min(0, min(vals))
285+ rng = (vmax - vmin) or 1
286+ # grille + graduations
287+ p.set_font("helvetica", "", 6.3)
288+ p.set_text_color(*INK3)
289+ p.set_draw_color(200, 200, 195)
290+ p.set_line_width(0.15)
291+ for g in range(5):
292+ gy = cy + ch - ch * g / 4
293+ p.line(cx, gy, cx + cw, gy)
294+ p.set_xy(x0 + 1, gy - 1.6)
295+ p.cell(10, 3, _fr(vmin + rng * g / 4), align="R")
296+
297+ def draw(series, color, width, dash=None):
298+ n = len(series)
299+ p.set_draw_color(*color)
300+ p.set_line_width(width)
301+ if dash:
302+ p.set_dash_pattern(dash=1.2, gap=1.2)
303+ last = None
304+ for i, pt in enumerate(series):
305+ px = cx + cw * (i / (n - 1))
306+ py = cy + ch - ch * ((pt["v"] - vmin) / rng)
307+ if last:
308+ p.line(last[0], last[1], px, py)
309+ last = (px, py)
310+ p.set_dash_pattern()
311+
312+ if s.get("compare"):
313+ draw(s["compare"], INK3, 0.35, dash=True)
314+ draw(pts, self.accent, 0.7)
315+ # libellés d'axe X (premier / milieu / dernier)
316+ p.set_text_color(*INK3)
317+ for frac, idx in ((0, 0), (0.5, len(pts) // 2), (1, -1)):
318+ p.set_xy(cx + cw * frac - 9, cy + ch + 1.5)
319+ p.cell(18, 3, str(pts[idx].get("t", ""))[:10], align="C")
320+ p.set_y(y0 + h + 4)
321+ if s.get("compare"):
322+ p.set_font("helvetica", "", 6.8)
323+ p.set_text_color(*INK3)
324+ p.cell(0, 4, "— période courante (accent) · ---- période comparée")
325+ p.ln(6)
326+ else:
327+ p.ln(2)
328+
329+ def _bars(self, title, items, unit=""):
330+ p = self.pdf
331+ items = [it for it in (items or []) if isinstance(it.get("value"), (int, float))][:12]
332+ if not items:
333+ return
334+ need = 10 + len(items) * 7
335+ if p.get_y() + need > 265:
336+ p.add_page()
337+ p.set_font("helvetica", "B", 10)
338+ p.set_text_color(*INK)
339+ p.cell(0, 6, title)
340+ p.ln(8)
341+ vmax = max(it["value"] for it in items) or 1
342+ for it in items:
343+ y = p.get_y()
344+ p.set_font("helvetica", "", 7.6)
345+ p.set_text_color(*INK)
346+ p.set_x(p.l_margin)
347+ p.cell(46, 5, str(it["label"])[:34])
348+ bw = 96 * (it["value"] / vmax)
349+ p.set_fill_color(*self.accent)
350+ p.set_draw_color(*INK)
351+ p.set_line_width(0.25)
352+ p.rect(p.l_margin + 48, y + 0.7, max(bw, 0.8), 3.6, style="DF")
353+ p.set_xy(p.l_margin + 148, y)
354+ p.set_font("helvetica", "B", 7.6)
355+ p.cell(26, 5, _fr(it["value"]) + (" " + unit if unit else ""), align="R")
356+ p.ln(6.4)
357+ p.ln(3)
358+
359+ def _donut(self, b):
360+ # anneau vectoriel simple (arcs) + légende
361+ p = self.pdf
362+ items = [it for it in (b.get("items") or []) if it.get("value")][:8]
363+ total = sum(it["value"] for it in items)
364+ if not items or not total:
365+ return
366+ if p.get_y() > 210:
367+ p.add_page()
368+ p.set_font("helvetica", "B", 10)
369+ p.set_text_color(*INK)
370+ p.cell(0, 6, b.get("title", ""))
371+ p.ln(8)
372+ cx, cy, r = p.l_margin + 26, p.get_y() + 24, 20
373+ shades = [1.0, 0.78, 0.58, 0.42, 0.30, 0.22, 0.15, 0.10]
374+ start = -90.0
375+ for i, it in enumerate(items):
376+ frac = it["value"] / total
377+ f = shades[i % len(shades)]
378+ col = tuple(int(PAPER[j] + (self.accent[j] - PAPER[j]) * f) for j in range(3))
379+ steps = max(2, int(72 * frac))
380+ p.set_fill_color(*col)
381+ p.set_draw_color(*col)
382+ for st in range(steps):
383+ a0 = math.radians(start + 360 * frac * st / steps)
384+ a1 = math.radians(start + 360 * frac * (st + 1) / steps)
385+ p.polygon(
386+ [(cx, cy),
387+ (cx + r * math.cos(a0), cy + r * math.sin(a0)),
388+ (cx + r * math.cos(a1), cy + r * math.sin(a1))],
389+ style="DF",
390+ )
391+ start += 360 * frac
392+ p.set_fill_color(*WHITE)
393+ p.set_draw_color(*INK)
394+ p.set_line_width(0.4)
395+ p.ellipse(cx - 11, cy - 11, 22, 22, style="DF")
396+ p.ellipse(cx - r, cy - r, 2 * r, 2 * r, style="D")
397+ # légende
398+ ly = cy - 22
399+ for i, it in enumerate(items):
400+ f = shades[i % len(shades)]
401+ col = tuple(int(PAPER[j] + (self.accent[j] - PAPER[j]) * f) for j in range(3))
402+ p.set_fill_color(*col)
403+ p.set_draw_color(*INK)
404+ p.rect(p.l_margin + 60, ly + 0.8, 4, 4, style="DF")
405+ p.set_xy(p.l_margin + 66, ly)
406+ p.set_font("helvetica", "", 7.6)
407+ p.set_text_color(*INK)
408+ pct = 100 * it["value"] / total
409+ p.cell(0, 5.6, f"{str(it['label'])[:40]} — {_fr(it['value'])} ({pct:.1f} %)".replace(".", ","))
410+ ly += 5.6
411+ p.set_y(max(cy + r, ly) + 6)
412+
413+ def _table(self, t):
414+ p = self.pdf
415+ cols = t.get("columns") or []
416+ rows = t.get("rows") or []
417+ if not cols or not rows:
418+ return
419+ self._section_title(t.get("title", "Tableau"))
420+ w = 174 / len(cols)
421+ def head():
422+ p.set_font("helvetica", "B", 7.6)
423+ p.set_fill_color(*INK)
424+ p.set_text_color(*WHITE)
425+ for c in cols:
426+ p.cell(w, 6, " " + str(c)[:30], fill=True)
427+ p.ln(6)
428+ head()
429+ p.set_text_color(*INK)
430+ for i, row in enumerate(rows[:200]):
431+ if p.get_y() > 262:
432+ p.add_page()
433+ head()
434+ p.set_text_color(*INK)
435+ p.set_font("helvetica", "", 7.4)
436+ p.set_fill_color(*(SURFACE2 if i % 2 else WHITE))
437+ for cell in row:
438+ txt = _fr(cell) if isinstance(cell, (int, float)) else str(cell)
439+ p.cell(w, 5.4, " " + txt[:34], fill=True)
440+ p.ln(5.4)
441+ if len(rows) > 200:
442+ p.set_font("helvetica", "", 7)
443+ p.set_text_color(*INK3)
444+ p.cell(0, 5, f"… {len(rows) - 200} lignes supplémentaires non imprimées")
445+ p.ln(6)
446+
447+ def _records(self):
448+ recs = self.d.get("records") or []
449+ if not recs:
450+ return
451+ self._section_title("Records & faits marquants")
452+ p = self.pdf
453+ for r in recs[:10]:
454+ if p.get_y() > 258:
455+ p.add_page()
456+ y = p.get_y()
457+ self._card(p.l_margin, y, 174, 11, fill=SURFACE2)
458+ p.set_xy(p.l_margin + 4, y + 2)
459+ p.set_font("helvetica", "", 8.6)
460+ p.set_text_color(*INK2)
461+ p.cell(96, 7, str(r.get("label", ""))[:70])
462+ p.set_font("helvetica", "B", 9)
463+ p.set_text_color(*INK)
464+ p.cell(52, 7, str(r.get("value", ""))[:36], align="R")
465+ p.set_font("helvetica", "", 7.6)
466+ p.set_text_color(*INK3)
467+ p.cell(20, 7, str(r.get("date", "") or ""), align="R")
468+ p.set_y(y + 13.5)
469+ p.ln(4)
470+
471+ def _final_page(self):
472+ p = self.pdf
473+ p.add_page()
474+ self._kicker("Groupe KA · contact")
475+ p.set_font("helvetica", "B", 15)
476+ p.set_text_color(*INK)
477+ p.cell(0, 8, "Coordonnées du Groupe KA")
478+ p.ln(12)
479+ for email, role in EMAILS:
480+ p.set_font("helvetica", "B", 10.5)
481+ p.set_text_color(*INK)
482+ p.cell(0, 6, email)
483+ p.ln(5.5)
484+ p.set_font("helvetica", "", 8.6)
485+ p.set_text_color(*INK3)
486+ p.cell(0, 5, role)
487+ p.ln(8)
488+ p.ln(2)
489+ p.set_font("helvetica", "B", 10)
490+ p.set_text_color(*GREEN)
491+ p.cell(0, 6, "groupe-ka.com — le portail de l'écosystème ·Ka")
492+ p.ln(10)
493+ p.set_draw_color(*self.accent)
494+ p.set_line_width(0.8)
495+ p.line(p.l_margin, p.get_y(), p.l_margin + 30, p.get_y())
496+ p.ln(4)
497+ p.set_font("helvetica", "", 8.6)
498+ p.set_text_color(*INK2)
499+ p.multi_cell(160, 4.6, DISCLAIMER)
500+ p.ln(4)
501+ p.set_font("helvetica", "", 7.6)
502+ p.set_text_color(*INK3)
503+ p.multi_cell(
504+ 160, 4.2,
505+ "Mentions : rapport généré automatiquement à partir des données réelles de la "
506+ "plateforme au moment indiqué en couverture. Conditions d'utilisation, politique "
507+ "de confidentialité et protection des renseignements personnels (Loi 25) : "
508+ "groupe-ka.com/conditions · /confidentialite · /loi-25.",
509+ )
510+
511+ def _toc_page(self):
512+ # insérée après coup ? fpdf ne réordonne pas : on écrit le sommaire en
513+ # page 2 en réservant la page lors du build (voir build()).
514+ pass
515+
516+ def build(self) -> bytes:
517+ p = self.pdf
518+ p.alias_nb_pages()
519+ self._cover()
520+ if self.mode == "synthese":
521+ p.add_page()
522+ self._kpis()
523+ self._records()
524+ self._final_page()
525+ else:
526+ p.add_page()
527+ toc_page_no = p.page_no()
528+ p.add_page()
529+ self._kpis()
530+ for s in self.d.get("series") or []:
531+ if s.get("kind") == "bar":
532+ self._bars(s.get("title", ""), [{"label": pt.get("t"), "value": pt.get("v")} for pt in (s.get("points") or [])], s.get("unit", ""))
533+ else:
534+ self._line_chart(s)
535+ for b in self.d.get("breakdowns") or []:
536+ if b.get("kind") == "donut":
537+ self._donut(b)
538+ else:
539+ self._bars(b.get("title", ""), b.get("items"))
540+ geo = self.d.get("geo")
541+ if geo:
542+ self._bars(geo.get("title", "Répartition géographique"), geo.get("items"))
543+ for t in self.d.get("tables") or []:
544+ self._table(t)
545+ self._records()
546+ self._final_page()
547+ # sommaire écrit sur la page réservée (page 2)
548+ last_page = p.page
549+ p.page = toc_page_no
550+ p.set_y(22)
551+ p.set_font("helvetica", "B", 15)
552+ p.set_text_color(*INK)
553+ p.cell(0, 8, "Sommaire")
554+ p.ln(12)
555+ p.set_font("helvetica", "", 9.5)
556+ for title, page_no in self.toc:
557+ p.set_text_color(*INK)
558+ p.cell(140, 6.5, title[:80])
559+ p.set_text_color(*INK3)
560+ p.cell(0, 6.5, str(page_no), align="R")
561+ p.ln(6.5)
562+ p.page = last_page
563+ return bytes(p.output())
564+
565+
566+def filename(platform_id: str, period: str) -> str:
567+ today = datetime.now(ZoneInfo("America/Toronto")).strftime("%Y-%m-%d")
568+ return f"groupe-ka_{platform_id}_stats_{period}_{today}.pdf"
modified src/api/main.py +25 −4
@@ -11,9 +11,10 @@ exposée publiquement uniquement via le tunnel ngrok www.api-ka.com."""
11 11
12 from __future__ import annotations 12 from __future__ import annotations
13 13
14+import asyncio
14 import json 15 import json
15 from collections.abc import AsyncIterator 16 from collections.abc import AsyncIterator
16−from contextlib import asynccontextmanager 17+from contextlib import asynccontextmanager, suppress
17 from pathlib import Path 18 from pathlib import Path
18 from typing import Any 19 from typing import Any
19 20
@@ -21,9 +22,10 @@ from fastapi import FastAPI, Request
21 from fastapi.responses import FileResponse 22 from fastapi.responses import FileResponse
22 from fastapi.staticfiles import StaticFiles 23 from fastapi.staticfiles import StaticFiles
23 24
25+from src.api import reqstats
24 from src.api.middleware.logging import RequestLoggingMiddleware 26 from src.api.middleware.logging import RequestLoggingMiddleware
25 from src.api.middleware.ratelimit import RateLimitMiddleware 27 from src.api.middleware.ratelimit import RateLimitMiddleware
26−from src.api.routes import auth, envelope, health, runs, services 28+from src.api.routes import auth, envelope, health, runs, services, stats
27 from src.config import SERVICES, get_settings, verify_node 29 from src.config import SERVICES, get_settings, verify_node
28 from src.database.db import init_db 30 from src.database.db import init_db
29 from src.utils.logger import get_logger 31 from src.utils.logger import get_logger
@@ -33,14 +35,21 @@ VERSION = "1.0.0"
33 35
34 @asynccontextmanager 36 @asynccontextmanager
35 async def lifespan(app: FastAPI) -> AsyncIterator[None]: 37 async def lifespan(app: FastAPI) -> AsyncIterator[None]:
36− """Au démarrage : vérifie le node m3u96b et initialise le schéma DB.""" 38+ """Au démarrage : vérifie le node m3u96b, initialise le schéma DB et lance
39+ la tâche de fond qui persiste le journal léger des requêtes (/stats)."""
37 verify_node() 40 verify_node()
38 init_db() 41 init_db()
39 get_logger("apika.api").info( 42 get_logger("apika.api").info(
40 "API-KA démarrée", 43 "API-KA démarrée",
41 extra={"version": VERSION, "node": get_settings().node_name}, 44 extra={"version": VERSION, "node": get_settings().node_name},
42 ) 45 )
43− yield 46+ flusher = asyncio.create_task(reqstats.flusher_task())
47+ try:
48+ yield
49+ finally:
50+ flusher.cancel()
51+ with suppress(asyncio.CancelledError):
52+ await flusher
44 53
45 54
46 app = FastAPI( 55 app = FastAPI(
@@ -63,12 +72,14 @@ app.add_middleware(RequestLoggingMiddleware)
63 app.include_router(health.router) 72 app.include_router(health.router)
64 app.include_router(auth.router) 73 app.include_router(auth.router)
65 app.include_router(runs.router) 74 app.include_router(runs.router)
75+app.include_router(stats.router)
66 app.include_router(services.router) 76 app.include_router(services.router)
67 77
68 78
69 WEB_DIR = Path(__file__).resolve().parent / "web" 79 WEB_DIR = Path(__file__).resolve().parent / "web"
70 WEB_INDEX = WEB_DIR / "index.html" 80 WEB_INDEX = WEB_DIR / "index.html"
71 WEB_CONTACT = WEB_DIR / "contact.html" 81 WEB_CONTACT = WEB_DIR / "contact.html"
82+WEB_STATS = WEB_DIR / "stats.html"
72 KA_DIR = WEB_DIR / "ka" 83 KA_DIR = WEB_DIR / "ka"
73 84
74 # Package ka-ui vendorisé (tokens.css, ecosystem.json, ka-shell.js) — servi 85 # Package ka-ui vendorisé (tokens.css, ecosystem.json, ka-shell.js) — servi
@@ -119,6 +130,16 @@ def contact(request: Request) -> Any:
119 ) 130 )
120 131
121 132
133+@app.get("/stats", tags=["root"], include_in_schema=False)
134+def stats_page() -> FileResponse:
135+ """Page /stats — tableau de bord analytique de la plateforme (HTML statique).
136+
137+ Les données proviennent de GET /api/stats/dashboard ; l'export PDF de
138+ GET /api/stats/report (gabarit Groupe-KA).
139+ """
140+ return FileResponse(WEB_STATS, media_type="text/html")
141+
142+
122 # Assets de marque Groupe KA servis à la racine du domaine (voir src/api/web/) : 143 # Assets de marque Groupe KA servis à la racine du domaine (voir src/api/web/) :
123 # favicon SVG, icône iOS et image Open Graph 1200x630 referencée par les meta og:image. 144 # favicon SVG, icône iOS et image Open Graph 1200x630 referencée par les meta og:image.
124 @app.get("/favicon.svg", include_in_schema=False) 145 @app.get("/favicon.svg", include_in_schema=False)
modified src/api/middleware/logging.py +14 −0
@@ -10,12 +10,14 @@
10 10
11 from __future__ import annotations 11 from __future__ import annotations
12 12
13+import datetime
13 import time 14 import time
14 15
15 from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint 16 from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
16 from starlette.requests import Request 17 from starlette.requests import Request
17 from starlette.responses import Response 18 from starlette.responses import Response
18 19
20+from src.api import reqstats
19 from src.utils.logger import get_logger 21 from src.utils.logger import get_logger
20 22
21 23
@@ -28,6 +30,18 @@ class RequestLoggingMiddleware(BaseHTTPMiddleware):
28 start = time.perf_counter() 30 start = time.perf_counter()
29 response = await call_next(request) 31 response = await call_next(request)
30 duration_ms = round((time.perf_counter() - start) * 1000, 2) 32 duration_ms = round((time.perf_counter() - start) * 1000, 2)
33+ # Journalisation légère pour /stats : simple append en mémoire (O(1),
34+ # aucune I/O ici) — la persistance se fait en lot par la tâche de fond.
35+ try:
36+ reqstats.record(
37+ ts=datetime.datetime.now(tz=datetime.UTC),
38+ method=request.method,
39+ path=request.url.path,
40+ status=response.status_code,
41+ duration_ms=duration_ms,
42+ )
43+ except Exception: # pragma: no cover — jamais bloquant pour l'API
44+ pass
31 get_logger("apika.api").info( 45 get_logger("apika.api").info(
32 "http_request", 46 "http_request",
33 extra={ 47 extra={
added src/api/reqstats.py +164 −0
@@ -0,0 +1,164 @@
1+# ============================================
2+# Projet : API-KA
3+# Fichier : src/api/reqstats.py
4+# Node : m3u96b
5+# Author : Simon-Pierre Boucher
6+# Contact : contact@spboucher.ai
7+# Date : 2026-08-17
8+# ============================================
9+"""Journalisation légère des requêtes API vers la table ``api_requests``.
10+
11+Conçu pour ne JAMAIS ralentir le chemin de requête : le middleware appelle
12+``record()`` (un simple ``deque.append``, O(1), sans I/O) et une tâche de
13+fond (démarrée dans le lifespan de l'app) vide le tampon en lot toutes les
14+``FLUSH_INTERVAL`` secondes via un thread, hors boucle d'événements.
15+
16+Rétention : ``RETENTION_DAYS`` jours — une purge s'exécute au plus une fois
17+par jour lors d'un flush. Alimente exclusivement la page /stats.
18+"""
19+
20+from __future__ import annotations
21+
22+import asyncio
23+import datetime
24+import threading
25+from collections import deque
26+
27+from src.config import SERVICES
28+from src.utils.logger import get_logger
29+
30+FLUSH_INTERVAL = 5.0 # secondes entre deux vidages du tampon
31+MAX_BUFFER = 10_000 # garde-fou mémoire : au-delà, les plus anciens sont perdus
32+RETENTION_DAYS = 90
33+
34+_buffer: deque[dict] = deque(maxlen=MAX_BUFFER)
35+_lock = threading.Lock()
36+_last_purge: datetime.date | None = None
37+
38+# Chemins statiques connus, conservés tels quels dans la colonne ``endpoint``.
39+_KNOWN_PATHS = frozenset(
40+ {
41+ "/",
42+ "/health",
43+ "/contact",
44+ "/stats",
45+ "/docs",
46+ "/redoc",
47+ "/openapi.json",
48+ "/favicon.svg",
49+ "/apple-touch-icon.png",
50+ "/og.png",
51+ "/api/v1/runs",
52+ "/api/stats/dashboard",
53+ "/api/stats/report",
54+ }
55+)
56+_SERVICE_SUBROUTES = frozenset({"latest", "stats"})
57+
58+
59+def normalize_endpoint(path: str) -> str:
60+ """Replie un chemin de requête vers un endpoint à cardinalité bornée.
61+
62+ Les paramètres de route (dates) sont remplacés par des gabarits, les
63+ assets statiques /ka/* sont agrégés et les chemins inconnus (scans de
64+ bots, 404) sont regroupés sous « (autre) ».
65+ """
66+ path = path.rstrip("/") or "/"
67+ if path in _KNOWN_PATHS:
68+ return path
69+ if path.startswith("/ka/"):
70+ return "/ka/*"
71+ if path.startswith("/api/v1/"):
72+ parts = path.split("/") # ['', 'api', 'v1', service, ...]
73+ service = parts[3] if len(parts) > 3 else ""
74+ if service in SERVICES:
75+ if len(parts) == 4:
76+ return f"/api/v1/{service}"
77+ sub = parts[4]
78+ if sub == "date":
79+ return f"/api/v1/{service}/date/{{date}}"
80+ if sub in _SERVICE_SUBROUTES and len(parts) == 5:
81+ return f"/api/v1/{service}/{sub}"
82+ return "/api/v1/(autre)"
83+ return "(autre)"
84+
85+
86+def record(
87+ ts: datetime.datetime, method: str, path: str, status: int, duration_ms: float
88+) -> None:
89+ """Empile une requête dans le tampon mémoire (non bloquant, jamais d'I/O)."""
90+ entry = {
91+ "ts": ts,
92+ "method": method,
93+ "endpoint": normalize_endpoint(path),
94+ "status": int(status),
95+ "duration_ms": round(float(duration_ms), 2),
96+ }
97+ with _lock:
98+ _buffer.append(entry)
99+
100+
101+def _drain() -> list[dict]:
102+ """Vide le tampon et retourne son contenu."""
103+ with _lock:
104+ entries = list(_buffer)
105+ _buffer.clear()
106+ return entries
107+
108+
109+def flush() -> int:
110+ """Insère en lot le contenu du tampon dans ``api_requests`` (synchrone).
111+
112+ Appelée depuis un thread par la tâche de fond ; purge les lignes plus
113+ vieilles que ``RETENTION_DAYS`` jours au plus une fois par jour.
114+ """
115+ global _last_purge
116+ entries = _drain()
117+ if not entries:
118+ _maybe_purge()
119+ return 0
120+ try:
121+ from src.database.db import session_scope
122+ from src.database.models import ApiRequest
123+
124+ with session_scope() as session:
125+ session.bulk_insert_mappings(ApiRequest.__mapper__, entries)
126+ except Exception: # pragma: no cover — la stat ne doit jamais casser l'API
127+ get_logger("apika.reqstats").exception("Échec du flush api_requests")
128+ return 0
129+ _maybe_purge()
130+ return len(entries)
131+
132+
133+def _maybe_purge() -> None:
134+ """Purge les requêtes plus vieilles que RETENTION_DAYS (1 fois/jour max)."""
135+ global _last_purge
136+ today = datetime.datetime.now(tz=datetime.UTC).date()
137+ if _last_purge == today:
138+ return
139+ _last_purge = today
140+ try:
141+ from sqlalchemy import delete
142+
143+ from src.database.db import session_scope
144+ from src.database.models import ApiRequest
145+
146+ cutoff = datetime.datetime.now(tz=datetime.UTC) - datetime.timedelta(
147+ days=RETENTION_DAYS
148+ )
149+ with session_scope() as session:
150+ session.execute(delete(ApiRequest).where(ApiRequest.ts < cutoff))
151+ except Exception: # pragma: no cover
152+ get_logger("apika.reqstats").exception("Échec de la purge api_requests")
153+
154+
155+async def flusher_task() -> None:
156+ """Tâche de fond : flush périodique du tampon, hors event loop (thread)."""
157+ try:
158+ while True:
159+ await asyncio.sleep(FLUSH_INTERVAL)
160+ await asyncio.to_thread(flush)
161+ except asyncio.CancelledError:
162+ # Dernier flush au shutdown pour ne rien perdre.
163+ await asyncio.to_thread(flush)
164+ raise
added src/api/routes/stats.py +580 −0
@@ -0,0 +1,580 @@
1+# ============================================
2+# Projet : API-KA
3+# Fichier : src/api/routes/stats.py
4+# Node : m3u96b
5+# Author : Simon-Pierre Boucher
6+# Contact : contact@spboucher.ai
7+# Date : 2026-08-17
8+# ============================================
9+"""Routes /api/stats : tableau de bord analytique de la plateforme + rapport PDF.
10+
11+Contrat commun Groupe KA (src/api/web/ka/stats/SPEC.md) :
12+- ``GET /api/stats/dashboard?period=…`` → JSON (KPI, séries, répartitions,
13+ heatmap, tableaux, records) calculé depuis les données réelles :
14+ table ``api_requests`` (journal du middleware) et ``collection_runs``.
15+- ``GET /api/stats/report?period=…&mode=complet|synthese`` → PDF Groupe-KA.
16+
17+Cache serveur : 5 minutes par période. Aucune stat inventée : une section
18+sans données est simplement absente (le front affiche « Pas encore mesuré »).
19+"""
20+
21+from __future__ import annotations
22+
23+import datetime
24+import threading
25+import time
26+from typing import Any
27+from zoneinfo import ZoneInfo
28+
29+from fastapi import APIRouter, Depends, HTTPException, Query, Response
30+from sqlalchemy import func, select
31+from sqlalchemy.orm import Session
32+
33+from src.api import kapdf
34+from src.api.routes import envelope
35+from src.database.db import get_db
36+from src.database.models import ApiRequest, CollectionRun
37+
38+router = APIRouter(prefix="/api/stats", tags=["stats"])
39+
40+TZ = ZoneInfo("America/Toronto")
41+CACHE_TTL = 300.0 # ≥ 5 min (SPEC)
42+PLATFORM_ID = "api-ka"
43+
44+SITE = {
45+ "wordmark": "API·Ka",
46+ "accent": "#3b5bdb",
47+ "domain": "www.api-ka.com",
48+ "tagline": "Plateforme centrale de l'écosystème Groupe KA",
49+}
50+
51+SERVICE_NAMES = {
52+ "louka": "Lou·Ka",
53+ "immoka": "Immo·Ka",
54+ "foodka": "Food·Ka",
55+ "autoka": "Auto·Ka",
56+ "fabrika": "Fabri·Ka",
57+ "restoka": "Resto·Ka",
58+ "sortika": "Sorti·Ka",
59+ "creaka": "Créa·Ka",
60+}
61+
62+PERIOD_LABELS = {
63+ "auj": "aujourd'hui",
64+ "7j": "7 jours",
65+ "30j": "30 jours",
66+ "3m": "3 mois",
67+ "6m": "6 mois",
68+ "12m": "12 mois",
69+ "annee": "année en cours",
70+ "tout": "toute la période",
71+}
72+PERIOD_DAYS = {"7j": 7, "30j": 30, "3m": 91, "6m": 182, "12m": 365}
73+
74+_cache: dict[tuple, tuple[float, dict]] = {}
75+_cache_lock = threading.Lock()
76+
77+
78+# ---------------------------------------------------------------- périodes
79+def _resolve_period(
80+ period: str,
81+ date_from: datetime.date | None,
82+ date_to: datetime.date | None,
83+ db: Session,
84+) -> tuple[datetime.date, datetime.date, str]:
85+ """Résout (from, to, label) en dates locales America/Toronto."""
86+ today = datetime.datetime.now(tz=TZ).date()
87+ if date_from and date_to:
88+ if date_from > date_to:
89+ raise HTTPException(status_code=422, detail="from doit précéder to")
90+ return date_from, date_to, f"du {date_from} au {date_to}"
91+ if period == "auj":
92+ return today, today, PERIOD_LABELS["auj"]
93+ if period == "annee":
94+ return datetime.date(today.year, 1, 1), today, PERIOD_LABELS["annee"]
95+ if period == "tout":
96+ firsts = [
97+ db.execute(select(func.min(ApiRequest.ts))).scalar(),
98+ db.execute(select(func.min(CollectionRun.started_at))).scalar(),
99+ ]
100+ dates = []
101+ for f in firsts:
102+ if f is not None:
103+ if f.tzinfo is None:
104+ f = f.replace(tzinfo=datetime.UTC)
105+ dates.append(f.astimezone(TZ).date())
106+ return (min(dates) if dates else today), today, PERIOD_LABELS["tout"]
107+ if period in PERIOD_DAYS:
108+ return (
109+ today - datetime.timedelta(days=PERIOD_DAYS[period] - 1),
110+ today,
111+ PERIOD_LABELS[period],
112+ )
113+ raise HTTPException(
114+ status_code=422,
115+ detail=f"Période invalide : {period}. Valides : {', '.join(PERIOD_LABELS)}",
116+ )
117+
118+
119+def _utc_bounds(
120+ d_from: datetime.date, d_to: datetime.date
121+) -> tuple[datetime.datetime, datetime.datetime]:
122+ """Bornes UTC [00:00 from, 24:00 to] exprimées depuis les dates locales."""
123+ start = datetime.datetime.combine(d_from, datetime.time.min, tzinfo=TZ)
124+ end = datetime.datetime.combine(
125+ d_to + datetime.timedelta(days=1), datetime.time.min, tzinfo=TZ
126+ )
127+ return start.astimezone(datetime.UTC), end.astimezone(datetime.UTC)
128+
129+
130+# ---------------------------------------------------------------- agrégats
131+def _fetch_requests(
132+ db: Session, d_from: datetime.date, d_to: datetime.date
133+) -> list[tuple[datetime.datetime, str, int, float]]:
134+ """Requêtes API de la fenêtre : (ts local, endpoint, status, duration_ms)."""
135+ lo, hi = _utc_bounds(d_from, d_to)
136+ rows = db.execute(
137+ select(
138+ ApiRequest.ts, ApiRequest.endpoint, ApiRequest.status, ApiRequest.duration_ms
139+ ).where(ApiRequest.ts >= lo, ApiRequest.ts < hi)
140+ ).all()
141+ out = []
142+ for ts, endpoint, status, duration in rows:
143+ if ts.tzinfo is None:
144+ ts = ts.replace(tzinfo=datetime.UTC)
145+ out.append((ts.astimezone(TZ), endpoint, status, duration or 0.0))
146+ return out
147+
148+
149+def _fetch_runs(
150+ db: Session, d_from: datetime.date, d_to: datetime.date
151+) -> list[CollectionRun]:
152+ """Runs de collecte de la fenêtre (par date logique ``date_key``)."""
153+ return (
154+ db.execute(
155+ select(CollectionRun)
156+ .where(CollectionRun.date_key >= d_from, CollectionRun.date_key <= d_to)
157+ .order_by(CollectionRun.started_at.desc())
158+ )
159+ .scalars()
160+ .all()
161+ )
162+
163+
164+def _p95(values: list[float]) -> float:
165+ if not values:
166+ return 0.0
167+ vs = sorted(values)
168+ idx = max(0, int(round(0.95 * (len(vs) - 1))))
169+ return vs[idx]
170+
171+
172+def _delta(cur: float, prev: float | None) -> float | None:
173+ if prev is None or prev == 0:
174+ return None
175+ return round((cur - prev) / prev * 100, 1)
176+
177+
178+def _daterange(d_from: datetime.date, d_to: datetime.date) -> list[datetime.date]:
179+ n = (d_to - d_from).days + 1
180+ return [d_from + datetime.timedelta(days=i) for i in range(n)]
181+
182+
183+def _fr_int(n: float) -> str:
184+ return f"{int(n):,}".replace(",", " ")
185+
186+
187+# ---------------------------------------------------------------- dashboard
188+def _build_dashboard(db: Session, period: str, d_from, d_to, label) -> dict[str, Any]:
189+ """Calcule le JSON complet du contrat SPEC depuis les données réelles."""
190+ hourly = period == "auj" or d_from == d_to
191+ span = (d_to - d_from).days + 1
192+ prev_to = d_from - datetime.timedelta(days=1)
193+ prev_from = prev_to - datetime.timedelta(days=span - 1)
194+
195+ reqs = _fetch_requests(db, d_from, d_to)
196+ prev_reqs = _fetch_requests(db, prev_from, prev_to)
197+ runs = _fetch_runs(db, d_from, d_to)
198+ prev_runs = _fetch_runs(db, prev_from, prev_to)
199+
200+ # -------- agrégats requêtes API (période courante)
201+ total_calls = len(reqs)
202+ durations = [r[3] for r in reqs]
203+ avg_ms = round(sum(durations) / total_calls, 2) if total_calls else 0.0
204+ p95_ms = round(_p95(durations), 2)
205+ errors = sum(1 for r in reqs if r[2] >= 400)
206+ err_rate = round(errors / total_calls * 100, 2) if total_calls else 0.0
207+ active_endpoints = len({r[1] for r in reqs})
208+
209+ # -------- agrégats requêtes API (période précédente, pour les deltas)
210+ prev_calls = len(prev_reqs)
211+ prev_durs = [r[3] for r in prev_reqs]
212+ prev_avg = (sum(prev_durs) / prev_calls) if prev_calls else None
213+ prev_err = (
214+ sum(1 for r in prev_reqs if r[2] >= 400) / prev_calls * 100
215+ if prev_calls
216+ else None
217+ )
218+
219+ # -------- agrégats runs de collecte
220+ ok_runs = [r for r in runs if r.status in ("success", "retried")]
221+ failed_runs = [r for r in runs if r.status == "failed"]
222+ records_total = sum(r.records_count for r in ok_runs)
223+ prev_ok = [r for r in prev_runs if r.status in ("success", "retried")]
224+ prev_records = sum(r.records_count for r in prev_ok)
225+
226+ # -------- KPI (uniquement des mesures réelles)
227+ kpis: list[dict[str, Any]] = []
228+
229+ def kpi(id_, label_, value, unit="", delta=None, invert=False):
230+ d: dict[str, Any] = {"id": id_, "label": label_, "value": value, "unit": unit}
231+ if delta is not None:
232+ d["delta_pct"] = delta
233+ good = (delta <= 0) if invert else (delta >= 0)
234+ d["direction"] = "up" if good else "down"
235+ if invert:
236+ d["invert"] = True
237+ kpis.append(d)
238+
239+ kpi("calls", "Appels API", total_calls, "", _delta(total_calls, prev_calls))
240+ kpi(
241+ "latency",
242+ "Latence moyenne",
243+ avg_ms,
244+ "ms",
245+ _delta(avg_ms, round(prev_avg, 2) if prev_avg else None),
246+ invert=True,
247+ )
248+ kpi("p95", "Latence p95", p95_ms, "ms")
249+ kpi(
250+ "errors",
251+ "Taux d'erreur (HTTP ≥ 400)",
252+ err_rate,
253+ "%",
254+ _delta(err_rate, round(prev_err, 2) if prev_err else None),
255+ invert=True,
256+ )
257+ kpi("endpoints", "Endpoints actifs", active_endpoints)
258+ kpi(
259+ "runs_ok",
260+ "Collectes réussies",
261+ len(ok_runs),
262+ "",
263+ _delta(len(ok_runs), len(prev_ok) or None),
264+ )
265+ kpi("runs_failed", "Collectes échouées", len(failed_runs), "", invert=True)
266+ kpi(
267+ "records",
268+ "Enregistrements collectés",
269+ records_total,
270+ "",
271+ _delta(records_total, prev_records or None),
272+ )
273+
274+ # -------- séries temporelles
275+ series: list[dict[str, Any]] = []
276+ days = _daterange(d_from, d_to)
277+
278+ if hourly:
279+ buckets = [f"{h:02d}h" for h in range(24)]
280+ calls_by = {b: 0 for b in buckets}
281+ lat_by: dict[str, list[float]] = {b: [] for b in buckets}
282+ for ts, _, _, dur in reqs:
283+ b = f"{ts.hour:02d}h"
284+ calls_by[b] += 1
285+ lat_by[b].append(dur)
286+ calls_pts = [{"t": b, "v": calls_by[b]} for b in buckets]
287+ lat_pts = [
288+ {
289+ "t": b,
290+ "v": round(sum(lat_by[b]) / len(lat_by[b]), 1) if lat_by[b] else 0,
291+ }
292+ for b in buckets
293+ ]
294+ calls_title = "Appels API par heure"
295+ lat_title = "Latence moyenne par heure (ms)"
296+ else:
297+ calls_by = {d: 0 for d in days}
298+ lat_by = {d: [] for d in days}
299+ for ts, _, _, dur in reqs:
300+ d = ts.date()
301+ if d in calls_by:
302+ calls_by[d] += 1
303+ lat_by[d].append(dur)
304+ calls_pts = [{"t": d.isoformat(), "v": calls_by[d]} for d in days]
305+ lat_pts = [
306+ {
307+ "t": d.isoformat(),
308+ "v": round(sum(lat_by[d]) / len(lat_by[d]), 1) if lat_by[d] else 0,
309+ }
310+ for d in days
311+ ]
312+ calls_title = "Appels API par jour"
313+ lat_title = "Latence moyenne par jour (ms)"
314+
315+ calls_serie: dict[str, Any] = {
316+ "id": "calls",
317+ "title": calls_title,
318+ "unit": "appels",
319+ "kind": "line",
320+ "points": calls_pts,
321+ }
322+ if prev_reqs and not hourly:
323+ prev_days = _daterange(prev_from, prev_to)
324+ prev_by = {d: 0 for d in prev_days}
325+ for ts, _, _, _ in prev_reqs:
326+ d = ts.date()
327+ if d in prev_by:
328+ prev_by[d] += 1
329+ calls_serie["compare"] = [
330+ {"t": d.isoformat(), "v": prev_by[d]} for d in prev_days
331+ ]
332+ if reqs:
333+ series.append(calls_serie)
334+ series.append(
335+ {
336+ "id": "latency",
337+ "title": lat_title,
338+ "unit": "ms",
339+ "kind": "line",
340+ "points": lat_pts,
341+ }
342+ )
343+
344+ if runs and not hourly:
345+ rec_by = {d: 0 for d in days}
346+ for r in ok_runs:
347+ if r.date_key in rec_by:
348+ rec_by[r.date_key] += r.records_count
349+ series.append(
350+ {
351+ "id": "records",
352+ "title": "Enregistrements collectés par jour",
353+ "unit": "enregistrements",
354+ "kind": "line",
355+ "points": [{"t": d.isoformat(), "v": rec_by[d]} for d in days],
356+ }
357+ )
358+
359+ # -------- répartitions
360+ breakdowns: list[dict[str, Any]] = []
361+ ep_stats: dict[str, dict[str, Any]] = {}
362+ for _, endpoint, status, dur in reqs:
363+ s = ep_stats.setdefault(endpoint, {"calls": 0, "durs": [], "errors": 0})
364+ s["calls"] += 1
365+ s["durs"].append(dur)
366+ if status >= 400:
367+ s["errors"] += 1
368+ top_eps = sorted(ep_stats.items(), key=lambda kv: kv[1]["calls"], reverse=True)
369+ if top_eps:
370+ breakdowns.append(
371+ {
372+ "id": "top_endpoints",
373+ "title": "Top endpoints (appels)",
374+ "kind": "bars",
375+ "items": [
376+ {"label": ep, "value": s["calls"]} for ep, s in top_eps[:12]
377+ ],
378+ }
379+ )
380+ svc_records: dict[str, int] = {}
381+ for r in ok_runs:
382+ svc_records[r.service] = svc_records.get(r.service, 0) + r.records_count
383+ if svc_records:
384+ breakdowns.append(
385+ {
386+ "id": "services",
387+ "title": "Enregistrements collectés par service",
388+ "kind": "donut",
389+ "items": sorted(
390+ (
391+ {"label": SERVICE_NAMES.get(s, s), "value": v}
392+ for s, v in svc_records.items()
393+ ),
394+ key=lambda it: it["value"],
395+ reverse=True,
396+ ),
397+ }
398+ )
399+
400+ # -------- heatmap : appels API par jour
401+ heatmap = None
402+ if reqs and not hourly:
403+ cells = [
404+ {"date": d.isoformat(), "value": calls_by[d]}
405+ for d in days
406+ if calls_by[d] > 0
407+ ]
408+ if cells:
409+ heatmap = {"title": "Appels API par jour", "cells": cells}
410+
411+ # -------- tableaux
412+ tables: list[dict[str, Any]] = []
413+ if top_eps:
414+ tables.append(
415+ {
416+ "id": "endpoints",
417+ "title": "Endpoints — appels, latence et erreurs",
418+ "columns": [
419+ "Endpoint",
420+ "Appels",
421+ "Latence moy. (ms)",
422+ "p95 (ms)",
423+ "Erreurs",
424+ "Taux d'erreur",
425+ ],
426+ "rows": [
427+ [
428+ ep,
429+ s["calls"],
430+ round(sum(s["durs"]) / len(s["durs"]), 1),
431+ round(_p95(s["durs"]), 1),
432+ s["errors"],
433+ f"{s['errors'] / s['calls'] * 100:.1f} %".replace(".", ","),
434+ ]
435+ for ep, s in top_eps
436+ ],
437+ }
438+ )
439+ if runs:
440+ tables.append(
441+ {
442+ "id": "runs",
443+ "title": "Derniers runs de collecte",
444+ "columns": ["Service", "Date", "Statut", "Enregistrements", "Durée (s)"],
445+ "rows": [
446+ [
447+ SERVICE_NAMES.get(r.service, r.service),
448+ r.date_key.isoformat(),
449+ {"success": "succès", "failed": "échec", "retried": "relancé"}.get(
450+ r.status, r.status
451+ ),
452+ r.records_count,
453+ round(r.duration_seconds, 1),
454+ ]
455+ for r in runs[:80]
456+ ],
457+ }
458+ )
459+
460+ # -------- records & faits marquants
461+ records: list[dict[str, Any]] = []
462+ if reqs and not hourly:
463+ best_day = max(calls_by.items(), key=lambda kv: kv[1])
464+ if best_day[1] > 0:
465+ records.append(
466+ {
467+ "label": "Jour record d'appels API",
468+ "value": f"{_fr_int(best_day[1])} appels",
469+ "date": best_day[0].isoformat(),
470+ }
471+ )
472+ if top_eps:
473+ records.append(
474+ {
475+ "label": "Endpoint le plus sollicité",
476+ "value": f"{top_eps[0][0]} — {_fr_int(top_eps[0][1]['calls'])} appels",
477+ }
478+ )
479+ if ok_runs:
480+ biggest = max(ok_runs, key=lambda r: r.records_count)
481+ records.append(
482+ {
483+ "label": "Run de collecte le plus volumineux",
484+ "value": (
485+ f"{_fr_int(biggest.records_count)} enregistrements "
486+ f"({SERVICE_NAMES.get(biggest.service, biggest.service)})"
487+ ),
488+ "date": biggest.date_key.isoformat(),
489+ }
490+ )
491+ fastest = min(ok_runs, key=lambda r: r.duration_seconds)
492+ records.append(
493+ {
494+ "label": "Collecte la plus rapide",
495+ "value": (
496+ f"{fastest.duration_seconds:.1f} s "
497+ f"({SERVICE_NAMES.get(fastest.service, fastest.service)})"
498+ ).replace(".", ","),
499+ "date": fastest.date_key.isoformat(),
500+ }
501+ )
502+
503+ # -------- couverture de mesure (depuis quand les appels sont journalisés)
504+ first_req = db.execute(select(func.min(ApiRequest.ts))).scalar()
505+ if first_req is not None and first_req.tzinfo is None:
506+ first_req = first_req.replace(tzinfo=datetime.UTC)
507+
508+ dash: dict[str, Any] = {
509+ "updated": datetime.datetime.now(tz=TZ).isoformat(timespec="seconds"),
510+ "period": {"from": d_from.isoformat(), "to": d_to.isoformat(), "label": label},
511+ "kpis": kpis,
512+ "series": series,
513+ "breakdowns": breakdowns,
514+ "tables": tables,
515+ "records": records,
516+ "coverage": {
517+ "api_requests_since": (
518+ first_req.astimezone(TZ).isoformat(timespec="seconds")
519+ if first_req
520+ else None
521+ )
522+ },
523+ }
524+ if heatmap:
525+ dash["heatmap"] = heatmap
526+ return dash
527+
528+
529+def _dashboard_cached(
530+ db: Session,
531+ period: str,
532+ date_from: datetime.date | None,
533+ date_to: datetime.date | None,
534+) -> dict[str, Any]:
535+ d_from, d_to, label = _resolve_period(period, date_from, date_to, db)
536+ key = (period, d_from.isoformat(), d_to.isoformat())
537+ now = time.monotonic()
538+ with _cache_lock:
539+ hit = _cache.get(key)
540+ if hit and now - hit[0] < CACHE_TTL:
541+ return hit[1]
542+ dash = _build_dashboard(db, period, d_from, d_to, label)
543+ with _cache_lock:
544+ if len(_cache) > 64:
545+ _cache.clear()
546+ _cache[key] = (now, dash)
547+ return dash
548+
549+
550+# ---------------------------------------------------------------- routes
551+@router.get("/dashboard")
552+def stats_dashboard(
553+ period: str = Query("30j", description="auj, 7j, 30j, 3m, 6m, 12m, annee, tout"),
554+ date_from: datetime.date | None = Query(None, alias="from"),
555+ date_to: datetime.date | None = Query(None, alias="to"),
556+ db: Session = Depends(get_db),
557+) -> dict[str, Any]:
558+ """Tableau de bord analytique de la plateforme (contrat commun Groupe KA)."""
559+ return envelope(_dashboard_cached(db, period, date_from, date_to))
560+
561+
562+@router.get("/report")
563+def stats_report(
564+ period: str = Query("30j", description="auj, 7j, 30j, 3m, 6m, 12m, annee, tout"),
565+ date_from: datetime.date | None = Query(None, alias="from"),
566+ date_to: datetime.date | None = Query(None, alias="to"),
567+ mode: str = Query("complet", description="complet ou synthese"),
568+ db: Session = Depends(get_db),
569+) -> Response:
570+ """Rapport statistique PDF estampillé Groupe-KA (complet ou synthèse)."""
571+ if mode not in ("complet", "synthese"):
572+ raise HTTPException(status_code=422, detail="mode : complet ou synthese")
573+ dash = _dashboard_cached(db, period, date_from, date_to)
574+ pdf_bytes = kapdf.GroupeKAReport(site=SITE, dashboard=dash, mode=mode).build()
575+ fname = kapdf.filename(PLATFORM_ID, period)
576+ return Response(
577+ content=pdf_bytes,
578+ media_type="application/pdf",
579+ headers={"Content-Disposition": f'attachment; filename="{fname}"'},
580+ )
modified src/api/web/contact.html +1 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
modified src/api/web/index.html +1 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
added src/api/web/ka/stats/SPEC.md +123 −0
Large diff (123 lines) is not loaded by default to keep the page fast.Load diff
added src/api/web/ka/stats/kacharts.tsx +389 −0
Large diff (389 lines) is not loaded by default to keep the page fast.Load diff
added src/api/web/ka/stats/kapdf.py +560 −0
Large diff (560 lines) is not loaded by default to keep the page fast.Load diff
added src/api/web/stats.html +572 −0
Large diff (572 lines) is not loaded by default to keep the page fast.Load diff
modified src/database/models.py +25 −0
Large diff (28 lines) is not loaded by default to keep the page fast.Load diff
added tests/test_stats.py +113 −0
Large diff (114 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere