SPB Git

spb/llmindex Public

The discriminative, contamination-resistant, fully transparent LLM ranking — updated live.

webappaillmbenchmark
TypeScript 77.9% TeX 15.2% Python 3.7% SQL 1.4% JavaScript 1.1% Shell 0.5%
  1. fix(web): show duel-stage lane in live banner when parallel lanes array is empty
    …
    Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 5 days ago (Aug 5, 2026) · 1 file changed +6 −1
  2. feat(web): dedicated full-page interactive live Pareto frontier chart (/frontier)
    …
    Hand-rolled SVG scatter: log-cost x-axis, Global Index y-axis with 95% CI
whiskers on every point, Pareto step line, direct labels on frontier models,
per-mark hover tooltips (score/CI, cost, latency, status) with enlarged hit
targets, click-through to model pages, 15s live polling with entry animation
for newly landed models. Two-class palette #047857/#7c3aed validated (CVD
deutan dE 23.9, contrast >=3:1). Nav link + home cross-link.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 5 days ago (Aug 5, 2026) · 6 files changed +423 −11
  3. docs(whitepaper): TikZ figures, landscape comparison table, sharp visual design
    …
    - TikZ-redrawn LLMindex logo on the title page
- Figures: evaluation pipeline flow, 2PL item characteristic curves +
  item information functions (pgfplots), score-composition diagram with
  the quality/economics wall, illustrative Pareto frontier
- Comparison table rotated to landscape (fixes right-edge overflow)
- Colored section titles, running headers, overfull boxes eliminated
- 19 pages, clean latexmk build

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 6 days ago (Aug 5, 2026) · 3 files changed +191 −11
  4. docs: methodology white paper (17-page LaTeX + PDF + BibTeX)
    …
    Full technical white paper: benchmark failure analysis, design principles,
2PL IRT and Bradley-Terry mathematics, robustness metrics, score structure
and interpretation guide, governance, limitations, comparison table, roadmap,
33 academic references. Compiles with latexmk -pdf.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 6 days ago (Aug 5, 2026) · 5 files changed +678 −1
  5. fix(worker): cap completion budget by model context window
    …
    max_tokens=16384 caused HTTP 400 on small-context models (e.g. 16k-window
models); budget is now min(16384, context_length - 4096) with a 2048 floor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 6 days ago (Aug 5, 2026) · 1 file changed +7 −2
  6. feat(web): deep-detail methodology, privacy policy, terms of service, live efficiency frontier
    …
    - Methodology: scoring mathematics (2PL MAP, Fisher SE, BT MM + SE, composite
  formulas), item lifecycle & bank hygiene, full judge protocol, known
  limitations, versioning/audit section, contact block
- New /privacy and /terms pages (contact: Simon-Pierre Boucher,
  contact@spboucher.ai), footer links
- Efficiency frontier + hero stats now poll live (/api/v1/efficiency)
- Rankings display current INDEX_VERSION only (old runs remain as
  immutable audit history)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 6 days ago (Aug 5, 2026) · 8 files changed +529 −77
  7. feat: LLM Index platform v0.2.0 — discriminative, contamination-resistant live LLM ranking
    …
    IRT 2PL + Bradley-Terry scoring over 12 domains with dynamically generated
items: hardened math/reasoning/code/knowledge/multilingual/instruction,
home-made simulated agentic (+context-load) and terminal benches, SVG logo
duels (3-judge cross-provider panel), vision OCR. Robust extraction cascade,
parallel evaluation lanes with live-updating leaderboard, per-model full
transparency pages, public API v1, detailed methodology with references.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 6 days ago (Aug 5, 2026) · 130 files changed +14,699