SPB Git

spb/llmindex Public

The discriminative, contamination-resistant, fully transparent LLM ranking — updated live.

webappaillmbenchmark
TypeScript 77.9% TeX 15.2% Python 3.7% SQL 1.4% JavaScript 1.1% Shell 0.5%

History of apps/worker/src/eval-runner.ts · clear filter

  1. fix(worker): cap completion budget by model context window
    …
    max_tokens=16384 caused HTTP 400 on small-context models (e.g. 16k-window
models); budget is now min(16384, context_length - 4096) with a 2048 floor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 5 days ago (Aug 5, 2026) · 1 file changed +7 −2
  2. feat: LLM Index platform v0.2.0 — discriminative, contamination-resistant live LLM ranking
    …
    IRT 2PL + Bradley-Terry scoring over 12 domains with dynamically generated
items: hardened math/reasoning/code/knowledge/multilingual/instruction,
home-made simulated agentic (+context-load) and terminal benches, SVG logo
duels (3-judge cross-provider panel), vision OCR. Robust extraction cascade,
parallel evaluation lanes with live-updating leaderboard, per-model full
transparency pages, public API v1, detailed methodology with references.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 5 days ago (Aug 5, 2026) · 1 file changed +229