SPB Git forge

spb/polyllm

Public
15commits 1branches 0releases
2.2 MBsize
maindefault branch
13 days agolast push
TypeScript 97.4% SQL 1% JavaScript 0.9% CSS 0.6%
12.9 KB

# Workstream C — Arena (2026-09-11)

Mobile-first rebuild of the Arena: adaptive layouts, live metrics, per-criterion votes, Arena Winner, Blind Arena, history, personal scoreboard, Markdown/JSON exports and public sharing. All existing backend behaviour is preserved (/api/arena GET/POST/PATCH, /api/arena/stream, arena_responses.ratings kept in sync).

# Files

# New

Path Role
src/lib/arena/scoring.ts Pure: criteria (BUILTIN_CRITERIA, customCriterion, criterionById, ratingKeyFor), categories (categoryFromTask), computeWinner, computeScoreboard, Blind helpers (shuffledOrder, normalizeOrder, blindLabel)
src/lib/arena/metrics.ts Pure: liveMetrics() (estimates from char count while streaming → exact on done), formatDelta
src/lib/arena/export.ts Pure: buildArenaMarkdown, buildArenaJson, buildArenaShareSnapshot, generationParameters, arenaExportFilename
src/app/api/arena/vote/route.ts POST / DELETE votes
src/app/api/arena/scoreboard/route.ts GET scoreboard
src/app/api/arena/[id]/route.ts GET one session (responses + votes), DELETE session
src/app/api/arena/[id]/export/route.ts POST export (Markdown / JSON)
src/app/api/arena/[id]/share/route.ts GET status / POST create-refresh / DELETE revoke
src/app/app/arena/scoreboard/page.tsx + src/components/arena/scoreboard-view.tsx Personal scoreboard
src/app/share/arena/[id]/page.tsx + src/components/arena/shared-arena-view.tsx Public share page (server page + client carousel)
src/components/arena/metrics-strip.tsx MetricsStrip, StatusDot, STATUS_META, useTick (shared 500 ms clock)
src/components/arena/vote-panel.tsx VotePanel chips + useCustomCriteria (localStorage polyllm:arena-criteria)
src/components/arena/winner-card.tsx Arena Winner card (animate-pop, Δ cost / Δ TTFT / Δ output)
src/components/arena/comparison-table.tsx End-of-run comparison: stacked metric rows < md, table ≥ md
src/components/arena/model-tabs.tsx Sticky phone tab bar (provider icon / blind letter, status dot, live tok/s, trophy)
src/components/arena/blind.tsx BlindAvatar, Flip, useFlip (two-phase rotateY reveal, no CSS additions)
src/components/arena/share-sheet.tsx ArenaShareSheet (ResponsiveDialog; privacy warning → link, views, refresh, revoke)
src/components/arena/download.ts downloadArenaExport() (blob download, filename from headers)
tests/unit/arena-scoring.test.ts 19 vitest units: winner (majority, tie → TTFT, deltas, dedupe), scoreboard (rates, category/criterion filters), criteria/categories, blind permutation, live metrics, exports/snapshot (no attachment data)

# Changed

Path Change
src/components/arena/arena-view.tsx Rewritten: header (layout Segmented, sync scroll, Reveal, New, Stop, menu), scrollable results, composer pinned at the bottom with model chips + ModelConfig + Blind toggle in leftSlot; phone .snap-row carousel + sticky ModelTabs; votes, winner, comparison table, history, share sheet, deep links
src/components/arena/arena-column.tsx Live MetricsStrip under every response (all statuses), VotePanel once final, blind identity (letter avatar + Flip), per-column Reveal, Winner badge; STATUS_META/StatusDot re-exported for compatibility
src/components/arena/arena-history.tsx Phone rows (tap = open read-only, long-press / … = ActionSheet), desktop table (Prompt · Models · Winner · Cost · Date · actions), ConfirmDialog delete, export/share entries
src/components/arena/model-picker.tsx Single scrollable row on phones (overflow-x-auto, md:flex-wrap), .tap targets
src/components/arena/types.ts ArenaVoteDto, ArenaSessionDto.votes, sessionBlind/sessionOrder/sessionAttachmentCount/sessionWinner/sessionCost
src/lib/arena/service.ts blind → settings.blind + shuffled settings.blindOrder; Arena flags stripped before filterSettings; sessions include votes; getArenaSession, deleteArenaSession, castArenaVote (upsert + legacy ratings mirror), retractArenaVote, getArenaScoreboard, exportArenaSession, shareArenaSession, revokeArenaShare, getArenaShare, getPublicArenaShare
src/lib/chat/schemas.ts (additive, arena-only) arenaRunSchema.blind, arenaCriterionSchema, arenaCategorySchema, arenaVoteSchema, arenaRetractVoteSchema

No new dependencies, no schema/migration changes (arena_votes and shared_arena_sessions from 0001_workspace_upgrade are used as-is).

# API routes

Method Path Body / query Response
GET /api/arena — { sessions: (ArenaSession & { responses, votes })[] } — now includes votes
POST /api/arena { prompt, systemPrompt?, modelKeys[1..4], settings?, attachmentIds?, blind? } { session } (settings carry blind: true, blindOrder: number[] when blind)
PATCH /api/arena { responseId, ratings } unchanged (legacy)
POST /api/arena/stream { sessionId, modelKey } SSE, unchanged
POST /api/arena/vote { sessionId, responseId, criterion, category? } — criterion best|accurate|writing|coding|value|fastest|custom:<slug>; category coding|research|writing|reasoning|general (client passes categoryFromTask(analyzePrompt(prompt).task)) { votes, ratings: { [responseId]: ratings } } — upsert on (session_id, criterion), mirrors ratingKeyFor(criterion) into arena_responses.ratings (exclusive)
DELETE /api/arena/vote { sessionId, criterion } { votes, ratings }
GET /api/arena/scoreboard ?category=&criterion= (Cost efficiency = criterion=value) { rows: ScoreboardRow[], sessions, votes } — per model: sessions, decided, wins, votes, winRate = wins / decided, avgCostUsd, avgTtftMs, criteria{}. Session category = voters' category if any, else analyzePrompt(prompt)
GET /api/arena/:id — { session } (responses + votes, owner only)
DELETE /api/arena/:id — { ok } — removes responses (cascade), votes, share links; usage records kept
POST /api/arena/:id/export?format=markdown|json — file (Content-Disposition, X-Filename) with models, parameters, metrics, votes/ratings, winner, responses
GET /api/arena/:id/share — { share: { id, createdAt, viewCount } | null }
POST /api/arena/:id/share — { share } — creates or refreshes the snapshot in shared_arena_sessions (no attachment data)
DELETE /api/arena/:id/share — { ok } — revokes
page /share/arena/:shareId — public, robots: noindex, increments view_count

Deep links on /app/arena: ?models=a,b preselects (unknown keys skipped with a toast), ?session=<id> opens a stored session read-only; the URL is cleaned with router.replace afterwards.

# Behaviour notes

  • Layout: desktop 1 → single, 2 → side by side, 3–4 → 2×2 grid by default, Columns toggle (Segmented in the header) switches to 3/4 columns from xl. Phone: one response per screen in a .snap-row (useSnapCarousel), sticky tab bar scrolls the carousel, swipe updates the tab; never more than one column. Composer + model chips are a fixed bottom bar (page is h-full min-h-0 flex-col; the shell reserves bottom-nav space).
  • Live metrics: liveMetrics() — while streaming: elapsed/TTFT from timestamps, output tokens from estimateTextTokens(text + reasoning), tok/s after 400 ms of generation, cost via estimateCost(model, promptTokens, outTokens) (≈ prefix). On done: exact server usage/latency/cost. One shared 500 ms clock (useTick) while anything streams.
  • Winner: most criteria won; tie → lowest TTFT (then latency, then order); needs all responses final + ≥ 1 vote. Deltas = winner − mean(other complete responses).
  • Blind Arena: toggle next to the composer (persists for re-runs). Server shuffles positions (blindOrder); the client shows Model A/B/C/D with neutral letter avatars, hides provider name/icon and the reasoning badge. Voting Best answer, the header Reveal button or a column's eye icon flips identities (Flip, 160 ms, reduced-motion safe). Stored blind sessions open revealed; exports/shares reveal names and mark the session as blind.
  • Custom criteria: polyllm:arena-criteria ([{id, label}], max 12). Ids custom:<slug>; legacy rating key bestCustom_<slug>.
  • Scoreboard math: a session is decided when it has ≥ 1 vote matching the filter; winRate = wins / decided (models that only appear in unvoted sessions are listed as unranked). Cost efficiency = criterion=value.

# Coming soon / not done

  • Nothing is labelled "Coming soon" in the Arena UI; every control is wired.
  • Arena votes are not (yet) surfaced in Usage analytics or the Models catalog — the scoreboard API (GET /api/arena/scoreboard) is ready for the Models workstream if it wants a "your win rate" badge.
  • Share snapshots are frozen: votes cast after sharing need Refresh snapshot (documented in the sheet).

# Integration requests (not in my files)

  • src/lib/ids.ts: I use newId("arv") for vote ids; optionally add arenaVote: () => newId("arv") to ids for consistency.
  • Chat "Compare with…" (workstream A) can open a stored comparison with /app/arena?session=<id> and preselect with ?models=.
  • Nothing else touches other areas; Composer/ModelSelector/ModelConfig are used with their current props only.

# Validation

  • pnpm typecheck: clean (whole project at the time of writing).
  • pnpm lint: arena paths clean; remaining project problems are in models/page.tsx, onboarding-flow.tsx, chat/*, search/* (other workstreams).
  • pnpm test: 124/125 — the single failure is formatPricePair in the Models tests (not arena); tests/unit/arena-scoring.test.ts 19/19.
  • React Compiler note: keep useSnapCarousel destructured ({ ref, index, scrollTo }) — the returned object is treated as a ref and reading carousel.index in render is flagged. Do not store reset (a ref-mutating callback) inside the header menu array; it is a header button instead.

# Visual QA checklist (integration phase, 375 / 390 / 393 / 430 px + 1440)

  1. /app/arena empty state: header 48 px with hamburger, bottom bar shows "Choose models" + presets in one scrollable row, composer above the bottom nav, no horizontal overflow.
  2. Pick 4 models → chips scroll horizontally on phone, wrap on desktop; 4/4 hides "Add model"; ModelConfig + Blind toggle sit in the composer's left slot.
  3. Run: phone shows the sticky tab bar (icon, name, dot, tok/s), one panel per screen, swipe left/right changes the active tab and vice-versa; "Swipe to compare · n / 4" updates. Desktop: 2×2 grid; header Segmented switches to 4 columns at ≥ 1280 px; sync scroll works at ≥ 1024 px.
  4. While streaming: metrics strip shows Elapsed → TTFT, ≈ tokens, tok/s appears after ~0.5 s, ≈ cost ticks; on done values lose the ≈ and match the stored numbers.
  5. Vote: chips under each final response; voting the same criterion on another response moves it; retract by tapping again; "+ Criterion" opens the PromptDialog (sheet on phone), custom chip shows an × (always visible on touch).
  6. After ≥ 1 vote with all responses final: Winner card pops in with deltas, the winning column gets the accent border + Winner badge, tabs show the trophy, Comparison section renders stacked rows on phone / a table on desktop with best-per-column highlighting.
  7. Blind: toggle before running → columns show Model A–D with letter avatars, shuffled order, no provider names; vote "Best answer" → flip reveal; "Reveal" also in the header and per column eye icon.
  8. History: phone rows (tap opens read-only with "stored · date" pill; long-press or … opens the ActionSheet with Open / Run again / Export MD / Export JSON / Share / Delete); desktop table with hover-revealed Run again + … menu; Delete asks for confirmation and removes the active session if open.
  9. Menu (…): Export Markdown / JSON downloads a file; Share… opens the sheet with the privacy warning → Create public link → URL, copy, open, views, Revoke → /share/arena/:id returns 404.
  10. /share/arena/:id: branded header, prompt bubble, Settings details, phone carousel with sticky tabs under the 56 px header, desktop grid, winner card + comparison, CTA; noindex.
  11. /app/arena/scoreboard: chip filters scroll on phone; ranking list with rank tile, provider icon, win-rate bar, stats; empty state copy; back arrow to the Arena.
  12. Deep links: /app/arena?models=openai/gpt-5.5,anthropic/claude-sonnet-5 preselects; /app/arena?session=arn_… opens read-only; URL is cleaned.
  13. Keyboard: ⌘↵ runs, Esc stops, ←/→ on the tab bar moves between models, Segmented is keyboard-navigable; all icon buttons have aria-labels; prefers-reduced-motion disables the flip/pop.