# PolyLLM — final production audit (v0.2.0, 2026-09-08)

# Build

pnpm typecheck — 0 errors. pnpm lint — 0 errors. pnpm build (on M3U96a, Next 16 production build) — ok.
pnpm test — vitest: unit (encryption, parameter normalization, model normalization for 3 providers, pricing, error normalization, stream accumulator, titles) + integration (registry sync with a real key against Postgres, encrypted key vault).

# Security

No secrets committed: .env, .env.* (except .env.example) and deploy/*.mld.json are gitignored; the repo was grepped for key prefixes before the first commit.
User provider keys: AES-256-GCM, data key derived with HKDF from API_KEY_ENCRYPTION_SECRET, AAD = user id + provider; verified in Postgres during E2E (encrypted_key starts with v1. and never contains the key); browser DOM never contains the key; API returns keyHint only.
Passwords: Argon2id (19 MiB, t=2). Sessions: HttpOnly, SameSite, Secure in production, 30-day expiry, cookie cache 5 min; revoke one / others / all.
Email verification mandatory (unverified login refused — verified by E2E). Password reset tokens expire in 1 h; sessions revoked on reset.
Rate limits: Better Auth (sign-in 8/min, sign-up 4/min, reset 4/min, resend 3/min…) + per-user limits on chat, arena, key save/validate, model sync, uploads, search, sharing.
Headers: CSP, HSTS, X-Frame-Options DENY, nosniff, Referrer-Policy, Permissions-Policy (verified on https://www.polyllm.io).
Logging: structured JSON, key-shaped strings redacted, sensitive field names redacted, prompts never logged.
Audit log per user (account, login, key added/replaced/removed, password/email events).

# Providers (documentation re-audited 2026-09-08; live test matrix `docs/provider-test-matrix.md`)

Nine providers, each with a dated research report (docs/provider-research/<id>.md), real probes (research/<id>/) and an adapter whose parameter translation only sends what the model accepts. The live matrix is all ✅ for 13 capabilities × 9 providers (auth, model list, text, streaming, system prompt, vision, tools, structured output, reasoning, token usage, invalid key, error normalization, long response).

Provider	SDK / endpoint	Models listed	Highlights verified
OpenAI	`openai` 7.10, Responses API	65 (+ alias gpt-5.6)	encrypted reasoning replay, sampling only at effort none, no seed/stop
Anthropic	`@anthropic-ai/sdk` 0.124, Messages API	11	live capabilities, adaptive thinking, Haiku budget mode
Google Gemini	`@google/genai` 2.21	18 chat	thinkingLevel/budget per family, thought signatures, free-tier quotas
xAI	`openai` @ api.x.ai	7	reasoning_content, live pricing, 400 on bad key
Mistral AI	`openai` @ api.mistral.ai (translated bodies)	28 ids / 10 billing models	listing capabilities, `reasoning_effort` none/high, thinking content arrays, `document_url` PDFs, 422 strictness
DeepSeek	`openai` @ api.deepseek.com	3	thinking on by default, `reasoning_effort` low/high/max, json_object only, cache-hit tokens
Kimi (Moonshot AI)	`openai` @ api.moonshot.ai	4	`/models/{id}` capabilities, fixed temperature/top_p, reasoning_content replay
OpenRouter	`openai` @ openrouter.ai	323 after filters	`supported_parameters` gating, `usage.cost`, reasoning effort/budget, web plugin, upstream provider surfaced
Cerebras	`openai` @ api.cerebras.ai	3 (gemma retired)	strict validation whitelist, `reasoning_effort`, tools+response_format rule, strict schema

# Email (Resend)

Sender: PolyLLM <polyllm@mail.spboucher.ai> (the key is not authorized for spboucher.ai or polyllm.io; mail.spboucher.ai verified — probed).
Verification email sent in production (Resend ids logged), welcome email after verification, reset/changed/email-change/deletion/new-login templates (HTML + plain text).

# UI

Desktop and mobile (390 px) screenshots of every screen in light and dark: qa/out/ (48 screens) — no horizontal overflow, no console errors (Playwright e2e/visual.spec.ts).
Real browser workflow (e2e/full-flow.spec.ts, 10 steps): sign up → verification refused → verify link → add 4 real keys → select model → real stream (Claude Haiku 4.5) → persistence after reload + second provider (GPT-5.4 nano) → capability-driven config → usage → change password → logout → login. All green.

# Production

Node M3U96a, PM2 polyllm-web (Next, :8240) + polyllm-ngrok (www.polyllm.io), registry entry in mld.
https://www.polyllm.io/api/health/ready → {"status":"ok","database":"ok"}; TLS via ngrok; HSTS.
Startup model sync on the node: 65 + 11 + 18 + 7 + 28 + 3 + 4 + 323 + 3 models (nine providers).
Production login tested and a real provider request streamed through ngrok (e2e/prod-smoke.spec.ts, account spbou4+polyllm-test@icloud.com).
Backups: deploy/backup/install.sh (daily pg_dump 03:45, 14-day retention) — to run on the node.

# Known limitations (honest)

Built-in tools are limited to calculator / clock / random (side-effect free); provider-native web search and code execution are wired where supported.
Audio input/output and image generation are catalogued as capabilities but not exposed in the chat UI (text, images, PDF, text files only).
Gemini web-search grounding and code execution were not live-tested (free-tier quota); Anthropic/OpenAI/xAI/OpenRouter web search paths are implemented from the audited docs and covered by the research probes, not by the matrix. Kimi $web_search and Mistral Conversations-API tools are not exposed (round trips not stable / different API).
OpenRouter reasoning_details signatures are not replayed yet (works without; docs recommend replay).
Conversation search is ILIKE-based (adequate for personal volumes); no full-text index yet.
English only (architecture ready for locales).