SPB Git forge
spboucher.ai ↗

spb/polyllm

Public
15commits 1branches 0releases
2.2 MBsize
maindefault branch
13 days agolast push
TypeScript 97.4% SQL 1% JavaScript 0.9% CSS 0.6%
5.4 KB
View source Raw Blame History

# PolyLLM — repository guide

PolyLLM (www.polyllm.io) is a BYOK multi-provider AI workspace: OpenAI, Anthropic, Gemini, xAI, Mistral, DeepSeek, Kimi, OpenRouter and Cerebras behind one normalized adapter layer (createOpenAICompatAdapter for the OpenAI-shaped ones), real streaming, capability-driven configuration, Arena comparisons, usage/cost tracking. The original product brief lives in docs/SPEC-original.md; the UI conventions in docs/UI-BRIEF.md.

# Non-negotiables

  • Research before provider changes. Read docs/provider-research/<provider>.md, re-fetch the official docs, probe the real API (pnpm providers:matrix), then update src/lib/ai/providers/<provider>/catalog*.ts. Never assume model ids, parameters, event names or limits from memory. Bump Last documentation audit.
  • Provider quirks stay inside src/lib/ai/providers/*. UI and services only see PolyModel, UnifiedChatRequest, UnifiedStreamEvent, PolyProviderError.
  • Capabilities drive the UI. Never branch on a model id in components; use model.capabilities / model.parameters. Never send a parameter the model does not accept (filterSettings() + adapter translation).
  • Keys. User keys are encrypted (src/lib/crypto/keys.ts) and decrypted only in getDecryptedKey() right before a request. Never log, return or serialize them; PublicConnection exposes keyHint only. Owner keys in .env are for the registry sync, admin diagnostics and tests — never for user requests.
  • No fake integrations. A capability is ✅ only after a real request passed (docs/provider-test-matrix.md).
  • Secrets only in .env (gitignored) and the mld manifest on M1M32; .env.example documents the variables.

# 1.0 architecture notes (2026-09-11)

  • Read docs/UPGRADE-PLAN.md (shared primitives, ownership map) and docs/upgrade-notes/*.md (per-area APIs, contracts) before touching chat, models, Arena, projects/library/prompts, usage/providers/endpoints, marketing or search/share.
  • Mobile first. Phones use BottomSheet / ResponsiveDialog / ActionSheet (components/ui/sheet.tsx), useLongPress, .snap-row carousels, .tap hit areas, .h-app (100dvh − keyboard) — never window.prompt/confirm, never 100vh.
  • Schema additions go in src/db/schema-workspace.ts (or a new schema-*.ts re-exported from src/db/index.ts); pnpm db:generate --name x && pnpm db:migrate (db:push needs a TTY). Never edit shipped migrations.
  • Cross-area contracts: prompt insert event polyllm:insert-prompt (components/prompts/insert.ts), pending attachments (components/library/use-in-chat.ts), palette events polyllm:open-attach / polyllm:switch-model, share via openShareSheet(), exports via exportConversation(). Custom endpoints resolve in prepareTurn / runArenaModel through resolveCustomEndpoint() (lib/endpoints/service.ts); their models are keyed custom/<endpointId>:<modelId>.
  • QA: one dev server only (.next conflicts); node qa/responsive-qa.mjs + node qa/flows.mjs before deploying.
  • Brand assets are generated: edit public/icon.svg / public/brand/*.svg, then scripts/brand-assets.sh.

# Commands

pnpm dev · pnpm typecheck · pnpm lint · pnpm test (vitest) · pnpm test:e2e (Playwright; dev server with EMAIL_DRY_RUN=1 EMAIL_DRY_RUN_PRINT=1) · pnpm db:generate / pnpm db:migrate · pnpm models:sync [provider] · pnpm providers:matrix [provider].

# Deploy

mld from the laptop (see docs/deployment-cluster.md): mld stage . polyllm && mld deploy polyllm --node M3U96a. Manifest with secrets: M1M32:~/dispatch/apps/polyllm.json (local gitignored copy deploy/polyllm.mld.json).

# Gotchas

  • React Compiler lint: no Date.now()/impure calls in render, no setState synchronously in effects without an explicit eslint-disable, memoize derived lists.
  • import "server-only" modules are used by CLI scripts through scripts/hooks.mjs (tsx --import) and by vitest via an alias.
  • Anthropic: adaptive thinking may legitimately produce zero thinking tokens on easy prompts; Haiku 4.5 needs budget_tokens.
  • Gemini: free-tier keys have tiny per-model daily quotas (429 limit: 20) and some models are paid-tier only (429 limit: 0); thought signatures must be replayed on function-call parts.
  • xAI: bad keys return HTTP 400 (Incorrect API key), frequency/presence_penalty always 400, web search only via /v1/responses.
  • OpenAI: Responses API has no seed/stop; sampling params only when reasoning.effort === "none" on reasoning models.
  • Mistral: strict 422 on unknown keys (random_seed, max_tokens, no developer role); reasoning is binary reasoning_effort none|high and streams as thinking content-array chunks; magistral-* are aliases.
  • DeepSeek: thinking ON by default (thinking:{type} + reasoning_effort low|high|max); only json_object; max_tokens caps reasoning + answer; images silently dropped except on the vision model.
  • Kimi: temperature must be 1 / top_p 0.95 (400 otherwise); /v1/models/{id} gives capabilities; reasoning_content replayed; json_schema not enforced (json_object + prompt).
  • OpenRouter: unsupported params are silently dropped — gate on supported_parameters; usage.cost is exact; :batch ids 404 on chat; finish chunk arrives twice.
  • Cerebras: unknown params → 400 (whitelist); reasoning via reasoning_effort only; tools+response_format → 400 on gpt-oss; strict schema needs additionalProperties:false.