SPB Git

spb/zyquo-router Public MIT

One local endpoint, every AI provider — a private OpenAI-compatible LLM gateway for your Mac (170 models, 12 providers).

Swift 95.7% Python 2.3% Shell 1.2% Makefile 0.9%
21.8 KB

# Zyquo Router — PLAN

Phase-by-phase execution log. One phase at a time, checkpoint at each gate.


# Phase 0 — Mandatory research + Zyquo Cloud study

  • 0.A.1 OpenAI API spec documented exhaustively (chat/completions request + response + SSE streaming + models + errors; /v1/responses decision)
  • 0.A.2 Reference gateways studied (LiteLLM, OpenRouter, Ollama/LM Studio/vLLM compat layers) with concrete patterns extracted
  • 0.A.3 Translation matrices: Anthropic Messages ⇄ OpenAI, Gemini generateContent ⇄ OpenAI, per-provider deviation table for OpenAI-compatible upstreams, reasoning-content decision
  • 0.A.4 Swift HTTP serving approach evaluated and chosen with rationale (SwiftNIO vs Network.framework vs Hummingbird/Vapor), incl. SSE, cancellation, CORS, timeouts
  • 0.A.5 Gateway concerns: logging/redaction, usage extraction, cost calc, rate limiting, retries/backoff, fallback chains, health
  • docs/ROUTER-RESEARCH.md complete
  • 0.B.1 Zyquo Cloud provider layer studied (base URLs, auth, Codable models, OpenAICompatibleClient, AnthropicClient, GeminiClient, SSE parsing)
  • 0.B.2 Complete model catalog + capabilities + pricing extracted
  • 0.B.3 SecureKeyStore vault design (AES-256-GCM, NO Keychain) documented for reuse
  • 0.B.4 Provider streaming quirks documented
  • docs/PROVIDER-REUSE.md complete

Phase gate: PASSED (2026-07-30).

Phase 0 summary: docs/ROUTER-RESEARCH.md (1,646 lines) compiled from four intensive web-research tracks: full current OpenAI chat/completions + SSE contract, gateway patterns from LiteLLM/OpenRouter/Ollama/LM Studio/vLLM, exact bidirectional Anthropic and Gemini translation matrices plus a 9-provider OpenAI-compat deviation table, SwiftNIO chosen for serving, and ops (redaction, usage/cost, rate limits, retries, fallbacks). 12 binding decisions (D1–D12) recorded up front. docs/PROVIDER-REUSE.md (788 lines) documents Zyquo Cloud's 12 providers / 170-model catalog / vault.zq AES-256-GCM SecureKeyStore (no Keychain) and a file-by-file port plan; key caveats: Cloud has no native GeminiClient (Router translates Gemini natively per D10) and no tool-calling in Cloud's wire types (Router extends them). All 12 real provider keys smoke-tested OK (HTTP 200).

# Phase 1 — Project setup (SPM, no Xcode IDE)

  • 1.1 Package.swift: executable target ZyquoRouter, macOS 13+, deps = SwiftNIO (NIOCore/NIOPosix/NIOHTTP1/NIOExtras) + swift-markdown only
  • 1.2 Makefile: make dev (debug build + ad-hoc-signed Zyquo Router.app + run), make build, make app (release universal), make clean; release/notarize targets stubbed for Phase 8
  • 1.3 Resources/Info.plist: CFBundleDisplayName = Zyquo Router, com.zyquo.router, LSMinimumSystemVersion 13.0, NSHighResolutionCapable, LSApplicationCategoryType developer-tools
  • 1.4 Entry point: @main SwiftUI App + AppDelegate activation from terminal; placeholder window
  • 1.5 swift build clean, zero warnings; make dev launches the app window
  • 1.6 Mandatory file headers on every code file created

Phase gate: PASSED (2026-07-30).

Phase 1 summary: SPM executable ZyquoRouter (macOS 13+, deps: SwiftNIO 2 + NIOExtras + swift-markdown) builds with zero warnings on the Xcode 6.3.3 toolchain — no SDKROOT pin needed here, unlike Cloud's CLT-only machine (noted in the Makefile). make dev assembles dist/Zyquo Router.app (com.zyquo.router, developer-tools category, ad-hoc signed, plist lints OK); the placeholder SwiftUI window launches from terminal, activates, and quits cleanly. Test target scaffolded and green. Family conventions ported from Zyquo Cloud's Makefile/write-info-plist.sh, incl. the Developer ID identity + notary profile constants for Phase 8.

# Phase 2 — Architecture + server skeleton

  • 2.1 Port from Zyquo Cloud (headers → Zyquo Router, type names kept): ProviderID, AIModel, Message+ChatParameters, ProviderProtocol, ProviderRegistry, OpenAICompatibleClient, AnthropicClient, StreamingService, SecureKeyStore, ModelCatalog, ModelCatalogData (170 models, byte-identical), PersistenceService, SSE + vault tests
  • 2.2 Vault decision recorded: own vault at ~/Library/Application Support/ZyquoRouter/vault.zq, HKDF info "ZyquoRouter.vault.v1"; User-Agent ZyquoRouter/1.0
  • 2.3 Server/: HTTPServer (NIOAsyncChannel bootstrap, bind host/port, EADDRINUSE detection, graceful shutdown), Routes (GET /health, GET /v1/models), SSEWriter skeleton, CORS, AuthMiddleware skeleton
  • 2.4 Router/: RequestRouter (namespaced provider/model + unambiguous bare ID resolution; alias/fallback hooks), RetryPolicy + UsageMeter stubs
  • 2.5 App owns server lifecycle (start/stop; placeholder UI button + port field)
  • 2.6 Phase gate: server starts on chosen port; /health ok; /v1/models returns full namespaced catalog (spec shape); port-in-use → clean error; graceful stop drains
  • 2.7 swift build zero warnings; headers sweep

Phase gate: PASSED (2026-07-30).

Phase 2 summary: Provider layer ported from Zyquo Cloud (12 providers, 170-model catalog byte-identical, SecureKeyStore with own vault ZyquoRouter/vault.zq + HKDF info ZyquoRouter.vault.v1, StreamingService with ZyquoRouter/1.0 UA); Cloud's SSE + vault tests ported and green. New: HTTPServer on NIOAsyncChannel structured concurrency (keep-alive, 32 MB body cap, EADDRINUSE → "Port N in use — try N+1"), Routes (/health, /v1/models, /v1/models/{id} incl. IDs containing "/"), SSEWriter, CORS (preflight 204), AuthMiddleware (hashed zyquo-sk-… gate, open when no keys), RequestRouter (namespaced/bare/alias/disabled resolution), RetryPolicy, UsageMeter, APIKeyRecord, ServerController + minimal Start/Stop UI, and a headless --serve [port] CLI mode for scripted verification. Gate verified live with curl (170 models, spec shapes, 404s, CORS) and by integration tests (graceful stop rebinds immediately; port-in-use typed error). 23 tests green; zero warnings; headers swept.

# Phase 3 — Standardized API (the core)

  • 3.1 OpenAI wire layer in Translate/OpenAINormalizer.swift: parsed canonical request (raw JSON + typed fields, unknown params preserved per D5), response/chunk emission helpers
  • 3.2 Translate/CompatAdjuster.swift: per-provider param strip/rename/clamp table (seeded from ParameterSupport + research §3.3) for the 10 OpenAI-compatible upstreams; response/chunk quirk normalization (Together eos/text, Mistral ThinkChunk, Perplexity citations, model echo → namespaced ID)
  • 3.3 Router/UpstreamCall.swift: executes the upstream request (auth per provider), non-streaming + SSE; pre-first-byte retries via RetryPolicy; fallback chains; client-disconnect cancellation
  • 3.4 Translate/AnthropicTranslator.swift: request (system extraction, tools, tool_choice, max_tokens required, temp clamp), response (stop_reason/usage mapping, tool_use → tool_calls), SSE state machine → byte-exact OpenAI chunks (text/tool/thinking deltas)
  • 3.5 Translate/GeminiTranslator.swift (net-new, native API per D10): contents/parts, systemInstruction, generationConfig, tools/functionDeclarations, finishReason table, streamGenerateContent?alt=sse → OpenAI chunks
  • 3.6 POST /v1/chat/completions route: parse → resolve (404/ambiguous) → capability checks → upstream → spec-exact response/stream; stream_options.include_usage; reasoning_content normalization (D6); usage estimation flagged when upstream omits (D9); OpenAI-format error mapping (D7)
  • 3.7 Access control: local zyquo-sk-… keys enforced on chat route (per-key model allow-list), LAN bind requires ≥1 key
  • 3.8 Fixture unit tests: Anthropic events → chunks, Gemini → chunks, CompatAdjuster tables, SSE edge cases
  • 3.9 docs/API.md written as the exact public contract
  • 3.10 PHASE GATE: streaming + non-streaming verified with curl AND official OpenAI Python SDK against one OpenAI-compatible provider, Anthropic, and Gemini (real keys, through the router only)

Phase gate: PASSED (2026-07-30).

Phase 3 summary: POST /v1/chat/completions is live and spec-exact. New layers: ChatCompletionRequest (typed parse + raw pass-through per D5), CompatAdjuster (per-provider strip/rename/clamp + quirk normalization: Together eos, Mistral thinking arrays, Perplexity <think> + citations, DeepSeek cache tokens, usage-chunk swallowing), AnthropicTranslator (full request map incl. turn merging/tools/thinking budget; SSE state machine → byte-exact chunks), GeminiTranslator (native generateContent per D10 incl. functionResponse object-wrapping, STOP+functionCall override, synthesized role chunk + router-added [DONE]), UpstreamCall (per-provider endpoints/auth + ProviderError→OpenAI wire mapping D7), ChatCompletionsRoute (capability gates, pre-first-byte retries D8, fallback chains, stream-aggregation for streaming-only models, estimated-and-flagged usage D9, UsageMeter recording). Local zyquo-sk keys enforced (401/403 paths verified live; /health stays open; LAN requires ≥1 key). --load-vault seeds the encrypted vault from env. docs/API.md written as the served contract. GATE: curl + OpenAI Python SDK 2.51 unmodified — non-streaming, streaming (role/finish/usage/[DONE] discipline), and streamed tool calls all PASS against xAI (compat), Anthropic (translated), Gemini (native). 31 unit/ fixture tests green; zero warnings; headers swept.

# Phase 4 — Design system & UI

  • 4.1 DesignSystem/ZyquoTheme.swift: Router palette per spec (bg #FAFBFC, surface #FFFFFF, accent signal-cyan #0891B2 + graphite #374151, subtle #E5F5F9, text #191C1F/#697077/#9BA3AA, border #E4E8EB, status greens/ambers/reds, chart tokens); dark "ops room" derived; family type/spacing/radii/motion tokens; SF Mono where data lives
  • 4.2 AppearanceStore: Light/Dark/System + accents cyan (default), graphite, sky, emerald, violet, copper
  • 4.3 Shared components: status pill (● Running on :port / ○ Stopped), cards, section headers, mono copy fields, capability badges, hairlines
  • 4.4 App shell: 240pt translucent navigator (Dashboard/Models/Requests/Keys/Playground/Docs + footer gear + permanent status pill with Start/Stop), 1280×820 default / 1020×660 min
  • 4.5 Dashboard: hero server card (status, big Start/Stop, port field, bind selector + LAN warning, endpoint URL with Copy + curl/Python/JS snippets), live tiles row (requests, tokens, cost, errors, uptime), first-run onboarding card
  • 4.6 Models: searchable/filterable catalog, mono namespaced IDs with one-click copy, capability badges, context + pricing columns
  • 4.7 Keys: Provider Keys tab (masked fields, save/delete, Test with status dot + latency, vault-backed) + Local API Keys tab (create/reveal-once/revoke)
  • 4.8 Requests: table + detail structure with empty state (live data lands in Phase 6)
  • 4.9 Playground: model picker, composer, params, streaming toggle — calls the router's own endpoint
  • 4.10 Docs: in-app rendering of docs/API.md (bundled into the app)
  • 4.11 Settings window (720×540 tabs): Server + Appearance functional; Logging/Usage/Shortcuts/Advanced structured
  • 4.12 Zero warnings; headers; make dev visual pass over stopped/starting/running/port-conflict/no-keys states

Phase gate: PASSED (2026-07-30) — layout/structure level; the full every-state design quality gate re-runs at the end of Phase 6 once all features are live.

Phase 4 summary: ZyquoTheme carries the spec palette exactly (graphite-cyan light flagship, dark ops-room derived, provider hues, chart tokens, family type/spacing/radii/ motion; SF Mono for all data). AppearanceStore with cyan/graphite/sky/emerald/violet/ copper accents. Shell: 240pt translucent navigator (NSVisualEffectView sidebar), six sections, footer gear + permanent StatusPill with Start/Stop; ⌘1–6 Go menu, ⌘⇧C copy endpoint, ⌘R start/stop. Screens live: Dashboard (hero server card with port/bind/LAN warning, endpoint + curl/Python/JS copy-as snippets, five 1Hz tiles fed by UsageMeter, first-run onboarding card), Models (search/provider filter over the 170-model catalog, mono IDs + hover copy, badges, context/pricing columns), Requests (table over UsageRecords + empty state), Keys (Provider tab identical to Cloud UX incl. Test with latency; Local tab create/reveal-once/enable/revoke), Playground (calls own endpoint, streaming + pretty JSON), Docs (bundled docs/API.md renderer with copyable code blocks), Settings 6 tabs (Server/Appearance fully live). Verified visually via screenshots: stopped + running dashboard, Models, Keys, Docs all match the spec. 31 tests green, zero warnings, headers swept.

# Phase 5 — App icon

  • 5.1 Author both motif candidates in SVG (viewBox 1024, family superellipse): zyquo-router-hub.svg (Z-hub: three gradient lines fan to node endpoints + cyan port dot) and zyquo-router-switch.svg (Z over crossing signal paths with chevrons); spec colors: #FCFCFD bg, #0B0B14→#1C1C2A Z, #2B6BFF→#4C2BE0 base, cyan #22B8D4 accents
  • 5.2 Render at 16→1024, inspect, pick the best → assets/icon/zyquo-router.svg (source of truth)
  • 5.3 Simplified small-size variant (drop lines/nodes) for 16/32 px
  • 5.4 scripts/generate-icon.sh: rsvg-convert → AppIcon.iconset (16→1024 + @2x) → iconutil -c icns → Resources/AppIcon.icns; menu bar template icon (18/36 px mono Z-with-signal-dot) from the same source
  • 5.5 App bundle carries the icon; visual check in Dock/Finder

Phase gate: PASSED (2026-07-30).

Phase 5 summary: Both spec'd motifs authored on the authentic family superellipse (extracted from Cloud's icon): zyquo-router-hub.svg (Z-hub — chunky #0B0B14→#1C1C2A Z, #2B6BFF→#4C2BE0 parallelogram base, three cyan→indigo fan lines anchored into the Z's diagonal, glowing cyan port dot, three soft-3D node squares) and zyquo-router-switch.svg (crossing signal paths + chevrons behind the centered Z). Hub chosen — more distinctive, tells "one in, many out". zyquo-router.svg is the source of truth; zyquo-router-small.svg (Z + base only) feeds 16/32 px; zyquo-router-template.svg renders the mono menu-bar glyph. scripts/generate-icon.sh (rsvg-convert → iconset → iconutil) produces Resources/AppIcon.icns + MenuBarIcon(@2x).png; make icon wired, bundle carries the icns, and the icon verified in the Dock at small size.

# Phase 6 — Features

  • 6.1 RouterConfigStore (aliases, fallback chains, disabled models, favorites) persisted to router-config.json; snapshot feeds RequestRouter at Start; Models screen gets enable/disable toggles, favorites, alias editor, fallback-chain editor
  • 6.2 RequestLogStore (ring buffer, redacted-by-default bodies, per-session reveal) wired into the chat route with timing (TTFB/duration); Requests screen: live table, filters (provider/status/model), pause/clear, detail pane with pretty JSON + timing waterfall, ⌘F focuses filter, export logs
  • 6.3 Dashboard: requests/min sparkline (cyan), per-provider breakdown bar, active streams tile; latency in tiles
  • 6.4 Menu bar extra (first-class): template glyph, status + port, Start/Stop, req/min, today's cost, Copy endpoint URL; toggleable in Settings
  • 6.5 Playground: params (temperature/top_p/max_tokens/reasoning), side-by-side raw request/response JSON with copy-as-code
  • 6.6 Launch at login (SMAppService) + auto-start (done) + keep-serving (done); ⌘K command palette (start/stop, sections, copy endpoint, model search)
  • 6.7 Settings: Logging (retention/export), Usage (reset counters), Advanced (export/import config without keys)
  • 6.8 Phase 4 design quality gate re-run across all states; zero warnings; headers sweep

Phase gate: PASSED (2026-07-30).

Phase 6 summary: RouterConfigStore (aliases/chains/disabled/favorites → router-config.json, snapshotted into RequestRouter at Start, honored by headless --serve too); Models screen gained enable/disable switches (strikethrough + 404), favorite stars, an alias editor and a drag-reorderable fallback-chain editor with CHAIN badges. RequestLogStore (500-entry ring, bodies in memory only) wired through the chat route with upstream TTFB; Requests is now a live 0.5 Hz table with provider/status/model filters (⌘F), pause/clear/export, and a detail inspector (timing waterfall, redacted-by- default bodies with per-session reveal, error detail). Dashboard: requests/min sparkline (cyan token), per-provider breakdown bar + legend, active-streams tile. First-class MenuBarExtra (template Z glyph, status, Start/Stop, req/min + cost, copy endpoint, toggleable). Playground: temperature/max_tokens/reasoning controls + side-by-side REQUEST JSON / RAW RESPONSE panes with copy. ⌘K command palette (server, sections, copy endpoint, model-ID search/copy); ⌘R moved to a Server menu so it works from every screen (bug found in live pass). Settings: launch-at-login (SMAppService), menu-bar toggle, usage reset, config export/import (never keys). Live-verified end-to-end with real traffic: tiles, sparkline, breakdown, log rows + SSE badges all populate. 31 tests green; zero warnings; headers swept.

# Phase 7 — Verification with real keys

  • 7.1 scripts/verify.py harness (official OpenAI Python SDK against the router only): sweeps GET /v1/models, runs non-stream + stream per model (SDK-parsed chunk shapes), tools on tool-capable, vision on vision-capable, reasoning content on reasoning models; produces the matrix in docs/VERIFICATION.md
  • 7.2 Full matrix green (or every failure diagnosed + fixed; upstream-side outages/rate limits documented as such)
  • 7.3 Client compatibility: OpenAI Python SDK + OpenAI JS SDK + curl, both modes incl. streamed tool calls, unmodified
  • 7.4 Gateway behavior: port conflict; graceful shutdown with an active stream; client-disconnect cancels upstream; retry policy on 429; fallback chain reports actually-used model; local-key auth paths (valid/invalid/revoked); LAN refuses without key
  • 7.5 Security: provider keys never in any response/log/error; logs redacted by default; vault round-trips
  • 7.6 Fix-until-green loop complete; results committed

Phase gate: PASSED (2026-07-30).

Phase 7 summary: scripts/verify.py drove all 170 catalog models through the local endpoint with the official OpenAI Python SDK — final matrix in docs/VERIFICATION.md: 170/170 green (2 rows carry documented upstream-side limitations: Together refuses tool_choice=required on Qwen3.7-Plus; DeepInfra 405s tool requests on its Llama-4-Maverick deployment). The fix-until-green loop produced real router improvements: Claude 4.7+/5 adaptive thinking, reasoning_effort stripped for models that 400 on it (Grok 4.20/code), reasoning_effort:"none" unlock for gpt-5.6 tools, Mistral prompt_mode/none-high clamping, Perplexity .done normalization + synthesized role/finish discipline for deviant streams, missing object injection, and transparent stream-aggregation (now wired for all three wire formats) for models whose buffered endpoints are broken/too slow. Client compat: OpenAI Python + JS SDKs + curl, both modes incl. streamed tool calls, unmodified. Gateway behavior integration-tested against mock upstreams: client-disconnect cancels the upstream in <1s, fallback chains answer with the actually-used model, transient 429 retries, graceful shutdown ends active streams and rebinds; LAN-without-key refusal verified in the app; auth paths (open/valid/invalid/revoked/malformed) unit-tested. Security: all 12 key fragments absent from every response, error, server log, and the vault file (ciphertext only); log bodies redacted by default. 26 tests green; zero warnings; headers swept.

# Design polish pass (post-Phase 7, user-requested)

  • Rounder, more modern surfaces: ZyquoRadius 10/16/22 with .continuous squircle corners everywhere, cards with a whisper of depth, capsule Start/Stop with icon + tinted shadow
  • Docs screen redesigned: gradient hero card with live endpoint + copy, code blocks with mac-dots header bar / language chip / one-click copy, real bordered tables with alternating rows, accent-bar H2s, cyan bullet dots with proper hard-wrap continuation
  • README.md written (quickstart incl. pointing the OpenAI SDKs at the router, highlights table, build targets, docs index)
  • Light theme remains the flagship; all tokens unchanged except radii.

# Phase 8 — Signing & notarization

  • 8.1 Inspect ~/Desktop/other/OTHER/zyquo-term: Developer ID identity, Team ID, notarytool profile, entitlements — reuse exactly (never print/commit secrets)
  • 8.2 entitlements.plist with Hardened Runtime, minimal set; NOT sandboxed (developer tool serving localhost) — documented
  • 8.3 make release: universal (arm64+x86_64 lipo) → assemble → codesign --options runtime --timestamp → ditto zip → notarytool submit --wait → staple
  • 8.4 spctl -a -vv says "accepted, source=Notarized Developer ID"; stapler validate passes
  • 8.5 Optional signed DMG
  • 8.6 make dev (ad-hoc) still works for iteration

Phase gate: PASSED (2026-07-31).

Phase 8 summary: Reused the zyquo-term pipeline exactly: identity Developer ID Application: Simon-Pierre Boucher (3YM54G49SN), notarytool keychain profile MacLustr-Notarize, and the same minimal non-sandboxed entitlements (Hardened Runtime, allow-jit false; rationale documented in ZyquoRouter.entitlements — localhost listening needs no entitlement outside the sandbox). make release: universal binary (arm64 + x86_64 via lipo), assemble, codesign --force --options runtime --timestamp (nested executable first), ditto -c -knotarytool submit --wait (Accepted), stapler staple. Verified: spctl -a -vv = "accepted, source=Notarized Developer ID"; stapler validate passes. Bonus: dist/ZyquoRouter.dmg built, signed, notarized (Accepted) and stapled. make dev (ad-hoc) still works for iteration.


ALL PHASES COMPLETE — Definition of Done satisfied (2026-07-31).