spb/zyquo-router Public MIT

1# Zyquo Router — API Reference23The exact public contract served on `http://localhost:<port>` (default port **8787**).4This document and the implementation are maintained together; the in-app Docs screen5renders this file. OpenAI-compatible: point any OpenAI SDK at6`base_url = http://localhost:8787/v1`.78```python9from openai import OpenAI10client = OpenAI(base_url="http://localhost:8787/v1", api_key="zyquo-sk-…")11r = client.chat.completions.create(12    model="anthropic/claude-sonnet-4-5",     # any provider/model from GET /v1/models13    messages=[{"role": "user", "content": "Hello"}],14)15```1617---1819## Authentication2021- **Localhost (default bind):** authentication is optional. With no local API keys22  configured, requests need no `Authorization` header.23- **When local keys exist** (created in *Keys → Local API Keys*), every endpoint except24  `GET /health` requires `Authorization: Bearer zyquo-sk-…`. Unknown, revoked, or25  malformed tokens → `401` (`authentication_error`, code `invalid_api_key`).26- **LAN bind (`0.0.0.0`)** refuses to start without at least one enabled local key.27- Keys may carry a **model allow-list**: requests for other models → `403`28  (`permission_error`, code `model_not_allowed`).29- Provider API keys (OpenAI, Anthropic, …) live only in the encrypted vault on the Mac30  running the router. **No endpoint ever returns them**, and they never appear in logs31  or error messages.3233## Model naming3435- Canonical IDs are **namespaced**: `provider/model-id` — e.g. `openai/gpt-5.2`,36  `anthropic/claude-sonnet-4-5`, `deepseek/deepseek-chat`,37  `deepinfra/meta-llama/Llama-4-Maverick`. The segment before the first `/` must be a38  provider (`openai, anthropic, xai, mistral, gemini, qwen, deepseek, kimi, perplexity,39  together, deepinfra, cerebras`); model IDs may themselves contain `/`.40- **Bare upstream IDs** are accepted when unambiguous across providers41  (`deepseek-chat` works; an ID hosted by two providers → `404` with the namespaced42  candidates listed).43- **Aliases** (user-defined, e.g. `fast`) resolve before anything else.44- Disabled models 404 exactly like unknown ones.45- Responses always echo the **namespaced ID** in `model` — including when a fallback46  chain routed the request to a different model than requested (honest reporting).4748---4950## POST /v1/chat/completions5152The full current OpenAI request schema is accepted. Highlights and router-specific53behavior:5455| Field | Behavior |56|---|---|57| `model` | Namespaced ID, unambiguous bare ID, or alias. Required. |58| `messages` | All roles: `system`, `developer`, `user`, `assistant` (incl. `tool_calls`), `tool`. Content may be a string or content-part array. |59| Image parts | `{"type":"image_url","image_url":{"url":…}}` with data-URI base64 or remote URL. Vision-capable models only (else `400`). **Gemini: data-URI only** — the router does not fetch remote URLs for Gemini (`400` with explanation). |60| `max_tokens` / `max_completion_tokens` | Both accepted; `max_completion_tokens` wins. Sent upstream under the name each provider documents. Anthropic requires one — when omitted the router fills the model's catalog max output (fallback 4096). |61| `temperature`, `top_p`, `stop`, `seed`, `frequency_penalty`, `presence_penalty`, `user`, `logprobs`… | Translated, clamped, renamed, or stripped per provider (see *Provider notes*). Unsupported params are stripped silently — never a 400 for asking. |62| `n` | **Only `n=1`.** `n>1` → `400` (`invalid_request_error`, param `n`). |63| `tools`, `tool_choice`, `parallel_tool_calls` | Full function calling on tool-capable models (else `400`). Translated natively for Anthropic (`input_schema`, `tool_choice` auto/any/none/tool, `disable_parallel_tool_use`) and Gemini (`functionDeclarations`, `functionCallingConfig`). |64| `response_format` | `json_object` / `json_schema` forwarded where supported (Gemini: `responseMimeType`/`responseJsonSchema`; OpenAI-compatible: pass-through). Anthropic: best-effort via system steering (documented limitation). |65| `reasoning_effort` | OpenAI-standard values, translated per provider (Anthropic `thinking` budget 1024/8192/24576; Gemini `thinkingConfig`; pass-through where native). Stripped on non-reasoning models. |66| `stream` | SSE streaming (below). |67| `stream_options.include_usage` | Adds the final usage chunk (empty `choices`). |68| **Unknown keys** | **Passed through** to OpenAI-compatible upstreams — use provider extras like Perplexity `search_domain_filter`, Qwen `enable_thinking`, Together `top_k`, Anthropic `thinking` (extra body). |6970### Non-streaming response7172Spec-exact `chat.completion`:7374```json75{76  "id": "chatcmpl-5f9d174703e1",77  "object": "chat.completion",78  "created": 1785462056,79  "model": "anthropic/claude-haiku-4-5-20251001",80  "choices": [{81    "index": 0,82    "message": { "role": "assistant", "content": "OK" },83    "finish_reason": "stop"84  }],85  "usage": { "prompt_tokens": 12, "completion_tokens": 4, "total_tokens": 16 }86}87```8889- `finish_reason` ∈ `stop | length | tool_calls | content_filter` (every upstream value90  is normalized into this set; e.g. Together `eos`→`stop`, Anthropic `tool_use`→91  `tool_calls`, Gemini `SAFETY`→`content_filter`).92- `usage` comes from the upstream when reported. When an upstream reports none, the93  router **estimates** (~4 chars/token) and flags it:94  `"usage": { …, "x_zyquo": {"usage_estimated": true} }`.95- Cached prompt tokens land in `usage.prompt_tokens_details.cached_tokens`; reasoning96  tokens in `usage.completion_tokens_details.reasoning_tokens`.9798### Reasoning output99100Reasoning/thinking text is normalized to **`reasoning_content`** — a sibling of101`content` on the message (non-streaming) and the delta (streaming) — the DeepSeek102convention that most tooling already understands. Sources: DeepSeek/Qwen/Kimi/xAI103native field, Anthropic `thinking` blocks, Gemini `thought` parts, Mistral Magistral104thinking chunks, Perplexity `<think>` tags (extracted).105106### Streaming (SSE)107108`Content-Type: text/event-stream`; each event is `data: <chat.completion.chunk JSON>`,109terminated by `data: [DONE]`. Byte-exact chunk discipline:1101111. First chunk: role delta `{"delta":{"role":"assistant","content":""}}`.1122. Content deltas `{"delta":{"content":"…"}}`; reasoning deltas113   `{"delta":{"reasoning_content":"…"}}`.1143. Tool calls stream as OpenAI deltas: first frame carries115   `{"index":N,"id":"…","type":"function","function":{"name":"…","arguments":""}}`,116   subsequent frames only `{"index":N,"function":{"arguments":"<fragment>"}}`.117   (Gemini delivers arguments whole; the router emits announce + one full fragment.)1184. Finish chunk: empty delta + `"finish_reason"`.1195. If `stream_options.include_usage`: one usage chunk with **empty `choices` array**.1206. `data: [DONE]`.121122The `id`/`created`/`model` envelope is constant across a stream. Comment lines123(`: keep-alive`) may appear and must be ignored (all OpenAI SDKs do).124125**Mid-stream upstream failure:** the router cannot change the HTTP status after bytes126are sent; it emits one error frame `data: {"error":{"message":…,"type":…,"code":…}}`127followed by `data: [DONE]`, and never retries after the first forwarded byte.128129**Client disconnect** cancels the upstream call immediately.130131### Retries & fallbacks132133- Transient upstream failures (429, 5xx, network) retry with exponential backoff +134  jitter (max 3 attempts), honoring `Retry-After` — only before any byte has been135  forwarded.136- User-configured **fallback chains** try the next model in the chain on upstream137  failure (rate limit, 5xx, network, missing/invalid provider key — never on request138  errors). The response `model` field reports the model that actually answered.139140### Errors141142Always OpenAI-shaped: `{"error": {"message", "type", "param", "code"}}`.143144| Status | When | type / code |145|---|---|---|146| 400 | Malformed body, missing `model`/`messages`, `n>1`, capability mismatch (tools/vision on unsupporting model), upstream rejected request, Gemini prompt block | `invalid_request_error` |147| 401 | Missing/invalid/revoked local key → `invalid_api_key` · provider key missing → `missing_provider_key` · provider key rejected upstream → `invalid_provider_key` | `authentication_error` |148| 403 | Local key not allowed for this model | `permission_error` / `model_not_allowed` |149| 404 | Unknown/disabled/ambiguous model (`model_not_found`), unknown route | `invalid_request_error` |150| 413 | Body over the request size limit (default 32 MB) | `invalid_request_error` |151| 429 | Upstream rate limit (with `Retry-After` when known) | `rate_limit_error` / `upstream_rate_limited` |152| 502 | Upstream 5xx / unreachable / malformed upstream response | `api_error` / `upstream_error` |153| 504 | Upstream timeout | `api_error` / `upstream_timeout` |154155Provider payload shapes and key material never leak into errors.156157---158159## GET /v1/models160161OpenAI list shape over the full enabled catalog (all providers, namespaced IDs), with162router metadata under the `x_zyquo` extension key:163164```json165{166  "object": "list",167  "data": [{168    "id": "anthropic/claude-sonnet-4-5",169    "object": "model",170    "created": 1785461333,171    "owned_by": "anthropic",172    "x_zyquo": {173      "display_name": "Claude Sonnet 4.5",174      "context_window": 200000, "max_output_tokens": 64000,175      "vision": true, "tools": true, "reasoning": true,176      "input_per_mtok": 3.0, "output_per_mtok": 15.0177    }178  }]179}180```181182`GET /v1/models/{id}` returns a single entry (namespaced, bare, or alias `id`;183URL-encode if needed — IDs containing `/` also work raw).184185## GET /health186187Unauthenticated readiness probe:188189```json190{ "status": "ok", "version": "1.0.0", "uptime": 42, "models": 170 }191```192193---194195## Provider notes (translation table summary)196197| Provider | Upstream API | Notes |198|---|---|---|199| `openai` | native chat/completions | Reference; pass-through. |200| `anthropic` | Messages API (translated) | `max_tokens` synthesized when omitted; `temperature` clamped to ≤1; system/developer → top-level `system`; consecutive turns merged; tool results become `tool_result` blocks; `stop_reason` mapped; usage includes cache reads in `prompt_tokens`. `response_format` best-effort. Extra body `thinking` / `top_k` forwarded. |201| `gemini` | native generateContent (translated) | Roles renamed (`assistant`→`model`); `tool` messages → `functionResponse` (object-wrapped, name resolved from `tool_call_id`); `STOP`+functionCall → `finish_reason:"tool_calls"`; images must be data URIs; `n>1` unsupported; blocked prompts → 400 naming the reason. |202| `xai` | compat | Reasoning models reject `presence_penalty`/`frequency_penalty`/`stop` — stripped. `search_parameters` pass-through. |203| `mistral` | compat | `seed`→`random_seed`; `logit_bias`/`user`/`logprobs` stripped. Magistral thinking arrays flattened into `reasoning_content`. |204| `qwen` (DashScope intl) | compat | `enable_thinking`/`thinking_budget` pass-through; streaming-only models transparently aggregated for non-streaming clients. |205| `deepseek` | compat | `reasoning_content` passed through natively; cache-hit tokens → `cached_tokens`; assistant `reasoning_content` echoed back **only** in tool loops (stripped otherwise). |206| `kimi` (Moonshot) | compat | `temperature` clamped to [0,1]. |207| `perplexity` | compat | `citations`/`search_results` pass through verbatim; `<think>` extracted to `reasoning_content`; search params (`search_domain_filter`, `web_search_options`, …) pass-through. No function calling. |208| `together` | compat | `finish_reason:"eos"`→`stop`; `top_k`/`min_p`/`repetition_penalty` pass-through. |209| `deepinfra` | compat | `logit_bias` stripped; usage `estimated_cost` used for cost metering. |210| `cerebras` | compat | `max_completion_tokens` naming; base64-only images. |211212## Copy-paste snippets213214```bash215curl http://localhost:8787/v1/chat/completions \216  -H "Content-Type: application/json" \217  -H "Authorization: Bearer zyquo-sk-…" \218  -d '{"model":"deepseek/deepseek-chat","messages":[{"role":"user","content":"Hi"}],"stream":true}'219```220221```javascript222import OpenAI from "openai";223const client = new OpenAI({ baseURL: "http://localhost:8787/v1", apiKey: "zyquo-sk-…" });224const stream = await client.chat.completions.create({225  model: "gemini/gemini-2.5-flash",226  messages: [{ role: "user", content: "Hi" }],227  stream: true,228});229for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");230```231232```python233# LangChain234from langchain_openai import ChatOpenAI235llm = ChatOpenAI(base_url="http://localhost:8787/v1", api_key="zyquo-sk-…",236                 model="anthropic/claude-sonnet-4-5")237```238