# ROUTER-RESEARCH.md — Zyquo Router

How to build a production-quality local LLM gateway, correctly. Compiled from intensive
web research (2026-07-30) against current official documentation, SDK sources, and
reference gateway implementations. Sections: (1) the OpenAI API specification the router
implements, (2) how existing gateways do it, (3) translation matrices for non-OpenAI
upstreams, (4) HTTP serving in Swift, (5) gateway concerns. Sources cited inline.

## Binding decisions (executive summary)

| # | Decision | Rationale (detail in section) |
|---|----------|-------------------------------|
| D1 | Implement `POST /v1/chat/completions`, `GET /v1/models`, `GET /v1/models/{id}`, `GET /health`. **Do NOT expose `POST /v1/responses` in v1.** | No upstream except OpenAI speaks it; its statefulness conflicts with the local/private posture; cheap to add later (§1.6). |
| D2 | Accept both `max_tokens` and `max_completion_tokens`; treat `max_completion_tokens` as canonical. | OpenAI deprecated `max_tokens`; real clients send either (§1.1, §2.4). |
| D3 | **SwiftNIO directly** (NIOCore/NIOPosix/NIOHTTP1 + NIOExtras), `NIOAsyncChannel` structured-concurrency APIs. Hummingbird 2 is the documented runner-up. | SPM-clean, Apple-maintained, full control over SSE flush/backpressure/disconnect, Swift 6-ready (§4.1). |
| D4 | Model namespace `provider/model-id`; bare IDs accepted when unambiguous; user aliases; disabled models 404. | LiteLLM/OpenRouter convention; avoids collisions like `deepseek-chat` on multiple hosts (§2.1, §2.5). |
| D5 | Param policy: known-param translation table per provider (strip/rename/clamp), unknown keys **passed through** to the upstream body. | Matches vLLM/OpenRouter behavior; enables provider extras (Perplexity search, Qwen `enable_thinking`) without schema churn (§2.5, §3.3). |
| D6 | Reasoning output normalized to DeepSeek-style **`reasoning_content`** on message and delta, with optional `reasoning_details` for signature round-trips. | Most tooling already understands DeepSeek's convention (§3.4). |
| D7 | Errors always in OpenAI `{"error":{...}}` shape: upstream 401→401 "provider key invalid (<provider>)", 429→429 with Retry-After, timeout→504, other upstream→502. Never leak raw provider payloads or key material. | §1.5, §2.5. |
| D8 | Retries: exponential backoff + jitter on 429/5xx/timeouts, respect `Retry-After`, **never retry once the first streamed byte has been forwarded**. Fallback chains report the actually-used model in `model`. | §5.5–5.6. |
| D9 | Usage: upstream-first; estimated (and flagged via `x_zyquo.usage_estimated`) only when the upstream provides none. Cost computed from the catalog's per-model pricing incl. cached-token rates. | §5.2–5.3. |
| D10 | **Gemini is translated natively** (`generateContent`/`streamGenerateContent?alt=sse`), even though Zyquo Cloud reaches Gemini through its OpenAI-compat endpoint. The Phase 3 gate requires a structurally different third upstream, and native translation avoids the compat layer's gaps (strict tool schemas, thinking metadata). | §3.2; PROVIDER-REUSE §1. |
| D11 | Streaming contract is byte-exact per §1.3: role-delta first chunk, content/tool-argument deltas, finish_reason chunk, optional usage chunk (empty `choices`) only when `stream_options.include_usage`, then `data: [DONE]`. Upstream streams are read to true EOF. | §1.3, §2.4. |
| D12 | Security: bind 127.0.0.1 by default; 0.0.0.0 opt-in forces ≥1 local API key (`zyquo-sk-…`, hashed at rest); logs redacted by default; provider keys never serialized into any response, log, or error. | §4.2–4.3, §5.1. |

## 1. The OpenAI API specification

> Research date: 2026-07-30. Primary sources: the OpenAI API reference (https://platform.openai.com/docs/api-reference/chat, mirrored at https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create), the streaming-events reference (https://developers.openai.com/api/reference/resources/chat/subresources/completions/streaming-events), and the official SDK type definitions, which are generated from OpenAI's OpenAPI spec and are therefore authoritative for the wire format (https://github.com/openai/openai-python/tree/main/src/openai/types/chat, https://github.com/openai/openai-node). This section is the implementation contract for Zyquo Router's public surface: **whatever the router serves must match this, byte-shape for byte-shape.**

---

### 1.1 `POST /v1/chat/completions` — request schema

Headers: `Authorization: Bearer <key>`, `Content-Type: application/json`. Only `model` and `messages` are required; every other parameter is optional and, when omitted, must be treated as "provider default" (the router must NOT inject its own defaults into upstream calls).

#### 1.1.1 `model` (string, required)

Model ID, e.g. `"gpt-4o"`. For Zyquo Router this is the namespaced `provider/model-id`, an unambiguous bare ID, or an alias. The response must echo a model string back (the router echoes the namespaced ID actually used).

#### 1.1.2 `messages` (array, required)

Ordered conversation. Each element is an object with a `role` and role-specific fields. Current roles (per `ChatCompletionMessageParam` in openai-python, https://github.com/openai/openai-python/tree/main/src/openai/types/chat):

| role | fields | notes |
|---|---|---|
| `system` | `content` (string or array of `text` parts), optional `name` | Classic system prompt. |
| `developer` | `content` (string or array of `text` parts), optional `name` | Introduced with o1; for OpenAI reasoning models `developer` replaces `system` ("with o1 models and newer, developer messages replace the previous system messages"). **Gateway rule: accept both; treat `developer` exactly like `system` when translating to upstreams that only know system prompts.** |
| `user` | `content` (string or array of content parts), optional `name` | Content parts may be multimodal (below). |
| `assistant` | `content` (string, array of `text`/`refusal` parts, or `null`), optional `name`, optional `refusal`, optional `tool_calls`, optional deprecated `function_call`, optional `audio` | "The contents of the assistant message. Required unless `tool_calls` or `function_call` is specified." So `content: null` + `tool_calls` is a legal and common history message. |
| `tool` | `content` (string or array of `text` parts, required), `tool_call_id` (string, required) | "Tool call that this message is responding to." One tool message per tool call ID. |
| `function` | `content`, `name` | Deprecated legacy of the pre-tools function API. Accept and map to `tool` semantics if seen. |

**User content parts** (array form of `content`):

- Text part: `{"type": "text", "text": "..."}`
- Image part (`ChatCompletionContentPartImageParam`, https://github.com/openai/openai-python/blob/main/src/openai/types/chat/chat_completion_content_part_image_param.py):

```json
{
  "type": "image_url",
  "image_url": {
    "url": "https://example.com/cat.png",
    "detail": "auto"
  }
}
```

  - `image_url.url` (required): "Either a URL of the image or the base64 encoded image data." The base64 form is a **data URI**: `"data:image/jpeg;base64,/9j/4AAQ..."` (`data:<mime>;base64,<payload>`; supported mimes: png, jpeg, webp, non-animated gif).
  - `image_url.detail` (optional): `"auto"` (default) | `"low"` | `"high"` — "Specifies the detail level of the image."
- Audio part: `{"type": "input_audio", "input_audio": {"data": "<base64>", "format": "wav"|"mp3"}}` (audio-capable models only; the router may reject with a clean error for providers without audio-in).
- File part: `{"type": "file", "file": {"file_id": "..."} }` or `{"file_data": "<base64 data URI>", "filename": "..."}` (PDF input on OpenAI; per-provider support varies).

Full request example with roles + multimodal:

```json
{
  "model": "gpt-4o",
  "messages": [
    {"role": "system", "content": "You are a terse assistant."},
    {"role": "user", "content": [
      {"type": "text", "text": "What is in this image?"},
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KG...", "detail": "high"}}
    ]},
    {"role": "assistant", "content": null, "tool_calls": [
      {"id": "call_abc123", "type": "function",
       "function": {"name": "lookup", "arguments": "{\"q\":\"cats\"}"}}
    ]},
    {"role": "tool", "tool_call_id": "call_abc123", "content": "{\"result\":\"a cat\"}"},
    {"role": "user", "content": "Thanks — summarize."}
  ]
}
```

#### 1.1.3 Sampling & length parameters

Types/defaults/deprecations verified against `CompletionCreateParams` (https://github.com/openai/openai-python/blob/main/src/openai/types/chat/completion_create_params.py) and the API reference:

| param | type | default | notes |
|---|---|---|---|
| `temperature` | number \| null | 1 | 0–2. "Higher values like 0.8 will make the output more random." Reasoning models (o-series, gpt-5) reject non-default values — the router passes through and lets upstreams reject, or strips per-provider via CompatAdjuster. |
| `top_p` | number \| null | 1 | Nucleus sampling. "We generally recommend altering this or `temperature` but not both." |
| `max_completion_tokens` | integer \| null | none | **The current parameter.** "An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens." |
| `max_tokens` | integer \| null | none | **Deprecated:** "This value is now deprecated in favor of `max_completion_tokens`, and is not compatible with o-series models." Still accepted for older models. (Sources: https://community.openai.com/t/why-was-max-tokens-changed-to-max-completion-tokens/938077, https://github.com/simonw/llm/issues/724, https://github.com/vercel/ai/issues/7863 — gpt-5 rejects `max_tokens` outright.) **Gateway rule: accept BOTH; if only `max_tokens` is given, treat it as `max_completion_tokens`; if both are given, prefer `max_completion_tokens`. Translate to each upstream's native cap (e.g., Anthropic's required `max_tokens`).** |
| `stop` | string \| string[] \| null | null | Up to 4 stop sequences. "Not supported with latest reasoning models `o3` and `o4-mini`." |
| `n` | integer \| null | 1 | Number of choices. Most non-OpenAI upstreams only support n=1 — the router should reject `n > 1` for those with a clear 400. |
| `frequency_penalty` | number \| null | 0 | −2.0 to 2.0. |
| `presence_penalty` | number \| null | 0 | −2.0 to 2.0. |
| `seed` | integer \| null | none | Beta. "Best effort to sample deterministically … same `seed` and parameters should return the same result." Pairs with `system_fingerprint` in the response. |
| `logit_bias` | map<string,int> \| null | null | Token-ID → bias −100..100. OpenAI-specific token IDs — pass through only to OpenAI-tokenizer upstreams; strip elsewhere. |
| `logprobs` | boolean \| null | false | "Whether to return log probabilities of the output tokens." Fills `choices[].logprobs`. |
| `top_logprobs` | integer \| null | none | 0–20; requires `logprobs: true`. |

#### 1.1.4 `response_format`

Three variants (see https://developers.openai.com/api/docs/guides/structured-outputs and `shared_params/response_format_*.py` in openai-python):

```json
{"type": "text"}
{"type": "json_object"}
{
  "type": "json_schema",
  "json_schema": {
    "name": "weather_report",
    "description": "optional",
    "schema": {
      "type": "object",
      "properties": {"city": {"type": "string"}, "temp_c": {"type": "number"}},
      "required": ["city", "temp_c"],
      "additionalProperties": false
    },
    "strict": true
  }
}
```

- `json_object` = legacy JSON mode ("an older method of generating JSON responses"); the prompt must mention JSON or OpenAI errors.
- `json_schema` = Structured Outputs. `json_schema.name` is required ("Must be a-z, A-Z, 0-9, or contain underscores and dashes, with a maximum length of 64"); `strict: true` means "the model will always follow the exact schema defined" (subset of JSON Schema: all fields `required`, `additionalProperties: false`).
- Router: translate to each provider's equivalent (Gemini `responseMimeType`/`responseSchema`, provider-specific json modes) or reject with a helpful 400 where unsupported.

#### 1.1.5 Tools / function calling

```json
{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"],
          "additionalProperties": false
        },
        "strict": true
      }
    }
  ],
  "tool_choice": "auto",
  "parallel_tool_calls": true
}
```

- `tools[]`: currently `type: "function"` for the public wire format (newer OpenAI additions include `custom` tools and hosted tools on the Responses API; a gateway needs only `function`). `function.parameters` is a JSON Schema object; `function.strict` optional.
- `tool_choice` — union per `ChatCompletionToolChoiceOptionParam` (https://github.com/openai/openai-python/blob/main/src/openai/types/chat/chat_completion_tool_choice_option_param.py):
  - `"none"` — never call tools ("default when no tools are present"),
  - `"auto"` — model decides (default when tools present),
  - `"required"` — model must call at least one tool,
  - named function: `{"type": "function", "function": {"name": "get_weather"}}`,
  - (newer) allowed-tools form `{"type": "allowed_tools", ...}` — pass-through/optional for a gateway.
- `parallel_tool_calls` (boolean, default true): "Whether to enable parallel function calling during tool use."
- Deprecated legacy: `functions` ("Deprecated in favor of `tools`") and `function_call` ("Deprecated in favor of `tool_choice`") — accept, map to tools/tool_choice internally.

#### 1.1.6 Streaming controls

- `stream` (boolean \| null, default false): "If set to true, the model response data will be streamed to the client as it is generated using server-sent events."
- `stream_options` (object \| null — "Only set this when you set `stream: true`"):
  - `include_usage` (boolean): "If set, an additional chunk will be streamed before the `data: [DONE]` message. The `usage` field on this chunk shows the token usage statistics for the entire request, and the `choices` field will always be an empty array. All other chunks will also include a `usage` field, but with a null value." (Source: https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156, and the SDK type docstring.)
  - `include_obfuscation` (boolean, newer): adds random `obfuscation` padding fields to chunks; a local gateway should not emit it.

#### 1.1.7 Identity, caching & misc parameters

| param | notes |
|---|---|
| `user` (string) | Legacy end-user ID. "This field is being replaced by `safety_identifier` and `prompt_cache_key`." Accept it; useful as a per-key attribution hint. |
| `safety_identifier` (string) | "A stable identifier used to help detect users … Maximum length of 64 characters." Pass through to OpenAI only. |
| `prompt_cache_key` (string) | Cache-affinity hint, "Replaces the `user` field" for caching. Pass through to OpenAI only. |
| `store` (bool), `metadata` (map, ≤16 keys) | OpenAI-side storage for distillation/evals. Pass through to OpenAI; strip elsewhere. |
| `service_tier` | `"auto" \| "default" \| "flex" \| "scale" \| "priority" \| "fast"`. OpenAI-only; strip elsewhere. |
| `reasoning_effort` | For reasoning models. Current values per SDK: "`none`, `minimal`, `low`, `medium`, `high`, `xhigh`, and `max`" (model-dependent subsets). The router should pass this through to reasoning-capable upstreams that accept it. |
| `verbosity` | `"low" \| "medium" \| "high"` (gpt-5 family). Pass through to OpenAI. |
| `modalities`, `audio`, `prediction`, `web_search_options` | Audio-out, predicted outputs, built-in web search — OpenAI-specific; a gateway may pass through to OpenAI and strip/400 elsewhere. |

**Unknown-key policy:** OpenAI itself returns 400 `unrecognized argument` for unknown top-level keys, but a *gateway* should follow LiteLLM/OpenRouter practice: accept unknown keys and pass them through to the upstream body (this is how provider-specific extras like Perplexity `search_domain_filter` or Qwen `enable_thinking` travel). Document accepted extras in `docs/API.md`.

---

### 1.2 Non-streaming response — `chat.completion` object

Fields verified against `ChatCompletion` (https://github.com/openai/openai-python/blob/main/src/openai/types/chat/chat_completion.py) and `CompletionUsage` (https://github.com/openai/openai-python/blob/main/src/openai/types/completion_usage.py):

```json
{
  "id": "chatcmpl-B9MBs8CjcvOU2jLn4n570S5qMJKcT",
  "object": "chat.completion",
  "created": 1741569952,
  "model": "gpt-4o-2024-08-06",
  "system_fingerprint": "fp_50cad350e4",
  "service_tier": "default",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I assist you today?",
        "refusal": null,
        "annotations": []
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 19,
    "completion_tokens": 10,
    "total_tokens": 29,
    "prompt_tokens_details": {"cached_tokens": 0, "audio_tokens": 0},
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  }
}
```

Field-by-field:

- `id` (string): "A unique identifier for the chat completion." Convention `chatcmpl-<base62>`; the router generates its own (`chatcmpl-` prefix keeps naive clients happy).
- `object`: literal `"chat.completion"`.
- `created` (integer): Unix seconds.
- `model` (string): "The model used" — router echoes the namespaced ID actually served (incl. after fallback).
- `system_fingerprint` (string, optional, now marked deprecated in OpenAI docs): backend-config fingerprint for use with `seed`. Optional — the router may omit or emit a static value.
- `service_tier` (optional): echo only for OpenAI upstreams.
- `choices[]`:
  - `index` (integer),
  - `message`:
    - `role`: always `"assistant"`,
    - `content` (string \| null): null when the model only called tools,
    - `refusal` (string \| null): structured-outputs refusal message,
    - `tool_calls` (array, optional): each `{"id": "call_…", "type": "function", "function": {"name": "...", "arguments": "<JSON string>"}}` — **`arguments` is a string containing JSON, not an object**,
    - `annotations` (array, optional): e.g. `url_citation` items from web search,
    - `audio` (optional, audio-out models),
    - reasoning models on other providers add `reasoning_content` (DeepSeek et al.) — not an OpenAI field, but the de-facto extension the router preserves (Phase 0 decision),
  - `logprobs` (object \| null): `{"content": [{token, logprob, bytes, top_logprobs: […]}], "refusal": […]}` when requested,
  - `finish_reason` — exact literal set per the SDK: **`"stop" | "length" | "tool_calls" | "content_filter" | "function_call"`**:
    - `stop` — natural stop or stop sequence hit,
    - `length` — token cap reached (`max_completion_tokens` or context limit),
    - `tool_calls` — the model called tools,
    - `content_filter` — content omitted by a filter,
    - `function_call` — deprecated legacy (only when using deprecated `functions`).
    Every upstream stop reason must be mapped into this set (e.g., Anthropic `end_turn`→`stop`, `max_tokens`→`length`, `tool_use`→`tool_calls`, `stop_sequence`→`stop`).
- `usage`:
  - `prompt_tokens`, `completion_tokens`, `total_tokens` (integers, total = prompt + completion),
  - `prompt_tokens_details` (optional): `cached_tokens` ("Cached tokens present in the prompt"), `audio_tokens`, and newer `cache_write_tokens`,
  - `completion_tokens_details` (optional): `reasoning_tokens` ("Tokens generated by the model for reasoning"), `audio_tokens`, `accepted_prediction_tokens`, `rejected_prediction_tokens`.
  - Gateway rule: use upstream-reported usage when present; when absent, estimate and flag (e.g. `"x_zyquo": {"usage_estimated": true}` — extension keys are tolerated by both SDKs).

Tool-call response example (non-streaming):

```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1699896916,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {"name": "get_weather", "arguments": "{\n\"city\": \"Boston\"\n}"}
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {"prompt_tokens": 82, "completion_tokens": 17, "total_tokens": 99}
}
```

---

### 1.3 The SSE streaming format — byte-level contract

Sources: streaming reference (https://developers.openai.com/api/reference/resources/chat/subresources/completions/streaming-events, https://platform.openai.com/docs/api-reference/chat-streaming/streaming), the cookbook (https://cookbook.openai.com/examples/how_to_stream_completions), and `ChatCompletionChunk` SDK types.

**Transport.** Response headers: `Content-Type: text/event-stream; charset=utf-8`, `Cache-Control: no-cache`, chunked transfer (no `Content-Length`), keep connection open. Each event is exactly:

```
data: <one-line JSON>\n
\n
```

i.e. the 6 bytes `data: `, the JSON serialized **without newlines**, then `\n\n`. OpenAI emits only `data:` lines — no `event:`, `id:`, or `retry:` fields, no SSE comments. The stream terminates with the sentinel:

```
data: [DONE]\n
\n
```

(`[DONE]` is not JSON; both official SDKs special-case this exact string.) An HTTP-level error that occurs *before* streaming starts is a plain JSON error body with a proper status code; the status is sent before any chunk, so a request that fails validation must NOT return 200 + SSE.

**Chunk object** (`object: "chat.completion.chunk"`): same `id` ("Each chunk has the same ID"), `created`, and `model` across all chunks of one completion; `choices[]` with `{index, delta, logprobs, finish_reason}`; `usage` null/absent except the final usage chunk. Delta fields: `role`, `content`, `refusal`, `tool_calls[]` (each with `index`, optional `id`, optional `type: "function"`, optional `function.name`, optional `function.arguments` fragment), deprecated `function_call`.

Chunk sequence rules:

1. **First chunk** carries the role delta: `"delta": {"role": "assistant", "content": ""}` (OpenAI includes the empty `content` string; emit it — some clients concatenate blindly). May also carry `refusal: null` — harmless.
2. **Content chunks**: `"delta": {"content": "<fragment>"}`, `finish_reason: null`.
3. **Final content chunk**: `"delta": {}` (empty object) with `"finish_reason": "stop"` (or `length`/`tool_calls`/`content_filter`). The finish_reason travels on a chunk whose delta is empty — not alongside content.
4. **Optional usage chunk** (only when `stream_options.include_usage` is true): `"choices": []` (empty array — quoted from the SDK: choices "can also be empty for the last chunk if you set `stream_options: {\"include_usage\": true}`") and a populated `usage` object. "All other chunks will also include a `usage` field, but with a null value" when include_usage is set.
5. `data: [DONE]`.

#### (a) Plain-text transcript (`stream: true`, `stream_options: {"include_usage": true}`)

```
data: {"id":"chatcmpl-9rORw7dbcqvUv0entIFewX1LLQBmy","object":"chat.completion.chunk","created":1722525116,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_611b667b19","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-9rORw7dbcqvUv0entIFewX1LLQBmy","object":"chat.completion.chunk","created":1722525116,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_611b667b19","choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-9rORw7dbcqvUv0entIFewX1LLQBmy","object":"chat.completion.chunk","created":1722525116,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_611b667b19","choices":[{"index":0,"delta":{"content":" there"},"logprobs":null,"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-9rORw7dbcqvUv0entIFewX1LLQBmy","object":"chat.completion.chunk","created":1722525116,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_611b667b19","choices":[{"index":0,"delta":{"content":"!"},"logprobs":null,"finish_reason":null}],"usage":null}

data: {"id":"chatcmpl-9rORw7dbcqvUv0entIFewX1LLQBmy","object":"chat.completion.chunk","created":1722525116,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_611b667b19","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}],"usage":null}

data: {"id":"chatcmpl-9rORw7dbcqvUv0entIFewX1LLQBmy","object":"chat.completion.chunk","created":1722525116,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_611b667b19","choices":[],"usage":{"prompt_tokens":19,"completion_tokens":3,"total_tokens":22,"prompt_tokens_details":{"cached_tokens":0,"audio_tokens":0},"completion_tokens_details":{"reasoning_tokens":0,"audio_tokens":0,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}}}

data: [DONE]

```

(Without `include_usage`, the usage chunk is absent and no `usage` key appears on chunks.)

#### (b) Streamed tool call transcript

Tool-call arguments stream as string fragments. **The first tool_call delta for a given `index` carries `id`, `type`, and `function.name` (with `"arguments":""`); every subsequent delta for that index carries ONLY `index` and `function.arguments` fragments — no id, no name.** Parallel tool calls interleave via `index` 0,1,…; clients accumulate by index and concatenate `arguments`.

```
data: {"id":"chatcmpl-9s0Ab","object":"chat.completion.chunk","created":1722530000,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"delta":{"role":"assistant","content":null},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9s0Ab","object":"chat.completion.chunk","created":1722530000,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_KSEnFnucOtZNQqEZF9wvfIWm","type":"function","function":{"name":"get_weather","arguments":""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9s0Ab","object":"chat.completion.chunk","created":1722530000,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"ci"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9s0Ab","object":"chat.completion.chunk","created":1722530000,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ty\": \"Bos"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9s0Ab","object":"chat.completion.chunk","created":1722530000,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"ton\"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9s0Ab","object":"chat.completion.chunk","created":1722530000,"model":"gpt-4o-2024-08-06","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}

data: [DONE]

```

#### What strict SDK clients require

- **openai-python** parses SSE by splitting on blank lines, reads only `data:` payloads, stops at the exact string `[DONE]`, and constructs a pydantic `ChatCompletionChunk`. Pydantic will **fail the whole stream** if `id`, `object`, `created`, `model`, or `choices` is missing/wrong-typed, if `object` isn't exactly `"chat.completion.chunk"`, or if `finish_reason` is a value outside the literal set. Extra unknown fields are tolerated (kept as extras) — so `reasoning_content` in deltas and `x_zyquo` extensions are safe. (See https://github.com/openai/openai-python; a real-world failure mode of non-conforming servers: https://github.com/janhq/jan/issues/8280 — "the chat.completion.chunk schema … requires choices[]; type validation failed" halts the stream.)
- **openai-node** is looser at runtime (TypeScript types are compile-time), but its stream accumulator (`stream.finalChatCompletion()`, and the Vercel AI SDK on top of it) indexes `choices[0].delta`, accumulates `tool_calls` strictly by `index`, and expects `id`/`function.name` on the first delta of each tool call; missing `index` or re-sending `name` fragments corrupts accumulation (see https://ai-sdk.dev/providers/openai-compatible-providers on buffering unreliable tool-call deltas).
- Both SDKs ignore SSE `event:` fields **only** if the line isn't a `data:` line; do not emit named events or comments — emit *only* `data:` lines exactly as above.
- Keep every JSON chunk on a single line; UTF-8; never split a multibyte character across chunks inside one JSON string (JSON-escape or buffer to codepoint boundaries).

---

### 1.4 `GET /v1/models` and `GET /v1/models/{id}`

Source: https://developers.openai.com/api/reference/resources/models (mirrors https://platform.openai.com/docs/api-reference/models).

`GET /v1/models` →

```json
{
  "object": "list",
  "data": [
    {"id": "gpt-4o", "object": "model", "created": 1686935002, "owned_by": "openai"},
    {"id": "gpt-4o-mini", "object": "model", "created": 1686935002, "owned_by": "openai"}
  ]
}
```

Model object: `id` (string — "The model identifier, which can be referenced in the API endpoints"), `object` (always `"model"`), `created` (Unix seconds), `owned_by` (string — "The organization that owns the model").

`GET /v1/models/{model}` → a single model object:

```json
{"id": "gpt-4o", "object": "model", "created": 1686935002, "owned_by": "openai"}
```

Unknown ID → 404 with the model-not-found error (§1.5). Router mapping: `id` = namespaced `provider/model-id` (aliases listed too), `owned_by` = provider name; enriched metadata (context window, pricing, capabilities) goes under an `x_zyquo` extension key on each entry — both SDKs tolerate extra fields.

---

### 1.5 Error response format

Every error is JSON with a single `error` object (see https://developers.openai.com/api/docs/guides/error-codes and https://community.openai.com/t/openai-chat-list-of-error-codes-and-types/357791):

```json
{
  "error": {
    "message": "<human-readable description>",
    "type": "<error family>",
    "param": "<offending request parameter or null>",
    "code": "<machine-readable code or null>"
  }
}
```

All four keys are always present (`param`/`code` may be `null`). The official SDKs map HTTP status → typed exceptions (`BadRequestError` 400, `AuthenticationError` 401, `PermissionDeniedError` 403, `NotFoundError` 404, `UnprocessableEntityError` 422, `RateLimitError` 429, `InternalServerError` ≥500, per https://github.com/openai/openai-python#handling-errors).

| HTTP | `type` | typical `code` values | when |
|---|---|---|---|
| 400 | `invalid_request_error` | `null`, `invalid_value`, `unsupported_parameter`, `context_length_exceeded`, `string_above_max_length`, `invalid_image_format` | Malformed body, bad param, context overflow |
| 401 | `invalid_request_error` / `authentication_error` | `invalid_api_key`, `no_organization` | Missing/invalid API key ("Incorrect API key provided: …") |
| 403 | `permission_error` / `invalid_request_error` | `unsupported_country_region_territory`, `insufficient_permissions` | Key valid but not allowed (region, scoped key) |
| 404 | `invalid_request_error` | `model_not_found` | Unknown model / resource |
| 422 | `invalid_request_error` | — | Semantically invalid (rare) |
| 429 | `rate_limit_error` | `rate_limit_exceeded` | "Rate limit reached for …" — retriable; honor/emit `Retry-After` |
| 429 | `insufficient_quota` | `insufficient_quota` | "You exceeded your current quota, plans & billing…" — NOT retriable (note: type AND code are both `insufficient_quota`) |
| 500 | `server_error` | `null` | "The server had an error while processing your request" |
| 502 | (gateway) `server_error` / `api_error` | `bad_gateway` | Used by gateways (OpenRouter/LiteLLM) for upstream failure — the router's choice for "upstream returned garbage / is down" |
| 503 | `server_error` / `service_unavailable` | `service_unavailable`, `slow_down` | "The engine is currently overloaded, please try again later" |
| 504 | (gateway) `timeout_error` | `timeout` | Gateway convention for upstream timeout |

**The exact model-not-found error** (HTTP 404) as OpenAI returns it (sources: https://community.openai.com/t/openai-error-invalidrequesterror-the-model-gpt-4-does-not-exist-or-you-do-not-have-access-to-it/376230, https://community.openai.com/t/api-returning-404-model-not-found-all-of-a-sudden-why-and-how-to-fix/679777):

```json
{
  "error": {
    "message": "The model `gpt-5-nonexistent` does not exist or you do not have access to it.",
    "type": "invalid_request_error",
    "param": null,
    "code": "model_not_found"
  }
}
```

The router must reproduce this shape verbatim (with its namespaced ID in backticks) for unknown/disabled models — SDK error-handling paths and agent frameworks string-match parts of it.

**Streaming errors:** if the failure happens before any chunk, return the JSON error with the real status code (no SSE). If the upstream dies mid-stream, OpenAI's own behavior is to emit an error payload as a `data:` line (`{"error": {...}}`) and close without `[DONE]`; a gateway should do the same — strict clients surface it as a stream error rather than hanging.

**Gateway error-mapping rules (Phase 3 contract):** upstream 401 (provider key bad) → 401 with "provider key for `<provider>` was rejected" (never echo the key); upstream 429 → 429 + `Retry-After` when given; upstream timeout → 504 `timeout`; upstream 5xx/unparseable → 502 `bad_gateway` with sanitized detail; never leak raw provider error shapes or key material.

---

### 1.6 `POST /v1/responses` — evaluate and decide

Summary (sources: https://platform.openai.com/docs/guides/migrate-to-responses, https://developers.openai.com/api/docs/guides/migrate-to-responses):

- **Shape:** flatter request — `input` (string or item array) + top-level `instructions` instead of a `messages` array; response is a typed `output` array of *items* (`message`, `reasoning` with `encrypted_content`, `function_call`, `function_call_output`, hosted-tool items) plus an `output_text` convenience; server-side state via `store: true` / `previous_response_id`; built-in hosted tools (web search, file search, code interpreter, computer use).
- **Streaming:** *semantic events*, not chunk deltas — named SSE events like `response.created`, `response.output_item.added`, `response.output_text.delta`, `response.completed` — a completely different event model from `chat.completion.chunk`.
- **Adoption (as of mid-2026):** OpenAI recommends Responses "for all new projects" and newest OpenAI-native features (encrypted reasoning, hosted tools) land there first, but they state Chat Completions "remains supported" indefinitely as the industry standard. Critically for a *multi-provider gateway*: the entire compatible-provider ecosystem (DeepSeek, Qwen, Mistral, xAI, Together, Cerebras, Ollama, LM Studio, vLLM…) standardized on **chat/completions**, and gateways (LiteLLM, OpenRouter) still treat it as the lingua franca (OpenRouter exposes chat/completions; LiteLLM added a Responses *bridge* that internally converts to chat/completions).

**DECISION for Zyquo Router: do NOT expose `/v1/responses` in v1.** Rationale:

1. Zero translation leverage — none of our 12 upstream providers speak Responses natively except OpenAI itself; we'd be building a second full bidirectional translation layer (items + semantic streaming events) purely as a front-end conversion to the same canonical internal request.
2. Statefulness (`store`, `previous_response_id`, `encrypted_content`) implies server-side conversation storage — out of scope and against the router's privacy posture.
3. Client compatibility target is met without it: every OpenAI-SDK-based tool can (and with third-party base URLs, typically does) use `chat.completions`.
4. Cheap to add later as a stateless subset (`input`/`instructions` → messages; emit `response.output_text.delta` events) once the chat/completions core is green — track as a post-v1 enhancement in `docs/PLAN.md`. `/health` should not advertise it; requests to `/v1/responses` return a 404 OpenAI-format error with a message pointing to `/v1/chat/completions`.

---

### 1.7 `POST /v1/embeddings` (optional gateway endpoint)

Source: https://platform.openai.com/docs/api-reference/embeddings and `embedding_create_params.py` / `create_embedding_response.py` in openai-python.

Request:

```json
{
  "model": "text-embedding-3-small",
  "input": "The food was delicious and the waiter...",
  "encoding_format": "float",
  "dimensions": 512,
  "user": "optional-end-user-id"
}
```

- `input` (required): "string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays." (Max ~2048 inputs per request; each within the model's token limit.)
- `model` (required); `encoding_format`: `"float"` (default) or `"base64"`; `dimensions`: "Only supported in `text-embedding-3` and later models"; `user`: abuse-monitoring ID.

Response:

```json
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0023064255, -0.009327292, -0.0028842222]
    }
  ],
  "model": "text-embedding-3-small",
  "usage": {"prompt_tokens": 8, "total_tokens": 8}
}
```

Notes for the router: `data[]` is ordered by `index` matching the input array; `usage` has only `prompt_tokens` + `total_tokens` (no completion tokens); `encoding_format: "base64"` returns each embedding as a base64 string of little-endian float32 — support it, since openai-python requests base64 by default when numpy is available. Route only to providers that offer embeddings; others 404 with `model_not_found`.

---

### 1.8 Contract checklist for Zyquo Router (derived from this section)

- [ ] Accept full request schema §1.1 incl. both `max_tokens` and `max_completion_tokens`, `developer` role, multimodal parts, all three `response_format` variants, all `tool_choice` forms, deprecated `functions`/`function_call`.
- [ ] Emit spec-exact `chat.completion` (§1.2) with mapped `finish_reason` from the closed literal set and real-or-flagged `usage` incl. details sub-objects when upstreams provide them.
- [ ] Emit byte-exact SSE (§1.3): role-first chunk, single-line JSON `data:` events, empty-delta finish chunk, empty-choices usage chunk only under `include_usage`, terminating `data: [DONE]`, correct tool-call delta id/name/index rules.
- [ ] `GET /v1/models` + `/v1/models/{id}` per §1.4 with namespaced IDs.
- [ ] All errors per §1.5 incl. verbatim model-not-found shape and gateway 502/504 conventions.
- [ ] `/v1/responses`: not exposed in v1 (documented 404 with pointer); `/v1/embeddings`: optional, per §1.7.
## 2. How existing gateways do it

Research date: 2026-07-30. Sources fetched live from docs.litellm.ai, openrouter.ai/docs, docs.ollama.com, lmstudio.ai/docs, docs.vllm.ai, and GitHub issue trackers. This section extracts the concrete, battle-tested patterns from the reference gateways that Zyquo Router should copy — and the compatibility landmines it must avoid.

---

### 2.1 LiteLLM proxy — the reference for config, translation, and error mapping

LiteLLM is the most complete open-source implementation of exactly what Zyquo Router is: an OpenAI-compatible front for ~100 providers. Its patterns are the closest prior art.

#### 2.1.1 Model naming: `provider/model`

LiteLLM routes on a **provider prefix in the model string**: `openai/gpt-4o`, `azure/gpt-4o`, `anthropic/claude-...`, `bedrock/anthropic.claude-instant-v1`, `ollama/mistral`, `gemini/gemini-2.5-pro`. The prefix selects the client implementation; the remainder is the upstream model ID sent to the provider. (Source: https://docs.litellm.ai/docs/proxy/configs)

The proxy adds a second layer of indirection: a **user-facing `model_name` alias** mapped to one or more concrete deployments in `config.yaml`:

```yaml
model_list:
  - model_name: gpt-4o                # what clients send in "model"
    litellm_params:
      model: azure/gpt-4o-eu          # provider/upstream-id actually called
      api_base: https://endpoint-europe.openai.azure.com/
      api_key: "os.environ/AZURE_API_KEY_EU"   # env-var indirection, secrets never in config
      rpm: 6                          # per-deployment rate limit
    model_info:                       # optional metadata (pricing/context overrides)
      max_input_tokens: 128000

litellm_settings:
  drop_params: true
  num_retries: 3
  request_timeout: 10
  fallbacks: [{"gpt-4o": ["claude-sonnet"]}]

router_settings:
  routing_strategy: simple-shuffle
  model_group_alias: {"gpt-4": "gpt-4o"}   # request-time alias remapping

general_settings:
  master_key: sk-1234                  # local bearer key gating the proxy
```

Key ideas to steal (https://docs.litellm.ai/docs/proxy/configs):

- **Two-level naming**: public alias (`model_name`) → concrete `provider/model` deployment. Zyquo Router's aliases (`fast` → `cerebras/...`) are exactly this.
- Multiple entries with the same `model_name` = load balancing group (Zyquo Router doesn't need multi-deployment balancing, but the alias→target indirection is the same shape).
- `model_group_alias` maps well-known client names (`gpt-4`) onto configured groups — useful for tools that hardcode OpenAI model names.
- Wildcards (`model_name: "*"`, `model: openai/*`) allow pass-through of any model given credentials — Zyquo Router's "accept unambiguous bare IDs" is a constrained version of this.
- `os.environ/VAR` indirection keeps keys out of the config file (Zyquo Router: keys live only in the vault; config export never includes them).

#### 2.1.2 Param translation and drop tables

LiteLLM maintains, per provider, a **mapping table of which OpenAI params the provider supports** (queryable via `litellm.get_supported_openai_params(model)`), and translates names where they differ. Handling of *unsupported* params is explicit policy, not accident (https://docs.litellm.ai/docs/completion/drop_params):

- **Default: raise an exception** if a param is sent to a model that doesn't support it — loud failure over silent behavior change.
- **`drop_params: true`** (global, per-deployment, or per-request): silently strip unsupported params instead of erroring. Most proxies run with this on.
- **`additional_drop_params: ["response_format"]`** — per-deployment list of specific params to strip even if nominally supported; supports JSONPath-ish nested syntax (`tools[*].input_examples`, `parent.child`, `array[0]`).
- **`allowed_openai_params: ["tools"]`** — the inverse escape hatch: force-forward a param LiteLLM believes is unsupported (settable in config or per-request via `extra_body`).
- **Provider-specific extras** ride through `extra_body` on the OpenAI SDK and are passed to the upstream unchanged.

Lesson for Zyquo Router's `CompatAdjuster`: implement a **per-provider param table** (supported / rename / strip / pass-through-extras) as data, not scattered `if`s, and make the strip-vs-error policy explicit and configurable.

#### 2.1.3 Error mapping to OpenAI exceptions

LiteLLM maps every upstream failure onto **exception types that inherit from the OpenAI SDK's own exceptions**, so client code catching `openai.RateLimitError` works against any provider (https://docs.litellm.ai/docs/exception_mapping). Status-code taxonomy:

| Status | Exceptions |
|---|---|
| 400 | `BadRequestError`, `UnsupportedParamsError`, `ContextWindowExceededError`, `ContentPolicyViolationError` |
| 401 | `AuthenticationError` |
| 403 | `PermissionDeniedError` |
| 404 | `NotFoundError` |
| 408 | `Timeout` |
| 422 | `UnprocessableEntityError` |
| 429 | `RateLimitError` |
| 500 | `APIConnectionError`, `APIError` |
| 503 | `ServiceUnavailableError` |
| ≥500 | `InternalServerError` |

Every mapped exception carries `status_code`, `message`, and **`llm_provider`** (which upstream failed) — Zyquo Router should likewise name the provider in error messages ("Anthropic key invalid") without leaking payloads. Note that `ContextWindowExceededError` and `ContentPolicyViolationError` are *distinguished subtypes of 400*: this is what makes context-window fallbacks and content-policy fallbacks possible. A `_should_retry(status_code)` helper centralizes the retryability decision (429, 5xx, timeouts → retry; 4xx auth/validation → don't).

#### 2.1.4 Retries, fallbacks, cooldowns

(Sources: https://docs.litellm.ai/docs/routing, https://docs.litellm.ai/docs/proxy/reliability)

- **`num_retries: 3`** with exponential backoff for `RateLimitError`, immediate retry for transient errors; `retry_after` sets a minimum wait. A `RetryPolicy` can set retry counts **per exception class** (e.g., `AuthenticationErrorRetries=0`, `RateLimitErrorRetries=3`, `TimeoutErrorRetries=2`) — never retry auth failures.
- **Three fallback kinds**, configured as ordered maps `{"primary": ["fallback1", "fallback2"]}`:
  - `fallbacks` — general retryable errors (429/5xx) after retries exhaust;
  - `context_window_fallbacks` — prompt too big → reroute to a bigger-context model;
  - `content_policy_fallbacks` — content filter tripped → reroute to a laxer model;
  - `default_fallbacks: ["claude-opus"]` — catch-all.
- **Per-request fallbacks** via a `fallbacks: [...]` array in the request body, and `disable_fallbacks: true` to opt out per request.
- **Cooldowns**: `allowed_fails: 3` failures per minute puts a deployment on a `cooldown_time: 30`s bench so the router stops hammering a failing upstream. For a single-deployment-per-model local router this maps to "mark provider degraded, surface in dashboard, fail fast or fall back."
- Execution order: retries on the primary first, then fallbacks in order until success or exhaustion. The actually-used deployment is reported via an `x-litellm-model-id` response header — Zyquo Router should report the actually-used model in the response `model` field (OpenRouter's approach, §2.2.4) and/or a header.

#### 2.1.5 Usage & cost tracking

(Source: https://docs.litellm.ai/docs/completion/token_usage)

- Pricing lives in one community-maintained JSON file, `model_prices_and_context_window.json` (https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) — the de-facto industry pricing database. Entry shape:

```json
{
  "gpt-4o": {
    "max_tokens": 4000,
    "input_cost_per_token": 1.5e-06,
    "output_cost_per_token": 2e-06,
    "litellm_provider": "openai",
    "mode": "chat"
  }
}
```

- `completion_cost(response)` computes USD from the usage in a response + this table; `cost_per_token(model, prompt_tokens, completion_tokens)` is the primitive. Every response carries `response_cost` in hidden params.
- `register_model({...})` lets users override/add pricing — Zyquo Router's Settings ▸ Usage & Pricing "pricing override" is the same feature.
- When upstream doesn't return usage (some streams), tokens are **estimated with a tokenizer** (tiktoken default, provider-specific where known) — and Zyquo Router should flag estimated usage as such.

Pattern: **pricing is data keyed by model ID, cost computed at response time from `usage`, estimation as fallback**. Zyquo Router already has per-model pricing in the Zyquo Cloud catalog; reuse it as the single pricing source.

#### 2.1.6 Streaming normalization

LiteLLM wraps every provider stream in a `CustomStreamWrapper` that re-emits **uniform OpenAI-shaped chunk objects** (`choices[0].delta.content`, etc.) regardless of upstream wire format, plus a `stream_chunk_builder(chunks)` helper that reassembles a full `chat.completion` from chunks (useful for logging/cost of streamed requests — Zyquo Router's request log needs exactly this) (https://docs.litellm.ai/docs/completion/stream). It also guards against pathological streams: `REPEATED_STREAMING_CHUNK_LIMIT = 100` aborts with an `InternalServerError` if the same chunk repeats endlessly.

Cautionary tales from LiteLLM's own tracker — even the reference implementation gets chunk shape wrong:

- Its synthetic usage chunk violated the OpenAI spec by carrying a **non-empty `choices`** array (spec: the `include_usage` final chunk has `"choices": []`) — https://github.com/BerriAI/litellm/issues/28735
- It **lost vLLM's usage** because vLLM sends usage in a separate empty-choices chunk *after* the `finish_reason` chunk and LiteLLM stopped reading at `finish_reason` — https://github.com/BerriAI/litellm/issues/25389 . Lesson: when consuming upstreams, read until the stream actually ends, not until `finish_reason`.
- Grok returned usage in the wrong chunk (extra empty final chunk) — https://github.com/BerriAI/litellm/issues/17136 ; and some providers reject `stream_options` as an unknown param, so the gateway must know per provider whether it can request usage-in-stream — https://github.com/BerriAI/litellm/issues/23847

---

### 2.2 OpenRouter — the reference for unified IDs, streaming discipline, and honest accounting

#### 2.2.1 Unified model IDs and variants

- IDs are **`vendor/model-name`** (`anthropic/claude-3.5-sonnet`, `openai/gpt-4o`), plus a permanent `canonical_slug` that survives renames. (https://openrouter.ai/docs/models)
- **Variant suffixes** append behavior to a slug: `:free` (free tier), `:thinking` (reasoning mode), `:nitro` (= `provider.sort: "throughput"`), `:floor` (sort by price). A suffix-on-the-ID is a very ergonomic way to encode routing preferences without extra params (https://openrouter.ai/docs/models, https://openrouter.ai/docs/features/provider-routing). Zyquo Router could reserve this pattern for future use (e.g., `model:nostore` to skip logging).
- `GET /api/v1/models` returns rich metadata per model: `id`, `canonical_slug`, `name`, `context_length`, `architecture` (input/output modalities, tokenizer), `pricing` (**USD per token as strings** — `"0"` means free; string avoids float precision issues), `supported_parameters` (array of OpenAI params this model accepts — clients can pre-check!), `top_provider`. Zyquo Router's plan to enrich `/v1/models` under an `x-zyquo` extension key mirrors this; adopting `context_length`, `pricing`, and `supported_parameters` fields is directly useful for the Playground and Docs UI.

#### 2.2.2 Provider-specific params & headers

- Default posture: **unsupported params are silently ignored by the receiving provider**; setting `"provider": {"require_parameters": true}` restricts routing to providers that support *every* param in the request (https://openrouter.ai/docs/features/provider-routing). For a single-upstream-per-model router, the equivalent decision is strip-vs-reject per param (see §2.5).
- Extra attribution headers `HTTP-Referer` and `X-Title` are optional and additive — the API remains pure OpenAI otherwise (https://openrouter.ai/docs/api-reference/overview).
- Provider-specific features arrive as **extra top-level body keys** (e.g., `models`, `provider`, `plugins`, `transforms`) that OpenAI SDKs send via `extra_body` — the standard pass-through idiom Zyquo Router should adopt for things like Perplexity search options.

#### 2.2.3 Streaming normalization — the details that matter

(Source: https://openrouter.ai/docs/api-reference/streaming)

- **Keep-alive SSE comments**: OpenRouter periodically emits `: OPENROUTER PROCESSING` comment lines to hold connections open during long prefill/queue waits. Per the SSE spec, lines starting with `:` are comments and must be ignored — but naive client loops that `JSON.parse` every line crash on them. Zyquo Router should (a) emit its own `: keep-alive` comments during long upstream silences (Anthropic thinking, queued requests), and (b) tolerate/strip comments when *consuming* upstream SSE.
- Termination is always `data: [DONE]`.
- **Mid-stream errors**: once tokens have flowed you can't change the HTTP status, so errors arrive as a final SSE event that is still a valid `chat.completion.chunk` with an added `error` object and `finish_reason: "error"`:

```
data: {"id":"gen-abc123","object":"chat.completion.chunk","created":1730000000,
       "model":"...","error":{"code":429,"message":"Rate limit exceeded",
       "metadata":{"error_type":"rate_limit_exceeded"}},
       "choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}
```

  (https://openrouter.ai/docs/api-reference/errors) This is the pattern Zyquo Router needs for "upstream died mid-stream": emit a well-formed error chunk, then `[DONE]`, then close — never a bare connection reset.
- **Cancellation**: aborting the connection cancels the upstream *only for streaming requests on providers that support cancellation*; otherwise the upstream finishes and bills anyway. Zyquo Router must cancel the upstream `URLSession`/NIO task on client disconnect (its clients are direct HTTP calls, so cancellation is always possible) — this is Phase 7 test 3.
- Usage stats ride in the **final chunk** (`chunk.usage`).

#### 2.2.4 Usage accounting

(Source: https://openrouter.ai/docs/use-cases/usage-accounting)

- Usage (with **cost**) is now included in every response automatically (the old `usage: {include: true}` opt-in is deprecated). Shape extends OpenAI's `usage`:

```json
"usage": {
  "prompt_tokens": 194,
  "completion_tokens": 2,
  "total_tokens": 196,
  "cost": 0.95,
  "cost_details": {"upstream_inference_cost": 19},
  "prompt_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 100},
  "completion_tokens_details": {"reasoning_tokens": 0}
}
```

  Putting `cost` inside `usage` is a compatible extension (SDKs ignore unknown fields on responses) — a good idea for Zyquo Router's estimated-cost surfacing.
- For streams, usage appears **in the last SSE message**; there's also a post-hoc `GET /api/v1/generation?id=...` stats endpoint keyed by the response `id` for auditing. Zyquo Router's request-log detail view is the local equivalent.

#### 2.2.5 Error format

(Source: https://openrouter.ai/docs/api-reference/errors)

- Error shape: `{"error": {"code": <number>, "message": "...", "metadata": {...}}}` with HTTP status = `error.code` for pre-stream failures. Status vocabulary: **400** bad params/CORS, **401** bad key, **402** out of credits, **403** moderation/guardrail, **408** timeout, **429** rate limited, **502** "your chosen model is down or we received an invalid response", **503** "no available provider meets your routing requirements".
- `metadata` carries structured context without leaking raw payloads: moderation errors include `reasons`, a truncated `flagged_input` (max 100 chars), `provider_name`, `model_slug`; provider errors include a canonical `error_type` plus the original `provider_code`. This "canonical code + original provider code" pair is exactly the honest-but-normalized surfacing Zyquo Router wants.
- **429/503 responses include a standard `Retry-After` header** — Zyquo Router should propagate upstream `Retry-After` to its own clients.
- Note: OpenRouter uses a numeric `error.code`; the strict OpenAI format is `{"error": {"message", "type", "param", "code"}}` with string-ish `code`. Zyquo Router should keep the **OpenAI field set** (per §1) and put router-specific context in the message and/or an extension key, since OpenAI SDKs construct exceptions from `type`/`code`.

#### 2.2.6 Fallbacks & routing preferences

(Sources: https://openrouter.ai/docs/guides/routing/model-fallbacks, https://openrouter.ai/docs/features/provider-routing)

- **Model fallbacks**: an extra `models: ["primary", "fallback1", ...]` array in the body (the `model` field is the first attempt; via OpenAI SDK it goes in `extra_body`). Fallback triggers on *any* error: context-length validation, moderation flags, rate limits, downtime. **The response's `model` field always reports the model actually used, and pricing follows the actually-used model.** This "honest `model` echo" is the contract Zyquo Router's CLAUDE.md already mandates for its fallback chains.
- **Provider preferences** (`provider` object): `order` (try providers in this order), `allow_fallbacks` (default true), `require_parameters`, `ignore`/`only` allow/deny lists, `sort` by price/throughput/latency, `max_price`. Default load balancing prefers providers without recent outages, weighted by **inverse square of price**. Mostly N/A for a local single-key-per-provider router, but the *cooldown-on-recent-outage* idea maps to LiteLLM cooldowns.
- `finish_reason` is **normalized to exactly five values** — `stop`, `length`, `tool_calls`, `content_filter`, `error` — with the raw upstream value preserved in a separate `native_finish_reason` field (https://openrouter.ai/docs/api-reference/overview). Recommended verbatim for Zyquo Router (OpenAI's own set is the first four plus `function_call` legacy; `error` only ever appears mid-stream).

---

### 2.3 Local OpenAI-compatible servers — what "compatible enough" looks like

These show which subset of the spec real clients actually depend on, and which deviations are tolerated.

#### 2.3.1 Ollama (`http://localhost:11434/v1`)

(Source: https://docs.ollama.com/api/openai-compatibility)

- Endpoints: `/v1/chat/completions`, `/v1/completions`, `/v1/models`, `/v1/models/{model}`, `/v1/embeddings`, `/v1/responses`.
- Supported on chat: `model`, `messages`, `temperature`, `top_p`, `max_tokens`, `frequency_penalty`, `presence_penalty`, `seed`, `stop`, `stream`, `stream_options.include_usage`, `response_format` (JSON mode), `tools`, `reasoning_effort`/`reasoning`, vision via **base64 images only**.
- **Not supported**: `logprobs`, `user`, `n`, `tool_choice`, `logit_bias`, image **URLs**. Unsupported params are ignored rather than erroring.
- Auth: any/no API key accepted (pure localhost trust). Zyquo Router improves on this with optional local keys.
- Notable: Ollama had to add `max_completion_tokens` support because OpenAI deprecated `max_tokens` (https://github.com/ollama/ollama/issues/7125) — see §2.4.5.

Takeaways: even a hugely popular compat layer omits `n`, `logprobs`, `logit_bias` and (long) omitted `tool_choice` — clients broadly tolerate missing niche params, but `tools`, `response_format`, `stream_options.include_usage`, and vision are table stakes in 2026.

#### 2.3.2 LM Studio (`http://localhost:1234/v1`)

(Source: https://lmstudio.ai/docs/app/api/endpoints/openai)

- Endpoints: `/v1/models`, `/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, and **`/v1/responses`** (added specifically so OpenAI Codex CLI works against it).
- Compatibility story is purely "change `base_url` on the official SDK" — the same acceptance test Zyquo Router's Phase 3 gate uses.
- Signal: local servers are converging on also exposing `/v1/responses` because new OpenAI tooling (Codex) speaks only the Responses API. Relevant to the Phase 0 "is `/v1/responses` worth it" decision: *optional now, trending toward expected*.

#### 2.3.3 vLLM OpenAI-compatible server

(Source: https://docs.vllm.ai/en/latest/serving/online_serving/)

- Implements `/v1/chat/completions` (+ batch), `/v1/completions` (no `suffix`), `/v1/responses`, `/v1/embeddings`, transcription/translation.
- Known deviations: `user` is ignored; `parallel_tool_calls` defaults to `true`; extra sampling params (`top_k`, `best_of`, guided decoding) accepted via `extra_body` — the standard pass-through idiom again.
- vLLM's habit of sending **usage in a separate empty-`choices` chunk after the `finish_reason` chunk** is spec-correct but broke LiteLLM's consumer (https://github.com/BerriAI/litellm/issues/25389) — Zyquo Router's normalizer must handle both orderings when consuming OpenAI-compatible upstreams.

---

### 2.4 Compatibility pitfalls that trip up real clients

Strict SDKs (openai-python uses Pydantic models; openai-node/Vercel AI SDK use zod-like validation) parse every chunk. These are the documented, real-world failure modes a gateway must design around:

1. **Usage chunk shape.** With `stream_options: {"include_usage": true}`, OpenAI's contract is: every content chunk has `"usage": null`, and one **final extra chunk before `[DONE]`** has `"choices": []` and the full `usage` object. Emitting usage with non-empty choices violates the spec (LiteLLM bug https://github.com/BerriAI/litellm/issues/28735); conversely, clients that stop at `finish_reason` miss the usage chunk (https://github.com/BerriAI/litellm/issues/25389); AutoGen crashed on the empty-choices chunk itself (https://github.com/microsoft/autogen/issues/5078); llama.cpp put usage in a slightly different chunk than OpenAI and broke clients (https://github.com/ggml-org/llama.cpp/issues/15443). **Rule: emit usage exactly like OpenAI (separate final empty-choices chunk, only when requested), and when consuming, read to true end-of-stream and tolerate usage in any late chunk.**
2. **Streamed tool-call deltas.** The first `tool_calls` delta must carry `index`, `id`, `type: "function"`, and `function.name`; subsequent deltas carry only `index` + `function.arguments` fragments. Real breakage: Gemini-behind-a-compat-shim omitting `index` (https://github.com/anomalyco/opencode/issues/17902); providers sending `function.name` only in a *later* chunk (https://github.com/anomalyco/opencode/issues/24137, https://github.com/anomalyco/opencode/issues/26412); vLLM omitting `"type":"function"` under forced `tool_choice` (https://github.com/vllm-project/vllm/issues/16340). **Rule: the translator owns tool-call chunk assembly — always emit index/id/type/name complete in the first delta for each call, arguments-only after.**
3. **Role delta discipline.** The first chunk of each choice must have `delta: {"role": "assistant"}` (optionally with `content: ""`); an **empty-string role** instead of `"assistant"` breaks strict parsers (https://github.com/anomalyco/opencode/issues/28427). Later deltas must omit `role` entirely rather than repeat it as `""`.
4. **All-or-nothing chunk validation.** Some client stacks silently drop an entire chunk if any field fails validation — provider quirks then surface as *silently missing content*, which is undebuggable (https://github.com/Effect-TS/effect-smol/issues/2337, https://github.com/pydantic/pydantic-ai/issues/3658 — OpenRouter `reasoning_details` variant missing a field broke pydantic-ai). **Rule: every field Zyquo Router emits must be exactly typed (`created` as integer epoch seconds, `object` exactly `"chat.completion.chunk"`, `index` present on every choice/tool_call); when *adding* fields (e.g. `reasoning_content`), add only well-formed, consistently shaped ones.** Also: null-valued token-detail fields inside `usage` broke the OpenAI Agents SDK (https://github.com/openai/openai-agents-python/issues/1179) — omit detail objects rather than sending them with `null` members.
5. **`max_tokens` vs `max_completion_tokens`.** OpenAI deprecated `max_tokens` in favor of `max_completion_tokens`; o-series/reasoning models hard-reject `max_tokens` ("Unsupported parameter"). Every ecosystem project had to patch (Ollama https://github.com/ollama/ollama/issues/7125, simonw/llm https://github.com/simonw/llm/issues/724, Home Assistant https://github.com/home-assistant/core/issues/137039, Spring AI https://github.com/spring-projects/spring-ai/issues/3300). **Rule: accept both on ingress, normalize internally to one limit value, emit whichever the upstream requires (per-provider table), never forward both.**
6. **`system_fingerprint`.** Optional in practice — OpenAI itself returns `null`/absent for many models (https://github.com/openai/openai-python/issues/1038, https://github.com/openai/openai-openapi/issues/167), and SDK type defs treat it as optional (https://github.com/openai/openai-node/issues/443). Gateways may safely omit it or set `null`; do not fabricate values (clients use it for determinism tracking with `seed`).
7. **`n > 1`.** Most non-OpenAI upstreams don't support multiple choices (Ollama: unsupported; Anthropic/Gemini: no direct equivalent). Options: reject with a clear 400, or fan out N upstream calls. LiteLLM/Ollama precedent: reject or ignore. `choices` must still always be an **array** with correct `index` fields even for n=1 (OpenRouter: "choices is always an array" — https://openrouter.ai/docs/api-reference/overview).
8. **Keep-alives, buffering, and timeouts.** Long prefills (big prompts, reasoning models) can be silent for 30s+; intermediaries and client idle timeouts kill the connection (e.g. https://github.com/microsoft/agent-framework/issues/6941). SSE **comment lines** (`: keep-alive`) every 15–30s are the only spec-compatible heartbeat; also send `Cache-Control: no-cache`, `Connection: keep-alive`, `X-Accel-Buffering: no`, and never gzip SSE (compression layers buffer the stream). Conversely, when *consuming*, tolerate comment lines from upstreams (OpenRouter emits `: OPENROUTER PROCESSING` — https://openrouter.ai/docs/api-reference/streaming). Streaming responses need effectively unlimited write timeouts; non-streaming needs a generous but bounded upstream timeout mapped to 504/408.
9. **CORS for browser clients.** A cross-origin `fetch` to the router fails before the first byte without correct `Access-Control-Allow-Origin` + preflight handling for `POST` with `Authorization`/`Content-Type: application/json` headers. OpenRouter even classifies CORS problems under 400 (https://openrouter.ai/docs/api-reference/errors). Zyquo Router's default-permissive-on-localhost CORS (with `OPTIONS` preflight support) is the right call for browser-based dev tools.
10. **Mid-stream failure surfacing.** After the first chunk, the status line is committed (200). The only clean options are OpenRouter's error-chunk-with-`finish_reason:"error"` (§2.2.3) followed by `[DONE]`, or an abrupt close (which strict SDKs report as a network error). Emit the error chunk.

---

### 2.5 Recommended pattern set for Zyquo Router

Synthesis of the above into the concrete policy for our gateway:

1. **Model namespacing** — `provider/model-id` (LiteLLM/OpenRouter convention), full catalog from Zyquo Cloud; also accept unambiguous bare upstream IDs (resolve via catalog; ambiguous → 400 listing candidates) and user aliases (LiteLLM `model_group_alias` pattern). `GET /v1/models` returns OpenAI list shape with `id` = namespaced ID, enriched per-model metadata (`context_length`, pricing as decimal strings, capability flags, `supported_parameters`) under an `x-zyquo` key, following OpenRouter's metadata precedent.
2. **Unknown/unsupported-param policy** — data-driven per-provider tables (LiteLLM style): *rename* where names differ (`max_tokens`/`max_completion_tokens`, `stop`→`stop_sequences`), *strip silently* what the upstream would reject (default `drop_params: true` behavior, log at debug in the request inspector), *reject with a helpful OpenAI-format 400* only when silently dropping would change semantics materially (e.g. `tools` on a no-tools model, `n>1`), and *pass through* unknown extra body keys to OpenAI-compatible upstreams (the `extra_body` idiom) — documented per provider in `docs/API.md`.
3. **finish_reason & usage normalization** — normalize `finish_reason` to `stop | length | tool_calls | content_filter` (+ `error` mid-stream only), preserving the raw upstream value as `native_finish_reason` (OpenRouter pattern). Usage: real upstream numbers when available (request usage-in-stream from upstreams that support it, per-provider flag); tokenizer-estimated otherwise, flagged (e.g. `"x-zyquo": {"usage_estimated": true}`); cost computed from the catalog's pricing at response time and exposed OpenRouter-style as extra `usage` fields. Streaming usage emitted **only** when the client sends `stream_options.include_usage`, as a final `"choices": []` chunk before `[DONE]` — byte-exact per §2.4.1–4.
4. **Upstream error surfacing** — map to OpenAI error JSON `{"error":{"message","type","param","code"}}` with LiteLLM's status taxonomy (401 provider-key invalid naming the provider, 429 with propagated `Retry-After`, 400 subtypes for context-window/content-policy, 502 "upstream returned an invalid response", 503 provider unavailable, 504/408 timeouts); include a canonical machine `code` plus the upstream's original code in the message (OpenRouter's `error_type`+`provider_code` idea) — never raw upstream payloads or key material. Mid-stream: OpenRouter-style error chunk with `finish_reason: "error"`, then `[DONE]`.
5. **Retry/fallback policy** — per-error-class retry policy (LiteLLM `RetryPolicy`): retries with exponential backoff + jitter on 429/5xx/timeouts (honoring `Retry-After`), zero retries on 400/401/403; then user-configured fallback chains (ordered model lists) triggered on retry exhaustion, context-window and content-policy errors; the response `model` field reports the model actually used and cost follows it (OpenRouter contract). Optional per-provider cooldown state feeding the dashboard's "degraded" indicator.
6. **Health & liveness** — `GET /health` (status, version, uptime, active streams) never touches upstreams; per-provider "Test key" in the UI does a minimal authenticated upstream call and reports latency; SSE keep-alive comments every ~20s of upstream silence; client disconnect cancels the upstream task immediately (guaranteed, since we own the upstream HTTP call).

---

*Primary sources:* https://docs.litellm.ai/docs/proxy/configs · https://docs.litellm.ai/docs/completion/drop_params · https://docs.litellm.ai/docs/exception_mapping · https://docs.litellm.ai/docs/routing · https://docs.litellm.ai/docs/proxy/reliability · https://docs.litellm.ai/docs/completion/token_usage · https://docs.litellm.ai/docs/completion/stream · https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json · https://openrouter.ai/docs/api-reference/streaming · https://openrouter.ai/docs/api-reference/errors · https://openrouter.ai/docs/api-reference/overview · https://openrouter.ai/docs/use-cases/usage-accounting · https://openrouter.ai/docs/guides/routing/model-fallbacks · https://openrouter.ai/docs/features/provider-routing · https://openrouter.ai/docs/models · https://docs.ollama.com/api/openai-compatibility · https://lmstudio.ai/docs/app/api/endpoints/openai · https://docs.vllm.ai/en/latest/serving/online_serving/ · GitHub issues linked inline (§2.4).
## 3. Translation matrices

> Research date: 2026-07-30. Verified against live official documentation (URLs cited inline).
> This section is the contract for `Translate/AnthropicTranslator.swift`, `Translate/GeminiTranslator.swift`,
> and `Translate/CompatAdjuster.swift`. The router's canonical internal format IS the OpenAI
> `chat/completions` wire format (Section 1); every non-OpenAI upstream is mapped bidirectionally onto it.

---

### 3.1 Anthropic Messages API ⇄ OpenAI chat/completions

Primary sources:
- Messages API reference: https://platform.claude.com/docs/en/api/messages (docs.anthropic.com 301-redirects here)
- Streaming: https://platform.claude.com/docs/en/docs/build-with-claude/streaming
- Tool use: https://platform.claude.com/docs/en/docs/agents-and-tools/tool-use/overview
- Errors: https://platform.claude.com/docs/en/api/errors

#### 3.1.1 Endpoint & auth

| | OpenAI (what our client sends us) | Anthropic (what we send upstream) |
|---|---|---|
| Endpoint | `POST /v1/chat/completions` | `POST https://api.anthropic.com/v1/messages` |
| Auth header | `Authorization: Bearer zyquo-sk-…` (local key) | `x-api-key: <ANTHROPIC_KEY>` |
| Version header | — | `anthropic-version: 2023-06-01` (required) |
| Content type | `application/json` | `application/json` |

The `anthropic-version` header is **mandatory**; requests without it are rejected. Pin `2023-06-01` (the stable version used by all official SDKs).

#### 3.1.2 Request translation (OpenAI → Anthropic)

##### Parameter map

| OpenAI request field | Anthropic field | Rule |
|---|---|---|
| `model` | `model` | Strip the `anthropic/` namespace prefix. |
| `messages[role=system]`, `messages[role=developer]` | top-level `system` | **Extract** all system/developer messages (in order), join text with `"\n\n"`. Anthropic has no `system` role inside `messages`. `developer` is treated identically to `system`. |
| `messages[role=user/assistant/tool]` | `messages` | See message-shape rules below. |
| `max_tokens` / `max_completion_tokens` | `max_tokens` | **REQUIRED by Anthropic.** If the client omits both, the router MUST synthesize a value. Strategy: use the model's catalog `max_output` (from Zyquo Cloud's `ModelCatalog`); fall back to `4096` if unknown. `max_completion_tokens` wins if both present. |
| `temperature` | `temperature` | OpenAI range **0–2**, Anthropic range **0–1** (default 1.0). Strategy: `min(temperature, 1.0)` (clamp). Do NOT divide by 2 — halving changes semantics for the common 0–1 sub-range that clients actually use. Log a warning when clamping. |
| `top_p` | `top_p` | Same 0–1 range, pass through. Anthropic advises using temperature OR top_p, not both — pass both if given (API accepts it). |
| — (extra body `top_k`) | `top_k` | Not an OpenAI param; accept as pass-through extra key. |
| `stop` (string or array) | `stop_sequences` (array) | Wrap a bare string in a 1-element array. |
| `n` | — | **Unsupported.** If `n > 1` → reject with OpenAI 400 error (`invalid_request_error`, param `n`). |
| `frequency_penalty`, `presence_penalty`, `logit_bias`, `seed`, `logprobs`, `top_logprobs` | — | **Unsupported → strip silently** (LiteLLM behavior); optionally record in the request log that params were dropped. |
| `stream` | `stream` | Pass through. |
| `stream_options` | — | Router-side only (controls our usage chunk emission). Never forwarded. |
| `user` | `metadata.user_id` | Direct map (Anthropic wants an opaque non-PII id — pass as-is). |
| `tools` | `tools` | See tools mapping. |
| `tool_choice` | `tool_choice` | See tool_choice mapping. |
| `parallel_tool_calls: false` | `tool_choice.disable_parallel_tool_use: true` | Set on whatever `tool_choice` object we send (`auto` if none was specified). `parallel_tool_calls: true` → omit (default). |
| `response_format` | `output_config.format` / workaround | See JSON-mode strategy. |
| `reasoning_effort` (OpenAI standard) | `output_config.effort` / `thinking` | See §3.4 (reasoning). |
| extra body `thinking` | `thinking` | Pass-through extra key for power users: `{"type":"enabled"\|"adaptive"\|"disabled","budget_tokens":≥1024,"display":"summarized"\|"omitted"}`. |

##### Message-shape rules (the tricky part)

Anthropic enforces constraints that OpenAI does not:

1. **Roles are only `user` and `assistant`** inside `messages`.
2. **Turns must alternate.** Consecutive same-role messages must be **merged** into a single message whose `content` is an array of blocks, preserving order.
3. **The first message must be `user`.** If the client's first non-system message is `assistant`, prepend a placeholder user message (e.g. a single text block `"(continue)"`) or reject — the router merges/prepends (LiteLLM's approach) so real clients keep working.
4. **OpenAI `tool` role messages → `user` messages containing `tool_result` blocks.** Consecutive `tool` messages (parallel tool results) merge into ONE user message with multiple `tool_result` blocks. `tool_result` blocks must come FIRST in that user message's content array if user text follows.
5. **Assistant `tool_calls` → `tool_use` content blocks**, after any text content, with `input` as a **parsed JSON object** (OpenAI `arguments` is a JSON *string* — parse it; if unparseable, send `{}` and log).

##### Content-part map

| OpenAI content part | Anthropic content block |
|---|---|
| `{"type":"text","text":T}` | `{"type":"text","text":T}` |
| `{"type":"image_url","image_url":{"url":"data:image/png;base64,XXX"}}` | `{"type":"image","source":{"type":"base64","media_type":"image/png","data":"XXX"}}` — media_type parsed from the data URL; allowed: `image/jpeg`, `image/png`, `image/gif`, `image/webp`. |
| `{"type":"image_url","image_url":{"url":"https://…"}}` | `{"type":"image","source":{"type":"url","url":"https://…"}}` — Anthropic supports URL sources natively. |
| plain string `content` | plain string `content` (both APIs accept a bare string). |

##### Side-by-side request example (tools + image + system)

OpenAI request received by the router:

```json
{
  "model": "anthropic/claude-sonnet-4-5",
  "max_tokens": 1024,
  "temperature": 1.4,
  "messages": [
    {"role": "system", "content": "You are terse."},
    {"role": "user", "content": [
      {"type": "text", "text": "What's in this image, and what's the weather there?"},
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ..."}}
    ]},
    {"role": "assistant", "content": null, "tool_calls": [
      {"id": "call_abc123", "type": "function",
       "function": {"name": "get_weather", "arguments": "{\"location\": \"Paris\"}"}}
    ]},
    {"role": "tool", "tool_call_id": "call_abc123", "content": "18°C, sunny"}
  ],
  "tools": [
    {"type": "function", "function": {
      "name": "get_weather",
      "description": "Get current weather",
      "parameters": {"type": "object",
        "properties": {"location": {"type": "string"}}, "required": ["location"]}}}
  ],
  "tool_choice": "auto",
  "parallel_tool_calls": false
}
```

Anthropic request the router sends upstream:

```json
{
  "model": "claude-sonnet-4-5",
  "max_tokens": 1024,
  "temperature": 1.0,
  "system": "You are terse.",
  "messages": [
    {"role": "user", "content": [
      {"type": "text", "text": "What's in this image, and what's the weather there?"},
      {"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": "/9j/4AAQ..."}}
    ]},
    {"role": "assistant", "content": [
      {"type": "tool_use", "id": "call_abc123", "name": "get_weather",
       "input": {"location": "Paris"}}
    ]},
    {"role": "user", "content": [
      {"type": "tool_result", "tool_use_id": "call_abc123", "content": "18°C, sunny"}
    ]}
  ],
  "tools": [
    {"name": "get_weather", "description": "Get current weather",
     "input_schema": {"type": "object",
       "properties": {"location": {"type": "string"}}, "required": ["location"]}}
  ],
  "tool_choice": {"type": "auto", "disable_parallel_tool_use": true}
}
```

Note the tool_use `id` is preserved verbatim in both directions so multi-turn tool loops round-trip.

##### Tools & tool_choice map

| OpenAI | Anthropic |
|---|---|
| `tools[].function.name` | `tools[].name` |
| `tools[].function.description` | `tools[].description` |
| `tools[].function.parameters` (JSON Schema) | `tools[].input_schema` (JSON Schema draft 2020-12) |
| `tools[].function.strict: true` | `tools[].strict: true` (now supported natively) |
| `tool_choice: "auto"` (or omitted with tools) | `{"type": "auto"}` |
| `tool_choice: "required"` | `{"type": "any"}` |
| `tool_choice: "none"` | `{"type": "none"}` |
| `tool_choice: {"type":"function","function":{"name":N}}` | `{"type": "tool", "name": N}` |
| `parallel_tool_calls: false` | `disable_parallel_tool_use: true` on the tool_choice object |

##### JSON mode / `response_format` strategy

Anthropic historically had **no native JSON mode**; the current API (2026) has **structured output** via
`output_config.format` (source: https://platform.claude.com/docs/en/api/messages — `output_config: { format: { type: "json_schema", schema: {...} } }`).

Router strategy, in priority order:

1. `response_format: {"type":"json_schema","json_schema":{"schema":S,...}}` → `output_config: {"format": {"type":"json_schema","schema": S}}` on models that support it (catalog capability flag).
2. On models WITHOUT structured-output support — **tool trick**: define a single synthetic tool (`name: "json_output"`, `input_schema: S`), force it with `tool_choice: {"type":"tool","name":"json_output"}`, and return the streamed/collected `input` object as the assistant `content` string (finish_reason `stop`, not `tool_calls`).
3. `response_format: {"type":"json_object"}` → append to `system`: `"You must respond with valid JSON only, no prose, no markdown fences."` and optionally **prefill** the assistant turn with `{` (append `{"role":"assistant","content":"{"}` and re-prepend `{` to the returned text). Prefill is Anthropic-sanctioned steering. Document in `docs/API.md` that json_object on Anthropic is best-effort.
4. `{"type":"text"}` → no-op.

##### Extended thinking (request side)

- `thinking: {"type":"enabled","budget_tokens":N}` (N ≥ 1024, must be < `max_tokens`) or `{"type":"adaptive"}`; `display: "summarized"|"omitted"` controls whether thinking text is streamed.
- Newer models also take `output_config.effort: "low"|"medium"|"high"|"xhigh"|"max"`.
- Router mapping for the standard OpenAI `reasoning_effort` param: `low → output_config.effort "low"` (or `thinking budget 1024`), `medium → "medium"` (8192), `high → "high"` (24576) — per-model capability gate from the catalog. Raw `thinking` extra-body always wins if provided.
- **Multi-turn constraint:** in tool-use loops with thinking enabled, Anthropic expects prior `thinking` blocks (with `signature`) to be passed back. See §3.4 for how the router preserves signatures via `reasoning_details`.

#### 3.1.3 Response translation (Anthropic → OpenAI), non-streaming

Anthropic response:

```json
{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "model": "claude-sonnet-4-5",
  "content": [
    {"type": "text", "text": "It's 18°C and sunny in Paris."},
    {"type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9",
     "name": "get_weather", "input": {"location": "Paris"}}
  ],
  "stop_reason": "tool_use",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 412, "output_tokens": 61,
    "cache_creation_input_tokens": 0, "cache_read_input_tokens": 128
  }
}
```

Router emits:

```json
{
  "id": "chatcmpl-9f3c1a2b7d4e",
  "object": "chat.completion",
  "created": 1753872000,
  "model": "anthropic/claude-sonnet-4-5",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "It's 18°C and sunny in Paris.",
      "tool_calls": [{
        "id": "toolu_01A09q90qw90lq917835lq9",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"location\":\"Paris\"}"}
      }]
    },
    "finish_reason": "tool_calls"
  }],
  "usage": {
    "prompt_tokens": 540,
    "completion_tokens": 61,
    "total_tokens": 601,
    "prompt_tokens_details": {"cached_tokens": 128}
  }
}
```

Rules:

- **`id`**: generate a fresh `chatcmpl-<hex>` (keep the upstream `msg_…` id in the request log for tracing). `object: "chat.completion"`, `created`: gateway clock (Unix seconds). `model`: echo the **namespaced** router id.
- **Content blocks →** concatenate all `text` block texts into `message.content` (`null` if none and tool_calls exist); each `tool_use` block → one `tool_calls[]` entry with `arguments` = `JSON.stringify(input)`; `thinking` blocks → `message.reasoning_content` (§3.4), `signature` → `reasoning_details`; `redacted_thinking` → `reasoning_details` only (opaque `data`).
- **`stop_reason` → `finish_reason`:**

| Anthropic `stop_reason` | OpenAI `finish_reason` | Note |
|---|---|---|
| `end_turn` | `stop` | |
| `max_tokens` | `length` | |
| `stop_sequence` | `stop` | OpenAI has no separate value; log the matched `stop_sequence`. |
| `tool_use` | `tool_calls` | |
| `refusal` | `content_filter` | Closest OpenAI semantic; `stop_details` (category/explanation) goes to the request log only. |
| `pause_turn` | `stop` | Long-running server-tool turns; router doesn't use server tools, treat as stop. |
| `model_context_window_exceeded` | `length` | |

- **`usage`:** Anthropic's `input_tokens` EXCLUDES cache reads/writes. Normalize:
  `prompt_tokens = input_tokens + cache_read_input_tokens + cache_creation_input_tokens`;
  `completion_tokens = output_tokens` (already includes thinking tokens);
  `total_tokens = prompt + completion`;
  `prompt_tokens_details.cached_tokens = cache_read_input_tokens`;
  `completion_tokens_details.reasoning_tokens = usage.output_tokens_details.thinking_tokens` when present.

#### 3.1.4 SSE event model → OpenAI `chat.completion.chunk` (streaming)

Anthropic stream grammar (source: https://platform.claude.com/docs/en/docs/build-with-claude/streaming):

```
message_start
( content_block_start → content_block_delta* → content_block_stop )*
message_delta+
message_stop
```
with `ping` events anywhere and possible `error` events. Each SSE frame is
`event: <name>\ndata: <json>\n\n`. `message_delta.usage.output_tokens` is **cumulative**.

Delta types inside `content_block_delta`: `text_delta` (`.text`), `input_json_delta` (`.partial_json`, a partial JSON **string** for `tool_use.input`), `thinking_delta` (`.thinking`), `signature_delta` (`.signature`, arrives just before the thinking block's `content_block_stop`).

##### Event → chunk mapping table

The translator keeps two counters: `toolIdx` = number of `tool_use` blocks seen so far (this is the OpenAI `tool_calls[].index`, 0-based, **independent of the Anthropic block `index`**), and cumulative usage.

| Anthropic event | Emitted OpenAI chunk delta | Notes |
|---|---|---|
| `message_start` | `{"delta":{"role":"assistant","content":""},"finish_reason":null}` | First chunk; role delta exactly once. Capture `message.usage.input_tokens` (+cache fields) for the final usage chunk. |
| `ping` | *(nothing)* — or forward as SSE comment `: keep-alive` | Comments keep clients' sockets warm without confusing SDK parsers. |
| `content_block_start` (`type:"text"`) | *(nothing)* | |
| `content_block_delta` / `text_delta` | `{"delta":{"content": text}}` | |
| `content_block_start` (`type:"tool_use"`) | `{"delta":{"tool_calls":[{"index": toolIdx, "id": block.id, "type": "function", "function": {"name": block.name, "arguments": ""}}]}}` | id + name announced once, `arguments:""` starts the accumulator — exactly the shape the OpenAI SDKs expect. |
| `content_block_delta` / `input_json_delta` | `{"delta":{"tool_calls":[{"index": toolIdx, "function": {"arguments": partial_json}}]}}` | No id/name repetition. Empty `partial_json` frames may be skipped. |
| `content_block_stop` (tool_use) | *(nothing)*; `toolIdx += 1` | |
| `content_block_delta` / `thinking_delta` | `{"delta":{"reasoning_content": thinking}}` | §3.4 convention. |
| `content_block_delta` / `signature_delta` | `{"delta":{"reasoning_details":[{"type":"anthropic.signature","signature":…,"index":blockIdx}]}}` — or drop if client didn't opt in | Needed only to round-trip thinking in tool loops. |
| `content_block_start/stop` (`type:"thinking"`) | *(nothing)* | |
| `message_delta` | `{"delta":{},"finish_reason": map(stop_reason)}` | finish_reason chunk (empty delta object, per OpenAI spec). Capture cumulative `usage.output_tokens`. |
| `message_stop` | If `stream_options.include_usage`: `{"choices":[],"usage":{…}}` chunk; then `data: [DONE]` | Usage chunk has an EMPTY `choices` array per OpenAI spec. Then terminate. |
| `error` | Emit `data: {"error":{"message":…,"type":"api_error","code":upstream_type}}` then close | OpenAI has no in-band stream-error spec; this LiteLLM-style error frame is the least-bad option — document it in `docs/API.md`. Map `overloaded_error` → our 529→503 semantics in logs. |

Every emitted chunk carries the constant envelope:
`{"id":"chatcmpl-…","object":"chat.completion.chunk","created":C,"model":"anthropic/…","choices":[{"index":0,"delta":…,"finish_reason":…}]}` — same `id`/`created` for the whole stream.

##### Full example transcript (tool-use stream)

Anthropic events (left) → OpenAI chunks emitted by the router (right). Envelope fields elided for readability; every right-hand line is a full `chat.completion.chunk`.

```
ANTHROPIC UPSTREAM                                          → ZYQUO ROUTER EMITS (OpenAI SSE)

event: message_start
data: {"type":"message_start","message":{"id":"msg_014p",  → data: {"id":"chatcmpl-a1","object":"chat.completion.chunk",
  "role":"assistant","content":[],"model":"claude-…",          "created":1753872000,"model":"anthropic/claude-sonnet-4-5",
  "usage":{"input_tokens":472,"output_tokens":2}, …}}          "choices":[{"index":0,"delta":{"role":"assistant",
                                                               "content":""},"finish_reason":null}]}

event: content_block_start
data: {"type":"content_block_start","index":0,             → (nothing)
  "content_block":{"type":"text","text":""}}

event: ping
data: {"type":"ping"}                                      → (nothing, or ": keep-alive" comment)

event: content_block_delta
data: {…,"delta":{"type":"text_delta","text":"Okay,"}}     → data: {…,"choices":[{"index":0,"delta":{"content":"Okay,"},
                                                               "finish_reason":null}]}

event: content_block_delta
data: {…,"delta":{"type":"text_delta",                     → data: {…,"choices":[{"index":0,"delta":{"content":
  "text":" checking the weather:"}}                            " checking the weather:"},"finish_reason":null}]}

event: content_block_stop
data: {"type":"content_block_stop","index":0}              → (nothing)

event: content_block_start
data: {"type":"content_block_start","index":1,             → data: {…,"choices":[{"index":0,"delta":{"tool_calls":[
  "content_block":{"type":"tool_use",                          {"index":0,"id":"toolu_01T1x","type":"function",
  "id":"toolu_01T1x","name":"get_weather","input":{}}}         "function":{"name":"get_weather","arguments":""}}]},
                                                               "finish_reason":null}]}

event: content_block_delta
data: {…,"delta":{"type":"input_json_delta",               → (skipped — empty partial_json)
  "partial_json":""}}

event: content_block_delta
data: {…,"delta":{"type":"input_json_delta",               → data: {…,"choices":[{"index":0,"delta":{"tool_calls":[
  "partial_json":"{\"location\":"}}                            {"index":0,"function":{"arguments":"{\"location\":"}}]},
                                                               "finish_reason":null}]}

event: content_block_delta
data: {…,"delta":{"type":"input_json_delta",               → data: {…,"choices":[{"index":0,"delta":{"tool_calls":[
  "partial_json":" \"San Francisco, CA\"}"}}                   {"index":0,"function":{"arguments":
                                                               " \"San Francisco, CA\"}"}}]},"finish_reason":null}]}

event: content_block_stop
data: {"type":"content_block_stop","index":1}              → (nothing; toolIdx→1)

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":      → data: {…,"choices":[{"index":0,"delta":{},
  "tool_use","stop_sequence":null},                            "finish_reason":"tool_calls"}]}
  "usage":{"output_tokens":89}}

event: message_stop                                        → data: {…,"choices":[],"usage":{"prompt_tokens":472,
data: {"type":"message_stop"}                                  "completion_tokens":89,"total_tokens":561}}
                                                               (only if stream_options.include_usage)
                                                           → data: [DONE]
```

Thinking streams follow the same pattern: `content_block_start {type:"thinking"}` opens nothing, each `thinking_delta` → `{"delta":{"reasoning_content":"…"}}`, `signature_delta` → `reasoning_details` (or dropped), then the text block streams as normal `content` deltas.

---

### 3.2 Gemini `generateContent` / `streamGenerateContent` ⇄ OpenAI

Primary sources:
- API reference: https://ai.google.dev/api/generate-content
- Part/Content schema: https://ai.google.dev/api/caching#Part
- Function calling: https://ai.google.dev/gemini-api/docs/function-calling
- Structured output: https://ai.google.dev/gemini-api/docs/structured-output
- Thinking: https://ai.google.dev/gemini-api/docs/thinking
- Google's own OpenAI-compat layer (used as a mapping oracle): https://ai.google.dev/gemini-api/docs/openai

#### 3.2.1 Endpoint & auth

| | Form |
|---|---|
| Non-streaming | `POST https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent` |
| Streaming (SSE) | `POST https://generativelanguage.googleapis.com/v1beta/models/{model}:streamGenerateContent?alt=sse` |
| Auth | Header `x-goog-api-key: <GEMINI_KEY>` (preferred) or query `?key=<GEMINI_KEY>` |

**Always use `?alt=sse`** — without it, streamGenerateContent returns a chunked **JSON array**, not SSE. The model name is in the **path**, not the body. Use the header for auth so the key never appears in URLs/logs.

#### 3.2.2 Request translation (OpenAI → Gemini)

| OpenAI field | Gemini field | Rule |
|---|---|---|
| `model` | URL path | Strip `gemini/` prefix. |
| system/developer messages | `systemInstruction: {"parts":[{"text": joined}]}` | Join multiple with `"\n\n"`. |
| `messages[role=user]` | `contents[]` entry `role: "user"` | |
| `messages[role=assistant]` | `contents[]` entry `role: "model"` | **Role rename user/assistant → user/model.** |
| `messages[role=tool]` | `contents[]` entry `role: "user"` with `functionResponse` part(s) | See tool round-trip below. Consecutive `tool` messages merge into one user-role content with multiple `functionResponse` parts. |
| `max_tokens`/`max_completion_tokens` | `generationConfig.maxOutputTokens` | Optional on Gemini (nice: no synthesis needed). |
| `temperature` | `generationConfig.temperature` | Both 0–2. Pass through unchanged. |
| `top_p` | `generationConfig.topP` | camelCase rename. |
| extra `top_k` | `generationConfig.topK` | |
| `stop` | `generationConfig.stopSequences` | Wrap string → array. |
| `n` | `generationConfig.candidateCount` (1–8) | Router policy: support `n` here (one of the few upstreams that can) or clamp to 1 for uniformity — **decide once; recommend rejecting n>1 router-wide** for consistent behavior across providers. |
| `seed` | `generationConfig.seed` | |
| `presence_penalty` | `generationConfig.presencePenalty` | −2..2, same semantics. |
| `frequency_penalty` | `generationConfig.frequencyPenalty` | |
| `logit_bias`, `logprobs`, `user` | — | Strip. |
| `response_format {"type":"json_object"}` | `generationConfig.responseMimeType: "application/json"` | |
| `response_format {"type":"json_schema",…}` | `responseMimeType: "application/json"` + `generationConfig.responseJsonSchema` (standard JSON Schema; older models: `responseSchema` OpenAPI-subset) | Prefer `responseJsonSchema`; scrub unsupported keywords (`$schema`, `additionalProperties` on old models) defensively. |
| `tools` | `tools: [{"functionDeclarations":[{name, description, parameters}]}]` | ALL functions go into ONE `functionDeclarations` array. `parameters` is JSON-Schema-like; scrub `strict`. |
| `tool_choice` | `toolConfig.functionCallingConfig` | `"auto"`→`{"mode":"AUTO"}`; `"required"`→`{"mode":"ANY"}`; `"none"`→`{"mode":"NONE"}`; `{"function":{"name":N}}`→`{"mode":"ANY","allowedFunctionNames":[N]}`. (A `VALIDATED` mode also exists; unused by the router.) |
| `parallel_tool_calls` | — | No Gemini equivalent; strip. Gemini decides parallelism itself (multiple `functionCall` parts in one candidate). |
| `reasoning_effort` | `generationConfig.thinkingConfig` | Gemini 2.5: `low→thinkingBudget 1024`, `medium→8192`, `high→24576`; Gemini 3+: `thinkingLevel: "low"/"medium"/"high"`. (This is Google's own mapping in their OpenAI-compat layer.) Add `includeThoughts: true` when the client opted into reasoning output. |
| `stream` | endpoint choice | `stream:true` → `:streamGenerateContent?alt=sse`. |

##### Content-part map

| OpenAI part | Gemini part |
|---|---|
| `{"type":"text","text":T}` | `{"text": T}` |
| `image_url` with `data:` URL | `{"inlineData": {"mimeType": "image/png", "data": "<base64>"}}` |
| `image_url` with `https://` URL | **Gemini cannot fetch arbitrary URLs** (`fileData.fileUri` requires the Google File API). Router policy: download the image itself (size-capped, e.g. 20 MB) and convert to `inlineData`; on failure return an OpenAI 400 error naming the URL. |

##### Tool round-trip (functionCall / functionResponse)

OpenAI assistant `tool_calls` → Gemini model turn:

```json
{"role": "model", "parts": [
  {"functionCall": {"id": "call_abc123", "name": "get_weather",
    "args": {"location": "Paris"}}}
]}
```

(`args` is a parsed object — parse the OpenAI `arguments` string. `functionCall.id` / `functionResponse.id` exist in current v1beta for parallel-call matching; include them when the OpenAI ids are available.)

OpenAI `tool` message → Gemini user turn:

```json
{"role": "user", "parts": [
  {"functionResponse": {"id": "call_abc123", "name": "get_weather",
    "response": {"result": "18°C, sunny"}}}
]}
```

Two traps:
1. **`functionResponse.response` must be a JSON OBJECT.** OpenAI tool content is a string → if it parses as a JSON object, pass it; otherwise wrap as `{"result": <string>}`.
2. **`name` is required**, but OpenAI `tool` messages carry only `tool_call_id`. The router must resolve `tool_call_id → name` from the preceding assistant message's `tool_calls` in the same request payload (always available in a well-formed OpenAI conversation).
3. **Thought signatures (Gemini 3+):** function-call parts may carry a `thoughtSignature` that should be echoed back on the following turn. Preserve it via `reasoning_details` (§3.4) and re-attach when translating the conversation back.

##### Side-by-side minimal request

```json
// OpenAI in                                    // Gemini out
{                                               {
  "model": "gemini/gemini-2.5-flash",             // POST …/models/gemini-2.5-flash:generateContent
  "messages": [                                   "systemInstruction": {"parts":[{"text":"Be brief."}]},
    {"role":"system","content":"Be brief."},      "contents": [
    {"role":"user","content":"Hi"},                 {"role":"user","parts":[{"text":"Hi"}]},
    {"role":"assistant","content":"Hello!"},        {"role":"model","parts":[{"text":"Hello!"}]},
    {"role":"user","content":"Name a color"}        {"role":"user","parts":[{"text":"Name a color"}]}
  ],                                              ],
  "temperature": 0.7,                             "generationConfig": {
  "max_tokens": 100,                                "temperature": 0.7,
  "stop": ["\n\n"]                                  "maxOutputTokens": 100,
}                                                   "stopSequences": ["\n\n"]
                                                  }
                                                }
```

#### 3.2.3 Response translation (Gemini → OpenAI)

Gemini response:

```json
{
  "candidates": [{
    "content": {"role": "model", "parts": [
      {"functionCall": {"name": "get_weather", "args": {"location": "Paris"}}}
    ]},
    "finishReason": "STOP",
    "index": 0,
    "safetyRatings": [ … ]
  }],
  "usageMetadata": {
    "promptTokenCount": 57, "candidatesTokenCount": 12,
    "thoughtsTokenCount": 88, "totalTokenCount": 157
  },
  "modelVersion": "gemini-2.5-flash"
}
```

Router emits:

```json
{
  "id": "chatcmpl-7be2f0c4",
  "object": "chat.completion",
  "created": 1753872000,
  "model": "gemini/gemini-2.5-flash",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": null,
      "tool_calls": [{"id": "call_9d1e2f3a", "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"location\":\"Paris\"}"}}]},
    "finish_reason": "tool_calls"
  }],
  "usage": {
    "prompt_tokens": 57,
    "completion_tokens": 100,
    "total_tokens": 157,
    "completion_tokens_details": {"reasoning_tokens": 88}
  }
}
```

Rules:

- Concatenate `parts[].text` (where `part.thought != true`) → `content`; parts with `"thought": true` → `reasoning_content`; each `functionCall` part → a `tool_calls[]` entry. **Gemini often omits ids** → synthesize `call_<12-hex>` (and remember name↔id for the return trip).
- **CRITICAL finish_reason rule:** Gemini reports `finishReason: "STOP"` even when the candidate contains `functionCall` parts. The router must emit `finish_reason: "tool_calls"` whenever any functionCall part is present, regardless of `finishReason`.

| Gemini `finishReason` | OpenAI `finish_reason` |
|---|---|
| `STOP` (with functionCall parts) | `tool_calls` |
| `STOP` | `stop` |
| `MAX_TOKENS` | `length` |
| `SAFETY`, `PROHIBITED_CONTENT`, `BLOCKLIST`, `SPII`, `IMAGE_SAFETY` | `content_filter` |
| `RECITATION` | `content_filter` (recitation = copyright block) |
| `MALFORMED_FUNCTION_CALL` | map to a **502-style OpenAI error** on non-streaming (the candidate is unusable); on streaming, emit finish_reason `stop` + log |
| `LANGUAGE`, `OTHER`, unknown | `stop` (+ log the raw value) |

- **Blocked prompts:** if `candidates` is empty and `promptFeedback.blockReason` is set (`SAFETY`, `BLOCKLIST`, `PROHIBITED_CONTENT`, `OTHER`, `IMAGE_SAFETY`), return an OpenAI 400 `invalid_request_error` with a clear message naming the block reason — never an empty 200.
- **usage:** `prompt_tokens = promptTokenCount`; `completion_tokens = candidatesTokenCount + thoughtsTokenCount` (OpenAI counts reasoning inside completion tokens); `total_tokens = totalTokenCount`; `completion_tokens_details.reasoning_tokens = thoughtsTokenCount`; `prompt_tokens_details.cached_tokens = cachedContentTokenCount`.

#### 3.2.4 Streaming (`streamGenerateContent?alt=sse`) → OpenAI chunks

Each SSE `data:` line is a **complete `GenerateContentResponse`** whose `candidates[0].content.parts` holds the *increment* — there is no delta envelope and **no `[DONE]` terminator** (the stream simply ends after the chunk carrying the final `finishReason`). `usageMetadata` appears on chunks with cumulative counts; the last chunk has the authoritative totals.

```
GEMINI SSE                                                 → ROUTER EMITS

data: {"candidates":[{"content":{"parts":[{"text":"The"}], → chunk 1 (synthesized role first):
  "role":"model"},"index":0}],                               data: {…,"delta":{"role":"assistant","content":""},…}
  "usageMetadata":{…},"modelVersion":"gemini-2.5-flash"}     data: {…,"delta":{"content":"The"},"finish_reason":null}

data: {"candidates":[{"content":{"parts":[{"text":          → data: {…,"delta":{"content":" sky is blue."},
  " sky is blue."}],"role":"model"},"index":0}],…}             "finish_reason":null}

data: {"candidates":[{"content":{"parts":[],"role":        → data: {…,"delta":{},"finish_reason":"stop"}
  "model"},"finishReason":"STOP","index":0}],              → data: {…,"choices":[],"usage":{"prompt_tokens":8,
  "usageMetadata":{"promptTokenCount":8,                       "completion_tokens":5,"total_tokens":13}}
  "candidatesTokenCount":5,"totalTokenCount":13}}              (if include_usage)
(stream ends — no [DONE] from Gemini)                      → data: [DONE]   (router ALWAYS adds it)
```

Rules:
- **Synthesize the role chunk**: Gemini has no role-only first frame; the router emits `{"role":"assistant","content":""}` before the first content delta.
- **Tool calls are NOT argument-streamed**: a `functionCall` part arrives complete in one chunk → emit ONE `tool_calls` delta containing `index`, generated `id`, `name`, and the FULL `arguments` string; strict SDKs accept whole-argument single deltas fine.
- Thought parts (`"thought": true`) → `reasoning_content` deltas.
- `finishReason` on the final chunk → the finish_reason chunk (apply the functionCall→`tool_calls` override).
- Router appends the OpenAI usage chunk (empty `choices`) and `data: [DONE]` itself.
- If Gemini aborts mid-stream with `finishReason: SAFETY`, emit `finish_reason: "content_filter"` and terminate normally.

---

### 3.3 OpenAI-compatible providers — deviation table

All nine below speak the OpenAI chat/completions wire format closely enough for **near-pass-through**: the router's `CompatAdjuster` only needs a per-provider strip/rename/allow table plus finish/usage normalization. Auth is `Authorization: Bearer <key>` for all of them. (OpenAI itself, `api.openai.com/v1`, is the reference and needs no adjustment.)

#### Summary matrix

| Provider | Base URL | Strip / rename | Extra params to allow (pass-through) | Quirks |
|---|---|---|---|---|
| **xAI** | `https://api.x.ai/v1` | For Grok-4-family reasoning models: **strip `presence_penalty`, `frequency_penalty`, `stop`** (they 400, not ignore). Strip `reasoning_effort` on models that reject it. | `reasoning_effort` (model-gated: grok-3-mini; grok-4.3 `none/low/medium/high`; grok-4.5 `low/medium/high` only), `search_parameters` (Live Search), `deferred` | grok-3-mini returns `message.reasoning_content`; grok-4 does NOT expose reasoning content, only `usage.completion_tokens_details.reasoning_tokens`. Vision via standard `image_url` (jpeg/png, ≤20 MiB). Structured outputs supported. Chat Completions is now labeled a "legacy" endpoint (Responses API is primary) but remains fully supported. |
| **Mistral** | `https://api.mistral.ai/v1` | **Rename `seed` → `random_seed`.** Strip `logit_bias`, `user`, `logprobs`. | `safe_prompt` (bool), `prompt_mode: "reasoning"`, `prediction`, `prompt_cache_key`, tool_choice value `"any"` | `tool_choice` accepts `auto/none/any/required` (`any`≈`required`). Temperature recommended 0–0.7. `response_format` supports `json_object` AND `json_schema`. Magistral reasoning models return `message.content` as an ARRAY of chunks: `{"type":"thinking","thinking":[{"type":"text","text":…}]}` + `{"type":"text","text":…}` — router must flatten: thinking chunks → `reasoning_content`, text chunks → `content` (same in streaming deltas, which shape-shift between array and string). SSE ends with `[DONE]`. |
| **DashScope / Qwen (intl)** | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | Strip `logit_bias`. `n` forced to 1 when `tools` present. | `enable_thinking` (bool), `thinking_budget` (int) — both extra-body; `translation_options` | Hybrid-thinking models (qwen3/qwen-plus) emit `delta.reasoning_content` then `delta.content`. **Some open-source thinking models are streaming-only** (non-streaming call errors) → router should transparently stream-and-aggregate when a client asks non-streaming. `stream_options.include_usage` supported (usage in final chunk). Vision (qwen-vl) uses standard `image_url` parts. Newer qwen models enable thinking by default — set `enable_thinking:false` when the client didn't ask for reasoning. |
| **DeepSeek** | `https://api.deepseek.com` (alias `…/v1`) | For reasoner/thinking mode: `temperature`, `top_p`, `presence_penalty`, `frequency_penalty` are **silently ignored** (no strip needed, but don't pretend they work); `logprobs`/`top_logprobs` error → strip. **Strip `reasoning_content` from incoming assistant messages** except in tool-call loops (see quirks). | `thinking: {"type":"enabled"/"disabled"}` (extra-body), `reasoning_effort` | `message.reasoning_content` + `delta.reasoning_content` (the convention we adopt, §3.4). Multi-turn: reasoning_content must NOT be resent in ordinary turns, but in TOOL-CALL loops it **must be passed back** or the API 400s (current docs). JSON mode: `response_format {"type":"json_object"}` requires the word "json" in the prompt — router auto-appends an instruction if missing. Tools supported. Usage includes `prompt_cache_hit_tokens` / `prompt_cache_miss_tokens` → map hit tokens to `prompt_tokens_details.cached_tokens`. |
| **Kimi / Moonshot** | `https://api.moonshot.ai/v1` | **Clamp `temperature` to [0,1]** (out-of-range 400s; default 0.0). `stop` max 5 sequences × 32 bytes → truncate/reject. K3 reasoning model: strip `temperature`, `top_p`, penalties. | `partial: true` on last assistant message (prefill mode), `thinking: {"type":…}`, `reasoning_effort` (`low/high/max` on K3), `thinking.keep` | `n` 1–5. `response_format`: `json_object` and `json_schema` (strict). Vision + video via `image_url`/`video_url` (base64 or `ms://<file_id>`). Thinking models return `reasoning_content` (message + delta). `stream_options.include_usage` → usage in final pre-`[DONE]` chunk. |
| **Perplexity** | `https://api.perplexity.ai` (`POST /chat/completions`; docs now also expose a gateway at `/router/v1/chat/completions`) | Strip `frequency_penalty`/`presence_penalty`/`top_k` if absent from current schema; **reject `tools`** on Sonar search models (no function calling) with a clear OpenAI error. Messages after `system` must **strictly alternate user/assistant** — merge consecutive same-role messages like the Anthropic path. | `search_mode` (`web/academic/sec`), `search_domain_filter`, `search_recency_filter`, `search_after_date_filter`/`search_before_date_filter`, `web_search_options` (`search_context_size`, `user_location`), `return_images`, `return_related_questions`, `disable_search`, `reasoning_effort`, `language_preference` | Response carries top-level `citations: [urls]` and `search_results: [{title,url,snippet,date}]` — the router passes these through verbatim as extension keys on the normalized response (and on the final stream chunk). `usage` extras: `citation_tokens`, `num_search_queries`, `reasoning_tokens`, `cost{…}` → keep under usage extension, fold `reasoning_tokens` into `completion_tokens_details`. sonar-reasoning models emit `<think>…</think>` inside `content` → router extracts to `reasoning_content`. |
| **Together** | `https://api.together.xyz/v1` | Nothing mandatory to strip. | `top_k`, `min_p`, `repetition_penalty`, `safety_model`, `echo`, `context_length_exceeded_behavior`, `chat_template_kwargs` | **finish_reason may be `"eos"`** → normalize to `"stop"` (also normalize `function_call`→`tool_calls` if seen). Usage appears in the final chunk (and on some models in every chunk) — always take the LAST non-null usage. `json_schema` support is model-dependent → 400s surface as OpenAI errors. Open-weights models vary in tool-calling quality; nothing structural to translate. |
| **DeepInfra** | `https://api.deepinfra.com/v1/openai` | Strip `logit_bias` (unsupported on most models). | `min_p`, `repetition_penalty`, `service_tier` (`priority`/`flex`), `fail_fast` | Usage object includes non-standard **`estimated_cost`** (USD) → feed it straight into `UsageMeter` as authoritative cost when present. Final stream chunk carries usage. Self-described as "not 100% compatible with all OpenAI parameters" — treat unknown-param 400s as strippable and retry once without extras. |
| **Cerebras** | `https://api.cerebras.ai/v1` | **Vision: base64 data-URI images only — remote `image_url` http(s) URLs are rejected** → router inlines (download + base64) or rejects with a clear error. `response_format {"type":"json_object"}` is **incompatible with streaming** → reject that combination or fall back to json_schema. | `reasoning_effort` (`low/medium/high/none`), `clear_thinking` (Cerebras-specific: drop prior-turn reasoning), `service_tier`, `prediction`, `prompt_cache_key` | Extremely high tokens/s — the SSE writer must handle very fast chunk cadence (backpressure!). Usage included in stream. Extra response fields `time_info` (queue/prompt/completion latencies) and `service_tier_used` → log, don't forward. `max_completion_tokens` preferred name. Supports `logprobs`, penalties, `logit_bias` per current docs (verify in Phase 7). |

Sources: xAI — https://docs.x.ai/developers/model-capabilities/legacy/chat-completions, https://www.promptfoo.dev/docs/providers/xai/ ; Mistral — https://docs.mistral.ai/api/ , https://docs.mistral.ai/capabilities/reasoning/ ; DashScope — https://www.alibabacloud.com/help/en/model-studio/compatibility-of-openai-with-dashscope , https://www.alibabacloud.com/help/en/model-studio/deep-thinking ; DeepSeek — https://api-docs.deepseek.com/guides/thinking_mode ; Kimi — https://platform.kimi.ai/docs/api/chat (platform.moonshot.ai redirects here; API host remains api.moonshot.ai) ; Perplexity — https://docs.perplexity.ai/api-reference/chat-completions-post , https://docs.perplexity.ai/getting-started/quickstart ; Together — https://docs.together.ai/reference/chat-completions-1 ; DeepInfra — https://docs.deepinfra.com/chat/overview ; Cerebras — https://inference-docs.cerebras.ai/api-reference/chat-completions .

#### Universal CompatAdjuster rules

1. **Unknown-parameter resilience:** providers split between *ignore-unknown* (DeepSeek, Together) and *400-on-unknown* (xAI reasoning models, Moonshot on bad ranges). The per-provider strip table is authoritative; additionally, on a 400 whose message names a parameter, retry ONCE with that parameter removed, then surface the error.
2. **`max_tokens` vs `max_completion_tokens`:** accept both from clients; send whichever the provider documents (`max_completion_tokens` for Cerebras/Moonshot-K-series; `max_tokens` elsewhere). Never send both.
3. **Streaming usage:** request `stream_options: {"include_usage": true}` upstream wherever supported (DashScope, Moonshot, DeepSeek, Together, DeepInfra, Cerebras, xAI, Mistral); when the upstream can't provide usage, estimate tokens locally and flag `"x-zyquo": {"usage_estimated": true}`.
4. **finish_reason normalization set:** everything must land in `stop | length | tool_calls | content_filter`; map `eos→stop`, provider-specific refusal/safety values → `content_filter`, anything unknown → `stop` + log.
5. **Keep-alive noise:** some upstreams emit SSE comment lines (`: keep-alive`) or `ping`-ish frames — the SSE parser must skip comment lines and blank frames without erroring, and the router's own SSEWriter may emit comments to keep client sockets alive during long thinking phases.

---

### 3.4 Reasoning content — normalization decision

**Decision: normalize on DeepSeek's `reasoning_content` convention** — a sibling of `content` on both the non-streaming `message` and the streaming `delta`:

```json
// non-streaming                              // streaming
"message": {                                  "delta": {
  "role": "assistant",                          "reasoning_content": "Let me check…"
  "reasoning_content": "Let me check…",       }
  "content": "The answer is 21."
}
```

Rationale: it is the oldest and most widely recognized wire convention (DeepSeek-R1 era) — Qwen/DashScope, Moonshot/Kimi, and xAI grok-3-mini already emit exactly this field, so most client tooling (chat UIs, LangChain, Continue, aider, etc.) knows to render it and to NOT confuse it with `content`. OpenRouter's richer `reasoning` + `reasoning_details` model (https://openrouter.ai/docs/use-cases/reasoning-tokens) is adopted only as a *supplement*: the router additionally emits `reasoning_details` (array of provider-native structured blocks) when signatures/opacity must round-trip.

#### Per-provider mapping into `reasoning_content`

| Provider | Native form | → Router normalization |
|---|---|---|
| DeepSeek | `message.reasoning_content` / `delta.reasoning_content` | Pass through unchanged. |
| Qwen/DashScope | same field | Pass through. |
| Kimi/Moonshot | same field | Pass through. |
| xAI grok-3-mini | same field | Pass through. (grok-4: nothing exposed — only `reasoning_tokens` in usage.) |
| Anthropic | `thinking` content blocks; streaming `thinking_delta` (+ `signature_delta`, `redacted_thinking`) | Thinking text → `reasoning_content`; signature + redacted blocks → `reasoning_details: [{"type":"anthropic.thinking_signature",…}]`. |
| Gemini | parts with `"thought": true` (needs `thinkingConfig.includeThoughts`); `thoughtSignature` on parts | Thought text → `reasoning_content`; `thoughtSignature` → `reasoning_details: [{"type":"gemini.thought_signature",…}]`. |
| Mistral Magistral | content chunk `{"type":"thinking",…}` inside the content array | Flatten to `reasoning_content`; text chunks → `content`. |
| Perplexity sonar-reasoning | `<think>…</think>` prefix inside `content` | Extract tags → `reasoning_content`; strip from `content`. |
| Cerebras (reasoning models) | model-dependent (`reasoning` field or `<think>` tags per hosted model) | Same extraction pipeline; verify per model in Phase 7. |

#### Usage normalization

All reasoning token counts land in the OpenAI-standard `usage.completion_tokens_details.reasoning_tokens` (Anthropic `thinking_tokens`, Gemini `thoughtsTokenCount`, xAI/Perplexity `reasoning_tokens`), and reasoning tokens are INCLUDED in `completion_tokens` (OpenAI semantics).

#### Request-side control

The router accepts the OpenAI-standard **`reasoning_effort`** (`"minimal"|"low"|"medium"|"high"`, plus provider extras like `"none"`/`"max"`) and translates per provider: Anthropic → `output_config.effort` / `thinking.budget_tokens`; Gemini → `thinkingConfig.thinkingBudget`/`thinkingLevel` (Google's own compat mapping: low=1024, medium=8192, high=24576 on 2.5-series); DeepSeek/Kimi/Cerebras/xAI/Perplexity → pass `reasoning_effort` through (gated by catalog capability); Qwen → `enable_thinking:true` (+ `thinking_budget`); Mistral → `prompt_mode:"reasoning"`. On models with no reasoning capability, `reasoning_effort` is stripped (never 400 the client for asking).

#### Echo-back rules (multi-turn)

Incoming assistant messages may contain `reasoning_content`/`reasoning_details` from prior router responses. Before forwarding:
- **Strip `reasoning_content`** for all providers by default (DeepSeek 400s in plain turns if it leaks into context via unknown-field-strict paths; others ignore it but it wastes tokens),
- **except**: DeepSeek tool-call loops (must be passed back per current docs), Anthropic thinking+tools loops (reconstruct `thinking` blocks with signatures from `reasoning_details`), Gemini 3 (re-attach `thoughtSignature`), Mistral Magistral (replay ThinkChunk to preserve the trace), Kimi with `thinking.keep`.
This asymmetry is exactly why `reasoning_details` exists: it carries the provider-native, signed material that some upstreams demand back, while `reasoning_content` stays a clean display string.

---

### 3.5 Source index

- Anthropic Messages API: https://platform.claude.com/docs/en/api/messages
- Anthropic streaming events: https://platform.claude.com/docs/en/docs/build-with-claude/streaming
- Anthropic errors: https://platform.claude.com/docs/en/api/errors
- Gemini generateContent reference: https://ai.google.dev/api/generate-content
- Gemini Part/Content schema: https://ai.google.dev/api/caching#Part
- Gemini function calling: https://ai.google.dev/gemini-api/docs/function-calling
- Gemini OpenAI-compat layer (mapping oracle): https://ai.google.dev/gemini-api/docs/openai
- xAI chat completions: https://docs.x.ai/developers/model-capabilities/legacy/chat-completions ; parameter-rejection field notes: https://www.promptfoo.dev/docs/providers/xai/ , https://github.com/vercel/ai/issues/12826
- Mistral API: https://docs.mistral.ai/api/ ; reasoning: https://docs.mistral.ai/capabilities/reasoning/
- DashScope OpenAI compat: https://www.alibabacloud.com/help/en/model-studio/compatibility-of-openai-with-dashscope ; deep thinking: https://www.alibabacloud.com/help/en/model-studio/deep-thinking
- DeepSeek thinking mode: https://api-docs.deepseek.com/guides/thinking_mode
- Kimi/Moonshot chat API: https://platform.kimi.ai/docs/api/chat
- Perplexity chat completions: https://docs.perplexity.ai/api-reference/chat-completions-post ; quickstart: https://docs.perplexity.ai/getting-started/quickstart
- Together chat completions: https://docs.together.ai/reference/chat-completions-1
- DeepInfra OpenAI API: https://docs.deepinfra.com/chat/overview
- Cerebras chat completions: https://inference-docs.cerebras.ai/api-reference/chat-completions
- OpenRouter reasoning normalization (prior art): https://openrouter.ai/docs/use-cases/reasoning-tokens
## 4. HTTP server in Swift without heavyweight deps

Research date: 2026-07-30. Target: an embedded HTTP/1.1 server inside a SwiftUI macOS 13+ app (SPM, no Xcode IDE) serving an OpenAI-compatible API on `http://localhost:<port>`, with spec-exact SSE streaming, long-lived streams, client-disconnect → upstream-cancellation, and graceful shutdown.

### 4.1 Options evaluated

#### Option A — SwiftNIO directly (NIOCore + NIOPosix + NIOHTTP1 + NIOExtras)

- **State as of mid-2026:** swift-nio is at **2.101.3** (released ~2026-07-23), actively maintained by Apple, compatible with Swift **6.0–6.3** and strict concurrency. NIO 3 is expected "sometime around Swift 6" per the Swift.org server guidelines, with NIO 2 continuing to receive bug fixes afterwards — NIO 2.x is a safe multi-year foundation. Sources: [swift-nio releases](https://github.com/apple/swift-nio/releases), [Swift Package Index — swift-nio](https://swiftpackageindex.com/apple/swift-nio), [Swift.org concurrency adoption guidelines](https://www.swift.org/documentation/server/guides/libraries/concurrency-adoption-guidelines.html).
- **Structured concurrency:** modern NIO exposes **`NIOAsyncChannel`**, which "abstracts the notion of a NIO `Channel` into something that can safely be used in a structured concurrency context". The recommended split: protocol-specific logic (HTTP parsing/encoding via `configureHTTPServerPipeline`) stays as `ChannelHandler`s; business logic consumes/produces via the `NIOAsyncChannel` inbound `AsyncSequence` / outbound writer. `executeThenClose` scopes the channel's lifetime to a closure — the channel closes when the closure returns, which maps perfectly onto "one inbound request = one cancellable `Task`". Sources: [NIOAsyncChannel docs](https://swiftinit.org/docs/swift-nio/niocore/nioasyncchannel), [NIO public async APIs](https://github.com/apple/swift-nio/blob/main/docs/public-async-nio-apis.md), [executeThenClose discussion](https://forums.swift.org/t/nioasyncchannel-executethenclose-is-too-restrictive/73460), [Using SwiftNIO — Channels](https://swiftonserver.com/using-swiftnio-channels/), [Building a web app with only SwiftNIO (2026)](https://blog.alexseifert.com/2026/06/29/building-a-web-app-in-swift-using-only-swiftnio/).
- **Reference server:** Apple's `NIOHTTP1Server` example shows the canonical `ServerBootstrap` setup — `backlog: 256`, `so_reuseaddr` on server and child channels, `configureHTTPServerPipeline(withErrorHandling: true)`, explicit keep-alive state machine (idle → waiting-for-body → sending-response) and adding `Connection: close`/`keep-alive` headers for HTTP/1.0 or explicit-close requests. (The example itself is future-based; we use the NIOAsyncChannel equivalent.) Source: [NIOHTTP1Server main.swift](https://github.com/apple/swift-nio/blob/main/Sources/NIOHTTP1Server/main.swift).
- **Graceful shutdown:** `swift-nio-extras` ships **`ServerQuiescingHelper`** — "helps to quiesce a server by notifying user code when all previously open connections have closed"; call `initiateShutdown(promise:)` to stop accepting and drain. There is a full demo (`HTTPServerWithQuiescingDemo`). Sources: [QuiescingHelper.swift](https://github.com/apple/swift-nio-extras/blob/main/Sources/NIOExtras/QuiescingHelper.swift), [HTTPServerWithQuiescingDemo](https://github.com/apple/swift-nio-extras/blob/main/Sources/HTTPServerWithQuiescingDemo/main.swift).
- **Weight:** NIOCore/NIOPosix/NIOHTTP1 (+ optionally NIOExtras) — one Apple-maintained dependency tree, no routing framework, no ServiceLifecycle/Logging/Metrics transitive stack. Full control over every byte of the SSE wire format.
- **Cost:** we write our own tiny router (we have ~5 routes), our own request-body accumulation with a size limit, and our own SSE writer. For this app that's a feature: the OpenAI chunk stream must be byte-exact, and owning `SSEWriter` end-to-end removes a framework abstraction between us and the wire.

#### Option B — Network.framework (`NWListener` + `NWProtocolFramer`)

`NWListener` replaces the BSD bind/listen/accept sequence, but HTTP itself must be brought along: either a custom `NWProtocolFramer` (real boilerplate via `NWProtocolFramerImplementation`) or hand-wiring a C parser (`http_parser.c`) onto raw connections — Helge Heß's NWHTTPProtocol does exactly that and its author notes that "for production use it's suggested to not use a protocol framer for HTTP" and to hook the parser up directly instead. No HTTP/1.1 pipeline, no chunked-encoding helpers, no keep-alive management, no quiescing utilities — all DIY. It buys nothing over NIO for a localhost server (its strengths are Wi-Fi/cellular path handling and Bonjour, irrelevant here). Sources: [NWHTTPProtocol](https://github.com/helje5/NWHTTPProtocol), [Intro to Network.framework servers](http://www.alwaysrightinstitute.com/network-framework/), [Apple Network framework docs](https://developer.apple.com/documentation/network). **Rejected.**

#### Option C — Hummingbird 2

The strongest framework candidate. Built from scratch on SwiftNIO with Swift concurrency central; at **2.25.1** as of July 2026; Swift 6.1+; "designed to require the minimum number of dependencies". It has first-class pieces we'd otherwise hand-roll: a `ResponseBodyWriter` closure body with backpressure-aware `await writer.write(...)`, a `ServerSentEvent` type, `consumeWithInboundCloseHandler` for client-disconnect detection, and graceful shutdown via **swift-service-lifecycle** ("currently running requests continue being handled, while new connections and requests will not be accepted"), started with `Application.runService(gracefulShutdownSignals:)`. There is a dedicated `server-sent-events` example. Sources: [hummingbird repo](https://github.com/hummingbird-project/hummingbird), [SPI — hummingbird](https://swiftpackageindex.com/hummingbird-project/hummingbird), [What's new in Hummingbird 2](https://swiftonserver.com/whats-new-in-hummingbird-2/), [Hummingbird 2 announcement](https://hummingbird.codes/news/hummingbird-2/), [SSE example](https://github.com/hummingbird-project/hummingbird-examples), [swift-service-lifecycle](https://github.com/swift-server/swift-service-lifecycle).

Why not choose it: (1) it drags in ServiceLifecycle/Logging/Metrics/Tracing abstractions designed for long-running server binaries with signal-driven lifecycles, whereas our lifecycle is owned by a SwiftUI app's Start/Stop button — bridging `runService` into an app-owned start/stop adds friction rather than removing it; (2) its router/middleware/extractor machinery is overhead for ~5 fixed routes; (3) an extra abstraction layer sits between us and the SSE bytes, and byte-exact OpenAI chunk emission is the core deliverable. Its SSE example is nonetheless the best public reference for the disconnect/shutdown patterns we will reimplement on raw NIO (see 4.2).

#### Option D — Vapor

Batteries-included (HTTP/2, TLS, auth, validation, WebSockets, …) and correspondingly heavy: bulks the executable, increases compile time, ~20–30 MB idle memory vs Hummingbird's ~5–10 MB, and a large transitive dependency graph. Everything it adds over NIO is something this app doesn't need. Sources: [Hummingbird vs Vapor discussion](https://github.com/hummingbird-project/hummingbird/discussions/150), [Beginner's guide to Hummingbird](https://theswiftdev.com/beginners-guide-to-server-side-swift-using-the-hummingbird-framework/). **Rejected.**

#### Decision

**SwiftNIO directly** (`NIOCore`, `NIOPosix`, `NIOHTTP1`, plus `NIOExtras` for `ServerQuiescingHelper`), using the `NIOAsyncChannel` structured-concurrency APIs. Rationale: Apple-maintained, SPM-clean, one dependency tree, macOS 13+ fine, Swift 6 strict-concurrency ready, full control over SSE emission/flushing/backpressure, first-class quiescing for graceful shutdown, and a natural one-request-one-`Task` model so client disconnect cancels the upstream call structurally. Hummingbird 2 is the documented fallback if raw-NIO plumbing proves costlier than expected — the migration path is easy since both are NIO underneath.

### 4.2 Implementation specifics (SwiftNIO)

**Bootstrap and binding.**

```swift
let group = MultiThreadedEventLoopGroup.singleton
let quiesce = ServerQuiescingHelper(group: group)

let serverChannel = try await ServerBootstrap(group: group)
    .serverChannelOption(ChannelOptions.backlog, value: 256)
    .serverChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
    .serverChannelInitializer { channel in
        channel.pipeline.addHandler(quiesce.makeServerChannelHandler(channel: channel))
    }
    .childChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
    .bind(host: bindHost, port: port) { channel in
        channel.eventLoop.makeCompletedFuture {
            try channel.pipeline.syncOperations.configureHTTPServerPipeline(withErrorHandling: true)
            return try NIOAsyncChannel<HTTPServerRequestPart, HTTPServerResponsePart>(
                wrappingChannelSynchronously: channel)
        }
    }
```

- **Bind host:** `"127.0.0.1"` by default (localhost only, unreachable from the LAN); `"0.0.0.0"` only when the user explicitly enables LAN exposure (which per our security policy forces a local API key). This is a plain string parameter to `bind(host:port:)` — no extra API.
- **Port-in-use:** `bind` throws; catch `IOError`/`NIOBSDSocket` errors and check `errno == EADDRINUSE` (48 on Darwin) → surface "Port 8787 is already in use" and probe upward (`try bind` on port+1, +2, …) to *suggest* the next free port (never silently switch). `EACCES` (ports < 1024 without privileges) gets its own message. `SO_REUSEADDR` on the server channel avoids spurious `EADDRINUSE` from sockets lingering in `TIME_WAIT` after a quick Stop→Start; note it does **not** let two live listeners share a port — a genuinely occupied port still fails, which is what we want. Sources: [ServerBootstrap docs](https://swiftinit.org/docs/swift-nio/nioposix/serverbootstrap), [Bind: address already in use](https://hea-www.harvard.edu/~fine/Tech/addrinuse.html), [NIOHTTP1Server example options](https://github.com/apple/swift-nio/blob/main/Sources/NIOHTTP1Server/main.swift).

**Concurrent connections with structured concurrency.** The async `bind` returns a `NIOAsyncChannel` of accepted-connection `NIOAsyncChannel`s. The serving loop:

```swift
try await serverChannel.executeThenClose { acceptedConnections in
    try await withThrowingDiscardingTaskGroup { group in
        for try await connection in acceptedConnections {
            group.addTask { await handleConnection(connection) }   // one Task per connection
        }
    }
}
```

Inside `handleConnection`, `connection.executeThenClose { inbound, outbound in ... }` gives an `AsyncSequence` of `HTTPServerRequestPart` (`.head`, `.body` buffers, `.end`) and an outbound writer for `HTTPServerResponsePart`. Each request is parsed, dispatched to `Routes`, and answered; the loop iterates for keep-alive. Cancelling the connection's `Task` tears everything down cleanly — this is the backbone of both client-disconnect handling and graceful shutdown. Sources: [NIO public async APIs](https://github.com/apple/swift-nio/blob/main/docs/public-async-nio-apis.md), [NIOAsyncChannel docs](https://swiftinit.org/docs/swift-nio/niocore/nioasyncchannel).

**Request body size limit.** Accumulate `.body` parts into a `ByteBuffer` with a hard cap (default e.g. 20 MiB — base64 images are large; user-configurable in Settings). On overflow: respond `413` in OpenAI error format, drain remaining parts (or close), never buffer further.

**Timeouts.**
- *Header/idle timeout:* add `IdleStateHandler` (NIOCore) ahead of the HTTP handlers, or a per-request `withTimeout` wrapper, to kill connections that never complete a request (~30 s read idle).
- *Streaming:* once a response stream has started, the idle clock must apply to *write* progress, not total duration — chat completions can legitimately stream for many minutes. Practical policy: generous upstream time-to-first-byte timeout (e.g. 120 s, configurable), then no total cap while chunks keep flowing; abort if the upstream stalls (no chunk for N seconds, e.g. 300 s).
- Remove/suspend the idle handler for the duration of an SSE response, restore for keep-alive reuse.

**Keep-alive.** `configureHTTPServerPipeline` parses `Connection` headers; our responder mirrors the NIOHTTP1Server example: honor `keepAlive` from the request head, set `Connection: keep-alive`/`close` explicitly for HTTP/1.0, close the channel after the response when keep-alive is false. After an SSE response we send the terminating `[DONE]` and `.end`; keeping the connection alive afterwards is legal (the response used chunked encoding with a proper terminator), but closing is also acceptable — OpenAI SDKs handle both. We keep it alive (SDKs reuse connections between calls). Source: [NIOHTTP1Server main.swift](https://github.com/apple/swift-nio/blob/main/Sources/NIOHTTP1Server/main.swift).

**SSE emission (`SSEWriter`).** Response head:

```
HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
X-Accel-Buffering: no          ← harmless; defeats proxy buffering if any intermediary appears
Transfer-Encoding: chunked     ← added automatically by HTTPResponseEncoder when no Content-Length
```

Each OpenAI chunk is one SSE event: `data: {json}\n\n` (UTF-8; the OpenAI format is single-line JSON per `data:` line, terminated by `data: [DONE]\n\n`). **Flush per event:** with `NIOAsyncChannel`, every `try await outbound.write(.body(.byteBuffer(eventBuffer)))` is a `writeAndFlush` — each event leaves the process immediately; no coalescing layer may sit above it. `await`-ing each write is also the **backpressure** mechanism: a slow client suspends us, which suspends consumption of the upstream `AsyncSequence`, which propagates backpressure to the upstream HTTP read. Optionally set `TCP_NODELAY` on child channels so small event frames aren't Nagle-delayed. SSE format/caching rules: [MDN — Using server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events). (Note: OpenAI's stream is consumed by SDK parsers, not browser `EventSource`, so `event:`/`id:`/`retry:` fields are never used — `data:` lines only.)

**Client-disconnect → cancel upstream.** Two complementary signals:
1. Structured: the write to a closed socket throws (`NIOAsyncWriterError`/`channelInactive`-driven); catching it in the request `Task` must cancel the upstream streaming `Task` — with structured concurrency this is automatic if the upstream call is a child task (`withThrowingTaskGroup` per request: one child consumes upstream and writes SSE; error/cancel of either cancels the other).
2. Proactive: watch the inbound side for EOF/half-closure while streaming — the pattern Hummingbird's SSE example uses (`consumeWithInboundCloseHandler` yielding a cancel event merged with the data stream). On raw NIO: run a second child task iterating `inbound`; when the sequence ends (client closed), cancel the group. This detects disconnects *between* writes, not only on the next failed write.
Cancellation must propagate into the provider client (`URLSession`/`AsyncHTTPClient` task cancelled) so upstream token spend stops — Phase 7 verifies "no orphaned upstream usage". Sources: [Hummingbird SSE example](https://github.com/hummingbird-project/hummingbird-examples), [NIOAsyncChannel docs](https://swiftinit.org/docs/swift-nio/niocore/nioasyncchannel).

**CORS.** Needed so browser-based tools can call the router. Implement in a small `CORS.swift`:
- Preflight: `OPTIONS` with `Origin` + `Access-Control-Request-Method` → `204` with `Access-Control-Allow-Origin: *` (default; configurable to a specific origin), `Access-Control-Allow-Methods: GET, POST, OPTIONS`, `Access-Control-Allow-Headers: Authorization, Content-Type` (or echo the requested headers), `Access-Control-Max-Age: 600`.
- Actual responses: add `Access-Control-Allow-Origin: *` (echo origin + `Access-Control-Allow-Credentials: true` only if credentials mode is ever needed — default permissive `*` for localhost tooling, per our spec).
- SSE responses need the CORS headers too (the stream is fetched cross-origin by browser clients).

**Graceful shutdown.** On Stop:
1. `quiesce.initiateShutdown(promise:)` — the `ServerQuiescingHelper` closes the *listening* channel (no new connections) and signals when all child channels have closed. Sources: [QuiescingHelper.swift](https://github.com/apple/swift-nio-extras/blob/main/Sources/NIOExtras/QuiescingHelper.swift), [HTTPServerWithQuiescingDemo](https://github.com/apple/swift-nio-extras/blob/main/Sources/HTTPServerWithQuiescingDemo/main.swift).
2. Wait up to a drain deadline (e.g. 10 s) for in-flight non-streaming requests to finish.
3. Long-lived SSE streams won't drain on their own: after the deadline (or immediately if the user chose "stop now"), cancel the connection task group — cancellation unwinds each request task, cancels upstream calls, and closes channels.
4. Do **not** `shutdownGracefully()` the singleton `EventLoopGroup` (it's shared and Start may be pressed again); just release the server channel. Semantics mirror Hummingbird/ServiceLifecycle: "currently running requests continue being handled, while new connections and requests will not be accepted" ([swift-service-lifecycle](https://github.com/swift-server/swift-service-lifecycle), [What's new in Hummingbird 2](https://swiftonserver.com/whats-new-in-hummingbird-2/)).

### 4.3 macOS specifics

- **Local network privacy prompt.** Since macOS 15 Sequoia there is an iOS-style *Local Network* permission (System Settings → Privacy & Security → Local Network). Binding and serving on `127.0.0.1` does not involve the local network and triggers nothing. Interacting with LAN peers can trigger the prompt; users can later toggle it, and there are known Sequoia bugs where access silently breaks after a restart until re-toggled. When the user enables LAN mode (`0.0.0.0`) the app must explain the prompt and the Settings toggle in the UI. Also note the responsible-process rules: tools run from Terminal inherit Terminal's exemption; root daemons are auto-granted. Sources: [Apple forums — local network privacy on Sequoia](https://developer.apple.com/forums/thread/763484), [mjtsai — Local Network Privacy on Sequoia](https://mjtsai.com/blog/2024/10/02/local-network-privacy-on-sequoia/), [Foldr — macOS 15 local network privacy](https://foldr.com/foldr-support/foldr-for-macos/macos-15-sequoia-local-network-privacy/), [Panic — granting local network access](https://help.panic.com/prompt/prompt-local-network/), [access lost after restart](https://developer.apple.com/forums/thread/769037?page=2).
- **App Sandbox vs Developer ID.** `com.apple.security.network.server` ("whether your app may listen for incoming network connections") is an **App Sandbox** entitlement; a **non-sandboxed, Hardened-Runtime Developer ID app needs no entitlement to listen on a socket** — "the sandbox was designed mainly for the App Store, while the hardened runtime was designed mainly for Developer ID". Decision (consistent with Phase 8 spec): ship non-sandboxed Developer ID + Hardened Runtime; document that if we ever adopt the sandbox we must add `com.apple.security.network.server` and `.client`. Sources: [Apple — com.apple.security.network.server](https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.security.network.server), [Hardened Runtime and Sandboxing (lapcatsoftware)](https://lapcatsoftware.com/articles/hardened-runtime-sandboxing.html).
- **Terminal vs Finder launch.** From Finder the app gets the standard GUI context. From Terminal (during `make dev`), the *Terminal* process is the responsible process for TCC purposes, so privacy prompts may attribute to Terminal; also a bare executable launched from Terminal needs explicit `NSApplication` activation to come frontmost (already planned in Phase 1's entry point). Environment differs too (Terminal's shell env vs launchd's user session) — never rely on env vars for configuration of the shipped app. Source: [Apple forums — CLI tools and local network privacy](https://developer.apple.com/forums/thread/767391).

---

## 5. Gateway concerns

### 5.1 Request logging with redaction

Pattern (validated against LiteLLM's proxy design):
- **Always log metadata:** timestamp, method/path, resolved model + provider, actually-used model (post-fallback), status, latency breakdown (queue → upstream TTFB → stream duration), tokens in/out, cost, local-key ID (its name/ID — never the token), stream flag, error class. This is safe and powers the whole dashboard.
- **Bodies are opt-in.** LiteLLM's `turn_off_message_logging` redacts `messages`/`prompt` and `choices[].message.content` (and `reasoning_content`) while still tracking spend; its bug tracker shows the classic failure mode — one storage path redacted, another (raw `proxy_server_request`) not. Lesson: **redact at the single ingestion point** of `RequestLogStore`, not per-sink. Store `"[redacted]"` placeholders (plus sizes/counts, e.g. message count, image count) unless the per-session "reveal bodies" switch is on. Sources: [LiteLLM logging docs](https://docs.litellm.ai/docs/proxy/logging), [redaction bug #16336](https://github.com/BerriAI/litellm/issues/16336), [message redaction overview](https://deepwiki.com/BerriAI/litellm/6.3-message-redaction-and-privacy-controls).
- **Keys never touch the log path:** strip/replace `Authorization`, `x-api-key`, `api-key`, and any `*key*`/`*token*` header before a request object is handed to the logger; scrub upstream error bodies (some providers echo the offending header) through the same filter. Verification (Phase 7): grep all logs/exports for known key material.
- Ring buffer in memory (e.g. last 1–5k entries) + persisted store with retention setting; export honors the current redaction state.

### 5.2 Token usage extraction per provider

- **Prefer upstream-reported usage.** Non-streaming: OpenAI-compatible providers return `usage`; Anthropic returns `usage.input_tokens/output_tokens` (in `message_start` + `message_delta` when streaming); Gemini returns `usageMetadata`. Streaming with OpenAI-compatible upstreams: request `stream_options: {"include_usage": true}` upstream where supported so a final usage chunk arrives; some compatible providers (per-provider quirk table from Section 3) send usage on the last chunk regardless, others never do.
- **Estimate only when the upstream gives nothing**, and **flag it** — e.g. `"usage": {..., "x_zyquo_estimated": true}` (or an `x-zyquo` extension block) so cost tiles can render "≈". Estimation options in Swift: [aespinilla/Tiktoken](https://github.com/aespinilla/Tiktoken) (pure-Swift tiktoken: cl100k_base etc.) and [narner/TiktokenSwift](https://github.com/narner/TiktokenSwift) (UniFFI bindings to the real tiktoken, incl. `o200k_base`). A dependency-free fallback — `chars/4` or `words × 4/3` — is acceptable for the *flagged-estimate* path given tokenizers differ per provider anyway; decide in Phase 3 whether pulling a tokenizer dep is worth it (recommendation: start with the heuristic, keep the field flagged, add Tiktoken later if users need tight estimates).
- Cached-token counts: OpenAI reports `usage.prompt_tokens_details.cached_tokens`; Anthropic reports `cache_read_input_tokens`/`cache_creation_input_tokens`. Preserve these in the normalized usage when present (they change cost — see 5.3).

### 5.3 Cost calculation

- Source of truth: the ported Zyquo Cloud model catalog's per-model pricing (USD per 1M input tokens / per 1M output tokens). `cost = prompt_tokens × in_price/1e6 + completion_tokens × out_price/1e6`.
- **Cached input tokens** are billed at a discount where reported (e.g. OpenAI cached input typically 50–90% off; Anthropic cache reads at 0.1× base input, cache writes at 1.25×): when the catalog has cached pricing and the upstream reports cached counts, split: `(prompt − cached) × in_price + cached × cached_price`. Where the catalog lacks a cached rate, fall back to full input price (over-estimate, never under).
- Reasoning tokens (OpenAI `completion_tokens_details.reasoning_tokens`) are already included in `completion_tokens` — do not double-count.
- Mark costs derived from estimated usage as estimated. Store per-request cost in `UsageRecord`; aggregate per key/model/provider/day for the dashboard. Settings allow a pricing override table (providers reprice frequently) and a display currency (store USD, convert for display only).
- Precedent: OpenRouter prices "using the model that was ultimately used, which will be returned in the `model` attribute of the response body" — cost must always be computed against the *actually-used* model after fallbacks. Source: [OpenRouter model fallbacks](https://openrouter.ai/docs/guides/routing/model-fallbacks).

### 5.4 Rate limiting per local API key

- **Algorithm: token bucket**, the production default (AWS, Stripe) because "real traffic is bursty" — it allows short bursts up to bucket capacity while enforcing an average rate; sliding-window counters give smoother limits but punish legitimate bursts. For a local single-process gateway, in-memory is all we need (no distributed store). Sources: [Arcjet — rate limiting algorithms](https://blog.arcjet.com/rate-limiting-algorithms-token-bucket-vs-sliding-window-vs-fixed-window/), [token bucket vs sliding window](https://medium.com/@tihomir.manushev/token-bucket-vs-sliding-window-the-rate-limiting-choice-that-shapes-your-apis-behavior-e04fb2646ee5), [APISIX — gateway rate limiting](https://apisix.apache.org/learning-center/api-gateway-rate-limiting/).
- Implementation: one `actor RateLimiter` holding `keyID → (tokens: Double, lastRefill: ContinuousClock.Instant)`. Lazy refill on each check: `tokens = min(capacity, tokens + elapsed × rate)`; admit iff `tokens ≥ 1` then decrement. Per-key config: requests/min (rate = rpm/60, capacity ≈ rpm burst allowance). No timers, O(1) per request, trivially Sendable.
- On limit: `429` in OpenAI error format (`type: "rate_limit_error"`-style) with `Retry-After: ceil((1 − tokens)/rate)` seconds. Count rejections in per-key stats (Keys screen mini-chart).
- Optional second dimension later: tokens-per-minute budget (LLM-style limits) using the same bucket with token-cost withdrawal after usage is known.

### 5.5 Retries with exponential backoff + jitter

Consensus best practice for LLM upstreams ([Zuplo 429 guide](https://zuplo.com/learning-center/http-429-too-many-requests-guide), [handling 429s in production LLM apps](https://www.getmaxim.ai/articles/handle-429-errors-in-production-llm-applications/), [retry strategies with backoff + jitter](https://callsphere.ai/blog/retry-strategies-llm-api-calls-exponential-backoff-jitter-tenacity)):
- **Retry on:** 429, 500, 502, 503, 504, connection reset/refused, and TTFB timeout. **Never** on 400/401/403/404/422 (client/config errors — fail fast with the mapped OpenAI error).
- **Schedule:** `delay = min(cap, base × 2^attempt) + random(0, jitter)` — e.g. base 1 s, cap 30 s, full jitter; **max 3 attempts** for these interactive, user-facing requests (total budget ≤ ~30 s before fallback/error).
- **Respect `Retry-After`** when the provider sends it: `wait = max(retryAfter, computedBackoff)`; if `Retry-After` exceeds our remaining budget, skip retrying this provider and go straight to fallback/error (propagating `Retry-After` to our own 429 response).
- **Idempotency / streaming rule:** a retry is only safe **before any response byte has been forwarded to the client**. Once the first SSE chunk has been written downstream, never retry or fall back — terminate the stream with an error event/close. (Chat completions are not idempotent upstream either: a "failed" request may still have consumed tokens; retrying after partial streaming double-bills and duplicates output.) So: retries apply to (a) whole non-streaming calls, (b) streaming calls that fail before the first upstream content delta.
- Jitter exists to desynchronize concurrent clients; even locally, parallel requests from one SDK justify it.

### 5.6 Fallback chains

Modeled on LiteLLM and OpenRouter:
- **Semantics:** an ordered model list tried in sequence. LiteLLM: "the router tries the primary model first; if it fails with a retry-able error (429, 5xx, context-limit, content-policy, timeout), it moves to the first fallback", in order. OpenRouter: a `models: [...]` array tried in order server-side. Sources: [LiteLLM reliability/fallbacks](https://docs.litellm.ai/docs/proxy/reliability), [LiteLLM router architecture](https://docs.litellm.ai/docs/router_architecture), [OpenRouter model fallbacks](https://openrouter.ai/docs/guides/routing/model-fallbacks).
- **Trigger errors:** exhausted retries on 429/5xx/timeout; upstream auth failure (missing/invalid provider key — jumping to a provider the user *has* a key for is exactly the point); model-not-available (404 upstream). Optionally context-window errors (LiteLLM has a distinct `context_window_fallbacks` class). **Not** on content-policy 400s by default (surprising model swaps on policy errors are a footgun; make it opt-in like LiteLLM's `content_policy_fallbacks`).
- **Same rule as retries:** no fallback after the first downstream byte.
- **Honest reporting of the actually-used model** (our Phase 3 spec requirement): OpenRouter returns "the model that was ultimately used … in the `model` attribute of the response body"; LiteLLM exposes the concrete deployment via `x-litellm-model-id` header / `_hidden_params`. Zyquo Router does both: the response/chunks' `model` field carries the namespaced ID that actually served the request, plus an `x-zyquo-served-model` response header and the fallback hop count in the request log. Beware LiteLLM's documented pitfall of fallbacks resetting the retry cycle and re-running fallback models ([issue #19985](https://github.com/BerriAI/litellm/issues/19985)) — our loop: `for model in chain { retryPolicy(model) }`, each model getting one bounded retry budget, no restarts.
- Per-chain config lives in the Models screen editor; a chain is addressable like a model/alias.

### 5.7 Health checks

- `GET /health` (no auth, no logging noise) returning:

```json
{
  "status": "ok",
  "version": "1.0.0",
  "uptime_seconds": 12345,
  "server": { "host": "127.0.0.1", "port": 8787 },
  "providers_configured": 7,
  "active_streams": 2
}
```

- `status` is `"ok"` if the server is accepting; no upstream probing on this path (it must be instant and side-effect-free — LiteLLM separates `/health` per-model probes, which cost real tokens, from cheap `/health/liveliness` liveness checks; ours is the cheap kind). Per-provider connectivity testing belongs to the Keys screen's explicit "Test" button, not the health endpoint. Source: [LiteLLM proxy docs](https://docs.litellm.ai/docs/proxy/reliability).
- Suitable for `curl`-based readiness in scripts and the Phase 7 harness; also the phase-gate check for Phase 2.

### Sources (consolidated)

- SwiftNIO: https://github.com/apple/swift-nio · https://github.com/apple/swift-nio/releases · https://swiftpackageindex.com/apple/swift-nio · https://github.com/apple/swift-nio/blob/main/docs/public-async-nio-apis.md · https://swiftinit.org/docs/swift-nio/niocore/nioasyncchannel · https://github.com/apple/swift-nio/blob/main/Sources/NIOHTTP1Server/main.swift · https://swiftonserver.com/using-swiftnio-channels/ · https://forums.swift.org/t/nioasyncchannel-executethenclose-is-too-restrictive/73460 · https://blog.alexseifert.com/2026/06/29/building-a-web-app-in-swift-using-only-swiftnio/
- NIOExtras quiescing: https://github.com/apple/swift-nio-extras/blob/main/Sources/NIOExtras/QuiescingHelper.swift · https://github.com/apple/swift-nio-extras/blob/main/Sources/HTTPServerWithQuiescingDemo/main.swift
- Swift.org server guidelines (NIO3 timing): https://www.swift.org/documentation/server/guides/libraries/concurrency-adoption-guidelines.html
- Hummingbird: https://github.com/hummingbird-project/hummingbird · https://swiftpackageindex.com/hummingbird-project/hummingbird · https://swiftonserver.com/whats-new-in-hummingbird-2/ · https://hummingbird.codes/news/hummingbird-2/ · https://github.com/hummingbird-project/hummingbird-examples (server-sent-events example) · https://github.com/swift-server/swift-service-lifecycle
- Vapor comparison: https://github.com/hummingbird-project/hummingbird/discussions/150 · https://theswiftdev.com/beginners-guide-to-server-side-swift-using-the-hummingbird-framework/
- Network.framework: https://github.com/helje5/NWHTTPProtocol · http://www.alwaysrightinstitute.com/network-framework/ · https://developer.apple.com/documentation/network
- SSE format: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
- macOS local network privacy: https://developer.apple.com/forums/thread/763484 · https://developer.apple.com/forums/thread/767391 · https://mjtsai.com/blog/2024/10/02/local-network-privacy-on-sequoia/ · https://foldr.com/foldr-support/foldr-for-macos/macos-15-sequoia-local-network-privacy/ · https://help.panic.com/prompt/prompt-local-network/ · https://developer.apple.com/forums/thread/769037?page=2
- Entitlements/sandbox: https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.security.network.server · https://lapcatsoftware.com/articles/hardened-runtime-sandboxing.html
- LiteLLM: https://docs.litellm.ai/docs/proxy/reliability · https://docs.litellm.ai/docs/router_architecture · https://docs.litellm.ai/docs/proxy/logging · https://github.com/BerriAI/litellm/issues/16336 · https://github.com/BerriAI/litellm/issues/19985 · https://deepwiki.com/BerriAI/litellm/6.3-message-redaction-and-privacy-controls
- OpenRouter: https://openrouter.ai/docs/guides/routing/model-fallbacks · https://openrouter.ai/blog/insights/reliability-failover/
- Tokenizers in Swift: https://github.com/aespinilla/Tiktoken · https://github.com/narner/TiktokenSwift
- Rate limiting: https://blog.arcjet.com/rate-limiting-algorithms-token-bucket-vs-sliding-window-vs-fixed-window/ · https://medium.com/@tihomir.manushev/token-bucket-vs-sliding-window-the-rate-limiting-choice-that-shapes-your-apis-behavior-e04fb2646ee5 · https://apisix.apache.org/learning-center/api-gateway-rate-limiting/
- Retries/backoff: https://zuplo.com/learning-center/http-429-too-many-requests-guide · https://www.getmaxim.ai/articles/handle-429-errors-in-production-llm-applications/ · https://callsphere.ai/blog/retry-strategies-llm-api-calls-exponential-backoff-jitter-tenacity