|
1 |
+<!-- |
|
2 |
+ PROVIDERS.md |
|
3 |
+ Zyquo Cloud |
|
4 |
+ Author: Simon-Pierre Boucher |
|
5 |
+ Mail: contact@spboucher.ai |
|
6 |
+--> |
|
7 |
+ |
|
8 |
+# Zyquo Cloud — Provider Research (Phase 0, single source of truth) |
|
9 |
+ |
|
10 |
+Compiled 2026-07-30 from **official documentation** cross-checked against **live API probes with real |
|
11 |
+keys** (see `research/live-probes.md`; raw `/models` captures in `research/probes/`). All 12 providers |
|
12 |
+authenticated and returned a successful streamed chat completion today. This document (with the |
|
13 |
+`research/` files it aggregates) drives `ModelCatalog` — the two must never drift apart. |
|
14 |
+ |
|
15 |
+## Provider matrix (quick reference) |
|
16 |
+ |
|
17 |
+| # | Provider | Base URL | Auth | Format | `/models` | Reasoning surface | Vision | |
|
18 |
+|---|---|---|---|---|---|---|---| |
|
19 |
+| 1 | OpenAI | `https://api.openai.com/v1` | `Authorization: Bearer` | OpenAI (origin) | ✅ (no metadata) | `reasoning_effort`; summaries not in chat API | ✅ `image_url` | |
|
20 |
+| 2 | Anthropic | `https://api.anthropic.com/v1` | `x-api-key` + `anthropic-version: 2023-06-01` | **Messages API** | ✅ (rich capability metadata) | `thinking` param; `thinking_delta` SSE; adaptive on 4.7+/5 | ✅ `image` content block | |
|
21 |
+| 3 | xAI | `https://api.x.ai/v1` | Bearer | OpenAI-compat | ✅ (pricing + aliases) | `reasoning_effort` (low/med/high) | ✅ | |
|
22 |
+| 4 | Mistral | `https://api.mistral.ai/v1` | Bearer | OpenAI-compat | ✅ | `reasoning_effort` on medium/small; ThinkChunk content arrays | ✅ (medium/small) | |
|
23 |
+| 5 | Google Gemini | native `…/v1beta` + compat `…/v1beta/openai` | `x-goog-api-key` / Bearer (compat) | Both | ✅ | `thinkingLevel` (3.x) / `thinkingBudget` (2.5); `thought_signature` in compat deltas | ✅ | |
|
24 |
+| 6 | Qwen (DashScope intl) | `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` | Bearer | OpenAI-compat | ✅ | `enable_thinking` (requires `stream:true`), `thinking_budget`, `reasoning_content` deltas | ✅ (VL models) | |
|
25 |
+| 7 | DeepSeek | `https://api.deepseek.com` | Bearer | OpenAI-compat | ✅ (2 models) | `thinking:{type}` + `reasoning_effort`; `reasoning_content` deltas (on by default, v4-flash) | ❌ (unverified) | |
|
26 |
+| 8 | Kimi (Moonshot) | `https://api.moonshot.ai/v1` | Bearer | OpenAI-compat | ✅ (capability metadata) | K-series: `thinking`/`reasoning_effort`, `reasoning_content` deltas | ✅ (moonshot-v1 vision-preview, K-series) | |
|
27 |
+| 9 | Perplexity | `https://api.perplexity.ai` | Bearer | OpenAI-compat + search extras | ❌ (404 — built-in catalog required) | `<think>` blocks (sonar-reasoning-pro) | ❌ | |
|
28 |
+| 10 | Together AI | `https://api.together.xyz/v1` | Bearer | OpenAI-compat | ✅ (bare array + pricing) | `message.reasoning` for hosted reasoning models | ✅ (Llama-4, Qwen-VL) | |
|
29 |
+| 11 | DeepInfra | `https://api.deepinfra.com/v1/openai` | Bearer | OpenAI-compat | ✅ (metadata incl. pricing) | model-dependent `reasoning_content` | ✅ (proxied Claude/Gemini, VL models) | |
|
30 |
+| 12 | Cerebras | `https://api.cerebras.ai/v1` | Bearer | OpenAI-compat | ✅ (3 models) | `reasoning_effort`, `clear_thinking`; use `max_completion_tokens` | ✅ (gemma-4-31b) | |
|
31 |
+ |
|
32 |
+## Cross-provider implementation notes |
|
33 |
+ |
|
34 |
+- **Decoders must ignore unknown JSON fields** (OpenAI `obfuscation`, Gemini `extra_content`, |
|
35 |
+ Perplexity `citations`/`search_results`, Together `eos` finish_reason, SSE `: keep-alive` comments). |
|
36 |
+- **Reasoning deltas** arrive as `delta.reasoning_content` (DeepSeek, Qwen, Kimi, some DeepInfra), |
|
37 |
+ Anthropic `thinking_delta`, Mistral ThinkChunk arrays, Perplexity inline `<think>` text. |
|
38 |
+- **Usage in streams**: OpenAI/Gemini-compat need `stream_options:{include_usage:true}`; Qwen, DeepInfra, |
|
39 |
+ Anthropic (message_delta), Perplexity include usage automatically; verify per provider in Phase 7. |
|
40 |
+- **Token cap param**: `max_completion_tokens` required by Cerebras & preferred by OpenAI reasoning |
|
41 |
+ models; `max_tokens` elsewhere; Anthropic `max_tokens` mandatory. |
|
42 |
+- **Catalog churn discovered today** (already reflected in per-provider files): DeepSeek retired |
|
43 |
+ `deepseek-chat`/`deepseek-reasoner` (2026-07-24) → `deepseek-v4-flash`/`-pro`; Mistral deprecates |
|
44 |
+ Magistral/Devstral/Nemo/small-2506 on 2026-07-31; xAI catalog fully replaced by grok-4.5/4.3/4.20/build; |
|
45 |
+ Cerebras removed all Llama/Qwen models; Perplexity removed `sonar-reasoning` and marks the Sonar chat |
|
46 |
+ API "(Deprecated)" while still supporting it — ship it but architect for its Agent API later. |
|
47 |
+ |
|
48 |
+--- |
|
49 |
+ |
|
50 |
+ |
|
51 |
+ |
|
52 |
+# OpenAI — API Research (Phase 0) |
|
53 |
+ |
|
54 |
+Researched: 2026-07-30. Sources: https://developers.openai.com/api/docs/models , https://developers.openai.com/api/docs/pricing , https://developers.openai.com/api/docs/api-reference/chat/create , https://developers.openai.com/api/docs/guides/reasoning (platform.openai.com 301-redirects to developers.openai.com). Live model list verified against `GET /v1/models` with a real key on 2026-07-30 (`/tmp/zyquo-probe/openai.json`). |
|
55 |
+ |
|
56 |
+## 1. Base URL & endpoints |
|
57 |
+ |
|
58 |
+- Base URL: `https://api.openai.com/v1` |
|
59 |
+- Chat completions (what Zyquo Cloud uses): `POST /v1/chat/completions` |
|
60 |
+- Responses API (newer, stateful; NOT used by the app, but note some models are Responses-only): `POST /v1/responses` |
|
61 |
+- Model listing: `GET /v1/models` |
|
62 |
+- Embeddings/audio/images exist but are out of scope. |
|
63 |
+ |
|
64 |
+## 2. Authentication |
|
65 |
+ |
|
66 |
+- Header: `Authorization: Bearer <OPENAI_API_KEY>` |
|
67 |
+- Optional: `OpenAI-Organization: <org_id>`, `OpenAI-Project: <proj_id>` |
|
68 |
+- No other required headers besides `Content-Type: application/json`. |
|
69 |
+ |
|
70 |
+## 3. Chat model catalog |
|
71 |
+ |
|
72 |
+Verified live 2026-07-30. Aliases listed; dated snapshots (e.g. `gpt-5.4-2026-03-05`, `gpt-5-2025-08-07`, `gpt-4.1-2025-04-14`, …) exist for most and are omitted. Pricing = USD per 1M tokens (input / cached input / output), from the official pricing page. All models below: streaming ✅, function calling/tools ✅, JSON/structured output (`response_format: json_schema`) ✅ unless noted. |
|
73 |
+ |
|
74 |
+### Flagship (featured in current docs — the GPT-5.6 trio) |
|
75 |
+ |
|
76 |
+| Model ID | Context | Max output | $/1M in / cached / out | Vision | Reasoning | Notes | |
|
77 |
+|---|---|---|---|---|---|---| |
|
78 |
+| `gpt-5.6-sol` | 1.05M | 128K | 5.00 / 0.50 / 30.00 | ✅ | ✅ | Frontier — "complex professional work". **Recommended default.** | |
|
79 |
+| `gpt-5.6-terra` | 1.05M | 128K | 2.50 / 0.25 / 15.00 | ✅ | ✅ | Balanced intelligence/cost | |
|
80 |
+| `gpt-5.6-luna` | 1.05M | 128K | 1.00 / 0.10 / 6.00 | ✅ | ✅ | Cost-optimized, high volume | |
|
81 |
+| `chat-latest` | unverified (~128K) | unverified | 5.00 / 0.50 / 30.00 | ✅ | ⚠️ non-reasoning chat tuning | Rolling alias to the current ChatGPT model; accepts `temperature` | |
|
82 |
+ |
|
83 |
+Note: there is **no plain `gpt-5.6`** model — only the sol/terra/luna variants. `ra-gpt-5.6-sol` also appears in `/models` (created 2026-07-25) but is undocumented — flag as experimental, exclude from the default catalog. |
|
84 |
+ |
|
85 |
+### Current / recent GPT-5.x (still served, not featured) |
|
86 |
+ |
|
87 |
+| Model ID | Context | Max output | $/1M in / cached / out | Vision | Reasoning | Notes | |
|
88 |
+|---|---|---|---|---|---|---| |
|
89 |
+| `gpt-5.5` | unverified (≥400K) | unverified | 5.00 / 0.50 / 30.00 | ✅ | ✅ | Previous flagship (Apr 2026) | |
|
90 |
+| `gpt-5.5-pro` | unverified | unverified | 30.00 / — / 180.00 | ✅ | ✅ | **Responses API only** — exclude from chat-completions catalog | |
|
91 |
+| `gpt-5.4` | unverified (400K) | unverified (128K) | 2.50 / 0.25 / 15.00 | ✅ | ✅ | Mar 2026 | |
|
92 |
+| `gpt-5.4-mini` | unverified | unverified | 0.75 / 0.075 / 4.50 | ✅ | ✅ | | |
|
93 |
+| `gpt-5.4-nano` | unverified | unverified | 0.20 / 0.02 / 1.25 | ✅ | ✅ | Cheapest current-gen | |
|
94 |
+| `gpt-5.4-pro` | unverified | unverified | 30.00 / — / 180.00 | ✅ | ✅ | **Responses API only** | |
|
95 |
+| `gpt-5.3-chat-latest` | unverified | unverified | unverified | ✅ | ⚠️ | Rolling chat alias; **no plain `gpt-5.3` exists** (only `gpt-5.3-codex`, Responses-oriented) | |
|
96 |
+| `gpt-5.2` | 400K | 128K | 1.75 / 0.175 / 14.00 | ✅ | ✅ | Dec 2025 | |
|
97 |
+| `gpt-5.2-pro` | 400K | unverified | 21.00 / — / 168.00 | ✅ | ✅ | **Responses API only** | |
|
98 |
+| `gpt-5.2-chat-latest` | 128K | 16K | ≈ gpt-5.2 | ✅ | ⚠️ | ChatGPT tuning; accepts temperature | |
|
99 |
+| `gpt-5.1` | 400K | 128K | 1.25 / 0.125 / 10.00 | ✅ | ✅ | Introduced `reasoning_effort: "none"` | |
|
100 |
+| `gpt-5.1-chat-latest` | 128K | 16K | ≈ gpt-5.1 | ✅ | ⚠️ | | |
|
101 |
+| `gpt-5` | 400K | 128K | 1.25 / 0.125 / 10.00 | ✅ | ✅ | | |
|
102 |
+| `gpt-5-mini` | 400K | 128K | 0.25 / 0.025 / 2.00 | ✅ | ✅ | | |
|
103 |
+| `gpt-5-nano` | 400K | 128K | 0.05 / 0.005 / 0.40 | ✅ | ✅ | | |
|
104 |
+| `gpt-5-pro` | 400K | 272K | 15.00 / — / 120.00 (unverified) | ✅ | ✅ | **Responses API only** | |
|
105 |
+| `gpt-5-chat-latest` | 128K | 16K | ≈ gpt-5 | ✅ | ⚠️ | | |
|
106 |
+ |
|
107 |
+Codex family (`gpt-5-codex`, `gpt-5.1-codex`, `gpt-5.1-codex-mini`, `gpt-5.1-codex-max`, `gpt-5.2-codex`, `gpt-5.3-codex`): agentic-coding tuned, **Responses API only** — exclude from the chat-completions catalog. `gpt-5-search-api` / `gpt-5-search-api-2025-10-14`: chat completions with built-in web search (returns citations/annotations), priced ≈ gpt-5 + per-search fee (unverified). |
|
108 |
+ |
|
109 |
+### o-series reasoning (legacy but served) |
|
110 |
+ |
|
111 |
+| Model ID | Context | Max output | $/1M in / cached / out | Vision | Reasoning | Notes | |
|
112 |
+|---|---|---|---|---|---|---| |
|
113 |
+| `o3` | 200K | 100K | 2.00 / 0.50 / 8.00 | ✅ | ✅ | | |
|
114 |
+| `o3-pro` | 200K | 100K | 20.00 / — / 80.00 | ✅ | ✅ | **Responses API only** | |
|
115 |
+| `o4-mini` | 200K | 100K | 1.10 / 0.275 / 4.40 | ✅ | ✅ | | |
|
116 |
+| `o3-mini` | 200K | 100K | 1.10 / 0.55 / 4.40 (unverified) | ❌ | ✅ | Text only | |
|
117 |
+| `o1` | 200K | 100K | 15.00 / 7.50 / 60.00 (unverified) | ✅ | ✅ | Legacy | |
|
118 |
+| `o1-pro` | 200K | 100K | 150.00 / — / 600.00 (unverified) | ✅ | ✅ | **Responses API only** | |
|
119 |
+ |
|
120 |
+`o3-deep-research`, `o4-mini-deep-research`: Responses API + background mode only — exclude. |
|
121 |
+ |
|
122 |
+### Legacy GPT-4.x / 3.5 (served; mark "legacy" in UI) |
|
123 |
+ |
|
124 |
+| Model ID | Context | Max output | $/1M in / cached / out | Vision | Reasoning | |
|
125 |
+|---|---|---|---|---|---| |
|
126 |
+| `gpt-4.1` | 1,047,576 | 32,768 | 2.00 / 0.50 / 8.00 | ✅ | ❌ | |
|
127 |
+| `gpt-4.1-mini` | 1,047,576 | 32,768 | 0.40 / 0.10 / 1.60 | ✅ | ❌ | |
|
128 |
+| `gpt-4.1-nano` | 1,047,576 | 32,768 | 0.10 / 0.025 / 0.40 | ✅ | ❌ | |
|
129 |
+| `gpt-4o` | 128K | 16,384 | 2.50 / 1.25 / 10.00 | ✅ | ❌ | |
|
130 |
+| `gpt-4o-mini` | 128K | 16,384 | 0.15 / 0.075 / 0.60 | ✅ | ❌ | |
|
131 |
+| `gpt-4o-search-preview`, `gpt-4o-mini-search-preview` | 128K | 16,384 | ≈ 4o / 4o-mini + search fee | ✅ | ❌ (built-in web search, citations) | |
|
132 |
+| `gpt-4-turbo` | 128K | 4,096 | 10.00 / — / 30.00 | ✅ | ❌ | |
|
133 |
+| `gpt-4` | 8,192 | 8,192 | 30.00 / — / 60.00 | ❌ | ❌ | |
|
134 |
+| `gpt-3.5-turbo` (+ `-16k`, `-1106`, `-0125`, `-instruct`) | 16,385 | 4,096 | 0.50 / — / 1.50 | ❌ | ❌ | |
|
135 |
+ |
|
136 |
+Skip entirely (non-chat): `text-embedding-*`, `whisper-1`, `tts-1*`, `gpt-4o-*-transcribe*`, `gpt-4o-mini-tts*`, `gpt-audio*`, `gpt-realtime*`, `gpt-transcribe`, `gpt-live-transcribe`, `gpt-image-*`, `chatgpt-image-latest`, `sora-2*`, `omni-moderation*`, `davinci-002`, `babbage-002`. |
|
137 |
+ |
|
138 |
+## 4. Request/response format |
|
139 |
+ |
|
140 |
+Fully OpenAI-native (this IS the reference format for the app's `OpenAICompatibleClient`). |
|
141 |
+ |
|
142 |
+```json |
|
143 |
+POST /v1/chat/completions |
|
144 |
+{ |
|
145 |
+ "model": "gpt-5.6-terra", |
|
146 |
+ "messages": [ |
|
147 |
+ {"role": "system", "content": "You are helpful."}, |
|
148 |
+ {"role": "user", "content": [ |
|
149 |
+ {"type": "text", "text": "What is in this image?"}, |
|
150 |
+ {"type": "image_url", "image_url": {"url": "data:image/png;base64,....", "detail": "auto"}} |
|
151 |
+ ]} |
|
152 |
+ ], |
|
153 |
+ "max_completion_tokens": 4096, |
|
154 |
+ "stream": true, |
|
155 |
+ "stream_options": {"include_usage": true} |
|
156 |
+} |
|
157 |
+``` |
|
158 |
+ |
|
159 |
+- Response: `{"id", "object": "chat.completion", "created", "model", "choices": [{"index", "message": {"role": "assistant", "content", "tool_calls?", "refusal?"}, "finish_reason"}], "usage": {"prompt_tokens", "completion_tokens", "total_tokens", "completion_tokens_details": {"reasoning_tokens"}, "prompt_tokens_details": {"cached_tokens"}}}` |
|
160 |
+- Vision input: content parts with `{"type": "image_url", "image_url": {"url": "<https URL or data:image/…;base64,…>", "detail": "auto|low|high"}}`. |
|
161 |
+- `finish_reason`: `stop`, `length`, `tool_calls`, `content_filter`. |
|
162 |
+ |
|
163 |
+## 5. Streaming (SSE) |
|
164 |
+ |
|
165 |
+- `stream: true`. Events are unnamed `data:` lines, each a `chat.completion.chunk`: |
|
166 |
+ `{"id","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role"?,"content"?,"tool_calls"?},"finish_reason":null|...}]}` |
|
167 |
+- First chunk carries `delta.role`; subsequent chunks `delta.content` fragments; final content chunk carries `finish_reason`. |
|
168 |
+- With `stream_options: {"include_usage": true}` a final extra chunk arrives with empty `choices: []` and a populated `usage` object — **use this for token/cost accounting**. |
|
169 |
+- Terminator: literal `data: [DONE]`. |
|
170 |
+- No SSE `event:` names — parse `data:` payloads only. |
|
171 |
+ |
|
172 |
+## 6. Special parameters |
|
173 |
+ |
|
174 |
+- `max_completion_tokens` — use this everywhere; `max_tokens` is **deprecated and rejected by reasoning models** (o-series, gpt-5.x). |
|
175 |
+- `reasoning_effort` — values `none | minimal | low | medium | high | xhigh | max`; support varies by model (gpt-5: minimal–high; `none` added with gpt-5.1; `xhigh`/`max` on newer 5.x; gpt-5.6 family supports the full set; default is `medium` on most, gpt-5.1+ default `none`/model-dependent). Send only on reasoning-capable models. |
|
176 |
+- `verbosity` — `low | medium | high` (gpt-5 family and later; controls answer length). |
|
177 |
+- Temperature restrictions: reasoning models (o-series, gpt-5/5.x reasoning variants) **reject `temperature`, `top_p`, `presence_penalty`, `frequency_penalty`, `logprobs`** (only default temperature=1 accepted). The `*-chat-latest` / `chat-latest` non-reasoning chat models DO accept `temperature` (0–2) and `top_p`. Zyquo Cloud must gate these sliders per model. |
|
178 |
+- Reasoning content is **not returned** via chat completions — only `usage.completion_tokens_details.reasoning_tokens`. (Reasoning summaries exist only on the Responses API via `summary: auto|concise|detailed`.) |
|
179 |
+- `response_format: {"type": "json_schema", "json_schema": {...}}` or `{"type": "json_object"}` for structured/JSON output. |
|
180 |
+- Tools: `tools: [{"type": "function", "function": {name, description, parameters}}]`, `tool_choice: none|auto|required|{...}`. |
|
181 |
+ |
|
182 |
+## 7. Rate limits & errors |
|
183 |
+ |
|
184 |
+- Error body: `{"error": {"message": "...", "type": "invalid_request_error|authentication_error|rate_limit_error|...", "param": null, "code": "model_not_found|insufficient_quota|..."}}` |
|
185 |
+- HTTP codes: 400 invalid request, 401 bad key, 403 permission/region, 404 model, 429 rate limit or quota (`insufficient_quota`), 500/503 server. Retry 429/5xx with exponential backoff; honor `retry-after`. |
|
186 |
+- Rate-limit headers: `x-ratelimit-limit-requests`, `x-ratelimit-remaining-requests`, `x-ratelimit-limit-tokens`, `x-ratelimit-remaining-tokens`, `x-ratelimit-reset-requests`, `x-ratelimit-reset-tokens`. Limits are tier-based (usage tiers 1–5), per model family. Docs: https://developers.openai.com/api/docs/guides/rate-limits |
|
187 |
+ |
|
188 |
+## 8. /models listing |
|
189 |
+ |
|
190 |
+- `GET /v1/models` exists. Shape: `{"object": "list", "data": [{"id", "object": "model", "created", "owned_by"}]}` — **no capability/context metadata**, so the built-in catalog must supply context windows and capabilities; use `/models` only to detect availability/new IDs. 130 models returned on 2026-07-30 with a standard key. |
|
191 |
+ |
|
192 |
+### Catalog guidance for ModelCatalog |
|
193 |
+ |
|
194 |
+- Default model: `gpt-5.6-terra` (balanced) or `gpt-5.6-sol` (flagship). |
|
195 |
+- Exclude from picker: all `*-pro`, `*-codex*`, `*deep-research*` (Responses-only), plus all audio/image/video/embedding/moderation IDs. |
|
196 |
+- Mark legacy: gpt-4.1/4o/4/3.5 families, o1/o3-mini. |
|
197 |
+- Unverified items above must be re-checked in Phase 7 with real calls. |
|
198 |
+ |
|
199 |
+--- |
|
200 |
+ |
|
201 |
+ |
|
202 |
+# Anthropic — API Research (Phase 0) |
|
203 |
+ |
|
204 |
+Researched: 2026-07-30. Sources: https://platform.claude.com/docs/en/about-claude/models/overview.md , https://platform.claude.com/docs/en/pricing.md , https://platform.claude.com/docs/en/build-with-claude/streaming.md , https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking.md , https://platform.claude.com/docs/en/api/errors.md , https://platform.claude.com/docs/en/api/rate-limits.md . Live model list verified against `GET /v1/models` with a real key on 2026-07-30 (`/tmp/zyquo-probe/anthropic.json`) — the live response includes `max_input_tokens`, `max_tokens`, and a full `capabilities` tree per model (authoritative below). |
|
205 |
+ |
|
206 |
+## 1. Base URL & endpoints |
|
207 |
+ |
|
208 |
+- Base URL: `https://api.anthropic.com/v1` |
|
209 |
+- Messages (chat): `POST /v1/messages` — **NOT OpenAI-compatible**; native client required |
|
210 |
+- Model listing: `GET /v1/models` (and `GET /v1/models/{id}`) |
|
211 |
+- Token counting: `POST /v1/messages/count_tokens` (useful for cost estimates) |
|
212 |
+- Batches/Files exist but are out of scope for the chat client. |
|
213 |
+ |
|
214 |
+## 2. Authentication |
|
215 |
+ |
|
216 |
+- Header: `x-api-key: <ANTHROPIC_API_KEY>` (NOT `Authorization: Bearer`) |
|
217 |
+- Required version header: `anthropic-version: 2023-06-01` (this exact value — it is the current stable version string) |
|
218 |
+- `Content-Type: application/json` |
|
219 |
+- Optional `anthropic-beta: <flag>` for beta features (not needed for the app's core flows). |
|
220 |
+ |
|
221 |
+## 3. Chat model catalog |
|
222 |
+ |
|
223 |
+All IDs verified live 2026-07-30. Context / max output come from the live `/v1/models` response. Pricing in USD per 1M tokens (input/output). All models: streaming ✅, tools/function calling ✅, vision (image input) ✅, PDF input ✅, structured outputs ✅. |
|
224 |
+ |
|
225 |
+| Model ID | Display name | Context | Max output | $/1M in/out | Thinking mode | Effort param | Status | |
|
226 |
+|---|---|---|---|---|---|---|---| |
|
227 |
+| `claude-opus-5` | Claude Opus 5 | 1M | 128K | 5.00 / 25.00 | adaptive (on by default) | low…max (all 5) | **Recommended default** | |
|
228 |
+| `claude-sonnet-5` | Claude Sonnet 5 | 1M | 128K | 3.00 / 15.00 (intro 2.00/10.00 through 2026-08-31) | adaptive (on by default) | low…max | **Recommended balanced** | |
|
229 |
+| `claude-fable-5` | Claude Fable 5 | 1M | 128K | 10.00 / 50.00 | always on (omit param; `disabled` → 400) | low…max | Most capable; premium | |
|
230 |
+| `claude-opus-4-8` | Claude Opus 4.8 | 1M | 128K | 5.00 / 25.00 | adaptive (off unless set) | low…max | Current-gen | |
|
231 |
+| `claude-opus-4-7` | Claude Opus 4.7 | 1M | 128K | 5.00 / 25.00 | adaptive (off unless set) | low…max | Previous-gen | |
|
232 |
+| `claude-opus-4-6` | Claude Opus 4.6 | 1M | 128K | 5.00 / 25.00 | adaptive or `enabled`+budget (deprecated) | low/med/high/max | Older | |
|
233 |
+| `claude-sonnet-4-6` | Claude Sonnet 4.6 | 1M | 128K | 3.00 / 15.00 | adaptive or `enabled`+budget (deprecated) | low/med/high/max | Older | |
|
234 |
+| `claude-opus-4-5-20251101` | Claude Opus 4.5 | 200K | 64K | 5.00 / 25.00 (unverified) | `enabled` + `budget_tokens` | low/med/high | Legacy | |
|
235 |
+| `claude-sonnet-4-5-20250929` | Claude Sonnet 4.5 | 1M | 64K | 3.00 / 15.00 | `enabled` + `budget_tokens` | ❌ | Legacy | |
|
236 |
+| `claude-haiku-4-5-20251001` | Claude Haiku 4.5 | 200K | 64K | 1.00 / 5.00 | `enabled` + `budget_tokens` | ❌ | Fast/cheap | |
|
237 |
+| `claude-opus-4-1-20250805` | Claude Opus 4.1 | 200K | 32K | 15.00 / 75.00 | `enabled` + `budget_tokens` | ❌ | Deprecated (retires 2026-08-05) | |
|
238 |
+ |
|
239 |
+Notes: date-suffixed aliases exist for the dated models (`claude-opus-4-5`, `claude-sonnet-4-5`, `claude-haiku-4-5` resolve); the 4.6+ IDs have **no** date suffix — never append one. `claude-mythos-5` exists but is invite-only (Project Glasswing) — exclude from catalog. |
|
240 |
+ |
|
241 |
+## 4. Request/response format (Messages API — native, not OpenAI-compatible) |
|
242 |
+ |
|
243 |
+Key differences from OpenAI: `system` is a **top-level parameter** (not a message role, though 4.8+/Opus 5/Fable 5 also accept mid-conversation `role:"system"` messages); `max_tokens` is **required**; message content is an array of typed content blocks; roles strictly alternate user/assistant. |
|
244 |
+ |
|
245 |
+```json |
|
246 |
+POST /v1/messages |
|
247 |
+{ |
|
248 |
+ "model": "claude-opus-5", |
|
249 |
+ "max_tokens": 4096, |
|
250 |
+ "system": "You are a helpful assistant.", |
|
251 |
+ "messages": [ |
|
252 |
+ {"role": "user", "content": [ |
|
253 |
+ {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "<base64, no newlines>"}}, |
|
254 |
+ {"type": "text", "text": "What is in this image?"} |
|
255 |
+ ]} |
|
256 |
+ ], |
|
257 |
+ "stream": true |
|
258 |
+} |
|
259 |
+``` |
|
260 |
+ |
|
261 |
+- Response: `{"id", "type": "message", "role": "assistant", "model", "content": [{"type": "text", "text": "..."} | {"type": "thinking", "thinking": "...", "signature": "..."} | {"type": "tool_use", ...}], "stop_reason", "usage": {"input_tokens", "output_tokens", "cache_creation_input_tokens", "cache_read_input_tokens"}}` |
|
262 |
+- Vision block: `{"type": "image", "source": {"type": "base64", "media_type": "image/jpeg|image/png|image/gif|image/webp", "data": "<b64>"}}` (also `{"type": "url", "url": ...}`). Image block goes **before** the text block. |
|
263 |
+- `stop_reason` values: `end_turn`, `max_tokens`, `stop_sequence`, `tool_use`, `pause_turn`, `refusal` (check before reading content — Fable 5/Opus 5 classifiers can refuse with HTTP 200), `model_context_window_exceeded`. |
|
264 |
+- Multi-turn: resend full history; pass assistant `thinking` blocks back **unchanged** on the same model. |
|
265 |
+ |
|
266 |
+## 5. Streaming (SSE) — named events, no [DONE] |
|
267 |
+ |
|
268 |
+`"stream": true`. Each SSE frame has an `event:` name AND a `data:` JSON whose `type` matches: |
|
269 |
+ |
|
270 |
+| Event | Payload | |
|
271 |
+|---|---| |
|
272 |
+| `message_start` | `{"message": {id, model, role, usage: {input_tokens, ...}}}` — input token count arrives here | |
|
273 |
+| `content_block_start` | `{"index", "content_block": {"type": "text"|"thinking"|"tool_use", ...}}` | |
|
274 |
+| `content_block_delta` | `{"index", "delta": {"type": "text_delta", "text"} | {"type": "thinking_delta", "thinking"} | {"type": "input_json_delta", "partial_json"} | {"type": "signature_delta", ...}}` | |
|
275 |
+| `content_block_stop` | `{"index"}` | |
|
276 |
+| `message_delta` | `{"delta": {"stop_reason", "stop_sequence"}, "usage": {"output_tokens"}}` — **final output token count here** | |
|
277 |
+| `message_stop` | `{}` — end of stream (no `[DONE]` sentinel) | |
|
278 |
+| `ping` | keepalive — ignore | |
|
279 |
+| `error` | mid-stream error, e.g. `{"error": {"type": "overloaded_error", ...}}` — handle | |
|
280 |
+ |
|
281 |
+Route `thinking_delta` into the collapsible "Thinking…" section; `text_delta` into the message body. Track block `index` to separate blocks. |
|
282 |
+ |
|
283 |
+## 6. Special parameters (per-model gating is critical) |
|
284 |
+ |
|
285 |
+- **Thinking config matrix** (send exactly this or get 400s): |
|
286 |
+ - `claude-fable-5`: OMIT `thinking` entirely (always on) or `{"type":"adaptive"}`; `disabled`/`budget_tokens` → 400. |
|
287 |
+ - `claude-opus-5`: default adaptive; `{"type":"adaptive"}` or `{"type":"disabled"}` (disabled only allowed at effort ≤ high); `budget_tokens` → 400. |
|
288 |
+ - `claude-sonnet-5`: default adaptive; `{"type":"adaptive"}` or `{"type":"disabled"}`; `budget_tokens` → 400. |
|
289 |
+ - `claude-opus-4-8` / `-4-7`: off unless `{"type":"adaptive"}` set; `budget_tokens` → 400. |
|
290 |
+ - `claude-opus-4-6` / `claude-sonnet-4-6`: `{"type":"adaptive"}` recommended; `{"type":"enabled","budget_tokens":N}` deprecated but works. |
|
291 |
+ - Older (4.5 / haiku-4-5 / 4.1): `{"type":"enabled","budget_tokens":N}` required for thinking; N ≥ 1024 and < `max_tokens`. |
|
292 |
+ - To see thinking text on 4.7+/Sonnet 5/Opus 5/Fable 5, set `"thinking": {"type":"adaptive","display":"summarized"}` — default `"omitted"` streams empty thinking. |
|
293 |
+- **Effort**: `"output_config": {"effort": "low|medium|high|xhigh|max"}` (default high). Supported per the table in §3; errors on Sonnet 4.5/Haiku 4.5. |
|
294 |
+- **Sampling restrictions**: `temperature`/`top_p`/`top_k` are **removed (400)** on Opus 5, Fable 5, Opus 4.8, Opus 4.7; Sonnet 5 rejects non-default values; allowed on 4.6 and older (never send temperature AND top_p together on Claude 4+). Zyquo Cloud must hide these sliders for 4.7+ models. |
|
295 |
+- `max_tokens` required on every request; large values (>~16K) should always be streamed. |
|
296 |
+- No assistant prefill on 4.6+ (400). |
|
297 |
+- Prompt caching available via `cache_control` blocks (optional optimization; min cacheable prefix 512–4096 tokens depending on model). |
|
298 |
+ |
|
299 |
+## 7. Rate limits & errors |
|
300 |
+ |
|
301 |
+- Error body: `{"type": "error", "error": {"type": "invalid_request_error|authentication_error|permission_error|not_found_error|request_too_large|rate_limit_error|api_error|overloaded_error", "message": "..."}, "request_id": "req_..."}` |
|
302 |
+- HTTP: 400, 401, 403, 404, 413, 429 (retry-after header), 500, **529 overloaded** (Anthropic-specific — retry with backoff). |
|
303 |
+- Rate limits are tier-based per model: RPM + input-tokens/min (ITPM) + output-tokens/min (OTPM). Headers: `retry-after`, `anthropic-ratelimit-requests-remaining`, `anthropic-ratelimit-*-tokens-remaining/reset`. Docs: https://platform.claude.com/docs/en/api/rate-limits |
|
304 |
+- Retry 429/500/529 with exponential backoff; never retry 4xx (except 408/429). |
|
305 |
+ |
|
306 |
+## 8. /models listing |
|
307 |
+ |
|
308 |
+- `GET /v1/models` exists and is **rich**: `{"data": [{"type": "model", "id", "display_name", "created_at", "max_input_tokens", "max_tokens", "capabilities": {"image_input": {...}, "thinking": {"types": {"enabled", "adaptive"}}, "effort": {"low"…"max"}, "structured_outputs", "pdf_input", ...}], "has_more", "first_id", "last_id"}` — paginated (`after_id`). |
|
309 |
+- Use it for dynamic refresh: context window, output cap, thinking mode, and effort support can all be read live per model. 11 models returned on 2026-07-30. |
|
310 |
+ |
|
311 |
+### Catalog guidance for ModelCatalog / AnthropicClient |
|
312 |
+ |
|
313 |
+- Default model: `claude-sonnet-5` (balanced) with `claude-opus-5` as the flagship option. |
|
314 |
+- The client must gate: thinking config shape, effort support, and sampling params per model (matrix in §6) — encode these as capability flags in `AIModel`. |
|
315 |
+- Auto-title generation: use `claude-haiku-4-5-20251001`. |
|
316 |
+ |
|
317 |
+--- |
|
318 |
+ |
|
319 |
+ |
|
320 |
+# xAI (Grok) — API Research (Phase 0) |
|
321 |
+ |
|
322 |
+Researched: 2026-07-30. Sources: https://docs.x.ai/docs/models , https://docs.x.ai/docs/guides/chat , https://docs.x.ai/docs/guides/reasoning , https://docs.x.ai/docs/guides/live-search (now "Web Search tool"), https://docs.x.ai/docs/api-reference . Live model list verified against `GET /v1/models` with a real key on 2026-07-30 (`/tmp/zyquo-probe/xai.json`) — the live response includes aliases, context length, and pricing per model (authoritative). |
|
323 |
+ |
|
324 |
+## 1. Base URL & endpoints |
|
325 |
+ |
|
326 |
+- Base URL: `https://api.x.ai/v1` |
|
327 |
+- Chat completions (what Zyquo Cloud uses): `POST /v1/chat/completions` — OpenAI-compatible, fully supported (not deprecated). |
|
328 |
+- Responses API also exists (`POST /v1/responses`, plus `GET/DELETE /v1/responses/{id}`, `POST /v1/responses/compact`) — xAI's docs now lead with it, but chat completions remains concurrently supported. |
|
329 |
+- Deferred completions: `GET /v1/chat/deferred-completion/{request_id}`. |
|
330 |
+- Model listing: `GET /v1/models` (verified live; xAI-extended shape, see §8). |
|
331 |
+ |
|
332 |
+## 2. Authentication |
|
333 |
+ |
|
334 |
+- Header: `Authorization: Bearer <XAI_API_KEY>` (standard OpenAI-style). |
|
335 |
+- `Content-Type: application/json`. No version header. |
|
336 |
+ |
|
337 |
+## 3. Chat model catalog |
|
338 |
+ |
|
339 |
+Verified live 2026-07-30. Pricing per 1M tokens (input / cached input / output); xAI uses **long-context tiered pricing**: above the `long_context_threshold` (200K tokens for all current models) input/output prices double. All chat models: streaming ✅, function calling/tools ✅, structured outputs (json_schema) ✅, vision (image input) ✅ (image tokens priced same as text input). |
|
340 |
+ |
|
341 |
+| Model ID | Key aliases | Context | Max output | $/1M in / cached / out (<200K) | ≥200K in/out | Vision | Reasoning | Status | |
|
342 |
+|---|---|---|---|---|---|---|---|---| |
|
343 |
+| `grok-4.5` | `grok-4.5-latest`, `grok-build-latest` | 500K | unverified | 2.00 / 0.30 / 6.00 | 4.00 / 12.00 | ✅ | ✅ (`reasoning_effort` low/medium/high, default high) | **Recommended default** — "most intelligent and fastest model"; knowledge cutoff 2026-02-01 | |
|
344 |
+| `grok-4.3` | `grok-4.3-latest`, **`grok-latest`** | 1M | unverified | 1.25 / 0.20 / 2.50 | 2.50 / 5.00 | ✅ | ✅ (unverified whether `reasoning_effort` accepted) | Previous flagship | |
|
345 |
+| `grok-4.20-0309-reasoning` | `grok-4.20`, `grok-4.20-reasoning`, `grok-4.20-reasoning-latest`, `grok-4.20-beta`, … | 1M | unverified | 1.25 / 0.20 / 2.50 | 2.50 / 5.00 | ✅ | ✅ (always reasons) | Older; reasoning variant | |
|
346 |
+| `grok-4.20-0309-non-reasoning` | `grok-4.20-non-reasoning`, `grok-4.20-non-reasoning-latest`, … | 1M | unverified | 1.25 / 0.20 / 2.50 | 2.50 / 5.00 | ✅ | ❌ | Older; fast non-reasoning variant | |
|
347 |
+| `grok-4.20-multi-agent-0309` | `grok-4.20-multi-agent`, `grok-4.20-multi-agent-latest`, … | 1M | unverified | 1.25 / 0.20 / 2.50 | 2.50 / 5.00 | ✅ | ✅ (`reasoning_effort` controls **agent count**, not depth) | Specialized multi-agent | |
|
348 |
+| `grok-build-0.1` | `grok-code-fast-1`, `grok-code-fast` | 256K | unverified | 1.00 / 0.20 / 2.00 | 2.00 / 4.00 | ✅ (price listed) | ✅ (fast coding reasoner, unverified effort support) | Coding/build model | |
|
349 |
+ |
|
350 |
+Excluded (non-chat): `grok-imagine-image`, `grok-imagine-image-quality` (image gen, per-image pricing), `grok-imagine-video`, `grok-imagine-video-1.5`. |
|
351 |
+ |
|
352 |
+Notes: |
|
353 |
+- Older models (`grok-4`, `grok-4-fast`, `grok-3`, `grok-2-vision`, …) are **gone** from the live `/models` list — do not ship them in the catalog. |
|
354 |
+- Max output token caps are not published per model — mark unverified and default `max_tokens` conservatively (e.g. 16K); verify in Phase 7. |
|
355 |
+- `logprobs` is NOT supported on grok-4.20 and newer. |
|
356 |
+ |
|
357 |
+## 4. Request/response format |
|
358 |
+ |
|
359 |
+OpenAI-compatible — the shared `OpenAICompatibleClient` works as-is: |
|
360 |
+ |
|
361 |
+```json |
|
362 |
+POST /v1/chat/completions |
|
363 |
+{ |
|
364 |
+ "model": "grok-4.5", |
|
365 |
+ "messages": [ |
|
366 |
+ {"role": "system", "content": "You are Grok."}, |
|
367 |
+ {"role": "user", "content": [ |
|
368 |
+ {"type": "text", "text": "Describe this image"}, |
|
369 |
+ {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,....", "detail": "high"}} |
|
370 |
+ ]} |
|
371 |
+ ], |
|
372 |
+ "reasoning_effort": "high", |
|
373 |
+ "stream": true, |
|
374 |
+ "stream_options": {"include_usage": true} |
|
375 |
+} |
|
376 |
+``` |
|
377 |
+ |
|
378 |
+- Response: standard OpenAI `chat.completion` object (`choices[].message.content`, `finish_reason`, `usage`). Reasoning token counts appear under `usage.completion_tokens_details.reasoning_tokens`. |
|
379 |
+- Vision: OpenAI-style `image_url` content parts (https URL or base64 data URI; images up to 20 MiB). |
|
380 |
+- The Responses API additionally offers stored conversations (`previous_response_id`, 30-day retention, `store: false` to opt out) and `include: ["reasoning.encrypted_content"]` — not needed for chat completions. |
|
381 |
+ |
|
382 |
+## 5. Streaming format |
|
383 |
+ |
|
384 |
+- OpenAI-style SSE: unnamed `data:` lines of `chat.completion.chunk` objects with `choices[].delta.content`, `finish_reason` on the last content chunk, terminated by `data: [DONE]`. |
|
385 |
+- `stream_options: {"include_usage": true}` → final chunk with `usage` (use for token/cost display). |
|
386 |
+- Reasoning models stream **summarized reasoning** alongside the answer; on chat completions this has historically surfaced as a `delta.reasoning_content` field for models that expose it (grok-4-era models did not expose raw thinking). Whether grok-4.5 emits `reasoning_content` chunks on `/v1/chat/completions` is **unverified** — the StreamingService should parse `delta.reasoning_content` if present and route it to the Thinking section; verify in Phase 7. |
|
387 |
+ |
|
388 |
+## 6. Special parameters |
|
389 |
+ |
|
390 |
+- `reasoning_effort`: `"low" | "medium" | "high"` (default `high`) on `grok-4.5` and `grok-4.20-multi-agent` (where it controls the number of collaborating agents). Not applicable to `grok-4.20-0309-non-reasoning`. NOTE: xAI's value set differs from OpenAI's — no `none/minimal/xhigh/max`. |
|
391 |
+- Reasoning cannot be disabled on reasoning models; reasoning tokens are billed as output. |
|
392 |
+- **Parameter restrictions on reasoning models:** `presence_penalty`, `frequency_penalty`, and `stop` are **rejected with errors** — the client must not send them to Grok reasoning models. `logprobs` unsupported on 4.20+. `temperature`/`top_p` are accepted. |
|
393 |
+- Web search / live search: the legacy `search_parameters` live-search field has been superseded by a **`web_search` tool** (tools array; options: `allowed_domains` / `excluded_domains` (max 5, mutually exclusive), `enable_image_understanding`, `enable_image_search`) — documented for the Responses API; citations retrievable from results (docs: https://docs.x.ai/developers/tools/citations). Support via plain chat completions is unverified — treat search as a vNext feature for the xAI provider and verify in Phase 7. |
|
394 |
+- Deferred completions (async) exist but are out of scope. |
|
395 |
+ |
|
396 |
+## 7. Rate limits & errors |
|
397 |
+ |
|
398 |
+- Errors are generally OpenAI-shaped (`{"error": {"message", "type", "code"}}`), but xAI has also returned a flat `{"code": "...", "error": "<message>"}` shape on some 4xx responses — the error mapper should handle **both** shapes defensively. |
|
399 |
+- HTTP: 400 invalid request/params (including sending `stop`/penalties to reasoning models), 401 bad key, 403 blocked, 404 model, 429 rate limited, 5xx server. Retry 429/5xx with exponential backoff. |
|
400 |
+- Rate limits are per-model, per-team, shown in the xAI Console; not exhaustively documented publicly (marked unverified). Reasoning requests can run long — use generous read timeouts (xAI docs suggest up to 3600 s for reasoning models). |
|
401 |
+ |
|
402 |
+## 8. /models listing |
|
403 |
+ |
|
404 |
+- `GET /v1/models` exists (verified live). xAI extends the OpenAI shape — each entry: `{"id", "aliases": [...], "context_length", "created", "object": "model", "owned_by": "xai", "prompt_text_token_price", "cached_prompt_text_token_price", "prompt_image_token_price", "completion_text_token_price", "prompt_text_token_price_long_context", "completion_text_token_price_long_context", "long_context_threshold"}` wrapped in `{"data": [...], "object": "list"}`. |
|
405 |
+- **Price unit:** divide the integer by 10,000 to get USD per 1M tokens (e.g. `20000` → $2.00/1M; cross-checked against the official pricing table for grok-4.5/4.3/build-0.1). |
|
406 |
+- Image models instead carry `image_price` (per-image, same 10,000 divisor → USD per 10K images… treat image models as excluded). |
|
407 |
+- Zyquo Cloud can use this endpoint for dynamic refresh of context length AND live pricing — unique among the three providers. Filter out entries lacking `completion_text_token_price` (image/video models). |
|
408 |
+ |
|
409 |
+### Catalog guidance for ModelCatalog |
|
410 |
+ |
|
411 |
+- Default: `grok-4.5`. Fast/cheap: `grok-4.20-0309-non-reasoning` (or its `grok-4.20-non-reasoning` alias). Coding: `grok-build-0.1`. |
|
412 |
+- Prefer the stable aliases (`grok-4.5`, `grok-4.3`, `grok-4.20`, `grok-4.20-non-reasoning`, `grok-4.20-multi-agent`, `grok-code-fast-1`) as user-facing IDs. |
|
413 |
+- Gate per model: `reasoning_effort` (low/medium/high only), and strip `stop`/`presence_penalty`/`frequency_penalty` for reasoning models. |
|
414 |
+ |
|
415 |
+--- |
|
416 |
+ |
|
417 |
+ |
|
418 |
+# Mistral (La Plateforme) — API Research (Phase 0) |
|
419 |
+ |
|
420 |
+Researched 2026-07-30 against official docs (docs.mistral.ai, mistral.ai/pricing) and a **live `/v1/models` probe with a real key** (`/tmp/zyquo-probe/mistral.json`). The live list is authoritative for model existence, aliases, context lengths, capability flags, and deprecation dates. |
|
421 |
+ |
|
422 |
+--- |
|
423 |
+ |
|
424 |
+## 1. Base URL & Endpoints |
|
425 |
+ |
|
426 |
+Base: `https://api.mistral.ai` |
|
427 |
+ |
|
428 |
+| Endpoint | Method | Purpose | |
|
429 |
+|---|---|---| |
|
430 |
+| `/v1/chat/completions` | POST | Chat (streaming + non-streaming) | |
|
431 |
+| `/v1/models` | GET | List models (rich metadata) | |
|
432 |
+| `/v1/models/{id}` | GET | Model metadata | |
|
433 |
+| `/v1/fim/completions` | POST | Fill-in-the-middle (Codestral; not needed for chat) | |
|
434 |
+| `/v1/embeddings` | POST | Embeddings (not needed) | |
|
435 |
+ |
|
436 |
+Docs: https://docs.mistral.ai/api/ |
|
437 |
+ |
|
438 |
+## 2. Authentication |
|
439 |
+ |
|
440 |
+`Authorization: Bearer $MISTRAL_API_KEY`. Keys created in the La Plateforme console. |
|
441 |
+ |
|
442 |
+## 3. Chat Model Catalog |
|
443 |
+ |
|
444 |
+From the live probe (capability flags `completion_chat`, `function_calling`, `reasoning`, `vision`, `max_context_length`, `deprecation` are all per-model in the `/v1/models` response) + https://docs.mistral.ai/getting-started/models/models_overview/ + https://mistral.ai/pricing/api. Max output tokens: Mistral does not publish a separate max-output limit — output is bounded by the shared context window (`max_tokens` ≤ context minus prompt). All chat models support **streaming, function calling, and JSON mode**. Pricing in USD per 1M tokens (input/output); cached input tokens get a 90% discount. |
|
445 |
+ |
|
446 |
+### Recommended (current, non-deprecated) |
|
447 |
+ |
|
448 |
+| Model ID (canonical) | `-latest` alias | Marketing name | Context | Vision | Reasoning | $/1M in | $/1M out | |
|
449 |
+|---|---|---|---|---|---|---|---| |
|
450 |
+| `mistral-medium-2604` | `mistral-medium-latest` (also `mistral-medium-3-5`, `mistral-medium-3.5`, `mistral-medium`, `mistral-medium-3`) | Mistral Medium 3.5 — **flagship** | 262,144 | ✅ | ✅ (`reasoning_effort`) | $1.50 | $7.50 | |
|
451 |
+| `mistral-large-2512` | `mistral-large-latest` | Mistral Large 3 | 262,144 | ✅ | ❌ | $0.50 | $1.50 | |
|
452 |
+| `mistral-small-2603` | `mistral-small-latest` (also `magistral-small-latest`) | Mistral Small 4 (hybrid instruct+reasoning+code) | 262,144 | ✅ | ✅ | $0.15 | $0.60 | |
|
453 |
+| `codestral-2508` | `codestral-latest` (also `mistral-code-latest`, `mistral-code-fim-latest`) | Codestral (coding) | 256,000 | ❌ | ❌ | $0.30 | $0.90 | |
|
454 |
+| `ministral-14b-2512` | `ministral-14b-latest` | Ministral 3 14B | 262,144 | ✅ | ❌ | $0.20 | $0.20 | |
|
455 |
+| `ministral-8b-2512` | `ministral-8b-latest` | Ministral 3 8B | 262,144 | ✅ | ❌ | $0.15 | $0.15 | |
|
456 |
+| `ministral-3b-2512` | `ministral-3b-latest` | Ministral 3 3B | 131,072 | ✅ | ❌ | $0.10 | $0.10 | |
|
457 |
+ |
|
458 |
+Note: `mistral-large-latest` (Large 3, open-weight) is now priced *below* `mistral-medium-latest` (Medium 3.5, frontier) — Medium 3.5 is the flagship despite the name. |
|
459 |
+ |
|
460 |
+### Legacy / deprecated (still live in the API — mark "legacy" in ModelCatalog, hide by default) |
|
461 |
+ |
|
462 |
+| Model ID | Context | Deprecated on | Replacement | $/1M in/out | |
|
463 |
+|---|---|---|---|---| |
|
464 |
+| `mistral-medium-2508`, `mistral-medium-2505` | 131,072 | 2026-08-31 | `mistral-medium-3-5` | unverified (old Medium 3 pricing) | |
|
465 |
+| `magistral-medium-2509` / `magistral-medium-latest` | 131,072 | **2026-07-31 (tomorrow!)** | `mistral-medium-3-5` | $2.00 / $5.00 | |
|
466 |
+| `magistral-small-2509` | 131,072 | **2026-07-31** | `mistral-small-latest` | $0.50 / $1.50 | |
|
467 |
+| `mistral-small-2506` | 131,072 | **2026-07-31** | `mistral-small-latest` | unverified | |
|
468 |
+| `devstral-2512` / `devstral-latest` / `devstral-medium-latest` | 262,144 | **2026-07-31** | `mistral-medium-3-5` | Devstral 2: $0.40 / $2.00 | |
|
469 |
+| `open-mistral-nemo` (+ `open-mistral-nemo-2407`, `mistral-tiny-2407`, `mistral-tiny-latest`) | 131,072 | **2026-07-31** | `ministral-8b-2512` | $0.15 / $0.15 | |
|
470 |
+ |
|
471 |
+**Notable:** the standalone **Magistral reasoning line is deprecated** — reasoning moved into `mistral-medium-latest` / `mistral-small-latest` (docs: "Native reasoning models (magistral-small/medium-latest) have been deprecated"; `magistral-small-latest` is literally an alias of `mistral-small-2603` in the live list). **Pixtral is gone** from the live list (absorbed into vision-capable mainline models). Excluded from catalog: `voxtral-*` (audio), `mistral-embed*`, `codestral-embed*`, `mistral-moderation-*`, `mistral-ocr-*`, `labs-leanstral-*` (Lean theorem-proving lab model), `mistral-vibe-cli-*` (product aliases of medium/small). |
|
472 |
+ |
|
473 |
+## 4. Request / Response Format |
|
474 |
+ |
|
475 |
+**OpenAI-compatible** with minor Mistral extensions. Full parameter list (https://docs.mistral.ai/api/): |
|
476 |
+ |
|
477 |
+`model` (required), `messages` (required; roles `system`/`user`/`assistant`/`tool`), `temperature` (recommend 0.0–0.7; default varies per model — live probe exposes `default_model_temperature`, e.g. 0.3 for most, 1.0 for medium-3.5), `top_p`, `max_tokens`, `stream` (default false), `stop` (string or array), `random_seed`, `response_format` (`{"type": "text" | "json_object" | "json_schema"}`), `tools`, `tool_choice` (`auto`/`none`/`any`/`required`), `parallel_tool_calls` (default true), `presence_penalty`, `frequency_penalty`, `n`, `prompt_cache_key`, and Mistral-specific `safe_prompt` (default false) and `prompt_mode` / `reasoning_effort` (see §6). |
|
478 |
+ |
|
479 |
+```json |
|
480 |
+POST /v1/chat/completions |
|
481 |
+{ |
|
482 |
+ "model": "mistral-medium-latest", |
|
483 |
+ "messages": [ |
|
484 |
+ { "role": "system", "content": "You are a helpful assistant." }, |
|
485 |
+ { "role": "user", "content": "Reply with exactly: OK" } |
|
486 |
+ ], |
|
487 |
+ "temperature": 0.3, |
|
488 |
+ "max_tokens": 128, |
|
489 |
+ "stream": false |
|
490 |
+} |
|
491 |
+``` |
|
492 |
+ |
|
493 |
+Response is standard OpenAI shape: `id`, `object: "chat.completion"`, `created`, `model`, `choices[].message`, `choices[].finish_reason`, `usage: {prompt_tokens, completion_tokens, total_tokens}`. |
|
494 |
+ |
|
495 |
+**Deviation to handle in the client:** for reasoning-capable models with `reasoning_effort: "high"`, `message.content` is **not a string** but an **array of chunks**: `{"type": "thinking", "thinking": [{"type":"text","text":"..."}]}` followed by `{"type": "text", "text": "..."}`. The Swift `Codable` model must accept `content` as string OR chunk array. |
|
496 |
+ |
|
497 |
+**Vision:** OpenAI style content parts — `{"type": "image_url", "image_url": "<https url or data:image/jpeg;base64,...>"}` (both URL and base64 data URI supported; https://docs.mistral.ai/capabilities/vision/). Vision models: mistral-medium, mistral-large-3, mistral-small-4, ministral-3 family (per live capability flags). |
|
498 |
+ |
|
499 |
+## 5. Streaming |
|
500 |
+ |
|
501 |
+Standard OpenAI SSE: `data: {chat.completion.chunk}` events with `choices[0].delta.content`, terminated by `data: [DONE]`. Usage arrives on the final chunk. **Reasoning wrinkle:** during the thinking phase, `delta.content` is a **list containing thinking chunks**, then transitions to **plain strings** for the answer phase — the SSE parser must handle both shapes for the same field. |
|
502 |
+ |
|
503 |
+## 6. Special Parameters |
|
504 |
+ |
|
505 |
+- **`safe_prompt`** (bool, default false): injects Mistral's safety guardrail system prompt before the conversation. |
|
506 |
+- **Reasoning** (https://docs.mistral.ai/capabilities/reasoning/): on `mistral-medium-latest` / `mistral-small-latest`, use **`reasoning_effort`**: `"high"` → full thinking chunks in output (recommended for agentic/code with medium-3.5); `"none"` → plain string output, no thinking. (`prompt_mode: "reasoning"` also exists in the API reference as a reasoning switch — appears to be the older Magistral-era mechanism; exact interaction with `reasoning_effort` unverified.) |
|
507 |
+- **Multi-turn with reasoning:** replay the **full** assistant message including the `thinking` chunk back into history — stripping reasoning traces degrades quality (official guidance). |
|
508 |
+- **`random_seed`** for reproducibility; **`prompt_cache_key`** for prompt caching (cached input: 90% discount). |
|
509 |
+- Temperature guidance: 0.0–0.7 recommended; per-model defaults exposed in `/v1/models` (`default_model_temperature`). |
|
510 |
+ |
|
511 |
+## 7. Rate Limits & Errors |
|
512 |
+ |
|
513 |
+(https://docs.mistral.ai/admin/user-management-finops/tier + help.mistral.ai) Two plans: **Free** (Experiment) and **Scale** (pay-as-you-go, higher limits). Limits are per model along two independent axes: **requests per second** and **tokens per minute** (plus tokens/month on free tier; ~1B tokens/month on the free Experiment tier — semi-verified, third-party). Mistral **no longer publishes exact numbers** — live values are in Admin Console → Limits. Org- and workspace-level monthly spending caps can suspend API access when reached. Higher limits: contact support with model/RPS/token-volume details. |
|
514 |
+ |
|
515 |
+Errors: OpenAI-style JSON. 401 invalid key, 422 invalid request/params (Mistral is stricter than OpenAI here — validation errors come back as 422 with `detail`), 429 rate limit, 5xx server. Exact error body shape varies (`{"object":"error","message",...}` vs FastAPI-style `{"detail":[...]}` on 422) — confirm both in Phase 7. Backoff on 429/5xx. |
|
516 |
+ |
|
517 |
+## 8. `/models` Listing |
|
518 |
+ |
|
519 |
+**Yes — the richest of all providers.** `GET /v1/models` → `{"object":"list","data":[...]}` where every entry includes: `id`, `name` (canonical), `description`, `max_context_length`, `aliases[]`, `deprecation` (ISO date or null), `deprecation_replacement_model`, `default_model_temperature`, `type`, and a full `capabilities` object (`completion_chat`, `completion_fim`, `function_calling`, `reasoning`, `vision`, `ocr`, `classification`, `moderation`, `audio`, `audio_transcription`, `audio_speech`, `fine_tuning`, …). Zyquo Cloud's dynamic refresh can build the entire Mistral catalog from this endpoint alone: filter `completion_chat == true && ocr == false && moderation == false && audio* == false`, dedupe by `name` (aliases each appear as separate entries), and flag `deprecation != null` as legacy. |
|
520 |
+ |
|
521 |
+--- |
|
522 |
+ |
|
523 |
+ |
|
524 |
+# Google Gemini — API Research (Phase 0) |
|
525 |
+ |
|
526 |
+Researched 2026-07-30 against official docs (ai.google.dev) and a **live `/v1beta/models` probe with a real key** (`/tmp/zyquo-probe/gemini.json`). Where docs and the live list disagree, the live list wins for model existence. |
|
527 |
+ |
|
528 |
+--- |
|
529 |
+ |
|
530 |
+## 1. Base URL & Endpoints |
|
531 |
+ |
|
532 |
+### Native Gemini API (v1beta) |
|
533 |
+ |
|
534 |
+Base: `https://generativelanguage.googleapis.com` |
|
535 |
+ |
|
536 |
+| Endpoint | Method | Purpose | |
|
537 |
+|---|---|---| |
|
538 |
+| `/v1beta/models/{model}:generateContent` | POST | Non-streaming chat | |
|
539 |
+| `/v1beta/models/{model}:streamGenerateContent?alt=sse` | POST | SSE streaming chat | |
|
540 |
+| `/v1beta/models` | GET | List models (paginated, `pageSize`/`pageToken`) | |
|
541 |
+| `/v1beta/models/{model}` | GET | Model metadata | |
|
542 |
+| `/v1beta/models/{model}:countTokens` | POST | Token counting | |
|
543 |
+| `/v1beta/cachedContents` | POST/GET | Explicit context caching | |
|
544 |
+ |
|
545 |
+Docs: https://ai.google.dev/api/generate-content |
|
546 |
+ |
|
547 |
+### OpenAI-compatible endpoint (recommended for Zyquo Cloud's `OpenAICompatibleClient`) |
|
548 |
+ |
|
549 |
+Base: `https://generativelanguage.googleapis.com/v1beta/openai/` |
|
550 |
+ |
|
551 |
+| Endpoint | Purpose | |
|
552 |
+|---|---| |
|
553 |
+| `/v1beta/openai/chat/completions` | Chat (streaming + non-streaming) | |
|
554 |
+| `/v1beta/openai/models` and `/models/{id}` | Model listing/retrieval | |
|
555 |
+| `/v1beta/openai/embeddings` | Embeddings | |
|
556 |
+| `/v1beta/openai/images/generations`, `/videos`, `/batches` | Media/batch (not needed for chat) | |
|
557 |
+ |
|
558 |
+Docs: https://ai.google.dev/gemini-api/docs/openai |
|
559 |
+ |
|
560 |
+**Compat-endpoint limitations (verified 2026-07):** officially "still in beta while we extend feature support". Supported: chat completions with streaming, function calling (`tools`), structured outputs (JSON schema), vision (base64 data URIs), `reasoning_effort`, and Gemini-specific extras via `extra_body` (`thinking_config`, `safety_settings`, `cached_content`, Google Search grounding `tools` on Gemini 3+). Batch file upload/download requires the native `genai` SDK. Native-only features (thought summaries as `thought: true` parts, `thoughtSignature` round-tripping, fine-grained safety feedback, `usageMetadata.thoughtsTokenCount`) are not fully exposed — if the app needs rich thinking display for Gemini, the native `GeminiClient` is the better path. |
|
561 |
+ |
|
562 |
+## 2. Authentication |
|
563 |
+ |
|
564 |
+- **Native API:** header `x-goog-api-key: $GEMINI_API_KEY` (preferred) or query param `?key=$GEMINI_API_KEY`. |
|
565 |
+- **OpenAI-compat endpoint:** `Authorization: Bearer $GEMINI_API_KEY`. |
|
566 |
+ |
|
567 |
+Same API key (from Google AI Studio) works for both. |
|
568 |
+ |
|
569 |
+## 3. Chat Model Catalog (text-generation chat models only) |
|
570 |
+ |
|
571 |
+Source of truth: live `/models` probe (2026-07-30) + https://ai.google.dev/gemini-api/docs/models + https://ai.google.dev/gemini-api/docs/pricing. All context/output limits below are the exact `inputTokenLimit`/`outputTokenLimit` from the live probe. All listed models support **streaming, vision (multimodal input), function calling, and JSON mode (structured output)**; the "Thinking" column is the probe's `thinking` flag. Pricing = standard tier, USD per 1M tokens, text input (audio input often costs more; noted where documented). Prices from the official pricing page unless marked. |
|
572 |
+ |
|
573 |
+### Recommended (current generation) |
|
574 |
+ |
|
575 |
+| Model ID | Status | Context | Max out | Input $/1M | Output $/1M | Thinking | |
|
576 |
+|---|---|---|---|---|---|---| |
|
577 |
+| `gemini-3.6-flash` | **Stable — flagship** | 1,048,576 | 65,536 | $1.50 | $7.50 | ✅ (default: medium) | |
|
578 |
+| `gemini-3.5-flash` | Stable | 1,048,576 | 65,536 | $1.50 | $9.00 | ✅ | |
|
579 |
+| `gemini-3.5-flash-lite` | Stable | 1,048,576 | 65,536 | $0.30 | $2.50 | ✅ (default: minimal) | |
|
580 |
+| `gemini-3.1-pro-preview` | Preview | 1,048,576 | 65,536 | $2.00 (≤200k) / $4.00 (>200k) | $12.00 / $18.00 | ✅ (default: high) | |
|
581 |
+| `gemini-3.1-flash-lite` | Stable | 1,048,576 | 65,536 | $0.25 (text/img/video), $0.50 (audio) | $1.50 | ✅ | |
|
582 |
+| `gemini-2.5-pro` | Stable | 1,048,576 | 65,536 | $1.25 (≤200k) / $2.50 (>200k) | $10.00 / $15.00 | ✅ (cannot be disabled) | |
|
583 |
+| `gemini-2.5-flash` | Stable | 1,048,576 | 65,536 | $0.30 (text), $1.00 (audio) | $2.50 | ✅ | |
|
584 |
+| `gemini-2.5-flash-lite` | Stable | 1,048,576 | 65,536 | $0.10 (text), $0.30 (audio) | $0.40 | ✅ (off by default) | |
|
585 |
+ |
|
586 |
+### Rolling aliases (auto-track latest release — good defaults for the app) |
|
587 |
+ |
|
588 |
+| Alias | Currently points at (probe) | Context / Max out | |
|
589 |
+|---|---|---| |
|
590 |
+| `gemini-pro-latest` | latest Pro release | 1,048,576 / 65,536 | |
|
591 |
+| `gemini-flash-latest` | latest Flash release | 1,048,576 / 65,536 | |
|
592 |
+| `gemini-flash-lite-latest` | latest Flash-Lite release | 1,048,576 / 65,536 | |
|
593 |
+ |
|
594 |
+### Preview / secondary (available, lower priority) |
|
595 |
+ |
|
596 |
+| Model ID | Status | Context | Max out | Pricing | Notes | |
|
597 |
+|---|---|---|---|---|---| |
|
598 |
+| `gemini-3-pro-preview` | Preview | 1,048,576 | 65,536 | $2.00/$12.00 (≤200k), $4.00/$18.00 (>200k) — corroborated by third-party trackers; not on official pricing page (semi-verified) | Superseded by 3.1 Pro | |
|
599 |
+| `gemini-3-flash-preview` | Preview | 1,048,576 | 65,536 | $0.50 (text) / $3.00 | Superseded by 3.5/3.6 Flash | |
|
600 |
+| `gemini-3.1-pro-preview-customtools` | Preview | 1,048,576 | 65,536 | unverified (assume 3.1 Pro pricing) | Tool-use-optimized variant | |
|
601 |
+| `gemini-3.1-flash-lite-preview` | Preview | 1,048,576 | 65,536 | unverified (assume 3.1 Flash-Lite pricing) | Preview channel of stable 3.1 Flash-Lite | |
|
602 |
+| `gemini-omni-flash-preview` | Preview | 131,072 | 65,536 | unverified | Omni multimodal preview | |
|
603 |
+| `gemini-2.0-flash` / `gemini-2.0-flash-001` | Stable, legacy | 1,048,576 | 8,192 | $0.10 (text), $0.70 (audio) / $0.40 | No thinking | |
|
604 |
+| `gemini-2.0-flash-lite` / `-001` | Stable, legacy | 1,048,576 | 8,192 | $0.075 / $0.30 | No thinking | |
|
605 |
+| `gemma-4-26b-a4b-it` | Open model | 262,144 | 32,768 | unverified (Gemma historically free tier) | Text-focused; function calling unverified | |
|
606 |
+| `gemma-4-31b-it` | Open model | 262,144 | 32,768 | unverified | Text-focused; function calling unverified | |
|
607 |
+ |
|
608 |
+**Excluded from the chat catalog** (present in live list but not chat-app material): TTS models (`gemini-2.5-*-tts`, `gemini-3.1-flash-tts-preview`), image models (`gemini-2.5-flash-image`, `gemini-3-pro-image[-preview]`, `nano-banana-pro-preview`, `gemini-3.1-flash[-lite]-image[-preview]`), Imagen/Veo/Lyria, embeddings (`gemini-embedding-001/2`), Live/native-audio (`bidiGenerateContent`-only), robotics (`gemini-robotics-er-1.5/1.6`), `gemini-2.5-computer-use-preview-10-2025`, `aqa`, and agentic specials (`antigravity-preview-05-2026`, `deep-research[-max|-pro]-preview-*` — generateContent-capable but agent products, not chat models). |
|
609 |
+ |
|
610 |
+Context caching: cached input is heavily discounted (e.g. 3.1 Flash-Lite $0.025/1M cached + $1.00/1M/hr storage; 2.0 Flash $0.025/1M). Batch/Flex modes: 50% off. (https://ai.google.dev/gemini-api/docs/pricing) |
|
611 |
+ |
|
612 |
+## 4. Request / Response Format |
|
613 |
+ |
|
614 |
+### Native (v1beta `generateContent`) — NOT OpenAI-compatible |
|
615 |
+ |
|
616 |
+Roles are `user` and `model` (not `assistant`). System prompt goes in top-level `systemInstruction`. Text and images are `parts`. |
|
617 |
+ |
|
618 |
+```json |
|
619 |
+POST /v1beta/models/gemini-3.6-flash:generateContent |
|
620 |
+{ |
|
621 |
+ "systemInstruction": { "parts": [{ "text": "You are a helpful assistant." }] }, |
|
622 |
+ "contents": [ |
|
623 |
+ { "role": "user", "parts": [ |
|
624 |
+ { "text": "What is in this image?" }, |
|
625 |
+ { "inline_data": { "mime_type": "image/jpeg", "data": "<base64>" } } |
|
626 |
+ ]} |
|
627 |
+ ], |
|
628 |
+ "generationConfig": { |
|
629 |
+ "temperature": 1.0, |
|
630 |
+ "topP": 0.95, |
|
631 |
+ "maxOutputTokens": 8192, |
|
632 |
+ "responseMimeType": "application/json", |
|
633 |
+ "thinkingConfig": { "thinkingLevel": "medium", "includeThoughts": true } |
|
634 |
+ }, |
|
635 |
+ "safetySettings": [ { "category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE" } ] |
|
636 |
+} |
|
637 |
+``` |
|
638 |
+ |
|
639 |
+Response: `candidates[0].content.parts[].text`, `candidates[0].finishReason` (`STOP`, `MAX_TOKENS`, `SAFETY`, …), `usageMetadata` (`promptTokenCount`, `candidatesTokenCount`, `thoughtsTokenCount`, `cachedContentTokenCount`, `totalTokenCount`), plus `modelVersion`, `responseId`, optional `promptFeedback.blockReason`. Thought summaries arrive as parts with `"thought": true`. Structured output: `responseMimeType: "application/json"` + `responseSchema`. |
|
640 |
+ |
|
641 |
+**Vision (native):** `inline_data: { mime_type, data }` with raw base64 (no data-URI prefix), or `file_data: { mime_type, file_uri }` via the Files API. |
|
642 |
+ |
|
643 |
+### OpenAI-compat endpoint |
|
644 |
+ |
|
645 |
+Standard OpenAI chat schema (`model`, `messages` with `system`/`user`/`assistant` roles, `temperature`, `top_p`, `max_tokens`, `stream`, `tools`, `response_format`). Vision uses OpenAI style: `{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<b64>"}}`. Gemini extras go in `extra_body` (`thinking_config`, `safety_settings`, `cached_content`). |
|
646 |
+ |
|
647 |
+```json |
|
648 |
+POST /v1beta/openai/chat/completions |
|
649 |
+{ |
|
650 |
+ "model": "gemini-3.6-flash", |
|
651 |
+ "reasoning_effort": "low", |
|
652 |
+ "messages": [ |
|
653 |
+ { "role": "system", "content": "You are a helpful assistant." }, |
|
654 |
+ { "role": "user", "content": "Reply with exactly: OK" } |
|
655 |
+ ], |
|
656 |
+ "stream": true |
|
657 |
+} |
|
658 |
+``` |
|
659 |
+ |
|
660 |
+## 5. Streaming |
|
661 |
+ |
|
662 |
+- **Native:** `:streamGenerateContent?alt=sse` → SSE stream of `data: {GenerateContentResponse}` chunks; each chunk carries incremental `candidates[0].content.parts` (thought-summary parts stream too, as rolling incremental summaries with `thought: true`). `usageMetadata` arrives on chunks (final chunk carries the complete counts). No `data: [DONE]` sentinel — stream ends when the HTTP body closes and the last chunk has a `finishReason`. Without `?alt=sse` the endpoint returns a streamed JSON array instead — always pass `alt=sse`. |
|
663 |
+- **OpenAI-compat:** standard OpenAI SSE — `data: {chat.completion.chunk}` with `choices[0].delta.content`, terminated by `data: [DONE]`; usage on the final chunk. |
|
664 |
+ |
|
665 |
+## 6. Special Parameters |
|
666 |
+ |
|
667 |
+- **Thinking** (https://ai.google.dev/gemini-api/docs/generate-content/thinking): |
|
668 |
+ - **Gemini 3.x models:** `thinkingConfig.thinkingLevel` = `"minimal" | "low" | "medium" | "high"` (minimal ≈ no thinking for most queries; default varies — 3.6/3.5 Flash: medium, 3.5 Flash-Lite: minimal, 3.1 Pro: high). Setting both `thinkingLevel` and `thinkingBudget` on a Gemini 3 model → error. |
|
669 |
+ - **Gemini 2.5 models:** `thinkingConfig.thinkingBudget` (token cap). 2.5 Pro: dynamic default, range 128–32,768, **cannot be disabled**; 2.5 Flash: 0–24,576 (0 disables); 2.5 Flash-Lite: off by default, 512–24,576. `-1` = dynamic. |
|
670 |
+ - `thinkingConfig.includeThoughts: true` → thought-summary parts (`thought: true`). Full thinking tokens are billed as output even though only summaries are returned. |
|
671 |
+ - **`thoughtSignature`:** encrypted reasoning state returned on parts; must be echoed back in multi-turn function-calling flows to preserve reasoning context. |
|
672 |
+ - **Compat endpoint:** `reasoning_effort` = `minimal|low|medium|high` maps to thinking levels; or `extra_body.thinking_config`. |
|
673 |
+- **Safety settings:** `safetySettings[]` with `category` (`HARM_CATEGORY_HARASSMENT`, `_HATE_SPEECH`, `_SEXUALLY_EXPLICIT`, `_DANGEROUS_CONTENT`, `_CIVIC_INTEGRITY`) and `threshold` (`BLOCK_NONE`, `BLOCK_ONLY_HIGH`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_LOW_AND_ABOVE`). Available via `extra_body.safety_settings` on the compat endpoint. |
|
674 |
+- **Context caching:** `cachedContent: "cachedContents/{id}"` (native) / `extra_body.cached_content` (compat). |
|
675 |
+- Sampling: `temperature` 0–2 (default 1.0 per probe), `topP` (default 0.95), `topK` (default 64 on 2.5+/3.x, 40 on 2.0), `candidateCount`, `stopSequences`. |
|
676 |
+- Google Search grounding via `tools` (Gemini 3+ on compat endpoint). |
|
677 |
+ |
|
678 |
+## 7. Rate Limits & Errors |
|
679 |
+ |
|
680 |
+(https://ai.google.dev/gemini-api/docs/rate-limits) Limits are per-project across three dimensions: RPM, TPM, RPD — exceeding any one triggers **HTTP 429 `RESOURCE_EXHAUSTED`**. Tiers auto-assigned by spend: Free; Tier 1 (billing linked, ~$250 cap); Tier 2 ($100+ spent, 3+ days); Tier 3 ($1,000+ spent, 30+ days). Additional **spend-based rolling 10-minute caps**: Tier 1 $10/10min, Tiers 2–3 $200/10min. Per-model RPM/TPM values are no longer published universally — shown live at https://aistudio.google.com/rate-limit. |
|
681 |
+ |
|
682 |
+**Error format (native):** standard Google API error envelope: |
|
683 |
+ |
|
684 |
+```json |
|
685 |
+{ "error": { "code": 429, "message": "...", "status": "RESOURCE_EXHAUSTED", "details": [ ... ] } } |
|
686 |
+``` |
|
687 |
+ |
|
688 |
+Common statuses: 400 `INVALID_ARGUMENT`, 403 `PERMISSION_DENIED` (bad key), 404 `NOT_FOUND` (bad model), 429 `RESOURCE_EXHAUSTED`, 500 `INTERNAL`, 503 `UNAVAILABLE`. The compat endpoint returns OpenAI-style `{"error": {"message", "type", "code"}}` (shape unverified in docs — confirm in Phase 7). Retry with exponential backoff on 429/5xx. |
|
689 |
+ |
|
690 |
+## 8. `/models` Listing |
|
691 |
+ |
|
692 |
+**Yes — both endpoints.** |
|
693 |
+- Native: `GET /v1beta/models` → `{ "models": [ { "name": "models/<id>", "displayName", "description", "inputTokenLimit", "outputTokenLimit", "supportedGenerationMethods": ["generateContent", ...], "temperature", "topP", "topK", "maxTemperature", "thinking" } ], "nextPageToken"? }`. **Rich**: Zyquo Cloud can derive context window, max output, thinking support, and chat capability (`generateContent` in `supportedGenerationMethods`) directly. Filter out non-chat models by method (`predict`, `bidiGenerateContent`, `embedContent`, `generateAnswer`) and by name patterns (`-tts`, `-image`, `imagen-`, `veo-`, `lyria-`, `embedding`, `robotics`, `computer-use`, `deep-research`, `antigravity`, `aqa`). |
|
694 |
+- Compat: `GET /v1beta/openai/models` → OpenAI-style `{"object":"list","data":[{"id","object":"model","owned_by"}]}` (IDs only, no metadata). |
|
695 |
+ |
|
696 |
+--- |
|
697 |
+ |
|
698 |
+ |
|
699 |
+# Alibaba Qwen (DashScope / Model Studio) — Provider Research |
|
700 |
+ |
|
701 |
+Research date: **2026-07-30**. Live `/models` probe (151 models) from `dashscope-intl.aliyuncs.com/compatible-mode/v1/models` with a real key is the authoritative ID list. Official docs: [Model Studio — Supported Models](https://www.alibabacloud.com/help/en/model-studio/models), [Text generation models](https://www.alibabacloud.com/help/en/model-studio/text-generation-model/), [Deep thinking](https://www.alibabacloud.com/help/en/model-studio/deep-thinking), [Vision models](https://www.alibabacloud.com/help/en/model-studio/vision-model/), [Rate limits](https://www.alibabacloud.com/help/en/model-studio/rate-limit), [Error codes](https://www.alibabacloud.com/help/en/model-studio/error-code). |
|
702 |
+ |
|
703 |
+--- |
|
704 |
+ |
|
705 |
+## 1. Base URL & Endpoints |
|
706 |
+ |
|
707 |
+**Zyquo Cloud uses the international (Singapore) OpenAI-compatible endpoint:** |
|
708 |
+ |
|
709 |
+- Base URL: `https://dashscope-intl.aliyuncs.com/compatible-mode/v1` |
|
710 |
+- Chat completions: `POST /chat/completions` |
|
711 |
+- Model listing: `GET /models` (verified live today — 151 models) |
|
712 |
+ |
|
713 |
+Other endpoints (documented, not used by the app): |
|
714 |
+- China (Beijing) endpoint: `https://dashscope.aliyuncs.com/compatible-mode/v1` — separate API keys, separate (60–70% cheaper) pricing, separate catalog. **Do not mix keys across regions** (a region-mismatched key returns 401 "Incorrect API key provided"). |
|
715 |
+- Native DashScope API: `https://dashscope-intl.aliyuncs.com/api/v1/services/aigc/text-generation/generation` — not needed; the OpenAI-compatible mode covers everything Zyquo Cloud does. |
|
716 |
+- An Anthropic-compatible mode also exists per the [models page](https://www.alibabacloud.com/help/en/model-studio/models) ("OpenAI-compatible, Anthropic-compatible, and DashScope" modes) — not used. |
|
717 |
+ |
|
718 |
+## 2. Authentication |
|
719 |
+ |
|
720 |
+Standard Bearer auth (confirmed by live probe): |
|
721 |
+ |
|
722 |
+``` |
|
723 |
+Authorization: Bearer sk-... |
|
724 |
+``` |
|
725 |
+ |
|
726 |
+Keys start with `sk-` and are region-bound (international console key for the intl endpoint). |
|
727 |
+ |
|
728 |
+## 3. Chat Model Catalog (text-generation chat models only) |
|
729 |
+ |
|
730 |
+Filtered from the live 151-model probe: image (`qwen-image*`, `wan*`, `z-image*`), TTS/ASR (`qwen3-tts*`, `qwen3-asr*`, `tongyi-tingwu-slp`), omni/realtime (`qwen*-omni*`, `qwen3-s2s*`), live-translate, machine-translation (`qwen-mt-*`), OCR (`qwen-vl-ocr*`), and embeddings (`text-embedding-v3/v4`) are excluded. |
|
731 |
+ |
|
732 |
+**Pricing caveat:** the official international docs pages do not render pricing tables to scrapers; USD prices below marked "(3p)" come from third-party trackers ([VentureBeat](https://venturebeat.com/technology/alibabas-qwen3-7-plus-supports-text-video-and-imagery-inputs-at-low-cost-of-0-4-1-6-per-1m-token-but-its-proprietary), [codersera](https://codersera.com/blog/qwen-3-7-max-launch-guide-2026/), OpenRouter) and should be re-verified in the Model Studio console; treat as **unverified**. Note Qwen commercial models use **tiered pricing by input length** on some SKUs, which explains conflicting third-party numbers (e.g. qwen3.7-plus quoted at $0.32/$1.28 and $0.40/$1.60 per 1M in/out). |
|
733 |
+ |
|
734 |
+### Flagship commercial (recommended) |
|
735 |
+ |
|
736 |
+| API ID | Context | Max output | Price /1M in/out (USD) | Vision | Tools | Thinking | Notes | |
|
737 |
+|---|---|---|---|---|---|---|---| |
|
738 |
+| `qwen3.7-max` | 1M | unverified | ~$2.50 / $7.50 (3p) | no (text-only) | yes | **on by default** | Flagship, agentic. Snapshots: `-2026-05-17`, `-2026-05-20`, `-2026-06-08`, plus `qwen3.7-max-preview` | |
|
739 |
+| `qwen3.7-plus` | 1M | 65,536 (3p) | ~$0.32 / $1.28 (3p) | yes (image+video, up to 2 h video) | yes | **on by default** | Multimodal hybrid agent (screen reading / GUI ops). Snapshot `-2026-05-26` | |
|
740 |
+| `qwen3.7-flash` | 1M | 65,536 (3p) | ~$0.03 / $0.13 (3p) | yes | yes | hybrid | Cheapest current-gen. Snapshot `-2026-07-15` | |
|
741 |
+| `qwen3.6-plus` | 1M | unverified | unverified | yes | yes | hybrid | Snapshot `-2026-04-02` | |
|
742 |
+| `qwen3.6-flash` | 1M | unverified | unverified | yes (2 h / 2 GB video) | yes | hybrid | "Near-flagship, same feature set as qwen3.7-plus at lower price". Snapshot `-2026-04-16` | |
|
743 |
+| `qwen3.6-max-preview` | 256K | unverified | unverified | no | yes | hybrid | Preview | |
|
744 |
+| `qwen3.5-plus` | 1M | unverified | unverified | yes (image+video) | yes | **on by default** | Snapshots `-2026-02-15`, `-2026-04-20` | |
|
745 |
+| `qwen3.5-flash` | 1M | unverified | unverified | yes | yes | hybrid | Snapshot `-2026-02-23` | |
|
746 |
+ |
|
747 |
+### Stable aliases (previous-gen commercial, still recommended fallbacks) |
|
748 |
+ |
|
749 |
+| API ID | Context | Thinking | Notes | |
|
750 |
+|---|---|---|---| |
|
751 |
+| `qwen-max` | 128K | off by default (`enable_thinking`) | Alias; `qwen3-max` + snapshots `-2025-09-23`, `-2026-01-23`, `qwen3-max-preview` also live | |
|
752 |
+| `qwen-plus` | 1M | off by default | Alias. Snapshots `-2025-01-25` … `-2025-12-01`, `qwen-plus-latest`, `qwen-plus-character` (role-play variant) | |
|
753 |
+| `qwen-turbo` | 1M | off by default | Legacy tier | |
|
754 |
+| `qwen-flash` | 1M | off by default | Highest rate quota (5M TPM). `qwen-flash-character` variant | |
|
755 |
+ |
|
756 |
+### Coder family |
|
757 |
+ |
|
758 |
+| API ID | Context | Thinking | Notes | |
|
759 |
+|---|---|---|---| |
|
760 |
+| `qwen3-coder-plus` | 1M | no | Snapshots `-2025-07-22`, `-2025-09-23`; alias `qwen-coder-plus` also live | |
|
761 |
+| `qwen3-coder-flash` | 1M | no | Cheap coding | |
|
762 |
+| `qwen3-coder-next` | unverified | no | Newest; "repository-level code understanding, multi-turn tool interactions" ([newly-released models](https://www.alibabacloud.com/help/en/model-studio/newly-released-models)) | |
|
763 |
+| `qwen3-coder-480b-a35b-instruct` | 256K (unverified) | no | Open-weights hosted | |
|
764 |
+ |
|
765 |
+### Vision-language chat models |
|
766 |
+ |
|
767 |
+Legacy `qwen-vl-max` / `qwen-vl-plus` are live but the docs mark them "Legacy … no longer recommended". Prefer qwen3-vl or the multimodal qwen3.5+/3.6+/3.7-plus/flash models. |
|
768 |
+ |
|
769 |
+| API ID | Context | Max output | Thinking | Notes | |
|
770 |
+|---|---|---|---|---| |
|
771 |
+| `qwen3-vl-plus` | 1M | 64K | hybrid | Video up to 1 h / 2 GB. Snapshots `-2025-09-23`, `-2025-12-19` | |
|
772 |
+| `qwen3-vl-flash` | 1M | 64K | hybrid | Snapshots `-2025-10-15`, `-2026-01-22` | |
|
773 |
+| `qwen3-vl-235b-a22b-instruct` | 128K (unverified) | — | no | Open-weights hosted | |
|
774 |
+| `qwen3-vl-235b-a22b-thinking` | 128K (unverified) | — | always | Open-weights hosted | |
|
775 |
+| `qvq-max` | 128K (unverified) | always (visual reasoning) | Streaming-only reasoning output | |
|
776 |
+| `qwen-vl-max`, `qwen-vl-plus` | 32K (unverified) | — | no | **Legacy** | |
|
777 |
+ |
|
778 |
+### Reasoning-only |
|
779 |
+ |
|
780 |
+| API ID | Context | Notes | |
|
781 |
+|---|---|---| |
|
782 |
+| `qwq-plus` | 128K | Always thinks — cannot be disabled; snapshot `-2025-03-05`. Streaming only (unverified for current build) | |
|
783 |
+ |
|
784 |
+### Open-weights Qwen hosted on DashScope |
|
785 |
+ |
|
786 |
+| API ID | Context | Thinking | |
|
787 |
+|---|---|---| |
|
788 |
+| `qwen3.5-397b-a17b` | unverified (likely 256K+) | **on by default** ([deep-thinking doc](https://www.alibabacloud.com/help/en/model-studio/deep-thinking): "Qwen3.5 open-source variants" think by default) | |
|
789 |
+| `qwen3.5-122b-a10b` | unverified | on by default | |
|
790 |
+| `qwen3.5-35b-a3b` | unverified | on by default | |
|
791 |
+| `qwen3.5-27b` | unverified | on by default | |
|
792 |
+| `qwen3.6-35b-a3b`, `qwen3.6-27b` | unverified | hybrid (unverified) | |
|
793 |
+| `qwen3-235b-a22b` | 128K | hybrid via `enable_thinking`; **streaming-only** (non-streaming calls error, per deep-thinking doc) | |
|
794 |
+| `qwen3-235b-a22b-instruct-2507` | 256K (unverified) | no | |
|
795 |
+| `qwen3-235b-a22b-thinking-2507` | 256K (unverified) | always | |
|
796 |
+| `qwen3-30b-a3b` (+ `-instruct-2507`, `-thinking-2507`) | 128K/256K (unverified) | as suffixed | |
|
797 |
+| `qwen3-next-80b-a3b-instruct` / `-thinking` | 256K (unverified) | as suffixed | |
|
798 |
+| `qwen3-32b`, `qwen3-14b`, `qwen3-8b` | 128K (unverified) | hybrid | |
|
799 |
+| `qwen2-7b-instruct` | 32K (unverified) | no — legacy | |
|
800 |
+ |
|
801 |
+### Third-party models hosted on DashScope (all in live probe) |
|
802 |
+ |
|
803 |
+| API ID | Context | Thinking | Notes | |
|
804 |
+|---|---|---|---| |
|
805 |
+| `deepseek-v4-pro` | 1M | yes (`enable_thinking`, off by default) | "Large-scale MoE, strong general reasoning" | |
|
806 |
+| `deepseek-v4-flash` | 1M | yes (off by default) | | |
|
807 |
+| `deepseek-v3.2` | 128K (unverified) | yes (off by default) | | |
|
808 |
+| `glm-5.2` | 198K | **on by default** (GLM series) | | |
|
809 |
+| `glm-5.2-fast-preview` | 198K (unverified) | on by default | | |
|
810 |
+| `glm-5.1` | 200K, max output 128K | on by default | | |
|
811 |
+| `kimi-k2.7-code` | 256K (unverified here; 262,144 on Moonshot direct) | yes | Same model Moonshot serves | |
|
812 |
+| `ccai-pro` | unverified | unverified | In live probe; not found in public docs — **unverified/unknown**, exclude from catalog until identified | |
|
813 |
+ |
|
814 |
+## 4. Request / Response Format |
|
815 |
+ |
|
816 |
+Fully **OpenAI-compatible** on `/compatible-mode/v1`. Minimal request: |
|
817 |
+ |
|
818 |
+```json |
|
819 |
+POST https://dashscope-intl.aliyuncs.com/compatible-mode/v1/chat/completions |
|
820 |
+Authorization: Bearer sk-... |
|
821 |
+Content-Type: application/json |
|
822 |
+ |
|
823 |
+{ |
|
824 |
+ "model": "qwen3.7-plus", |
|
825 |
+ "messages": [{ "role": "user", "content": "Hello" }] |
|
826 |
+} |
|
827 |
+``` |
|
828 |
+ |
|
829 |
+Response is the standard OpenAI `chat.completion` object (`choices[0].message.content`, `usage.prompt_tokens/completion_tokens/total_tokens`). Deviations: |
|
830 |
+ |
|
831 |
+- Thinking models add `message.reasoning_content` (and `delta.reasoning_content` when streaming) alongside `content` ([deep-thinking doc](https://www.alibabacloud.com/help/en/model-studio/deep-thinking)). |
|
832 |
+- Vision input uses standard OpenAI parts: `{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}` (https URLs also accepted). Video via `video_url` parts on video-capable models. |
|
833 |
+- Some open-weights thinking models are **streaming-only** — non-streaming calls return a 400. |
|
834 |
+ |
|
835 |
+## 5. Streaming (SSE) |
|
836 |
+ |
|
837 |
+Standard OpenAI SSE: `data: {chunk}` lines, terminated by `data: [DONE]`. Add `"stream_options": {"include_usage": true}` to get usage in the final chunk. |
|
838 |
+ |
|
839 |
+Thinking models stream in two phases: chunks first carry `delta.reasoning_content` (with `content` null), then `reasoning_content` goes null and `delta.content` carries the answer. Zyquo Cloud's collapsible "Thinking…" section maps directly to `reasoning_content` deltas. |
|
840 |
+ |
|
841 |
+`incremental_output` is a **native DashScope-API** parameter (must be enabled there for proper delta streaming); in OpenAI-compatible mode incremental deltas are the default and no such parameter is sent. Important compat-mode constraint (from the [error-code doc](https://www.alibabacloud.com/help/en/model-studio/error-code)): **`enable_thinking: true` requires `stream: true`** — a non-streaming call with thinking enabled returns a 400 parameter error. |
|
842 |
+ |
|
843 |
+## 6. Special Parameters |
|
844 |
+ |
|
845 |
+- `enable_thinking` (bool) — turn reasoning on/off for **hybrid** models. Defaults ([deep-thinking doc](https://www.alibabacloud.com/help/en/model-studio/deep-thinking)): |
|
846 |
+ - Default **on**: qwen3.7-max/plus series, qwen3.5 open-weights, GLM series. |
|
847 |
+ - Default **off**: qwen3 commercial (qwen-max/plus/flash/turbo, qwen3-max…), DeepSeek v3.2/v4. |
|
848 |
+ - Always on (cannot disable): `qwq-plus`, qvq, `*-thinking-*` models. |
|
849 |
+- `thinking_budget` (int) — cap reasoning tokens; "when the limit is reached, the model stops reasoning and responds immediately". |
|
850 |
+- `preserve_thinking` (bool, supported models) — feeds prior turns' `reasoning_content` back into context. |
|
851 |
+- Sampling bounds (from error docs): `temperature` ∈ `[0.0, 2.0)`, `top_p` ∈ `(0.0, 1.0]`. `presence_penalty`, `max_tokens`, `stop`, `tools`/`tool_choice`, `response_format: {"type":"json_object"}` supported (JSON mode not available on thinking output — unverified detail). |
|
852 |
+- Translation-specific `translation_options` applies only to `qwen-mt-*` models — **not applicable** to Zyquo Cloud (MT models excluded). |
|
853 |
+- `vl_high_resolution_images` (bool) for VL models — unverified in compat mode. |
|
854 |
+ |
|
855 |
+## 7. Rate Limits & Errors |
|
856 |
+ |
|
857 |
+Limits are **account-level** (all keys/workspaces aggregated), RPM + TPM, with RPS/TPS enforcement at RPM/60 ([rate-limit doc](https://www.alibabacloud.com/help/en/model-studio/rate-limit)). Singapore examples: `qwen3.7-max` 600 RPM / 1M TPM; `qwen-plus` 600 RPM / 1.5M TPM; `qwen-flash` 600 RPM / 5M TPM. Stable aliases get higher quotas than dated snapshots. Recovery "within one minute"; TPM can be temporarily raised in the console. |
|
858 |
+ |
|
859 |
+Errors are OpenAI-shaped JSON (`error.code/message/type`) with meaningful HTTP statuses ([error-code doc](https://www.alibabacloud.com/help/en/model-studio/error-code)): |
|
860 |
+- 401 `InvalidApiKey` — "Incorrect API key provided" (also fires on region mismatch) |
|
861 |
+- 429 — `RateQuota` (RPM), `AllocationQuota` (TPM), `BurstRate` ("Request rate increased too quickly") → retry with exponential backoff |
|
862 |
+- 400 — parameter errors (`enable_thinking` without stream; temperature/top_p out of range), `DataInspectionFailed` (content filter on input or output) |
|
863 |
+- 403/404 — model not activated in workspace / deprecated model ID |
|
864 |
+ |
|
865 |
+## 8. `/models` Listing |
|
866 |
+ |
|
867 |
+**Yes.** `GET /compatible-mode/v1/models` (Bearer auth) — verified live today, 151 entries. Shape is the OpenAI list format, minimal fields only: |
|
868 |
+ |
|
869 |
+```json |
|
870 |
+{ "object": "list", |
|
871 |
+ "data": [ { "id": "qwen3.7-flash", "object": "model", "created": 1785147332, "owned_by": "system" } ] } |
|
872 |
+``` |
|
873 |
+ |
|
874 |
+No capability/context metadata is returned — Zyquo Cloud must join IDs against the built-in catalog and can use the endpoint for "Refresh from API" diffing only. |
|
875 |
+ |
|
876 |
+--- |
|
877 |
+ |
|
878 |
+ |
|
879 |
+# DeepSeek — API Research (Phase 0) |
|
880 |
+ |
|
881 |
+Researched 2026-07-30 against official docs (api-docs.deepseek.com) and a **live `/models` probe with a real key** (`/tmp/zyquo-probe/deepseek.json`). Major change vs older integrations: **`deepseek-chat` and `deepseek-reasoner` are RETIRED** — only `deepseek-v4-flash` and `deepseek-v4-pro` exist now. |
|
882 |
+ |
|
883 |
+--- |
|
884 |
+ |
|
885 |
+## 1. Base URL & Endpoints |
|
886 |
+ |
|
887 |
+Base: `https://api.deepseek.com` (OpenAI-compatible). An Anthropic-Messages-compatible surface also exists at `https://api.deepseek.com/anthropic` (not needed — Zyquo Cloud uses the OpenAI surface via `OpenAICompatibleClient`). |
|
888 |
+ |
|
889 |
+| Endpoint | Method | Purpose | |
|
890 |
+|---|---|---| |
|
891 |
+| `/chat/completions` (also `/v1/chat/completions`) | POST | Chat (streaming + non-streaming) | |
|
892 |
+| `/models` | GET | List models | |
|
893 |
+| `/user/balance` | GET | Account balance (nice-to-have for the usage footer) | |
|
894 |
+ |
|
895 |
+Docs: https://api-docs.deepseek.com/ and https://api-docs.deepseek.com/api/create-chat-completion |
|
896 |
+ |
|
897 |
+## 2. Authentication |
|
898 |
+ |
|
899 |
+`Authorization: Bearer $DEEPSEEK_API_KEY`. Keys from platform.deepseek.com. |
|
900 |
+ |
|
901 |
+## 3. Chat Model Catalog |
|
902 |
+ |
|
903 |
+Live probe (2026-07-30) returns exactly two models. Specs from https://api-docs.deepseek.com/quick_start/pricing. Both support **streaming, vision — NO (text-only; unverified whether V4 added image input — docs do not document vision input; treat as text-only), tools/function calling (up to 128 functions), JSON mode (`json_object`), and thinking mode** (default ON). |
|
904 |
+ |
|
905 |
+| Model ID | Context | Max output | $/1M input (cache hit) | $/1M input (cache miss) | $/1M output | Thinking | Concurrency | |
|
906 |
+|---|---|---|---|---|---|---|---| |
|
907 |
+| `deepseek-v4-flash` | 1,000,000 | 384K | $0.0028 | $0.14 | $0.28 | ✅ default on (also non-thinking) | 2,500 | |
|
908 |
+| `deepseek-v4-pro` | 1,000,000 | 384K | $0.003625 | $0.435 | $0.87 | ✅ default on (also non-thinking) | 500 | |
|
909 |
+ |
|
910 |
+**Retired legacy IDs (do NOT ship in ModelCatalog):** `deepseek-chat` and `deepseek-reasoner` were routing aliases (chat = non-thinking mode, reasoner = thinking mode of the current generation, latterly resolving to `deepseek-v4-flash`). They stopped resolving on **2026-07-24 15:59 UTC** and now return errors (404-class). V4 launched in preview 2026-04-24, GA 2026-07-20. If migrating user configs: `deepseek-chat` → `deepseek-v4-flash` with `thinking.type: "disabled"`; `deepseek-reasoner` → `deepseek-v4-flash` with thinking enabled (note: reasoner mapped to **Flash**, not Pro). Sources: [DeepSeek V4 migration](https://ecorpit.com/deepseek-v4-migration-legacy-api-cutoff-july-2026/), [Developers Digest guide](https://www.developersdigest.tech/blog/deepseek-chat-to-v4-migration-guide), [TheRouter.ai](https://therouter.ai/news/deepseek-chat-reasoner-deprecation-v4-migration-routing/) (third-party corroboration of official release notes). |
|
911 |
+ |
|
912 |
+Max-output default vs maximum split is not documented beyond "384K max" — default `max_tokens` when omitted: unverified. |
|
913 |
+ |
|
914 |
+## 4. Request / Response Format |
|
915 |
+ |
|
916 |
+**OpenAI-compatible** with DeepSeek extensions/deviations (https://api-docs.deepseek.com/api/create-chat-completion): |
|
917 |
+ |
|
918 |
+- `model`: `"deepseek-v4-flash" | "deepseek-v4-pro"` |
|
919 |
+- `messages`: roles `system` / `user` / `assistant` / `tool`; optional `name` |
|
920 |
+- `temperature`: 0–2, **default 1**; `top_p`: 0–1, default 1 |
|
921 |
+- `max_tokens`: nullable int (input + output ≤ context) |
|
922 |
+- **`thinking`**: `{ "type": "enabled" | "disabled" }` — default **enabled**; plus **`reasoning_effort`**: `"high" | "max"` (defaults vary by request type; exact default unverified) |
|
923 |
+- `stream`, `stream_options: {"include_usage": true}` |
|
924 |
+- `response_format`: `{"type": "text" | "json_object"}` (no `json_schema` documented) |
|
925 |
+- `tools` (≤128 functions), `tool_choice`, `logprobs` + `top_logprobs` (0–20), `stop` (≤16 sequences) |
|
926 |
+- **Deprecated/removed:** `frequency_penalty` and `presence_penalty` — "no longer supported". Do not surface these sliders for DeepSeek in the per-conversation parameters UI. |
|
927 |
+ |
|
928 |
+```json |
|
929 |
+POST /chat/completions |
|
930 |
+{ |
|
931 |
+ "model": "deepseek-v4-flash", |
|
932 |
+ "messages": [ |
|
933 |
+ { "role": "system", "content": "You are a helpful assistant." }, |
|
934 |
+ { "role": "user", "content": "Reply with exactly: OK" } |
|
935 |
+ ], |
|
936 |
+ "thinking": { "type": "enabled" }, |
|
937 |
+ "reasoning_effort": "high", |
|
938 |
+ "max_tokens": 4096, |
|
939 |
+ "stream": false |
|
940 |
+} |
|
941 |
+``` |
|
942 |
+ |
|
943 |
+Response deviations from vanilla OpenAI: |
|
944 |
+- `choices[0].message.reasoning_content` (nullable string): "For thinking mode only. The reasoning contents of the assistant message, before the final answer." → this feeds Zyquo Cloud's collapsible "Thinking…" section. |
|
945 |
+- `finish_reason` values: `stop`, `length`, `content_filter`, `tool_calls`, **`insufficient_system_resource`** (DeepSeek-specific — surface as "DeepSeek servers overloaded"). |
|
946 |
+- `usage`: `prompt_tokens` (= `prompt_cache_hit_tokens` + `prompt_cache_miss_tokens`), `completion_tokens`, `total_tokens`, `prompt_cache_hit_tokens`, `prompt_cache_miss_tokens`, `completion_tokens_details.reasoning_tokens`. Cost calc must use hit/miss split (cache-hit input is ~50× cheaper). |
|
947 |
+ |
|
948 |
+**Vision:** not documented for V4 — treat DeepSeek as text-only (no `image_url` support; unverified whether it errors or ignores). |
|
949 |
+ |
|
950 |
+**Legacy V3-era rule** ("do not pass `reasoning_content` back in the next request or the API errors") is **unverified for V4** — the V4 docs no longer state it. Safe client behavior: never send `reasoning_content` back in `messages` (matches OpenAI-compat semantics either way). Whether `temperature`/`top_p` are ignored in thinking mode (as with the old reasoner): unverified for V4 — docs list them without model restrictions. |
|
951 |
+ |
|
952 |
+## 5. Streaming |
|
953 |
+ |
|
954 |
+Standard OpenAI SSE: `data: {chunk}` lines, terminated by `data: [DONE]`. DeepSeek specifics: |
|
955 |
+- Thinking mode streams `choices[0].delta.reasoning_content` first (while reasoning), then `choices[0].delta.content` for the final answer. The parser must watch both fields and switch the UI from the "Thinking…" section to the answer body when `content` starts. |
|
956 |
+- With `stream_options.include_usage: true`, one extra chunk with full `usage` arrives just before `[DONE]`. |
|
957 |
+- **Keep-alive under load** (https://api-docs.deepseek.com/quick_start/rate_limit): non-streaming responses may pad with empty lines; streaming sends SSE comments `: keep-alive`. The SSE parser must ignore comment lines and empty lines. If inference hasn't started after **10 minutes**, the server closes the connection — use long read timeouts. |
|
958 |
+ |
|
959 |
+## 6. Special Parameters & Notes |
|
960 |
+ |
|
961 |
+- **`thinking: {"type": "enabled"|"disabled"}`** — per-request toggle; default enabled on both models. Map Zyquo Cloud's "reasoning" toggle to this. |
|
962 |
+- **`reasoning_effort: "high" | "max"`** — extra thinking depth control (thinking mode). |
|
963 |
+- **Context caching is automatic** (no opt-in): repeated prefixes hit the cache and bill at the cache-hit input rate; observable via `prompt_cache_hit_tokens`/`prompt_cache_miss_tokens`. |
|
964 |
+- **`user_id`** (optional): scheduling isolation; on expanded-capacity accounts each distinct `user_id` gets its own concurrency budget. |
|
965 |
+- No `frequency_penalty`/`presence_penalty`; no `json_schema` response format; `n` not documented (unverified). |
|
966 |
+ |
|
967 |
+## 7. Rate Limits & Errors |
|
968 |
+ |
|
969 |
+(https://api-docs.deepseek.com/quick_start/rate_limit, https://api-docs.deepseek.com/quick_start/error_codes) DeepSeek uses **concurrency limits, not RPM/TPM**: 2,500 concurrent requests for v4-flash, 500 for v4-pro (a slot is held from send until the response completes). Exceeding → HTTP 429. Free capacity-expansion requests available. Under load, expect keep-alive padding (§5) rather than fast failures. |
|
970 |
+ |
|
971 |
+| HTTP | Meaning | Handling | |
|
972 |
+|---|---|---| |
|
973 |
+| 400 | Invalid request body format | fix request | |
|
974 |
+| 401 | Wrong API key | "Invalid API key for DeepSeek" | |
|
975 |
+| 402 | **Insufficient balance** (DeepSeek-specific) | tell user to top up | |
|
976 |
+| 422 | Invalid parameters | fix params per message | |
|
977 |
+| 429 | Concurrency limit | backoff + retry | |
|
978 |
+| 500 | Server error | retry with backoff | |
|
979 |
+| 503 | Server overloaded | retry after wait | |
|
980 |
+ |
|
981 |
+Error body: OpenAI-style `{"error": {"message", "type", "code"}}` — exact JSON shape not specified in official docs (unverified; confirm in Phase 7). |
|
982 |
+ |
|
983 |
+## 8. `/models` Listing |
|
984 |
+ |
|
985 |
+**Yes.** `GET /models` → minimal OpenAI shape (verified live): `{"object":"list","data":[{"id":"deepseek-v4-flash","object":"model","owned_by":"deepseek"},{"id":"deepseek-v4-pro","object":"model","owned_by":"deepseek"}]}`. IDs only — context/pricing/capabilities must come from the built-in catalog. |
|
986 |
+ |
|
987 |
+--- |
|
988 |
+ |
|
989 |
+ |
|
990 |
+# Kimi (Moonshot AI) — Provider Research |
|
991 |
+ |
|
992 |
+Research date: **2026-07-30**. Live `GET /v1/models` probe from `api.moonshot.ai` with a real key (12 models) is the authoritative ID list. Official docs now live at **platform.kimi.ai** (`platform.moonshot.ai` 301-redirects there): [Chat API](https://platform.kimi.ai/docs/api/chat.md), [Pricing](https://platform.kimi.ai/docs/pricing/chat), per-model pricing pages ([K3](https://platform.kimi.ai/docs/pricing/chat-k3.md), [K2.7 Code](https://platform.kimi.ai/docs/pricing/chat-k27-code.md), [K2.6](https://platform.kimi.ai/docs/pricing/chat-k26.md), [K2.5](https://platform.kimi.ai/docs/pricing/chat-k25.md), [V1](https://platform.kimi.ai/docs/pricing/chat-v1.md)), [Rate limits](https://platform.kimi.ai/docs/pricing/limits.md), [Errors](https://platform.kimi.ai/docs/api/errors.md), [Vision](https://platform.kimi.ai/docs/guide/use-kimi-vision-model.md), [Web search](https://platform.kimi.ai/docs/guide/use-web-search.md), [Partial mode](https://platform.kimi.ai/docs/guide/use-partial-mode-feature-of-kimi-api.md). |
|
993 |
+ |
|
994 |
+--- |
|
995 |
+ |
|
996 |
+## 1. Base URL & Endpoints |
|
997 |
+ |
|
998 |
+- **Base URL (international — Zyquo Cloud uses this):** `https://api.moonshot.ai/v1` |
|
999 |
+- Chat completions: `POST /v1/chat/completions` |
|
1000 |
+- Model listing: `GET /v1/models` (verified live today) |
|
1001 |
+- Also available: token estimation endpoint (for pre-computing image/video token cost), files API (`ms://<file-id>` video refs), context caching, batch. |
|
1002 |
+- **China variant:** `https://api.moonshot.cn/v1` — separate platform and keys. Keys are **not cross-platform**: an `.ai` key fails on `.cn` and vice versa (errors doc). Zyquo Cloud uses `.ai` only. |
|
1003 |
+ |
|
1004 |
+## 2. Authentication |
|
1005 |
+ |
|
1006 |
+Standard Bearer (confirmed): |
|
1007 |
+ |
|
1008 |
+``` |
|
1009 |
+Authorization: Bearer sk-... |
|
1010 |
+``` |
|
1011 |
+ |
|
1012 |
+## 3. Chat Model Catalog |
|
1013 |
+ |
|
1014 |
+All 12 models from the live probe are chat models (nothing to filter). Capability flags below come straight from the probe (`supports_image_in`, `supports_video_in`, `supports_reasoning`, `context_length`); pricing from the official per-model pricing pages (USD, per 1M tokens; input is billed by cache hit vs miss — automatic context caching is built in). |
|
1015 |
+ |
|
1016 |
+| API ID | Context | Vision | Video | Reasoning | In (cache hit / miss) | Out | Notes | |
|
1017 |
+|---|---|---|---|---|---|---|---| |
|
1018 |
+| `kimi-k3` | 1,048,576 | yes | yes | **always on** (`reasoning_effort`) | $0.30 / $3.00 | $15.00 | Flagship. Default max output 131,072, settable up to 1,048,576. Tools, JSON mode, structured output, partial mode, web search. **Recommended** | |
|
1019 |
+| `kimi-k2.7-code` | 262,144 | yes | yes | always on (`thinking: {"type":"enabled","keep":"all"}`) | $0.19 / $0.95 | $4.00 | Dedicated coding/agent model. **Recommended for code** | |
|
1020 |
+| `kimi-k2.7-code-highspeed` | 262,144 | yes | yes | always on | $0.38 / $1.90 | $8.00 | Same model, ~180 tok/s (up to 260 tok/s short-context) | |
|
1021 |
+| `kimi-k2.6` | 262,144 | yes | yes | hybrid (`thinking: {"type":"enabled"\|"disabled","keep":null\|"all"}`) | $0.16 / $0.95 | $4.00 | Tools, JSON mode, partial mode, web search | |
|
1022 |
+| `kimi-k2.5` | 262,144 | yes | yes | hybrid (`thinking: {"type":"enabled"\|"disabled"}`) | $0.10 / $0.60 | $3.00 | Cheapest K-series | |
|
1023 |
+| `moonshot-v1-8k` | 8,192 | no | no | no | $0.20 | $2.00 | Legacy "classic" series | |
|
1024 |
+| `moonshot-v1-32k` | 32,768 | no | no | no | $1.00 | $3.00 | Legacy | |
|
1025 |
+| `moonshot-v1-128k` | 131,072 | no | no | no | $2.00 | $5.00 | Legacy | |
|
1026 |
+| `moonshot-v1-auto` | 131,072 | no | no | no | billed as selected size (unverified) | | Auto-routes to 8k/32k/128k by context length | |
|
1027 |
+| `moonshot-v1-8k-vision-preview` | 8,192 | yes | no | no | $0.20 | $2.00 | Legacy vision | |
|
1028 |
+| `moonshot-v1-32k-vision-preview` | 32,768 | yes | no | no | $1.00 | $3.00 | Legacy vision | |
|
1029 |
+| `moonshot-v1-128k-vision-preview` | 131,072 | yes | no | no | $2.00 | $5.00 | Legacy vision | |
|
1030 |
+ |
|
1031 |
+No dated snapshots exist in the live catalog. Max output for K2.x/moonshot-v1 not documented separately from the context window (unverified). moonshot-v1 cache-hit pricing not published (single input price). |
|
1032 |
+ |
|
1033 |
+## 4. Request / Response Format |
|
1034 |
+ |
|
1035 |
+Fully **OpenAI-compatible**. Minimal request: |
|
1036 |
+ |
|
1037 |
+```json |
|
1038 |
+POST https://api.moonshot.ai/v1/chat/completions |
|
1039 |
+Authorization: Bearer sk-... |
|
1040 |
+Content-Type: application/json |
|
1041 |
+ |
|
1042 |
+{ |
|
1043 |
+ "model": "kimi-k3", |
|
1044 |
+ "messages": [{ "role": "user", "content": "Hello" }] |
|
1045 |
+} |
|
1046 |
+``` |
|
1047 |
+ |
|
1048 |
+Response: standard `chat.completion`. Deviations / additions (from [chat API doc](https://platform.kimi.ai/docs/api/chat.md)): |
|
1049 |
+ |
|
1050 |
+- `choices[0].message.reasoning_content` — "returned only when thinking mode is enabled". `finish_reason`: `stop | length | tool_calls`. |
|
1051 |
+- `usage` includes `cached_tokens` (automatic context caching). |
|
1052 |
+- Vision content parts: `{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}` — content must be a real JSON array, "do not serialize the JSON array … as a string". Video via `{"type":"video_url","video_url":{"url":"ms://<file-id>"}}` (uploaded to Moonshot storage first). Recommended limits: images ≤ 4096×2160, video ≤ 1920×1080, request body ≤ 100 MB. |
|
1053 |
+- Prefer `max_completion_tokens` over `max_tokens` (K3 default 131,072, max 1,048,576). |
|
1054 |
+- `response_format`: `{"type":"text"}` (default), `{"type":"json_object"}`, `{"type":"json_schema","json_schema":{...}}` (structured output). |
|
1055 |
+ |
|
1056 |
+## 5. Streaming (SSE) |
|
1057 |
+ |
|
1058 |
+`"stream": true` → standard SSE: each event line is `data: {json}`; terminates with `data: [DONE]`. `stream_options: {"include_usage": true}` puts complete usage in the final chunk. Thinking models stream `delta.reasoning_content` before `delta.content` — map to Zyquo Cloud's collapsible thinking section. Tool-call deltas follow the OpenAI incremental `tool_calls` convention. |
|
1059 |
+ |
|
1060 |
+## 6. Special Parameters |
|
1061 |
+ |
|
1062 |
+- **Sampling params are documented for moonshot-v1 only** (chat API doc): `temperature` 0–1 (default 0 — Moonshot's historic cap of 1 still holds), `top_p` 0–1 (default 1), `n` 1–5, `presence_penalty`/`frequency_penalty` −2.0–2.0. For K-series models the docs expose reasoning controls instead — Zyquo Cloud should hide/limit sampling controls per model accordingly. |
|
1063 |
+- **Reasoning controls:** |
|
1064 |
+ - `kimi-k3`: `reasoning_effort`: `"low" | "high" | "max"` (default `"max"`); "Kimi K3 always enables thinking with Preserved Thinking". |
|
1065 |
+ - `kimi-k2.6`: `thinking: {"type":"enabled"|"disabled","keep":null|"all"}`; `kimi-k2.5`: `thinking: {"type":"enabled"|"disabled"}`; `kimi-k2.7-code`: always `{"type":"enabled","keep":"all"}`. |
|
1066 |
+- **Partial mode** (assistant prefill): append `{"role":"assistant","content":"Dear customer, hello,","partial":true}` as the last message; the model continues from the prefix (client must concatenate). Use cases: forced openings, resuming `finish_reason=length` truncations (include prior `reasoning_content` when resuming thinking output), role-play consistency via `name` field. |
|
1067 |
+- **Built-in web search tool:** declare `{"type":"builtin_function","function":{"name":"$web_search"}}` (the `$` prefix marks Kimi built-ins). Model emits a tool call; search runs server-side; echo the tool result back as a `role: tool` message. Charged a per-call fee on top of tokens; search-result tokens count toward `prompt_tokens`. Supported on `kimi-k3` and `kimi-k2.6` (with reasoning enabled). **Caution (2026-07-30):** K3 pricing page says `web_search` "is currently being updated" and recommends against using it right now. |
|
1068 |
+- `prompt_cache_key` (cache-hit optimization), `safety_identifier` (hashed end-user ID), `tool_choice` (`auto|none|required|{function}`), `stop` (≤5 strings, ≤32 bytes each). |
|
1069 |
+ |
|
1070 |
+## 7. Rate Limits & Errors |
|
1071 |
+ |
|
1072 |
+Tiers by **cumulative recharge** ([limits doc](https://platform.kimi.ai/docs/pricing/limits.md)); minimum $1 to start: |
|
1073 |
+ |
|
1074 |
+| Tier | Recharge | Concurrency | RPM | TPM | TPD | |
|
1075 |
+|---|---|---|---|---|---| |
|
1076 |
+| 0 | $1 | 1 | 3 | 500K | 1.5M | |
|
1077 |
+| 1 | $10 | 50 | 200 | 2M | ∞ | |
|
1078 |
+| 2 | $20 | 100 | 500 | 3M | ∞ | |
|
1079 |
+| 3 | $100 | 200 | 5,000 | 3M | ∞ | |
|
1080 |
+| 4 | $1,000 | 400 | 5,000 | 4M | ∞ | |
|
1081 |
+| 5 | $3,000 | 1,000 | 10,000 | 5M | ∞ | |
|
1082 |
+ |
|
1083 |
+Higher limits: api-service@moonshot.ai. No per-model limits. |
|
1084 |
+ |
|
1085 |
+Error body ([errors doc](https://platform.kimi.ai/docs/api/errors.md)): |
|
1086 |
+ |
|
1087 |
+```json |
|
1088 |
+{ "error": { "type": "error_type_string", "message": "Human-readable description" } } |
|
1089 |
+``` |
|
1090 |
+ |
|
1091 |
+Key types: 401 `invalid_authentication_error` / `incorrect_api_key_error`; 400 `invalid_request_error` (incl. "Input token length too long") and `content_filter`; **429 covers two distinct cases** — `exceeded_current_quota_error` ("Account balance is insufficient" — do NOT retry, tell the user to top up) vs `rate_limit_reached_error` (back off and retry); 500/503/504 `server_error` / `server_unavailable` (retry with backoff). Zyquo Cloud must branch on `error.type`, not just HTTP status. |
|
1092 |
+ |
|
1093 |
+## 8. `/models` Listing |
|
1094 |
+ |
|
1095 |
+**Yes.** `GET https://api.moonshot.ai/v1/models` — OpenAI list shape **plus rich extension fields**, verified live: |
|
1096 |
+ |
|
1097 |
+```json |
|
1098 |
+{ "object": "list", "data": [ { |
|
1099 |
+ "id": "kimi-k3", "object": "model", "created": 1785311147, "owned_by": "moonshot", |
|
1100 |
+ "permission": [ { "...": "..." } ], |
|
1101 |
+ "supports_image_in": true, "supports_video_in": true, |
|
1102 |
+ "supports_reasoning": true, "context_length": 1048576 } ] } |
|
1103 |
+``` |
|
1104 |
+ |
|
1105 |
+The extras (`supports_image_in`, `supports_video_in`, `supports_reasoning`, `context_length`) let Zyquo Cloud derive capabilities dynamically at refresh time — the only one of the three providers in this batch that exposes capability metadata. |
|
1106 |
+ |
|
1107 |
+--- |
|
1108 |
+ |
|
1109 |
+ |
|
1110 |
+# Perplexity — Provider Research |
|
1111 |
+ |
|
1112 |
+Research date: **2026-07-30**. Perplexity has **no `/models` listing for the Sonar chat API** (confirmed 404 with a real key today) — this catalog comes from the official docs: [Models](https://docs.perplexity.ai/docs/sonar/models.md), [Chat Completions reference](https://docs.perplexity.ai/api-reference/sonar-post.md), [Pricing](https://docs.perplexity.ai/docs/getting-started/pricing.md), [Rate limits](https://docs.perplexity.ai/docs/admin/rate-limits-usage-tiers.md), [Stream mode](https://docs.perplexity.ai/docs/sonar/pro-search/stream-mode.md), [Agent API migration](https://docs.perplexity.ai/docs/agent-api/migrate-from-sonar/overview.md). |
|
1113 |
+ |
|
1114 |
+> **Strategic note (July 2026):** Perplexity is steering users from "Sonar Chat Completions" to a new **Agent API** (`POST https://api.perplexity.ai/v1/agent`, presets `fast/low/medium/high/xhigh`, third-party models incl. GPT/Claude/Gemini/Grok/Kimi). The migration page says Sonar Chat Completions "remains supported" with **no announced sunset date**, but the rate-limits page already labels it "Sonar API (Deprecated)". Zyquo Cloud ships on the OpenAI-compatible `/chat/completions` Sonar API (fits our shared client), and should keep the Agent API on the roadmap. |
|
1115 |
+ |
|
1116 |
+--- |
|
1117 |
+ |
|
1118 |
+## 1. Base URL & Endpoints |
|
1119 |
+ |
|
1120 |
+- **Base URL:** `https://api.perplexity.ai` |
|
1121 |
+- Chat completions (what Zyquo Cloud uses): `POST https://api.perplexity.ai/chat/completions` |
|
1122 |
+- No model-listing endpoint for the chat API (see §8). |
|
1123 |
+- Other product APIs, not used: Agent API `POST /v1/agent` (and `GET /v1/models` — Agent API models only), Search API, Embeddings API. |
|
1124 |
+ |
|
1125 |
+## 2. Authentication |
|
1126 |
+ |
|
1127 |
+Standard Bearer (confirmed live): |
|
1128 |
+ |
|
1129 |
+``` |
|
1130 |
+Authorization: Bearer pplx-... |
|
1131 |
+``` |
|
1132 |
+ |
|
1133 |
+## 3. Chat Model Catalog |
|
1134 |
+ |
|
1135 |
+The [API reference](https://docs.perplexity.ai/api-reference/sonar-post.md) enumerates exactly **four** valid `model` values. All are search-grounded chat models with streaming; none accept image input (`return_images` returns image *search results*, it is not vision input). **`sonar-reasoning` (non-Pro) is gone from the current docs enum — treat as removed/deprecated.** |
|
1136 |
+ |
|
1137 |
+Pricing is official ([pricing doc](https://docs.perplexity.ai/docs/getting-started/pricing.md)); note the **per-request fee scaled by `search_context_size`** on top of token fees — Zyquo Cloud's cost estimator must include it. Context windows are not stated in the current official docs; figures below are from trackers ([OpenRouter](https://openrouter.ai/perplexity)) — **unverified**. |
|
1138 |
+ |
|
1139 |
+| API ID | Context (unverified) | Max output | $/1M in | $/1M out | Request fee /1K (low/med/high ctx) | Capabilities | |
|
1140 |
+|---|---|---|---|---|---|---| |
|
1141 |
+| `sonar` | 128K | ≤128,000 (API cap) | $1 | $1 | $5 / $8 / $12 | Fast grounded Q&A; streaming, JSON schema output, citations | |
|
1142 |
+| `sonar-pro` | 200K | 8,000 (unverified) | $3 | $15 | $6 / $10 / $14 | "Complex queries and follow-ups"; streaming, citations | |
|
1143 |
+| `sonar-reasoning-pro` | 128K | — | $2 | $8 | $6 / $10 / $14 | Chain-of-thought (`<think>` block in content); citations | |
|
1144 |
+| `sonar-deep-research` | 128K | — | $2 | $8 | Search queries $5/1K + citation tokens $2/1M + reasoning tokens $3/1M | Long-running exhaustive research reports; `reasoning_effort` applies | |
|
1145 |
+ |
|
1146 |
+## 4. Request / Response Format |
|
1147 |
+ |
|
1148 |
+**OpenAI-compatible request shape with Perplexity extensions.** Minimal request: |
|
1149 |
+ |
|
1150 |
+```json |
|
1151 |
+POST https://api.perplexity.ai/chat/completions |
|
1152 |
+Authorization: Bearer pplx-... |
|
1153 |
+Content-Type: application/json |
|
1154 |
+ |
|
1155 |
+{ |
|
1156 |
+ "model": "sonar", |
|
1157 |
+ "messages": [{ "role": "user", "content": "How many stars are in the Milky Way?" }] |
|
1158 |
+} |
|
1159 |
+``` |
|
1160 |
+ |
|
1161 |
+Response = standard `chat.completion` plus Perplexity-specific top-level fields (all from the [API reference](https://docs.perplexity.ai/api-reference/sonar-post.md)): |
|
1162 |
+ |
|
1163 |
+- `citations`: array of source URLs ("URLs of sources used to generate the response") — render as Zyquo Cloud's numbered chips. |
|
1164 |
+- `search_results`: array of result objects (title/url/date) used for context — richer than `citations`; prefer it for chip tooltips. |
|
1165 |
+- `images` (when `return_images: true`), `related_questions` (when `return_related_questions: true`). |
|
1166 |
+- `usage` is extended: `prompt_tokens`, `completion_tokens`, `total_tokens`, `search_context_size`, `citation_tokens`, `num_search_queries`, `reasoning_tokens`, and a **`cost` object** (input/output/reasoning/request/citation/search-queries costs + `total_cost`, USD) — Zyquo Cloud can display exact billed cost instead of estimating. |
|
1167 |
+- Reasoning models emit CoT inside `<think>…</think>` in `content` (parse into the collapsible thinking section); `sonar-deep-research` reports `reasoning_tokens` in usage. |
|
1168 |
+ |
|
1169 |
+Deviations from OpenAI: no `n`, no `presence_penalty`/`frequency_penalty`/`top_k` in the current schema (historical `top_k`/penalties have been dropped — send none); no vision/image input; no tool/function calling on the Sonar chat API. |
|
1170 |
+ |
|
1171 |
+## 5. Streaming (SSE) |
|
1172 |
+ |
|
1173 |
+`"stream": true` → SSE `data:` lines. Two formats via `stream_mode` ([stream-mode doc](https://docs.perplexity.ai/docs/sonar/pro-search/stream-mode.md)): |
|
1174 |
+ |
|
1175 |
+- `full` (default): every chunk is `chat.completion.chunk` and each chunk carries the accumulated `message` server-side — **search results/citations appear repeatedly during the stream**; higher bandwidth. OpenAI-style `delta` parsing still works. |
|
1176 |
+- `concise`: delta-only, with distinct chunk `object` types: `chat.reasoning` (live reasoning steps) → `chat.reasoning.done` ("includes all search results — web, images, videos") → `chat.completion.chunk` (answer deltas) → `chat.completion.done` (final metadata; "cost information is only available in the `chat.completion.done` chunk"). |
|
1177 |
+ |
|
1178 |
+Practical rule for Zyquo Cloud's shared SSE parser: with default `full` mode, take `citations`/`search_results`/`usage` from the **final** chunk. `sonar-deep-research` streams slowly (long research phase) — needs the long read-timeout path. |
|
1179 |
+ |
|
1180 |
+## 6. Special Parameters (Perplexity extensions) |
|
1181 |
+ |
|
1182 |
+From the [API reference](https://docs.perplexity.ai/api-reference/sonar-post.md): |
|
1183 |
+ |
|
1184 |
+- Generation: `max_tokens` (0 < n ≤ 128,000), `temperature` (0–2), `top_p` (0–1), `stop`, `stream`, `stream_mode` (`full|concise`), `reasoning_effort` (`minimal|low|medium|high` — for reasoning/deep-research models), `response_format` (`text` or `json_schema` structured output), `language_preference` (ISO 639-1). |
|
1185 |
+- Search behavior: `disable_search` (bool), `enable_search_classifier` (bool — model decides if search is needed), `search_mode` (`web|academic|sec`), `web_search_options` `{ search_context_size: low|medium|high, search_type: fast|pro|auto, user_location, image_results_enhanced_relevance }`. |
|
1186 |
+- Search filters: `search_domain_filter` (array; `-domain` excludes), `search_language_filter` (ISO 639-1 array), `search_recency_filter` (`hour|day|week|month|year`), `search_after_date_filter` / `search_before_date_filter` and `last_updated_after_filter` / `last_updated_before_filter` (all `MM/DD/YYYY`), `image_domain_filter`, `image_format_filter`. |
|
1187 |
+- Response extras: `return_images` (bool), `return_related_questions` (bool). |
|
1188 |
+ |
|
1189 |
+UI mapping: expose `search_mode`, recency filter, domain filter, `search_context_size`, `return_images`, `return_related_questions`, and `reasoning_effort` in Zyquo Cloud's per-conversation parameter sheet for Perplexity only. |
|
1190 |
+ |
|
1191 |
+## 7. Rate Limits & Errors |
|
1192 |
+ |
|
1193 |
+[Rate limits doc](https://docs.perplexity.ai/docs/admin/rate-limits-usage-tiers.md): six tiers by **cumulative lifetime spend** — Tier 0 $0, T1 $50+, T2 $250+, T3 $500+, T4 $1,000+, T5 $5,000+. Sonar API tiered RPM historically ranged "from 5 RPM (deep-research at Tier 0) to 100 RPM (Tier 5)" (page now brands the Sonar table "Deprecated" — exact current per-model RPM: unverified). On breach: HTTP **429 "Too Many Requests"**, with continuous token-bucket refill (retry after short backoff). |
|
1194 |
+ |
|
1195 |
+Error format: OpenAI-style JSON error body with HTTP statuses 400 (invalid model/params), 401 (bad key), 429 (rate limit). Exact error JSON schema is not published in the docs — **unverified**; handle by HTTP status + message string defensively. |
|
1196 |
+ |
|
1197 |
+## 8. `/models` Listing |
|
1198 |
+ |
|
1199 |
+**No — for the chat API.** `GET https://api.perplexity.ai/models` returns **404** (confirmed live today with a real key). The four Sonar model IDs must ship hard-coded in `ModelCatalog` from `docs/PROVIDERS.md`. |
|
1200 |
+ |
|
1201 |
+Caveat: the docs do describe `GET https://api.perplexity.ai/v1/models` ([reference](https://docs.perplexity.ai/api-reference/models-get.md)), but it "returns model identifiers that can be used with the `POST /v1/agent` endpoint" — i.e. **Agent API** models (claude-opus-5, gpt-5.6-sol, gemini-3.6-flash, grok-4.5, kimi-k3, glm-5.2, sonar, …), not Sonar chat-completions models. Do not use it to populate the Perplexity chat catalog. |
|
1202 |
+ |
|
1203 |
+## Vision Input |
|
1204 |
+ |
|
1205 |
+Not supported on the Sonar chat models — no `image_url` content parts. (`return_images` only returns image *search results*.) Zyquo Cloud must mark all Perplexity models as non-vision. |
|
1206 |
+ |
|
1207 |
+--- |
|
1208 |
+ |
|
1209 |
+ |
|
1210 |
+# Together AI — Provider Research (Phase 0) |
|
1211 |
+ |
|
1212 |
+Researched 2026-07-30 against official docs (https://docs.together.ai) and a live authenticated |
|
1213 |
+`GET /v1/models` probe (`/tmp/zyquo-probe/together.json`, 271 models returned: 164 `chat` + |
|
1214 |
+14 `language` + image/video/audio/embedding/rerank/etc.). The probe is authoritative for model |
|
1215 |
+existence, context lengths, and pricing (embedded `pricing` object, USD per 1M tokens). |
|
1216 |
+ |
|
1217 |
+--- |
|
1218 |
+ |
|
1219 |
+## 1. Base URL & Endpoints |
|
1220 |
+ |
|
1221 |
+- **Base URL:** `https://api.together.xyz/v1` (docs also present `https://api.together.ai/v1` — both resolve to the same API; Zyquo Cloud uses `https://api.together.xyz/v1`) |
|
1222 |
+- `POST /v1/chat/completions` — chat (the endpoint Zyquo Cloud uses) |
|
1223 |
+- `POST /v1/completions` — legacy text completion |
|
1224 |
+- `GET /v1/models` — model listing |
|
1225 |
+- `POST /v1/embeddings`, `/v1/images/generations`, `/v1/audio/*`, `/v1/rerank` — other modalities (out of scope) |
|
1226 |
+- Docs also mention an optimized inference host `https://api-inference.together.ai/v2` (not needed; standard v1 is the documented OpenAI-compatible surface) |
|
1227 |
+ |
|
1228 |
+Reference: https://docs.together.ai/reference/chat-completions-1 |
|
1229 |
+ |
|
1230 |
+## 2. Authentication |
|
1231 |
+ |
|
1232 |
+`Authorization: Bearer <TOGETHER_API_KEY>` — standard Bearer header. Confirmed by docs and by the live probe. |
|
1233 |
+ |
|
1234 |
+## 3. Chat Model Catalog (curated serverless chat models) |
|
1235 |
+ |
|
1236 |
+The live `/models` response contains 178 chat/language entries, many of which are dedicated-endpoint-only |
|
1237 |
+or free-tier/lora artifacts (pricing `0/0`). The curated list below covers the main **serverless** chat |
|
1238 |
+models (cross-checked against https://docs.together.ai/docs/serverless-models). Pricing = USD / 1M tokens |
|
1239 |
+from the live probe. Context = `context_length` from the probe. |
|
1240 |
+ |
|
1241 |
+Capabilities legend: **V** vision, **T** tools/function calling, **R** reasoning/thinking output, **J** JSON mode / structured output. All models stream. Capability flags marked * are inferred from model family docs, not individually verified on Together. |
|
1242 |
+ |
|
1243 |
+| Model ID (exact) | Context | $ In | $ Out | $ Cached | Caps | |
|
1244 |
+|---|---|---|---|---|---| |
|
1245 |
+| `moonshotai/Kimi-K3` | 1,000,000 | 3.00 | 15.00 | 0.30 | T J R | |
|
1246 |
+| `moonshotai/Kimi-K2.7-Code` | 262,144 | 0.95 | 4.00 | 0.19 | T J R* | |
|
1247 |
+| `moonshotai/Kimi-K2.6` | 262,144 | 1.20 | 4.50 | 0.20 | T J R* | |
|
1248 |
+| `moonshotai/Kimi-K2.5-fp4` | 262,144 | 0.50 | 2.80 | — | T* J* | |
|
1249 |
+| `deepseek-ai/DeepSeek-V4-Pro` | 512,000 | 1.74 | 3.48 | 0.20 | T J R* | |
|
1250 |
+| `deepseek-ai/DeepSeek-V3.1` | 131,072 | 0.60 | 1.70 | — | T* J* R (hybrid) | |
|
1251 |
+| `deepseek-ai/DeepSeek-R1-0528` | 163,840 | 3.00 | 7.00 | — | R | |
|
1252 |
+| `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | 131,072 | 2.00 | 2.00 | — | R | |
|
1253 |
+| `zai-org/GLM-5.2` | 512,000 | 1.40 | 4.40 | 0.26 | T J R* | |
|
1254 |
+| `zai-org/GLM-5.1` | 202,752 | 1.40 | 4.40 | 0.26 | T* J* R* | |
|
1255 |
+| `zai-org/GLM-5` | 202,752 | 1.00 | 3.20 | — | T* J* R* | |
|
1256 |
+| `zai-org/GLM-4.7` | 202,752 | 0.45 | 2.00 | — | T* J* R* | |
|
1257 |
+| `zai-org/GLM-4.6` | 202,752 | 0.60 | 2.20 | — | T* J* R* | |
|
1258 |
+| `zai-org/GLM-4.5-Air-FP8` | 131,072 | 0.20 | 1.10 | — | T* J* R* | |
|
1259 |
+| `Qwen/Qwen3.7-Max` | 1,000,000 | 1.25 | 3.75 | 0.125 | T* J* R* (unverified caps) | |
|
1260 |
+| `Qwen/Qwen3.7-Plus` | 1,000,000 | 0.32 | 1.28 | — | T* J* (unverified caps) | |
|
1261 |
+| `Qwen/Qwen3.6-Plus` | 1,000,000 | 0.50 | 3.00 | — | T* J* (unverified caps) | |
|
1262 |
+| `Qwen/Qwen3.5-397B-A17B` | 262,144 | 0.60 | 3.60 | 0.35 | T* J* R* | |
|
1263 |
+| `Qwen/Qwen3.5-9B` | 262,144 | 0.17 | 0.25 | — | T J | |
|
1264 |
+| `Qwen/Qwen3-Next-80B-A3B-Instruct` | 262,144 | 0.15 | 1.50 | — | T* J* | |
|
1265 |
+| `Qwen/Qwen3-Next-80B-A3B-Thinking` | 262,144 | 0.15 | 1.50 | — | R T* J* | |
|
1266 |
+| `Qwen/Qwen3-Coder-Next-FP8` | 262,144 | 0.50 | 1.20 | — | T* J* | |
|
1267 |
+| `Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8` | 262,144 | 2.00 | 2.00 | — | T* J* | |
|
1268 |
+| `Qwen/Qwen3-VL-32B-Instruct` | 262,144 | 0.50 | 1.50 | — | V T* J* | |
|
1269 |
+| `Qwen/Qwen3-VL-8B-Instruct` | 262,144 | 0.18 | 0.68 | — | V T* J* | |
|
1270 |
+| `Qwen/QwQ-32B` | 131,072 | 1.20 | 1.20 | — | R | |
|
1271 |
+| `Qwen/Qwen2.5-VL-72B-Instruct` | 32,768 | 1.95 | 8.00 | — | V | |
|
1272 |
+| `Qwen/Qwen2.5-72B-Instruct-Turbo` | 131,072 | 1.20 | 1.20 | — | T* J* | |
|
1273 |
+| `Qwen/Qwen2.5-7B-Instruct-Turbo` | 32,768 | 0.30 | 0.30 | — | T J | |
|
1274 |
+| `Qwen/Qwen2.5-Coder-32B-Instruct` | 16,384 | 0.80 | 0.80 | — | — | |
|
1275 |
+| `meta-llama/Llama-4-Scout-17B-16E-Instruct` | 1,048,576 | 0.18 | 0.59 | — | V T* J* | |
|
1276 |
+| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | 131,072 | 1.04 | 1.04 | — | T J | |
|
1277 |
+| `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` | 131,072 | 0.88 | 0.88 | — | T* J* | |
|
1278 |
+| `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | 131,072 | 0.18 | 0.18 | — | T* J* | |
|
1279 |
+| `meta-llama/Llama-3.1-405B-Instruct` | 4,096 (probe value — likely a listing quirk; flag in Phase 7) | 3.50 | 3.50 | — | T* | |
|
1280 |
+| `meta-llama/Llama-3.2-3B-Instruct` | 131,072 | 0.06 | 0.06 | — | — | |
|
1281 |
+| `openai/gpt-oss-120b` | 131,072 | 0.15 | 0.60 | — | R T J (`reasoning_effort`) | |
|
1282 |
+| `openai/gpt-oss-20b` | 131,072 | 0.05 | 0.20 | — | R T J (`reasoning_effort`) | |
|
1283 |
+| `nvidia/nemotron-3-ultra-550b-a55b` | 512,288 | 0.60 | 3.60 | 0.20 | T J R* | |
|
1284 |
+| `nvidia/NVIDIA-Nemotron-Nano-9B-v2` | 131,072 | 0.06 | 0.25 | — | R* | |
|
1285 |
+| `MiniMaxAI/MiniMax-M3` | 524,288 | 0.30 | 1.20 | 0.06 | T J R* | |
|
1286 |
+| `MiniMaxAI/MiniMax-M2.7` | 196,608 | 0.30 | 1.20 | 0.06 | T* J* R* | |
|
1287 |
+| `mistralai/Mistral-Small-24B-Instruct-2501` | 32,768 | 0.10 | 0.30 | — | T* J* | |
|
1288 |
+| `mistralai/Ministral-3-14B-Instruct-2512` | 262,144 | 0.20 | 0.20 | — | T* J* | |
|
1289 |
+| `mistralai/Mixtral-8x7B-Instruct-v0.1` | 32,768 | 0.60 | 0.60 | — | J* | |
|
1290 |
+| `google/gemma-4-31B-it` | 262,144 | 0.39 | 0.97 | — | T J V* | |
|
1291 |
+| `google/gemma-3n-E4B-it` | 32,768 | 0.06 | 0.12 | — | — | |
|
1292 |
+| `thinkingmachines/Inkling` | 524,288 | 1.00 | 4.05 | 0.17 | T J R* | |
|
1293 |
+| `deepcogito/cogito-v2-1-671b` | 163,840 | 1.25 | 1.25 | — | R* | |
|
1294 |
+| `LiquidAI/LFM2.5-8B-A1B` | 128,000 | 0.03 | 0.12 | — | — | |
|
1295 |
+| `arcee-ai/trinity-mini` | 128,000 | 0.045 | 0.15 | — | — | |
|
1296 |
+ |
|
1297 |
+Notes: |
|
1298 |
+- The probe also lists `zai-org/GLM-4.7-fp4`, `zai-org/GLM-5-FP4`, `Qwen/Qwen3.6-35B-A3B-FP8`, `Qwen/Qwen3.5-35B-A3B`, `MiniMaxAI/MiniMax-M2`, `pearl-ai/gemma-4-31b-it`, `Qwen/Qwen3-235B-A22B-Instruct-2507-FP8` etc. with pricing `0/0` — these appear to be dedicated-endpoint variants; exclude from the serverless catalog (revisit in Phase 7). |
|
1299 |
+- `moonshotai/Kimi-K3` at 1M context and `Qwen/Qwen3.7-Max`/`Qwen3.7-Plus`/`Qwen3.6-Plus` (1M) are the largest-context serverless options. |
|
1300 |
+- Vision on Together: `image_url` content parts (OpenAI format); `video_url`/`audio_url`/`input_audio` also accepted on multimodal models (docs). |
|
1301 |
+ |
|
1302 |
+## 4. Request/Response Format |
|
1303 |
+ |
|
1304 |
+**Fully OpenAI-compatible** (`POST /v1/chat/completions`). Drop-in with OpenAI SDKs — change only base URL + key (https://docs.together.ai/docs/openai-api-compatibility). |
|
1305 |
+ |
|
1306 |
+Deviations to handle: |
|
1307 |
+- Extra `finish_reason` value **`eos`** in addition to `stop|length|tool_calls`. |
|
1308 |
+- Assistant messages from reasoning models carry a **top-level `reasoning` field** (`choices[].message.reasoning`) with the chain of thought — not OpenAI's shape. Zyquo Cloud's "Thinking…" section should read this field. |
|
1309 |
+- Usage token details vary: reasoning models nest cached/reasoning tokens under `usage.prompt_tokens_details` / `usage.completion_tokens_details`; some models return them flat — read both defensively. |
|
1310 |
+- `logprobs` returns Together's own richer shape; `seed` is best-effort; `n` (1–128) not supported on all models; `logit_bias` unsupported on most models; `reasoning_effort` only honored on GPT-OSS models; vision `detail` accepted but ignored. |
|
1311 |
+- Ignored params: `service_tier`, `store`, `metadata`, `prediction`. |
|
1312 |
+- Response may include `warnings: [{message}]` and an echoed `prompt` array. |
|
1313 |
+ |
|
1314 |
+## 5. Streaming |
|
1315 |
+ |
|
1316 |
+- `stream: true` → SSE, `data: {chat.completion.chunk JSON}` lines, terminated by `data: [DONE]`. |
|
1317 |
+- Chunk deltas follow the OpenAI `choices[].delta` shape; reasoning models stream a `reasoning` delta field alongside `content` (mirror of the non-streaming `message.reasoning`). |
|
1318 |
+- **Usage in stream:** Together returns a `usage` object on the final chunk(s) by default (observed behavior; `stream_options.include_usage` is accepted for OpenAI compat — verify exact behavior in Phase 7; marked partially unverified). |
|
1319 |
+ |
|
1320 |
+## 6. Special Parameters |
|
1321 |
+ |
|
1322 |
+(Full list: https://docs.together.ai/reference/chat-completions-1) |
|
1323 |
+ |
|
1324 |
+- `context_length_exceeded_behavior`: `"truncate"` or `"error"` — Together-specific control of over-context prompts (default error). Expose as an Advanced setting. |
|
1325 |
+- `safety_model`: model ID of a moderation model (e.g., Llama Guard) run against output tokens. |
|
1326 |
+- `repetition_penalty` (number) and `min_p` (0–1) — sampler params beyond the OpenAI set; `top_k` (integer) also supported. |
|
1327 |
+- `echo` (bool, include prompt in response), `logprobs` (0–20, Together shape), `seed`, `n` (1–128). |
|
1328 |
+- `response_format`: `text` | `json_object` | `json_schema` (structured outputs supported). |
|
1329 |
+- `tools` / `tool_choice` (`none`/`auto`/named) — OpenAI shape. Deprecated `function_call` still accepted. |
|
1330 |
+- `reasoning_effort` (`low|medium|high`) — GPT-OSS models only; `reasoning` object also documented for reasoning control. |
|
1331 |
+- `chat_template_kwargs` (engine-specific), `compliance: "hipaa"`. |
|
1332 |
+- Temperature documented range 0–1 on Together (vs 0–2 at OpenAI) — clamp the UI slider per provider. |
|
1333 |
+ |
|
1334 |
+## 7. Rate Limits & Errors |
|
1335 |
+ |
|
1336 |
+(https://docs.together.ai/docs/rate-limits) |
|
1337 |
+ |
|
1338 |
+- **Dynamic rate limits** — no published fixed RPM/TPM tiers. Limits adapt per organization and per model based on model capacity and your recent sustained usage; steady traffic raises your limit, spikes get throttled. |
|
1339 |
+- `429` with error types `"dynamic_request_limited"` / `"dynamic_token_limited"`; `503` when the platform itself is capacity-constrained even under your rate. |
|
1340 |
+- Header `x-ratelimit-reset` = seconds to wait before retrying — use it for backoff. |
|
1341 |
+- Error body: OpenAI-shaped `{"error": {"message", "type", "param", "code"}}` but with Together-specific `type`/`code` values; branch on HTTP status (400/401/404/429/500/503/504) for portable handling. |
|
1342 |
+ |
|
1343 |
+## 8. `/models` Listing |
|
1344 |
+ |
|
1345 |
+`GET /v1/models` returns a **bare JSON array** (NOT wrapped in `{"object":"list","data":[...]}` — deviation from OpenAI; confirmed by live probe). Each entry: |
|
1346 |
+ |
|
1347 |
+```json |
|
1348 |
+{ |
|
1349 |
+ "id": "moonshotai/Kimi-K3", |
|
1350 |
+ "object": "model", |
|
1351 |
+ "type": "chat", // chat | language | code | image | video | audio | transcribe | embedding | rerank | moderation |
|
1352 |
+ "display_name": "Kimi K3", |
|
1353 |
+ "organization": "Moonshot AI", |
|
1354 |
+ "context_length": 1000000, |
|
1355 |
+ "config": { "chat_template": null, "stop": [], ... }, |
|
1356 |
+ "pricing": { "input": 3, "output": 15, "cached_input": 0.3, "hourly": 0, ... } // USD per 1M tokens |
|
1357 |
+} |
|
1358 |
+``` |
|
1359 |
+ |
|
1360 |
+Excellent for dynamic refresh: filter `type in {chat, language}`, use `context_length` + `pricing` directly. No capability flags (vision/tools) in the listing — keep those in the built-in catalog. |
|
1361 |
+ |
|
1362 |
+--- |
|
1363 |
+ |
|
1364 |
+ |
|
1365 |
+# DeepInfra — Provider Research (Phase 0) |
|
1366 |
+ |
|
1367 |
+Researched 2026-07-30 against official docs (https://docs.deepinfra.com — note: `deepinfra.com/docs/*` |
|
1368 |
+now 308-redirects there) and a live authenticated `GET /v1/openai/models` probe |
|
1369 |
+(`/tmp/zyquo-probe/deepinfra.json`, 174 models, mixed modalities). Each entry embeds |
|
1370 |
+`metadata.context_length`, `metadata.max_tokens`, and `metadata.pricing` (USD per 1M tokens) — |
|
1371 |
+authoritative for existence, context, and pricing. |
|
1372 |
+ |
|
1373 |
+--- |
|
1374 |
+ |
|
1375 |
+## 1. Base URL & Endpoints |
|
1376 |
+ |
|
1377 |
+- **OpenAI-compatible base URL:** `https://api.deepinfra.com/v1/openai` (what Zyquo Cloud uses) |
|
1378 |
+ - `POST /v1/openai/chat/completions` |
|
1379 |
+ - `POST /v1/openai/completions` |
|
1380 |
+ - `GET /v1/openai/models` |
|
1381 |
+ - `POST /v1/openai/embeddings` (out of scope) |
|
1382 |
+- **Native inference endpoints** also exist (`https://api.deepinfra.com/v1/inference/{model}`) with per-model schemas — documented for completeness only; Zyquo Cloud uses the OpenAI-compatible surface exclusively. |
|
1383 |
+ |
|
1384 |
+Reference: https://docs.deepinfra.com/chat/overview |
|
1385 |
+ |
|
1386 |
+## 2. Authentication |
|
1387 |
+ |
|
1388 |
+`Authorization: Bearer <DEEPINFRA_TOKEN>` — standard Bearer. Confirmed by docs and live probe. |
|
1389 |
+ |
|
1390 |
+## 3. Chat Model Catalog (curated main chat models) |
|
1391 |
+ |
|
1392 |
+Notable: DeepInfra proxies **Anthropic Claude** and **Google Gemini** models under its own billing — |
|
1393 |
+useful as a single-key fallback. Pricing/context below are from the live probe (`metadata`), USD per 1M |
|
1394 |
+tokens. `metadata.max_tokens` equals `context_length` for every chat entry (listing does not expose a |
|
1395 |
+distinct max-output figure — treat real max output as unverified per model). Caps legend: **V** vision, |
|
1396 |
+**T** tools, **R** reasoning, **J** JSON mode. All stream. Flags marked * inferred from model family, not |
|
1397 |
+individually verified on DeepInfra. |
|
1398 |
+ |
|
1399 |
+### Proxied frontier models |
|
1400 |
+ |
|
1401 |
+| Model ID (exact) | Context | $ In | $ Out | $ Cached | Caps | |
|
1402 |
+|---|---|---|---|---|---| |
|
1403 |
+| `anthropic/claude-fable-5` | 1,000,000 | 10.00 | 50.00 | — | V* T R J* | |
|
1404 |
+| `anthropic/claude-opus-5` | 1,000,000 | 5.00 | 25.00 | — | V* T R J* | |
|
1405 |
+| `anthropic/claude-opus-4-8` | 1,000,000 | 5.00 | 25.00 | — | V* T R J* | |
|
1406 |
+| `anthropic/claude-opus-4-7` | 1,000,000 | 5.00 | 25.00 | — | V* T R J* | |
|
1407 |
+| `anthropic/claude-sonnet-5` | 1,000,000 | 2.00 | 10.00 | — | V* T R J* | |
|
1408 |
+| `anthropic/claude-sonnet-4-6` | 1,000,000 | 3.00 | 15.00 | — | V* T R J* | |
|
1409 |
+| `anthropic/claude-haiku-4-5` | 200,000 | 1.00 | 5.00 | — | V* T R J* | |
|
1410 |
+| `google/gemini-3.1-pro` | 1,000,000 | 2.00 | 12.00 | — | V* T R J* | |
|
1411 |
+| `google/gemini-3.5-flash` | 1,000,000 | 1.50 | 9.00 | — | V* T R J* | |
|
1412 |
+| `google/gemini-3.1-flash-lite` | 1,000,000 | 0.25 | 1.50 | — | V* T J* | |
|
1413 |
+| `google/gemini-2.5-pro` | 1,000,000 | 1.25 | 10.00 | — | V* T R J* | |
|
1414 |
+| `google/gemini-2.5-flash` | 1,000,000 | 0.30 | 2.50 | — | V* T R J* | |
|
1415 |
+ |
|
1416 |
+### Open-weight chat models |
|
1417 |
+ |
|
1418 |
+| Model ID (exact) | Context | $ In | $ Out | $ Cached | Caps | |
|
1419 |
+|---|---|---|---|---|---| |
|
1420 |
+| `deepseek-ai/DeepSeek-V4-Pro` | 1,048,576 | 1.30 | 2.60 | 0.10 | T J R* | |
|
1421 |
+| `deepseek-ai/DeepSeek-V4-Flash` | 1,048,576 | 0.09 | 0.18 | 0.018 | T* J* | |
|
1422 |
+| `deepseek-ai/DeepSeek-V3.2` | 163,840 | 0.26 | 0.38 | 0.13 | T* J* R* | |
|
1423 |
+| `deepseek-ai/DeepSeek-V3.1-Terminus` | 163,840 | 0.27 | 0.95 | 0.13 | T* J* R (hybrid) | |
|
1424 |
+| `deepseek-ai/DeepSeek-V3.1` | 163,840 | 0.25 | 0.95 | 0.13 | T* J* R (hybrid) | |
|
1425 |
+| `deepseek-ai/DeepSeek-V3-0324` | 163,840 | 0.24 | 0.90 | 0.135 | T* J* | |
|
1426 |
+| `deepseek-ai/DeepSeek-R1-0528` | 163,840 | 0.50 | 2.15 | 0.35 | R | |
|
1427 |
+| `moonshotai/Kimi-K2.7-Code` | 262,144 | 0.74 | 3.50 | 0.15 | T J R* | |
|
1428 |
+| `moonshotai/Kimi-K2.6` | 262,144 | 0.75 | 3.50 | 0.15 | T J R* | |
|
1429 |
+| `moonshotai/Kimi-K2.5` | 262,144 | 0.45 | 2.25 | 0.07 | T J | |
|
1430 |
+| `zai-org/GLM-5.2` | 1,048,576 | 0.75 | 2.40 | 0.14 | T J R* | |
|
1431 |
+| `zai-org/GLM-5.1` | 202,752 | 1.05 | 3.50 | 0.205 | T* J* R* | |
|
1432 |
+| `zai-org/GLM-5` | 202,752 | 0.60 | 2.08 | 0.12 | T* J* R* | |
|
1433 |
+| `zai-org/GLM-4.7` | 202,752 | 0.40 | 1.75 | 0.08 | T* J* R* | |
|
1434 |
+| `zai-org/GLM-4.7-Flash` | 202,752 | 0.06 | 0.40 | 0.01 | T* J* R* | |
|
1435 |
+| `zai-org/GLM-4.6` | 202,752 | 0.50 | 2.00 | 0.10 | T* J* R* | |
|
1436 |
+| `Qwen/Qwen3.7-Max` | 256,000 | 2.50 | 7.50 | 0.50 | T* J* R* | |
|
1437 |
+| `Qwen/Qwen3.6-35B-A3B` | 262,144 | 0.10 | 0.95 | — | T* J* | |
|
1438 |
+| `Qwen/Qwen3.6-27B` | 262,144 | 0.32 | 3.20 | — | T* J* | |
|
1439 |
+| `Qwen/Qwen3.5-397B-A17B` | 262,144 | 0.45 | 3.00 | 0.22 | T* J* R* | |
|
1440 |
+| `Qwen/Qwen3.5-122B-A10B` | 262,144 | 0.29 | 2.40 | — | T* J* R* | |
|
1441 |
+| `Qwen/Qwen3.5-35B-A3B` | 262,144 | 0.14 | 1.00 | 0.05 | T* J* | |
|
1442 |
+| `Qwen/Qwen3.5-9B` | 262,144 | 0.10 | 0.15 | — | T* J* | |
|
1443 |
+| `Qwen/Qwen3-Max` | 256,000 | 1.20 | 6.00 | 0.24 | T* J* | |
|
1444 |
+| `Qwen/Qwen3-Max-Thinking` | 256,000 | 1.20 | 6.00 | 0.24 | R T* J* | |
|
1445 |
+| `Qwen/Qwen3-235B-A22B-Instruct-2507` | 262,144 | 0.09 | 0.55 | — | T* J* | |
|
1446 |
+| `Qwen/Qwen3-235B-A22B-Thinking-2507` | 262,144 | 0.23 | 2.30 | 0.20 | R T* J* | |
|
1447 |
+| `Qwen/Qwen3-Next-80B-A3B-Instruct` | 262,144 | 0.09 | 1.10 | — | T* J* | |
|
1448 |
+| `Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo` | 262,144 | 0.30 | 1.00 | 0.10 | T J | |
|
1449 |
+| `Qwen/Qwen3-VL-235B-A22B-Instruct` | 262,144 | 0.20 | 0.88 | 0.11 | V T* J* | |
|
1450 |
+| `Qwen/Qwen3-VL-30B-A3B-Instruct` | 262,144 | 0.15 | 0.60 | — | V T* J* | |
|
1451 |
+| `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | 1,048,576 | 0.20 | 0.80 | — | V T J | |
|
1452 |
+| `meta-llama/Llama-4-Scout-17B-16E-Instruct` | 327,680 | 0.10 | 0.30 | — | V T J | |
|
1453 |
+| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | 131,072 | 0.10 | 0.32 | — | T J | |
|
1454 |
+| `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | 131,072 | 0.02 | 0.04 | — | T J | |
|
1455 |
+| `openai/gpt-oss-120b` | 131,072 | 0.037 | 0.17 | — | R T J (`reasoning_effort`) | |
|
1456 |
+| `openai/gpt-oss-120b-Turbo` | 131,072 | 0.15 | 0.60 | — | R T J | |
|
1457 |
+| `openai/gpt-oss-20b` | 131,072 | 0.03 | 0.14 | — | R T J | |
|
1458 |
+| `MiniMaxAI/MiniMax-M3` | 524,288 | 0.30 | 1.20 | 0.06 | T* J* R* | |
|
1459 |
+| `MiniMaxAI/MiniMax-M2.7` | 196,608 | 0.25 | 1.00 | 0.05 | T* J* R* | |
|
1460 |
+| `nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B` | 262,144 | 0.50 | 2.20 | 0.10 | T* J* R* | |
|
1461 |
+| `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B` | 262,144 | 0.085 | 0.40 | — | T* J* R* | |
|
1462 |
+| `nvidia/Nemotron-3-Nano-30B-A3B` | 262,144 | 0.05 | 0.20 | 0.025 | R* | |
|
1463 |
+| `ByteDance/Seed-2.0-pro` | 256,000 | 0.50 | 3.00 | 0.10 | T* J* R* (unverified caps) | |
|
1464 |
+| `ByteDance/Seed-2.0-mini` | 256,000 | 0.10 | 0.40 | 0.02 | T* J* (unverified caps) | |
|
1465 |
+| `XiaomiMiMo/MiMo-V2.5` | 262,144 | 0.40 | 2.00 | 0.08 | R* (unverified caps) | |
|
1466 |
+| `XiaomiMiMo/MiMo-V2.5-Pro` | 1,048,576 | 1.00 | 3.00 | 0.20 | R* (unverified caps) | |
|
1467 |
+| `mistralai/Mistral-Small-3.2-24B-Instruct-2506` | 128,000 | 0.075 | 0.20 | — | V T* J* | |
|
1468 |
+| `mistralai/Mistral-Nemo-Instruct-2407` | 131,072 | 0.019 | 0.03 | — | — | |
|
1469 |
+| `google/gemma-4-31B-it` | 262,144 | 0.13 | 0.38 | — | V* T* J* | |
|
1470 |
+| `google/gemma-3-27b-it` | 131,072 | 0.08 | 0.16 | — | V* | |
|
1471 |
+| `tencent/Hy3` | 262,144 | 0.14 | 0.58 | 0.035 | (unverified caps) | |
|
1472 |
+| `stepfun-ai/Step-3.7-Flash` | 262,144 | 0.20 | 1.15 | 0.04 | (unverified caps) | |
|
1473 |
+| `thinkingmachines/Inkling` | 524,288 | 1.00 | 4.05 | 0.17 | T* J* R* | |
|
1474 |
+| `microsoft/phi-4` | 16,384 | 0.07 | 0.14 | — | — | |
|
1475 |
+ |
|
1476 |
+Also present (skip or list under "more"): Hermes-3 405B/70B, Llama-Guard-4 (moderation), MythoMax, |
|
1477 |
+Sao10K roleplay models, gemma-4 turbo/ultra variants, `deepcogito`, `Qwen3-32B/30B-A3B/14B`, etc. |
|
1478 |
+ |
|
1479 |
+## 4. Request/Response Format |
|
1480 |
+ |
|
1481 |
+**OpenAI-compatible.** Point any OpenAI SDK at `base_url=https://api.deepinfra.com/v1/openai`. Standard |
|
1482 |
+`choices[].message`, `usage`, `finish_reason`. Docs caveat: "We may not be 100% compatible with all |
|
1483 |
+OpenAI parameters" — unknown params are generally ignored. DeepInfra also has native per-model inference |
|
1484 |
+endpoints (`/v1/inference/{model}`) with different schemas — **not used by Zyquo Cloud**. |
|
1485 |
+ |
|
1486 |
+Supported params include: `model`, `messages`, `max_tokens`, `temperature` (0–2), `top_p`, `stop`, `n`, |
|
1487 |
+`presence_penalty`, `frequency_penalty`, `stream`, `response_format` (JSON mode), `tools`/`tool_choice`, |
|
1488 |
+`reasoning_effort` (reasoning models). Reasoning models return chain-of-thought as |
|
1489 |
+`message.reasoning_content` (DeepSeek convention; verify per model in Phase 7 — unverified field name for |
|
1490 |
+non-DeepSeek models). |
|
1491 |
+ |
|
1492 |
+## 5. Streaming |
|
1493 |
+ |
|
1494 |
+(https://docs.deepinfra.com/chat/streaming) |
|
1495 |
+ |
|
1496 |
+- `stream: true` → OpenAI-style SSE: `data: {chunk}` lines with `choices[].delta.content`, final `data: [DONE]`. |
|
1497 |
+- **Usage is included automatically in the final chunk before `[DONE]`** (when `finish_reason` is populated; token counts match non-streaming). No `stream_options.include_usage` needed (parameter not documented; sending it is harmless). |
|
1498 |
+- On completion the delta becomes `{}` and `finish_reason` is set. |
|
1499 |
+ |
|
1500 |
+## 6. Special Parameters |
|
1501 |
+ |
|
1502 |
+- `service_tier`: `"priority"` (faster TTFT/throughput, +50% price) or `"flex"` (−20% price, may queue up to ~10 min). Default tier if omitted. |
|
1503 |
+- `fail_fast: true` — reject immediately with HTTP 429 (`"engine_overloaded"` code) instead of queueing when capacity is exceeded; rejected requests are not billed. |
|
1504 |
+- Response continuation pattern: to continue a truncated answer, resend with the truncated assistant message appended (documented technique, not a parameter). |
|
1505 |
+- `reasoning_effort` supported on reasoning-capable models (gpt-oss etc.). |
|
1506 |
+- Prompt caching: automatic on supported models; discounted `cached_input_tokens` pricing appears in the model metadata (no request parameter required — unverified whether a cache-control knob exists). |
|
1507 |
+ |
|
1508 |
+## 7. Rate Limits & Errors |
|
1509 |
+ |
|
1510 |
+(https://deepinfra.com/docs/advanced/rate-limits) |
|
1511 |
+ |
|
1512 |
+- **200 concurrent requests per account** (the primary limit; no published RPM/TPM). Increases via Dashboard → Account request. |
|
1513 |
+- Exceeding it → HTTP 429 "Rate limited"; capacity overload with `fail_fast` → 429 `"engine_overloaded"`. |
|
1514 |
+- Errors are OpenAI-shaped JSON; branch on HTTP status (400/401/404/429/5xx). Retry with backoff on 429/5xx. |
|
1515 |
+ |
|
1516 |
+## 8. `/models` Listing |
|
1517 |
+ |
|
1518 |
+`GET https://api.deepinfra.com/v1/openai/models` returns OpenAI shape `{"object":"list","data":[...]}` — |
|
1519 |
+but each model carries a rich non-standard `metadata` object: |
|
1520 |
+ |
|
1521 |
+```json |
|
1522 |
+{ |
|
1523 |
+ "id": "deepseek-ai/DeepSeek-V4-Pro", |
|
1524 |
+ "object": "model", |
|
1525 |
+ "owned_by": "deepinfra", |
|
1526 |
+ "metadata": { |
|
1527 |
+ "description": "...", |
|
1528 |
+ "context_length": 1048576, |
|
1529 |
+ "max_tokens": 1048576, |
|
1530 |
+ "pricing": { "input_tokens": 1.3, "output_tokens": 2.6, "cached_input_tokens": 0.1 } |
|
1531 |
+ } |
|
1532 |
+} |
|
1533 |
+``` |
|
1534 |
+ |
|
1535 |
+Ideal for dynamic refresh: filter entries where `metadata.context_length` and `pricing.output_tokens` |
|
1536 |
+are non-null (text-gen models); image/video/TTS/ASR entries have null context/pricing fields. |
|
1537 |
+No capability flags (vision/tools) — keep those in the built-in catalog. |
|
1538 |
+ |
|
1539 |
+--- |
|
1540 |
+ |
|
1541 |
+ |
|
1542 |
+# Cerebras — Provider Research (Phase 0) |
|
1543 |
+ |
|
1544 |
+Researched 2026-07-30 against official docs (https://inference-docs.cerebras.ai) and a live |
|
1545 |
+authenticated `GET /v1/models` probe (`/tmp/zyquo-probe/cerebras.json`). Cerebras runs inference on |
|
1546 |
+wafer-scale hardware — its differentiator is **extreme speed** (1,000–3,000 output tokens/s), not |
|
1547 |
+catalog breadth. |
|
1548 |
+ |
|
1549 |
+--- |
|
1550 |
+ |
|
1551 |
+## 1. Base URL & Endpoints |
|
1552 |
+ |
|
1553 |
+- **Base URL:** `https://api.cerebras.ai/v1` |
|
1554 |
+- `POST /v1/chat/completions` — chat (used by Zyquo Cloud) |
|
1555 |
+- `POST /v1/completions` — text completion |
|
1556 |
+- `GET /v1/models`, `GET /v1/models/{id}` — model listing |
|
1557 |
+ |
|
1558 |
+Reference: https://inference-docs.cerebras.ai/api-reference/chat-completions |
|
1559 |
+ |
|
1560 |
+## 2. Authentication |
|
1561 |
+ |
|
1562 |
+`Authorization: Bearer <CEREBRAS_API_KEY>` — standard Bearer (key from cloud.cerebras.ai console). Confirmed. |
|
1563 |
+ |
|
1564 |
+## 3. Chat Model Catalog |
|
1565 |
+ |
|
1566 |
+Live `/models` probe returned exactly **3 models** — and the docs models page |
|
1567 |
+(https://inference-docs.cerebras.ai/models/overview) lists the **same 3** public models, so there is |
|
1568 |
+**no discrepancy** between `/models` and docs today (older models like llama-3.3-70b, qwen-3-*, and |
|
1569 |
+llama3.1-8b no longer appear in either — treat as removed). Additional models exist only via paid |
|
1570 |
+Dedicated Endpoints. Context windows are **tier-dependent** (free vs paid). |
|
1571 |
+ |
|
1572 |
+| Model ID (exact) | Status | Context (free / paid) | Max output (free / paid) | $ In / $ Out per 1M | Speed | Caps | |
|
1573 |
+|---|---|---|---|---|---|---| |
|
1574 |
+| `gpt-oss-120b` | Production | 65k / 131k | 32k / 40k | 0.35 / 0.75 (docs page; a search snippet also showed 0.25/0.69 — treat exact price as needs-confirmation in Phase 7) | ~3,000 tok/s | Reasoning (`reasoning_effort`, default `medium`), tools, structured outputs, streaming, prompt caching | |
|
1575 |
+| `gemma-4-31b` | Preview | 65k / 131k | 32k / 40k | 0.99 / 1.49 | ~1,850 tok/s | **Vision** (base64 images only), tools (parallel), structured outputs (constrained decoding), reasoning (off by default, enable via `reasoning_effort`), streaming, prompt caching | |
|
1576 |
+| `zai-glm-4.7` | Preview — **deprecated 2026-08-17** | 64k / 131k | 40k / 40k | 2.25 / 2.75 | ~1,000 tok/s | Reasoning (on by default), tools (parallel, `strict: true`), structured outputs, streaming, prompt caching | |
|
1577 |
+ |
|
1578 |
+Notes: |
|
1579 |
+- `zai-glm-4.7` is scheduled for discontinuation on **August 17, 2026** (migration guide: https://inference-docs.cerebras.ai/resources/glm-47-migration). Ship it flagged "deprecating soon" in `ModelCatalog`. |
|
1580 |
+- `gemma-4-31b` image limits: max 2 images/request (free) or 10 (paid), 10 MB total, base64 PNG/JPEG data URIs only — **external image URLs not supported**; chat endpoint only. |
|
1581 |
+- Note the probe listing order (`gemma-4-31b`, `gpt-oss-120b`, `zai-glm-4.7`) matches docs exactly. |
|
1582 |
+ |
|
1583 |
+## 4. Request/Response Format |
|
1584 |
+ |
|
1585 |
+OpenAI-compatible with notable deviations: |
|
1586 |
+ |
|
1587 |
+- **`max_tokens` is NOT supported — use `max_completion_tokens`** (includes reasoning tokens). This is the biggest client-side difference; Zyquo Cloud's Cerebras path must map its max-tokens setting to `max_completion_tokens`. |
|
1588 |
+- `temperature` range 0–2 (default 0); `top_p` 0–1; `stop` up to 4 sequences; `seed` for determinism; `frequency_penalty`/`presence_penalty`; `logit_bias`; `logprobs` (boolean) + `top_logprobs` (0–20). |
|
1589 |
+- Roles: `system`, `user`, `assistant`, `tool`, plus `developer` (gpt-oss-120b only; `system` maps to developer-level). |
|
1590 |
+- Reasoning models return chain-of-thought in **`choices[].message.reasoning`** (top-level field). |
|
1591 |
+- Response includes extras: `time_info` (queue/prompt/completion/total seconds — great for a latency badge), `usage.prompt_tokens_details.cached_tokens`, `usage.completion_tokens_details.reasoning_tokens`. |
|
1592 |
+- `n` (multiple completions) and `stream_options` are not documented (assume unsupported — unverified). |
|
1593 |
+- Optional: `Content-Type: application/vnd.msgpack`, gzip request encoding, `queue_threshold` header (50–20000 ms) for flex/auto tiers. |
|
1594 |
+ |
|
1595 |
+## 5. Streaming |
|
1596 |
+ |
|
1597 |
+- `stream: true` → SSE `chat.completion.chunk` objects with `choices[].delta` (content, tool_calls, and `reasoning` deltas on reasoning models), terminated by `data: [DONE]`. |
|
1598 |
+- **`usage` appears in the final streamed chunk** per the documented streaming response schema (no `stream_options.include_usage` needed — the parameter is not documented). |
|
1599 |
+- Streams are extremely fast (up to ~3,000 tok/s) — Zyquo Cloud's renderer must batch UI updates (Phase 4.3 "no layout thrash" rule matters most here). |
|
1600 |
+ |
|
1601 |
+## 6. Special Parameters |
|
1602 |
+ |
|
1603 |
+- `reasoning_effort`: `low | medium | high | none`. gpt-oss-120b default `medium`; gemma-4-31b reasoning off by default (enable via this param; no `raw`/`hidden` formats); zai-glm-4.7 reasoning on by default. |
|
1604 |
+- `clear_thinking` (boolean): whether prior turns' thinking content is included in prompt context — off by default, recommended on for agentic use (zai-glm-4.7). |
|
1605 |
+- `response_format`: `text` | `json_object` | `json_schema` with **`strict: true` schema enforcement via constrained decoding** — Cerebras's structured outputs are genuinely strict. |
|
1606 |
+- `tools`/`tool_choice` (`none|auto|required|named`) + `parallel_tool_calls` (default true); `strict: true` supported on tool schemas. |
|
1607 |
+- `prediction` (predicted outputs), `prompt_cache_key` (grouping for prompt caching), `service_tier` (`priority|default|auto|flex`), `user`. |
|
1608 |
+- Known quirks (docs): gpt-oss-120b may emit unexpected EOS with `min_tokens`, and may hallucinate tool calls outside schema (reprompt to correct). |
|
1609 |
+ |
|
1610 |
+## 7. Rate Limits & Errors |
|
1611 |
+ |
|
1612 |
+(https://inference-docs.cerebras.ai/support/rate-limits) |
|
1613 |
+ |
|
1614 |
+- Measured on RPM/RPH/RPD **and** TPM/TPH/TPD — whichever trips first. **Dual token buckets:** uncached TPM and total TPM (total ≈ 3× uncached) — good caching effectively triples throughput. |
|
1615 |
+- **Free Trial** ($5 credit): **5 RPM across all models, 30k TPM, 1M TPH, 1M TPD** — very tight; Zyquo Cloud error messages should mention upgrading when free-tier users hit 429. |
|
1616 |
+- **Developer (pay-as-you-go):** `gpt-oss-120b` 1M TPM / 1,000 RPM; `zai-glm-4.7` 500k TPM / 500 RPM; no hourly/daily caps. (gemma-4-31b developer-tier numbers not published — unverified.) |
|
1617 |
+- **Enterprise:** custom. |
|
1618 |
+- 429 on limit, with the message stating which bucket (uncached vs total) was exceeded. Error bodies follow the OpenAI `{"error": {...}}` shape (message/type/param/code — shape unverified in fine detail; confirm in Phase 7). |
|
1619 |
+- Free-tier context is also capped (65k vs 131k paid) — a context-length error on free tier may occur below the model's nominal window. |
|
1620 |
+ |
|
1621 |
+## 8. `/models` Listing |
|
1622 |
+ |
|
1623 |
+`GET /v1/models` returns standard OpenAI shape, minimal fields only: |
|
1624 |
+ |
|
1625 |
+```json |
|
1626 |
+{"object":"list","data":[ |
|
1627 |
+ {"id":"gemma-4-31b","object":"model","created":0,"owned_by":"Cerebras"}, |
|
1628 |
+ {"id":"gpt-oss-120b","object":"model","created":0,"owned_by":"Cerebras"}, |
|
1629 |
+ {"id":"zai-glm-4.7","object":"model","created":0,"owned_by":"Cerebras"} |
|
1630 |
+]} |
|
1631 |
+``` |
|
1632 |
+ |
|
1633 |
+No context/pricing/capability metadata — dynamic refresh can only diff IDs; everything else must come |
|
1634 |
+from the built-in catalog (this document). |
|
1635 |
+ |
|
1636 |
+--- |