spb/polyllm

1# Cerebras Inference — provider research for PolyLLM23Last documentation audit: **2026-09-08**4Probes executed 2026-09-08 with a real **Developer-tier (pay-as-you-go)** key against `https://api.cerebras.ai/v1` (scripts in `research/cerebras/`, raw outputs in `research/cerebras/out/`). Everything marked **(probed)** was observed live; everything marked **(docs)** comes from the pages listed at the end.56> Heads-up: `GET /v1/models` for this key returns **3 models**: `gpt-oss-120b`, `qwen-3.8-27b`, `gemma-4-31b`. The changelog and deprecation page say **`gemma-4-31b` was removed from public endpoints on 2026-09-03** ("remains on dedicated endpoints"), its model page is 404 and it has no public price — yet it is still listed and **served normally for this key (all probes 200)**. Treat it as *grandfathered / may disappear without notice*; do not make it a default.78---910## 1. Base URL, auth, headers1112| Item | Value |13|---|---|14| REST base URL | `https://api.cerebras.ai/v1` (docs + probed). Behind Cloudflare (`cf-ray`, `server: cloudflare`). |15| Auth | `Authorization: Bearer <CEREBRAS_API_KEY>` (keys start with `csk-`). |16| Content type | `application/json`. Malformed JSON → **400** `{"message":": Invalid JSON: key must be a string at line 1 column 2","type":"invalid_request_error","param":"validation_error","code":"wrong_api_format"}` **(probed)**. Docs: payload optimisation supports msgpack + gzip (not needed). |17| API version header | `X-Cerebras-Version-Patch: 2` (docs). **v2 is the default since 2026-07-22**; sending `1` → **400** `API version 1 is not supported by this model (minimum supported version 2).` **(probed)**. Don't send the header. |18| Response headers **(probed, inference calls only, not on `/models`)** | `x-request-id`, and **per-model, per-window** rate limits: `x-ratelimit-limit-requests-{minute,hour,day}`, `x-ratelimit-limit-tokens-{minute,hour,day}`, `x-ratelimit-remaining-requests-{minute,hour,day}`, `x-ratelimit-remaining-tokens-{minute,hour,day}`. 429 adds `retry-after: 60`. Observed limits for this Developer key: gpt-oss-120b **1000 RPM / 500 000 TPM** (60k RPH, 1.44M RPD, 30M TPH, 720M TPD); gemma-4-31b **500 RPM / 250 000 TPM**; qwen-3.8-27b **450 RPM / 150 000 TPM** (docs say 300 RPM / 150K uncached TPM — headers show the *uncached* bucket as `limit` and the *total* bucket in `remaining`, e.g. `remaining-tokens-hour 89 999 979` > `limit-tokens-hour 30 000 000`). |19| Key introspection | None. Cheapest "validate key" call is `GET /v1/models` (200 vs **401**) **(probed)**. |2021## 2. SDK recommendation (TypeScript / Node)2223- Official TS SDK **`@cerebras/cerebras_cloud_sdk@1.91.0`** (Stainless-generated, Node ≥ 18, TS ≥ 4.5). Options `apiKey`, `baseURL`, `timeout` (default 1 min), `maxRetries` (default 2; retries 408/429/≥500 + connection errors), `warmTCPConnection` (opens a TCP connection at construction — set `false` in a server). Errors: `APIError` subclasses `BadRequestError` 400, `AuthenticationError` 401, `PermissionDeniedError` 403, `NotFoundError` 404, `RateLimitError` 429, `InternalServerError` ≥500, `APIConnectionError`. **(probed: invalid key → `AuthenticationError`, `.status 401`, `.message "401 Wrong API Key"`)**. Its types know `reasoning`, `time_info`, `reasoning_effort`, `reasoning_format`, `clear_thinking`.24- **OpenAI SDK also works unchanged** (`openai@7.10.0` probed) with `baseURL: "https://api.cerebras.ai/v1"`: streaming, `delta.reasoning`, final usage chunk, tools, `response_format` all fine. Cerebras-only params (`reasoning_format`, `clear_thinking`) must go through `extra_body`/a cast (docs).25- **Recommendation for PolyLLM:** reuse the OpenAI-compatible chat-completions path (same as xAI) with `baseURL` swapped and a Cerebras translation layer; `maxRetries: 0` (we do our own backoff), `timeout` 120 s is plenty (fastest provider we have: 1 200–2 600 tok/s). No need to add the Cerebras SDK unless we want its typed `time_info`.26- Vercel AI SDK: `@ai-sdk/cerebras` exists (docs integration page) — not needed.2728## 3. Endpoints2930| Endpoint | Status | Notes |31|---|---|---|32| `POST /v1/chat/completions` | **the** inference endpoint | OpenAI-compatible; SSE streaming; `time_info` extra. **(probed)** |33| `POST /v1/completions` | legacy, works | `{model, prompt, max_tokens}` → `choices[0].text` **(probed 200)**. Docs page exists. Not for the adapter. |34| `GET /v1/models`, `GET /v1/models/{id}` | active | Minimal `{id, object:"model", created: 0, owned_by:"Cerebras"}`. **No context/pricing/capabilities.** Unknown id → 404 `model_not_found`. **(probed)** |35| `POST /v1/responses` | **404** **(probed)** | No Responses API. |36| `POST /v1/embeddings` | **404** **(probed)** | No embeddings. |37| Batch (`/v1/batches`), Files (`/v1/files`), Metrics | active (docs) | Batch = async JSONL jobs; out of scope. |38| Dedicated-endpoint management API | active (docs) | Enterprise only. |3940## 4. Chat Completions request/response (probed shapes)4142Request body fields (docs API ref + probes): `model`, `messages`, `max_completion_tokens` (alias `max_tokens`; **sending both → 400** `Setting "max_tokens" and "max_completion_tokens" at the same time is not supported.`), `min_completion_tokens`/`min_tokens`, `temperature` **0–2** (2.5 → 400 `temperature: Input should be less than or equal to 2`), `top_p` 0–1, `stop` (≤4), `seed`, `stream`, `stream_options` (accepted; **usage is sent anyway**, see §5), `response_format` (`text|json_object|json_schema`), `tools`, `tool_choice` (`none|auto|required|{type:"function",function:{name}}`), `parallel_tool_calls` (default true), `reasoning_effort` (`none|low|medium|high`, model-dependent), `reasoning_format` (`parsed|raw|hidden|none`), `clear_thinking` (qwen only), `logprobs`/`top_logprobs` (0–20), `n` (**only 1**: `n: 2` → 400 `"n" > 1 is not currently supported`), `frequency_penalty`/`presence_penalty` (−2..2, all models since 2026-03-31), `logit_bias`, `user`, `service_tier` (`default|priority|auto|flex`, dedicated only — **silently accepted on shared**, no `service_tier` echoed **(probed)**), `prompt_cache_key` (≤1024 chars), `prediction` (dedicated only).4344**Strict schema validation (v2): unknown top-level params are REJECTED**, e.g. `top_k` → 400 `top_k: property 'top_k' is unsupported`, `foo_bar` → 400 `foo_bar: property 'foo_bar' is unsupported` **(probed)**. Never forward unknown/experimental keys.4546Roles: `system`, `user`, `assistant`, `tool`. **`developer` role: OK on gpt-oss-120b and gemma-4-31b, 400 on qwen-3.8-27b** (`Failed to apply chat template to messages due to error: Unexpected message role.`) **(probed)** → always send `system`. Content parts: `{type:"text"}` (all roles), `{type:"image_url", image_url:{url}}` (user only, data URI only — §10).4748Non-streaming response **(probed)**:4950```json51{52  "id": "chatcmpl-61794ede-…", "object": "chat.completion", "created": 1788849167, "model": "gpt-oss-120b",53  "system_fingerprint": "fp_b546658c8e93d2e57ef2",54  "choices": [{ "index": 0, "finish_reason": "stop",55    "message": { "role": "assistant", "content": "2 + 2 equals 4.",56                 "reasoning": "User asks: \"What is 2+2? …\" Just answer: …" } }],57  "usage": { "prompt_tokens": 80, "completion_tokens": 51, "total_tokens": 131,58             "prompt_tokens_details": { "cached_tokens": 0 },59             "completion_tokens_details": { "reasoning_tokens": 32,60                                            "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } },61  "time_info": { "queue_time": 0.004089397, "prompt_time": 0.00175769,62                 "completion_time": 0.019383306, "total_time": 0.026791095733642648, "created": 1788849167.0522857 }63}64```6566- **Reasoning is inside `completion_tokens`** (51 = 19 visible + 32 reasoning); `total_tokens = prompt + completion`. Reasoning also counts toward `max_completion_tokens` (docs + probed: `max_completion_tokens: 20` on qwen → `finish_reason: "length"`, `reasoning` = 20 tokens, **no `content` key at all**). The adapter must treat a missing `content` as empty string, and the UI needs a "budget consumed by reasoning" hint.67- The `reasoning` field is **absent** (not `null`) when the model did not reason (gemma default, `reasoning_effort: "none"`).68- `time_info` (seconds): use `completion_tokens / completion_time` for tokens/s. Observed: gemma ≈ **1 216 tok/s**, qwen ≈ **1 393**, gpt-oss ≈ **2 631** (high effort, 785 tokens: 1 472). `queue_time` occasionally 0.2–0.3 s on qwen.69- `prompt_tokens_details.image_tokens` present on gemma/qwen (0 without images); gpt-oss has none. gpt-oss adds `accepted/rejected_prediction_tokens`.70- `finish_reason`: `stop`, `length`, `tool_calls` **(probed)**; docs also `content_filter`.71- No hidden system prompt to speak of: a one-line prompt = 20–26 prompt tokens on gemma, ~60 on qwen, ~75–80 on gpt-oss (harmony preamble).72- `max_completion_tokens` far above the documented cap (60 000, even 999 999) is **accepted silently** **(probed)** — the API does not error, it just caps output; the docs' 40k "max output" is a soft figure. Omit or clamp client-side.7374## 5. Streaming protocol **(probed)**7576SSE, `Content-Type: text/event-stream`, no `event:` field, `data: {json}` lines, terminated by `data: [DONE]`.7778```json79data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1788849167,"model":"gpt-oss-120b","system_fingerprint":"fp_…",80       "choices":[{"index":0,"delta":{"role":"assistant"}}]}81data: {"…","choices":[{"index":0,"delta":{"reasoning":"The"}}]}82data: {"…","choices":[{"index":0,"delta":{"reasoning":" user asks: \"Say hello"}}]}83…84data: {"…","choices":[{"index":0,"delta":{"content":"Bonjour"}}]}85data: {"…","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],86       "usage":{"prompt_tokens":77,"completion_tokens":152,"total_tokens":229,"prompt_tokens_details":{"cached_tokens":0},87                "completion_tokens_details":{"reasoning_tokens":136,…}},88       "time_info":{"queue_time":0.0033,"prompt_time":0.0017,"completion_time":0.1104,"total_time":0.1175,"created":1788849167.17}}89data: [DONE]90```9192- Delta keys observed: `role`, `reasoning`, `content`, `tool_calls`. **Reasoning streams as `delta.reasoning` (plain text)** — not `reasoning_content`, no `<think>` tags in `content` (unless `reasoning_format: "raw"`, §8).93- **`usage` + `time_info` ride on the SAME chunk as `finish_reason`** (no separate `choices: []` chunk) and are present **with or without `stream_options.include_usage`** **(probed on all 3)**. Standard OpenAI accumulation code works; just read `chunk.usage` whenever it is defined.94- Chunks are big (whole phrases per delta) because generation is so fast — 5–25 events per short answer; TTFB 107–221 ms.95- `json_object` + `stream: true`: docs say "not compatible… stream must be false", but it **works on all 3 models** **(probed, valid JSON streamed)**. Keep the docs rule in mind if it starts failing.9697## 6. Tool / function calling **(probed on all 3 models)**9899- OpenAI nested format `{type:"function", function:{name, description, parameters, strict?}}`; `tool_choice` `auto|required|none|{type:"function",function:{name}}` all OK; `parallel_tool_calls: false` OK; `strict: true` OK (requires `additionalProperties:false` on every object; qwen forbids `pattern/minLength/maxLength` in strict tool schemas — docs).100- Round trip (`assistant.tool_calls` + `{role:"tool", tool_call_id, content}`) → 200 with a grounded answer on all 3. Tool-call ids are **9-hex-char strings** (`"c1ded86e6"`), not `call_…`.101- **Streaming shape differs per model**:102  - gemma-4-31b, qwen-3.8-27b: **one chunk** with the complete call `{"tool_calls":[{"index":0,"id":"7f46ce3e9","type":"function","function":{"name":"get_weather","arguments":"{\"city\": \"Montreal\"}"}}]}`.103  - gpt-oss-120b: **true deltas** — chunk 1 `{id, type, index, function:{name, arguments:""}}`, then `{index, type, function:{arguments:"{\"city\":\""}}`, `{… arguments:"Montreal\"}"}`. Standard index-keyed accumulation handles both.104  - then `finish_reason: "tool_calls"` (usage on that chunk).105- **`tools` + `response_format` together: 400 on gpt-oss-120b** `"tools" is incompatible with "response_format"` (`param: "tools"`); accepted on gemma/qwen (the model called the tool). Adapter: drop `response_format` when tools are present for gpt-oss, or refuse the combination.106- gpt-oss-120b docs warning: "may call tools that aren't directly specified due to its training" (hallucinated tool names) — validate the name before dispatch.107- v2 validation is strict on multi-turn tool history (docs): no orphan `tool` messages, every `tool_call_id` must match, unique ids.108109## 7. Structured output **(probed on all 3 models)**110111- `response_format: {type:"json_schema", json_schema:{name, schema, strict:true}}` → valid JSON matching the schema on all 3 (non-stream and stream, gpt-oss stream verified in probe 07). `{type:"json_object"}` works on all 3 (also streamed).112- **`strict: true` without `additionalProperties:false` → 400** `{"message":"'additionalProperties' is required to be supplied and set to false.","type":"invalid_request_error","param":"response_format","code":"wrong_api_format"}` **(probed all 3)** — the adapter must inject `additionalProperties:false` recursively (same requirement as OpenAI strict mode).113- `pattern` (documented as prohibited) was **accepted** on all 3 — apparently ignored (output happened to match). Don't rely on it.114- Docs limits (strict): schema text ≤ 5 000 chars, depth ≤ 10, ≤ 500 properties, ≤ 500 enum values total; supported: primitive types, `enum`, `const`, `anyOf`, nested objects/arrays, `required`, `$ref/$defs`, tuple `items`, number bounds, `multipleOf`, annotations. Prohibited: recursion, external `$ref`, `oneOf`, `allOf`, `not`, `if/then/else`, `patternProperties`, `unevaluatedProperties`, `pattern`, `format`, `minItems/maxItems`. Output keys follow schema order.115- gpt-oss: `min_tokens`/`min_completion_tokens` → 400 `Structured outputs is not currently supported with min_tokens` **even without response_format** **(probed)** — never send `min_tokens` to gpt-oss.116117## 8. Reasoning controls **(probed matrix)**118119| Model | Default | `reasoning_effort` accepted | Rejected | `reasoning_format` | `reasoning` field |120|---|---|---|---|---|---|121| gpt-oss-120b | `medium`, always on | `low`, `medium`, `high` | **`none` → 400** `Failed to apply chat template to messages due to error: Unsupported reasoning effort: none. Supported values are 'low', 'medium', and 'high'.`; `minimal` → 400 `reasoning_effort: Input should be 'none', 'low', 'medium' or 'high'` | `parsed` (default), `raw` (reasoning **prepended to `content` with no delimiter**: `"Need answer \"Yes\". One word.Yes"`), `hidden` (no `reasoning` key, `reasoning_tokens` still billed) | yes (`message.reasoning`, `delta.reasoning`) |122| qwen-3.8-27b | `high`, on | `none`, `low`, `medium`, `high` | `minimal` → 400 | `parsed` (default), `raw` (**`reasoning\n</think>\n\ncontent`** — closing tag only, no opening `<think>`), **`hidden` → 400** `qwen-3.8-27b does not support 'hidden' reasoning format.` | yes; absent when `none`. `content` starts with `"\n\n"` after reasoning — trim. |123| gemma-4-31b | **off** (`none`) | `none`, `low`, `medium`, `high` (docs: low/medium/high all equivalent — probed reasoning_tokens 34–40 regardless) | `minimal` → 400 | `parsed` only; **`raw`/`hidden` → 400** `Gemma-4 does not support 'raw' reasoning format.` | yes when enabled (`delta.reasoning` streamed **(probed)**); absent otherwise |124125- **`disable_reasoning`** (deprecated 2026-07-21): still **accepted on gemma/qwen** (behaves like `none`), **400 on gpt-oss** (same "Unsupported reasoning effort: none" message) **(probed)**. Use `reasoning_effort: "none"` only.126- **`clear_thinking: true`**: OK on qwen only; gemma/gpt-oss → 400 `"clear_thinking" is not supported for this model`. When replaying history, the adapter should **not** send prior `reasoning` back (qwen keeps it unless `clear_thinking`; others ignore).127- Effort has little effect on trivial prompts (qwen 22→31→29 reasoning tokens for low/medium/high; gpt-oss 18/33/37); gpt-oss `high` on "count the r's" → **758 reasoning tokens**, 785 completion in 0.53 s.128- No thinking budget parameter. Reasoning tokens are billed at the output rate and consume `max_completion_tokens` → for reasoning-on models set `max_completion_tokens` ≥ 1 000 or expect empty `content` with `finish_reason: "length"`.129- `logprobs: true` with reasoning → `choices[0].logprobs` (content tokens) **and `choices[0].reasoning_logprobs`** (reasoning tokens, same `{content:[{token, logprob, bytes, top_logprobs}]}` shape) **(probed)**.130131## 9. Sampling & other parameters — support matrix **(probed, chat completions, HTTP status)**132133| Param | gemma-4-31b | qwen-3.8-27b | gpt-oss-120b |134|---|---|---|---|135| `temperature` 0–2 | ✓ | ✓ | ✓ (>2 → 400 everywhere) |136| `top_p` | ✓ | ✓ | ✓ |137| `top_k` | **400 unsupported property** | **400** | **400** |138| `max_completion_tokens` / `max_tokens` (not both) | ✓ | ✓ | ✓ |139| `min_completion_tokens` | ✓ | ✓ | **400** (`Structured outputs is not currently supported with min_tokens`) |140| `stop` | ✓ (`stop` honoured, `finish_reason: stop`) | ✓ | ✓ |141| `seed` | ✓ (**deterministic**: same 6-digit "random" number twice, same `system_fingerprint`) | ✓ | ✓ |142| `frequency_penalty` / `presence_penalty` | ✓ | ✓ | ✓ |143| `logit_bias` | ✓ | ✓ | ✓ |144| `logprobs` + `top_logprobs` | ✓ | ✓ (+`reasoning_logprobs`) | ✓ (+`reasoning_logprobs`) |145| `n: 2` | **400** | **400** | **400** |146| `reasoning_effort` | none/low/medium/high | none/low/medium/high | low/medium/high (**no none**) |147| `reasoning_format` | parsed | parsed, raw | parsed, raw, hidden |148| `clear_thinking` | **400** | ✓ | **400** |149| `disable_reasoning` (deprecated) | ✓ (=none) | ✓ (=none) | **400** |150| `response_format` json_object / json_schema | ✓ / ✓ | ✓ / ✓ | ✓ / ✓ |151| `tools` + `response_format` | ✓ | ✓ | **400** |152| `developer` role | ✓ | **400** | ✓ |153| `system` role | ✓ | ✓ | ✓ |154| `service_tier: "priority"` | accepted, ignored | accepted, ignored | accepted, ignored |155| `prompt_cache_key`, `user` | ✓ | ✓ | ✓ |156| unknown param (`foo_bar`) | **400** | **400** | **400** |157| `image_url` content | ✓ | ✓ | **400** |158159Error body for every 400: `{"message": "...", "type": "invalid_request_error", "param": "<field or validation_error>", "code": "wrong_api_format", "id": ""}`. **Adapter rules: strip any non-whitelisted key; never send `top_k`, `n>1`, `min_tokens` (gpt-oss), `clear_thinking` (non-qwen), `developer` role (map to `system`), `reasoning_effort: "none"` to gpt-oss, `response_format` with tools on gpt-oss, `reasoning_format` other than `parsed`.**160161## 10. Modalities, context, output limits162163- **Input**: text on all; **images on gemma-4-31b and qwen-3.8-27b** (docs + **probed OK**: 32×32 PNG → gemma `"Red"`, `image_tokens: 256`; qwen `image_tokens: 64`). **gpt-oss-120b: 400** `{"message":"Content type 'image_url' is not supported by selected model. Only 'text' content type can be used.","type":"invalid_request_error","param":"prompt","code":"wrong_api_format"}`. **Output**: text only. No audio, no image generation, no files, no web search, no embeddings.164- **Images**: base64 data URI only (`data:image/png;base64,…` / JPEG); **HTTPS URL → 400** `{"message":"Remote image URLs are not supported; send images as data URIs (data:image/png;base64,...)","type":"invalid_request_error","param":"messages","code":"invalid_multimodal_input"}` **(probed)** → PolyLLM must fetch/inline remote images itself. `detail` is documented unsupported but **`detail: "high"` was silently accepted** **(probed)** — strip it anyway. Limits (docs): ≤ 15 000 px per side, ≤ 10 MiB request payload (→ 413 `image_too_large`), **2 images/request on Free Trial, 10 on Developer/Enterprise**, user messages only. Token formula: qwen 32×32 px/token (max 2 304), gemma 48×48 (max 280 — probe shows a 32×32 image is padded to 256 tokens). Image tokens are inside `prompt_tokens` and repeated in `prompt_tokens_details.image_tokens`.165- **Context windows (docs, Free → Paid)**: gpt-oss-120b **65k → 131k**; qwen-3.8-27b **64k → 128k** (65 536 → 131 072); gemma-4-31b: no page — **probed limit 131 072**. Live errors with a ~186k-token prompt: gpt-oss `Please reduce the length of the messages or completion. Current length is 186745 while limit is 131000` (`code: "context_length_exceeded"`, `param: "messages"`, 400); qwen/gemma same text with `limit is 131072`. A ~93k-token prompt succeeded on all 3 (this key is paid tier; free keys will be refused above ~65k). "Current length" = prompt + `max_completion_tokens`.166- **Max output (docs)**: gpt-oss-120b 32k free / **40k paid**; qwen-3.8-27b 32 768 free / **40 960 paid**; gemma unknown (null). Not enforced as a 400 (§4).167- Free Trial vs Developer for this key: headers show Developer-tier limits; a 150K-TPM qwen bucket was exhausted by one 93k-token request + retry → 429 `{"message":"Tokens per minute limit exceeded - too many tokens processed.","type":"too_many_tokens_error","param":"quota","code":"token_quota_exceeded"}` with `retry-after: 60` **(probed)**. Large-context use on qwen is effectively rate-limited to ~1 request/min.168169## 11. Server-side tools, search, citations170171None. Cerebras has no web search, code interpreter, file search or MCP tool; only client-side function calling. No citations/annotations.172173## 12. Model listing & pricing units174175`GET /v1/models` gives ids only. Pricing is **not in the API**; the public pricing page (`cerebras.ai/pricing`) only states tiers (Free Trial $5 credits / 30 days, Developer from $10, Enterprise). Per-model prices come from the model doc pages **(docs)**:176177| Model | ctx free / paid | max out free / paid | speed (docs / probed) | in / out ($/M) | cached |178|---|---|---|---|---|---|179| gpt-oss-120b | 65k / 131k (probed 131 000) | 32k / 40k | ~3000 / 2 631 tok/s | **0.35 / 0.75** | no discount |180| qwen-3.8-27b | 64k / 128k (probed 131 072) | 32 768 / 40 960 | ~1500 / 1 393 | **0.99 / 1.49** | no discount |181| gemma-4-31b | — / probed 131 072 | — | — / 1 216 | **unknown (removed from public price list 2026-09-03)** | — |182183Prompt caching gives **no price discount** ("billed at the standard input token rate") — it only reduces latency and the *uncached* TPM bucket. Reasoning tokens billed as output. Batch API and service tiers have no differentiated pricing during preview (docs).184185## 13. Prompt caching & provider-side state186187- **Automatic** on all models, 128-token blocks (prefixes < 128 tokens may not be cached), TTL guaranteed 5 min (up to 1 h), reported in `usage.prompt_tokens_details.cached_tokens`; optional `prompt_cache_key` (≤ 1024 chars) as a routing hint — docs warn *against* sharing one key across many users. **(probed: two identical 404-token prompts back-to-back both reported `cached_tokens: 0`** — cache warm-up is not instantaneous; don't promise hits.) No price change either way, so caching is purely a rate-limit/latency optimisation for PolyLLM.188- **Stateless**: no stored conversations, no `previous_response_id`, no ZDR flag. Docs policy page covers retention.189190## 14. Errors, rate limits, retries191192Error body: OpenAI-style flat object `{"message","type","param","code"}` (+ `"id": ""` on some). The OpenAI SDK maps it to `APIError` with `e.error` = that object.193194| Case **(probed)** | HTTP | body |195|---|---|---|196| Invalid key / no Authorization header | **401** | `{"message":"Wrong API Key","type":"invalid_request_error","param":"api_key","code":"wrong_api_key"}` |197| Unknown or retired model (`llama-3.3-70b`, `grok-99`) | **404** | `{"message":"Model does not exist or you do not have access to it.","type":"not_found_error","param":"model","code":"model_not_found"}` |198| Validation (unknown param, range, `n`, `top_k`, both max_*) | 400 | `type: invalid_request_error`, `code: wrong_api_format`, `param: validation_error` or the field |199| Missing `messages` | 400 | `messages: Field required` |200| Bad JSON / wrong type | 400 | `: Invalid JSON: …` / `messages: Input should be a valid array` |201| Context too long | 400 | `code: context_length_exceeded`, `param: messages` |202| Image on text model | 400 | `code: wrong_api_format`, `param: prompt` |203| Remote image URL | 400 | `code: invalid_multimodal_input`, `param: messages` |204| TPM bucket exhausted | **429** | `{"message":"Tokens per minute limit exceeded - too many tokens processed.","type":"too_many_tokens_error","param":"quota","code":"token_quota_exceeded"}` + `retry-after: 60` |205| Unsupported API version header | 400 | `code: wrong_api_format`, `param: api_version` |206207Docs also list 402 PaymentRequired (credits exhausted), 403, 413 ContentTooLarge (`image_too_large`), 422 (legacy; validation now 400 since 2026-04-24), 500, 503. Rate limits (docs): Free Trial 5 RPM / 30K uncached TPM / 90K total TPM / 1M TPH / 1M TPD per model; Developer: gpt-oss 1K RPM / 1M uncached / 3M total TPM, qwen 300 RPM / 150K / 450K, no hourly/daily caps; dual-bucket (uncached vs total = 3× uncached), token-bucket refill; 429 tells which bucket. Status page: https://status.cerebras.ai (docs).208209**Retry/timeout recommendation for the adapter:** retry 429/5xx with backoff honouring `retry-after` (60 s observed — for a chat UI show "rate limited, retry in 60 s" rather than blocking); never retry 400/401/404; 401 → *invalid credentials* state (clean, unlike xAI); 404 `model_not_found` → refresh the model list (Cerebras retires models often — 15 retirements since 2025). Short timeouts are fine (whole answers in < 1 s) but keep ≥ 60 s for 40k-token outputs and queueing.210211## 15. Lifecycle / aliases212213- **No aliases** (no `-latest`); ids are stable slugs. Retired ids return 404 immediately (no transparent redirect).214- Retirements (docs deprecation page): `gemma-4-31b` public 2026-09-03 (→ qwen-3.8-27b), `zai-glm-4.7` 2026-08-17, `llama3.1-8b` + `qwen-3-235b-a22b-instruct-2507` 2026-05-27, `qwen-3-32b` + `llama-3.3-70b` 2026-02-16, `zai-glm-4.6` 2026-01-20, `qwen-3-235b-a22b-thinking-2507` 2025-11-14, `qwen-3-coder-480b` 2025-11-05, llama-4 scout/maverick Oct–Nov 2025, `deepseek-r1-distill-llama-70b` 2025-08-12. Parameter deprecation: `disable_reasoning` 2026-07-21.215- `kimi-k2.7-code` appears in capability docs but is "customer trials only" (not in `/v1/models` for this key).216- Because `/v1/models` is the only truth and it lags the docs (gemma), the registry sync should **filter to the intersection** of `/v1/models` and our catalog, and show gemma with a "legacy / dedicated-only" badge if present.217218## 16. Exact streaming code that worked219220OpenAI SDK (`openai@7.10.0`) against Cerebras — reasoning + usage:221222```ts223import OpenAI from "openai";224const client = new OpenAI({ apiKey: process.env.CEREBRAS_API_KEY, baseURL: "https://api.cerebras.ai/v1", timeout: 120_000, maxRetries: 0 });225226const stream = await client.chat.completions.create({227  model: "gpt-oss-120b",228  messages: [{ role: "user", content: "Say hi." }],229  stream: true,                       // stream_options not needed: usage rides on the finish chunk230  max_completion_tokens: 1000,        // includes reasoning tokens!231  reasoning_effort: "low",            // gpt-oss: low|medium|high ; qwen/gemma: none|low|medium|high232} as any);233234for await (const chunk of stream as any) {235  const d = chunk.choices?.[0]?.delta ?? {};236  if (d.reasoning) onThinking(d.reasoning);         // Cerebras field name is `reasoning`237  if (d.content) onText(d.content);238  if (d.tool_calls) onToolCallDeltas(d.tool_calls); // gemma/qwen: complete in 1 chunk; gpt-oss: real deltas239  if (chunk.choices?.[0]?.finish_reason) onFinish(chunk.choices[0].finish_reason);240  if (chunk.usage) onUsage(chunk.usage, chunk.time_info);   // same chunk as finish_reason241}242```243244Raw fetch SSE (`research/cerebras/lib.ts` → `rawSSE`): POST JSON, read `res.body`, split on `\n\n`, parse `data:` lines, stop at `[DONE]` — identical parser to xAI.245246## 17. Probe results table247248| # | Probe | Model(s) | Result |249|---|---|---|---|250| 00 | `GET /models`, `/models/{id}`, unknown id, headers on a tiny call | — | 3 models (gpt-oss-120b, gemma-4-31b, qwen-3.8-27b), `created: 0`; unknown → 404; per-model `x-ratelimit-*-{minute,hour,day}` headers; Developer-tier limits |251| 01a | tiny chat completion `max_completion_tokens: 200` | 3 models | 200 in 124–399 ms; `message.reasoning` on qwen/gpt-oss, absent on gemma; `usage.completion_tokens_details.reasoning_tokens`; `time_info` → 1 216 / 1 393 / 2 631 tok/s |252| 01b | streaming ± `stream_options.include_usage` | 3 models | `delta.role` → `delta.reasoning`* → `delta.content`; `usage` + `time_info` on the `finish_reason` chunk in both cases; `[DONE]`; no `<think>` tags |253| 02 | param matrix (44 variants) | 3 models | see §9; unknown params & `top_k` 400; `n>1` 400; both max_* 400; `temperature > 2` 400; `reasoning_effort` per §8; `developer` 400 on qwen; `tools`+`response_format` 400 on gpt-oss; `min_tokens` 400 on gpt-oss; `service_tier` ignored |254| 03 | function call round trip, streaming | 3 models | gemma/qwen: single complete tool_call chunk; gpt-oss: name chunk + 2 argument deltas; 9-hex ids; `finish_reason: tool_calls`; round 2 answers correctly on all |255| 04 | `json_schema` strict (non-stream + stream), forbidden `pattern`, missing `additionalProperties` | 3 models | valid JSON on all; `pattern` silently accepted; missing `additionalProperties:false` → 400 `'additionalProperties' is required to be supplied and set to false.` |256| 05 | vision 32×32 PNG data URL; HTTPS URL; `detail` | 3 models | gemma 200 (`image_tokens 256`), qwen 200 (`64`), gpt-oss 400 "Content type 'image_url' is not supported"; HTTPS URL 400 `invalid_multimodal_input`; `detail` ignored |257| 06 | invalid key / no auth / unknown model / bad body / no messages / huge max / `/completions` / `/embeddings` / `/responses` / version header / SDK errors | — | 401 `wrong_api_key` / 401 / 404 `model_not_found` / 400 / 400 / 200 (cap ignored) / 200 / 404 / 404 / 400 (v1 unsupported) / `AuthenticationError` |258| 07 | gpt-oss json_schema stream, `reasoning_format: raw`, gemma reasoning stream, ~93k & ~186k-token prompts, `max_completion_tokens: 60000`, gpt-oss `high`, qwen cap 20 | mixed | valid; raw = reasoning glued to content (gpt-oss) / `…\n</think>\n\n…` (qwen) / 400 (gemma); gemma streams `delta.reasoning`; 93k OK on all, 186k → 400 `context_length_exceeded` limit 131000 (gpt-oss) / 131072 (qwen, gemma), qwen first hit 429 TPM; 60000 accepted; high → 758 reasoning tokens, 1 472 tok/s; cap 20 → `length`, no `content` |259| 08 | `json_object`+stream, qwen ctx retry, `logprobs` on reasoning models, `stop`, `seed` ×2, cache ×2, OpenAI SDK stream | mixed | json_object streams fine on all 3 (docs say no); qwen limit 131072; `logprobs` + `reasoning_logprobs`; `stop` honoured; seed deterministic; `cached_tokens` 0 on immediate repeat; OpenAI SDK sees `delta.reasoning` + usage |260261\* gemma only when `reasoning_effort` ≠ none.262263## Documentation pages used (all fetched 2026-09-08)264265- https://inference-docs.cerebras.ai/introduction · https://inference-docs.cerebras.ai/llms.txt (index used to find the real paths) · https://inference-docs.cerebras.ai/quickstart266- https://inference-docs.cerebras.ai/api-reference/chat-completions (+ `.md`) · https://inference-docs.cerebras.ai/api-reference/models · https://inference-docs.cerebras.ai/api-reference/models/retrieve-model.md · https://inference-docs.cerebras.ai/api-reference/versions.md267- https://inference-docs.cerebras.ai/models/overview · https://inference-docs.cerebras.ai/models/openai-oss · https://inference-docs.cerebras.ai/models/qwen-3.8-27b · https://inference-docs.cerebras.ai/models/choose-a-model.md · https://inference-docs.cerebras.ai/dedicated/overview.md268- https://inference-docs.cerebras.ai/capabilities/streaming · …/capabilities/tool-use · …/capabilities/structured-outputs · …/capabilities/reasoning · …/capabilities/image-inputs.md · …/capabilities/prompt-caching · …/capabilities/service-tiers.md269- https://inference-docs.cerebras.ai/support/rate-limits · https://inference-docs.cerebras.ai/support/error.md · https://inference-docs.cerebras.ai/support/change-log.md · https://inference-docs.cerebras.ai/support/deprecation.md270- https://inference-docs.cerebras.ai/resources/openai · https://www.cerebras.ai/pricing (tiers only, no per-model table; `/support/pricing` 307-redirects here) · https://registry.npmjs.org/@cerebras/cerebras_cloud_sdk/latest (1.91.0) · https://github.com/Cerebras/cerebras-cloud-sdk-node (README)271- 404 at audit time: `/support/error-codes`, `/support/error-handling`, `/resources/changelog`, `/models/deprecations`, `/models/gemma-4-31b` (+ `.md`), `/capabilities/vision`, `/capabilities/prompt-caching.md` was fine but `/support/pricing` is a redirect.272