# Cerebras Inference — provider research for PolyLLM

Last documentation audit: 2026-09-08 Probes executed 2026-09-08 with a real Developer-tier (pay-as-you-go) key against https://api.cerebras.ai/v1 (scripts in research/cerebras/, raw outputs in research/cerebras/out/). Everything marked (probed) was observed live; everything marked (docs) comes from the pages listed at the end.

Heads-up: GET /v1/models for this key returns 3 models: gpt-oss-120b, qwen-3.8-27b, gemma-4-31b. The changelog and deprecation page say gemma-4-31b was removed from public endpoints on 2026-09-03 ("remains on dedicated endpoints"), its model page is 404 and it has no public price — yet it is still listed and served normally for this key (all probes 200). Treat it as grandfathered / may disappear without notice; do not make it a default.

# 1. Base URL, auth, headers

# 2. SDK recommendation (TypeScript / Node)

• Official TS SDK @cerebras/cerebras_cloud_sdk@1.91.0 (Stainless-generated, Node ≥ 18, TS ≥ 4.5). Options apiKey, baseURL, timeout (default 1 min), maxRetries (default 2; retries 408/429/≥500 + connection errors), warmTCPConnection (opens a TCP connection at construction — set false in a server). Errors: APIError subclasses BadRequestError 400, AuthenticationError 401, PermissionDeniedError 403, NotFoundError 404, RateLimitError 429, InternalServerError ≥500, APIConnectionError. (probed: invalid key → AuthenticationError, .status 401, .message "401 Wrong API Key"). Its types know reasoning, time_info, reasoning_effort, reasoning_format, clear_thinking.
• OpenAI SDK also works unchanged (openai@7.10.0 probed) with baseURL: "https://api.cerebras.ai/v1": streaming, delta.reasoning, final usage chunk, tools, response_format all fine. Cerebras-only params (reasoning_format, clear_thinking) must go through extra_body/a cast (docs).
• Recommendation for PolyLLM: reuse the OpenAI-compatible chat-completions path (same as xAI) with baseURL swapped and a Cerebras translation layer; maxRetries: 0 (we do our own backoff), timeout 120 s is plenty (fastest provider we have: 1 200–2 600 tok/s). No need to add the Cerebras SDK unless we want its typed time_info.
• Vercel AI SDK: @ai-sdk/cerebras exists (docs integration page) — not needed.

# 3. Endpoints

# 4. Chat Completions request/response (probed shapes)

Request body fields (docs API ref + probes): model, messages, max_completion_tokens (alias max_tokens; sending both → 400 Setting "max_tokens" and "max_completion_tokens" at the same time is not supported.), min_completion_tokens/min_tokens, temperature 0–2 (2.5 → 400 temperature: Input should be less than or equal to 2), top_p 0–1, stop (≤4), seed, stream, stream_options (accepted; usage is sent anyway, see §5), response_format (text|json_object|json_schema), tools, tool_choice (none|auto|required|{type:"function",function:{name}}), parallel_tool_calls (default true), reasoning_effort (none|low|medium|high, model-dependent), reasoning_format (parsed|raw|hidden|none), clear_thinking (qwen only), logprobs/top_logprobs (0–20), n (only 1: n: 2 → 400 "n" > 1 is not currently supported), frequency_penalty/presence_penalty (−2..2, all models since 2026-03-31), logit_bias, user, service_tier (default|priority|auto|flex, dedicated only — silently accepted on shared, no service_tier echoed (probed)), prompt_cache_key (≤1024 chars), prediction (dedicated only).

Strict schema validation (v2): unknown top-level params are REJECTED, e.g. top_k → 400 top_k: property 'top_k' is unsupported, foo_bar → 400 foo_bar: property 'foo_bar' is unsupported (probed). Never forward unknown/experimental keys.

Roles: system, user, assistant, tool. developer role: OK on gpt-oss-120b and gemma-4-31b, 400 on qwen-3.8-27b (Failed to apply chat template to messages due to error: Unexpected message role.) (probed) → always send system. Content parts: {type:"text"} (all roles), {type:"image_url", image_url:{url}} (user only, data URI only — §10).

Non-streaming response (probed):

{
  "id": "chatcmpl-61794ede-…", "object": "chat.completion", "created": 1788849167, "model": "gpt-oss-120b",
  "system_fingerprint": "fp_b546658c8e93d2e57ef2",
  "choices": [{ "index": 0, "finish_reason": "stop",
    "message": { "role": "assistant", "content": "2 + 2 equals 4.",
                 "reasoning": "User asks: \"What is 2+2? …\" Just answer: …" } }],
  "usage": { "prompt_tokens": 80, "completion_tokens": 51, "total_tokens": 131,
             "prompt_tokens_details": { "cached_tokens": 0 },
             "completion_tokens_details": { "reasoning_tokens": 32,
                                            "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } },
  "time_info": { "queue_time": 0.004089397, "prompt_time": 0.00175769,
                 "completion_time": 0.019383306, "total_time": 0.026791095733642648, "created": 1788849167.0522857 }
}

• Reasoning is inside completion_tokens (51 = 19 visible + 32 reasoning); total_tokens = prompt + completion. Reasoning also counts toward max_completion_tokens (docs + probed: max_completion_tokens: 20 on qwen → finish_reason: "length", reasoning = 20 tokens, no content key at all). The adapter must treat a missing content as empty string, and the UI needs a "budget consumed by reasoning" hint.
• The reasoning field is absent (not null) when the model did not reason (gemma default, reasoning_effort: "none").
• time_info (seconds): use completion_tokens / completion_time for tokens/s. Observed: gemma ≈ 1 216 tok/s, qwen ≈ 1 393, gpt-oss ≈ 2 631 (high effort, 785 tokens: 1 472). queue_time occasionally 0.2–0.3 s on qwen.
• prompt_tokens_details.image_tokens present on gemma/qwen (0 without images); gpt-oss has none. gpt-oss adds accepted/rejected_prediction_tokens.
• finish_reason: stop, length, tool_calls (probed); docs also content_filter.
• No hidden system prompt to speak of: a one-line prompt = 20–26 prompt tokens on gemma, ~60 on qwen, ~75–80 on gpt-oss (harmony preamble).
• max_completion_tokens far above the documented cap (60 000, even 999 999) is accepted silently (probed) — the API does not error, it just caps output; the docs' 40k "max output" is a soft figure. Omit or clamp client-side.

# 5. Streaming protocol (probed)

SSE, Content-Type: text/event-stream, no event: field, data: {json} lines, terminated by data: [DONE].

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1788849167,"model":"gpt-oss-120b","system_fingerprint":"fp_…",
       "choices":[{"index":0,"delta":{"role":"assistant"}}]}
data: {"…","choices":[{"index":0,"delta":{"reasoning":"The"}}]}
data: {"…","choices":[{"index":0,"delta":{"reasoning":" user asks: \"Say hello"}}]}
…
data: {"…","choices":[{"index":0,"delta":{"content":"Bonjour"}}]}
data: {"…","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],
       "usage":{"prompt_tokens":77,"completion_tokens":152,"total_tokens":229,"prompt_tokens_details":{"cached_tokens":0},
                "completion_tokens_details":{"reasoning_tokens":136,…}},
       "time_info":{"queue_time":0.0033,"prompt_time":0.0017,"completion_time":0.1104,"total_time":0.1175,"created":1788849167.17}}
data: [DONE]

• Delta keys observed: role, reasoning, content, tool_calls. Reasoning streams as delta.reasoning (plain text) — not reasoning_content, no <think> tags in content (unless reasoning_format: "raw", §8).
• usage + time_info ride on the SAME chunk as finish_reason (no separate choices: [] chunk) and are present with or without stream_options.include_usage (probed on all 3). Standard OpenAI accumulation code works; just read chunk.usage whenever it is defined.
• Chunks are big (whole phrases per delta) because generation is so fast — 5–25 events per short answer; TTFB 107–221 ms.
• json_object + stream: true: docs say "not compatible… stream must be false", but it works on all 3 models (probed, valid JSON streamed). Keep the docs rule in mind if it starts failing.

# 6. Tool / function calling (probed on all 3 models)

• OpenAI nested format {type:"function", function:{name, description, parameters, strict?}}; tool_choice auto|required|none|{type:"function",function:{name}} all OK; parallel_tool_calls: false OK; strict: true OK (requires additionalProperties:false on every object; qwen forbids pattern/minLength/maxLength in strict tool schemas — docs).
• Round trip (assistant.tool_calls + {role:"tool", tool_call_id, content}) → 200 with a grounded answer on all 3. Tool-call ids are 9-hex-char strings ("c1ded86e6"), not call_….
• Streaming shape differs per model:
  • gemma-4-31b, qwen-3.8-27b: one chunk with the complete call {"tool_calls":[{"index":0,"id":"7f46ce3e9","type":"function","function":{"name":"get_weather","arguments":"{\"city\": \"Montreal\"}"}}]}.
  • gpt-oss-120b: true deltas — chunk 1 {id, type, index, function:{name, arguments:""}}, then {index, type, function:{arguments:"{\"city\":\""}}, {… arguments:"Montreal\"}"}. Standard index-keyed accumulation handles both.
  • then finish_reason: "tool_calls" (usage on that chunk).
• tools + response_format together: 400 on gpt-oss-120b "tools" is incompatible with "response_format" (param: "tools"); accepted on gemma/qwen (the model called the tool). Adapter: drop response_format when tools are present for gpt-oss, or refuse the combination.
• gpt-oss-120b docs warning: "may call tools that aren't directly specified due to its training" (hallucinated tool names) — validate the name before dispatch.
• v2 validation is strict on multi-turn tool history (docs): no orphan tool messages, every tool_call_id must match, unique ids.

# 7. Structured output (probed on all 3 models)

• response_format: {type:"json_schema", json_schema:{name, schema, strict:true}} → valid JSON matching the schema on all 3 (non-stream and stream, gpt-oss stream verified in probe 07). {type:"json_object"} works on all 3 (also streamed).
• strict: true without additionalProperties:false → 400 {"message":"'additionalProperties' is required to be supplied and set to false.","type":"invalid_request_error","param":"response_format","code":"wrong_api_format"} (probed all 3) — the adapter must inject additionalProperties:false recursively (same requirement as OpenAI strict mode).
• pattern (documented as prohibited) was accepted on all 3 — apparently ignored (output happened to match). Don't rely on it.
• Docs limits (strict): schema text ≤ 5 000 chars, depth ≤ 10, ≤ 500 properties, ≤ 500 enum values total; supported: primitive types, enum, const, anyOf, nested objects/arrays, required, $ref/$defs, tuple items, number bounds, multipleOf, annotations. Prohibited: recursion, external $ref, oneOf, allOf, not, if/then/else, patternProperties, unevaluatedProperties, pattern, format, minItems/maxItems. Output keys follow schema order.
• gpt-oss: min_tokens/min_completion_tokens → 400 Structured outputs is not currently supported with min_tokens even without response_format (probed) — never send min_tokens to gpt-oss.

# 8. Reasoning controls (probed matrix)

• disable_reasoning (deprecated 2026-07-21): still accepted on gemma/qwen (behaves like none), 400 on gpt-oss (same "Unsupported reasoning effort: none" message) (probed). Use reasoning_effort: "none" only.
• clear_thinking: true: OK on qwen only; gemma/gpt-oss → 400 "clear_thinking" is not supported for this model. When replaying history, the adapter should not send prior reasoning back (qwen keeps it unless clear_thinking; others ignore).
• Effort has little effect on trivial prompts (qwen 22→31→29 reasoning tokens for low/medium/high; gpt-oss 18/33/37); gpt-oss high on "count the r's" → 758 reasoning tokens, 785 completion in 0.53 s.
• No thinking budget parameter. Reasoning tokens are billed at the output rate and consume max_completion_tokens → for reasoning-on models set max_completion_tokens ≥ 1 000 or expect empty content with finish_reason: "length".
• logprobs: true with reasoning → choices[0].logprobs (content tokens) and choices[0].reasoning_logprobs (reasoning tokens, same {content:[{token, logprob, bytes, top_logprobs}]} shape) (probed).

# 9. Sampling & other parameters — support matrix (probed, chat completions, HTTP status)

Error body for every 400: {"message": "...", "type": "invalid_request_error", "param": "<field or validation_error>", "code": "wrong_api_format", "id": ""}. Adapter rules: strip any non-whitelisted key; never send top_k, n>1, min_tokens (gpt-oss), clear_thinking (non-qwen), developer role (map to system), reasoning_effort: "none" to gpt-oss, response_format with tools on gpt-oss, reasoning_format other than parsed.

# 10. Modalities, context, output limits

• Input: text on all; images on gemma-4-31b and qwen-3.8-27b (docs + probed OK: 32×32 PNG → gemma "Red", image_tokens: 256; qwen image_tokens: 64). gpt-oss-120b: 400 {"message":"Content type 'image_url' is not supported by selected model. Only 'text' content type can be used.","type":"invalid_request_error","param":"prompt","code":"wrong_api_format"}. Output: text only. No audio, no image generation, no files, no web search, no embeddings.
• Images: base64 data URI only (data:image/png;base64,… / JPEG); HTTPS URL → 400 {"message":"Remote image URLs are not supported; send images as data URIs (data:image/png;base64,...)","type":"invalid_request_error","param":"messages","code":"invalid_multimodal_input"} (probed) → PolyLLM must fetch/inline remote images itself. detail is documented unsupported but detail: "high" was silently accepted (probed) — strip it anyway. Limits (docs): ≤ 15 000 px per side, ≤ 10 MiB request payload (→ 413 image_too_large), 2 images/request on Free Trial, 10 on Developer/Enterprise, user messages only. Token formula: qwen 32×32 px/token (max 2 304), gemma 48×48 (max 280 — probe shows a 32×32 image is padded to 256 tokens). Image tokens are inside prompt_tokens and repeated in prompt_tokens_details.image_tokens.
• Context windows (docs, Free → Paid): gpt-oss-120b 65k → 131k; qwen-3.8-27b 64k → 128k (65 536 → 131 072); gemma-4-31b: no page — probed limit 131 072. Live errors with a ~186k-token prompt: gpt-oss Please reduce the length of the messages or completion. Current length is 186745 while limit is 131000 (code: "context_length_exceeded", param: "messages", 400); qwen/gemma same text with limit is 131072. A ~93k-token prompt succeeded on all 3 (this key is paid tier; free keys will be refused above ~65k). "Current length" = prompt + max_completion_tokens.
• Max output (docs): gpt-oss-120b 32k free / 40k paid; qwen-3.8-27b 32 768 free / 40 960 paid; gemma unknown (null). Not enforced as a 400 (§4).
• Free Trial vs Developer for this key: headers show Developer-tier limits; a 150K-TPM qwen bucket was exhausted by one 93k-token request + retry → 429 {"message":"Tokens per minute limit exceeded - too many tokens processed.","type":"too_many_tokens_error","param":"quota","code":"token_quota_exceeded"} with retry-after: 60 (probed). Large-context use on qwen is effectively rate-limited to ~1 request/min.

# 11. Server-side tools, search, citations

None. Cerebras has no web search, code interpreter, file search or MCP tool; only client-side function calling. No citations/annotations.

# 12. Model listing & pricing units

GET /v1/models gives ids only. Pricing is not in the API; the public pricing page (cerebras.ai/pricing) only states tiers (Free Trial $5 credits / 30 days, Developer from $10, Enterprise). Per-model prices come from the model doc pages (docs):

Prompt caching gives no price discount ("billed at the standard input token rate") — it only reduces latency and the uncached TPM bucket. Reasoning tokens billed as output. Batch API and service tiers have no differentiated pricing during preview (docs).

# 13. Prompt caching & provider-side state

• Automatic on all models, 128-token blocks (prefixes < 128 tokens may not be cached), TTL guaranteed 5 min (up to 1 h), reported in usage.prompt_tokens_details.cached_tokens; optional prompt_cache_key (≤ 1024 chars) as a routing hint — docs warn against sharing one key across many users. (probed: two identical 404-token prompts back-to-back both reported cached_tokens: 0 — cache warm-up is not instantaneous; don't promise hits.) No price change either way, so caching is purely a rate-limit/latency optimisation for PolyLLM.
• Stateless: no stored conversations, no previous_response_id, no ZDR flag. Docs policy page covers retention.

# 14. Errors, rate limits, retries

Error body: OpenAI-style flat object {"message","type","param","code"} (+ "id": "" on some). The OpenAI SDK maps it to APIError with e.error = that object.

Docs also list 402 PaymentRequired (credits exhausted), 403, 413 ContentTooLarge (image_too_large), 422 (legacy; validation now 400 since 2026-04-24), 500, 503. Rate limits (docs): Free Trial 5 RPM / 30K uncached TPM / 90K total TPM / 1M TPH / 1M TPD per model; Developer: gpt-oss 1K RPM / 1M uncached / 3M total TPM, qwen 300 RPM / 150K / 450K, no hourly/daily caps; dual-bucket (uncached vs total = 3× uncached), token-bucket refill; 429 tells which bucket. Status page: https://status.cerebras.ai (docs).

Retry/timeout recommendation for the adapter: retry 429/5xx with backoff honouring retry-after (60 s observed — for a chat UI show "rate limited, retry in 60 s" rather than blocking); never retry 400/401/404; 401 → invalid credentials state (clean, unlike xAI); 404 model_not_found → refresh the model list (Cerebras retires models often — 15 retirements since 2025). Short timeouts are fine (whole answers in < 1 s) but keep ≥ 60 s for 40k-token outputs and queueing.

# 15. Lifecycle / aliases

• No aliases (no -latest); ids are stable slugs. Retired ids return 404 immediately (no transparent redirect).
• Retirements (docs deprecation page): gemma-4-31b public 2026-09-03 (→ qwen-3.8-27b), zai-glm-4.7 2026-08-17, llama3.1-8b + qwen-3-235b-a22b-instruct-2507 2026-05-27, qwen-3-32b + llama-3.3-70b 2026-02-16, zai-glm-4.6 2026-01-20, qwen-3-235b-a22b-thinking-2507 2025-11-14, qwen-3-coder-480b 2025-11-05, llama-4 scout/maverick Oct–Nov 2025, deepseek-r1-distill-llama-70b 2025-08-12. Parameter deprecation: disable_reasoning 2026-07-21.
• kimi-k2.7-code appears in capability docs but is "customer trials only" (not in /v1/models for this key).
• Because /v1/models is the only truth and it lags the docs (gemma), the registry sync should filter to the intersection of /v1/models and our catalog, and show gemma with a "legacy / dedicated-only" badge if present.

# 16. Exact streaming code that worked

OpenAI SDK (openai@7.10.0) against Cerebras — reasoning + usage:

import OpenAI from "openai";
const client = new OpenAI({ apiKey: process.env.CEREBRAS_API_KEY, baseURL: "https://api.cerebras.ai/v1", timeout: 120_000, maxRetries: 0 });

const stream = await client.chat.completions.create({
  model: "gpt-oss-120b",
  messages: [{ role: "user", content: "Say hi." }],
  stream: true,                       // stream_options not needed: usage rides on the finish chunk
  max_completion_tokens: 1000,        // includes reasoning tokens!
  reasoning_effort: "low",            // gpt-oss: low|medium|high ; qwen/gemma: none|low|medium|high
} as any);

for await (const chunk of stream as any) {
  const d = chunk.choices?.[0]?.delta ?? {};
  if (d.reasoning) onThinking(d.reasoning);         // Cerebras field name is `reasoning`
  if (d.content) onText(d.content);
  if (d.tool_calls) onToolCallDeltas(d.tool_calls); // gemma/qwen: complete in 1 chunk; gpt-oss: real deltas
  if (chunk.choices?.[0]?.finish_reason) onFinish(chunk.choices[0].finish_reason);
  if (chunk.usage) onUsage(chunk.usage, chunk.time_info);   // same chunk as finish_reason
}

Raw fetch SSE (research/cerebras/lib.ts → rawSSE): POST JSON, read res.body, split on \n\n, parse data: lines, stop at [DONE] — identical parser to xAI.

# 17. Probe results table

* gemma only when reasoning_effort ≠ none.

# Documentation pages used (all fetched 2026-09-08)

• https://inference-docs.cerebras.ai/introduction · https://inference-docs.cerebras.ai/llms.txt (index used to find the real paths) · https://inference-docs.cerebras.ai/quickstart
• https://inference-docs.cerebras.ai/api-reference/chat-completions (+ .md) · https://inference-docs.cerebras.ai/api-reference/models · https://inference-docs.cerebras.ai/api-reference/models/retrieve-model.md · https://inference-docs.cerebras.ai/api-reference/versions.md
• https://inference-docs.cerebras.ai/models/overview · https://inference-docs.cerebras.ai/models/openai-oss · https://inference-docs.cerebras.ai/models/qwen-3.8-27b · https://inference-docs.cerebras.ai/models/choose-a-model.md · https://inference-docs.cerebras.ai/dedicated/overview.md
• https://inference-docs.cerebras.ai/capabilities/streaming · …/capabilities/tool-use · …/capabilities/structured-outputs · …/capabilities/reasoning · …/capabilities/image-inputs.md · …/capabilities/prompt-caching · …/capabilities/service-tiers.md
• https://inference-docs.cerebras.ai/support/rate-limits · https://inference-docs.cerebras.ai/support/error.md · https://inference-docs.cerebras.ai/support/change-log.md · https://inference-docs.cerebras.ai/support/deprecation.md
• https://inference-docs.cerebras.ai/resources/openai · https://www.cerebras.ai/pricing (tiers only, no per-model table; /support/pricing 307-redirects here) · https://registry.npmjs.org/@cerebras/cerebras_cloud_sdk/latest (1.91.0) · https://github.com/Cerebras/cerebras-cloud-sdk-node (README)
• 404 at audit time: /support/error-codes, /support/error-handling, /resources/changelog, /models/deprecations, /models/gemma-4-31b (+ .md), /capabilities/vision, /capabilities/prompt-caching.md was fine but /support/pricing is a redirect.