SPB Git forge

spb/polyllm

Public
15commits 1branches 0releases
2.2 MBsize
maindefault branch
13 days agolast push
TypeScript 97.4% SQL 1% JavaScript 0.9% CSS 0.6%
38.3 KB

# Mistral AI (La Plateforme) — provider research for PolyLLM

Last documentation audit: 2026-09-08 Probes executed 2026-09-08 with a real key against https://api.mistral.ai/v1 (scripts in research/mistral/*.ts, raw outputs in research/mistral/out/*.json). Everything marked (probed) was observed live; everything marked (docs) comes from the pages listed at the end.

Heads-up 1: the docs site was restructured (May 2026, "Vibe/Studio/Models/Admin"). Most URLs in the brief (/getting-started/models/models_overview/ still works, but /capabilities/structured-output/*, /capabilities/document/, /deployment/laplateforme/tier/, /models/deprecation) are 404. Current paths: /studio/conversations/*, /studio-api/conversations/advanced/*, /inference/*, /models/<card>, /admin/*, /api/endpoint/*. Several FAQ answers (vision limits, structured-output model list, predicted-output caveats) are collapsed accordions that WebFetch cannot expand — marked (docs, FAQ not extractable).

Heads-up 2: Magistral no longer exists as a model. magistral-medium-latest / magistral-small-latest are plain aliases of mistral-medium-2604 (Medium 3.5) and mistral-small-2603 (Small 4), which are hybrid models: reasoning is off by default and only enabled by reasoning_effort: "high". prompt_mode: "reasoning" is dead (400 everywhere). All magistral-*-25xx dated ids return 400.


# 1. Base URL, auth, headers

Item Value
REST base URL https://api.mistral.ai/v1 (docs also mention regional inference: EU/US endpoints via SDK server: "eu" | "us" at +10 % price; default global is fine for BYOK)
Auth Authorization: Bearer <MISTRAL_API_KEY>; no header → 401 {"detail":"Invalid API Key"} (probed)
Content type application/json; wrong types → 422 pydantic body ({"detail":[{"type":"list_type","loc":["body","messages"],"msg":"Input should be a valid list",...}]}) (probed)
Gateway Kong behind Cloudflare: response headers x-kong-request-id (use as request id), x-kong-proxy-latency, x-kong-upstream-latency, x-envoy-upstream-service-time, cf-ray (probed)
Rate-limit headers (probed, per model) x-ratelimit-limit-req-minute, x-ratelimit-remaining-req-minute, x-ratelimit-limit-tokens-minute, x-ratelimit-remaining-tokens-minute, x-ratelimit-tokens-query-cost (tokens this request consumed from the TPM budget — cached tokens are not counted: 3 276 → 28 on a cache hit). Present on inference calls only (not on /models, not on SSE). Limits differ per model (see §14).
Key introspection None (GET /v1/usage, /v1/tokenize → 404 (probed)). Cheapest "validate key" call is GET /v1/models (401 on a bad key (probed)).

# 2. SDK recommendation (TypeScript / Node)

  • Official SDK: @mistralai/mistralai 2.6.4 (npm, checked 2026-09-08; v2 is ESM-only). client.chat.complete(...) / client.chat.stream(...) (async iterable of {data: chunk}), client.fim.complete, client.models.list, client.beta.conversations.*, client.ocr.process, client.files.*. Params are camelCase (maxTokens, randomSeed, reasoningEffort, promptCacheKey) and responses are camelCased too (finishReason, usage.promptTokens) except usage.prompt_tokens_details which stays snake_case (probed) — a mapping trap. Errors: SDKError with statusCode, body (401 API error occurred: Status 401. Body: {"detail":"Invalid API Key"}) (probed); retry via retryConfig: {strategy: "backoff" | "none"}.
  • OpenAI SDK (openai@7.10.0) with baseURL: "https://api.mistral.ai/v1" works for non-streaming, streaming (incl. the array-typed delta.content thinking chunks, which the OpenAI types do not model — cast to any), and errors map to AuthenticationError 401 (probed). It is not documented as an official compatibility mode on docs.mistral.ai (the migration guide says "switch the client"); third-party guides call it wire-compatible. Constraints: Mistral rejects unknown fields with 422 (seed, max_completion_tokens, developer role, any extra key) — see §9 — so the adapter must translate rather than pass OpenAI bodies through.
  • Recommendation for PolyLLM: keep the current fetch-based SSE client (or OpenAI SDK) — the wire format is OpenAI-shaped and the deltas below are simple; use random_seed, max_tokens, system role only. If you want typed responses for thinking chunks and the Conversations API, the official SDK is the only typed option. Vercel @ai-sdk/mistral exists (not evaluated).

# 3. Endpoints

Endpoint Status Notes
POST /v1/chat/completions primary OpenAI-shaped; SSE streaming; tools; response_format; reasoning_effort; document_url / image_url / (docs) input_audio content parts; prediction; prompt_cache_key; service_tier; guardrails. (probed)
POST /v1/fim/completions active Fill-in-the-middle, Codestral only (FIM is not enabled for this model elsewhere) (probed)
GET /v1/models, GET /v1/models/{id} active Rich objects (§12). 50 entries incl. aliases as separate rows. (probed)
POST /v1/conversations (+/{id}, /{id}/restart, #stream, GET /{id}/history, /messages, DELETE) beta, working Agents & Conversations API — the only place for server-side tools (web_search, web_search_premium, code_interpreter, image_generation, document_library, MCP connectors). store:false honoured. (probed)
POST /v1/agents beta Persistent agent definitions (not needed for PolyLLM). (docs)
POST /v1/ocr active mistral-ocr-latest (= 4.1), $4 / 1 000 pages; not a chat model (400 Invalid model on chat) (probed)
POST /v1/audio/transcriptions, /audio/speech, /audio/voices, realtime WS active Voxtral Mini Transcribe 2 ($0.003/min), Voxtral TTS ($0.016 / 1k chars); out of scope. (docs)
POST /v1/embeddings active mistral-embed, codestral-embed (8 192 ctx).
POST /v1/moderations, /v1/chat/moderations active mistral-moderation-2603, free. (docs)
POST /v1/files, /v1/batch/jobs, fine-tuning, libraries, workflows, observability, admin API active Out of scope. Batch = −50 %. (docs)
Token counting none No tokenize endpoint (404 on 3 guesses) (probed). Use mistral-common tokenizer offline or estimate.

# 4. Chat Completions request/response (probed shapes)

Request fields (API ref + probes): model, messages (roles system, user, assistant, tool — developer → 422 union_tag_invalid), temperature 0–1.5, top_p 0–1, max_tokens (any int accepted; docs: "prompt + max_tokens cannot exceed context"; 10 M and 200 k > context were accepted and simply ignored (probed)), n (works on all but Large 3; "input tokens billed once"), stop (string or array, excluded from the output text (probed)), random_seed, frequency_penalty / presence_penalty [-2, 2], stream, stream_options (accepted, redundant — usage is always in the last chunk), safe_prompt, parallel_tool_calls, tools, tool_choice (auto | none | any | required | {type:"function",function:{name}} — all four string values + object probed OK), response_format (text | json_object | json_schema), reasoning_effort (enum none | minimal | low | medium | high | xhigh | max, model-gated §8), prompt_mode ("reasoning" — dead, 400 on every model), prompt_cache_key, prediction {type:"content", content}, service_tier (auto | standard_only), guardrails, metadata, assistant prefix: true (forces the reply to start with that text — probed: "Sure thing: **OK**! 😊").

Strict validation: any unknown top-level key → 422 {"object":"error","message":{"detail":[{"type":"extra_forbidden","loc":["body","foo_bar"],"msg":"Extra inputs are not permitted"}]},"type":"invalid_request_error"} (probed). So seed, max_completion_tokens, top_k(→400 top_k sampling is not enabled for this model), logprobs (→400 Logprobs are not enabled for this model) must never be sent.

Non-streaming response (probed):

json
{ "id": "c2a8…", "object": "chat.completion", "created": 1788849048, "model": "mistral-large-latest",
  "choices": [{ "index": 0, "finish_reason": "stop",
     "message": { "role": "assistant", "tool_calls": null, "content": "2 + 2 equals 4." } }],
  "usage": { "prompt_tokens": 16, "completion_tokens": 9, "total_tokens": 25,
             "prompt_tokens_details": { "cached_tokens": 0 }, "service_tier": "standard" } }
  • message.tool_calls is always present (null when none). message.content is a string normally and an array of chunks when reasoning is on (§8). Small 4 / Medium 3.5 sometimes return both content text and tool_calls in one message ("I'll get the current weather for Paris for you." + call) (probed).
  • finish_reason: stop, length (probed with max_tokens: 5, and when truncated inside thinking), tool_calls. No content_filter observed.
  • usage: no separate reasoning-token field — thinking tokens are inside completion_tokens. service_tier echoed (standard); GLM omits it. Docs mention prompt_audio_seconds for Voxtral.
  • Hidden system prompt: Small 4 / Medium 3.5 add ~12 prompt tokens (Say OK. = 6 tokens on Large/Ministral/Codestral/GLM, 18 on Small/Medium). With safe_prompt: true Small's count dropped to 6 (the injected safety prompt is not counted and seems to replace the hidden one) (probed).
  • model in the response echoes the alias you sent (mistral-large-latest), except retired slugs which echo the redirect target (§15).

# 5. Streaming protocol (probed)

SSE, Content-Type: text/event-stream; charset=utf-8, no event: field on chat completions, data: {json} lines, terminated by data: [DONE]. Every data chunk after the first carries a random-length "p": "abcdefghijklmnopq…" padding field (ignore it).

text
data: {"id":"…","object":"chat.completion.chunk","created":…,"model":"mistral-medium-latest","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}
data: {…,"choices":[{"index":0,"delta":{"content":"Bon"},"finish_reason":null}],"p":"abcdefghijklmnopqrstu"}
data: {…,"choices":[{"index":0,"delta":{"content":"jour !"},"finish_reason":"stop"}],"usage":{"prompt_tokens":25,"completion_tokens":4,"total_tokens":29,"prompt_tokens_details":{"cached_tokens":0},"service_tier":"standard"},"p":"abc"}
data: [DONE]
  • Delta keys: role, content, tool_calls. Usage arrives in the same chunk as finish_reason (no extra choices: [] chunk, stream_options.include_usage not needed). The last content delta can be non-empty in that chunk.
  • Reasoning stream (Medium 3.5 / Small 4 / GLM with reasoning_effort ≥ high): delta.content changes type during the stream — sequence observed "string" (role chunk, "") → [{type:"thinking", thinking:[{type:"text", text:"Let"}]}] × N → transition chunk [{type:"thinking", thinking:[{type:"text",text:"."}], closed:true}, {type:"text", text:"No."}] → "string" deltas for the rest of the answer → final chunk with finish_reason + usage. When the answer is cut during thinking, the closing chunk has only the closed:true thinking part and finish_reason: "length" (probed). Adapter: treat each thinking[].text as a thinking delta, {type:"text"} parts and plain strings as text deltas.
  • Tool calls: for Mistral models the whole call arrives in one chunk with complete arguments (§6). GLM 5.2 streams arguments across several chunks and adds delta.index and logprobs: null — use standard OpenAI accumulation by tool_calls[].index to cover both.
  • Conversations API streams do use event: names (§11). Docs: streaming connections time out after 10 min of inactivity.

# 6. Tool / function calling (probed on 7 models + GLM)

  • OpenAI nested format {type:"function", function:{name, description, parameters}}; max 128 tools per request (docs, known limitations); parallel_tool_calls default true; tool_choice auto | none | any | required | {type:"function",function:{name}}.
  • Streamed shape (Mistral models): single delta {"tool_calls":[{"id":"m82GbeK4G","type":"function","function":{"name":"get_weather","arguments":"{\"city\": \"Montreal\"}"},"index":0}]} then finish_reason: "tool_calls" + usage. Ids are 9 alphanumeric chars (m82GbeK4G, DxeWRMxcs…). GLM ids: chatcmpl-tool-ba59f9d9d6ffc20b, arguments chunked.
  • Round trip: {role:"assistant", content:"", tool_calls:[…]} then {role:"tool", tool_call_id, name, content} → 200 on all models. A long OpenAI-style id (call_abc123def456ghi789) in the replayed history was accepted (200) — the historical 9-char constraint is no longer enforced on input (probed), but keep ids verbatim anyway.
  • No strict flag on tools (docs). Citations from tool results: model can emit {type:"reference", reference_ids:[…]} content chunks when the tool result is a reference dictionary (docs, chat completions).

# 7. Structured output (probed on 7 models + GLM)

  • response_format: {type:"json_schema", json_schema:{name, schema, strict:true, description?}} → valid JSON matching the schema on all models incl. Codestral, Ministral 8B, GLM. minimum/maximum keywords accepted (200). Output is the JSON string in content (pretty-printed by some models — parse, don't display raw).
  • {type:"json_object"} works even without "JSON" in the prompt (probed on Small 4), though docs insist you must instruct the model to output JSON.
  • No documented list of unsupported schema keywords (docs, FAQ not extractable); SDK helper client.chat.parse() (Python) / Zod helpers in TS docs not surfaced. Docs: "Custom structured outputs are more reliable and are recommended".

# 8. Reasoning controls (probed matrix)

Model reasoning flag in /models Accepted reasoning_effort Default Shape
mistral-medium-2604 (Medium 3.5, magistral-medium-latest) true high, none only; others → 400 code 3051 reasoning_effort X is not supported for this model, supported values: [<ReasoningEffort.high: 'high'>, <ReasoningEffort.none: 'none'>] none (no thinking unless asked) content chunks
mistral-small-2603 (Small 4, magistral-small-latest) true high, none only (same error, slightly different wording reasoning_effort='low' is not supported …) none content chunks
zai-glm-5-2 true all 7 values (none, minimal → plain string; low…max → thinking) none content chunks
mistral-large-2512, ministral-*, codestral, voxtral-small false any value → 400 reasoning_effort is not enabled for this model — string
labs-leanstral-1-5 true not probed (403 Labs) — —
any model, bad value — 422 enum error listing 'none', 'minimal', 'low', 'medium', 'high', 'xhigh' or 'max'

Non-stream reasoning message (probed):

json
"content": [
  {"type":"thinking","thinking":[{"type":"text","text":"Let me think about this. A prime number is …"}],"closed":true},
  {"type":"text","text":"No, 221 is not a prime number (13 × 17)."}
]
  • No thinking budget, no summary control, no encrypted reasoning. Reasoning tokens are billed as output and counted in completion_tokens (Medium 3.5 spent 345 tokens on "is 221 prime"; max_tokens caps thinking + answer together → set a generous cap or you get finish_reason: "length" with only a thinking chunk).
  • Multi-turn: docs say to replay the full assistant message including the thinking chunk; replaying it verbatim worked (probed). Replaying an assistant message whose content is an empty string → 400 code 3240 Assistant message must have either content or tool_calls, but not none. (happens if you strip thinking from a truncated answer). Safe rule: replay the whole content array as received; when the model returned tool_calls with content: "" that is fine.
  • prompt_mode: "reasoning" (old Magistral switch) → 400 Reasoning prompt mode is not enabled for this model on every model — drop it.

# 9. Sampling & other parameters — support matrix (probed, chat completions)

Param Large 3 Medium 3.5 Small 4 Ministral 8B Codestral GLM 5.2
temperature 0–1.5 (2.0 → 422 less_than_equal … 1.5) ✓ ✓ ✓ ✓ ✓ ✓
top_p (1.5 → 422) ✓ ✓ ✓ ✓ ✓ ✓
top_k 400 400 400 400 400 —
max_tokens ✓ ✓ ✓ ✓ ✓ ✓
max_completion_tokens, seed, developer role, unknown keys 422 422 422 422 422 —
random_seed ✓ ✓ ✓ ✓ ✓ ✓
stop (string / array) ✓ ✓ ✓ ✓ ✓ ✓
presence_penalty / frequency_penalty ∈ [-2,2] (outside → 422) ✓ ✓ ✓ ✓ ✓ ✓
n: 2 400 Max allowed: 1 ✓ ✓ ✓ ✓ ✓
logprobs 400 400 400 400 400 —
safe_prompt ✓ ✓ ✓ ✓ ✓ ✓
reasoning_effort 400 high/none high/none 400 400 all
prompt_mode: "reasoning" 400 400 400 400 400 —
response_format json_object / json_schema ✓ ✓ ✓ ✓ ✓ ✓
tool_choice any/required/none/object, parallel_tool_calls ✓ ✓ ✓ ✓ ✓ ✓
prediction, prompt_cache_key, service_tier, metadata, stream_options, assistant prefix ✓ ✓ ✓ ✓ ✓ ✓
image_url ✓ ✓ ✓ ✓ 400 400
document_url (PDF) (docs ✓) (docs ✓) ✓ ✓ ✓ (docs ✓)

Adapter rules: clamp temperature to [0, 1.5]; map seed → random_seed; map maxTokens → max_tokens (never max_completion_tokens); map developer → system; never send top_k, logprobs, stream_options(harmless but useless), prompt_mode; send reasoning_effort only when model.capabilities.reasoning and only high/none for Mistral hybrids (all values for GLM); send n > 1 never to Large 3.

# 10. Modalities, context, output limits

  • Input: text everywhere; images on Large 3, Medium 3.5, Small 4, Ministral 3/8/14B (and Leanstral per listing) — capabilities.vision in /models is authoritative (probed 4/4 vision models OK, Codestral & GLM → 400 code 3051 Image input is not enabled for this model). Audio input only on voxtral-small-* (input_audio part, docs). Output: text only (image generation exists only as a Conversations-API tool, $100 / 1k images).
  • Image format: {type:"image_url", image_url:"<https url or data:image/png;base64,…>"} — string form (docs) and OpenAI object form {url, detail} both accepted (probed). Accepted formats (from the error text): JPEG, PNG, WEBP, GIF, MPO, HEIF, AVIF, BMP, TIFF; invalid data → 400 code 3310 invalid_request_file. No minimum size (2×2 PNG accepted). Token cost: 32×32 ≈ 15 tokens, 512×512 ≈ 390 tokens on Small 4 (≈ 28×28 px per token); billed at the text input price (no separate image price). Limits (docs, known-limitations page): 20 MB per image, resolution > 10 000×10 000 rejected (older FAQ: 8 images / request, 10 MB) — FAQ answers not extractable, treat 8 images as the safe cap.
  • Documents: {type:"document_url", document_url:"https://…pdf" | "data:application/pdf;base64,…", document_name?} → server-side OCR then QnA; probed OK on Small 4, Ministral 8B and Codestral; usage shows only ~50 prompt tokens for a 1-page PDF (OCR cost is not in usage — pricing of Document QnA unclear, docs say OCR $4/1k pages). {type:"file", file_id} also valid (bogus id → 400 3310 File … could not be found or may have expired). Limits (docs, FAQ not extractable).
  • Context: from /models.max_context_length — 262 144 (Large 3, Medium 3.5, Small 4, Ministral 8B/14B, Leanstral), 256 000 (Codestral), 131 072 (Ministral 3B), 1 048 576 (GLM 5.2), 32 768 (Voxtral Small). Exceeding it → 400 (docs). Max output: not published per model except GLM (128k, model card); max_tokens above context is accepted silently (probed) → registry maxOutputTokens: null, cap in UI at context − prompt.

# 11. Server-side tools, Conversations API, citations (probed)

  • On /v1/chat/completions: tools:[{type:"web_search"}] → 400 code 1800 {"type":"invalid_tools","message":"WebSearchTool connector is not supported"}. Docs: web_search, web_search_premium, code_interpreter are Conversations/Agents-API only; image_generation is claimed to work on chat completions (not probed, $0.10/image).
  • POST /v1/conversations {model, inputs:"…", tools:[{type:"web_search"}], store:false, completion_args:{max_tokens, temperature, response_format, …}} → 200 (probed):
json
{ "object":"conversation.response", "conversation_id":"conv_01a0…",
  "outputs":[
    {"object":"entry","type":"tool.execution","name":"web_search","arguments":"{\"query\": \"…\"}","info":{"result":"{\"D5PmwtGh\": {\"url\":…,\"title\":…,\"snippets\":[…]}}"},"id":"tool_exec_…","created_at":…,"completed_at":…},
    {"object":"entry","type":"message.output","role":"assistant","id":"msg_…","content":[
       {"type":"text","text":"The latest stable Node.js version is **26.8.1**…"},
       {"type":"tool_reference","tool":"web_search","title":"…","url":"https://versionlog.com/nodejs/","favicon":"https://imgs.search.brave.com/…","description":"…"},
       {"type":"text","text":"."}]}],
  "usage":{"prompt_tokens":779,"completion_tokens":66,"total_tokens":6582,"connector_tokens":5737,"connectors":{"web_search":1}},
  "guardrails":null }

Citations = inline tool_reference chunks (title, url, favicon, description) placed where the model cites; the raw search results sit in tool.execution.info.result (Brave-backed). Billing trap: connector_tokens (5 737 here, the search results fed to the model) are billed as input tokens on top of the $30 / 1k web_search calls.

  • Streaming (stream:true): SSE with event: names — conversation.response.started {conversation_id}, message.output.delta {output_index, id, content_index, role, content:"Hi"} (content is a plain string per delta; tool_reference deltas presumably arrive as chunk objects — not observed), conversation.response.done {usage}; docs also list tool.execution.started/done, function.call.delta, agent.handoff.*, conversation.response.error.
  • Client function tools, handoff_execution, instructions, previous-style continuation (POST /v1/conversations/{id}), restart, history GET/DELETE exist (docs). store defaults to true — send store:false for BYOK privacy.
  • Pricing (docs): web search $30 / 1k calls, premium news $50 / 1k, code interpreter $30 / 1k, image generation $100 / 1k, document library $0.01 / call + OCR/indexing.

# 12. Model listing & pricing (probed + docs)

GET /v1/models → {object:"list", data:[…]} with 50 rows (aliases are separate rows). Fields: id, object:"model", created (= request time, useless), owned_by ("mistralai"), name (canonical, e.g. "mistral-large-2512"), description, max_context_length, aliases[], deprecation (null everywhere today), deprecation_replacement_model, default_model_temperature (0.3 most; **1** for Medium 3.5 & Leanstral; 0.2 Voxtral; null GLM), billing_model_name (canonical billing id — use it to dedupe), type ("base"), capabilities {completion_chat, function_calling, reasoning, completion_fim, fine_tuning, vision, ocr, classification, moderation, audio, audio_transcription, audio_transcription_realtime, audio_speech, unified_resources}. No pricing, no max output in the API. capabilities.audio is false even for voxtral-small (looks wrong vs docs). Docs' /models schema (root, archived, job) is outdated.

Dedupe by billing_model_name → 10 chat-capable models (the 10 entries in mistral.models.json):

billing id display ctx in / cached / out $/M (docs pricing page) notes
mistral-medium-3-5 (mistral-medium-2604) Mistral Medium 3.5 256k 1.50 / 0.15 / 7.50 hybrid reasoning; 9 aliases incl. magistral-medium-latest
mistral-small-2603 Mistral Small 4 256k 0.15 / 0.015 / 0.60 hybrid reasoning; magistral-small-latest
mistral-large-2512 Mistral Large 3 256k 0.50 / 0.05 / 1.50 no reasoning, n≤1
ministral-14b-2512 / 8b / 3b Ministral 3 256k / 256k / 128k 0.20/0.02/0.20 · 0.15/0.015/0.15 · 0.10/0.01/0.10 vision, tools
codestral-2508 Codestral 256k (card says 128k) 0.30 / 0.03 / 0.90 FIM, no vision
zai-glm-5-2 Z.ai GLM 5.2 1M 1.40 / 0.14 / 4.40 third-party preview
voxtral-small-2507 Voxtral Small 32k 0.10 / ? / 0.40 audio in
labs-leanstral-1-5 Leanstral 1.5 256k free 403 unless org opts in; retiring 2026-09-30

Cached input = 10 % of input everywhere. Batch −50 %. Priority tier ×1.75. Regional +10 %. Non-chat: OCR 4.1 $4 / 1k pages, Voxtral Mini Transcribe 2 $0.003/min, TTS $0.016 / 1k chars, embeddings $0.10–0.15, moderation free.

# 13. Prompt caching & provider-side state

  • Explicit, key-based: send the same prompt_cache_key (session/conversation id) on requests sharing a prefix; 64-token blocks (cached_tokens is a multiple of 64, prompts < 64 tokens never hit); billed at 10 %; reported in usage.prompt_tokens_details.cached_tokens; no TTL documented (docs). Probed: identical 3 273-token prefix — call 1 cached_tokens: 0, call 2 (same key) cached_tokens: 3248, call 3 (no key) 0; the rate-limit header x-ratelimit-tokens-query-cost went 3 276 → 28, so cache hits also spare TPM. Adapter: always send prompt_cache_key = conversationId.
  • Chat completions are stateless. The Conversations API stores by default (store:true) — pass store:false.

# 14. Errors, rate limits, retries

Two error envelopes (probed):

  1. Business errors: {"object":"error","message":"…","type":"<type>","param":null,"code":"<numeric string>","raw_status_code":400}.
  2. Validation errors (422) and auth: {"detail": …} (pydantic list, or "Invalid API Key"); some 422s are wrapped in envelope 1 with message.detail[].
Case (probed) HTTP type / code message
Invalid or missing key (any endpoint) 401 — {"detail":"Invalid API Key"}
Unknown / retired / non-chat model 400 invalid_model / 1500 Invalid model: mistral-99-ultra
Unsupported feature for model 400 invalid_request_invalid_args / 3051 reasoning_effort is not enabled for this model, top_k sampling is not enabled…, Image input is not enabled…, FIM is not enabled…, Logprobs are not enabled…, Invalid value 2 for parameter \n`. Max allowed: 1.`
Bad range / unknown field / bad enum / bad role 422 invalid_request_error / null pydantic detail[] (less_than_equal, extra_forbidden, enum, union_tag_invalid)
Empty messages 400 invalid_request_message_order / 3230 Conversation must have at least one message
Assistant message without content/tool_calls 400 invalid_request_assistant_message / 3240
Bad image / missing file 400 invalid_request_file / 3310 lists allowed formats
Server tool on chat completions 400 invalid_tools / 1800 WebSearchTool connector is not supported
Labs model not enabled 403 — Model labs-leanstral-1-5 is a Labs model. To use Labs models, an admin must enable them…
Rate limit 429 (docs) Too Many Requests; no retry-after observed (none triggered)

Rate limits (docs + headers): scoped per workspace, per model, expressed as requests/min + tokens/min headers (docs talk RPS + tokens/month). Tiers by cumulative spend: Free → Tier 1 (pay-as-you-go on) → Tier 2 (> $20) → Tier 3 (> $100) → Tier 4 (> $500) → custom. Observed on this key (req/min · tokens/min): Large 3 75 · 1M (!), Medium 3.5 3000 · 2M, Small 4 2000 · 2M, Ministral 8B 3800 · 2.5M, 14B 500 · 3.8M, 3B 15000 · 5M, Codestral 2500 · 2.5M, GLM 3000 · 4M, Voxtral Small 1440 · 500k. Users see theirs at admin.mistral.ai/plateforme/limits.

Retry/timeout recommendation: retry 429 / 5xx with jittered exponential backoff (respect retry-after if present), never retry 400/401/403/422; surface x-kong-request-id in error toasts; read timeout ≥ 5 min for reasoning_effort: "high" (thinking deltas arrive quickly as liveness). Map 401 Invalid API Key → invalid credentials; map 403 Labs → model not enabled for this workspace.

# 15. Lifecycle / aliases / deprecations

  • Policy (docs /inference/model-lifecycle): notice before retirement = 6 months GA, 1 month Labs / Public Preview / third-party; retired ids "fail with 404" (actually 400 invalid_model (probed)). -latest aliases move automatically to the next GA model ("silent updates in model behaviour and pricing") → pin dated ids in the registry, show -latest as aliases.
  • deprecation / deprecation_replacement_model are null for every current model (probed); the docs table (models overview) lists retired ones: mistral-medium-2508 (ret. 2026-08-31), mistral-small-2506 (2026-07-31), magistral-medium-2509 / magistral-small-2509 (2026-07-31), devstral-2512 (2026-07-31), mistral-large-2411, pixtral-large-2411, devstral-*-2507 (2026-05-31), open-mistral-nemo-2407 (2026-07-31), labs-leanstral-2603 (2026-06-30), mistral-moderation-2411, voxtral-mini-2507, older 2024/25 models.
  • Silent redirects (probed): mistral-small-2506 → served as mistral-small-latest; mistral-medium-2508 / -2505 → mistral-medium-3-5; pixtral-12b-2409 → ministral-14b-latest; open-mistral-7b, ministral-8b-2410, open-mistral-nemo → Ministral 8B; codestral-2501 → codestral-latest; mistral-small-2501, mistral-saba-latest → Small 4; mistral-large-2407 → Large 3. Hard 400: magistral-*-25xx, mistral-large-2411, pixtral-large-latest, devstral-medium-2507. Don't list any of these; if a user types one, show the redirect from response.model.
  • Alias sprawl worth knowing: mistral-medium, mistral-medium-3, mistral-medium-3.5, mistral-vibe-cli-latest, mistral-vibe-cli-with-tools (Vibe CLI product aliases) → Medium 3.5; mistral-vibe-cli-fast → Small 4; mistral-code-latest, mistral-code-fim-latest → Codestral; glm-5-2 ↔ zai-glm-5-2.
  • Changelog 2026: Medium 3.5 GA 2026-04-28; Small 4 (2603); GLM 5.2 preview 2026-08-06; OCR 4.1 GA Aug 2026; Leanstral 1.5 June 2026 (retire 2026-09-30); docs moved to /vibe/*, /studio/* May 2026.

# 16. Exact streaming code that worked

Raw fetch SSE (research/mistral/lib.ts → rawSSE) and the OpenAI SDK both worked; the reasoning-aware accumulation used in 08-sdks-cache.ts:

ts
import OpenAI from "openai";
const client = new OpenAI({ apiKey: process.env.MISTRAL_API_KEY, baseURL: "https://api.mistral.ai/v1", timeout: 300_000, maxRetries: 0 });

const stream = await client.chat.completions.create({
  model: "mistral-medium-2604",
  messages: [{ role: "system", content: "Be concise." }, { role: "user", content: "Is 17 prime? One sentence." }],
  stream: true,
  max_tokens: 400,                       // covers thinking + answer
  random_seed: 42,                       // NOT `seed` (422)
  prompt_cache_key: conversationId,      // enables prefix caching
  reasoning_effort: "high",              // only "high" | "none" on Mistral hybrids; omit on Large/Ministral/Codestral
} as any);

for await (const chunk of stream as any) {
  const choice = chunk.choices?.[0];
  const c = choice?.delta?.content;
  if (typeof c === "string") { if (c) onText(c); }
  else if (Array.isArray(c)) {
    for (const part of c) {
      if (part.type === "thinking") for (const t of part.thinking ?? []) if (t.text) onThinking(t.text);
      else if (part.type === "text" && part.text) onText(part.text);
    }
  }
  if (choice?.delta?.tool_calls) accumulateByIndex(choice.delta.tool_calls); // 1 chunk on Mistral, many on GLM
  if (choice?.finish_reason) onFinish(choice.finish_reason);
  if (chunk.usage) onUsage(chunk.usage);  // same chunk as finish_reason; usage.prompt_tokens_details.cached_tokens
}

Official SDK equivalent: for await (const ev of await mistral.chat.stream({ model, messages, maxTokens, reasoningEffort: "high", promptCacheKey })) { const d = ev.data.choices[0].delta; … ev.data.choices[0].finishReason; ev.data.usage?.promptTokens } (camelCase, except prompt_tokens_details).

# 17. Probe results table

# Probe Model(s) Result
00 GET /models, /models/mistral-large-latest — 200; 50 rows; fields incl. capabilities.reasoning, billing_model_name, default_model_temperature, deprecation (all null); no pricing/max output
01 tiny chat + raw SSE stream 7 models all 200; string content; no thinking by default on magistral-*; usage in final chunk; p padding field; per-model rate-limit headers
02 param matrix (45 variants) 7 models §9: temperature ≤ 1.5, penalties ∈ [-2,2], unknown keys 422, top_k/logprobs/prompt_mode 400, n 400 on Large only, reasoning_effort high/none on hybrids, developer 422, prefix works
03 tools streamed + round trip; long tool_call_id 7 models single-chunk tool call, 9-char ids, finish_reason: tool_calls, round 2 OK; long id accepted
04 json_schema strict; min/max keywords; json_object w/o "JSON" 7 models valid JSON on all; extras accepted
05 vision 32 px / 2 px / 512 px / object form / invalid 4 vision models + Codestral + GLM OK on vision models (~15 / ~390 tokens); Codestral & GLM 400 3051; invalid → 400 3310 with format list
06 invalid key, no auth, unknown model, 17 retired slugs, malformed body, empty messages, huge max_tokens, temp 3, top_p 1.5, embed/OCR on chat — 401 Invalid API Key; 400 1500; redirects vs hard 400 per §15; 422 pydantic; 400 3230; 200 (ignored); 422; 422; 400 1500
07 document_url base64 + public URL, file type Small 4, Ministral 8B, Codestral 200 and correct answer ("pamplemousse") on all incl. Codestral; ~50 prompt tokens; bogus file id → 400 3310
08 OpenAI SDK non-stream/stream/bad key; Mistral SDK complete/stream/bad key; caching ×3 Small 4, Medium 3.5 all OK; AuthenticationError 401 / SDKError 401; cached_tokens 0 → 3248 → 0, TPM cost 3276 → 28
09 reasoning high non-stream + stream + replay + text-only replay; prompt_mode; truncation; Labs/3B/14B/Voxtral; FIM; web_search on chat; Conversations (web_search, stream); tokenize/usage Medium 3.5, Small 4, GLM, others shapes §8/§5; replay OK; empty-content replay 400 3240; prompt_mode 400; Labs 403; FIM Codestral-only; chat web_search 400 1800; Conversations 200 with tool_reference + connector_tokens; no tokenize/usage endpoints
10–11 GLM param matrix + stream + tools; max on Medium; hidden prompt sizes; stop exclusion; max_tokens > ctx GLM, Medium, Small, Ministral 3B GLM accepts all efforts (none/minimal → string), multi-chunk tool args, chatcmpl-tool- ids; Medium max 400; Small/Medium +12 hidden tokens; stop excluded; over-context max_tokens accepted

# Documentation pages used (all fetched 2026-09-08)