ORIX Astra — Research log
Research date: 2026-11-21. Every fact below was read on that date from the URL cited next to it, unless it is explicitly marked NOT verified.
Method and conventions:
- "verified on 2026-11-21" means the statement was read in the cited official page (or returned by the cited public API) on that date. Official pages were fetched as raw Markdown/JSON (
.mdvariants of the Anthropic, OpenAI and OpenRouter docs; Apple's DocC JSON athttps://developer.apple.com/tutorials/data/documentation/<path>.json; GitHub and Hugging Face public APIs). - "verified live" means it was observed in a real HTTP response to an unauthenticated request (no key was ever sent anywhere).
- "NOT verified" means I did not find it in an official source during this session. Nothing in this file is filled in from memory.
- No authenticated call was made, so no streaming response was observed on the wire. All SSE payloads below are the documented examples, not captures.
- Apple's developer documentation pages are JS-rendered; the HTML was not used. The DocC JSON endpoint above returned full content for every page cited in sections 4, 5 and 7.
1. OpenAI
Docs base: https://developers.openai.com/api/docs/… (platform.openai.com/docs/… redirects there). Append .md to any page for Markdown.
1.1 Recommended API
- The Responses API (
POST https://api.openai.com/v1/responses) is the recommended API: "If you're building any text generation app, we recommend using the Responses API over the older Chat Completions API." Reasoning models "perform better … when used with the Responses API". — https://developers.openai.com/api/docs/guides/text.md — verified on 2026-11-21 - For the current flagship family, Chat Completions is still supported but tool calling requires Responses ("GPT-6.2 Astra and GPT-6.2 Sol support Chat Completions, but tool calling requires Responses"). — https://developers.openai.com/api/docs/guides/latest-model.md — verified on 2026-11-21
- Auth header:
Authorization: Bearer <key>; optionalOpenAI-Organization,OpenAI-Project; optional client-suppliedX-Client-Request-Id(ASCII, ≤ 512 chars, else HTTP 400). — https://developers.openai.com/api/reference/overview.md — verified on 2026-11-21
1.2 Request shape (POST /v1/responses)
Top-level parameters, from the reference (https://developers.openai.com/api/reference/resources/responses/methods/create.md — verified on 2026-11-21):
| Parameter | Notes (as documented) |
|---|---|
model |
e.g. gpt-6.2-astra |
input |
string, or array of input items / messages |
instructions |
string; "A system (or developer) message inserted into the model's context" |
stream |
boolean, default false |
stream_options |
{ include_obfuscation } only. Obfuscation is on by default (adds an obfuscation field to delta events); set false to save bandwidth |
max_output_tokens |
upper bound "including visible output tokens and reasoning tokens" |
temperature |
number 0–2, default 1 (rejected by current flagship models, see 1.7) |
top_p |
default 1 (same caveat) |
reasoning |
{ effort, summary, context, mode } (see 1.6) |
tools, tool_choice, parallel_tool_calls (default true), max_tool_calls |
|
store |
boolean, default true |
previous_response_id, conversation |
server-side conversation state |
include |
extra output data |
truncation |
"auto" or "disabled" (default "disabled") |
text |
output format config (plain text / JSON / structured outputs) |
prompt_cache_key, prompt_cache_options (gpt-5.6 and later), safety_identifier, service_tier, metadata, background |
|
user |
deprecated, replaced by safety_identifier and prompt_cache_key |
prompt_cache_retention |
deprecated, use prompt_cache_options.ttl |
Message roles accepted in input: user, assistant, system, developer. Assistant messages may carry phase: "commentary" | "final_answer". — same reference page — verified on 2026-11-21
Minimal streaming request, as documented (same reference page — verified on 2026-11-21):
{
"model": "gpt-6.2-astra",
"instructions": "You are a helpful assistant.",
"input": "Hello!",
"stream": true
}Multi-turn with a message array, as documented in the streaming guide (https://developers.openai.com/api/docs/guides/streaming-responses.md — verified on 2026-11-21):
{ "model": "gpt-6.2-astra", "input": [ { "role": "user", "content": "…" } ], "stream": true }Behaviour change that matters for instructions: "Models starting with GPT-6.2 append top-level instructions after input in the Responses API. To make reusable instructions eligible for prompt caching, put them in a developer message as the first item in input." — https://developers.openai.com/api/docs/guides/latest-model.md — verified on 2026-11-21
Stateless multi-turn (what a BYOK client that sets store: false must do): "When you create a response in stateless mode, reasoning items in the response's output array include an encrypted_content property by default. Stateless mode applies when store is false or when your organization uses Zero Data Retention." Pass the prior output items back in order, including encrypted reasoning items. Also: "Starting with GPT-6.2, the Responses API drops reasoning items after mid-conversation edits to the conversation history (prefix edits)". — https://developers.openai.com/api/docs/guides/reasoning.md and https://developers.openai.com/api/docs/guides/latest-model.md — verified on 2026-11-21
1.3 Image input (base64)
Content parts are input_text and input_image; a base64 image is sent as a data URL in image_url, with optional detail ("auto" shown):
{
"role": "user",
"content": [
{ "type": "input_text", "text": "what's in this image?" },
{ "type": "input_image", "image_url": "data:image/jpeg;base64,<BASE64>", "detail": "auto" }
]
}— https://developers.openai.com/api/docs/guides/images-vision.md — verified on 2026-11-21
- Accepted image MIME types, maximum image size, maximum images per request, and the full list of
detailvalues: NOT verified (not extracted from the page during this session).
1.4 Streaming (SSE)
Wire format, from the documented example response (https://developers.openai.com/api/reference/resources/responses/methods/create.md — verified on 2026-11-21): each event is an event: <type> line followed by a data: <json> line whose JSON repeats the same value in "type" and carries a "sequence_number". SSE comment lines (starting with :) appear in the example.
- Whether a Responses stream ends with a
data: [DONE]sentinel: NOT verified. The documented example ends atresponse.completedand shows no[DONE]. Treatresponse.completed/response.incomplete/response.failed/erroras terminal and tolerate[DONE].
Documented order for a plain text answer (same page): response.created → response.in_progress → response.output_item.added → response.content_part.added → response.output_text.delta (repeated) → response.output_text.done → response.content_part.done → response.output_item.done → response.completed.
Complete list of event type values present in the streaming reference (https://developers.openai.com/api/reference/resources/responses/streaming-events.md — verified on 2026-11-21):
- Lifecycle:
response.created,response.queued,response.in_progress,response.completed,response.incomplete,response.failed,error - Structure:
response.output_item.added,response.output_item.done,response.content_part.added,response.content_part.done - Text:
response.output_text.delta,response.output_text.done,response.output_text.annotation.added,response.refusal.delta,response.refusal.done - Reasoning:
response.reasoning_summary_part.added,response.reasoning_summary_part.done,response.reasoning_summary_text.delta,response.reasoning_summary_text.done,response.reasoning_text.delta,response.reasoning_text.done - Function tools:
response.function_call_arguments.delta,response.function_call_arguments.done,response.custom_tool_call_input.delta,response.custom_tool_call_input.done - Hosted tools:
response.web_search_call.{in_progress,searching,completed},response.file_search_call.{in_progress,searching,completed},response.code_interpreter_call.{in_progress,interpreting,completed},response.code_interpreter_call_code.{delta,done},response.image_generation_call.{in_progress,generating,partial_image,completed},response.mcp_call.{in_progress,completed,failed},response.mcp_call_arguments.{delta,done},response.mcp_list_tools.{in_progress,completed,failed},response.shell_call_command.{added,delta,done},response.shell_call_output_content.{delta,done} - Other:
response.compaction.compacting,response.audio.delta,response.audio.done,response.audio.transcript.delta,response.audio.transcript.done
Documented payload examples (same page — verified on 2026-11-21):
{"type":"response.output_text.delta","item_id":"msg_123","output_index":0,"content_index":0,"delta":"In","sequence_number":1,"logprobs":[]}
{"type":"response.output_text.done","item_id":"msg_123","output_index":0,"content_index":0,"text":"…","sequence_number":1,"logprobs":[]}
{"type":"response.output_item.added","output_index":0,"item":{"id":"msg_123","status":"in_progress","type":"message","role":"assistant","content":[]},"sequence_number":1}
{"type":"response.reasoning_summary_part.added","item_id":"rs_…","output_index":0,"summary_index":0,"part":{"type":"summary_text","text":""},"sequence_number":1}
{"type":"response.reasoning_summary_text.delta","item_id":"rs_…","output_index":0,"summary_index":0,"delta":"…","sequence_number":1}
{"type":"response.reasoning_text.delta","item_id":"rs_123","output_index":0,"content_index":0,"delta":"The","sequence_number":1}
{"type":"response.function_call_arguments.delta","item_id":"item-abc","output_index":0,"delta":"{ \"arg\":","sequence_number":1}
{"type":"response.function_call_arguments.done","item_id":"item-abc","output_index":1,"arguments":"{ \"arg\": 123 }","sequence_number":1}
{"type":"response.refusal.delta","item_id":"msg_123","output_index":0,"content_index":0,"delta":"refusal text so far","sequence_number":1}
{"type":"error","code":"ERR_SOMETHING","message":"Something went wrong","param":null,"sequence_number":1}- The stream
errorevent is flat:code(string or null),message,param(string or null),sequence_number,type: "error". It is not wrapped in anerrorobject. — same page — verified on 2026-11-21 response.failedcarries the full response withstatus: "failed"andresponse.error: { "code": "server_error", "message": "…" }. — same page — verified on 2026-11-21response.incompletecarriesresponse.incomplete_details: { "reason": "max_output_tokens" }(aContentFilterreason also exists in the SDK enums shown in the reasoning guide). — same page and https://developers.openai.com/api/docs/guides/reasoning.md — verified on 2026-11-21- The function name and
call_idare not in the arguments-delta events; they arrive in thefunction_callitem (response.output_item.added/.done). The item shape is documented (1.5); that theaddedevent already carriesnameandcall_idfor a function call is NOT verified (the reference example foroutput_item.addedshows a message item only).
Token usage appears in the terminal response.completed event at response.usage (it is null in response.created, and null in the documented response.failed / response.incomplete examples):
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"output_tokens_details": { "reasoning_tokens": 0 },
"total_tokens": 0,
"input_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 0 }
}— https://developers.openai.com/api/reference/resources/responses/streaming-events.md — verified on 2026-11-21
1.5 Function tools
Tool definition (flat, not nested under function as in Chat Completions):
{
"type": "function",
"name": "get_horoscope",
"strict": true,
"description": "Get today's horoscope for an astrological sign.",
"parameters": {
"type": "object",
"properties": { "sign": { "type": "string", "description": "…" } },
"required": ["sign"],
"additionalProperties": false
}
}The model's call is an output item; arguments is a JSON-encoded string:
{ "id": "fc_12345xyz", "call_id": "call_12345xyz", "type": "function_call", "name": "get_weather", "arguments": "{\"location\":\"Paris, France\"}" }Results are sent back by appending the model's output items to input, then one item per call:
{ "type": "function_call_output", "call_id": "call_12345xyz", "output": "<string result>" }"Since model responses can include zero, one, or multiple calls, it is best practice to assume there are several." — https://developers.openai.com/api/docs/guides/function-calling.md — verified on 2026-11-21
1.6 Reasoning
reasoning.effort: "Currently supported values arenone,minimal,low,medium,high,xhigh, andmax", defaultmedium; "Not all reasoning models support every value." — https://developers.openai.com/api/reference/resources/responses/methods/create.md — verified on 2026-11-21reasoning.summary:"auto" | "concise" | "detailed"; summaries are opt-in ("This output will not be included unless you explicitly opt in").generate_summaryis deprecated. — same page and https://developers.openai.com/api/docs/guides/reasoning.md — verified on 2026-11-21reasoning.mode:"standard" | "pro";reasoning.context:"auto" | "current_turn" | "all_turns". — reference page above — verified on 2026-11-21- Per-model effort restrictions (https://developers.openai.com/api/docs/guides/reasoning.md and the model pages — verified on 2026-11-21):
gpt-6.2-astra,gpt-6.2-sol: rejecteffort: "none"(HTTP 400).gpt-6-astra: does not supportnone(HTTP 400).- GPT-6.1 Sol: does not support
noneorminimal; defaultmedium. gpt-6.1-terra,gpt-6-luna: supportnone,low,medium(default),high,xhigh,max.
- Budgeting: "OpenAI recommends reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models"; hitting the limit gives
status: "incomplete"withincomplete_details.reason: "max_output_tokens", possibly before any visible output. — https://developers.openai.com/api/docs/guides/reasoning.md — verified on 2026-11-21
1.7 Parameters the current models reject
"Unsupported parameters: When migrating Chat Completions requests, omit temperature, top_p, logprobs, and top_logprobs. For Responses requests, omit temperature, top_p, and top_logprobs and remove message.output_text.logprobs from include." (stated for gpt-6.2-astra and gpt-6.2-sol). — https://developers.openai.com/api/docs/guides/latest-model.md — verified on 2026-11-21
- The exact HTTP status and error code returned when one of these is sent: NOT verified (the page says to omit them; it does not show the error).
- Whether
gpt-6.1-terra/gpt-6-lunaaccepttemperature: NOT verified.
1.8 GET /v1/models
{
"object": "list",
"data": [
{ "id": "model-id-0", "object": "model", "created": 1686935002, "owned_by": "organization-owner", "shutdown_date": null }
]
}Fields per model: id, created (unix seconds), object: "model", owned_by, shutdown_date (date string or null). No context window, modality or pricing fields: the app needs its own capability table. — https://developers.openai.com/api/reference/resources/models/methods/list.md — verified on 2026-11-21
1.9 Errors and rate-limit headers
HTTP error body, verified live on 2026-11-21 with an unauthenticated GET https://api.openai.com/v1/models (HTTP 401, www-authenticate: Bearer realm="OpenAI API", x-request-id present):
{ "error": { "message": "Missing bearer authentication in header", "type": "invalid_request_error", "param": null, "code": null } }The same four-field shape in the documentation itself: NOT verified (no JSON example found in the error-codes guide or the reference overview).
Documented status codes (https://developers.openai.com/api/docs/guides/error-codes.md — verified on 2026-11-21): 400 (invalid service_tier), 401 (invalid authentication / incorrect API key / not a member of an organization / IP not authorized), 403 (country, region or territory not supported), 429 (rate limit; credit_balance_exhausted; slow_down with type rate_limit_error; organization_spend_limit_exceeded; project_spend_limit_exceeded; organization_usage_limit_exceeded), 500, 503 (server_is_overloaded, type service_unavailable_error).
Response headers (https://developers.openai.com/api/reference/overview.md and https://developers.openai.com/api/docs/guides/rate-limits.md — verified on 2026-11-21):
x-request-id,openai-organization,openai-processing-ms,openai-version("currently2020-10-01")x-ratelimit-limit-requests,x-ratelimit-limit-tokens,x-ratelimit-remaining-requests,x-ratelimit-remaining-tokens,x-ratelimit-reset-requests,x-ratelimit-reset-tokens(reset values are durations such as1s,6m0s)x-ratelimit-limit-project-tokens,x-ratelimit-remaining-project-tokens,x-ratelimit-reset-project-tokens(only when a project-scoped token limit applies)Retry-After(seconds) "may be present on429responses caused by a temporary rate limit and503responses caused by temporary model overload".
1.10 Current models and prices
Source for the lineup: https://developers.openai.com/api/docs/models.md. Source for each row: the model page https://developers.openai.com/api/docs/models/<id>.md. Pricing page: https://developers.openai.com/api/docs/pricing. All read on 2026-11-21. Prices are USD per 1M tokens, Standard processing, short context.
| Model ID | Role (as described) | Context window | Max input | Max output | Input | Cached input | Cache write (30 min) | Output |
|---|---|---|---|---|---|---|---|---|
gpt-6.2-astra |
flagship, "most capable" | 1,050,000 | 922,000 | 128,000 | $8 | $0.40 | $10 | $40 |
gpt-6.2-sol |
"go-to model for everyday work" | 1,050,000 | 922,000 | 128,000 | $2 | $0.10 | $2.50 | $10 |
gpt-6.1-terra |
"fast, efficient" | 1,050,000 | 922,000 | 128,000 | $1 | $0.05 | $1.25 | $5 |
gpt-6-luna |
"most efficient", high volume | 1,050,000 | 922,000 | 128,000 | $0.10 | $0.01 | $0.125 | $0.50 |
All four: text + image in, text out, reasoning token support — verified on 2026-11-21.
- Long-context surcharge on all four: prompts above 272K input tokens are billed at 2x input and cache rates and 1.5x output for the whole request. Batch is 50% of Standard. — model pages — verified on 2026-11-21
- The naming scheme is no longer "mini/nano": the tiers are Astra / Sol / Terra / Luna. The only model with "mini" in the current catalogue list is
gpt-5.4-mini(older generation). Other IDs listed:gpt-6-astra,gpt-6-sol,gpt-6.1-sol,gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5,gpt-5.5-pro,gpt-5.4,gpt-5.4-pro,gpt-5.2,gpt-5.2-pro,chat-latest. — https://developers.openai.com/api/docs/models.md — verified on 2026-11-21 - The consolidated pricing page itself (https://developers.openai.com/api/docs/pricing) was fetched but its table was not parsed; the figures above come from the per-model pages. Prices of the non-featured models: NOT verified.
2. Anthropic
Docs base: https://platform.claude.com/docs/en/… (append .md for Markdown).
2.1 Endpoint and headers
POST https://api.anthropic.com/v1/messages. Header table from https://platform.claude.com/docs/en/api/overview — verified on 2026-11-21:
| Header | Value | Required |
|---|---|---|
Authorization |
Bearer <token> (API key or short-lived access token) |
Yes, unless x-api-key is set |
x-api-key |
API key. Described as "Legacy fallback for Authorization, still supported" |
No |
anthropic-version |
API version, e.g. 2026-10-21 |
Yes |
content-type |
application/json |
Yes |
anthropic-workspace-id |
workspace ID (wrkspc_…) |
"Required with an API key that isn't scoped to a workspace. Optional for other API keys." |
- Every curl example in the current docs uses
x-api-key+anthropic-version: 2026-10-21. — e.g. https://platform.claude.com/docs/en/build-with-claude/thinking — verified on 2026-11-21 - Verified live on 2026-11-21: an unauthenticated
GET https://api.anthropic.com/v1/modelsreturns HTTP 401 with message "x-api-key header is required".
anthropic-version values (https://platform.claude.com/docs/en/api/versioning — verified on 2026-11-21): 2026-10-21 (latest, recommended), 2023-06-01, 2023-01-01. The streaming SSE format is the one introduced in 2023-06-01; 2026-10-21 does not change it.
Breaking changes in 2026-10-21 for the Messages API (https://platform.claude.com/docs/en/api/migration-2026-10-21 — verified on 2026-11-21):
- "The
temperature,top_p, andtop_krequest parameters are removed. A request that sends any of them returns a 400 error on every model", "even at the parameter's default value". - A
documentblock placed directly in anassistantmessage returns 400. - A
textblock with a non-emptycitationsarray in ausermessage returns 400. - The guide states that a request relying on a non-default sampling value can "keep that request on
2023-06-01", which implies2023-06-01is still accepted. An explicit sunset date for2023-06-01: NOT verified.
2.2 Request body essentials
model,max_tokens,messages(rolesuser/assistant), top-levelsystem(string),stream: true. — examples in https://platform.claude.com/docs/en/build-with-claude/prompt-caching and https://platform.claude.com/docs/en/build-with-claude/streaming — verified on 2026-11-21- Prefilling the final assistant turn is rejected (400) on Claude 4.6 and later models. — https://platform.claude.com/docs/en/build-with-claude/thinking — verified on 2026-11-21
- Request size limit: 128 MB for the Messages API (413
request_too_largeabove it). — https://platform.claude.com/docs/en/api/errors — verified on 2026-11-21 - Full list of
stop_reasonvalues: NOT verified (onlyend_turnandtool_usewere seen in the examples read).
2.3 Streaming SSE events
Source: https://platform.claude.com/docs/en/build-with-claude/streaming — verified on 2026-11-21. Each event has an SSE event: name and a data: JSON whose type matches. There is no [DONE] sentinel (removed in 2023-06-01, per the versioning page).
Flow: message_start → for each content block: content_block_start, zero or more content_block_delta, content_block_stop → one or more message_delta → message_stop. ping events may appear anywhere. "new event types may be added, and your code should handle unknown event types gracefully."
Documented payloads:
event: message_start
data: {"type": "message_start", "message": {"id": "msg_…", "type": "message", "role": "assistant", "content": [], "model": "claude-opus-5-5", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 25, "output_tokens": 1}}}
event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}
event: ping
data: {"type": "ping"}
event: content_block_delta
data: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "Hello"}}
event: content_block_stop
data: {"type": "content_block_stop", "index": 0}
event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence": null}, "usage": {"output_tokens": 15}}
event: message_stop
data: {"type": "message_stop"}
event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}Delta types inside content_block_delta:
delta.type |
Field | Notes |
|---|---|---|
text_delta |
text |
|
input_json_delta |
partial_json |
partial JSON string fragments for a tool_use block; accumulate and parse at content_block_stop. The first fragment can be "" |
thinking_delta |
thinking |
|
signature_delta |
signature |
sent just before content_block_stop of a thinking block |
Tool-use block start: {"type":"content_block_start","index":1,"content_block":{"type":"tool_use","id":"toolu_…","name":"get_weather","input":{}}}; the stream then ends with message_delta carrying "stop_reason":"tool_use".
Usage:
message_start.message.usagecarriesinput_tokens(and, per the caching page,cache_creation_input_tokens/cache_read_input_tokens).message_delta.usage.output_tokensis cumulative ("The token counts shown in theusagefield of themessage_deltaevent are cumulative").- Some blocks arrive whole with no deltas:
redacted_thinking, server tool results, and thefallbackblock.
With thinking.display: "omitted" (the default on current models, see 2.6) no thinking text is streamed: the block opens, gets one thinking_delta with an empty string, then one signature_delta, then closes.
2.4 Image input
{ "type": "image", "source": { "type": "base64", "media_type": "image/jpeg", "data": "<BASE64>" } }placed in a user message's content array (the documented example puts the image before the text block). — https://platform.claude.com/docs/en/build-with-claude/vision — verified on 2026-11-21
- Formats:
image/jpeg,image/png,image/gif,image/webp. - Limits: 10 MB per image (base64-encoded) on the Claude API; max 8000x8000 px; up to 600 images per request (100 for 200k-context models); above 20 images in one request a stricter per-image dimension limit applies (keep each side ≤ 3,000 px).
- Cost:
⌈width/28⌉ × ⌈height/28⌉visual tokens. Claude 4.7 and later: max long edge 2,576 px / 4,784 visual tokens; older models 1,568 px / 1,568 tokens. Larger images are downscaled server-side.
All from the same page — verified on 2026-11-21.
2.5 Tool use cycle
Tool definition: name (regex ^[a-zA-Z0-9_-]{1,128}$), description, input_schema (JSON Schema), optional input_examples, cache_control, strict. — https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools — verified on 2026-11-21
Assistant turn (response stop_reason is tool_use):
{ "type": "tool_use", "id": "toolu_01A09q90qw90lq917835lq9", "name": "get_weather", "input": { "location": "San Francisco, CA" } }Follow-up request: keep earlier messages, add the response content unchanged as an assistant message, then a user message with one block per tool call:
{ "type": "tool_result", "tool_use_id": "toolu_01A09q90qw90lq917835lq9", "content": "…", "is_error": true }content is a string or a list of text / image / document / search_result blocks; is_error is optional. Send the same tools again. — https://platform.claude.com/docs/en/agents-and-tools/tool-use/handle-tool-calls — verified on 2026-11-21
tool_choice: auto (default), none, any, tool. Forced tool use (any / tool) returns 400 on claude-fable-5-2, claude-opus-5-5, claude-sonnet-5-5 and the 5.1 Fable/Mythos models. — https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools — verified on 2026-11-21
2.6 Thinking and effort
Source: https://platform.claude.com/docs/en/build-with-claude/thinking — verified on 2026-11-21.
thinking request values and what each model does:
| Model | no thinking field |
{"type":"adaptive"} |
{"type":"enabled","budget_tokens":N} |
{"type":"disabled"} |
|---|---|---|---|---|
| Claude Fable 5.2 / 5.1 / 5, Mythos 5.x | adaptive | adaptive | 400 | 400 |
| Claude Opus 5.5 | adaptive | adaptive | 400 | 400 |
| Claude Sonnet 5.5 | adaptive | adaptive | 400 | 400 (use {"type":"between_tools"} to turn off up-front thinking, at high effort or below) |
| Claude Haiku 5.5 | adaptive | adaptive | 400 | off, at high effort or below |
| Claude Opus 5 | adaptive | adaptive | 400 | off, at high effort or below |
| Claude Sonnet 5 | adaptive | adaptive | 400 | off |
| Claude Opus 4.8, 4.7 | off | adaptive | 400 | off |
| Claude Opus 4.6, Sonnet 4.6 | off | adaptive | extended thinking (deprecated) | off |
| Claude Opus 4.5, Sonnet 4.5, Haiku 4.5 | off | 400 | extended thinking | off |
display:"summarized"returns readable summary text;"omitted"returns thinking blocks with an emptythinkingfield and is the default on all Claude 5 models, Opus 4.8 and Opus 4.7. To show thinking:"thinking": {"type": "adaptive", "display": "summarized"}."updates"is beta (headerthinking-display-updates-2026-08-18).- Thinking is billed at full thinking tokens regardless of
display. Thinking blocks (withsignature) must be passed back unchanged in multi-turn and tool-use conversations. - Sampling parameters: on all Claude 5 models, Opus 4.8 and Opus 4.7, non-default
temperature,top_portop_kreturn 400 on every request; withanthropic-version: 2026-10-21any of the three returns 400 on every model.
Effort (https://platform.claude.com/docs/en/build-with-claude/effort — verified on 2026-11-21): set output_config.effort at the top level of the request:
{ "model": "claude-opus-5-5", "max_tokens": 4096, "messages": [ … ], "output_config": { "effort": "medium" } }Levels: low, medium, high, xhigh, max. Default is high, except medium on Claude Opus 5.5 and Claude Haiku 5.5. xhigh is available on all Claude 5 models, Opus 4.8 and 4.7; max additionally on Opus 4.6 and Sonnet 4.6. Changing top-level effort between requests can invalidate the prompt cache.
2.7 Prompt caching
Source: https://platform.claude.com/docs/en/build-with-claude/prompt-caching — verified on 2026-11-21.
- Automatic caching: one top-level field,
"cache_control": {"type": "ephemeral"}. - Explicit breakpoints:
cache_controlon individual content blocks; up to 4 breakpoint slots (the automatic one uses a slot). - Prefix order:
tools,system,messages. - TTL: 5 minutes by default;
{"type": "ephemeral", "ttl": "1h"}for one hour.ephemeralis the only cache type. - Minimum cacheable length: 512 tokens (Fable 5.x, Mythos 5.x, Opus 5.5, Opus 5, Sonnet 5.5, Haiku 5.5); 1,024 (Opus 4.8, Sonnet 5, Sonnet 4.6, Sonnet 4.5); 2,048 (Opus 4.7); 4,096 (Opus 4.6, Opus 4.5, Haiku 4.5). Shorter prompts are processed uncached without error.
- Usage fields:
cache_creation_input_tokens,cache_read_input_tokens,input_tokens(only the tokens after the last breakpoint). Total input = sum of the three. - Supported on all active Claude models.
2.8 GET /v1/models
Source: https://platform.claude.com/docs/en/api/models/list — verified on 2026-11-21. Query: limit (default 20, 1–1000), after_id, before_id, lifecycle (active / deprecated / retired; default returns active + deprecated). Response:
{
"data": [ {
"type": "model", "id": "claude-opus-5-5", "display_name": "Claude Opus 5.5",
"created_at": "2026-09-21T00:00:00Z", "lifecycle": "active", "line": "opus",
"max_input_tokens": 0, "max_tokens": 0, "deprecated_at": "…", "retires_at": "…",
"capabilities": {
"image_input": {"supported": true}, "pdf_input": {"supported": true},
"effort": {"supported": true, "low": {"supported": true}, "medium": {…}, "high": {…}, "xhigh": {…}, "max": {…}},
"thinking": {"supported": true, "types": {"adaptive": {…}, "enabled": {…}, "disabled": {…}, "between_tools": {…}}},
"structured_outputs": {…}, "batch": {…}, "citations": {…}, "code_execution": {…},
"context_management": {…}, "server_tools": {…}
}
} ],
"first_id": "…", "has_more": true, "last_id": "…"
}max_input_tokens, max_tokens and capabilities let a client build its model picker without a hard-coded table ("Read line instead of inferring it from the model's id"). — also https://platform.claude.com/docs/en/about-claude/models/overview — verified on 2026-11-21
2.9 Errors and rate-limit headers
Error body (https://platform.claude.com/docs/en/api/errors — verified on 2026-11-21, and verified live):
{ "type": "error", "error": { "type": "not_found_error", "message": "…" }, "request_id": "req_…" }| HTTP | error.type |
|---|---|
| 400 | invalid_request_error (also returned when a self-set spend limit is reached) |
| 401 | authentication_error |
| 402 | billing_error |
| 403 | permission_error |
| 404 | not_found_error |
| 409 | conflict_error |
| 413 | request_too_large |
| 429 | rate_limit_error (a tier spend-cap 429 has no retry-after) |
| 500 | api_error |
| 504 | timeout_error (can also appear with other statuses) |
| 529 | overloaded_error |
Mid-stream errors arrive as the error SSE event after a 200 (see 2.3).
Response headers (https://platform.claude.com/docs/en/api/rate-limits and https://platform.claude.com/docs/en/api/overview — verified on 2026-11-21): request-id, anthropic-organization-id, anthropic-workspace-id, retry-after (seconds), and anthropic-ratelimit-{requests,tokens,input-tokens,output-tokens}-{limit,remaining,reset} (reset is RFC 3339; token "remaining" values are rounded to the nearest thousand), plus anthropic-priority-{input,output}-tokens-{limit,remaining,reset} on Priority Tier.
2.10 Current models and prices
Sources: https://platform.claude.com/docs/en/about-claude/models/overview and https://platform.claude.com/docs/en/about-claude/pricing — verified on 2026-11-21. USD per million tokens.
| Model | API ID | Context | Max output | Input | Output | 5 min cache write | 1 h cache write | Cache read |
|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.2 | claude-fable-5-2 |
1M | 128K | $8 | $40 | $10 | $16 | $0.20 |
| Claude Opus 5.5 | claude-opus-5-5 |
1M | 128K | $4 | $20 | $5 | $8 | $0.20 |
| Claude Sonnet 5.5 | claude-sonnet-5-5 |
1M | 128K | $2 | $10 | $2.50 | $4 | $0.10 |
| Claude Haiku 5.5 (prompt ≤ 100,000 tokens) | claude-haiku-5-5 |
1M | 128K | $0.10 | $0.50 | $0.125 | $0.20 | $0.01 |
| Claude Haiku 5.5 (prompt > 100,000 tokens) | claude-haiku-5-5 |
1M | 128K | $0.50 | $2.50 | $0.625 | $1 | $0.05 |
| Claude Opus 5 | claude-opus-5 |
1M | 128K | $5 | $25 | $6.25 | $10 | $0.50 |
| Claude Sonnet 5 | claude-sonnet-5 |
1M | 128K | $2 | $10 | $2.50 | $4 | $0.20 |
| Claude Opus 4.8 | claude-opus-4-8 |
1M | 128K | $5 | $25 | $6.25 | $10 | $0.50 |
| Claude Sonnet 4.6 | claude-sonnet-4-6 |
1M | 128K | $3 | $15 | $3.75 | $6 | $0.30 |
| Claude Haiku 4.5 | claude-haiku-4-5-20251001 |
200K | 64K | $1 | $5 | $1.25 | $2 | $0.10 |
- The overview recommends Claude Opus 5.5 as the default starting point and Claude Fable 5.2 "for demanding reasoning and long-horizon agentic work".
- "Every Claude model ID is a pinned snapshot, including the dateless IDs used from the 4.6 generation on."
- Also listed:
claude-fable-5-1,claude-fable-5($10 / $50),claude-opus-4-7,claude-opus-4-6,claude-opus-4-5-20251101,claude-sonnet-4-5-20250929, and the limited-availability Mythos models. - Claude 4.7 and later use a newer tokenizer that "produces approximately 30% more tokens for the same text".
- Batch API: 50% off. Full 1M context at standard pricing on 4.6 and later, except Haiku 5.5's 100,000-token threshold above.
3. OpenRouter
Docs base: https://openrouter.ai/docs/… (index at https://openrouter.ai/docs/llms.txt; pages have .md variants).
Note: OpenRouter's guides now lead with its own Responses API (POST /api/v1/responses), and also expose an Anthropic-style POST /api/v1/messages. Chat Completions (POST https://openrouter.ai/api/v1/chat/completions) remains documented and supported; each model in GET /api/v1/models lists the endpoints it supports in an endpoints array. — https://openrouter.ai/docs/api_reference/overview.md and live GET https://openrouter.ai/api/v1/models — verified on 2026-11-21
3.1 Streaming on Chat Completions
Source: https://openrouter.ai/docs/api_reference/streaming.md — verified on 2026-11-21.
- Lines are
data: <json>; the stream ends withdata: [DONE]. - SSE comment lines must be skipped. The doc's own parser comment: "Skip SSE comments (lines starting with "😊, e.g. the ": OPENROUTER PROCESSING" keep-alive — they are not JSON".
- Chunk schema (
ChatStreamChunkin the OpenAPI spec at https://openrouter.ai/docs/api/api-reference/chat/send-chat-completion-request.md — verified on 2026-11-21):id,object: "chat.completion.chunk",created,model,choices[],usage,error,system_fingerprint,service_tier,openrouter_metadata,safeguard_results,alignment. Each choice:index,delta,finish_reason,logprobs.deltafields:role,content,reasoning,reasoning_details,refusal,tool_calls,audio. - The generation ID is also returned in the
X-Generation-Idresponse header.
Usage in the final chunk: "every stream ends with an extra chunk that carries the usage object for the request, sent just before the [DONE] message." OpenRouter deviates from OpenAI here: the usage chunk has a non-empty choices array, with a content-free delta that repeats finish_reason and native_finish_reason, so the terminal finish_reason appears twice:
data: {"id":"gen-abc123",...,"choices":[{"index":0,"delta":{"content":"","role":"assistant"},"finish_reason":"stop","native_finish_reason":"stop"}]}
data: {"id":"gen-abc123",...,"choices":[{"index":0,"delta":{"content":"","role":"assistant"},"finish_reason":"stop","native_finish_reason":"stop"}],"usage":{...}}
data: [DONE]How to request usage: nothing to send. "On Chat Completions, the usage: { include: true } and stream_options: { include_usage: true } parameters are deprecated and have no effect. … Full usage details are always included automatically in every response." — https://openrouter.ai/docs/cookbook/administration/usage-accounting.md — verified on 2026-11-21
Usage object on Chat Completions (https://openrouter.ai/docs/api_reference/overview.md — verified on 2026-11-21): prompt_tokens, completion_tokens, total_tokens, prompt_tokens_details { cached_tokens, cache_write_tokens?, audio_tokens?, video_tokens? }, completion_tokens_details { reasoning_tokens?, audio_tokens?, image_tokens? }, cost (credits), is_byok, cost_details { upstream_inference_cost?, upstream_inference_prompt_cost, upstream_inference_completions_cost, server_tool_cost? }.
Errors (streaming doc and https://openrouter.ai/docs/api_reference/errors-and-debugging.md — verified on 2026-11-21):
- Before the response is committed: normal HTTP status with body
{"error": {"code": 400, "message": "Invalid model specified"}}(codeis a number; optionalmetadata). Statuses: 400, 401, 402 (insufficient credits), 403 (permissions, guardrail block, moderation), 408, 429, 502 (provider error), 503 (no provider meets routing requirements).Retry-Aftermay be present on 429 and 503. - Mid-stream (HTTP stays 200): a final chunk with a top-level
errorandfinish_reason: "error":
data: {"id":"cmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"openai/gpt-4o","provider":"openai","error":{"code":502,"message":"Provider disconnected unexpectedly"},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}- "The error can be the first and only event in the stream, so treat a
200carrying anerrorchunk with no content as a failure, not a success." - For Chat Completions, a provider error carries a typed
error.metadata.error_type(e.g.rate_limit_exceeded,provider_overloaded,provider_unavailable,context_length_exceeded,payment_required,authentication, …). - Upstream token-limit errors become
finish_reason: "length". - Stream cancellation: aborting the connection stops processing and billing only for some providers (supported list includes OpenAI, Azure, Anthropic, Fireworks, DeepInfra, Together, XAI, DeepSeek; not supported: AWS Bedrock, Groq, Google, Mistral, Perplexity, and others).
3.2 Image input on Chat Completions
Content part {"type": "image_url", "image_url": {"url": "<https URL or data: URL>", "detail": "auto"}}; "URL of the image (data: URLs supported)"; detail enum auto, low, high, original (original is an OpenRouter extension). — OpenAPI spec, ChatContentImage, https://openrouter.ai/docs/api/api-reference/chat/send-chat-completion-request.md — verified on 2026-11-21
3.3 GET /api/v1/models
Public, no key needed. Verified live on 2026-11-21 (476 models returned). Top-level keys observed across all entries: id, canonical_slug, hugging_face_id, name, created, description, context_length, architecture, pricing, top_provider, per_request_limits, supported_parameters, default_parameters, supported_voices, knowledge_cutoff, expiration_date, links, endpoints, benchmarks, reasoning, alias_target.
Example entry (trimmed), observed live:
{
"id": "anthropic/claude-sonnet-5.5",
"canonical_slug": "anthropic/claude-sonnet-5.5-20260928",
"name": "Anthropic: Claude Sonnet 5.5",
"context_length": 1000000,
"architecture": { "modality": "text+image+file->text", "input_modalities": ["text","image","file"], "output_modalities": ["text"], "tokenizer": "Claude", "instruct_type": null },
"pricing": { "prompt": "0.000002", "completion": "0.00001", "web_search": "0.01", "input_cache_read": "0.0000001", "input_cache_write": "0.0000025", "input_cache_write_1h": "0.000004" },
"top_provider": { "context_length": 1000000, "max_completion_tokens": 128000, "is_moderated": true },
"supported_parameters": ["include_reasoning","max_completion_tokens","max_tokens","reasoning","reasoning_effort","response_format","stop","structured_outputs","tool_choice","tools","verbosity"],
"endpoints": ["/api/v1/chat/completions","/api/v1/completions","/api/v1/messages","/api/v1/responses"]
}- Pricing units: strings, "All pricing values are in USD per token/request/unit. A value of
"0"indicates the feature is free." Multiplyprompt/completionby 1,000,000 for a per-million figure. Documented keys:prompt,completion,request,image,web_search,internal_reasoning,input_cache_read,input_cache_write, optionaloverrides[](conditional pricing, e.g.min_prompt_tokens). — https://openrouter.ai/docs/guides/overview/models.md — verified on 2026-11-21 supported_parametersdoes not containtemperaturefor the model above, which matches Anthropic's removal of sampling parameters: use this array to decide which controls to show.- Optional per-model
reasoningobject:supported_efforts(descending),default_effort,default_enabled,supports_max_tokens,mandatory(whentrue, do not sendeffort: "none"). — https://openrouter.ai/docs/guides/best-practices/reasoning-tokens.md — verified on 2026-11-21 - Query filters:
?supported_parameters=tools,?sort=pricing-low-to-high, etc. — models doc above — verified on 2026-11-21
3.4 GET /api/v1/key
GET https://openrouter.ai/api/v1/key with Authorization: Bearer <key>. Documented response (https://openrouter.ai/docs/api_reference/limits.md — verified on 2026-11-21), under data: label, is_management_key, is_provisioning_key, limit, limit_reset, limit_remaining, include_byok_in_limit, budgets[], effective_budget { scope, limit_usd, spend_usd, remaining_usd, reset_interval, resets_at, … }, usage, usage_daily, usage_weekly, usage_monthly, byok_usage*, is_free_tier, expires_at, organization_id, workspace_id, allowed_data_regions, free_model_daily_requests { used, limit }.
- Verified live on 2026-11-21: without a key the endpoint returns HTTP 401
{"error":{"message":"No cookie auth credentials found","code":401}}. limit/limit_remainingarenullfor a key with no cap: NOT verified.- Rate limits:
X-RateLimit-*headers appear "on free-model and key-management errors". — same page — verified on 2026-11-21
3.5 Attribution headers
Source: https://openrouter.ai/docs/app-attribution.md and https://openrouter.ai/docs/api_reference/overview.md — verified on 2026-11-21. All optional for the API to work.
HTTP-Referer: the app's URL; "required for app attribution" (without it no app page is created).X-OpenRouter-Title: display name. "X-Titleis still supported for backwards compatibility."X-OpenRouter-Categories: comma-separated, up to 2 recognised lowercase categories.X-OpenRouter-App-Visibility: hidden: creates a new app unlisted.
3.6 Provider routing, fallbacks, reasoning
provider object fields (https://openrouter.ai/docs/guides/routing/provider-selection.md — verified on 2026-11-21):
| Field | Type | Default |
|---|---|---|
order |
string[] of provider slugs | – |
allow_fallbacks |
boolean | true |
require_parameters |
boolean | false |
data_collection |
"allow" | "deny" |
"allow" |
zdr |
boolean | – |
only |
string[] | – |
ignore |
string[] | – |
quantizations |
string[] | – |
sort |
"price" | "throughput" | "latency", or { by, partition } |
– |
preferred_min_throughput, preferred_max_latency |
number or percentile object | – |
max_price |
object | – |
enforce_distillable_text |
boolean | – |
models fallback array (https://openrouter.ai/docs/guides/routing/model-fallbacks.md — verified on 2026-11-21): model IDs in priority order; any error can trigger the next one (context-length validation, moderation flags, rate limiting, downtime). "Requests are priced using the model that was ultimately used, which will be returned in the model attribute of the response body."
reasoning parameter (https://openrouter.ai/docs/guides/best-practices/reasoning-tokens.md — verified on 2026-11-21):
"reasoning": { "effort": "high", "max_tokens": 2000, "exclude": false, "enabled": true }effort:"max","xhigh","high","medium","low","minimal","none".max_tokens: explicit budget (Anthropic-style).enabled: truealone means medium effort.exclude: truehides reasoning from the response and is honoured on Chat Completions only.- On Chat Completions the reasoning text arrives in
delta.reasoning/message.reasoning, and structured blocks inreasoning_details, which must be sent back unmodified to preserve reasoning across tool calls. - Reasoning tokens are billed as output and, on most providers, count against
max_tokens; if the budget is consumed by reasoning the result isfinish_reason: "length"with emptycontent.
3.7 How the actual model and provider are reported
- Model: the response
modelfield is the model actually used (fallback doc, 3.6). — verified on 2026-11-21 - Provider: the documented mid-stream error example carries a top-level
"provider": "openai"on the chunk (streaming doc). However the OpenAPI schema forChatStreamChunk/ChatResultdoes not declare a top-levelproviderproperty. Decode it as optional. Whether every successful chunk carriesprovider: NOT verified. - The documented, schema-backed way: send the
X-OpenRouter-Metadata: enabledheader (older nameX-OpenRouter-Experimental-Metadatastill accepted); the response then includes anopenrouter_metadataobject withendpoints,attempts,region,is_byok,strategy, and per-attemptprovider/model/status. — https://openrouter.ai/docs/guides/routing/routers/router-metadata.md and the OpenAPI spec — verified on 2026-11-21 native_finish_reasonis the provider's raw finish reason, next to the normalisedfinish_reason. — streaming doc — verified on 2026-11-21GET /api/v1/generation?id=…response fields (e.g.provider_name): NOT verified (page not read).
4. Apple Foundation Models framework
All pages below were read through the DocC JSON endpoint on 2026-11-21. Human URLs are given.
4.1 Minimum OS and platforms
- Framework availability: iOS 26.0, iPadOS 26.0, Mac Catalyst 26.0, macOS 26.0, visionOS 26.0, watchOS 27.0.
streamResponse(to:options:)andTooladditionally list tvOS 27.0. — https://developer.apple.com/documentation/foundationmodels — verified on 2026-11-21 SystemLanguageModel(the on-device model): iOS 26.0, iPadOS 26.0, Mac Catalyst 26.0, macOS 26.0, visionOS 26.0. — https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel — verified on 2026-11-21
4.2 Device eligibility
- "Model availability depends on whether the device and region supports Apple Intelligence." — SystemLanguageModel page above — verified on 2026-11-21
- Apple Intelligence devices for the 27 releases: "iPhone Duo, iPhone 16 models or later, iPhone 15 Pro, iPhone 15 Pro Max, iPhone Air, iPad mini (A17 Pro and A20 Pro), iPad models with M1 or later, MacBook Neo (A18 Pro), Mac with M1 or later, Apple Vision Pro, Apple Watch Series 9 or later, Apple Watch Ultra 2 or later, and Apple Watch SE 3 when paired with an Apple Intelligence-enabled iPhone". Storage: up to 14 GB on iPhone Duo, iPhone 17 Pro and later, iPhone Air, M4+ iPads and M3+ Macs with at least 12 GB memory; up to 8 GB on other eligible devices. — https://support.apple.com/en-us/121115 — verified on 2026-11-21
- Two on-device variants exist and the app cannot choose: "AFM Core Advanced | 20 billion parameters, 8K tokens context" and "AFM Core | 3 billion parameters, 4-8K tokens context"; a server variant "AFM Cloud | 32K tokens context" is exposed as
PrivateCloudComputeLanguageModel. Detect the on-device variant at runtime with thevariantproperty. — https://developer.apple.com/documentation/foundationmodels/detecting-available-apple-foundation-models — verified on 2026-11-21 - Which device gets which variant: NOT verified (not stated on the pages read).
4.3 Availability API
SystemLanguageModel.default.availability is SystemLanguageModel.Availability (@frozen enum): .available or .unavailable(UnavailableReason). UnavailableReason cases: .deviceNotEligible, .appleIntelligenceNotEnabled, .modelNotReady. There is also a Boolean isAvailable. Apple's sample switch includes @unknown default. — https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel/availability-swift.enum and the "Detecting available…" article — verified on 2026-11-21
- "In iOS 27, iPadOS 27, macOS 27, and visionOS 27 or later, Apple Intelligence is on by default and can no longer be turned off." So
.appleIntelligenceNotEnabledis in practice an OS 26 case. .modelNotReadyis typically the model downloading after an OS update; show "temporarily unavailable, try again later".
4.4 Sessions and streaming
LanguageModelSession (https://developer.apple.com/documentation/foundationmodels/languagemodelsession — verified on 2026-11-21):
- Init:
init(model:tools:instructions:),init(model:tools:transcript:); OS 27 addsinit(profile:history:)andinit(model:dynamicInstructions:history:). prewarm(promptPrefix:),isResponding,transcript,usage.- Non-streaming:
respond(to:options:),respond(to:generating:includeSchemaInPrompt:options:),respond(to:schema:…). - Streaming:
final func streamResponse(to prompt: Prompt, options: GenerationOptions = GenerationOptions()) -> sending LanguageModelSession.ResponseStream<String>; returns "A response stream that produces aggregated tokens" (snapshots, not deltas). Variants:streamResponse(to:generating:includeSchemaInPrompt:options:),streamResponse(to:schema:…). — https://developer.apple.com/documentation/foundationmodels/languagemodelsession/streamresponse(to:options:) — verified on 2026-11-21 - "If running in the background, use the non-streaming
respond(to:options:)method to reduce the likelihood of encounteringLanguageModelError.rateLimited(_:)errors." — same page — verified on 2026-11-21
4.5 Guided generation and tools
@Generable(description:)on a type and@Guide(description:, …)on properties, e.g.@Guide(description: "The age of the cat", .range(0...20)). Related types:GenerationSchema,DynamicGenerationSchema,GeneratedContent. — https://developer.apple.com/documentation/foundationmodels/supporting-languages-and-locales-with-foundation-models and the framework index — verified on 2026-11-21protocol Tool<Arguments, Output> : Sendablewithname,description, a@Generable struct Arguments, andfunc call(arguments: Arguments) async throws -> Output. Tool definitions and outputs count against the context window. — https://developer.apple.com/documentation/foundationmodels/tool — verified on 2026-11-21- OS 27:
GenerationOptions.ToolCallingModecontrols how the model interacts with tools. — https://developer.apple.com/documentation/updates/foundationmodels — verified on 2026-11-21
4.6 Context window and the overflow error
- Error:
LanguageModelSession.GenerationError.exceededContextWindowSize(_:)(case exceededContextWindowSize(LanguageModelSession.GenerationError.Context)). "This error occurs when you use the available tokens for the context window of 4,096 tokens. The token count includes instructions, prompts, and outputs for a session instance." Recovery: start a new session with a shorter prompt or output. — https://developer.apple.com/documentation/foundationmodels/languagemodelsession/generationerror/exceededcontextwindowsize(_:) — verified on 2026-11-21 - The size is no longer a constant: "Context window size can vary, even within the same model variant and version. To get the exact number of tokens available, check the
contextSizeproperty".contextSizeis@backDeployed(before: iOS 26.4, macOS 26.4, visionOS 26.4);tokenCount(for:)was added in the February 2026 (26.4) update. — https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel/contextsize and the updates page — verified on 2026-11-21 - Other
GenerationErrorcases:assetsUnavailable,decodingFailure,guardrailViolation,rateLimited,refusal,concurrentRequests,unsupportedGuide,unsupportedLanguageOrLocale. — https://developer.apple.com/documentation/foundationmodels/languagemodelsession/generationerror — verified on 2026-11-21 - OS 27 adds
LanguageModelError(@nonexhaustive enum, iOS 27.0+). Apple's guidance for apps that deploy to 26: catchLanguageModelErrorunderif #available(iOS 27.0, …)and fall back toLanguageModelSession.GenerationError. Which exact case ofLanguageModelErrorreports a context overflow on OS 27: NOT verified. — https://developer.apple.com/documentation/foundationmodels/languagemodelerror — verified on 2026-11-21
4.7 Languages
SystemLanguageModel.supportedLanguages: Set<Locale.Language>andsupportsLocale(_:)are the runtime checks. — https://developer.apple.com/documentation/foundationmodels/systemlanguagemodel/supportedlanguages — verified on 2026-11-21- "The on-device system language model is multilingual, which means the same model understands and generates text in any language that Apple Intelligence supports." — languages article above — verified on 2026-11-21
- Apple Intelligence languages on iOS 27 / iPadOS 27 / macOS 27: English, Danish, Dutch, French, German, Italian, Norwegian, Portuguese, Spanish, Swedish, Turkish, Vietnamese, Chinese (simplified), Chinese (traditional), Japanese, Korean. — https://support.apple.com/en-us/121115 — verified on 2026-11-21
- That this consumer list equals
supportedLanguageson a given device: NOT verified. Use the runtime property.
4.8 Changes in the OS 27 releases
Source: https://developer.apple.com/documentation/updates/foundationmodels ("June 2026") and the SystemLanguageModel page — verified on 2026-11-21.
- A new on-device model ships with iOS 27 / iPadOS 27 / macOS 27 / visionOS 27 ("test your prompts with the new model"). There are now three model versions: 26.0–26.3, 26.4, 27.0.
LanguageModelprotocol: "use any large language model — server or on-device — with the Foundation Models framework" (withLanguageModelExecutor,LanguageModelCapabilities).PrivateCloudComputeLanguageModel(entitlementcom.apple.developer.private-cloud-compute, daily quota viaquotaUsage.isLimitReached); without the entitlement the app crashes on the first prompt.- Image input in prompts (
Attachment,ImageAttachmentContent), plus Vision tools such asOCRToolandBarcodeReaderTool. LanguageModelSession.DynamicProfile, new error types (LanguageModelError,SystemLanguageModel.Error,LanguageModelSession.Error),GenerationOptions.ToolCallingMode.- Open source:
CoreAILanguageModelandMLXLanguageModel"to integrate on-device models with the Foundation Models framework" (see section 5).
5. MLX Swift on iOS
5.1 Releases
From the GitHub releases API (https://api.github.com/repos/ml-explore/<repo>/releases) on 2026-11-21:
| Package | Latest release | Published | URL |
|---|---|---|---|
mlx-swift |
0.33.0 | 2026-10-27 | https://github.com/ml-explore/mlx-swift/releases/tag/0.33.0 |
mlx-swift-lm |
3.33.0 | 2026-11-17 | https://github.com/ml-explore/mlx-swift-lm/releases/tag/3.33.0 |
mlx-swift-examples |
2.29.1 | 2025-10-16 | https://github.com/ml-explore/mlx-swift-examples/releases/tag/2.29.1 |
verified on 2026-11-21. Notes from the release bodies:
mlx-swift-lm3.33.0 "picks up mlx-swift 0.33.0 (upToNextMinor)";mainPackage.swiftpins.upToNextMinor(from: "0.33.0").mlx-swift0.33.0 is source-breaking aroundcompile(...)andMLXCustomFunction(@Sendableclosures).mlx-swift0.32.2 crashed on launch "on macOS/iOS earlier than 26.4 (SDK issue)"; fixed in 0.32.3. Do not pin 0.32.2.mlx-swift-lm3.x (first 3.x release 3.31.3, 2026-04-15) decoupled the tokenizer and downloader packages; 3.32.3 addedMLXFoundationModelsandMLXGuidedGeneration.- The README's documented integration: products
MLXLLM,MLXLMCommon,MLXHuggingFace, plushuggingface/swift-huggingface(from 0.9.0) andhuggingface/swift-transformers(from 1.3.0);#huggingFaceLoadModelContainer(configuration:)thenChatSession(model). The README's install snippet still saysfrom: "3.32.3". — https://github.com/ml-explore/mlx-swift-lm/blob/main/README.md — verified on 2026-11-21 MLXFoundationModels(bridges anMLXLanguageModelintoLanguageModelSession) "Requires the macOS/iOS/visionOS 27.0 SDK." — same README — verified on 2026-11-21
5.2 Minimum OS and toolchain
From Package.swift on main of each repo, read on 2026-11-21:
mlx-swift:// swift-tools-version: 6.3; platforms.macOS("14.0"),.iOS(.v17),.tvOS(.v17),.visionOS(.v1). — https://github.com/ml-explore/mlx-swift/blob/main/Package.swift — verified on 2026-11-21mlx-swift-lm:// swift-tools-version: 6.2; platforms.macOS(.v14),.iOS(.v17),.tvOS(.v17),.visionOS(.v1). — https://github.com/ml-explore/mlx-swift-lm/blob/main/Package.swift — verified on 2026-11-21- Minimum Xcode version implied by swift-tools-version 6.3: NOT verified. The
Package.swiftat the 0.33.0 / 3.33.0 tags (as opposed tomain): NOT verified.
5.3 Memory entitlement
- MLX's own doc: "if the device your code runs on has more RAM than the jetsam limit would normally allow, you can use the Increased Memory Limit entitlement", linking to
com.apple.developer.kernel.increased-memory-limit. It also recommends capping the buffer cache, e.g.MLX.Memory.cacheLimit = 20 * 1024 * 1024. — https://github.com/ml-explore/mlx-swift/blob/main/Source/MLX/Documentation.docc/Articles/running-on-ios.md — verified on 2026-11-21 - The LLMEval example: "LLM models are large so this uses the Increased Memory Limit entitlement on iOS". — https://github.com/ml-explore/mlx-swift-examples/blob/main/Applications/LLMEval/README.md — verified on 2026-11-21
- Apple:
com.apple.developer.kernel.increased-memory-limitis a Boolean, iOS 15.0+. "An increased memory limit is only available on some device models. Call theos_proc_available_memoryfunction to determine the amount of memory available." The app must still behave correctly without the extra memory. — https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.kernel.increased-memory-limit — verified on 2026-11-21
5.4 iOS Simulator
"It isn't possible to use the iOS simulator for developing MLX applications, since MLX requires a modern Metal MTLGPUFamily and the simulator does not provide that." Symptom: failed assertion 'Dispatch Threads with Non-Uniform Threadgroup Size is not supported on this device'. Recommended workarounds: add the Mac (Designed for iPad) destination, or build a multiplatform target and develop on macOS. "you can also use the simulator for developing UI features, you just won't be able to evaluate any MLXArray." — https://github.com/ml-explore/mlx-swift/blob/main/Source/MLX/Documentation.docc/Articles/running-on-ios.md — verified on 2026-11-21
5.5 Background GPU restrictions
- "iOS and tvOS restrict a background app's access to the GPU… If a Metal command queue tries to schedule command buffers after the app moves in the background, the system prevents those commands from executing." A command buffer submitted in the background fails with
MTLCommandBufferStatus.errorandMTLCommandBufferError.Code.notPermitted. — https://developer.apple.com/documentation/metal/preparing-your-metal-app-to-run-in-the-background — verified on 2026-11-21 - iOS 26+ exception: a
BGContinuedProcessingTask, started in the foreground from an explicit user action, can keep using the GPU "on supported devices" after the app is backgrounded, if the app has the Background GPU Access capability, entitlementcom.apple.developer.background-tasks.continued-processing.gpu(iOS 26.0, iPadOS 26.0). The system shows progress UI and the user can cancel. — https://developer.apple.com/documentation/bundleresources/entitlements/com.apple.developer.background-tasks.continued-processing.gpu and https://developer.apple.com/documentation/backgroundtasks/performing-long-running-tasks-on-ios-and-ipados — verified on 2026-11-21 - How MLX itself behaves when the app is backgrounded mid-generation (crash, thrown error, or stall), and whether MLX works inside a
BGContinuedProcessingTask: NOT verified. The MLX docs read do not address it. Safe design: stop generation onscenePhase != .active.
6. Candidate small MLX models for iPhone
Source for every row: https://huggingface.co/api/models/mlx-community/<repo>?blobs=true, queried without a key on 2026-11-21. "Weights" is the sum of siblings[].size for *.safetensors files (each repo has a single model.safetensors). "All files" is the sum over every file in the repo. License and base model are the repo's cardData / tags, reported as-is.
Repo (mlx-community/…) |
Exists | Commit sha | License tag | Base model (card) | Weights (bytes) | All files (bytes) | Params (safetensors) |
|---|---|---|---|---|---|---|---|
Qwen3-0.6B-4bit |
yes | 73e3e38d981303bc594367cd910ea6eb48349da8 |
apache-2.0 |
Qwen/Qwen3-0.6B |
335,450,584 | 351,386,061 | 596,049,920 |
Qwen3-1.7B-4bit |
yes | 3b1b1768f8f8cf8351c712464f906e86c2b8269e |
apache-2.0 |
Qwen/Qwen3-1.7B |
968,080,210 | 984,015,687 | 1,720,574,976 |
Qwen3-4B-4bit |
yes | 4dcb3d101c2a062e5c1d4bb173588c54ea6c4d25 |
apache-2.0 |
Qwen/Qwen3-4B |
2,263,022,529 | 2,278,972,183 | 4,022,468,096 |
gemma-3-1b-it-qat-4bit |
yes | 15fed4eafb456c6fcb2a1165f19ac609670ed14b |
gemma |
google/gemma-3-1b-it |
732,577,304 | 771,863,021 | 1,301,875,840 |
gemma-3-4b-it-qat-4bit |
yes | 3d9ef289111449933c22761961f16a5df237ce2a |
other (license_name: qwen) — see warning |
OpenGVLab/InternVL3-1B-Instruct — see warning |
2,995,351,814 | 3,034,685,868 | 4,971,331,952 |
Llama-3.2-1B-Instruct-4bit |
yes | 08231374eeacb049a0eade7922910865b8fce912 |
llama3.2 |
mlx-community/Llama-3.2-1B-Instruct-bf16 |
695,283,921 | 712,593,855 | 1,235,814,400 |
Llama-3.2-3B-Instruct-4bit |
yes | 7f0dc925e0d0afb0322d96f9255cfddf2ba5636e |
llama3.2 |
mlx-community/Llama-3.2-3B-Instruct |
1,807,496,278 | 1,824,825,759 | 3,212,749,824 |
Phi-4-mini-instruct-4bit |
yes | ac1c269cb4222a4e136a3d09edad301056c1f36a |
mit |
microsoft/Phi-4-mini-instruct |
2,158,100,796 | 2,180,067,259 | 3,836,021,760 |
SmolLM2-1.7B-Instruct-4bit |
no (HTTP 401) | – | – | – | – | – | – |
SmolLM2-360M-Instruct-4bit |
no (HTTP 401) | – | – | – | – | – | – |
Qwen2.5-0.5B-Instruct-4bit |
yes | a5339a4131f135d0fdc6a5c8b5bbed2753bbe0f3 |
apache-2.0 |
Qwen/Qwen2.5-0.5B (relation tagged finetune) |
278,064,920 | 289,601,064 | 494,032,768 |
All rows verified on 2026-11-21. None of the nine existing repos is gated or private.
Warnings and gaps:
gemma-3-4b-it-qat-4bithas inconsistent card metadata. The API returnslicense: other,license_name: qwen,license_linkpointing at a Qwen2.5-72B license,base_model: ["OpenGVLab/InternVL3-1B-Instruct"], tagsinternvlandcustom_code, pipelineimage-text-to-text; whileconfig.model_typeisgemma3. The license and base model in the table are what the card says, not what the name implies. That the weights are really Gemma 3 4B under the Gemma terms: NOT verified. Do not ship this repo before reading itsconfig.jsonand resolving the license. It is also a vision-language repo (haspreprocessor_config.json/processor_config.json), so it needsMLXVLM, not onlyMLXLLM.- The two SmolLM2 4-bit repos do not resolve. Both the API URL and the web page return HTTP 401 with
{"error":"Invalid username or password."}. A search of themlx-communitynamespace (https://huggingface.co/api/models?author=mlx-community&search=SmolLM2) returned only:SmolLM2-1.7B-Instruct,SmolLM2-360M-Instruct,SmolLM2-135M-Instruct,SmolLM2-135M-Instruct-8bit,SmolLM2-360M-Instruct-bf16-mlx,SmolLM2-360M-Instruct-6bit— verified on 2026-11-21. That HTTP 401 means "does not exist" rather than "private": NOT verified (it is the consistent reading with the search result). The quantisation, size and license of the six repos that do exist were not queried: NOT verified. - The Llama 3.2 repos carry the
llama3.2license tag (Meta's community license) and the Gemma repo thegemmatag: both are custom terms, not OSI licenses. Their redistribution and acceptable-use conditions were not read: NOT verified. - Whether each architecture is supported by
mlx-swift-lm3.33.0 (LLMRegistryentries, chat template handling): NOT verified, except that the README usesLLMRegistry.gemma3_1B_qat_4bitand the LLMEval README mentionsLLMRegistry.qwen3_8b_4bitandLLMRegistry.phi4bit. - Runtime memory on device is higher than the weight size (KV cache, buffers); no measurement was made.
7. App Store
7.1 App Review Guidelines
Source: https://developer.apple.com/app-store/review/guidelines/ — text read on 2026-11-21. The page's "last updated" date was not captured: NOT verified.
Downloading model weights after install:
- 2.5.2: "Apps should be self-contained in their bundles, and may not read or write data outside the designated container area, nor may they download, install, or execute code which introduces or changes features or functionality of the app". Model weights are data, not executable code; that reading is mine, the guideline does not mention ML models: NOT verified as an Apple statement.
- 4.2.3 (ii): "If your app needs to download additional resources in order to function on initial launch, disclose the size of the download and prompt users before doing so." Show the size and ask before each model download.
- 4.2.3 (i): "Your app should work on its own without requiring installation of another app to function."
- 2.4.2: "Apps should not rapidly drain battery, generate excessive heat, or put unnecessary strain on device resources"; relevant to long on-device generation.
- 2.5.4: background services only for their intended purposes; relevant if generation or downloads continue in the background.
- 2.5.1: public APIs only.
- 5.2.1 / 5.2.2: no third-party trademarks or protected material without permission; "If your app uses, accesses, monetizes access to, or displays content from a third-party service, ensure that you are specifically permitted to do so under the service's terms of use." Applies to provider names and logos, and to redistributing or linking model weights under their licenses.
BYOK cloud AI chat:
- 5.1.2 (i): "You must clearly disclose where personal data will be shared with third parties, including with third-party AI, and obtain explicit permission before doing so." Add an explicit consent step naming the provider before the first request to OpenAI, Anthropic or OpenRouter.
- 5.1.1 (i): privacy policy link in App Store Connect and inside the app. 5.1.1 (ii): consent for collected data, and a way to withdraw it.
- 1.2 User-Generated Content: covers "AI content creation apps"; requires a method for filtering objectionable material, a reporting mechanism, the ability to block abusive users, and published contact information.
- 4.7: "Apps may offer certain software that is not embedded in the binary, specifically HTML5 and JavaScript mini apps and mini games, streaming games, chatbots, and plug-ins." 4.7.1 requires such software to follow the privacy guidelines, "include a method for filtering objectionable material, a mechanism to report content and timely responses to concerns", and follow 3.1 for digital goods. 4.7.5 requires an age-restriction mechanism for content exceeding the app's age rating. Whether a BYOK chat client talking to third-party models is treated as a "chatbot" under 4.7: NOT verified.
- 2.1 (a): the app must be fully functional for review; a BYOK-only app gives the reviewer nothing to test without a key. Provide a working path (the on-device models, or a review key in the review notes). The guideline text covers demo accounts and demo modes; it does not mention API keys: NOT verified as an explicit rule.
- 2.3.6: answer the age-rating questionnaire honestly. Which age-rating answer Apple currently expects for unrestricted AI chat: NOT verified.
- 3.1.1 / 3.1.3: in-app purchase rules. That entering a user's own third-party API key is not a "purchase" under 3.1.1 is not stated in the guidelines: NOT verified. The app must not sell credits or link to provider billing pages outside what 3.1.1(a) / 3.1.3 allow.
7.2 Privacy manifest (PrivacyInfo.xcprivacy)
- "Starting May 1, 2024, apps that don't describe their use of required reason API in their privacy manifest file aren't accepted by App Store Connect." Required on iOS, iPadOS, tvOS, visionOS, watchOS. Each used category goes in
NSPrivacyAccessedAPITypesas a dictionary withNSPrivacyAccessedAPITypeandNSPrivacyAccessedAPITypeReasons. A third-party SDK must ship its own manifest; the app cannot declare on its behalf. — https://developer.apple.com/documentation/bundleresources/describing-use-of-required-reason-api and https://developer.apple.com/documentation/bundleresources/privacy-manifest-files — verified on 2026-11-21 - The manifest also carries
NSPrivacyTracking,NSPrivacyTrackingDomains,NSPrivacyCollectedDataTypes. — privacy-manifest-files page — verified on 2026-11-21
Reason codes per category (https://developer.apple.com/documentation/bundleresources/app-privacy-configuration/nsprivacyaccessedapitypes/nsprivacyaccessedapitype — verified on 2026-11-21):
| Category | Code | Meaning (abridged from Apple's text) |
|---|---|---|
NSPrivacyAccessedAPICategoryUserDefaults |
CA92.1 |
read/write information only accessible to the app itself |
1C8F.1 |
information shared within the same App Group | |
C56D.1 |
third-party SDK wrapper only | |
AC6B.1 |
MDM managed app configuration / feedback keys | |
NSPrivacyAccessedAPICategoryFileTimestamp |
DDA9.1 |
display file timestamps to the user; not sent off-device |
C617.1 |
timestamps, size or metadata of files inside the app container, app group container or CloudKit container | |
3B52.1 |
files or directories the user specifically granted access to (document picker) | |
0A2A.1 |
third-party SDK wrapper only | |
NSPrivacyAccessedAPICategoryDiskSpace |
85F4.1 |
display disk space to the user |
E174.1 |
check there is sufficient disk space to write files, or delete files when space is low; behaviour must be observable to users. Not sent off-device, with "an exception that allows the app to avoid downloading files from a server when disk space is insufficient" | |
7D9E.1 |
optional user-submitted bug report | |
B728.1 |
health research apps | |
NSPrivacyAccessedAPICategorySystemBootTime |
35F9.1 |
measure elapsed time between events in the app, or enable timers |
8FFB.1 |
calculate absolute timestamps for in-app events | |
3D61.1 |
optional user-submitted bug report | |
NSPrivacyAccessedAPICategoryActiveKeyboards |
3EC4.1, 54BD.1 |
custom keyboard apps / customised UI |
Likely declarations for this app (my mapping of the app's behaviour onto Apple's text, not an Apple statement): CA92.1 for settings in UserDefaults; C617.1 for listing downloaded model files with size and date; E174.1 for the free-space check before a model download; 35F9.1 for tokens-per-second timing if it uses mach_absolute_time() / systemUptime.
- The exact list of Swift/Foundation symbols in each category (e.g. whether
ContinuousClock,Date,URLResourceValues.volumeAvailableCapacityForImportantUsage,FileManager.attributesOfItemcount): only the C-level lists were seen (mach_absolute_time(),statfamily,statfsfamily,getattrlistfamily); the Foundation symbol lists were not extracted: NOT verified. - Whether
mlx-swift,mlx-swift-lm,swift-transformersandswift-huggingfaceship their own privacy manifests: NOT verified.
7.3 Export compliance
Source: https://developer.apple.com/documentation/security/complying-with-encryption-export-regulations and https://developer.apple.com/documentation/bundleresources/information-property-list/itsappusesnonexemptencryption — verified on 2026-11-21.
- Any app distributed outside the U.S. or Canada that "uses, accesses, contains, implements, or incorporates encryption" is subject to U.S. export compliance requirements.
ITSAppUsesNonExemptEncryptioninInfo.plist: "Set the value toNOif your app—including any third-party libraries it links against—doesn't use encryption, or if it only uses forms of encryption that are exempt from export compliance documentation requirements. Otherwise, set it toYES."- "Typically, the use of encryption that's built into the operating system—for example, when your app makes HTTPS connections using URLSession—is exempt from export documentation upload requirements, whereas the use of proprietary encryption is not."
- Without the key, "App Store Connect walks you through an export compliance questionnaire every time you upload a new version".
- "If your app uses exempt forms of encryption, you might alternatively be required to submit a year-end self-classification report to the U.S. government."
- For an app that uses only HTTPS via
URLSession, the Keychain and CryptoKit (Apple-provided crypto):ITSAppUsesNonExemptEncryption = NOis consistent with the text above. Whether a given app owes the year-end self-classification report, and the French encryption declaration for distribution in France: NOT verified. - The App Store Connect help page on encryption documentation (https://developer.apple.com/help/app-store-connect/reference/app-information/export-compliance-documentation-for-encryption) was fetched but its body was not extracted: NOT verified.
8. Open items (not verified in this session)
- Any real streaming capture from the three providers (needs a key).
- OpenAI: image limits and MIME types; presence of
[DONE]on Responses streams; the exact error for a rejectedtemperature; whethergpt-6.1-terra/gpt-6-lunaaccept sampling parameters; the consolidated pricing table. - Anthropic: full
stop_reasonlist; sunset date ofanthropic-version: 2023-06-01. - OpenRouter: top-level
provideron successful chunks;GET /api/v1/generationfields;limit: nullsemantics on/api/v1/key. - Apple: which devices get AFM Core vs AFM Core Advanced; the OS 27
LanguageModelErrorcase for context overflow; whether the consumer language list matchessupportedLanguages. - MLX: behaviour when backgrounded mid-generation;
Package.swiftat the release tags; minimum Xcode; privacy manifests of the dependencies; architecture support for each candidate model in 3.33.0. - Hugging Face: real provenance and license of
mlx-community/gemma-3-4b-it-qat-4bit; a replacement for the two missing SmolLM2 4-bit repos. - App Review: how BYOK key entry, age rating and guideline 4.7 are applied in practice to AI chat clients.