OpenAI provider audit for PolyLLM
Last documentation audit: 2026-09-08
SDK probed: openai@7.10.0 (npm), Node 25.9.0, tsx 4.23.13. Probe scripts and raw results: research/openai/ (results/*.json, no secrets).
Model registry produced from this audit: docs/provider-research/openai.models.json (84 entries).
Docs moved: every
platform.openai.com/docs/...URL now 301-redirects tohttps://developers.openai.com/api/docs/.... The API reference pages there render only partially through a headless fetch (the "create response" and "streaming events" pages surfaced only the cancel endpoint), so event names, enums and parameter lists below were cross-checked against the SDK type definitions (node_modules/openai/resources/responses/responses.d.ts,shared.d.ts) and against live probes. Where docs and probes disagree, the probe wins and the disagreement is called out.
1. Endpoint, auth, headers
| Item | Value |
|---|---|
| Base URL | https://api.openai.com/v1/ (docs also mention data-residency endpoints in the SDK 7.6 release notes; not needed for BYOK) |
| Auth | Authorization: Bearer <OPENAI_API_KEY> |
| Optional headers | OpenAI-Organization: org_..., OpenAI-Project: proj_... (billing scope), X-Client-Request-Id (ASCII, <= 512 chars), OpenAI-Safety-Identifier (Realtime only; for HTTP use the safety_identifier body param) |
| Response headers seen | x-request-id, openai-processing-ms, openai-version: 2020-10-01, openai-organization, openai-project, x-ratelimit-* (see section 13) |
| Header size limits | total < 64 KiB, single custom header value < 60 KiB |
OpenAI-Beta |
Not required for Responses, Chat Completions, tools or structured outputs (none sent by SDK 7.x for these). |
Source: https://developers.openai.com/api/docs/api-reference/introduction
2. Which API: Responses vs Chat Completions
- Use the Responses API (
POST /v1/responses) as the primary adapter. Docs: "While Chat Completions remains supported, Responses is recommended for all new projects." Chat Completions is not labelled legacy/deprecated and has no shutdown date, but:- Pro models (
gpt-5.5-pro,gpt-5.4-pro,gpt-5.2-pro,gpt-5-pro,o3-pro,o1-pro),gpt-5.3-codexandgpt-5.6-cyberare Responses-only. - Built-in tools (web_search, file_search, code_interpreter, MCP, image_generation, computer use) exist only in Responses (Chat Completions has only
web_search_optionson the*-search-*models). - Reasoning docs: "Reasoning models work better with the Responses API"; GPT-6 Astra page: "Requires Responses API for tool calling (Chat Completions available but limited)".
- Reasoning summaries, encrypted reasoning,
previous_response_id,conversation, background mode,reasoning.contextare Responses-only.
- Pro models (
- Chat Completions still works (verified 2026-09-08 on gpt-4.1-mini, gpt-5.5, o4-mini): chunk shape
chat.completion.chunkwithchoices[0].delta,usageonly in the final chunk whenstream_options.include_usage: true; usage fieldsprompt_tokens/completion_tokens/prompt_tokens_details.cached_tokens/completion_tokens_details.reasoning_tokens.max_tokensis rejected on new models (gpt-6-astra:400 unsupported_parameter "'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead."). - The Chat-Completions-only web search models (
gpt-5-search-api,gpt-4o-search-preview,gpt-4o-mini-search-preview) return400 model_not_found "The requested model 'gpt-5-search-api' is not supported with the Responses API."on Responses. Do not expose them; use theweb_searchtool on a normal model. - Assistants API shut down 2026-08-26 (replaced by Responses + Conversations).
v1/prompts(reusable prompts), Evals, Agent Builder shut down 2026-11-30.
Sources: https://developers.openai.com/api/docs/api-reference/chat , https://developers.openai.com/api/docs/guides/migrate-to-responses , https://developers.openai.com/api/docs/deprecations
Chat Completions -> Responses mapping (for the adapter)
| Chat Completions | Responses |
|---|---|
messages |
input (string, or array of items: messages {role, content} / function_call / function_call_output / reasoning ...) |
system message |
instructions (top-level) or a developer (or system) role message; all three verified to work |
max_tokens / max_completion_tokens |
max_output_tokens (min 16; includes reasoning tokens) |
response_format |
text.format ({type:"text"} / {type:"json_schema", name, schema, strict} / {type:"json_object"}) |
tools[].function.{name,parameters} |
flattened tools[] = {type:"function", name, description, parameters, strict} |
tool_calls[] in message |
separate function_call output items (call_id, name, arguments) |
role:"tool" message |
{type:"function_call_output", call_id, output} input item |
reasoning_effort, verbosity |
reasoning.effort, text.verbosity |
usage.prompt_tokens / completion_tokens |
usage.input_tokens / output_tokens (+ input_tokens_details.cached_tokens, cache_write_tokens, output_tokens_details.reasoning_tokens, total_tokens) |
n, seed, stop, logprobs (bool) |
n removed; seed and stop do not exist (400 unknown_parameter); logprobs via top_logprobs + include:["message.output_text.logprobs"] |
stream_options.include_usage |
not needed: response.completed carries the full response.usage |
3. Request shape (Responses) - parameters that matter
From SDK ResponseCreateParams (7.10.0) and probes:
model, input, instructions, max_output_tokens (>= 16), reasoning {effort, summary, mode, context}, text {format, verbosity}, tools, tool_choice ("auto"|"none"|"required"|{type:"function",name}|{type:"allowed_tools",...}), parallel_tool_calls, temperature, top_p, top_logprobs, include[], store (default true), previous_response_id, conversation, stream, stream_options {include_obfuscation}, background, truncation ("auto"|"disabled", default disabled), metadata, service_tier (auto|default|flex|priority|fast), prompt_cache_key, prompt_cache_retention (in_memory|24h, pre-5.6), prompt_cache_options {mode, ttl:"30m"} (5.6+), safety_identifier, user, context_management (compaction), moderation, prompt.
include values: file_search_call.results, web_search_call.results, web_search_call.action.sources, message.input_image.image_url, computer_call_output.output.image_url, code_interpreter_call.outputs, reasoning.encrypted_content, message.output_text.logprobs.
Response object: id (resp_...), object:"response", status (completed|failed|in_progress|cancelled|queued|incomplete), incomplete_details.reason (max_output_tokens|max_messages|content_filter|steered), error, model (resolved snapshot, e.g. gpt-5.5-2026-04-23), output[] items, usage, plus echoes of reasoning, text, temperature, top_p, truncation, store, service_tier, previous_response_id, tools. The SDK adds a convenience output_text getter only on non-streaming create() results (the response object inside a response.completed event does not have it - accumulate deltas or read output[].content[].text).
4. Streaming protocol
SSE; every event is JSON with type and a monotonically increasing sequence_number (verified: strictly +1 per event). Text deltas also carry an obfuscation padding string (disable via stream_options.include_obfuscation:false).
Event names observed in probes (exact strings):
response.created > response.in_progress
> response.output_item.added {item:{type:"reasoning"...}} (reasoning models, effort >= low)
> response.reasoning_summary_part.added
> response.reasoning_summary_text.delta {delta} (x N) (only when reasoning.summary is set AND the model emits one)
> response.reasoning_summary_text.done {text}
> response.reasoning_summary_part.done
> response.output_item.done
> response.output_item.added {item:{type:"message", role:"assistant"}}
> response.content_part.added {part:{type:"output_text", text:"", annotations:[]}}
> response.output_text.delta {delta, item_id, output_index, content_index, logprobs, obfuscation} (x N)
> response.output_text.annotation.added {annotation:{type:"url_citation",...}, annotation_index} (web search)
> response.output_text.done {text}
> response.content_part.done
> response.output_item.done
> response.completed {response:{...usage}} | response.incomplete | response.failed | errorFunction calls (verified on all six models):
response.output_item.added {item:{type:"function_call", id:"fc_...", call_id:"call_...", name, arguments:"", status:"in_progress"}, output_index}
response.function_call_arguments.delta {delta, item_id, output_index}
response.function_call_arguments.done {arguments:"{...}", item_id}
response.output_item.done {item:{type:"function_call", ..., arguments:"{...}", status:"completed"}}
response.completedWeb search: response.web_search_call.in_progress > .searching > .completed (item type web_search_call).
Full list of event types in the SDK (for exhaustive switch statements): response.created, response.in_progress, response.queued, response.completed, response.incomplete, response.failed, error, response.output_item.added/done, response.content_part.added/done, response.output_text.delta/done, response.output_text.annotation.added, response.refusal.delta/done, response.reasoning_text.delta/done, response.reasoning_summary_part.added/done, response.reasoning_summary_text.delta/done, response.function_call_arguments.delta/done, response.custom_tool_call_input.delta/done, response.web_search_call.in_progress/searching/completed, response.file_search_call.in_progress/searching/completed, response.code_interpreter_call.in_progress/interpreting/completed, response.code_interpreter_call_code.delta/done, response.image_generation_call.in_progress/generating/partial_image/completed, response.mcp_call.in_progress/completed/failed, response.mcp_call_arguments.delta/done, response.mcp_list_tools.in_progress/completed/failed, response.shell_call_command.added/delta/done, response.shell_call_output_content.delta/done, response.audio.delta/done, response.audio.transcript.delta/done, response.steer.*.
Resumable streams: with background:true, stream:true, reconnect with GET /v1/responses/{id}?stream=true&starting_after={sequence_number}.
Sources: https://developers.openai.com/api/docs/guides/streaming-responses , https://developers.openai.com/api/docs/guides/background , SDK types.
Exact SDK streaming code that worked (research/openai/03-stream.ts, 05-tools-stream.ts)
import OpenAI from "openai";
const client = new OpenAI({ maxRetries: 1, timeout: 180_000 }); // key from OPENAI_API_KEY
const stream = await client.responses.create({
model: "gpt-5.5",
input: [{ role: "user", content: "Say hello in five words." }],
reasoning: { effort: "low", summary: "auto" },
max_output_tokens: 200,
stream: true,
});
let text = "", final: OpenAI.Responses.Response | null = null;
for await (const ev of stream) {
switch (ev.type) {
case "response.output_text.delta": text += ev.delta; break;
case "response.reasoning_summary_text.delta": /* show thinking */ break;
case "response.function_call_arguments.delta": /* accumulate by ev.item_id */ break;
case "response.output_item.done": if (ev.item.type === "function_call") { /* ev.item.call_id, name, arguments */ } break;
case "response.completed": case "response.incomplete": final = ev.response; break; // final.usage, final.output
case "error": throw new Error(ev.message);
}
}Tool round trip (stateless): replay [...previousInput, ...final.output, { type: "function_call_output", call_id, output: JSON.stringify(result) }]. Replaying the entire output array (including reasoning items) is what the docs require for reasoning models; verified on gpt-5.5, gpt-5.4-mini, gpt-5.6-sol, gpt-6-astra, gpt-4.1-mini. Stateful alternative verified on gpt-5.4-mini: previous_response_id: final.id, input: [function_call_output].
5. Tool calling format
- Definition:
{ type: "function", name, description, parameters: <JSON Schema>, strict: true }(flattened;strictrecommended; strict requiresadditionalProperties:falseand all properties inrequired). - Output item:
{ id: "fc_...", type: "function_call", status: "completed", call_id: "call_...", name, arguments: "<json string>" }(verified shape).call_idis what you echo back;idis the item id. - Result item:
{ type: "function_call_output", call_id, output: string | [content parts for images/files] }(SDK 7.7 made theidoptional). tool_choice:"auto"(default) |"none"|"required"|{type:"function", name}|{type:"allowed_tools", mode:"auto"|"required", tools:[...]}.parallel_tool_callsdefault true.- Other tool types accepted in
tools[]:web_search(and legacyweb_search_preview),file_search,code_interpreter,image_generation,mcp,computer/computer_use_preview,shell,local_shell,apply_patch,custom(free-form text tool),namespace(grouping,defer_loading),tool_search,programmatic_tool_calling.
Sources: https://developers.openai.com/api/docs/guides/function-calling , https://developers.openai.com/api/docs/guides/tools
6. Structured output / JSON schema
text: { format: { type: "json_schema", name, schema, strict: true } }- verified on gpt-5.5, gpt-5.4-mini, gpt-5.6-sol, gpt-6-astra, gpt-4.1-mini (parsed{name:"Marie Curie",age:66,city:"Paris"}); o4-mini accepted it but ran out of the 200-token budget on reasoning.- Schema subset: root must be
object; every property inrequired;additionalProperties:false; optional fields viatype:["string","null"]; supportsenum,anyOf,$ref/$defs, recursion; nominLength/pattern/formatetc. Numeric limits (depth/property count/enum size) exist but the page did not expose them - keep schemas modest. - Refusals arrive as a
refusalcontent part (response.refusal.delta/done) instead of JSON. {type:"json_object"}(JSON mode) is the fallback for gpt-4 / gpt-3.5-turbo which lack structured outputs.- SDK helpers:
zodTextFormat(schema, "name")+client.responses.parse(...)->response.output_parsed.
Source: https://developers.openai.com/api/docs/guides/structured-outputs
7. Reasoning controls
reasoning.effortvalues in the SDK enum:none | minimal | low | medium | high | xhigh | max. Per-model acceptance (verified, exact error strings in section 20):- gpt-6-astra:
low, medium, high, xhigh, max(defaultmedium;none/minimal-> 400) - gpt-5.6-sol/terra/luna:
none, low, medium, high, xhigh, max(defaultmedium) - gpt-5.5:
none, low, medium, high, xhigh(defaultmedium) - gpt-5.4 / 5.4-mini / 5.4-nano, gpt-5.2:
none, low, medium, high, xhigh(defaultnone- verified echo) - gpt-5.1:
none, low, medium, high(defaultnone) - gpt-5 / 5-mini / 5-nano:
minimal, low, medium, high(defaultmedium; nonone) - o3, o4-mini, o3-mini, o1:
low, medium, high(defaultmedium) - pro models: 5.5-pro
medium|high|xhigh(default high), 5.4-pro / 5.2-promedium|high|xhigh, gpt-5-prohighonly - gpt-4.1 / gpt-4o / chat-latest:
reasoning.effort->400 unsupported_parameter
- gpt-6-astra:
reasoning.summary:auto | concise | detailed(server echoesdetailedforauto). Summary text arrives inreasoningitems'summary[]and viaresponse.reasoning_summary_*events; gpt-5.5 and gpt-6-astra returned no summary on a trivial prompt while gpt-5.6-sol and o4-mini did. Never assume a summary will exist. Non-reasoning models silently ignoresummary.reasoning.mode:standard | pro(GPT-5.6; pro replaces the*-promodels). Not probed (cost).reasoning.context:current_turn | all_turns(all_turnsis the default on 5.6 and 6; earlier models defaultcurrent_turn). Reasoning persists only within a family.- Reasoning tokens are billed as output and count against
max_output_tokens;usage.output_tokens_details.reasoning_tokensreports them. Docs recommend reserving >= 25k tokens; with tiny caps, o4-mini / gpt-5-nano returnstatus:"incomplete", incomplete_details.reason:"max_output_tokens"with an empty message (verified). Adapter rule: never cap reasoning models below ~1-2k output tokens; treatincomplete+ empty text as "reasoning exhausted the budget". - Encrypted reasoning for stateless (store:false) multi-turn:
store:false, include:["reasoning.encrypted_content"]->reasoningitems carryencrypted_content(~1.2 kB for a trivial turn); replaying them in the nextinputworks (verified on gpt-5.5).previous_response_idon astore:falseresponse fails:400 previous_response_not_found. - GPT-6 Astra: mid-conversation
configuration_updateitems to change effort; async tool calling (async:true); nononeeffort.
Sources: https://developers.openai.com/api/docs/guides/reasoning , https://developers.openai.com/api/docs/guides/latest-model
8. Sampling parameter support matrix (Responses API, verified 2026-09-08)
| Param | gpt-6-astra | gpt-5.6-sol | gpt-5.5 | gpt-5.4-mini (default effort none) | gpt-4.1-mini | o4-mini |
|---|---|---|---|---|---|---|
temperature |
rejected | rejected unless effort:"none" (then accepted) |
rejected unless effort:"none" |
accepted | accepted | rejected |
top_p |
rejected | same as temperature | same | accepted | accepted | rejected |
frequency_penalty |
rejected | rejected (effort medium) | rejected | accepted | accepted | rejected |
presence_penalty |
rejected | rejected | rejected | accepted | accepted | rejected |
seed |
unknown_parameter on every model (does not exist in Responses) |
|||||
stop |
unknown_parameter on every model ("Did you mean 'store'?") |
|||||
logprobs (top_logprobs+include) |
rejected (reasoning) | rejected | rejected | accepted (logprobs[] on content part) |
accepted | rejected |
reasoning.effort |
low..max | none..max | none..xhigh | none..xhigh | rejected | low..high |
reasoning.summary |
accepted | accepted | accepted | accepted | ignored (no error) | accepted |
text.verbosity: "low" |
accepted | accepted | accepted | accepted | rejected (Supported values are: 'medium') |
rejected |
max_output_tokens |
min 16 on all (integer_below_min_value) |
|||||
truncation:"auto" |
accepted on all | |||||
store:false + include:["reasoning.encrypted_content"] |
accepted on all (non-reasoning models just have no reasoning item) | |||||
prompt_cache_key |
accepted on all |
Default top_p echoed by GPT-5.x/6 is 0.98, by GPT-4.1 1. temperature default 1. top_k does not exist. There is no thinking budget parameter (effort only). Chat Completions on gpt-5.5: temperature:0.3 -> 400 unsupported_value "'temperature' does not support 0.3 with this model. Only the default (1) value is supported."
Practical rule for the adapter: send sampling params only when (model is non-reasoning) or (reasoning-capable model with reasoning.effort === "none"); otherwise drop them client-side and show "not supported" in the UI. Never send seed/stop to Responses.
9. Roles and instructions
Input roles accepted: user, assistant, developer, system (both system and developer verified on gpt-5.5 and gpt-4.1-mini; docs describe developer as the canonical name and treat instructions as equivalent to a developer message). instructions is not inherited through previous_response_id; resend it each call. Content parts: input_text, input_image, input_file, (input_audio rejected, see 10); assistant output parts: output_text, refusal.
Source: https://developers.openai.com/api/docs/guides/text
10. Modalities
| Modality | Responses API support | Notes |
|---|---|---|
| Text in/out | yes | |
| Image in | yes on every current model except o3-mini, gpt-4, gpt-3.5-turbo |
{type:"input_image", image_url:"data:image/png;base64,..." | https URL | file_id, detail:"low"|"high"|"auto"|"original"}. PNG/JPEG/WEBP/non-animated GIF; <= 512 MB per request, <= 1,500 images, <= 30,000 patches after resize. Tokens ~ ceil(w/32)*ceil(h/32) x model multiplier (1.2-2.46). Verified with a 2x2 PNG on all six probe models (31-33 input tokens). |
| PDF / file in | yes (vision models) | {type:"input_file", filename, file_data:"data:application/pdf;base64,..." | file_id | file_url}; <= 50 MB per file and per request; both text and page images are sent (costly). Verified on gpt-5.4-mini (answered 42 from the PDF). |
| Image out | via tool only | tools:[{type:"image_generation"}] uses gpt-image-2; returns base64 in an image_generation_call item; events response.image_generation_call.partial_image. Not probed. Older gpt-image-1* shut down 2026-10-23 / 2026-12-01. |
| Audio in/out | no in Responses | probe: 400 invalid_request_error param=input "Audio input is not available.". Audio chat = Chat Completions with gpt-audio-1.5 (input_audio parts, modalities:["text","audio"]) or Realtime (gpt-realtime-2.1). TTS: gpt-4o-mini-tts; STT: gpt-transcribe, gpt-live-transcribe (whisper-1/gpt-4o-transcribe shut down 2027-02-26). |
| Video | no | Sora 2 / Videos API shut down 2026-09-24. |
Sources: https://developers.openai.com/api/docs/guides/images-vision , https://developers.openai.com/api/docs/guides/pdf-files , https://developers.openai.com/api/docs/guides/audio
11. Context windows, max output, pricing (per 1M tokens, USD, standard tier)
| Model | Context | Max output | Input | Cached | Output | Notes |
|---|---|---|---|---|---|---|
| gpt-6-astra | 1,050,000 | 128,000 | 10 | 1 | 50 | cache write 12.50; >272K input: 2x in / 1.5x out |
gpt-5.6-sol (gpt-5.6) |
1,050,000 | 128,000 | 4 | 0.40 | 20 | |
| gpt-5.6-terra | 1,050,000 (922K input) | 128,000 | 2 | 0.20 | 12 | |
| gpt-5.6-luna | 1,050,000 | 128,000 | 0.20 | 0.02 | 1.20 | |
| gpt-5.5 | 1,050,000 | 128,000 | 5 | 0.50 | 30 | >272K: 2x/1.5x |
| gpt-5.5-pro | 1,050,000 | 128,000 | 30 | - | 180 | Responses only |
| gpt-5.4 | 1,050,000 | 128,000 | 2.50 | 0.25 | 15 | |
| gpt-5.4-mini | 400,000 | 128,000 | 0.75 | 0.075 | 4.50 | |
| gpt-5.4-nano | 400,000 | 128,000 | 0.20 | 0.02 | 1.25 | |
| gpt-5.4-pro | 1,050,000 | 128,000 | 30 | - | 180 | Responses only |
| gpt-5.3-codex | 400,000 | 128,000 | 1.75 | 0.175 | 14 | Responses only, coding |
| gpt-5.2 | 400,000 | 128,000 | 1.75 | 0.175 | 14 | |
| gpt-5.2-pro | 400,000 | 128,000 | 21 | - | 168 | Responses only |
| gpt-5.1 | 400,000 | 128,000 | 1.25 | 0.125 | 10 | |
| gpt-5 | 400,000 | 128,000 | 1.25 | 0.125 | 10 | snapshot shuts down 2026-12-11 |
| gpt-5-mini | 400,000 | 128,000 | 0.25 | 0.025 | 2 | snapshot shuts down 2026-12-11 |
| gpt-5-nano | 400,000 | 128,000 | 0.05 | 0.005 | 0.40 | snapshot shuts down 2026-12-11 |
| gpt-5-pro | 400,000 | 272,000 | 15 | - | 120 | Responses only |
| chat-latest | 400,000 (272K input) | 128,000 | 5 | 0.50 | 30 | ChatGPT Instant, non-reasoning |
| gpt-4.1 | 1,047,576 | 32,768 | 2 | 0.50 | 8 | |
| gpt-4.1-mini | 1,047,576 | 32,768 | 0.40 | 0.10 | 1.60 | |
| gpt-4.1-nano | 1,047,576 | 32,768 | 0.10 | 0.025 | 0.40 | shuts down 2026-10-23 |
| gpt-4o | 128,000 | 16,384 | 2.50 | 1.25 | 10 | alias -> 2024-08-06 |
| gpt-4o-mini | 128,000 | 16,384 | 0.15 | 0.075 | 0.60 | |
| o3 | 200,000 | 100,000 | 2 | 0.50 | 8 | snapshot shuts down 2026-12-11 |
| o3-pro | 200,000 | 100,000 | 20 | - | 80 | Responses only |
| o4-mini | 200,000 | 100,000 | 1.10 | 0.275 | 4.40 | shuts down 2026-10-23 |
| o3-mini | 200,000 | 100,000 | 1.10 | 0.55 | 4.40 | text only; shuts down 2026-10-23 |
| o1 | 200,000 | 100,000 | 15 | 7.50 | 60 | shuts down 2026-10-23 |
| o1-pro | 200,000 | 100,000 | 150 | - | 600 | shuts down 2026-10-23 |
| gpt-4-turbo | 128,000 | 4,096 | 10 | - | 30 | shuts down 2026-10-23 |
| gpt-4 | 8,192 | 8,192 | 30 | - | 60 | shuts down 2026-10-23 |
| gpt-3.5-turbo | 16,385 | 4,096 | 0.50 | - | 1.50 | shuts down 2026-10-23 |
Tiers: Batch and Flex = ~50% of standard; Priority higher; Fast = 2x (not for gpt-6-astra with EU residency). Regional data residency +10% for models released after 2026-03-05. Tools: web search $10 / 1k calls (+ tokens), file search $2.50 / 1k calls + $0.10/GB/day storage (1 GB free), code interpreter $0.03-$1.92 per 20-min session by memory tier, MCP = tokens only. Unknown for this key: gpt-5-search-api, gpt-4o-mini-search-preview, the dead codex snapshots (set to null in the JSON).
Sources: per-model pages under https://developers.openai.com/api/docs/models/ , https://developers.openai.com/api/docs/pricing
12. Prompt caching
Automatic prefix caching; minimum cacheable prefix 1,024 tokens (GPT-5.6+) / 2,048 (earlier). Reads at the "cached" price (0.1x on 5.6+, 0.5x on gpt-4o/o3-mini/o1); GPT-5.6+ also bill cache writes at 1.25x (reported in usage.input_tokens_details.cache_write_tokens, present in every usage object we saw). prompt_cache_key (routing hint, accepted on every model) ; prompt_cache_retention: "in_memory" | "24h" for pre-5.6 models (accepted on gpt-5.5 and even gpt-6-astra, but docs say it is deprecated for 6 -> use prompt_cache_options: { ttl: "30m" }, verified accepted on gpt-5.6-sol, echoed {mode:"implicit", ttl:"30m"}). cached_tokens is exact on 5.6+, rounded to 128 earlier. Cache lifetime 5-10 min (up to 1 h) or 24 h; not shared across orgs.
Source: https://developers.openai.com/api/docs/guides/prompt-caching
13. Rate limits, headers, retries, timeouts
- Dimensions: RPM, TPM, RPD, TPD, IPM (+ audio minutes). Tiers Free..Tier 5 by cumulative spend; this key is Tier 5 (headers show 15,000 RPM / 40M TPM on gpt-5.5, 30,000 RPM / 150M TPM on gpt-4.1-mini).
- Headers (verified on every 200):
x-ratelimit-limit-requests,x-ratelimit-remaining-requests,x-ratelimit-reset-requests(e.g.4ms,6m0s),x-ratelimit-limit-tokens,x-ratelimit-remaining-tokens,x-ratelimit-reset-tokens; docs also listx-ratelimit-limit-project-tokens/x-ratelimit-remaining-project-tokensandRetry-After(seconds) on 429/503. Alsox-request-idandopenai-processing-ms. - 429 codes:
rate_limit_exceeded,slow_down(traffic ramped too fast),insufficient_quota/credit_balance_exhausted,organization_spend_limit_exceeded,project_spend_limit_exceeded,organization_usage_limit_exceeded. 503server_is_overloaded. Billing/quota 429s must not be retried. - Retry guidance: honour
Retry-Afterwhen present, else exponential backoff with jitter; ramp traffic <= +50% per 15 min above 1M TPM. SDK default:maxRetries: 2, retries 408/409/429/>=500 and any response withx-should-retry: true, honoursretry-after-ms/retry-after, backoff 0.5 s x 2^n capped at 8 s with up to 25% jitter. For a BYOK chat UI usemaxRetries: 1-2for non-streaming, and do not retry once streaming has begun. - Timeouts: SDK default 10 minutes (
OpenAI.DEFAULT_TIMEOUT = 600000). Recommend: 60 s connect/first-byte for chat, overall 5-10 min for high/xhigh/max effort, andbackground: true(+ polling / resumable stream) for pro models and anything expected to run > 1-2 min. Docs: streaming is "the single most effective approach" to perceived latency;service_tier: "priority" | "fast"buys lower latency at higher price (flexaccepted on gpt-5.4-mini: slower, cheaper).
Sources: https://developers.openai.com/api/docs/guides/rate-limits , https://developers.openai.com/api/docs/guides/error-codes , https://developers.openai.com/api/docs/guides/latency-optimization , SDK client.js
14. Error schema
HTTP body: { "error": { "message", "type", "param", "code" } } (raw fetch with a bad key also returned a top-level "status": 401). SDK maps to BadRequestError (400), AuthenticationError (401), PermissionDeniedError (403), NotFoundError (404), RateLimitError (429), InternalServerError (>=500), APIConnectionError, APITimeoutError; each exposes .status, .code, .type, .param, .requestID, .headers, .error (raw body).
Verified examples:
| Case | Status | type |
code |
param |
message |
|---|---|---|---|---|---|
| invalid key | 401 | invalid_request_error | invalid_api_key |
null | Incorrect API key provided: sk-inva*****key. You can find your API key at https://platform.openai.com/account/api-keys. |
| unknown model | 404 | invalid_request_error | model_not_found |
null | The model `gpt-does-not-exist` does not exist or you do not have access to it. |
| CC-only model on Responses | 400 | invalid_request_error | model_not_found |
model | The requested model 'gpt-5-search-api' is not supported with the Responses API. |
| wrong type | 400 | invalid_request_error | invalid_type |
max_output_tokens | Invalid type for 'max_output_tokens': expected an integer, but got a string instead. |
| below min | 400 | invalid_request_error | integer_below_min_value |
max_output_tokens | Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead. |
| unsupported param | 400 | invalid_request_error | null | temperature | Unsupported parameter: 'temperature' is not supported with this model. |
| unsupported param | 400 | invalid_request_error | unsupported_parameter |
reasoning.effort | Unsupported parameter: 'reasoning.effort' is not supported with this model. |
| unsupported value | 400 | invalid_request_error | unsupported_value |
reasoning.effort | Unsupported value: 'minimal' is not supported with the 'gpt-5.5' model. Supported values are: 'none', 'low', 'medium', 'high', and 'xhigh'. |
| unknown param | 400 | invalid_request_error | unknown_parameter |
stop | Unknown parameter: 'stop'. Did you mean 'store'? |
| logprobs on reasoning | 400 | invalid_request_error | unsupported_parameter |
- | logprobs are not supported with reasoning models. |
| prev response gone | 400 | invalid_request_error | previous_response_not_found |
previous_response_id | Previous response with id 'resp_...' not found. |
| audio in Responses | 400 | invalid_request_error | null | input | Audio input is not available. |
Note code is sometimes null for "Unsupported parameter" errors; match on param + message prefix as well. Other documented codes: context_length_exceeded, invalid_service_tier, insufficient_quota, rate_limit_exceeded, slow_down, server_is_overloaded.
15. Usage reporting and token counting
- Every response (and
response.completed) hasusage: { input_tokens, input_tokens_details: { cached_tokens, cache_write_tokens }, output_tokens, output_tokens_details: { reasoning_tokens }, total_tokens }. Web search inflatesinput_tokensheavily (4.7k-8.8k for a one-line question) because search results are injected as context and billed as input. - Chat Completions usage:
prompt_tokens,completion_tokens,total_tokens,prompt_tokens_details {cached_tokens, audio_tokens},completion_tokens_details {reasoning_tokens, audio_tokens, accepted_prediction_tokens, rejected_prediction_tokens}; streaming needsstream_options: {include_usage: true}(usage arrives in a final chunk with emptychoices). - Pre-flight counting:
POST /v1/responses/input_tokens({model, input, instructions?, tools?, conversation?, previous_response_id?}->{object:"response.input_tokens", input_tokens}); SDK:client.responses.inputTokens.count(...). Client-side:tiktoken/js-tiktokenwitho200k_basefor gpt-4o and newer (approximation only for images/PDF). Rule of thumb 1 token ~ 4 chars English. - SDK 7.8 added
compute_unitstracking on usage for some tiers (not observed in our responses).
Source: https://developers.openai.com/api/docs/api-reference/responses/input-tokens
16. Model listing
GET /v1/models (SDK client.models.list() auto-paginates) returns {id, object:"model", created, owned_by, shutdown_date}. shutdown_date is the only lifecycle signal (ISO date or null; documented as "The date when the model will shut down, or null if not announced"). No capability, context or pricing metadata is exposed, so a static registry is mandatory (openai.models.json). 131 ids were returned for this key on 2026-09-08; 55 carried a shutdown_date, several already in the past (see 18). Filtering heuristics for chat models: exclude ids matching babbage|davinci|instruct|transcribe|tts|whisper|audio|realtime|image|sora|embedding|moderation|search-preview|search-api|deep-research|codex and anything whose shutdown_date < today; keep the rest and look it up in the registry (unknown ids -> show as "unverified").
Source: https://developers.openai.com/api/docs/api-reference/models
17. Aliases
gpt-5.6->gpt-5.6-sol(documented; verifiedresponse.model === "gpt-5.6-sol"; not itself listed by/v1/models).gpt-5.5->gpt-5.5-2026-04-23;gpt-5.4->-2026-03-05;gpt-5.4-mini/-nano->-2026-03-17;gpt-5.2->-2025-12-11;gpt-5.1->-2025-11-13;gpt-5*->-2025-08-07;gpt-4.1*->-2025-04-14;gpt-4o->-2024-08-06(not the newest 11-20 snapshot);gpt-4o-mini->-2024-07-18;o3->-2025-04-16;o4-mini->-2025-04-16;o3-mini->-2025-01-31;gpt-4-turbo->-2024-04-09;gpt-3.5-turbo->-0125.gpt-6-astra,gpt-5.6-*,gpt-5.3-codex,chat-latesthave no dated snapshot (echoed as-is).*-chat-latestaliases (gpt-5 / 5.1 / 5.2 / 5.3) are dead (404) although still listed;chat-latestis the live ChatGPT-Instant alias.
18. Lifecycle / deprecations relevant to a chat app
| Shutdown | Models | Replacement |
|---|---|---|
| 2026-07-23 (passed; 404 today) | gpt-5-chat-latest, gpt-5.1-chat-latest, gpt-5-codex, gpt-5.1-codex(-max/-mini), gpt-5.2-codex, o3-deep-research, o4-mini-deep-research, *-search-preview-2025-03-11 snapshots | gpt-6-astra / gpt-5.3-codex |
| 2026-08-10 (passed; 404) | gpt-5.2-chat-latest, gpt-5.3-chat-latest | chat-latest / gpt-6-astra |
| 2026-08-26 | Assistants API | Responses + Conversations |
| 2026-09-24 | sora-2, sora-2-pro, Videos API | - |
| 2026-09-28 | gpt-3.5-turbo-1106, gpt-3.5-turbo-instruct, babbage-002, davinci-002 | gpt-5.6-terra |
| 2026-10-23 | gpt-3.5-turbo(-0125), gpt-4(-0613), gpt-4-turbo, gpt-4.1-nano, gpt-4o-2024-05-13, gpt-image-1, o1, o1-pro, o3-mini, o4-mini | gpt-5.6-sol / terra / luna (o1-pro -> 5.6-sol reasoning.mode: pro) |
| 2026-11-30 | v1/prompts, Evals, Agent Builder | - |
| 2026-12-01 | gpt-image-1-mini, gpt-image-1.5, chatgpt-image-latest | gpt-image-2 |
| 2026-12-11 | gpt-5-2025-08-07, gpt-5-mini-2025-08-07, gpt-5-nano-2025-08-07, gpt-5-pro-2025-10-06, o3-2025-04-16, o3-pro-2025-06-10 (the bare aliases gpt-5, o3 currently have shutdown_date: null) |
gpt-5.6-sol / terra / luna |
| 2027-01-20 / 2027-02-26 | legacy audio/realtime/transcribe models | gpt-audio-1.5, gpt-realtime-2.1, gpt-transcribe |
Fine-tuning: closed to new orgs since 2026-05-07; no new jobs at all after 2027-01-06.
Source: https://developers.openai.com/api/docs/deprecations + shutdown_date from /v1/models
19. Built-in tools, state, safety
- Conversation state:
storedefaults to true (30-day retention, visible in dashboard logs) - setstore:falsefor privacy-minded BYOK usage and manage history client-side; then useinclude:["reasoning.encrypted_content"]and replay all output items.previous_response_id(verified) andconversation(Conversations API,conv_..., not subject to 30-day TTL) requirestore:true. Server-side compaction:context_management:[{type:"compaction", compact_threshold}]. - Web search:
{type:"web_search", search_context_size?:"low"|"medium"|"high", user_location?:{type:"approximate", country, city, region, timezone}, filters?:{allowed_domains[] | blocked_domains[]} (<=100), external_web_access?, search_content_types?:["text","image"], return_token_budget?}(legacyweb_search_previewstill accepted). Output:web_search_callitems (action: {type:"search", queries[], query} | {type:"open_page", url} | {type:"find_in_page"}) then amessagewhoseoutput_textpart hasannotations: [{type:"url_citation", start_index, end_index, url, title}]and inline markdown links with?utm_source=openai. Events:response.web_search_call.{in_progress,searching,completed},response.output_text.annotation.added. Verified on gpt-4.1-mini and gpt-5.5 (effort low); the gpt-5.5 default-effort attempt at 200 tokens ended with three searches and no message - budget again. Search context capped at 128k. Supported per docs on gpt-6, 5.6, 5.5, 5.4 family, 5.2, 5.1, 5(-mini/-nano), 4.1(-mini), 4o(-mini), o3, o4-mini, chat-latest; not on gpt-4.1-nano, o3-mini, o1, gpt-4, gpt-3.5. - Code execution:
{type:"code_interpreter", container: {type:"auto", memory_limit?:"1g"|"4g"|"16g"|"64g", file_ids?} | "cntr_..."}->code_interpreter_call {code, outputs, container_id}; generated files ascontainer_file_citationannotations; 100 RPM/org; eventsresponse.code_interpreter_call*. Alsoshell(hosted shell),local_shell,apply_patch, skills. - File search:
{type:"file_search", vector_store_ids[], max_num_results?, filters?, ranking_options?}->file_search_call+file_citation {file_id, filename, index}annotations;include:["file_search_call.results"]. - Computer use:
{type:"computer"}(new) orcomputer_use_preview {display_width, display_height, environment};computer_call-> reply withcomputer_call_output {call_id, output:{type:"computer_screenshot", image_url}}; docs recommend GPT-6 Astra with code execution instead. Not relevant for PolyLLM v1. - MCP:
{type:"mcp", server_label, server_url | connector_id, server_description?, authorization?, headers?, allowed_tools?, require_approval:"never"|"always"|{...}, defer_loading?}->mcp_list_tools,mcp_call,mcp_approval_request(answer withmcp_approval_response {approval_request_id, approve}); eventsresponse.mcp_call*,response.mcp_list_tools*. Billing = tokens only. 8 first-party connectors (Gmail, Drive, Calendar, Dropbox, Teams, Outlook x2, SharePoint). - Citations/annotations:
url_citation,file_citation,container_file_citation,file_pathtypes onoutput_text.annotations[]; streamed throughresponse.output_text.annotation.added. - Safety: send
safety_identifier(hashed user id) on every request;incomplete_details.reason:"content_filter"andrefusalparts must be surfaced; free Moderation API (omni-moderation-latest) available; revoke leaked keys immediately.moderationrequest param exists in the SDK types.
Sources: https://developers.openai.com/api/docs/guides/conversation-state , https://developers.openai.com/api/docs/guides/tools-web-search , https://developers.openai.com/api/docs/guides/tools-code-interpreter , https://developers.openai.com/api/docs/guides/tools-file-search , https://developers.openai.com/api/docs/guides/tools-computer-use , https://developers.openai.com/api/docs/guides/tools-remote-mcp , https://developers.openai.com/api/docs/guides/safety-best-practices
20. Probe results (2026-09-08, key = Tier 5 project key)
Legend: OK = completed with expected output; INC = accepted but status:"incomplete" (reasoning consumed the <= 200-token cap); 4xx = rejected with the quoted message.
| Probe | gpt-5.5 | gpt-5.4-mini | gpt-5.6-sol | gpt-6-astra | gpt-4.1-mini | o4-mini |
|---|---|---|---|---|---|---|
| (a) basic Responses, 200 tok | OK (gpt-5.5-2026-04-23, 12 reasoning tok) |
OK (0 reasoning) | OK (0 reasoning) | OK (0 reasoning) | OK | OK (64 reasoning) |
| (b) streaming events | OK, reasoning item but no summary events | OK | OK, reasoning_summary_* events seen |
OK, no summary events | OK | INC: summary events, then response.incomplete |
| (c) temperature / top_p | 400 Unsupported parameter: 'temperature' is not supported with this model. (OK with effort:"none") |
OK (echo 0.5) | 400 same (OK with effort:"none") |
400 same; effort:"none" itself rejected |
OK | 400 same |
| (c) frequency/presence_penalty | 400 Unsupported parameter: 'frequency_penalty' is not supported with this model. |
OK | 400 | 400 | OK | 400 |
| (c) seed | 400 unknown_parameter Unknown parameter: 'seed'. (all models) |
|||||
| (c) stop | 400 unknown_parameter Unknown parameter: 'stop'. Did you mean 'store'? (all models) |
|||||
| (c) logprobs | 400 logprobs are not supported with reasoning models. |
OK | 400 | 400 | OK | 400 |
| (c) effort none | OK | OK | OK | 400 Unsupported value: 'none' is not supported with the 'gpt-6-astra' model. Supported values are: 'low', 'medium', 'high', 'xhigh', and 'max'. |
400 Unsupported parameter: 'reasoning.effort' is not supported with this model. |
400 Supported values are: 'low', 'medium', and 'high'. |
| (c) effort minimal | 400 ... Supported values are: 'none', 'low', 'medium', 'high', and 'xhigh'. |
400 same | 400 ... 'none', 'low', 'medium', 'high', 'xhigh', and 'max'. |
400 | 400 | 400 |
| (c) effort low/medium/high | OK | OK | OK | OK | 400 | OK |
| (c) effort xhigh | OK | OK | OK | OK | 400 | 400 |
| (c) effort max | 400 | 400 | OK | OK | 400 | 400 |
| (c) reasoning.summary auto | OK (echo detailed) |
OK | OK | OK | OK (ignored) | OK |
| (c) text.verbosity low | OK | OK | OK | OK | 400 Unsupported value: 'low' is not supported with the 'gpt-4.1-mini' model. Supported values are: 'medium'. |
400 same wording |
| (c) max_output_tokens 5 | 400 integer_below_min_value Expected a value >= 16, but got 5 instead. (all) |
|||||
| (c) truncation auto, prompt_cache_key, store:false+encrypted | OK on all six | |||||
| (d) function call, streaming | OK: function_call item, args {"city":"Paris","unit":"c"}, round trip -> "18°C and cloudy" |
OK (+ previous_response_id variant OK) |
OK | OK | OK | INC (192 reasoning tokens, no call) |
| (e) json_schema strict | OK parsed | OK | OK | OK | OK | INC |
| (f) vision 2x2 PNG | INC at default effort; OK with effort:"none" |
OK | INC default; OK with effort:"none" |
OK (effort:"low", detail high) |
OK | INC (even at effort:"low") |
| (h) web_search tool | OK with effort:"low": web_search_call + url_citation annotation; default effort at 200 tok: 3 searches, no message |
OK (3 searches + message, no citation) | not probed | not probed | OK with citation | not probed |
| (i) Chat Completions | temperature:0.3 -> 400 Only the default (1) value is supported; reasoning_effort+verbosity OK |
- | - | max_tokens -> 400 Use 'max_completion_tokens' instead. |
OK streaming, usage in last chunk | OK streaming |
| (g) invalid key | 401 invalid_api_key (AuthenticationError) - see section 14 |
Other probes: gpt-5.6 alias OK; 26-model sweep OK except gpt-5.3-chat-latest (404); gpt-5-nano INC at default effort; o1 INC at 32 tokens; gpt-5.5-pro OK (7.3 s, "pong"); gpt-4.1-nano, gpt-4-turbo, gpt-3.5-turbo, o3-mini still answer despite announced shutdowns; gpt-5-search-api works only via chat.completions; PDF input OK on gpt-5.4-mini; previous_response_id OK (remembered "7"); service_tier:"flex" OK; input_audio -> 400 Audio input is not available.; prompt_cache_options.ttl:"30m" OK on gpt-5.6-sol.
21. Adapter recommendations (summary)
- Responses API only; Chat Completions merely as an optional legacy toggle (needed for
gpt-audio-*,*-search-*). - Registry-driven parameter gating: sampling params allowed iff model is non-reasoning or
effort === "none"; never sendseed/stop;max_output_tokens >= 16and default it high (>= 4k) for reasoning models; effort dropdown filtered per model (reasoningEffortsin the JSON). - Stream handler keyed on
event.type; treatresponse.incompleteas a normal terminal event (show partial text + "reasoning exhausted budget" hint whenreason === "max_output_tokens"and no text). - Default
store:false+include:["reasoning.encrypted_content"], replay fulloutputitems each turn; offerprevious_response_idmode only when the user opts into server-side storage. - Persist
usage(incl.cached_tokens,cache_write_tokens,reasoning_tokens) per message and price with the registry table; web-search turns cost 5-9k input tokens. - Model list =
GET /v1/modelsfiltered by regex +shutdown_date, then joined withopenai.models.json; show shutdown badges for dates < 90 days away. - Errors: map on
status+code+param;codecan be null for "Unsupported parameter" - fall back to message prefix; never retry 401/400/quota-429.