SPB Git forge

spb/polyllm

Public
15commits 1branches 0releases
2.2 MBsize
maindefault branch
14 days agolast push
TypeScript 97.4% SQL 1% JavaScript 0.9% CSS 0.6%
43.9 KB

# OpenAI provider audit for PolyLLM

Last documentation audit: 2026-09-08 SDK probed: openai@7.10.0 (npm), Node 25.9.0, tsx 4.23.13. Probe scripts and raw results: research/openai/ (results/*.json, no secrets). Model registry produced from this audit: docs/provider-research/openai.models.json (84 entries).

Docs moved: every platform.openai.com/docs/... URL now 301-redirects to https://developers.openai.com/api/docs/.... The API reference pages there render only partially through a headless fetch (the "create response" and "streaming events" pages surfaced only the cancel endpoint), so event names, enums and parameter lists below were cross-checked against the SDK type definitions (node_modules/openai/resources/responses/responses.d.ts, shared.d.ts) and against live probes. Where docs and probes disagree, the probe wins and the disagreement is called out.


# 1. Endpoint, auth, headers

Item Value
Base URL https://api.openai.com/v1/ (docs also mention data-residency endpoints in the SDK 7.6 release notes; not needed for BYOK)
Auth Authorization: Bearer <OPENAI_API_KEY>
Optional headers OpenAI-Organization: org_..., OpenAI-Project: proj_... (billing scope), X-Client-Request-Id (ASCII, <= 512 chars), OpenAI-Safety-Identifier (Realtime only; for HTTP use the safety_identifier body param)
Response headers seen x-request-id, openai-processing-ms, openai-version: 2020-10-01, openai-organization, openai-project, x-ratelimit-* (see section 13)
Header size limits total < 64 KiB, single custom header value < 60 KiB
OpenAI-Beta Not required for Responses, Chat Completions, tools or structured outputs (none sent by SDK 7.x for these).

Source: https://developers.openai.com/api/docs/api-reference/introduction

# 2. Which API: Responses vs Chat Completions

  • Use the Responses API (POST /v1/responses) as the primary adapter. Docs: "While Chat Completions remains supported, Responses is recommended for all new projects." Chat Completions is not labelled legacy/deprecated and has no shutdown date, but:
    • Pro models (gpt-5.5-pro, gpt-5.4-pro, gpt-5.2-pro, gpt-5-pro, o3-pro, o1-pro), gpt-5.3-codex and gpt-5.6-cyber are Responses-only.
    • Built-in tools (web_search, file_search, code_interpreter, MCP, image_generation, computer use) exist only in Responses (Chat Completions has only web_search_options on the *-search-* models).
    • Reasoning docs: "Reasoning models work better with the Responses API"; GPT-6 Astra page: "Requires Responses API for tool calling (Chat Completions available but limited)".
    • Reasoning summaries, encrypted reasoning, previous_response_id, conversation, background mode, reasoning.context are Responses-only.
  • Chat Completions still works (verified 2026-09-08 on gpt-4.1-mini, gpt-5.5, o4-mini): chunk shape chat.completion.chunk with choices[0].delta, usage only in the final chunk when stream_options.include_usage: true; usage fields prompt_tokens/completion_tokens/prompt_tokens_details.cached_tokens/completion_tokens_details.reasoning_tokens. max_tokens is rejected on new models (gpt-6-astra: 400 unsupported_parameter "'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.").
  • The Chat-Completions-only web search models (gpt-5-search-api, gpt-4o-search-preview, gpt-4o-mini-search-preview) return 400 model_not_found "The requested model 'gpt-5-search-api' is not supported with the Responses API." on Responses. Do not expose them; use the web_search tool on a normal model.
  • Assistants API shut down 2026-08-26 (replaced by Responses + Conversations). v1/prompts (reusable prompts), Evals, Agent Builder shut down 2026-11-30.

Sources: https://developers.openai.com/api/docs/api-reference/chat , https://developers.openai.com/api/docs/guides/migrate-to-responses , https://developers.openai.com/api/docs/deprecations

# Chat Completions -> Responses mapping (for the adapter)

Chat Completions Responses
messages input (string, or array of items: messages {role, content} / function_call / function_call_output / reasoning ...)
system message instructions (top-level) or a developer (or system) role message; all three verified to work
max_tokens / max_completion_tokens max_output_tokens (min 16; includes reasoning tokens)
response_format text.format ({type:"text"} / {type:"json_schema", name, schema, strict} / {type:"json_object"})
tools[].function.{name,parameters} flattened tools[] = {type:"function", name, description, parameters, strict}
tool_calls[] in message separate function_call output items (call_id, name, arguments)
role:"tool" message {type:"function_call_output", call_id, output} input item
reasoning_effort, verbosity reasoning.effort, text.verbosity
usage.prompt_tokens / completion_tokens usage.input_tokens / output_tokens (+ input_tokens_details.cached_tokens, cache_write_tokens, output_tokens_details.reasoning_tokens, total_tokens)
n, seed, stop, logprobs (bool) n removed; seed and stop do not exist (400 unknown_parameter); logprobs via top_logprobs + include:["message.output_text.logprobs"]
stream_options.include_usage not needed: response.completed carries the full response.usage

# 3. Request shape (Responses) - parameters that matter

From SDK ResponseCreateParams (7.10.0) and probes:

model, input, instructions, max_output_tokens (>= 16), reasoning {effort, summary, mode, context}, text {format, verbosity}, tools, tool_choice ("auto"|"none"|"required"|{type:"function",name}|{type:"allowed_tools",...}), parallel_tool_calls, temperature, top_p, top_logprobs, include[], store (default true), previous_response_id, conversation, stream, stream_options {include_obfuscation}, background, truncation ("auto"|"disabled", default disabled), metadata, service_tier (auto|default|flex|priority|fast), prompt_cache_key, prompt_cache_retention (in_memory|24h, pre-5.6), prompt_cache_options {mode, ttl:"30m"} (5.6+), safety_identifier, user, context_management (compaction), moderation, prompt.

include values: file_search_call.results, web_search_call.results, web_search_call.action.sources, message.input_image.image_url, computer_call_output.output.image_url, code_interpreter_call.outputs, reasoning.encrypted_content, message.output_text.logprobs.

Response object: id (resp_...), object:"response", status (completed|failed|in_progress|cancelled|queued|incomplete), incomplete_details.reason (max_output_tokens|max_messages|content_filter|steered), error, model (resolved snapshot, e.g. gpt-5.5-2026-04-23), output[] items, usage, plus echoes of reasoning, text, temperature, top_p, truncation, store, service_tier, previous_response_id, tools. The SDK adds a convenience output_text getter only on non-streaming create() results (the response object inside a response.completed event does not have it - accumulate deltas or read output[].content[].text).

# 4. Streaming protocol

SSE; every event is JSON with type and a monotonically increasing sequence_number (verified: strictly +1 per event). Text deltas also carry an obfuscation padding string (disable via stream_options.include_obfuscation:false).

Event names observed in probes (exact strings):

text
response.created > response.in_progress
  > response.output_item.added {item:{type:"reasoning"...}}            (reasoning models, effort >= low)
      > response.reasoning_summary_part.added
      > response.reasoning_summary_text.delta {delta}   (x N)          (only when reasoning.summary is set AND the model emits one)
      > response.reasoning_summary_text.done {text}
      > response.reasoning_summary_part.done
    > response.output_item.done
  > response.output_item.added {item:{type:"message", role:"assistant"}}
    > response.content_part.added {part:{type:"output_text", text:"", annotations:[]}}
      > response.output_text.delta {delta, item_id, output_index, content_index, logprobs, obfuscation}   (x N)
      > response.output_text.annotation.added {annotation:{type:"url_citation",...}, annotation_index}    (web search)
      > response.output_text.done {text}
    > response.content_part.done
  > response.output_item.done
> response.completed {response:{...usage}}   |  response.incomplete  |  response.failed  |  error

Function calls (verified on all six models):

text
response.output_item.added   {item:{type:"function_call", id:"fc_...", call_id:"call_...", name, arguments:"", status:"in_progress"}, output_index}
response.function_call_arguments.delta {delta, item_id, output_index}
response.function_call_arguments.done  {arguments:"{...}", item_id}
response.output_item.done    {item:{type:"function_call", ..., arguments:"{...}", status:"completed"}}
response.completed

Web search: response.web_search_call.in_progress > .searching > .completed (item type web_search_call).

Full list of event types in the SDK (for exhaustive switch statements): response.created, response.in_progress, response.queued, response.completed, response.incomplete, response.failed, error, response.output_item.added/done, response.content_part.added/done, response.output_text.delta/done, response.output_text.annotation.added, response.refusal.delta/done, response.reasoning_text.delta/done, response.reasoning_summary_part.added/done, response.reasoning_summary_text.delta/done, response.function_call_arguments.delta/done, response.custom_tool_call_input.delta/done, response.web_search_call.in_progress/searching/completed, response.file_search_call.in_progress/searching/completed, response.code_interpreter_call.in_progress/interpreting/completed, response.code_interpreter_call_code.delta/done, response.image_generation_call.in_progress/generating/partial_image/completed, response.mcp_call.in_progress/completed/failed, response.mcp_call_arguments.delta/done, response.mcp_list_tools.in_progress/completed/failed, response.shell_call_command.added/delta/done, response.shell_call_output_content.delta/done, response.audio.delta/done, response.audio.transcript.delta/done, response.steer.*.

Resumable streams: with background:true, stream:true, reconnect with GET /v1/responses/{id}?stream=true&starting_after={sequence_number}.

Sources: https://developers.openai.com/api/docs/guides/streaming-responses , https://developers.openai.com/api/docs/guides/background , SDK types.

# Exact SDK streaming code that worked (research/openai/03-stream.ts, 05-tools-stream.ts)

ts
import OpenAI from "openai";
const client = new OpenAI({ maxRetries: 1, timeout: 180_000 }); // key from OPENAI_API_KEY

const stream = await client.responses.create({
  model: "gpt-5.5",
  input: [{ role: "user", content: "Say hello in five words." }],
  reasoning: { effort: "low", summary: "auto" },
  max_output_tokens: 200,
  stream: true,
});
let text = "", final: OpenAI.Responses.Response | null = null;
for await (const ev of stream) {
  switch (ev.type) {
    case "response.output_text.delta": text += ev.delta; break;
    case "response.reasoning_summary_text.delta": /* show thinking */ break;
    case "response.function_call_arguments.delta": /* accumulate by ev.item_id */ break;
    case "response.output_item.done": if (ev.item.type === "function_call") { /* ev.item.call_id, name, arguments */ } break;
    case "response.completed": case "response.incomplete": final = ev.response; break; // final.usage, final.output
    case "error": throw new Error(ev.message);
  }
}

Tool round trip (stateless): replay [...previousInput, ...final.output, { type: "function_call_output", call_id, output: JSON.stringify(result) }]. Replaying the entire output array (including reasoning items) is what the docs require for reasoning models; verified on gpt-5.5, gpt-5.4-mini, gpt-5.6-sol, gpt-6-astra, gpt-4.1-mini. Stateful alternative verified on gpt-5.4-mini: previous_response_id: final.id, input: [function_call_output].

# 5. Tool calling format

  • Definition: { type: "function", name, description, parameters: <JSON Schema>, strict: true } (flattened; strict recommended; strict requires additionalProperties:false and all properties in required).
  • Output item: { id: "fc_...", type: "function_call", status: "completed", call_id: "call_...", name, arguments: "<json string>" } (verified shape). call_id is what you echo back; id is the item id.
  • Result item: { type: "function_call_output", call_id, output: string | [content parts for images/files] } (SDK 7.7 made the id optional).
  • tool_choice: "auto" (default) | "none" | "required" | {type:"function", name} | {type:"allowed_tools", mode:"auto"|"required", tools:[...]}.
  • parallel_tool_calls default true.
  • Other tool types accepted in tools[]: web_search (and legacy web_search_preview), file_search, code_interpreter, image_generation, mcp, computer / computer_use_preview, shell, local_shell, apply_patch, custom (free-form text tool), namespace (grouping, defer_loading), tool_search, programmatic_tool_calling.

Sources: https://developers.openai.com/api/docs/guides/function-calling , https://developers.openai.com/api/docs/guides/tools

# 6. Structured output / JSON schema

  • text: { format: { type: "json_schema", name, schema, strict: true } } - verified on gpt-5.5, gpt-5.4-mini, gpt-5.6-sol, gpt-6-astra, gpt-4.1-mini (parsed {name:"Marie Curie",age:66,city:"Paris"}); o4-mini accepted it but ran out of the 200-token budget on reasoning.
  • Schema subset: root must be object; every property in required; additionalProperties:false; optional fields via type:["string","null"]; supports enum, anyOf, $ref/$defs, recursion; no minLength/pattern/format etc. Numeric limits (depth/property count/enum size) exist but the page did not expose them - keep schemas modest.
  • Refusals arrive as a refusal content part (response.refusal.delta/done) instead of JSON.
  • {type:"json_object"} (JSON mode) is the fallback for gpt-4 / gpt-3.5-turbo which lack structured outputs.
  • SDK helpers: zodTextFormat(schema, "name") + client.responses.parse(...) -> response.output_parsed.

Source: https://developers.openai.com/api/docs/guides/structured-outputs

# 7. Reasoning controls

  • reasoning.effort values in the SDK enum: none | minimal | low | medium | high | xhigh | max. Per-model acceptance (verified, exact error strings in section 20):
    • gpt-6-astra: low, medium, high, xhigh, max (default medium; none/minimal -> 400)
    • gpt-5.6-sol/terra/luna: none, low, medium, high, xhigh, max (default medium)
    • gpt-5.5: none, low, medium, high, xhigh (default medium)
    • gpt-5.4 / 5.4-mini / 5.4-nano, gpt-5.2: none, low, medium, high, xhigh (default none - verified echo)
    • gpt-5.1: none, low, medium, high (default none)
    • gpt-5 / 5-mini / 5-nano: minimal, low, medium, high (default medium; no none)
    • o3, o4-mini, o3-mini, o1: low, medium, high (default medium)
    • pro models: 5.5-pro medium|high|xhigh (default high), 5.4-pro / 5.2-pro medium|high|xhigh, gpt-5-pro high only
    • gpt-4.1 / gpt-4o / chat-latest: reasoning.effort -> 400 unsupported_parameter
  • reasoning.summary: auto | concise | detailed (server echoes detailed for auto). Summary text arrives in reasoning items' summary[] and via response.reasoning_summary_* events; gpt-5.5 and gpt-6-astra returned no summary on a trivial prompt while gpt-5.6-sol and o4-mini did. Never assume a summary will exist. Non-reasoning models silently ignore summary.
  • reasoning.mode: standard | pro (GPT-5.6; pro replaces the *-pro models). Not probed (cost).
  • reasoning.context: current_turn | all_turns (all_turns is the default on 5.6 and 6; earlier models default current_turn). Reasoning persists only within a family.
  • Reasoning tokens are billed as output and count against max_output_tokens; usage.output_tokens_details.reasoning_tokens reports them. Docs recommend reserving >= 25k tokens; with tiny caps, o4-mini / gpt-5-nano return status:"incomplete", incomplete_details.reason:"max_output_tokens" with an empty message (verified). Adapter rule: never cap reasoning models below ~1-2k output tokens; treat incomplete + empty text as "reasoning exhausted the budget".
  • Encrypted reasoning for stateless (store:false) multi-turn: store:false, include:["reasoning.encrypted_content"] -> reasoning items carry encrypted_content (~1.2 kB for a trivial turn); replaying them in the next input works (verified on gpt-5.5). previous_response_id on a store:false response fails: 400 previous_response_not_found.
  • GPT-6 Astra: mid-conversation configuration_update items to change effort; async tool calling (async:true); no none effort.

Sources: https://developers.openai.com/api/docs/guides/reasoning , https://developers.openai.com/api/docs/guides/latest-model

# 8. Sampling parameter support matrix (Responses API, verified 2026-09-08)

Param gpt-6-astra gpt-5.6-sol gpt-5.5 gpt-5.4-mini (default effort none) gpt-4.1-mini o4-mini
temperature rejected rejected unless effort:"none" (then accepted) rejected unless effort:"none" accepted accepted rejected
top_p rejected same as temperature same accepted accepted rejected
frequency_penalty rejected rejected (effort medium) rejected accepted accepted rejected
presence_penalty rejected rejected rejected accepted accepted rejected
seed unknown_parameter on every model (does not exist in Responses)
stop unknown_parameter on every model ("Did you mean 'store'?")
logprobs (top_logprobs+include) rejected (reasoning) rejected rejected accepted (logprobs[] on content part) accepted rejected
reasoning.effort low..max none..max none..xhigh none..xhigh rejected low..high
reasoning.summary accepted accepted accepted accepted ignored (no error) accepted
text.verbosity: "low" accepted accepted accepted accepted rejected (Supported values are: 'medium') rejected
max_output_tokens min 16 on all (integer_below_min_value)
truncation:"auto" accepted on all
store:false + include:["reasoning.encrypted_content"] accepted on all (non-reasoning models just have no reasoning item)
prompt_cache_key accepted on all

Default top_p echoed by GPT-5.x/6 is 0.98, by GPT-4.1 1. temperature default 1. top_k does not exist. There is no thinking budget parameter (effort only). Chat Completions on gpt-5.5: temperature:0.3 -> 400 unsupported_value "'temperature' does not support 0.3 with this model. Only the default (1) value is supported."

Practical rule for the adapter: send sampling params only when (model is non-reasoning) or (reasoning-capable model with reasoning.effort === "none"); otherwise drop them client-side and show "not supported" in the UI. Never send seed/stop to Responses.

# 9. Roles and instructions

Input roles accepted: user, assistant, developer, system (both system and developer verified on gpt-5.5 and gpt-4.1-mini; docs describe developer as the canonical name and treat instructions as equivalent to a developer message). instructions is not inherited through previous_response_id; resend it each call. Content parts: input_text, input_image, input_file, (input_audio rejected, see 10); assistant output parts: output_text, refusal.

Source: https://developers.openai.com/api/docs/guides/text

# 10. Modalities

Modality Responses API support Notes
Text in/out yes
Image in yes on every current model except o3-mini, gpt-4, gpt-3.5-turbo {type:"input_image", image_url:"data:image/png;base64,..." | https URL | file_id, detail:"low"|"high"|"auto"|"original"}. PNG/JPEG/WEBP/non-animated GIF; <= 512 MB per request, <= 1,500 images, <= 30,000 patches after resize. Tokens ~ ceil(w/32)*ceil(h/32) x model multiplier (1.2-2.46). Verified with a 2x2 PNG on all six probe models (31-33 input tokens).
PDF / file in yes (vision models) {type:"input_file", filename, file_data:"data:application/pdf;base64,..." | file_id | file_url}; <= 50 MB per file and per request; both text and page images are sent (costly). Verified on gpt-5.4-mini (answered 42 from the PDF).
Image out via tool only tools:[{type:"image_generation"}] uses gpt-image-2; returns base64 in an image_generation_call item; events response.image_generation_call.partial_image. Not probed. Older gpt-image-1* shut down 2026-10-23 / 2026-12-01.
Audio in/out no in Responses probe: 400 invalid_request_error param=input "Audio input is not available.". Audio chat = Chat Completions with gpt-audio-1.5 (input_audio parts, modalities:["text","audio"]) or Realtime (gpt-realtime-2.1). TTS: gpt-4o-mini-tts; STT: gpt-transcribe, gpt-live-transcribe (whisper-1/gpt-4o-transcribe shut down 2027-02-26).
Video no Sora 2 / Videos API shut down 2026-09-24.

Sources: https://developers.openai.com/api/docs/guides/images-vision , https://developers.openai.com/api/docs/guides/pdf-files , https://developers.openai.com/api/docs/guides/audio

# 11. Context windows, max output, pricing (per 1M tokens, USD, standard tier)

Model Context Max output Input Cached Output Notes
gpt-6-astra 1,050,000 128,000 10 1 50 cache write 12.50; >272K input: 2x in / 1.5x out
gpt-5.6-sol (gpt-5.6) 1,050,000 128,000 4 0.40 20
gpt-5.6-terra 1,050,000 (922K input) 128,000 2 0.20 12
gpt-5.6-luna 1,050,000 128,000 0.20 0.02 1.20
gpt-5.5 1,050,000 128,000 5 0.50 30 >272K: 2x/1.5x
gpt-5.5-pro 1,050,000 128,000 30 - 180 Responses only
gpt-5.4 1,050,000 128,000 2.50 0.25 15
gpt-5.4-mini 400,000 128,000 0.75 0.075 4.50
gpt-5.4-nano 400,000 128,000 0.20 0.02 1.25
gpt-5.4-pro 1,050,000 128,000 30 - 180 Responses only
gpt-5.3-codex 400,000 128,000 1.75 0.175 14 Responses only, coding
gpt-5.2 400,000 128,000 1.75 0.175 14
gpt-5.2-pro 400,000 128,000 21 - 168 Responses only
gpt-5.1 400,000 128,000 1.25 0.125 10
gpt-5 400,000 128,000 1.25 0.125 10 snapshot shuts down 2026-12-11
gpt-5-mini 400,000 128,000 0.25 0.025 2 snapshot shuts down 2026-12-11
gpt-5-nano 400,000 128,000 0.05 0.005 0.40 snapshot shuts down 2026-12-11
gpt-5-pro 400,000 272,000 15 - 120 Responses only
chat-latest 400,000 (272K input) 128,000 5 0.50 30 ChatGPT Instant, non-reasoning
gpt-4.1 1,047,576 32,768 2 0.50 8
gpt-4.1-mini 1,047,576 32,768 0.40 0.10 1.60
gpt-4.1-nano 1,047,576 32,768 0.10 0.025 0.40 shuts down 2026-10-23
gpt-4o 128,000 16,384 2.50 1.25 10 alias -> 2024-08-06
gpt-4o-mini 128,000 16,384 0.15 0.075 0.60
o3 200,000 100,000 2 0.50 8 snapshot shuts down 2026-12-11
o3-pro 200,000 100,000 20 - 80 Responses only
o4-mini 200,000 100,000 1.10 0.275 4.40 shuts down 2026-10-23
o3-mini 200,000 100,000 1.10 0.55 4.40 text only; shuts down 2026-10-23
o1 200,000 100,000 15 7.50 60 shuts down 2026-10-23
o1-pro 200,000 100,000 150 - 600 shuts down 2026-10-23
gpt-4-turbo 128,000 4,096 10 - 30 shuts down 2026-10-23
gpt-4 8,192 8,192 30 - 60 shuts down 2026-10-23
gpt-3.5-turbo 16,385 4,096 0.50 - 1.50 shuts down 2026-10-23

Tiers: Batch and Flex = ~50% of standard; Priority higher; Fast = 2x (not for gpt-6-astra with EU residency). Regional data residency +10% for models released after 2026-03-05. Tools: web search $10 / 1k calls (+ tokens), file search $2.50 / 1k calls + $0.10/GB/day storage (1 GB free), code interpreter $0.03-$1.92 per 20-min session by memory tier, MCP = tokens only. Unknown for this key: gpt-5-search-api, gpt-4o-mini-search-preview, the dead codex snapshots (set to null in the JSON).

Sources: per-model pages under https://developers.openai.com/api/docs/models/ , https://developers.openai.com/api/docs/pricing

# 12. Prompt caching

Automatic prefix caching; minimum cacheable prefix 1,024 tokens (GPT-5.6+) / 2,048 (earlier). Reads at the "cached" price (0.1x on 5.6+, 0.5x on gpt-4o/o3-mini/o1); GPT-5.6+ also bill cache writes at 1.25x (reported in usage.input_tokens_details.cache_write_tokens, present in every usage object we saw). prompt_cache_key (routing hint, accepted on every model) ; prompt_cache_retention: "in_memory" | "24h" for pre-5.6 models (accepted on gpt-5.5 and even gpt-6-astra, but docs say it is deprecated for 6 -> use prompt_cache_options: { ttl: "30m" }, verified accepted on gpt-5.6-sol, echoed {mode:"implicit", ttl:"30m"}). cached_tokens is exact on 5.6+, rounded to 128 earlier. Cache lifetime 5-10 min (up to 1 h) or 24 h; not shared across orgs.

Source: https://developers.openai.com/api/docs/guides/prompt-caching

# 13. Rate limits, headers, retries, timeouts

  • Dimensions: RPM, TPM, RPD, TPD, IPM (+ audio minutes). Tiers Free..Tier 5 by cumulative spend; this key is Tier 5 (headers show 15,000 RPM / 40M TPM on gpt-5.5, 30,000 RPM / 150M TPM on gpt-4.1-mini).
  • Headers (verified on every 200): x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests (e.g. 4ms, 6m0s), x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, x-ratelimit-reset-tokens; docs also list x-ratelimit-limit-project-tokens / x-ratelimit-remaining-project-tokens and Retry-After (seconds) on 429/503. Also x-request-id and openai-processing-ms.
  • 429 codes: rate_limit_exceeded, slow_down (traffic ramped too fast), insufficient_quota / credit_balance_exhausted, organization_spend_limit_exceeded, project_spend_limit_exceeded, organization_usage_limit_exceeded. 503 server_is_overloaded. Billing/quota 429s must not be retried.
  • Retry guidance: honour Retry-After when present, else exponential backoff with jitter; ramp traffic <= +50% per 15 min above 1M TPM. SDK default: maxRetries: 2, retries 408/409/429/>=500 and any response with x-should-retry: true, honours retry-after-ms / retry-after, backoff 0.5 s x 2^n capped at 8 s with up to 25% jitter. For a BYOK chat UI use maxRetries: 1-2 for non-streaming, and do not retry once streaming has begun.
  • Timeouts: SDK default 10 minutes (OpenAI.DEFAULT_TIMEOUT = 600000). Recommend: 60 s connect/first-byte for chat, overall 5-10 min for high/xhigh/max effort, and background: true (+ polling / resumable stream) for pro models and anything expected to run > 1-2 min. Docs: streaming is "the single most effective approach" to perceived latency; service_tier: "priority" | "fast" buys lower latency at higher price (flex accepted on gpt-5.4-mini: slower, cheaper).

Sources: https://developers.openai.com/api/docs/guides/rate-limits , https://developers.openai.com/api/docs/guides/error-codes , https://developers.openai.com/api/docs/guides/latency-optimization , SDK client.js

# 14. Error schema

HTTP body: { "error": { "message", "type", "param", "code" } } (raw fetch with a bad key also returned a top-level "status": 401). SDK maps to BadRequestError (400), AuthenticationError (401), PermissionDeniedError (403), NotFoundError (404), RateLimitError (429), InternalServerError (>=500), APIConnectionError, APITimeoutError; each exposes .status, .code, .type, .param, .requestID, .headers, .error (raw body).

Verified examples:

Case Status type code param message
invalid key 401 invalid_request_error invalid_api_key null Incorrect API key provided: sk-inva*****key. You can find your API key at https://platform.openai.com/account/api-keys.
unknown model 404 invalid_request_error model_not_found null The model `gpt-does-not-exist` does not exist or you do not have access to it.
CC-only model on Responses 400 invalid_request_error model_not_found model The requested model 'gpt-5-search-api' is not supported with the Responses API.
wrong type 400 invalid_request_error invalid_type max_output_tokens Invalid type for 'max_output_tokens': expected an integer, but got a string instead.
below min 400 invalid_request_error integer_below_min_value max_output_tokens Invalid 'max_output_tokens': integer below minimum value. Expected a value >= 16, but got 5 instead.
unsupported param 400 invalid_request_error null temperature Unsupported parameter: 'temperature' is not supported with this model.
unsupported param 400 invalid_request_error unsupported_parameter reasoning.effort Unsupported parameter: 'reasoning.effort' is not supported with this model.
unsupported value 400 invalid_request_error unsupported_value reasoning.effort Unsupported value: 'minimal' is not supported with the 'gpt-5.5' model. Supported values are: 'none', 'low', 'medium', 'high', and 'xhigh'.
unknown param 400 invalid_request_error unknown_parameter stop Unknown parameter: 'stop'. Did you mean 'store'?
logprobs on reasoning 400 invalid_request_error unsupported_parameter - logprobs are not supported with reasoning models.
prev response gone 400 invalid_request_error previous_response_not_found previous_response_id Previous response with id 'resp_...' not found.
audio in Responses 400 invalid_request_error null input Audio input is not available.

Note code is sometimes null for "Unsupported parameter" errors; match on param + message prefix as well. Other documented codes: context_length_exceeded, invalid_service_tier, insufficient_quota, rate_limit_exceeded, slow_down, server_is_overloaded.

# 15. Usage reporting and token counting

  • Every response (and response.completed) has usage: { input_tokens, input_tokens_details: { cached_tokens, cache_write_tokens }, output_tokens, output_tokens_details: { reasoning_tokens }, total_tokens }. Web search inflates input_tokens heavily (4.7k-8.8k for a one-line question) because search results are injected as context and billed as input.
  • Chat Completions usage: prompt_tokens, completion_tokens, total_tokens, prompt_tokens_details {cached_tokens, audio_tokens}, completion_tokens_details {reasoning_tokens, audio_tokens, accepted_prediction_tokens, rejected_prediction_tokens}; streaming needs stream_options: {include_usage: true} (usage arrives in a final chunk with empty choices).
  • Pre-flight counting: POST /v1/responses/input_tokens ({model, input, instructions?, tools?, conversation?, previous_response_id?} -> {object:"response.input_tokens", input_tokens}); SDK: client.responses.inputTokens.count(...). Client-side: tiktoken / js-tiktoken with o200k_base for gpt-4o and newer (approximation only for images/PDF). Rule of thumb 1 token ~ 4 chars English.
  • SDK 7.8 added compute_units tracking on usage for some tiers (not observed in our responses).

Source: https://developers.openai.com/api/docs/api-reference/responses/input-tokens

# 16. Model listing

GET /v1/models (SDK client.models.list() auto-paginates) returns {id, object:"model", created, owned_by, shutdown_date}. shutdown_date is the only lifecycle signal (ISO date or null; documented as "The date when the model will shut down, or null if not announced"). No capability, context or pricing metadata is exposed, so a static registry is mandatory (openai.models.json). 131 ids were returned for this key on 2026-09-08; 55 carried a shutdown_date, several already in the past (see 18). Filtering heuristics for chat models: exclude ids matching babbage|davinci|instruct|transcribe|tts|whisper|audio|realtime|image|sora|embedding|moderation|search-preview|search-api|deep-research|codex and anything whose shutdown_date < today; keep the rest and look it up in the registry (unknown ids -> show as "unverified").

Source: https://developers.openai.com/api/docs/api-reference/models

# 17. Aliases

  • gpt-5.6 -> gpt-5.6-sol (documented; verified response.model === "gpt-5.6-sol"; not itself listed by /v1/models).
  • gpt-5.5 -> gpt-5.5-2026-04-23; gpt-5.4 -> -2026-03-05; gpt-5.4-mini/-nano -> -2026-03-17; gpt-5.2 -> -2025-12-11; gpt-5.1 -> -2025-11-13; gpt-5* -> -2025-08-07; gpt-4.1* -> -2025-04-14; gpt-4o -> -2024-08-06 (not the newest 11-20 snapshot); gpt-4o-mini -> -2024-07-18; o3 -> -2025-04-16; o4-mini -> -2025-04-16; o3-mini -> -2025-01-31; gpt-4-turbo -> -2024-04-09; gpt-3.5-turbo -> -0125. gpt-6-astra, gpt-5.6-*, gpt-5.3-codex, chat-latest have no dated snapshot (echoed as-is).
  • *-chat-latest aliases (gpt-5 / 5.1 / 5.2 / 5.3) are dead (404) although still listed; chat-latest is the live ChatGPT-Instant alias.

# 18. Lifecycle / deprecations relevant to a chat app

Shutdown Models Replacement
2026-07-23 (passed; 404 today) gpt-5-chat-latest, gpt-5.1-chat-latest, gpt-5-codex, gpt-5.1-codex(-max/-mini), gpt-5.2-codex, o3-deep-research, o4-mini-deep-research, *-search-preview-2025-03-11 snapshots gpt-6-astra / gpt-5.3-codex
2026-08-10 (passed; 404) gpt-5.2-chat-latest, gpt-5.3-chat-latest chat-latest / gpt-6-astra
2026-08-26 Assistants API Responses + Conversations
2026-09-24 sora-2, sora-2-pro, Videos API -
2026-09-28 gpt-3.5-turbo-1106, gpt-3.5-turbo-instruct, babbage-002, davinci-002 gpt-5.6-terra
2026-10-23 gpt-3.5-turbo(-0125), gpt-4(-0613), gpt-4-turbo, gpt-4.1-nano, gpt-4o-2024-05-13, gpt-image-1, o1, o1-pro, o3-mini, o4-mini gpt-5.6-sol / terra / luna (o1-pro -> 5.6-sol reasoning.mode: pro)
2026-11-30 v1/prompts, Evals, Agent Builder -
2026-12-01 gpt-image-1-mini, gpt-image-1.5, chatgpt-image-latest gpt-image-2
2026-12-11 gpt-5-2025-08-07, gpt-5-mini-2025-08-07, gpt-5-nano-2025-08-07, gpt-5-pro-2025-10-06, o3-2025-04-16, o3-pro-2025-06-10 (the bare aliases gpt-5, o3 currently have shutdown_date: null) gpt-5.6-sol / terra / luna
2027-01-20 / 2027-02-26 legacy audio/realtime/transcribe models gpt-audio-1.5, gpt-realtime-2.1, gpt-transcribe

Fine-tuning: closed to new orgs since 2026-05-07; no new jobs at all after 2027-01-06.

Source: https://developers.openai.com/api/docs/deprecations + shutdown_date from /v1/models

# 19. Built-in tools, state, safety

  • Conversation state: store defaults to true (30-day retention, visible in dashboard logs) - set store:false for privacy-minded BYOK usage and manage history client-side; then use include:["reasoning.encrypted_content"] and replay all output items. previous_response_id (verified) and conversation (Conversations API, conv_..., not subject to 30-day TTL) require store:true. Server-side compaction: context_management:[{type:"compaction", compact_threshold}].
  • Web search: {type:"web_search", search_context_size?:"low"|"medium"|"high", user_location?:{type:"approximate", country, city, region, timezone}, filters?:{allowed_domains[] | blocked_domains[]} (<=100), external_web_access?, search_content_types?:["text","image"], return_token_budget?} (legacy web_search_preview still accepted). Output: web_search_call items (action: {type:"search", queries[], query} | {type:"open_page", url} | {type:"find_in_page"}) then a message whose output_text part has annotations: [{type:"url_citation", start_index, end_index, url, title}] and inline markdown links with ?utm_source=openai. Events: response.web_search_call.{in_progress,searching,completed}, response.output_text.annotation.added. Verified on gpt-4.1-mini and gpt-5.5 (effort low); the gpt-5.5 default-effort attempt at 200 tokens ended with three searches and no message - budget again. Search context capped at 128k. Supported per docs on gpt-6, 5.6, 5.5, 5.4 family, 5.2, 5.1, 5(-mini/-nano), 4.1(-mini), 4o(-mini), o3, o4-mini, chat-latest; not on gpt-4.1-nano, o3-mini, o1, gpt-4, gpt-3.5.
  • Code execution: {type:"code_interpreter", container: {type:"auto", memory_limit?:"1g"|"4g"|"16g"|"64g", file_ids?} | "cntr_..."} -> code_interpreter_call {code, outputs, container_id}; generated files as container_file_citation annotations; 100 RPM/org; events response.code_interpreter_call*. Also shell (hosted shell), local_shell, apply_patch, skills.
  • File search: {type:"file_search", vector_store_ids[], max_num_results?, filters?, ranking_options?} -> file_search_call + file_citation {file_id, filename, index} annotations; include:["file_search_call.results"].
  • Computer use: {type:"computer"} (new) or computer_use_preview {display_width, display_height, environment}; computer_call -> reply with computer_call_output {call_id, output:{type:"computer_screenshot", image_url}}; docs recommend GPT-6 Astra with code execution instead. Not relevant for PolyLLM v1.
  • MCP: {type:"mcp", server_label, server_url | connector_id, server_description?, authorization?, headers?, allowed_tools?, require_approval:"never"|"always"|{...}, defer_loading?} -> mcp_list_tools, mcp_call, mcp_approval_request (answer with mcp_approval_response {approval_request_id, approve}); events response.mcp_call*, response.mcp_list_tools*. Billing = tokens only. 8 first-party connectors (Gmail, Drive, Calendar, Dropbox, Teams, Outlook x2, SharePoint).
  • Citations/annotations: url_citation, file_citation, container_file_citation, file_path types on output_text.annotations[]; streamed through response.output_text.annotation.added.
  • Safety: send safety_identifier (hashed user id) on every request; incomplete_details.reason:"content_filter" and refusal parts must be surfaced; free Moderation API (omni-moderation-latest) available; revoke leaked keys immediately. moderation request param exists in the SDK types.

Sources: https://developers.openai.com/api/docs/guides/conversation-state , https://developers.openai.com/api/docs/guides/tools-web-search , https://developers.openai.com/api/docs/guides/tools-code-interpreter , https://developers.openai.com/api/docs/guides/tools-file-search , https://developers.openai.com/api/docs/guides/tools-computer-use , https://developers.openai.com/api/docs/guides/tools-remote-mcp , https://developers.openai.com/api/docs/guides/safety-best-practices

# 20. Probe results (2026-09-08, key = Tier 5 project key)

Legend: OK = completed with expected output; INC = accepted but status:"incomplete" (reasoning consumed the <= 200-token cap); 4xx = rejected with the quoted message.

Probe gpt-5.5 gpt-5.4-mini gpt-5.6-sol gpt-6-astra gpt-4.1-mini o4-mini
(a) basic Responses, 200 tok OK (gpt-5.5-2026-04-23, 12 reasoning tok) OK (0 reasoning) OK (0 reasoning) OK (0 reasoning) OK OK (64 reasoning)
(b) streaming events OK, reasoning item but no summary events OK OK, reasoning_summary_* events seen OK, no summary events OK INC: summary events, then response.incomplete
(c) temperature / top_p 400 Unsupported parameter: 'temperature' is not supported with this model. (OK with effort:"none") OK (echo 0.5) 400 same (OK with effort:"none") 400 same; effort:"none" itself rejected OK 400 same
(c) frequency/presence_penalty 400 Unsupported parameter: 'frequency_penalty' is not supported with this model. OK 400 400 OK 400
(c) seed 400 unknown_parameter Unknown parameter: 'seed'. (all models)
(c) stop 400 unknown_parameter Unknown parameter: 'stop'. Did you mean 'store'? (all models)
(c) logprobs 400 logprobs are not supported with reasoning models. OK 400 400 OK 400
(c) effort none OK OK OK 400 Unsupported value: 'none' is not supported with the 'gpt-6-astra' model. Supported values are: 'low', 'medium', 'high', 'xhigh', and 'max'. 400 Unsupported parameter: 'reasoning.effort' is not supported with this model. 400 Supported values are: 'low', 'medium', and 'high'.
(c) effort minimal 400 ... Supported values are: 'none', 'low', 'medium', 'high', and 'xhigh'. 400 same 400 ... 'none', 'low', 'medium', 'high', 'xhigh', and 'max'. 400 400 400
(c) effort low/medium/high OK OK OK OK 400 OK
(c) effort xhigh OK OK OK OK 400 400
(c) effort max 400 400 OK OK 400 400
(c) reasoning.summary auto OK (echo detailed) OK OK OK OK (ignored) OK
(c) text.verbosity low OK OK OK OK 400 Unsupported value: 'low' is not supported with the 'gpt-4.1-mini' model. Supported values are: 'medium'. 400 same wording
(c) max_output_tokens 5 400 integer_below_min_value Expected a value >= 16, but got 5 instead. (all)
(c) truncation auto, prompt_cache_key, store:false+encrypted OK on all six
(d) function call, streaming OK: function_call item, args {"city":"Paris","unit":"c"}, round trip -> "18°C and cloudy" OK (+ previous_response_id variant OK) OK OK OK INC (192 reasoning tokens, no call)
(e) json_schema strict OK parsed OK OK OK OK INC
(f) vision 2x2 PNG INC at default effort; OK with effort:"none" OK INC default; OK with effort:"none" OK (effort:"low", detail high) OK INC (even at effort:"low")
(h) web_search tool OK with effort:"low": web_search_call + url_citation annotation; default effort at 200 tok: 3 searches, no message OK (3 searches + message, no citation) not probed not probed OK with citation not probed
(i) Chat Completions temperature:0.3 -> 400 Only the default (1) value is supported; reasoning_effort+verbosity OK - - max_tokens -> 400 Use 'max_completion_tokens' instead. OK streaming, usage in last chunk OK streaming
(g) invalid key 401 invalid_api_key (AuthenticationError) - see section 14

Other probes: gpt-5.6 alias OK; 26-model sweep OK except gpt-5.3-chat-latest (404); gpt-5-nano INC at default effort; o1 INC at 32 tokens; gpt-5.5-pro OK (7.3 s, "pong"); gpt-4.1-nano, gpt-4-turbo, gpt-3.5-turbo, o3-mini still answer despite announced shutdowns; gpt-5-search-api works only via chat.completions; PDF input OK on gpt-5.4-mini; previous_response_id OK (remembered "7"); service_tier:"flex" OK; input_audio -> 400 Audio input is not available.; prompt_cache_options.ttl:"30m" OK on gpt-5.6-sol.

# 21. Adapter recommendations (summary)

  1. Responses API only; Chat Completions merely as an optional legacy toggle (needed for gpt-audio-*, *-search-*).
  2. Registry-driven parameter gating: sampling params allowed iff model is non-reasoning or effort === "none"; never send seed/stop; max_output_tokens >= 16 and default it high (>= 4k) for reasoning models; effort dropdown filtered per model (reasoningEfforts in the JSON).
  3. Stream handler keyed on event.type; treat response.incomplete as a normal terminal event (show partial text + "reasoning exhausted budget" hint when reason === "max_output_tokens" and no text).
  4. Default store:false + include:["reasoning.encrypted_content"], replay full output items each turn; offer previous_response_id mode only when the user opts into server-side storage.
  5. Persist usage (incl. cached_tokens, cache_write_tokens, reasoning_tokens) per message and price with the registry table; web-search turns cost 5-9k input tokens.
  6. Model list = GET /v1/models filtered by regex + shutdown_date, then joined with openai.models.json; show shutdown badges for dates < 90 days away.
  7. Errors: map on status + code + param; code can be null for "Unsupported parameter" - fall back to message prefix; never retry 401/400/quota-429.