State, caching and cost
Conversation state
| Method | How | Trade-off |
|---|---|---|
previous_response_id |
Send only the new input plus the id of the last response | Simplest. Responses are stored (store defaults to true). instructions are not carried over: resend them |
| Your own history | Append response.output items to your input array each turn |
You control retention and can edit history |
| Stateless with reasoning | store: false + include: ["reasoning.encrypted_content"], pass reasoning items back |
Nothing retained by the API; you keep the encrypted reasoning |
Background mode (background: true) runs a long response asynchronously; poll
responses.retrieve(id). It needs stored responses (store: false is rejected with it).
Prompt caching
Automatic for long prompts: a repeated prefix is billed at the cached-input rate (often a tenth of
input; see references/models.md). Keep stable content (instructions, tool definitions, reference
text) first and volatile content last. prompt_cache_key groups requests that share a prefix;
prompt_cache_retention: "24h" keeps it longer. Verify with
usage.input_tokens_details.cached_tokens.
Cost levers, cheapest first
- Stable prefixes so caching applies. 2. Shorter prompts and tool outputs. 3. Batch or flex processing (about half price) for non-urgent work. 4. Lower reasoning effort. 5. A cheaper model. Reasoning tokens are billed as output. Above 272K input tokens long-context rates apply on the newest models. Measure cost per completed task, not per request.
bash
node scripts/openai-api.mjs cost gpt-6.1-sol 10000 2000 --requests 500 --batch
node scripts/openai-api.mjs compare 10000 2000 --requests 500