spb/llm-api-toolkits

1# State, caching and cost23## Conversation state45| Method | How | Trade-off |6|---|---|---|7| `previous_response_id` | Send only the new input plus the id of the last response | Simplest. Responses are stored (`store` defaults to true). `instructions` are **not** carried over: resend them |8| Your own history | Append `response.output` items to your input array each turn | You control retention and can edit history |9| Stateless with reasoning | `store: false` + `include: ["reasoning.encrypted_content"]`, pass reasoning items back | Nothing retained by the API; you keep the encrypted reasoning |1011Background mode (`background: true`) runs a long response asynchronously; poll12`responses.retrieve(id)`. It needs stored responses (`store: false` is rejected with it).1314## Prompt caching1516Automatic for long prompts: a repeated prefix is billed at the cached-input rate (often a tenth of17input; see `references/models.md`). Keep stable content (instructions, tool definitions, reference18text) first and volatile content last. `prompt_cache_key` groups requests that share a prefix;19`prompt_cache_retention: "24h"` keeps it longer. Verify with20`usage.input_tokens_details.cached_tokens`.2122## Cost levers, cheapest first23241. Stable prefixes so caching applies. 2. Shorter prompts and tool outputs. 3. Batch or flex25processing (about half price) for non-urgent work. 4. Lower reasoning effort. 5. A cheaper model.26Reasoning tokens are billed as output. Above 272K input tokens long-context rates apply on the27newest models. Measure cost per completed task, not per request.2829```bash30node scripts/openai-api.mjs cost gpt-6.1-sol 10000 2000 --requests 500 --batch31node scripts/openai-api.mjs compare 10000 2000 --requests 50032```33