Caching, context and cost
Prompt caching
The cache matches a prefix in the order tools → system → messages. Put stable content
first and mark the end of it:
"system": [{"type": "text", "text": "<long stable instructions>", "cache_control": {"type": "ephemeral"}}]or set "cache_control": {"type": "ephemeral"} at the top level for automatic placement. At most
4 breakpoints. Anything that changes before a breakpoint invalidates what follows it: a timestamp
in the system prompt, JSON serialised in a different key order, a tool list that varies.
Verify, do not assume: usage.cache_creation_input_tokens on the first call,
usage.cache_read_input_tokens on the next. Zero reads on repeated calls means the prefix is
changing or is below the model's minimum cacheable size (a ~9K-token prefix cached on
claude-haiku-5-5 in this skill's tests). Writes cost 1.25× input (5 min) or 2× (1 h); reads cost
a small fraction of input — see references/models.md.
Long conversations
| Tool | Beta flag | What it does | Not on |
|---|---|---|---|
| Compaction | compact-2026-01-12 |
Summarises older turns (context_management.edits: [{"type": "compact_20260112"}]) |
4.5 models |
| Context editing | context-management-2025-06-27 |
Drops old tool results (clear_tool_uses_20250919) or thinking (clear_thinking_20251015) |
— |
| Task budget | task-budgets-2026-03-13 |
output_config.task_budget: {"type": "tokens", "total": ≥20000} lets the model pace a run |
4.5 and 4.6 models |
With compaction, keep appending the full response.content: compaction blocks must be preserved.
Mid-conversation operator instructions: append {"role": "system", "content": "…"} to messages
instead of editing the top-level system, which would reset the cache. Placement is strict: the
entry goes directly after the user message it applies to — […, user, system] — never right
after an assistant turn ([…, assistant, system, user] is a 400). Rejected entirely by the 4.5,
4.6 and Opus 4.7 models.
Cost levers, cheapest first
- Prompt caching. 2. Shorter prompts and tool results. 3. Batches (50%) for non-urgent work.
- Lower
effort. 5. A cheaper model. Measure cost per completed task, not per request: a cheaper request that needs more turns is not cheaper. Thinking tokens are billed as output tokens.
node scripts/claude-api.mjs cost claude-sonnet-5-5 10000 2000 --requests 500 --batch
node scripts/claude-api.mjs compare 10000 2000 --requests 500