# LLM Rater — Anthropic API Reference (August 2026)

Practical, current reference for the task-rating pipeline in `apps/worker/src/raters/` (CLAUDE.md §6).
Scope: model choice, Message Batches API, structured outputs, prompt caching, rate limits, TypeScript SDK
patterns, and a concrete cost estimate for rating ~18,000 O*NET task statements × 3 samples.

All facts verified against the official docs on **2026-08-05**. Note: `https://docs.claude.com/en/api/overview`
(the URL in CLAUDE.md §6) now 301-redirects to `https://platform.claude.com/docs/en/api/overview` — the docs
moved to `platform.claude.com`.

Sources:

- Models overview — https://platform.claude.com/docs/en/about-claude/models/overview
- Pricing — https://platform.claude.com/docs/en/about-claude/pricing
- Batch processing — https://platform.claude.com/docs/en/build-with-claude/batch-processing
- Rate limits — https://platform.claude.com/docs/en/api/rate-limits
- Structured outputs — https://platform.claude.com/docs/en/build-with-claude/structured-outputs
- Prompt caching — https://platform.claude.com/docs/en/build-with-claude/prompt-caching

---

## 1. Current model lineup and pricing (per MTok, standard API)

| Model | ID (set via `RATER_MODEL`) | Context / max output | Input | Output | Batch input | Batch output | Cache read |
|---|---|---|---|---|---|---|---|
| Claude Fable 5 | `claude-fable-5` | 1M / 128K | $10 | $50 | $5 | $25 | $1.00 |
| Claude Opus 5 | `claude-opus-5` | 1M / 128K | $5 | $25 | $2.50 | $12.50 | $0.50 |
| Claude Opus 4.8 | `claude-opus-4-8` | 1M / 128K | $5 | $25 | $2.50 | $12.50 | $0.50 |
| Claude Sonnet 5 (intro, **through 2026-08-31**) | `claude-sonnet-5` | 1M / 128K | $2 | $10 | $1 | $5 | $0.20 |
| Claude Sonnet 5 (from 2026-09-01) | `claude-sonnet-5` | 1M / 128K | $3 | $15 | $1.50 | $7.50 | $0.30 |
| Claude Sonnet 4.6 | `claude-sonnet-4-6` | 1M / 128K | $3 | $15 | $1.50 | $7.50 | $0.30 |
| Claude Haiku 4.5 | `claude-haiku-4-5` | 200K / 64K | $1 | $5 | $0.50 | $2.50 | $0.10 |

Cache writes: 1.25× base input (5-min TTL) or 2× base input (1-hour TTL). Cache multipliers **stack with the
batch discount** (confirmed in the pricing doc: "These multipliers stack with other pricing modifiers,
including the Batch API discount"). All model IDs from the 4.6 generation onward are dateless pinned
snapshots — no date suffix to append.

Tokenizer note: models from Opus 4.7 onward (incl. Fable 5, Opus 5, Sonnet 5) use a tokenizer that yields
~30% more tokens for the same text than Sonnet 4.6/Haiku 4.5. Budget token estimates per model, not globally.

### Which tier for large-scale 5-point task rating?

For a structured, rubric-guided 5-point rating with a short rationale (a classification-plus-justification
task, not open-ended reasoning):

- **Haiku 4.5** — cheapest ($0.50/$2.50 batch), fast, supports structured outputs. Likely adequate for the
  bulk of clear-cut tasks, but weakest calibration on ambiguous tasks — expect more >1-point disagreements
  flowing into the expert-panel queue (which costs human time).
- **Claude Sonnet 5** — **recommended default.** Near-Opus quality on judgment tasks; at the introductory
  price ($1/$5 batch through Aug 31, 2026) the full 54k-rating job costs ~$160–250 (see §7) — the marginal
  cost over Haiku is trivial relative to the human-review pipeline it feeds. Caveat: adaptive thinking is on
  by default and thinking tokens bill as output; for this task set `thinking: {type: "disabled"}` (accepted
  on Sonnet 5) or keep it on with `output_config: {effort: "low"}` and budget extra output tokens.
- **Opus 5 / Fable 5** — overkill for the volume run. Total quality of the index is bounded by the
  methodology and human validation, not by Opus-vs-Sonnet deltas on a 5-point scale. Best use: (a) rate the
  expert-panel calibration subset with Opus 5 as a second opinion, or (b) adjudicate flagged disagreements.

Practical plan: pilot ~500 tasks on both Haiku 4.5 and Sonnet 5, compare agreement with the human 5% sample,
and pick per the disagreement rate. Sampling-variance note for the 3-samples design: **Sonnet 5, Opus 5, and
Opus 4.7+ reject non-default `temperature`/`top_p`/`top_k` (400 error)** — you cannot set temperature for
the 3 samples on those models; between-sample variance is whatever the model naturally produces. Haiku 4.5
and Sonnet 4.6 still accept temperature.

## 2. Message Batches API

Docs: https://platform.claude.com/docs/en/build-with-claude/batch-processing

- **Endpoints:** `POST /v1/messages/batches` (create), `GET /v1/messages/batches/{id}` (poll),
  `GET <results_url>` (stream `.jsonl` results), `POST /v1/messages/batches/{id}/cancel`, `GET /v1/messages/batches` (list).
- **Discount:** flat **50% off both input and output tokens**, all models, all features (vision, tools,
  structured outputs, prompt caching all supported inside batches).
- **Limits:** max **100,000 requests or 256 MB** per batch, whichever comes first. Each request needs a
  `custom_id` matching `^[a-zA-Z0-9_-]{1,64}$`. `max_tokens` must be ≥ 1 (`max_tokens: 0` cache pre-warming
  is rejected inside batches).
- **Timing:** most batches finish < 1 hour; hard **24-hour expiration** — requests not processed by then
  come back as `expired` (not billed) and must be resubmitted.
- **Results:** available at `results_url` once `processing_status === "ended"`; delivered as JSONL, **in
  arbitrary order — always key by `custom_id`, never by position**. Results are downloadable for **29 days**
  after batch creation; persist them to `data/derived/ratings/` + DB immediately.
- **Per-request result types:** `succeeded` (has `.result.message`), `errored` (invalid request → fix and
  resubmit; server error → safe to retry; not billed), `canceled` (not billed), `expired` (not billed —
  resubmit).
- **Extended output:** Opus 5/4.8/4.7/4.6, Sonnet 5/4.6 support up to 300k output tokens in batches via the
  `output-300k-2026-03-24` beta header (not needed for 500-token ratings).
- **No server-side idempotency key on batch create** — dedupe is your job (below).

### Marrying batches with BullMQ

Our invariant (CLAUDE.md §6): BullMQ job ID = deterministic hash of `task_id + prompt_version`. Extend to
`hash(task_id + prompt_version + sample_index)` since each task is rated 3×. Recommended architecture:

1. **`rating-request` rows, not per-rating jobs.** Persist one DB row per (task, prompt_version, sample)
   with status `pending`. A hex SHA-256 (truncated to 32–48 chars) of `taskId:promptVersion:sampleIdx`
   satisfies both the BullMQ job-ID and the batch `custom_id` charset — **use the same string for both**, so
   a batch result maps 1:1 to a job/row.
2. **`batch-submitter` job** (BullMQ, repeatable or triggered): collects up to 100k `pending` rows, calls
   `batches.create()`, stores `batch_id` on the rows *before* flipping them to `submitted` — if the process
   dies after create but before persist, on restart list recent batches and reconcile by `custom_id` rather
   than re-creating (this is the idempotency seam; the API will happily accept duplicate custom_ids across
   batches and bill you twice).
3. **`batch-poller` job** with BullMQ job ID = `poll:${batchId}` (deterministic → re-enqueue is a no-op),
   repeat/delay ~60s until `processing_status === "ended"`.
4. **`batch-ingester`** streams results, and per result: store raw JSON response, parsed score, model,
   prompt version, timestamp (full audit trail); `errored`(server)/`expired` → flip row back to `pending`
   so the next submitter run resubmits; `errored`(invalid_request) → dead-letter for inspection.
5. A prompt-version bump changes every hash → new custom_ids → old cached rows are naturally invalidated,
   exactly matching the CLAUDE.md rule.

The whole 54,000-rating run fits in **one batch** (well under 100k requests and 256 MB), even at the Start
tier queue limit (200k requests in processing queue).

## 3. Structured outputs for the 5-point rating

**Structured outputs is GA** (no beta header; the old `structured-outputs-2025-11-13` header and top-level
`output_format` request param are deprecated transition shims). Two mechanisms:

1. **JSON outputs** — `output_config: {format: {type: "json_schema", schema: {...}}}` constrains the
   response text to schema-valid JSON.
2. **Strict tool use** — `strict: true` on a tool definition; guarantees `tool_use.input` validates.

**Recommendation: use JSON outputs (`output_config.format`), not tool-forced JSON.** Rationale:

- It's the purpose-built mechanism for "the response *is* the structured object" — no fake tool, no
  `tool_choice` forcing, one fewer moving part in the audit trail.
- Guaranteed-valid JSON with `required` fields → the "parsed score" column can be extracted without retry
  loops.
- Works with the **Batches API**, streaming, and thinking. (Incompatible with citations and prefilling —
  neither is used here.)
- Schema limits that matter to us: `enum` is supported (use `"score": {"enum": [1,2,3,4,5]}` — do **not**
  use `minimum`/`maximum`, numeric range constraints are unsupported); no `minLength`/`maxLength` on the
  rationale string (enforce length in the prompt); `additionalProperties: false` is mandatory on every
  object.
- First use of a schema pays a one-time grammar-compilation latency; compiled grammars are cached 24h —
  irrelevant inside a batch run that reuses one schema 54,000×. Note that **changing `output_config.format`
  invalidates the prompt cache**, so treat the schema like the rubric: versioned with `RATER_PROMPT_VERSION`.

Suggested schema:

```json
{
  "type": "object",
  "properties": {
    "score": { "type": "integer", "enum": [1, 2, 3, 4, 5] },
    "rationale": { "type": "string", "description": "2-3 sentence justification citing the rubric" },
    "confidence": { "type": "string", "enum": ["low", "medium", "high"] }
  },
  "required": ["score", "rationale", "confidence"],
  "additionalProperties": false
}
```

Structured outputs injects a system-prompt preamble explaining the format (small, fixed token overhead per
request).

## 4. Prompt caching for the shared rubric

Layout: `tools` → `system` → `messages` renders in that order and caching is a **byte-exact prefix match**.
Put the ~2k-token rubric in `system` with `cache_control` on its last block; the per-task variable content
(task statement, occupation context) goes in the user message, after the breakpoint:

```ts
system: [
  { type: "text", text: RUBRIC_V3,            // frozen per RATER_PROMPT_VERSION — no timestamps, no task data
    cache_control: { type: "ephemeral", ttl: "1h" } },
],
messages: [{ role: "user", content: `Occupation: ${occ}\nTask: ${taskStatement}\nRate this task.` }]
```

Key facts:

- **Pricing:** cache read = 0.1× base input; write = 1.25× (5-min TTL) or 2× (1-hour TTL). **Stacks with the
  batch discount** → a cached rubric token inside a batch costs 0.05× base input.
- **Inside batches use the 1-hour TTL** (official recommendation): batch requests process concurrently over
  up to an hour, so 5-min entries can lapse between hits. Caveat: cache hits inside a batch are
  best-effort — parallel workers may each miss; treat the §7 "with caching" numbers as the optimistic bound.
- **Minimum cacheable prefix is model-dependent and non-monotonic:** 512 tokens (Opus 5/Fable 5), 1,024
  (Opus 4.8, Sonnet 5, Sonnet 4.6), 2,048 (Opus 4.7), **4,096 (Haiku 4.5, Opus 4.6)**. **A 2k-token rubric
  silently will not cache on Haiku 4.5** — no error, just `cache_creation_input_tokens: 0`. If Haiku is
  chosen, either accept uncached input (still cheap) or grow the cached prefix ≥4,096 tokens (e.g. include
  the scoring examples/anchors in the system block).
- Cost math for the rubric alone (Sonnet 5 intro, batch, 54k requests, 2,000 tokens):
  uncached = 54,000 × 2,000 × $1/MTok = **$108**; cached (1 write + 54k reads at 0.05×) ≈ 108M × $0.10/MTok
  ≈ **$10.80**. ~10× saving on the shared-prefix portion.
- Verify via `usage.cache_read_input_tokens` in each batch result; zero across the run means a silent
  invalidator (non-deterministic serialization, per-request content above the breakpoint).

## 5. Rate limits relevant to batch rating throughput

Docs: https://platform.claude.com/docs/en/api/rate-limits — organizations sit on Start / Build / Scale /
Custom tiers (auto-assigned by usage history; monthly spend caps of $500 / $1,000 / $200,000).

**Message Batches API has its own limits, shared across all models** (separate from Messages ITPM/OTPM):

| Tier | API requests/min | Max batch requests in processing queue | Max requests per batch |
|---|---|---|---|
| Start | 1,000 | 200,000 | 100,000 |
| Build | 2,000 | 300,000 | 100,000 |
| Scale | 4,000 | 500,000 | 100,000 |

Implications for us: 54,000 ratings fit in a single batch at any tier; even a full-index recompute with
several prompt versions in flight stays under the Start-tier queue (200k). Batch throughput inside the
queue is demand-based, not tier-based — under load, more requests may hit the 24h expiry; the ingester's
resubmit path (§2) handles that.

For any **synchronous** rating path (e.g. on-demand re-rate of a single task): limits are per-model RPM +
ITPM/OTPM (e.g. Start tier, Sonnet 5: 1,000 RPM / 2M ITPM / 400k OTPM). **Cache reads do not count toward
ITPM** on current models, so the cached rubric also multiplies effective sync throughput. Opus 5 and
Sonnet 5 each have rate-limit buckets separate from the combined Opus 4.x / Sonnet 4.x pools. On 429, honor
`retry-after`; the SDK does this automatically (default 2 retries).

## 6. TypeScript SDK (`@anthropic-ai/sdk`) patterns

The SDK auto-retries 408/409/429/5xx with exponential backoff (`maxRetries` default 2; timeout default
10 min, in **milliseconds** on TS). Sketch of the worker pieces (model from `RATER_MODEL`, never hardcoded):

```ts
import Anthropic from "@anthropic-ai/sdk";
import { createHash } from "node:crypto";

const client = new Anthropic(); // ANTHROPIC_API_KEY from env
const MODEL = process.env.RATER_MODEL!;
const PROMPT_VERSION = process.env.RATER_PROMPT_VERSION!;

export const ratingId = (taskId: string, sample: number) =>
  createHash("sha256").update(`${taskId}:${PROMPT_VERSION}:${sample}`).digest("hex").slice(0, 48);
// valid as BullMQ job ID *and* batch custom_id (^[a-zA-Z0-9_-]{1,64}$)

// --- batch-submitter job ---
export async function submitBatch(rows: PendingRating[]) {
  const batch = await client.messages.batches.create({
    requests: rows.map((r) => ({
      custom_id: r.id, // = ratingId(...)
      params: {
        model: MODEL,
        max_tokens: 1024, // headroom over the ~500-token rating
        system: [{ type: "text" as const, text: RUBRIC,
                   cache_control: { type: "ephemeral" as const, ttl: "1h" as const } }],
        output_config: { format: { type: "json_schema", schema: RATING_SCHEMA } },
        messages: [{ role: "user" as const, content: r.taskPrompt }],
      },
    })),
  });
  await db.markSubmitted(rows.map((r) => r.id), batch.id); // persist batch_id BEFORE returning
  return batch.id;
}

// --- batch-poller job (BullMQ delayed/repeatable, jobId: `poll:${batchId}`) ---
export async function pollBatch(batchId: string): Promise<boolean> {
  const batch = await client.messages.batches.retrieve(batchId);
  return batch.processing_status === "ended"; // else re-schedule in ~60s
}

// --- batch-ingester job ---
export async function ingestResults(batchId: string) {
  for await (const result of await client.messages.batches.results(batchId)) {
    switch (result.result.type) {
      case "succeeded": {
        const msg = result.result.message;
        const text = msg.content.find((b) => b.type === "text")?.text ?? "";
        await db.storeRating({
          customId: result.custom_id,          // → task_id + prompt_version + sample
          model: msg.model,
          promptVersion: PROMPT_VERSION,
          rawResponse: JSON.stringify(msg),    // full audit trail (CLAUDE.md §6)
          parsed: JSON.parse(text),            // schema-guaranteed {score, rationale, confidence}
          usage: msg.usage,                    // incl. cache_read_input_tokens for cost telemetry
          ratedAt: new Date().toISOString(),
        });
        break;
      }
      case "errored":
        if (result.result.error.type === "invalid_request") await db.deadLetter(result.custom_id, result.result);
        else await db.markPending(result.custom_id); // server error — next submitter run retries
        break;
      case "expired":
      case "canceled":
        await db.markPending(result.custom_id);
        break;
    }
  }
}
```

Notes: results arrive in arbitrary order — key everything by `custom_id`. For sync one-off calls, prefer
`client.messages.parse()` with `zodOutputFormat(...)` from `@anthropic-ai/sdk/helpers/zod` (typed
`parsed_output`); for batches, validate the JSON against the same Zod schema at ingest time. Handle errors
with typed classes (`Anthropic.RateLimitError`, `Anthropic.APIError`), never string-matching.

## 7. Cost estimate: 18,000 tasks × 3 samples = 54,000 ratings

Assumptions: rubric 2,000 tokens (shared, cacheable), per-task suffix ~300 tokens (statement + occupation
context + instruction), completion ~500 tokens. Totals: **input 124.2M tokens** (108M rubric + 16.2M
variable), **output 27M tokens**. "With caching" = 1h-TTL cache, optimistic ~100% hit rate (real batch runs
will land between the last two columns), cache read = 0.1× input, stacked with the 50% batch discount
(0.05× net). Prices per §1.

| Model | Sync, no cache | Batch only (50%) | Batch + rubric caching |
|---|---:|---:|---:|
| Haiku 4.5 ($1/$5) | $259 | **$130** | ≈$130 (2k rubric **below Haiku's 4,096-token cache minimum** — won't cache; pad rubric ≥4k to reach ≈$92) |
| Sonnet 5 — intro thru 2026-08-31 ($2/$10) | $518 | $259 | **≈$162** ($10.80 cached rubric + $16.20 variable input + $135 output) |
| Sonnet 5 / Sonnet 4.6 — standard ($3/$15) | $778 | $389 | **≈$243** ($16.20 + $24.30 + $202.50) |
| Opus 5 ($5/$25) | $1,296 | $648 | **≈$405** ($27 + $40.50 + $337.50) |
| Fable 5 ($10/$50) | $2,592 | $1,296 | ≈$810 (not recommended for this workload) |

Takeaways:

- **Output tokens dominate** once caching is on (83% of the Sonnet cost). Keep rationales tight in the
  prompt, and control thinking (Sonnet 5/Opus 5 think by default; thinking bills as output — disable it or
  set `effort: "low"` or the 500-token completion assumption breaks).
- The entire volume run costs **$130–$405** depending on model — negligible against the human-expert
  pipeline. This argues for Sonnet 5 (or even an Opus 5 second-pass on flagged tasks) over Haiku
  penny-pinching; if the run happens before **2026-09-01**, Sonnet 5's intro pricing makes it ~$162.
- A full re-run per `INDEX_VERSION`/`RATER_PROMPT_VERSION` bump is affordable, which supports the
  methodology-integrity rule that recomputations create new immutable runs.

## Where the docs contradict / update CLAUDE.md assumptions

1. **Docs URL moved:** CLAUDE.md §6 points to `https://docs.claude.com/en/api/overview`; that 301-redirects
   to `https://platform.claude.com/docs/en/api/overview`. Update the reference.
2. **Temperature-based sampling is gone on current models:** if the "3 samples per task" design assumed
   `temperature > 0` resampling, note that Sonnet 5 / Opus 5 / Opus 4.7+ **reject** non-default
   `temperature`/`top_p`/`top_k` with a 400. Variance across samples is natural model stochasticity only
   (or use Haiku 4.5 / Sonnet 4.6, which still accept temperature).
3. **`output_format` param is deprecated** — any prototype code using it should move to
   `output_config: {format: ...}` (GA, no beta header).
4. No contradiction on the audit-trail/idempotency requirements — the Batch API's `custom_id` +
   result-streaming model fits the deterministic-hash design directly; the only gap is that batch **create**
   has no server-side idempotency key, so the submitter must persist `batch_id` transactionally (§2.2).