# LLM Rater — Anthropic API Reference (August 2026)

Practical, current reference for the task-rating pipeline in apps/worker/src/raters/ (CLAUDE.md §6). Scope: model choice, Message Batches API, structured outputs, prompt caching, rate limits, TypeScript SDK patterns, and a concrete cost estimate for rating ~18,000 O*NET task statements × 3 samples.

All facts verified against the official docs on 2026-08-05. Note: https://docs.claude.com/en/api/overview (the URL in CLAUDE.md §6) now 301-redirects to https://platform.claude.com/docs/en/api/overview — the docs moved to platform.claude.com.

Sources:

• Models overview — https://platform.claude.com/docs/en/about-claude/models/overview
• Pricing — https://platform.claude.com/docs/en/about-claude/pricing
• Batch processing — https://platform.claude.com/docs/en/build-with-claude/batch-processing
• Rate limits — https://platform.claude.com/docs/en/api/rate-limits
• Structured outputs — https://platform.claude.com/docs/en/build-with-claude/structured-outputs
• Prompt caching — https://platform.claude.com/docs/en/build-with-claude/prompt-caching

# 1. Current model lineup and pricing (per MTok, standard API)

Cache writes: 1.25× base input (5-min TTL) or 2× base input (1-hour TTL). Cache multipliers stack with the batch discount (confirmed in the pricing doc: "These multipliers stack with other pricing modifiers, including the Batch API discount"). All model IDs from the 4.6 generation onward are dateless pinned snapshots — no date suffix to append.

Tokenizer note: models from Opus 4.7 onward (incl. Fable 5, Opus 5, Sonnet 5) use a tokenizer that yields ~30% more tokens for the same text than Sonnet 4.6/Haiku 4.5. Budget token estimates per model, not globally.

# Which tier for large-scale 5-point task rating?

For a structured, rubric-guided 5-point rating with a short rationale (a classification-plus-justification task, not open-ended reasoning):

• Haiku 4.5 — cheapest ($0.50/$2.50 batch), fast, supports structured outputs. Likely adequate for the bulk of clear-cut tasks, but weakest calibration on ambiguous tasks — expect more >1-point disagreements flowing into the expert-panel queue (which costs human time).
• Claude Sonnet 5 — recommended default. Near-Opus quality on judgment tasks; at the introductory price ($1/$5 batch through Aug 31, 2026) the full 54k-rating job costs ~$160–250 (see §7) — the marginal cost over Haiku is trivial relative to the human-review pipeline it feeds. Caveat: adaptive thinking is on by default and thinking tokens bill as output; for this task set thinking: {type: "disabled"} (accepted on Sonnet 5) or keep it on with output_config: {effort: "low"} and budget extra output tokens.
• Opus 5 / Fable 5 — overkill for the volume run. Total quality of the index is bounded by the methodology and human validation, not by Opus-vs-Sonnet deltas on a 5-point scale. Best use: (a) rate the expert-panel calibration subset with Opus 5 as a second opinion, or (b) adjudicate flagged disagreements.

Practical plan: pilot ~500 tasks on both Haiku 4.5 and Sonnet 5, compare agreement with the human 5% sample, and pick per the disagreement rate. Sampling-variance note for the 3-samples design: Sonnet 5, Opus 5, and Opus 4.7+ reject non-default temperature/top_p/top_k (400 error) — you cannot set temperature for the 3 samples on those models; between-sample variance is whatever the model naturally produces. Haiku 4.5 and Sonnet 4.6 still accept temperature.

# 2. Message Batches API

Docs: https://platform.claude.com/docs/en/build-with-claude/batch-processing

• Endpoints: POST /v1/messages/batches (create), GET /v1/messages/batches/{id} (poll), GET <results_url> (stream .jsonl results), POST /v1/messages/batches/{id}/cancel, GET /v1/messages/batches (list).
• Discount: flat 50% off both input and output tokens, all models, all features (vision, tools, structured outputs, prompt caching all supported inside batches).
• Limits: max 100,000 requests or 256 MB per batch, whichever comes first. Each request needs a custom_id matching ^[a-zA-Z0-9_-]{1,64}$. max_tokens must be ≥ 1 (max_tokens: 0 cache pre-warming is rejected inside batches).
• Timing: most batches finish < 1 hour; hard 24-hour expiration — requests not processed by then come back as expired (not billed) and must be resubmitted.
• Results: available at results_url once processing_status === "ended"; delivered as JSONL, in arbitrary order — always key by custom_id, never by position. Results are downloadable for 29 days after batch creation; persist them to data/derived/ratings/ + DB immediately.
• Per-request result types: succeeded (has .result.message), errored (invalid request → fix and resubmit; server error → safe to retry; not billed), canceled (not billed), expired (not billed — resubmit).
• Extended output: Opus 5/4.8/4.7/4.6, Sonnet 5/4.6 support up to 300k output tokens in batches via the output-300k-2026-03-24 beta header (not needed for 500-token ratings).
• No server-side idempotency key on batch create — dedupe is your job (below).

# Marrying batches with BullMQ

Our invariant (CLAUDE.md §6): BullMQ job ID = deterministic hash of task_id + prompt_version. Extend to hash(task_id + prompt_version + sample_index) since each task is rated 3×. Recommended architecture:

1. rating-request rows, not per-rating jobs. Persist one DB row per (task, prompt_version, sample) with status pending. A hex SHA-256 (truncated to 32–48 chars) of taskId:promptVersion:sampleIdx satisfies both the BullMQ job-ID and the batch custom_id charset — use the same string for both, so a batch result maps 1:1 to a job/row.
2. batch-submitter job (BullMQ, repeatable or triggered): collects up to 100k pending rows, calls batches.create(), stores batch_id on the rows before flipping them to submitted — if the process dies after create but before persist, on restart list recent batches and reconcile by custom_id rather than re-creating (this is the idempotency seam; the API will happily accept duplicate custom_ids across batches and bill you twice).
3. batch-poller job with BullMQ job ID = poll:${batchId} (deterministic → re-enqueue is a no-op), repeat/delay ~60s until processing_status === "ended".
4. batch-ingester streams results, and per result: store raw JSON response, parsed score, model, prompt version, timestamp (full audit trail); errored(server)/expired → flip row back to pending so the next submitter run resubmits; errored(invalid_request) → dead-letter for inspection.
5. A prompt-version bump changes every hash → new custom_ids → old cached rows are naturally invalidated, exactly matching the CLAUDE.md rule.

The whole 54,000-rating run fits in one batch (well under 100k requests and 256 MB), even at the Start tier queue limit (200k requests in processing queue).

# 3. Structured outputs for the 5-point rating

Structured outputs is GA (no beta header; the old structured-outputs-2025-11-13 header and top-level output_format request param are deprecated transition shims). Two mechanisms:

1. JSON outputs — output_config: {format: {type: "json_schema", schema: {...}}} constrains the response text to schema-valid JSON.
2. Strict tool use — strict: true on a tool definition; guarantees tool_use.input validates.

Recommendation: use JSON outputs (output_config.format), not tool-forced JSON. Rationale:

• It's the purpose-built mechanism for "the response is the structured object" — no fake tool, no tool_choice forcing, one fewer moving part in the audit trail.
• Guaranteed-valid JSON with required fields → the "parsed score" column can be extracted without retry loops.
• Works with the Batches API, streaming, and thinking. (Incompatible with citations and prefilling — neither is used here.)
• Schema limits that matter to us: enum is supported (use "score": {"enum": [1,2,3,4,5]} — do not use minimum/maximum, numeric range constraints are unsupported); no minLength/maxLength on the rationale string (enforce length in the prompt); additionalProperties: false is mandatory on every object.
• First use of a schema pays a one-time grammar-compilation latency; compiled grammars are cached 24h — irrelevant inside a batch run that reuses one schema 54,000×. Note that changing output_config.format invalidates the prompt cache, so treat the schema like the rubric: versioned with RATER_PROMPT_VERSION.

Suggested schema:

{
  "type": "object",
  "properties": {
    "score": { "type": "integer", "enum": [1, 2, 3, 4, 5] },
    "rationale": { "type": "string", "description": "2-3 sentence justification citing the rubric" },
    "confidence": { "type": "string", "enum": ["low", "medium", "high"] }
  },
  "required": ["score", "rationale", "confidence"],
  "additionalProperties": false
}

Structured outputs injects a system-prompt preamble explaining the format (small, fixed token overhead per request).

# 4. Prompt caching for the shared rubric

Layout: tools → system → messages renders in that order and caching is a byte-exact prefix match. Put the ~2k-token rubric in system with cache_control on its last block; the per-task variable content (task statement, occupation context) goes in the user message, after the breakpoint:

system: [
  { type: "text", text: RUBRIC_V3,            // frozen per RATER_PROMPT_VERSION — no timestamps, no task data
    cache_control: { type: "ephemeral", ttl: "1h" } },
],
messages: [{ role: "user", content: `Occupation: ${occ}\nTask: ${taskStatement}\nRate this task.` }]

Key facts:

• Pricing: cache read = 0.1× base input; write = 1.25× (5-min TTL) or 2× (1-hour TTL). Stacks with the batch discount → a cached rubric token inside a batch costs 0.05× base input.
• Inside batches use the 1-hour TTL (official recommendation): batch requests process concurrently over up to an hour, so 5-min entries can lapse between hits. Caveat: cache hits inside a batch are best-effort — parallel workers may each miss; treat the §7 "with caching" numbers as the optimistic bound.
• Minimum cacheable prefix is model-dependent and non-monotonic: 512 tokens (Opus 5/Fable 5), 1,024 (Opus 4.8, Sonnet 5, Sonnet 4.6), 2,048 (Opus 4.7), 4,096 (Haiku 4.5, Opus 4.6). A 2k-token rubric silently will not cache on Haiku 4.5 — no error, just cache_creation_input_tokens: 0. If Haiku is chosen, either accept uncached input (still cheap) or grow the cached prefix ≥4,096 tokens (e.g. include the scoring examples/anchors in the system block).
• Cost math for the rubric alone (Sonnet 5 intro, batch, 54k requests, 2,000 tokens): uncached = 54,000 × 2,000 × $1/MTok = $108; cached (1 write + 54k reads at 0.05×) ≈ 108M × $0.10/MTok ≈ $10.80. ~10× saving on the shared-prefix portion.
• Verify via usage.cache_read_input_tokens in each batch result; zero across the run means a silent invalidator (non-deterministic serialization, per-request content above the breakpoint).

# 5. Rate limits relevant to batch rating throughput

Docs: https://platform.claude.com/docs/en/api/rate-limits — organizations sit on Start / Build / Scale / Custom tiers (auto-assigned by usage history; monthly spend caps of $500 / $1,000 / $200,000).

Message Batches API has its own limits, shared across all models (separate from Messages ITPM/OTPM):

Implications for us: 54,000 ratings fit in a single batch at any tier; even a full-index recompute with several prompt versions in flight stays under the Start-tier queue (200k). Batch throughput inside the queue is demand-based, not tier-based — under load, more requests may hit the 24h expiry; the ingester's resubmit path (§2) handles that.

For any synchronous rating path (e.g. on-demand re-rate of a single task): limits are per-model RPM + ITPM/OTPM (e.g. Start tier, Sonnet 5: 1,000 RPM / 2M ITPM / 400k OTPM). Cache reads do not count toward ITPM on current models, so the cached rubric also multiplies effective sync throughput. Opus 5 and Sonnet 5 each have rate-limit buckets separate from the combined Opus 4.x / Sonnet 4.x pools. On 429, honor retry-after; the SDK does this automatically (default 2 retries).

# 6. TypeScript SDK (@anthropic-ai/sdk) patterns

The SDK auto-retries 408/409/429/5xx with exponential backoff (maxRetries default 2; timeout default 10 min, in milliseconds on TS). Sketch of the worker pieces (model from RATER_MODEL, never hardcoded):

import Anthropic from "@anthropic-ai/sdk";
import { createHash } from "node:crypto";

const client = new Anthropic(); // ANTHROPIC_API_KEY from env
const MODEL = process.env.RATER_MODEL!;
const PROMPT_VERSION = process.env.RATER_PROMPT_VERSION!;

export const ratingId = (taskId: string, sample: number) =>
  createHash("sha256").update(`${taskId}:${PROMPT_VERSION}:${sample}`).digest("hex").slice(0, 48);
// valid as BullMQ job ID *and* batch custom_id (^[a-zA-Z0-9_-]{1,64}$)

// --- batch-submitter job ---
export async function submitBatch(rows: PendingRating[]) {
  const batch = await client.messages.batches.create({
    requests: rows.map((r) => ({
      custom_id: r.id, // = ratingId(...)
      params: {
        model: MODEL,
        max_tokens: 1024, // headroom over the ~500-token rating
        system: [{ type: "text" as const, text: RUBRIC,
                   cache_control: { type: "ephemeral" as const, ttl: "1h" as const } }],
        output_config: { format: { type: "json_schema", schema: RATING_SCHEMA } },
        messages: [{ role: "user" as const, content: r.taskPrompt }],
      },
    })),
  });
  await db.markSubmitted(rows.map((r) => r.id), batch.id); // persist batch_id BEFORE returning
  return batch.id;
}

// --- batch-poller job (BullMQ delayed/repeatable, jobId: `poll:${batchId}`) ---
export async function pollBatch(batchId: string): Promise<boolean> {
  const batch = await client.messages.batches.retrieve(batchId);
  return batch.processing_status === "ended"; // else re-schedule in ~60s
}

// --- batch-ingester job ---
export async function ingestResults(batchId: string) {
  for await (const result of await client.messages.batches.results(batchId)) {
    switch (result.result.type) {
      case "succeeded": {
        const msg = result.result.message;
        const text = msg.content.find((b) => b.type === "text")?.text ?? "";
        await db.storeRating({
          customId: result.custom_id,          // → task_id + prompt_version + sample
          model: msg.model,
          promptVersion: PROMPT_VERSION,
          rawResponse: JSON.stringify(msg),    // full audit trail (CLAUDE.md §6)
          parsed: JSON.parse(text),            // schema-guaranteed {score, rationale, confidence}
          usage: msg.usage,                    // incl. cache_read_input_tokens for cost telemetry
          ratedAt: new Date().toISOString(),
        });
        break;
      }
      case "errored":
        if (result.result.error.type === "invalid_request") await db.deadLetter(result.custom_id, result.result);
        else await db.markPending(result.custom_id); // server error — next submitter run retries
        break;
      case "expired":
      case "canceled":
        await db.markPending(result.custom_id);
        break;
    }
  }
}

Notes: results arrive in arbitrary order — key everything by custom_id. For sync one-off calls, prefer client.messages.parse() with zodOutputFormat(...) from @anthropic-ai/sdk/helpers/zod (typed parsed_output); for batches, validate the JSON against the same Zod schema at ingest time. Handle errors with typed classes (Anthropic.RateLimitError, Anthropic.APIError), never string-matching.

# 7. Cost estimate: 18,000 tasks × 3 samples = 54,000 ratings

Assumptions: rubric 2,000 tokens (shared, cacheable), per-task suffix ~300 tokens (statement + occupation context + instruction), completion ~500 tokens. Totals: input 124.2M tokens (108M rubric + 16.2M variable), output 27M tokens. "With caching" = 1h-TTL cache, optimistic ~100% hit rate (real batch runs will land between the last two columns), cache read = 0.1× input, stacked with the 50% batch discount (0.05× net). Prices per §1.

Takeaways:

• Output tokens dominate once caching is on (83% of the Sonnet cost). Keep rationales tight in the prompt, and control thinking (Sonnet 5/Opus 5 think by default; thinking bills as output — disable it or set effort: "low" or the 500-token completion assumption breaks).
• The entire volume run costs $130–$405 depending on model — negligible against the human-expert pipeline. This argues for Sonnet 5 (or even an Opus 5 second-pass on flagged tasks) over Haiku penny-pinching; if the run happens before 2026-09-01, Sonnet 5's intro pricing makes it ~$162.
• A full re-run per INDEX_VERSION/RATER_PROMPT_VERSION bump is affordable, which supports the methodology-integrity rule that recomputations create new immutable runs.

# Where the docs contradict / update CLAUDE.md assumptions

1. Docs URL moved: CLAUDE.md §6 points to https://docs.claude.com/en/api/overview; that 301-redirects to https://platform.claude.com/docs/en/api/overview. Update the reference.
2. Temperature-based sampling is gone on current models: if the "3 samples per task" design assumed temperature > 0 resampling, note that Sonnet 5 / Opus 5 / Opus 4.7+ reject non-default temperature/top_p/top_k with a 400. Variance across samples is natural model stochasticity only (or use Haiku 4.5 / Sonnet 4.6, which still accept temperature).
3. output_format param is deprecated — any prototype code using it should move to output_config: {format: ...} (GA, no beta header).
4. No contradiction on the audit-trail/idempotency requirements — the Batch API's custom_id + result-streaming model fits the deterministic-hash design directly; the only gap is that batch create has no server-side idempotency key, so the submitter must persist batch_id transactionally (§2.2).