Compute and cost budget
Version 0.1 · 2026-11-09. All money is in USD. Prices come from configs/rewriting/pricing.json, retrieved
2026-11-09; they must be re-verified before every paid run. The engine refuses --live for a model that is
missing from the file.
1. API cost model
| Model | Input / MTok | Output / MTok | Cache read / MTok | Role |
|---|---|---|---|---|
claude-opus-5-5 |
4.00 | 20.00 | 0.20 | primary rewriter (configs/rewriting/pilot.json) |
claude-sonnet-5-5 |
2.00 | 10.00 | 0.20 | held-out model (RQ4) |
claude-haiku-5-5 |
0.10 | 0.50 | 0.01 | held-out model (RQ4) |
The Message Batches discount is 0.5, applied to the whole request. The post-hoc cost computed per response
(estimate_cost) is:
disc × (in·p_in + cache_write·1.25·p_in + cache_read·p_cr + out·p_out) / 10⁶
Pre-flight estimate (RewriteEngine.estimate):
- tokens ≈ chars / 4;
- input per request = 260 (prompt overhead) + |protected passage| / 4;
- output per request = 1.1 × |passage| / 4.
The measured system and user overhead of build_messages is 987 characters, about 247 tokens.
Passage statistics. From data/parsed/pilot_pmc, 23 658 eligible passages:
| Statistic | Value |
|---|---|
| Mean length | 941.9 characters (median 835) |
| Mean words | 142.9 |
| Mean protected length | 928.6 characters (first 3 000 passages) |
| Input tokens per request | ≈ 492 |
| Output tokens per request | ≈ 259 |
| Scenario | Requests | Opus 5.5 standard | Opus 5.5 batch | Sonnet 5.5 batch | Haiku 5.5 batch |
|---|---|---|---|---|---|
| per 1 000 requests | 1 000 | 7.15 | 3.57 | 1.79 | 0.09 |
| Pilot (Gate B): 1 000 passages × 6 generations | 6 000 | 42.89 | 21.44 | 10.72 | 0.54 |
| Held-out prompts H1+H2 on 500 test passages | 1 000 | 7.15 | 3.57 | — | — |
| Held-out models, 500 test passages × 6 | 3 000 | — | — | 5.36 | 0.27 |
| Medium: 20 000 passages × 6 | 120 000 | 857.9 | 428.9 | 214.5 | 10.7 |
| Medium target: 200 000 passages × 6 | 1 200 000 | 8 579 | 4 289.5 | 2 144.7 | 107.2 |
Caveats.
- Thinking tokens. Thinking is adaptive on Claude 5.x. Any thinking tokens are billed as output but are
not in the pre-flight estimate. Because of this, effort is pinned to
low, and the first batch is capped with--max-requests 200. Real tokens per request must be measured fromrewrites.jsonl[].usagebefore the remainder is submitted. - Live mode at pilot scale. The pilot exceeds the 30 USD ceiling in live mode (≈ 42.9). It is feasible only via batches (≈ 21.4).
- Batch spend enforcement. In batch mode the ceiling is enforced only when results are collected
(
collect_batches→ ledger).submit_batchesdoes not compare the estimate with the ceiling. Until the code is fixed, the operator must check theestimate:line and pass--max-requestsaccordingly.
2. Training compute
The trainer logs FLOPs as 6·N·D, where N = parameters and D = tokens_seen = source + target non-pad tokens
(trainer.flops_estimate). For an encoder–decoder this counts every token through all N parameters. It is
therefore an upper bound, roughly 2× the forward-backward cost when the encoder and decoder have similar size.
Planning assumption. About 495 tokens per pair (≈ 259 source + 236 target, from chars/4 at vocabulary
32 000). The real value is to be read from train_summary.json. N is the parameter count at initialisation for
M1, vocabulary 32 000 (RESEARCH_PLAN §5).
| FLOPs per epoch (6ND) | Pilot, 6 000 pairs (≈ 3.0 M tok) | Medium, 120 000 pairs (≈ 59 M tok) | Medium target, 1.2 M pairs (≈ 594 M tok) | fp32 weights |
|---|---|---|---|---|
| tiny (33.5 M) | 6.0e14 | 1.2e16 | 1.2e17 | 0.13 GB |
| small (130.8 M) | 2.4e15 | 4.7e16 | 4.7e17 | 0.52 GB |
| base (334.8 M) | 6.0e15 | 1.2e17 | 1.2e18 | 1.34 GB |
| large (1 068.6 M) | 1.9e16 | 3.8e17 | 3.8e18 | 4.27 GB |
Measured throughput (only one data point). Run vs_smoke-m1-nano (8.33 M parameters, MPS, fp32, Apple M5
Max) processed 1 658 892 tokens in 33.8 accelerator-seconds. That is ≈ 49 100 tok/s, or ≈ 2.45 TFLOP/s
effective by 6ND (artifacts/runs/vs_smoke-m1-nano/train_summary.json).
Naive extrapolation, not a measurement. At that effective rate, one Medium epoch (59 M tokens) would take ≈ 1.8 h (tiny), 7.9 h (small), 19.8 h (base) or 64 h (large). Utilisation changes with model size, so each design point must be benchmarked:
- What to run: 200 steps per design point,
--max-steps 200 --eval-every 100. - Which nodes: the laptop (M5 Max, 48 GB, torch 2.12.0 MPS) and M4M64a (M4 Max, 64 GB, torch 2.12.1).
- What to record:
accel_seconds,tokens_seenand peak memory intoartifacts/runs/bench-<size>-<node>/.
M3U96a and M3U96b (M3 Ultra, 96 GB) have only MLX installed; torch may be installed there with the author's
permission. MLX is used only if a benchmark justifies it (mandate Phase 5). Energy is reported only if measured
(e.g. powermetrics). Otherwise it is stated as not measured.
Memory. fp32 AdamW needs ≈ 16 bytes per parameter for weights, gradients and two moments. The measured
trainer_state.pt is 2× the weight file (66.7 MB vs 33.3 MB). Estimated needs, before activations:
| Size | Memory before activations | Feasibility |
|---|---|---|
| base | ≈ 7.8 GB | fits all GPU nodes listed above |
| large | ≈ 25 GB | fits 48–96 GB nodes only with small batches; activation checkpointing is not implemented |
3. Storage (measured per-document rates × stage)
| Item | Measured | Per unit | Medium (10 000 docs) | Large (100 000 docs) |
|---|---|---|---|---|
| Raw PMC JATS | 107 MB / 1 200 docs | ≈ 89 KB/doc | ≈ 0.9 GB | ≈ 9 GB |
Parsed corpus (pilot_pmc) |
76 MB / 1 180 docs | ≈ 64 KB/doc | ≈ 0.65 GB | ≈ 6.4 GB |
Rewrites (rewrites.jsonl) |
1.11 MB / 720 rows (mock) | ≈ 1.5 KB/row | 0.18 GB (120 k rows) | 1.8 GB (1.2 M rows) |
Pairs (pairs.jsonl) |
1.76 MB / 557 rows | ≈ 3.2 KB/row | 0.38 GB | 3.8 GB |
| Checkpoint (best + last + optimizer state) | 130 MB for nano | ≈ 4 × fp32 weights | base ≈ 8 GB/run | — |
Raw data, pairs and checkpoints are never committed (.gitignore).
4. Ceilings and approvals
| Item | Ceiling | Approves |
|---|---|---|
| Pilot rewriting (Gate B) | 30 USD (budget_usd_ceiling, hard stop) |
author (approved in the mandate scope) |
| Held-out prompt and model runs (E6, E7) | ≤ 15 USD total | author, before submission |
| Medium rewriting (≥ 20 000 passages) | per-stage quote from §1; nothing submitted without written approval | author |
| Any training run > 24 h of accelerator time, or on a node other than the laptop or M4M64a | — | author |
| Installing torch on other nodes | allowed by the author (2026-11-09); record versions in the run manifest | — |
| Cloud compute, public uploads (weights, data) | 0 USD unless explicitly approved | author |
Every spend is recorded in data/pairs/<job>/ledger.json (spent_usd, per-status counts, batch ids) and summarised in the gate report.
Cloud GPU (author-approved 2026-11-10, D22)
| Instance | Type | Price | Launched (UTC) | Jobs |
|---|---|---|---|---|
1 (68.209.75.3) |
1×H100 SXM 80 GB | 4.29 USD/h | 2026-11-10 ≈ 15:30 | HTR-Base denoise 30 k → FT 30 k → predictions (effective training from 18:00 after I3) |
2 (68.209.74.137) |
2×H100 SXM | 8.38 USD/h | 2026-11-10 18:08 | GPU 0 HTR-Large (40 k + 30 k, 8 192/6 144 tokens); GPU 1 HTR-Base seed 2 |
3 (68.209.73.247) |
2×H100 SXM | 8.38 USD/h | 2026-11-10 18:31 | GPU 0 flan-t5-large backbone; GPU 1 RoBERTa-large discriminator → MRT (Tiny, then Base) |
Running cost ≈ 21 USD/h for the three instances; the author asked to add instances to finish sooner. Terminate each instance
when its chain is done and its artifacts are fetched (scripts/lambda_sync.sh in reverse: artifacts/runs/*/best,
artifacts/predictions, logs). Spend is read from the Lambda console at termination and recorded in docs/STATUS.md.
Closed 2026-11-11 ≈ 01:30 UTC. All three instances terminated after their artifacts were fetched (scripts/lambda_fetch.sh):
instance 1 ≈ 15:30 → 00:40 (≈ 9.2 h × 4.29 ≈ 39 USD), instance 2 18:08 → 01:30 (≈ 7.4 h × 8.38 ≈ 62 USD), instance 3 18:31 → 01:20
(≈ 6.8 h × 8.38 ≈ 57 USD): ≈ 158 USD of GPU time (to be confirmed against the Lambda invoice). Pangram runs of 2026-11-10/11:
7.52 + 3.64 + 3.80 + 3.28 = 18.24 USD on top of the 38.93 USD already recorded.