spb/wpai0002_uqo

1# Compute and cost budget23Version 0.1 · 2026-11-09. All money is in USD. Prices come from `configs/rewriting/pricing.json`, retrieved42026-11-09; they must be re-verified before every paid run. The engine refuses `--live` for a model that is5missing from the file.67## 1. API cost model89| Model | Input / MTok | Output / MTok | Cache read / MTok | Role |10|---|---|---|---|---|11| `claude-opus-5-5` | 4.00 | 20.00 | 0.20 | primary rewriter (`configs/rewriting/pilot.json`) |12| `claude-sonnet-5-5` | 2.00 | 10.00 | 0.20 | held-out model (RQ4) |13| `claude-haiku-5-5` | 0.10 | 0.50 | 0.01 | held-out model (RQ4) |1415The Message Batches discount is 0.5, applied to the whole request. The post-hoc cost computed per response16(`estimate_cost`) is:1718`disc × (in·p_in + cache_write·1.25·p_in + cache_read·p_cr + out·p_out) / 10⁶`1920**Pre-flight estimate** (`RewriteEngine.estimate`):2122- tokens ≈ chars / 4;23- input per request = 260 (prompt overhead) + |protected passage| / 4;24- output per request = 1.1 × |passage| / 4.2526The measured system and user overhead of `build_messages` is 987 characters, about 247 tokens.2728**Passage statistics.** From `data/parsed/pilot_pmc`, 23 658 eligible passages:2930| Statistic | Value |31|---|---|32| Mean length | 941.9 characters (median 835) |33| Mean words | 142.9 |34| Mean protected length | 928.6 characters (first 3 000 passages) |35| Input tokens per request | ≈ 492 |36| Output tokens per request | ≈ 259 |3738| Scenario | Requests | Opus 5.5 standard | Opus 5.5 batch | Sonnet 5.5 batch | Haiku 5.5 batch |39|---|---|---|---|---|---|40| per 1 000 requests | 1 000 | 7.15 | 3.57 | 1.79 | 0.09 |41| **Pilot (Gate B)**: 1 000 passages × 6 generations | 6 000 | 42.89 | **21.44** | 10.72 | 0.54 |42| Held-out prompts H1+H2 on 500 test passages | 1 000 | 7.15 | 3.57 | — | — |43| Held-out models, 500 test passages × 6 | 3 000 | — | — | 5.36 | 0.27 |44| Medium: 20 000 passages × 6 | 120 000 | 857.9 | 428.9 | 214.5 | 10.7 |45| Medium target: 200 000 passages × 6 | 1 200 000 | 8 579 | 4 289.5 | 2 144.7 | 107.2 |4647**Caveats.**48491. **Thinking tokens.** Thinking is adaptive on Claude 5.x. Any thinking tokens are billed as output but are50   **not** in the pre-flight estimate. Because of this, effort is pinned to `low`, and the first batch is51   capped with `--max-requests 200`. Real tokens per request must be measured from `rewrites.jsonl[].usage`52   before the remainder is submitted.532. **Live mode at pilot scale.** The pilot exceeds the 30 USD ceiling in live mode (≈ 42.9). It is feasible54   only via batches (≈ 21.4).553. **Batch spend enforcement.** In batch mode the ceiling is enforced only when results are *collected*56   (`collect_batches` → ledger). `submit_batches` does not compare the estimate with the ceiling. Until the57   code is fixed, the operator must check the `estimate:` line and pass `--max-requests` accordingly.5859## 2. Training compute6061The trainer logs FLOPs as 6·N·D, where N = parameters and D = `tokens_seen` = source + target non-pad tokens62(`trainer.flops_estimate`). For an encoder–decoder this counts every token through all N parameters. It is63therefore an **upper bound**, roughly 2× the forward-backward cost when the encoder and decoder have similar size.6465**Planning assumption.** About 495 tokens per pair (≈ 259 source + 236 target, from chars/4 at vocabulary6632 000). The real value is to be read from `train_summary.json`. N is the parameter count at initialisation for67M1, vocabulary 32 000 (RESEARCH_PLAN §5).6869| FLOPs per epoch (6ND) | Pilot, 6 000 pairs (≈ 3.0 M tok) | Medium, 120 000 pairs (≈ 59 M tok) | Medium target, 1.2 M pairs (≈ 594 M tok) | fp32 weights |70|---|---|---|---|---|71| tiny (33.5 M) | 6.0e14 | 1.2e16 | 1.2e17 | 0.13 GB |72| small (130.8 M) | 2.4e15 | 4.7e16 | 4.7e17 | 0.52 GB |73| base (334.8 M) | 6.0e15 | 1.2e17 | 1.2e18 | 1.34 GB |74| large (1 068.6 M) | 1.9e16 | 3.8e17 | 3.8e18 | 4.27 GB |7576**Measured throughput (only one data point).** Run `vs_smoke-m1-nano` (8.33 M parameters, MPS, fp32, Apple M577Max) processed 1 658 892 tokens in 33.8 accelerator-seconds. That is ≈ 49 100 tok/s, or ≈ 2.45 TFLOP/s78effective by 6ND (`artifacts/runs/vs_smoke-m1-nano/train_summary.json`).7980**Naive extrapolation, not a measurement.** At that effective rate, one Medium epoch (59 M tokens) would take81≈ 1.8 h (tiny), 7.9 h (small), 19.8 h (base) or 64 h (large). Utilisation changes with model size, so each82design point **must be benchmarked**:8384- **What to run:** 200 steps per design point, `--max-steps 200 --eval-every 100`.85- **Which nodes:** the laptop (M5 Max, 48 GB, torch 2.12.0 MPS) and M4M64a (M4 Max, 64 GB, torch 2.12.1).86- **What to record:** `accel_seconds`, `tokens_seen` and peak memory into `artifacts/runs/bench-<size>-<node>/`.8788M3U96a and M3U96b (M3 Ultra, 96 GB) have only MLX installed; torch may be installed there with the author's89permission. MLX is used only if a benchmark justifies it (mandate Phase 5). Energy is reported only if measured90(e.g. `powermetrics`). Otherwise it is stated as not measured.9192**Memory.** fp32 AdamW needs ≈ 16 bytes per parameter for weights, gradients and two moments. The measured93`trainer_state.pt` is 2× the weight file (66.7 MB vs 33.3 MB). Estimated needs, before activations:9495| Size | Memory before activations | Feasibility |96|---|---|---|97| base | ≈ 7.8 GB | fits all GPU nodes listed above |98| large | ≈ 25 GB | fits 48–96 GB nodes only with small batches; activation checkpointing is not implemented |99100## 3. Storage (measured per-document rates × stage)101102| Item | Measured | Per unit | Medium (10 000 docs) | Large (100 000 docs) |103|---|---|---|---|---|104| Raw PMC JATS | 107 MB / 1 200 docs | ≈ 89 KB/doc | ≈ 0.9 GB | ≈ 9 GB |105| Parsed corpus (`pilot_pmc`) | 76 MB / 1 180 docs | ≈ 64 KB/doc | ≈ 0.65 GB | ≈ 6.4 GB |106| Rewrites (`rewrites.jsonl`) | 1.11 MB / 720 rows (mock) | ≈ 1.5 KB/row | 0.18 GB (120 k rows) | 1.8 GB (1.2 M rows) |107| Pairs (`pairs.jsonl`) | 1.76 MB / 557 rows | ≈ 3.2 KB/row | 0.38 GB | 3.8 GB |108| Checkpoint (best + last + optimizer state) | 130 MB for nano | ≈ 4 × fp32 weights | base ≈ 8 GB/run | — |109110Raw data, pairs and checkpoints are never committed (`.gitignore`).111112## 4. Ceilings and approvals113114| Item | Ceiling | Approves |115|---|---|---|116| Pilot rewriting (Gate B) | **30 USD** (`budget_usd_ceiling`, hard stop) | author (approved in the mandate scope) |117| Held-out prompt and model runs (E6, E7) | ≤ 15 USD total | author, before submission |118| Medium rewriting (≥ 20 000 passages) | per-stage quote from §1; nothing submitted without written approval | author |119| Any training run > 24 h of accelerator time, or on a node other than the laptop or M4M64a | — | author |120| Installing torch on other nodes | allowed by the author (2026-11-09); record versions in the run manifest | — |121| Cloud compute, public uploads (weights, data) | 0 USD unless explicitly approved | author |122123Every spend is recorded in `data/pairs/<job>/ledger.json` (`spent_usd`, per-status counts, batch ids) and summarised in the gate report.124125## Cloud GPU (author-approved 2026-11-10, D22)126127| Instance | Type | Price | Launched (UTC) | Jobs |128|---|---|---|---|---|129| 1 (`68.209.75.3`) | 1×H100 SXM 80 GB | 4.29 USD/h | 2026-11-10 ≈ 15:30 | HTR-Base denoise 30 k → FT 30 k → predictions (effective training from 18:00 after I3) |130| 2 (`68.209.74.137`) | 2×H100 SXM | 8.38 USD/h | 2026-11-10 18:08 | GPU 0 HTR-Large (40 k + 30 k, 8 192/6 144 tokens); GPU 1 HTR-Base seed 2 |131| 3 (`68.209.73.247`) | 2×H100 SXM | 8.38 USD/h | 2026-11-10 18:31 | GPU 0 flan-t5-large backbone; GPU 1 RoBERTa-large discriminator → MRT (Tiny, then Base) |132133Running cost ≈ 21 USD/h for the three instances; the author asked to add instances to finish sooner. Terminate each instance134when its chain is done and its artifacts are fetched (`scripts/lambda_sync.sh` in reverse: `artifacts/runs/*/best`,135`artifacts/predictions`, logs). Spend is read from the Lambda console at termination and recorded in `docs/STATUS.md`.136137**Closed 2026-11-11 ≈ 01:30 UTC.** All three instances terminated after their artifacts were fetched (`scripts/lambda_fetch.sh`):138instance 1 ≈ 15:30 → 00:40 (≈ 9.2 h × 4.29 ≈ 39 USD), instance 2 18:08 → 01:30 (≈ 7.4 h × 8.38 ≈ 62 USD), instance 3 18:31 → 01:20139(≈ 6.8 h × 8.38 ≈ 57 USD): **≈ 158 USD of GPU time** (to be confirmed against the Lambda invoice). Pangram runs of 2026-11-10/11:1407.52 + 3.64 + 3.80 + 3.28 = 18.24 USD on top of the 38.93 USD already recorded.141