SPB Git

spb/wp41_uqo

Public
2.8 KB · 37 lines markdown
1# PHASE 4 — Scale: profiling, feasible Base architecture, main ingestion, frozen test protocol (2026-11-02)23## Profiling (`scripts/benchmark.py`, M3U96b, Apple M3 Ultra, torch 2.15.0 MPS, FP32, real pilot grids)4Data loader (batch 64, memory-mapped grids, spawned workers, after warm-up): 3 971 ex/s (0 workers), 13 016 (4), 38 486 (8)5→ the loader is never the bottleneck (training consumes < 800 ex/s).67| Preset | Parameters (measured) | Batch | Steps/s | Examples/s | Peak MPS memory | Inference ms/origin (batched) |8|---|---|---|---|---|---|---|9| Small (d192, 4L) | 3 455 619 | 32 / 64 / 128 | 20.8 / 12.2 / 5.9 | 666 / 782 / 758 | 54 / 69 / 97 MB | 0.34 / 0.25 / 0.27 |10| Spec (d256, 6L, the brief) | 7 677 955 | 32 / 64 / 128 | 11.5 / 6.3 / 3.0 | 369 / 402 / 380 | 114 / 119 / 147 MB | 0.59 / 0.48 / 0.53 |11| Base (d384, 8L) | 20 718 339 | 32 / 64 / 128 | 6.1 / 3.2 / 1.5 | 195 / 207 / 193 | 275 / 284 / 315 MB | 5.0 / 1.6 / 1.8 |12Batch-1 latency 4.3 / 5.1 / 5.9 ms. Losses finite in FP32 at every setting. Mixed precision was not enabled (no stability13study yet). Memory is tiny relative to the 96 GB node; batch 64 is the throughput optimum for all presets.1415## Decision: Base = 20.7 M parameters (inside the 15–30 M target); Small = 3.5 M (inside 3–5 M)16Budget at batch 64 for the frozen main protocol (12 000 steps = 768 000 examples ≈ 1.3 passes over the 600 k training17origins drawn at 2 per instrument-session): Small ≈ 16 min, Spec ≈ 32 min, Base ≈ 62 min of pure training plus18validation passes (60 batches every 1 000 steps) and prediction (≈ 1.45 M evaluation origins at stride 15 → ≈ 6 min19per Small model, ≈ 40 min for Base at 1.6 ms/origin). The 50–100 M "Large" preset is not trained in v1.020(no scaling evidence yet; documented limitation).2122## Main ingestion (completed before this phase, `docs/PHASE_1_REPORT.md`)23300 instruments × 2018–2025: 233 958 776 rows (220 142 835 regular hours), 3.5 GB Parquet, 414 s of exports, 19 s of24grids; disk usage ≈ 8 GB of 423 GB free.2526## Frozen test protocol (`configs/main.yaml`, `configs/final.yaml`)27- Train ≤ 2022-12-31, validation 2023, development test 2024 (model selection and ablations), final 2025 untouched28  (run once with the main-study checkpoints, `--models-from results/metrics/main`).29- Evaluation origins every 15 minutes from 10:00 (24 per normal session); 200 Monte Carlo draws per origin.30- Held-out instruments (60) evaluated zero-shot with the pooled checkpoints.31- Ablations (`configs/ablations.yaml`): Small scale, 4 000 steps for every variant **and** for its reference32  (`mrpt_small_b4k`), seed 1 (+ seeds 2, 3 for the reference).33- Statistics: day-cluster bootstrap, HAC Diebold–Mariano with lag = overlap × instruments, Holm/BH.3435## Go / no-go36GO to P5 (`main_chain` launched on M3U96b 2026-11-02 18:5x ET: benchmark → `configs/main.yaml`).37
Files · main
Menu

Theme