TeX 53.6%
Python 46.2%
1# COMPUTE_LOG.md23## 2026-07-13 — Stage A pilot benchmarks (local: Apple Silicon, macOS, Python 3.14, polars 1.40)45| Operation | Input | Wall time | Notes |6|---|---|---|---|7| Download 1992-06 (24 MB) | HF anonymous | 3.3 s | ~7 MB/s |8| Download 2009-06 (270 MB) | HF anonymous | 24.4 s | ~11 MB/s |9| Download 2025-12 (401 MB) | HF anonymous | 19.0 s | ~21 MB/s |10| Full audit of 3 months (58.8M rows) | polars eager | ~40 s total | fits comfortably in RAM per month |1112## Projections for full run1314- Full download 87.7 GB: ~1.5–2.5 h serial anonymous; ~10× faster sharded across MacLustr15 nodes with `--shard k/n` (downloader supports node sharding natively).16- Storage: raw 88 GB; clean RTH panel est. 60–80 GB; features est. 100–150 GB;17 total ≲ 350 GB. Local free: ~1.0 TB (checked 2026-07-13).18- Largest single month 464 MB compressed (~2–4 GB in RAM) — per-month processing is19 single-node safe on every cluster node with ≥16 GB RAM.20