spb/modelmap Public License

1---2project: modelmap3document: expH_capture_cost_frontier — hypothesis (run #1, registered before run)4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6website: https://modelmap.io7created: 2026-08-128modified: 2026-08-129status: reviewed10---1112# Hypothesis — expH run #1 (storage formats + hook overhead)1314> Capture cost frontier — what can you map on which Mac (macOS-specific).15> Run #1 scope: (A) activation-store formats on APFS, (B) capture/hook16> overhead in MLX vs PyTorch-MPS on a synthetic transformer. Registered17> 2026-08-12 **before** the first benchmark execution.1819```text20Hypothesis              : (A) For the SAE-shuffle access pattern (random row21                          batches), raw np.memmap sustains ≥ 2× the read22                          throughput of chunked zarr on APFS (warm cache),23                          because dense fp16 activations gain little from24                          compression while chunk decode adds latency.25                          (B) Activation capture (retain per-layer outputs)26                          costs < 2× plain inference on both MLX and27                          PyTorch-MPS at the 12-layer/d1024 scale, and the28                          device→CPU copy + disk write — not the hook/retain29                          mechanism — dominates capture cost.3031Falsification criterion : (A) dies if zarr (any codec) matches or beats mmap32                          on random-batch reads (ratio < 1.25×).33                          (B) dies if retain-only capture overhead exceeds34                          2× plain inference on either backend; the "capture35                          is cheap on consumer Macs" premise dies at > 3×36                          including copy+write.3738Method                  : Storage: 100k rows × 4096 dims fp16 (~0.8 GB) per39                          format (raw mmap, safetensors-mmap, zarr zstd, zarr40                          uncompressed); sequential write; sequential scan;41                          random-batch reads (4096 rows/batch, SAE-shuffle42                          pattern); 3 repeats/cell. Compute: synthetic43                          pre-norm transformer (12 layers, d_model 1024,44                          4 heads, seq 512, batch 8, fp16) implemented45                          identically in torch-MPS and MLX; modes = plain /46                          retain-on-device / retain+CPU-copy+mmap-write;47                          20 timed forwards (10 for write mode), 3 repeats,48                          explicit device synchronization.4950Baseline / null         : plain forward (no capture) per backend; sequential51                          scan as the reference for random-read degradation.5253Result                  : (pending)5455Interpretation          : (pending — with explicit confidence level)5657Next experiment         : (pending)58```5960**Known limitations, declared in advance.** Warm-cache regime (the 0.8 GB61store fits in the unified buffer cache): read numbers upper-bound the cold62case; a purge/F_NOCACHE cold pass is a registered follow-up (run #2), and the63sister project's cold-cache NVMe ceiling (localvm expH: ~13.1 GB/s at64≥256 KiB, QD≥4) bounds what run #2 can find. Synthetic model ≠ real65checkpoint: run #3 will repeat mode timings on a real 0.5B model via mlx-lm.6667---6869# Hypothesis — expH run #2 (cold-cache storage, second hardware)7071Registered 2026-08-12 **before** the run. Host: MacLustr node M3U96a72(Mac Studio, 32 cores, 96 GB), where `sudo purge` empties the unified buffer73cache between repetitions — also our first cross-hardware replication point.7475```text76Hypothesis              : The warm-cache format ordering survives cold:77                          raw mmap sustains ≥ 1.5× zarr-uncompressed on78                          random-batch reads with a purged cache; absolute79                          throughputs drop for all formats; zarr-zstd's80                          relative gap narrows (decode overlaps disk IO)81                          but does not close.82Falsification criterion : Dies if any zarr variant matches or beats mmap83                          cold (ratio < 1.25×) — that would rehabilitate84                          chunked stores and overturn run #1's design call.85Method                  : Same grid as run #1 Part A, store enlarged to86                          200k×4096 fp16 (~1.6 GB), `sudo purge` before87                          every timed read repetition, 3 repeats/cell,88                          writes measured warm (unchanged claim).89Baseline / null         : run #1 warm numbers on M5 Max; sequential scan90                          as the random-read reference.91Result                  : (pending)92Interpretation          : (pending)93Next experiment         : (pending)94```9596---9798# Hypothesis — expH run #3 (real quantized checkpoint via mlx-lm)99100Registered 2026-08-12 **before** the run. Local M5 Max, model =101mlx-community Qwen3-0.6B (4-bit), capture by wrapping decoder layers.102103```text104Hypothesis              : On a real 0.6B 4-bit model under MLX, per-layer105                          residual retention costs ≤ 1.15× plain prefill106                          (lazy-graph retention, as in run #1's synthetic107                          result), and retention + NumPy conversion + mmap108                          write stays ≤ 1.5×.109Falsification criterion : Dies at retain > 1.5× or retain+write > 2× —110                          would mean synthetic overheads do not transfer to111                          real checkpoints and all run #1 Part B numbers112                          must be re-derived on real models.113Method                  : Prefill of a fixed 1024-token prompt, batch 1,114                          10 timed forwards after 3 warmups, 3 repeats;115                          modes = plain / retain-all-layers / retain+write;116                          mx.eval barriers around each forward.117Baseline / null         : plain prefill throughput (tok/s).118Result                  : (pending)119Interpretation          : (pending)120Next experiment         : (pending)121```122