spb/localvm-research Public License

1---2project: localvm-research3document: expH_ssd_feasibility/analysis4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6created: 2026-08-127status: reviewed8---910# Analysis — expH_ssd_feasibility1112Run: `results/expH_ssd_feasibility/20260812T034359Z/` · code committed at `b4a652d`13Hardware: Apple M5 Max (6P+12E, 40 GPU cores), 48 GB unified memory, APPLE SSD AP2048Z14(2 TB, internal APFS), macOS 27.0 (26A5388g), Python 3.14.4, MLX 0.32.0.153 repeats/cell, adaptive budget (~3 s wall/repeat, 256 MiB floor, 8 GiB cap),168 GiB incompressible test file written with `F_NOCACHE`. `iostat -d -w 1` logged17alongside as controller-level ground truth.1819```text20Hypothesis21  See hypothesis.md: ≥2 GB/s uncached random reads at ≥1 MiB blocks; sharp22  degradation below 64 KiB; modest degradation under concurrent Metal load.2324Falsification criterion25  <500 MB/s uncached random @1 MiB, or >50% collapse under GPU load.26  → NOT triggered. Hypothesis survives, and understates the hardware.2728Method / Baseline29  As registered in hypothesis.md (internal sequential baseline; no straw men).3031Result (mean of 3 repeats; std in results.json, mostly <2%)32  Cold random reads (F_NOCACHE file, iostat-validated):33    4 KiB   QD1:     67 MB/s   (16.6k IOPS)      QD8:    480 MB/s (117k IOPS)34    64 KiB  QD1:    770 MB/s                     QD8:   5138 MB/s35    256 KiB QD1:   2302 MB/s                     QD8:  11590 MB/s36    1 MiB   QD1:   4556 MB/s   QD4: 13628 MB/s   QD8:  13789 MB/s  ← at ceiling37  iostat peak disk throughput across the whole run: 13112 MB/s → the true38  device ceiling is ~13 GB/s. Warm-cache peak (page cache / unified memory):39  136 GB/s — a ~10× RAM:SSD ratio, far better than the ~100–250:1 cliff on40  discrete-GPU systems (PCIe + separate VRAM).41  Under concurrent MLX fp16 4096² matmul load (10 032 iterations, GPU42  saturated): 4 KiB QD8 480→478 MB/s; 1 MiB QD8 13789→13393 MB/s (−2.9%).4344  INVALID CELLS (flagged, not used): every cell whose mean exceeds the45  iostat ceiling is cache-contaminated — all 4 MiB cells (48–55 GB/s) and46  threaded sequential cells at ≥256 KiB (24–34 GB/s). Cause: budget ≈ file47  size, so repeats re-read blocks whose F_NOCACHE pages had not yet been48  evicted. Sequential single-thread cells ≤14 GB/s are plausible but treated49  as upper bounds only.5051Interpretation52  1. SSD weight-streaming is VIABLE on this hardware — with the right access53     pattern. At ~13 GB/s, a runtime spending 50% of a 100 ms/token budget on54     I/O can read ~650 MB/token; even QD1 1 MiB reads supply ~450 MB/token.55  2. The access-pattern contract is strict: ≥256 KiB blocks at QD≥4 (or56     ≥1 MiB at any QD) reach ≥60% of ceiling; 4 KiB random is 200× slower57     than ceiling. Weight block layout must therefore bundle to ≥256 KiB —58     consistent with LLM-in-a-flash's row-column bundling rationale.59  3. Unified-memory contention is a non-issue (<5% under full GPU matmul60     load): prefetch threads can run concurrently with Metal compute.61  4. macOS measurement discipline established: F_NOCACHE does not bypass62     already-resident pages (first run read 42–115 GB/s from cache);63     test data must be written uncached; any figure above the concurrent64     iostat ceiling is contamination; 16 KiB pages make sub-16 KiB uncached65     I/O impossible in principle.66  5. Bytes/token budget implication for candidates: interactive ≥5 tok/s67     leaves ≤~1.3 GB/token of SSD reads in the theoretical limit, ≤~65068     MB/token realistically. A 100 GB-class checkpoint is usable only if69     per-token touched bytes stay ~two orders of magnitude below total size70     — which is exactly the decoupling this project investigates.7172Next experiment73  expD/expG (progressive reconstruction + decision stability) become the74  binding questions: the storage substrate is not the bottleneck; the open75  question is whether ≤650 MB/token of *useful* residual/expert/sparse bytes76  suffice to preserve model behavior. Also rerun key cells with a 64 GiB77  test file (> RAM) to close the cache-contamination hole for sequential78  cells before publishing any sequential numbers.79```80