--- project: localvm-research document: expH_ssd_feasibility/analysis author: Simon-Pierre Boucher contact: contact@spboucher.ai created: 2026-08-12 status: reviewed --- # Analysis — expH_ssd_feasibility Run: `results/expH_ssd_feasibility/20260812T034359Z/` · code committed at `b4a652d` Hardware: Apple M5 Max (6P+12E, 40 GPU cores), 48 GB unified memory, APPLE SSD AP2048Z (2 TB, internal APFS), macOS 27.0 (26A5388g), Python 3.14.4, MLX 0.32.0. 3 repeats/cell, adaptive budget (~3 s wall/repeat, 256 MiB floor, 8 GiB cap), 8 GiB incompressible test file written with `F_NOCACHE`. `iostat -d -w 1` logged alongside as controller-level ground truth. ```text Hypothesis See hypothesis.md: ≥2 GB/s uncached random reads at ≥1 MiB blocks; sharp degradation below 64 KiB; modest degradation under concurrent Metal load. Falsification criterion <500 MB/s uncached random @1 MiB, or >50% collapse under GPU load. → NOT triggered. Hypothesis survives, and understates the hardware. Method / Baseline As registered in hypothesis.md (internal sequential baseline; no straw men). Result (mean of 3 repeats; std in results.json, mostly <2%) Cold random reads (F_NOCACHE file, iostat-validated): 4 KiB QD1: 67 MB/s (16.6k IOPS) QD8: 480 MB/s (117k IOPS) 64 KiB QD1: 770 MB/s QD8: 5138 MB/s 256 KiB QD1: 2302 MB/s QD8: 11590 MB/s 1 MiB QD1: 4556 MB/s QD4: 13628 MB/s QD8: 13789 MB/s ← at ceiling iostat peak disk throughput across the whole run: 13112 MB/s → the true device ceiling is ~13 GB/s. Warm-cache peak (page cache / unified memory): 136 GB/s — a ~10× RAM:SSD ratio, far better than the ~100–250:1 cliff on discrete-GPU systems (PCIe + separate VRAM). Under concurrent MLX fp16 4096² matmul load (10 032 iterations, GPU saturated): 4 KiB QD8 480→478 MB/s; 1 MiB QD8 13789→13393 MB/s (−2.9%). INVALID CELLS (flagged, not used): every cell whose mean exceeds the iostat ceiling is cache-contaminated — all 4 MiB cells (48–55 GB/s) and threaded sequential cells at ≥256 KiB (24–34 GB/s). Cause: budget ≈ file size, so repeats re-read blocks whose F_NOCACHE pages had not yet been evicted. Sequential single-thread cells ≤14 GB/s are plausible but treated as upper bounds only. Interpretation 1. SSD weight-streaming is VIABLE on this hardware — with the right access pattern. At ~13 GB/s, a runtime spending 50% of a 100 ms/token budget on I/O can read ~650 MB/token; even QD1 1 MiB reads supply ~450 MB/token. 2. The access-pattern contract is strict: ≥256 KiB blocks at QD≥4 (or ≥1 MiB at any QD) reach ≥60% of ceiling; 4 KiB random is 200× slower than ceiling. Weight block layout must therefore bundle to ≥256 KiB — consistent with LLM-in-a-flash's row-column bundling rationale. 3. Unified-memory contention is a non-issue (<5% under full GPU matmul load): prefetch threads can run concurrently with Metal compute. 4. macOS measurement discipline established: F_NOCACHE does not bypass already-resident pages (first run read 42–115 GB/s from cache); test data must be written uncached; any figure above the concurrent iostat ceiling is contamination; 16 KiB pages make sub-16 KiB uncached I/O impossible in principle. 5. Bytes/token budget implication for candidates: interactive ≥5 tok/s leaves ≤~1.3 GB/token of SSD reads in the theoretical limit, ≤~650 MB/token realistically. A 100 GB-class checkpoint is usable only if per-token touched bytes stay ~two orders of magnitude below total size — which is exactly the decoupling this project investigates. Next experiment expD/expG (progressive reconstruction + decision stability) become the binding questions: the storage substrate is not the bottleneck; the open question is whether ≤650 MB/token of *useful* residual/expert/sparse bytes suffice to preserve model behavior. Also rerun key cells with a 64 GiB test file (> RAM) to close the cache-contamination hole for sequential cells before publishing any sequential numbers. ```