spb/localvm-research Public License
Running LLMs larger than memory on a consumer Mac — falsification-driven research: margin-gated deferred refinement, out-of-core verification on Apple Silicon. TR-01 published.
Python 63.2%
JavaScript 23.5%
CSS 11.8%
Shell 0.9%
Makefile 0.5%
1---2project: localvm-research3document: expH_ssd_feasibility/hypothesis4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6created: 2026-08-117modified: 2026-08-128status: draft9---1011# Hypothesis — expH_ssd_feasibility1213```text14Hypothesis15 The internal Apple NVMe SSD (AP2048Z, 2 TB) sustains enough *random* read16 bandwidth at weight-block-sized granularities (256 KB–4 MB) to stream a17 meaningful fraction of model weights per token: we expect ≥ 2 GB/s random18 reads at ≥ 1 MB blocks with F_NOCACHE (true storage path), degrading19 sharply below 64 KB, and only modest degradation under concurrent Metal20 GPU compute (unified memory contention is expected to be small relative21 to SSD ceiling).2223Falsification criterion24 If uncached random reads at 1 MB blocks sustain < 500 MB/s, or collapse25 by > 50% under concurrent GPU matmul load, SSD weight streaming cannot26 supply even ~0.5 GB/token at interactive rates and candidate designs27 must not assume per-token SSD reads on this class of hardware.2829Method30 Create an 8 GiB incompressible test file on the internal APFS volume.31 Measure read throughput at block sizes {4 KB, 16 KB, 64 KB, 256 KB,32 1 MB, 4 MB} × {sequential, random} × {F_NOCACHE on, off/warm cache} ×33 {1, 4, 8 reader threads}. Cold-cache condition enforced via F_NOCACHE34 (macOS has no O_DIRECT); warm condition by pre-reading the file.35 Repeat each cell 3 times, report mean/median/std. Then repeat the key36 uncached random cells while an MLX fp16 4096×4096 matmul loop saturates37 the GPU. Record hardware manifest and thermal level before/after.3839Baseline40 Apple's advertised sequential throughput for this SSD class (~5-7 GB/s)41 and the sequential-read measurement from the same harness (internal42 baseline; no straw men — we compare random vs our own sequential).4344Result45 Hypothesis CONFIRMED and understated. Cold random reads: 67 MB/s (4 KiB46 QD1) → 13.8 GB/s (1 MiB QD8, at the iostat-validated ~13.1 GB/s device47 ceiling). GPU load cost <5%. 4 MiB and threaded-sequential cells were48 cache-contaminated and are flagged invalid. Full numbers:49 results/expH_ssd_feasibility/20260812T034359Z/ and analysis.md.5051Interpretation52 SSD streaming is viable at ~650 MB/token (realistic) if and only if53 blocks are ≥256 KiB at QD≥4. Prefetch can overlap Metal compute freely.54 See analysis.md for the full argument and the measurement-discipline55 findings (F_NOCACHE semantics).5657Next experiment58 expD/expG — the substrate is not the bottleneck; whether ≤650 MB/token of59 useful bytes preserves behavior is. Secondary: rerun sequential cells with60 a 64 GiB (> RAM) file.61```62