SPB Git forge
spboucher.ai ↗

spb/localvm-research

Public License

Running LLMs larger than memory on a consumer Mac — falsification-driven research: margin-gated deferred refinement, out-of-core verification on Apple Silicon. TR-01 published.

34commits 1branches 0releases
113.5 MBsize
maindefault branch
21 days agolast push
Python 63.2% JavaScript 23.5% CSS 11.8% Shell 0.9% Makefile 0.5%

History of experiments/candidate_01/implementation/runtime.py · clear filter

  1. Wed, Aug 12, 2026 4

    1. candidate_01 scale: StreamingVerifier (per-layer materialize->compute->re-lazify) + 32B benchmark
      …
      q4-32B resident (17.5 GB) + q8-32B (34.8 GB) layer-streamed from SSD per
sweep; peak residency = base + a few layers; SSD sees one sequential
checkpoint pass per sweep. Judge: independent 8B bf16.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 1 mo ago · 1 file +8 −2
    2. candidate_01 v2: incremental q8 verify-cache (O(window) sweeps) + bf16 quality judge
      …
      Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 1 mo ago · 1 file +18 −3
    3. candidate_01: fix truncating final-window rollback; verify EOS decisions; loop safety bound
      …
      Diagnostic: q8 prefill-vs-decode argmax divergence = 1.56% — the
fidelity ceiling for any teacher-forced verification on Metal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 1 mo ago · 1 file +30 −14
    4. candidate_01: margin-gated deferred-refinement runtime + benchmark + hypothesis
      …
      q4 resident base generates with margin recording; low-margin tokens
deferred; windowed q8 sweeps verify (margin mode) or verify-all
(QSpec-style exact); rollback via KV trim. Baselines: pure q4/q8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 1 mo ago · 1 file +129