spb/localvm-research Public License

Running LLMs larger than memory on a consumer Mac — falsification-driven research: margin-gated deferred refinement, out-of-core verification on Apple Silicon. TR-01 published.

Python 63.2% JavaScript 23.5% CSS 11.8% Shell 0.9% Makefile 0.5%

1.2 KB · 29 lines text

Raw Blame History

1resident q4: 0.97 GB · streamed q8: 1.83 GB2baseline: pure q8 greedy …3baseline: pure q4 greedy …4runtime: mode=margin tau=1.0 W=32 …5  fidelity=0.5482 tok/s=110.3 defer=0.37 rollback=0.064 MB/token(logical)=1506runtime: mode=margin tau=2.0 W=32 …7  fidelity=0.6611 tok/s=78.1 defer=1.00 rollback=0.103 MB/token(logical)=2118runtime: mode=verify-all tau=2.0 W=32 …9  fidelity=0.8864 tok/s=68.7 defer=1.09 rollback=0.121 MB/token(logical)=23710judging outputs with bf16 reference …11
Fetching 9 files:   0%|          | 0/9 [00:00<?, ?it/s]
Fetching 9 files: 100%|██████████| 9/9 [00:00<00:00, 2338.83it/s]12             pure_q4: mean logprob (bf16 judge) = -0.387213             pure_q8: mean logprob (bf16 judge) = -0.132814       margin_tau1.0: mean logprob (bf16 judge) = -0.267115       margin_tau2.0: mean logprob (bf16 judge) = -0.176116   verify-all_tau2.0: mean logprob (bf16 judge) = -0.12971718wrote /Users/simon-pierreboucher/Desktop/localvm-research/results/candidate_01/20260812T052415Z/results.json19baselines: {20 "pure_q4": {21  "tokens_per_s": 294.29358902494874,22  "fidelity_vs_q8_mean": 0.48632812523 },24 "pure_q8": {25  "tokens_per_s": 204.40615588770143,26  "fidelity_vs_q8_mean": 1.027 }28}29