README: continuous update — status, headline measured results
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Showing 1 changed file with +27 and −1
modified
README.md
+27 −1
@@ -4,6 +4,7 @@ document: README | ||
| 4 | 4 | author: Simon-Pierre Boucher |
| 5 | 5 | contact: contact@spboucher.ai |
| 6 | 6 | created: 2026-08-11 |
| 7 | +modified: 2026-08-12 | |
| 7 | 8 | status: draft |
| 8 | 9 | --- |
| 9 | 10 | |
@@ -23,7 +24,32 @@ total model size ≠ resident model size ≠ bytes read per token ≠ parameters | ||
| 23 | 24 | |
| 24 | 25 | - **Author:** Simon-Pierre Boucher — <contact@spboucher.ai> |
| 25 | 26 | - **Primary platform:** Apple Silicon Mac (unified memory, Metal, internal NVMe), macOS 14+ |
| 26 | −- **Status:** Phase 1 (literature research) in progress. See `research/LOG.md` for the audit trail. | |
| 27 | +- **Status:** Phases 1–4 complete (≈300-source literature sweep → state-of-the-art map → | |
| 28 | + 24 falsifiable gaps → ranked candidates). Micro-experiment campaign complete | |
| 29 | + (8 experiments: 3 confirmed, 3 usefully refuted, plus substrate + prototype). | |
| 30 | + A candidate architecture — **margin-gated deferred refinement with a layer-streamed | |
| 31 | + verifier** — was selected by the evidence and prototyped. See `research/LOG.md` for the | |
| 32 | + full audit trail and https://www.localvm.dev for the browsable paper trail. | |
| 33 | + | |
| 34 | +## Headline measured results (Apple M5 Max, 48 GB, macOS 27) | |
| 35 | + | |
| 36 | +- **SSD substrate (expH):** internal NVMe ceiling ≈ 13.1 GB/s (iostat-validated); | |
| 37 | + 67 MB/s at 4 KiB QD1 → 13.8 GB/s at 1 MiB QD8; Metal GPU load costs < 5%. | |
| 38 | + `F_NOCACHE` does not bypass resident pages — cold benchmarks must write uncached. | |
| 39 | +- **Decision stability (expG):** low-bit top-1 margin predicts disagreement with the | |
| 40 | + full-precision model (AUROC 0.85–0.98); improves with scale (esc@99%: 36.6% at 1.7B → | |
| 41 | + 22.6% at 8B, 4-bit). Naive affine 2-bit destroys the model (3% agreement). | |
| 42 | +- **Negative results (expF/expA/expB):** quantization damage is diffuse across depth; | |
| 43 | + SwiGLU FFN importance has no pageable block structure and no temporal working set — | |
| 44 | + dynamic sparsity paging (DejaVu / LLM-in-a-flash paradigm) measurably does not | |
| 45 | + transfer to dense modern architectures. | |
| 46 | +- **Prototype (candidate_01, 1.7B):** q4 resident + windowed q8 verification sweeps → | |
| 47 | + output quality statistically equal to resident-q8 (bf16 judge) at 237 MB/token | |
| 48 | + streamed (9× under the checkpoint), with a runtime knob (τ) trading bytes for quality. | |
| 49 | +- **Scale run (32B on 48 GB):** q8-32B (34.8 GB) cannot co-reside with the q4 base — | |
| 50 | + it is layer-streamed per sweep at ~11.6 GB/s sequential; verified output beats the | |
| 51 | + only fitting alternative (pure q4) by +0.28 nats (8B judge) at 3.7 GB/token | |
| 52 | + (9.4× under the checkpoint), 1.69 tok/s (prototype speed; headroom documented). | |
| 27 | 53 | |
| 28 | 54 | ## Layout |
| 29 | 55 | |
| 30 | 56 | |