candidate_01: W=64 lever NEGATIVE — rollback rate bounds sweep amortization
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Showing 4 changed files with +138 and −0
modified
experiments/candidate_01/analysis.md
+25 −0
@@ -151,3 +151,28 @@ Findings | ||
| 151 | 151 | Next: Phase 11 novelty check on the assembled architecture (margin-gated |
| 152 | 152 | deferral + layer-streamed amortized verification on unified memory), then |
| 153 | 153 | either the throughput levers above or the 14B/70B scale points. |
| 154 | + | |
| 155 | +## Addendum — window lever test: W=64 (2026-08-12) | |
| 156 | + | |
| 157 | +Run: `results/candidate_01_scale32b/20260812T061115Z/` (verify-all, 32B, same | |
| 158 | +12 prompts × 96 tokens). | |
| 159 | + | |
| 160 | +```text | |
| 161 | + W=32 W=64 expected if sweeps ∝ 1/W | |
| 162 | +GB/token 3.72 3.53 ~1.9 | |
| 163 | +tok/s 1.69 1.59 higher | |
| 164 | +judge logprob -0.467 -0.514 equal | |
| 165 | +``` | |
| 166 | + | |
| 167 | +NEGATIVE (lever mostly ineffective): doubling the window cut bytes/token by | |
| 168 | +only 5%, not ~2×. Cause: in verify-all mode the rollback rate (~0.12/token) | |
| 169 | +means a 64-token window almost always contains a flip; the sweep fires at the | |
| 170 | +flip, the window restarts, and the effective inter-sweep distance is bounded | |
| 171 | +by ~1/rollback-rate (≈8–15 tokens), not by W. Quality drifted slightly down | |
| 172 | +(n=12, no error bars — treat as noise until repeated). | |
| 173 | + | |
| 174 | +Implication: sweep amortization is governed by ROLLBACK dynamics, not window | |
| 175 | +size. The levers that can actually move throughput/bytes are (a) margin-mode | |
| 176 | +with τ calibrated on the real quantized artifact's margin distribution | |
| 177 | +(fewer checked positions → fewer flips that trigger rollbacks), and | |
| 178 | +(b) overlapping sweep I/O with continued generation. W-tuning alone is dead. | |
modified
experiments/candidate_01/scale32b_w64.log
+8 −0
@@ -27,3 +27,11 @@ runtime: mode=verify-all tau=2.0 W=64 … | ||
| 27 | 27 | 9/12 (96 tok, 6 sweeps, 5 rollbacks, last sweep io 3.0s) |
| 28 | 28 | 10/12 (95 tok, 11 sweeps, 11 rollbacks, last sweep io 3.0s) |
| 29 | 29 | 11/12 (93 tok, 13 sweeps, 13 rollbacks, last sweep io 3.0s) |
| 30 | + 12/12 (96 tok, 10 sweeps, 9 rollbacks, last sweep io 2.8s) | |
| 31 | + tok/s=1.59 sweepLat=3.0s GB/token(logical)=3.53 | |
| 32 | +freeing 32B models; loading 8B bf16 judge … | |
| 33 | + Fetching 12 files: 0%| | 0/12 [00:00<?, ?it/s] Fetching 12 files: 100%|██████████| 12/12 [00:00<00:00, 5958.52it/s] | |
| 34 | + pure_q4: -0.7506 | |
| 35 | + verify-all_tau2.0: -0.5144 | |
| 36 | + | |
| 37 | +wrote /Users/simon-pierreboucher/Desktop/localvm-research/results/candidate_01_scale32b/20260812T061115Z/results.json | |
modified
research/LOG.md
+13 −0
@@ -340,3 +340,16 @@ Format per entry: date/time (local, with timezone) · question · experiment · | ||
| 340 | 340 | Lever/CATS's regime with (a) precision-cast drafts (no training), (b) margin-priced |
| 341 | 341 | verification I/O, (c) bytes/token accounting. Re-run this check before publication — |
| 342 | 342 | prior-art density in this exact area is rising quarter over quarter. |
| 343 | + | |
| 344 | +--- | |
| 345 | + | |
| 346 | +## 2026-08-12 18:05 EDT — W=64 lever test: NEGATIVE — rollback rate, not window size, bounds amortization | |
| 347 | + | |
| 348 | +- **Question:** Does doubling the sweep window halve bytes/token? | |
| 349 | +- **Experiment:** 32B verify-all, W=64 vs W=32 (`results/candidate_01_scale32b/20260812T061115Z/`). | |
| 350 | +- **Result:** GB/token 3.72→3.53 (−5%, not −50%); tok/s 1.69→1.59; judge −0.467→−0.514 (n=12, noise). | |
| 351 | +- **Interpretation:** Effective inter-sweep distance is bounded by ~1/rollback-rate (≈8–15 tokens | |
| 352 | + at 12%/token), so W beyond that is dead weight. Real levers: τ-calibrated margin mode | |
| 353 | + (fewer checked positions → fewer rollback triggers) and sweep-I/O/generation overlap. | |
| 354 | +- **Decision:** Session milestone reached; next cycle: honest baselines (resident q5/q6 at 32B), | |
| 355 | + τ calibration on the real q4-32B margin distribution, task-level benchmarks, 70B point. | |
added
results/candidate_01_scale32b/20260812T061115Z/results.json
+92 −0
@@ -0,0 +1,92 @@ | ||
| 1 | +{ | |
| 2 | + "experiment": "candidate_01_scale32b", | |
| 3 | + "author": "Simon-Pierre Boucher", | |
| 4 | + "contact": "contact@spboucher.ai", | |
| 5 | + "manifest": { | |
| 6 | + "author": "Simon-Pierre Boucher", | |
| 7 | + "contact": "contact@spboucher.ai", | |
| 8 | + "project": "localvm-research", | |
| 9 | + "collected_utc": "2026-08-12T06:11:15.614864+00:00", | |
| 10 | + "chip": { | |
| 11 | + "brand": "Apple M5 Max", | |
| 12 | + "arch": "arm64", | |
| 13 | + "cores_total": 18, | |
| 14 | + "cores_performance": 6, | |
| 15 | + "cores_efficiency": 12, | |
| 16 | + "gpu_cores": 40 | |
| 17 | + }, | |
| 18 | + "memory": { | |
| 19 | + "unified_bytes": 51539607552, | |
| 20 | + "unified_gb": 48.0, | |
| 21 | + "pagesize": 16384 | |
| 22 | + }, | |
| 23 | + "ssd": { | |
| 24 | + "model": "APPLE SSD AP2048Z", | |
| 25 | + "size": "2 TB", | |
| 26 | + "smart_status": "Verified" | |
| 27 | + }, | |
| 28 | + "os": { | |
| 29 | + "product": "macOS", | |
| 30 | + "version": "27.0", | |
| 31 | + "build": "26A5388g", | |
| 32 | + "kernel": "27.0.0" | |
| 33 | + }, | |
| 34 | + "software": { | |
| 35 | + "python": "3.14.4", | |
| 36 | + "mlx": "0.32.0", | |
| 37 | + "mlx_lm": "0.31.3", | |
| 38 | + "torch": null, | |
| 39 | + "numpy": "2.5.2" | |
| 40 | + }, | |
| 41 | + "git": { | |
| 42 | + "commit": "ce7b70032b95506529bfc1d4c59e9b55dd7df958", | |
| 43 | + "dirty_tree": true | |
| 44 | + }, | |
| 45 | + "thermal_level_at_collect": null | |
| 46 | + }, | |
| 47 | + "config": { | |
| 48 | + "per_domain": 2, | |
| 49 | + "max_tokens": 96, | |
| 50 | + "window": 64, | |
| 51 | + "taus": "2.0", | |
| 52 | + "modes": "verify-all" | |
| 53 | + }, | |
| 54 | + "models": { | |
| 55 | + "base": "mlx-community/Qwen3-32B-4bit", | |
| 56 | + "verify": "mlx-community/Qwen3-32B-8bit", | |
| 57 | + "judge": "mlx-community/Qwen3-8B-bf16" | |
| 58 | + }, | |
| 59 | + "q8_streamed_bytes": 34810574466, | |
| 60 | + "baseline_pure_q4_tokens_per_s": 26.338828755243266, | |
| 61 | + "runs": [ | |
| 62 | + { | |
| 63 | + "mode": "verify-all", | |
| 64 | + "tau": 2.0, | |
| 65 | + "window": 64, | |
| 66 | + "tokens_per_s": 1.5931855479899018, | |
| 67 | + "deferral_rate": 1.6887892376681615, | |
| 68 | + "rollback_rate": 0.09596412556053811, | |
| 69 | + "sweep_latency_s_mean": 2.9809600431238654, | |
| 70 | + "logical_verify_bytes_per_token": 3527887815.8367715, | |
| 71 | + "raw": { | |
| 72 | + "tokens": 1115, | |
| 73 | + "deferred": 1883, | |
| 74 | + "sweeps": 113, | |
| 75 | + "rollbacks": 107, | |
| 76 | + "sweep_s": 336.8484848729968, | |
| 77 | + "gen_s": 363.0072233380015, | |
| 78 | + "logical_bytes": 3933594914658 | |
| 79 | + } | |
| 80 | + } | |
| 81 | + ], | |
| 82 | + "quality_8b_judge": { | |
| 83 | + "pure_q4": { | |
| 84 | + "mean_logprob_8b_judge": -0.7506343560914198, | |
| 85 | + "n": 12 | |
| 86 | + }, | |
| 87 | + "verify-all_tau2.0": { | |
| 88 | + "mean_logprob_8b_judge": -0.5144313722848892, | |
| 89 | + "n": 12 | |
| 90 | + } | |
| 91 | + } | |
| 92 | +} | |
| \ No newline at end of file | ||
| 93 | ||