spb/localvm-research Public License

1---2project: localvm-research3document: expD_progressive_reconstruction/hypothesis4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6created: 2026-08-127modified: 2026-08-128status: reviewed9---1011# Hypothesis — expD_progressive_reconstruction1213Follows expG (margin gating promoted; naive affine 2-bit base dead; 4-bit14escalation need 36.6%). This experiment measures how rapidly the token15decision and distribution converge as residual quantization stages are added16— the quality-vs-cumulative-bits curve that, combined with expG's escalation17rates and expH's byte budget, decides candidate C1's arithmetic.1819```text20Hypothesis21  Weights represented as base + residual stages (each stage an affine22  group-quantization of the previous stage's error) converge rapidly:23  one residual stage over a 3-bit base (≈6.4 cumulative bits/param)24  reaches ≥95% greedy agreement, and a margin-gated two-tier policy25  (stage-k decision when margin ≥ τ, stage-(k+1) decision otherwise)26  attains ≥97% agreement while consulting the residual for ≤40% of27  tokens. Hidden-state error shrinks monotonically with each stage.2829Falsification criterion30  If base3+1 residual (≈6.4 bits) stays below 90% agreement, or the31  two-tier margin policy cannot beat the flat next-stage agreement while32  escalating <50% of tokens, or hidden-state error does NOT decrease33  monotonically with stages (residual coding unstable), then progressive34  residual representations lose to simply shipping a flat higher-bit35  model, and C1 must pivot to expert/sparsity paging (C3) or amortized36  verification (C2).3738Method39  Model: Qwen3-1.7B bf16 reference (as expG). Residual ladders, affine40  group-64 quantization at every stage, applied to all divisible Linear41  layers: A) 3 → 3+3 → 3+3+3 bits; B) 4 → 4+4 bits.42  Same 48 trajectories × 128 tokens protocol as expG (teacher-forced).43  Per cumulative stage: agreement, KL(ref||stage), margin stats, AUROC,44  escalation curve. Two-tier policy simulation from recorded per-stage45  argmax/margins across a τ grid. Hidden-state relative L2 error vs46  reference at layer depths {25%, 50%, 75%, 100%} on 8 trajectories.47  Bits accounting includes scale/bias overhead (group 64 → +0.5 bits/param48  per stage at bf16 scales+biases).4950Baseline51  Flat MLX affine quantization at matched cumulative bit-widths from expG52  (3, 4, 8-bit rows) — the "just ship a bigger flat model" alternative.53  No straw men: residual ladders must beat or match flat models at equal54  bytes to be interesting.5556Result57  CONFIRMED (no kill criterion triggered). 3+3 bits: 95.7% agreement58  (≥95% target met); two-tier 4-bit policy: 97.6% @ 35% escalation59  (≥97% @ ≤40% met); hidden-state error monotone (≈5×/stage). Static60  parity caveat: flat quantization mildly beats ladders at equal bytes.61  Full numbers: results/expD_progressive_reconstruction/20260812T043508Z/.6263Interpretation64  Progressive coding's value is dynamic quality (one artifact, runtime-65  chosen precision), not compression. C1 operating point exists at66  4-bit base + 25–35% escalation. Open variable: bytes-per-escalation67  (full-residual re-run is too big at scale) → layer-restricted68  escalation (expF) or temporal locality (expB).6970Next experiment71  expF layer-sensitivity map; then expB temporal locality.72```73