project: localvm-research document: expF_error_accumulation/analysis author: Simon-Pierre Boucher contact: contact@spboucher.ai created: 2026-08-12 status: reviewed
Analysis — expF_error_accumulation
Run: results/expF_error_accumulation/20260812T043945Z/ · code committed before
run. Qwen3-1.7B, 28 layers in 7 depth-groups of 4, affine g64 4-bit, 48
trajectories × 128 tokens teacher-forced, 17 configurations.
text
Hypothesis / Falsification
Hypothesized ≥3× sensitivity spread across depth groups and ≥40% of lost
agreement recovered by repairing the top ~25% of layers.
Kill criteria: <2× spread, or top-25% repair recovering <20%.
Result — HYPOTHESIS REFUTED (spread criterion killed; repair borderline-negative)
All-4-bit floor: 87.53% agreement (loss = 12.47 points vs bf16).
DEGRADE-ONE (one group 4-bit, rest bf16): agreement drops span only
3.7–5.6 points across the 7 groups — a 1.5× spread (< the 2× kill
line). Mid-depth groups (L12–19) are marginally most sensitive; the
LAST group is among the LEAST sensitive (3.96) despite expD showing
the largest hidden-state error there — late-layer error is large in
norm but decision-benign.
REPAIR-ONE (one group bf16, rest 4-bit): best single group (L12–15)
recovers +1.5 points = 12% of the loss; worst 5%.
REPAIR-TOP-K: top-2 groups (29% of layers) recover 24% of the loss;
top-3 (43% of layers) recover 33% — consistently SUB-proportional.
Cross-check of additivity: individual degrade-one drops sum to ~33
points, yet degrading everything at once costs only 12.5 — errors
partially mask each other; symmetrically, repair values sum to 7.4
of the 12.5 lost — repair requires cooperation across depth.
Interpretation
1. NEGATIVE RESULT (recorded per charter §10/§17): quantization damage
to token decisions is DIFFUSE and cooperative across depth, not
concentrated. Layer-restricted escalation cannot materially cut
bytes-per-escalation: paying 29% of the residual bytes buys only
24% of the repair — worse than linear, no leverage.
2. expD's depth-concentrated hidden-state error was a red herring for
decision repair: large late-layer representation drift coexists with
benign decisions (norms ≠ decisions — a caution for any design that
gates on hidden-state error instead of decision margin).
3. Consequently C1's bytes-per-escalation must come from the remaining
mechanisms: (b) temporal locality — the residual working set of
escalated tokens staying hot in RAM (expB); (c) sub-layer/block-level
selection — repair only the weight blocks that matter for THIS token
(expA/expE, finer grain than layers); or (d) batch-amortized
escalation — G17/G18-style verification sweeps sharing one residual
read across many queued low-margin tokens.
4. Design implication: if (b) also fails, C1 degrades into "resident
4-bit + rare whole-model refinement passes" — which is exactly
candidate C2 (amortized verification). The two candidates are
converging on the same mechanism from opposite ends; this is
useful, not disappointing.
Next experiment
expA (weight contribution concentration at BLOCK granularity within
layers) — the finer-grained version of the question expF just answered
negatively at layer granularity; feeds expE (partial GEMM) directly.
In parallel, expB (temporal stability of important blocks) decides the
page-cache path (b).