spb/localvm-research Public License

1---2project: localvm-research3document: expB_token_stability/analysis4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6created: 2026-08-127status: reviewed8---910# Analysis — expB_token_stability1112Run: `results/expB_token_stability/20260812T045224Z/` · consumes expA's trace13(Qwen3-1.7B, 6 144 positions × 28 layers), 64-neuron sets covering 95% of14per-token FFN energy.1516```text17Hypothesis / Falsification18  Hoped: Jaccard(t,t+1) ≥ 0.5 with slow decay; 128-token union ≤80%.19  Kill: Jaccard(t,t+1) < 0.3 OR 128-token union ≥95%.2021Result — KILL CRITERION TRIGGERED (union), stickiness ILLUSORY22  Jaccard(t,t+δ): 0.690 (δ=1) → 0.688 (δ=16) → 0.686 (δ=32) — flat.23  Random-set null at matched sizes: 0.640. Real stickiness beyond24  set-size artifact: +0.05 — negligible. (The registered ≥0.5 threshold25  is technically met at 0.69, but the null exposes it as a size effect:26  sets covering ~77% of blocks overlap ~64% by chance alone.)27  Union working set: 98.0% of blocks after 8 tokens, 99.3% after 32,28  99.9% after 128 — the entire FFN is touched within a few tokens.29  Domain locality (expC-lite): within-domain = across-domain = 1.0 —30  every trajectory's union covers everything; no domain-specific caches.3132Interpretation33  1. NEGATIVE, decisively: for dense SwiGLU models, per-token FFN energy34     working sets are neither sparse at pageable granularity (expA) nor35     temporally reusable (here) nor domain-clustered (here). The36     DejaVu / LLM-in-a-flash paradigm does NOT transfer to modern37     non-ReLU architectures without ReLUfication retraining — now38     measured, not just suspected from the literature. G06 (TealPager),39     G12 (HiddenPrefetch for dense FFNs) and G13 (DomainCache) are dead40     on this model family; G07 (WandaTiers) survives only for STATIC41     (input-independent) importance tiering.42  2. Scope note (honesty): we measured activation-ENERGY sets on a 1.7B43     dense model. Residual-relevance sets (which bytes fix a decision)44     could differ, and MoE models have architectural, not emergent,45     sparsity — C3 is untouched by this result. But the prior for any46     per-token dynamic weight selection on dense SwiGLU is now strongly47     unfavorable.48  3. PIVOT (with expF, expA): all three fine-grained routes for cutting49     C1's per-token escalation bytes are closed — layers (expF), blocks50     (expA), temporal caching (expB). What remains is BATCH AMORTIZATION,51     which expH makes attractive: the full 1 GB residual streams52     sequentially in ~80 ms at 13 GB/s; a margin-gated queue that defers53     low-margin tokens and refines them in periodic sweeps shares one54     sequential residual pass across many tokens. C1 thereby converges55     with C2 (Amortized Verification Sweeps) — the evidence has selected56     the candidate.5758Next experiment59  Prototype candidate: margin-gated deferred-refinement runtime60  (C1→C2 merge): 4-bit resident base generates optimistically; low-margin61  tokens queue; a periodic sequential residual sweep verifies/corrects62  (speculative-decoding-style rollback on flips). Measure end-to-end63  bytes/token, tok/s, and agreement on this Mac. Also rerun expG at 8B to64  check margin-signal scaling before committing the prototype design.65```66