---
project: localvm-research
document: expG_decision_stability/hypothesis
author: Simon-Pierre Boucher
contact: contact@spboucher.ai
created: 2026-08-12
modified: 2026-08-12
status: reviewed
---

# Hypothesis — expG_decision_stability

The Gate-Zero experiment (candidate_ranking.md C4): measures the joint
distribution of (cheap-pass top-1 margin × agreement with the full-precision
model) that the literature does not report, and which decides whether
margin-gated escalation (G02 MarginGate, and with it candidate C1) is viable.

```text
Hypothesis
  For a small pretrained LLM, the top-1 logit margin of a low-bit quantized
  variant is a strong predictor of whether its greedy token decision matches
  the full-precision model: disagreements concentrate at small margins.
  Concretely, margin achieves AUROC ≥ 0.8 as a disagreement detector at
  3–4 bits, so that escalating only the low-margin fraction of tokens
  (expected 10–30%) recovers ≥98–99% greedy agreement.

Falsification criterion
  If AUROC < 0.65 at every bit-width, or if reaching 99% agreement requires
  escalating > 50% of tokens at 4-bit (i.e., disagreements are NOT
  concentrated at low margins), then decision-uncertainty is not a usable
  gating signal: G02/G23 die and candidate C1 must be redesigned around
  amortized verification (C2) instead.

Method
  Model: Qwen3-1.7B (bf16 reference; representative modern architecture,
  runs fully in memory). Variants: MLX affine quantization at 2, 3, 4, 8
  bits (group size 64), quantizing all linear layers.
  For each of 48 prompts across 6 domains (code, math, chat, FR, EN,
  reasoning; benchmarks/datasets/eval_prompts.json): generate 128 tokens
  greedily with the bf16 reference, then teacher-force every variant over
  the same token sequence and record, per position: variant top-1 margin
  (logit gap between its top-2 candidates), variant argmax, reference
  argmax, agreement bit, and KL(ref||variant).
  Analysis: per bit-width and per domain — agreement rate; margin
  distributions conditioned on agree/disagree; AUROC of margin as a
  disagreement detector; escalation curve (fraction of tokens with margin
  below τ vs residual disagreement rate after escalating those tokens);
  the operating point (escalation fraction) that reaches 99% agreement.
  Seed fixed (greedy decode; deterministic). Hardware manifest embedded.

Baseline
  The 8-bit variant (known near-lossless: expected ≥99% agreement without
  any gating) anchors the top of the scale; published aggregate agreement
  rates for 4-bit (~90–91%, llama.cpp 'same top p' data) sanity-check our
  pipeline. No straw men: the comparison is against unconditional
  escalation (loading full weights for all tokens).

Result
  CONFIRMED (kill criteria not triggered). AUROC: 0.852 (3-bit), 0.898
  (4-bit), 0.975 (8-bit); esc@99%: 60.2% / 36.6% / 5.1%. Naive affine
  2-bit collapsed entirely (3% agreement, AUROC 0.605) — the ParetoQ
  cliff, measured. Full numbers: results/expG_decision_stability/
  20260812T042744Z/ and analysis.md.

Interpretation
  Margin is a strong, free, domain-uniform gating signal → G02 promoted.
  4-bit escalation need (36.6%) exceeds the hoped 10–30% band, so C1's
  economics now hinge on bytes-per-escalation (expD). Resident base must
  be ≥3-bit affine or codebook 2-bit (G05).

Next experiment
  expD — bytes of residual needed per escalated token; then expG rerun at
  3B–8B scale.
```