spb/modelmap Public License

1---2project: modelmap3document: Research log (append-only)4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6website: https://modelmap.io7created: 2026-08-128status: draft9---1011# modelmap — Research log1213Append-only. Every entry records: date/time with timezone, question, experiment,14result, interpretation, decision. The entire reasoning process must be auditable.1516---1718## 2026-08-12 00:30 EDT — Project bootstrap1920**Question.** None yet — administrative bootstrap.2122**Action.** Repository initialized from the research charter (`CLAUDE.md`):23full skeleton per charter §3, mandatory author headers on every source file,24`tools/check_headers.py` written first (charter §0.1 rule 3), experiment25directories A–H scaffolded with empty seven-field scientific blocks26(hypotheses will be registered before any run), and the public platform27(`site/`) built and deployed to https://www.modelmap.io on MacLustr node28M3U96b (PM2 + ngrok), mirroring the architecture of the sister project29localvm-research (www.localvm.dev) so both research platforms stay coherent.3031**Decision.** Next step is Phase 1: ultra-deep literature research across the32ten areas of charter §4 (observational probing, causal methods, feature33decomposition, circuits, weight-space cartography, knowledge editing,34cross-model comparison, statistics/epistemology, systems, atlas design).35No mapping code is written before the state-of-the-art map (Phase 2) exists.3637---3839## 2026-08-12 02:00 EDT — Phase 1 sweep #1: ten-area literature survey4041**Question.** What does the field already know, per charter §4.1–4.10, and42which of modelmap's core bets survive first contact with the literature?4344**Method.** Five parallel web-verified survey passes (two charter areas each);45every source logged with URL + access date in `research/bibliography.md`46(~200 entries); per-area reading notes written to `research/notes/4.X_*.md`.47This is sweep #1 of Phase 1 — deeper follow-ups (reading full papers behind48the highest-stakes claims, backward/forward citation chasing) remain.4950**Key results (detail in the notes):**51521. **The quantization gap is real (Q4/expF bet confirmed).** ~5 shallow papers53   exist on quantization × interpretability; none does probe-drift, circuit54   survival, Q2/Q3 internals, or GGUF-runtime introspection. Open territory.552. **The weight-only pre-screen gap is real but partially anticipated (expE).**56   WeightWatcher (behavioral quality from spectra), Super Weight (weight-visible57   outliers → massive activations), Watch-the-Weights (2025) are adjacent; no58   systematic weight-only-statistics → activation-level-properties study exists.593. **Published noise floors to build on:** only ~30% of SAE latents replicate60   across seeds (Paulo & Belrose 2025); only 1–5% of neurons are universal61   across seeds (Gurnee 2024); global platonic-convergence metrics deflate62   under calibration (2026). These calibrate expA/expD and our63   replication-rate metric.644. **Central methodological caution:** localization ≠ editability (Hase 2023);65   faithfulness scores are not robust to ablation-scheme choice (Miller 2024);66   subspace patching can hit dormant pathways (Makelov vs Wu, unresolved);67   random-init baselines + FDR control are mandatory (Dead Salmons 2025).685. **Apple Silicon tooling is a desert (Q4 confirmed):** TransformerLens MPS is69   opt-in/unstable; editing repos are CUDA-first; only mlxterp (tiny, 2025)70   targets MLX; llama.cpp eval-callback enables quantized-runtime capture but71   nothing wraps it; no macOS activation-store or hook-overhead benchmarks72   exist anywhere.736. **Atlas prior-art check:** no existing artifact host combines versioned74   entries + regeneration recipes + confidence levels + negative results75   (verified across Neuronpedia, Neuroscope, Transluce, circuit-tracer,76   Gemma Scope). "Model atlas" name collision with Horwitz 2025 (inter-model77   graph, not internals) noted for novelty_check.md.7879**Interpretation (confidence: this is a literature map, not a result).**80The charter's bets (quantization maps, weight-only pre-screens, Mac-local81tooling, provenance-first atlas) all survive; several have adjacent 2025–202682work converging on them, so speed matters. The field's own rigor wave83(MIB, Tracr/InterpBench validation, statistical-inference framing) supplies84ready-made harnesses for our confidence taxonomy.8586**Decision.** Proceed to Phase 2 (`research/state_of_the_art.md`): organize87these techniques into the charter §5 table with epistemic status per88technique, then Phase 3 gap generation (≥20 candidates). Validation-first89rule adopted: every future pipeline is validated on Tracr/InterpBench-style90ground truth before touching a real model.9192---9394## 2026-08-12 02:45 EDT — Phase 2: state-of-the-art map written9596**Question.** Organize Phase 1 into the charter §5 taxonomy: what does each97technique actually measure, at what cost on a Mac, with what failure modes98and epistemic status — and where are the overlaps and novelty traps?99100**Action.** Wrote `research/state_of_the_art.md`: ~40 techniques across six101families + cross-cutting instruments, each with the eleven-field block102(condensed), epistemic flags (established / contested / debunked-in-part),103an overlap analysis (6 "secretly the same thing" identities), a list of104already-tried combinations, a novelty-trap list, and an estimated Mac cost105frontier table (16/32/64 GB) that Experiment H must convert to measured.106107**Key synthesis judgments (recorded, not yet evidence):**108- Screening/verification split adopted: attribution patching (EAP-IG) screens,109  real patching verifies; auto-interp labels never exceed Level 1 without110  intervention scoring.111- One-estimator-family insight: SAE/transcoder/crosscoder/CLT share the112  seed-replication pathology — replication findings transfer across them.113- CKA-class metrics are never standalone evidence: ≥2 metrics + nulls +114  outlier-stripped variants required.115- The atlas map-card schema fields fall out of Phase 2: pinned patching116  protocol, ablation-scheme curves, featurizer class, seed replication rate,117  per-dataset agreement.118119**Decision.** Proceed to Phase 3 (`research/research_gaps.md`): ≥20120substantially different candidate directions, each with the six-field gap121block and a smallest falsifying experiment runnable on a Mac. Seeds from122Phase 2: quantization × internals; weight-only pre-screens vs activation123ground truth; replication-rate-first artifacts; quantized-runtime capture;124provenance-first atlas schema; cross-model coordinate systems anchored on125induction heads.126127---128129## 2026-08-12 03:20 EDT — Phase 3: 24 research gaps registered130131**Question.** Which substantially different directions are (a) genuinely132open per the Phase 1/2 evidence, (b) falsifiable with a Mac-scale133experiment, (c) worth ranking in Phase 4?134135**Action.** Wrote `research/research_gaps.md`: 24 gaps (G01–G24) in seven136clusters — quantization×internals (G01–G05), weight-only pre-screens137(G06–G09), replication/method-agreement science (G10–G13), cross-model138coordinates (G14–G16), localization science (G17–G19), systems/tooling139(G20–G22), atlas methodology & localvm bridge (G23–G24). Every gap carries140the six-field block including its smallest falsifying Mac experiment.141All charter §6 example categories are covered; four gaps (G05, G08, G10,142G19) fall outside them. Noted that the 24 gaps share ~5 falsification143kernels, so harnesses amortize across gaps.144145**Interpretation.** The falsifiers are deliberately harsh (e.g. G06 dies at146cross-validated R²<0.2; G11 dies at κ≤0; G17 must separate Tracr from147random-init before touching real models). Several gaps are informative in148both outcomes (G05's runtime-vs-simulated discrepancy, G24's working-set149negative), which is the profile the charter asks for.150151**Decision.** Proceed to Phase 4 (`research/candidate_ranking.md`): score152all 24 gaps on the ten charter axes with written reasoning, select 3–5153prototype candidates. Prior expectation (to be tested by the scoring, not154assumed): the quantization cluster and the weight-only pre-screen cluster155combine high novelty with low cost; G10/G23 are infrastructure that every156other candidate needs regardless of ranking.157158---159160## 2026-08-12 03:50 EDT — Phase 4: candidates ranked and selected161162**Question.** Which of the 24 gaps become prototype candidates, and in what163order does everything else get built?164165**Action.** Wrote `research/candidate_ranking.md`: full 24×10 score table166(convention: 10 always favorable, including complexity and risk), grouped167written reasoning, and selection. Updated `experiments/candidate_01–04`168READMEs with assignments.169170**Selection:**171- **candidate_01 — Weight-only pre-screen** (G06 + G08 anchor, G03/G09):172  kill at cross-validated R² < 0.2 on Pythia-410M.173- **candidate_02 — Quantization deformation atlas** (G01 + G02, later174  G04/G05): Qwen3-0.6B FP16-vs-Q4 probe drift vs noise floor.175- **candidate_03 — Localization index** (G17 + G11/G12 as measurement176  layers): must separate Tracr (≈1) from random-init (≈0) first.177- **candidate_04 — localvm working-set bridge** (G24 + G09), reduced scope:178  run the domain-conditioned ablation falsifier before committing.179- **Mandatory substrate first, outside the ranking:** G20 (expH cost180  frontier), G10 (expA/expD noise floors), G23 (map cards v0), G13181  (ablation-curves rule, adopted at zero cost).182183**Interpretation.** The infrastructure quartet (G23 87, G20 86, G10 82,184G13 75) tops the table by construction — every candidate presupposes it.185The three main candidates fail on independent kernels (weights-regression /186drift-vs-floor / index-validation), satisfying the charter's demand for187genuinely distinct bets; each yields versioned artifacts even when negative.188Cross-model coordinate work (G14–G16) is deliberately blocked on noise189floors — alignment claims without floors are the field's known failure mode.190191**Decision.** Phases 1–4 complete. Next: Phase 5 (experimental framework) —192implement the substrate in order: (1) expH harness (hook overhead MLX vs193MPS, storage formats on APFS, streaming throughput) reusing localvm194patterns; (2) expA probe-reliability harness with the full controls195doctrine; (3) map-cards v0 schema in src/modelmap/atlas enforced by196tools/publish.py. Register hypotheses in experiments/micro/expH and expA197before their first runs.198199---200201## 2026-08-12 05:30 EDT — Phase 5 substrate + expH run #1 (FIRST MEASURED RESULT)202203**Question.** Build the substrate; then: which activation-store format wins204on APFS, and what does capture actually cost on MLX vs PyTorch-MPS?205206**Built.**207- `src/modelmap/atlas/mapcard.py` — map-card schema v0 (G23): provenance +208  per-level requirements (L1: ≥3 seeds, ≥2 promptsets, replication_rate;209  L2: ≥2 methods; L3: interventions + featurizer class). `tools/publish.py`210  is the gate: refuses any atlas entry without a valid card.211- `src/modelmap/stats/replication.py` — bootstrap CIs, BH-FDR, selectivity,212  Jaccard replication_rate (the §8.3 metric).213- `src/modelmap/probes/linear.py` — probe_with_control: every probe carries214  its shuffled-label control by construction. **The unit test caught a real215  bug before any science: `rng.shuffle(y[tr])` shuffles a fancy-indexed216  COPY, so the "control" was training on true labels.** Fixed; tests now217  verify the probe separates planted structure (selectivity>0.3) from pure218  noise (|selectivity|<0.15). 5/5 tests pass; ruff clean; env = Python219  3.14.4, numpy 2.5.2, torch 2.13 (MPS), mlx 0.32, zarr 3.3.220221**expH run #1** (hypothesis registered BEFORE the run; M5 Max 48 GB,222macOS 27.0; results/expH_capture_cost_frontier/20260812T052441Z/):223- **(A) CONFIRMED.** Random-batch reads (SAE-shuffle pattern, warm cache):224  raw-mmap 4.09 GB/s, safetensors(mmap) 4.26, zarr-uncompressed 1.27,225  zarr-zstd 0.38 → mmap beats zarr 3.2×/10.8×, past the pre-registered 2×226  bar. safetensors write wins at 8.10 GB/s (single large write).227- **(B) CONFIRMED, asymmetrically.** Capture overhead: torch-MPS retain228  1.22× plain, +copy+write 1.47×; **MLX retain 1.02× (capture nearly free,229  lazy-graph retention), +copy+write 1.11×.** MLX baseline also 1.29×230  faster than torch-MPS on the identical synthetic 12-layer model.231- **Confidence: Level 0–1** (3 repeats, tight variance, but single machine,232  warm cache, synthetic model — all declared in advance).233- **Decisions licensed:** activation stores = mmap-backed flat files (zarr234  eliminated for the shuffle pattern pending the cold-cache pass); MLX is235  the preferred capture backend; capture cost is not the bottleneck — the236  storage wall is.237238**Next.** expH run #2 (cold cache) and #3 (real 0.5B checkpoint via239mlx-lm); then expA first run (hypothesis registered: dataset variance >240seed variance; harness validity gate = selectivity<0.05 on random-init).241expA's output becomes the first real map card. Replicate expH grid on242M3U96a/M2U64 to move the cost table cross-hardware.243244---245246## 2026-08-12 06:30 EDT — expH runs #2 & #3: first FALSIFIED hypothesis + first quantized capture247248**Questions.** Does the warm-cache storage ordering survive a cold cache249(run #2, on M3U96a with `sudo purge` per repetition — also the first250cross-hardware point)? Do synthetic capture overheads transfer to a real2514-bit checkpoint (run #3, mlx-lm Qwen3-0.6B-4bit)?252253**Run #2 — FALSIFIED (the project's first registered-and-killed254hypothesis).** Cold random-batch reads on M3 Ultra: zarr-uncompressed2550.62 GB/s > safetensors-mmap 0.32 > zarr-zstd 0.19 > raw-mmap 0.14 —256the warm ordering (mmap 3.2–10.8× ahead) fully INVERTS. Mechanism: cold257mmap is page-fault IO (~8–16 KiB, QD1), exactly the regime localvm's SSD258grid measured at ~0.07–0.14 GB/s; zarr read()s 32 MiB chunks — the259≥256 KiB regime where Apple NVMe delivers. Run #1 measured memory, not260disk. **The rule is IO granularity, not the container.** Store design261revised: chunk-aligned batched access mandatory for larger-than-RAM262stores; run #4 registered (chunk-sorted ≥1 MiB mmap reads vs zarr, cold).263Cross-project convergence with localvm expH is strong — the bridge works264in both directions.265266**Run #3 — CONFIRMED, stronger than predicted.** Real Qwen3-0.6B-4bit267prefill (901 tokens, 28 layers): plain 28.5 ms → retain 28.6 ms268(**1.004× — capture is free under MLX lazy evaluation**) → retain+cast+269mmap-write 36.4 ms (1.28×). First quantized-model activation capture in270Python tooling (the §4.9 gap, now demonstrated). Engineering notes:271d_model must be inferred from runtime activations (quantized embeddings272pack weights); bf16 must be cast in MLX before NumPy.273274**Decisions.** (1) Promote the Tap-wrapper pattern into275src/modelmap/capture as the standard MLX capture layer — expA depends on276it. (2) The published cost table will carry cache-regime as a first-class277dimension (warm ≠ cold columns), and run #2's negative goes on the site's278negative-results ledger once the atlas pipeline exists. (3) expA first279run is now unblocked (capture layer + probes + stats all in place).280281---282283## 2026-08-12 06:45 EDT — expA run #1: validity gate FAILED → first atlas entry is a negative result284285**Question.** Are per-layer probe maps seed-stable but dataset-sensitive286(registered hypothesis), and does the harness pass its registered validity287gate (random-init twin selectivity < 0.05)?288289**Built first.** src/modelmap/capture/mlx_capture.py (Tap layer promoted290from expH run #3, + random_init_twin architecture null); versioned291promptsets v1 (6 files × 240 items, sha256 manifest, template-generated —292limitation declared in the manifest itself).293294**Result (48 s on M5 Max, 2,160 captures + 1,260 probes).**295- Primary hypothesis UNTESTABLE: ceiling everywhere (acc 1.000, seed SD296  0.0000, dataset shift 0.0000, all 28 layers, all 3 properties, both sets).297- **Validity gate FAILED, decisively: the random-init twin also reaches298  acc 1.00 at every layer** (twin max selectivity 0.56–0.88 vs the 0.05299  bar; mean real−twin selectivity within ±0.06). The probe map contains300  zero measurable trained-model signal — it reads the tokenizer +301  architecture prior. The shuffled-label control passed everywhere and302  would NOT have caught this; only the twin null did.303304**Interpretation.** Level 1 for the negative claim (5 seeds × 2 sets × 3305properties). The probing-pitfalls literature reproduced in our own hands,306caught by a pre-registered gate. Exactly what expA exists for: the noise307floor turned out to be a ceiling.308309**Published.** First atlas entry: atlas/qwen3-0.6b-4bit/probes/v1 with310map.json (all per-layer profiles incl. twin), provenance.json, mapcard.json311(negative_result=true, replication 1.00, Level 1) and confidence.md —312validated by tools/publish.py and exported to the site. The atlas begins313with a negative result, by design.314315**Decisions.** (1) Probe maps are publishable only as REAL−TWIN316differentials from now on — schema rule added to the map-card doctrine.317(2) Promptsets v1 are re-designated as the positive-control corpus; v2318must remove lexical separability (shared vocabulary, structure-borne319properties). (3) The twin null becomes a permanent fixture of every320observational map. Next: expA run #2 on v2 promptsets.321322---323324## 2026-08-12 07:15 EDT — expA run #2: first POSITIVE maps + first real noise floor325326**Question.** On token-balanced, structure-borne properties (v2 promptsets327with overlap certificates), does the twin fall to chance, and does the328trained model rise above it?329330**Result (88.6 s; results/expA_probe_reliability/20260812T063856Z).**331- Strict twin gate (<0.05) falsified AGAIN, with an instructive gradient:332  word_order twin acc 0.96 (tokenization statistics of scrambles);333  agreement twin 0.73; arith_valid twin 0.56. Even matched word sets leave334  subword-level class signal. The binary gate is retired; the twin becomes335  a mandatory reported baseline and the DIFFERENTIAL is the map.336- **agreement: first positive map** — real−twin selectivity > 0.10 on 25/28337  layers (max Δ +0.379; real acc 0.967 vs twin 0.729).338- **arith_valid: positive** — real 0.86/0.90 (mean/last) vs twin ~0.56;339  strongest in last-token reps (computation completes at "=").340- word_order: null-dominated (2/28 signal layers) — scrambled-vs-grammatical341  is NOT valid evidence of learned syntax at this granularity.342- **First real noise floor:** off ceiling, top-5-layer replication drops to343  ~0.54 with seed SD finally nonzero; dataset shift exceeds seed SD on only344  3–8/28 layers (registered 1/3 threshold NOT met — seed and dataset345  variance are comparable in this regime).346- Positive control: v1 lang_id_A still at ceiling — harness intact.347348**Published.** atlas/qwen3-0.6b-4bit/probes/v2 (Level 1, differential349claims only, per-property verdicts incl. the word_order null-domination350flag). Site charts generalized to render any probes atlas entry; home now351leads with the agreement differential map next to v1's negative.352353**Decisions.** (1) Twin gate: binary → reported baseline; differential354maps are the standard probe artifact. (2) Level-2 path for agreement/355arith_valid: LEACE erasure as second method, then ablation on top layers356(expC entry). (3) The ~0.54 replication number seeds expD's design (more357seeds, tighter CIs). (4) The agreement differential map is candidate_02's358quantization-drift target.359360---361362## 2026-08-12 07:45 EDT — expC run #1: the agreement map FAILS causal verification (survival ledger opens 0/1)363364**Question.** Do the top-5 layers of the agreement differential probe map365causally support agreement behavior under layer-skip ablation?366367**Result (5.7 s; pre-registered binary verdict).** FALSIFIED.368Baseline grammatical margin +4.63 (the 0.6B model robustly prefers correct369agreement — sanity holds). Skip damage: top-5 differential layers370(17,18,19,21,22) = +2.17, BELOW the random-5 mean (+3.14; p95 +4.85, 20371draws); bottom-5 differential (layers 0–4) = +4.99, the largest of all372conditions. **Where agreement information is most decodable above the373architecture null is not where the computation is causally load-bearing.**374The Hase-class dissociation (localization ≠ causal support), measured375end-to-end in our own pipeline within one day of standing it up.376377**Ledger.** The correlational→causal survival rate — the charter §8.3378metric — is now live: 0/1. The atlas entry probes/v2 records the failed379check in mapcard.interventions and confidence.md ("causal verification:380attempted and failed"); the map stays Level 1 and its layer ranking is381explicitly flagged as non-causal.382383**Caveats (registered in advance, both bit).** Layer-skip is coarse:384bottom-5 damage plausibly reflects GENERAL degradation (early layers break385everything), not agreement-specific structure. Held-out pairs 89 < planned386200 (dedup exhausted the combo pool).387388**Decisions / next.** expC run #2: (1) perplexity-normalized specificity389per skip condition; (2) direction-level intervention — LEACE-erase the390agreement direction in the forward pass at layer ℓ (a surgical test the391probe map can legitimately pass); (3) larger held-out bank; then the same392protocol on arith_valid. The pipeline now demonstrably runs the full393charter loop: register → measure → verify causally → publish either way.394395---396397## 2026-08-12 08:15 EDT — expC run #2: ledger 0/2 — and the causal scan finds what the probes missed398399**Results (192 pairs, baseline margin +4.24; both hypotheses registered).**400- **P1 CONFIRMED:** normalized by general (NLL) damage, run #1's bottom-5401  spike was unspecific (specificity 0.51 < random mean 1.44); top-5 (2.51)402  sits above the random mean but below p95 — the skip family never singles403  out the probe map's layers.404- **P2 FALSIFIED (ρ = −0.136, p = 0.76): survival ledger 0/2.** The probe405  map's layer ranking anti-correlates with the causal profile.406- **The discovery:** erasing the diff-of-means agreement direction at ANY407  single layer 2–15 destroys most of the behavior (specific damage up to408  +3.97/+4.24 at L12), with random-direction controls netted out; the409  late layers the probes ranked highest carry little, and L18/L22 erasure410  slightly HELPS (suppressive components). The load-bearing object is a411  low-dimensional DIRECTION constructed early — not a late "place" where412  information is merely readable.413414**Doctrine consequence.** Decodability-peak maps and causal-joint maps are415different map types and the atlas must never conflate them (this is charter416§2's distinction, now measured in-house at survival 0/2). The causal417profile is NOT yet publishable by our own rules (single direction estimate,418no seed replication) — expC run #3 will replicate it (direction from set B419+ bootstrap seeds); if stable it becomes the atlas's first INTERVENTIONS420map and probes/v2 gets a pointer to its causal counterpart.421422**Also noted.** Early residual streams are fragile to ANY rank-1 deletion423(random-direction damage +0.9…+2.5 at layers 2–9) — relevant to424quantization sensitivity (candidate_02) and to localvm's working-set425question (which layers tolerate compression).426427---428429## 2026-08-12 08:45 EDT — expC run #3: the publication gate refused the causal profile430431**Result (pre-registered rule).** Replication FAILED: mean pairwise432Spearman ρ = 0.495 (< 0.7 required; min 0.176); the early≥3×late band433claim fails on 2 of 5 direction sources. `make_interventions_mapcard.py`434refused the atlas entry (exit 1) — **the gate fired exactly as designed;435the atlas stays at two entries.**436437**The decomposition the refusal exposed.** Early band (layers 2–15):438tightly replicated across all five sources (+2.98…+3.34 specific damage439on a +4.24 baseline — erasing the estimated agreement direction anywhere440early reliably destroys ~70–79% of the behavior). Late band (20–27):441estimator noise (−0.44…+2.77), which retroactively explains run #2's442anti-correlation — the probe ranking lives exactly where the causal443profile is noise.444445**Decision.** Run #4 will pre-register the NARROWER claim (early-band mean446damage ≥ 2.5, fresh estimates + fresh behavioral bank; late band excluded447as unstable). If it passes, the interventions map is published as a BAND448claim, not a per-layer ranking, at Level 2. Meta-lesson for methodology.md:449map artifacts must declare their stable granularity — per-layer rankings450were too fine for this object; bands are the honest resolution.451452---453454## 2026-08-12 09:15 EDT — expC run #4: BAND claim passes — FIRST LEVEL-2 ATLAS ENTRY455456**Result (pre-registered; everything fresh).** CONFIRMED — early-band457(layers 2–15) mean specific damage per source: +3.35 / +3.23 / +3.30 /458+3.26 / +3.33 / +3.27 (min +3.228 vs bar 2.5; spread 0.12 across six459independent direction estimates; fresh bank baseline +4.45). Late band460again unstable (+0.27…+2.83) and carries no claim, as declared.461462**Published.** atlas/qwen3-0.6b-4bit/interventions/v1 — the project's463first causal, Level-2 map: *a single diff-of-means agreement direction,464erased at any one layer in the early band, removes ~73–75% of the model's465grammatical preference, replicated across six fresh estimators, netted466against random-direction damage.* The gate that refused run #3's467per-layer version passed run #4's band version — granularity discipline468enforced by tooling, start to finish. Site: the band map now leads the469home page and the atlas entry renders mean / worst-source / random-470reference curves.471472**The day's arc, as the atlas now shows it:** probes/v1 (negative —473architecture null wins), probes/v2 (Level 1 — differential signal,474per-property verdicts), interventions/v1 (Level 2 — causal band claim),475survival ledger 0/2 for correlational layer rankings, one publication-gate476refusal on record. Every claim at its measured level.477478**Next.** (1) Level-3 path: activation-addition steering (the direction479should *raise* the margin where it is weak); (2) same band protocol on480arith_valid; (3) candidate_02 entry: quantization drift of the band map481(FP16 vs Q8 vs Q4); (4) expG entry: does the band replicate on Qwen3-1.7B?482483---484485## 2026-08-12 09:45 EDT — expC run #5: L3 gate refused — layer 12 is a textbook handle anyway486487**Result (conjunctive criterion, pre-registered).** FAILED as a488conjunction; make_l3_mapcard.py refused v2 (exit 1); interventions/v1489stays Level 2.490- **L12: passes everything** — strictly monotone dose-response491  (+1.95/+3.00/+4.45/+5.82/+6.79 across −2σ…+2σ), halved at −2σ,492  specific (random |Δ| 0.45 vs bound 1.11). A textbook causal handle.493- L08: near-monotone, not specific at 2σ (random |Δ| 2.97).494- L04: overdose regime — ±2σ collapses the margin AND random directions495  are equally destructive (|Δ| 4.40). Run #2's rank-1 fragility,496  dose-resolved: early layers tolerate no ~2σ perturbation of any kind.497498**Reading.** Necessity everywhere in the band (Level-2 entry) coexists499with *controllability only where the layer tolerates perturbation*. The500dose scale (σ from mean-pooled stats) is a declared confound at early501layers. Gate record now: refusal (run #3, per-layer) → pass (run #4,502band, L2) → refusal (run #5, L3 conjunction). The atlas has never503received a claim its evidence didn't carry.504505**Next.** Run #6 candidates (to register before running): layer-local506dose calibration + handle claim on the perturbation-tolerant sub-band;507or the minimal L12 single-layer L3 claim with fresh direction estimates.508Parallel tracks unchanged: arith_valid band, candidate_02 quantization509drift, expG cross-model.510511---512513## 2026-08-12 10:15 EDT — expC run #6: L12 handle fails replication — Level 3 abandoned, program closed514515**Result (pre-registered kill rule).** FALSIFIED on both prongs: only 1/4516fresh sources monotone (the POSITIVE dose arm is unstable); specificity517did not replicate (random-direction |Δ| 3.36 vs bound 1.14 on fresh bank +518fresh seeds — run #5's 0.45 at L12 was sampling luck of 3 draws). Halving519at −2σ held 4/4: the negative (erasure-like) arm is robust, consistent520with the Level-2 entry. **Verdict: the agreement direction is NECESSARY521but NOT a reliable additive handle. Level 3 abandoned for this object;522steering program closed; interventions/v1 (Level 2) stands as the final523causal claim for agreement at 0.6B.**524525**The gate earned its keep a third time:** had run #5's L12 observation526been published without fresh re-registration, the atlas would now hold a527false Level-3 claim. Final arc record: refuse (r3) → pass L2 (r4) →528refuse (r5) → refuse (r6).529530**Methodology rule adopted:** specificity controls need ≥10 random-531direction draws with a percentile bound (3 was too few) — goes into532methodology.md and the map-card doctrine.533534**Proceeding tracks (from the Level-2 base):** arith_valid band protocol;535candidate_02 quantization drift of the band map; expG cross-model536replication on Qwen3-1.7B.537