# project: modelmap document: Phase 1 notes — §4.8 Statistics and epistemology of interpretability author: Simon-Pierre Boucher contact: contact@spboucher.ai website: https://modelmap.io created: 2026-08-12 status: draft

# §4.8 — Statistics and epistemology of interpretability

# B1. The BERT interpretability illusion (dataset sensitivity)

• Key paper: Bolukbasi et al. 2021, https://arxiv.org/abs/2104.07143 — the same neuron looks monosemantic on one dataset and encodes a different "concept" on another; top-activating-example analysis systematically over-reads.
• Consequence (already in charter): every unit-level claim needs ≥2 prompt sets; atlas entries report per-dataset agreement.
• Status: Established.

# B2. Subspace-patching illusions — Makelov vs Wu (unresolved)

• Key papers: Makelov, Lange, Nanda 2023, https://arxiv.org/abs/2311.17030 (subspace patching can activate dormant parallel pathways — interventional success ≠ correct localization) vs Wu et al. reply 2024, https://arxiv.org/abs/2401.12631 (the illusion criterion also rejects intuitively correct explanations).
• Common ground: interchange-intervention success alone under-determines mechanism. Rule: Level-3 labels record which intervention family and which basis; DAS-style optimized subspaces get a distinct flag.
• Status: Contested (productive open debate).

# B3. Illusions from simplified proxies (SVD, clustering, low-rank views)

• Key paper: Friedman et al., ICML 2024, https://arxiv.org/abs/2312.03656 — simplifications matching the model in-distribution diverge OOD; the proxy captures ID behavior, not mechanism.
• Direct warning for expE: SVD/effective-rank maps are hypotheses, not mechanisms — every geometry map gets an OOD promptset check.
• Status: Established.

# B4. Methodological sensitivity of activation patching

• Key papers: Zhang & Nanda, ICLR 2024, https://arxiv.org/abs/2309.16042 ; Heimersheim & Nanda 2024, https://arxiv.org/abs/2404.15255
• Substance: results flip with corruption type (Gaussian vs symmetric counterfactual), metric (prob vs logit-diff vs KL), window, position. ROME-style Gaussian corruption inflates mid-layer MLP localization. The closest thing to a pre-registration standard: fix corruption + metric before scanning — methodology.md pins these per map type.
• Status: Established.

# B5. Faithfulness vs plausibility

• Key paper: Jacovi & Goldberg, ACL 2020, https://arxiv.org/abs/2004.03685 — the vocabulary underlying modelmap's confidence taxonomy; plausibility (human appeal) must never borrow causal authority. Visualization design (§4.10) must respect this firewall.
• Status: Established (canonical).

# B6. Probing controls, selectivity

• Key papers: Hewitt & Liang, EMNLP 2019, https://arxiv.org/abs/1909.03368 ; Belinkov, CL 2022, https://arxiv.org/abs/2102.12452
• Substance: the mandatory-controls doctrine expA implements — shuffled-label nulls, capacity-matched probes, multi-dataset replication, selectivity over raw accuracy.
• Status: Established.

# B7. Seed sensitivity of SAEs — the noise-floor number

• Key paper: Paulo & Belrose 2025, https://arxiv.org/abs/2501.16615 — SAEs differing only in seed share as little as ~30% of features (131k latents, Llama-3-8B); TopK more seed-dependent than ReLU+L1.
• Consequence: cross-seed overlap is the noise floor that cross-model universality claims must beat. Combined with 1–5% neuron universality (Gurnee), it makes modelmap's replication-rate metric first-class. Replicating at 0.5–1B scale is feasible and would be a strong early result.
• Status: Established (multi-LLM, multi-architecture).

# B8. Multiple comparisons — interpretability as statistical inference

• Key sources: The Dead Salmons of AI Interpretability (2025), https://arxiv.org/abs/2512.18792 — attribution, probing, SAEs and causal analyses yield plausible "explanations" of randomly initialized networks; reframe interpretability as statistical inference (uncertainty, explicit alternatives, multiplicity control). Historical anchor: Bennett et al. 2009, https://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf
• Consequence: random-init baselines non-negotiable (charter §8.4); add per-map FDR control to src/modelmap/stats.
• Status: Emerging position; underlying statistics established.

# B9. Ground-truth benchmarks

• Tracr (compiled transformers, exact circuits): https://arxiv.org/abs/2301.05062 , code https://github.com/google-deepmind/tracr (JAX, CPU-runnable on Mac)
• InterpBench (17 semi-synthetic transformers via Strict IIT, NeurIPS 2024): https://arxiv.org/abs/2407.14494
• RAVEL (attribute disentanglement, intervention-based): https://arxiv.org/abs/2402.17700
• MIB — Mechanistic Interpretability Benchmark (ICML 2025): https://arxiv.org/abs/2504.13151 — two tracks (circuit localization; causal-variable localization), 4 tasks × 5 models; attribution/mask-optimization best for circuits; supervised DAS beats SAEs on causal variables; BlackboxNLP 2025 shared task https://arxiv.org/html/2510.06811v1
• Consequence: validate every modelmap pipeline on Tracr/InterpBench first (tiny, CPU/MPS-feasible), report recovery rates, then apply to real models. MIB pairs are within Mac budget.
• Status: Established, rapidly standardizing.

# B10. Hypothesis testing for circuits

• Key papers: Shi et al., NeurIPS 2024, https://arxiv.org/abs/2410.13032 (code https://github.com/blei-lab/circuitry) — formal equivalence/independence/minimality tests; published circuits pass only partially, Tracr circuits pass fully. Hanna et al. 2024, https://arxiv.org/abs/2403.17806 (faithfulness > overlap).
• Consequence: run circuitry-style tests on every promoted circuit; report pass/fail per criterion in confidence.md — a ready-made harness for the causal confirmation rate.
• Status: Established methodology.

# B11. Causal abstraction — the formal framework and its limit

• Key papers: Geiger et al., JMLR 2025, https://arxiv.org/abs/2301.04709 — unifies patching/scrubbing/circuits/SAEs/steering as causal abstraction; Non-Linear Representation Dilemma (2025): https://arxiv.org/abs/2507.08802 — with arbitrary nonlinear featurizers, any model "abstracts" any algorithm (accuracy ~1 even for random networks); causal abstraction is only meaningful jointly with a constrained featurizer class.
• Consequence: confidence.md records the featurizer class (linear / orthogonal / DAS-optimized) for every causal claim.
• Status: Framework established; sufficiency contested.

# B12. Calls for rigor (2023–2026)

• Saphra & Wiegreffe, Mechanistic? (BlackboxNLP 2024): https://arxiv.org/abs/2410.09087 — four senses of "mechanistic"; state which one is used (modelmap: the narrow causal one).
• Casper, The Engineer's Interpretability Sequence (2023): https://www.alignmentforum.org/s/a6ne2ve5uturEEQK7
• Sharkey et al., Open Problems in Mechanistic Interpretability (2025): https://arxiv.org/abs/2501.16496 — lists validation/method-evaluation gaps modelmap targets.
• Mechanistic Interpretability Needs Philosophy (2025): https://arxiv.org/abs/2506.18852 ; Make Mechanistic Interpretability Auditable (ACL 2026): https://arxiv.org/pdf/2606.00033 — auditability guidelines are close cousins of modelmap's provenance/confidence schema.
• Status: Converging consensus: ground-truth benchmarks + statistical controls + causal verification — exactly the charter's bar.

# Synthesis for modelmap

1. Noise floors first (expA): selectivity, ≥2 datasets, ≥3–5 seeds, random-init baselines, FDR control.
2. Pipeline validation on Tracr/InterpBench before any real-model map.
3. confidence.md schema additions: intervention family, basis/featurizer class, ablation-scheme curves, per-dataset agreement.
4. The 30%-SAE-seed-overlap and 1–5%-neuron-universality numbers are the published calibration points our replication-rate metric extends.