Phase 1 sweep #1: ten-area literature survey (§4.1-4.10)

- research/notes/: 10 theme notes with verified sources, Apple Silicon
  status, failure modes, epistemic status per technique
- research/bibliography.md: ~200 sources with URLs + access dates
- research/LOG.md: sweep results — quantization gap confirmed (~5 shallow
  papers), weight-only pre-screen gap open, noise floors (30% SAE seed
  overlap, 1-5% neuron universality), atlas provenance gap verified

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Simon-Pierre Boucher committed 5 h ago (Aug 12, 2026) parent 6d7f0fa

Showing 12 changed files with +1,078 and −6

modified research/LOG.md +55 −0

		`@@ -33,3 +33,58 @@ ten areas of charter §4 (observational probing, causal methods, feature`
33	33	decomposition, circuits, weight-space cartography, knowledge editing,
34	34	cross-model comparison, statistics/epistemology, systems, atlas design).
35	35	No mapping code is written before the state-of-the-art map (Phase 2) exists.
	36	+
	37	+---
	38	+
	39	+## 2026-08-12 02:00 EDT — Phase 1 sweep #1: ten-area literature survey
	40	+
	41	+Question. What does the field already know, per charter §4.1–4.10, and
	42	+which of modelmap's core bets survive first contact with the literature?
	43	+
	44	+Method. Five parallel web-verified survey passes (two charter areas each);
	45	+every source logged with URL + access date in `research/bibliography.md`
	46	+(~200 entries); per-area reading notes written to `research/notes/4.X_*.md`.
	47	+This is sweep #1 of Phase 1 — deeper follow-ups (reading full papers behind
	48	+the highest-stakes claims, backward/forward citation chasing) remain.
	49	+
	50	+Key results (detail in the notes):
	51	+
	52	+1. The quantization gap is real (Q4/expF bet confirmed). ~5 shallow papers
	53	+ exist on quantization × interpretability; none does probe-drift, circuit
	54	+ survival, Q2/Q3 internals, or GGUF-runtime introspection. Open territory.
	55	+2. The weight-only pre-screen gap is real but partially anticipated (expE).
	56	+ WeightWatcher (behavioral quality from spectra), Super Weight (weight-visible
	57	+ outliers → massive activations), Watch-the-Weights (2025) are adjacent; no
	58	+ systematic weight-only-statistics → activation-level-properties study exists.
	59	+3. Published noise floors to build on: only ~30% of SAE latents replicate
	60	+ across seeds (Paulo & Belrose 2025); only 1–5% of neurons are universal
	61	+ across seeds (Gurnee 2024); global platonic-convergence metrics deflate
	62	+ under calibration (2026). These calibrate expA/expD and our
	63	+ replication-rate metric.
	64	+4. Central methodological caution: localization ≠ editability (Hase 2023);
	65	+ faithfulness scores are not robust to ablation-scheme choice (Miller 2024);
	66	+ subspace patching can hit dormant pathways (Makelov vs Wu, unresolved);
	67	+ random-init baselines + FDR control are mandatory (Dead Salmons 2025).
	68	+5. Apple Silicon tooling is a desert (Q4 confirmed): TransformerLens MPS is
	69	+ opt-in/unstable; editing repos are CUDA-first; only mlxterp (tiny, 2025)
	70	+ targets MLX; llama.cpp eval-callback enables quantized-runtime capture but
	71	+ nothing wraps it; no macOS activation-store or hook-overhead benchmarks
	72	+ exist anywhere.
	73	+6. Atlas prior-art check: no existing artifact host combines versioned
	74	+ entries + regeneration recipes + confidence levels + negative results
	75	+ (verified across Neuronpedia, Neuroscope, Transluce, circuit-tracer,
	76	+ Gemma Scope). "Model atlas" name collision with Horwitz 2025 (inter-model
	77	+ graph, not internals) noted for novelty_check.md.
	78	+
	79	+Interpretation (confidence: this is a literature map, not a result).
	80	+The charter's bets (quantization maps, weight-only pre-screens, Mac-local
	81	+tooling, provenance-first atlas) all survive; several have adjacent 2025–2026
	82	+work converging on them, so speed matters. The field's own rigor wave
	83	+(MIB, Tracr/InterpBench validation, statistical-inference framing) supplies
	84	+ready-made harnesses for our confidence taxonomy.
	85	+
	86	+Decision. Proceed to Phase 2 (`research/state_of_the_art.md`): organize
	87	+these techniques into the charter §5 table with epistemic status per
	88	+technique, then Phase 3 gap generation (≥20 candidates). Validation-first
	89	+rule adopted: every future pipeline is validated on Tracr/InterpBench-style
	90	+ground truth before touching a real model.

modified research/bibliography.md +303 −6

		`@@ -5,16 +5,313 @@ author: Simon-Pierre Boucher`
5	5	contact: contact@spboucher.ai
6	6	website: https://modelmap.io
7	7	created: 2026-08-12
	8	+modified: 2026-08-12
8	9	status: draft
9	10	---
10	11
11	12	# Bibliography
12	13
13		−Every consulted source, with URL and access date. Populated during Phase 1
14		−(charter §4). Entries use the form:
	14	+Every consulted source, with URL and access date. Populated during Phase 1 (charter §4).
	15	+Entries: `- Authors — Title (year). URL — accessed YYYY-MM-DD. [theme tags]`
	16	+Duplicates across themes are listed once, under their primary theme. Entries flagged
	17	+(spot-check) carry stable canonical IDs cited from memory by the survey pass and should
	18	+be re-verified before citation in a published document.
15	19
16		−```text
17		−- <Authors> — <Title> (<year>). <URL> — accessed <YYYY-MM-DD>. [themes]
18		−```
	20	+## §4.1 — Observational probing
19	21
20		−(No sources logged yet — Phase 1 has not started.)
	22	+- Alain & Bengio — Understanding intermediate layers using linear classifier probes (2016). https://arxiv.org/abs/1610.01644 — accessed 2026-08-12. [probing]
	23	+- Belinkov — Probing Classifiers: Promises, Shortcomings, and Advances (2021/2022). https://arxiv.org/abs/2102.12452 ; https://aclanthology.org/2022.cl-1.7/ — accessed 2026-08-12. [probing, survey] (spot-check)
	24	+- Hewitt & Liang — Designing and Interpreting Probes with Control Tasks (2019). https://arxiv.org/abs/1909.03368 — accessed 2026-08-12. [probing, controls] (spot-check)
	25	+- Voita & Titov — Information-Theoretic Probing with Minimum Description Length (2020). https://arxiv.org/pdf/2003.12298 — accessed 2026-08-12. [probing, controls]
	26	+- nostalgebraist — interpreting GPT: the logit lens (2020). https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens — accessed 2026-08-12. [logit-lens]
	27	+- Belrose et al. — Eliciting Latent Predictions from Transformers with the Tuned Lens (2023). https://arxiv.org/abs/2303.08112 ; https://github.com/AlignmentResearch/tuned-lens — accessed 2026-08-12. [tuned-lens]
	28	+- Chuang et al. — DoLa: Decoding by Contrasting Layers (2023). https://arxiv.org/abs/2309.03883 — accessed 2026-08-12. [early-decoding]
	29	+- Pal et al. — Future Lens: Anticipating Subsequent Tokens from a Single Hidden State (2023). https://www.alphaxiv.org/overview/2311.04897v1 ; https://future.baulab.info/ — accessed 2026-08-12. [lens-successors]
	30	+- Ghandeharioun et al. — Patchscopes (2024). https://arxiv.org/abs/2401.06102 ; https://pair-code.github.io/interpretability/patchscopes/ — accessed 2026-08-12. [lens-successors]
	31	+- SimLens for Early Exit (2025). https://arxiv.org/pdf/2507.17618 — accessed 2026-08-12. [lens-successors, 2025]
	32	+- Zou et al. — Representation Engineering: A Top-Down Approach to AI Transparency (2023). https://arxiv.org/abs/2310.01405 ; https://github.com/andyzoujm/representation-engineering — accessed 2026-08-12. [RepE]
	33	+- Representation Engineering: Survey and Research Challenges (2025). https://arxiv.org/pdf/2502.17601 — accessed 2026-08-12. [RepE, critique]
	34	+- Vogel — repeng (control-vector library). https://github.com/vgel/repeng ; llama.cpp PR https://github.com/ggml-org/llama.cpp/pull/5970 — accessed 2026-08-12. [tooling, steering, GGUF]
	35	+- Ravfogel et al. — Null It Out: INLP (ACL 2020). https://aclanthology.org/2020.acl-main.647.pdf — accessed 2026-08-12. [concept-erasure]
	36	+- Elazar et al. — Amnesic Probing (TACL 2021). https://aclanthology.org/2021.tacl-1.10/ — accessed 2026-08-12. [concept-erasure]
	37	+- Belrose et al. — LEACE: Perfect linear concept erasure in closed form (2023). https://arxiv.org/abs/2306.03819 ; https://github.com/EleutherAI/concept-erasure ; https://blog.eleuther.ai/oracle-leace/ — accessed 2026-08-12. [concept-erasure]
	38	+- Nonlinear Concept Erasure: a Density Matching Approach (2025). https://arxiv.org/abs/2507.12341 — accessed 2026-08-12. [concept-erasure, 2025]
	39	+- Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE (2025). https://arxiv.org/html/2506.11673 — accessed 2026-08-12. [concept-erasure, critique]
	40	+
	41	+## §4.2 — Causal / interventional methods
	42	+
	43	+- Vig et al. — Investigating Gender Bias in LMs Using Causal Mediation Analysis (NeurIPS 2020). https://papers.nips.cc/paper/2020/hash/92650b2e92217715fe312e6fa7b90d82-Abstract.html — accessed 2026-08-12. [patching-origins]
	44	+- Zhang & Nanda — Towards Best Practices of Activation Patching (ICLR 2024). https://arxiv.org/abs/2309.16042 — accessed 2026-08-12. [patching, methodology]
	45	+- Heimersheim & Nanda — How to use and interpret activation patching (2024). https://arxiv.org/pdf/2404.15255 — accessed 2026-08-12. [patching, methodology]
	46	+- Nanda — How to Think About Activation Patching. https://www.alignmentforum.org/posts/xh85KbTFhbCz7taD4/how-to-think-about-activation-patching — accessed 2026-08-12. [patching]
	47	+- Goldowsky-Dill et al. — Localizing Model Behavior with Path Patching (2023). https://arxiv.org/abs/2304.05969 — accessed 2026-08-12. [path-patching]
	48	+- Nanda — Attribution Patching: Activation Patching At Industrial Scale (2023). https://www.neelnanda.io/mechanistic-interpretability/attribution-patching — accessed 2026-08-12. [attribution-patching]
	49	+- Syed, Rager, Conmy — Attribution Patching Outperforms Automated Circuit Discovery (BlackboxNLP 2024). https://aclanthology.org/2024.blackboxnlp-1.25.pdf ; https://arxiv.org/abs/2310.10348 — accessed 2026-08-12. [EAP]
	50	+- Kramár, Lieberum, Shah, Nanda — AtP*: efficient localization of LLM behaviour (2024). https://arxiv.org/abs/2403.00745 ; https://github.com/koayon/atp_star — accessed 2026-08-12. [attribution-patching]
	51	+- When Attribution Patching Lies (2026). https://arxiv.org/pdf/2606.09899 — accessed 2026-08-12. [attribution-patching, critique, 2026]
	52	+- RelP: Relevance Patching (2025). https://arxiv.org/html/2508.21258v1 — accessed 2026-08-12. [attribution-patching, 2025]
	53	+- Position-aware Automatic Circuit Discovery (2025). https://arxiv.org/pdf/2502.04577 — accessed 2026-08-12. [circuits, 2025]
	54	+- APP: Accelerated Path Patching (2025). https://arxiv.org/html/2511.05442 — accessed 2026-08-12. [path-patching, 2025]
	55	+- Chan et al. (Redwood) — Causal Scrubbing (2022). https://www.alignmentforum.org/posts/JvZhhzycHu2Yd57RN/causal-scrubbing-a-method-for-rigorously-testing ; appendix https://www.alignmentforum.org/posts/kcZZAsEjwrbczxN2i/causal-scrubbing-appendix — accessed 2026-08-12. [causal-scrubbing]
	56	+- Practical Pitfalls of Causal Scrubbing (2023). https://www.alignmentforum.org/posts/DFarDnQjMnjsKvW8s/practical-pitfalls-of-causal-scrubbing — accessed 2026-08-12. [causal-scrubbing, critique]
	57	+- Geiger et al. — Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations (DAS, CLeaR 2024). https://arxiv.org/abs/2303.02536 — accessed 2026-08-12. [DAS, IIA]
	58	+- Wu et al. — Interpretability at Scale: Boundless DAS in Alpaca (2023). https://arxiv.org/abs/2305.08809 — accessed 2026-08-12. [DAS]
	59	+- Wu et al. — pyvene (NAACL 2024). https://arxiv.org/abs/2403.07809 ; https://github.com/stanfordnlp/pyvene — accessed 2026-08-12. [tooling, DAS]
	60	+- Turner et al. — Steering Language Models With Activation Engineering (ActAdd, 2023). https://arxiv.org/abs/2308.10248 — accessed 2026-08-12. [steering]
	61	+- Rimsky et al. — Steering Llama 2 via Contrastive Activation Addition (ACL 2024). https://aclanthology.org/2024.acl-long.828/ ; https://arxiv.org/abs/2312.06681 — accessed 2026-08-12. [steering]
	62	+- Li et al. — Inference-Time Intervention (NeurIPS 2023). https://arxiv.org/pdf/2306.03341 — accessed 2026-08-12. [steering]
	63	+- Tan et al. — Analysing the Generalisation and Reliability of Steering Vectors (NeurIPS 2024). https://arxiv.org/abs/2407.12404 — accessed 2026-08-12. [steering, critique]
	64	+- Steering off Course: Reliability Challenges in Steering LMs (ACL 2025). https://aclanthology.org/2025.acl-long.974/ ; https://arxiv.org/html/2504.04635v1 — accessed 2026-08-12. [steering, critique, 2025]
	65	+- Understanding Unreliability of Steering Vectors (2026). https://arxiv.org/pdf/2602.17881 — accessed 2026-08-12. [steering, 2026]
	66	+- On the Non-Identifiability of Steering Vectors (2026). https://arxiv.org/pdf/2602.06801 — accessed 2026-08-12. [steering, 2026]
	67	+- The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching (2026). https://arxiv.org/html/2606.27510v1 — accessed 2026-08-12. [patching, critique, 2026]
	68	+- Bereska & Gavves — Mechanistic Interpretability for AI Safety: A Review (2024). https://arxiv.org/pdf/2404.14082 — accessed 2026-08-12. [survey]
	69	+- Fiotto-Kaufman et al. — NNsight and NDIF (ICLR 2025). https://arxiv.org/abs/2407.14561 ; https://github.com/ndif-team/nnsight ; https://ndif.us/ — accessed 2026-08-12. [tooling, remote-execution]
	70	+- Bau — baukit. https://github.com/davidbau/baukit — accessed 2026-08-12. [tooling]
	71	+- TransformerLens — Getting Started (MPS opt-in note). https://transformerlensorg.github.io/TransformerLens/content/getting_started.html ; https://github.com/TransformerLensOrg/TransformerLens — accessed 2026-08-12. [tooling, Apple-Silicon]
	72	+- IOI circuit in Mamba (2024). https://arxiv.org/html/2407.14008v1 — accessed 2026-08-12. [circuits, replication]
	73	+
	74	+## §4.3 — Feature decomposition
	75	+
	76	+- Elhage et al. — Toy Models of Superposition (2022). https://transformer-circuits.pub/2022/toy_model/index.html — accessed 2026-08-12. [superposition]
	77	+- Yun, Chen, Olshausen, LeCun — Transformer visualization via dictionary learning (2021). https://arxiv.org/abs/2103.15949 — accessed 2026-08-12. [dictionary-learning]
	78	+- Sharkey, Braun, Millidge — Interim report: taking features out of superposition with SAEs (2022). https://www.lesswrong.com/posts/z6QQJbtpkEAX3Aojj/interim-research-report-taking-features-out-of-superposition — accessed 2026-08-12. [SAE-precursor]
	79	+- Cunningham et al. — Sparse Autoencoders Find Highly Interpretable Features in LMs (2023). https://arxiv.org/abs/2309.08600 — accessed 2026-08-12. [SAE]
	80	+- Bricken et al. — Towards Monosemanticity (2023). https://transformer-circuits.pub/2023/monosemantic-features/index.html — accessed 2026-08-12. [SAE, feature-splitting]
	81	+- Templeton et al. — Scaling Monosemanticity (2024). https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html — accessed 2026-08-12. [SAE-scaling, pipeline, feature-browser]
	82	+- Rajamanoharan et al. — Gated Sparse Autoencoders (2024). https://arxiv.org/abs/2404.16014 — accessed 2026-08-12. [SAE-variant]
	83	+- Gao et al. — Scaling and evaluating sparse autoencoders (TopK, 2024). https://arxiv.org/abs/2406.04093 — accessed 2026-08-12. [SAE-variant]
	84	+- Rajamanoharan et al. — JumpReLU Sparse Autoencoders (2024). https://arxiv.org/abs/2407.14435 — accessed 2026-08-12. [SAE-variant]
	85	+- Bussmann, Leask, Nanda — BatchTopK Sparse Autoencoders (2024). https://arxiv.org/abs/2412.06410 — accessed 2026-08-12. [SAE-variant]
	86	+- Bussmann et al. — Matryoshka Sparse Autoencoders (2025). https://arxiv.org/abs/2503.17547 — accessed 2026-08-12. [SAE-variant, absorption]
	87	+- Lindsey et al. — Sparse Crosscoders for Cross-Layer Features and Model Diffing (2024). https://transformer-circuits.pub/2024/crosscoders/index.html — accessed 2026-08-12. [crosscoders]
	88	+- Dunefsky, Chlenski, Nanda — Transcoders Find Interpretable LLM Feature Circuits (2024). https://arxiv.org/abs/2406.11944 — accessed 2026-08-12. [transcoders]
	89	+- Paulo, Shabalin, Belrose — Transcoders Beat Sparse Autoencoders for Interpretability (2025). https://arxiv.org/abs/2501.18823 — accessed 2026-08-12. [skip-transcoders]
	90	+- Chanin et al. — A is for Absorption (2024). https://arxiv.org/abs/2409.14507 — accessed 2026-08-12. [feature-absorption]
	91	+- Engels et al. — Not All Language Model Features Are Linear (2024). https://arxiv.org/abs/2405.14860 — accessed 2026-08-12. [feature-geometry]
	92	+- Bills et al. — Language models can explain neurons in language models (2023). https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html — accessed 2026-08-12. [auto-interp]
	93	+- Paulo et al. — Automatically Interpreting Millions of Features (2024). https://arxiv.org/abs/2410.13928 — accessed 2026-08-12. [auto-interp]
	94	+- Huang et al. — Rigorously Assessing Natural Language Explanations of Neurons (2023). https://arxiv.org/abs/2309.10312 — accessed 2026-08-12. [auto-interp-critique]
	95	+- Karvonen et al. — SAEBench (2025). https://arxiv.org/abs/2503.09532 ; https://www.neuronpedia.org/sae-bench/info ; https://github.com/adamkarvonen/SAEBench — accessed 2026-08-12. [SAE-evaluation]
	96	+- Chanin et al. — Are Sparse Autoencoder Benchmarks Reliable? (2026). https://arxiv.org/abs/2605.18229 — accessed 2026-08-12. [evaluation-critique, 2026]
	97	+- Smith et al. (GDM) — Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (2025). https://deepmindsafetyresearch.medium.com/negative-results-for-sparse-autoencoders-on-downstream-tasks-and-deprioritising-sae-research-6cadcfc125b9 — accessed 2026-08-12. [SAE-debate]
	98	+- Kantamneni et al. — Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (2025). https://arxiv.org/abs/2502.16681 — accessed 2026-08-12. [SAE-debate]
	99	+- Wu et al. — AxBench (2025). https://arxiv.org/abs/2501.17148 — accessed 2026-08-12. [steering, SAE-debate]
	100	+- Kissane et al. — SAEs are highly dataset dependent (2024). https://www.alignmentforum.org/posts/rtp6n7Z23uJpEH7od/saes-are-highly-dataset-dependent-a-case-study-on-the — accessed 2026-08-12. [dataset-dependence]
	101	+- Paulo, Belrose — SAEs Trained on the Same Data Learn Different Features (2025). https://arxiv.org/abs/2501.16615 — accessed 2026-08-12. [seed-dependence, replication]
	102	+- Movva et al. — Use SAEs to Discover Unknown Concepts, Not to Act on Known Concepts (2025). https://arxiv.org/abs/2506.23845 — accessed 2026-08-12. [SAE-debate]
	103	+- Lieberum et al. — Gemma Scope (2024). https://arxiv.org/abs/2408.05147 ; https://huggingface.co/google/gemma-scope — accessed 2026-08-12. [open-SAEs, artifacts]
	104	+- He et al. — Llama Scope (2024). https://arxiv.org/abs/2410.20526 — accessed 2026-08-12 (spot-check). [open-SAEs]
	105	+- Huang et al. — RAVEL (ACL 2024). https://arxiv.org/abs/2402.17700 ; https://github.com/explanare/ravel — accessed 2026-08-12. [evaluation, causal]
	106	+- CE-Bench (2025). https://arxiv.org/abs/2509.00691 — accessed 2026-08-12. [SAE-evaluation, 2025]
	107	+- SynthSAEBench (2026). https://arxiv.org/abs/2602.14687 — accessed 2026-08-12. [SAE-evaluation, synthetic, 2026]
	108	+- SAELens (Bloom, Tigges, Chanin et al.). https://github.com/jbloomAus/SAELens ; https://github.com/decoderesearch/SAELens — accessed 2026-08-12. [tooling, MPS]
	109	+- dictionary_learning (Marks, Mueller). https://github.com/saprmarks/dictionary_learning — accessed 2026-08-12. [tooling]
	110	+- EleutherAI — sparsify. https://github.com/EleutherAI/sparsify — accessed 2026-08-12. [tooling, streaming]
	111	+- EleutherAI — delphi (auto-interp). https://github.com/EleutherAI/delphi — accessed 2026-08-12. [tooling]
	112	+- EleutherAI — clt-training. https://github.com/EleutherAI/clt-training — accessed 2026-08-12. [tooling, CLT]
	113	+- ckkissane — crosscoder-model-diff-replication. https://github.com/ckkissane/crosscoder-model-diff-replication — accessed 2026-08-12. [tooling]
	114	+
	115	+## §4.4 — Circuits and computational graphs
	116	+
	117	+- Olsson et al. — In-context Learning and Induction Heads (2022). https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html ; https://arxiv.org/abs/2209.11895 — accessed 2026-08-12. [induction-heads, universality]
	118	+- Wang et al. — Interpretability in the Wild: IOI circuit (2022). https://arxiv.org/abs/2211.00593 — accessed 2026-08-12. [IOI, circuits, path-patching]
	119	+- McDougall et al. — Copy Suppression (2023). https://arxiv.org/abs/2310.04625 — accessed 2026-08-12. [head-taxonomy]
	120	+- Gould et al. — Successor Heads (2023). https://arxiv.org/abs/2312.09230 — accessed 2026-08-12. [head-taxonomy, universality]
	121	+- McGrath et al. — The Hydra Effect (2023). https://arxiv.org/abs/2307.15771 — accessed 2026-08-12. [self-repair, ablation-critique]
	122	+- Geva et al. — Transformer Feed-Forward Layers Are Key-Value Memories (2021). https://arxiv.org/abs/2012.14913 — accessed 2026-08-12. [MLP-KV]
	123	+- Geva et al. — FFN Layers Build Predictions by Promoting Concepts in Vocabulary Space (2022). https://arxiv.org/abs/2203.14680 — accessed 2026-08-12. [MLP-KV]
	124	+- Ameisen, Lindsey et al. — Circuit Tracing: Revealing Computational Graphs (2025). https://transformer-circuits.pub/2025/attribution-graphs/methods.html — accessed 2026-08-12. [attribution-graphs]
	125	+- Lindsey et al. — On the Biology of a Large Language Model (2025). https://transformer-circuits.pub/2025/attribution-graphs/biology.html — accessed 2026-08-12. [attribution-graphs]
	126	+- Anthropic — Tracing Attention Computation Through Feature Interactions (2025). https://transformer-circuits.pub/2025/attention-qk/index.html — accessed 2026-08-12. [QK-attribution]
	127	+- Anthropic — Open-sourcing circuit-tracing tools (2025). https://www.anthropic.com/research/open-source-circuit-tracing — accessed 2026-08-12. [circuit-tracer]
	128	+- Hanna, Piotrowski et al. — circuit-tracer (2025). https://github.com/safety-research/circuit-tracer ; https://github.com/decoderesearch/circuit-tracer ; https://aclanthology.org/2025.blackboxnlp-1.14.pdf — accessed 2026-08-12. [tooling]
	129	+- Conmy et al. — Towards Automated Circuit Discovery (ACDC, NeurIPS 2023). https://arxiv.org/abs/2304.14997 ; https://github.com/ArthurConmy/Automatic-Circuit-Discovery — accessed 2026-08-12. [circuit-discovery]
	130	+- Hanna, Pezzelle, Belinkov — Have Faith in Faithfulness (EAP-IG, COLM 2024). https://arxiv.org/abs/2403.17806 ; https://github.com/hannamw/EAP-IG — accessed 2026-08-12. [EAP-IG, faithfulness]
	131	+- Bhaskar et al. — Finding Transformer Circuits with Edge Pruning (NeurIPS 2024). https://arxiv.org/abs/2406.16778 ; https://github.com/princeton-nlp/Edge-Pruning — accessed 2026-08-12. [edge-pruning]
	132	+- Miller, Chughtai, Saunders — Transformer Circuit Faithfulness Metrics are not Robust (2024). https://arxiv.org/abs/2407.08734 — accessed 2026-08-12. [faithfulness-critique]
	133	+- Shi et al. — Hypothesis Testing the Circuit Hypothesis (NeurIPS 2024). https://arxiv.org/abs/2410.13032 ; https://github.com/blei-lab/circuitry — accessed 2026-08-12. [faithfulness, statistics]
	134	+- Marks et al. — Sparse Feature Circuits (ICLR 2025). https://arxiv.org/abs/2403.19647 ; https://github.com/saprmarks/feature-circuits — accessed 2026-08-12. [feature-circuits, SHIFT]
	135	+- Gao, Rajaram et al. (OpenAI) — Weight-sparse transformers have interpretable circuits (2025). https://arxiv.org/abs/2511.13653 ; https://github.com/openai/circuit_sparsity — accessed 2026-08-12. [weight-sparsity]
	136	+- Interpretable individual parameters follow-up (2026). https://arxiv.org/abs/2607.02964 — accessed 2026-08-12. [weight-sparsity, 2026]
	137	+- ADAG — Automatically Describing Attribution Graphs (2026). https://arxiv.org/abs/2604.07615 — accessed 2026-08-12. [automation, 2026]
	138	+- Circuit Insights: Towards Interpretability Beyond Activations (2025). https://arxiv.org/abs/2510.14936 — accessed 2026-08-12. [weight-based, 2025]
	139	+- Anthropic — Circuits Updates June 2026. https://transformer-circuits.pub/2026/june-update/index.html — accessed 2026-08-12. [2026, safety-auditing]
	140	+- Neuronpedia — Circuits Research Landscape (Aug 2025). https://www.neuronpedia.org/graph/info — accessed 2026-08-12. [community, attribution-graphs]
	141	+- UFO-101 — auto-circuit (multi-ablation library). https://github.com/UFO-101/auto-circuit — accessed 2026-08-12. [tooling, ablations]
	142	+- anthropics — attribution-graphs-frontend. https://github.com/anthropics/attribution-graphs-frontend — accessed 2026-08-12. [tooling, viz]
	143	+- mlxterp — Mechanistic Interpretability Library for Apple Silicon (COAI). https://github.com/coairesearch/mlxterp — accessed 2026-08-12. [tooling, MLX, Apple-Silicon]
	144	+
	145	+## §4.5 — Weight-space cartography
	146	+
	147	+- Sharma, Ash, Misra — The Truth is in There: LASER (ICLR 2024). https://arxiv.org/abs/2312.13558 ; https://github.com/pratyushasharma/laser — accessed 2026-08-12. [weight-SVD, intervention]
	148	+- Martin & Mahoney — Heavy-Tailed Universality Predicts Trends in Test Accuracies (2019). https://arxiv.org/abs/1901.08278 — accessed 2026-08-12. [weight-only, spectra]
	149	+- Martin, Peng, Mahoney — Predicting NN quality without training/testing data (2020). https://arxiv.org/abs/2002.06716 — accessed 2026-08-12. [weight-only, pre-screen-prior-art]
	150	+- WeightWatcher. https://github.com/CalculatedContent/WeightWatcher — accessed 2026-08-12. [tooling, Mac-native]
	151	+- Martin & Mahoney — Post-mortem: Simpson's paradox in weight metrics (2021). https://arxiv.org/abs/2106.00734 — accessed 2026-08-12. [critique]
	152	+- Dettmers et al. — LLM.int8() (2022). https://arxiv.org/abs/2208.07339 — accessed 2026-08-12. [outliers]
	153	+- Sun, Chen, Kolter, Liu — Massive Activations in LLMs (2024). https://arxiv.org/abs/2402.17762 — accessed 2026-08-12. [outliers, sinks]
	154	+- Xiao et al. — Attention Sinks / StreamingLLM (2023). https://arxiv.org/abs/2309.17453 — accessed 2026-08-12. [sinks]
	155	+- Gu et al. — When Attention Sink Emerges (2024). https://arxiv.org/abs/2410.10781 — accessed 2026-08-12. [sinks]
	156	+- Yu et al. — The Super Weight in LLMs (2024). https://arxiv.org/abs/2411.07191 ; https://machinelearning.apple.com/research/the-super-weight — accessed 2026-08-12. [weight-visible-outliers, causal]
	157	+- Attention Sink in Transformers: A Survey (2026). https://arxiv.org/pdf/2604.10098 — accessed 2026-08-12. [sinks, 2026]
	158	+- A Single Layer to Explain Them All: Massive Activations (2026). https://arxiv.org/html/2605.08504 — accessed 2026-08-12. [outliers, 2026]
	159	+- Kornblith et al. — CKA (2019). https://arxiv.org/abs/1905.00414 — accessed 2026-08-12 (spot-check). [similarity]
	160	+- Raghu et al. — SVCCA (2017). https://arxiv.org/abs/1706.05806 — accessed 2026-08-12 (spot-check). [similarity]
	161	+- Davari et al. — Reliability of CKA (ICLR 2023). https://arxiv.org/abs/2210.16156 — accessed 2026-08-12. [CKA-pathologies]
	162	+- Ding, Denain, Steinhardt — Grounding Representation Similarity with Statistical Testing (2021). https://arxiv.org/abs/2108.01661 — accessed 2026-08-12. [similarity-critique]
	163	+- Projection kernel between attention-head weight subspaces (2026). https://arxiv.org/pdf/2601.10266 — accessed 2026-08-12. [weight-only-similarity, 2026]
	164	+- Gromov et al. — The Unreasonable Ineffectiveness of the Deeper Layers (2024). https://arxiv.org/abs/2403.17887 — accessed 2026-08-12. [layer-redundancy]
	165	+- Men et al. — ShortGPT (2024). https://arxiv.org/abs/2403.03853 — accessed 2026-08-12. [layer-redundancy]
	166	+- Lenc & Vedaldi — Equivariance and equivalence (2015). https://arxiv.org/abs/1411.5908 — accessed 2026-08-12. [stitching]
	167	+- Bansal, Nakkiran, Barak — Revisiting Model Stitching (2021). https://arxiv.org/abs/2106.07682 — accessed 2026-08-12. [stitching]
	168	+- Ainsworth et al. — Git Re-Basin (ICLR 2023). https://arxiv.org/abs/2209.04836 ; https://github.com/samuela/git-re-basin — accessed 2026-08-12. [permutation-alignment]
	169	+- Jordan et al. — REPAIR (2022). https://arxiv.org/abs/2211.08403 — accessed 2026-08-12. [permutation-alignment]
	170	+- Peña et al. — Re-basin via implicit Sinkhorn differentiation (2022). https://arxiv.org/abs/2212.12042 — accessed 2026-08-12. [permutation-alignment]
	171	+- Sharma et al. — Simultaneous linear connectivity modulo permutation (2024). https://arxiv.org/abs/2404.06498 — accessed 2026-08-12. [mode-connectivity]
	172	+- Garipov et al. — Loss Surfaces, Mode Connectivity, Fast Ensembling (2018). https://arxiv.org/abs/1802.10026 — accessed 2026-08-12. [mode-connectivity]
	173	+- Frankle et al. — Linear Mode Connectivity and the Lottery Ticket Hypothesis (2019). https://arxiv.org/abs/1912.05671 — accessed 2026-08-12. [mode-connectivity]
	174	+- Entezari et al. — Role of permutation invariance in linear mode connectivity (2021). https://arxiv.org/abs/2110.06296 — accessed 2026-08-12. [mode-connectivity]
	175	+- Lubana et al. — Mechanistic Mode Connectivity (2022). https://arxiv.org/abs/2211.08422 — accessed 2026-08-12. [mode-connectivity]
	176	+- Ilharco et al. — Editing Models with Task Arithmetic (ICLR 2023). https://arxiv.org/abs/2212.04089 ; https://github.com/mlfoundations/task_vectors — accessed 2026-08-12. [task-vectors]
	177	+- Yadav et al. — TIES-Merging (2023). https://arxiv.org/abs/2306.01708 — accessed 2026-08-12. [delta-analysis]
	178	+- Yu et al. — DARE (2023). https://arxiv.org/abs/2311.03099 — accessed 2026-08-12. [delta-sparsity]
	179	+- Minder et al. — Overcoming Sparsity Artifacts in Crosscoders (2025). https://arxiv.org/abs/2504.02922 — accessed 2026-08-12. [model-diffing, pitfalls]
	180	+- Anthropic — Stage-Wise Model Diffing (2024). https://transformer-circuits.pub/2024/model-diffing/index.html — accessed 2026-08-12. [model-diffing]
	181	+- Anthropic — Insights on Crosscoder Model Diffing (2025). https://transformer-circuits.pub/2025/crosscoder-diffing-update/index.html — accessed 2026-08-12. [model-diffing]
	182	+- Narrow Finetuning Leaves Clearly Readable Traces (2025). https://arxiv.org/abs/2510.13900 — accessed 2026-08-12. [model-diffing, 2025]
	183	+- Cross-Architecture Model Diffing with Crosscoders (2026). https://arxiv.org/pdf/2602.11729 — accessed 2026-08-12. [model-diffing, 2026]
	184	+- What We Learned Trying to Diff Base and Chat Models (LessWrong, 2025). https://www.lesswrong.com/posts/xmpauEXEerzYcJKNm/what-we-learned-trying-to-diff-base-and-chat-models-and-why — accessed 2026-08-12. [diffing-post-mortem]
	185	+- Watch the Weights: Unsupervised Monitoring of Fine-tuned LLMs (2025). https://arxiv.org/pdf/2508.00161 — accessed 2026-08-12. [weight-only, 2025]
	186	+- Learning to Interpret Weight Differences (2025). https://arxiv.org/html/2510.05092v4 — accessed 2026-08-12. [weight-diff, 2025]
	187	+- Steering LMs with Weight Arithmetic (Alignment Forum, 2025). https://www.alignmentforum.org/posts/HYTbakdHpxfaCowYp/steering-language-models-with-weight-arithmetic — accessed 2026-08-12. [weight-only-steering, 2025]
	188	+- Jiang et al. — Mixtral of Experts (2024). https://arxiv.org/abs/2401.04088 — accessed 2026-08-12. [MoE-routing]
	189	+- Dai et al. — DeepSeekMoE (2024). https://arxiv.org/abs/2401.06066 — accessed 2026-08-12. [MoE-structure]
	190	+- The Myth of Expert Specialization in MoEs (2026). https://arxiv.org/html/2604.09780v1 — accessed 2026-08-12. [MoE-critique, 2026]
	191	+- Unterthiner et al. — Predicting NN Accuracy from Weights (2020). https://arxiv.org/abs/2002.11448 — accessed 2026-08-12. [weights-as-data]
	192	+- Eilertsen et al. — Classifying the classifier (2020). https://arxiv.org/abs/2002.05688 — accessed 2026-08-12. [weights-as-data]
	193	+- Schürholt — hyper-representations publications. https://kschuerholt.github.io/publications/ — accessed 2026-08-12. [weight-space-learning]
	194	+- ICLR 2025 Workshop on Weight Space Learning. https://weight-space-learning.github.io/ — accessed 2026-08-12. [weight-space-learning, 2025]
	195	+- WeightCLIP / survey of weight space learning (2026). https://arxiv.org/html/2607.03551v1 — accessed 2026-08-12. [weight-space-learning, 2026]
	196	+
	197	+## §4.6 — Knowledge localization and editing
	198	+
	199	+- Meng, Bau, Andonian, Belinkov — Locating and Editing Factual Associations in GPT (ROME, NeurIPS 2022). https://arxiv.org/abs/2202.05262 ; https://rome.baulab.info/ ; https://github.com/kmeng01/rome — accessed 2026-08-12. [causal-tracing, editing]
	200	+- Meng et al. — MEMIT: Mass-Editing Memory in a Transformer (ICLR 2023). https://arxiv.org/abs/2210.07229 ; https://github.com/kmeng01/memit — accessed 2026-08-12. [mass-editing]
	201	+- Li et al. — PMET (AAAI 2024). https://arxiv.org/abs/2308.08742 — accessed 2026-08-12. [editing]
	202	+- Gupta et al. — EMMET / A Unified Framework for Model Editing (EMNLP Findings 2024). https://arxiv.org/abs/2403.14236 — accessed 2026-08-12. [editing-theory]
	203	+- Fang et al. — AlphaEdit (ICLR 2025 Outstanding Paper). https://arxiv.org/abs/2410.02355 ; https://github.com/jianghoucheng/alphaedit — accessed 2026-08-12. [editing, 2025]
	204	+- Reproducibility Study of AlphaEdit (2026). https://arxiv.org/pdf/2606.26783 — accessed 2026-08-12. [editing, 2026]
	205	+- UltraEdit — lifelong editing (2025). https://arxiv.org/abs/2505.14679 — accessed 2026-08-12. [lifelong-editing, 2025]
	206	+- Efficient Knowledge Editing via Minimal Precomputation (2025). https://arxiv.org/pdf/2506.04226 — accessed 2026-08-12. [editing-cost, 2025]
	207	+- Dai et al. — Knowledge Neurons in Pretrained Transformers (ACL 2022). https://arxiv.org/abs/2104.08696 — accessed 2026-08-12. [knowledge-neurons]
	208	+- Chen et al. — Journey to the Center of the Knowledge Neurons (2023). https://arxiv.org/html/2308.13198 — accessed 2026-08-12. [multilingual-neurons, critique]
	209	+- Neuron-Level Knowledge Attribution in LLMs (EMNLP 2024). https://arxiv.org/abs/2312.12141 — accessed 2026-08-12. [attribution-critique]
	210	+- Geva et al. — Dissecting Recall of Factual Associations (EMNLP 2023). https://arxiv.org/abs/2304.14767 — accessed 2026-08-12. [fact-recall-circuit]
	211	+- Hernandez et al. — Linearity of Relation Decoding / attribute lens (ICLR 2024). https://arxiv.org/abs/2308.09124 — accessed 2026-08-12. [attribute-lens]
	212	+- Chughtai, Cooney, Nanda — Summing Up the Facts (2024). https://arxiv.org/abs/2402.07321 — accessed 2026-08-12. [fact-recall-mechanisms]
	213	+- Hase et al. — Does Localization Inform Editing? (NeurIPS 2023). https://arxiv.org/abs/2301.04213 — accessed 2026-08-12. [localization-critique]
	214	+- Cohen et al. — Evaluating the Ripple Effects of Knowledge Editing (TACL 2024). https://arxiv.org/abs/2307.12976 — accessed 2026-08-12. [ripple-effects]
	215	+- Pressure-aware joint neighborhood optimization for ripple effects (2026). https://arxiv.org/pdf/2606.01610 — accessed 2026-08-12. [ripple-effects, 2026]
	216	+- Gupta, Rao, Anumanchipalli — Model Editing at Scale Leads to Gradual and Catastrophic Forgetting (2024). https://arxiv.org/abs/2401.07453 — accessed 2026-08-12. [sequential-collapse]
	217	+- Gupta et al. — Rebuilding ROME (2024). https://arxiv.org/abs/2403.07175 — accessed 2026-08-12. [collapse-diagnosis]
	218	+- Gu et al. — Model Editing Harms General Abilities of LLMs (EMNLP 2024). https://arxiv.org/abs/2401.04700 — accessed 2026-08-12. [side-effects]
	219	+- Should We Really Edit Language Models? (2024). https://arxiv.org/pdf/2410.18785 — accessed 2026-08-12. [evaluation-critique]
	220	+- Energy-Regularized Sequential Model Editing on Hyperspheres (2025). https://arxiv.org/html/2510.01172v1 — accessed 2026-08-12. [lifelong-editing, 2025]
	221	+- Wang et al. — Cross-Lingual Knowledge Editing in LLMs (2023). https://arxiv.org/abs/2309.08952 — accessed 2026-08-12. [multilingual-editing]
	222	+- MPN: Multilingual Patch Neuron (2024). https://arxiv.org/abs/2401.03190 — accessed 2026-08-12. [multilingual-editing]
	223	+- Multilingual Knowledge Editing with Language-Agnostic Factual Neurons (2024). https://arxiv.org/abs/2406.16416 — accessed 2026-08-12. [multilingual-neurons]
	224	+- How Do Multilingual Language Models Remember Facts? (2024). https://arxiv.org/abs/2410.14387 — accessed 2026-08-12. [multilingual-recall]
	225	+- Paths Not Taken: Multilingual Factual Recall Pipeline (2025). https://arxiv.org/html/2505.20546v2 — accessed 2026-08-12. [multilingual-circuits, 2025]
	226	+- Editing Across Languages: Survey of Multilingual Knowledge Editing (EMNLP 2025). https://arxiv.org/html/2505.14393 — accessed 2026-08-12. [survey, 2025]
	227	+- Wang et al. — Knowledge Editing for LLMs: A Survey (2023). https://arxiv.org/pdf/2310.16218 — accessed 2026-08-12. [survey]
	228	+- Zhang et al. — A Comprehensive Study of Knowledge Editing for LLMs (2024). https://arxiv.org/pdf/2401.01286 — accessed 2026-08-12. [survey]
	229	+- zjunlp/EasyEdit (ACL 2024). https://github.com/zjunlp/EasyEdit — accessed 2026-08-12. [tooling, CUDA-first]
	230	+
	231	+## §4.7 — Cross-model and cross-scale comparison
	232	+
	233	+- Klabunde et al. — Similarity of Neural Network Models: A Survey (2023, rev. 2025). https://arxiv.org/abs/2305.06329 — accessed 2026-08-12. [similarity, survey]
	234	+- Klabunde et al. — Towards Measuring Representational Similarity of LLMs (2023). https://arxiv.org/abs/2312.02730 — accessed 2026-08-12. [similarity, LLM]
	235	+- Klabunde, Wald et al. — ReSi: A Comprehensive Benchmark for Representational Similarity Measures (ICLR 2025). https://arxiv.org/abs/2408.00531 — accessed 2026-08-12. [similarity, benchmark]
	236	+- Gurnee et al. — Universal Neurons in GPT2 Language Models (2024). https://arxiv.org/abs/2401.12181 — accessed 2026-08-12. [universality, neurons]
	237	+- Universal Neurons in GPT-2: Emergence, Persistence, and Functional Impact (2025). https://arxiv.org/abs/2508.00903 — accessed 2026-08-12. [universality, 2025]
	238	+- Crosbie & Shutova — Induction Heads as an Essential Mechanism for Pattern Matching in ICL (2024). https://arxiv.org/abs/2407.07011 — accessed 2026-08-12. [universality, circuits]
	239	+- Wang et al. — Towards Universality: Mechanistic Similarity Across Architectures (2024). https://arxiv.org/abs/2410.06672 — accessed 2026-08-12. [universality, cross-architecture]
	240	+- Universal Response and Emergence of Induction in LLMs (2024). https://arxiv.org/abs/2411.07071 — accessed 2026-08-12. [universality]
	241	+- Lieberum et al. — Does Circuit Analysis Interpretability Scale? Chinchilla 70B (2023). https://arxiv.org/abs/2307.09458 — accessed 2026-08-12. [cross-scale, circuits]
	242	+- Tigges et al. — LLM Circuit Analyses Are Consistent Across Training and Scale (2024). https://arxiv.org/abs/2407.10827 — accessed 2026-08-12. [cross-scale, circuits]
	243	+- Lan et al. — Quantifying Feature Space Universality Across LLMs via SAEs (2024/2025). https://arxiv.org/abs/2410.06981 — accessed 2026-08-12. [universality, SAE]
	244	+- Thasarathan et al. — Universal Sparse Autoencoders (2025). https://arxiv.org/abs/2502.03714 — accessed 2026-08-12. [universality, SAE]
	245	+- Open-source replication of Anthropic's crosscoder paper (LessWrong). https://www.lesswrong.com/posts/srt6JXsRMtmqAJavD/open-source-replication-of-anthropic-s-crosscoder-paper-for — accessed 2026-08-12. [crosscoders, replication]
	246	+- Huh, Cheung, Wang, Isola — The Platonic Representation Hypothesis (ICML 2024). https://arxiv.org/abs/2405.07987 — accessed 2026-08-12. [PRH]
	247	+- Ziyin et al. — Proof of a Perfect Platonic Representation Hypothesis (2025). https://arxiv.org/abs/2507.01098 — accessed 2026-08-12. [PRH, theory]
	248	+- Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2026). https://arxiv.org/abs/2602.14486 — accessed 2026-08-12. [PRH, critique, 2026]
	249	+- Causes and Consequences of Representational Similarity in ML Models (2025). https://arxiv.org/pdf/2505.13899 — accessed 2026-08-12. [similarity, convergence]
	250	+- Model Stitching: Looking For Functional Similarity Between Representations (2023). https://arxiv.org/abs/2303.11277 — accessed 2026-08-12. [stitching]
	251	+- How Not to Stitch Representations to Measure Similarity (2024). https://arxiv.org/pdf/2412.11299 — accessed 2026-08-12. [stitching, pitfalls]
	252	+- Moschella et al. line — Latent Communication in Artificial Neural Networks (2024). https://arxiv.org/pdf/2406.11014 — accessed 2026-08-12. [relative-representations]
	253	+- Dravid, Gandelsman, Efros, Shocher — Rosetta Neurons (ICCV 2023). https://arxiv.org/abs/2306.09346 ; https://github.com/yossigandelsman/rosetta_neurons — accessed 2026-08-12. [universality, vision]
	254	+- TokSuite: Measuring the Impact of Tokenizer Choice on LM Behavior (2025). https://arxiv.org/pdf/2512.20757 — accessed 2026-08-12. [tokenizers]
	255	+- Minixhofer, Vulić, Ponti — Cross-Tokenizer Distillation via Approximate Likelihood Matching (2025). https://arxiv.org/abs/2503.20083 — accessed 2026-08-12. [tokenizers]
	256	+- Tokenisation is NP-Complete (2024). https://arxiv.org/pdf/2412.15210 — accessed 2026-08-12. [tokenizers]
	257	+- Interpreting the Effects of Quantization on LLMs (IJCNLP-AACL 2025). https://arxiv.org/abs/2508.16785 — accessed 2026-08-12. [quantization, interpretability]
	258	+- Through a Compressed Lens: Impact of Quantization on LLM Explainability and Interpretability (2025). https://arxiv.org/abs/2505.13963 — accessed 2026-08-12. [quantization, interpretability]
	259	+- How Quantization Changes Interpretable Features: An SAE Analysis of LMs (2026). https://arxiv.org/html/2606.03002v1 — accessed 2026-08-12. [quantization, SAE, 2026]
	260	+- The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs (2026). https://arxiv.org/pdf/2607.08734 — accessed 2026-08-12. [quantization, statistics, 2026]
	261	+- Does Quantization Affect Models' Performance on Long-Context Tasks? (2025). https://arxiv.org/pdf/2505.20276 — accessed 2026-08-12. [quantization, behavior]
	262	+
	263	+## §4.8 — Statistics and epistemology of interpretability
	264	+
	265	+- Bolukbasi et al. — An Interpretability Illusion for BERT (2021). https://arxiv.org/abs/2104.07143 — accessed 2026-08-12. [illusions, dataset-sensitivity]
	266	+- Makelov, Lange, Nanda — Is This the Subspace You Are Looking For? (2023). https://arxiv.org/abs/2311.17030 — accessed 2026-08-12. [illusions, patching]
	267	+- Wu et al. — A Reply to Makelov et al.'s "Interpretability Illusion" Arguments (2024). https://arxiv.org/abs/2401.12631 ; https://www.alphaxiv.org/abs/2401.12631 — accessed 2026-08-12. [illusions, debate]
	268	+- Friedman, Lampinen, Dixon, Chen, Ghandeharioun — Interpretability Illusions in the Generalization of Simplified Models (ICML 2024). https://arxiv.org/abs/2312.03656 — accessed 2026-08-12. [illusions, OOD]
	269	+- Jacovi & Goldberg — Towards Faithfully Interpretable NLP Systems (ACL 2020). https://arxiv.org/abs/2004.03685 — accessed 2026-08-12 (spot-check). [faithfulness]
	270	+- The Dead Salmons of AI Interpretability (2025). https://arxiv.org/abs/2512.18792 — accessed 2026-08-12. [statistics, multiplicity, 2025]
	271	+- Bennett et al. — Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon (2009). https://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf — accessed 2026-08-12. [statistics, historical]
	272	+- Lindner et al. — Tracr: Compiled Transformers as a Laboratory for Interpretability (NeurIPS 2023). https://arxiv.org/abs/2301.05062 ; https://github.com/google-deepmind/tracr — accessed 2026-08-12 (spot-check). [benchmark, ground-truth]
	273	+- Gupta et al. — InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability (NeurIPS 2024 D&B). https://arxiv.org/abs/2407.14494 — accessed 2026-08-12. [benchmark, ground-truth]
	274	+- Mueller, Geiger et al. — MIB: A Mechanistic Interpretability Benchmark (ICML 2025). https://arxiv.org/abs/2504.13151 — accessed 2026-08-12. [benchmark, 2025]
	275	+- BlackboxNLP-2025 MIB Shared Task (2025). https://arxiv.org/html/2510.06811v1 — accessed 2026-08-12. [benchmark, 2025]
	276	+- Geiger et al. — Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability (JMLR 2025). https://arxiv.org/abs/2301.04709 — accessed 2026-08-12. [causal-abstraction, theory]
	277	+- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough? (2025). https://arxiv.org/abs/2507.08802 — accessed 2026-08-12. [causal-abstraction, critique, 2025]
	278	+- Saphra & Wiegreffe — Mechanistic? (BlackboxNLP 2024). https://arxiv.org/abs/2410.09087 — accessed 2026-08-12. [epistemology, position]
	279	+- Casper — The Engineer's Interpretability Sequence (2023). https://www.alignmentforum.org/s/a6ne2ve5uturEEQK7 — accessed 2026-08-12. [rigor, position]
	280	+- Sharkey, Chughtai, Batson, Lindsey et al. — Open Problems in Mechanistic Interpretability (2025). https://arxiv.org/abs/2501.16496 — accessed 2026-08-12. [rigor, survey, 2025]
	281	+- Williams, Oldenburg et al. — Mechanistic Interpretability Needs Philosophy (2025). https://arxiv.org/abs/2506.18852 — accessed 2026-08-12. [epistemology, 2025]
	282	+- Make Mechanistic Interpretability Auditable (ACL 2026). https://arxiv.org/pdf/2606.00033 — accessed 2026-08-12. [rigor, auditing, 2026]
	283	+
	284	+## §4.9 — Systems side (local mapping at scale)
	285	+
	286	+- safetensors contributors — safetensors. https://github.com/safetensors/safetensors — accessed 2026-08-12. [storage]
	287	+- zarr-developers — zarr-python performance discussion. https://lightrun.com/answers/zarr-developers-zarr-python-zarr-slower-than-npy-hdf5-etc — accessed 2026-08-12. [storage, benchmarks]
	288	+- h5py. https://www.h5py.org/ — accessed 2026-08-12. [storage]
	289	+- SAELens training docs (ActivationsStore, CacheActivationsRunner). https://github.com/decoderesearch/SAELens/blob/main/docs/training_saes.md — accessed 2026-08-12. [SAE-pipeline, caching]
	290	+- Hugging Face — PyTorch training on Apple silicon. https://huggingface.co/docs/transformers/main/perf_train_special — accessed 2026-08-12. [MPS-limits]
	291	+- PyTorch — Apple Silicon Support (TorchServe docs). https://docs.pytorch.org/serve/hardware_support/apple_silicon_support.html — accessed 2026-08-12. [MPS]
	292	+- Apple — MLX Lazy Evaluation (docs). https://ml-explore.github.io/mlx/build/html/usage/lazy_evaluation.html — accessed 2026-08-12. [MLX-capture]
	293	+- Apple ML Research — Exploring LLMs with MLX and the Neural Accelerators in the M5 GPU (2025). https://machinelearning.apple.com/research/exploring-llms-mlx-m5 — accessed 2026-08-12. [Apple-Silicon]
	294	+- ml-explore — mlx-lm. https://github.com/ml-explore/mlx-lm — accessed 2026-08-12. [MLX]
	295	+- ggml-org — llama.cpp ; examples/eval-callback. https://github.com/ggml-org/llama.cpp ; https://github.com/ggml-org/llama.cpp/tree/master/examples/eval-callback — accessed 2026-08-12. [quantized-runtime-introspection]
	296	+- Apple — MTLCaptureManager (developer docs). https://developer.apple.com/documentation/metal/mtlcapturemanager — accessed 2026-08-12. [Metal-capture]
	297	+- vllm-metal — GPU Profiling docs. https://docs.vllm.ai/projects/vllm-metal/en/latest/profiling/ — accessed 2026-08-12. [Metal-profiling]
	298	+- alia-traces — Metal frame capture outside of Xcode (2020). https://alia-traces.github.io/metal/tools/xcode/2020/07/18/adding-framecapture-outside-of-xcode.html — accessed 2026-08-12. [Metal-capture]
	299	+- eDIF: A European Deep Inference Fabric for Remote Interpretability of LLM (2025). https://arxiv.org/pdf/2508.10553 — accessed 2026-08-12. [remote-execution, 2025]
	300	+- Oakley (Eclectic Light) — APFS: How sparse files work (2024) ; Sparse files are common in APFS (2021). https://eclecticlight.co/2024/06/08/apfs-how-sparse-files-work/ ; https://eclecticlight.co/2021/03/29/sparse-files-are-common-in-apfs/ — accessed 2026-08-12. [APFS, disk-IO]
	301	+- Enabling Performant and Flexible Model-Internal Observability for LLM Inference (2026). https://arxiv.org/pdf/2605.11093 — accessed 2026-08-12. [systems-observability, 2026]
	302	+
	303	+## §4.10 — Visualization and atlas prior art
	304	+
	305	+- Nanda — Neuroscope. https://github.com/neelnanda-io/Neuroscope — accessed 2026-08-12. [atlas-prior-art]
	306	+- Lin / Decode Research — Neuronpedia. https://www.neuronpedia.org/ ; https://github.com/hijohnnylin/neuronpedia ; https://www.neuronpedia.org/blog/neuronpedia-is-now-open-source — accessed 2026-08-12. [atlas-prior-art, API]
	307	+- OpenAI — Microscope (2020). https://openai.com/index/microscope/ ; https://microscope.openai.com/models — accessed 2026-08-12. [vision-atlas]
	308	+- Carter, Armstrong, Schubert, Johnson, Olah — Activation Atlas (Distill, 2019). https://distill.pub/2019/activation-atlas/ — accessed 2026-08-12. [atlas-prior-art]
	309	+- Transluce — Monitor ; Scaling Automatic Neuron Description (2024). https://transluce.org/observability-interface ; https://transluce.org/neuron-descriptions ; https://github.com/TransluceAI/observatory ; https://neurons.transluce.org/ — accessed 2026-08-12. [observability, neuron-catalog]
	310	+- Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers (2025). https://arxiv.org/pdf/2501.17727 — accessed 2026-08-12. [auto-interp-reliability, 2025]
	311	+- Vig — BertViz (2019). https://github.com/jessevig/bertviz — accessed 2026-08-12. [attention-viz]
	312	+- Hoover, Strobelt, Gehrmann — exBERT (2019). https://github.com/bhoov/exbert ; https://arxiv.org/abs/1910.05276 — accessed 2026-08-12. [viz]
	313	+- Alammar — Ecco (2021). https://github.com/jalammar/ecco — accessed 2026-08-12. [viz]
	314	+- Geva et al. — LM-Debugger (2022). https://github.com/mega002/lm-debugger ; https://arxiv.org/abs/2204.12130 — accessed 2026-08-12. [viz, intervention-UI]
	315	+- Gebru et al. — Datasheets for Datasets (2018). https://arxiv.org/abs/1803.09010 — accessed 2026-08-12. [provenance]
	316	+- Mitchell et al. — Model Cards for Model Reporting (2018). https://arxiv.org/abs/1810.03993 — accessed 2026-08-12. [provenance]
	317	+- Horwitz, Kurer, Kahana, Amar, Hoshen — Charting and Navigating Hugging Face's Model Atlas (2025). https://arxiv.org/abs/2503.10633 ; https://openreview.net/forum?id=BzFMBNqg7R ; https://horwitz.ai/model-atlas — accessed 2026-08-12. [model-atlas-naming, novelty]

added research/notes/4.10_visualization_atlas_design.md +39 −0

		`@@ -0,0 +1,39 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.10 Visualization and atlas prior art
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.10 — Visualization and atlas design
	12	+
	13	+## Prior atlases and browsers
	14	+- Neuroscope https://github.com/neelnanda-io/Neuroscope — max-activating examples for every MLP neuron in 25 small models. The canonical suggestive-but-unfalsifiable atlas: max-activating examples are actively misleading (Bolukbasi illusion); no controls, no uncertainty, no versioned provenance; unmaintained. Generation itself is cheap and Mac-feasible.
	15	+- Neuronpedia https://www.neuronpedia.org/ (open-sourced 2025: https://github.com/hijohnnylin/neuronpedia) — the field's central artifact host: 50M+ latents, 5+ TB, per-feature dashboards (activation histograms, logit effects, auto-interp scores), steering, attribution-graph hosting, full REST API. The modelmap gap: no per-entry reproducibility contract — no commit/config/hardware manifest that regenerates a dashboard, no confidence taxonomy; hosted data changes without versioned releases. A local Neuronpedia instance is a plausible serving layer for modelmap viewers rather than rebuilding dashboards.
	16	+- OpenAI Microscope (2020, vision) https://openai.com/index/microscope/ — exhaustive per-unit coverage + canonical "model organisms": both ideas modelmap inherits. No uncertainty/provenance.
	17	+- Activation Atlases (Distill 2019) https://distill.pub/2019/activation-atlas/ — the literal "atlas" ancestor; UMAP layouts are seed/hyperparameter-sensitive with no stability analysis — the beautiful-but-unfalsifiable failure mode charter §2 warns about. Never done for LLMs with replication controls.
	18	+- Anthropic Scaling Monosemanticity interface https://transformer-circuits.pub/2024/scaling-monosemanticity/ — activation spectra across strength deciles = an explicit anti-cherry-picking device worth copying; but features hand-picked, underlying model closed, artifact unversioned.
	19	+- circuit-tracer + attribution-graph frontend https://github.com/decoderesearch/circuit-tracer ; hosted graphs https://www.neuronpedia.org/graph/info ; library paper https://aclanthology.org/2025.blackboxnlp-1.14.pdf — first mainstream browser with a built-in verification affordance (feature interventions to test graph hypotheses). Limitations: graphs describe a replacement model; faithfulness bounded, not displayed per-edge; pruning thresholds shape what you see; no confidence levels or provenance manifests.
	20	+- Transluce Monitor / neuron descriptions https://transluce.org/observability-interface ; https://github.com/TransluceAI/observatory — LLM-generated descriptions of all 458,752 neurons of Llama-3.1-8B + live steer/ablate UI. Auto-interp descriptions unreliable (metrics can fail to distinguish trained from random transformers: https://arxiv.org/pdf/2501.17727); no causal-verification labels, no versioned provenance.
	21	+- BertViz https://github.com/jessevig/bertviz — most-used interp viz ever; attention ≠ explanation; a caution: high adoption of a low-epistemics tool shaped years of over-claiming.
	22	+- exBERT https://arxiv.org/abs/1910.05276 — early corpus-level aggregation (anecdote → statistics). Ecco https://github.com/jalammar/ecco — NMF factors without stability analysis. LM-Debugger https://arxiv.org/abs/2204.12130 — early observation+intervention coupling; single interpretive frame presented as ground truth.
	23	+- Gemma Scope / Llama Scope https://arxiv.org/abs/2408.05147 ; https://huggingface.co/google/gemma-scope ; https://arxiv.org/abs/2410.20526 — best existing example of interpretability artifacts released with reproducible identity (fixed weights, named hookpoints, versioned HF artifacts). Still: no per-feature confidence labels; seed-replication not surfaced. Running (not training) Gemma Scope SAEs on a 2B fits 16 GB — natural third-party input for modelmap cross-checks.
	24	+
	25	+## Provenance templates
	26	+- Datasheets for Datasets (Gebru et al. 2018) https://arxiv.org/abs/1803.09010 ; Model Cards (Mitchell et al. 2018) https://arxiv.org/abs/1810.03993 — direct templates for modelmap's "map cards": commit hash, config, model hash+quantization, promptset checksums, hardware manifest, seeds, controls run, replication stats, confidence level. No interpretability artifact host ships anything equivalent (verified across Neuronpedia, Neuroscope, Transluce, circuit-tracer).
	27	+
	28	+## Uncertainty encoding — cross-cutting finding
	29	+Across all tools surveyed: uncertainty is essentially never encoded visually. Partial exceptions: Neuronpedia auto-interp scores (a scalar of simulation fit, not a CI, not causal validity); Anthropic activation-spectrum deciles (anti-cherry-picking, not uncertainty); circuit-tracer prunable thresholds (manipulable, not summarized). Nobody displays seed-replication rates, bootstrap CIs, or correlational-vs-causal status on the artifact. The Level 0–3 badges have no prior art to copy — and none to compete with.
	30	+
	31	+## Novelty check — "model atlas" naming collision
	32	+- Horwitz et al. 2025, Charting and Navigating Hugging Face's Model Atlas, https://arxiv.org/abs/2503.10633 ; https://horwitz.ai/model-atlas — maps the inter-model graph of HF (which checkpoint fine-tuned from which), not model internals. Name collision to record in novelty_check.md, not a scope overlap.
	33	+- Verdict (2026-08-12): the project's bet is confirmed — no existing atlas of LLM internals combines versioned entries + regeneration recipes + explicit confidence levels + published negative results. Closest partials: Gemma Scope (versioning without confidence), Neuronpedia (hosting/API without reproducibility contracts), circuit-tracer (verification affordance without provenance).
	34	+
	35	+## Design conclusions for modelmap.io
	36	+1. Copy: exhaustive per-unit coverage (Microscope), activation-spectrum deciles (Anthropic), intervention affordances (circuit-tracer, LM-Debugger), corpus-level aggregation (exBERT).
	37	+2. Avoid: max-activating-example-only views (Neuroscope), unstable DR layouts without stability analysis (Activation Atlases), auto-interp labels presented as ground truth (Transluce).
	38	+3. Differentiate: map cards (datasheet-style provenance), visible Level 0–3 badges, seed-replication rates and ablation-scheme curves rendered on the artifact, negative-results section.
	39	+4. Consider a local Neuronpedia instance as serving infrastructure; keep modelmap.io as the provenance-first publication layer.

added research/notes/4.1_observational_probing.md +62 −0

		`@@ -0,0 +1,62 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.1 Observational probing
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.1 — Observational probing
	12	+
	13	+## A1. Linear probes (probing classifiers)
	14	+- Key papers: Alain & Bengio 2016, https://arxiv.org/abs/1610.01644 ; Belinkov, CL 2022, https://arxiv.org/abs/2102.12452
	15	+- Reveals: whether a property is linearly decodable from hidden states per layer — an "information availability" profile.
	16	+- Type: Observational (decodability ≠ use by the model).
	17	+- Cost: Cheap — one forward pass per prompt + logistic/ridge on cached activations. Feasible 0.5B–14B on 16–64 GB (14B fp16 ≈ 28 GB weights → 64 GB tier or 4-bit). Main cost = activation storage; stream to disk.
	18	+- Implementations: TransformerLens `run_with_cache` (MPS opt-in — some ops lack stable MPS kernels: https://transformerlensorg.github.io/TransformerLens/content/getting_started.html); baukit https://github.com/davidbau/baukit (device-agnostic). Trivial in MLX.
	19	+- Failure modes: probe learns the task (Hewitt & Liang); decodable-but-unused information; dataset/seed sensitivity; high-capacity probes find structure in random models.
	20	+- Epistemic status: Established as decodability measurement; contested as evidence of use.
	21	+
	22	+## A2. Logit lens
	23	+- Key source: nostalgebraist 2020, https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens
	24	+- Reveals: layer-by-layer refinement of the next-token prediction (final LN + unembedding applied to intermediate residual).
	25	+- Cost: Near-free; ideal Mac technique; trivial in MLX.
	26	+- Failure modes: works on GPT-2 family, poorly/misleadingly on BLOOM/OPT/some Llamas (intermediate residual bases differ from final — documented in Belrose et al. 2023); early layers unreadable.
	27	+- Epistemic status: Established quick diagnostic; debunked-in-part as a quantitative claim about intermediate beliefs.
	28	+
	29	+## A3. Tuned lens
	30	+- Key paper: Belrose et al. 2023, https://arxiv.org/abs/2303.08112 ; code https://github.com/AlignmentResearch/tuned-lens
	31	+- Reveals: same as logit lens with trained per-layer affine translators — less biased prediction trajectories.
	32	+- Cost: Moderate (hours on consumer hardware ≤7B; pretrained lenses exist). Model frozen — no backward through model.
	33	+- Failure modes: translators can "do the work" (mitigated by affine-only capacity); lens must be retrained per checkpoint AND per quantization level — directly relevant to Experiment F.
	34	+- Epistemic status: Established refinement; validated to 20B (Pythia/NeoX/OPT).
	35	+
	36	+## A4. Early decoding / vocabulary-space successors (DoLa, Future Lens, Patchscopes)
	37	+- Key papers: DoLa: Chuang et al. 2023, https://arxiv.org/abs/2309.03883 ; Future Lens: Pal et al., CoNLL 2023, https://future.baulab.info/ ; Patchscopes: Ghandeharioun et al., ICML 2024, https://arxiv.org/abs/2401.06102
	38	+- Reveals: DoLa: which layers hold "factual" signal. Future Lens: hidden states encode ≥2 tokens ahead. Patchscopes: unifying framework — patch a hidden state into another prompt/model and let the LLM verbalize it; subsumes logit/tuned lens and fixes early-layer unreadability.
	39	+- Cost: Cheap–moderate; all ≤14B on Mac. Patchscopes ≈ 2× inference.
	40	+- Failure modes: verbalized readouts inherit the model's confabulation; DoLa's premise unreliable across families ("Steering off Course", ACL 2025).
	41	+- Epistemic status: Established tools; DoLa mechanism contested. Also: SimLens (2025) https://arxiv.org/pdf/2507.17618
	42	+
	43	+## A5. Representation reading / RepE
	44	+- Key paper: Zou et al. 2023, https://arxiv.org/abs/2310.01405 ; code https://github.com/andyzoujm/representation-engineering ; survey of challenges (2025): https://arxiv.org/pdf/2502.17601
	45	+- Reveals: population-level "reading vectors" for high-level concepts via PCA on activation differences over stimulus pairs.
	46	+- Cost: Cheap (dozens–hundreds of forwards + PCA). Very Mac-friendly.
	47	+- Implementations: repeng https://github.com/vgel/repeng — control vector "in under sixty seconds", works on CPU/MPS, exports GGUF control vectors for llama.cpp (PR https://github.com/ggml-org/llama.cpp/pull/5970) — directly relevant to mapping quantized local models.
	48	+- Failure modes: concept vectors conflate correlated stimulus features; single-direction assumption; LLM-judge evaluation (AxBench: prompting baselines beat most representation methods).
	49	+- Epistemic status: Linear concept directions established; methodology reliability contested.
	50	+
	51	+## A6. Concept erasure — INLP, amnesic probing, LEACE
	52	+- Key papers: INLP: Ravfogel et al., ACL 2020, https://aclanthology.org/2020.acl-main.647.pdf ; amnesic probing: Elazar et al., TACL 2021, https://aclanthology.org/2021.tacl-1.10/ ; LEACE: Belrose et al., NeurIPS 2023, https://arxiv.org/abs/2306.03819 ; code https://github.com/EleutherAI/concept-erasure
	53	+- Reveals: whether the model uses a property: erase it (LEACE: closed-form, provably-minimal-damage linear guarding) and observe behavioral change. Interventional on representations — bridges to §4.2.
	54	+- Cost: Cheap–moderate; LEACE needs only class-conditional means/covariances. Mac-feasible to 14B.
	55	+- Failure modes: INLP damages representations broadly (rank loss); linear erasure can leave or increase nonlinearly-decodable information; erasure-operator choice changes amnesic conclusions (2025: https://arxiv.org/html/2506.11673). Post-LEACE: Oracle-LEACE https://blog.eleuther.ai/oracle-leace/ ; nonlinear erasure via density matching https://arxiv.org/abs/2507.12341
	56	+- Epistemic status: LEACE established for the linear guarantee; amnesic-probing inferences contested.
	57	+
	58	+## A7. Probing pitfalls: control tasks, selectivity, MDL
	59	+- Key papers: Hewitt & Liang, EMNLP 2019, https://arxiv.org/abs/1909.03368 ; Voita & Titov 2020 (MDL), https://arxiv.org/pdf/2003.12298
	60	+- Substance: selectivity = task accuracy − control-task accuracy; MDL replaces accuracy with codelength. Probes overfit to: word identity/frequency memorization, spurious dataset correlates, annotation artifacts, probe capacity, shared vocabulary between splits.
	61	+- Implication for modelmap (Experiment A is exactly this): every probe map needs shuffled-label controls, selectivity or MDL reporting, ≥2 datasets, ≥3–5 seeds, randomly-initialized-model baselines.
	62	+- Epistemic status: Established best practice; ignoring it is the field's canonical failure.

added research/notes/4.2_causal_interventions.md +73 −0

		`@@ -0,0 +1,73 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.2 Causal / interventional methods
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.2 — Causal / interventional methods
	12	+
	13	+## B1. Activation patching (causal mediation / interchange intervention)
	14	+- Key papers: origins: Vig et al., NeurIPS 2020, https://arxiv.org/abs/2004.12265 ; best practices: Zhang & Nanda, ICLR 2024, https://arxiv.org/abs/2309.16042 ; Heimersheim & Nanda 2024, https://arxiv.org/pdf/2404.15255
	15	+- Reveals: which components causally mediate a behavior (clean/corrupted runs; noising vs denoising directions differ in meaning).
	16	+- Cost: 2 forwards per (component, prompt-pair); exhaustive sweeps O(#components). Time-heavy, memory-light — well-suited to Macs 0.5B–8B; 14B with patience/quantization. Hook overhead MLX vs MPS = Experiment H.
	17	+- Implementations: TransformerLens (MPS opt-in), NNsight https://github.com/ndif-team/nnsight , pyvene, baukit.
	18	+- Failure modes: metric (logit-diff vs prob) and corruption type (Gaussian vs symmetric token swap) change localization conclusions; backup/hydra compensation; dormant-pathway subspace illusions; multi-mediator interactions (Curse of Multiple Mediators, 2026, https://arxiv.org/html/2606.27510v1).
	19	+- Epistemic status: Established core method; hyperparameter sensitivity well-documented. methodology.md must pin corruption method + metric per map type, before scanning.
	20	+
	21	+## B2. Path patching
	22	+- Key papers: Wang et al. (IOI), ICLR 2023, https://arxiv.org/abs/2211.00593 ; Goldowsky-Dill et al. 2023, https://arxiv.org/abs/2304.05969 ; accelerated: APP 2025, https://arxiv.org/html/2511.05442
	23	+- Reveals: causal importance of specific edges/paths, not just nodes.
	24	+- Cost: combinatorially heavier than node patching. Practical ≤3B–7B on Mac; 14B painful.
	25	+- Failure modes: metric sensitivity + hypothesis-space explosion; faithfulness not guaranteed (Miller et al. 2024, https://arxiv.org/pdf/2407.08734).
	26	+- Epistemic status: Established; IOI replicated (even in Mamba, https://arxiv.org/html/2407.14008v1) but faithfulness swings with ablation details.
	27	+
	28	+## B3. Attribution patching (AtP/AtP*) and gradient-based screening
	29	+- Key papers: Nanda 2023, https://www.neelnanda.io/mechanistic-interpretability/attribution-patching ; Syed, Rager, Conmy, BlackboxNLP 2024, https://aclanthology.org/2024.blackboxnlp-1.25.pdf ; AtP*: Kramár et al. 2024, https://arxiv.org/abs/2403.00745 ; EAP-IG: Hanna et al., COLM 2024, https://arxiv.org/abs/2403.17806
	30	+- Reveals: first-order approximation of every patch effect in ~2 forwards + 1 backward; AtP* fixes false negatives (QK softmax linearization + GradDrop).
	31	+- Cost: cheapest per-component causal estimate, but backward ≈ 2–3× weight memory: ≤3B fp16 on 16–32 GB, ≤7–8B on 64 GB. Gradients through 4-bit quantized models are awkward — a real Mac constraint. MLX autodiff is the native path worth benchmarking (Experiment H).
	32	+- Failure modes: bad approximation for large activations (residual, MLP0, saturated attention); systematic false negatives (quantified in AtP); second-order failures (When Attribution Patching Lies*, 2026, https://arxiv.org/pdf/2606.09899); LRP alternative RelP (2025, https://arxiv.org/html/2508.21258v1).
	33	+- Epistemic status: Established as screening; never a substitute for real patching on final claims.
	34	+
	35	+## B4. Causal tracing / ROME-style locating
	36	+- Key papers: Meng et al., NeurIPS 2022, https://arxiv.org/abs/2202.05262 (code https://rome.baulab.info/) ; critique: Hase et al., NeurIPS 2023, https://arxiv.org/abs/2301.04213
	37	+- Reveals: where factual associations appear "stored" (noise subject tokens, restore states; mid-layer MLPs at last subject token).
	38	+- THE central cautionary tale for modelmap: Hase et al. — edit success essentially uncorrelated with tracing-identified layers (ρ≈−0.13 at GPT-J layer 6); editing works almost anywhere; localization ≠ editability. Gaussian noising also distorts activations off-distribution.
	39	+- Epistemic status: Contested / debunked-in-part (the locating→editing inference); the tracing observation itself replicates.
	40	+
	41	+## B5. Ablations (zero, mean, resample)
	42	+- Key analyses: Zhang & Nanda (B1); Miller et al. 2024 (ablation choice flips faithfulness scores).
	43	+- Reveals: necessity (not sufficiency) of a component.
	44	+- Cost: cheapest intervention class; 1 forward per ablation. Ideal for Experiment B (localization vs diffusion).
	45	+- Failure modes: zero-ablation is off-distribution (can create signal); mean-ablation destroys within-distribution variance; resample > mean > zero is the emerging norm; hydra effects mask importance; unit scans → multiple-comparison inflation.
	46	+- Epistemic status: Established, with distribution-dependence caveats.
	47	+
	48	+## B6. Causal scrubbing
	49	+- Key sources: Chan et al. (Redwood) 2022, https://www.alignmentforum.org/posts/JvZhhzycHu2Yd57RN/causal-scrubbing-a-method-for-rigorously-testing ; pitfalls: https://www.alignmentforum.org/posts/DFarDnQjMnjsKvW8s/practical-pitfalls-of-causal-scrubbing
	50	+- Reveals: quantitative faithfulness score for a full interpretability hypothesis via all claimed-behavior-preserving resample ablations.
	51	+- Cost: expensive; fine ≤1B on Mac, research-grade beyond.
	52	+- Note: Redwood's rust_circuit is archived; no maintained macOS-friendly implementation exists — a genuine gap.
	53	+- Epistemic status: Established conceptually, little-used; succeeded in spirit by circuit-faithfulness metrics.
	54	+
	55	+## B7. DAS / interchange intervention accuracy / pyvene
	56	+- Key papers: Geiger et al., CLeaR 2024, https://arxiv.org/abs/2303.02536 ; Boundless DAS: Wu et al., NeurIPS 2023, https://arxiv.org/abs/2305.08809 ; pyvene: https://arxiv.org/abs/2403.07809 , https://github.com/stanfordnlp/pyvene
	57	+- Reveals: whether a high-level causal model is implemented in learned subspaces (train a rotation; IIA = fraction of counterfactuals matched).
	58	+- Cost: moderate–high; Boundless DAS ran on 7B — Mac-feasible ≤7B (gradients only through rotation + partial graph).
	59	+- The headline dispute: Makelov, Lange, Nanda 2023, https://arxiv.org/abs/2311.17030 (trained subspaces can work via dormant causally-disconnected pathways) vs Wu, Geiger et al. reply 2024, https://www.alphaxiv.org/abs/2401.12631 . Genuinely unresolved — trained-subspace claims need convergent evidence and a distinct flag in confidence.md.
	60	+- Epistemic status: Framework established; subspace-illusion question contested. RAVEL provides standardized evaluation.
	61	+
	62	+## B8. Steering vectors (ActAdd, CAA, ITI)
	63	+- Key papers: ActAdd: Turner et al. 2023, https://arxiv.org/abs/2308.10248 ; CAA: Rimsky et al., ACL 2024, https://arxiv.org/abs/2312.06681 ; ITI: Li et al., NeurIPS 2023, https://arxiv.org/pdf/2306.03341
	64	+- Reveals: behaviors as linear residual-stream directions — sufficiency-style causal evidence.
	65	+- Cost: very cheap; among the most Mac-friendly causal techniques. repeng + llama.cpp applies control vectors to GGUF quantized models.
	66	+- Critiques (the 2024–2026 evaluation wave): Tan et al., NeurIPS 2024, https://arxiv.org/abs/2407.12404 (steerability input-heterogeneous, sometimes anti-steers); Steering off Course, ACL 2025, https://aclanthology.org/2025.acl-long.974/ (fails to transfer across 36 models); AxBench 2025, https://arxiv.org/abs/2501.17148 (prompting/finetuning beat representation steering); non-identifiability (2026, https://arxiv.org/pdf/2602.06801); geometric predictors of failure (2026, https://arxiv.org/pdf/2602.17881).
	67	+- Epistemic status: Steering can work — established; reliability contested. *Always report against prompting baselines + random-direction controls. Opportunity: publish steerability maps with variance.*
	68	+
	69	+## Cross-cutting 2024–2026 signals
	70	+1. Faithfulness (not circuit overlap) became the standard comparison metric (EAP-IG) — then shown non-robust itself (Miller et al.).
	71	+2. Patchscopes unifies lens-style readouts with interventions.
	72	+3. Field syntheses: Open Problems in Mechanistic Interpretability (Sharkey et al. 2025, https://arxiv.org/abs/2501.16496); Bereska & Gavves review 2024, https://arxiv.org/pdf/2404.14082 ; Dead Salmons (2025, https://arxiv.org/pdf/2512.18792) — null-model controls, aligned with charter §8.4.
	73	+4. Tooling: NNsight/NDIF paper https://arxiv.org/abs/2407.14561 benchmarks NNsight vs TransformerLens/pyvene/baukit. TransformerLens MPS remains opt-in. Plan: baukit-style raw hooks on MPS + native MLX hooks, benchmark both (Experiment H). No mainstream interpretability library targets MLX — publishable gap.

added research/notes/4.3_feature_decomposition.md +93 −0

		`@@ -0,0 +1,93 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.3 Feature decomposition (SAEs, superposition, dictionary methods)
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.3 — Feature decomposition
	12	+
	13	+Cost classes used below: L (laptop-trivial, minutes), M (hours + tens of GB
	14	+activations), H (multi-day / >64 GB working sets / cluster-typical). Apple Silicon
	15	+status as of 2026-08-12: almost all tooling is PyTorch and thus MPS-capable in
	16	+principle but CUDA-assumed in practice — expect `device="mps"` patching, float64
	17	+fallbacks, no flash-attn.
	18	+
	19	+## A1. Superposition theory (Toy Models of Superposition)
	20	+- Key paper: Elhage, Hume, Olsson, … Olah (Anthropic), Toy Models of Superposition, 2022. https://transformer-circuits.pub/2022/toy_model/index.html
	21	+- Claims: Networks represent more features than dimensions as non-orthogonal directions; occurrence sparsity governs superposition; predicts polysemantic neurons and phase transitions in toy ReLU models.
	22	+- Type: Theoretical/constructive (toy models with known ground truth).
	23	+- Cost: L. Toy models train in seconds on any Mac; ideal for MLX reimplementation and as planted-ground-truth testbeds.
	24	+- Implementation: Anthropic notebooks; many replications (ARENA, TransformerLens-based). Trivial on MPS/MLX.
	25	+- Failure modes / critiques: Toy results extrapolate uncertainly to real LLMs; "features as linear directions" is an assumption, not a theorem (multi-dimensional/circular features — Engels et al. 2024, https://arxiv.org/abs/2405.14860); degree of superposition in real models is inferred, not measured directly.
	26	+- Epistemic status: Established as theory of toy models; contested as a complete account of real-LLM representation.
	27	+
	28	+## A2. Dictionary learning precursors
	29	+- Key papers: Yun, Chen, Olshausen, LeCun 2021, https://arxiv.org/abs/2103.15949 ; Sharkey, Braun, Millidge 2022 (interim report), https://www.lesswrong.com/posts/z6QQJbtpkEAX3Aojj/interim-research-report-taking-features-out-of-superposition ; Cunningham, Ewart, Riggs, Huben, Sharkey 2023, https://arxiv.org/abs/2309.08600
	30	+- Claims: Activations decompose into a sparse nonnegative combination of an overcomplete dictionary of interpretable directions.
	31	+- Type: Observational (Cunningham et al. add ablation checks).
	32	+- Cost: L–M at Pythia-70M/410M scale.
	33	+- Status: Established as lineage; methods superseded by SAELens-era tooling. Classical sparse coding (ISTA/OMP) too slow at LLM scale; early SAEs had dead latents and shrinkage.
	34	+
	35	+## A3. ReLU SAEs — Towards / Scaling Monosemanticity
	36	+- Key papers: Bricken et al. 2023, https://transformer-circuits.pub/2023/monosemantic-features/index.html ; Templeton et al. 2024, https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html
	37	+- Claims: L1-penalized overcomplete autoencoders on MLP/residual activations recover largely monosemantic features; scale to frontier models; features can steer behavior.
	38	+- Type: Primarily observational; steering demos are weak causal evidence about behavior, not about decomposition correctness.
	39	+- Cost — THE cost driver for modelmap: competitive SAEs need ~100M–2B tokens of activations; at d_model=2048 fp32, one hook point = 8 KB/token → 500M tokens ≈ 4 TB if materialized (hence streaming/regenerating activations + shuffle buffer; bf16 halves it). On a 32–64 GB Mac: SAEs on 0.5B–2B models at 4k–65k width feasible (M, hours–days); 7B–14B residual SAEs at ≥131k width are H, marginal.
	40	+- Implementations: SAELens https://github.com/jbloomAus/SAELens (PyTorch; official tutorial cites "~2 hours on an M3 Max" — MPS works); dictionary_learning https://github.com/saprmarks/dictionary_learning (device-agnostic, MPS-workable); EleutherAI sparsify https://github.com/EleutherAI/sparsify (top-k recipe, CUDA-oriented, MPS untested); MLX-native: mlxterp https://github.com/coairesearch/mlxterp (young, purpose-built for Apple Silicon — novelty-check target).
	41	+- Failure modes: Shrinkage from L1; dead latents; substantial reconstruction error ("dark matter"); interpretability of a feature ≠ its causal role.
	42	+- Epistemic status: Established as a decomposition method; contested as a route to faithful "true features".
	43	+
	44	+## A4. SAE architecture variants (Gated, TopK, JumpReLU, BatchTopK, Matryoshka)
	45	+- Key papers: Gated: Rajamanoharan et al. 2024, https://arxiv.org/abs/2404.16014 ; TopK: Gao et al. (OpenAI) 2024, https://arxiv.org/abs/2406.04093 ; JumpReLU: Rajamanoharan et al. 2024, https://arxiv.org/abs/2407.14435 ; BatchTopK: Bussmann, Leask, Nanda 2024, https://arxiv.org/abs/2412.06410 ; Matryoshka: Bussmann et al. 2025, https://arxiv.org/abs/2503.17547
	46	+- Claims: Fix shrinkage (Gated); direct sparsity control + clean scaling laws (TopK/BatchTopK); Pareto-improve fidelity-vs-sparsity (JumpReLU); reduce feature absorption via nested dictionaries (Matryoshka).
	47	+- Cost: Same class as A3; TopK/BatchTopK slightly cheaper to tune (no L1 sweep). All implementable in MLX.
	48	+- Implementations: All in SAELens and SAEBench's suite (200+ open SAEs, 7 architectures); dictionary_learning implements most; sparsify has TopK/transcoders. JumpReLU straight-through estimators port fine to MPS.
	49	+- Failure modes: SAEBench showed proxy-metric gains (loss recovered, L0) do not reliably transfer to downstream utility; Matryoshka underperforms proxies while winning on disentanglement — proxy metrics are unreliable selection criteria.
	50	+- Epistemic status: Established engineering improvements; ranking among them contested and metric-dependent.
	51	+
	52	+## A5. Transcoders, skip transcoders, cross-layer transcoders (CLTs), crosscoders
	53	+- Key papers: Dunefsky, Chlenski, Nanda 2024, https://arxiv.org/abs/2406.11944 ; Paulo, Shabalin, Belrose 2025 (skip transcoders), https://arxiv.org/abs/2501.18823 ; Lindsey, Templeton et al. 2024 (crosscoders), https://transformer-circuits.pub/2024/crosscoders/index.html
	54	+- Claims: Transcoders sparsely approximate an MLP's input→output map, enabling feature-to-feature weight analysis and circuit tracing; skip transcoders Pareto-dominate SAEs on interpretability+fidelity; crosscoders learn features shared across layers (cross-layer superposition) or across models (base-vs-instruct diffing).
	55	+- Cost: Like SAEs per layer (M for ≤2B on a Mac); CLTs/crosscoders read many layers at once — capture cost multiplies by layer count, pushing 7B+ toward H on 64 GB.
	56	+- Implementations: sparsify; EleutherAI clt-training https://github.com/EleutherAI/clt-training ; dictionary_learning; crosscoder diffing replication https://github.com/ckkissane/crosscoder-model-diff-replication . PyTorch; MPS untested but no exotic kernels.
	57	+- Failure modes: Transcoders only cover MLPs (attention needs separate treatment — cf. Anthropic 2025 QK-tracing); replacement-model error nodes can dominate; crosscoder diffing latents can be training-asymmetry artifacts.
	58	+- Epistemic status: Established and ascendant (they power attribution graphs); details contested.
	59	+
	60	+## A6. Polysemanticity, feature splitting, feature absorption
	61	+- Key papers: Bricken et al. 2023 (splitting); Chanin et al. 2024, A is for Absorption, https://arxiv.org/abs/2409.14507
	62	+- Claims: As dictionary width grows, features split hierarchically; "absorption" makes a general latent silently fail on tokens claimed by more specific child latents — latents are not reliable classifiers of their nominal concept.
	63	+- Cost: L–M (evaluation on existing SAEs, e.g. Gemma Scope, on any Mac).
	64	+- Implementation: SAEBench includes absorption metrics — https://github.com/adamkarvonen/SAEBench . MPS-workable.
	65	+- Epistemic status: Established phenomenon; Matryoshka SAEs partially mitigate.
	66	+
	67	+## A7. Automated interpretability (auto-interp) and its failure modes
	68	+- Key papers: Bills et al. (OpenAI) 2023, https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html ; Paulo, Mallen, Juang, Belrose 2024, https://arxiv.org/abs/2410.13928 ; critique: Huang et al. 2023, https://arxiv.org/abs/2309.10312
	69	+- Claims: An explainer LLM labels each latent from top-activating examples; scorers (simulation, detection, fuzzing, intervention scoring) grade at scale.
	70	+- Cost: Dominated by LLM inference; with a local MLX-served model (e.g. the maclustr llm endpoint) it is M-class.
	71	+- Implementations: EleutherAI Delphi https://github.com/EleutherAI/delphi (local models OK); Neuronpedia pipeline https://www.neuronpedia.org
	72	+- Failure modes: High-scoring explanations had low precision/recall under targeted tests (Huang et al.); top-activation bias; explanations describe activation, not downstream causal role.
	73	+- Epistemic status: Established as tooling; explanation quality claims debunked-in-part — never publish auto-interp labels above Level 1 without intervention scoring.
	74	+
	75	+## A8. SAE evaluation: SAEBench, RAVEL, the metrics problem
	76	+- Key papers: SAEBench: Karvonen et al. 2025, https://arxiv.org/abs/2503.09532 (https://www.neuronpedia.org/sae-bench/info) ; RAVEL: Huang et al., ACL 2024, https://arxiv.org/abs/2402.17700 ; reliability critique: Chanin et al. 2026, https://arxiv.org/abs/2605.18229
	77	+- Claims: SAEBench: 8 metrics over 200+ open SAEs — key finding: proxy metrics ≠ downstream utility. RAVEL: counterfactual-intervention benchmark; distributed methods (DAS/MDAS) beat neurons and SAE latents.
	78	+- Cost: L–M — SAEBench on Gemma-2-2B SAEs fits a 32 GB Mac; strong candidate to port as modelmap's SAE gate.
	79	+- Implementations: https://github.com/adamkarvonen/SAEBench ; https://github.com/explanare/ravel . MPS-workable with patching.
	80	+- Failure modes: Benchmark rankings unstable across tasks/seeds (Chanin 2026); Goodharting.
	81	+- Epistemic status: Established best practice; individual metrics contested.
	82	+
	83	+## A9. The 2024–2026 "SAEs disappointing?" debate; seed/dataset dependence; universality
	84	+- Key sources: GDM deprioritization: Smith et al., Mar 2025, https://deepmindsafetyresearch.medium.com/negative-results-for-sparse-autoencoders-on-downstream-tasks-and-deprioritising-sae-research-6cadcfc125b9 ; sparse probing negative: Kantamneni et al. 2025, https://arxiv.org/abs/2502.16681 ; AxBench: Wu et al. 2025, https://arxiv.org/abs/2501.17148 ; dataset dependence: Kissane et al. 2024, https://www.alignmentforum.org/posts/rtp6n7Z23uJpEH7od/saes-are-highly-dataset-dependent-a-case-study-on-the ; seed dependence: Paulo & Belrose 2025, https://arxiv.org/abs/2501.16615 ; reframing: Movva et al. 2025, https://arxiv.org/abs/2506.23845 ; universality: Lan et al. 2024, https://arxiv.org/abs/2410.06981 ; Gemma Scope: https://arxiv.org/abs/2408.05147
	85	+- Substance: SAE probes underperform plain linear probes OOD (GDM, deprioritized); SAE probes lose to logistic regression across 100+ datasets (Kantamneni); prompting/finetuning beat SAE steering, difference-in-means beats SAE dictionaries (AxBench). Counterpoints: dataset-matched SAEs close much of the gap (Kissane — chat-trained SAE recovers the refusal direction Pile-trained SAEs miss); only ~30% of latents replicate across seeds at 131k width on Llama-3-8B (Paulo & Belrose) — a direct replication-rate result modelmap must reproduce; nevertheless significant cross-model SAE-space similarity exists (Lan). Emerging 2025–2026 position: SAEs are hypothesis-generation/discovery tools, not optimal actuators or probes for known concepts. Anthropic doubled down via transcoder-based attribution graphs and continues SAE-based safety auditing (Circuits Updates, June 2026).
	86	+- Epistemic status: Negative downstream results established and replicated; "SAEs are useless" is an overreach — utility-for-discovery contested, exactly what modelmap's replication/causal-confirmation metrics can quantify.
	87	+- Mac note: every experiment in this debate (probing vs SAE probing, steering baselines, seed-replication Jaccard) is L–M on ≤2B models with public SAEs — a realistic early modelmap experiment family.
	88	+
	89	+## Implications for modelmap
	90	+1. SAE training cost is the capture-pipeline design driver (Experiment H must measure streaming activation capture throughput on MLX vs MPS).
	91	+2. Seed replication (~30% latent overlap) is the noise floor Experiment A/D must beat or report.
	92	+3. SAEBench (or a subset) should be ported as the SAE gate; never select SAEs on proxy metrics alone.
	93	+4. mlxterp is direct prior art for the MLX capture layer — study and cite; differentiate via provenance/replication discipline.

added research/notes/4.4_circuits_and_graphs.md +97 −0

		`@@ -0,0 +1,97 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.4 Circuits and computational graphs
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.4 — Circuits and computational graphs
	12	+
	13	+Cost classes: L (laptop-trivial), M (hours + tens of GB), H (multi-day / >64 GB / cluster-typical).
	14	+
	15	+## B1. Induction heads
	16	+- Key paper: Olsson et al. (Anthropic) 2022, https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
	17	+- Claims: Two-head circuit (previous-token + induction head) implements [A][B]…[A]→[B] copying; forms in a phase change coincident with in-context learning.
	18	+- Type: Both — pattern-based detection plus ablations ("medium-confidence" causal per the paper itself).
	19	+- Cost: L. Prefix-matching scores on any model that fits memory; classic first map layer for modelmap (cheap, replicable, cross-model).
	20	+- Implementation: TransformerLens https://github.com/TransformerLensOrg/TransformerLens (MPS works, occasional dtype/op fallbacks); trivially portable to MLX hooks.
	21	+- Failure modes: "Induction head" is behaviorally defined — heads heterogeneous and multifunctional; ICL attribution contested for large models.
	22	+- Epistemic status: Established (most-replicated finding in mech interp).
	23	+
	24	+## B2. IOI circuit and attention-head taxonomies
	25	+- Key papers: IOI: Wang et al. 2022, https://arxiv.org/abs/2211.00593 ; Copy suppression: McDougall et al. 2023, https://arxiv.org/abs/2310.04625 ; Successor heads: Gould et al. 2023, https://arxiv.org/abs/2312.09230
	26	+- Claims: 26-head circuit computes indirect-object identification in GPT-2-small; established faithfulness/completeness/minimality criteria. Copy suppression explains "negative heads" and self-repair. Successor heads recur across families 31M–12B on shared "mod-10" numeric features.
	27	+- Type: Causal (path patching, ablations) + observational weight analysis.
	28	+- Cost: L–M. GPT-2-small path patching laptop-trivial; head-taxonomy scans across a 7B model are M on 32–64 GB.
	29	+- Failure modes: Backup heads / self-repair make ablation-based importance unstable (Hydra effect — McGrath et al. 2023, https://arxiv.org/abs/2307.15771); IOI only ~87% faithful, node-level; taxonomy labels are task-distribution-relative.
	30	+- Epistemic status: Established (IOI heavily replicated cross-model); taxonomy generality contested at margins.
	31	+
	32	+## B3. MLP layers as key–value memories
	33	+- Key papers: Geva et al. 2021, https://arxiv.org/abs/2012.14913 ; Geva et al. 2022, https://arxiv.org/abs/2203.14680
	34	+- Claims: MLP input weights = pattern detectors (keys), output weights = vocabulary-space updates (values); predictions built by additive sub-updates.
	35	+- Cost: L — weight-only plus light forward passes; pairs naturally with Experiment E (weight-space cartography) and logit lens.
	36	+- Implementation: simple to reimplement; baukit https://github.com/davidbau/baukit for hooks. MPS/MLX trivial.
	37	+- Failure modes: Vocabulary projection misleads in early/mid layers; neuron-level keys polysemantic; "memory" metaphor overclaims vs editing critiques (Hase et al. 2023, https://arxiv.org/abs/2301.04213).
	38	+- Epistemic status: Established as a lens; literal KV-memory reading contested.
	39	+
	40	+## B4. Activation patching / attribution patching
	41	+- Key papers: causal tracing: Meng et al. 2022, https://arxiv.org/abs/2202.05262 ; attribution patching: Nanda 2023, https://www.neelnanda.io/mechanistic-interpretability/attribution-patching ; best practices: Heimersheim & Nanda 2024, https://arxiv.org/abs/2404.15255
	42	+- Claims: Swapping activations between clean/corrupted runs localizes causally necessary components; attribution patching approximates all components in 2 forward + 1 backward pass.
	43	+- Cost: Patching scans M at 7B (thousands of forwards); attribution patching L–M — the key trick making circuit scans Mac-feasible, but needs gradients (MLX autodiff or PyTorch-MPS backward; memory ≈ 2–3× inference).
	44	+- Implementations: TransformerLens, nnsight https://github.com/ndif-team/nnsight , pyvene.
	45	+- Failure modes: Corrupted-baseline choice changes conclusions (noising vs denoising, zero vs mean vs resample); gradient approximation fails near zero-gradient (fixed by IG); localization ≠ editability.
	46	+- Epistemic status: Established core method; interpretation pitfalls well-documented.
	47	+
	48	+## B5. Attribution graphs / circuit tracing (Anthropic 2025)
	49	+- Key papers: Ameisen, Lindsey et al., Circuit Tracing, 2025, https://transformer-circuits.pub/2025/attribution-graphs/methods.html ; Lindsey et al., On the Biology of a Large Language Model, 2025, https://transformer-circuits.pub/2025/attribution-graphs/biology.html ; QK follow-up: https://transformer-circuits.pub/2025/attention-qk/index.html
	50	+- Claims: Replace MLPs with cross-layer transcoders → per-prompt linear attribution graph (nodes = features/errors/tokens, edges = direct causal contributions). Revealed planning-in-poetry, multilingual shared circuits, multi-step reasoning, unfaithful CoT, refusal mechanics.
	51	+- Type: Hybrid — graph construction is attributional through a lossy proxy model; hypotheses validated with steering/suppression interventions (the modelmap Level-2→Level-3 pattern).
	52	+- Cost: With pretrained CLTs (Gemma-2-2B, Llama-3.2-1B, Qwen3-4B): M — community reports ~15–40 GB for Gemma-2-2B graphs → 64 GB Mac plausible, 16 GB not. Training own CLTs for 7B: H.
	53	+- Implementations: circuit-tracer https://github.com/safety-research/circuit-tracer (announcement: https://www.anthropic.com/research/open-source-circuit-tracing); Neuronpedia hosts interactive graphs; frontend https://github.com/anthropics/attribution-graphs-frontend . CUDA-oriented; MPS port is a concrete modelmap engineering task.
	54	+- Failure modes: Error nodes are unexplained dark matter; attention patterns frozen (QK side has substantial interference); replacement-model faithfulness varies by prompt; per-prompt graphs need aggregation to become "maps"; many graphs unreadable.
	55	+- Epistemic status: Established methodology with candid limitation reporting; individual findings Level 2–3.
	56	+
	57	+## B6. Automated circuit discovery: ACDC → EAP → EAP-IG → edge pruning
	58	+- Key papers: ACDC: Conmy et al., NeurIPS 2023, https://arxiv.org/abs/2304.14997 ; EAP: Syed, Rager, Conmy 2023, https://arxiv.org/abs/2310.10348 ; EAP-IG: Hanna et al., COLM 2024, https://arxiv.org/abs/2403.17806 ; Edge pruning: Bhaskar et al., NeurIPS 2024, https://arxiv.org/abs/2406.16778
	59	+- Claims: ACDC: greedy edge removal recovers known circuits. EAP: gradient attribution scores all edges in ~3 passes. EAP-IG: integrated gradients fix zero-gradient failure; circuits must be selected by faithfulness, not node-overlap. Edge pruning: continuous masks; half-size circuits at equal faithfulness; scaled to CodeLlama-13B.
	60	+- Cost: ACDC M–H (slow beyond GPT-2). EAP/EAP-IG: L–M — the Mac-feasible default for 0.5B–8B. Edge pruning M–H (~1 GPU-day class; marginal on 64 GB Mac, feasible on M3U96a-class nodes).
	61	+- Implementations: https://github.com/ArthurConmy/Automatic-Circuit-Discovery ; https://github.com/hannamw/EAP-IG ; https://github.com/princeton-nlp/Edge-Pruning . PyTorch; MPS-workable (edge pruning assumes CUDA-scale memory).
	62	+- Epistemic status: Established tooling; "which method is right" contested — see B7.
	63	+
	64	+## B7. Faithfulness metrics for circuits — and their critiques
	65	+- Key papers: Miller, Chughtai, Saunders 2024, https://arxiv.org/abs/2407.08734 ; Shi et al. 2024, https://arxiv.org/abs/2410.13032 ; subspace-patching illusion: Makelov, Lange, Nanda 2023, https://arxiv.org/abs/2311.17030
	66	+- Substance: Measured faithfulness swings wildly with ablation type (zero/mean/resample), direction, token positions, and metric (logit diff vs KL) — the same circuit can look faithful or unfaithful by experimenter choice. Shi et al. formalize equivalence/independence/minimality tests; published circuits pass only partially. Subspace patching can activate dormant pathways → illusory localization.
	67	+- Cost: L–M at ≤2B — running a battery of ablation variants is exactly the charter §8.4 multi-control discipline.
	68	+- Implementations: auto-circuit https://github.com/UFO-101/auto-circuit (efficient multi-ablation); https://github.com/blei-lab/circuitry
	69	+- Consequence for modelmap: report faithfulness curves across ablation schemes, never single numbers.
	70	+- Epistemic status: Critiques established; no consensus null for "faithful enough".
	71	+
	72	+## B8. Sparse feature circuits
	73	+- Key paper: Marks et al., ICLR 2025, https://arxiv.org/abs/2403.19647
	74	+- Claims: Circuits over SAE features are far more interpretable than head/neuron circuits; SHIFT removes spurious signals; unsupervised discovery at scale.
	75	+- Type: Causal — IG attribution patching validated by ablation.
	76	+- Cost: M on Pythia-70M/Gemma-2-2B with public SAEs (Mac-feasible); needs SAEs for every stream, so capture cost multiplies.
	77	+- Implementation: https://github.com/saprmarks/feature-circuits (dictionary_learning + nnsight). MPS-workable.
	78	+- Failure modes: Inherits SAE pathologies — circuits over non-replicating features inherit their fragility.
	79	+- Epistemic status: Established method, actively extended; robustness contested.
	80	+
	81	+## B9. Weight-sparse transformers (OpenAI 2025) — circuits by construction
	82	+- Key paper: Gao, Rajaram et al. 2025, https://arxiv.org/abs/2511.13653 (toolkit: https://github.com/openai/circuit_sparsity)
	83	+- Claims: Training at ~1/1000 weight density yields compact, human-legible, causally-verifiable task circuits.
	84	+- Relevance: applies only to models trained this way — useful to modelmap as a planted-ground-truth benchmark for validating discovery methods, not for mapping existing checkpoints.
	85	+- Epistemic status: Established for tiny models; scaling unresolved.
	86	+
	87	+## 2024–2026 developments
	88	+- Mar 2025: attribution-graph diptych reorients field from head-level to feature-level circuits; May 2025 circuit-tracer open-sourced; Aug 2025 Neuronpedia "Circuits Research Landscape" https://www.neuronpedia.org/graph/info
	89	+- Nov–Dec 2025: OpenAI weight-sparse circuits; 2026 follow-up on interpretable individual parameters (https://arxiv.org/abs/2607.02964)
	90	+- 2026: ADAG — automated attribution-graph description (https://arxiv.org/abs/2604.07615); Circuit Insights — interpretability beyond activations toward weights (https://arxiv.org/abs/2510.14936); Anthropic Circuits Updates May/June 2026 apply features+graphs to safety auditing (https://transformer-circuits.pub/2026/june-update/index.html)
	91	+
	92	+## Implications for modelmap
	93	+1. Induction-head and head-taxonomy maps are the cheapest replicable cross-model map type — good Experiment B/D substrate.
	94	+2. EAP-IG is the default Mac-feasible circuit-discovery method; ACDC only for validation on tiny models.
	95	+3. Faithfulness must be published as curves across ablation schemes (charter §8.4 alignment).
	96	+4. An MPS/MLX port of circuit-tracer (or a minimal reimplementation) is a concrete high-value engineering contribution.
	97	+5. The Hydra effect / self-repair is a first-class threat to any ablation-based localization map — Experiment C must include backup-head detection.

added research/notes/4.5_weight_space_cartography.md +82 −0

		`@@ -0,0 +1,82 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.5 Weight-space cartography (no forward pass required)
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.5 — Weight-space cartography
	12	+
	13	+## A1. LASER — layer-selective rank reduction (SVD spectra as intervention targets)
	14	+- Key paper: Sharma, Ash, Misra, ICLR 2024, https://arxiv.org/abs/2312.13558 ; code https://github.com/pratyushasharma/laser
	15	+- Claims: Replacing individual weight matrices (esp. late-layer MLP down-projections) with low-rank truncations can improve QA accuracy up to 20–30 points.
	16	+- Type: Causal on behavior; correlational on interpretation ("noise in high components").
	17	+- Cost: SVD of 4096×11008 = seconds on M-series CPU (Accelerate/LAPACK); full per-matrix sweep of a 7B = CPU-minutes; behavioral validation is the expensive part (hours).
	18	+- Failure modes: task/matrix-specific gains; (layer, matrix, rank) sweep invites multiple-comparison illusions.
	19	+- Epistemic status: Established phenomenon; contested interpretation.
	20	+
	21	+## A2. Heavy-tailed spectral analysis / WeightWatcher — closest prior art to the weight-only pre-screen hypothesis
	22	+- Key papers: Martin & Mahoney 2019, https://arxiv.org/abs/1901.08278 ; Martin, Peng, Mahoney 2020 (Nature Comms 2021), https://arxiv.org/abs/2002.06716 ; tool: https://github.com/CalculatedContent/WeightWatcher (pip, pure CPU, runs natively on Apple Silicon today)
	23	+- Claims: Power-law exponents ("alpha") of per-matrix spectral densities predict relative test quality across model zoos with zero data, zero forward passes; per-layer alpha profiles diagnose over/under-training.
	24	+- Cost: cheapest technique surveyed — full 7B scan in minutes on any Mac.
	25	+- Failure modes: power-law fitting finicky (fit-range sensitivity); documented Simpson's-paradox failure (https://arxiv.org/abs/2106.00734); predicts trends, not absolute capability; largely untested as a predictor of internal/activation-level structure — that's the modelmap gap.
	26	+- Epistemic status: Established for accuracy-trend prediction; contested at fine granularity.
	27	+
	28	+## A3. Outlier channels, super weights, massive activations, attention sinks
	29	+- Key papers: LLM.int8 outliers: Dettmers et al. 2022, https://arxiv.org/abs/2208.07339 ; massive activations: Sun et al. 2024, https://arxiv.org/abs/2402.17762 ; sinks: Xiao et al. 2023, https://arxiv.org/abs/2309.17453 ; Gu et al. 2024, https://arxiv.org/abs/2410.10781 ; super weights: Yu et al. (Apple) 2024, https://arxiv.org/abs/2411.07191 (https://machinelearning.apple.com/research/the-super-weight)
	30	+- Weight-visible vs activation-only: LLM.int8 outliers and massive activations are activation-level; super weights are weight-visible (extreme entries in specific matrices, typically early-layer `mlp.down_proj`) — a single scalar whose removal collapses perplexity by 3 orders of magnitude and which induces the massive activations. The crucial bridge from weight space to activation space.
	31	+- Cost: weight scan CPU-seconds; verification (zero one weight, measure PPL) minutes on a Mac at 7B 4-bit/MLX.
	32	+- Epistemic status: Established phenomena; causal story settled for super weights, active for sinks (2026: https://arxiv.org/html/2605.08504 ; survey https://arxiv.org/pdf/2604.10098).
	33	+
	34	+## A4. Cross-layer/cross-model similarity (CKA, SVCCA, Procrustes) and CKA's pathologies
	35	+- Key papers: CKA: Kornblith et al. 2019, https://arxiv.org/abs/1905.00414 ; SVCCA: https://arxiv.org/abs/1706.05806 ; pathologies: Davari et al., ICLR 2023, https://arxiv.org/abs/2210.16156 ; Ding, Denain, Steinhardt 2021, https://arxiv.org/abs/2108.01661
	36	+- Substance: CKA dominated by outlier directions, arbitrarily manipulable without functional change; CKA/CCA disagree and fail sanity checks. Interacts badly with A3: LLM CKA scores can be dominated by sink dimensions unless removed. Canonical CKA needs activations; weight-only analogues (Procrustes/subspace angles between weight matrices, 2026: https://arxiv.org/pdf/2601.10266) exist but are less validated.
	37	+- Rule for modelmap: report ≥2 similarity metrics + Ding-style statistical tests; strip outlier dims as a control.
	38	+- Epistemic status: Established tool, debunked-in-part as a standalone measure.
	39	+
	40	+## A5. Layer redundancy maps (similarity-guided depth pruning)
	41	+- Key papers: Gromov et al. 2024, https://arxiv.org/abs/2403.17887 ; ShortGPT: Men et al. 2024, https://arxiv.org/abs/2403.03853
	42	+- Claims: Angular similarity between layer input/output identifies deletable deep-layer blocks — an internal depth-redundancy map. Causal (delete-and-measure).
	43	+- Note: needs a small calibration set (minutes on Mac). A candidate target that weight-only spectra (A2) might predict — explicit modelmap experiment. Caveat: benchmark loss ≠ capability loss (deep-layer deletion selectively harms reasoning even when perplexity survives).
	44	+
	45	+## A6. Model stitching
	46	+- Key papers: Lenc & Vedaldi 2015, https://arxiv.org/abs/1411.5908 ; Bansal, Nakkiran, Barak 2021, https://arxiv.org/abs/2106.07682
	47	+- Claims: low stitching penalty = functionally interchangeable representations — stronger, task-grounded similarity than CKA. Quasi-causal.
	48	+- Cost: train stitching layers — hours per pair on Mac for small LLMs.
	49	+- Failure modes: stitcher capacity confounds (expressive stitcher "repairs" dissimilar reps); direction asymmetry. Mostly vision; LLM stitching thin.
	50	+- Epistemic status: Established in vision; under-explored for LLMs.
	51	+
	52	+## A7. Permutation alignment / Git Re-Basin
	53	+- Key papers: Ainsworth et al., ICLR 2023, https://arxiv.org/abs/2209.04836 ; REPAIR: https://arxiv.org/abs/2211.08403 ; Sinkhorn re-basin: https://arxiv.org/abs/2212.12042 ; 2024: https://arxiv.org/abs/2404.06498
	54	+- Reality check: transformers align poorly — permutation symmetry broken by residual streams, LayerNorm, head structure; unproven at 7B scale; moot for LLMs anyway (nobody retrains from different seeds). The useful LLM analogue is alignment across sizes/families — open (Q3 / Experiment G).
	55	+- Epistemic status: Established for MLPs/ResNets; contested/unproven for LLM transformers.
	56	+
	57	+## A8. Mode connectivity
	58	+- Key papers: Garipov et al. 2018, https://arxiv.org/abs/1802.10026 ; Frankle et al. 2019, https://arxiv.org/abs/1912.05671 ; Entezari conjecture 2021, https://arxiv.org/abs/2110.06296 ; mechanistic mode connectivity: Lubana et al. 2022, https://arxiv.org/abs/2211.08422
	59	+- For modelmap: the cheap artifact is the base↔instruct linear interpolation loss/behavior profile — weight lerp is free, evals are the cost. Loss connectivity ≠ mechanism connectivity.
	60	+
	61	+## A9. Task arithmetic / delta analysis
	62	+- Key papers: task vectors: Ilharco et al., ICLR 2023, https://arxiv.org/abs/2212.04089 (code https://github.com/mlfoundations/task_vectors — device-agnostic state-dict math) ; TIES: https://arxiv.org/abs/2306.01708 ; DARE: https://arxiv.org/abs/2311.03099 (90%+ of delta entries droppable)
	63	+- Claims: θ_ft − θ_base is a manipulable task vector; deltas are highly sparse/redundant. Causal (weight edit → behavior).
	64	+- Cost: weight arithmetic free; fully Mac-feasible 7–14B.
	65	+- ⚑ Gap: delta maps (base vs instruct: per-matrix delta norms, delta effective rank, delta spectra) have no published systematic atlas — an ideal cheap modelmap artifact.
	66	+
	67	+## A10. Model diffing via crosscoders — and 2025 weight-first convergence
	68	+- Key papers: Anthropic crosscoders 2024, https://transformer-circuits.pub/2024/crosscoders/index.html ; sparsity artifacts fix: Minder et al. 2025, https://arxiv.org/abs/2504.02922 ; narrow-finetuning traces: https://arxiv.org/abs/2510.13900 ; cross-architecture (2026): https://arxiv.org/pdf/2602.11729 ; post-mortem: https://www.lesswrong.com/posts/xmpauEXEerzYcJKNm/
	69	+- Cost: heavy (paired activation capture + SAE-style training). Mac: ≤2B pairs feasible with MLX + streamed stores; 7B pairs at the edge of 64 GB (Experiment H territory).
	70	+- Weight-side complements (2025) — direct evidence the field converges on modelmap's weight-first thesis: Watch the Weights https://arxiv.org/pdf/2508.00161 (monitoring fine-tunes from weight diffs alone); Learning to Interpret Weight Differences https://arxiv.org/html/2510.05092v4 ; Steering LMs with Weight Arithmetic https://www.alignmentforum.org/posts/HYTbakdHpxfaCowYp/
	71	+- Epistemic status: Promising but contested; methodology moving.
	72	+
	73	+## A11. MoE router / expert structure
	74	+- Key papers: Mixtral: https://arxiv.org/abs/2401.04088 (routing more syntactic/positional than semantic) ; DeepSeekMoE: https://arxiv.org/abs/2401.06066 ; Myth of Expert Specialization (2026): https://arxiv.org/html/2604.09780v1
	75	+- Substance: router weights are weight-only artifacts (expert similarity, router row norms: CPU-minutes); "expert = topic" readings largely debunked; load-balancing losses confound structure. Structure-of-routers is an open, cheap mapping target. Mixtral 8x7B strains 64 GB except 4-bit.
	76	+
	77	+## A12. Weight-space learning (weights as data) — and the pre-screen novelty check
	78	+- Key sources: Unterthiner et al. 2020, https://arxiv.org/abs/2002.11448 ; Eilertsen et al. 2020, https://arxiv.org/abs/2002.05688 ; Schürholt hyper-representations: https://kschuerholt.github.io/publications/ ; ICLR 2025 Workshop on Weight Space Learning: https://weight-space-learning.github.io/ ; WeightCLIP (2026): https://arxiv.org/html/2607.03551v1
	79	+- Claims: model properties predictable from raw weights; but almost all results on tiny CNNs/ViTs — the LLM regime (few, huge models) breaks the model-zoo assumption.
	80	+
	81	+## ⚑ Novelty check for the core hypothesis (weight-only maps → activation-level properties)
	82	+Partially anticipated but not occupied: WeightWatcher predicts behavioral quality (not internal structure); Super Weight links one weight-visible phenomenon to activations; Watch-the-Weights monitors fine-tuning; weight-space learning predicts global properties; LASER flags behaviorally-loaded components; QK/OV composition scores (Elhage et al. 2021 framework) are weight-only circuit hints. Missing and open after aggressive 2025–2026 search: a systematic study of weight-only statistics predicting probe accuracy, localization, quantization drift, or working sets on the same model. This is Experiment E's claim to novelty.

added research/notes/4.6_knowledge_localization_editing.md +62 −0

		`@@ -0,0 +1,62 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.6 Knowledge localization and editing
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.6 — Knowledge localization and editing
	12	+
	13	+## B1–B2. ROME and MEMIT
	14	+- Key papers: ROME: Meng et al., NeurIPS 2022, https://arxiv.org/abs/2202.05262 (code https://github.com/kmeng01/rome — CUDA-hardcoded in places, no Apple Silicon support); MEMIT: Meng et al., ICLR 2023, https://arxiv.org/abs/2210.07229 (https://github.com/kmeng01/memit)
	15	+- Claims: causal tracing localizes fact recall to mid-layer MLPs at subject tokens; rank-one (ROME) / spread least-squares (MEMIT, 10k edits) weight updates rewrite facts.
	16	+- Cost: tracing minutes/fact on Mac at 7B; edits cheap, but covariance statistics need ~100k Wikipedia forward passes (precomputed stats exist only for GPT-2/GPT-J; 2025 reduction: https://arxiv.org/pdf/2506.04226).
	17	+- Epistemic status: Landmark; localization claim debunked-in-part (B8); editing mechanics established.
	18	+
	19	+## B3–B5. PMET, EMMET, AlphaEdit
	20	+- PMET: Li et al., AAAI 2024, https://arxiv.org/abs/2308.08742 — writes only FFN weights; its finding that MHSA weights need no update is itself interpretability evidence (aligned with B7).
	21	+- EMMET: Gupta et al., EMNLP Findings 2024, https://arxiv.org/abs/2403.14236 — unifies ROME/MEMIT as one preservation–memorization objective; useful theory for an MLX reimplementation.
	22	+- AlphaEdit: Fang et al., ICLR 2025 Outstanding Paper, https://arxiv.org/abs/2410.02355 (code https://github.com/jianghoucheng/alphaedit) — null-space projection onto preserved-knowledge keys; +36.7% over locate-then-edit baselines; mitigates sequential collapse; one-line projection + one SVD. 2026 reproducibility study broadly confirms but tempers (https://arxiv.org/pdf/2606.26783). Successor: UltraEdit (2025), https://arxiv.org/abs/2505.14679
	23	+- Epistemic status: Established SOTA line, still fresh.
	24	+
	25	+## B6. Knowledge neurons
	26	+- Key paper: Dai et al., ACL 2022, https://arxiv.org/abs/2104.08696
	27	+- Critiques: neurons express token/routing patterns rather than stored knowledge; language-independent and degenerate (redundant) knowledge neurons undermine one-fact-one-neuron (https://arxiv.org/html/2308.13198); attribution pitfalls (EMNLP 2024, https://arxiv.org/abs/2312.12141).
	28	+- Epistemic status: Debunked-in-part; historically important. Mac-feasible to reproduce.
	29	+
	30	+## B7. Storage vs retrieval — the current consensus scaffold
	31	+- Key papers: Geva et al., EMNLP 2023, https://arxiv.org/abs/2304.14767 (three-stage recall: early-MLP subject enrichment → relation propagation → attribute extraction by attention heads; causal via attention knockouts); attribute lens / linear relation decoding: Hernandez et al., ICLR 2024, https://arxiv.org/abs/2308.09124 (~48% of relations well-fit by a single affine map — linearity NOT universal); additive mechanisms: Chughtai, Cooney, Nanda 2024, https://arxiv.org/abs/2402.07321 (recall = sum of qualitatively different mechanisms — complicates clean localization).
	32	+- Consensus: MLPs store, attention retrieves — established; details (additivity, linearity coverage) contested.
	33	+- Cost: all Mac-feasible ≤7B.
	34	+
	35	+## B8. The localization-vs-editing critique — central calibration datum for Q1
	36	+- Key paper: Hase et al., NeurIPS 2023, https://arxiv.org/abs/2301.04213
	37	+- Substance: causal-tracing localization does not predict which layer edits best; edit success near-uniform across layers; tracing effects explain a tiny fraction of edit-success variance. "Edit success" cannot be used as evidence of localization — design Experiment C accordingly.
	38	+- Related: Rebuilding ROME (collapse partly implementation asymmetry, https://arxiv.org/abs/2403.07175); Should We Really Edit Language Models? (evaluation misleading, https://arxiv.org/pdf/2410.18785)
	39	+- Epistemic status: Established — a causal audit of a causal method. Reproduction Mac-feasible on GPT-2-XL.
	40	+
	41	+## B9. Ripple effects
	42	+- Key paper: Cohen et al., TACL 2024, https://arxiv.org/abs/2307.12976 — editors fail to propagate logical consequences (multi-hop, sibling, aggregation); in-context editing beats parametric editors. 2025–2026 follow-ups: ChainEdit, RippleCOT, https://arxiv.org/pdf/2606.01610
	43	+- Epistemic status: Established.
	44	+
	45	+## B10. Sequential-editing collapse and general-ability damage
	46	+- Key papers: Gupta et al. 2024, https://arxiv.org/abs/2401.07453 (gradual forgetting → abrupt collapse); Gu et al., EMNLP 2024, https://arxiv.org/abs/2401.04700 (few edits degrade reasoning/NLI/QA); 2025 energy-regularized line: https://arxiv.org/html/2510.01172v1
	47	+- Consequence: central caveat for any "editing as causal verification" protocol in modelmap.
	48	+
	49	+## B11. Multilingual knowledge sharing and cross-lingual editing
	50	+- Key papers: cross-lingual editing: https://arxiv.org/abs/2309.08952 ; language-independent neurons: https://arxiv.org/html/2308.13198 ; MPN: https://arxiv.org/abs/2401.03190 ; language-agnostic factual neurons: https://arxiv.org/abs/2406.16416 ; multilingual recall: https://arxiv.org/abs/2410.14387 ; multilingual pipeline replication (2025): https://arxiv.org/html/2505.20546v2 ; survey (EMNLP 2025): https://arxiv.org/html/2505.14393
	51	+- Substance: monolingual edits transfer poorly; a shared language-agnostic substrate exists (English-centric hub behavior common); Geva pipeline partially replicates multilingually with language-specific extraction stages. Directly matches modelmap's FR/EN promptset plans (Experiment B).
	52	+- Epistemic status: Sharing established qualitatively; degree and mechanism contested.
	53	+
	54	+## B12. Tooling and Apple Silicon reality check
	55	+- EasyEdit https://github.com/zjunlp/EasyEdit (ACL 2024) unifies ROME/MEMIT/PMET/AlphaEdit + KnowEdit benchmark; surveys: https://arxiv.org/abs/2310.16218 , https://arxiv.org/pdf/2401.01286
	56	+- All editing repos are CUDA-first; none advertises MPS support. The math is small-matrix linear algebra — an MLX/MPS reimplementation is tractable and itself a publishable modelmap artifact ("knowledge editing on consumer Macs"). The only expensive step is covariance precomputation (hours of forward passes at 7B on 64 GB).
	57	+
	58	+## Implications for modelmap
	59	+1. Never infer localization from edit success (Hase). Experiment C's correlational→causal survival rate must use patching/ablation, with editing as a separate axis.
	60	+2. The storage-vs-retrieval scaffold (B7) gives concrete map types: subject-enrichment maps (MLP), extraction maps (attention heads).
	61	+3. Multilingual localization atlas (FR/EN) has solid prior art to build on and clear open questions.
	62	+4. MLX knowledge-editing port = concrete tooling contribution.

added research/notes/4.7_cross_model_comparison.md +72 −0

		`@@ -0,0 +1,72 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.7 Cross-model and cross-scale comparison
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.7 — Cross-model and cross-scale comparison
	12	+
	13	+## A1. CKA / SVCCA / RSA as similarity metrics for LLMs
	14	+- Key papers: SVCCA: https://arxiv.org/abs/1706.05806 ; CKA: https://arxiv.org/abs/1905.00414 ; survey: Klabunde et al. 2023 (rev. 2025), https://arxiv.org/abs/2305.06329 ; ReSi benchmark (24 measures, ICLR 2025): https://arxiv.org/abs/2408.00531 ; LLM-specific: https://arxiv.org/abs/2312.02730
	15	+- Pathologies: outlier-direction sensitivity (Davari et al., https://arxiv.org/abs/2210.16156); score saturation for LLM pairs (most non-Llama pairs >0.8); naive estimator drifts to 1 as feature/sample ratio grows → false alignment discoveries. ReSi: different measures rank model pairs inconsistently — metric choice is itself a confounder, must be part of provenance.
	16	+- Cost: cheap (minibatch-CKA on cached activations, NumPy/Accelerate/MLX). Fully feasible 0.5B–14B.
	17	+- Epistemic status: Established tool; contested as evidence — every convergence claim needs baseline calibration (A6).
	18	+
	19	+## A2. Universal neurons across seeds (Gurnee et al.)
	20	+- Key papers: https://arxiv.org/abs/2401.12181 (2024); follow-up with ablation-based functional impact (2025): https://arxiv.org/abs/2508.00903
	21	+- Claims: only 1–5% of MLP neurons are universal across independently seeded GPT-2 runs (activation correlation over 100M tokens); these are disproportionately interpretable (unigram, alphabet, position, entropy-regulation families). Negative half: ~95% of neurons are seed-specific — single-neuron catalogs are mostly noise.
	22	+- Methodology template for modelmap: correlation-over-corpus → taxonomize → ablate.
	23	+- Cost: paired capture over ~10⁸ tokens — heavy but streamable; feasible ≤1B pairs on 32–64 GB with disk-backed store (expH regime).
	24	+- Epistemic status: Established (replicated, causally confirmed).
	25	+
	26	+## A3. Universality of induction heads
	27	+- Key papers: Olsson et al. 2022, https://arxiv.org/abs/2209.11895 ; ICL essentiality: https://arxiv.org/abs/2407.07011 ; cross-architecture (Transformer vs Mamba): https://arxiv.org/abs/2410.06672 ; https://arxiv.org/abs/2411.07071
	28	+- Substance: strongest known case of circuit universality — appears in essentially every transformer LM, and analogues in Mamba. A natural "known landmark" for calibrating any cross-model coordinate system: if an alignment method can't match induction heads across two models, it fails the sanity check.
	29	+- Cost: cheap — one forward pass with attention capture; feasible to 32B quantized via MLX.
	30	+- Epistemic status: Established (multi-team, cross-architecture).
	31	+
	32	+## A4. Cross-scale circuit evolution
	33	+- Key papers: Lieberum et al. 2023 (Chinchilla 70B), https://arxiv.org/abs/2307.09458 ; Tigges et al. 2024 (Pythia 70M–2.8B across checkpoints): https://arxiv.org/abs/2407.10827
	34	+- Substance: circuit-analysis methods scale to 70B but head-level semantics get muddier; task circuits emerge at consistent token budgets and algorithms stay roughly stable while component identities drift — supports longitudinal/cross-scale maps (Experiment G). The Tigges regime (70M–2.8B, full patching) is comfortably Mac-feasible — the right scale band for expG.
	35	+- Epistemic status: Established small-scale; partially established at 70B.
	36	+
	37	+## A5. Feature universality via SAEs; universal SAEs; crosscoder diffing
	38	+- Key papers: Lan et al. 2024/2025, https://arxiv.org/abs/2410.06981 ; universal SAEs: Thasarathan et al. 2025, https://arxiv.org/abs/2502.03714 ; Anthropic model diffing: https://transformer-circuits.pub/2024/model-diffing/index.html , https://transformer-circuits.pub/2025/crosscoder-diffing-update/index.html ; sparsity artifacts: Minder et al. 2025, https://arxiv.org/abs/2504.02922 ; open replication: https://www.lesswrong.com/posts/srt6JXsRMtmqAJavD/
	39	+- Substance: moderate feature-space overlap across families, strongest mid-layers and concrete concepts. Caveat chain: SAE features are seed-unstable (~30% overlap — see §4.8); crosscoder "model-exclusive" latents partly sparsity artifacts. Any cross-model feature atlas must control for both.
	40	+- Cost: SAE training is the expensive item (10⁸–10⁹ activations/layer); ≤3B feasible; crosscoders double storage. SAELens MPS throughput unbenchmarked — genuine expH deliverable.
	41	+- Epistemic status: Contested/emerging — real signal, magnitude method- and seed-dependent.
	42	+
	43	+## A6. Platonic Representation Hypothesis and its deflation
	44	+- Key papers: Huh et al., ICML 2024, https://arxiv.org/abs/2405.07987 ; linear-net proof: https://arxiv.org/abs/2507.01098 ; Aristotelian critique (2026): https://arxiv.org/abs/2602.14486 ; causes/consequences: https://arxiv.org/abs/2505.13899
	45	+- The load-bearing negative result: after calibrating for width/depth confounders, global convergence metrics largely evaporate (partly a look-elsewhere effect from reporting max similarity over many layer pairs). What survives is local neighborhood/topological alignment.
	46	+- Practical rule for the atlas: cross-model claims use calibrated, local metrics with null baselines from random/width-matched models.
	47	+- Epistemic status: Contested; global-metric version debunked-in-part (2026); local version live.
	48	+
	49	+## A7. Model stitching as functional similarity
	50	+- Key papers: Bansal et al. 2021, https://arxiv.org/abs/2106.07682 ; https://arxiv.org/abs/2303.11277 ; "How Not to Stitch" (2024): https://arxiv.org/abs/2412.11299 ; relative representations line: https://arxiv.org/abs/2406.11014
	51	+- Substance: low CKA can stitch perfectly and vice versa; results depend heavily on connector expressivity/protocol (affine stitch can "succeed" by relearning); LLM pairs collide with tokenizer mismatch (A9).
	52	+- Epistemic status: Established as probe; contested as measure.
	53	+
	54	+## A8. Rosetta Neurons (vision precedent)
	55	+- Key paper: Dravid et al., ICCV 2023, https://arxiv.org/abs/2306.09346 (code https://github.com/yossigandelsman/rosetta_neurons)
	56	+- Substance: activation-correlation mining finds shared-concept units across 8 heterogeneous vision models without supervision. The matching algorithm (normalized cross-correlation over a shared stimulus set) transfers directly to LLMs with a shared prompt corpus — a concrete candidate for modelmap's cross-model coordinate system. Untested at LLM scale (gap).
	57	+
	58	+## A9. Tokenizer effects on comparability
	59	+- Key papers: TokSuite (2025, models identical except tokenizer): https://arxiv.org/pdf/2512.20757 ; cross-tokenizer distillation: https://arxiv.org/abs/2503.20083 ; tokenisation NP-complete: https://arxiv.org/abs/2412.15210
	60	+- Substance: no standard solution for aligning activations across tokenizers; workarounds (shared-substring boundaries, word/byte pooling, prompt-level mean pooling) each destroy information. Rule: within-family cross-scale comparison (shared tokenizer) is the safe default; cross-family maps must document the alignment convention in the map schema.
	61	+- Epistemic status: Established confound; unsolved methodology (gap).
	62	+
	63	+## A10. Quantization effects on internal representations — the thin shelf (project bet confirmed)
	64	+As of 2026-08-12 this literature is ~5 papers, recent, shallow — no systematic multi-technique internal atlas across quantization levels:
	65	+- Interpreting the Effects of Quantization on LLMs (IJCNLP-AACL 2025): https://arxiv.org/abs/2508.16785 — 4/8-bit vs FP16; breadth-first, small depth.
	66	+- Through a Compressed Lens (2025): https://arxiv.org/abs/2505.13963 — GPTQ/AWQ on Qwen2.5 7B–72B; effects inconsistent in direction, dependent on method × technique × protocol.
	67	+- How Quantization Changes Interpretable Features: An SAE Analysis (2026): https://arxiv.org/html/2606.03002v1 — first SAE-feature-drift-under-quantization study; closest existing work to expF; single-method, worth replicating and extending.
	68	+- The Illusion of Equivalency (2026): https://arxiv.org/pdf/2607.08734 — benchmark-equivalence masks internal distributional divergence.
	69	+- Behavioral-only adjacent: long-context degradation https://arxiv.org/pdf/2505.20276
	70	+
	71	+Verified absent: probe-drift-across-Q-levels with controls; circuit survival under quantization; Q2/Q3 extreme-quantization internals; GGUF/llama.cpp-runtime introspection; weight-geometry-predicts-quantization-sensitivity. expF sits in genuinely open territory — and quantized models are precisely what runs on the target hardware. MLX ships 4/8-bit natively; FP16/Q8/Q4 triplets fit sequentially in 32–64 GB up to ~14B.
	72	+- Epistemic status: Emerging; the field's own words ("inconsistent", "illusion of equivalency") mark it unsettled.

added research/notes/4.8_statistics_epistemology.md +81 −0

		`@@ -0,0 +1,81 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.8 Statistics and epistemology of interpretability
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.8 — Statistics and epistemology of interpretability
	12	+
	13	+## B1. The BERT interpretability illusion (dataset sensitivity)
	14	+- Key paper: Bolukbasi et al. 2021, https://arxiv.org/abs/2104.07143 — the same neuron looks monosemantic on one dataset and encodes a different "concept" on another; top-activating-example analysis systematically over-reads.
	15	+- Consequence (already in charter): every unit-level claim needs ≥2 prompt sets; atlas entries report per-dataset agreement.
	16	+- Status: Established.
	17	+
	18	+## B2. Subspace-patching illusions — Makelov vs Wu (unresolved)
	19	+- Key papers: Makelov, Lange, Nanda 2023, https://arxiv.org/abs/2311.17030 (subspace patching can activate dormant parallel pathways — interventional success ≠ correct localization) vs Wu et al. reply 2024, https://arxiv.org/abs/2401.12631 (the illusion criterion also rejects intuitively correct explanations).
	20	+- Common ground: interchange-intervention success alone under-determines mechanism. Rule: Level-3 labels record which intervention family and which basis; DAS-style optimized subspaces get a distinct flag.
	21	+- Status: Contested (productive open debate).
	22	+
	23	+## B3. Illusions from simplified proxies (SVD, clustering, low-rank views)
	24	+- Key paper: Friedman et al., ICML 2024, https://arxiv.org/abs/2312.03656 — simplifications matching the model in-distribution diverge OOD; the proxy captures ID behavior, not mechanism.
	25	+- Direct warning for expE: SVD/effective-rank maps are hypotheses, not mechanisms — every geometry map gets an OOD promptset check.
	26	+- Status: Established.
	27	+
	28	+## B4. Methodological sensitivity of activation patching
	29	+- Key papers: Zhang & Nanda, ICLR 2024, https://arxiv.org/abs/2309.16042 ; Heimersheim & Nanda 2024, https://arxiv.org/abs/2404.15255
	30	+- Substance: results flip with corruption type (Gaussian vs symmetric counterfactual), metric (prob vs logit-diff vs KL), window, position. ROME-style Gaussian corruption inflates mid-layer MLP localization. The closest thing to a pre-registration standard: fix corruption + metric before scanning — methodology.md pins these per map type.
	31	+- Status: Established.
	32	+
	33	+## B5. Faithfulness vs plausibility
	34	+- Key paper: Jacovi & Goldberg, ACL 2020, https://arxiv.org/abs/2004.03685 — the vocabulary underlying modelmap's confidence taxonomy; plausibility (human appeal) must never borrow causal authority. Visualization design (§4.10) must respect this firewall.
	35	+- Status: Established (canonical).
	36	+
	37	+## B6. Probing controls, selectivity
	38	+- Key papers: Hewitt & Liang, EMNLP 2019, https://arxiv.org/abs/1909.03368 ; Belinkov, CL 2022, https://arxiv.org/abs/2102.12452
	39	+- Substance: the mandatory-controls doctrine expA implements — shuffled-label nulls, capacity-matched probes, multi-dataset replication, selectivity over raw accuracy.
	40	+- Status: Established.
	41	+
	42	+## B7. Seed sensitivity of SAEs — the noise-floor number
	43	+- Key paper: Paulo & Belrose 2025, https://arxiv.org/abs/2501.16615 — SAEs differing only in seed share as little as ~30% of features (131k latents, Llama-3-8B); TopK more seed-dependent than ReLU+L1.
	44	+- Consequence: cross-seed overlap is the noise floor that cross-model universality claims must beat. Combined with 1–5% neuron universality (Gurnee), it makes modelmap's replication-rate metric first-class. Replicating at 0.5–1B scale is feasible and would be a strong early result.
	45	+- Status: Established (multi-LLM, multi-architecture).
	46	+
	47	+## B8. Multiple comparisons — interpretability as statistical inference
	48	+- Key sources: The Dead Salmons of AI Interpretability (2025), https://arxiv.org/abs/2512.18792 — attribution, probing, SAEs and causal analyses yield plausible "explanations" of randomly initialized networks; reframe interpretability as statistical inference (uncertainty, explicit alternatives, multiplicity control). Historical anchor: Bennett et al. 2009, https://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf
	49	+- Consequence: random-init baselines non-negotiable (charter §8.4); add per-map FDR control to src/modelmap/stats.
	50	+- Status: Emerging position; underlying statistics established.
	51	+
	52	+## B9. Ground-truth benchmarks
	53	+- Tracr (compiled transformers, exact circuits): https://arxiv.org/abs/2301.05062 , code https://github.com/google-deepmind/tracr (JAX, CPU-runnable on Mac)
	54	+- InterpBench (17 semi-synthetic transformers via Strict IIT, NeurIPS 2024): https://arxiv.org/abs/2407.14494
	55	+- RAVEL (attribute disentanglement, intervention-based): https://arxiv.org/abs/2402.17700
	56	+- MIB — Mechanistic Interpretability Benchmark (ICML 2025): https://arxiv.org/abs/2504.13151 — two tracks (circuit localization; causal-variable localization), 4 tasks × 5 models; attribution/mask-optimization best for circuits; supervised DAS beats SAEs on causal variables; BlackboxNLP 2025 shared task https://arxiv.org/html/2510.06811v1
	57	+- Consequence: validate every modelmap pipeline on Tracr/InterpBench first (tiny, CPU/MPS-feasible), report recovery rates, then apply to real models. MIB pairs are within Mac budget.
	58	+- Status: Established, rapidly standardizing.
	59	+
	60	+## B10. Hypothesis testing for circuits
	61	+- Key papers: Shi et al., NeurIPS 2024, https://arxiv.org/abs/2410.13032 (code https://github.com/blei-lab/circuitry) — formal equivalence/independence/minimality tests; published circuits pass only partially, Tracr circuits pass fully. Hanna et al. 2024, https://arxiv.org/abs/2403.17806 (faithfulness > overlap).
	62	+- Consequence: run circuitry-style tests on every promoted circuit; report pass/fail per criterion in confidence.md — a ready-made harness for the causal confirmation rate.
	63	+- Status: Established methodology.
	64	+
	65	+## B11. Causal abstraction — the formal framework and its limit
	66	+- Key papers: Geiger et al., JMLR 2025, https://arxiv.org/abs/2301.04709 — unifies patching/scrubbing/circuits/SAEs/steering as causal abstraction; Non-Linear Representation Dilemma (2025): https://arxiv.org/abs/2507.08802 — with arbitrary nonlinear featurizers, any model "abstracts" any algorithm (accuracy ~1 even for random networks); causal abstraction is only meaningful jointly with a constrained featurizer class.
	67	+- Consequence: confidence.md records the featurizer class (linear / orthogonal / DAS-optimized) for every causal claim.
	68	+- Status: Framework established; sufficiency contested.
	69	+
	70	+## B12. Calls for rigor (2023–2026)
	71	+- Saphra & Wiegreffe, Mechanistic? (BlackboxNLP 2024): https://arxiv.org/abs/2410.09087 — four senses of "mechanistic"; state which one is used (modelmap: the narrow causal one).
	72	+- Casper, The Engineer's Interpretability Sequence (2023): https://www.alignmentforum.org/s/a6ne2ve5uturEEQK7
	73	+- Sharkey et al., Open Problems in Mechanistic Interpretability (2025): https://arxiv.org/abs/2501.16496 — lists validation/method-evaluation gaps modelmap targets.
	74	+- Mechanistic Interpretability Needs Philosophy (2025): https://arxiv.org/abs/2506.18852 ; Make Mechanistic Interpretability Auditable (ACL 2026): https://arxiv.org/pdf/2606.00033 — auditability guidelines are close cousins of modelmap's provenance/confidence schema.
	75	+- Status: Converging consensus: ground-truth benchmarks + statistical controls + causal verification — exactly the charter's bar.
	76	+
	77	+## Synthesis for modelmap
	78	+1. Noise floors first (expA): selectivity, ≥2 datasets, ≥3–5 seeds, random-init baselines, FDR control.
	79	+2. Pipeline validation on Tracr/InterpBench before any real-model map.
	80	+3. confidence.md schema additions: intervention family, basis/featurizer class, ablation-scheme curves, per-dataset agreement.
	81	+4. The 30%-SAE-seed-overlap and 1–5%-neuron-universality numbers are the published calibration points our replication-rate metric extends.

added research/notes/4.9_systems_local_mapping.md +59 −0

		`@@ -0,0 +1,59 @@`
	1	+---
	2	+project: modelmap
	3	+document: Phase 1 notes — §4.9 Systems side — local mapping at scale on macOS
	4	+author: Simon-Pierre Boucher
	5	+contact: contact@spboucher.ai
	6	+website: https://modelmap.io
	7	+created: 2026-08-12
	8	+status: draft
	9	+---
	10	+
	11	+# §4.9 — Systems side of local interpretability at scale
	12	+
	13	+## Activation storage size math (framing)
	14	+Llama-class 7–8B (d_model 4096, 32 layers): fp16 residual capture = 8 KB/token/layer; all layers = 256 KB/token. MLP hidden (d_mlp 14336) is 3.5× larger per layer.
	15	+- 10M tokens × 1 layer ≈ 80 GB — cacheable on a 1–2 TB SSD for one or two hookpoints.
	16	+- SAE budget (500M tokens × 1 layer) ≈ 4 TB fp16 — pre-caching infeasible; on-the-fly generation mandatory.
	17	+- Full-model capture for 100k tokens ≈ 25–90 GB — fine for probing/patching corpora (10⁴–10⁶ tokens).
	18	+The Mac regime splits cleanly: small causal/probing corpora → cache everything; SAE-scale corpora → stream, never store.
	19	+
	20	+## Storage formats
	21	+- safetensors https://github.com/safetensors/safetensors — zero-copy mmap, lazy per-tensor; no appendable writes, no chunk index; free-form metadata dict (exploit for provenance embedding).
	22	+- zarr https://github.com/zarr-developers/zarr-python — chunked, compressed, appendable N-D; ideal shape for token×layer×d_model with random chunk access; but dense fp16 activations are high-entropy → compression gains modest; reported slower than h5py/npy raw throughput. Benchmark on macOS, don't assume.
	23	+- HDF5 — mature; single-writer pain; little advantage over zarr/mmap here.
	24	+- Raw mmap / np.memmap — best random-access latency (the pattern SAE shuffling needs); on unified memory + Apple NVMe likely the throughput winner; needs external index + own content hashes. No published macOS numbers exist for any of these — Experiment H claims this gap.
	25	+
	26	+## SAE training pipelines (streaming on limited memory)
	27	+- SAELens https://github.com/decoderesearch/SAELens — on-the-fly buffer shuffle (approximate); optional CacheActivationsRunner (safetensors shards); buffer size = the RAM knob; no provenance schema for caches; shuffle quality is a replication variable modelmap should control.
	28	+- EleutherAI sparsify https://github.com/EleutherAI/sparsify — TopK SAEs/transcoders with no activation caching at all; bitsandbytes 8-bit path is CUDA-only (useless on MPS).
	29	+- Anthropic (disclosed): collect billions → full on-disk shuffle → stream (https://transformer-circuits.pub/2024/scaling-monosemanticity/). The gold standard a Mac cannot replicate. Open testable question: how much does buffer-shuffled streaming degrade feature replication vs full shuffles? Testable locally at 0.5–3B.
	30	+- Budget: (weights + KV cache + buffer + SAE) < unified memory. 3B fp16 + 2 GB buffer + 131k SAE fits 16 GB; 8B needs 32 GB+.
	31	+
	32	+## Hooks and capture paths
	33	+- PyTorch MPS: forward hooks fire normally (eager); costs: forced syncs on `.cpu()` copies, operator gaps (`PYTORCH_ENABLE_MPS_FALLBACK=1` silently round-trips via CPU), no float64, `device_map="auto"` CPU offload does not work on MPS — model must fit unified memory. fp16 numerics on MPS can differ from CUDA — a confound when comparing to published results. Hook overhead unmeasured in literature (Experiment H).
	34	+- TransformerLens https://github.com/TransformerLensOrg/TransformerLens — HookPoints everywhere; MPS opt-in; materializes all hook activations eagerly (memory-hungry); HF checkpoints only, no GGUF.
	35	+- MLX / mlx_lm: lazy evaluation (https://ml-explore.github.io/mlx/build/html/usage/lazy_evaluation.html) — intermediates can be retained for free during graph construction; early materialization wrecks throughput; no built-in hook API — introspection = wrapping modules (plain Python, easy). MLX runs 4-bit quantized models natively → the only realistic path to quantized-model activation capture with a Python API (llama.cpp aside). M5 neural-accelerator work: https://machinelearning.apple.com/research/exploring-llms-mlx-m5
	36	+- mlxterp https://github.com/coairesearch/mlxterp — the one existing MLX mechinterp library: `model.trace()` context manager, ~196 activations/forward, composable interventions, mlx-lm model support. Very early (~11 stars); no streaming store, no SAE training, no provenance, no overhead benchmarks. Direct prior art for modelmap's capture layer — evaluate before building.
	37	+
	38	+## Quantized-runtime introspection (llama.cpp)
	39	+- eval-callback https://github.com/ggml-org/llama.cpp/tree/master/examples/eval-callback — ggml graph callback intercepting every op; can dump every intermediate tensor during inference including the Metal backend — real C-level activation extraction from actually-quantized compute.
	40	+- imatrix — per-channel mean-squared activation statistics guiding quantization; itself a crude weight-importance map.
	41	+- Verified absent: no interpretability tooling wraps this. A thin exporter (eval-callback → mmap store) would give Q8/Q4/Q2 activation capture no Python framework offers — key for Experiment F. Caveats: dequantized dumps op-by-op have real overhead; no token bookkeeping; graph names shift across versions — pin a commit.
	42	+
	43	+## Metal capture & profiling
	44	+- `MTLCaptureManager` (https://developer.apple.com/documentation/metal/mtlcapturemanager) records `.gputrace` (30k+ buffers per forward) — a debugging tool, not a data pipeline. Instruments' Metal System Trace for kernel timings/occupancy → Experiment H instrumentation, and for validating that Python-level capture matches GPU reality.
	45	+
	46	+## The competing philosophy: NNsight + NDIF
	47	+- https://arxiv.org/abs/2407.14561 (ICLR 2025); https://ndif.us/ ; European replication eDIF: https://arxiv.org/pdf/2508.10553
	48	+- Deferred-execution intervention graphs shipped to a shared GPU fabric — don't run locally; send the experiment away. modelmap's antithesis and best contrast case: NDIF serves unquantized checkpoints and cannot study the quantized runtimes people actually use locally; reproducibility depends on a remote service. NNsight's deferred-graph idiom is conceptually close to MLX laziness — study for API design.
	49	+
	50	+## Disk/IO on macOS
	51	+- APFS: native sparse files (pre-sized mmap stores written out of order allocate nothing); copy-on-write clones = instant snapshot of an activation store — a genuinely nice provenance primitive; metadata-only checksums (data corruption NOT detected — store own content hashes). Refs: https://eclecticlight.co/2024/06/08/apfs-how-sparse-files-work/ ; https://eclecticlight.co/2021/03/29/sparse-files-are-common-in-apfs/
	52	+- fio-style numbers for mmap workloads on Apple NVMe essentially unpublished — measure, don't import Linux ext4 folklore. (Sister project localvm-research already measured cold-cache NVMe: ~13.1 GB/s ceiling at ≥256 KiB blocks, QD≥4 — reuse.)
	53	+
	54	+## Verified open gaps (modelmap's to claim)
	55	+1. No published MLX-vs-MPS hook-overhead benchmarks.
	56	+2. No macOS activation-store format benchmarks (mmap vs zarr vs safetensors on APFS).
	57	+3. No quantized-runtime interpretability tooling (llama.cpp eval-callback unwrapped).
	58	+4. Buffer-shuffle vs full-shuffle effect on SAE feature replication unpublished.
	59	+5. 2026 systems-side interest appearing (https://arxiv.org/pdf/2605.11093 — model-internal observability for LLM inference) — move fast.
	60