SPB Git

spb/modelmap Public License

Internal cartography of local LLMs on Apple Silicon — registered, gated, negative-first. Public atlas at modelmap.io.

🔗 https://modelmap.io
interpretabilityllmapple-siliconmlxreproducibility
Python 66.3% JavaScript 24.5% CSS 8.1% Shell 0.7%

Phase 2: state-of-the-art map (charter §5)

~40 techniques, six families + cross-cutting instruments, eleven-field
blocks with epistemic flags; overlap analysis, already-tried combinations,
novelty traps, estimated Mac cost frontier (16/32/64 GB).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed 5 h ago (Aug 12, 2026) parent f7ae38f

Showing 2 changed files with +445 and −0

modified research/LOG.md +35 −0
@@ -88,3 +88,38 @@ these techniques into the charter §5 table with epistemic status per
88 88 technique, then Phase 3 gap generation (≥20 candidates). Validation-first
89 89 rule adopted: every future pipeline is validated on Tracr/InterpBench-style
90 90 ground truth before touching a real model.
91 +
92 +---
93 +
94 +## 2026-08-12 02:45 EDT — Phase 2: state-of-the-art map written
95 +
96 +**Question.** Organize Phase 1 into the charter §5 taxonomy: what does each
97 +technique actually measure, at what cost on a Mac, with what failure modes
98 +and epistemic status — and where are the overlaps and novelty traps?
99 +
100 +**Action.** Wrote `research/state_of_the_art.md`: ~40 techniques across six
101 +families + cross-cutting instruments, each with the eleven-field block
102 +(condensed), epistemic flags (established / contested / debunked-in-part),
103 +an overlap analysis (6 "secretly the same thing" identities), a list of
104 +already-tried combinations, a novelty-trap list, and an estimated Mac cost
105 +frontier table (16/32/64 GB) that Experiment H must convert to measured.
106 +
107 +**Key synthesis judgments (recorded, not yet evidence):**
108 +- Screening/verification split adopted: attribution patching (EAP-IG) screens,
109 + real patching verifies; auto-interp labels never exceed Level 1 without
110 + intervention scoring.
111 +- One-estimator-family insight: SAE/transcoder/crosscoder/CLT share the
112 + seed-replication pathology — replication findings transfer across them.
113 +- CKA-class metrics are never standalone evidence: ≥2 metrics + nulls +
114 + outlier-stripped variants required.
115 +- The atlas map-card schema fields fall out of Phase 2: pinned patching
116 + protocol, ablation-scheme curves, featurizer class, seed replication rate,
117 + per-dataset agreement.
118 +
119 +**Decision.** Proceed to Phase 3 (`research/research_gaps.md`): ≥20
120 +substantially different candidate directions, each with the six-field gap
121 +block and a smallest falsifying experiment runnable on a Mac. Seeds from
122 +Phase 2: quantization × internals; weight-only pre-screens vs activation
123 +ground truth; replication-rate-first artifacts; quantized-runtime capture;
124 +provenance-first atlas schema; cross-model coordinate systems anchored on
125 +induction heads.
added research/state_of_the_art.md +410 −0
@@ -0,0 +1,410 @@
1 +---
2 +project: modelmap
3 +document: State of the art — technique taxonomy (Phase 2)
4 +author: Simon-Pierre Boucher
5 +contact: contact@spboucher.ai
6 +website: https://modelmap.io
7 +created: 2026-08-12
8 +modified: 2026-08-12
9 +status: reviewed
10 +---
11 +
12 +# State of the art — what can currently be measured inside an open-weight LLM
13 +
14 +Phase 2 deliverable (charter §5). Built from the Phase 1 sweep
15 +(`research/notes/4.1–4.10`, `research/bibliography.md`, ~200 sources accessed
16 +2026-08-12). Each technique carries the charter's eleven-field block, condensed.
17 +Citations live in the notes and bibliography; this document is the synthesis.
18 +
19 +**Cost classes.** L = laptop-trivial (minutes, fits any 16 GB Mac).
20 +M = hours + tens of GB of activations (32–64 GB Mac, careful engineering).
21 +H = multi-day / >64 GB working sets / cluster-typical (out of local scope or marginal).
22 +
23 +**Mac feasibility bands** (unified memory; fp16 unless stated): 16 GB → ≤3B fp16
24 +or ≤8B 4-bit; 32 GB → ≤8B fp16 or ≤14B 4-bit; 64 GB → ≤14B fp16 or ~32B 4-bit.
25 +Backward passes ≈ 2–3× forward memory. All feasibility judgments are **estimated**
26 +from published numbers, not yet measured — Experiment H converts them to measured.
27 +
28 +**Epistemic flags.** ✅ established · ⚠️ contested · ❌ debunked-in-part.
29 +
30 +---
31 +
32 +## Family 1 — Observational probing
33 +
34 +### 1.1 Linear probes ✅(as decodability)/⚠️(as "use")
35 +- **Reveals:** per-layer linear decodability of a property (language, domain, truth, syntax).
36 +- **Observational/causal:** observational. **Access:** activations.
37 +- **Cost:** L. **Mac:** yes, 0.5B–14B; storage is the binding cost (stream to disk).
38 +- **Failure modes:** probe learns the task; decodable-but-unused; seed/dataset sensitivity; structure found in random models.
39 +- **Reproducibility:** good *with* controls; historically often uncontrolled.
40 +- **Implementation:** any framework; TransformerLens (MPS opt-in), baukit (device-agnostic); trivial in MLX.
41 +- **Main limitation:** decodability ≠ causality — ceiling is Level 1 without intervention.
42 +- **Extension opportunity:** probe maps with selectivity/MDL + FDR control as the atlas's *baseline layer*; probe-drift across quantization levels (unoccupied).
43 +
44 +### 1.2 Logit lens ✅(diagnostic)/❌(quantitative)
45 +- **Reveals:** layer-by-layer refinement of next-token prediction.
46 +- **Observational/causal:** observational. **Access:** activations + unembedding.
47 +- **Cost:** L. **Mac:** every runnable size; trivial in MLX.
48 +- **Failure modes:** family-dependent validity (fails on BLOOM/OPT/some Llamas); early layers unreadable.
49 +- **Reproducibility:** deterministic; validity varies by family.
50 +- **Main limitation:** intermediate residual bases differ from the final one.
51 +- **Extension:** use only as sanity layer; superseded by 1.3/1.4.
52 +
53 +### 1.3 Tuned lens ✅
54 +- **Reveals:** less-biased per-layer prediction trajectories via trained affine translators.
55 +- **Observational/causal:** observational. **Access:** activations; gradients w.r.t. lens only.
56 +- **Cost:** L–M (train lenses once per checkpoint). **Mac:** yes ≤14B.
57 +- **Failure modes:** translator capacity objection (mitigated: affine-only).
58 +- **Reproducibility:** good (validated to 20B).
59 +- **Main limitation:** must be retrained per checkpoint *and per quantization level*.
60 +- **Extension:** tuned-lens drift as a cheap quantization-deformation signal (expF) — not in the literature.
61 +
62 +### 1.4 Patchscopes / early decoding (DoLa, Future Lens) ✅(tools)/⚠️(DoLa mechanism)
63 +- **Reveals:** verbalized readouts of hidden states; multi-token-ahead information.
64 +- **Observational/causal:** observational readout implemented via patching.
65 +- **Cost:** L–M (≈2× inference). **Mac:** yes ≤14B.
66 +- **Failure modes:** readouts inherit model confabulation; DoLa's layer story unreliable across families.
67 +- **Main limitation:** qualitative; hard to aggregate into maps.
68 +- **Extension:** structured Patchscopes corpora as *labels* for probe maps (combination not systematically tried).
69 +
70 +### 1.5 Representation reading (RepE) ⚠️
71 +- **Reveals:** concept directions via PCA on stimulus-pair activation differences.
72 +- **Cost:** L. **Mac:** very friendly (repeng: CPU/MPS, exports GGUF control vectors).
73 +- **Failure modes:** direction conflates stimulus correlates; single-direction assumption; weak evaluation (AxBench: prompting beats most methods).
74 +- **Main limitation:** unreliable without baselines.
75 +- **Extension:** reading-vector *stability maps* (across seeds of stimulus sets) — fits our replication-first program.
76 +
77 +### 1.6 Concept erasure (INLP → LEACE) ✅(LEACE guarantee)/⚠️(amnesic inference)
78 +- **Reveals:** behavioral dependence on a property, by provably-minimal linear removal.
79 +- **Observational/causal:** interventional on representations (bridge to Family 2).
80 +- **Cost:** L–M (LEACE closed-form). **Mac:** yes ≤14B.
81 +- **Failure modes:** nonlinear residue can survive or grow; erasure-operator choice changes conclusions; INLP rank damage.
82 +- **Main limitation:** linear-only guarantee.
83 +- **Extension:** erasure-based capability maps (erase concept → measure per-task damage) as a *causal upgrade of probe maps* — partially tried (amnesic probing), never as a systematic atlas layer.
84 +
85 +**Meta (1.x): the controls doctrine.** Selectivity (Hewitt & Liang), MDL (Voita &
86 +Titov), shuffled labels, capacity-matched probes, random-init baselines, ≥2 datasets,
87 +≥3–5 seeds. Non-negotiable; encoded in Experiment A.
88 +
89 +---
90 +
91 +## Family 2 — Causal / interventional methods
92 +
93 +### 2.1 Activation patching ✅ (with protocol pinned)
94 +- **Reveals:** components causally mediating a behavior (single-mediator, input-relative).
95 +- **Access:** activations (read/write hooks). **Cost:** L–M (2 forwards × components × pairs).
96 +- **Mac:** yes 0.5B–8B; 14B slow. Memory-light, time-heavy — good Mac fit.
97 +- **Failure modes:** conclusions flip with corruption type and metric; hydra/backup compensation; multi-mediator interactions missed.
98 +- **Reproducibility:** good on classic tasks when protocol fixed.
99 +- **Implementation:** TransformerLens/NNsight/pyvene/baukit; all MPS-workable; MLX = wrap modules.
100 +- **Main limitation:** O(components) sweeps; single-mediator blindness.
101 +- **Extension:** pre-registered patching protocols per map type (methodology.md); patched-maps across quantization levels (unoccupied).
102 +
103 +### 2.2 Path patching ✅
104 +- **Reveals:** causal importance of edges/paths (the IOI instrument).
105 +- **Cost:** M (combinatorial). **Mac:** ≤3B–7B practical.
106 +- **Failure modes:** hypothesis-space explosion + all of 2.1's.
107 +- **Main limitation:** does not scale to full-model edge maps locally.
108 +- **Extension:** use only to verify circuits proposed by 2.3/4.5 (screen-then-verify).
109 +
110 +### 2.3 Attribution patching (AtP*, EAP, EAP-IG) ✅ (as screening)
111 +- **Reveals:** first-order estimate of every patch effect in ~3 passes.
112 +- **Access:** activations **and gradients**. **Cost:** L–M.
113 +- **Mac:** ≤3B fp16 @16–32 GB, ≤8B @64 GB; **gradients through 4-bit models are the open constraint; MLX autodiff is the native path to benchmark (expH).**
114 +- **Failure modes:** fails near-zero gradients (fixed by IG), saturated attention, residual stream; documented false negatives; second-order lies (2026).
115 +- **Reproducibility:** good; public reimplementations replicate.
116 +- **Main limitation:** an *estimator* — final claims need real patching.
117 +- **Extension:** the default Mac circuit-scanner; calibration study (AtP error vs true patching) per model family is publishable.
118 +
119 +### 2.4 Causal tracing (ROME-style) ⚠️/❌ (locating→editing inference)
120 +- **Reveals:** where restored states recover corrupted behavior.
121 +- **Cost:** M per fact corpus. **Mac:** yes ≤8B.
122 +- **Failure modes:** **Hase et al.: tracing does not predict editability**; Gaussian noising is off-distribution and inflates mid-layer MLP localization.
123 +- **Main limitation:** interpretation, not mechanics.
124 +- **Extension:** rerun with symmetric-token corruption + resample baselines; treat as one voter in method-agreement maps, never alone.
125 +
126 +### 2.5 Ablations (zero / mean / resample) ✅
127 +- **Reveals:** necessity of components. **Cost:** L. **Mac:** all sizes; ideal for expB.
128 +- **Failure modes:** zero/mean off-distribution; hydra effect; multiple comparisons.
129 +- **Norm:** resample > mean > zero.
130 +- **Main limitation:** necessity only; distribution-relative.
131 +- **Extension:** ablation-scheme *curves* (all three schemes) as the atlas standard — no one publishes this systematically.
132 +
133 +### 2.6 Causal scrubbing ✅(concept)/little-used
134 +- **Reveals:** quantitative faithfulness of a full hypothesis graph.
135 +- **Cost:** M–H. **Mac:** ≤1B realistically.
136 +- **Failure modes:** unfaithful hypotheses can score well; combinatorial resample structure.
137 +- **Implementation:** rust_circuit archived — **no maintained macOS-friendly implementation exists (gap).**
138 +- **Extension:** a minimal MLX scrubbing kernel for micro-models would be new tooling.
139 +
140 +### 2.7 DAS / interchange interventions (pyvene) ⚠️ (subspace-illusion debate unresolved)
141 +- **Reveals:** whether a high-level causal model lives in learned subspaces (IIA).
142 +- **Access:** activations + gradients (rotation trained). **Cost:** M. **Mac:** ≤7B.
143 +- **Failure modes:** dormant-pathway illusion (Makelov) vs over-strict criterion (Wu reply); with unconstrained featurizers, causal abstraction is vacuous (Non-Linear Representation Dilemma).
144 +- **Main limitation:** optimized bases need convergent evidence.
145 +- **Extension:** confidence.md records featurizer class; DAS results always paired with natural-basis patching.
146 +
147 +### 2.8 Steering vectors (ActAdd/CAA/ITI) ✅(existence)/⚠️(reliability)
148 +- **Reveals:** linear sufficiency of behavior directions.
149 +- **Cost:** L. **Mac:** the most Mac-friendly causal technique; **works on GGUF via llama.cpp control vectors.**
150 +- **Failure modes:** input-heterogeneous, anti-steering, OOD-brittle, non-identifiable; loses to prompting baselines (AxBench).
151 +- **Main limitation:** evaluation, not mechanism.
152 +- **Extension:** *steerability maps with variance and baselines* per model/quantization — the literature reports means, not distributions.
153 +
154 +---
155 +
156 +## Family 3 — Feature decomposition
157 +
158 +### 3.1 Superposition theory ✅(toy)/⚠️(real models)
159 +- **Reveals:** why polysemanticity exists; predicts feature geometry under sparsity.
160 +- **Cost:** L (toy models in seconds). **Mac:** trivially; ideal MLX testbed with planted ground truth.
161 +- **Main limitation:** linear-feature assumption is not a theorem (multi-dimensional features exist).
162 +- **Extension:** planted-superposition testbeds to validate our SAE pipeline before real models (validation-first rule).
163 +
164 +### 3.2 Sparse autoencoders (ReLU → TopK/JumpReLU/BatchTopK/Matryoshka) ✅(method)/⚠️(faithful "true features")
165 +- **Reveals:** overcomplete sparse dictionaries of activation directions.
166 +- **Access:** activations at scale. **Cost:** **the cost driver** — 10⁸–10⁹ tokens/layer; 500M tokens ≈ 4 TB fp16 if materialized → streaming mandatory.
167 +- **Mac:** M for 0.5B–2B models at 4k–65k width (SAELens documents an M3 Max run); H for 7B+ at ≥131k width.
168 +- **Failure modes:** ~30% seed replication (the noise floor); dataset dependence; absorption/splitting; proxy metrics ≠ downstream utility; dark-matter reconstruction error.
169 +- **Reproducibility:** the weak point — quantified and low.
170 +- **Implementation:** SAELens (MPS works), dictionary_learning, sparsify (no-cache streaming, CUDA-leaning), mlxterp (MLX, embryonic).
171 +- **Main limitation:** decompositions are pragmatic, not canonical.
172 +- **Extension:** seed-replication rate as a *published per-SAE metric*; buffer-shuffle vs full-shuffle replication study (unpublished); SAE feature drift under quantization (one 2026 paper — extend it).
173 +
174 +### 3.3 Transcoders / CLTs / crosscoders ✅(ascendant)
175 +- **Reveals:** sparse input→output MLP maps (analyzable feature-to-feature weights); cross-layer features; cross-model diffs.
176 +- **Cost:** M (≤2B) to H (7B+, cross-layer multiplies capture). **Mac:** ≤2B pairs realistic.
177 +- **Failure modes:** MLP-only coverage; error nodes dominate on some prompts; diffing latents can be sparsity artifacts (fixed by BatchTopK + latent scaling).
178 +- **Main limitation:** analyses describe a *replacement model*.
179 +- **Extension:** an MPS/MLX transcoder recipe at 0.5–2B with replication reporting.
180 +
181 +### 3.4 Automated interpretability (auto-interp) ✅(tooling)/❌(label quality claims)
182 +- **Reveals:** natural-language labels for latents, at scale.
183 +- **Cost:** M, dominated by LLM inference — **local via the maclustr LLM endpoint.**
184 +- **Failure modes:** low precision/recall under targeted tests; top-activation bias; metrics fail to distinguish trained from random transformers.
185 +- **Rule:** never above Level 1 without intervention scoring.
186 +- **Extension:** intervention-scored labels only; label *stability* across explainer models as an uncertainty channel.
187 +
188 +### 3.5 SAE evaluation (SAEBench, RAVEL, MIB causal-variable track) ✅(best practice)/⚠️(metric stability)
189 +- **Reveals:** which decompositions are worth anything (proxy + downstream + intervention metrics).
190 +- **Cost:** L–M with public SAEs (Gemma Scope on 2B fits 16 GB). **Mac:** yes.
191 +- **Failure modes:** benchmark rankings unstable across tasks/seeds; Goodharting.
192 +- **Extension:** port a SAEBench subset as modelmap's SAE gate; add our replication-rate metric to it.
193 +
194 +---
195 +
196 +## Family 4 — Circuits and computational graphs
197 +
198 +### 4.1 Induction heads ✅ (most replicated finding in the field)
199 +- **Reveals:** the [A][B]…[A]→[B] two-head circuit; cross-architecture universality.
200 +- **Cost:** L (one forward with attention capture). **Mac:** to 32B 4-bit via MLX.
201 +- **Main limitation:** behaviorally defined; heads are multifunctional.
202 +- **Extension:** **the calibration landmark for any cross-model coordinate system** (if alignment can't match induction heads, it fails); first atlas map type.
203 +
204 +### 4.2 IOI circuit & head taxonomies ✅ (details ⚠️)
205 +- **Reveals:** name movers, S-inhibition, copy suppression, successor heads; faithfulness criteria.
206 +- **Cost:** L–M. **Mac:** GPT-2-scale trivial; 7B head scans M.
207 +- **Failure modes:** hydra/backup heads destabilize ablation importance; ~87% faithfulness; task-distribution-relative labels.
208 +- **Extension:** head-taxonomy maps across sizes/quantizations with stability scores (expD substrate).
209 +
210 +### 4.3 MLP key–value memory view ✅(lens)/⚠️(literal reading)
211 +- **Reveals:** input weights as pattern keys, output weights as vocabulary updates.
212 +- **Cost:** L, near-weight-only. **Mac:** trivial.
213 +- **Extension:** pairs with Family 5 as a weight-first map layer; vocabulary-projection maps with early-layer validity flags.
214 +
215 +### 4.4 Attribution graphs / circuit tracing (CLT-based) ✅ (candid limits)
216 +- **Reveals:** per-prompt causal graphs over features (planning, multi-hop, refusal mechanics).
217 +- **Cost:** M with pretrained CLTs (Gemma-2-2B graphs ≈ 15–40 GB → 64 GB Mac plausible, 16 GB not); H to train CLTs at 7B.
218 +- **Failure modes:** error nodes unexplained; frozen attention (QK interference); per-prompt graphs need aggregation to become maps; many graphs unreadable.
219 +- **Implementation:** circuit-tracer (CUDA-first) — **MPS/MLX port is a concrete high-value engineering task.**
220 +- **Extension:** graph *aggregation across prompt corpora* with replication stats — the "map" layer Anthropic hasn't published.
221 +
222 +### 4.5 Automated circuit discovery (ACDC → EAP-IG → edge pruning) ✅ (method choice ⚠️)
223 +- **Reveals:** minimal subgraphs reproducing behavior under faithfulness objectives.
224 +- **Cost:** ACDC M–H; **EAP-IG L–M (the Mac default)**; edge pruning M–H (scaled to 13B on cluster-class compute — fits M3U96a/b).
225 +- **Failure modes:** inherits patching baselines; greedy/independence assumptions; metric overfitting.
226 +- **Extension:** circuit discovery under *multiple ablation schemes* reported as curves.
227 +
228 +### 4.6 Circuit hypothesis testing (circuitry; faithfulness critiques) ✅
229 +- **Reveals:** pass/fail on equivalence/independence/minimality; robustness of faithfulness itself.
230 +- **Cost:** L–M ≤2B. **Mac:** yes.
231 +- **Rule adopted:** every promoted circuit runs the battery; results in confidence.md.
232 +
233 +### 4.7 Sparse feature circuits ✅(method)/⚠️(robustness)
234 +- **Reveals:** circuits over SAE/transcoder features (more interpretable than heads); SHIFT debiasing.
235 +- **Cost:** M with public SAEs. **Mac:** Pythia/Gemma-2-2B scale yes.
236 +- **Failure modes:** inherits SAE seed-fragility — circuits over non-replicating features are sandcastles.
237 +- **Extension:** feature-circuit replication rate across SAE seeds — directly quantifies the sandcastle problem; not in the literature.
238 +
239 +### 4.8 Weight-sparse transformers (circuits by construction) ✅(tiny models)
240 +- **Relevance:** planted-ground-truth benchmark for validating discovery pipelines, not for mapping existing checkpoints.
241 +
242 +---
243 +
244 +## Family 5 — Weight-space cartography (zero/low forward-pass)
245 +
246 +### 5.1 SVD spectra & LASER ✅(phenomenon)/⚠️(interpretation)
247 +- **Reveals:** rank structure per matrix; behaviorally-loaded high-order components.
248 +- **Cost:** L (SVD of 4096×11008 = seconds on Accelerate). **Mac:** fully native.
249 +- **Failure modes:** (layer, matrix, rank) sweeps invite multiple comparisons; ID-only proxies diverge OOD (Friedman illusion).
250 +- **Extension:** per-matrix spectral maps as the atlas's cheapest layer; **test whether they predict A–D properties (expE — the unoccupied gap).**
251 +
252 +### 5.2 Heavy-tailed spectral diagnostics (WeightWatcher) ✅(trends)/⚠️(fine grain)
253 +- **Reveals:** training quality per layer from ESD power-law fits, zero data.
254 +- **Cost:** L; pip-installable, runs on Apple Silicon today.
255 +- **Failure modes:** fit-range sensitivity; Simpson's paradox documented.
256 +- **Extension:** alpha profiles vs probe/localization maps on the same checkpoints — the direct expE experiment.
257 +
258 +### 5.3 Outliers: super weights, massive activations, sinks ✅
259 +- **Reveals:** a handful of weight-visible scalars with catastrophic causal load; the weight→activation bridge.
260 +- **Cost:** L (scan) + minutes (verify by zeroing). **Mac:** yes (MLX 4-bit PPL checks).
261 +- **Extension:** super-weight/outlier maps per model+quantization; **outlier-aware controls for all similarity metrics** (CKA is dominated by these dims).
262 +
263 +### 5.4 Representation/weight similarity (CKA/SVCCA/Procrustes) ✅(tool)/❌(standalone evidence)
264 +- **Reveals:** cross-layer/cross-model geometry alignment.
265 +- **Cost:** L on cached activations. **Mac:** yes.
266 +- **Failure modes:** outlier domination; manipulability; metric disagreement (ReSi); saturation on LLM pairs; estimator drift.
267 +- **Rule:** ≥2 metrics + statistical tests + outlier-stripped variants + random/width-matched nulls.
268 +
269 +### 5.5 Layer redundancy / depth maps ✅
270 +- **Reveals:** deletable contiguous deep blocks (angular similarity).
271 +- **Cost:** L (small calibration set). **Failure modes:** perplexity survives while reasoning dies.
272 +- **Extension:** redundancy maps as a *target* for weight-only predictors; feeds localvm working-set questions (Q5).
273 +
274 +### 5.6 Stitching ⚠️ | 5.7 Git re-basin ⚠️(transformers) | 5.8 Mode connectivity ✅/⚠️
275 +- Stitching: protocol-sensitive; LLM literature thin; tokenizer collision cross-family.
276 +- Re-basin: **fails on transformers**; the LLM-relevant analogue (cross-size alignment) is open = Experiment G.
277 +- Mode connectivity: cheap base↔instruct interpolation profiles are the Mac-friendly artifact.
278 +
279 +### 5.9 Task arithmetic / delta maps ✅(engineering)/⚠️(interpretation)
280 +- **Reveals:** θ_ft − θ_base as sparse, composable task vectors (90%+ droppable).
281 +- **Cost:** L (state-dict math). **Mac:** 7–14B fine.
282 +- **Extension:** **systematic base-vs-instruct delta atlas (per-matrix norms, delta rank, delta spectra) — no published atlas exists; ideal cheap modelmap product.**
283 +
284 +### 5.10 Crosscoder model diffing ⚠️ + weight-side monitoring (2025) — see 3.3
285 +- The field is converging on weight-first diffing (Watch the Weights; weight-diff interpretation) — adjacent to, not overlapping, our pre-screen program.
286 +
287 +### 5.11 MoE router structure ✅(negative result known)
288 +- "Expert = topic" debunked; router-weight geometry is an open, cheap target. Mixtral-class needs 4-bit on 64 GB.
289 +
290 +### 5.12 Weight-space learning ✅(field)/nascent for LLMs
291 +- Model-zoo regime breaks at LLM scale; borrow representations (per-matrix statistics vectors), not the zoo methodology.
292 +
293 +---
294 +
295 +## Family 6 — Knowledge localization and editing
296 +
297 +### 6.1 ROME/MEMIT/PMET/EMMET/AlphaEdit ✅(mechanics)/❌(localization inference)
298 +- **Reveals:** facts are *rewritable* via mid-layer MLP updates; AlphaEdit's null-space projection tames sequential collapse.
299 +- **Cost:** M (covariance stats = hours of forwards; edits cheap). **Mac:** feasible ≤8B, **but all repos are CUDA-first — an MLX editing port is an open tooling contribution.**
300 +- **Failure modes:** localization ≠ editability (Hase); sequential collapse; ripple-effect failures (in-context editing beats parametric); general-ability damage.
301 +- **Extension:** editing as *perturbation probe* (measure what else moves), never as localization evidence.
302 +
303 +### 6.2 Knowledge neurons ❌ (one-fact-one-neuron)
304 +- Degenerate/redundant neurons; token-pattern confound. Keep only as a historical baseline for expB nulls.
305 +
306 +### 6.3 Factual recall pipeline (Geva) + attribute lens ✅(scaffold)/⚠️(details)
307 +- **Reveals:** subject enrichment (MLPs) → relation propagation → attribute extraction (attention heads); ~48% of relations linearly decodable.
308 +- **Cost:** L–M. **Mac:** ≤7B yes.
309 +- **Extension:** the concrete map types for Q1: *enrichment maps* (MLP) and *extraction maps* (heads), with the additivity caveat (multiple mechanisms sum).
310 +
311 +### 6.4 Multilingual knowledge sharing ✅(qualitative)/⚠️(mechanism)
312 +- **Reveals:** partially language-agnostic substrate, English-hub behavior, poor cross-lingual edit transfer.
313 +- **Mac:** ≤8B multilingual models fine. **Extension:** FR/EN localization atlas (expB) has clean prior art to extend and a personal-fit corpus advantage.
314 +
315 +---
316 +
317 +## Cross-cutting instruments
318 +
319 +### 7.1 Cross-model comparison
320 +Universal neurons (1–5% across seeds — methodology template: correlate → taxonomize →
321 +ablate); induction-head landmark; Tigges consistency across training/scale (the expG
322 +band is 70M–2.8B); SAE feature matching (moderate overlap, seed-noise-limited);
323 +platonic convergence deflated to *local/topological* alignment (2026); Rosetta-neuron
324 +mining untested on LLMs (candidate coordinate system); **tokenizer alignment is
325 +unsolved — within-family comparison is the safe default; cross-family maps must
326 +declare their convention in the schema.**
327 +
328 +### 7.2 Ground-truth validation & statistics
329 +Tracr / InterpBench / MIB as pipeline validators (all Mac-runnable); circuitry
330 +hypothesis tests; causal abstraction as the Level-3 formal spine *plus* featurizer-class
331 +disclosure; Dead-Salmons doctrine (random-init nulls, FDR control, uncertainty
332 +quantification) — adopted into `src/modelmap/stats` requirements.
333 +
334 +### 7.3 Quantization × internals (the thin shelf)
335 +Five shallow papers; effects "inconsistent in direction"; one SAE-drift study (2026).
336 +Nothing on probe drift, circuit survival, Q2/Q3, GGUF-runtime capture, or
337 +weight-geometry-predicts-quantization-sensitivity. **The single clearest open field
338 +this survey found.**
339 +
340 +### 7.4 Systems substrate (macOS)
341 +Split regime: probing/patching corpora (≤10⁶ tokens) → cache everything; SAE corpora →
342 +stream, never store. mmap likely wins on Apple NVMe (unmeasured); APFS clones = free
343 +store snapshots for provenance; MLX = only Python path to quantized-model capture;
344 +llama.cpp eval-callback = only path into *actually-quantized* compute (unwrapped);
345 +MPS hooks work but sync/fallback costs unmeasured. All quantitative claims here are
346 +expH's to measure.
347 +
348 +---
349 +
350 +## Overlaps and redundancies (what is secretly the same thing)
351 +
352 +1. **Patching ≡ interchange intervention ≡ causal mediation** — one method, three
353 + vocabularies (mech-interp, causal-abstraction, NLP-fairness). DAS adds a learned
354 + basis; Patchscopes adds a verbal readout; causal tracing adds Gaussian noise (its
355 + weakest link).
356 +2. **Tuned lens ≈ per-layer linear probes onto vocabulary** — a probe family, not a
357 + separate epistemic category; inherits all probe pitfalls.
358 +3. **LEACE ≈ steering's inverse** — erasure removes a direction, steering adds one;
359 + both test linear-direction causality and should share controls (random directions).
360 +4. **SAE / transcoder / crosscoder / CLT = one estimator family** (sparse dictionary
361 + on activations) differing in domain (layer, MLP map, model pair, layer stack);
362 + they share the seed-replication pathology, so replication findings transfer.
363 +5. **Faithfulness metrics ≈ causal scrubbing lite** — circuit faithfulness under
364 + resample ablation is scrubbing's score without the hypothesis-graph machinery.
365 +6. **Layer-redundancy maps ≈ cross-layer CKA** (angular similarity is a similarity
366 + metric) — inherits CKA's outlier pathology; strip sink dimensions first.
367 +
368 +## Combinations already tried (do not claim as new)
369 +
370 +- Probes + causal verification: amnesic probing; RAVEL; MIB causal-variable track.
371 +- SAE features + circuits: sparse feature circuits; attribution graphs.
372 +- SAE + steering: AxBench (negative); feature steering in Scaling Monosemanticity.
373 +- Weight diffs + interpretation: Watch the Weights; Learning to Interpret Weight Differences (2025).
374 +- Patching + automated search: ACDC/EAP/EAP-IG/edge pruning.
375 +- Editing + localization: ROME→Hase (the canonical failed combination).
376 +- Atlas + API hosting: Neuronpedia (without provenance contracts).
377 +
378 +## Ideas that seem novel but are known (novelty traps)
379 +
380 +- "Label every neuron/feature with an LLM" → Transluce, OpenAI 2023, Delphi. Known, and known-unreliable.
381 +- "Compare models with CKA to find shared structure" → large literature *and* a debunking literature.
382 +- "Find the layers where facts live and edit them" → ROME line + Hase refutation.
383 +- "A browsable atlas of features" → Neuronpedia/Neuroscope. The *provenance/confidence* layer is the gap, not the browser.
384 +- "Predict quality from weight spectra" → WeightWatcher (2019–2021).
385 +- "Mine correlated units across models" → Rosetta Neurons (vision) — LLM transfer is open but the method is not new.
386 +- "MLX interpretability library" → mlxterp exists (embryonic) — extend or differentiate, cite regardless.
387 +
388 +## Cost frontier summary (estimated; expH converts to measured)
389 +
390 +| Map type | 16 GB | 32 GB | 64 GB |
391 +|---|---|---|---|
392 +| Weight-only maps (spectra, outliers, deltas) | ≤14B | ≤32B | ≤70B (disk-bound) |
393 +| Probe / lens / erasure maps | ≤3B fp16, 8B 4-bit | ≤8B fp16 | ≤14B fp16 |
394 +| Patching / ablation scans | ≤3B | ≤8B | ≤14B (slow) |
395 +| Attribution patching (gradients) | ≤1.5B | ≤3B fp16 | ≤8B fp16 |
396 +| SAE/transcoder training | ≤0.5B narrow | ≤2B | ≤3B (7B marginal) |
397 +| Attribution graphs (pretrained CLTs) | no | Gemma-2-2B tight | ≤4B |
398 +| Editing (with covariance precompute) | ≤3B | ≤8B | ≤8B+ |
399 +
400 +## Conclusions feeding Phase 3
401 +
402 +1. The unoccupied high-value squares: **quantization × internal maps**, **weight-only
403 + pre-screens validated against activation-level ground truth**, **replication-rate
404 + reporting as a first-class artifact metric**, **quantized-runtime capture
405 + (llama.cpp/MLX)**, **provenance-first atlas schema**.
406 +2. The methodological spine is buyable off the shelf: Tracr/InterpBench validation,
407 + circuitry tests, SAEBench subset, controls doctrine — integrate, don't reinvent.
408 +3. Everything promoted to the atlas needs: pinned patching protocol, ablation-scheme
409 + curves, ≥2 similarity metrics with nulls, featurizer-class disclosure, seed
410 + replication rate, per-dataset agreement. These become the map-card schema fields.
411