# project: modelmap document: Phase 1 notes — §4.1 Observational probing author: Simon-Pierre Boucher contact: contact@spboucher.ai website: https://modelmap.io created: 2026-08-12 status: draft

# §4.1 — Observational probing

Key papers: Alain & Bengio 2016, https://arxiv.org/abs/1610.01644 ; Belinkov, CL 2022, https://arxiv.org/abs/2102.12452
Reveals: whether a property is linearly decodable from hidden states per layer — an "information availability" profile.
Type: Observational (decodability ≠ use by the model).
Cost: Cheap — one forward pass per prompt + logistic/ridge on cached activations. Feasible 0.5B–14B on 16–64 GB (14B fp16 ≈ 28 GB weights → 64 GB tier or 4-bit). Main cost = activation storage; stream to disk.
Implementations: TransformerLens run_with_cache (MPS opt-in — some ops lack stable MPS kernels: https://transformerlensorg.github.io/TransformerLens/content/getting_started.html); baukit https://github.com/davidbau/baukit (device-agnostic). Trivial in MLX.
Failure modes: probe learns the task (Hewitt & Liang); decodable-but-unused information; dataset/seed sensitivity; high-capacity probes find structure in random models.
Epistemic status: Established as decodability measurement; contested as evidence of use.

Key source: nostalgebraist 2020, https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens
Reveals: layer-by-layer refinement of the next-token prediction (final LN + unembedding applied to intermediate residual).
Cost: Near-free; ideal Mac technique; trivial in MLX.
Failure modes: works on GPT-2 family, poorly/misleadingly on BLOOM/OPT/some Llamas (intermediate residual bases differ from final — documented in Belrose et al. 2023); early layers unreadable.
Epistemic status: Established quick diagnostic; debunked-in-part as a quantitative claim about intermediate beliefs.

Key paper: Belrose et al. 2023, https://arxiv.org/abs/2303.08112 ; code https://github.com/AlignmentResearch/tuned-lens
Reveals: same as logit lens with trained per-layer affine translators — less biased prediction trajectories.
Cost: Moderate (hours on consumer hardware ≤7B; pretrained lenses exist). Model frozen — no backward through model.
Failure modes: translators can "do the work" (mitigated by affine-only capacity); lens must be retrained per checkpoint AND per quantization level — directly relevant to Experiment F.
Epistemic status: Established refinement; validated to 20B (Pythia/NeoX/OPT).

Key papers: DoLa: Chuang et al. 2023, https://arxiv.org/abs/2309.03883 ; Future Lens: Pal et al., CoNLL 2023, https://future.baulab.info/ ; Patchscopes: Ghandeharioun et al., ICML 2024, https://arxiv.org/abs/2401.06102
Reveals: DoLa: which layers hold "factual" signal. Future Lens: hidden states encode ≥2 tokens ahead. Patchscopes: unifying framework — patch a hidden state into another prompt/model and let the LLM verbalize it; subsumes logit/tuned lens and fixes early-layer unreadability.
Cost: Cheap–moderate; all ≤14B on Mac. Patchscopes ≈ 2× inference.
Failure modes: verbalized readouts inherit the model's confabulation; DoLa's premise unreliable across families ("Steering off Course", ACL 2025).
Epistemic status: Established tools; DoLa mechanism contested. Also: SimLens (2025) https://arxiv.org/pdf/2507.17618

Key paper: Zou et al. 2023, https://arxiv.org/abs/2310.01405 ; code https://github.com/andyzoujm/representation-engineering ; survey of challenges (2025): https://arxiv.org/pdf/2502.17601
Reveals: population-level "reading vectors" for high-level concepts via PCA on activation differences over stimulus pairs.
Cost: Cheap (dozens–hundreds of forwards + PCA). Very Mac-friendly.
Implementations: repeng https://github.com/vgel/repeng — control vector "in under sixty seconds", works on CPU/MPS, exports GGUF control vectors for llama.cpp (PR https://github.com/ggml-org/llama.cpp/pull/5970) — directly relevant to mapping quantized local models.
Failure modes: concept vectors conflate correlated stimulus features; single-direction assumption; LLM-judge evaluation (AxBench: prompting baselines beat most representation methods).
Epistemic status: Linear concept directions established; methodology reliability contested.

Key papers: INLP: Ravfogel et al., ACL 2020, https://aclanthology.org/2020.acl-main.647.pdf ; amnesic probing: Elazar et al., TACL 2021, https://aclanthology.org/2021.tacl-1.10/ ; LEACE: Belrose et al., NeurIPS 2023, https://arxiv.org/abs/2306.03819 ; code https://github.com/EleutherAI/concept-erasure
Reveals: whether the model uses a property: erase it (LEACE: closed-form, provably-minimal-damage linear guarding) and observe behavioral change. Interventional on representations — bridges to §4.2.
Cost: Cheap–moderate; LEACE needs only class-conditional means/covariances. Mac-feasible to 14B.
Failure modes: INLP damages representations broadly (rank loss); linear erasure can leave or increase nonlinearly-decodable information; erasure-operator choice changes amnesic conclusions (2025: https://arxiv.org/html/2506.11673). Post-LEACE: Oracle-LEACE https://blog.eleuther.ai/oracle-leace/ ; nonlinear erasure via density matching https://arxiv.org/abs/2507.12341
Epistemic status: LEACE established for the linear guarantee; amnesic-probing inferences contested.

Key papers: Hewitt & Liang, EMNLP 2019, https://arxiv.org/abs/1909.03368 ; Voita & Titov 2020 (MDL), https://arxiv.org/pdf/2003.12298
Substance: selectivity = task accuracy − control-task accuracy; MDL replaces accuracy with codelength. Probes overfit to: word identity/frequency memorization, spurious dataset correlates, annotation artifacts, probe capacity, shared vocabulary between splits.
Implication for modelmap (Experiment A is exactly this): every probe map needs shuffled-label controls, selectivity or MDL reporting, ≥2 datasets, ≥3–5 seeds, randomly-initialized-model baselines.
Epistemic status: Established best practice; ignoring it is the field's canonical failure.