SPB Git

Repository Public Research preview

RESEARCH_GAP.md 6.4 KB 85 lines markdown

1# Research gap — WP-AI-0001 (written 2026-11-09, before any transplant experiment)23Literature status: **verified via arXiv API metadata and abstracts on 2026-11-09** for every record in4`literature/arxiv_verified_2026-11-09.txt` (identifiers, titles, first-submission dates, authors) and5`literature/references_verified.bib`. Full texts were not re-read for every item; claims below about6*what each paper does* come from the verified abstracts and the project's search notes. Anything not in7the verified list is marked (unverified) and must not be cited.89## 1. What is already known1011**Representation similarity across models.** SVCCA (Raghu et al. 2017, 1706.05806), CKA (Kornblith et al.122019, 1905.00414), the Platonic Representation Hypothesis (Huh et al. 2024, 2405.07987) and Multi-Way13Representation Alignment (Achara et al. 2026, 2602.06205) establish that independently trained networks14have *linearly related* representation spaces. Shared embedding geometry allows steering vectors to move15between models (Lee et al. 2025, 2503.21073).1617**Model stitching.** Bansal, Nakkiran & Barak (2021, 2106.07682) revisit stitching as a representation18comparison tool (vision). Chen, Merullo, Stolfo & Pavlick (2025, 2506.06609; NeurIPS 2025) show that affine19maps between residual streams of *same-family* language models of different sizes transfer SAEs, probes20and steering vectors. Kim, Mun & Han (2026, 2608.09521) learn a shared dense "activation bus" with one21encoder–decoder pair per model and show that an intermediate activation of one model can be consumed by22another model's frozen upper layers.2324**Cross-model feature dictionaries.** Lan et al. (2024, 2410.06981), Universal SAEs (Thasarathan et al.252025, 2502.03714), SPARC (2025, 2507.06265) and SharedSAE (Ognev et al. 2026, 2609.04344) build concept26spaces shared across models, mostly for *interpretation*, not for conferring capabilities.2728**Cross-model steering and circuit transfer.** Agarwal (2026, 2608.05164) transfers SAE-derived steering29directions across five models (0.8B–8B) and reports functional transfer above ≈1.7B parameters for30*semantic concepts*. Cristofano (2026, 2601.16034) transfers *refusal* interventions across eight model31pairs by concept-basis reconstruction. Sinii et al. (2025, 2509.06608) find RL-induced reasoning steering32vectors transfer only within a family.3334**Capability transfer across scales.** Panigrahi et al. (2023, 2302.06600) graft ≈0.01 % of fine-tuned35parameters back onto a pre-trained model ("skill localization", same model). APM (Fan et al. 2026,362608.13596) prunes a larger donor to a recipient's architecture and mixes it in with a micro coefficient;37SemAlign (Gu et al. 2025, 2510.24208) uses activations as the transfer medium but *trains* the recipient38layer by layer. Task arithmetic (Ilharco et al. 2022, 2212.04089), TIES (Yadav et al. 2023, 2306.01708) and39Git Re-Basin (Ainsworth et al. 2022, 2209.04836) merge *weights* of models with a common initialization.40Zero-shot tokenizer transfer (Minixhofer et al. 2024, 2405.07883) addresses the tokenizer boundary.4142**Causal methods.** Causal mediation (Vig et al. 2020, 2004.12265), causal tracing / ROME (Meng et al. 2022,432202.05262), interchange interventions and causal abstraction (Geiger et al. 2023, 2301.04709 and DAS,442303.02536), circuit discovery (Wang et al. 2022, 2211.00593; Conmy et al. 2023, 2304.14997), function /45task vectors (Todd et al. 2023, 2310.15213; Hendel et al. 2023, 2310.15916), activation steering (Turner46et al. 2023, 2308.10248; Li et al. 2023, 2306.03341; Zou et al. 2023, 2310.01405).4748**The closest negative result.** Zhang & Xin (2026, 2606.03280) learn a linear bridge from Pythia-160M to49Pythia-410M hidden states (cosine ≈ 0.97) and find that injected activations **do not** improve multi-hop50answering: additive injection is null, replacement is destructive. Their conclusion: *offline51representational alignment is not sufficient for useful causal communication inside the receiver.*5253## 2. What is missing (the gap this project targets)54551. **No study starts from a causally identified subsystem.** Stitching, activation-bus and steering-transfer56   work map *whole residual streams* or *concept directions*; APM prunes by activation salience. None first57   establishes, by interchange interventions in the donor, that a bounded component is necessary and58   sufficient for a *measurable capability*, and then asks whether *that* component transfers.592. **No controlled donor/recipient capability gap.** Prior work takes two off-the-shelf models and measures60   benchmark deltas. We *manufacture* the gap (recipient trained without the capability; Tier 0) or61   verify it with repeated held-out measurement (Tier 1–2), so that "acquisition" is identifiable.623. **No matched-control battery.** Sham mappings, norm-matched random grafts, unrelated-skill donors,63   shuffled-token/layer grafts and matched-size adapter fine-tuning are not reported together in any of64   the above; the negative result of 2606.03280 is itself uncontrolled with respect to these.654. **Persistence is conflated.** Inference-time steering, saved adapters and weight edits are different66   claims; we label single-token, within-sequence, cross-prompt and saved-checkpoint effects separately.675. **Rule versus cache.** Whether a transferred component carries a *rule* (generalizing to unseen68   compositions / symbols) rather than a bias toward answer tokens is not tested by existing transfer work.696. **Failure boundaries.** Agarwal (2026) reports a scale discontinuity and Sinii et al. (2025) a family70   boundary for *steering*; nothing comparable exists for *capability* transfer with causal controls.7172## 3. What would *not* be novel (and we will not claim)7374- Affine residual-stream mappings as such (Chen et al. 2025; Kim et al. 2026).75- Cross-model steering of concepts (Agarwal 2026; Lee et al. 2025).76- Grafting fine-tuned parameters within one model (Panigrahi et al. 2023).77- Any "organ" metaphor as a theoretical contribution; the contribution must be the causal design and the78  measured transfer / non-transfer boundaries.7980## 4. Positioning sentence (to be refined after results)8182*We ask whether a component that is causally necessary and sufficient for a capability in a donor can be83installed in a recipient that demonstrably lacks it, under sham, random and matched-training controls,84and we map where this fails.* A rigorous negative result with boundary conditions is an acceptable outcome.85
Files main
Menu

Theme