SPB Git

spb/wpai0001_uqo

Public
6.4 KB

# Research gap — WP-AI-0001 (written 2026-11-09, before any transplant experiment)

Literature status: verified via arXiv API metadata and abstracts on 2026-11-09 for every record in literature/arxiv_verified_2026-11-09.txt (identifiers, titles, first-submission dates, authors) and literature/references_verified.bib. Full texts were not re-read for every item; claims below about what each paper does come from the verified abstracts and the project's search notes. Anything not in the verified list is marked (unverified) and must not be cited.

# 1. What is already known

Representation similarity across models. SVCCA (Raghu et al. 2017, 1706.05806), CKA (Kornblith et al. 2019, 1905.00414), the Platonic Representation Hypothesis (Huh et al. 2024, 2405.07987) and Multi-Way Representation Alignment (Achara et al. 2026, 2602.06205) establish that independently trained networks have linearly related representation spaces. Shared embedding geometry allows steering vectors to move between models (Lee et al. 2025, 2503.21073).

Model stitching. Bansal, Nakkiran & Barak (2021, 2106.07682) revisit stitching as a representation comparison tool (vision). Chen, Merullo, Stolfo & Pavlick (2025, 2506.06609; NeurIPS 2025) show that affine maps between residual streams of same-family language models of different sizes transfer SAEs, probes and steering vectors. Kim, Mun & Han (2026, 2608.09521) learn a shared dense "activation bus" with one encoder–decoder pair per model and show that an intermediate activation of one model can be consumed by another model's frozen upper layers.

Cross-model feature dictionaries. Lan et al. (2024, 2410.06981), Universal SAEs (Thasarathan et al. 2025, 2502.03714), SPARC (2025, 2507.06265) and SharedSAE (Ognev et al. 2026, 2609.04344) build concept spaces shared across models, mostly for interpretation, not for conferring capabilities.

Cross-model steering and circuit transfer. Agarwal (2026, 2608.05164) transfers SAE-derived steering directions across five models (0.8B–8B) and reports functional transfer above ≈1.7B parameters for semantic concepts. Cristofano (2026, 2601.16034) transfers refusal interventions across eight model pairs by concept-basis reconstruction. Sinii et al. (2025, 2509.06608) find RL-induced reasoning steering vectors transfer only within a family.

Capability transfer across scales. Panigrahi et al. (2023, 2302.06600) graft ≈0.01 % of fine-tuned parameters back onto a pre-trained model ("skill localization", same model). APM (Fan et al. 2026, 2608.13596) prunes a larger donor to a recipient's architecture and mixes it in with a micro coefficient; SemAlign (Gu et al. 2025, 2510.24208) uses activations as the transfer medium but trains the recipient layer by layer. Task arithmetic (Ilharco et al. 2022, 2212.04089), TIES (Yadav et al. 2023, 2306.01708) and Git Re-Basin (Ainsworth et al. 2022, 2209.04836) merge weights of models with a common initialization. Zero-shot tokenizer transfer (Minixhofer et al. 2024, 2405.07883) addresses the tokenizer boundary.

Causal methods. Causal mediation (Vig et al. 2020, 2004.12265), causal tracing / ROME (Meng et al. 2022, 2202.05262), interchange interventions and causal abstraction (Geiger et al. 2023, 2301.04709 and DAS, 2303.02536), circuit discovery (Wang et al. 2022, 2211.00593; Conmy et al. 2023, 2304.14997), function / task vectors (Todd et al. 2023, 2310.15213; Hendel et al. 2023, 2310.15916), activation steering (Turner et al. 2023, 2308.10248; Li et al. 2023, 2306.03341; Zou et al. 2023, 2310.01405).

The closest negative result. Zhang & Xin (2026, 2606.03280) learn a linear bridge from Pythia-160M to Pythia-410M hidden states (cosine ≈ 0.97) and find that injected activations do not improve multi-hop answering: additive injection is null, replacement is destructive. Their conclusion: offline representational alignment is not sufficient for useful causal communication inside the receiver.

# 2. What is missing (the gap this project targets)

  1. No study starts from a causally identified subsystem. Stitching, activation-bus and steering-transfer work map whole residual streams or concept directions; APM prunes by activation salience. None first establishes, by interchange interventions in the donor, that a bounded component is necessary and sufficient for a measurable capability, and then asks whether that component transfers.
  2. No controlled donor/recipient capability gap. Prior work takes two off-the-shelf models and measures benchmark deltas. We manufacture the gap (recipient trained without the capability; Tier 0) or verify it with repeated held-out measurement (Tier 1–2), so that "acquisition" is identifiable.
  3. No matched-control battery. Sham mappings, norm-matched random grafts, unrelated-skill donors, shuffled-token/layer grafts and matched-size adapter fine-tuning are not reported together in any of the above; the negative result of 2606.03280 is itself uncontrolled with respect to these.
  4. Persistence is conflated. Inference-time steering, saved adapters and weight edits are different claims; we label single-token, within-sequence, cross-prompt and saved-checkpoint effects separately.
  5. Rule versus cache. Whether a transferred component carries a rule (generalizing to unseen compositions / symbols) rather than a bias toward answer tokens is not tested by existing transfer work.
  6. Failure boundaries. Agarwal (2026) reports a scale discontinuity and Sinii et al. (2025) a family boundary for steering; nothing comparable exists for capability transfer with causal controls.

# 3. What would not be novel (and we will not claim)

  • Affine residual-stream mappings as such (Chen et al. 2025; Kim et al. 2026).
  • Cross-model steering of concepts (Agarwal 2026; Lee et al. 2025).
  • Grafting fine-tuned parameters within one model (Panigrahi et al. 2023).
  • Any "organ" metaphor as a theoretical contribution; the contribution must be the causal design and the measured transfer / non-transfer boundaries.

# 4. Positioning sentence (to be refined after results)

We ask whether a component that is causally necessary and sufficient for a capability in a donor can be installed in a recipient that demonstrably lacks it, under sham, random and matched-training controls, and we map where this fails. A rigorous negative result with boundary conditions is an acceptable outcome.

Files · main
Menu

Theme