SPB Git

spb/wpai0001_uqo

Public
4.8 KB · 55 lines latex
1\section{Introduction}\label{sec:intro}23Independently trained language models appear to converge toward similar linear representation4geometries \citep{huh2024platonic,kornblith2019similarity,raghu2017svcca}, and affine maps between the5residual streams of related models transfer probes, sparse-autoencoder dictionaries and steering vectors6\citep{chen2025transferring,kim2026one,agarwal2026cross}. Whether such maps can carry a7\emph{capability}---a reusable computation that the recipient could not perform before---is open. The one8controlled attempt we know of reports a null: a linear bridge between two Pythia models aligns hidden states9almost perfectly yet does not improve multi-hop answering when injected \citep{zhang2026negative}.1011This paper asks the question under conditions where it can be answered. We (i) \emph{manufacture} a12capability gap by training donor and recipient transformers on the same micro-world except for one invented13operator $\Omega$ that the recipient never sees, (ii) \emph{causally localize} where the donor executes14$\Omega$ by interchange interventions before attempting any transfer, and (iii) compare six transplant15families against a battery of matched controls (sham map, norm-matched random rows, shuffled items, shuffled16layers, unrelated-skill donors) and against conventional learning on the same data. The word \emph{organ} is a17metaphor for a bounded, causally identified subsystem; it carries no biological or phenomenal claim. We then18test the same protocol on three open-weight models (Qwen2.5-1.5B, Qwen2.5-3B, Llama-3.2-3B) with the19capability defined as performing a verifiable task without demonstrations.2021\paragraph{What we find.} In the micro-world the answer is positive, specific and bounded.22A donor-in-the-loop \emph{activation graft}---replacing the recipient's post-layer-0 residual row at every23read position by the donor's row through an orthogonal Procrustes map fitted on \emph{task-free} rows---lifts24the recipient from chance to near-perfect accuracy on held-out $\Omega$ programs (confirmatory: $\Delta_T^{\mathrm{adj}}$25reported in \S\ref{sec:results}) while leaving the control stratum intact, for same- and different-initialization26pairs, across a width mismatch, a depth mismatch and a harder stride-2 variant. The graft carries the27\emph{rule}: maps fitted on six of the eleven operand values transfer to the five unseen values. An28eight-dimensional subspace of the 128-dimensional residual stream suffices; a random subspace of the same29size does nothing. Ablating the donor's layer-0 attention output destroys the donor; grafting the same30component from an intact donor of another seed rescues it; a sham map does not. All controls sit at chance,31and the sham map and random rows also destroy the control stratum, so retention is itself diagnostic.3233Three findings bound the claim. First, the choice of alignment map is not a detail: maps fitted by regression34toward the recipient on rows where the donor computes $\Omega$ \emph{remove} the $\Omega$-specific component35(Proposition~\ref{prop:annihilation}), which is why ridge and CCA grafts fail where orthogonal and task-free36maps succeed, and why rescaling or more data cannot help them. Second, donor-free variants are weaker and37qualitatively different: a saved affine adapter conveys part of the effect in a fresh process but behaves as a38cache (zero on unseen operands); weight grafts partly transfer $\Omega$ at the cost of the control stratum;39steering vectors transfer nothing. Third, on this task a supervised adapter with the same parameter count, or40distillation on the same items, reaches perfect accuracy: transplantation is a way to move a computation41\emph{without} labels or gradient steps, not a cheaper way to learn. On open-weight models, task vectors are42causally sufficient within a model for antonyms and capitals but not for an invented morphology, and43cross-model transfer of mapped vectors is \emph{reported as measured} in \S\ref{sec:results}; we do not44extrapolate from the micro-world to them.4546\paragraph{Contributions.}47(1) An operational theory of transplantable subsystems with explicit estimands, a control battery and a48persistence vocabulary (\S\ref{sec:theory}), including an elementary annihilation result for regression-fitted49alignment maps. (2) A controlled micro-world and an open-weight framework with exact hook semantics,50preregistered hypotheses and a registry of every run (\S\ref{sec:design}). (3) Confirmatory evidence on51localization, sufficiency, necessity, rescue, selectivity, dimensionality, rule-versus-cache and portability52of an activation graft in the micro-world (\S\ref{sec:results}--\S\ref{sec:mechanism}). (4) A map of the53failure boundaries: regression-fitted maps, donor-free adapters, weight grafts and steering vectors, and the54open-weight task-vector transfer (\S\ref{sec:robustness}--\S\ref{sec:discussion}).55
Files · main
Menu

Theme