spb/wpai0001_uqo

1\section{Operational Theory}\label{sec:theory}23\subsection{Objects}4Let $D$ and $R$ be donor and recipient models, $\mathcal{T}$ a task distribution with a target stratum5(items that require the capability) and control strata (items that do not), $S_D$ a candidate subsystem of6$D$ (a set of layers, heads, MLP units, residual subspaces or sparse features), $A$ an alignment operator7mapping donor-side quantities into recipient coordinates, and $I$ an intervention operator. The transplanted8recipient is $R' = I(R, A(S_D))$. Interventions act on the residual stream at \emph{read positions}: the9token positions whose next-token prediction is scored.1011\subsection{Estimands}12With $Q_T(\cdot)$ the exact-match score on held-out target items and $Q_c(\cdot)$ on control stratum $c$,13\begin{align}14\Delta_T &= Q_T(R') - Q_T(R), \qquad15\Delta_T^{\mathrm{adj}} = \Delta_T - \big[Q_T(R_{\mathrm{sham}}) - Q_T(R)\big], \label{eq:adj}\\16\rho_c &= Q_c(R')/Q_c(R) \ \text{(if } Q_c(R)\geq 0.5\text{, else report } \Delta_c), \qquad17N_D = Q_T(D) - Q_T(D \setminus S_D). \label{eq:ret}18\end{align}19$R_{\mathrm{sham}}$ is the matched sham intervention: identical positions, layers, dimensionality and20activation norm, with the donor content replaced by the control defined for each graft family (for mapped21activation grafts, the same donor activation through a random orthogonal map). Items are nested in22generating programs (clusters); uncertainty is a cluster bootstrap over programs within a model pair, and23model pairs form a second level summarized by mean, standard deviation and range.2425\subsection{Definitions}26\emph{Candidate organ}: a component whose perturbation produces repeatable, task-selective causal effects27in $D$ (necessity $N_D$, restored-answer rate under interchange, and rescue by re-insertion).28\emph{Transplant}: a predeclared intervention $I$ that injects, replaces, composes or maps $A(S_D)$ into $R$.29\emph{Acquisition}: $\Delta_T^{\mathrm{adj}}$ above the smallest effect of interest on held-out items, with30retention and no leakage. \emph{Selectivity}: target improvement relative to control strata and matched31interventions. \emph{Persistence}: single-token, within-sequence, cross-prompt, or saved checkpoint,32reported separately; a donor-in-the-loop graft is an inference-time dependency, not a saved modification.33\emph{Portability}: replication across seeds, initializations, widths, depths, families and tokenizers.3435\subsection{Transplant families}36We implement six families (Table~\ref{tab:families}).3738\begin{table}[t]\centering\small39\caption{Transplant families, their inference-time dependency on the donor, what is fitted, and the matched40control used for the control-adjusted effect.}41\label{tab:families}42\resizebox{\textwidth}{!}{\begin{tabular}{llll}43\toprule44Family & donor at inference & fitted object (data) & matched control \\45\midrule46Activation graft & yes (same token ids) & map $A$ (task-free or task rows) & sham map (random orthogonal) \\47Subspace graft & yes & $A$ + $k$ principal directions & random $k$-subspace \\48Steering graft & no (vector precomputed) & $A$ + mean-difference direction & norm-matched random vector \\49Adapter graft & no & affine $f: x_R \to A(x_D)$ (ridge) & supervised adapter with labels (reference) \\50Weight graft & no & none (copy of blocks / rows) & shuffled-layer / unrelated-donor weights \\51Persistent graft & no & saved adapter, fresh process & unmodified recipient on new items \\52\bottomrule53\end{tabular}}54\end{table} The \emph{activation graft} replaces the recipient's55residual row at a read position by a mapped donor row computed on the same token ids56($x_R \leftarrow x_R + \alpha(A(x_D) - x_R)$); the \emph{subspace graft} restricts the replacement to a57$k$-dimensional subspace; the \emph{steering graft} adds a mapped mean-difference direction; the58\emph{adapter graft} is a donor-free affine interface $f$ fitted offline to predict $A(x_D)$ from $x_R$; the59\emph{weight graft} copies shape-compatible parameter blocks; the \emph{persistent graft} saves an adapter60and is evaluated in a separate process on new items.6162\subsection{What alignment maps can and cannot carry}63A natural choice for $A$ is a least-squares map fitted to predict recipient activations from donor64activations on paired inputs. The following elementary statement, proved in Appendix~\ref{app:theory},65explains an empirical regularity of \S\ref{sec:results}: such maps annihilate exactly the donor directions66that carry what the recipient lacks.6768\begin{proposition}[Annihilation of recipient-unpredictable directions]\label{prop:annihilation}69Let $X \in \mathbb{R}^{n\times d_D}$ and $Y \in \mathbb{R}^{n\times d_R}$ be centred paired activations and let70$X = X_c + X_u$ be an orthogonal decomposition of the donor space into a subspace $U$ and its complement with71$X_u^{\top} X_c = 0$ and $X_u^{\top} Y = 0$ (the coordinates of $X$ in $U$ are uncorrelated with every72recipient coordinate). Then every ridge solution $W_\lambda = (X^{\top}X + \lambda I)^{-1} X^{\top} Y$,73$\lambda \geq 0$, satisfies $P_U W_\lambda = 0$, where $P_U$ is the orthogonal projector onto $U$: the map74sends every component of a donor activation in $U$ to zero. The same holds for the least-squares map75restricted to the top-$k$ canonical directions (CCA), because $U$ has zero canonical correlation with $Y$.76In contrast, any orthogonal map $Q$ (e.g.\ the Procrustes solution, with or without isotropic scale) is77injective and satisfies $\|Q^{\top} u\| = \|u\|$ for all $u \in U$.78\end{proposition}7980The proposition does not say that orthogonal maps place the preserved component where the recipient can81\emph{use} it; that is an empirical question answered by the interchange experiments. It does say that82fitting $A$ by regression on rows where the donor computes the target capability and the recipient does not83is self-defeating by construction: the capability-specific variance is, by hypothesis, unpredictable from the84recipient and is removed. Two remedies follow: fit $A$ on \emph{task-free} rows (so that the donor's85capability-specific directions never enter the regression as unpredictable variance), or constrain $A$ to be86orthogonal. Both are tested.8788\subsection{What is a theorem and what is a conjecture}89Proposition~\ref{prop:annihilation} is a statement about linear regression; nothing in it depends on90transformers. The interchange-commutation property under exact causal abstraction \citep{geiger2023causal}91is restated with its assumptions in Appendix~\ref{app:theory}; everything about real models is empirical.92\textbf{Conjecture (transfer boundary).} An activation graft through a linear map succeeds only if the93recipient's downstream computation reads the mapped subspace in the donor's role; this is the default94expectation motivated by \citet{zhang2026negative} and is tested, not assumed.95