SPB Git

spb/wpai0001_uqo

Public
2.9 KB · 36 lines latex
1\section{Mechanistic Analysis}\label{sec:mechanism}23\paragraph{Where the donor computes $\Omega$.} The pilot localization (\S\ref{sec:results}) shows that4patching the residual row at the read position restores the answer at every layer, that layer-0 attention5(mostly head 0) moves the operator identity from the $\Omega$ token to the read position, and that layer-06to layer-2 MLPs compute the value map. The candidate organ is therefore the post-layer-0 residual row at the7read position, whose content is the (operator, operand) pair.89\paragraph{Necessity, loss and rescue.} Zeroing the donor's layer-0 attention output at read positions10reduces donor accuracy to $0.086$ on the target stratum and $0.102$ on the control stratum (all operators11pass through the same component). Grafting the post-layer-0 row of an \emph{intact donor of another seed}12through the task-free Procrustes map into the ablated donor restores $1.000$ on both strata; the same graft13through a sham map restores nothing ($0.115$ / $0.101$; Figure~\ref{fig:mechanism}, left). The reverse graft,14recipient $\to$ donor, leaves the donor's control stratum intact ($1.00$) and the target stratum at $0.965$:15the donor's deeper MLPs can recover $\Omega$ from the recipient's row because the row still carries the16operator identity and the operand; what the recipient lacks is not the information but the map.1718\paragraph{Rule, not cache.} A Procrustes map fitted only on $\Omega$ rows whose operand is in19$\{0,\dots,5\}$ yields $\Omega$-step accuracy $0.966$ on items whose operands are all seen and $0.831$ on items20whose operands are all unseen (ratio $0.86$; confirmatory values in \S\ref{sec:results}). The donor-free21adapter fitted on the same rows gives $0.765$ seen and $0.000$ unseen (Figure~\ref{fig:mechanism}, right):22the activation graft moves the donor's computation, the adapter memorizes the alignment rows.2324\paragraph{Specificity.} The unrelated-skill donor (competent on \texttt{vex}, ignorant of $\Omega$), the25shuffled-item rows, the shuffled-layer rows and the norm-matched random rows all leave the target stratum at26chance; the latter three also destroy the control stratum, whereas the unrelated-skill donor and the27shuffled-layer rows do not (Table~\ref{tab:main}). Transfer therefore requires (i) a donor that has the28capability, (ii) rows computed on the same item, (iii) the right layer, and (iv) a map that preserves the29capability-specific directions.3031\paragraph{What is transferred.} Eight principal directions of the mapped $\Omega$-contrast suffice32(\S\ref{sec:robustness}); the directions are those along which donor rows on $\Omega$ steps differ from the33recipient's rows on the same steps after the orthogonal map, i.e.\ the directions the recipient's34downstream MLPs read as ``apply $\Omega$ to this operand''. We do not claim that these directions are35interpretable features; we claim that they are causally sufficient, low-dimensional and rule-carrying.36
Files · main
Menu

Theme