SPB Git

spb/wpai0001_uqo

Public
2.9 KB · 38 lines latex
1\section{Related Work}\label{sec:related}23\paragraph{Representation similarity and stitching.} SVCCA \citep{raghu2017svcca}, CKA4\citep{kornblith2019similarity}, the Platonic Representation Hypothesis \citep{huh2024platonic} and5multi-way alignment \citep{achara2026multi} quantify cross-model similarity; stitching6\citep{bansal2021revisiting} turns similarity into a functional test. For language models, affine7residual-stream maps transfer SAEs, probes and steering vectors across sizes within a family8\citep{chen2025transferring}, and a shared ``activation bus'' lets one model's intermediate activation be9consumed by another's upper layers \citep{kim2026one}. Shared geometry of unembeddings supports steering10transfer \citep{lee2025shared}. Our work differs in starting from a causally identified component and in11measuring acquisition against matched controls rather than representational agreement.1213\paragraph{Cross-model feature dictionaries and steering transfer.} Universal and shared SAEs14\citep{lan2024quantifying,thasarathan2025universal,nasirisarvi2025sparc,ognev2026sharedsae} build concept15spaces across models; cross-architecture steering transfer \citep{agarwal2026cross} and refusal-circuit16transfer \citep{cristofano2026universal} move \emph{behavioural directions}. RL-induced reasoning vectors17transfer only within a family \citep{sinii2025small}. These concern concept- or behaviour-level control;18we test capability acquisition on verifiable tasks.1920\paragraph{Capability transfer and merging.} Skill localization by grafting \citep{panigrahi2023task}21operates within one model. Activation-guided pruning and merging across scales \citep{fan2026training} and22latent semantic alignment with layer-wise training \citep{gu2025beyond} transfer knowledge across sizes;23task arithmetic \citep{ilharco2022editing}, TIES \citep{yadav2023ties} and re-basin \citep{ainsworth2022git}24merge weights sharing an initialization. Tokenizer transfer \citep{minixhofer2024zero} addresses the25vocabulary boundary.2627\paragraph{Causal methods.} We rely on causal mediation and tracing \citep{vig2020causal,meng2022locating},28interchange interventions and causal abstraction \citep{geiger2023causal,geiger2023finding}, circuit29discovery \citep{wang2022interpretability,conmy2023towards}, function and task vectors30\citep{todd2023function,hendel2023context}, activation steering31\citep{turner2023steering,li2023inference,zou2023representation} and sparse dictionaries32\citep{cunningham2023sparse,gurnee2023finding}. Massive activations \citep{sun2024massive} and concept33relearning after removal \citep{lo2024large} inform our ablation and persistence tests.3435\paragraph{Closest negative result.} \citet{zhang2026negative} find that a well-fitting linear bridge does36not yield useful causal communication inside the receiver. We treat this as the default expectation and37design controls that can distinguish ``no transfer'' from ``transfer masked by misplacement''.38
Files · main
Menu

Theme