SPB Git

spb/wp31_uqo

Public
3.2 KB · 45 lines latex
1\section{Mechanistic studies and ablations}\label{sec:mechanistic}23\paragraph{Where the gains are.}4The variance decomposition of Section~\ref{sec:results-oracle} is the central mechanistic fact of this study:5for the trained editor, whether an edit helps is mostly a property of the \emph{state} (\labelscaliballpos{}\,\%6of audited states have every action helpful, \labelscaliballneg{}\,\% every action harmful), and only7\labelscalibwithin{}\,\% of the variance of gains is attributable to which slot is written. Opportunity is8concentrated in high-loss states, where the spread between the best and a random action is largest9(\labelscalibtopdecoracle{} versus \labelscalibtopdecrandom{} nats in the top decile). A router that can only10detect opportunity but not rank actions therefore edits exactly where editing is riskiest---which is what the11learned controller did.1213\paragraph{Harm and overthinking.}14Across fixed schedules the loss is U-shaped in the budget (Table~\ref{tab:seedmatrix}): one or two random15edits help, eight hurt, for both the sparse editor and (less so) the global adapter; the fraction of positive16logged gains falls from 48\,\% at the first edit to 37--40\,\% after three (Figure~\ref{fig:labels}, left). Edits17compound noise faster than they compound signal in the forced-training regime, and nothing in the forced recipe18teaches the editor to leave a good state alone.1920\paragraph{Slot usage and fixed pairs.}21In the exhaustive audits no slot or pair is preferred: the best action's identity agrees between consecutive22states 11--12\,\% of the time (uniform guess 4.2\,\%), singles and fixed pairs have indistinguishable mean gains on23the editor-unseen stream, and the round-robin schedule, which visits every slot, matches random selection.24The address embeddings $e_j$ therefore do not induce functional specialisation in 3,000 steps of forced training;25we make no interpretability claim about slots.2627\paragraph{Myopia.}28The two-action oracle exceeds the one-step oracle by \oracletwostep{} versus \oracleonestep{} nats, but29``greedy first action, then best second'' recovers nearly all of it; the one-step greedy rule is not what limits30the learned controller.3132\paragraph{Cost decomposition.}33Analytically (Section~\ref{sec:compute}), at the 103M configuration and $T=512$ a single-slot edit costs34\costsingle{} and a pair \costpair{} of the backbone's operations per token, of which the slot-key refresh of35all $M$ slots is the largest single term; measured at the tiny scale, host dispatch dominates36(Section~\ref{sec:results-latency}).3738\paragraph{Ablations executed.}39Readout initialisation (zero vs $\sigma\in\{0.2,0.5\}$), readout scale, editor width ($w\in\{64,128\}$),40learning rate ($10^{-3}$ vs $3\times10^{-3}$), budget schedule (mixed vs budget-8 only), write locality41(1--2 slots vs all 16), global vs local recurrence, controller features ($z$ vs $\hat h$ + entropy), label stream42(editor-seen vs editor-unseen), penalties ($\lambda,\kappa$). Not executed: $M\in\{4,8,32\}$, editor depth43$\in\{1,4\}$, rematched pairs, joint fine-tuning (one attempt degraded the backbone and was stopped), distillation,44quantisation. All executed runs, including failed ones, are listed in Appendix~\ref{app:runs}.45
Files · 7846b4f1c44cd53b7d0886106bbb4f52eb1f2ab6
Menu

Theme