SPB Git

spb/wp31_uqo

Public
12.7 KB · 154 lines latex
1\section{Results}\label{sec:results}23All numbers in this section are generated from result files by \texttt{scripts/make\_tables.py} and are indexed4in \texttt{results/claim\_registry.csv} with their run identifiers. Unless stated otherwise, ``loss'' is the5teacher-forced cross-entropy on the answer span of the synthetic suite (nats per token), ``$b$'' a forced budget6applied to every position with uniformly random actions, and relative changes are with respect to the frozen7backbone (budget 0) of the same run.89\subsection{Learnability and the bootstrap problem (Phase 4)}\label{sec:results-learnability}10With the readout initialised at zero, as prescribed, the editor did not learn at all in 700 steps on the frozen112.1M-parameter backbone: validation loss was identical at every budget (0.4144, 0.4143, 0.4143, 0.4146, 0.4164)12and the trainable gradient norm stayed at 0.01--0.02. The cause is a scale mismatch: the backbone residual stream13has root-mean-square 5.6 while the difference of the two attention reads has root-mean-square 0.06, so $U$ must14grow by two orders of magnitude before any edit can move a logit, and until it does the editor receives no15gradient. Three remedies were compared under the identical recipe (Figure~\ref{fig:learnability}): the fixed16scale $s$ alone (zero $U$), $s$ with $U\sim\mathcal N(0,0.2^2)$, and $s$ with $U\sim\mathcal N(0,0.5^2)$. Only17the non-zero initialisations learned within 2,000 steps (dev-iid loss $-1.4\,\%$ at budgets 1--2 for18$\sigma=0.2$); with $\sigma=0.5$ the budget-8 loss was worse than the backbone throughout, and a learning rate19of $3\times10^{-3}$ with $w=128$ collapsed to inert edits (every budget exactly equal to budget 0, gradient20norm 0.00). The deliberate-overfitting test passed: on 64 items at budget 8 the frozen-backbone model reached a21training loss of 0.0024 in 200 steps through the workspace path alone while the held-out loss rose, confirming22that the gradient path and the capacity exist. Failure mode 13.1 of the proposal is therefore confirmed, and23$U\sim\mathcal N(0,0.2^2)$ with $s=\rms(H)$ is used for every adapter below, including the baselines.2425\begin{figure}[t]\centering26\includegraphics[width=\textwidth]{../results/figures/learnability_bootstrap.pdf}27\caption{Stage B bootstrap on the frozen tiny backbone (dev-iid answer-span loss under random forced actions,28budgets 0--8). Left: zero-initialised $U$ with the scale $s$; middle and right: $U\sim\mathcal N(0,\sigma^2)$ with29$\sigma=0.2$ and $0.5$. Budget 0 is the frozen backbone (identical in all panels).}30\label{fig:learnability}31\end{figure}3233\subsection{Sparse editor versus global recurrence and dense writes (H1, H2)}\label{sec:results-matrix}34Table~\ref{tab:seedmatrix} reports the pre-registered decisive comparison on the second tiny backbone35(5.6M parameters, char-level): \vtwoprismeseeds{} independent adapter seeds for the sparse editor,36\vtwogrseeds{} for the global-recurrence adapter and \vtwodenseseeds{} for the dense sixteen-slot editor, all37trained with the same forced-budget recipe for 3,000 steps. H1 is supported in its weak form: random sparse38edits reduce the held-out answer-span loss by \vtwoprismegainbone{}\,\% (budget 1), \vtwoprismegainbtwo{}\,\%39(budget 2) and \vtwoprismegainbfour{}\,\% (budget 4), with seed standard deviations below 0.002 nats, and40raise dev-iid exact match from \vtwoprismeembzero{}\,\% to \vtwoprismeembtwo{}\,\%. The effect is not41specific to sparse local writes, however: the global-recurrence adapter---one shared block rewriting the whole42latent, parameter-matched, same bootstrap---reduces the loss by \vtwogrgainbone{}\,\%, \vtwogrgainbtwo{}\,\% and43\vtwogrgainbfour{}\,\% at the same budgets and reaches \vtwogrembtwo{}\,\% exact match, i.e.\ it is at least as44good at every budget and better at budget 8, where the sparse editor's gain shrinks to \vtwoprismegainbeight{}\,\%.45Dense writes (Table~\ref{tab:seedmatrix}, third row) answer the second half of H2 directly. None of the46adapters improves the held-out template, depth or compose regimes by more than one point (H6).47Figure~\ref{fig:frontier} places the same runs on the quality-versus-analytic-cost plane: the global adapter is48both cheaper per step (one block at width $w$ over the whole latent costs less than two shared blocks with slot49re-keying for two lanes) and better at every budget, so it dominates the sparse editor along the entire frontier;50H4 is refuted at this scale for the comparison the proposal considered most important.51\input{tables/seed_matrix.tex}52\begin{figure}[t]\centering53\includegraphics[width=0.72\textwidth]{../results/figures/frontier_v2.pdf}54\caption{Quality versus analytic extra operations per token at the tiny scale (five adapter seeds, mean $\pm$ SD),55random forced budgets $b\in\{0,1,2,4,8\}$, context 160. Costs from \texttt{prisme/flops.py}; the sparse editor's56cost is the average of single and pair actions.}57\label{fig:frontier}58\end{figure}5960\subsection{Oracle opportunity and the structure of intervention gains (Phase 5)}\label{sec:results-oracle}61Figure~\ref{fig:labels} and Table~\ref{tab:controller} summarise the counterfactual labels collected with the62fixed editor on a stream the editor never trained on (\labelstrainnstates{} and \labelscalibnstates{} exhaustive-audit63states in the training and calibration partitions). Three facts shape everything downstream.64First, the exhaustive one-step oracle finds real value: choosing the best of the 24 actions, or stopping when65none helps, gains \labelscaliboracle{} nats per state on calibration against \labelscalibrandom{} for a random66action; rolled out greedily on dev-iid it lowers the loss by \routingoraclebone{}\,\% with67\routingoracleeditsone{} edits per token at budget 1 and by \routingoraclebeight{}\,\% with68\routingoracleeditseight{} edits per token at budget 8 (Table~\ref{tab:routing}). The two-action oracle gains69\oracletwostep{} nats against \oracleonestep{} for one step, and ``greedy first action then best second'' recovers70almost all of it, so myopia of the one-step rule is not the binding constraint at this scale.71Second, most of the variance of the gains is between states, not between actions: \labelscalibwithin{}\,\% of72the variance of the audited gains is within-state (action identity), the rest is the state's mean gain;73\labelscaliballpos{}\,\% of states have all 24 actions positive and \labelscaliballneg{}\,\% have all 2474negative. Decomposing the oracle, a perfect continue/stop decision with a random action captures75\labelscalibstoponly{} of the \labelscaliboracle{} nats; the choice of \emph{which} slot captures the rest.76Third, opportunity is concentrated where the backbone is uncertain and is dangerous there: in states above the77median base loss the oracle gains \labelscaliboraclehigh{} nats while a random action loses78\labelscalibrandomhigh{}; in the top decile of base loss the oracle gains \labelscalibtopdecoracle{} and a79random action loses \labelscalibtopdecrandom{} nats, with only \labelscalibfracposhigh{}\,\% of actions helpful80(\labelscalibfracposlow{}\,\% in the lower half, where all gains are negligible).8182\begin{figure}[t]\centering83\includegraphics[width=\textwidth]{../results/figures/labels_analysis.pdf}84\caption{Counterfactual labels (calibration partition, editor-unseen stream). Left: fraction of positive gains by85number of edits already applied. Middle: variance decomposition of the exhaustive 24-action audit. Right: mean86gain per state of a random action, of a perfect continue/stop decision with a random action, and of the87one-step oracle.}88\label{fig:labels}89\end{figure}9091\subsection{Learned routing versus reference routers (H2, H3)}\label{sec:results-routing}92Table~\ref{tab:routing} compares, on dev-iid with the same editor, the frozen backbone, random, round-robin and93fixed-pair schedules at budgets 1--8, entropy and norm-change routers over a threshold grid, the learned value94controller at its locked penalty and along a $\lambda$ grid, and the diagnostic oracles. Three results follow.95(i) Spatial selection does not matter for fixed schedules: at budget 1 random, round-robin and fixed-pair give96\routingrandombone{}, \routinguniformbone{} and \routingpairbone{}\,\% respectively, within the paired97bootstrap noise of one another; more edits hurt (\routingrandombeight{}\,\% at budget 8). H2 is not supported.98(ii) The learned controller is worse than not routing: at its calibration-selected penalty99($\lambda=\routingctrllambda$) it edits \routingctrlledits{} times per token and \emph{raises} the dev-iid loss by100\routingctrllocked{}\,\%; every smaller $\lambda$ is worse still, although the same controller had lowered the101calibration loss by 1.1\,\% (Table~\ref{tab:controller}). The reference routers do no better: the best entropy102threshold gives \routingentropybest{}\,\% and the best norm-change threshold \routingnormbest{}\,\%. H3 is103refuted at this scale.104(iii) The mechanism of the failure is visible in the labels. With backbone-residual and entropy features the105controller predicts the \emph{sign} of a gain with \ctrlhentsignacc{}\,\% accuracy (chance 50\,\%) and finds106the states with opportunity, which are the high-loss states; but in exactly those states a wrong action costs107more than the right action gains (\labelscalibtopdecrandom{} versus \labelscalibtopdecoracle{} nats in the top108decile), and the controller's ranking of actions within a state is close to chance (top-1 agreement with the109oracle 7--8\,\% against 4.2\,\% for a uniform guess; captured fraction of the oracle opportunity $\le 5\,\%$).110The pre-registered null ``an oracle can find beneficial edits, but the learned controller cannot predict111them'' is the outcome at this scale.112(iv) The result replicates across adapter seeds on the second backbone (Table~\ref{tab:routingseeds}):113over \vtwoctrlseeds{} seeds the locked controller changes the dev-iid loss by \vtwoctrlrel{}\,\% (SD114\vtwoctrlrelsd{}) while editing a small fraction of tokens, the best fixed schedule lowers it by115\vtwobestfixedrel{}\,\% and the greedy oracle by \vtwooraclerel{}\,\%; the controller beats the frozen backbone116on \vtwoctrlbetter{} of \vtwoctrlseeds{} seeds and never approaches the fixed schedules at any cost.117(v) Adding within-state ranking losses (pairwise on the logged candidates, listwise on the exhaustive audits;118Appendix~\ref{app:failures}) did not change the picture: the locked policy edits 0.07 tokens and matches119budget 0 (0.4333 vs 0.4334), and every smaller penalty is worse.120\input{tables/routing.tex}121\input{tables/controller.tex}122\input{tables/routing_seeds.tex}123124\subsection{Token loss versus task success}\label{sec:results-reasoning}125Exact match of free greedy generation moves with the loss on dev-iid but by less than the pre-registered126minimum effect of three points (Table~\ref{tab:stats}): the sparse editor gains $+1.9$ points at budget 2127(95\,\% $t$-interval over five seeds $[+1.6,+2.2]$; every per-seed paired item bootstrap excludes zero), global128recurrence gains $+1.9$ at budget 2 and $+2.3$ at budget 8, and the paired difference between the two systems129at budget 2 is $+0.1$ points $[-0.5,+0.7]$---a null---while at budget 1 global recurrence is ahead by $0.6$130points $[0.3,0.9]$. Transitions are two-sided: at budget 2 the sparse editor turns \vtwoprismetransplus{}131items per thousand from wrong to right and \vtwoprismetransminus{} from right to wrong132(Table~\ref{tab:emfamilies}); gains concentrate in \textsf{fsm} and \textsf{bind}, \textsf{excl} does not move.133No adapter changes the held-out regimes by more than a point, and the compose family stays at 2--3\,\%.134\input{tables/stats.tex}135136\subsection{Latency and memory}\label{sec:results-latency}137Table~\ref{tab:latency} gives measured batch-1 and batch-8 decode latencies for the tiny model on the138development laptop (Apple M5 Max, fp32, PyTorch 2.15, MPS). Budget 0 costs \latencybzero{} ms per token; each139executed single-slot edit adds about 2.3 ms, so eight edits cost seven times the backbone although their140analytic operation count is only 4.2 times; the dense sixteen-slot write at budget 8 is \emph{not} slower than141eight sparse writes (17.3 vs 19.3 ms), and one controller evaluation adds 2.9 ms even when it stops immediately.142At this scale the per-action host dispatch and kernel-launch overhead dominates the arithmetic, exactly as the143proposal's failure mode 13.6 anticipated; Section~\ref{sec:results-pilot} repeats the measurement at 103M, where144the arithmetic is 60 times larger per token.145\input{tables/latency.tex}146147\subsection{The 103M pilot on web text (Phase 8)}\label{sec:results-pilot}148\input{sections/results_pilot}149150\subsection{Larger configurations}151The 459M and 1.18B configurations were instantiated on the meta device for exact parameter accounting152(Table~\ref{tab:configs}) and the 459M operation table was reproduced (Table~\ref{tab:ops}); neither was153trained. No scaling claim is made (H7 untested).154
Files · 20e88d79efe28fb13610284e7be1ef0ef40c265d
Menu

Theme