SPB Git

spb/wp31_uqo

Public
2.1 KB · 23 lines latex
1\section{Threats to validity and limitations}\label{sec:limitations}23\paragraph{Scale.} The decisive seeded comparisons were run on 2--6M-parameter character-level backbones trained4on a synthetic suite; the 103M experiments (Section~\ref{sec:results-pilot}) use a single backbone seed and a5196M-token budget (two tokens per parameter), far below compute-optimal. Nothing here speaks to 459M or 1.18B.6\paragraph{Frozen backbones.} Every adapter was trained on a frozen backbone, which bounds what any adapter can7do; the one joint-fine-tuning attempt degraded the backbone and was not pursued.8\paragraph{Recipe dependence.} The bootstrap device ($U\sim\mathcal N(0,0.2^2)$, $s=\rms(H)$), the forced-budget9distribution and the learning rate were chosen on development data after the zero-initialised recipe failed;10other recipes could change the ranking of adapters. The global-recurrence baseline is an ``inspired''11simplification, not the authors' implementations of Huginn, Ouro or ELR.12\paragraph{Label quality.} Intervention gains were collected with one editor checkpoint and one logging mixture;13propensities were recorded but no inverse-propensity correction was needed because the logging policy was14uniform over the audit; labels are exact for the token but noisy as a function of state features.15\paragraph{Measurement.} Latency was measured on Apple Silicon in fp32 with PyTorch's MPS backend and includes16Python dispatch; a fused implementation would change the absolute numbers and possibly the sparse-vs-dense17ordering. Operation counts ignore normalisation and activation costs.18\paragraph{Evaluation scope.} Slot count, editor depth and pairing were ablated with three seeds each; GSM8K and code benchmarks were pre-registered as out of scope for a ~200M-token19backbone and were not run; the held-out compose family remained at 2--3\,\% for every system.20\paragraph{Novelty.} The literature moved quickly in 2025--2026 (Section~\ref{sec:related}); the21execution-time audit may still miss relevant work.22\paragraph{Interpretability.} A slot is a vector, not a thought; no slot-level semantics are claimed.23
Files · main
Menu

Theme