SPB Git

spb/wp31_uqo

Public
2.2 KB · 21 lines latex
1We design, implement, train and evaluate PRISME, a small-language-model architecture that augments a frozen2causal decoder with a read-only source bank, an ephemeral sixteen-slot latent workspace that is re-created at every3position, a shared sparse editor that rewrites one or two slots per action, a value controller trained on4counterfactual intervention gains measured by executing paired edit/no-edit branches, and a differential readout5with an exact zero-budget bypass. We reproduce the proposal's exact backbone parameter counts and its analytic6operation table, prove its structural properties (causality, bitwise identity, exact locality, bounded7displacement, one-step selection regret), and test seven pre-registered hypotheses with a complete implementation,8a fifteen-invariant test suite, an incremental inference engine and controlled, seeded comparisons against a9parameter-matched plain decoder, a dense-slot editor, a global-recurrence adapter trained with the identical10recipe, five reference routers and exhaustive one- and two-step oracles. The zero-initialised readout prescribed by11the proposal does not learn because of a scale mismatch; a scale-aware non-zero initialisation does. Sparse edits12then lower a frozen backbone's held-out loss by \vtwoprismegainbtwo{}\,\% and raise compositional exact match by13about two points, but the global-recurrence adapter matches or exceeds this at every budget. Where to write does14not matter for fixed schedules. An oracle that reads the target finds large opportunity (\routingoraclebeight{}\,\%15of loss), but \labelscalibwithin{}\,\% of the variance of intervention gains is state-level rather than action-level,16the opportunity is concentrated in high-uncertainty states where a wrong action costs more than a right one gains,17and the learned controller ranks actions within a state near chance; it is worse than no routing on held-out data.18Measured on Apple Silicon, per-action dispatch overhead erases the arithmetic savings of sparsity at small scale.19We report the 103M web-text pilot, the complete negative result on learned spatial routing, the mechanistic20explanation, and the conditions under which the hypothesis could still be rescued.21
Files · d6746e03e0d6828cde427e384b9ef1d2b92c62d6
Menu

Theme