SPB Git

spb/wp31_uqo

Public
1.2 KB · 17 lines latex
1The backbone counts of Table~\ref{tab:configs} are reproduced exactly by2$N_{\mathrm{bb}} = Vd + L\,(2d^2 + 2d\,d_{kv} + 3df + 2d) + d$ with $d_{kv}=n_{kv}\cdot 64$ (tied embeddings, two3RMSNorm vectors per block, one final norm). The proposal's workspace+controller figures (3,088,260 / 12,336,900 /425,779,844) are reproduced exactly---for all three configurations---by the closed form5\[6N^{\mathrm{prop}}_{\mathrm{ws}} = 4dw + Mw + 17w^2 + 6wf_w + 15.5\,w + 4,7\]8which we inferred algebraically from the three published values. The implemented design counts9\[10N^{\mathrm{impl}}_{\mathrm{ws}} = d + 3dw + Mw + \underbrace{(6w^2 + 3wf_w + 3w)}_{\text{editor block}} + \underbrace{(4w + 2w + 1)}_{\text{ctx, gate}}11 + \underbrace{(2w^2 + wd)}_{\text{readout}} + \underbrace{(2w^2 + 3w + w + 3wf_w + 2w + 4w + 4)}_{\text{controller}},12\]13i.e.\ $10w^2$ square projections instead of $17w^2$ and a few extra vectors: the editor reads the cached bank14without re-projecting it (no $K/V$ projections, $-2w^2$), the readout uses raw slot values ($-w^2$), and the15controller shares one state MLP across actions ($-4w^2$). The difference is 457,087 (103M), 1,831,679 (459M) and164,124,031 (1.18B) parameters. We did not add inert projections to match the proposal's figures.17
Files · main
Menu

Theme