SPB Git

spb/wp31_uqo

Public
3.5 KB

# PRISME v1 — frozen architecture specification (Phase 1, 2026-10-29)

Notation: prefix x_{1:t}, backbone residual stream H_t = B_phi(x_{1:t}) ∈ R^{t×d}, h_t = H_t[t]. Workspace width w, M = 16 slots, editor blocks L_w = 2 (shared weights), hard cap R_max = 8.

# Backbone (prisme/backbone.py)

Pre-norm causal decoder: RMSNorm → GQA attention (RoPE θ = 10⁴, head dim 64) → RMSNorm → SwiGLU; tied input/output embeddings; one final RMSNorm. logits = E · RMSNorm(h). Exact backbone counts match the brief for the three configurations (results/tables/param_counts.json).

# Source bank and workspace initialisation (prisme/model.py)

ĥ = RMSNorm_bank(H) (a dedicated norm, d parameters, added for numerical hygiene — see DECISIONS D2), K = ĥ W_K, V = ĥ W_V (read-only, causal: query t reads keys 1..t only), z_t = P ĥ_t, S0[j] = e[j] + z_t for j = 0..15; S is re-initialised at every position (no cross-token memory).

# Action catalog (prisme/workspace.py)

SINGLE_0..15, PAIR_0_1, PAIR_2_3, …, PAIR_14_15, STOP = 25 actions (24 nonterminal). STOP is absorbing.

# Shared sparse editor (prisme/editor.py)

Active queries x = S[I(a)] + ctx_proj([step/R, remaining/R, k/2]). Each of the two shared blocks: (1) RMSNorm → multi-head cross-attention of the active queries to the bank (q, o projections only; the bank is already in width w) with the strict per-query causal mask; (2) RMSNorm → multi-head attention of the active queries over all M slots of the pre-write snapshot (same norm applied to the snapshot); (3) RMSNorm → SwiGLU. Write: δ = proposed − S[j], δ_c = δ · min(1, c / max(RMS(δ), ε)), η = η_max σ(gate([S[j]; proposed])), S'[j] = S[j] + η δ_c. Non-selected slots are returned bitwise unchanged (one-hot scatter from the same snapshot ⇒ pair writes are order-independent). Defaults: η_max = 0.25, c = 1 (untested research values).

# Value controller (prisme/controller.py)

Target-free features: u = f(z_t, step, remaining) (one SwiGLU MLP per state), per action h_a = SiLU(u + W_s s_a + k_emb[k(a)]) with s_a the mean of the slots in I(a); heads → ĝ, ĉ, log σ̂, harm. Q(s,a) = ĝ − λ c(a) − κ σ̂, Q(STOP) = 0, greedy argmax, STOP when max Q ≤ 0. c(a) is the analytic ops cost of the action relative to the backbone ops/token (prisme/flops.py), not a learned quantity.

# Differential readout and exact bypass (prisme/readout.py, prisme/model.py)

r(S, z) = Σ_j softmax_j((W_q z)·(W_k S_j)/√w) S_j; Δ = U (r(S_final, z) − r(S0, z)), U = 0 at init; h_corr = h + Δ. max_budget = 0 ⇒ backbone head returned without allocating slots (explicit branch); positions with zero executed edits also bypass the readout. Identity is bitwise (tests 02–04).

# Causal / cache contract (prisme/cache.py, prisme/generation.py)

Separate per-request caches for backbone KV and bank K/V; the editor never writes to H, K, V, z; incremental decoding equals full-prefix recomputation within 1e-4 (tests in tests/integration/).

# Parameter accounting

Implemented workspace+controller: 2,631,173 (103M) / 10,505,221 (459M) / 21,655,813 (1.18B) versus the brief's 3,088,260 / 12,336,900 / 25,779,844. The brief's three numbers are reproduced exactly by 4dw + Mw + 17w² + 6w f_w + 15.5w + 4; the implementation has 10w² square projections instead of 17w² (no K/V re-projection of the bank inside the editor, no value projection in the readout, leaner controller) and a few extra vectors (bank norm, context projections, k-embedding). See DECISIONS D3.

Files · main
Menu

Theme