WP31 research mandate (verbatim summary of the author's brief, 2026-10-29)
Working title: PRISME: Intervention-Supervised Sparse Workspace Editing for Small Language Models. Author: Simon-Pierre Boucher (UQO). Status at initiation: research proposal + untrained reference prototype; no demonstrated language-model advantage. Target: honest, reproducible, submission-quality paper + implementation.
Immutable rules
- Scientific truth first (no fabricated training, metrics, runtimes, citations, hardware, screenshots, proofs).
- No novelty/superiority claims without evidence; up-to-date primary-source prior-art review; negative results acceptable.
- Phases in order with acceptance gates; failures documented, never relabelled.
- Minimal falsifiable version first (no persistent memory, RAG, tools, MoE, verifiers, RL).
- Respect resource/permission boundaries (existing approved access only).
- Never hide costs (params, executed ops, training FLOPs incl. counterfactual branches, wall-clock, RAM, caches).
- Independent train / controller-calibration / dev / final-test splits; no contamination across paired branches.
- Pre-register primary outcome and decision criteria before unsealing final test evaluations.
- Provenance for every number (run id, config hash, code revision, data manifest, log, result file).
- Runnable by another researcher (setup, scripts, seeds, model/data cards, licences, compiled PDF).
Central question and hypotheses
Can counterfactual supervision of where to edit a small latent workspace improve the quality–compute frontier of autoregressive LM beyond uniform workspace computation and adaptive global recurrence? H1 editor usefulness; H2 spatial selection (learned 1–2 slot selection beats uniform/random at fixed cost); H3 intervention supervision beats uncertainty/entropy/norm/fixed-budget routing (controller overhead included); H4 compute adaptivity moves the Pareto frontier vs the best eligible matched baseline; H5 differential identity at budget 0; H6 generalisation (unseen templates, deeper compositions, distractors, paraphrases, seeds, independent distribution); H7 scaling robustness at 459M (conditional). Null possibilities: oracle finds edits but controller cannot predict them; loss improves but answers worsen; larger ordinary Transformer / shared-depth recurrence wins at equal compute; fixed two-slot editing matches learned routing; dispatch overhead dominates sparse savings; source bank hurts long-context latency/memory.
Architecture contract (Section 3 of the brief)
Configurations 103M / 459M / 1.18B (L 12/16/24, d 768/1536/2048, FFN 2048/4096/5632, heads 12/3, 24/6, 32/8, w 256/512/768, FFN_w 768/1536/2048; vocab 32768, M = 16, L_w = 2, R_max = 8, tied head, RoPE, GQA, single final norm). Reference counts: backbone 99,502,848 / 446,744,064 / 1,149,339,648; workspace+controller 3,088,260 / 12,336,900 / 25,779,844. Source bank = backbone final representations projected to width w, strictly causal. S0[j] = e[j] + z_t, reset per token. 24 nonterminal actions (16 singles + 8 fixed pairs) + STOP. Two shared editor blocks (bank cross-attention, slot attention over a coherent snapshot, SwiGLU, RMSNorm/residuals, step/budget context); clipped gated writes (η_max = 0.25, c = 1); exact locality. Controller predicts ĝ, ĉ, σ̂, p_harm; Q = ĝ − λĉ − κσ̂, Q(STOP) = 0; never executes candidate edits at deployment. Differential readout Δ = U(r(S_final) − r(S0)), U = 0 at init, explicit budget-0 bypass (bitwise). Separate backbone KV and bank caches. Reference ops/token at 459M, T = 4096: 1.296 / 1.645 / 1.767 / 2.195 / 5.608 billion (backbone; 4×k1; 4×k2; 8×k2; 8×dense).
Training (Section 4)
Paired target g(s,a;y) = loss(D(s),y) − loss(D(T_a(s)),y). Label sampling: 4 candidates/state (policy preference, competitive alternative, 2 random; balanced singles/pairs; propensities logged); exhaustive 24-action audit subset. Stages A backbone → B editor bootstrap (frozen backbone, forced budgets {0,1,2,4,8}) → C counterfactual collection → D controller fit (Huber gain, BCE harm, calibrated σ; λ, κ tuned on calibration data) → E alternation/lock. Delayed benefits: compare 1-step oracle, 2-action lookahead oracle, learned controller. Token loss ≠ task success: evaluate free generation, track correct↔incorrect transitions vs budget.
Phases 0–15, baselines (Section 7), ablations (Section 8), data/evaluation (Section 9), hardware policy (10),
tracking (11), paper structure (12), failure modes (13), decision rules (14), final report (15): as in the brief. Suggested (non-binding) decision criteria: advance if PRISME exceeds the strongest eligible baseline by ≥ 3 pp at comparable latency, or cuts median latency ≥ 20 % at non-inferior quality, with the uncertainty interval excluding a null practical effect; zero-budget perplexity preservation within 1 %.