SPB Git
spboucher.ai

spb/wp31_uqo

Public

WP31: final assembly — locked evaluations, audit PASS, 48-page manuscript, handoff

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Oct 30, 2026) parent dc69523 Browse files

44 changed files +778 −111

44 changed files
Unified Split
modified PROGRESS.md +6 −0
@@ -21,3 +21,9 @@
21 | 2026-10-30 | 4 dense-writes ablation (v2 backbone, 5 seeds) | done | `p4v2_dense.yaml` → `results/analyzed/seed_matrix_v2.json` | dev-iid loss −2.5/−3.2/−3.2/−1.6 % at b=1/2/4/8 (vs sparse −2.1/−2.8/−3.0/−1.0, GR −3.6/−4.1/−3.7/−3.3); EM +2.0 pp (b1), t-CI [1.6, 2.4] | sparse local writes ≤ dense (H2 second half null; 5 seeds: −2.4/−3.1/−3.1/−1.5 %, EM +2.0 pp [1.6, 2.4]) | — | 21 | 2026-10-30 | 4 dense-writes ablation (v2 backbone, 5 seeds) | done | `p4v2_dense.yaml` → `results/analyzed/seed_matrix_v2.json` | dev-iid loss −2.5/−3.2/−3.2/−1.6 % at b=1/2/4/8 (vs sparse −2.1/−2.8/−3.0/−1.0, GR −3.6/−4.1/−3.7/−3.3); EM +2.0 pp (b1), t-CI [1.6, 2.4] | sparse local writes ≤ dense (H2 second half null; 5 seeds: −2.4/−3.1/−3.1/−1.5 %, EM +2.0 pp [1.6, 2.4]) | — |
22 | 2026-10-30 | 8–10 pilot stages B–D + routing + latency (103M) | done | `editor_pilot_103m_lr3e4` (3000 steps, 90 min, peak 25.9 GB), `labels_103m` (491k labels, 32,768 audit states), `controller_103m`, `routing_103m_val.json`, `latency_103m_mps.json`, `eval_gr_103m_val.json` | editor −0.04/−0.06/−0.08/−0.06 % at b=1/2/4/8; GR −0.08 % (ppl 48.16 → 48.01); controller locked (λ=0.3, κ=0.5) −0.01 % with 0.61 edits/token, calib sign acc 51 %, captured 7 %; oracle b8 −3.6 % (4.3 edits/token); two-step oracle 0.0162 vs one-step 0.0088 nats; latency batch 1: 7.55 ms (b0) → 22.9 (8 single) / 26.9 (8 dense) / 11.0 (controller) ms; dense 103M probe collapsed twice | H1 not supported on web text; H3 null; dispatch overhead still dominant (sparse 3.0× vs dense 3.6× measured, analytic 1.6× vs 3.1×) | enlarged baseline eval, final audit | 22 | 2026-10-30 | 8–10 pilot stages B–D + routing + latency (103M) | done | `editor_pilot_103m_lr3e4` (3000 steps, 90 min, peak 25.9 GB), `labels_103m` (491k labels, 32,768 audit states), `controller_103m`, `routing_103m_val.json`, `latency_103m_mps.json`, `eval_gr_103m_val.json` | editor −0.04/−0.06/−0.08/−0.06 % at b=1/2/4/8; GR −0.08 % (ppl 48.16 → 48.01); controller locked (λ=0.3, κ=0.5) −0.01 % with 0.61 edits/token, calib sign acc 51 %, captured 7 %; oracle b8 −3.6 % (4.3 edits/token); two-step oracle 0.0162 vs one-step 0.0088 nats; latency batch 1: 7.55 ms (b0) → 22.9 (8 single) / 26.9 (8 dense) / 11.0 (controller) ms; dense 103M probe collapsed twice | H1 not supported on web text; H3 null; dispatch overhead still dominant (sparse 3.0× vs dense 3.6× measured, analytic 1.6× vs 3.1×) | enlarged baseline eval, final audit |
23 | 2026-10-30 | 9 locked test evaluation (synthetic, pre-registered primary outcome) | done | prereg frozen v1.0 04:04, manifest unsealed 04:04; `test_eval_v2_s*.yaml` on 5 nodes → `results/raw/eval_p4v2_*_test_*.json`, `results/analyzed/test_statistics.json` | pooled EM (iid+template+depth) b2 vs b0: PRISME +0.9 pp [+0.9, +1.0]; PRISME − GR −0.3 [−0.6, +0.1]; PRISME − dense +0.4 [+0.1, +0.7]; compose 4–5 % all systems | pre-registered threshold (3 pp) NOT met → do not advance | FineWeb test perplexity (4 systems) running | 23 | 2026-10-30 | 9 locked test evaluation (synthetic, pre-registered primary outcome) | done | prereg frozen v1.0 04:04, manifest unsealed 04:04; `test_eval_v2_s*.yaml` on 5 nodes → `results/raw/eval_p4v2_*_test_*.json`, `results/analyzed/test_statistics.json` | pooled EM (iid+template+depth) b2 vs b0: PRISME +0.9 pp [+0.9, +1.0]; PRISME − GR −0.3 [−0.6, +0.1]; PRISME − dense +0.4 [+0.1, +0.7]; compose 4–5 % all systems | pre-registered threshold (3 pp) NOT met → do not advance | FineWeb test perplexity (4 systems) running |
24+| 2026-10-30 | 9 locked FineWeb test (4 systems) | done | `locked_evaluation.yaml` → `results/raw/eval_{backbone_103m,prisme_103m_editor,gr_103m,enlarged_102m}_{val,test}.json` | test ppl: backbone 45.72; editor/GR −0.08 % loss at b=4; enlarged plain decoder 45.64 (val 47.89 vs 48.16) | larger ordinary Transformer wins at matched training (pre-registered null) | — |
25+| 2026-10-30 | 11 MLX / MacLustr engineering | not executed | — | PyTorch/MPS measurements only | SKIPPED (resource/time; declared in paper) | — |
26+| 2026-10-30 | 12 scaling (459M / 1.18B) | meta-instantiation only | `scripts/count_params.py`, `scripts/estimate_flops.py` | exact counts + ops table | no training (declared) | — |
27+| 2026-10-30 | 13 statistics + audit | done | `scripts/statistics.py`, `scripts/verify_manuscript_claims.py` → AUDIT PASS (25 claims), `reproducibility/audit_report.md`, `checksums.sha256` | — | PASS | — |
28+| 2026-10-30 | 14 manuscript | done | `paper/main.pdf` 48 p., 0 undefined refs, 78 verified refs, tables/figures generated | — | PASS | — |
29+| 2026-10-30 | 15 handoff | done | `REPRODUCE.md`, `scripts/reproduce_pilot.sh` (56 s CPU end-to-end), `reproducibility/run_index.md` (52 runs), model/data cards | — | PASS | next experiment: within-state ranking labels at a scale where arithmetic dominates dispatch, GR as baseline |
modified experiments/runs/p4_editor_v1_seed01_20261029233624/manifest.json +1 −1
@@ -9,7 +9,7 @@
9 "python": "3.12.13", 9 "python": "3.12.13",
10 "torch": "2.15.0", 10 "torch": "2.15.0",
11 "command": "/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_editor.py --config configs/experiments/p4_editor_bootstrap.yaml --device mps", 11 "command": "/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_editor.py --config configs/experiments/p4_editor_bootstrap.yaml --device mps",
12− "state": "running", 12+ "state": "stopped_by_operator_scale_mismatch_diagnosed",
13 "wall_seconds": 0.02, 13 "wall_seconds": 0.02,
14 "device": "mps", 14 "device": "mps",
15 "backend": "mps", 15 "backend": "mps",
modified paper/appendix/runs_index.tex +38 −4
@@ -2,6 +2,8 @@
2 2
3 \begin{itemize} 3 \begin{itemize}
4 \tightlist 4 \tightlist
5+\item
6+ \textbf{run\_id}: controller\_103m\_seed01\_20261030034451; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 315.08; \textbf{commit}: unknown
5 \item 7 \item
6 \textbf{run\_id}: controller\_mini\_seed01\_20261030005154; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: cpu; \textbf{wall s}: 3.14; \textbf{commit}: ede67055 8 \textbf{run\_id}: controller\_mini\_seed01\_20261030005154; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: cpu; \textbf{wall s}: 3.14; \textbf{commit}: ede67055
7 \item 9 \item
@@ -10,6 +12,10 @@
10 \textbf{run\_id}: controller\_p4\_hent\_seed01\_20261030002015; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 584.36; \textbf{commit}: unknown 12 \textbf{run\_id}: controller\_p4\_hent\_seed01\_20261030002015; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 584.36; \textbf{commit}: unknown
11 \item 13 \item
12 \textbf{run\_id}: controller\_v1ctrl\_seed01\_20261030004131; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 164.25; \textbf{commit}: 33298d96 14 \textbf{run\_id}: controller\_v1ctrl\_seed01\_20261030004131; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 164.25; \textbf{commit}: 33298d96
15+\item
16+ \textbf{run\_id}: controller\_v1rank\_seed01\_20261030005431; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 171.86; \textbf{commit}: d6746e03
17+\item
18+ \textbf{run\_id}: controller\_v2s1\_seed01\_20261030005758; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 141.05; \textbf{commit}: unknown
13 \item 19 \item
14 \textbf{run\_id}: controller\_v2s2\_seed01\_20261030004717; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 193.23; \textbf{commit}: unknown 20 \textbf{run\_id}: controller\_v2s2\_seed01\_20261030004717; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 193.23; \textbf{commit}: unknown
15 \item 21 \item
@@ -18,6 +24,14 @@
18 \textbf{run\_id}: controller\_v2s4\_seed01\_20261030004719; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 217.1; \textbf{commit}: unknown 24 \textbf{run\_id}: controller\_v2s4\_seed01\_20261030004719; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 217.1; \textbf{commit}: unknown
19 \item 25 \item
20 \textbf{run\_id}: controller\_v2s5\_seed01\_20261030004719; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 214.63; \textbf{commit}: unknown 26 \textbf{run\_id}: controller\_v2s5\_seed01\_20261030004719; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 214.63; \textbf{commit}: unknown
27+\item
28+ \textbf{run\_id}: editor\_pilot\_103m\_lr3e4\_seed01\_20261030021200; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 5397.54; \textbf{train tokens}: 24576000; \textbf{peak mem (GB)}: 25.9; \textbf{commit}: unknown; \textbf{notes}: prisme
29+\item
30+ \textbf{run\_id}: editor\_pilot\_103m\_seed01\_20261030015139; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.24; \textbf{commit}: unknown; \textbf{notes}: prisme
31+\item
32+ \textbf{run\_id}: gr\_pilot\_103m\_seed01\_20261030015117; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 1230.52; \textbf{train tokens}: 24576000; \textbf{peak mem (GB)}: 9.7; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence
33+\item
34+ \textbf{run\_id}: labels\_103m\_seed01\_20261030034314; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 94.09; \textbf{commit}: unknown
21 \item 35 \item
22 \textbf{run\_id}: labels\_mini\_seed01\_20261030005150; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: cpu; \textbf{wall s}: 3.44; \textbf{commit}: ede67055 36 \textbf{run\_id}: labels\_mini\_seed01\_20261030005150; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: cpu; \textbf{wall s}: 3.44; \textbf{commit}: ede67055
23 \item 37 \item
@@ -26,6 +40,8 @@
26 \textbf{run\_id}: labels\_v1ctrl\_seed01\_20261030003957; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: 33298d96 40 \textbf{run\_id}: labels\_v1ctrl\_seed01\_20261030003957; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: 33298d96
27 \item 41 \item
28 \textbf{run\_id}: labels\_v1ctrl\_seed01\_20261030004018; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 72.1; \textbf{commit}: 33298d96 42 \textbf{run\_id}: labels\_v1ctrl\_seed01\_20261030004018; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 72.1; \textbf{commit}: 33298d96
43+\item
44+ \textbf{run\_id}: labels\_v2s1\_seed01\_20261030005712; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 44.65; \textbf{commit}: unknown
29 \item 45 \item
30 \textbf{run\_id}: labels\_v2s2\_seed01\_20261030004543; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 92.45; \textbf{commit}: unknown 46 \textbf{run\_id}: labels\_v2s2\_seed01\_20261030004543; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 92.45; \textbf{commit}: unknown
31 \item 47 \item
@@ -43,7 +59,7 @@
43 \item 59 \item
44 \textbf{run\_id}: p4\_editor\_v1\_scaled\_seed01\_20261029234334; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.16; \textbf{commit}: HEAD; \textbf{notes}: prisme 60 \textbf{run\_id}: p4\_editor\_v1\_scaled\_seed01\_20261029234334; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.16; \textbf{commit}: HEAD; \textbf{notes}: prisme
45 \item 61 \item
46− \textbf{run\_id}: p4\_editor\_v1\_seed01\_20261029233624; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: HEAD; \textbf{notes}: prisme 62+ \textbf{run\_id}: p4\_editor\_v1\_seed01\_20261029233624; \textbf{state}: stopped\_by\_operator\_scale\_mismatch\_diagnosed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: HEAD; \textbf{notes}: prisme
47 \item 63 \item
48 \textbf{run\_id}: p4\_overfit64\_b8\_seed01\_20261029235111; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 519.6; \textbf{train tokens}: 95773; \textbf{peak mem (GB)}: 6.7; \textbf{commit}: a9d94e8a; \textbf{notes}: prisme 64 \textbf{run\_id}: p4\_overfit64\_b8\_seed01\_20261029235111; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 519.6; \textbf{train tokens}: 95773; \textbf{peak mem (GB)}: 6.7; \textbf{commit}: a9d94e8a; \textbf{notes}: prisme
49 \item 65 \item
@@ -51,7 +67,23 @@
51 \item 67 \item
52 \textbf{run\_id}: p4\_tiny\_backbone\_v2\_seed01\_20261029233624; \textbf{state}: stopped\_overfitting\_step9300\_checkpoint\_step5000\_used; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: HEAD; \textbf{notes}: backbone\_only 68 \textbf{run\_id}: p4\_tiny\_backbone\_v2\_seed01\_20261029233624; \textbf{state}: stopped\_overfitting\_step9300\_checkpoint\_step5000\_used; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: HEAD; \textbf{notes}: backbone\_only
53 \item 69 \item
54− \textbf{run\_id}: p4v2\_dense\_seed05\_20261030001839; \textbf{state}: running; \textbf{host}: M4BP36; \textbf{device}: mps; \textbf{wall s}: 0.33; \textbf{commit}: unknown; \textbf{notes}: dense 70+ \textbf{run\_id}: p4v2\_dense\_seed01\_20261030013421; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 2465.35; \textbf{train tokens}: 496298; \textbf{peak mem (GB)}: 30.3; \textbf{commit}: unknown; \textbf{notes}: dense
71+\item
72+ \textbf{run\_id}: p4v2\_dense\_seed02\_20261030001749; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.51; \textbf{commit}: unknown; \textbf{notes}: dense
73+\item
74+ \textbf{run\_id}: p4v2\_dense\_seed02\_20261030004544; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 2601.32; \textbf{train tokens}: 498543; \textbf{peak mem (GB)}: 30.3; \textbf{commit}: unknown; \textbf{notes}: dense
75+\item
76+ \textbf{run\_id}: p4v2\_dense\_seed03\_20261030001749; \textbf{state}: running; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 0.54; \textbf{commit}: unknown; \textbf{notes}: dense
77+\item
78+ \textbf{run\_id}: p4v2\_dense\_seed03\_20261030004544; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 2620.1; \textbf{train tokens}: 499017; \textbf{peak mem (GB)}: 30.3; \textbf{commit}: unknown; \textbf{notes}: dense
79+\item
80+ \textbf{run\_id}: p4v2\_dense\_seed04\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 3983.98; \textbf{train tokens}: 497821; \textbf{peak mem (GB)}: 28.4; \textbf{commit}: unknown; \textbf{notes}: dense
81+\item
82+ \textbf{run\_id}: p4v2\_dense\_seed05\_20261030001839; \textbf{state}: completed; \textbf{host}: M4BP36; \textbf{device}: mps; \textbf{wall s}: 3249.7; \textbf{train tokens}: 497078; \textbf{peak mem (GB)}: 30.3; \textbf{commit}: unknown; \textbf{notes}: dense
83+\item
84+ \textbf{run\_id}: p4v2\_dense\_seed05\_20261030004544; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 2620.45; \textbf{train tokens}: 497078; \textbf{peak mem (GB)}: 28.2; \textbf{commit}: unknown; \textbf{notes}: dense
85+\item
86+ \textbf{run\_id}: p4v2\_gr\_seed01\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 1120.54; \textbf{train tokens}: 496298; \textbf{peak mem (GB)}: 2.3; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence
55 \item 87 \item
56 \textbf{run\_id}: p4v2\_gr\_seed02\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 951.12; \textbf{train tokens}: 498543; \textbf{peak mem (GB)}: 3.4; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence 88 \textbf{run\_id}: p4v2\_gr\_seed02\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 951.12; \textbf{train tokens}: 498543; \textbf{peak mem (GB)}: 3.4; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence
57 \item 89 \item
@@ -60,6 +92,8 @@
60 \textbf{run\_id}: p4v2\_gr\_seed04\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 430.63; \textbf{train tokens}: 497821; \textbf{peak mem (GB)}: 2.3; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence 92 \textbf{run\_id}: p4v2\_gr\_seed04\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 430.63; \textbf{train tokens}: 497821; \textbf{peak mem (GB)}: 2.3; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence
61 \item 93 \item
62 \textbf{run\_id}: p4v2\_gr\_seed05\_20261030001839; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 370.27; \textbf{train tokens}: 497078; \textbf{peak mem (GB)}: 3.4; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence 94 \textbf{run\_id}: p4v2\_gr\_seed05\_20261030001839; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 370.27; \textbf{train tokens}: 497078; \textbf{peak mem (GB)}: 3.4; \textbf{commit}: unknown; \textbf{notes}: global\_recurrence
95+\item
96+ \textbf{run\_id}: p4v2\_prisme\_seed01\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 2333.13; \textbf{train tokens}: 496298; \textbf{peak mem (GB)}: 8.9; \textbf{commit}: unknown; \textbf{notes}: prisme
63 \item 97 \item
64 \textbf{run\_id}: p4v2\_prisme\_seed02\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 1529.74; \textbf{train tokens}: 498543; \textbf{peak mem (GB)}: 8.8; \textbf{commit}: unknown; \textbf{notes}: prisme 98 \textbf{run\_id}: p4v2\_prisme\_seed02\_20261030001749; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 1529.74; \textbf{train tokens}: 498543; \textbf{peak mem (GB)}: 8.8; \textbf{commit}: unknown; \textbf{notes}: prisme
65 \item 99 \item
@@ -69,7 +103,7 @@
69 \item 103 \item
70 \textbf{run\_id}: p4v2\_prisme\_seed05\_20261030001839; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 939.64; \textbf{train tokens}: 497078; \textbf{peak mem (GB)}: 9.9; \textbf{commit}: unknown; \textbf{notes}: prisme 104 \textbf{run\_id}: p4v2\_prisme\_seed05\_20261030001839; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 939.64; \textbf{train tokens}: 497078; \textbf{peak mem (GB)}: 9.9; \textbf{commit}: unknown; \textbf{notes}: prisme
71 \item 105 \item
72− \textbf{run\_id}: pretrain103m\_seed01\_20261029232706; \textbf{state}: running; \textbf{host}: M3U96a; \textbf{device}: mps; \textbf{wall s}: 0.03; \textbf{commit}: unknown 106+ \textbf{run\_id}: pretrain103m\_seed01\_20261029232706; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 8279.79; \textbf{train tokens}: 98304000; \textbf{peak mem (GB)}: 28.0; \textbf{commit}: unknown
73 \item 107 \item
74− \textbf{run\_id}: pretrain\_enlarged102m\_seed01\_20261029232914; \textbf{state}: running; \textbf{host}: M3U96b; \textbf{device}: mps; \textbf{wall s}: 0.02; \textbf{commit}: unknown 108+ \textbf{run\_id}: pretrain\_enlarged102m\_seed01\_20261030015017; \textbf{state}: completed; \textbf{host}: laptop; \textbf{device}: mps; \textbf{wall s}: 8527.03; \textbf{train tokens}: 98304000; \textbf{peak mem (GB)}: 28.5; \textbf{commit}: unknown
75 \end{itemize} 109 \end{itemize}
modified paper/main.log +42 −45
@@ -1,4 +1,4 @@
1−This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026) (preloaded format=pdflatex 2026.3.22) 30 OCT 2026 04:15 1+This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026) (preloaded format=pdflatex 2026.3.22) 30 OCT 2026 04:21
2 entering extended mode 2 entering extended mode
3 restricted \write18 enabled. 3 restricted \write18 enabled.
4 %&-line parsing enabled. 4 %&-line parsing enabled.
@@ -1297,18 +1297,19 @@ Overfull \hbox (36.62411pt too wide) in paragraph at lines 19--24
1297 t/m/n/12 data/processed/fineweb_edu_300m/manifest.json\T1/ptm/m/n/12 (-20) ). 1297 t/m/n/12 data/processed/fineweb_edu_300m/manifest.json\T1/ptm/m/n/12 (-20) ).
1298 [] 1298 []
1299 1299
1300−[16] 1300+[16] (./tables/controls.tex
1301− 1301+Overfull \hbox (344.15573pt too wide) in paragraph at lines 7--16
1302−LaTeX Warning: Reference `tab:controls' on page 17 undefined on input line 27. 1302+ [][]
1303+ []
1303 1304
1304−) (./sections/results.tex 1305+)) (./sections/results.tex
1305−Overfull \hbox (71.70984pt too wide) in paragraph at lines 38--1 1306+Overfull \hbox (71.70984pt too wide) in paragraph at lines 39--1
1306 \T1/ptm/m/n/12 (-20) The pri-mary out-come, met-rics, de-ci-sion thresh-olds an 1307 \T1/ptm/m/n/12 (-20) The pri-mary out-come, met-rics, de-ci-sion thresh-olds an
1307 d seed plan were writ-ten to \T1/cmtt/m/n/12 protocol/preregistration.md 1308 d seed plan were writ-ten to \T1/cmtt/m/n/12 protocol/preregistration.md
1308 [] 1309 []
1309 1310
1310 [17] 1311 [17]
1311−<../results/figures/learnability_bootstrap.pdf, id=494, 353.75287pt x 191.45125 1312+<../results/figures/learnability_bootstrap.pdf, id=496, 353.75287pt x 191.45125
1312 pt> 1313 pt>
1313 File: ../results/figures/learnability_bootstrap.pdf Graphic file (type pdf) 1314 File: ../results/figures/learnability_bootstrap.pdf Graphic file (type pdf)
1314 <use ../results/figures/learnability_bootstrap.pdf> 1315 <use ../results/figures/learnability_bootstrap.pdf>
@@ -1321,14 +1322,14 @@ Overfull \hbox (133.05286pt too wide) in paragraph at lines 7--21
1321 [] 1322 []
1322 1323
1323 ) 1324 )
1324−<../results/figures/frontier_v2.pdf, id=499, 305.58919pt x 208.87035pt> 1325+<../results/figures/frontier_v2.pdf, id=501, 305.58919pt x 208.87035pt>
1325 File: ../results/figures/frontier_v2.pdf Graphic file (type pdf) 1326 File: ../results/figures/frontier_v2.pdf Graphic file (type pdf)
1326 <use ../results/figures/frontier_v2.pdf> 1327 <use ../results/figures/frontier_v2.pdf>
1327 Package pdftex.def Info: ../results/figures/frontier_v2.pdf used on input line 1328 Package pdftex.def Info: ../results/figures/frontier_v2.pdf used on input line
1328 59. 1329 59.
1329 (pdftex.def) Requested size: 338.22418pt x 231.19547pt. 1330 (pdftex.def) Requested size: 338.22418pt x 231.19547pt.
1330− [18 <../results/figures/learnability_bootstrap.pdf>] [19] 1331+ [18] [19 <../results/figures/learnability_bootstrap.pdf>]
1331−<../results/figures/labels_analysis.pdf, id=536, 548.77519pt x 192.17598pt> 1332+<../results/figures/labels_analysis.pdf, id=538, 548.77519pt x 192.17598pt>
1332 File: ../results/figures/labels_analysis.pdf Graphic file (type pdf) 1333 File: ../results/figures/labels_analysis.pdf Graphic file (type pdf)
1333 <use ../results/figures/labels_analysis.pdf> 1334 <use ../results/figures/labels_analysis.pdf>
1334 Package pdftex.def Info: ../results/figures/labels_analysis.pdf used on input 1335 Package pdftex.def Info: ../results/figures/labels_analysis.pdf used on input
@@ -1340,12 +1341,12 @@ Overfull \hbox (109.74734pt too wide) in paragraph at lines 8--15
1340 [][] 1341 [][]
1341 [] 1342 []
1342 1343
1343−) [20 <../results/figures/frontier_v2.pdf>] (./tables/stats.tex) [21 <../result 1344+) [20] (./tables/stats.tex) [21 <../results/figures/frontier_v2.pdf>]
1344−s/figures/labels_analysis.pdf>] (./tables/test.tex) (./tables/latency.tex) 1345+(./tables/test.tex) [22 <../results/figures/labels_analysis.pdf>]
1345−(./sections/results_pilot.tex [22] (./tables/pilot_ppl.tex) 1346+(./tables/latency.tex) (./sections/results_pilot.tex (./tables/pilot_ppl.tex)
1346−(./tables/pilot_routing.tex) (./tables/pilot_latency.tex))) 1347+[23] (./tables/pilot_routing.tex) (./tables/pilot_latency.tex)))
1347−(./sections/mechanistic.tex [23]) (./sections/limitations.tex [24]) 1348+(./sections/mechanistic.tex [24]) (./sections/limitations.tex [25])
1348−(./sections/conclusion.tex [25]) (./sections/reproducibility.tex) 1349+(./sections/conclusion.tex) (./sections/reproducibility.tex)
1349 Overfull \hbox (51.18217pt too wide) in paragraph at lines 2--101 1350 Overfull \hbox (51.18217pt too wide) in paragraph at lines 2--101
1350 \T1/ptm/m/n/12 (-20) num-ber in the manuscript is pro-duced by \T1/cmtt/m/n/12 1351 \T1/ptm/m/n/12 (-20) num-ber in the manuscript is pro-duced by \T1/cmtt/m/n/12
1351 scripts/make_tables.py \T1/ptm/m/n/12 (-20) and \T1/cmtt/m/n/12 scripts/reprodu 1352 scripts/make_tables.py \T1/ptm/m/n/12 (-20) and \T1/cmtt/m/n/12 scripts/reprodu
@@ -1359,18 +1360,18 @@ cript_claims.py \T1/ptm/m/n/12 (-20) checks the chain. \T1/cmtt/m/n/12 REPRODUC
1359 E.md 1360 E.md
1360 [] 1361 []
1361 1362
1362−[26] (./main.bbl [27] [28] 1363+[26] [27] (./main.bbl [28] [29]
1363 Underfull \hbox (badness 1297) in paragraph at lines 208--211 1364 Underfull \hbox (badness 1297) in paragraph at lines 208--211
1364 []\T1/ptm/b/n/12 (+20) You, Yichen, Tianyu Fu, Aosong Feng, Xing-tai Lv, Xue-fe 1365 []\T1/ptm/b/n/12 (+20) You, Yichen, Tianyu Fu, Aosong Feng, Xing-tai Lv, Xue-fe
1365 i Ning, Ning Ding, and 1366 i Ning, Ning Ding, and
1366 [] 1367 []
1367 1368
1368−[29]) (./appendix/appendix.tex (./appendix/notation.tex 1369+[30]) (./appendix/appendix.tex (./appendix/notation.tex
1369 Overfull \hbox (12.19864pt too wide) in paragraph at lines 2--16 1370 Overfull \hbox (12.19864pt too wide) in paragraph at lines 2--16
1370 [] 1371 []
1371 [] 1372 []
1372 1373
1373−) (./appendix/proofs.tex [30]) (./appendix/params.tex [31] 1374+) (./appendix/proofs.tex [31]) (./appendix/params.tex [32]
1374 Overfull \hbox (92.32437pt too wide) detected at line 12 1375 Overfull \hbox (92.32437pt too wide) detected at line 12
1375 \OML/ztmcm/m/it/12 N[] \OT1/ztmcm/m/n/12 = \OML/ztmcm/m/it/12 d \OT1/ztmcm/m/n/ 1376 \OML/ztmcm/m/it/12 N[] \OT1/ztmcm/m/n/12 = \OML/ztmcm/m/it/12 d \OT1/ztmcm/m/n/
1376 12 + 3\OML/ztmcm/m/it/12 dw \OT1/ztmcm/m/n/12 + \OML/ztmcm/m/it/12 Mw \OT1/ztmc 1377 12 + 3\OML/ztmcm/m/it/12 dw \OT1/ztmcm/m/n/12 + \OML/ztmcm/m/it/12 Mw \OT1/ztmc
@@ -1479,7 +1480,7 @@ LaTeX Warning: `h' float specifier changed to `ht'.
1479 ) (./appendix/prereg.tex 1480 ) (./appendix/prereg.tex
1480 LaTeX Font Info: Font shape `T1/cmtt/bx/n' in size <12> not available 1481 LaTeX Font Info: Font shape `T1/cmtt/bx/n' in size <12> not available
1481 (Font) Font shape `T1/cmtt/m/n' tried instead on input line 1. 1482 (Font) Font shape `T1/cmtt/m/n' tried instead on input line 1.
1482− [32] 1483+ [33]
1483 LaTeX Font Info: Trying to load font information for TS1+ptm on input line 8 1484 LaTeX Font Info: Trying to load font information for TS1+ptm on input line 8
1484 . 1485 .
1485 1486
@@ -1492,7 +1493,7 @@ Overfull \hbox (80.09856pt too wide) in paragraph at lines 24--26
1492 T1/cmtt/m/n/12 data/generators/synthetic.py\T1/ptm/m/n/12 (-20) , 1493 T1/cmtt/m/n/12 data/generators/synthetic.py\T1/ptm/m/n/12 (-20) ,
1493 [] 1494 []
1494 1495
1495−[33] [34]) (./appendix/runs.tex 1496+[34] [35]) (./appendix/runs.tex
1496 Overfull \hbox (121.91187pt too wide) in paragraph at lines 1--14 1497 Overfull \hbox (121.91187pt too wide) in paragraph at lines 1--14
1497 \T1/cmtt/m/n/12 training.pretrain\T1/ptm/m/n/12 (-20) , \T1/cmtt/m/n/12 trainin 1498 \T1/cmtt/m/n/12 training.pretrain\T1/ptm/m/n/12 (-20) , \T1/cmtt/m/n/12 trainin
1498 g.train_editor\T1/ptm/m/n/12 (-20) , \T1/cmtt/m/n/12 training.collect_intervent 1499 g.train_editor\T1/ptm/m/n/12 (-20) , \T1/cmtt/m/n/12 training.collect_intervent
@@ -1507,14 +1508,14 @@ tion.latency \T1/ptm/m/n/12 (-20) or \T1/cmtt/m/n/12 evaluation.run\T1/ptm/m/n/
1507 12 (-20) ; launch scripts un-der \T1/cmtt/m/n/12 scripts/ 1508 12 (-20) ; launch scripts un-der \T1/cmtt/m/n/12 scripts/
1508 [] 1509 []
1509 1510
1510−(./appendix/runs_index.tex [35] [36] 1511+(./appendix/runs_index.tex [36] [37]
1511−Overfull \hbox (25.12868pt too wide) in paragraph at lines 52--54 1512+Overfull \hbox (25.12868pt too wide) in paragraph at lines 68--70
1512 []\T1/ptm/b/n/10 (-20) run_id\T1/ptm/m/n/10 (-20) : p4_tiny_backbone_v2_seed01_ 1513 []\T1/ptm/b/n/10 (-20) run_id\T1/ptm/m/n/10 (-20) : p4_tiny_backbone_v2_seed01_
1513 20261029233624; \T1/ptm/b/n/10 (-20) state\T1/ptm/m/n/10 (-20) : stopped_overfi 1514 20261029233624; \T1/ptm/b/n/10 (-20) state\T1/ptm/m/n/10 (-20) : stopped_overfi
1514 tting_step9300_checkpoint_step5000_used; 1515 tting_step9300_checkpoint_step5000_used;
1515 [] 1516 []
1516 1517
1517−)) (./appendix/results.tex (./tables/em_families.tex)) 1518+[38])) (./appendix/results.tex (./tables/em_families.tex))
1518 Overfull \hbox (168.4564pt too wide) in paragraph at lines 2--16 1519 Overfull \hbox (168.4564pt too wide) in paragraph at lines 2--16
1519 \T1/ptm/m/n/12 (-20) records (\T1/cmtt/m/n/12 experiments/runs/*/em_*.json\T1/p 1520 \T1/ptm/m/n/12 (-20) records (\T1/cmtt/m/n/12 experiments/runs/*/em_*.json\T1/p
1520 tm/m/n/12 (-20) , \T1/cmtt/m/n/12 results/raw/items_*.json\T1/ptm/m/n/12 (-20) 1521 tm/m/n/12 (-20) , \T1/cmtt/m/n/12 results/raw/items_*.json\T1/ptm/m/n/12 (-20)
@@ -1538,14 +1539,14 @@ ileNotFoundError\T1/ptm/m/n/12 (-20) ).; \T1/ptm/b/n/12 (-20) ev-i-dence\T1/ptm
1538 g 1539 g
1539 [] 1540 []
1540 1541
1541−[37] 1542+
1542 Overfull \hbox (49.9369pt too wide) in paragraph at lines 10--12 1543 Overfull \hbox (49.9369pt too wide) in paragraph at lines 10--12
1543 \T1/ptm/m/n/12 (-20) be-fore ed-its can af-fect log-its.; \T1/ptm/b/n/12 (-20) 1544 \T1/ptm/m/n/12 (-20) be-fore ed-its can af-fect log-its.; \T1/ptm/b/n/12 (-20)
1544 ev-i-dence\T1/ptm/m/n/12 (-20) : \T1/cmtt/m/n/12 experiments/runs/p4_editor_v1_ 1545 ev-i-dence\T1/ptm/m/n/12 (-20) : \T1/cmtt/m/n/12 experiments/runs/p4_editor_v1_
1545 seed01_*/metrics.jsonl 1546 seed01_*/metrics.jsonl
1546 [] 1547 []
1547 1548
1548− 1549+[39]
1549 Overfull \hbox (47.44559pt too wide) in paragraph at lines 12--14 1550 Overfull \hbox (47.44559pt too wide) in paragraph at lines 12--14
1550 \T1/cmtt/m/n/12 experiments/logs/p4_diag_mixed_w128_lr3e3.log\T1/ptm/m/n/12 (-2 1551 \T1/cmtt/m/n/12 experiments/logs/p4_diag_mixed_w128_lr3e3.log\T1/ptm/m/n/12 (-2
1551 0) , run \T1/cmtt/m/n/12 p4_diag_mixed_w128_lr3e3_seed01_*\T1/ptm/m/n/12 (-20) 1552 0) , run \T1/cmtt/m/n/12 p4_diag_mixed_w128_lr3e3_seed01_*\T1/ptm/m/n/12 (-20)
@@ -1570,15 +1571,15 @@ Overfull \hbox (12.79033pt too wide) in paragraph at lines 18--20
1570 run \T1/cmtt/m/n/12 p4_tiny_backbone_v2_seed01_* \T1/ptm/m/n/12 (-20) (state: 1571 run \T1/cmtt/m/n/12 p4_tiny_backbone_v2_seed01_* \T1/ptm/m/n/12 (-20) (state:
1571 [] 1572 []
1572 1573
1573−[38{/usr/local/texlive/2026/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.en 1574+
1574−c}]
1575 Overfull \hbox (12.01894pt too wide) in paragraph at lines 22--24 1575 Overfull \hbox (12.01894pt too wide) in paragraph at lines 22--24
1576 \T1/ptm/m/n/12 (-20) 0.4334); $\OML/ztmcm/m/it/12 ^^U$\T1/ptm/m/n/12 (-20) =0 i 1576 \T1/ptm/m/n/12 (-20) 0.4334); $\OML/ztmcm/m/it/12 ^^U$\T1/ptm/m/n/12 (-20) =0 i
1577 s worse (+1.3 %).; \T1/ptm/b/n/12 (-20) ev-i-dence\T1/ptm/m/n/12 (-20) : \T1/cm 1577 s worse (+1.3 %).; \T1/ptm/b/n/12 (-20) ev-i-dence\T1/ptm/m/n/12 (-20) : \T1/cm
1578 tt/m/n/12 experiments/runs/controller_v1rank_seed01_*\T1/ptm/m/n/12 (-20) , 1578 tt/m/n/12 experiments/runs/controller_v1rank_seed01_*\T1/ptm/m/n/12 (-20) ,
1579 [] 1579 []
1580 1580
1581− 1581+[40{/usr/local/texlive/2026/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.en
1582+c}]
1582 Overfull \hbox (146.55865pt too wide) in paragraph at lines 24--26 1583 Overfull \hbox (146.55865pt too wide) in paragraph at lines 24--26
1583 \T1/ptm/m/n/12 (-20) un-til step ~6,540 and was killed.; \T1/ptm/b/n/12 (-20) e 1584 \T1/ptm/m/n/12 (-20) un-til step ~6,540 and was killed.; \T1/ptm/b/n/12 (-20) e
1584 v-i-dence\T1/ptm/m/n/12 (-20) : \T1/cmtt/m/n/12 experiments/runs/pretrain103m_s 1585 v-i-dence\T1/ptm/m/n/12 (-20) : \T1/cmtt/m/n/12 experiments/runs/pretrain103m_s
@@ -1599,26 +1600,22 @@ dence\T1/ptm/m/n/12 (-20) : M2U64 \T1/cmtt/m/n/12 experiments/logs/dense_probe_
1599 103m*_seed1.log\T1/ptm/m/n/12 (-20) ; 1600 103m*_seed1.log\T1/ptm/m/n/12 (-20) ;
1600 [] 1601 []
1601 1602
1602−)) [39] [40] [41] [42] [43] [44] [45] (./main.aux) 1603+)) [41] [42] [43] [44] [45] [46] [47] [48] (./main.aux)
1603 *********** 1604 ***********
1604 LaTeX2e <2025-11-01> 1605 LaTeX2e <2025-11-01>
1605 L3 programming layer <2026-01-19> 1606 L3 programming layer <2026-01-19>
1606 *********** 1607 ***********
1607−
1608−
1609−LaTeX Warning: There were undefined references.
1610−
1611 Package rerunfilecheck Info: File `main.out' has not changed. 1608 Package rerunfilecheck Info: File `main.out' has not changed.
1612−(rerunfilecheck) Checksum: 5C078824F258CF894B89F741D112AD48;9386. 1609+(rerunfilecheck) Checksum: 2518BAC872CA29A47E8E1B5E242EBEEE;9386.
1613 ) 1610 )
1614 Here is how much of TeX's memory you used: 1611 Here is how much of TeX's memory you used:
1615− 36174 strings out of 467525 1612+ 36217 strings out of 467525
1616− 703202 string characters out of 5418982 1613+ 703862 string characters out of 5418982
1617− 1152637 words of memory out of 5000000 1614+ 1163186 words of memory out of 5000000
1618− 63458 multiletter control sequences out of 15000+600000 1615+ 63476 multiletter control sequences out of 15000+600000
1619− 793836 words of font info for 673 fonts, out of 8000000 for 9000 1616+ 794607 words of font info for 685 fonts, out of 8000000 for 9000
1620 1141 hyphenation exceptions out of 8191 1617 1141 hyphenation exceptions out of 8191
1621− 84i,13n,104p,1001b,1484s stack positions out of 10000i,1000n,20000p,200000b,200000s 1618+ 84i,13n,104p,1001b,1530s stack positions out of 10000i,1000n,20000p,200000b,200000s
1622 </usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmex10.pfb 1619 </usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmex10.pfb
1623 ></usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb> 1620 ></usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb>
1624 </usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></ 1621 </usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></
@@ -1634,10 +1631,10 @@ exlive/2026/texmf-dist/fonts/type1/urw/symbol/usyr.pfb></usr/local/texlive/2026
1634 e1/urw/times/utmbi8a.pfb></usr/local/texlive/2026/texmf-dist/fonts/type1/urw/ti 1631 e1/urw/times/utmbi8a.pfb></usr/local/texlive/2026/texmf-dist/fonts/type1/urw/ti
1635 mes/utmr8a.pfb></usr/local/texlive/2026/texmf-dist/fonts/type1/urw/times/utmri8 1632 mes/utmr8a.pfb></usr/local/texlive/2026/texmf-dist/fonts/type1/urw/times/utmri8
1636 a.pfb> 1633 a.pfb>
1637−Output written on main.pdf (45 pages, 447044 bytes). 1634+Output written on main.pdf (48 pages, 450927 bytes).
1638 PDF statistics: 1635 PDF statistics:
1639− 824 PDF objects out of 1000 (max. 8388607) 1636+ 839 PDF objects out of 1000 (max. 8388607)
1640− 709 compressed objects within 8 object streams 1637+ 721 compressed objects within 8 object streams
1641− 167 named destinations out of 1000 (max. 500000) 1638+ 171 named destinations out of 1000 (max. 500000)
1642 143785 words of extra memory for PDF output out of 154059 (max. 10000000) 1639 143785 words of extra memory for PDF output out of 154059 (max. 10000000)
1643 1640
modified paper/main.pdf +0 −0

Binary file not shown.

modified paper/sections/conclusion.tex +1 −1
@@ -1,7 +1,7 @@
1 \section{Conclusion}\label{sec:conclusion} 1 \section{Conclusion}\label{sec:conclusion}
2 The central question was whether counterfactual supervision of \emph{where} to edit a small latent workspace 2 The central question was whether counterfactual supervision of \emph{where} to edit a small latent workspace
3 improves the quality--compute frontier beyond uniform workspace computation and adaptive global recurrence. 3 improves the quality--compute frontier beyond uniform workspace computation and adaptive global recurrence.
4−On the evidence collected here the answer is no. The sparse editor is learnable and helps a frozen backbone 4+On the evidence collected here the answer is no, and the pre-registered decision rule says so: on the locked test regimes the primary effect is $\testprismebtwovsbzero$ points against a three-point threshold. The sparse editor is learnable and helps a frozen backbone
5 by a few percent of loss and two points of exact match when it is applied once or twice at every position, but 5 by a few percent of loss and two points of exact match when it is applied once or twice at every position, but
6 a global-recurrence adapter trained with the identical recipe helps at least as much at every budget; where to 6 a global-recurrence adapter trained with the identical recipe helps at least as much at every budget; where to
7 write barely matters for fixed schedules; an exhaustive oracle shows large opportunity (up to 7 write barely matters for fixed schedules; an exhaustive oracle shows large opportunity (up to
modified paper/sections/design.tex +1 −0
@@ -33,6 +33,7 @@ recipe and the identical symmetry-breaking device. Reference routers (random, un
33 entropy threshold, norm-change threshold) reuse the trained \prisme{} editor and are therefore matched in 33 entropy threshold, norm-change threshold) reuse the trained \prisme{} editor and are therefore matched in
34 everything except the choice of action. The exhaustive one-step and two-step oracles read the target and are 34 everything except the choice of action. The exhaustive one-step and two-step oracles read the target and are
35 reported as ceilings only. 35 reported as ceilings only.
36+\input{tables/controls.tex}
36 37
37 \subsection{Pre-registration, seeds and uncertainty} 38 \subsection{Pre-registration, seeds and uncertainty}
38 The primary outcome, metrics, decision thresholds and seed plan were written to 39 The primary outcome, metrics, decision thresholds and seed plan were written to
modified paper/sections/results_pilot.tex +6 −2
@@ -17,8 +17,12 @@ global adapter, which did not collapse at $10^{-3}$, gives $-\pilotgrgainbone$,
17 $-\pilotgrgainbfour$ and $-\pilotgrgainbeight\,\%$ in \pilotgrminutes{} min. The dense sixteen-slot probe 17 $-\pilotgrgainbfour$ and $-\pilotgrgainbeight\,\%$ in \pilotgrminutes{} min. The dense sixteen-slot probe
18 collapsed at both learning rates and yields no 103M result. On general web text, therefore, neither adapter 18 collapsed at both learning rates and yields no 103M result. On general web text, therefore, neither adapter
19 moves a frozen 100M backbone by more than a tenth of a percent of loss with 24.6M adapter tokens---less than the 19 moves a frozen 100M backbone by more than a tenth of a percent of loss with 24.6M adapter tokens---less than the
20−run-to-run noise of pretraining itself---so H1 is not supported outside the synthetic suite, and the 20+run-to-run noise of pretraining itself---so H1 is not supported outside the synthetic suite. On the locked
21−parameter-matched plain decoder is the relevant comparison (Table~\ref{tab:pilotppl}). 21+FineWeb-Edu test split (327,680 tokens) the backbone's perplexity is \pilotpplbackbonemtest{}; forced edits
22+change the test loss by \pilottestprismemeditorbfour\,\% (sparse, budget 4) and \pilottestgrmbfour\,\%
23+(global); the parameter-matched plain decoder trained with the identical recipe reaches \pilotpplenlargedmtest{}
24+(Table~\ref{tab:pilotppl})---the proposal's null ``a larger ordinary Transformer wins after equalising training
25+compute'' holds at this budget.
22 \input{tables/pilot_ppl.tex} 26 \input{tables/pilot_ppl.tex}
23 27
24 \paragraph{Stages C--D and routing.} Counterfactual labels were collected on a validation-disjoint stream of 28 \paragraph{Stages C--D and routing.} Counterfactual labels were collected on a validation-disjoint stream of
modified paper/tables/controls.tex +11 −10
@@ -1,15 +1,16 @@
1 \begin{table}[t]\centering\small 1 \begin{table}[t]\centering\small
2−\caption{Systems and controls. ``bb'' = backbone, ``6NP'' = the usual training-FLOP estimate $6\,N\,P$ for $N$ tokens 2+\caption{Systems and controls. ``bb'' = backbone, ``$6NP$'' = the usual training-FLOP estimate for $N$ tokens and $P$
3−and $P$ parameters; adapter training additionally executes up to 8 editor applications per position. Adapter 3+parameters; adapter training additionally executes up to 8 editor applications per position. Adapter parameters are
4−parameters are unique parameters outside the backbone (counted by instantiation).} 4+unique parameters outside the backbone (counted by instantiation; the 103M editor includes the residual-plus-entropy
5+controller features).}
5 \label{tab:controls} 6 \label{tab:controls}
6 \begin{tabular}{lrrlll}\toprule 7 \begin{tabular}{lrrlll}\toprule
7 System & backbone & adapter & training exposure & training compute & inference regime\\\midrule 8 System & backbone & adapter & training exposure & training compute & inference regime\\\midrule
8−Frozen backbone v2 (budget 0) & 4,754,688 & 182,725 & synthetic train, 5k steps x 64 & backbone only & no workspace (bitwise bypass)\\ 9+Frozen backbone v2 (budget 0) & 4,754,688 & 182,725 & synthetic train, 5k steps $\times$ 64 & backbone only & no workspace (bitwise bypass)\\
9−PRISME sparse editor (tiny) & 4,754,688 & 182,725 & same + 3k adapter steps x 32 & frozen bb fwd + editor & 1-2 slots/action, 0-8 actions\\ 10+PRISME sparse editor (tiny) & 4,754,688 & 182,725 & same + 3k adapter steps $\times$ 32 & frozen bb fwd + editor & 1--2 slots/action, 0--8 actions\\
10−Global-recurrence adapter (tiny) & 4,754,688 & 172,608 & same & frozen bb fwd + block & whole latent/action, 0-8 actions\\ 11+Global-recurrence adapter (tiny) & 4,754,688 & 172,608 & same & frozen bb fwd + block & whole latent/action, 0--8 actions\\
11−Dense 16-slot editor (tiny) & 4,754,688 & 182,725 & same & frozen bb fwd + 16 lanes & 16 slots/action, 0-8 actions\\ 12+Dense 16-slot editor (tiny) & 4,754,688 & 182,725 & same & frozen bb fwd + 16 lanes & 16 slots/action, 0--8 actions\\
12−PRISME-103M (editor pilot) & 99,502,848 & 2,762,757 & FineWeb-Edu 196.6M tok + 24.6M adapter tok & 6NP + editor & 1-2 slots/action\\ 13+PRISME-103M sparse editor & 99,502,848 & 2,762,757 & FineWeb-Edu 98.3M tok + 24.6M adapter tok & $6NP$ + editor & 1--2 slots/action\\
13−Global-recurrence adapter (103M) & 99,502,848 & 2,623,488 & same & 6NP + block & whole latent/action\\ 14+Global-recurrence adapter (103M) & 99,502,848 & 2,623,488 & same & $6NP$ + block & whole latent/action\\
14−Plain decoder, parameter-matched (102.2M) & 102,157,056 & 0 & FineWeb-Edu 196.6M tok (same recipe) & 6NP & no workspace\\ 15+Plain decoder, parameter-matched (102.2M) & 102,157,056 & 0 & FineWeb-Edu 98.3M tok (same recipe) & $6NP$ & no workspace\\
15 \bottomrule\end{tabular}\end{table} 16 \bottomrule\end{tabular}\end{table}
modified paper/tables/numbers.tex +16 −1
@@ -63,12 +63,27 @@
63 \newcommand{\pilotlatctrl}{11.0} 63 \newcommand{\pilotlatctrl}{11.0}
64 \newcommand{\pilotlatdeight}{26.9} 64 \newcommand{\pilotlatdeight}{26.9}
65 \newcommand{\pilotlatseight}{22.9} 65 \newcommand{\pilotlatseight}{22.9}
66−\newcommand{\pilotpplgrm}{48.16} 66+\newcommand{\pilotpplbackbonemtest}{45.72}
67+\newcommand{\pilotpplbackbonemval}{48.16}
68+\newcommand{\pilotpplenlargedmtest}{45.64}
69+\newcommand{\pilotpplenlargedmval}{47.89}
70+\newcommand{\pilotpplgrmtest}{45.72}
71+\newcommand{\pilotpplgrmval}{48.16}
72+\newcommand{\pilotpplprismemeditortest}{45.72}
73+\newcommand{\pilotpplprismemeditorval}{48.16}
67 \newcommand{\pilotroutingctrl}{-0.01} 74 \newcommand{\pilotroutingctrl}{-0.01}
68 \newcommand{\pilotroutingctrledits}{0.61} 75 \newcommand{\pilotroutingctrledits}{0.61}
69 \newcommand{\pilotroutingoracle}{-3.6} 76 \newcommand{\pilotroutingoracle}{-3.6}
70 \newcommand{\pilotroutingoracleedits}{4.28} 77 \newcommand{\pilotroutingoracleedits}{4.28}
71 \newcommand{\pilotroutinguniformone}{-0.02} 78 \newcommand{\pilotroutinguniformone}{-0.02}
79+\newcommand{\pilottestbackbonembfour}{+0.00}
80+\newcommand{\pilottestbackbonembtwo}{+0.00}
81+\newcommand{\pilottestenlargedmbfour}{+0.00}
82+\newcommand{\pilottestenlargedmbtwo}{+0.00}
83+\newcommand{\pilottestgrmbfour}{-0.08}
84+\newcommand{\pilottestgrmbtwo}{-0.06}
85+\newcommand{\pilottestprismemeditorbfour}{-0.08}
86+\newcommand{\pilottestprismemeditorbtwo}{-0.06}
72 \newcommand{\routingbzero}{0.4334} 87 \newcommand{\routingbzero}{0.4334}
73 \newcommand{\routingctrllambda}{0.3} 88 \newcommand{\routingctrllambda}{0.3}
74 \newcommand{\routingctrlledits}{0.22} 89 \newcommand{\routingctrlledits}{0.22}
modified paper/tables/pilot_ppl.tex +45 −9
@@ -1,12 +1,48 @@
1 \begin{table}[t]\centering\small 1 \begin{table}[t]\centering\small
2−\caption{103M pilot: held-out cross-entropy (nats/token) and perplexity on 327,680 FineWeb-Edu validation tokens 2+\caption{103M pilot: held-out cross-entropy (nats/token) and perplexity on 327,680 FineWeb-Edu validation tokens and
3−(40 $\times$ 16 $\times$ 512) under forced random budgets and, for PRISME, under the locked controller.} 3+on the LOCKED test split (40 $\times$ 16 $\times$ 512 tokens each) under forced random budgets. Budget 0 of the adapters
4+equals the backbone bitwise.}
4 \label{tab:pilotppl} 5 \label{tab:pilotppl}
5−\begin{tabular}{llrr}\toprule 6+\begin{tabular}{lllrr}\toprule
6−System & policy & loss & ppl\\\midrule 7+System & split & policy & loss & ppl\\\midrule
7−gr 103m & forced b0 & 3.8746 & 48.16\\ 8+backbone 103m & val & forced b0 & 3.8746 & 48.16\\
8−gr 103m & forced b1 & 3.8731 & 48.09\\ 9+backbone 103m & val & forced b1 & 3.8746 & 48.16\\
9−gr 103m & forced b2 & 3.8722 & 48.05\\ 10+backbone 103m & val & forced b2 & 3.8746 & 48.16\\
10−gr 103m & forced b4 & 3.8714 & 48.01\\ 11+backbone 103m & val & forced b4 & 3.8746 & 48.16\\
11−gr 103m & forced b8 & 3.8718 & 48.03\\ 12+backbone 103m & val & forced b8 & 3.8746 & 48.16\\
13+enlarged 102m & val & forced b0 & 3.8689 & 47.89\\
14+enlarged 102m & val & forced b1 & 3.8689 & 47.89\\
15+enlarged 102m & val & forced b2 & 3.8689 & 47.89\\
16+enlarged 102m & val & forced b4 & 3.8689 & 47.89\\
17+enlarged 102m & val & forced b8 & 3.8689 & 47.89\\
18+gr 103m & val & forced b0 & 3.8746 & 48.16\\
19+gr 103m & val & forced b1 & 3.8731 & 48.09\\
20+gr 103m & val & forced b2 & 3.8722 & 48.05\\
21+gr 103m & val & forced b4 & 3.8714 & 48.01\\
22+gr 103m & val & forced b8 & 3.8718 & 48.03\\
23+prisme 103m editor & val & forced b0 & 3.8746 & 48.16\\
24+prisme 103m editor & val & forced b1 & 3.8731 & 48.09\\
25+prisme 103m editor & val & forced b2 & 3.8722 & 48.05\\
26+prisme 103m editor & val & forced b4 & 3.8714 & 48.01\\
27+prisme 103m editor & val & forced b8 & 3.8718 & 48.03\\
28+backbone 103m & test & forced b0 & 3.8226 & 45.72\\
29+backbone 103m & test & forced b1 & 3.8226 & 45.72\\
30+backbone 103m & test & forced b2 & 3.8226 & 45.72\\
31+backbone 103m & test & forced b4 & 3.8226 & 45.72\\
32+backbone 103m & test & forced b8 & 3.8226 & 45.72\\
33+gr 103m & test & forced b0 & 3.8226 & 45.72\\
34+gr 103m & test & forced b1 & 3.8211 & 45.66\\
35+gr 103m & test & forced b2 & 3.8203 & 45.62\\
36+gr 103m & test & forced b4 & 3.8195 & 45.58\\
37+gr 103m & test & forced b8 & 3.8201 & 45.61\\
38+prisme 103m editor & test & forced b0 & 3.8226 & 45.72\\
39+prisme 103m editor & test & forced b1 & 3.8211 & 45.66\\
40+prisme 103m editor & test & forced b2 & 3.8202 & 45.61\\
41+prisme 103m editor & test & forced b4 & 3.8195 & 45.58\\
42+prisme 103m editor & test & forced b8 & 3.8200 & 45.60\\
43+enlarged 102m & test & forced b0 & 3.8207 & 45.64\\
44+enlarged 102m & test & forced b1 & 3.8207 & 45.64\\
45+enlarged 102m & test & forced b2 & 3.8207 & 45.64\\
46+enlarged 102m & test & forced b4 & 3.8207 & 45.64\\
47+enlarged 102m & test & forced b8 & 3.8207 & 45.64\\
12 \bottomrule\end{tabular}\end{table} 48 \bottomrule\end{tabular}\end{table}
added reproducibility/audit_report.md +16 −0
@@ -0,0 +1,16 @@
1+# Results-to-claim audit report (WP31, 2026-10-30)
2+
3+* `scripts/verify_manuscript_claims.py`: **AUDIT PASS** — 25 registered claims; every artefact path exists; every run id has a
4+ manifest (states `completed`, or `stopped-run-used` for the two deliberately stopped runs cited as failures); every numeric
5+ macro used in the manuscript is generated by `scripts/make_tables.py`; no promotional phrase; no placeholder section.
6+* Tests: `python -m pytest tests -q` → 23 passed (15 mandated invariants, controller target-freedom with and without the
7+ residual/entropy features, dense-write ablation, 5 cache-equivalence tests).
8+* Manuscript: `paper/main.pdf`, 48 pages, latexmk with 0 undefined references/citations (one harmless font-shape warning).
9+* Pre-registration: `protocol/preregistration.md` frozen v1.0 at 2026-10-30T04:04-0400; `protocol/locked_test_manifest.json`
10+ unsealed at the same time; test splits were read only afterwards (synthetic test regimes: 5 nodes; FineWeb test: 4 systems).
11+* Deviations: `protocol/deviations.md` (bootstrap init/scale, controller features, label stream, dense 103M infeasible,
12+ backbone budget 98.3M instead of 196.6M tokens).
13+* Not executed: Phase 11 (MLX port), Phase 12 training (459M/1.18B only instantiated on meta), GSM8K/EvalPlus, M ∈ {4,8,32},
14+ editor depth ∈ {1,4}, rematched pairs, quantisation, distillation, joint fine-tuning (one degraded attempt), dense editor at 103M
15+ (collapsed twice), MLX/Metal profiling.
16+* Checksums of the key artefacts: `reproducibility/checksums.sha256`.
added reproducibility/checksums.sha256 +213 −0
@@ -0,0 +1,213 @@
1+7d2419ffa1f2892bc7dafc95685f767cc79de6554e219f7f1a76570c8d3790b1 paper/main.pdf
2+a1a3468ffe50fba1beb150871b8d5ad575163ba1cd96373838d7ff989380853d results/claim_registry.csv
3+70cb86ea329cb0052e0956db0374d8e55b43305056edd778e7512cf5ee6fef02 results/analyzed/labels_labels_p4_seed01_20261030000543.json
4+e5765cbde70d9faf34b4aec875a44184b8ae368531d5ccca29e5ecfddb4a020e results/analyzed/labels_labels_v1ctrl_seed01_20261030004018.json
5+77f6c3406df0ed977b90c4b5dbeccfecfff63a5054a87dbe1c6ddb3e23acc53d results/analyzed/routing_seeds_v2.json
6+c121c2bfd141a3ffbc161c8213bb9189507a965434c22b65005432f9cb723a58 results/analyzed/seed_matrix_v2.json
7+de0677e8600a9d82c293ad49469fdb5346a3fe55f05d8f6d24315dadcd5f147c results/analyzed/statistics_v2.json
8+b7923aef253503c8d8ea1740c9791b691a2b620519eba798c1d34fc12dd1e64d results/analyzed/test_statistics.json
9+7945a43fde621b133add69976be3d1f50bfa6a3e976bacae6a8075f6663d4312 results/raw/eval_backbone_103m_test.json
10+1db7d53edf42e2065ee66749b5b1231a0e793a51d3b7b3bad0475e907fd65fad results/raw/eval_backbone_103m_val.json
11+56efdc8762fefc5ee2a6dd0e3ee3a88edfe51450576838862ca4528f85df8f99 results/raw/eval_enlarged_102m_test.json
12+4ed5f6d040e0bb84741a54de6de385a76a4db78d26a883c31ccd4442223ae877 results/raw/eval_enlarged_102m_val.json
13+24a17a932585a6a46116375b71c4e72847892b58070d3d979e1c17e41cee13db results/raw/eval_gr_103m_test.json
14+94d76e5cdf6da940f16ec113eca2daf25fe9069b35c6537f83eea513ea6caad6 results/raw/eval_gr_103m_val.json
15+240a85d62d9ebe4a274314e0d50a2e8c2ad4b4b581bfdb5f95d0579d132c5604 results/raw/eval_p4v2_dense_s1_test_compose.json
16+b4601f1bfb75d0618c8a48e2a7b9f15d8b9f0906192c1235c3a355d0c95d25e9 results/raw/eval_p4v2_dense_s1_test_depth.json
17+0de49b5dc700f1db63066d69d8bc5fcfb6dc12a1efaefb72be033840f1be13c7 results/raw/eval_p4v2_dense_s1_test_iid.json
18+a4bf328b91b69fd77878f7d278d3d81e8d708ea809e84eb82e55dd08e3e580ae results/raw/eval_p4v2_dense_s1_test_template.json
19+59348dced041971ace2836f13d744ae07e1e1be9ecf21ab35b254db499d0ea7e results/raw/eval_p4v2_dense_s2_test_compose.json
20+f9b5bd5d98040d5556d5b624987c1d4f10f89f3af74480ce909167ab14b9465f results/raw/eval_p4v2_dense_s2_test_depth.json
21+e0c772078e0277a185890e6a0f37bee555be1e2856fd181f62ce5b067a9a36e9 results/raw/eval_p4v2_dense_s2_test_iid.json
22+b8a8f4bd4feccf6468755ee8eee6cc73baf50726ff3e4a7384a849d845ce9460 results/raw/eval_p4v2_dense_s2_test_template.json
23+ff25f532fa987ba08027d677b77e730308eb98debe5b2861e9d63ad896daf047 results/raw/eval_p4v2_dense_s3_test_compose.json
24+4c22fe9f58e85b80c7a04d66a367dc9d8468df7a37feab0a55e96a488d86108c results/raw/eval_p4v2_dense_s3_test_depth.json
25+ccae7be5c414503e6916af1b496ac5d76e8ccf10ed02938bc302b3063dd55bcd results/raw/eval_p4v2_dense_s3_test_iid.json
26+19a05738052fdb253b6b0cca58386c80a98b4a460d399d4eb86587b499760d7e results/raw/eval_p4v2_dense_s3_test_template.json
27+553412dbdae6f569ff2a4aaba12cebbaf46b545e63f17b2c92af727db3f2cacc results/raw/eval_p4v2_dense_s4_test_compose.json
28+9c9a7cb00ba1e9e2edf419961966508cf00fb8bd9b8c5e19bc4297f875d4f3db results/raw/eval_p4v2_dense_s4_test_depth.json
29+6d16a9462d9e90ed1210f095b30445a1abc8e95fabf4d5262d26b8bcc9f75821 results/raw/eval_p4v2_dense_s4_test_iid.json
30+cb601cd8f310f29e19ce6b880d19dd68178d83970d8eb2659c79dd24bcef7f55 results/raw/eval_p4v2_dense_s4_test_template.json
31+a6cabd657169d423a1d49befb485d161f79ca7d7c31c457e6467870a315fc60b results/raw/eval_p4v2_dense_s5_test_compose.json
32+40cb9aa807935ef72c86d1cd2a41d3a3a76fa933c9f36c50a5216aa31c50245f results/raw/eval_p4v2_dense_s5_test_depth.json
33+7d6bc6039c4c2be34c18d9016350765e35bccc4f2a8e7f7b13dbddabe22af1ce results/raw/eval_p4v2_dense_s5_test_iid.json
34+6a18b127adeb7267d2a41c3113215c3d9bd56aa48189461bba2ba383fc131437 results/raw/eval_p4v2_dense_s5_test_template.json
35+a89e20b97e580615df0d3818d5280af6a722d8ae442d898bd5c945ea0550936d results/raw/eval_p4v2_gr_s1_test_compose.json
36+bd42a30c976c62752907bb84e71dd43a3634a27c937d5e82093ce7dd6fe87905 results/raw/eval_p4v2_gr_s1_test_depth.json
37+5636d906e8c4aadf73b89a2e10fd4e0a6dd1c62099949eda5bf9ee92fa630e47 results/raw/eval_p4v2_gr_s1_test_iid.json
38+88c25f1afa4e344e2a09f95fa4356bef066288459a5237a4efa2fc6cbcb4deaf results/raw/eval_p4v2_gr_s1_test_template.json
39+e83025dcb455a1e417098583271de0a5f460d86086a0b8beb034fef12bf304cd results/raw/eval_p4v2_gr_s2_test_compose.json
40+c93d645871dd63aebaef1e9c5b43725bbf9c7b0cd9805ebe5c7c67683742ef89 results/raw/eval_p4v2_gr_s2_test_depth.json
41+4adae524b80f03a7b3db761f6449e28a5d191b9d35d864eef208a2ad014cd9b5 results/raw/eval_p4v2_gr_s2_test_iid.json
42+ff3b6070ecca2de27276c2b6c81a045bc05cc32ed696d30c581b7b5b4c490a86 results/raw/eval_p4v2_gr_s2_test_template.json
43+cc3310f947c2ab0556978bd721d68d11b8cb252fb52a4e5c44710956b4339415 results/raw/eval_p4v2_gr_s3_test_compose.json
44+c48b3b0ba3533819c092ebcde1a578f69d44cec34a38247e2a2eb1c59489e01a results/raw/eval_p4v2_gr_s3_test_depth.json
45+f9df369e3f358110e301413d6a6071aefbed562c08f06957d71514af738f9827 results/raw/eval_p4v2_gr_s3_test_iid.json
46+b2281f2b745aa5be1972dbb0d111e60f3b39ec6b2e0a7e228a852a5a158b629b results/raw/eval_p4v2_gr_s3_test_template.json
47+daff4678d7c288ffc1c891dd8fc73ea63e9c00ef165fee1c786f7ab6910d27ef results/raw/eval_p4v2_gr_s4_test_compose.json
48+edcdfebbc348086b2578c00c4ffd757e0d135bf24b67c202598b09961e39080a results/raw/eval_p4v2_gr_s4_test_depth.json
49+5275701b43e763cea01617bbc6ba406622d7dada9975e17d152543e4d86e9b37 results/raw/eval_p4v2_gr_s4_test_iid.json
50+781772b860a0d8171b135122d05bcbef9402bfa75e302d64f75a5eff54d297bd results/raw/eval_p4v2_gr_s4_test_template.json
51+3f44da7d05b20404c12639942152172942faf325b81b89391e1b7d300ac1ec6d results/raw/eval_p4v2_gr_s5_test_compose.json
52+01ea31f8424a5a00b90164cb641e6e8af66dbb0e8176bdb4da8aa572a7f1268f results/raw/eval_p4v2_gr_s5_test_depth.json
53+8dd943be0d13f10b3eab74249e96bf474e8cb761f111323df42b3619e46ae875 results/raw/eval_p4v2_gr_s5_test_iid.json
54+6dbccfdd4ea90d91b2bc37adba864c4d44866b2de489956e379e7f6f5e106e04 results/raw/eval_p4v2_gr_s5_test_template.json
55+21fd0d5905ca637f9b79a7be000909865197774dca6f47252d007caa620e834a results/raw/eval_p4v2_prisme_s1_test_compose.json
56+f313c6d0183813f2a1273d7a79f34d91436c1951933482c368392df81d779f31 results/raw/eval_p4v2_prisme_s1_test_depth.json
57+2912cf4bb956a7d1483e8246909e0f6fcf44b4fd66cdf41c87dd70f89403407f results/raw/eval_p4v2_prisme_s1_test_iid.json
58+e7ce172fdbcd7daee5b41c1cce7e7935201f55a862b748b47f767c5f0fbde186 results/raw/eval_p4v2_prisme_s1_test_template.json
59+9cf33ba1b80f7bd9e0740f2de0bbb51a9044b573094a6178c629447e386aba41 results/raw/eval_p4v2_prisme_s2_test_compose.json
60+7b09000cca0ce54acdd7b989ee52ebbe8c977af5967949d4ff2d6f052b152640 results/raw/eval_p4v2_prisme_s2_test_depth.json
61+7b029010b158cb1654792bb03c31c3348ebbc3d1e1fb2e02dc0ecc2b4ca9d4d2 results/raw/eval_p4v2_prisme_s2_test_iid.json
62+432e089082d068e591934353180b450c6f1b6899e8146af8f058fdbc0e041705 results/raw/eval_p4v2_prisme_s2_test_template.json
63+32d899d80fe6e96c75145a0cad12cbb0b1d46743e0bc5097ebb2b3a1dceba931 results/raw/eval_p4v2_prisme_s3_test_compose.json
64+0cb5219b36308c768f96fd23008284b2470d773e87c9cf54200cdc5b95ee53c3 results/raw/eval_p4v2_prisme_s3_test_depth.json
65+9f614b3f111cd9f97e5b5fe376f508f96311a1db33c67a3862dcd0eb8cf8e2e4 results/raw/eval_p4v2_prisme_s3_test_iid.json
66+aa3eeac5cc2c680170fa82e19e5e6a748f92771285d110fc80b001f04b7f7a5d results/raw/eval_p4v2_prisme_s3_test_template.json
67+508ba5332b8d84219703c07971c6212f9e8d8c5fee0e7e15bec90bb19bac836c results/raw/eval_p4v2_prisme_s4_test_compose.json
68+b667a0ddc3794ee2412a1ff183631a5ba9f67c417ea21c8a46a849ce6c04d888 results/raw/eval_p4v2_prisme_s4_test_depth.json
69+833f5eed3a48185c17ea58dfa76e297d93f6b875bb08651bc324120a30f8b3bf results/raw/eval_p4v2_prisme_s4_test_iid.json
70+b2e41313e851338880f5ae5ac181fb665775f5ef90ffe8a458e79952b60079da results/raw/eval_p4v2_prisme_s4_test_template.json
71+5a78fa1db60b6f2d699e404540c43cdfffa8264290d76804d7cc88741d7d663c results/raw/eval_p4v2_prisme_s5_test_compose.json
72+6fae55bfa3d1531da0047057efab493fad5d59387631a6af5ce50d59c2f17898 results/raw/eval_p4v2_prisme_s5_test_depth.json
73+26b429fd5eb464ad19d7502d4de4bfc1bf8ff3c138c63e035da2d36bb8ac0dc2 results/raw/eval_p4v2_prisme_s5_test_iid.json
74+169577d4954ca6786db7e95a0b10d3f5e67821154908097895fd67e54ab62020 results/raw/eval_p4v2_prisme_s5_test_template.json
75+83200604d660282d28acbf8eca212562915edd1bff3ed19d18010195ce98fb54 results/raw/eval_prisme_103m_editor_test.json
76+bb1080bbcfb2ff25471be1dc13409dab4531b7a07f4f133e1427f41b2c4d047c results/raw/eval_prisme_103m_editor_val.json
77+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_dense_s1_test_compose_budget0.json
78+2e999f28ca2c658f23598b80bc9952815f9f23d385c960bd1d5d95e51cca1d17 results/raw/items_p4v2_dense_s1_test_compose_forced_b2.json
79+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_dense_s1_test_depth_budget0.json
80+c509d6ebf76ed9fd8f3ba93c7c0bed1d9dc062f599f1572f9b814b25b76e1ece results/raw/items_p4v2_dense_s1_test_depth_forced_b2.json
81+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_dense_s1_test_iid_budget0.json
82+c767436f7a6c79a2c93cdf99096249263cd1432d7a1a4562945ddf407eaca69d results/raw/items_p4v2_dense_s1_test_iid_forced_b2.json
83+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_dense_s1_test_template_budget0.json
84+8448cc79e9d1e9ab28274143142207f345427d12dccea5a23b5a4417eece564b results/raw/items_p4v2_dense_s1_test_template_forced_b2.json
85+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_dense_s2_test_compose_budget0.json
86+16a7d8b011b8a10caef1ba95393d488e19a2d850eeaf80a8aa95ba14d5b04c1d results/raw/items_p4v2_dense_s2_test_compose_forced_b2.json
87+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_dense_s2_test_depth_budget0.json
88+a02eaf64716ddddcfb10708cdd215597f525323881fff3e1831731be22814924 results/raw/items_p4v2_dense_s2_test_depth_forced_b2.json
89+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_dense_s2_test_iid_budget0.json
90+dd07b669a62500527d704d123584184be2aa19e032e1b933e3a83d3d7c466fc1 results/raw/items_p4v2_dense_s2_test_iid_forced_b2.json
91+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_dense_s2_test_template_budget0.json
92+f51973ecfd545bc9780240e219a2ad5bd98cf83ca1085add5c65eb800a8653fe results/raw/items_p4v2_dense_s2_test_template_forced_b2.json
93+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_dense_s3_test_compose_budget0.json
94+307a8c7781f3c75a5a65edb17140ef94be91fbf2d5776607a4f75a32a2f3f575 results/raw/items_p4v2_dense_s3_test_compose_forced_b2.json
95+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_dense_s3_test_depth_budget0.json
96+7ef70865bf95cd33268c5e2124ab6f7a649e128f09d51053891d16ca63e950b8 results/raw/items_p4v2_dense_s3_test_depth_forced_b2.json
97+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_dense_s3_test_iid_budget0.json
98+ab88e74b8581317c7e4fbd0c6e3f4a55c51d4209f344fcbf90d004787fbef4c0 results/raw/items_p4v2_dense_s3_test_iid_forced_b2.json
99+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_dense_s3_test_template_budget0.json
100+60a01032ada2b17ae0b1cc80bee58da6167eba49f678e206f6dca3e832eca4fe results/raw/items_p4v2_dense_s3_test_template_forced_b2.json
101+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_dense_s4_test_compose_budget0.json
102+f93c20b054f2760e287a8dab70fd6fed984f0d7b4e137a488d7a515e4748ca0b results/raw/items_p4v2_dense_s4_test_compose_forced_b2.json
103+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_dense_s4_test_depth_budget0.json
104+a1c523c689ee99410142d4e89cb3997a10342a00e712dab11bd37d560d1a5928 results/raw/items_p4v2_dense_s4_test_depth_forced_b2.json
105+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_dense_s4_test_iid_budget0.json
106+c9cfad851190b133393b90636bb74625f1434226cc1c13c7744fad5714bb1b46 results/raw/items_p4v2_dense_s4_test_iid_forced_b2.json
107+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_dense_s4_test_template_budget0.json
108+ddd0fe249aa523c6c8b5e65666829710af812c4503001b425a8c33af637a6761 results/raw/items_p4v2_dense_s4_test_template_forced_b2.json
109+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_dense_s5_test_compose_budget0.json
110+1ed8e804351e8c2daf199cba00355d83db19d363f4830ab024f7fd0a37f6fe60 results/raw/items_p4v2_dense_s5_test_compose_forced_b2.json
111+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_dense_s5_test_depth_budget0.json
112+c99c501680904e3cf2d2f6845a6bc4f4a15b10a581ecc7592da8bba5a9d60924 results/raw/items_p4v2_dense_s5_test_depth_forced_b2.json
113+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_dense_s5_test_iid_budget0.json
114+91568b5ec9200783cf78320454f1cf988d14d418a93d5198e5bd1e62a1cd1a29 results/raw/items_p4v2_dense_s5_test_iid_forced_b2.json
115+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_dense_s5_test_template_budget0.json
116+505f27c6fffe6875e91f62221bb14693a3bf0467fd091bfd41b2b1e032872dd9 results/raw/items_p4v2_dense_s5_test_template_forced_b2.json
117+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_gr_s1_test_compose_budget0.json
118+e464fdc8a2b60c7609843dce2a8c22747bd897c1adb1886616eb9d4210f35c17 results/raw/items_p4v2_gr_s1_test_compose_forced_b2.json
119+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_gr_s1_test_depth_budget0.json
120+8aad9b15f27d52155e4928ebe77e36aea8bf19390f044531fcf0733dd5f5adfc results/raw/items_p4v2_gr_s1_test_depth_forced_b2.json
121+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_gr_s1_test_iid_budget0.json
122+5fd4cde26848fb97275e93925c6e24c8777da268be39dad2e8962dc604c4aaa8 results/raw/items_p4v2_gr_s1_test_iid_forced_b2.json
123+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_gr_s1_test_template_budget0.json
124+3e77bad1a73c4a811c92611002499a7e948f19c2bdec93c442ad4578f2e8eaec results/raw/items_p4v2_gr_s1_test_template_forced_b2.json
125+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_gr_s2_test_compose_budget0.json
126+7fee922e3864adcf6eb2d7d66353393ec4df5948769c1c8adb06d920ded6c0cb results/raw/items_p4v2_gr_s2_test_compose_forced_b2.json
127+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_gr_s2_test_depth_budget0.json
128+93db1e816557fdcfbdf374cfc12f6eb08cc8b88d1bfce4c49a594756f5247b07 results/raw/items_p4v2_gr_s2_test_depth_forced_b2.json
129+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_gr_s2_test_iid_budget0.json
130+187f6a7fc6131419ac322a7005e11b58bd829938042d5ab0eae81285ed62c189 results/raw/items_p4v2_gr_s2_test_iid_forced_b2.json
131+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_gr_s2_test_template_budget0.json
132+c9f640c16d349590b8dde575cc8218e93e653e0e9f5f060b87e144dee4a2d8af results/raw/items_p4v2_gr_s2_test_template_forced_b2.json
133+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_gr_s3_test_compose_budget0.json
134+b05f703226a7c4db4484e46c027bbbe768f8e41d8a4cdcdff07a64aaf0d67e7f results/raw/items_p4v2_gr_s3_test_compose_forced_b2.json
135+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_gr_s3_test_depth_budget0.json
136+ecbb22971316f411cccc02692f4eefe40a297771824c8b94a74645399f1a3379 results/raw/items_p4v2_gr_s3_test_depth_forced_b2.json
137+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_gr_s3_test_iid_budget0.json
138+d292ff34868f76e572e1ad4daf6545e222f0240f5fcb8fdd3a8921e3840cdb0d results/raw/items_p4v2_gr_s3_test_iid_forced_b2.json
139+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_gr_s3_test_template_budget0.json
140+8b43517446e9f59c9153606c135bf048feec04406b30bc9d26f4a4a43c8d9484 results/raw/items_p4v2_gr_s3_test_template_forced_b2.json
141+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_gr_s4_test_compose_budget0.json
142+ab3e9552cd2a6a230cdb8f33a4973893bbb107ba4f5cdb1c31cc74c86d8c2a26 results/raw/items_p4v2_gr_s4_test_compose_forced_b2.json
143+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_gr_s4_test_depth_budget0.json
144+487cd30e77ff443531b33cd047f9eb401a9e4177cbad513bb6e75e8e09e68abf results/raw/items_p4v2_gr_s4_test_depth_forced_b2.json
145+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_gr_s4_test_iid_budget0.json
146+1d353dfdbc37d45ecf8a90d8846c5325830c91fc4ac3f69a06ce8fe265a28595 results/raw/items_p4v2_gr_s4_test_iid_forced_b2.json
147+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_gr_s4_test_template_budget0.json
148+476b357454472720aeb0730eb10903c7251683fe8d3a7991b0df2eeaffb70aad results/raw/items_p4v2_gr_s4_test_template_forced_b2.json
149+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_gr_s5_test_compose_budget0.json
150+b79c36c2b71c270f90e54776c0e470fce0d15f8376bf751b7b7781c6fcf63332 results/raw/items_p4v2_gr_s5_test_compose_forced_b2.json
151+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_gr_s5_test_depth_budget0.json
152+f2f016d4cd3346e60eacc131cc100d462a21638d3490c5a240d51a9d46315dc4 results/raw/items_p4v2_gr_s5_test_depth_forced_b2.json
153+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_gr_s5_test_iid_budget0.json
154+2d5df71b91fe76a80463efde1a216de1a2a6339aacd9e0c584cfbfa0938a629d results/raw/items_p4v2_gr_s5_test_iid_forced_b2.json
155+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_gr_s5_test_template_budget0.json
156+32eb5e11f354321177fbd7dbd0841473cf2270e4cf9df5a0f9464dcbfb59ad9e results/raw/items_p4v2_gr_s5_test_template_forced_b2.json
157+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_prisme_s1_test_compose_budget0.json
158+80586b49e01e888dfb56a0a5db4e54d58cb4856668ca21278c86ebe4e11506aa results/raw/items_p4v2_prisme_s1_test_compose_forced_b2.json
159+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_prisme_s1_test_depth_budget0.json
160+4ab1a75242a89a0af0e6d024749d2cf088c3b8e58c9f2f663940be9564c52b2b results/raw/items_p4v2_prisme_s1_test_depth_forced_b2.json
161+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_prisme_s1_test_iid_budget0.json
162+8391d02c1a270e306ee86ebfc6fed306ab2817bbef784e6b807b993a08b9e5bf results/raw/items_p4v2_prisme_s1_test_iid_forced_b2.json
163+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_prisme_s1_test_template_budget0.json
164+d969667cd1fd32f07dd4033ede38e756200b6af096f9d2aa5764b5848013e9ac results/raw/items_p4v2_prisme_s1_test_template_forced_b2.json
165+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_prisme_s2_test_compose_budget0.json
166+81f46e521c20c05cc1981017682681adf39aee9e685f6e3aaf9888d3768bd639 results/raw/items_p4v2_prisme_s2_test_compose_forced_b2.json
167+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_prisme_s2_test_depth_budget0.json
168+b82f82f9dda0335045995f92f005abe6cd48c27e54b5de65ed453c9f0c16dd51 results/raw/items_p4v2_prisme_s2_test_depth_forced_b2.json
169+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_prisme_s2_test_iid_budget0.json
170+8761df4b6ae82ce5848e5b8736f6d4d6cb3453ec1daac23317416e791fa59236 results/raw/items_p4v2_prisme_s2_test_iid_forced_b2.json
171+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_prisme_s2_test_template_budget0.json
172+b84990037836a760f101b381216c8ba32455ee65f7ae14871b0ad72373399b15 results/raw/items_p4v2_prisme_s2_test_template_forced_b2.json
173+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_prisme_s3_test_compose_budget0.json
174+2233020039ad7f833d596c4d29b0ea8544302eff1a418eb051d9a7c4692c8c74 results/raw/items_p4v2_prisme_s3_test_compose_forced_b2.json
175+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_prisme_s3_test_depth_budget0.json
176+c38c9207609aaf1d8012712bfbf69898ceb8c2e3947517131c00f574e2df0a19 results/raw/items_p4v2_prisme_s3_test_depth_forced_b2.json
177+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_prisme_s3_test_iid_budget0.json
178+474f66cc8bae2acaffb15163852b1a32565706637ddc349a8434870778f168cd results/raw/items_p4v2_prisme_s3_test_iid_forced_b2.json
179+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_prisme_s3_test_template_budget0.json
180+07c6750a3bdd19e78fce4a252440a150ce45b2b47f98feefe058d16ede0fd9fb results/raw/items_p4v2_prisme_s3_test_template_forced_b2.json
181+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_prisme_s4_test_compose_budget0.json
182+bd79b85556e9d87ea59daa7429b932d03f8c5557e24318e99a953a934a0ebfe3 results/raw/items_p4v2_prisme_s4_test_compose_forced_b2.json
183+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_prisme_s4_test_depth_budget0.json
184+f03d3e5be849eda84999cea23d73ce170cf99fbb5286816763bcdd229317577a results/raw/items_p4v2_prisme_s4_test_depth_forced_b2.json
185+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_prisme_s4_test_iid_budget0.json
186+89999afb68660788f080c070e8a53c5377700403ba40c6deafa70162f4834046 results/raw/items_p4v2_prisme_s4_test_iid_forced_b2.json
187+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_prisme_s4_test_template_budget0.json
188+128df74db3be1d568b86ac2e158af651dd8ff5390cef15f81de6f81afbd600a2 results/raw/items_p4v2_prisme_s4_test_template_forced_b2.json
189+3d20172b7720ee1d094c7cd1d2c897832c16361d28c40f3fd41969177bf8ef06 results/raw/items_p4v2_prisme_s5_test_compose_budget0.json
190+e604df74deb203b3d80f0fd271237506da912b87d04dc969e6f1fee513556aca results/raw/items_p4v2_prisme_s5_test_compose_forced_b2.json
191+4f9f12c4648eba2ca8b664faadb3d797ea85fd97b812952aa6dc24fe567eba34 results/raw/items_p4v2_prisme_s5_test_depth_budget0.json
192+99e37b0a00330ebdb8790092fa2b194fb065e27fc03c4d58dd0886048e8545d0 results/raw/items_p4v2_prisme_s5_test_depth_forced_b2.json
193+7bc153555ff0b72561aea0e33d85c96c3e65e4f77cfd065d694bb81c6032766a results/raw/items_p4v2_prisme_s5_test_iid_budget0.json
194+d5eb96b933c0b67966734d733579e0885ce3710f782d101eee95d629fda4bdc9 results/raw/items_p4v2_prisme_s5_test_iid_forced_b2.json
195+2f00de69a1f10a905c93d3d6f01488e6590303e7e59db903785acece1bce9025 results/raw/items_p4v2_prisme_s5_test_template_budget0.json
196+4a273387f2204f907717961a3e5d1f58133ef2b3b03f97f8a947931d4f8f4eae results/raw/items_p4v2_prisme_s5_test_template_forced_b2.json
197+5ac057e7323760fe0774e0016dad4e4480a3b0fb2bcb2a71fab9b595596f00fa results/raw/latency_103m_mps.json
198+17fd8d751c915fa60c506718da6e862fb4e973b1b4996599d9ade54512982634 results/raw/latency_p4_editor_v1_Ustd02_seed01_20261029235050_mps.json
199+c7f8878b9b007546d7312cb29b1904992235c243d2d29446c1bec64bcfc9b1e4 results/raw/routing_103m_val.json
200+a2d6f4fc3f5b1591e6f02215d9d72a537bedc6afec076210b04eab44550a2822 results/raw/routing_controller_p4_hent_seed01_20261030002015_dev_iid.json
201+f7748c20a6446a22b1e307c80cea49a30cc89dc2c4c951d2486659cdbcfc3cef results/raw/routing_mini.json
202+f20c434c42090ba17603819bdfc127da73ca46163fc6823b6a78013dda08e555 results/raw/routing_v1ctrl.json
203+95cf28b285b26f6e1a54564b2a2b8ee691e89886df9875c850a1b5606d8831b1 results/raw/routing_v1rank.json
204+bb85e8001ba798a203fc01e55f212777430ae816acf893eb32ebdfc88a779e4b results/raw/routing_v2s1.json
205+224965164912b215d4e0be99c6a470b8b6d42a7fc390198424cfeb1a4e8be893 results/raw/routing_v2s2.json
206+152654d5ded3c0a643dcb2c296580ee8c330877b299d484057eec06f2a32b6a1 results/raw/routing_v2s3.json
207+909442a2374d2a2929719a7dc0142933ed1a4bf6f1feba2c82c547751d711b7e results/raw/routing_v2s4.json
208+38c615486b0c298ba1b179327f4e03827d26f7db3662f00657f2ddbfc30a10c3 results/raw/routing_v2s5.json
209+06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2 data/splits/synthetic_v1/manifest.json
210+d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54 data/processed/fineweb_edu_300m/manifest.json
211+54ef965fa85e96ebfd88bcdc745a8d19efa56c461d73444a4592dbedc678ac4e protocol/preregistration.md
212+acd9f9c178b9ab8979d527fed75733100a11eec63fb201f892c4ea016daeb1b9 protocol/locked_test_manifest.json
213+e9763b53e11f13ed41ccdba86fe95da896bc7b249762be6e4136ccc8a318b71b requirements-lock.txt
modified reproducibility/run_index.md +21 −4
@@ -2,18 +2,26 @@
2 2
3 | run_id | state | host | device | wall s | train tokens | peak mem (GB) | commit | notes | 3 | run_id | state | host | device | wall s | train tokens | peak mem (GB) | commit | notes |
4 |---|---|---|---|---|---|---|---|---| 4 |---|---|---|---|---|---|---|---|---|
5+| controller_103m_seed01_20261030034451 | completed | laptop | mps | 315.08 | | | unknown | |
5 | controller_mini_seed01_20261030005154 | completed | laptop | cpu | 3.14 | | | ede67055 | | 6 | controller_mini_seed01_20261030005154 | completed | laptop | cpu | 3.14 | | | ede67055 | |
6 | controller_p4_hent_seed01_20261030001419 | running | laptop | mps | 0.02 | | | unknown | | 7 | controller_p4_hent_seed01_20261030001419 | running | laptop | mps | 0.02 | | | unknown | |
7 | controller_p4_hent_seed01_20261030002015 | completed | laptop | mps | 584.36 | | | unknown | | 8 | controller_p4_hent_seed01_20261030002015 | completed | laptop | mps | 584.36 | | | unknown | |
8 | controller_v1ctrl_seed01_20261030004131 | completed | laptop | mps | 164.25 | | | 33298d96 | | 9 | controller_v1ctrl_seed01_20261030004131 | completed | laptop | mps | 164.25 | | | 33298d96 | |
10+| controller_v1rank_seed01_20261030005431 | completed | laptop | mps | 171.86 | | | d6746e03 | |
11+| controller_v2s1_seed01_20261030005758 | completed | laptop | mps | 141.05 | | | unknown | |
9 | controller_v2s2_seed01_20261030004717 | completed | laptop | mps | 193.23 | | | unknown | | 12 | controller_v2s2_seed01_20261030004717 | completed | laptop | mps | 193.23 | | | unknown | |
10 | controller_v2s3_seed01_20261030004720 | completed | laptop | mps | 197.97 | | | unknown | | 13 | controller_v2s3_seed01_20261030004720 | completed | laptop | mps | 197.97 | | | unknown | |
11 | controller_v2s4_seed01_20261030004719 | completed | laptop | mps | 217.1 | | | unknown | | 14 | controller_v2s4_seed01_20261030004719 | completed | laptop | mps | 217.1 | | | unknown | |
12 | controller_v2s5_seed01_20261030004719 | completed | laptop | mps | 214.63 | | | unknown | | 15 | controller_v2s5_seed01_20261030004719 | completed | laptop | mps | 214.63 | | | unknown | |
16+| editor_pilot_103m_lr3e4_seed01_20261030021200 | completed | laptop | mps | 5397.54 | 24576000 | 25.9 | unknown | prisme |
17+| editor_pilot_103m_seed01_20261030015139 | running | laptop | mps | 0.24 | | | unknown | prisme |
18+| gr_pilot_103m_seed01_20261030015117 | completed | laptop | mps | 1230.52 | 24576000 | 9.7 | unknown | global_recurrence |
19+| labels_103m_seed01_20261030034314 | completed | laptop | mps | 94.09 | | | unknown | |
13 | labels_mini_seed01_20261030005150 | completed | laptop | cpu | 3.44 | | | ede67055 | | 20 | labels_mini_seed01_20261030005150 | completed | laptop | cpu | 3.44 | | | ede67055 | |
14 | labels_p4_seed01_20261030000543 | completed | laptop | mps | 33.33 | | | unknown | | 21 | labels_p4_seed01_20261030000543 | completed | laptop | mps | 33.33 | | | unknown | |
15 | labels_v1ctrl_seed01_20261030003957 | running | laptop | mps | 0.02 | | | 33298d96 | | 22 | labels_v1ctrl_seed01_20261030003957 | running | laptop | mps | 0.02 | | | 33298d96 | |
16 | labels_v1ctrl_seed01_20261030004018 | completed | laptop | mps | 72.1 | | | 33298d96 | | 23 | labels_v1ctrl_seed01_20261030004018 | completed | laptop | mps | 72.1 | | | 33298d96 | |
24+| labels_v2s1_seed01_20261030005712 | completed | laptop | mps | 44.65 | | | unknown | |
17 | labels_v2s2_seed01_20261030004543 | completed | laptop | mps | 92.45 | | | unknown | | 25 | labels_v2s2_seed01_20261030004543 | completed | laptop | mps | 92.45 | | | unknown | |
18 | labels_v2s3_seed01_20261030004544 | completed | laptop | mps | 95.57 | | | unknown | | 26 | labels_v2s3_seed01_20261030004544 | completed | laptop | mps | 95.57 | | | unknown | |
19 | labels_v2s4_seed01_20261030004544 | completed | laptop | mps | 94.03 | | | unknown | | 27 | labels_v2s4_seed01_20261030004544 | completed | laptop | mps | 94.03 | | | unknown | |
@@ -22,18 +30,27 @@
22 | mini_editor_seed01_20261030005118 | completed | laptop | cpu | 30.59 | 9847 | 2.0 | ede67055 | prisme | 30 | mini_editor_seed01_20261030005118 | completed | laptop | cpu | 30.59 | 9847 | 2.0 | ede67055 | prisme |
23 | p4_editor_v1_Ustd02_seed01_20261029235050 | completed | laptop | mps | 650.11 | 331026 | 8.8 | unknown | prisme | 31 | p4_editor_v1_Ustd02_seed01_20261029235050 | completed | laptop | mps | 650.11 | 331026 | 8.8 | unknown | prisme |
24 | p4_editor_v1_scaled_seed01_20261029234334 | running | laptop | mps | 0.16 | | | HEAD | prisme | 32 | p4_editor_v1_scaled_seed01_20261029234334 | running | laptop | mps | 0.16 | | | HEAD | prisme |
25−| p4_editor_v1_seed01_20261029233624 | running | laptop | mps | 0.02 | | | HEAD | prisme | 33+| p4_editor_v1_seed01_20261029233624 | stopped_by_operator_scale_mismatch_diagnosed | laptop | mps | 0.02 | | | HEAD | prisme |
26 | p4_overfit64_b8_seed01_20261029235111 | completed | laptop | mps | 519.6 | 95773 | 6.7 | a9d94e8a | prisme | 34 | p4_overfit64_b8_seed01_20261029235111 | completed | laptop | mps | 519.6 | 95773 | 6.7 | a9d94e8a | prisme |
27 | p4_tiny_backbone_seed01_20261029232626 | completed | laptop | mps | 457.67 | 44367329 | 3.4 | HEAD | backbone_only | 35 | p4_tiny_backbone_seed01_20261029232626 | completed | laptop | mps | 457.67 | 44367329 | 3.4 | HEAD | backbone_only |
28 | p4_tiny_backbone_v2_seed01_20261029233624 | stopped_overfitting_step9300_checkpoint_step5000_used | laptop | mps | 0.02 | | | HEAD | backbone_only | 36 | p4_tiny_backbone_v2_seed01_20261029233624 | stopped_overfitting_step9300_checkpoint_step5000_used | laptop | mps | 0.02 | | | HEAD | backbone_only |
29−| p4v2_dense_seed05_20261030001839 | running | M4BP36 | mps | 0.33 | | | unknown | dense | 37+| p4v2_dense_seed01_20261030013421 | completed | laptop | mps | 2465.35 | 496298 | 30.3 | unknown | dense |
38+| p4v2_dense_seed02_20261030001749 | running | laptop | mps | 0.51 | | | unknown | dense |
39+| p4v2_dense_seed02_20261030004544 | completed | laptop | mps | 2601.32 | 498543 | 30.3 | unknown | dense |
40+| p4v2_dense_seed03_20261030001749 | running | laptop | mps | 0.54 | | | unknown | dense |
41+| p4v2_dense_seed03_20261030004544 | completed | laptop | mps | 2620.1 | 499017 | 30.3 | unknown | dense |
42+| p4v2_dense_seed04_20261030001749 | completed | laptop | mps | 3983.98 | 497821 | 28.4 | unknown | dense |
43+| p4v2_dense_seed05_20261030001839 | completed | M4BP36 | mps | 3249.7 | 497078 | 30.3 | unknown | dense |
44+| p4v2_dense_seed05_20261030004544 | completed | laptop | mps | 2620.45 | 497078 | 28.2 | unknown | dense |
45+| p4v2_gr_seed01_20261030001749 | completed | laptop | mps | 1120.54 | 496298 | 2.3 | unknown | global_recurrence |
30 | p4v2_gr_seed02_20261030001749 | completed | laptop | mps | 951.12 | 498543 | 3.4 | unknown | global_recurrence | 46 | p4v2_gr_seed02_20261030001749 | completed | laptop | mps | 951.12 | 498543 | 3.4 | unknown | global_recurrence |
31 | p4v2_gr_seed03_20261030001749 | completed | laptop | mps | 915.78 | 499017 | 2.3 | unknown | global_recurrence | 47 | p4v2_gr_seed03_20261030001749 | completed | laptop | mps | 915.78 | 499017 | 2.3 | unknown | global_recurrence |
32 | p4v2_gr_seed04_20261030001749 | completed | laptop | mps | 430.63 | 497821 | 2.3 | unknown | global_recurrence | 48 | p4v2_gr_seed04_20261030001749 | completed | laptop | mps | 430.63 | 497821 | 2.3 | unknown | global_recurrence |
33 | p4v2_gr_seed05_20261030001839 | completed | laptop | mps | 370.27 | 497078 | 3.4 | unknown | global_recurrence | 49 | p4v2_gr_seed05_20261030001839 | completed | laptop | mps | 370.27 | 497078 | 3.4 | unknown | global_recurrence |
50+| p4v2_prisme_seed01_20261030001749 | completed | laptop | mps | 2333.13 | 496298 | 8.9 | unknown | prisme |
34 | p4v2_prisme_seed02_20261030001749 | completed | laptop | mps | 1529.74 | 498543 | 8.8 | unknown | prisme | 51 | p4v2_prisme_seed02_20261030001749 | completed | laptop | mps | 1529.74 | 498543 | 8.8 | unknown | prisme |
35 | p4v2_prisme_seed03_20261030001749 | completed | laptop | mps | 1578.01 | 499017 | 10.0 | unknown | prisme | 52 | p4v2_prisme_seed03_20261030001749 | completed | laptop | mps | 1578.01 | 499017 | 10.0 | unknown | prisme |
36 | p4v2_prisme_seed04_20261030001749 | completed | laptop | mps | 1504.41 | 497821 | 9.3 | unknown | prisme | 53 | p4v2_prisme_seed04_20261030001749 | completed | laptop | mps | 1504.41 | 497821 | 9.3 | unknown | prisme |
37 | p4v2_prisme_seed05_20261030001839 | completed | laptop | mps | 939.64 | 497078 | 9.9 | unknown | prisme | 54 | p4v2_prisme_seed05_20261030001839 | completed | laptop | mps | 939.64 | 497078 | 9.9 | unknown | prisme |
38−| pretrain103m_seed01_20261029232706 | running | M3U96a | mps | 0.03 | | | unknown | | 55+| pretrain103m_seed01_20261029232706 | completed | laptop | mps | 8279.79 | 98304000 | 28.0 | unknown | |
39−| pretrain_enlarged102m_seed01_20261029232914 | running | M3U96b | mps | 0.02 | | | unknown | | 56+| pretrain_enlarged102m_seed01_20261030015017 | completed | laptop | mps | 8527.03 | 98304000 | 28.5 | unknown | |
modified results/claim_registry.csv +26 −25
@@ -1,25 +1,26 @@
1−claim,metric,value,benchmark,run_ids,artifact,analysis_script,status 1+claim,metric,value,benchmark,run_ids,artifact,analysis_script,status
2−Backbone parameter counts equal the proposal for 103M/459M/1.18B,unique parameters,"99502848;446744064;1149339648",meta instantiation,,results/tables/param_counts.json,scripts/count_params.py,complete 2+Backbone parameter counts equal the proposal for 103M/459M/1.18B,unique parameters,99502848;446744064;1149339648,meta instantiation,,results/tables/param_counts.json,scripts/count_params.py,complete
3−Implemented workspace+controller counts differ from the proposal by a closed-form 7w^2,unique parameters,"2631173;10505221;21655813",meta instantiation,,results/tables/param_counts.json,scripts/count_params.py,complete 3+Implemented workspace+controller counts differ from the proposal by a closed-form 7w^2,unique parameters,2631173;10505221;21655813,meta instantiation,,results/tables/param_counts.json,scripts/count_params.py,complete
4−Analytic ops table reproduces the proposal within 1.3%,ops/token at 459M T=4096,"1.296;1.666;1.787;2.219;5.633 e9",analytic,,results/tables/ops_459m_T4096.json,scripts/estimate_flops.py,complete 4+Analytic ops table reproduces the proposal within 1.3%,ops/token at 459M T=4096,1.296;1.666;1.787;2.219;5.633 e9,analytic,,results/tables/ops_459m_T4096.json,scripts/estimate_flops.py,complete
5−15 mandated invariants + cache equivalence pass,tests,23 passed,pytest,,tests/,python -m pytest tests,complete 5+15 mandated invariants + cache equivalence pass,tests,23 passed,pytest,,tests/,python -m pytest tests,complete
6−Zero-U bootstrap does not learn (scale mismatch),val loss by budget after 700 steps,0.4144/0.4143/0.4143/0.4146/0.4164,synthetic dev-iid v1 backbone,p4_editor_v1_seed01,experiments/runs/p4_editor_v1_seed01_20261029233315/metrics.jsonl,,stopped-run-used 6+Zero-U bootstrap does not learn (scale mismatch),val loss by budget after 700 steps,0.4144/0.4143/0.4143/0.4146/0.4164,synthetic dev-iid v1 backbone,p4_editor_v1_seed01_20261029233624,experiments/runs/p4_editor_v1_seed01_20261029233624/metrics.jsonl,,stopped-run-used
7−Overfitting test passes through the workspace path,train loss 64 items budget 8,0.347->0.0024 in 200 steps,synthetic train subset,p4_overfit64_b8_seed01,experiments/runs/p4_overfit64_b8_seed01_20261029235111/metrics.jsonl,,complete 7+Overfitting test passes through the workspace path,train loss 64 items budget 8,0.347->0.0024 in 200 steps,synthetic train subset,p4_overfit64_b8_seed01,experiments/runs/p4_overfit64_b8_seed01_20261029235111/metrics.jsonl,,complete
8−Sparse editor lowers held-out loss vs frozen backbone (H1 weak),dev-iid answer loss relative change by budget,see numbers.tex vtwoprismegainb*,synthetic dev-iid v2 backbone 5 seeds,p4v2_prisme,results/analyzed/seed_matrix_v2.json,scripts/make_results_tables.py,complete 8+Sparse editor lowers held-out loss vs frozen backbone (H1 weak),dev-iid answer loss relative change by budget,see numbers.tex vtwoprismegainb*,synthetic dev-iid v2 backbone 5 seeds,p4v2_prisme,results/analyzed/seed_matrix_v2.json,scripts/make_results_tables.py,complete
9−Global-recurrence adapter matches or exceeds the sparse editor at every budget,dev-iid answer loss by budget,see numbers.tex vtwogrgainb*,synthetic dev-iid v2 backbone 5 seeds,p4v2_gr,results/analyzed/seed_matrix_v2.json,scripts/make_results_tables.py,complete 9+Global-recurrence adapter matches or exceeds the sparse editor at every budget,dev-iid answer loss by budget,see numbers.tex vtwogrgainb*,synthetic dev-iid v2 backbone 5 seeds,p4v2_gr,results/analyzed/seed_matrix_v2.json,scripts/make_results_tables.py,complete
10−Variance of intervention gains is mostly between states,within-state share,see numbers.tex labelscalibwithin,editor-unseen label stream (calib audit),labels_v1ctrl_seed01_20261030004018,results/analyzed/labels_labels_v1ctrl_seed01_20261030004018.json,evaluation/label_analysis.py,complete 10+Variance of intervention gains is mostly between states,within-state share,see numbers.tex labelscalibwithin,editor-unseen label stream (calib audit),labels_v1ctrl_seed01_20261030004018,results/analyzed/labels_labels_v1ctrl_seed01_20261030004018.json,evaluation/label_analysis.py,complete
11−One-step oracle lowers dev-iid loss by ~15% at budget 8 (diagnostic),loss relative change,see numbers.tex routingoraclebeight,synthetic dev-iid v1 editor,controller_v1ctrl_seed01,results/raw/routing_v1ctrl.json,evaluation/routing.py,complete 11+One-step oracle lowers dev-iid loss by ~15% at budget 8 (diagnostic),loss relative change,see numbers.tex routingoraclebeight,synthetic dev-iid v1 editor,controller_v1ctrl_seed01,results/raw/routing_v1ctrl.json,evaluation/routing.py,complete
12−Learned controller is worse than no routing on dev-iid (H3 null),loss relative change at locked lambda,see numbers.tex routingctrllocked;vtwoctrlrel,synthetic dev-iid v1 and v2 editors,controller_v1ctrl_seed01;controller_v2s2;controller_v2s3;controller_v2s4;controller_v2s5,results/raw/routing_v1ctrl.json;results/analyzed/routing_seeds_v2.json,evaluation/routing.py,complete 12+Learned controller is worse than no routing on dev-iid (H3 null),loss relative change at locked lambda,see numbers.tex routingctrllocked;vtwoctrlrel,synthetic dev-iid v1 and v2 editors,controller_v1ctrl_seed01;controller_v2s2;controller_v2s3;controller_v2s4;controller_v2s5,results/raw/routing_v1ctrl.json;results/analyzed/routing_seeds_v2.json,evaluation/routing.py,complete
13−Fixed schedules: random = uniform = fixed pair (H2 null),loss at budget 1,see numbers.tex routing*bone,synthetic dev-iid v1 editor,controller_v1ctrl_seed01,results/raw/routing_v1ctrl.json,evaluation/routing.py,complete 13+Fixed schedules: random = uniform = fixed pair (H2 null),loss at budget 1,see numbers.tex routing*bone,synthetic dev-iid v1 editor,controller_v1ctrl_seed01,results/raw/routing_v1ctrl.json,evaluation/routing.py,complete
14−Two-step oracle vs one-step: myopia regret small,gain nats,see numbers.tex oracletwostep;oracleonestep,synthetic dev-iid v1 editor (1 batch),controller_v1ctrl_seed01,results/raw/routing_v1ctrl.json,evaluation/routing.py,complete 14+Two-step oracle vs one-step: myopia regret small,gain nats,see numbers.tex oracletwostep;oracleonestep,synthetic dev-iid v1 editor (1 batch),controller_v1ctrl_seed01,results/raw/routing_v1ctrl.json,evaluation/routing.py,complete
15−Dispatch overhead dominates at tiny scale; dense not slower than sparse,median ms/token batch 1,see paper/tables/latency.tex,laptop MPS tiny model,p4_editor_v1_Ustd02_seed01,results/raw/latency_p4_editor_v1_Ustd02_seed01_20261029235050_mps.json,evaluation/latency.py,complete 15+Dispatch overhead dominates at tiny scale; dense not slower than sparse,median ms/token batch 1,see paper/tables/latency.tex,laptop MPS tiny model,p4_editor_v1_Ustd02_seed01,results/raw/latency_p4_editor_v1_Ustd02_seed01_20261029235050_mps.json,evaluation/latency.py,complete
16−Miniature end-to-end pipeline runs on CPU,wall seconds,56,mini configs,mini_backbone_seed01;mini_editor_seed01;labels_mini_seed01;controller_mini_seed01,results/raw/routing_mini.json,scripts/reproduce_pilot.sh,complete 16+Miniature end-to-end pipeline runs on CPU,wall seconds,56,mini configs,mini_backbone_seed01;mini_editor_seed01;labels_mini_seed01;controller_mini_seed01,results/raw/routing_mini.json,scripts/reproduce_pilot.sh,complete
17−103M backbone pretrained on 98.3M tokens (ppl 48.2),val loss/ppl,3.8746 / 48.16,FineWeb-Edu val,pretrain103m_seed01_20261029232706,experiments/runs/pretrain103m_seed01_20261029232706/manifest.json,training/pretrain.py,complete 17+103M backbone pretrained on 98.3M tokens (ppl 48.2),val loss/ppl,3.8746 / 48.16,FineWeb-Edu val,pretrain103m_seed01_20261029232706,experiments/runs/pretrain103m_seed01_20261029232706/manifest.json,training/pretrain.py,complete
18−103M sparse editor (lr 3e-4) changes held-out loss by less than 0.1% at any budget,val loss relative change,see numbers.tex piloted*,FineWeb-Edu val (training-time eval),editor_pilot_103m_lr3e4_seed01_20261030021200,experiments/runs/editor_pilot_103m_lr3e4_seed01_20261030021200/metrics.jsonl,training/train_editor.py,complete 18+103M sparse editor (lr 3e-4) changes held-out loss by less than 0.1% at any budget,val loss relative change,see numbers.tex piloted*,FineWeb-Edu val (training-time eval),editor_pilot_103m_lr3e4_seed01_20261030021200,experiments/runs/editor_pilot_103m_lr3e4_seed01_20261030021200/metrics.jsonl,training/train_editor.py,complete
19−103M sparse editor at lr 1e-3 collapsed to inert edits,val loss by budget,identical 3.9889,FineWeb-Edu val,editor_pilot_103m_seed01_20261030015139,experiments/runs/editor_pilot_103m_seed01_20261030015139/metrics.jsonl,,stopped-run-used 19+103M sparse editor at lr 1e-3 collapsed to inert edits,val loss by budget,identical 3.9889,FineWeb-Edu val,editor_pilot_103m_seed01_20261030015139,experiments/runs/editor_pilot_103m_seed01_20261030015139/metrics.jsonl,,stopped-run-used
20−103M global-recurrence adapter changes held-out loss by less than 0.1%,val ppl by budget,48.16 -> 48.01,FineWeb-Edu val,gr_pilot_103m_seed01_20261030015117,results/raw/eval_gr_103m_val.json,evaluation/run.py,complete 20+103M global-recurrence adapter changes held-out loss by less than 0.1%,val ppl by budget,48.16 -> 48.01,FineWeb-Edu val,gr_pilot_103m_seed01_20261030015117,results/raw/eval_gr_103m_val.json,evaluation/run.py,complete
21−103M controller: oracle opportunity exists (-3.6% at b8) but the controller captures none (-0.01%),val loss relative change,see numbers.tex pilotrouting*,FineWeb-Edu val,controller_103m_seed01,results/raw/routing_103m_val.json,evaluation/routing.py,complete 21+103M controller: oracle opportunity exists (-3.6% at b8) but the controller captures none (-0.01%),val loss relative change,see numbers.tex pilotrouting*,FineWeb-Edu val,controller_103m_seed01,results/raw/routing_103m_val.json,evaluation/routing.py,complete
22−103M latency: 8 sparse edits 3.0x budget-0 median latency; dense 3.6x; controller 1.5x,median ms/token batch 1,7.55 / 22.9 / 26.9 / 11.0,M3 Ultra MPS fp32,controller_103m_seed01,results/raw/latency_103m_mps.json,evaluation/latency.py,complete 22+103M latency: 8 sparse edits 3.0x budget-0 median latency; dense 3.6x; controller 1.5x,median ms/token batch 1,7.55 / 22.9 / 26.9 / 11.0,M3 Ultra MPS fp32,controller_103m_seed01,results/raw/latency_103m_mps.json,evaluation/latency.py,complete
23−Pre-registered primary outcome: pooled test EM gain of sparse editor at b2 is +0.9 pp (< 3 pp threshold),exact match pp,see numbers.tex testprismebtwovsbzero*,synthetic locked test regimes (1000 items each),p4v2_prisme (5 seeds),results/analyzed/test_statistics.json;paper/tables/test.tex,scripts/make_results_tables.py,complete 23+Pre-registered primary outcome: pooled test EM gain of sparse editor at b2 is +0.9 pp (< 3 pp threshold),exact match pp,see numbers.tex testprismebtwovsbzero*,synthetic locked test regimes (1000 items each),p4v2_prisme_seed01_20261030001749;p4v2_prisme_seed02_20261030001749;p4v2_prisme_seed03_20261030001749;p4v2_prisme_seed04_20261030001749;p4v2_prisme_seed05_20261030001839,results/analyzed/test_statistics.json;paper/tables/test.tex,scripts/make_results_tables.py,complete
24−Sparse editor vs global recurrence on locked test: -0.3 pp [-0.6 +0.1]; vs dense +0.4 pp [+0.1 +0.7],exact match pp,see numbers.tex testprismevs*,synthetic locked test regimes,p4v2_prisme;p4v2_gr;p4v2_dense,results/analyzed/test_statistics.json,scripts/make_results_tables.py,complete 24+Sparse editor vs global recurrence on locked test: -0.3 pp [-0.6 +0.1]; vs dense +0.4 pp [+0.1 +0.7],exact match pp,see numbers.tex testprismevs*,synthetic locked test regimes,p4v2_prisme;p4v2_gr;p4v2_dense,results/analyzed/test_statistics.json,scripts/make_results_tables.py,complete
25−Parameter-matched plain decoder (102.2M) beats backbone+adapters on validation text,val loss,3.8689 vs 3.8746,FineWeb-Edu val,pretrain_enlarged102m_seed01_20261030015017,experiments/runs/pretrain_enlarged102m_seed01_20261030015017/manifest.json,training/pretrain.py,complete 25+Parameter-matched plain decoder (102.2M) beats backbone+adapters on validation text,val loss,3.8689 vs 3.8746,FineWeb-Edu val,pretrain_enlarged102m_seed01_20261030015017,experiments/runs/pretrain_enlarged102m_seed01_20261030015017/manifest.json,training/pretrain.py,complete
26+Locked FineWeb test: backbone ppl 45.72; sparse editor and global adapter change test loss by -0.08% at budget 4; enlarged decoder lower,test ppl / relative loss,see numbers.tex pilotppl*test;pilottest*,FineWeb-Edu locked test (327680 tokens),pretrain103m_seed01_20261029232706;editor_pilot_103m_lr3e4_seed01_20261030021200;gr_pilot_103m_seed01_20261030015117;pretrain_enlarged102m_seed01_20261030015017,results/raw/eval_backbone_103m_test.json;results/raw/eval_prisme_103m_editor_test.json;results/raw/eval_gr_103m_test.json;results/raw/eval_enlarged_102m_test.json,evaluation/run.py,complete
added results/figures/exact_match_regimes.pdf +0 −0

Binary file not shown.

added results/figures/exact_match_regimes.png +0 −0

Binary file not shown.

modified results/figures/frontier_v2.pdf +0 −0

Binary file not shown.

modified results/figures/labels_analysis.pdf +0 −0

Binary file not shown.

modified results/figures/latency_103m_mps.pdf +0 −0

Binary file not shown.

modified results/figures/latency_p4_editor_v1_Ustd02_seed01_20261029235050_mps.pdf +0 −0

Binary file not shown.

modified results/figures/learnability_bootstrap.pdf +0 −0

Binary file not shown.

modified results/figures/learnability_v2.pdf +0 −0

Binary file not shown.

modified results/figures/pretrain_curves.pdf +0 −0

Binary file not shown.

modified results/figures/pretrain_curves.png +0 −0

Binary file not shown.

modified results/figures/routing_103m_val.pdf +0 −0

Binary file not shown.

modified results/figures/routing_controller_p4_hent_seed01_20261030002015_dev_iid.pdf +0 −0

Binary file not shown.

modified results/figures/routing_mini.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v1ctrl.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v1rank.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v2s1.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v2s2.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v2s3.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v2s4.pdf +0 −0

Binary file not shown.

modified results/figures/routing_v2s5.pdf +0 −0

Binary file not shown.

added results/raw/eval_backbone_103m_test.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "backbone_103m",
3+ "split": "test",
4+ "checkpoint": "experiments/runs/pretrain103m_seed01_20261029232706/checkpoint",
5+ "checkpoint_sha256": "1f05af70bb9223b63dacb765c1816212ec828ff6e81bf8c0d6d46f33ebc77db2",
6+ "config_sha256": "3962409f5a94a4dcd6a9898a024ac618f1fbacb74e5f1466318b8989dcf3857b",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:15:27-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.8226005017757414,
15+ "ppl": 45.722956481048506,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.8226005017757414,
20+ "ppl": 45.722956481048506,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.8226005017757414,
25+ "ppl": 45.722956481048506,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.8226005017757414,
30+ "ppl": 45.722956481048506,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.8226005017757414,
35+ "ppl": 45.722956481048506,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 38.6
40+}
\ No newline at end of file
added results/raw/eval_backbone_103m_val.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "backbone_103m",
3+ "split": "val",
4+ "checkpoint": "experiments/runs/pretrain103m_seed01_20261029232706/checkpoint",
5+ "checkpoint_sha256": "1f05af70bb9223b63dacb765c1816212ec828ff6e81bf8c0d6d46f33ebc77db2",
6+ "config_sha256": "3962409f5a94a4dcd6a9898a024ac618f1fbacb74e5f1466318b8989dcf3857b",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:14:49-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.8746298491954803,
15+ "ppl": 48.16486672694994,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.8746298491954803,
20+ "ppl": 48.16486672694994,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.8746298491954803,
25+ "ppl": 48.16486672694994,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.8746298491954803,
30+ "ppl": 48.16486672694994,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.8746298491954803,
35+ "ppl": 48.16486672694994,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 38.7
40+}
\ No newline at end of file
added results/raw/eval_enlarged_102m_test.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "enlarged_102m",
3+ "split": "test",
4+ "checkpoint": "experiments/runs/pretrain_enlarged102m_seed01_20261030015017/checkpoint",
5+ "checkpoint_sha256": "1552f1ec984ec895a40814cd466bde59abaf35e4bfe1ea35b9d43ff4cfe9b9d0",
6+ "config_sha256": "0cc9406173e04dcb44f4facbf0dc448da684d5e7a0ba3362385d292e6a125357",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:20:19-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.8207094609737395,
15+ "ppl": 45.63657420669954,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.8207094609737395,
20+ "ppl": 45.63657420669954,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.8207094609737395,
25+ "ppl": 45.63657420669954,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.8207094609737395,
30+ "ppl": 45.63657420669954,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.8207094609737395,
35+ "ppl": 45.63657420669954,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 36.8
40+}
\ No newline at end of file
added results/raw/eval_enlarged_102m_val.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "enlarged_102m",
3+ "split": "val",
4+ "checkpoint": "experiments/runs/pretrain_enlarged102m_seed01_20261030015017/checkpoint",
5+ "checkpoint_sha256": "1552f1ec984ec895a40814cd466bde59abaf35e4bfe1ea35b9d43ff4cfe9b9d0",
6+ "config_sha256": "0cc9406173e04dcb44f4facbf0dc448da684d5e7a0ba3362385d292e6a125357",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:19:44-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.868902176618576,
15+ "ppl": 47.88978268831879,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.868902176618576,
20+ "ppl": 47.88978268831879,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.868902176618576,
25+ "ppl": 47.88978268831879,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.868902176618576,
30+ "ppl": 47.88978268831879,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.868902176618576,
35+ "ppl": 47.88978268831879,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 35.2
40+}
\ No newline at end of file
added results/raw/eval_gr_103m_test.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "gr_103m",
3+ "split": "test",
4+ "checkpoint": "experiments/runs/gr_pilot_103m_seed01_20261030015117/checkpoint",
5+ "checkpoint_sha256": "1e3e5cdd103672376ad23bbfaafc50bcc68b937d808fae74308285a54564938a",
6+ "config_sha256": "a3f0c8dc75a01e38b6eb8a9d199aa2e29f574b887bdd0e62092e2f550e7a6373",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:16:54-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.8226005017757414,
15+ "ppl": 45.722956481048506,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.8211239874362946,
20+ "ppl": 45.65549569582661,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.8202611207962036,
25+ "ppl": 45.61611808291132,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.8195126056671143,
30+ "ppl": 45.581986503991125,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.8201463341712953,
35+ "ppl": 45.61088226318195,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 45.8
40+}
\ No newline at end of file
modified results/raw/eval_gr_103m_val.json +2 −2
@@ -7,7 +7,7 @@
7 "code_commit": "unknown", 7 "code_commit": "unknown",
8 "device": "mps", 8 "device": "mps",
9 "precision": "fp32", 9 "precision": "fp32",
10− "evaluated_at": "2026-10-30T03:43:14-0400", 10+ "evaluated_at": "2026-10-30T04:16:08-0400",
11 "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54", 11 "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12 "perplexity": { 12 "perplexity": {
13 "forced_b0": { 13 "forced_b0": {
@@ -36,5 +36,5 @@
36 "n_tokens": 327680 36 "n_tokens": 327680
37 } 37 }
38 }, 38 },
39− "seconds": 45.8 39+ "seconds": 45.9
40 } 40 }
\ No newline at end of file
added results/raw/eval_prisme_103m_editor_test.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "prisme_103m_editor",
3+ "split": "test",
4+ "checkpoint": "experiments/runs/editor_pilot_103m_lr3e4_seed01_20261030021200/checkpoint",
5+ "checkpoint_sha256": "1203ea2e786b258268ea56204e9676a6b900d41628d9fce32af5a01ab0181ea3",
6+ "config_sha256": "a3f0c8dc75a01e38b6eb8a9d199aa2e29f574b887bdd0e62092e2f550e7a6373",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:16:30-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.8226005017757414,
15+ "ppl": 45.722956481048506,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.821114146709442,
20+ "ppl": 45.65504641477478,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.8202012896537783,
25+ "ppl": 45.61338890009928,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.819450855255127,
30+ "ppl": 45.57917188444815,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.8199622869491576,
35+ "ppl": 45.60248847945214,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 86.6
40+}
\ No newline at end of file
added results/raw/eval_prisme_103m_editor_val.json +40 −0
@@ -0,0 +1,40 @@
1+{
2+ "system": "prisme_103m_editor",
3+ "split": "val",
4+ "checkpoint": "experiments/runs/editor_pilot_103m_lr3e4_seed01_20261030021200/checkpoint",
5+ "checkpoint_sha256": "1203ea2e786b258268ea56204e9676a6b900d41628d9fce32af5a01ab0181ea3",
6+ "config_sha256": "a3f0c8dc75a01e38b6eb8a9d199aa2e29f574b887bdd0e62092e2f550e7a6373",
7+ "code_commit": "unknown",
8+ "device": "mps",
9+ "precision": "fp32",
10+ "evaluated_at": "2026-10-30T04:14:49-0400",
11+ "data_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
12+ "perplexity": {
13+ "forced_b0": {
14+ "loss": 3.8746298491954803,
15+ "ppl": 48.16486672694994,
16+ "n_tokens": 327680
17+ },
18+ "forced_b1": {
19+ "loss": 3.8731415927410127,
20+ "ppl": 48.093238367070285,
21+ "n_tokens": 327680
22+ },
23+ "forced_b2": {
24+ "loss": 3.872187703847885,
25+ "ppl": 48.04738463431747,
26+ "n_tokens": 327680
27+ },
28+ "forced_b4": {
29+ "loss": 3.871379178762436,
30+ "ppl": 48.00855281891424,
31+ "n_tokens": 327680
32+ },
33+ "forced_b8": {
34+ "loss": 3.871813231706619,
35+ "ppl": 48.02939559571834,
36+ "n_tokens": 327680
37+ }
38+ },
39+ "seconds": 100.4
40+}
\ No newline at end of file
modified scripts/make_results_tables.py +53 −7
@@ -345,22 +345,29 @@ def table_pilot(macro):
345 macro(f"{tag}gainb{bn}", "n/a") 345 macro(f"{tag}gainb{bn}", "n/a")
346 macro(f"{tag}minutes", "n/a"); macro(f"{tag}peakgb", "n/a"); macro(f"{tag}params", "n/a") 346 macro(f"{tag}minutes", "n/a"); macro(f"{tag}peakgb", "n/a"); macro(f"{tag}params", "n/a")
347 # perplexity table from evaluation.run outputs 347 # perplexity table from evaluation.run outputs
348− evs = sorted(glob.glob("results/raw/eval_*_val.json")) 348+ evs = sorted(glob.glob("results/raw/eval_*_val.json")) + sorted(glob.glob("results/raw/eval_*103m*_test.json")) + sorted(glob.glob("results/raw/eval_enlarged*_test.json"))
349+ evs = [f for f in evs if "p4v2" not in f]
349 if evs: 350 if evs:
350 head = r"""\begin{table}[t]\centering\small 351 head = r"""\begin{table}[t]\centering\small
351−\caption{103M pilot: held-out cross-entropy (nats/token) and perplexity on 327,680 FineWeb-Edu validation tokens 352+\caption{103M pilot: held-out cross-entropy (nats/token) and perplexity on 327,680 FineWeb-Edu validation tokens and
352−(40 $\times$ 16 $\times$ 512) under forced random budgets and, for PRISME, under the locked controller.} 353+on the LOCKED test split (40 $\times$ 16 $\times$ 512 tokens each) under forced random budgets. Budget 0 of the adapters
354+equals the backbone bitwise.}
353 \label{tab:pilotppl} 355 \label{tab:pilotppl}
354−\begin{tabular}{llrr}\toprule 356+\begin{tabular}{lllrr}\toprule
355−System & policy & loss & ppl\\\midrule 357+System & split & policy & loss & ppl\\\midrule
356 """ 358 """
357 body = "" 359 body = ""
358 for f in evs: 360 for f in evs:
359 d = json.load(open(f)) 361 d = json.load(open(f))
360 for pol, r in d.get("perplexity", {}).items(): 362 for pol, r in d.get("perplexity", {}).items():
361− body += f"{d['system'].replace('_', ' ')} & {pol.replace('_', ' ')} & {r['loss']:.4f} & {r['ppl']:.2f}\\\\\n" 363+ if pol == "controller":
364+ continue
365+ body += f"{d['system'].replace('_', ' ')} & {d['split']} & {pol.replace('_', ' ')} & {r['loss']:.4f} & {r['ppl']:.2f}\\\\\n"
362 if pol == "forced_b0": 366 if pol == "forced_b0":
363− macro("pilotppl" + re.sub(r"[^a-z]", "", d["system"]), f"{r['ppl']:.2f}") 367+ macro("pilotppl" + re.sub(r"[^a-z]", "", d["system"]) + d["split"], f"{r['ppl']:.2f}")
368+ if d["split"] == "test" and pol in ("forced_b2", "forced_b4"):
369+ b0 = d["perplexity"]["forced_b0"]["loss"]
370+ macro("pilottest" + re.sub(r"[^a-z]", "", d["system"]) + {"forced_b2": "btwo", "forced_b4": "bfour"}[pol], f"{100*(r['loss']/b0-1):+.2f}")
364 (OUT / "pilot_ppl.tex").write_text(head + body + r"\bottomrule\end{tabular}\end{table}" + "\n") 371 (OUT / "pilot_ppl.tex").write_text(head + body + r"\bottomrule\end{tabular}\end{table}" + "\n")
365 rt = Path("results/raw/routing_103m_val.json") 372 rt = Path("results/raw/routing_103m_val.json")
366 if rt.exists(): 373 if rt.exists():
@@ -485,3 +492,42 @@ System & budget & iid & template & depth & compose & pooled\\\midrule
485 _old_make_all4 = make_all 492 _old_make_all4 = make_all
486 def make_all(macro): # noqa: F811 493 def make_all(macro): # noqa: F811
487 _old_make_all4(macro); table_test(macro) 494 _old_make_all4(macro); table_test(macro)
495+
496+
497+def table_controls(macro):
498+ import torch
499+ from training.train_editor import build_model
500+ from training.common import load_yaml
501+ rows = []
502+ def count(exp, label, data, compute, regime):
503+ cfg, m = build_model(exp)
504+ nb = sum(p.numel() for p in m.backbone.parameters()); tot = sum(p.numel() for p in m.parameters())
505+ rows.append((label, nb, tot - nb, data, compute, regime))
506+ with torch.device("meta"):
507+ count(load_yaml("configs/experiments/p4v2_prisme.yaml"), "Frozen backbone v2 (budget 0)", "synthetic train, 5k steps $\\times$ 64", "backbone only", "no workspace (bitwise bypass)")
508+ count(load_yaml("configs/experiments/p4v2_prisme.yaml"), "PRISME sparse editor (tiny)", "same + 3k adapter steps $\\times$ 32", "frozen bb fwd + editor", "1--2 slots/action, 0--8 actions")
509+ count(load_yaml("configs/experiments/p4v2_gr.yaml"), "Global-recurrence adapter (tiny)", "same", "frozen bb fwd + block", "whole latent/action, 0--8 actions")
510+ count(load_yaml("configs/experiments/p4v2_dense.yaml"), "Dense 16-slot editor (tiny)", "same", "frozen bb fwd + 16 lanes", "16 slots/action, 0--8 actions")
511+ e = load_yaml("configs/experiments/editor_pilot_103m_lr3e4.yaml"); e["init"] = {}
512+ count(e, "PRISME-103M sparse editor", "FineWeb-Edu 98.3M tok + 24.6M adapter tok", "$6NP$ + editor", "1--2 slots/action")
513+ g = load_yaml("configs/experiments/gr_pilot_103m.yaml"); g["init"] = {}
514+ count(g, "Global-recurrence adapter (103M)", "same", "$6NP$ + block", "whole latent/action")
515+ count({"model": "configs/baselines/transformer_enlarged_102m.yaml", "mode": "backbone_only"}, "Plain decoder, parameter-matched (102.2M)", "FineWeb-Edu 98.3M tok (same recipe)", "$6NP$", "no workspace")
516+ with open(OUT / "controls.tex", "w") as f:
517+ f.write(r"""\begin{table}[t]\centering\small
518+\caption{Systems and controls. ``bb'' = backbone, ``$6NP$'' = the usual training-FLOP estimate for $N$ tokens and $P$
519+parameters; adapter training additionally executes up to 8 editor applications per position. Adapter parameters are
520+unique parameters outside the backbone (counted by instantiation; the 103M editor includes the residual-plus-entropy
521+controller features).}
522+\label{tab:controls}
523+\begin{tabular}{lrrlll}\toprule
524+System & backbone & adapter & training exposure & training compute & inference regime\\\midrule
525+""")
526+ for lab, nb, ad, data, comp, reg in rows:
527+ f.write(f"{lab} & {nb:,} & {ad:,} & {data} & {comp} & {reg}\\\\\n")
528+ f.write(r"\bottomrule\end{tabular}\end{table}" + "\n")
529+
530+
531+_old_make_all5 = make_all
532+def make_all(macro): # noqa: F811
533+ _old_make_all5(macro); table_controls(macro)
488 534
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere