SPB Git
spboucher.ai

spb/wp31_uqo

Public

WP31: locked test outcome, findings, abstract

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Oct 30, 2026) parent 7846b4f Browse files

203 changed files +3,263 −81

203 changed files
Load all 203 files Unified Split
modified PROGRESS.md +1 −0
@@ -20,3 +20,4 @@
20 20 | 2026-10-30 | 6 ranking-loss controller (D12, tiny v1 editor, editor-unseen labels) | done | `controller_v1rank.yaml` → `results/raw/routing_v1rank.json` | locked λ=0.3 κ=0.5: dev-iid loss 0.4333 vs 0.4334 (0.07 edits/token); λ=0: +1.3 % | H3 null stands | — |
21 21 | 2026-10-30 | 4 dense-writes ablation (v2 backbone, 5 seeds) | done | `p4v2_dense.yaml` → `results/analyzed/seed_matrix_v2.json` | dev-iid loss −2.5/−3.2/−3.2/−1.6 % at b=1/2/4/8 (vs sparse −2.1/−2.8/−3.0/−1.0, GR −3.6/−4.1/−3.7/−3.3); EM +2.0 pp (b1), t-CI [1.6, 2.4] | sparse local writes ≤ dense (H2 second half null; 5 seeds: −2.4/−3.1/−3.1/−1.5 %, EM +2.0 pp [1.6, 2.4]) | — |
22 22 | 2026-10-30 | 8–10 pilot stages B–D + routing + latency (103M) | done | `editor_pilot_103m_lr3e4` (3000 steps, 90 min, peak 25.9 GB), `labels_103m` (491k labels, 32,768 audit states), `controller_103m`, `routing_103m_val.json`, `latency_103m_mps.json`, `eval_gr_103m_val.json` | editor −0.04/−0.06/−0.08/−0.06 % at b=1/2/4/8; GR −0.08 % (ppl 48.16 → 48.01); controller locked (λ=0.3, κ=0.5) −0.01 % with 0.61 edits/token, calib sign acc 51 %, captured 7 %; oracle b8 −3.6 % (4.3 edits/token); two-step oracle 0.0162 vs one-step 0.0088 nats; latency batch 1: 7.55 ms (b0) → 22.9 (8 single) / 26.9 (8 dense) / 11.0 (controller) ms; dense 103M probe collapsed twice | H1 not supported on web text; H3 null; dispatch overhead still dominant (sparse 3.0× vs dense 3.6× measured, analytic 1.6× vs 3.1×) | enlarged baseline eval, final audit |
23 +| 2026-10-30 | 9 locked test evaluation (synthetic, pre-registered primary outcome) | done | prereg frozen v1.0 04:04, manifest unsealed 04:04; `test_eval_v2_s*.yaml` on 5 nodes → `results/raw/eval_p4v2_*_test_*.json`, `results/analyzed/test_statistics.json` | pooled EM (iid+template+depth) b2 vs b0: PRISME +0.9 pp [+0.9, +1.0]; PRISME − GR −0.3 [−0.6, +0.1]; PRISME − dense +0.4 [+0.1, +0.7]; compose 4–5 % all systems | pre-registered threshold (3 pp) NOT met → do not advance | FineWeb test perplexity (4 systems) running |
modified configs/experiments/locked_evaluation.yaml +2 −1
@@ -4,6 +4,7 @@ data:
4 4 dir: data/processed/fineweb_edu_300m
5 5 splits:
6 6 - val
7 +- test
7 8 seq_len: 512
8 9 batch_size: 16
9 10 n_batches: 40
@@ -22,7 +23,7 @@ systems:
22 23 enlarged_102m:
23 24 mode: backbone_only
24 25 model: from_checkpoint
25 − checkpoint: ENLARGED_CKPT
26 + checkpoint: experiments/runs/pretrain_enlarged102m_seed01_20261030015017/checkpoint
26 27 controller: false
27 28 prisme_103m_editor:
28 29 mode: prisme
added configs/experiments/test_eval_v2_s1.yaml +43 −0
@@ -0,0 +1,43 @@
1 +out_dir: results/raw
2 +data:
3 + kind: synthetic
4 + tokenizer: char
5 + splits_dir: data/splits/synthetic_v1
6 +splits:
7 +- test_iid
8 +- test_template
9 +- test_depth
10 +- test_compose
11 +batch_size: 64
12 +seq_len: 320
13 +max_items: 1000
14 +systems:
15 + p4v2_prisme_s1:
16 + mode: prisme
17 + model: from_checkpoint
18 + checkpoint: experiments/runs/p4v2_prisme_seed01_20261030001749/checkpoint
19 + controller: false
20 + policies:
21 + - budget0
22 + - forced_b2
23 + seed: 1
24 + p4v2_gr_s1:
25 + mode: global_recurrence
26 + model: from_checkpoint
27 + checkpoint: experiments/runs/p4v2_gr_seed01_20261030001749/checkpoint
28 + controller: false
29 + policies:
30 + - budget0
31 + - forced_b2
32 + seed: 1
33 + baseline_kwargs:
34 + ffn_w: 640
35 + p4v2_dense_s1:
36 + mode: dense
37 + model: from_checkpoint
38 + checkpoint: experiments/runs/p4v2_dense_seed01_20261030013421/checkpoint
39 + controller: false
40 + policies:
41 + - budget0
42 + - forced_b2
43 + seed: 1
added configs/experiments/test_eval_v2_s2.yaml +43 −0
@@ -0,0 +1,43 @@
1 +out_dir: results/raw
2 +data:
3 + kind: synthetic
4 + tokenizer: char
5 + splits_dir: data/splits/synthetic_v1
6 +splits:
7 +- test_iid
8 +- test_template
9 +- test_depth
10 +- test_compose
11 +batch_size: 64
12 +seq_len: 320
13 +max_items: 1000
14 +systems:
15 + p4v2_prisme_s2:
16 + mode: prisme
17 + model: from_checkpoint
18 + checkpoint: experiments/runs/p4v2_prisme_seed02_20261030001749/checkpoint
19 + controller: false
20 + policies:
21 + - budget0
22 + - forced_b2
23 + seed: 2
24 + p4v2_gr_s2:
25 + mode: global_recurrence
26 + model: from_checkpoint
27 + checkpoint: experiments/runs/p4v2_gr_seed02_20261030001749/checkpoint
28 + controller: false
29 + policies:
30 + - budget0
31 + - forced_b2
32 + seed: 2
33 + baseline_kwargs:
34 + ffn_w: 640
35 + p4v2_dense_s2:
36 + mode: dense
37 + model: from_checkpoint
38 + checkpoint: experiments/runs/p4v2_dense_seed02_20261030004544/checkpoint
39 + controller: false
40 + policies:
41 + - budget0
42 + - forced_b2
43 + seed: 2
added configs/experiments/test_eval_v2_s3.yaml +43 −0
@@ -0,0 +1,43 @@
1 +out_dir: results/raw
2 +data:
3 + kind: synthetic
4 + tokenizer: char
5 + splits_dir: data/splits/synthetic_v1
6 +splits:
7 +- test_iid
8 +- test_template
9 +- test_depth
10 +- test_compose
11 +batch_size: 64
12 +seq_len: 320
13 +max_items: 1000
14 +systems:
15 + p4v2_prisme_s3:
16 + mode: prisme
17 + model: from_checkpoint
18 + checkpoint: experiments/runs/p4v2_prisme_seed03_20261030001749/checkpoint
19 + controller: false
20 + policies:
21 + - budget0
22 + - forced_b2
23 + seed: 3
24 + p4v2_gr_s3:
25 + mode: global_recurrence
26 + model: from_checkpoint
27 + checkpoint: experiments/runs/p4v2_gr_seed03_20261030001749/checkpoint
28 + controller: false
29 + policies:
30 + - budget0
31 + - forced_b2
32 + seed: 3
33 + baseline_kwargs:
34 + ffn_w: 640
35 + p4v2_dense_s3:
36 + mode: dense
37 + model: from_checkpoint
38 + checkpoint: experiments/runs/p4v2_dense_seed03_20261030004544/checkpoint
39 + controller: false
40 + policies:
41 + - budget0
42 + - forced_b2
43 + seed: 3
added configs/experiments/test_eval_v2_s4.yaml +43 −0
@@ -0,0 +1,43 @@
1 +out_dir: results/raw
2 +data:
3 + kind: synthetic
4 + tokenizer: char
5 + splits_dir: data/splits/synthetic_v1
6 +splits:
7 +- test_iid
8 +- test_template
9 +- test_depth
10 +- test_compose
11 +batch_size: 64
12 +seq_len: 320
13 +max_items: 1000
14 +systems:
15 + p4v2_prisme_s4:
16 + mode: prisme
17 + model: from_checkpoint
18 + checkpoint: experiments/runs/p4v2_prisme_seed04_20261030001749/checkpoint
19 + controller: false
20 + policies:
21 + - budget0
22 + - forced_b2
23 + seed: 4
24 + p4v2_gr_s4:
25 + mode: global_recurrence
26 + model: from_checkpoint
27 + checkpoint: experiments/runs/p4v2_gr_seed04_20261030001749/checkpoint
28 + controller: false
29 + policies:
30 + - budget0
31 + - forced_b2
32 + seed: 4
33 + baseline_kwargs:
34 + ffn_w: 640
35 + p4v2_dense_s4:
36 + mode: dense
37 + model: from_checkpoint
38 + checkpoint: experiments/runs/p4v2_dense_seed04_20261030001749/checkpoint
39 + controller: false
40 + policies:
41 + - budget0
42 + - forced_b2
43 + seed: 4
added configs/experiments/test_eval_v2_s5.yaml +43 −0
@@ -0,0 +1,43 @@
1 +out_dir: results/raw
2 +data:
3 + kind: synthetic
4 + tokenizer: char
5 + splits_dir: data/splits/synthetic_v1
6 +splits:
7 +- test_iid
8 +- test_template
9 +- test_depth
10 +- test_compose
11 +batch_size: 64
12 +seq_len: 320
13 +max_items: 1000
14 +systems:
15 + p4v2_prisme_s5:
16 + mode: prisme
17 + model: from_checkpoint
18 + checkpoint: experiments/runs/p4v2_prisme_seed05_20261030001839/checkpoint
19 + controller: false
20 + policies:
21 + - budget0
22 + - forced_b2
23 + seed: 5
24 + p4v2_gr_s5:
25 + mode: global_recurrence
26 + model: from_checkpoint
27 + checkpoint: experiments/runs/p4v2_gr_seed05_20261030001839/checkpoint
28 + controller: false
29 + policies:
30 + - budget0
31 + - forced_b2
32 + seed: 5
33 + baseline_kwargs:
34 + ffn_w: 640
35 + p4v2_dense_s5:
36 + mode: dense
37 + model: from_checkpoint
38 + checkpoint: experiments/runs/p4v2_dense_seed05_20261030004544/checkpoint
39 + controller: false
40 + policies:
41 + - budget0
42 + - forced_b2
43 + seed: 5
added experiments/runs/pretrain_enlarged102m_seed01_20261030015017/config.yaml +52 −0
@@ -0,0 +1,52 @@
1 +experiment:
2 + name: pretrain_enlarged102m
3 + seed: 1
4 + model: configs/baselines/transformer_enlarged_102m.yaml
5 + data:
6 + train_bin: data/processed/fineweb_edu_300m/train.bin
7 + val_bin: data/processed/fineweb_edu_300m/val.bin
8 + train: &id001
9 + batch_size: 32
10 + seq_len: 512
11 + steps: 6000
12 + lr: 0.0006
13 + warmup: 300
14 + weight_decay: 0.1
15 + grad_clip: 1.0
16 + log_every: 20
17 + eval_every: 500
18 + eval_batches: 20
19 + ckpt_every: 1000
20 +model:
21 + name: transformer_enlarged_102m
22 + vocab_size: 32768
23 + d_model: 768
24 + n_layers: 12
25 + n_heads: 12
26 + n_kv_heads: 3
27 + ffn_width: 2144
28 + max_seq_len: 4096
29 + rope_theta: 10000.0
30 + norm_eps: 1.0e-05
31 + tie_embeddings: true
32 + init_std: 0.02
33 + use_workspace: false
34 + w: 256
35 + ffn_w: 768
36 + n_slots: 16
37 + n_editor_blocks: 2
38 + n_editor_heads: 4
39 + r_max: 8
40 + eta_max: 0.25
41 + clip_c: 1.0
42 + clip_eps: 1.0e-06
43 + readout_zero_init: true
44 + readout_init_std: 0.02
45 + readout_scale: 1.0
46 + controller_hidden: 0
47 + ctrl_input: z
48 + ctrl_use_entropy: false
49 + lambda_cost: 0.0
50 + kappa_risk: 0.0
51 + dense_writes: false
52 +resolved_train: *id001
added experiments/runs/pretrain_enlarged102m_seed01_20261030015017/manifest.json +26 −0
@@ -0,0 +1,26 @@
1 +{
2 + "run_id": "pretrain_enlarged102m_seed01_20261030015017",
3 + "name": "pretrain_enlarged102m",
4 + "seed": 1,
5 + "code_commit": "unknown",
6 + "config_sha256": "efbc5ac5e13ac94c05ef6d273e2701851307b5066ea5160e171e89ea81bb83ed",
7 + "started_at": "2026-10-30T01:50:17-0400",
8 + "host": "m3u96b",
9 + "python": "3.12.15",
10 + "torch": "2.15.0",
11 + "command": "/Users/simon-pierreboucher/wp31/prisme-slm/training/pretrain.py --config configs/experiments/pretrain_enlarged_102m.yaml --device mps",
12 + "state": "completed",
13 + "wall_seconds": 8527.03,
14 + "device": "mps",
15 + "backend": "mps",
16 + "precision": "fp32",
17 + "n_params": 102157056,
18 + "dataset_manifest_sha256": "d8753bdfa9609fce56a7ec2da7c108666185d021e3f921721e4d2241e1d4fe54",
19 + "checkpoint_parent": null,
20 + "finished_at": "2026-10-30T04:12:24-0400",
21 + "train_tokens": 98304000,
22 + "estimated_train_flops": 60254683398144000,
23 + "peak_memory_bytes": 28522004480,
24 + "final_val_loss": 3.8689021825790406,
25 + "metrics_file": "experiments/runs/pretrain_enlarged102m_seed01_20261030015017/metrics.jsonl"
26 +}
\ No newline at end of file
added experiments/runs/pretrain_enlarged102m_seed01_20261030015017/metrics.jsonl +319 −0
@@ -0,0 +1,319 @@
1 +{"step": 0, "loss": 10.538154602050781, "lr": 2e-06, "grad_norm": 8.826038360595703, "tokens": 16384, "tok_per_s": 12267.008820282133, "t": 1.98}
2 +{"step": 20, "loss": 9.062786102294922, "lr": 4.2e-05, "grad_norm": 2.396116018295288, "tokens": 344064, "tok_per_s": 12236.186788926587, "t": 28.76}
3 +{"step": 40, "loss": 8.023894309997559, "lr": 8.199999999999999e-05, "grad_norm": 1.9327870607376099, "tokens": 671744, "tok_per_s": 12224.43886163398, "t": 55.59}
4 +{"step": 60, "loss": 7.147858619689941, "lr": 0.00012199999999999998, "grad_norm": 1.0073579549789429, "tokens": 999424, "tok_per_s": 12227.853585676065, "t": 82.38}
5 +{"step": 80, "loss": 6.872986793518066, "lr": 0.00016199999999999998, "grad_norm": 0.9701820015907288, "tokens": 1327104, "tok_per_s": 12350.690989394596, "t": 108.09}
6 +{"step": 100, "loss": 6.5623087882995605, "lr": 0.00020199999999999998, "grad_norm": 1.135318398475647, "tokens": 1654784, "tok_per_s": 12246.719225784744, "t": 135.76}
7 +{"step": 120, "loss": 6.4833784103393555, "lr": 0.000242, "grad_norm": 0.9940797090530396, "tokens": 1982464, "tok_per_s": 12290.734902855935, "t": 161.94}
8 +{"step": 140, "loss": 6.285987854003906, "lr": 0.00028199999999999997, "grad_norm": 0.9160900115966797, "tokens": 2310144, "tok_per_s": 12263.175739855946, "t": 189.02}
9 +{"step": 160, "loss": 6.127978324890137, "lr": 0.00032199999999999997, "grad_norm": 0.8789034485816956, "tokens": 2637824, "tok_per_s": 12244.953978378258, "t": 216.06}
10 +{"step": 180, "loss": 6.162482261657715, "lr": 0.00036199999999999996, "grad_norm": 1.3229773044586182, "tokens": 2965504, "tok_per_s": 12291.164784312696, "t": 241.91}
11 +{"step": 200, "loss": 6.06372594833374, "lr": 0.00040199999999999996, "grad_norm": 0.9360699653625488, "tokens": 3293184, "tok_per_s": 12260.65061054621, "t": 269.24}
12 +{"step": 220, "loss": 5.960588455200195, "lr": 0.000442, "grad_norm": 0.9670940637588501, "tokens": 3620864, "tok_per_s": 12250.598133371244, "t": 296.21}
13 +{"step": 240, "loss": 5.984297752380371, "lr": 0.00048199999999999995, "grad_norm": 0.8978189826011658, "tokens": 3948544, "tok_per_s": 12291.705053241867, "t": 321.88}
14 +{"step": 260, "loss": 5.8681182861328125, "lr": 0.000522, "grad_norm": 0.8115763068199158, "tokens": 4276224, "tok_per_s": 12275.620218191996, "t": 348.99}
15 +{"step": 280, "loss": 5.783034324645996, "lr": 0.0005619999999999999, "grad_norm": 0.8142603635787964, "tokens": 4603904, "tok_per_s": 12285.887365268558, "t": 375.37}
16 +{"step": 300, "loss": 5.728802680969238, "lr": 0.0006, "grad_norm": 0.7503604292869568, "tokens": 4931584, "tok_per_s": 12283.22840230745, "t": 402.13}
17 +{"step": 320, "loss": 5.7670793533325195, "lr": 0.0005999835963914726, "grad_norm": 1.6453608274459839, "tokens": 5259264, "tok_per_s": 12274.431994656205, "t": 429.12}
18 +{"step": 340, "loss": 5.742429733276367, "lr": 0.0005999343875590635, "grad_norm": 0.7031630277633667, "tokens": 5586944, "tok_per_s": 12280.981116658615, "t": 455.57}
19 +{"step": 360, "loss": 5.577595233917236, "lr": 0.00059985237948205, "grad_norm": 0.8332432508468628, "tokens": 5914624, "tok_per_s": 12268.441190882228, "t": 482.74}
20 +{"step": 380, "loss": 5.530177116394043, "lr": 0.0005997375821250868, "grad_norm": 0.6786928176879883, "tokens": 6242304, "tok_per_s": 12279.671362677862, "t": 508.99}
21 +{"step": 400, "loss": 5.6273932456970215, "lr": 0.0005995900094369953, "grad_norm": 6.397047519683838, "tokens": 6569984, "tok_per_s": 12264.629455104921, "t": 536.33}
22 +{"step": 420, "loss": 5.520440101623535, "lr": 0.000599409679349069, "grad_norm": 0.6246231198310852, "tokens": 6897664, "tok_per_s": 12287.867050340814, "t": 561.98}
23 +{"step": 440, "loss": 5.382430076599121, "lr": 0.0005991966137728945, "grad_norm": 0.761988639831543, "tokens": 7225344, "tok_per_s": 12279.624106426805, "t": 589.04}
24 +{"step": 460, "loss": 5.283364772796631, "lr": 0.0005989508385976885, "grad_norm": 0.6207935214042664, "tokens": 7553024, "tok_per_s": 12270.523383262516, "t": 616.18}
25 +{"step": 480, "loss": 5.33247184753418, "lr": 0.000598672383687153, "grad_norm": 0.5935987234115601, "tokens": 7880704, "tok_per_s": 12246.607024038762, "t": 644.14}
26 +{"step": 499, "val_loss": 5.376418542861939, "val_ppl": 216.246409641857, "tokens": 8192000, "t": 676.9}
27 +{"step": 500, "loss": 5.335311412811279, "lr": 0.0005983612828758463, "grad_norm": 0.6082054972648621, "tokens": 8208384, "tok_per_s": 12117.563445660973, "t": 678.04}
28 +{"step": 520, "loss": 5.3863677978515625, "lr": 0.0005980175739650714, "grad_norm": 0.6699115037918091, "tokens": 8536064, "tok_per_s": 12089.311821260695, "t": 706.73}
29 +{"step": 540, "loss": 5.296011924743652, "lr": 0.0005976412987182833, "grad_norm": 0.757933497428894, "tokens": 8863744, "tok_per_s": 12075.933905206339, "t": 734.64}
30 +{"step": 560, "loss": 5.353665351867676, "lr": 0.000597232502856014, "grad_norm": 0.6468153595924377, "tokens": 9191424, "tok_per_s": 12053.971988970368, "t": 763.16}
31 +{"step": 580, "loss": 5.164031982421875, "lr": 0.0005967912360503179, "grad_norm": 0.5246732831001282, "tokens": 9519104, "tok_per_s": 12039.288807349485, "t": 791.31}
32 +{"step": 600, "loss": 5.383706092834473, "lr": 0.000596317551918735, "grad_norm": 0.5707435607910156, "tokens": 9846784, "tok_per_s": 12027.311249556898, "t": 819.34}
33 +{"step": 620, "loss": 5.237800598144531, "lr": 0.0005958115080177768, "grad_norm": 0.5295183658599854, "tokens": 10174464, "tok_per_s": 12017.056818105553, "t": 847.31}
34 +{"step": 640, "loss": 5.215066909790039, "lr": 0.0005952731658359328, "grad_norm": 0.548611581325531, "tokens": 10502144, "tok_per_s": 11992.821053121737, "t": 876.34}
35 +{"step": 660, "loss": 5.153785705566406, "lr": 0.0005947025907861984, "grad_norm": 0.5935801863670349, "tokens": 10829824, "tok_per_s": 11977.674335609541, "t": 904.81}
36 +{"step": 680, "loss": 5.202521324157715, "lr": 0.0005940998521981274, "grad_norm": 0.5564954876899719, "tokens": 11157504, "tok_per_s": 11959.257592686838, "t": 933.6}
37 +{"step": 700, "loss": 5.146511554718018, "lr": 0.0005934650233094076, "grad_norm": 0.5552934408187866, "tokens": 11485184, "tok_per_s": 11941.760258932192, "t": 962.41}
38 +{"step": 720, "loss": 5.095799446105957, "lr": 0.0005927981812569615, "grad_norm": 0.567599892616272, "tokens": 11812864, "tok_per_s": 11935.94153600311, "t": 990.33}
39 +{"step": 740, "loss": 4.996639251708984, "lr": 0.0005920994070675743, "grad_norm": 0.578127920627594, "tokens": 12140544, "tok_per_s": 11925.37125969469, "t": 1018.69}
40 +{"step": 760, "loss": 4.951390266418457, "lr": 0.000591368785648048, "grad_norm": 0.671257734298706, "tokens": 12468224, "tok_per_s": 11910.112552015506, "t": 1047.5}
41 +{"step": 780, "loss": 5.068446159362793, "lr": 0.0005906064057748841, "grad_norm": 0.5128580331802368, "tokens": 12795904, "tok_per_s": 11887.9204368979, "t": 1077.02}
42 +{"step": 800, "loss": 5.168520927429199, "lr": 0.0005898123600834975, "grad_norm": 5.2463555335998535, "tokens": 13123584, "tok_per_s": 11875.612775467493, "t": 1105.73}
43 +{"step": 820, "loss": 5.072286605834961, "lr": 0.00058898674505696, "grad_norm": 0.6444844603538513, "tokens": 13451264, "tok_per_s": 11876.670710730987, "t": 1133.22}
44 +{"step": 840, "loss": 4.948967456817627, "lr": 0.0005881296610142765, "grad_norm": 0.5277834534645081, "tokens": 13778944, "tok_per_s": 11873.161622668349, "t": 1161.15}
45 +{"step": 860, "loss": 4.968014717102051, "lr": 0.0005872412120981961, "grad_norm": 0.5329476594924927, "tokens": 14106624, "tok_per_s": 11869.584852756028, "t": 1189.11}
46 +{"step": 880, "loss": 4.756638526916504, "lr": 0.0005863215062625574, "grad_norm": 0.5423914194107056, "tokens": 14434304, "tok_per_s": 11877.591588374811, "t": 1215.9}
47 +{"step": 900, "loss": 4.976390838623047, "lr": 0.0005853706552591712, "grad_norm": 0.5418165326118469, "tokens": 14761984, "tok_per_s": 11875.467479769606, "t": 1243.71}
48 +{"step": 920, "loss": 4.84976863861084, "lr": 0.0005843887746242426, "grad_norm": 0.6431712508201599, "tokens": 15089664, "tok_per_s": 11873.266409096448, "t": 1271.54}
49 +{"step": 940, "loss": 4.840450286865234, "lr": 0.000583375983664331, "grad_norm": 0.5837947726249695, "tokens": 15417344, "tok_per_s": 11877.077855763022, "t": 1298.72}
50 +{"step": 960, "loss": 4.9250969886779785, "lr": 0.0005823324054418544, "grad_norm": 0.5661423206329346, "tokens": 15745024, "tok_per_s": 11868.1409528405, "t": 1327.31}
51 +{"step": 980, "loss": 5.006077289581299, "lr": 0.0005812581667601359, "grad_norm": 1.4778692722320557, "tokens": 16072704, "tok_per_s": 11861.968977733293, "t": 1355.62}
52 +{"step": 999, "val_loss": 4.868359494209289, "val_ppl": 130.10729994547023, "tokens": 16384000, "t": 1387.8}
53 +{"step": 999, "checkpoint_sha256": "cda289d1d0f642b19ecbf6e6c12a162167897565760a2f30d1652474fd2adb60", "t": 1388.18}
54 +{"step": 1000, "loss": 4.746236324310303, "lr": 0.0005801533981479964, "grad_norm": 0.5520177483558655, "tokens": 16400384, "tok_per_s": 11808.062608862347, "t": 1389.56}
55 +{"step": 1020, "loss": 4.937170028686523, "lr": 0.0005790182338438938, "grad_norm": 0.5781713128089905, "tokens": 16728064, "tok_per_s": 11805.29816888464, "t": 1417.64}
56 +{"step": 1040, "loss": 4.894624710083008, "lr": 0.0005778528117796123, "grad_norm": 0.5315879583358765, "tokens": 17055744, "tok_per_s": 11803.357443436133, "t": 1445.63}
57 +{"step": 1060, "loss": 4.880748271942139, "lr": 0.0005766572735635022, "grad_norm": 1.2597907781600952, "tokens": 17383424, "tok_per_s": 11790.355502316252, "t": 1475.02}
58 +{"step": 1080, "loss": 5.013078689575195, "lr": 0.0005754317644632738, "grad_norm": 0.6085506081581116, "tokens": 17711104, "tok_per_s": 11787.496721408717, "t": 1503.18}
59 +{"step": 1100, "loss": 4.827304840087891, "lr": 0.000574176433388346, "grad_norm": 0.5576329827308655, "tokens": 18038784, "tok_per_s": 11797.208143102953, "t": 1529.71}
60 +{"step": 1120, "loss": 4.837977409362793, "lr": 0.0005728914328717517, "grad_norm": 0.5152426958084106, "tokens": 18366464, "tok_per_s": 11795.445317768213, "t": 1557.72}
61 +{"step": 1140, "loss": 4.640369415283203, "lr": 0.000571576919051606, "grad_norm": 0.5521364212036133, "tokens": 18694144, "tok_per_s": 11795.230672817028, "t": 1585.53}
62 +{"step": 1160, "loss": 4.757654666900635, "lr": 0.0005702330516521316, "grad_norm": 0.5247706770896912, "tokens": 19021824, "tok_per_s": 11788.59224630508, "t": 1614.24}
63 +{"step": 1180, "loss": 4.691868782043457, "lr": 0.0005688599939642524, "grad_norm": 0.5326834917068481, "tokens": 19349504, "tok_per_s": 11791.14385723584, "t": 1641.66}
64 +{"step": 1200, "loss": 4.747924327850342, "lr": 0.000567457912825752, "grad_norm": 0.5180054903030396, "tokens": 19677184, "tok_per_s": 11792.949766833508, "t": 1669.2}
65 +{"step": 1220, "loss": 4.812901496887207, "lr": 0.000566026978601002, "grad_norm": 0.5599668622016907, "tokens": 20004864, "tok_per_s": 11783.796474617193, "t": 1698.3}
66 +{"step": 1240, "loss": 4.624703407287598, "lr": 0.00056456736516026, "grad_norm": 0.6033844351768494, "tokens": 20332544, "tok_per_s": 11779.719556755437, "t": 1726.71}
67 +{"step": 1260, "loss": 4.742227077484131, "lr": 0.000563079249858545, "grad_norm": 0.5360792875289917, "tokens": 20660224, "tok_per_s": 11779.076173728605, "t": 1754.62}
68 +{"step": 1280, "loss": 4.559293270111084, "lr": 0.000561562813514085, "grad_norm": 0.557126522064209, "tokens": 20987904, "tok_per_s": 11770.653351125746, "t": 1783.71}
69 +{"step": 1300, "loss": 4.704896926879883, "lr": 0.0005600182403863482, "grad_norm": 0.552845299243927, "tokens": 21315584, "tok_per_s": 11774.175946588024, "t": 1811.01}
70 +{"step": 1320, "loss": 4.649643898010254, "lr": 0.0005584457181536525, "grad_norm": 0.5440144538879395, "tokens": 21643264, "tok_per_s": 11778.277753235605, "t": 1838.2}
71 +{"step": 1340, "loss": 4.542902946472168, "lr": 0.0005568454378903616, "grad_norm": 0.5623109340667725, "tokens": 21970944, "tok_per_s": 11782.242522633296, "t": 1865.39}
72 +{"step": 1360, "loss": 4.589754104614258, "lr": 0.0005552175940436679, "grad_norm": 0.5271830558776855, "tokens": 22298624, "tok_per_s": 11772.282296335274, "t": 1894.81}
73 +{"step": 1380, "loss": 4.611910820007324, "lr": 0.0005535623844099653, "grad_norm": 0.5605332255363464, "tokens": 22626304, "tok_per_s": 11767.477063289036, "t": 1923.43}
74 +{"step": 1400, "loss": 4.653350830078125, "lr": 0.0005518800101108161, "grad_norm": 0.5224345326423645, "tokens": 22953984, "tok_per_s": 11763.642777929157, "t": 1951.91}
75 +{"step": 1420, "loss": 4.504340171813965, "lr": 0.000550170675568512, "grad_norm": 0.5628354549407959, "tokens": 23281664, "tok_per_s": 11759.425495362444, "t": 1980.47}
76 +{"step": 1440, "loss": 4.538120746612549, "lr": 0.0005484345884812357, "grad_norm": 0.5646089911460876, "tokens": 23609344, "tok_per_s": 11763.342947627734, "t": 2007.67}
77 +{"step": 1460, "loss": 4.548748970031738, "lr": 0.000546671959797824, "grad_norm": 0.5454878211021423, "tokens": 23937024, "tok_per_s": 11766.325077504327, "t": 2035.01}
78 +{"step": 1480, "loss": 4.4955878257751465, "lr": 0.0005448830036921353, "grad_norm": 0.5409777760505676, "tokens": 24264704, "tok_per_s": 11760.413489326585, "t": 2063.9}
79 +{"step": 1499, "val_loss": 4.55281765460968, "val_ppl": 94.8994257612488, "tokens": 24576000, "t": 2098.38}
80 +{"step": 1500, "loss": 4.449386119842529, "lr": 0.0005430679375370263, "grad_norm": 0.5416464805603027, "tokens": 24592384, "tok_per_s": 11715.626049715813, "t": 2099.75}
81 +{"step": 1520, "loss": 4.506931304931641, "lr": 0.0005412269818779387, "grad_norm": 0.5411835312843323, "tokens": 24920064, "tok_per_s": 11709.046886550848, "t": 2128.92}
82 +{"step": 1540, "loss": 4.46286678314209, "lr": 0.0005393603604061018, "grad_norm": 0.5418587327003479, "tokens": 25247744, "tok_per_s": 11712.523059721985, "t": 2156.26}
83 +{"step": 1560, "loss": 4.550535202026367, "lr": 0.0005374682999313519, "grad_norm": 0.9948550462722778, "tokens": 25575424, "tok_per_s": 11706.191924192692, "t": 2185.42}
84 +{"step": 1580, "loss": 4.395641326904297, "lr": 0.000535551030354573, "grad_norm": 0.5468602776527405, "tokens": 25903104, "tok_per_s": 11708.36429591705, "t": 2213.0}
85 +{"step": 1600, "loss": 4.452836036682129, "lr": 0.000533608784639762, "grad_norm": 0.5829842686653137, "tokens": 26230784, "tok_per_s": 11711.802983617088, "t": 2240.33}
86 +{"step": 1620, "loss": 4.391358375549316, "lr": 0.0005316417987857215, "grad_norm": 0.5233222246170044, "tokens": 26558464, "tok_per_s": 11704.320435493784, "t": 2269.76}
87 +{"step": 1640, "loss": 4.432625770568848, "lr": 0.0005296503117973845, "grad_norm": 0.5276442766189575, "tokens": 26886144, "tok_per_s": 11713.19016145784, "t": 2296.02}
88 +{"step": 1660, "loss": 4.441397666931152, "lr": 0.0005276345656567725, "grad_norm": 0.5338090062141418, "tokens": 27213824, "tok_per_s": 11709.577315533745, "t": 2324.71}
89 +{"step": 1680, "loss": 4.608345985412598, "lr": 0.000525594805293594, "grad_norm": 0.5458671450614929, "tokens": 27541504, "tok_per_s": 11706.816270307252, "t": 2353.25}
90 +{"step": 1700, "loss": 4.404858589172363, "lr": 0.0005235312785554818, "grad_norm": 0.5006349682807922, "tokens": 27869184, "tok_per_s": 11703.181470092059, "t": 2381.98}
91 +{"step": 1720, "loss": 4.4875993728637695, "lr": 0.000521444236177879, "grad_norm": 0.5281708240509033, "tokens": 28196864, "tok_per_s": 11706.901990885435, "t": 2409.21}
92 +{"step": 1740, "loss": 4.452470779418945, "lr": 0.0005193339317535718, "grad_norm": 0.5706871747970581, "tokens": 28524544, "tok_per_s": 11709.705702930381, "t": 2436.62}
93 +{"step": 1760, "loss": 4.406674385070801, "lr": 0.0005172006217018763, "grad_norm": 0.5338311195373535, "tokens": 28852224, "tok_per_s": 11705.233072294539, "t": 2465.54}
94 +{"step": 1780, "loss": 4.415951728820801, "lr": 0.0005150445652374809, "grad_norm": 0.54909747838974, "tokens": 29179904, "tok_per_s": 11705.049532957939, "t": 2493.58}
95 +{"step": 1800, "loss": 4.3329620361328125, "lr": 0.0005128660243389501, "grad_norm": 0.523220419883728, "tokens": 29507584, "tok_per_s": 11699.735345727024, "t": 2522.72}
96 +{"step": 1820, "loss": 4.487624645233154, "lr": 0.0005106652637168917, "grad_norm": 0.579258143901825, "tokens": 29835264, "tok_per_s": 11702.42285860611, "t": 2550.14}
97 +{"step": 1840, "loss": 4.522067070007324, "lr": 0.0005084425507817925, "grad_norm": 0.5385577082633972, "tokens": 30162944, "tok_per_s": 11697.895599891974, "t": 2579.14}
98 +{"step": 1860, "loss": 4.351419448852539, "lr": 0.0005061981556115255, "grad_norm": 0.5320832133293152, "tokens": 30490624, "tok_per_s": 11699.751976479256, "t": 2606.73}
99 +{"step": 1880, "loss": 4.346895694732666, "lr": 0.0005039323509185333, "grad_norm": 0.5495853424072266, "tokens": 30818304, "tok_per_s": 11702.60960328932, "t": 2634.1}
100 +{"step": 1900, "loss": 4.382790565490723, "lr": 0.0005016454120166914, "grad_norm": 0.6258963346481323, "tokens": 31145984, "tok_per_s": 11697.128749691117, "t": 2663.35}
101 +{"step": 1920, "loss": 4.427793502807617, "lr": 0.0004993376167878549, "grad_norm": 0.5516924858093262, "tokens": 31473664, "tok_per_s": 11691.72184032782, "t": 2692.6}
102 +{"step": 1940, "loss": 4.197183609008789, "lr": 0.0004970092456480945, "grad_norm": 0.588075578212738, "tokens": 31801344, "tok_per_s": 11690.4435519863, "t": 2720.93}
103 +{"step": 1960, "loss": 4.288908958435059, "lr": 0.0004946605815136225, "grad_norm": 0.5377606749534607, "tokens": 32129024, "tok_per_s": 11687.307367410403, "t": 2749.7}
104 +{"step": 1980, "loss": 4.317612171173096, "lr": 0.0004922919097664167, "grad_norm": 0.5370826125144958, "tokens": 32456704, "tok_per_s": 11689.41867836808, "t": 2777.23}
105 +{"step": 1999, "val_loss": 4.377403140068054, "val_ppl": 79.63097418334951, "tokens": 32768000, "t": 2810.66}
106 +{"step": 1999, "checkpoint_sha256": "0aa6a9f604a8834952c90f7fdbc128abdeda90a2fab99192c8d6e82ae49cb5fd", "t": 2811.09}
107 +{"step": 2000, "loss": 4.231821537017822, "lr": 0.0004899035182195446, "grad_norm": 0.5784196257591248, "tokens": 32784384, "tok_per_s": 11658.385223092659, "t": 2812.73}
108 +{"step": 2020, "loss": 4.3884429931640625, "lr": 0.00048749569708219075, "grad_norm": 0.525882363319397, "tokens": 33112064, "tok_per_s": 11656.431473337303, "t": 2841.31}
109 +{"step": 2040, "loss": 4.128255844116211, "lr": 0.0004850687389243948, "grad_norm": 0.5654372572898865, "tokens": 33439744, "tok_per_s": 11655.083826786376, "t": 2869.75}
110 +{"step": 2060, "loss": 4.392373561859131, "lr": 0.00048262293864150137, "grad_norm": 0.5647616982460022, "tokens": 33767424, "tok_per_s": 11651.209102049805, "t": 2898.83}
111 +{"step": 2080, "loss": 4.272017478942871, "lr": 0.0004801585934183281, "grad_norm": 0.5246736407279968, "tokens": 34095104, "tok_per_s": 11654.059240044917, "t": 2926.24}
112 +{"step": 2100, "loss": 4.351164817810059, "lr": 0.0004776760026930552, "grad_norm": 0.5639843344688416, "tokens": 34422784, "tok_per_s": 11651.157758820587, "t": 2955.09}
113 +{"step": 2120, "loss": 4.381256103515625, "lr": 0.0004751754681208411, "grad_norm": 0.5990827679634094, "tokens": 34750464, "tok_per_s": 11654.334181549966, "t": 2982.41}
114 +{"step": 2140, "loss": 4.293796539306641, "lr": 0.0004726572935371689, "grad_norm": 0.5889109373092651, "tokens": 35078144, "tok_per_s": 11657.838494768248, "t": 3009.62}
115 +{"step": 2160, "loss": 4.318106651306152, "lr": 0.00047012178492092805, "grad_norm": 0.5541678667068481, "tokens": 35405824, "tok_per_s": 11652.700123326464, "t": 3039.06}
116 +{"step": 2180, "loss": 4.370950698852539, "lr": 0.00046756925035723536, "grad_norm": 0.523361086845398, "tokens": 35733504, "tok_per_s": 11651.574937822988, "t": 3067.48}
117 +{"step": 2200, "loss": 4.3684163093566895, "lr": 0.00046499999999999997, "grad_norm": 0.5235266089439392, "tokens": 36061184, "tok_per_s": 11650.641940686393, "t": 3095.85}
118 +{"step": 2220, "loss": 4.416513442993164, "lr": 0.0004624143460342372, "grad_norm": 0.5547477006912231, "tokens": 36388864, "tok_per_s": 11650.49520771614, "t": 3124.02}
119 +{"step": 2240, "loss": 4.400991439819336, "lr": 0.00045981260263813534, "grad_norm": 0.6226858496665955, "tokens": 36716544, "tok_per_s": 11652.8716753309, "t": 3151.5}
120 +{"step": 2260, "loss": 4.15883731842041, "lr": 0.0004571950859448811, "grad_norm": 0.5616633296012878, "tokens": 37044224, "tok_per_s": 11655.973818399472, "t": 3178.77}
121 +{"step": 2280, "loss": 4.316667556762695, "lr": 0.0004545621140042457, "grad_norm": 0.568236231803894, "tokens": 37371904, "tok_per_s": 11653.730269233454, "t": 3207.5}
122 +{"step": 2300, "loss": 4.269564628601074, "lr": 0.00045191400674394013, "grad_norm": 0.5399973392486572, "tokens": 37699584, "tok_per_s": 11650.245063967917, "t": 3236.59}
123 +{"step": 2320, "loss": 4.225350379943848, "lr": 0.000449251085930741, "grad_norm": 0.5322178602218628, "tokens": 38027264, "tok_per_s": 11646.621966280994, "t": 3265.73}
124 +{"step": 2340, "loss": 4.254165172576904, "lr": 0.00044657367513139303, "grad_norm": 0.5262772440910339, "tokens": 38354944, "tok_per_s": 11649.606915520488, "t": 3293.02}
125 +{"step": 2360, "loss": 4.305544853210449, "lr": 0.00044388209967329364, "grad_norm": 0.5276159644126892, "tokens": 38682624, "tok_per_s": 11646.145968824367, "t": 3322.14}
126 +{"step": 2380, "loss": 4.318025588989258, "lr": 0.00044117668660496233, "grad_norm": 0.5362470149993896, "tokens": 39010304, "tok_per_s": 11649.147628536659, "t": 3349.41}
127 +{"step": 2400, "loss": 4.189087390899658, "lr": 0.00043845776465630177, "grad_norm": 0.5912144184112549, "tokens": 39337984, "tok_per_s": 11652.166209961133, "t": 3376.67}
128 +{"step": 2420, "loss": 4.167083263397217, "lr": 0.00043572566419865466, "grad_norm": 0.5396695733070374, "tokens": 39665664, "tok_per_s": 11653.654786173189, "t": 3404.35}
129 +{"step": 2440, "loss": 4.282366752624512, "lr": 0.0004329807172046608, "grad_norm": 0.5638473629951477, "tokens": 39993344, "tok_per_s": 11650.457568695849, "t": 3433.41}
130 +{"step": 2460, "loss": 4.179342269897461, "lr": 0.00043022325720791954, "grad_norm": 0.5641481876373291, "tokens": 40321024, "tok_per_s": 11647.07079450147, "t": 3462.54}
131 +{"step": 2480, "loss": 4.261195182800293, "lr": 0.000427453619262463, "grad_norm": 0.55649334192276, "tokens": 40648704, "tok_per_s": 11647.617384047178, "t": 3490.52}
132 +{"step": 2499, "val_loss": 4.253906965255737, "val_ppl": 70.37984751220374, "tokens": 40960000, "t": 3524.46}
133 +{"step": 2500, "loss": 4.270415782928467, "lr": 0.00042467213990204363, "grad_norm": 0.5214126706123352, "tokens": 40976384, "tok_per_s": 11623.909543881557, "t": 3525.82}
134 +{"step": 2520, "loss": 4.200478553771973, "lr": 0.0004218791570992436, "grad_norm": 0.5286369919776917, "tokens": 41304064, "tok_per_s": 11627.020290283132, "t": 3553.06}
135 +{"step": 2540, "loss": 4.271385192871094, "lr": 0.00041907501022440735, "grad_norm": 1.1362669467926025, "tokens": 41631744, "tok_per_s": 11624.355791304675, "t": 3582.07}
136 +{"step": 2560, "loss": 4.153648376464844, "lr": 0.00041626004000440593, "grad_norm": 0.5819516181945801, "tokens": 41959424, "tok_per_s": 11623.037056115738, "t": 3610.69}
137 +{"step": 2580, "loss": 4.239153861999512, "lr": 0.00041343458848123576, "grad_norm": 0.5144444108009338, "tokens": 42287104, "tok_per_s": 11619.319612031708, "t": 3640.02}
138 +{"step": 2600, "loss": 4.200573444366455, "lr": 0.0004105989989704575, "grad_norm": 0.562723696231842, "tokens": 42614784, "tok_per_s": 11622.04027606792, "t": 3667.36}
139 +{"step": 2620, "loss": 4.275785446166992, "lr": 0.0004077536160194806, "grad_norm": 0.5386180877685547, "tokens": 42942464, "tok_per_s": 11620.13482113355, "t": 3696.16}
140 +{"step": 2640, "loss": 4.140957355499268, "lr": 0.000404898785365698, "grad_norm": 0.51993727684021, "tokens": 43270144, "tok_per_s": 11622.032355083207, "t": 3723.76}
141 +{"step": 2660, "loss": 4.120570659637451, "lr": 0.000402034853894476, "grad_norm": 0.5389391183853149, "tokens": 43597824, "tok_per_s": 11624.461428055718, "t": 3751.17}
142 +{"step": 2680, "loss": 4.026679992675781, "lr": 0.000399162169597005, "grad_norm": 0.542602002620697, "tokens": 43925504, "tok_per_s": 11621.979828131509, "t": 3780.16}
143 +{"step": 2700, "loss": 4.190106391906738, "lr": 0.00039628108152801574, "grad_norm": 0.53343266248703, "tokens": 44253184, "tok_per_s": 11618.268985670818, "t": 3809.6}
144 +{"step": 2720, "loss": 4.038628578186035, "lr": 0.000393391939763366, "grad_norm": 0.5840466022491455, "tokens": 44580864, "tok_per_s": 11619.40309369032, "t": 3837.4}
145 +{"step": 2740, "loss": 4.085831642150879, "lr": 0.00039049509535750383, "grad_norm": 0.5408041477203369, "tokens": 44908544, "tok_per_s": 11618.701568373139, "t": 3865.84}
146 +{"step": 2760, "loss": 4.053369045257568, "lr": 0.0003875909003008116, "grad_norm": 0.5366416573524475, "tokens": 45236224, "tok_per_s": 11620.9278906342, "t": 3893.29}
147 +{"step": 2780, "loss": 4.154467582702637, "lr": 0.0003846797074768358, "grad_norm": 0.5264991521835327, "tokens": 45563904, "tok_per_s": 11618.84091486484, "t": 3922.2}
148 +{"step": 2800, "loss": 4.168743133544922, "lr": 0.00038176187061940935, "grad_norm": 0.536626398563385, "tokens": 45891584, "tok_per_s": 11616.338803380988, "t": 3951.25}
149 +{"step": 2820, "loss": 4.153240203857422, "lr": 0.0003788377442696697, "grad_norm": 0.533578634262085, "tokens": 46219264, "tok_per_s": 11615.253346568814, "t": 3979.83}
150 +{"step": 2840, "loss": 4.130046844482422, "lr": 0.00037590768373297907, "grad_norm": 0.5270524024963379, "tokens": 46546944, "tok_per_s": 11618.32541559027, "t": 4006.98}
151 +{"step": 2860, "loss": 4.124772071838379, "lr": 0.0003729720450357526, "grad_norm": 0.5412513613700867, "tokens": 46874624, "tok_per_s": 11617.065799558944, "t": 4035.62}
152 +{"step": 2880, "loss": 4.045408248901367, "lr": 0.0003700311848821973, "grad_norm": 0.5562382936477661, "tokens": 47202304, "tok_per_s": 11620.032790486088, "t": 4062.79}
153 +{"step": 2900, "loss": 4.053845405578613, "lr": 0.00036708546061097017, "grad_norm": 0.5519654154777527, "tokens": 47529984, "tok_per_s": 11622.144412185111, "t": 4090.25}
154 +{"step": 2920, "loss": 4.138660430908203, "lr": 0.00036413523015175846, "grad_norm": 0.5388080477714539, "tokens": 47857664, "tok_per_s": 11620.688842362795, "t": 4118.96}
155 +{"step": 2940, "loss": 4.136221885681152, "lr": 0.0003611808519817883, "grad_norm": 0.5571575164794922, "tokens": 48185344, "tok_per_s": 11617.611410784506, "t": 4148.25}
156 +{"step": 2960, "loss": 4.039947509765625, "lr": 0.0003582226850822664, "grad_norm": 0.535615861415863, "tokens": 48513024, "tok_per_s": 11618.557089478945, "t": 4176.12}
157 +{"step": 2980, "loss": 4.180006504058838, "lr": 0.0003552610888947618, "grad_norm": 0.5578852891921997, "tokens": 48840704, "tok_per_s": 11620.276163903865, "t": 4203.7}
158 +{"step": 2999, "val_loss": 4.158720552921295, "val_ppl": 63.989598897214684, "tokens": 49152000, "t": 4237.64}
159 +{"step": 2999, "checkpoint_sha256": "01088cbe14a4bbf4ef3d3f45ae99d689c24d2752c909afe60703401d5c293072", "t": 4238.01}
160 +{"step": 3000, "loss": 4.180983543395996, "lr": 0.0003522964232775297, "grad_norm": 0.5751718878746033, "tokens": 49168384, "tok_per_s": 11599.737328140918, "t": 4239.39}
161 +{"step": 3020, "loss": 4.092770576477051, "lr": 0.0003493290484617865, "grad_norm": 0.529112696647644, "tokens": 49496064, "tok_per_s": 11597.092272818163, "t": 4268.61}
162 +{"step": 3040, "loss": 4.0629377365112305, "lr": 0.0003463593250079383, "grad_norm": 0.5474892854690552, "tokens": 49823744, "tok_per_s": 11595.560939643, "t": 4297.44}
163 +{"step": 3060, "loss": 4.105502128601074, "lr": 0.00034338761376177, "grad_norm": 0.8342363834381104, "tokens": 50151424, "tok_per_s": 11597.205212677118, "t": 4325.08}
164 +{"step": 3080, "loss": 4.145478248596191, "lr": 0.00034041427581059977, "grad_norm": 0.5806731581687927, "tokens": 50479104, "tok_per_s": 11594.81427453882, "t": 4354.24}
165 +{"step": 3100, "loss": 4.065998554229736, "lr": 0.0003374396724394037, "grad_norm": 0.5266734957695007, "tokens": 50806784, "tok_per_s": 11597.401930260172, "t": 4381.52}
166 +{"step": 3120, "loss": 4.046976089477539, "lr": 0.0003344641650869168, "grad_norm": 0.5373362302780151, "tokens": 51134464, "tok_per_s": 11599.940600673119, "t": 4408.81}
167 +{"step": 3140, "loss": 4.0326666831970215, "lr": 0.0003314881153017151, "grad_norm": 0.5418212413787842, "tokens": 51462144, "tok_per_s": 11602.451890150127, "t": 4436.1}
168 +{"step": 3160, "loss": 4.104043006896973, "lr": 0.00032851188469828487, "grad_norm": 0.5828465223312378, "tokens": 51789824, "tok_per_s": 11599.187619918594, "t": 4465.6}
169 +{"step": 3180, "loss": 3.948253870010376, "lr": 0.0003255358349130832, "grad_norm": 0.5222381353378296, "tokens": 52117504, "tok_per_s": 11598.717562240729, "t": 4494.03}
170 +{"step": 3200, "loss": 4.206734657287598, "lr": 0.00032256032756059627, "grad_norm": 0.5611771941184998, "tokens": 52445184, "tok_per_s": 11598.044955429035, "t": 4522.54}
171 +{"step": 3220, "loss": 4.076472282409668, "lr": 0.00031958572418940007, "grad_norm": 0.5682035088539124, "tokens": 52772864, "tok_per_s": 11597.313869426645, "t": 4551.08}
172 +{"step": 3240, "loss": 3.881274700164795, "lr": 0.00031661238623822995, "grad_norm": 0.5410988330841064, "tokens": 53100544, "tok_per_s": 11599.778026789865, "t": 4578.36}
173 +{"step": 3260, "loss": 4.112787246704102, "lr": 0.00031364067499206177, "grad_norm": 0.5453398823738098, "tokens": 53428224, "tok_per_s": 11598.3489765245, "t": 4607.18}
174 +{"step": 3280, "loss": 4.042259216308594, "lr": 0.00031067095153821354, "grad_norm": 0.5254319906234741, "tokens": 53755904, "tok_per_s": 11596.062737123084, "t": 4636.35}
175 +{"step": 3300, "loss": 4.027780532836914, "lr": 0.0003077035767224703, "grad_norm": 0.5368713736534119, "tokens": 54083584, "tok_per_s": 11593.837629205618, "t": 4665.5}
176 +{"step": 3320, "loss": 4.048270225524902, "lr": 0.0003047389111052382, "grad_norm": 0.5798897743225098, "tokens": 54411264, "tok_per_s": 11594.94706767128, "t": 4693.31}
177 +{"step": 3340, "loss": 3.9507741928100586, "lr": 0.0003017773149177336, "grad_norm": 0.5504516363143921, "tokens": 54738944, "tok_per_s": 11594.143632157271, "t": 4721.9}
178 +{"step": 3360, "loss": 3.9647669792175293, "lr": 0.0002988191480182117, "grad_norm": 0.5416991114616394, "tokens": 55066624, "tok_per_s": 11596.184635204852, "t": 4749.33}
179 +{"step": 3380, "loss": 4.000699996948242, "lr": 0.0002958647698482415, "grad_norm": 0.5480679273605347, "tokens": 55394304, "tok_per_s": 11595.102761281029, "t": 4778.03}
180 +{"step": 3400, "loss": 3.971863031387329, "lr": 0.0002929145393890298, "grad_norm": 0.5375354290008545, "tokens": 55721984, "tok_per_s": 11594.402055720479, "t": 4806.58}
181 +{"step": 3420, "loss": 4.104773998260498, "lr": 0.0002899688151178026, "grad_norm": 0.5447196364402771, "tokens": 56049664, "tok_per_s": 11590.86988710522, "t": 4836.32}
182 +{"step": 3440, "loss": 4.046260833740234, "lr": 0.00028702795496424736, "grad_norm": 0.5488489866256714, "tokens": 56377344, "tok_per_s": 11592.886162092072, "t": 4863.74}
183 +{"step": 3460, "loss": 3.960139274597168, "lr": 0.00028409231626702093, "grad_norm": 0.5675356984138489, "tokens": 56705024, "tok_per_s": 11591.251066099452, "t": 4892.7}
184 +{"step": 3480, "loss": 3.994968891143799, "lr": 0.00028116225573033036, "grad_norm": 0.5636969208717346, "tokens": 57032704, "tok_per_s": 11592.733713437376, "t": 4920.34}
185 +{"step": 3499, "val_loss": 4.076284086704254, "val_ppl": 58.926098247434794, "tokens": 57344000, "t": 4953.42}
186 +{"step": 3500, "loss": 4.110962390899658, "lr": 0.00027823812938059065, "grad_norm": 0.5642924308776855, "tokens": 57360384, "tok_per_s": 11577.222286224182, "t": 4955.23}
187 +{"step": 3520, "loss": 3.9599759578704834, "lr": 0.0002753202925231642, "grad_norm": 0.5764612555503845, "tokens": 57688064, "tok_per_s": 11575.88157647093, "t": 4984.14}
188 +{"step": 3540, "loss": 3.9944944381713867, "lr": 0.0002724090996991884, "grad_norm": 0.5443633198738098, "tokens": 58015744, "tok_per_s": 11574.98417124486, "t": 5012.81}
189 +{"step": 3560, "loss": 4.018226623535156, "lr": 0.00026950490464249606, "grad_norm": 0.5742700099945068, "tokens": 58343424, "tok_per_s": 11573.307568254504, "t": 5041.85}
190 +{"step": 3580, "loss": 4.130144119262695, "lr": 0.0002666080602366339, "grad_norm": 0.5585811138153076, "tokens": 58671104, "tok_per_s": 11574.399101452585, "t": 5069.68}
191 +{"step": 3600, "loss": 4.045418739318848, "lr": 0.00026371891847198426, "grad_norm": 0.5662236213684082, "tokens": 58998784, "tok_per_s": 11573.813791413262, "t": 5098.25}
192 +{"step": 3620, "loss": 4.177013397216797, "lr": 0.00026083783040299483, "grad_norm": 0.5743661522865295, "tokens": 59326464, "tok_per_s": 11576.377095529744, "t": 5125.43}
193 +{"step": 3640, "loss": 3.959064483642578, "lr": 0.0002579651461055239, "grad_norm": 0.5499097108840942, "tokens": 59654144, "tok_per_s": 11578.98264556307, "t": 5152.58}
194 +{"step": 3660, "loss": 3.7604146003723145, "lr": 0.000255101214634302, "grad_norm": 0.5641483664512634, "tokens": 59981824, "tok_per_s": 11576.51833666578, "t": 5181.98}
195 +{"step": 3680, "loss": 4.021690845489502, "lr": 0.0002522463839805194, "grad_norm": 0.5463368892669678, "tokens": 60309504, "tok_per_s": 11575.705473660024, "t": 5210.65}
196 +{"step": 3700, "loss": 3.943697452545166, "lr": 0.00024940100102954256, "grad_norm": 0.5520758628845215, "tokens": 60637184, "tok_per_s": 11575.606176705642, "t": 5239.02}
197 +{"step": 3720, "loss": 3.904292583465576, "lr": 0.00024656541151876424, "grad_norm": 0.5493237972259521, "tokens": 60964864, "tok_per_s": 11577.859974252468, "t": 5266.28}
198 +{"step": 3740, "loss": 4.000242233276367, "lr": 0.00024373995999559398, "grad_norm": 0.5580956339836121, "tokens": 61292544, "tok_per_s": 11577.373754446042, "t": 5294.81}
199 +{"step": 3760, "loss": 3.917473554611206, "lr": 0.0002409249897755926, "grad_norm": 0.5592621564865112, "tokens": 61620224, "tok_per_s": 11575.636270353167, "t": 5323.91}
200 +{"step": 3780, "loss": 3.902782917022705, "lr": 0.00023812084290075633, "grad_norm": 0.5623103380203247, "tokens": 61947904, "tok_per_s": 11573.932404902851, "t": 5353.01}
201 +{"step": 3800, "loss": 4.045053482055664, "lr": 0.00023532786009795634, "grad_norm": 0.5493343472480774, "tokens": 62275584, "tok_per_s": 11576.10886221121, "t": 5380.31}
202 +{"step": 3820, "loss": 4.035120010375977, "lr": 0.0002325463807375371, "grad_norm": 0.5667306184768677, "tokens": 62603264, "tok_per_s": 11574.20379848079, "t": 5409.5}
203 +{"step": 3840, "loss": 4.002408027648926, "lr": 0.00022977674279208034, "grad_norm": 0.5492165088653564, "tokens": 62930944, "tok_per_s": 11576.377089696263, "t": 5436.79}
204 +{"step": 3860, "loss": 4.010754108428955, "lr": 0.00022701928279533916, "grad_norm": 0.5650079846382141, "tokens": 63258624, "tok_per_s": 11578.523310325681, "t": 5464.09}
205 +{"step": 3880, "loss": 4.072173118591309, "lr": 0.00022427433580134536, "grad_norm": 0.598127543926239, "tokens": 63586304, "tok_per_s": 11580.383044802815, "t": 5491.51}
206 +{"step": 3900, "loss": 3.9902126789093018, "lr": 0.0002215422353436983, "grad_norm": 0.5532431602478027, "tokens": 63913984, "tok_per_s": 11577.041542373525, "t": 5521.4}
207 +{"step": 3920, "loss": 3.8755486011505127, "lr": 0.00021882331339503767, "grad_norm": 0.5454591512680054, "tokens": 64241664, "tok_per_s": 11577.402404052884, "t": 5549.53}
208 +{"step": 3940, "loss": 3.9723997116088867, "lr": 0.00021611790032670633, "grad_norm": 0.5648732781410217, "tokens": 64569344, "tok_per_s": 11577.031145315133, "t": 5578.01}
209 +{"step": 3960, "loss": 4.01093053817749, "lr": 0.0002134263248686069, "grad_norm": 0.5775900483131409, "tokens": 64897024, "tok_per_s": 11578.616200441384, "t": 5605.55}
210 +{"step": 3980, "loss": 3.90216064453125, "lr": 0.00021074891406925903, "grad_norm": 0.5466880202293396, "tokens": 65224704, "tok_per_s": 11577.200550459853, "t": 5634.54}
211 +{"step": 3999, "val_loss": 4.0112474918365475, "val_ppl": 55.21570876292144, "tokens": 65536000, "t": 5668.99}
212 +{"step": 3999, "checkpoint_sha256": "1defac6e92e64f97536e0c6d03a5ad0f431d8bc6cbc78392fcd088a15efaf557", "t": 5669.37}
213 +{"step": 4000, "loss": 3.962883234024048, "lr": 0.00020808599325605992, "grad_norm": 0.5598580241203308, "tokens": 65552384, "tok_per_s": 11561.020262288332, "t": 5670.76}
214 +{"step": 4020, "loss": 3.9114115238189697, "lr": 0.0002054378859957543, "grad_norm": 0.5523264408111572, "tokens": 65880064, "tok_per_s": 11559.453836876024, "t": 5699.88}
215 +{"step": 4040, "loss": 3.91835355758667, "lr": 0.00020280491405511884, "grad_norm": 0.5684453845024109, "tokens": 66207744, "tok_per_s": 11561.263558356273, "t": 5727.33}
216 +{"step": 4060, "loss": 4.060446739196777, "lr": 0.00020018739736186444, "grad_norm": 0.5896223783493042, "tokens": 66535424, "tok_per_s": 11560.698828062677, "t": 5755.95}
217 +{"step": 4080, "loss": 4.083343029022217, "lr": 0.00019758565396576288, "grad_norm": 0.568533182144165, "tokens": 66863104, "tok_per_s": 11562.029005289642, "t": 5783.63}
218 +{"step": 4100, "loss": 3.9439985752105713, "lr": 0.00019500000000000005, "grad_norm": 0.5773836374282837, "tokens": 67190784, "tok_per_s": 11563.784358023158, "t": 5811.09}
219 +{"step": 4120, "loss": 3.813251256942749, "lr": 0.0001924307496427646, "grad_norm": 0.5718711614608765, "tokens": 67518464, "tok_per_s": 11560.73883504852, "t": 5840.97}
220 +{"step": 4140, "loss": 3.793492317199707, "lr": 0.000189878215079072, "grad_norm": 0.5597081184387207, "tokens": 67846144, "tok_per_s": 11560.29938081825, "t": 5869.53}
221 +{"step": 4160, "loss": 3.9493250846862793, "lr": 0.00018734270646283112, "grad_norm": 0.5616011023521423, "tokens": 68173824, "tok_per_s": 11560.322394282883, "t": 5897.87}
222 +{"step": 4180, "loss": 3.8800625801086426, "lr": 0.0001848245318791589, "grad_norm": 0.550301194190979, "tokens": 68501504, "tok_per_s": 11561.836976458999, "t": 5925.44}
223 +{"step": 4200, "loss": 3.836061954498291, "lr": 0.00018232399730694481, "grad_norm": 0.5758439898490906, "tokens": 68829184, "tok_per_s": 11561.382130904814, "t": 5954.01}
224 +{"step": 4220, "loss": 3.8811402320861816, "lr": 0.00017984140658167184, "grad_norm": 0.5658969879150391, "tokens": 69156864, "tok_per_s": 11560.042820097335, "t": 5983.05}
225 +{"step": 4240, "loss": 3.8178927898406982, "lr": 0.00017737706135849857, "grad_norm": 0.5584120750427246, "tokens": 69484544, "tok_per_s": 11558.705750259509, "t": 6012.09}
226 +{"step": 4260, "loss": 3.942742347717285, "lr": 0.00017493126107560525, "grad_norm": 0.5712693929672241, "tokens": 69812224, "tok_per_s": 11561.05464668217, "t": 6039.21}
227 +{"step": 4280, "loss": 3.9611594676971436, "lr": 0.00017250430291780933, "grad_norm": 0.5624386668205261, "tokens": 70139904, "tok_per_s": 11559.63639961476, "t": 6068.3}
228 +{"step": 4300, "loss": 4.0740766525268555, "lr": 0.0001700964817804554, "grad_norm": 0.5950373411178589, "tokens": 70467584, "tok_per_s": 11561.908158682603, "t": 6095.45}
229 +{"step": 4320, "loss": 3.9431636333465576, "lr": 0.00016770809023358322, "grad_norm": 0.5978004932403564, "tokens": 70795264, "tok_per_s": 11564.137920502184, "t": 6122.61}
230 +{"step": 4340, "loss": 3.7616939544677734, "lr": 0.00016533941848637757, "grad_norm": 0.5960965752601624, "tokens": 71122944, "tok_per_s": 11564.849615749808, "t": 6150.57}
231 +{"step": 4360, "loss": 3.8475029468536377, "lr": 0.00016299075435190547, "grad_norm": 0.5656423568725586, "tokens": 71450624, "tok_per_s": 11564.091210004042, "t": 6179.31}
232 +{"step": 4380, "loss": 3.911325454711914, "lr": 0.000160662383212145, "grad_norm": 0.5660343170166016, "tokens": 71778304, "tok_per_s": 11561.790236658328, "t": 6208.88}
233 +{"step": 4400, "loss": 3.8917365074157715, "lr": 0.00015835458798330867, "grad_norm": 0.5705775022506714, "tokens": 72105984, "tok_per_s": 11563.96163458225, "t": 6236.05}
234 +{"step": 4420, "loss": 3.8357012271881104, "lr": 0.00015606764908146668, "grad_norm": 0.6175817847251892, "tokens": 72433664, "tok_per_s": 11563.629199508314, "t": 6264.56}
235 +{"step": 4440, "loss": 3.9402458667755127, "lr": 0.00015380184438847444, "grad_norm": 0.5682572722434998, "tokens": 72761344, "tok_per_s": 11565.777961491118, "t": 6291.73}
236 +{"step": 4460, "loss": 3.9123647212982178, "lr": 0.0001515574492182075, "grad_norm": 0.5742660760879517, "tokens": 73089024, "tok_per_s": 11564.787390080484, "t": 6320.61}
237 +{"step": 4480, "loss": 3.8569695949554443, "lr": 0.00014933473628310837, "grad_norm": 0.5776249766349792, "tokens": 73416704, "tok_per_s": 11563.360530311495, "t": 6349.72}
238 +{"step": 4499, "val_loss": 3.957085204124451, "val_ppl": 52.304646175065805, "tokens": 73728000, "t": 6384.15}
239 +{"step": 4500, "loss": 3.900219440460205, "lr": 0.00014713397566104992, "grad_norm": 0.5595879554748535, "tokens": 73744384, "tok_per_s": 11549.85744394998, "t": 6385.52}
240 +{"step": 4520, "loss": 3.9554238319396973, "lr": 0.0001449554347625191, "grad_norm": 0.5791251063346863, "tokens": 74072064, "tok_per_s": 11551.858150088292, "t": 6412.78}
241 +{"step": 4540, "loss": 3.8150556087493896, "lr": 0.00014279937829812366, "grad_norm": 0.5590927600860596, "tokens": 74399744, "tok_per_s": 11550.828260871835, "t": 6441.72}
242 +{"step": 4560, "loss": 3.792018413543701, "lr": 0.00014066606824642808, "grad_norm": 0.5811460018157959, "tokens": 74727424, "tok_per_s": 11552.671204872138, "t": 6469.05}
243 +{"step": 4580, "loss": 3.759868621826172, "lr": 0.0001385557638221209, "grad_norm": 0.572047770023346, "tokens": 75055104, "tok_per_s": 11554.365421175793, "t": 6496.46}
244 +{"step": 4600, "loss": 3.800534248352051, "lr": 0.0001364687214445182, "grad_norm": 0.5741000175476074, "tokens": 75382784, "tok_per_s": 11554.324567762942, "t": 6524.85}
245 +{"step": 4620, "loss": 3.8696978092193604, "lr": 0.00013440519470640598, "grad_norm": 0.6237900853157043, "tokens": 75710464, "tok_per_s": 11554.072040147503, "t": 6553.35}
246 +{"step": 4640, "loss": 3.943920612335205, "lr": 0.00013236543434322733, "grad_norm": 0.5705210566520691, "tokens": 76038144, "tok_per_s": 11552.62606413038, "t": 6582.54}
247 +{"step": 4660, "loss": 3.7643725872039795, "lr": 0.00013034968820261554, "grad_norm": 0.5851072669029236, "tokens": 76365824, "tok_per_s": 11554.693671882093, "t": 6609.72}
248 +{"step": 4680, "loss": 3.8417506217956543, "lr": 0.00012835820121427852, "grad_norm": 0.5822275876998901, "tokens": 76693504, "tok_per_s": 11554.239013189506, "t": 6638.34}
249 +{"step": 4700, "loss": 3.8552558422088623, "lr": 0.00012639121536023803, "grad_norm": 0.5749747157096863, "tokens": 77021184, "tok_per_s": 11555.92077454996, "t": 6665.73}
250 +{"step": 4720, "loss": 3.877544403076172, "lr": 0.00012444896964542695, "grad_norm": 0.5795513987541199, "tokens": 77348864, "tok_per_s": 11555.125192265208, "t": 6694.54}
251 +{"step": 4740, "loss": 3.855560302734375, "lr": 0.00012253170006864812, "grad_norm": 0.5813860297203064, "tokens": 77676544, "tok_per_s": 11553.848127747267, "t": 6723.64}
252 +{"step": 4760, "loss": 3.9683942794799805, "lr": 0.00012063963959389822, "grad_norm": 0.5823618173599243, "tokens": 78004224, "tok_per_s": 11552.457505885966, "t": 6752.82}
253 +{"step": 4780, "loss": 3.827885150909424, "lr": 0.00011877301812206132, "grad_norm": 0.5629019141197205, "tokens": 78331904, "tok_per_s": 11554.108498220257, "t": 6780.21}
254 +{"step": 4800, "loss": 3.8732705116271973, "lr": 0.00011693206246297376, "grad_norm": 0.6126952171325684, "tokens": 78659584, "tok_per_s": 11553.165489637673, "t": 6809.13}
255 +{"step": 4820, "loss": 3.753878355026245, "lr": 0.00011511699630786462, "grad_norm": 0.6059414744377136, "tokens": 78987264, "tok_per_s": 11554.472144458165, "t": 6836.72}
256 +{"step": 4840, "loss": 4.004238605499268, "lr": 0.00011332804020217591, "grad_norm": 0.5791417360305786, "tokens": 79314944, "tok_per_s": 11555.287329584366, "t": 6864.6}
257 +{"step": 4860, "loss": 3.7956504821777344, "lr": 0.00011156541151876422, "grad_norm": 0.594039261341095, "tokens": 79642624, "tok_per_s": 11554.656880093267, "t": 6893.33}
258 +{"step": 4880, "loss": 3.903764486312866, "lr": 0.00010982932443148792, "grad_norm": 0.5707221031188965, "tokens": 79970304, "tok_per_s": 11551.781497839473, "t": 6923.42}
259 +{"step": 4900, "loss": 3.8113834857940674, "lr": 0.00010811998988918382, "grad_norm": 0.6023657917976379, "tokens": 80297984, "tok_per_s": 11553.437096988495, "t": 6950.78}
260 +{"step": 4920, "loss": 3.8437676429748535, "lr": 0.00010643761559003459, "grad_norm": 0.5618969798088074, "tokens": 80625664, "tok_per_s": 11552.96747676362, "t": 6979.43}
261 +{"step": 4940, "loss": 3.911839008331299, "lr": 0.00010478240595633218, "grad_norm": 0.5965155959129333, "tokens": 80953344, "tok_per_s": 11554.25529240919, "t": 7007.01}
262 +{"step": 4960, "loss": 3.9762771129608154, "lr": 0.00010315456210963839, "grad_norm": 0.5922918915748596, "tokens": 81281024, "tok_per_s": 11554.737882554518, "t": 7035.08}
263 +{"step": 4980, "loss": 3.800461530685425, "lr": 0.0001015542818463474, "grad_norm": 0.5719321966171265, "tokens": 81608704, "tok_per_s": 11554.03089926349, "t": 7063.87}
264 +{"step": 4999, "val_loss": 3.9170217752456664, "val_ppl": 50.250564225568326, "tokens": 81920000, "t": 7098.86}
265 +{"step": 4999, "checkpoint_sha256": "f09f039d5f72ac5fb5bdab0899b9f829650b3c0101cf509c62d1f4aaf0e6bb5f", "t": 7099.15}
266 +{"step": 5000, "loss": 3.9287047386169434, "lr": 9.998175961365173e-05, "grad_norm": 0.5943207144737244, "tokens": 81936384, "tok_per_s": 11540.023799157347, "t": 7100.83}
267 +{"step": 5020, "loss": 3.853404998779297, "lr": 9.843718648591489e-05, "grad_norm": 0.5844020247459412, "tokens": 82264064, "tok_per_s": 11541.101026861128, "t": 7128.56}
268 +{"step": 5040, "loss": 3.861715793609619, "lr": 9.692075014145502e-05, "grad_norm": 0.586184024810791, "tokens": 82591744, "tok_per_s": 11539.954421434737, "t": 7157.67}
269 +{"step": 5060, "loss": 3.7985501289367676, "lr": 9.543263483973994e-05, "grad_norm": 0.5837913155555725, "tokens": 82919424, "tok_per_s": 11541.87261737775, "t": 7184.87}
270 +{"step": 5080, "loss": 3.9008865356445312, "lr": 9.39730213989981e-05, "grad_norm": 0.5807478427886963, "tokens": 83247104, "tok_per_s": 11543.789493249493, "t": 7212.06}
271 +{"step": 5100, "loss": 3.8417551517486572, "lr": 9.254208717424797e-05, "grad_norm": 0.5791102647781372, "tokens": 83574784, "tok_per_s": 11545.686514623943, "t": 7239.26}
272 +{"step": 5120, "loss": 3.8045313358306885, "lr": 9.114000603574761e-05, "grad_norm": 0.5828171372413635, "tokens": 83902464, "tok_per_s": 11543.956937651712, "t": 7268.73}
273 +{"step": 5140, "loss": 3.9080166816711426, "lr": 8.976694834786837e-05, "grad_norm": 0.5843819379806519, "tokens": 84230144, "tok_per_s": 11543.657775211985, "t": 7297.3}
274 +{"step": 5160, "loss": 3.749263286590576, "lr": 8.842308094839385e-05, "grad_norm": 0.5649449229240417, "tokens": 84557824, "tok_per_s": 11544.058148752227, "t": 7325.43}
275 +{"step": 5180, "loss": 3.858259677886963, "lr": 8.710856712824807e-05, "grad_norm": 0.5771579742431641, "tokens": 84885504, "tok_per_s": 11544.122094995391, "t": 7353.78}
276 +{"step": 5200, "loss": 3.790924549102783, "lr": 8.582356661165409e-05, "grad_norm": 0.5768647789955139, "tokens": 85213184, "tok_per_s": 11545.853513065293, "t": 7381.06}
277 +{"step": 5220, "loss": 3.7866573333740234, "lr": 8.456823553672612e-05, "grad_norm": 0.56633460521698, "tokens": 85540864, "tok_per_s": 11545.18819581383, "t": 7409.86}
278 +{"step": 5240, "loss": 3.8559517860412598, "lr": 8.334272643649774e-05, "grad_norm": 0.6036170721054077, "tokens": 85868544, "tok_per_s": 11543.96703405992, "t": 7439.03}
279 +{"step": 5260, "loss": 3.857818603515625, "lr": 8.214718822038769e-05, "grad_norm": 0.590832531452179, "tokens": 86196224, "tok_per_s": 11543.041741204966, "t": 7468.02}
280 +{"step": 5280, "loss": 3.8089985847473145, "lr": 8.098176615610612e-05, "grad_norm": 0.5837736129760742, "tokens": 86523904, "tok_per_s": 11544.735299253369, "t": 7495.31}
281 +{"step": 5300, "loss": 3.729093313217163, "lr": 7.984660185200352e-05, "grad_norm": 0.5829435586929321, "tokens": 86851584, "tok_per_s": 11543.877992195296, "t": 7524.25}
282 +{"step": 5320, "loss": 3.79866886138916, "lr": 7.874183323986404e-05, "grad_norm": 0.5758870244026184, "tokens": 87179264, "tok_per_s": 11545.581373394647, "t": 7551.52}
283 +{"step": 5340, "loss": 3.85446834564209, "lr": 7.76675945581456e-05, "grad_norm": 0.5752629637718201, "tokens": 87506944, "tok_per_s": 11547.36954656337, "t": 7578.73}
284 +{"step": 5360, "loss": 3.789114475250244, "lr": 7.662401633566895e-05, "grad_norm": 0.5718693733215332, "tokens": 87834624, "tok_per_s": 11546.979012343047, "t": 7607.38}
285 +{"step": 5380, "loss": 3.6952404975891113, "lr": 7.561122537575734e-05, "grad_norm": 0.5910956859588623, "tokens": 88162304, "tok_per_s": 11546.87632803104, "t": 7635.81}
286 +{"step": 5400, "loss": 3.8815152645111084, "lr": 7.462934474082865e-05, "grad_norm": 0.5759338140487671, "tokens": 88489984, "tok_per_s": 11545.790709781168, "t": 7664.91}
287 +{"step": 5420, "loss": 4.046452045440674, "lr": 7.36784937374426e-05, "grad_norm": 0.6042420864105225, "tokens": 88817664, "tok_per_s": 11545.50851820901, "t": 7693.48}
288 +{"step": 5440, "loss": 3.7572991847991943, "lr": 7.27587879018039e-05, "grad_norm": 0.5804364681243896, "tokens": 89145344, "tok_per_s": 11546.780616024904, "t": 7721.01}
289 +{"step": 5460, "loss": 3.8507866859436035, "lr": 7.187033898572343e-05, "grad_norm": 0.5794432759284973, "tokens": 89473024, "tok_per_s": 11547.378430238676, "t": 7748.98}
290 +{"step": 5480, "loss": 3.731966733932495, "lr": 7.101325494303994e-05, "grad_norm": 0.5920389890670776, "tokens": 89800704, "tok_per_s": 11546.92697569402, "t": 7777.66}
291 +{"step": 5499, "val_loss": 3.887117636203766, "val_ppl": 48.770110536442886, "tokens": 90112000, "t": 7813.41}
292 +{"step": 5500, "loss": 3.9196672439575195, "lr": 7.018763991650238e-05, "grad_norm": 0.6001965403556824, "tokens": 90128384, "tok_per_s": 11534.021598855534, "t": 7814.78}
293 +{"step": 5520, "loss": 3.8390917778015137, "lr": 6.939359422511588e-05, "grad_norm": 0.5888873934745789, "tokens": 90456064, "tok_per_s": 11535.491951955515, "t": 7842.19}
294 +{"step": 5540, "loss": 3.8811357021331787, "lr": 6.863121435195203e-05, "grad_norm": 0.5845345258712769, "tokens": 90783744, "tok_per_s": 11534.633071129776, "t": 7871.18}
295 +{"step": 5560, "loss": 3.7738800048828125, "lr": 6.790059293242561e-05, "grad_norm": 0.5928100943565369, "tokens": 91111424, "tok_per_s": 11535.80731025614, "t": 7898.78}
296 +{"step": 5580, "loss": 3.835395574569702, "lr": 6.720181874303845e-05, "grad_norm": 0.5779862403869629, "tokens": 91439104, "tok_per_s": 11537.383975466677, "t": 7926.1}
297 +{"step": 5600, "loss": 3.815725803375244, "lr": 6.653497669059239e-05, "grad_norm": 0.579407811164856, "tokens": 91766784, "tok_per_s": 11535.913175032281, "t": 7955.52}
298 +{"step": 5620, "loss": 3.761955499649048, "lr": 6.590014780187247e-05, "grad_norm": 0.5867651700973511, "tokens": 92094464, "tok_per_s": 11535.090516233922, "t": 7984.5}
299 +{"step": 5640, "loss": 3.8077659606933594, "lr": 6.529740921380147e-05, "grad_norm": 0.5797607898712158, "tokens": 92422144, "tok_per_s": 11535.485589128413, "t": 8012.63}
300 +{"step": 5660, "loss": 3.789388656616211, "lr": 6.47268341640671e-05, "grad_norm": 0.5859403610229492, "tokens": 92749824, "tok_per_s": 11535.285421023631, "t": 8041.17}
301 +{"step": 5680, "loss": 3.8692054748535156, "lr": 6.418849198222306e-05, "grad_norm": 0.5814026594161987, "tokens": 93077504, "tok_per_s": 11536.586077977248, "t": 8068.67}
302 +{"step": 5700, "loss": 3.8076140880584717, "lr": 6.368244808126497e-05, "grad_norm": 0.5866917371749878, "tokens": 93405184, "tok_per_s": 11536.014408981831, "t": 8097.48}
303 +{"step": 5720, "loss": 3.7942938804626465, "lr": 6.320876394968206e-05, "grad_norm": 0.572772741317749, "tokens": 93732864, "tok_per_s": 11535.139908600137, "t": 8126.5}
304 +{"step": 5740, "loss": 3.8309621810913086, "lr": 6.276749714398588e-05, "grad_norm": 0.5950015783309937, "tokens": 94060544, "tok_per_s": 11534.120242321984, "t": 8155.62}
305 +{"step": 5760, "loss": 3.825157642364502, "lr": 6.23587012817167e-05, "grad_norm": 0.609603762626648, "tokens": 94388224, "tok_per_s": 11535.867126376139, "t": 8182.79}
306 +{"step": 5780, "loss": 3.729236125946045, "lr": 6.198242603492857e-05, "grad_norm": 0.5921071171760559, "tokens": 94715904, "tok_per_s": 11534.939763906073, "t": 8211.86}
307 +{"step": 5800, "loss": 3.8878514766693115, "lr": 6.163871712415363e-05, "grad_norm": 0.6161739826202393, "tokens": 95043584, "tok_per_s": 11536.696837804526, "t": 8239.01}
308 +{"step": 5820, "loss": 3.8525753021240234, "lr": 6.132761631284691e-05, "grad_norm": 0.5878046751022339, "tokens": 95371264, "tok_per_s": 11538.254949744933, "t": 8266.3}
309 +{"step": 5840, "loss": 3.8129560947418213, "lr": 6.10491614023115e-05, "grad_norm": 0.5896515846252441, "tokens": 95698944, "tok_per_s": 11538.439787215593, "t": 8294.57}
310 +{"step": 5860, "loss": 3.7627148628234863, "lr": 6.08033862271055e-05, "grad_norm": 0.6002752780914307, "tokens": 96026624, "tok_per_s": 11537.950833802404, "t": 8323.32}
311 +{"step": 5880, "loss": 3.917717933654785, "lr": 6.059032065093087e-05, "grad_norm": 0.5864716172218323, "tokens": 96354304, "tok_per_s": 11536.854347275696, "t": 8352.51}
312 +{"step": 5900, "loss": 3.671173572540283, "lr": 6.040999056300463e-05, "grad_norm": 0.596362292766571, "tokens": 96681984, "tok_per_s": 11537.971918323201, "t": 8380.1}
313 +{"step": 5920, "loss": 3.816864013671875, "lr": 6.026241787491321e-05, "grad_norm": 0.5997350811958313, "tokens": 97009664, "tok_per_s": 11537.623532511629, "t": 8408.76}
314 +{"step": 5940, "loss": 3.8800463676452637, "lr": 6.014762051794997e-05, "grad_norm": 0.5878621339797974, "tokens": 97337344, "tok_per_s": 11539.057999128861, "t": 8436.11}
315 +{"step": 5960, "loss": 3.8143668174743652, "lr": 6.0065612440936474e-05, "grad_norm": 0.599541425704956, "tokens": 97665024, "tok_per_s": 11540.68232762661, "t": 8463.32}
316 +{"step": 5980, "loss": 3.7237539291381836, "lr": 6.001640360852739e-05, "grad_norm": 0.5826242566108704, "tokens": 97992704, "tok_per_s": 11539.89272352153, "t": 8492.29}
317 +{"step": 5999, "loss": 3.8935680389404297, "lr": 6.000004100943553e-05, "grad_norm": 0.5838854908943176, "tokens": 98304000, "tok_per_s": 11538.776028145761, "t": 8520.09}
318 +{"step": 5999, "val_loss": 3.8689021825790406, "val_ppl": 47.88978297376415, "tokens": 98304000, "t": 8526.68}
319 +{"step": 5999, "checkpoint_sha256": "1552f1ec984ec895a40814cd466bde59abaf35e4bfe1ea35b9d43ff4cfe9b9d0", "t": 8527.03}
modified paper/appendix/prereg.tex +16 −2
@@ -1,6 +1,20 @@
1 −\subsection*{\texorpdfstring{Pre-registration --- WP31 PRISME pilot (DRAFT v0.1, 2026-10-29; to be FROZEN before any \texttt{test\_*} split is read)}{Pre-registration --- WP31 PRISME pilot (DRAFT v0.1, 2026-10-29; to be FROZEN before any test\_* split is read)}}\label{pre-registration-wp31-prisme-pilot-draft-v0.1-2026-10-29-to-be-frozen-before-any-test_-split-is-read}
1 +\subsection*{\texorpdfstring{Pre-registration --- WP31 PRISME pilot (v1.0, FROZEN 2026-10-30T04:04-0400 before any \texttt{test\_*} split was read)}{Pre-registration --- WP31 PRISME pilot (v1.0, FROZEN 2026-10-30T04:04-0400 before any test\_* split was read)}}\label{pre-registration-wp31-prisme-pilot-v1.0-frozen-2026-10-30t0404-0400-before-any-test_-split-was-read}
2 2
3 −Status: DRAFT. No file named \texttt{test*.jsonl} / \texttt{test.bin} has been opened by any evaluation script so far (\texttt{protocol/locked\_test\_manifest.json} records the split hashes and the unsealing timestamp once frozen).
3 +Status: FROZEN. No file named \texttt{test*.jsonl} / \texttt{test.bin} had been opened by any evaluation script at freeze time (\texttt{protocol/locked\_test\_manifest.json} records the split hashes and the unsealing timestamp).
4 +
5 +\subsubsection*{0. Primary comparisons as executed (fixed at freeze, before unsealing)}\label{primary-comparisons-as-executed-fixed-at-freeze-before-unsealing}
6 +
7 +\begin{itemize}
8 +\tightlist
9 +\item
10 + P1 (H1/H2, synthetic): exact match on the pooled test regimes at forced budget 2 for the sparse editor vs the frozen backbone, vs the global-recurrence adapter and vs the dense editor, 5 adapter seeds each (v2 backbone), paired by item.
11 +\item
12 + P2 (H3, synthetic): dev-iid loss of the locked controller vs the best fixed schedule and vs budget 0 (already reported on dev; the test regimes are evaluated at budget 0 and 2 with forced random actions only, since the controller was worse than budget 0 on dev for every seed --- the controller is therefore evaluated on test for completeness, not as the primary system).
13 +\item
14 + P3 (H1/H4, language): FineWeb-Edu locked test cross-entropy at budgets 0/1/2/4/8 for the 103M backbone, the sparse editor, the global adapter and the parameter-matched plain decoder.
15 +\item
16 + Decision thresholds as in Section 4 below; no other analysis is pre-registered.
17 +\end{itemize}
4 18
5 19 \subsubsection*{1. Primary benchmark suite}\label{primary-benchmark-suite}
6 20
modified paper/main.log +66 −65
@@ -1,4 +1,4 @@
1 −This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026) (preloaded format=pdflatex 2026.3.22) 30 OCT 2026 04:04
1 +This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026) (preloaded format=pdflatex 2026.3.22) 30 OCT 2026 04:15
2 2 entering extended mode
3 3 restricted \write18 enabled.
4 4 %&-line parsing enabled.
@@ -1159,7 +1159,7 @@ Package hyperref Info: Link coloring ON on input line 85.
1159 1159
1160 1160 \c@lstlisting=\count403
1161 1161 (./sections/titlepage.tex
1162 −<uq_logo.jpg, id=199, 722.7pt x 361.35pt>
1162 +<uq_logo.jpg, id=203, 722.7pt x 361.35pt>
1163 1163 File: uq_logo.jpg Graphic file (type jpg)
1164 1164 <use uq_logo.jpg>
1165 1165 Package pdftex.def Info: uq_logo.jpg used on input line 9.
@@ -1168,10 +1168,9 @@ Package pdftex.def Info: uq_logo.jpg used on input line 9.
1168 1168
1169 1169 {/usr/local/texlive/2026/texmf-var/fonts/map/pdftex/updmap/pdftex.map}{/usr/loc
1170 1170 al/texlive/2026/texmf-dist/fonts/enc/dvips/base/8r.enc} <./uq_logo.jpg>] (./sec
1171 −tions/abstract.tex)
1172 −[2]) (./sections/introduction.tex
1171 +tions/abstract.tex
1173 1172 LaTeX Font Info: Trying to load font information for OT1+ztmcm on input line
1174 − 8.
1173 + 18.
1175 1174
1176 1175 (/usr/local/texlive/2026/texmf-dist/tex/latex/psnfss/ot1ztmcm.fd
1177 1176 File: ot1ztmcm.fd 2000/01/03 Fontinst v1.801 font definitions for OT1/ztmcm.
@@ -1181,50 +1180,51 @@ Package microtype Info: Loading generic protrusion settings for font family
1181 1180 (microtype) For optimal results, create family-specific settings.
1182 1181 (microtype) See the microtype manual for details.
1183 1182 LaTeX Font Info: Trying to load font information for OML+ztmcm on input line
1184 − 8.
1183 + 18.
1185 1184
1186 1185 (/usr/local/texlive/2026/texmf-dist/tex/latex/psnfss/omlztmcm.fd
1187 1186 File: omlztmcm.fd 2000/01/03 Fontinst v1.801 font definitions for OML/ztmcm.
1188 1187 )
1189 1188 LaTeX Font Info: Trying to load font information for OMS+ztmcm on input line
1190 − 8.
1189 + 18.
1191 1190
1192 1191 (/usr/local/texlive/2026/texmf-dist/tex/latex/psnfss/omsztmcm.fd
1193 1192 File: omsztmcm.fd 2000/01/03 Fontinst v1.801 font definitions for OMS/ztmcm.
1194 1193 )
1195 1194 LaTeX Font Info: Trying to load font information for OMX+ztmcm on input line
1196 − 8.
1195 + 18.
1197 1196
1198 1197 (/usr/local/texlive/2026/texmf-dist/tex/latex/psnfss/omxztmcm.fd
1199 1198 File: omxztmcm.fd 2000/01/03 Fontinst v1.801 font definitions for OMX/ztmcm.
1200 1199 )
1201 −LaTeX Font Info: Trying to load font information for OT1+ptm on input line 8
1202 −.
1200 +LaTeX Font Info: Trying to load font information for OT1+ptm on input line 1
1201 +8.
1203 1202
1204 1203 (/usr/local/texlive/2026/texmf-dist/tex/latex/psnfss/ot1ptm.fd
1205 1204 File: ot1ptm.fd 2001/06/04 font definitions for OT1/ptm.
1206 1205 )
1207 1206 LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <12> not available
1208 −(Font) Font shape `OT1/ptm/b/n' tried instead on input line 8.
1207 +(Font) Font shape `OT1/ptm/b/n' tried instead on input line 18.
1209 1208 LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <9> not available
1210 −(Font) Font shape `OT1/ptm/b/n' tried instead on input line 8.
1209 +(Font) Font shape `OT1/ptm/b/n' tried instead on input line 18.
1211 1210 LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <7> not available
1212 −(Font) Font shape `OT1/ptm/b/n' tried instead on input line 8.
1211 +(Font) Font shape `OT1/ptm/b/n' tried instead on input line 18.
1213 1212
1214 1213 (/usr/local/texlive/2026/texmf-dist/tex/latex/microtype/mt-msa.cfg
1215 1214 File: mt-msa.cfg 2006/02/04 v1.1 microtype config. file: AMS symbols (a) (RS)
1216 1215 )
1217 1216 (/usr/local/texlive/2026/texmf-dist/tex/latex/microtype/mt-msb.cfg
1218 1217 File: mt-msb.cfg 2005/06/01 v1.0 microtype config. file: AMS symbols (b) (RS)
1219 −) [3]
1220 −(./sections/findings.tex)) (./sections/related.tex
1218 +)) [2]
1219 +[3]) (./sections/introduction.tex [4] (./sections/findings.tex))
1220 +(./sections/related.tex [5]
1221 1221 Package microtype Info: Loading generic protrusion settings for font family
1222 1222 (microtype) `cmtt' (encoding: T1).
1223 1223 (microtype) For optimal results, create family-specific settings.
1224 1224 (microtype) See the microtype manual for details.
1225 − [4{/usr/local/texlive/2026/texmf-dist/fonts/enc/dvips/cm-super/cm-super-t1.enc
1226 −}]) (./sections/problem.tex
1227 −[5]) (./sections/architecture.tex (./figures/architecture.tex
1225 + [6{/usr/local/texlive/2026/texmf-dist/fonts/enc/dvips/cm-super/cm-super-t1.enc
1226 +}]) (./sections/problem.tex [7])
1227 +(./sections/architecture.tex (./figures/architecture.tex
1228 1228 LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <10> not available
1229 1229 (Font) Font shape `OT1/ptm/b/n' tried instead on input line 9.
1230 1230 LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <7.4> not available
@@ -1240,7 +1240,7 @@ LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <10.95> not available
1240 1240 (Font) Font shape `OT1/ptm/b/n' tried instead on input line 13.
1241 1241 LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <8> not available
1242 1242 (Font) Font shape `OT1/ptm/b/n' tried instead on input line 13.
1243 −[6]
1243 +
1244 1244 Overfull \hbox (0.70027pt too wide) detected at line 28
1245 1245 [][][][] \OT1/ztmcm/m/n/12 = [][](\OML/ztmcm/m/it/12 H[]\OT1/ztmcm/m/n/12 )\OML
1246 1246 /ztmcm/m/it/12 ; K[] \OT1/ztmcm/m/n/12 = []\OML/ztmcm/m/it/12 W[]; V[] \OT1/z
@@ -1249,12 +1249,12 @@ t/12 P []; W[]; W[]; P[] \OMS/ztmcm/m/n/12 2 \U/msb/m/n/12 R[]\OML/ztmcm/m/it/
1249 1249 12 :
1250 1250 []
1251 1251
1252 −[7]
1252 +[8]
1253 1253 Overfull \hbox (8.35876pt too wide) detected at line 60
1254 1254 []
1255 1255 []
1256 1256
1257 −
1257 +[9]
1258 1258 Overfull \hbox (22.86838pt too wide) detected at line 74
1259 1259 [][][]\OML/ztmcm/m/it/12 ^^N[] \OT1/ztmcm/m/n/12 = [] \OMS/ztmcm/m/n/12 ^^@ \OM
1260 1260 L/ztmcm/m/it/12 S[]; ^^N[] \OT1/ztmcm/m/n/12 = \OML/ztmcm/m/it/12 ^^N[] [] []
@@ -1273,60 +1273,62 @@ mcm/m/n/12 (\OML/ztmcm/m/it/12 S; z[]\OT1/ztmcm/m/n/12 ) = [][] \OML/ztmcm/m/it
1273 1273 ][](\OML/ztmcm/m/it/12 W[]z[]\OT1/ztmcm/m/n/12 ) \OMS/ztmcm/m/n/12 ^^A
1274 1274 []
1275 1275
1276 −[8] (./tables/configs.tex)) (./sections/policy.tex [9])
1277 −(./sections/learning.tex [10]) (./sections/theory.tex [11]
1276 +(./tables/configs.tex)) (./sections/policy.tex [10]) (./sections/learning.tex
1277 +[11] [12]) (./sections/theory.tex [13]
1278 1278
1279 1279 LaTeX Font Warning: Font shape `T1/ptm/m/scit' undefined
1280 1280 (Font) using `T1/ptm/m/sc' instead on input line 37.
1281 1281
1282 −[12]) (./sections/implementation.tex (./tables/ops.tex
1282 +) (./sections/implementation.tex [14] (./tables/ops.tex
1283 1283 Underfull \hbox (badness 10000) in paragraph at lines 5--5
1284 1284 []\T1/ptm/b/n/10.95 (+20) Table 2. |\T1/ptm/m/n/10.95 (+20) An-a-lytic op-er-a-
1285 1285 tions per gen-er-ated to-ken, PRISME-459M, con-text $\OML/ztmcm/m/it/10.95 T \O
1286 1286 T1/ztmcm/m/n/10.95 = 4096$
1287 1287 []
1288 1288
1289 −) [13]) (./sections/design.tex
1289 +)) (./sections/design.tex [15]
1290 1290 Package microtype Info: Loading generic protrusion settings for font family
1291 1291 (microtype) `cmss' (encoding: T1).
1292 1292 (microtype) For optimal results, create family-specific settings.
1293 1293 (microtype) See the microtype manual for details.
1294 − (./tables/families.tex) [14]
1294 + (./tables/families.tex)
1295 1295 Overfull \hbox (36.62411pt too wide) in paragraph at lines 19--24
1296 1296 \T1/ptm/m/n/12 (-20) 2.91M val-i-da-tion and 2.88M locked test to-kens (\T1/cmt
1297 1297 t/m/n/12 data/processed/fineweb_edu_300m/manifest.json\T1/ptm/m/n/12 (-20) ).
1298 1298 []
1299 1299
1300 +[16]
1300 1301
1301 −LaTeX Warning: Reference `tab:controls' on page 15 undefined on input line 27.
1302 +LaTeX Warning: Reference `tab:controls' on page 17 undefined on input line 27.
1302 1303
1303 −[15]) (./sections/results.tex
1304 +) (./sections/results.tex
1304 1305 Overfull \hbox (71.70984pt too wide) in paragraph at lines 38--1
1305 1306 \T1/ptm/m/n/12 (-20) The pri-mary out-come, met-rics, de-ci-sion thresh-olds an
1306 1307 d seed plan were writ-ten to \T1/cmtt/m/n/12 protocol/preregistration.md
1307 1308 []
1308 1309
1309 −<../results/figures/learnability_bootstrap.pdf, id=482, 353.75287pt x 191.45125
1310 +[17]
1311 +<../results/figures/learnability_bootstrap.pdf, id=494, 353.75287pt x 191.45125
1310 1312 pt>
1311 1313 File: ../results/figures/learnability_bootstrap.pdf Graphic file (type pdf)
1312 1314 <use ../results/figures/learnability_bootstrap.pdf>
1313 1315 Package pdftex.def Info: ../results/figures/learnability_bootstrap.pdf used on
1314 1316 input line 26.
1315 1317 (pdftex.def) Requested size: 469.75502pt x 254.23257pt.
1316 −(./tables/seed_matrix.tex
1318 + (./tables/seed_matrix.tex
1317 1319 Overfull \hbox (133.05286pt too wide) in paragraph at lines 7--21
1318 1320 [][]
1319 1321 []
1320 1322
1321 1323 )
1322 −<../results/figures/frontier_v2.pdf, id=487, 305.58919pt x 208.87035pt>
1324 +<../results/figures/frontier_v2.pdf, id=499, 305.58919pt x 208.87035pt>
1323 1325 File: ../results/figures/frontier_v2.pdf Graphic file (type pdf)
1324 1326 <use ../results/figures/frontier_v2.pdf>
1325 1327 Package pdftex.def Info: ../results/figures/frontier_v2.pdf used on input line
1326 1328 59.
1327 1329 (pdftex.def) Requested size: 338.22418pt x 231.19547pt.
1328 − [16] [17 <../results/figures/learnability_bootstrap.pdf>] [18]
1329 −<../results/figures/labels_analysis.pdf, id=528, 548.77519pt x 192.17598pt>
1330 + [18 <../results/figures/learnability_bootstrap.pdf>] [19]
1331 +<../results/figures/labels_analysis.pdf, id=536, 548.77519pt x 192.17598pt>
1330 1332 File: ../results/figures/labels_analysis.pdf Graphic file (type pdf)
1331 1333 <use ../results/figures/labels_analysis.pdf>
1332 1334 Package pdftex.def Info: ../results/figures/labels_analysis.pdf used on input
@@ -1338,12 +1340,12 @@ Overfull \hbox (109.74734pt too wide) in paragraph at lines 8--15
1338 1340 [][]
1339 1341 []
1340 1342
1341 −) [19 <../results/figures/frontier_v2.pdf>] (./tables/stats.tex)
1342 −(./tables/latency.tex) [20 <../results/figures/labels_analysis.pdf>]
1343 −(./sections/results_pilot.tex (./tables/pilot_ppl.tex)
1344 −(./tables/pilot_routing.tex) [21] (./tables/pilot_latency.tex)))
1345 −(./sections/mechanistic.tex [22]) (./sections/limitations.tex [23])
1346 −(./sections/conclusion.tex) (./sections/reproducibility.tex)
1343 +) [20 <../results/figures/frontier_v2.pdf>] (./tables/stats.tex) [21 <../result
1344 +s/figures/labels_analysis.pdf>] (./tables/test.tex) (./tables/latency.tex)
1345 +(./sections/results_pilot.tex [22] (./tables/pilot_ppl.tex)
1346 +(./tables/pilot_routing.tex) (./tables/pilot_latency.tex)))
1347 +(./sections/mechanistic.tex [23]) (./sections/limitations.tex [24])
1348 +(./sections/conclusion.tex [25]) (./sections/reproducibility.tex)
1347 1349 Overfull \hbox (51.18217pt too wide) in paragraph at lines 2--101
1348 1350 \T1/ptm/m/n/12 (-20) num-ber in the manuscript is pro-duced by \T1/cmtt/m/n/12
1349 1351 scripts/make_tables.py \T1/ptm/m/n/12 (-20) and \T1/cmtt/m/n/12 scripts/reprodu
@@ -1357,18 +1359,18 @@ cript_claims.py \T1/ptm/m/n/12 (-20) checks the chain. \T1/cmtt/m/n/12 REPRODUC
1357 1359 E.md
1358 1360 []
1359 1361
1360 −[24] [25] (./main.bbl [26] [27]
1362 +[26] (./main.bbl [27] [28]
1361 1363 Underfull \hbox (badness 1297) in paragraph at lines 208--211
1362 1364 []\T1/ptm/b/n/12 (+20) You, Yichen, Tianyu Fu, Aosong Feng, Xing-tai Lv, Xue-fe
1363 1365 i Ning, Ning Ding, and
1364 1366 []
1365 1367
1366 −[28]) (./appendix/appendix.tex (./appendix/notation.tex
1368 +[29]) (./appendix/appendix.tex (./appendix/notation.tex
1367 1369 Overfull \hbox (12.19864pt too wide) in paragraph at lines 2--16
1368 1370 []
1369 1371 []
1370 1372
1371 −) (./appendix/proofs.tex [29]) (./appendix/params.tex [30]
1373 +) (./appendix/proofs.tex [30]) (./appendix/params.tex [31]
1372 1374 Overfull \hbox (92.32437pt too wide) detected at line 12
1373 1375 \OML/ztmcm/m/it/12 N[] \OT1/ztmcm/m/n/12 = \OML/ztmcm/m/it/12 d \OT1/ztmcm/m/n/
1374 1376 12 + 3\OML/ztmcm/m/it/12 dw \OT1/ztmcm/m/n/12 + \OML/ztmcm/m/it/12 Mw \OT1/ztmc
@@ -1477,20 +1479,20 @@ LaTeX Warning: `h' float specifier changed to `ht'.
1477 1479 ) (./appendix/prereg.tex
1478 1480 LaTeX Font Info: Font shape `T1/cmtt/bx/n' in size <12> not available
1479 1481 (Font) Font shape `T1/cmtt/m/n' tried instead on input line 1.
1480 − [31]
1482 + [32]
1481 1483 LaTeX Font Info: Trying to load font information for TS1+ptm on input line 8
1482 1484 .
1483 1485
1484 1486 (/usr/local/texlive/2026/texmf-dist/tex/latex/psnfss/ts1ptm.fd
1485 1487 File: ts1ptm.fd 2001/06/04 font definitions for TS1/ptm.
1486 1488 )
1487 −Overfull \hbox (80.09856pt too wide) in paragraph at lines 10--12
1489 +Overfull \hbox (80.09856pt too wide) in paragraph at lines 24--26
1488 1490 []\T1/ptm/b/n/12 (-20) Synthetic com-po-si-tional suite v1 \T1/ptm/m/n/12 (-20)
1489 1491 (\T1/cmtt/m/n/12 data/splits/synthetic_v1\T1/ptm/m/n/12 (-20) , gen-er-a-tor \
1490 1492 T1/cmtt/m/n/12 data/generators/synthetic.py\T1/ptm/m/n/12 (-20) ,
1491 1493 []
1492 1494
1493 −[32]) (./appendix/runs.tex
1495 +[33] [34]) (./appendix/runs.tex
1494 1496 Overfull \hbox (121.91187pt too wide) in paragraph at lines 1--14
1495 1497 \T1/cmtt/m/n/12 training.pretrain\T1/ptm/m/n/12 (-20) , \T1/cmtt/m/n/12 trainin
1496 1498 g.train_editor\T1/ptm/m/n/12 (-20) , \T1/cmtt/m/n/12 training.collect_intervent
@@ -1505,14 +1507,14 @@ tion.latency \T1/ptm/m/n/12 (-20) or \T1/cmtt/m/n/12 evaluation.run\T1/ptm/m/n/
1505 1507 12 (-20) ; launch scripts un-der \T1/cmtt/m/n/12 scripts/
1506 1508 []
1507 1509
1508 −[33] (./appendix/runs_index.tex [34]
1510 +(./appendix/runs_index.tex [35] [36]
1509 1511 Overfull \hbox (25.12868pt too wide) in paragraph at lines 52--54
1510 1512 []\T1/ptm/b/n/10 (-20) run_id\T1/ptm/m/n/10 (-20) : p4_tiny_backbone_v2_seed01_
1511 1513 20261029233624; \T1/ptm/b/n/10 (-20) state\T1/ptm/m/n/10 (-20) : stopped_overfi
1512 1514 tting_step9300_checkpoint_step5000_used;
1513 1515 []
1514 1516
1515 −[35])) (./appendix/results.tex (./tables/em_families.tex))
1517 +)) (./appendix/results.tex (./tables/em_families.tex))
1516 1518 Overfull \hbox (168.4564pt too wide) in paragraph at lines 2--16
1517 1519 \T1/ptm/m/n/12 (-20) records (\T1/cmtt/m/n/12 experiments/runs/*/em_*.json\T1/p
1518 1520 tm/m/n/12 (-20) , \T1/cmtt/m/n/12 results/raw/items_*.json\T1/ptm/m/n/12 (-20)
@@ -1536,7 +1538,7 @@ ileNotFoundError\T1/ptm/m/n/12 (-20) ).; \T1/ptm/b/n/12 (-20) ev-i-dence\T1/ptm
1536 1538 g
1537 1539 []
1538 1540
1539 −
1541 +[37]
1540 1542 Overfull \hbox (49.9369pt too wide) in paragraph at lines 10--12
1541 1543 \T1/ptm/m/n/12 (-20) be-fore ed-its can af-fect log-its.; \T1/ptm/b/n/12 (-20)
1542 1544 ev-i-dence\T1/ptm/m/n/12 (-20) : \T1/cmtt/m/n/12 experiments/runs/p4_editor_v1_
@@ -1550,7 +1552,6 @@ Overfull \hbox (47.44559pt too wide) in paragraph at lines 12--14
1550 1552 ;
1551 1553 []
1552 1554
1553 −[36]
1554 1555 Package microtype Info: Loading generic protrusion settings for font family
1555 1556 (microtype) `cmtt' (encoding: TS1).
1556 1557 (microtype) For optimal results, create family-specific settings.
@@ -1569,7 +1570,8 @@ Overfull \hbox (12.79033pt too wide) in paragraph at lines 18--20
1569 1570 run \T1/cmtt/m/n/12 p4_tiny_backbone_v2_seed01_* \T1/ptm/m/n/12 (-20) (state:
1570 1571 []
1571 1572
1572 −
1573 +[38{/usr/local/texlive/2026/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.en
1574 +c}]
1573 1575 Overfull \hbox (12.01894pt too wide) in paragraph at lines 22--24
1574 1576 \T1/ptm/m/n/12 (-20) 0.4334); $\OML/ztmcm/m/it/12 ^^U$\T1/ptm/m/n/12 (-20) =0 i
1575 1577 s worse (+1.3 %).; \T1/ptm/b/n/12 (-20) ev-i-dence\T1/ptm/m/n/12 (-20) : \T1/cm
@@ -1590,15 +1592,14 @@ Overfull \hbox (107.55685pt too wide) in paragraph at lines 26--28
1590 1592 gs/editor_pilot_103m_seed1.log\T1/ptm/m/n/12 (-20) ,
1591 1593 []
1592 1594
1593 −[37{/usr/local/texlive/2026/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.en
1594 −c}]
1595 +
1595 1596 Overfull \hbox (46.17226pt too wide) in paragraph at lines 28--29
1596 1597 \T1/ptm/m/n/12 (-20) Killed; no 103M dense re-sult.; \T1/ptm/b/n/12 (-20) ev-i-
1597 1598 dence\T1/ptm/m/n/12 (-20) : M2U64 \T1/cmtt/m/n/12 experiments/logs/dense_probe_
1598 1599 103m*_seed1.log\T1/ptm/m/n/12 (-20) ;
1599 1600 []
1600 1601
1601 −)) [38] [39] [40] [41] [42] [43] [44] (./main.aux)
1602 +)) [39] [40] [41] [42] [43] [44] [45] (./main.aux)
1602 1603 ***********
1603 1604 LaTeX2e <2025-11-01>
1604 1605 L3 programming layer <2026-01-19>
@@ -1608,16 +1609,16 @@ L3 programming layer <2026-01-19>
1608 1609 LaTeX Warning: There were undefined references.
1609 1610
1610 1611 Package rerunfilecheck Info: File `main.out' has not changed.
1611 −(rerunfilecheck) Checksum: 6517FE2FB3D993CA26B075D85418F650;9020.
1612 +(rerunfilecheck) Checksum: 5C078824F258CF894B89F741D112AD48;9386.
1612 1613 )
1613 1614 Here is how much of TeX's memory you used:
1614 − 36143 strings out of 467525
1615 − 702671 string characters out of 5418982
1616 − 1149081 words of memory out of 5000000
1617 − 63439 multiletter control sequences out of 15000+600000
1618 − 793579 words of font info for 669 fonts, out of 8000000 for 9000
1615 + 36174 strings out of 467525
1616 + 703202 string characters out of 5418982
1617 + 1152637 words of memory out of 5000000
1618 + 63458 multiletter control sequences out of 15000+600000
1619 + 793836 words of font info for 673 fonts, out of 8000000 for 9000
1619 1620 1141 hyphenation exceptions out of 8191
1620 − 84i,13n,104p,1001b,1501s stack positions out of 10000i,1000n,20000p,200000b,200000s
1621 + 84i,13n,104p,1001b,1484s stack positions out of 10000i,1000n,20000p,200000b,200000s
1621 1622 </usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmex10.pfb
1622 1623 ></usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb>
1623 1624 </usr/local/texlive/2026/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></
@@ -1633,10 +1634,10 @@ exlive/2026/texmf-dist/fonts/type1/urw/symbol/usyr.pfb></usr/local/texlive/2026
1633 1634 e1/urw/times/utmbi8a.pfb></usr/local/texlive/2026/texmf-dist/fonts/type1/urw/ti
1634 1635 mes/utmr8a.pfb></usr/local/texlive/2026/texmf-dist/fonts/type1/urw/times/utmri8
1635 1636 a.pfb>
1636 −Output written on main.pdf (44 pages, 442504 bytes).
1637 +Output written on main.pdf (45 pages, 447044 bytes).
1637 1638 PDF statistics:
1638 − 812 PDF objects out of 1000 (max. 8388607)
1639 − 699 compressed objects within 7 object streams
1640 − 164 named destinations out of 1000 (max. 500000)
1641 − 143777 words of extra memory for PDF output out of 154059 (max. 10000000)
1639 + 824 PDF objects out of 1000 (max. 8388607)
1640 + 709 compressed objects within 8 object streams
1641 + 167 named destinations out of 1000 (max. 500000)
1642 + 143785 words of extra memory for PDF output out of 154059 (max. 10000000)
1642 1643
modified paper/main.pdf +0 −0

Binary file not shown.

modified paper/sections/abstract.tex +6 −3
@@ -15,6 +15,9 @@ not matter for fixed schedules. An oracle that reads the target finds large oppo
15 15 of loss), but \labelscalibwithin{}\,\% of the variance of intervention gains is state-level rather than action-level,
16 16 the opportunity is concentrated in high-uncertainty states where a wrong action costs more than a right one gains,
17 17 and the learned controller ranks actions within a state near chance; it is worse than no routing on held-out data.
18 −Measured on Apple Silicon, per-action dispatch overhead erases the arithmetic savings of sparsity at small scale.
19 −We report the 103M web-text pilot, the complete negative result on learned spatial routing, the mechanistic
20 −explanation, and the conditions under which the hypothesis could still be rescued.
18 +On the locked test regimes the pre-registered primary effect is $\testprismebtwovsbzero$ points against a three-point
19 +threshold. At 103M parameters on 98M tokens of web text, neither the sparse editor nor the global adapter moves the frozen
20 +backbone by more than 0.1\,\% of loss and a parameter-matched plain decoder does better than both. Measured on Apple
21 +Silicon, per-action dispatch overhead erases most of the arithmetic savings of sparsity. We report the complete negative
22 +result on learned spatial routing, its mechanistic explanation, and the conditions under which the hypothesis could
23 +still be rescued.
modified paper/sections/findings.tex +16 −1
@@ -1 +1,16 @@
1 −% findings paragraph is written after the locked evaluation
1 +On a frozen synthetic backbone with five adapter seeds, sparse one-or-two-slot edits lower the held-out
2 +answer-span loss by about 3\,\% and raise exact match by about two points at budget 2---but a parameter-matched
3 +global-recurrence adapter trained with the identical recipe does at least as well at every budget and dominates
4 +the quality-versus-cost frontier, and dense sixteen-slot writes do slightly better than sparse ones; on the locked
5 +test regimes the pre-registered primary effect is $\testprismebtwovsbzero$ points against a three-point threshold.
6 +Which slot is written does not matter for fixed schedules. An exhaustive one-step oracle that reads the target
7 +lowers the loss by 15\,\% on the synthetic suite and 3.6\,\% on web text, but the learned value controller---four
8 +feature sets, three loss variants, labels from an editor-unseen stream, calibration-based early stopping---never
9 +beats not routing on held-out data, because \labelscalibwithin{}\,\% of the variance of intervention gains is
10 +action-level, opportunity concentrates in high-uncertainty states where a wrong action costs more than a right one
11 +gains, and the controller's within-state ranking is at chance. On 98M tokens of web text at 103M parameters, neither
12 +adapter moves the frozen backbone by more than a tenth of a percent of loss, the parameter-matched plain decoder
13 +does better than both, and the zero-initialised readout prescribed by the proposal does not learn without a
14 +scale-aware bootstrap. Measured on Apple Silicon, eight sparse edits cost three times the backbone's decode
15 +latency at 103M against an analytic 1.6 times, and dense writes are only 17\,\% slower than sparse ones. The exact
16 +zero-budget bypass holds bitwise; the parameter and operation accounting of the proposal is reproduced.
modified paper/sections/results.tex +14 −0
@@ -139,6 +139,20 @@ items per thousand from wrong to right and \vtwoprismetransminus{} from right to
139 139 No adapter changes the held-out regimes by more than a point, and the compose family stays at 2--3\,\%.
140 140 \input{tables/stats.tex}
141 141
142 +\subsection{Pre-registered primary outcome on the locked test regimes}\label{sec:results-test}
143 +The test streams (seed 9) were unsealed on 2026-10-30 after the pre-registration was frozen
144 +(Appendix~\ref{app:prereg}); every system was evaluated once, at budget 0 and at forced random budget 2, on 1,000
145 +stratified items per regime (Table~\ref{tab:test}). The pre-registered primary outcome---pooled exact match over
146 +the iid, template and depth regimes at budget 2 versus the frozen backbone---is $\testprismebtwovsbzero$ points
147 +for the sparse editor (95\,\% $t$-interval over \testprismebtwovsbzeron{} seeds
148 +$[\testprismebtwovsbzerolo,\testprismebtwovsbzerohi]$), far below the pre-registered minimum effect of three
149 +points; against the global-recurrence adapter the paired difference is $\testprismevsgr$ points
150 +$[\testprismevsgrlo,\testprismevsgrhi]$ and against the dense editor $\testprismevsdense$ points
151 +$[\testprismevsdenselo,\testprismevsdensehi]$. The compose regime stays at 4--5\,\% for every system. By the
152 +decision rule fixed in advance, the architecture does not advance: it neither exceeds the strongest matched
153 +baseline by three points at comparable cost nor reduces cost at non-inferior quality.
154 +\input{tables/test.tex}
155 +
142 156 \subsection{Latency and memory}\label{sec:results-latency}
143 157 Table~\ref{tab:latency} gives measured batch-1 and batch-8 decode latencies for the tiny model on the
144 158 development laptop (Apple M5 Max, fp32, PyTorch 2.15, MPS). Budget 0 costs \latencybzero{} ms per token; each
modified paper/tables/numbers.tex +14 −2
@@ -50,8 +50,8 @@
50 50 \newcommand{\pilotedminutes}{90}
51 51 \newcommand{\pilotedparams}{1,907,713}
52 52 \newcommand{\pilotedpeakgb}{25.9}
53 −\newcommand{\pilotenhours}{n/a}
54 −\newcommand{\pilotenppl}{n/a}
53 +\newcommand{\pilotenhours}{2.4}
54 +\newcommand{\pilotenppl}{47.9}
55 55 \newcommand{\pilotgrgainbeight}{0.07}
56 56 \newcommand{\pilotgrgainbfour}{0.08}
57 57 \newcommand{\pilotgrgainbone}{0.04}
@@ -88,6 +88,18 @@
88 88 \newcommand{\routinguniformbeight}{+0.6}
89 89 \newcommand{\routinguniformbone}{-0.8}
90 90 \newcommand{\routinguniformbtwo}{-0.7}
91 +\newcommand{\testprismebtwovsbzero}{+0.9}
92 +\newcommand{\testprismebtwovsbzerohi}{+1.0}
93 +\newcommand{\testprismebtwovsbzerolo}{+0.9}
94 +\newcommand{\testprismebtwovsbzeron}{5}
95 +\newcommand{\testprismevsdense}{+0.4}
96 +\newcommand{\testprismevsdensehi}{+0.7}
97 +\newcommand{\testprismevsdenselo}{+0.1}
98 +\newcommand{\testprismevsdensen}{5}
99 +\newcommand{\testprismevsgr}{-0.3}
100 +\newcommand{\testprismevsgrhi}{+0.1}
101 +\newcommand{\testprismevsgrlo}{-0.6}
102 +\newcommand{\testprismevsgrn}{5}
91 103 \newcommand{\vtwobestfixedrel}{-4.2}
92 104 \newcommand{\vtwobzeroloss}{0.4621}
93 105 \newcommand{\vtwoctrlbetter}{3}
added paper/tables/test.tex +14 −0
@@ -0,0 +1,14 @@
1 +\begin{table}[t]\centering\small
2 +\caption{LOCKED test regimes (synthetic suite, 1,000 stratified items per regime, unsealed 2026-10-30 after the
3 +pre-registration was frozen): exact match (\%) at budget 0 (frozen backbone, identical for all systems) and at forced
4 +random budget 2, mean $\pm$ SD over adapter seeds. ``pooled'' = the pre-registered primary outcome (iid+template+depth).}
5 +\label{tab:test}
6 +\begin{tabular}{llrrrrr}\toprule
7 +System & budget & iid & template & depth & compose & pooled\\\midrule
8 +PRISME sparse editor & 0 & 36.8 $\pm$ 0.0 & 25.4 $\pm$ 0.0 & 31.2 $\pm$ 0.0 & 4.6 $\pm$ 0.0 & 31.1 $\pm$ 0.0\\
9 +PRISME sparse editor & 2 & 38.3 $\pm$ 0.3 & 25.9 $\pm$ 0.5 & 32.1 $\pm$ 0.2 & 4.3 $\pm$ 0.3 & 32.1 $\pm$ 0.1\\
10 +Global recurrence & 0 & 36.8 $\pm$ 0.0 & 25.4 $\pm$ 0.0 & 31.2 $\pm$ 0.0 & 4.6 $\pm$ 0.0 & 31.1 $\pm$ 0.0\\
11 +Global recurrence & 2 & 38.3 $\pm$ 0.4 & 25.6 $\pm$ 0.4 & 33.2 $\pm$ 0.4 & 4.3 $\pm$ 0.5 & 32.3 $\pm$ 0.3\\
12 +Dense 16-slot & 0 & 36.8 $\pm$ 0.0 & 25.4 $\pm$ 0.0 & 31.2 $\pm$ 0.0 & 4.6 $\pm$ 0.0 & 31.1 $\pm$ 0.0\\
13 +Dense 16-slot & 2 & 37.7 $\pm$ 0.1 & 25.4 $\pm$ 0.4 & 32.0 $\pm$ 0.2 & 4.7 $\pm$ 0.2 & 31.7 $\pm$ 0.2\\
14 +\bottomrule\end{tabular}\end{table}
modified protocol/locked_test_manifest.json +5 −4
@@ -1,8 +1,8 @@
1 1 {
2 − "status": "sealed",
3 − "unsealed": false,
2 + "status": "unsealed",
3 + "unsealed": true,
4 4 "sealed_at": "2026-10-29T23:30:00-0400",
5 − "unsealed_at": null,
5 + "unsealed_at": "2026-10-30T04:04:55-0400",
6 6 "locked_splits": {
7 7 "test_iid": "f342d78bb14c8d19c513e9040a1b4f07dee5f866ad6c1a0fc5729a9d011ef45b",
8 8 "test_template": "f9a7402e39b4304695625cd82a8b7ddd7d9b4f607860a5b26d9d7b2ceda50705",
@@ -11,5 +11,6 @@
11 11 },
12 12 "fineweb_test_sha256": "1ce9ca3ab6d659fe08042844fc0589951e09386ee79d2ef76504ffaaebb0947b",
13 13 "fineweb_test_tokens": 2883436,
14 − "rule": "evaluation.run refuses any split whose name starts with 'test' while unsealed is false; unsealing requires protocol/preregistration.md to be frozen (status FROZEN) and is recorded here with a timestamp."
14 + "rule": "evaluation.run refuses any split whose name starts with 'test' while unsealed is false; unsealing requires protocol/preregistration.md to be frozen (status FROZEN) and is recorded here with a timestamp.",
15 + "preregistration_version": "1.0 frozen"
15 16 }
\ No newline at end of file
modified protocol/preregistration.md +14 −3
@@ -1,7 +1,18 @@
1 −# Pre-registration — WP31 PRISME pilot (DRAFT v0.1, 2026-10-29; to be FROZEN before any `test_*` split is read)
1 +# Pre-registration — WP31 PRISME pilot (v1.0, FROZEN 2026-10-30T04:04-0400 before any `test_*` split was read)
2 2
3 −Status: DRAFT. No file named `test*.jsonl` / `test.bin` has been opened by any evaluation script so far
4 −(`protocol/locked_test_manifest.json` records the split hashes and the unsealing timestamp once frozen).
3 +Status: FROZEN. No file named `test*.jsonl` / `test.bin` had been opened by any evaluation script at freeze time
4 +(`protocol/locked_test_manifest.json` records the split hashes and the unsealing timestamp).
5 +
6 +## 0. Primary comparisons as executed (fixed at freeze, before unsealing)
7 +* P1 (H1/H2, synthetic): exact match on the pooled test regimes at forced budget 2 for the sparse editor vs the frozen
8 + backbone, vs the global-recurrence adapter and vs the dense editor, 5 adapter seeds each (v2 backbone), paired by item.
9 +* P2 (H3, synthetic): dev-iid loss of the locked controller vs the best fixed schedule and vs budget 0 (already reported
10 + on dev; the test regimes are evaluated at budget 0 and 2 with forced random actions only, since the controller was
11 + worse than budget 0 on dev for every seed — the controller is therefore evaluated on test for completeness, not as the
12 + primary system).
13 +* P3 (H1/H4, language): FineWeb-Edu locked test cross-entropy at budgets 0/1/2/4/8 for the 103M backbone, the sparse
14 + editor, the global adapter and the parameter-matched plain decoder.
15 +* Decision thresholds as in Section 4 below; no other analysis is pre-registered.
5 16
6 17 ## 1. Primary benchmark suite
7 18 * **Synthetic compositional suite v1** (`data/splits/synthetic_v1`, generator `data/generators/synthetic.py`,
added results/analyzed/test_statistics.json +69 −0
@@ -0,0 +1,69 @@
1 +{
2 + "pooled_per_seed": {
3 + "p4v2_prisme|budget0": [
4 + 0.3113333333333333,
5 + 0.3113333333333333,
6 + 0.3113333333333333,
7 + 0.3113333333333333,
8 + 0.3113333333333333
9 + ],
10 + "p4v2_prisme|forced_b2": [
11 + 0.3216666666666667,
12 + 0.32033333333333336,
13 + 0.3213333333333333,
14 + 0.32066666666666666,
15 + 0.32
16 + ],
17 + "p4v2_gr|budget0": [
18 + 0.3113333333333333,
19 + 0.3113333333333333,
20 + 0.3113333333333333,
21 + 0.3113333333333333,
22 + 0.3113333333333333
23 + ],
24 + "p4v2_gr|forced_b2": [
25 + 0.322,
26 + 0.3236666666666667,
27 + 0.32033333333333336,
28 + 0.3273333333333333,
29 + 0.3233333333333333
30 + ],
31 + "p4v2_dense|budget0": [
32 + 0.3113333333333333,
33 + 0.3113333333333333,
34 + 0.3113333333333333,
35 + 0.3113333333333333,
36 + 0.3113333333333333
37 + ],
38 + "p4v2_dense|forced_b2": [
39 + 0.3153333333333333,
40 + 0.31833333333333336,
41 + 0.3156666666666667,
42 + 0.31866666666666665,
43 + 0.316
44 + ]
45 + },
46 + "stats": {
47 + "prismebtwovsbzero": {
48 + "mean": 0.009466666666666668,
49 + "sd": 0.000691214711777591,
50 + "ci_low": 0.008608410244023642,
51 + "ci_high": 0.010324923089309695,
52 + "n": 5
53 + },
54 + "prismevsgr": {
55 + "mean": -0.0025333333333333336,
56 + "sd": 0.002987008908962647,
57 + "ci_low": -0.0062421947579061575,
58 + "ci_high": 0.0011755280912394898,
59 + "n": 5
60 + },
61 + "prismevsdense": {
62 + "mean": 0.004,
63 + "sd": 0.002013840995599095,
64 + "ci_low": 0.0014994861376577788,
65 + "ci_high": 0.006500513862342221,
66 + "n": 5
67 + }
68 + }
69 +}
\ No newline at end of file
modified results/claim_registry.csv +3 −0
@@ -20,3 +20,6 @@ Miniature end-to-end pipeline runs on CPU,wall seconds,56,mini configs,mini_back
20 20 103M global-recurrence adapter changes held-out loss by less than 0.1%,val ppl by budget,48.16 -> 48.01,FineWeb-Edu val,gr_pilot_103m_seed01_20261030015117,results/raw/eval_gr_103m_val.json,evaluation/run.py,complete
21 21 103M controller: oracle opportunity exists (-3.6% at b8) but the controller captures none (-0.01%),val loss relative change,see numbers.tex pilotrouting*,FineWeb-Edu val,controller_103m_seed01,results/raw/routing_103m_val.json,evaluation/routing.py,complete
22 22 103M latency: 8 sparse edits 3.0x budget-0 median latency; dense 3.6x; controller 1.5x,median ms/token batch 1,7.55 / 22.9 / 26.9 / 11.0,M3 Ultra MPS fp32,controller_103m_seed01,results/raw/latency_103m_mps.json,evaluation/latency.py,complete
23 +Pre-registered primary outcome: pooled test EM gain of sparse editor at b2 is +0.9 pp (< 3 pp threshold),exact match pp,see numbers.tex testprismebtwovsbzero*,synthetic locked test regimes (1000 items each),p4v2_prisme (5 seeds),results/analyzed/test_statistics.json;paper/tables/test.tex,scripts/make_results_tables.py,complete
24 +Sparse editor vs global recurrence on locked test: -0.3 pp [-0.6 +0.1]; vs dense +0.4 pp [+0.1 +0.7],exact match pp,see numbers.tex testprismevs*,synthetic locked test regimes,p4v2_prisme;p4v2_gr;p4v2_dense,results/analyzed/test_statistics.json,scripts/make_results_tables.py,complete
25 +Parameter-matched plain decoder (102.2M) beats backbone+adapters on validation text,val loss,3.8689 vs 3.8746,FineWeb-Edu val,pretrain_enlarged102m_seed01_20261030015017,experiments/runs/pretrain_enlarged102m_seed01_20261030015017/manifest.json,training/pretrain.py,complete
added results/raw/eval_p4v2_dense_s1_test_compose.json +31 −0
@@ -0,0 +1,31 @@
1 +{
2 + "system": "p4v2_dense_s1",
3 + "split": "test_compose",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed01_20261030013421/checkpoint",
5 + "checkpoint_sha256": "a31c97935f20348d4763244744bc1606292a50f1bcd8fddfc0df68b7550c29aa",
6 + "config_sha256": "ed25665359232b92666bd3a813caa56cf6b32ba4c23af58a3c562f5c4b72d9e2",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:06:12-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.046296296296296294,
15 + "n": 540,
16 + "by_family": {
17 + "compose": 0.046296296296296294
18 + },
19 + "edits_per_token": 0.0
20 + },
21 + "forced_b2": {
22 + "accuracy": 0.046296296296296294,
23 + "n": 540,
24 + "by_family": {
25 + "compose": 0.046296296296296294
26 + },
27 + "edits_per_token": 2.0
28 + }
29 + },
30 + "seconds": 1.8
31 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s1_test_depth.json +39 −0
@@ -0,0 +1,39 @@
1 +{
2 + "system": "p4v2_dense_s1",
3 + "split": "test_depth",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed01_20261030013421/checkpoint",
5 + "checkpoint_sha256": "a31c97935f20348d4763244744bc1606292a50f1bcd8fddfc0df68b7550c29aa",
6 + "config_sha256": "ed25665359232b92666bd3a813caa56cf6b32ba4c23af58a3c562f5c4b72d9e2",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:05:59-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.312,
15 + "n": 1000,
16 + "by_family": {
17 + "arith": 0.0,
18 + "bind": 0.02,
19 + "fsm": 0.425,
20 + "excl": 0.81,
21 + "sched": 0.305
22 + },
23 + "edits_per_token": 0.0
24 + },
25 + "forced_b2": {
26 + "accuracy": 0.317,
27 + "n": 1000,
28 + "by_family": {
29 + "arith": 0.0,
30 + "bind": 0.025,
31 + "fsm": 0.42,
32 + "excl": 0.805,
33 + "sched": 0.335
34 + },
35 + "edits_per_token": 2.0
36 + }
37 + },
38 + "seconds": 12.7
39 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s1_test_iid.json +39 −0
@@ -0,0 +1,39 @@
1 +{
2 + "system": "p4v2_dense_s1",
3 + "split": "test_iid",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed01_20261030013421/checkpoint",
5 + "checkpoint_sha256": "a31c97935f20348d4763244744bc1606292a50f1bcd8fddfc0df68b7550c29aa",
6 + "config_sha256": "ed25665359232b92666bd3a813caa56cf6b32ba4c23af58a3c562f5c4b72d9e2",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:05:31-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.368,
15 + "n": 1000,
16 + "by_family": {
17 + "bind": 0.2119815668202765,
18 + "fsm": 0.4358974358974359,
19 + "arith": 0.06701030927835051,
20 + "excl": 0.3348623853211009,
21 + "sched": 0.7813953488372093
22 + },
23 + "edits_per_token": 0.0
24 + },
25 + "forced_b2": {
26 + "accuracy": 0.378,
27 + "n": 1000,
28 + "by_family": {
29 + "bind": 0.2119815668202765,
30 + "fsm": 0.4807692307692308,
31 + "arith": 0.07731958762886598,
32 + "excl": 0.3348623853211009,
33 + "sched": 0.786046511627907
34 + },
35 + "edits_per_token": 2.0
36 + }
37 + },
38 + "seconds": 15.0
39 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s1_test_template.json +39 −0
@@ -0,0 +1,39 @@
1 +{
2 + "system": "p4v2_dense_s1",
3 + "split": "test_template",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed01_20261030013421/checkpoint",
5 + "checkpoint_sha256": "a31c97935f20348d4763244744bc1606292a50f1bcd8fddfc0df68b7550c29aa",
6 + "config_sha256": "ed25665359232b92666bd3a813caa56cf6b32ba4c23af58a3c562f5c4b72d9e2",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:05:45-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.254,
15 + "n": 1000,
16 + "by_family": {
17 + "fsm": 0.430939226519337,
18 + "arith": 0.024875621890547265,
19 + "bind": 0.10679611650485436,
20 + "sched": 0.4854368932038835,
21 + "excl": 0.23786407766990292
22 + },
23 + "edits_per_token": 0.0
24 + },
25 + "forced_b2": {
26 + "accuracy": 0.251,
27 + "n": 1000,
28 + "by_family": {
29 + "fsm": 0.4530386740331492,
30 + "arith": 0.029850746268656716,
31 + "bind": 0.0825242718446602,
32 + "sched": 0.4854368932038835,
33 + "excl": 0.22330097087378642
34 + },
35 + "edits_per_token": 2.0
36 + }
37 + },
38 + "seconds": 13.6
39 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s2_test_compose.json +31 −0
@@ -0,0 +1,31 @@
1 +{
2 + "system": "p4v2_dense_s2",
3 + "split": "test_compose",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed02_20261030004544/checkpoint",
5 + "checkpoint_sha256": "a7371f8c402f8a7fd98f784bb1c6750d70ccb46f53060656ba762df1166582b0",
6 + "config_sha256": "dbf7526c8c09e5cb0257a92cb81256910e8697c6a3507f64b37a3af184e37f46",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:06:59-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.046296296296296294,
15 + "n": 540,
16 + "by_family": {
17 + "compose": 0.046296296296296294
18 + },
19 + "edits_per_token": 0.0
20 + },
21 + "forced_b2": {
22 + "accuracy": 0.04814814814814815,
23 + "n": 540,
24 + "by_family": {
25 + "compose": 0.04814814814814815
26 + },
27 + "edits_per_token": 2.0
28 + }
29 + },
30 + "seconds": 1.5
31 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s2_test_depth.json +39 −0
@@ -0,0 +1,39 @@
1 +{
2 + "system": "p4v2_dense_s2",
3 + "split": "test_depth",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed02_20261030004544/checkpoint",
5 + "checkpoint_sha256": "a7371f8c402f8a7fd98f784bb1c6750d70ccb46f53060656ba762df1166582b0",
6 + "config_sha256": "dbf7526c8c09e5cb0257a92cb81256910e8697c6a3507f64b37a3af184e37f46",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:06:49-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.312,
15 + "n": 1000,
16 + "by_family": {
17 + "arith": 0.0,
18 + "bind": 0.02,
19 + "fsm": 0.425,
20 + "excl": 0.81,
21 + "sched": 0.305
22 + },
23 + "edits_per_token": 0.0
24 + },
25 + "forced_b2": {
26 + "accuracy": 0.319,
27 + "n": 1000,
28 + "by_family": {
29 + "arith": 0.0,
30 + "bind": 0.02,
31 + "fsm": 0.425,
32 + "excl": 0.805,
33 + "sched": 0.345
34 + },
35 + "edits_per_token": 2.0
36 + }
37 + },
38 + "seconds": 10.3
39 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s2_test_iid.json +39 −0
@@ -0,0 +1,39 @@
1 +{
2 + "system": "p4v2_dense_s2",
3 + "split": "test_iid",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed02_20261030004544/checkpoint",
5 + "checkpoint_sha256": "a7371f8c402f8a7fd98f784bb1c6750d70ccb46f53060656ba762df1166582b0",
6 + "config_sha256": "dbf7526c8c09e5cb0257a92cb81256910e8697c6a3507f64b37a3af184e37f46",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:06:25-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.368,
15 + "n": 1000,
16 + "by_family": {
17 + "bind": 0.2119815668202765,
18 + "fsm": 0.4358974358974359,
19 + "arith": 0.06701030927835051,
20 + "excl": 0.3348623853211009,
21 + "sched": 0.7813953488372093
22 + },
23 + "edits_per_token": 0.0
24 + },
25 + "forced_b2": {
26 + "accuracy": 0.377,
27 + "n": 1000,
28 + "by_family": {
29 + "bind": 0.22119815668202766,
30 + "fsm": 0.4807692307692308,
31 + "arith": 0.07731958762886598,
32 + "excl": 0.3256880733944954,
33 + "sched": 0.7813953488372093
34 + },
35 + "edits_per_token": 2.0
36 + }
37 + },
38 + "seconds": 12.2
39 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s2_test_template.json +39 −0
@@ -0,0 +1,39 @@
1 +{
2 + "system": "p4v2_dense_s2",
3 + "split": "test_template",
4 + "checkpoint": "experiments/runs/p4v2_dense_seed02_20261030004544/checkpoint",
5 + "checkpoint_sha256": "a7371f8c402f8a7fd98f784bb1c6750d70ccb46f53060656ba762df1166582b0",
6 + "config_sha256": "dbf7526c8c09e5cb0257a92cb81256910e8697c6a3507f64b37a3af184e37f46",
7 + "code_commit": "unknown",
8 + "device": "mps",
9 + "precision": "fp32",
10 + "evaluated_at": "2026-10-30T04:06:37-0400",
11 + "data_sha256": "06620e1cf60b7250aac71f4fb5ee74e3705f4b9aab102e22b5c5bdbed76c4ae2",
12 + "exact_match": {
13 + "budget0": {
14 + "accuracy": 0.254,
15 + "n": 1000,
16 + "by_family": {
17 + "fsm": 0.430939226519337,
18 + "arith": 0.024875621890547265,
19 + "bind": 0.10679611650485436,
20 + "sched": 0.4854368932038835,
21 + "excl": 0.23786407766990292
22 + },
23 + "edits_per_token": 0.0
24 + },
25 + "forced_b2": {
26 + "accuracy": 0.259,
27 + "n": 1000,
28 + "by_family": {
29 + "fsm": 0.44751381215469616,
30 + "arith": 0.03980099502487562,
31 + "bind": 0.07766990291262135,
32 + "sched": 0.5097087378640777,
33 + "excl": 0.23786407766990292
34 + },
35 + "edits_per_token": 2.0
36 + }
37 + },
38 + "seconds": 11.5
39 +}
\ No newline at end of file
added results/raw/eval_p4v2_dense_s3_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s3_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s3_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s3_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s4_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s4_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s4_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s4_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s5_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s5_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s5_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_dense_s5_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s1_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s1_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s1_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s1_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s2_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s2_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s2_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s2_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s3_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s3_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s3_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s3_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s4_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s4_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s4_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s4_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s5_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s5_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s5_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_gr_s5_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s1_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s1_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s1_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s1_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s2_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s2_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s2_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s2_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s3_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s3_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s3_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s3_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s4_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s4_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s4_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s4_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s5_test_compose.json +31 −0
Large diff (32 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s5_test_depth.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s5_test_iid.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/eval_p4v2_prisme_s5_test_template.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s1_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s2_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s3_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s4_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_dense_s5_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s1_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s2_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s3_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s4_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_gr_s5_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s1_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s2_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s3_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s4_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_compose_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_compose_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_depth_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_depth_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_iid_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_iid_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_template_budget0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/items_p4v2_prisme_s5_test_template_forced_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
modified scripts/make_results_tables.py +71 −0
Large diff (75 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere