SPB Git
spboucher.ai

spb/wp31_uqo

Public

WP31: v2 routing seeds, mini pipeline, README, cards, run index

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Oct 30, 2026) parent ede6705 Browse files

81 changed files +4,600 −105

81 changed files
Load all 81 files Unified Split
added README.md +44 −0
@@ -0,0 +1,44 @@
1 +<!-- Author: Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca -->
2 +# 🧩 PRISME: Intervention-Supervised Sparse Workspace Editing for Small Language Models
3 +
4 +**UQO Working Paper No. 31** — *Design, controlled evaluation, and negative and positive evidence at the 100M-parameter scale*
5 +
6 +[![Paper](https://img.shields.io/badge/paper-PDF-b31b1b?logo=latex&logoColor=white)](paper/main.pdf)
7 +[![Python](https://img.shields.io/badge/python-3.12-3776AB?logo=python&logoColor=white)](requirements-lock.txt)
8 +[![Tests](https://img.shields.io/badge/tests-23%20passing-2e7d32)](tests/)
9 +[![Reproducible](https://img.shields.io/badge/reproducible-end--to--end%20pipeline-2e7d32)](REPRODUCE.md)
10 +[![Institution](https://img.shields.io/badge/UQO-D%C3%A9pt.%20des%20sciences%20administratives-16365c)](https://uqo.ca)
11 +
12 +> **TL;DR** — PRISME adds to a frozen causal decoder a read-only source bank, a 16-slot ephemeral workspace, a
13 +> shared sparse editor (1–2 slots per action, 24 actions + STOP), a value controller trained on executed
14 +> counterfactual gains, and a differential readout with an exact zero-budget bypass. We implement, test, train and
15 +> evaluate it against matched baselines. Result: the editor is learnable and helps a frozen backbone by a few
16 +> percent of loss, but a global-recurrence adapter trained identically helps as much; where to write does not
17 +> matter for fixed schedules; a target-reading oracle shows large opportunity that the learned controller cannot
18 +> capture (it is worse than no routing on held-out data); and dispatch overhead erases sparse savings at small
19 +> scale. A careful negative result with a mechanistic explanation.
20 +
21 +## 📌 Status
22 +| Item | Value |
23 +|---|---|
24 +| Version | 1.0 (October 2026) |
25 +| Phases executed | 0–10, 13–15 (see `PROGRESS.md`); 11 (MLX port) not executed; 12 limited to meta-instantiation |
26 +| Decisive experiments | 5 adapter seeds × {sparse editor, global recurrence, dense writes} on a frozen synthetic backbone; 103M FineWeb-Edu pilot (one backbone seed) |
27 +| Code & data | `REPRODUCE.md`; synthetic suite regenerated from seeds; FineWeb-Edu subset documented, not redistributed |
28 +
29 +## 🗂 Repository layout
30 +`prisme/` reference implementation · `training/` stages A–D · `baselines/` global recurrence + reference routers ·
31 +`evaluation/` routing, latency, perplexity, locked driver, statistics · `data/` generators, processors, splits ·
32 +`tests/` invariants + cache equivalence · `scripts/` audits, counts, FLOPs, launchers, tables, figures, claim audit ·
33 +`protocol/` pre-registration, locked test manifest, deviations · `literature/` verified bibliography + novelty audit ·
34 +`experiments/runs/` manifests + metrics (checkpoints on the training nodes) · `results/` raw/analyzed/tables/figures ·
35 +`paper/` LaTeX + PDF · `reproducibility/` environment, cards, run index, audit · `PROGRESS.md`, `DECISIONS.md`, `FAILURES.md`.
36 +
37 +## 🚀 Quick start
38 +```bash
39 +uv venv --python 3.12 .venv && uv pip install -p .venv/bin/python -r requirements-lock.txt
40 +.venv/bin/python -m pytest tests -q
41 +.venv/bin/python scripts/count_params.py --all-configs --device meta
42 +scripts/reproduce_pilot.sh # miniature end-to-end pipeline on CPU (~1 min)
43 +.venv/bin/python scripts/make_tables.py && .venv/bin/python scripts/reproduce_figures.py
44 +```
added REPRODUCE.md +21 −0
@@ -0,0 +1,21 @@
1 +# REPRODUCE.md — WP31 PRISME
2 +
3 +All commands run from the repository root with the project venv (`uv venv --python 3.12 .venv && uv pip install -p .venv/bin/python -r requirements-lock.txt`).
4 +
5 +| Step | Command | Expected |
6 +|---|---|---|
7 +| Environment audit | `.venv/bin/python scripts/environment_audit.py --smoke` | `hardware/device_inventory.json`, CPU (and MPS/CUDA if present) forward/backward OK |
8 +| Unit + property + integration tests | `.venv/bin/python -m pytest tests -q` | 23 passed (15 mandated invariants, controller target-freedom, dense ablation, 5 cache-equivalence tests) |
9 +| Exact parameter counts | `.venv/bin/python scripts/count_params.py --all-configs --device meta` | backbone 99,502,848 / 446,744,064 / 1,149,339,648 (= proposal); workspace 2,631,173 / 10,505,221 / 21,655,813 |
10 +| Analytic ops table | `.venv/bin/python scripts/estimate_flops.py --config configs/prisme_459m.yaml --T 4096` | 1.296 / 1.666 / 1.787 / 2.219 / 5.633 ×10⁹ ops/token |
11 +| Synthetic data | `.venv/bin/python -c "import sys; sys.path.insert(0,'.'); from data.generators.synthetic import write_splits; print(write_splits('data/splits/synthetic_v1'))"` | 11 split files + manifest (sha256 as in `data/splits/synthetic_v1/manifest.json`) |
12 +| Web corpus (optional, ~10 min, network) | `.venv/bin/python data/processors/prepare_corpus.py` | 290M train / 2.9M val / 2.9M locked test tokens, tokenizer 32,768 |
13 +| Miniature end-to-end pipeline (CPU, ~minutes) | `scripts/reproduce_pilot.sh` | tiny backbone → editor bootstrap → labels → controller → routing JSON under `experiments/runs/` and `results/raw/` |
14 +| Dry runs of every trainer | `.venv/bin/python -m training.pretrain --config configs/experiments/pretrain_103m.yaml --dry-run` etc. | exit 0 |
15 +| Tables and figures | `.venv/bin/python scripts/make_tables.py && .venv/bin/python scripts/reproduce_figures.py --results results/raw/` | `paper/tables/*.tex`, `results/figures/*.pdf` |
16 +| Claim audit | `.venv/bin/python scripts/verify_manuscript_claims.py` | `AUDIT PASS` |
17 +| Manuscript | `cd paper && latexmk -pdf main.tex` | `paper/main.pdf` |
18 +
19 +Full-scale runs (Stage A 103M on an M3 Ultra ≈ 4 h; seed matrices on several nodes) are launched with
20 +`scripts/cluster_setup.sh <node>`, `scripts/launch_seed_matrix.sh`, `scripts/run_stage_cd.sh` and
21 +`scripts/launch_103m.sh`; their run ids are listed in `reproducibility/run_index.md`.
added configs/experiments/controller_mini.yaml +27 −0
@@ -0,0 +1,27 @@
1 +name: controller_mini
2 +seed: 1
3 +mode: prisme
4 +model: from_checkpoint
5 +checkpoint: experiments/runs/mini_editor_seed01_20261030005118/checkpoint
6 +labels:
7 + train: experiments/runs/labels_mini_seed01_20261030005150/labels_train.pt
8 + calib: experiments/runs/labels_mini_seed01_20261030005150/labels_calib.pt
9 +train:
10 + steps: 60
11 + lr: 0.001
12 + weight_decay: 0.01
13 + alpha: 1.0
14 + beta: 0.5
15 + gamma: 0.1
16 + huber_delta: 0.1
17 + log_every: 20
18 + eval_every: 30
19 + seq_len: 320
20 +lambda_grid:
21 +- 0.0
22 +- 0.3
23 +kappa_grid:
24 +- 0.0
25 +sweep_batches: 2
26 +max_budget: 4
27 +selection_rule: max_loss_reduction_per_ops
added configs/experiments/labels_mini.yaml +25 −0
@@ -0,0 +1,25 @@
1 +name: labels_mini
2 +seed: 1
3 +mode: prisme
4 +model: from_checkpoint
5 +checkpoint: experiments/runs/mini_editor_seed01_20261030005118/checkpoint
6 +use_policy: false
7 +data:
8 + kind: synthetic
9 + tokenizer: char
10 + splits_dir: data/splits/synthetic_v1
11 + loss_span: answer
12 +collect:
13 + batch_size: 16
14 + seq_len: 320
15 + max_prefix: 2
16 + audit_every: 2
17 +partitions:
18 + train:
19 + split: ctrl_train
20 + n_batches: 12
21 + stream_seed: 11
22 + calib:
23 + split: calib
24 + n_batches: 4
25 + stream_seed: 12
added configs/experiments/mini_backbone.yaml +43 −0
@@ -0,0 +1,43 @@
1 +name: mini_backbone
2 +seed: 1
3 +mode: backbone_only
4 +model:
5 + name: mini
6 + vocab_size: 128
7 + d_model: 96
8 + n_layers: 2
9 + n_heads: 4
10 + n_kv_heads: 2
11 + ffn_width: 256
12 + max_seq_len: 320
13 + w: 32
14 + ffn_w: 96
15 + n_slots: 16
16 + n_editor_heads: 2
17 + r_max: 8
18 + readout_zero_init: false
19 + readout_init_std: 0.2
20 + ctrl_input: h
21 + ctrl_use_entropy: true
22 +init:
23 + freeze_backbone: false
24 +data:
25 + kind: synthetic
26 + tokenizer: char
27 + splits_dir: data/splits/synthetic_v1
28 + loss_span: all
29 + eval_splits:
30 + - dev_iid
31 +train:
32 + batch_size: 16
33 + seq_len: 320
34 + steps: 150
35 + lr: 0.001
36 + warmup: 10
37 + weight_decay: 0.1
38 + grad_clip: 1.0
39 + log_every: 50
40 + eval_every: 150
41 + eval_batches: 2
42 + ckpt_every: 150
43 +em_max_items: 32
added configs/experiments/mini_editor.yaml +61 −0
@@ -0,0 +1,61 @@
1 +name: mini_editor
2 +seed: 1
3 +mode: prisme
4 +model:
5 + name: mini
6 + vocab_size: 128
7 + d_model: 96
8 + n_layers: 2
9 + n_heads: 4
10 + n_kv_heads: 2
11 + ffn_width: 256
12 + max_seq_len: 320
13 + w: 32
14 + ffn_w: 96
15 + n_slots: 16
16 + n_editor_heads: 2
17 + r_max: 8
18 + readout_zero_init: false
19 + readout_init_std: 0.2
20 + ctrl_input: h
21 + ctrl_use_entropy: true
22 +init:
23 + backbone_checkpoint: experiments/runs/mini_backbone_seed01_20261030005114/checkpoint
24 + freeze_backbone: true
25 + auto_readout_scale: true
26 +data:
27 + kind: synthetic
28 + tokenizer: char
29 + splits_dir: data/splits/synthetic_v1
30 + loss_span: answer
31 + eval_splits:
32 + - dev_iid
33 +train:
34 + batch_size: 16
35 + seq_len: 320
36 + steps: 120
37 + lr: 0.001
38 + warmup: 10
39 + weight_decay: 0.1
40 + grad_clip: 1.0
41 + log_every: 50
42 + eval_every: 120
43 + eval_batches: 2
44 + ckpt_every: 120
45 +em_max_items: 32
46 +budgets:
47 +- 0
48 +- 1
49 +- 2
50 +- 4
51 +- 8
52 +budget_probs:
53 +- 0.2
54 +- 0.2
55 +- 0.2
56 +- 0.2
57 +- 0.2
58 +budget_per: position
59 +em_budgets:
60 +- 0
61 +- 2
added configs/experiments/routing_mini.yaml +24 −0
@@ -0,0 +1,24 @@
1 +seed: 1
2 +mode: prisme
3 +model: from_checkpoint
4 +checkpoint: experiments/runs/controller_mini_seed01_20261030005154/checkpoint
5 +data:
6 + kind: synthetic
7 + tokenizer: char
8 + splits_dir: data/splits/synthetic_v1
9 + loss_span: answer
10 + split: dev_iid
11 +batch_size: 16
12 +seq_len: 320
13 +n_batches: 2
14 +budgets:
15 +- 1
16 +- 2
17 +oracle_batches: 1
18 +two_step_batches: 1
19 +entropy_thresholds:
20 +- 1.0
21 +norm_thresholds:
22 +- 0.1
23 +lambda_grid:
24 +- 0.0
added experiments/logs/controller_mini.log +14 −0
@@ -0,0 +1,14 @@
1 +/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_controller.py:131: UserWarning: Converting a tensor with requires_grad=True to a scalar may lead to unexpected behavior.
2 +Consider using tensor.detach() first. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/autograd/generated/python_variable_methods.cpp:821.)
3 + run.log(step=step, loss=float(loss), gain=float(L["gain"]), harm=float(L["harm"]), nll=float(L["nll"]))
4 +step 0 loss 0.45115 gain 0.02663 harm 0.8158
5 +step 20 loss 0.13172 gain 0.00543 harm 0.5627
6 + calib: {'huber': 0.0175, 'sign_acc': 0.5467, 'harm_brier': 0.1334, 'harm_ece': 0.0955, 'n_labels': 1156, 'oracle_opportunity_per_state': 0.2508, 'regret_per_state': 0.1381, 'captured_fraction': 0.4494, 'top1_match': 0.0, 'n_audit_states': 124}
7 +step 40 loss 0.07597 gain 0.00382 harm 0.4947
8 +step 59 loss 0.05327 gain 0.00333 harm 0.4847
9 + calib: {'huber': 0.0162, 'sign_acc': 0.7344, 'harm_brier': 0.1264, 'harm_ece': 0.0872, 'n_labels': 1156, 'oracle_opportunity_per_state': 0.2508, 'regret_per_state': 0.1129, 'captured_fraction': 0.5501, 'top1_match': 0.0, 'n_audit_states': 124}
10 +restored controller from step 60 (calibration score -0.11286)
11 +sweep {'lambda': 0.0, 'kappa': 0.0, 'loss': 2.059080786738835, 'edits_per_token': 2.5673758865248226, 'extra_ops_ratio': 1.4467924402115193}
12 +sweep {'lambda': 0.3, 'kappa': 0.0, 'loss': 2.307212234388852, 'edits_per_token': 0.04964539007092199, 'extra_ops_ratio': 0.02571701665296622}
13 +selected {'lambda': 0.3, 'kappa': 0.0, 'loss': 2.307212234388852, 'edits_per_token': 0.04964539007092199, 'extra_ops_ratio': 0.02571701665296622, 'loss_reduction': 0.04320390850094169, 'reduction_per_ops': 1.6799735787377388}
14 +run controller_mini_seed01_20261030005154
added experiments/logs/labels_mini.log +8 −0
@@ -0,0 +1,8 @@
1 +/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_editor.py:57: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:214.)
2 + return torch.from_numpy(acts).to(device), torch.from_numpy(np.ascontiguousarray(b)).to(device)
3 +batch 0/12 r_pre=2 mean gain 0.02953 frac>0 0.615
4 +batch 10/12 r_pre=1 mean gain 0.02505 frac>0 0.578
5 +train {'file': 'experiments/runs/labels_mini_seed01_20261030005150/labels_train.pt', 'n_states': 1047, 'n_labels': 4188, 'mean_gain': 0.031750522553920746, 'frac_positive': 0.5690066814422607, 'frac_harm': 0.43099331855773926, 'seconds': 2.5, 'n_audit_batches': 6}
6 +batch 0/4 r_pre=2 mean gain 0.05232 frac>0 0.560
7 +calib {'file': 'experiments/runs/labels_mini_seed01_20261030005150/labels_calib.pt', 'n_states': 289, 'n_labels': 1156, 'mean_gain': 0.19328418374061584, 'frac_positive': 0.6349480748176575, 'frac_harm': 0.36505189538002014, 'seconds': 0.8, 'n_audit_batches': 2}
8 +run labels_mini_seed01_20261030005150
added experiments/logs/mini_backbone.log +9 −0
@@ -0,0 +1,9 @@
1 +step 0 loss 4.8830 lr 1.00e-04 gn 2.63 (0s)
2 +step 50 loss 2.5086 lr 8.31e-04 gn 0.98 (1s)
3 +step 100 loss 1.9095 lr 3.55e-04 gn 1.14 (2s)
4 +step 149 loss 1.5540 lr 1.00e-04 gn 1.46 (3s)
5 +/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_editor.py:57: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:214.)
6 + return torch.from_numpy(acts).to(device), torch.from_numpy(np.ascontiguousarray(b)).to(device)
7 + val loss by budget: {0: 1.661}
8 + exact match dev_iid budget 0: 0.0000 (n=32) by family {'bind': 0.0, 'arith': 0.0, 'fsm': 0.0, 'excl': 0.0, 'sched': 0.0}
9 +run mini_backbone_seed01_20261030005114 done
added experiments/logs/mini_editor.log +12 −0
@@ -0,0 +1,12 @@
1 +loaded backbone from experiments/runs/mini_backbone_seed01_20261030005114/checkpoint missing: 36
2 +readout_scale set to backbone residual rms = 0.120
3 +/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_editor.py:57: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:214.)
4 + return torch.from_numpy(acts).to(device), torch.from_numpy(np.ascontiguousarray(b)).to(device)
5 +step 0 loss 2.2980 lr 1.00e-04 gn 0.09 (0s)
6 +step 50 loss 2.0584 lr 7.37e-04 gn 0.12 (14s)
7 +step 100 loss 1.9985 lr 1.71e-04 gn 0.21 (25s)
8 +step 119 loss 2.2012 lr 1.00e-04 gn 0.21 (29s)
9 + val loss by budget: {0: 2.391, 1: 2.178, 2: 2.1457, 4: 2.0783, 8: 2.0136}
10 + exact match dev_iid budget 0: 0.0000 (n=32) by family {'bind': 0.0, 'arith': 0.0, 'fsm': 0.0, 'excl': 0.0, 'sched': 0.0}
11 + exact match dev_iid budget 2: 0.0000 (n=32) by family {'bind': 0.0, 'arith': 0.0, 'fsm': 0.0, 'excl': 0.0, 'sched': 0.0}
12 +run mini_editor_seed01_20261030005118 done
added experiments/logs/routing_mini.log +17 −0
@@ -0,0 +1,17 @@
1 +{"policy": "budget0", "loss": 2.3909947187051004, "edits_per_token": 0.0, "extra_ops_ratio": 0.0, "n_tokens": 174}
2 +/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_editor.py:57: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:214.)
3 + return torch.from_numpy(acts).to(device), torch.from_numpy(np.ascontiguousarray(b)).to(device)
4 +{"policy": "random_forced", "budget": 1, "loss": 2.177954575111126, "edits_per_token": 1.0, "extra_ops_ratio": 0.6340132088496767, "n_tokens": 174}
5 +{"policy": "uniform", "budget": 1, "loss": 2.187390908427622, "edits_per_token": 1.0, "extra_ops_ratio": 0.5180141843598465, "n_tokens": 174}
6 +{"policy": "fixed_pair", "budget": 1, "loss": 2.1671100484913794, "edits_per_token": 1.0, "extra_ops_ratio": 0.7239716190031205, "n_tokens": 174}
7 +{"policy": "random_forced", "budget": 2, "loss": 2.145720470910785, "edits_per_token": 2.0, "extra_ops_ratio": 1.2502715012122845, "n_tokens": 174}
8 +{"policy": "uniform", "budget": 2, "loss": 2.167156175635327, "edits_per_token": 2.0, "extra_ops_ratio": 1.036028368719693, "n_tokens": 174}
9 +{"policy": "fixed_pair", "budget": 2, "loss": 2.1463586215315194, "edits_per_token": 2.0, "extra_ops_ratio": 1.447943238006241, "n_tokens": 174}
10 +{"policy": "entropy", "threshold": 1.0, "budget": 2, "loss": 2.147084027871318, "edits_per_token": 1.632183908045977, "extra_ops_ratio": 0.9839830727412783, "n_tokens": 174}
11 +{"policy": "norm", "threshold": 0.1, "budget": 2, "loss": 2.149580155295887, "edits_per_token": 1.9885057471264367, "extra_ops_ratio": 1.194603536320829, "n_tokens": 174}
12 +{"policy": "controller_locked", "lam": 0.3, "kap": 0.0, "budget": 2, "loss": 2.3621927897135415, "edits_per_token": 0.06896551724137931, "extra_ops_ratio": 0.03572511535951461, "n_tokens": 174}
13 +{"policy": "controller", "lam": 0.0, "kap": 0.0, "budget": 2, "loss": 2.1407089233398438, "edits_per_token": 1.5344827586206897, "extra_ops_ratio": 0.8812912908093683, "n_tokens": 174}
14 +{"policy": "oracle1_greedy", "budget": 1, "diagnostic": true, "loss": 2.1457042020909927, "edits_per_token": 0.6470588235294118, "extra_ops_ratio": 0.4587601156795726, "n_tokens": 85}
15 +{"policy": "oracle1_greedy", "budget": 2, "diagnostic": true, "loss": 1.9766888786764707, "edits_per_token": 1.2470588235294118, "extra_ops_ratio": 0.854374605066636, "n_tokens": 85}
16 +{"policy": "oracle_two_step", "diagnostic": true, "one_step_gain": 0.25905925526338464, "two_step_gain": 0.429955561020795, "greedy_then_best_gain": 0.42805216172162225, "n_tokens": 85}
17 +wrote results/raw/routing_mini.json
added experiments/runs/controller_mini_seed01_20261030005154/checkpoint/checkpoint.json +49 −0
@@ -0,0 +1,49 @@
1 +{
2 + "config": {
3 + "name": "mini",
4 + "vocab_size": 128,
5 + "d_model": 96,
6 + "n_layers": 2,
7 + "n_heads": 4,
8 + "n_kv_heads": 2,
9 + "ffn_width": 256,
10 + "max_seq_len": 320,
11 + "rope_theta": 10000.0,
12 + "norm_eps": 1e-05,
13 + "tie_embeddings": true,
14 + "init_std": 0.02,
15 + "use_workspace": true,
16 + "w": 32,
17 + "ffn_w": 96,
18 + "n_slots": 16,
19 + "n_editor_blocks": 2,
20 + "n_editor_heads": 2,
21 + "r_max": 8,
22 + "eta_max": 0.25,
23 + "clip_c": 1.0,
24 + "clip_eps": 1e-06,
25 + "readout_zero_init": false,
26 + "readout_init_std": 0.2,
27 + "readout_scale": 0.11967716366052628,
28 + "controller_hidden": 0,
29 + "ctrl_input": "h",
30 + "ctrl_use_entropy": true,
31 + "lambda_cost": 0.3,
32 + "kappa_risk": 0.0,
33 + "dense_writes": false
34 + },
35 + "saved_at": "2026-10-30T00:51:57-0400",
36 + "run_id": "controller_mini_seed01_20261030005154",
37 + "parent": "experiments/runs/mini_editor_seed01_20261030005118/checkpoint",
38 + "selected": {
39 + "lambda": 0.3,
40 + "kappa": 0.0,
41 + "loss": 2.307212234388852,
42 + "edits_per_token": 0.04964539007092199,
43 + "extra_ops_ratio": 0.02571701665296622,
44 + "loss_reduction": 0.04320390850094169,
45 + "reduction_per_ops": 1.6799735787377388
46 + },
47 + "locked": true,
48 + "weights_sha256": "bb56e9c7b34c773d7cb314b50da687d910b37fa126243f4cfbe9b36c931da83e"
49 +}
\ No newline at end of file
added experiments/runs/controller_mini_seed01_20261030005154/config.yaml +60 −0
@@ -0,0 +1,60 @@
1 +experiment:
2 + name: controller_mini
3 + seed: 1
4 + mode: prisme
5 + model: from_checkpoint
6 + checkpoint: experiments/runs/mini_editor_seed01_20261030005118/checkpoint
7 + labels:
8 + train: experiments/runs/labels_mini_seed01_20261030005150/labels_train.pt
9 + calib: experiments/runs/labels_mini_seed01_20261030005150/labels_calib.pt
10 + train:
11 + steps: 60
12 + lr: 0.001
13 + weight_decay: 0.01
14 + alpha: 1.0
15 + beta: 0.5
16 + gamma: 0.1
17 + huber_delta: 0.1
18 + log_every: 20
19 + eval_every: 30
20 + seq_len: 320
21 + lambda_grid:
22 + - 0.0
23 + - 0.3
24 + kappa_grid:
25 + - 0.0
26 + sweep_batches: 2
27 + max_budget: 4
28 + selection_rule: max_loss_reduction_per_ops
29 +model:
30 + name: mini
31 + vocab_size: 128
32 + d_model: 96
33 + n_layers: 2
34 + n_heads: 4
35 + n_kv_heads: 2
36 + ffn_width: 256
37 + max_seq_len: 320
38 + rope_theta: 10000.0
39 + norm_eps: 1.0e-05
40 + tie_embeddings: true
41 + init_std: 0.02
42 + use_workspace: true
43 + w: 32
44 + ffn_w: 96
45 + n_slots: 16
46 + n_editor_blocks: 2
47 + n_editor_heads: 2
48 + r_max: 8
49 + eta_max: 0.25
50 + clip_c: 1.0
51 + clip_eps: 1.0e-06
52 + readout_zero_init: false
53 + readout_init_std: 0.2
54 + readout_scale: 0.11967716366052628
55 + controller_hidden: 0
56 + ctrl_input: h
57 + ctrl_use_entropy: true
58 + lambda_cost: 0.0
59 + kappa_risk: 0.0
60 + dense_writes: false
added experiments/runs/controller_mini_seed01_20261030005154/manifest.json +79 −0
@@ -0,0 +1,79 @@
1 +{
2 + "run_id": "controller_mini_seed01_20261030005154",
3 + "name": "controller_mini",
4 + "seed": 1,
5 + "code_commit": "ede6705574e7140bdc16eff11931815070b471d1",
6 + "config_sha256": "0471d90913fcc1fc2be8399872a1cb24835993f8ca0091d1534b4be963a5d588",
7 + "started_at": "2026-10-30T00:51:54-0400",
8 + "host": "MacBookPro-de-simon-pierre-2.local",
9 + "python": "3.12.13",
10 + "torch": "2.15.0",
11 + "command": "/Users/simon-pierreboucher/Desktop/Recherche-UQO/ia/prisme-slm/training/train_controller.py --config configs/experiments/controller_mini.yaml --device cpu",
12 + "state": "completed",
13 + "wall_seconds": 3.14,
14 + "device": "cpu",
15 + "checkpoint_parent": "experiments/runs/mini_editor_seed01_20261030005118/checkpoint",
16 + "labels_train": "experiments/runs/labels_mini_seed01_20261030005150/labels_train.pt",
17 + "labels_calib": "experiments/runs/labels_mini_seed01_20261030005150/labels_calib.pt",
18 + "finished_at": "2026-10-30T00:51:57-0400",
19 + "eval_train": {
20 + "huber": 0.0030732022628629334,
21 + "sign_acc": 0.6184336198662846,
22 + "harm_brier": 0.1599525462590521,
23 + "harm_ece": 0.04834684729576111,
24 + "n_labels": 4188,
25 + "oracle_opportunity_per_state": 0.1721912329537528,
26 + "regret_per_state": 0.1273838915143694,
27 + "captured_fraction": 0.2602184830828045,
28 + "top1_match": 0.0838095238095238,
29 + "n_audit_states": 525
30 + },
31 + "eval_calib": {
32 + "huber": 0.016214936043564422,
33 + "sign_acc": 0.7344290657439446,
34 + "harm_brier": 0.1263718605041504,
35 + "harm_ece": 0.08718162029981613,
36 + "n_labels": 1156,
37 + "oracle_opportunity_per_state": 0.2508483394499748,
38 + "regret_per_state": 0.11286073538564867,
39 + "captured_fraction": 0.550083785154353,
40 + "top1_match": 0.0,
41 + "n_audit_states": 124
42 + },
43 + "sweep": [
44 + {
45 + "lambda": 0.0,
46 + "kappa": 0.0,
47 + "loss": 2.059080786738835,
48 + "edits_per_token": 2.5673758865248226,
49 + "extra_ops_ratio": 1.4467924402115193,
50 + "loss_reduction": 0.2913353561509586,
51 + "reduction_per_ops": 0.20136637989922435
52 + },
53 + {
54 + "lambda": 0.3,
55 + "kappa": 0.0,
56 + "loss": 2.307212234388852,
57 + "edits_per_token": 0.04964539007092199,
58 + "extra_ops_ratio": 0.02571701665296622,
59 + "loss_reduction": 0.04320390850094169,
60 + "reduction_per_ops": 1.6799735787377388
61 + }
62 + ],
63 + "baseline_budget0": {
64 + "loss": 2.350416142889794,
65 + "edits_per_token": 0.0,
66 + "extra_ops_ratio": 0.0
67 + },
68 + "selected": {
69 + "lambda": 0.3,
70 + "kappa": 0.0,
71 + "loss": 2.307212234388852,
72 + "edits_per_token": 0.04964539007092199,
73 + "extra_ops_ratio": 0.02571701665296622,
74 + "loss_reduction": 0.04320390850094169,
75 + "reduction_per_ops": 1.6799735787377388
76 + },
77 + "checkpoint_sha256": "bb56e9c7b34c773d7cb314b50da687d910b37fa126243f4cfbe9b36c931da83e",
78 + "selection_rule": "max_loss_reduction_per_ops"
79 +}
\ No newline at end of file
added experiments/runs/controller_mini_seed01_20261030005154/metrics.jsonl +7 −0
@@ -0,0 +1,7 @@
1 +{"step": 0, "loss": 0.45115095376968384, "gain": 0.026630813255906105, "harm": 0.8158032894134521, "nll": 0.16618479788303375, "t": 0.54}
2 +{"step": 20, "loss": 0.13172240555286407, "gain": 0.005428161472082138, "harm": 0.5627143383026123, "nll": -1.5506292581558228, "t": 1.06}
3 +{"step": 29, "calib_huber": 0.01750746204366321, "calib_sign_acc": 0.5467128027681661, "calib_harm_brier": 0.1334372787739579, "calib_harm_ece": 0.09551648795604706, "calib_n_labels": 1156, "calib_oracle_opportunity_per_state": 0.2508483394499748, "calib_regret_per_state": 0.13811139137514175, "calib_captured_fraction": 0.4494227401386307, "calib_top1_match": 0.0, "calib_n_audit_states": 124, "t": 1.38}
4 +{"step": 40, "loss": 0.07597307860851288, "gain": 0.0038186139427125454, "harm": 0.4946993291378021, "nll": -1.751952052116394, "t": 1.71}
5 +{"step": 59, "loss": 0.053273364901542664, "gain": 0.0033322509843856096, "harm": 0.48466554284095764, "nll": -1.9239165782928467, "t": 2.29}
6 +{"step": 59, "calib_huber": 0.016214936043564422, "calib_sign_acc": 0.7344290657439446, "calib_harm_brier": 0.1263718605041504, "calib_harm_ece": 0.08718162029981613, "calib_n_labels": 1156, "calib_oracle_opportunity_per_state": 0.2508483394499748, "calib_regret_per_state": 0.11286073538564867, "calib_captured_fraction": 0.550083785154353, "calib_top1_match": 0.0, "calib_n_audit_states": 124, "t": 2.37}
7 +{"event": "restored_best_calibration_checkpoint", "step": 60, "score": -0.11286073538564867, "t": 2.37}
added experiments/runs/controller_v2s2_seed01_20261030004717/checkpoint/checkpoint.json +49 −0
@@ -0,0 +1,49 @@
1 +{
2 + "config": {
3 + "name": "tiny_p4v2",
4 + "vocab_size": 128,
5 + "d_model": 256,
6 + "n_layers": 6,
7 + "n_heads": 8,
8 + "n_kv_heads": 4,
9 + "ffn_width": 768,
10 + "max_seq_len": 320,
11 + "rope_theta": 10000.0,
12 + "norm_eps": 1e-05,
13 + "tie_embeddings": true,
14 + "init_std": 0.02,
15 + "use_workspace": true,
16 + "w": 64,
17 + "ffn_w": 192,
18 + "n_slots": 16,
19 + "n_editor_blocks": 2,
20 + "n_editor_heads": 2,
21 + "r_max": 8,
22 + "eta_max": 0.25,
23 + "clip_c": 1.0,
24 + "clip_eps": 1e-06,
25 + "readout_zero_init": false,
26 + "readout_init_std": 0.2,
27 + "readout_scale": 38.51451873779297,
28 + "controller_hidden": 0,
29 + "ctrl_input": "h",
30 + "ctrl_use_entropy": true,
31 + "lambda_cost": 0.3,
32 + "kappa_risk": 0.5,
33 + "dense_writes": false
34 + },
35 + "saved_at": "2026-10-30T00:50:30-0400",
36 + "run_id": "controller_v2s2_seed01_20261030004717",
37 + "parent": "experiments/runs/p4v2_prisme_seed02_20261030001749/checkpoint",
38 + "selected": {
39 + "lambda": 0.3,
40 + "kappa": 0.5,
41 + "loss": 0.4873708813215914,
42 + "edits_per_token": 0.020166073546856466,
43 + "extra_ops_ratio": 0.0021535491621649844,
44 + "loss_reduction": 3.514453892466962e-05,
45 + "reduction_per_ops": 0.016319357617700477
46 + },
47 + "locked": true,
48 + "weights_sha256": "809bef042a202eb089b8c2860bdd01a30faec25414290983bc7de06b17c27445"
49 +}
\ No newline at end of file
added experiments/runs/controller_v2s2_seed01_20261030004717/config.yaml +67 −0
@@ -0,0 +1,67 @@
1 +experiment:
2 + name: controller_v2s2
3 + seed: 1
4 + mode: prisme
5 + model: from_checkpoint
6 + checkpoint: experiments/runs/p4v2_prisme_seed02_20261030001749/checkpoint
7 + labels:
8 + train: experiments/runs/labels_v2s2_seed01_20261030004543/labels_train.pt
9 + calib: experiments/runs/labels_v2s2_seed01_20261030004543/labels_calib.pt
10 + train:
11 + steps: 2000
12 + lr: 0.001
13 + weight_decay: 0.01
14 + alpha: 1.0
15 + beta: 0.5
16 + gamma: 0.1
17 + huber_delta: 0.1
18 + log_every: 100
19 + eval_every: 250
20 + seq_len: 320
21 + lambda_grid:
22 + - 0.0
23 + - 0.01
24 + - 0.03
25 + - 0.1
26 + - 0.3
27 + kappa_grid:
28 + - 0.0
29 + - 0.5
30 + sweep_batches: 10
31 + max_budget: 8
32 + selection_rule: max_loss_reduction_per_ops
33 + model_overrides:
34 + ctrl_input: h
35 + ctrl_use_entropy: true
36 +model:
37 + name: tiny_p4v2
38 + vocab_size: 128
39 + d_model: 256
40 + n_layers: 6
41 + n_heads: 8
42 + n_kv_heads: 4
43 + ffn_width: 768
44 + max_seq_len: 320
45 + rope_theta: 10000.0
46 + norm_eps: 1.0e-05
47 + tie_embeddings: true
48 + init_std: 0.02
49 + use_workspace: true
50 + w: 64
51 + ffn_w: 192
52 + n_slots: 16
53 + n_editor_blocks: 2
54 + n_editor_heads: 2
55 + r_max: 8
56 + eta_max: 0.25
57 + clip_c: 1.0
58 + clip_eps: 1.0e-06
59 + readout_zero_init: false
60 + readout_init_std: 0.2
61 + readout_scale: 38.51451873779297
62 + controller_hidden: 0
63 + ctrl_input: h
64 + ctrl_use_entropy: true
65 + lambda_cost: 0.0
66 + kappa_risk: 0.0
67 + dense_writes: false
added experiments/runs/controller_v2s2_seed01_20261030004717/manifest.json +151 −0
@@ -0,0 +1,151 @@
1 +{
2 + "run_id": "controller_v2s2_seed01_20261030004717",
3 + "name": "controller_v2s2",
4 + "seed": 1,
5 + "code_commit": "unknown",
6 + "config_sha256": "8cdd361620875bda3cfcae976e78a784064bfff845c0433fa1fafec7f9805f54",
7 + "started_at": "2026-10-30T00:47:17-0400",
8 + "host": "M4M64c",
9 + "python": "3.12.14",
10 + "torch": "2.15.0",
11 + "command": "/Users/simon-pierreboucher/wp31/prisme-slm/training/train_controller.py --config configs/experiments/controller_v2s2.yaml --device mps",
12 + "state": "completed",
13 + "wall_seconds": 193.23,
14 + "device": "mps",
15 + "checkpoint_parent": "experiments/runs/p4v2_prisme_seed02_20261030001749/checkpoint",
16 + "labels_train": "experiments/runs/labels_v2s2_seed01_20261030004543/labels_train.pt",
17 + "labels_calib": "experiments/runs/labels_v2s2_seed01_20261030004543/labels_calib.pt",
18 + "finished_at": "2026-10-30T00:50:30-0400",
19 + "eval_train": {
20 + "huber": 0.0010070554593935765,
21 + "sign_acc": 0.4106531766106234,
22 + "harm_brier": 0.1394085687570071,
23 + "harm_ece": 0.02013043873012066,
24 + "n_labels": 26884,
25 + "oracle_opportunity_per_state": 0.01949981483372454,
26 + "regret_per_state": 0.013346665076057068,
27 + "captured_fraction": 0.3155491377808227,
28 + "top1_match": 0.059802712700369916,
29 + "n_audit_states": 1622
30 + },
31 + "eval_calib": {
32 + "huber": 0.001002937499951388,
33 + "sign_acc": 0.412654183385347,
34 + "harm_brier": 0.1532348860141019,
35 + "harm_ece": 0.03501247614622116,
36 + "n_labels": 26916,
37 + "oracle_opportunity_per_state": 0.020023393703449098,
38 + "regret_per_state": 0.013364227492033892,
39 + "captured_fraction": 0.3325693091809977,
40 + "top1_match": 0.06930091185410335,
41 + "n_audit_states": 1645
42 + },
43 + "sweep": [
44 + {
45 + "lambda": 0.0,
46 + "kappa": 0.0,
47 + "loss": 0.5010346970382817,
48 + "edits_per_token": 6.505338078291815,
49 + "extra_ops_ratio": 0.842747028753568,
50 + "loss_reduction": -0.013628671177765672,
51 + "reduction_per_ops": -0.016171722608055498
52 + },
53 + {
54 + "lambda": 0.0,
55 + "kappa": 0.5,
56 + "loss": 0.4934385713733388,
57 + "edits_per_token": 3.9359430604982206,
58 + "extra_ops_ratio": 0.4958741628119671,
59 + "loss_reduction": -0.006032545512822729,
60 + "reduction_per_ops": -0.012165476576988422
61 + },
62 + {
63 + "lambda": 0.01,
64 + "kappa": 0.0,
65 + "loss": 0.49886353244985127,
66 + "edits_per_token": 6.35349940688019,
67 + "extra_ops_ratio": 0.815659252632838,
68 + "loss_reduction": -0.011457506589335198,
69 + "reduction_per_ops": -0.01404692774875282
70 + },
71 + {
72 + "lambda": 0.01,
73 + "kappa": 0.5,
74 + "loss": 0.49234691180933976,
75 + "edits_per_token": 3.7265717674970342,
76 + "extra_ops_ratio": 0.4669636967343368,
77 + "loss_reduction": -0.004940885948823692,
78 + "reduction_per_ops": -0.010580878092616784
79 + },
80 + {
81 + "lambda": 0.03,
82 + "kappa": 0.0,
83 + "loss": 0.49679631949319536,
84 + "edits_per_token": 5.992289442467379,
85 + "extra_ops_ratio": 0.7534280869743012,
86 + "loss_reduction": -0.009390293632679292,
87 + "reduction_per_ops": -0.012463423908696925
88 + },
89 + {
90 + "lambda": 0.03,
91 + "kappa": 0.5,
92 + "loss": 0.4901708600642701,
93 + "edits_per_token": 3.0759193357058128,
94 + "extra_ops_ratio": 0.37601326142624464,
95 + "loss_reduction": -0.0027648342037540075,
96 + "reduction_per_ops": -0.007353023117500691
97 + },
98 + {
99 + "lambda": 0.1,
100 + "kappa": 0.0,
101 + "loss": 0.4935920529801085,
102 + "edits_per_token": 3.5118623962040334,
103 + "extra_ops_ratio": 0.4023556692320257,
104 + "loss_reduction": -0.00618602711959243,
105 + "reduction_per_ops": -0.015374524562807005
106 + },
107 + {
108 + "lambda": 0.1,
109 + "kappa": 0.5,
110 + "loss": 0.48869777602515896,
111 + "edits_per_token": 1.2004744958481612,
112 + "extra_ops_ratio": 0.1326800710648687,
113 + "loss_reduction": -0.0012917501646428908,
114 + "reduction_per_ops": -0.009735826596078174
115 + },
116 + {
117 + "lambda": 0.3,
118 + "kappa": 0.0,
119 + "loss": 0.4878548206949291,
120 + "edits_per_token": 0.6435349940688019,
121 + "extra_ops_ratio": 0.06894260369161694,
122 + "loss_reduction": -0.00044879483441301327,
123 + "reduction_per_ops": -0.006509687919830977
124 + },
125 + {
126 + "lambda": 0.3,
127 + "kappa": 0.5,
128 + "loss": 0.4873708813215914,
129 + "edits_per_token": 0.020166073546856466,
130 + "extra_ops_ratio": 0.0021535491621649844,
131 + "loss_reduction": 3.514453892466962e-05,
132 + "reduction_per_ops": 0.016319357617700477
133 + }
134 + ],
135 + "baseline_budget0": {
136 + "loss": 0.48740602586051607,
137 + "edits_per_token": 0.0,
138 + "extra_ops_ratio": 0.0
139 + },
140 + "selected": {
141 + "lambda": 0.3,
142 + "kappa": 0.5,
143 + "loss": 0.4873708813215914,
144 + "edits_per_token": 0.020166073546856466,
145 + "extra_ops_ratio": 0.0021535491621649844,
146 + "loss_reduction": 3.514453892466962e-05,
147 + "reduction_per_ops": 0.016319357617700477
148 + },
149 + "checkpoint_sha256": "809bef042a202eb089b8c2860bdd01a30faec25414290983bc7de06b17c27445",
150 + "selection_rule": "max_loss_reduction_per_ops"
151 +}
\ No newline at end of file
added experiments/runs/controller_v2s2_seed01_20261030004717/metrics.jsonl +30 −0
@@ -0,0 +1,30 @@
1 +{"step": 0, "loss": 9.83135986328125, "gain": 0.14342276751995087, "harm": 0.6234025359153748, "nll": 93.76235961914062, "t": 0.65}
2 +{"step": 100, "loss": 0.036009281873703, "gain": 0.0035093766637146473, "harm": 0.44542667269706726, "nll": -1.9021341800689697, "t": 8.75}
3 +{"step": 200, "loss": 0.1008618026971817, "gain": 0.003002654295414686, "harm": 0.6249974966049194, "nll": -2.1463959217071533, "t": 15.93}
4 +{"step": 249, "calib_huber": 0.01094102960476852, "calib_sign_acc": 0.6051790756427404, "calib_harm_brier": 0.19434303400473638, "calib_harm_ece": 0.09549890458583832, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.017705129490072604, "calib_captured_fraction": 0.11577778710794495, "calib_top1_match": 0.08206686930091185, "calib_n_audit_states": 1645, "t": 21.98}
5 +{"step": 300, "loss": 0.0225842148065567, "gain": 0.0018381704576313496, "harm": 0.5111701488494873, "nll": -2.3483903408050537, "t": 25.41}
6 +{"step": 400, "loss": -0.003825247287750244, "gain": 0.0019429537933319807, "harm": 0.47566935420036316, "nll": -2.4360287189483643, "t": 32.64}
7 +{"step": 499, "calib_huber": 0.0015247461168314978, "calib_sign_acc": 0.43884678258285037, "calib_harm_brier": 0.17252413148351367, "calib_harm_ece": 0.017360275611281395, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.01528218683863121, "calib_captured_fraction": 0.23678338123078502, "calib_top1_match": 0.06565349544072949, "calib_n_audit_states": 1645, "t": 41.7}
8 +{"step": 500, "loss": 0.060030609369277954, "gain": 0.0015156494919210672, "harm": 0.6202768683433533, "nll": -2.5162346363067627, "t": 41.75}
9 +{"step": 600, "loss": 0.13166989386081696, "gain": 0.00890484731644392, "harm": 0.4930534064769745, "nll": -1.2376166582107544, "t": 48.59}
10 +{"step": 700, "loss": 0.18399077653884888, "gain": 0.010280882008373737, "harm": 0.4504131078720093, "nll": -0.514966607093811, "t": 55.35}
11 +{"step": 749, "calib_huber": 0.004172799588667016, "calib_sign_acc": 0.5681007579135087, "calib_harm_brier": 0.16633579595993525, "calib_harm_ece": 0.0329064205288887, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.014974688251692474, "calib_captured_fraction": 0.2521403477616768, "calib_top1_match": 0.060790273556231005, "calib_n_audit_states": 1645, "t": 61.43}
12 +{"step": 800, "loss": 0.009334906935691833, "gain": 0.008806868456304073, "harm": 0.4048343598842621, "nll": -2.0188913345336914, "t": 64.89}
13 +{"step": 900, "loss": 0.028008311986923218, "gain": 0.002588255563750863, "harm": 0.4612581431865692, "nll": -2.0520901679992676, "t": 71.84}
14 +{"step": 999, "calib_huber": 0.0018093500733109814, "calib_sign_acc": 0.5104398870560262, "calib_harm_brier": 0.15887767702744113, "calib_harm_ece": 0.02854321524500847, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.014404436375232452, "calib_captured_fraction": 0.28061962979076627, "calib_top1_match": 0.08389057750759879, "calib_n_audit_states": 1645, "t": 81.28}
15 +{"step": 1000, "loss": -0.015576347708702087, "gain": 0.0019178803777322173, "harm": 0.40998339653015137, "nll": -2.2248592376708984, "t": 81.38}
16 +{"step": 1100, "loss": -0.06739456951618195, "gain": 0.0009511433891020715, "harm": 0.4489712417125702, "nll": -2.9283132553100586, "t": 88.41}
17 +{"step": 1200, "loss": -0.08154422044754028, "gain": 0.0012713662581518292, "harm": 0.46403712034225464, "nll": -3.148341417312622, "t": 95.24}
18 +{"step": 1249, "calib_huber": 0.0010799360305828226, "calib_sign_acc": 0.48424728785852283, "calib_harm_brier": 0.15457595845603858, "calib_harm_ece": 0.024711085483431816, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.014418237549918039, "calib_captured_fraction": 0.2799303772649465, "calib_top1_match": 0.07355623100303951, "calib_n_audit_states": 1645, "t": 101.23}
19 +{"step": 1300, "loss": -0.10255365073680878, "gain": 0.0005639962619170547, "harm": 0.4233770966529846, "nll": -3.148061990737915, "t": 104.82}
20 +{"step": 1400, "loss": -0.06496067345142365, "gain": 0.0011079360265284777, "harm": 0.4857495129108429, "nll": -3.0894336700439453, "t": 112.24}
21 +{"step": 1499, "calib_huber": 0.0010084362830958407, "calib_sign_acc": 0.6550750482984099, "calib_harm_brier": 0.15728407888289173, "calib_harm_ece": 0.022302670404314995, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.014015099197897868, "calib_captured_fraction": 0.3000637451640519, "calib_top1_match": 0.06930091185410335, "calib_n_audit_states": 1645, "t": 121.73}
22 +{"step": 1500, "loss": -0.09400822222232819, "gain": 0.0005882722325623035, "harm": 0.45213499665260315, "nll": -3.2066397666931152, "t": 121.82}
23 +{"step": 1600, "loss": -0.06699615716934204, "gain": 0.001017246162518859, "harm": 0.4579717516899109, "nll": -2.9699926376342773, "t": 128.88}
24 +{"step": 1700, "loss": -0.149380624294281, "gain": 0.0009441319270990789, "harm": 0.3887898921966553, "nll": -3.4471969604492188, "t": 135.87}
25 +{"step": 1749, "calib_huber": 0.0011374952911285398, "calib_sign_acc": 0.75817357705454, "calib_harm_brier": 0.15767023378786882, "calib_harm_ece": 0.019358007237315178, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.01419691836580317, "calib_captured_fraction": 0.29098340790463995, "calib_top1_match": 0.07537993920972644, "calib_n_audit_states": 1645, "t": 141.68}
26 +{"step": 1800, "loss": -0.07670816779136658, "gain": 0.0007706652977503836, "harm": 0.4373716115951538, "nll": -2.961646318435669, "t": 145.4}
27 +{"step": 1900, "loss": -0.06387145817279816, "gain": 0.0019935278687626123, "harm": 0.4595317840576172, "nll": -2.956308603286743, "t": 152.6}
28 +{"step": 1999, "loss": -0.08407506346702576, "gain": 0.0018677458865568042, "harm": 0.47809404134750366, "nll": -3.2498981952667236, "t": 159.38}
29 +{"step": 1999, "calib_huber": 0.001002937499951388, "calib_sign_acc": 0.412654183385347, "calib_harm_brier": 0.1532348860141019, "calib_harm_ece": 0.03501247614622116, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.020023393703449098, "calib_regret_per_state": 0.013364227492033892, "calib_captured_fraction": 0.3325693091809977, "calib_top1_match": 0.06930091185410335, "calib_n_audit_states": 1645, "t": 161.96}
30 +{"event": "restored_best_calibration_checkpoint", "step": 2000, "score": -0.013364227492033892, "t": 161.97}
added experiments/runs/controller_v2s3_seed01_20261030004720/checkpoint/checkpoint.json +49 −0
@@ -0,0 +1,49 @@
1 +{
2 + "config": {
3 + "name": "tiny_p4v2",
4 + "vocab_size": 128,
5 + "d_model": 256,
6 + "n_layers": 6,
7 + "n_heads": 8,
8 + "n_kv_heads": 4,
9 + "ffn_width": 768,
10 + "max_seq_len": 320,
11 + "rope_theta": 10000.0,
12 + "norm_eps": 1e-05,
13 + "tie_embeddings": true,
14 + "init_std": 0.02,
15 + "use_workspace": true,
16 + "w": 64,
17 + "ffn_w": 192,
18 + "n_slots": 16,
19 + "n_editor_blocks": 2,
20 + "n_editor_heads": 2,
21 + "r_max": 8,
22 + "eta_max": 0.25,
23 + "clip_c": 1.0,
24 + "clip_eps": 1e-06,
25 + "readout_zero_init": false,
26 + "readout_init_std": 0.2,
27 + "readout_scale": 38.98434829711914,
28 + "controller_hidden": 0,
29 + "ctrl_input": "h",
30 + "ctrl_use_entropy": true,
31 + "lambda_cost": 0.3,
32 + "kappa_risk": 0.0,
33 + "dense_writes": false
34 + },
35 + "saved_at": "2026-10-30T00:50:38-0400",
36 + "run_id": "controller_v2s3_seed01_20261030004720",
37 + "parent": "experiments/runs/p4v2_prisme_seed03_20261030001749/checkpoint",
38 + "selected": {
39 + "lambda": 0.3,
40 + "kappa": 0.0,
41 + "loss": 0.4887911139292508,
42 + "edits_per_token": 4.198102016607355,
43 + "extra_ops_ratio": 0.4771132757655242,
44 + "loss_reduction": -0.001385088068734719,
45 + "reduction_per_ops": -0.002903059166635757
46 + },
47 + "locked": true,
48 + "weights_sha256": "34ec25bc13935a5276c68155f7bd978233d297fd543a288cd9612686fa7e6c75"
49 +}
\ No newline at end of file
added experiments/runs/controller_v2s3_seed01_20261030004720/config.yaml +67 −0
@@ -0,0 +1,67 @@
1 +experiment:
2 + name: controller_v2s3
3 + seed: 1
4 + mode: prisme
5 + model: from_checkpoint
6 + checkpoint: experiments/runs/p4v2_prisme_seed03_20261030001749/checkpoint
7 + labels:
8 + train: experiments/runs/labels_v2s3_seed01_20261030004544/labels_train.pt
9 + calib: experiments/runs/labels_v2s3_seed01_20261030004544/labels_calib.pt
10 + train:
11 + steps: 2000
12 + lr: 0.001
13 + weight_decay: 0.01
14 + alpha: 1.0
15 + beta: 0.5
16 + gamma: 0.1
17 + huber_delta: 0.1
18 + log_every: 100
19 + eval_every: 250
20 + seq_len: 320
21 + lambda_grid:
22 + - 0.0
23 + - 0.01
24 + - 0.03
25 + - 0.1
26 + - 0.3
27 + kappa_grid:
28 + - 0.0
29 + - 0.5
30 + sweep_batches: 10
31 + max_budget: 8
32 + selection_rule: max_loss_reduction_per_ops
33 + model_overrides:
34 + ctrl_input: h
35 + ctrl_use_entropy: true
36 +model:
37 + name: tiny_p4v2
38 + vocab_size: 128
39 + d_model: 256
40 + n_layers: 6
41 + n_heads: 8
42 + n_kv_heads: 4
43 + ffn_width: 768
44 + max_seq_len: 320
45 + rope_theta: 10000.0
46 + norm_eps: 1.0e-05
47 + tie_embeddings: true
48 + init_std: 0.02
49 + use_workspace: true
50 + w: 64
51 + ffn_w: 192
52 + n_slots: 16
53 + n_editor_blocks: 2
54 + n_editor_heads: 2
55 + r_max: 8
56 + eta_max: 0.25
57 + clip_c: 1.0
58 + clip_eps: 1.0e-06
59 + readout_zero_init: false
60 + readout_init_std: 0.2
61 + readout_scale: 38.98434829711914
62 + controller_hidden: 0
63 + ctrl_input: h
64 + ctrl_use_entropy: true
65 + lambda_cost: 0.0
66 + kappa_risk: 0.0
67 + dense_writes: false
added experiments/runs/controller_v2s3_seed01_20261030004720/manifest.json +151 −0
@@ -0,0 +1,151 @@
1 +{
2 + "run_id": "controller_v2s3_seed01_20261030004720",
3 + "name": "controller_v2s3",
4 + "seed": 1,
5 + "code_commit": "unknown",
6 + "config_sha256": "e56154e7ec9666ab6d7e4a94427e0c31397f8975c9b339b0344bd8cbb90bb651",
7 + "started_at": "2026-10-30T00:47:20-0400",
8 + "host": "MacStudnpierre6",
9 + "python": "3.12.13",
10 + "torch": "2.15.0",
11 + "command": "/Users/simon-pierreboucher/wp31/prisme-slm/training/train_controller.py --config configs/experiments/controller_v2s3.yaml --device mps",
12 + "state": "completed",
13 + "wall_seconds": 197.97,
14 + "device": "mps",
15 + "checkpoint_parent": "experiments/runs/p4v2_prisme_seed03_20261030001749/checkpoint",
16 + "labels_train": "experiments/runs/labels_v2s3_seed01_20261030004544/labels_train.pt",
17 + "labels_calib": "experiments/runs/labels_v2s3_seed01_20261030004544/labels_calib.pt",
18 + "finished_at": "2026-10-30T00:50:38-0400",
19 + "eval_train": {
20 + "huber": 0.007815325425114948,
21 + "sign_acc": 0.48322422258592473,
22 + "harm_brier": 0.19074533085368287,
23 + "harm_ece": 0.061203483492136,
24 + "n_labels": 26884,
25 + "oracle_opportunity_per_state": 0.019982138044590134,
26 + "regret_per_state": 0.01430556020078177,
27 + "captured_fraction": 0.28408260573223354,
28 + "top1_match": 0.1029593094944513,
29 + "n_audit_states": 1622
30 + },
31 + "eval_calib": {
32 + "huber": 0.007724088622620563,
33 + "sign_acc": 0.5044954673799971,
34 + "harm_brier": 0.19917026808329547,
35 + "harm_ece": 0.08023828268051147,
36 + "n_labels": 26916,
37 + "oracle_opportunity_per_state": 0.0200653997960424,
38 + "regret_per_state": 0.013944871664771917,
39 + "captured_fraction": 0.30502896495875775,
40 + "top1_match": 0.08936170212765958,
41 + "n_audit_states": 1645
42 + },
43 + "sweep": [
44 + {
45 + "lambda": 0.0,
46 + "kappa": 0.0,
47 + "loss": 0.4941370365599862,
48 + "edits_per_token": 5.279359430604982,
49 + "extra_ops_ratio": 0.6436597794683236,
50 + "loss_reduction": -0.00673101069947013,
51 + "reduction_per_ops": -0.010457404539134145
52 + },
53 + {
54 + "lambda": 0.0,
55 + "kappa": 0.5,
56 + "loss": 0.49054730687984227,
57 + "edits_per_token": 2.98220640569395,
58 + "extra_ops_ratio": 0.34909897257848993,
59 + "loss_reduction": -0.003141281019326203,
60 + "reduction_per_ops": -0.00899825340683273
61 + },
62 + {
63 + "lambda": 0.01,
64 + "kappa": 0.0,
65 + "loss": 0.4926429264344637,
66 + "edits_per_token": 5.233096085409253,
67 + "extra_ops_ratio": 0.6357123396433404,
68 + "loss_reduction": -0.005236900573947656,
69 + "reduction_per_ops": -0.008237846345543273
70 + },
71 + {
72 + "lambda": 0.01,
73 + "kappa": 0.5,
74 + "loss": 0.4901237397290068,
75 + "edits_per_token": 2.9187425860023724,
76 + "extra_ops_ratio": 0.3409675064177417,
77 + "loss_reduction": -0.0027177138684907343,
78 + "reduction_per_ops": -0.007970594902263456
79 + },
80 + {
81 + "lambda": 0.03,
82 + "kappa": 0.0,
83 + "loss": 0.49175829451844927,
84 + "edits_per_token": 5.177935943060498,
85 + "extra_ops_ratio": 0.6265161045929716,
86 + "loss_reduction": -0.0043522686579332,
87 + "reduction_per_ops": -0.006946778584024965
88 + },
89 + {
90 + "lambda": 0.03,
91 + "kappa": 0.5,
92 + "loss": 0.4906825974035659,
93 + "edits_per_token": 2.820284697508897,
94 + "extra_ops_ratio": 0.3285812454857153,
95 + "loss_reduction": -0.0032765715430498332,
96 + "reduction_per_ops": -0.009971876326070711
97 + },
98 + {
99 + "lambda": 0.1,
100 + "kappa": 0.0,
101 + "loss": 0.49028573992130736,
102 + "edits_per_token": 4.982799525504152,
103 + "extra_ops_ratio": 0.5937093098958334,
104 + "loss_reduction": -0.002879714060791294,
105 + "reduction_per_ops": -0.004850377133713031
106 + },
107 + {
108 + "lambda": 0.1,
109 + "kappa": 0.5,
110 + "loss": 0.4904432398568693,
111 + "edits_per_token": 2.4905100830367735,
112 + "extra_ops_ratio": 0.285856970971448,
113 + "loss_reduction": -0.003037213996353205,
114 + "reduction_per_ops": -0.010624942907747275
115 + },
116 + {
117 + "lambda": 0.3,
118 + "kappa": 0.0,
119 + "loss": 0.4887911139292508,
120 + "edits_per_token": 4.198102016607355,
121 + "extra_ops_ratio": 0.4771132757655242,
122 + "loss_reduction": -0.001385088068734719,
123 + "reduction_per_ops": -0.002903059166635757
124 + },
125 + {
126 + "lambda": 0.3,
127 + "kappa": 0.5,
128 + "loss": 0.48979161444649294,
129 + "edits_per_token": 1.568208778173191,
130 + "extra_ops_ratio": 0.17398185096460164,
131 + "loss_reduction": -0.002385588585976872,
132 + "reduction_per_ops": -0.0137117094268772
133 + }
134 + ],
135 + "baseline_budget0": {
136 + "loss": 0.48740602586051607,
137 + "edits_per_token": 0.0,
138 + "extra_ops_ratio": 0.0
139 + },
140 + "selected": {
141 + "lambda": 0.3,
142 + "kappa": 0.0,
143 + "loss": 0.4887911139292508,
144 + "edits_per_token": 4.198102016607355,
145 + "extra_ops_ratio": 0.4771132757655242,
146 + "loss_reduction": -0.001385088068734719,
147 + "reduction_per_ops": -0.002903059166635757
148 + },
149 + "checkpoint_sha256": "34ec25bc13935a5276c68155f7bd978233d297fd543a288cd9612686fa7e6c75",
150 + "selection_rule": "max_loss_reduction_per_ops"
151 +}
\ No newline at end of file
added experiments/runs/controller_v2s3_seed01_20261030004720/metrics.jsonl +30 −0
@@ -0,0 +1,30 @@
1 +{"step": 0, "loss": 0.5471222400665283, "gain": 0.08188310265541077, "harm": 0.7777696847915649, "nll": 0.763542890548706, "t": 1.29}
2 +{"step": 100, "loss": 0.10401472449302673, "gain": 0.003353619482368231, "harm": 0.5848105549812317, "nll": -1.917441725730896, "t": 10.42}
3 +{"step": 200, "loss": 0.06753906607627869, "gain": 0.002697773976251483, "harm": 0.5347252488136292, "nll": -2.0252132415771484, "t": 17.76}
4 +{"step": 249, "calib_huber": 0.007724088622620563, "calib_sign_acc": 0.5044954673799971, "calib_harm_brier": 0.19917026808329547, "calib_harm_ece": 0.08023828268051147, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.013944871664771917, "calib_captured_fraction": 0.30502896495875775, "calib_top1_match": 0.08936170212765958, "calib_n_audit_states": 1645, "t": 23.98}
5 +{"step": 300, "loss": 0.03441177308559418, "gain": 0.0029057699721306562, "harm": 0.5025796890258789, "nll": -2.197838306427002, "t": 27.54}
6 +{"step": 400, "loss": 0.10745498538017273, "gain": 0.005344087723642588, "harm": 0.5453636646270752, "nll": -1.7057093381881714, "t": 34.69}
7 +{"step": 499, "calib_huber": 0.0029885766587333207, "calib_sign_acc": 0.4259919750334374, "calib_harm_brier": 0.1813072681356139, "calib_harm_ece": 0.0780940130352974, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.014313748107492743, "calib_captured_fraction": 0.2866452573590925, "calib_top1_match": 0.0547112462006079, "calib_n_audit_states": 1645, "t": 44.21}
8 +{"step": 500, "loss": 0.07936693727970123, "gain": 0.002874889876693487, "harm": 0.5229513049125671, "nll": -1.849835991859436, "t": 44.29}
9 +{"step": 600, "loss": 0.006543099880218506, "gain": 0.001932281069457531, "harm": 0.48412805795669556, "nll": -2.3745319843292236, "t": 51.1}
10 +{"step": 700, "loss": 0.06510673463344574, "gain": 0.00877080112695694, "harm": 0.4454459547996521, "nll": -1.6638703346252441, "t": 58.48}
11 +{"step": 749, "calib_huber": 0.002168223534212865, "calib_sign_acc": 0.49041462327240304, "calib_harm_brier": 0.16566909980518818, "calib_harm_ece": 0.042970336973667145, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.015758255694774872, "calib_captured_fraction": 0.21465528447218118, "calib_top1_match": 0.041945288753799395, "calib_n_audit_states": 1645, "t": 64.88}
12 +{"step": 800, "loss": 0.06329897046089172, "gain": 0.0020338173490017653, "harm": 0.5008138418197632, "nll": -1.8914177417755127, "t": 68.52}
13 +{"step": 900, "loss": -0.044697314500808716, "gain": 0.001048971083946526, "harm": 0.44541195034980774, "nll": -2.6845223903656006, "t": 75.49}
14 +{"step": 999, "calib_huber": 0.0013120168749990958, "calib_sign_acc": 0.3987591023926289, "calib_harm_brier": 0.1603591404196447, "calib_harm_ece": 0.026692669838666916, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.01603862224741185, "calib_captured_fraction": 0.2006826472216502, "calib_top1_match": 0.04012158054711246, "calib_n_audit_states": 1645, "t": 85.15}
15 +{"step": 1000, "loss": -0.02209603786468506, "gain": 0.0009799482068046927, "harm": 0.5161257386207581, "nll": -2.8113887310028076, "t": 85.23}
16 +{"step": 1100, "loss": -0.10085465013980865, "gain": 0.0010887801181524992, "harm": 0.434581995010376, "nll": -3.1923441886901855, "t": 92.09}
17 +{"step": 1200, "loss": -0.10687923431396484, "gain": 0.0008391870651394129, "harm": 0.4322952926158905, "nll": -3.2386605739593506, "t": 99.33}
18 +{"step": 1249, "calib_huber": 0.0011769543454628994, "calib_sign_acc": 0.36168078466339726, "calib_harm_brier": 0.15637497314462195, "calib_harm_ece": 0.017439356073737144, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.01591340959253282, "calib_captured_fraction": 0.20692287448608404, "calib_top1_match": 0.044984802431610946, "calib_n_audit_states": 1645, "t": 105.4}
19 +{"step": 1300, "loss": -0.10567505657672882, "gain": 0.0011399664217606187, "harm": 0.4573471248149872, "nll": -3.3548858165740967, "t": 109.09}
20 +{"step": 1400, "loss": -0.1049814522266388, "gain": 0.0012958016013726592, "harm": 0.45608627796173096, "nll": -3.3432040214538574, "t": 116.11}
21 +{"step": 1499, "calib_huber": 0.001097509661557363, "calib_sign_acc": 0.5441744687174914, "calib_harm_brier": 0.15651853762368848, "calib_harm_ece": 0.01747792400419712, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.015713384810914385, "calib_captured_fraction": 0.21689151621022695, "calib_top1_match": 0.0486322188449848, "calib_n_audit_states": 1645, "t": 125.65}
22 +{"step": 1500, "loss": -0.1010504961013794, "gain": 0.0016412652330473065, "harm": 0.4739455282688141, "nll": -3.3966450691223145, "t": 125.7}
23 +{"step": 1600, "loss": -0.06810225546360016, "gain": 0.0016716807149350643, "harm": 0.47334086894989014, "nll": -3.064443826675415, "t": 132.59}
24 +{"step": 1700, "loss": -0.10438083112239838, "gain": 0.0014919735258445144, "harm": 0.4370725750923157, "nll": -3.2440907955169678, "t": 139.85}
25 +{"step": 1749, "calib_huber": 0.001152203617478253, "calib_sign_acc": 0.6283251597562788, "calib_harm_brier": 0.15444377882729704, "calib_harm_ece": 0.021278375759720802, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.01586139586196482, "calib_captured_fraction": 0.20951508451412748, "calib_top1_match": 0.0547112462006079, "calib_n_audit_states": 1645, "t": 145.83}
26 +{"step": 1800, "loss": -0.12716279923915863, "gain": 0.0009756564977578819, "harm": 0.42070525884628296, "nll": -3.384910821914673, "t": 149.48}
27 +{"step": 1900, "loss": -0.11546054482460022, "gain": 0.0011908027809113264, "harm": 0.4737454354763031, "nll": -3.535240411758423, "t": 156.56}
28 +{"step": 1999, "loss": -0.14543230831623077, "gain": 0.001172536169178784, "harm": 0.42771992087364197, "nll": -3.6046481132507324, "t": 163.78}
29 +{"step": 1999, "calib_huber": 0.0010707473310660352, "calib_sign_acc": 0.38646158418784365, "calib_harm_brier": 0.15663047594378546, "calib_harm_ece": 0.018951764330267906, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.0200653997960424, "calib_regret_per_state": 0.01485436325015268, "calib_captured_fraction": 0.2597026024329462, "calib_top1_match": 0.06504559270516717, "calib_n_audit_states": 1645, "t": 166.28}
30 +{"event": "restored_best_calibration_checkpoint", "step": 250, "score": -0.013944871664771917, "t": 166.29}
added experiments/runs/controller_v2s4_seed01_20261030004719/checkpoint/checkpoint.json +49 −0
@@ -0,0 +1,49 @@
1 +{
2 + "config": {
3 + "name": "tiny_p4v2",
4 + "vocab_size": 128,
5 + "d_model": 256,
6 + "n_layers": 6,
7 + "n_heads": 8,
8 + "n_kv_heads": 4,
9 + "ffn_width": 768,
10 + "max_seq_len": 320,
11 + "rope_theta": 10000.0,
12 + "norm_eps": 1e-05,
13 + "tie_embeddings": true,
14 + "init_std": 0.02,
15 + "use_workspace": true,
16 + "w": 64,
17 + "ffn_w": 192,
18 + "n_slots": 16,
19 + "n_editor_blocks": 2,
20 + "n_editor_heads": 2,
21 + "r_max": 8,
22 + "eta_max": 0.25,
23 + "clip_c": 1.0,
24 + "clip_eps": 1e-06,
25 + "readout_zero_init": false,
26 + "readout_init_std": 0.2,
27 + "readout_scale": 39.0492057800293,
28 + "controller_hidden": 0,
29 + "ctrl_input": "h",
30 + "ctrl_use_entropy": true,
31 + "lambda_cost": 0.3,
32 + "kappa_risk": 0.5,
33 + "dense_writes": false
34 + },
35 + "saved_at": "2026-10-30T00:50:56-0400",
36 + "run_id": "controller_v2s4_seed01_20261030004719",
37 + "parent": "experiments/runs/p4v2_prisme_seed04_20261030001749/checkpoint",
38 + "selected": {
39 + "lambda": 0.3,
40 + "kappa": 0.5,
41 + "loss": 0.48625035585740567,
42 + "edits_per_token": 0.1951364175563464,
43 + "extra_ops_ratio": 0.02227253153089385,
44 + "loss_reduction": 0.0011556700031103984,
45 + "reduction_per_ops": 0.05188768063959807
46 + },
47 + "locked": true,
48 + "weights_sha256": "ce86e3a12b5d5af217859afaccfe238677ad488c5ee0b4f219200742f99a7fa2"
49 +}
\ No newline at end of file
added experiments/runs/controller_v2s4_seed01_20261030004719/config.yaml +67 −0
@@ -0,0 +1,67 @@
1 +experiment:
2 + name: controller_v2s4
3 + seed: 1
4 + mode: prisme
5 + model: from_checkpoint
6 + checkpoint: experiments/runs/p4v2_prisme_seed04_20261030001749/checkpoint
7 + labels:
8 + train: experiments/runs/labels_v2s4_seed01_20261030004544/labels_train.pt
9 + calib: experiments/runs/labels_v2s4_seed01_20261030004544/labels_calib.pt
10 + train:
11 + steps: 2000
12 + lr: 0.001
13 + weight_decay: 0.01
14 + alpha: 1.0
15 + beta: 0.5
16 + gamma: 0.1
17 + huber_delta: 0.1
18 + log_every: 100
19 + eval_every: 250
20 + seq_len: 320
21 + lambda_grid:
22 + - 0.0
23 + - 0.01
24 + - 0.03
25 + - 0.1
26 + - 0.3
27 + kappa_grid:
28 + - 0.0
29 + - 0.5
30 + sweep_batches: 10
31 + max_budget: 8
32 + selection_rule: max_loss_reduction_per_ops
33 + model_overrides:
34 + ctrl_input: h
35 + ctrl_use_entropy: true
36 +model:
37 + name: tiny_p4v2
38 + vocab_size: 128
39 + d_model: 256
40 + n_layers: 6
41 + n_heads: 8
42 + n_kv_heads: 4
43 + ffn_width: 768
44 + max_seq_len: 320
45 + rope_theta: 10000.0
46 + norm_eps: 1.0e-05
47 + tie_embeddings: true
48 + init_std: 0.02
49 + use_workspace: true
50 + w: 64
51 + ffn_w: 192
52 + n_slots: 16
53 + n_editor_blocks: 2
54 + n_editor_heads: 2
55 + r_max: 8
56 + eta_max: 0.25
57 + clip_c: 1.0
58 + clip_eps: 1.0e-06
59 + readout_zero_init: false
60 + readout_init_std: 0.2
61 + readout_scale: 39.0492057800293
62 + controller_hidden: 0
63 + ctrl_input: h
64 + ctrl_use_entropy: true
65 + lambda_cost: 0.0
66 + kappa_risk: 0.0
67 + dense_writes: false
added experiments/runs/controller_v2s4_seed01_20261030004719/manifest.json +151 −0
@@ -0,0 +1,151 @@
1 +{
2 + "run_id": "controller_v2s4_seed01_20261030004719",
3 + "name": "controller_v2s4",
4 + "seed": 1,
5 + "code_commit": "unknown",
6 + "config_sha256": "e2fd878b8eb564a3b6415335bf21da1ff862fdf95dfbdb47fefa2577cda90381",
7 + "started_at": "2026-10-30T00:47:19-0400",
8 + "host": "M5M36",
9 + "python": "3.12.14",
10 + "torch": "2.15.0",
11 + "command": "/Users/simon-pierreboucher/wp31/prisme-slm/training/train_controller.py --config configs/experiments/controller_v2s4.yaml --device mps",
12 + "state": "completed",
13 + "wall_seconds": 217.1,
14 + "device": "mps",
15 + "checkpoint_parent": "experiments/runs/p4v2_prisme_seed04_20261030001749/checkpoint",
16 + "labels_train": "experiments/runs/labels_v2s4_seed01_20261030004544/labels_train.pt",
17 + "labels_calib": "experiments/runs/labels_v2s4_seed01_20261030004544/labels_calib.pt",
18 + "finished_at": "2026-10-30T00:50:56-0400",
19 + "eval_train": {
20 + "huber": 0.0012207997952024066,
21 + "sign_acc": 0.6253905668799286,
22 + "harm_brier": 0.14596635829549232,
23 + "harm_ece": 0.06639833003282547,
24 + "n_labels": 26884,
25 + "oracle_opportunity_per_state": 0.017455400138836164,
26 + "regret_per_state": 0.012565521732358545,
27 + "captured_fraction": 0.28013556650576177,
28 + "top1_match": 0.059802712700369916,
29 + "n_audit_states": 1622
30 + },
31 + "eval_calib": {
32 + "huber": 0.0012847311736074417,
33 + "sign_acc": 0.6326348640213999,
34 + "harm_brier": 0.15521432896712006,
35 + "harm_ece": 0.07186548411846161,
36 + "n_labels": 26916,
37 + "oracle_opportunity_per_state": 0.023318092004144083,
38 + "regret_per_state": 0.011166786567780747,
39 + "captured_fraction": 0.52111062235297,
40 + "top1_match": 0.06018237082066869,
41 + "n_audit_states": 1645
42 + },
43 + "sweep": [
44 + {
45 + "lambda": 0.0,
46 + "kappa": 0.0,
47 + "loss": 0.49249181147854804,
48 + "edits_per_token": 3.6684460260972718,
49 + "extra_ops_ratio": 0.470991978583104,
50 + "loss_reduction": -0.005085785618031968,
51 + "reduction_per_ops": -0.010798030219817446
52 + },
53 + {
54 + "lambda": 0.0,
55 + "kappa": 0.5,
56 + "loss": 0.4869922624364018,
57 + "edits_per_token": 1.9780545670225385,
58 + "extra_ops_ratio": 0.23909690739279824,
59 + "loss_reduction": 0.00041376342411425515,
60 + "reduction_per_ops": 0.0017305260391114452
61 + },
62 + {
63 + "lambda": 0.01,
64 + "kappa": 0.0,
65 + "loss": 0.4923982970937998,
66 + "edits_per_token": 3.5539739027283512,
67 + "extra_ops_ratio": 0.4550833967939673,
68 + "loss_reduction": -0.004992271233283718,
69 + "reduction_per_ops": -0.010970013998431808
70 + },
71 + {
72 + "lambda": 0.01,
73 + "kappa": 0.5,
74 + "loss": 0.48735612482362795,
75 + "edits_per_token": 1.8612099644128113,
76 + "extra_ops_ratio": 0.22442850155903918,
77 + "loss_reduction": 4.990103688812253e-05,
78 + "reduction_per_ops": 0.00022234714638058275
79 + },
80 + {
81 + "lambda": 0.03,
82 + "kappa": 0.0,
83 + "loss": 0.4913539094574228,
84 + "edits_per_token": 3.295373665480427,
85 + "extra_ops_ratio": 0.4207166149098678,
86 + "loss_reduction": -0.003947883596906732,
87 + "reduction_per_ops": -0.009383712116414766
88 + },
89 + {
90 + "lambda": 0.03,
91 + "kappa": 0.5,
92 + "loss": 0.48911659177370626,
93 + "edits_per_token": 1.6779359430604983,
94 + "extra_ops_ratio": 0.20157079821096763,
95 + "loss_reduction": -0.0017105659131901874,
96 + "reduction_per_ops": -0.008486179190499004
97 + },
98 + {
99 + "lambda": 0.1,
100 + "kappa": 0.0,
101 + "loss": 0.4855112390303244,
102 + "edits_per_token": 2.543890865954923,
103 + "extra_ops_ratio": 0.3220053658083806,
104 + "loss_reduction": 0.0018947868301916748,
105 + "reduction_per_ops": 0.0058843330931299676
106 + },
107 + {
108 + "lambda": 0.1,
109 + "kappa": 0.5,
110 + "loss": 0.486984067398741,
111 + "edits_per_token": 1.0504151838671412,
112 + "extra_ops_ratio": 0.12493916508141786,
113 + "loss_reduction": 0.0004219584617750627,
114 + "reduction_per_ops": 0.0033773113618943205
115 + },
116 + {
117 + "lambda": 0.3,
118 + "kappa": 0.0,
119 + "loss": 0.4817445196149471,
120 + "edits_per_token": 1.0581257413997627,
121 + "extra_ops_ratio": 0.1268179996440702,
122 + "loss_reduction": 0.005661506245568992,
123 + "reduction_per_ops": 0.04464276570722361
124 + },
125 + {
126 + "lambda": 0.3,
127 + "kappa": 0.5,
128 + "loss": 0.48625035585740567,
129 + "edits_per_token": 0.1951364175563464,
130 + "extra_ops_ratio": 0.02227253153089385,
131 + "loss_reduction": 0.0011556700031103984,
132 + "reduction_per_ops": 0.05188768063959807
133 + }
134 + ],
135 + "baseline_budget0": {
136 + "loss": 0.48740602586051607,
137 + "edits_per_token": 0.0,
138 + "extra_ops_ratio": 0.0
139 + },
140 + "selected": {
141 + "lambda": 0.3,
142 + "kappa": 0.5,
143 + "loss": 0.48625035585740567,
144 + "edits_per_token": 0.1951364175563464,
145 + "extra_ops_ratio": 0.02227253153089385,
146 + "loss_reduction": 0.0011556700031103984,
147 + "reduction_per_ops": 0.05188768063959807
148 + },
149 + "checkpoint_sha256": "ce86e3a12b5d5af217859afaccfe238677ad488c5ee0b4f219200742f99a7fa2",
150 + "selection_rule": "max_loss_reduction_per_ops"
151 +}
\ No newline at end of file
added experiments/runs/controller_v2s4_seed01_20261030004719/metrics.jsonl +30 −0
@@ -0,0 +1,30 @@
1 +{"step": 0, "loss": 0.43164244294166565, "gain": 0.027255289256572723, "harm": 0.7605108022689819, "nll": 0.24131746590137482, "t": 1.48}
2 +{"step": 100, "loss": 0.2405385822057724, "gain": 0.004398028831928968, "harm": 0.4923504889011383, "nll": -0.10034700483083725, "t": 11.1}
3 +{"step": 200, "loss": 0.0673132985830307, "gain": 0.0024042660370469093, "harm": 0.5429790616035461, "nll": -2.065804958343506, "t": 19.72}
4 +{"step": 249, "calib_huber": 0.002094036332741413, "calib_sign_acc": 0.4631817506315946, "calib_harm_brier": 0.17309671079692593, "calib_harm_ece": 0.048935920000076294, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.014540166550494255, "calib_captured_fraction": 0.3764426974595442, "calib_top1_match": 0.07173252279635259, "calib_n_audit_states": 1645, "t": 26.52}
5 +{"step": 300, "loss": 0.005617067217826843, "gain": 0.0021593354176729918, "harm": 0.48432907462120056, "nll": -2.387068033218384, "t": 30.22}
6 +{"step": 400, "loss": -0.01675678789615631, "gain": 0.001087905722670257, "harm": 0.4958101809024811, "nll": -2.6574976444244385, "t": 38.24}
7 +{"step": 499, "calib_huber": 0.001623512151461332, "calib_sign_acc": 0.5320627136275821, "calib_harm_brier": 0.15747589369836365, "calib_harm_ece": 0.029524514451622963, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.014382726008406526, "calib_captured_fraction": 0.38319455957844095, "calib_top1_match": 0.0668693009118541, "calib_n_audit_states": 1645, "t": 48.85}
8 +{"step": 500, "loss": -0.02444365620613098, "gain": 0.0012579343747347593, "harm": 0.5083814263343811, "nll": -2.7989227771759033, "t": 48.92}
9 +{"step": 600, "loss": -0.011498332023620605, "gain": 0.001862199860624969, "harm": 0.4400762915611267, "nll": -2.333986759185791, "t": 56.83}
10 +{"step": 700, "loss": 0.02981466054916382, "gain": 0.0019247914897277951, "harm": 0.4267236590385437, "nll": -1.854719638824463, "t": 65.16}
11 +{"step": 749, "calib_huber": 0.0018654652089345578, "calib_sign_acc": 0.4522588794768911, "calib_harm_brier": 0.16276104536324382, "calib_harm_ece": 0.09062658995389938, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.012852072280953359, "calib_captured_fraction": 0.4488368825944553, "calib_top1_match": 0.0662613981762918, "calib_n_audit_states": 1645, "t": 71.46}
12 +{"step": 800, "loss": -0.009733706712722778, "gain": 0.0018562733894214034, "harm": 0.3807581663131714, "nll": -2.01969051361084, "t": 75.0}
13 +{"step": 900, "loss": -0.04677686095237732, "gain": 0.0012354672653600574, "harm": 0.4526765048503876, "nll": -2.7435057163238525, "t": 82.81}
14 +{"step": 999, "calib_huber": 0.0014452590856372841, "calib_sign_acc": 0.48970872343587457, "calib_harm_brier": 0.14695231233172473, "calib_harm_ece": 0.03946586325764656, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.011811167273477943, "calib_captured_fraction": 0.493476255631084, "calib_top1_match": 0.05531914893617021, "calib_n_audit_states": 1645, "t": 93.47}
15 +{"step": 1000, "loss": -0.0617438405752182, "gain": 0.0008112452342174947, "harm": 0.4560224115848541, "nll": -2.905662775039673, "t": 93.57}
16 +{"step": 1100, "loss": -0.10207153856754303, "gain": 0.001203094725497067, "harm": 0.3867413103580475, "nll": -2.9664528369903564, "t": 101.64}
17 +{"step": 1200, "loss": -0.09816358983516693, "gain": 0.001390167511999607, "harm": 0.41572046279907227, "nll": -3.0741398334503174, "t": 109.49}
18 +{"step": 1249, "calib_huber": 0.0012847311736074417, "calib_sign_acc": 0.6326348640213999, "calib_harm_brier": 0.15521432896712006, "calib_harm_ece": 0.07186548411846161, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.011166786567780747, "calib_captured_fraction": 0.52111062235297, "calib_top1_match": 0.06018237082066869, "calib_n_audit_states": 1645, "t": 116.26}
19 +{"step": 1300, "loss": -0.11031846702098846, "gain": 0.0005297428579069674, "harm": 0.4200277030467987, "nll": -3.208620548248291, "t": 120.53}
20 +{"step": 1400, "loss": -0.08227270841598511, "gain": 0.0012564704520627856, "harm": 0.41779661178588867, "nll": -2.9242746829986572, "t": 128.5}
21 +{"step": 1499, "calib_huber": 0.0011141103800169725, "calib_sign_acc": 0.5541313716748403, "calib_harm_brier": 0.14586461934988063, "calib_harm_ece": 0.04005922004580498, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.011563696035135844, "calib_captured_fraction": 0.5040890981525956, "calib_top1_match": 0.0668693009118541, "calib_n_audit_states": 1645, "t": 139.64}
22 +{"step": 1500, "loss": -0.11083218455314636, "gain": 0.0005095681408420205, "harm": 0.46461519598960876, "nll": -3.436493396759033, "t": 139.74}
23 +{"step": 1600, "loss": -0.06091587245464325, "gain": 0.0010667721508070827, "harm": 0.4672386944293976, "nll": -2.956019878387451, "t": 147.54}
24 +{"step": 1700, "loss": -0.08807265758514404, "gain": 0.0010536278132349253, "harm": 0.44384080171585083, "nll": -3.110466718673706, "t": 155.32}
25 +{"step": 1749, "calib_huber": 0.0010563022508134387, "calib_sign_acc": 0.4426363501263189, "calib_harm_brier": 0.141879980127247, "calib_harm_ece": 0.02590574137866497, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.01162012717644132, "calib_captured_fraction": 0.5016690398864454, "calib_top1_match": 0.06322188449848025, "calib_n_audit_states": 1645, "t": 161.98}
26 +{"step": 1800, "loss": -0.1428903043270111, "gain": 0.0009566989610902965, "harm": 0.3898843228816986, "nll": -3.3878915309906006, "t": 166.42}
27 +{"step": 1900, "loss": -0.10342848300933838, "gain": 0.0008749199332669377, "harm": 0.5072088241577148, "nll": -3.579077959060669, "t": 174.07}
28 +{"step": 1999, "loss": -0.0823371410369873, "gain": 0.0009618307230994105, "harm": 0.4634293019771576, "nll": -3.1501359939575195, "t": 181.79}
29 +{"step": 1999, "calib_huber": 0.0010995379567305744, "calib_sign_acc": 0.45567692079060784, "calib_harm_brier": 0.14569287350422597, "calib_harm_ece": 0.0276187714189291, "calib_n_labels": 26916, "calib_oracle_opportunity_per_state": 0.023318092004144083, "calib_regret_per_state": 0.011232054849526078, "calib_captured_fraction": 0.5183115819454731, "calib_top1_match": 0.06930091185410335, "calib_n_audit_states": 1645, "t": 184.54}
30 +{"event": "restored_best_calibration_checkpoint", "step": 1250, "score": -0.011166786567780747, "t": 184.54}
added experiments/runs/controller_v2s5_seed01_20261030004719/checkpoint/checkpoint.json +49 −0
@@ -0,0 +1,49 @@
1 +{
2 + "config": {
3 + "name": "tiny_p4v2",
4 + "vocab_size": 128,
5 + "d_model": 256,
6 + "n_layers": 6,
7 + "n_heads": 8,
8 + "n_kv_heads": 4,
9 + "ffn_width": 768,
10 + "max_seq_len": 320,
11 + "rope_theta": 10000.0,
12 + "norm_eps": 1e-05,
13 + "tie_embeddings": true,
14 + "init_std": 0.02,
15 + "use_workspace": true,
16 + "w": 64,
17 + "ffn_w": 192,
18 + "n_slots": 16,
19 + "n_editor_blocks": 2,
20 + "n_editor_heads": 2,
21 + "r_max": 8,
22 + "eta_max": 0.25,
23 + "clip_c": 1.0,
24 + "clip_eps": 1e-06,
25 + "readout_zero_init": false,
26 + "readout_init_std": 0.2,
27 + "readout_scale": 39.13882827758789,
28 + "controller_hidden": 0,
29 + "ctrl_input": "h",
30 + "ctrl_use_entropy": true,
31 + "lambda_cost": 0.3,
32 + "kappa_risk": 0.5,
33 + "dense_writes": false
34 + },
35 + "saved_at": "2026-10-30T00:50:54-0400",
36 + "run_id": "controller_v2s5_seed01_20261030004719",
37 + "parent": "experiments/runs/p4v2_prisme_seed05_20261030001839/checkpoint",
38 + "selected": {
39 + "lambda": 0.3,
40 + "kappa": 0.5,
41 + "loss": 0.48367020866059185,
42 + "edits_per_token": 0.2568208778173191,
43 + "extra_ops_ratio": 0.027824353634391955,
44 + "loss_reduction": 0.0037358171999242185,
45 + "reduction_per_ops": 0.13426429411487234
46 + },
47 + "locked": true,
48 + "weights_sha256": "958788a3ebabeed125e43ca842c6e0c0a26b2eb9741f65b990ede4325bfbceb9"
49 +}
\ No newline at end of file
added experiments/runs/controller_v2s5_seed01_20261030004719/config.yaml +67 −0
@@ -0,0 +1,67 @@
1 +experiment:
2 + name: controller_v2s5
3 + seed: 1
4 + mode: prisme
5 + model: from_checkpoint
6 + checkpoint: experiments/runs/p4v2_prisme_seed05_20261030001839/checkpoint
7 + labels:
8 + train: experiments/runs/labels_v2s5_seed01_20261030004544/labels_train.pt
9 + calib: experiments/runs/labels_v2s5_seed01_20261030004544/labels_calib.pt
10 + train:
11 + steps: 2000
12 + lr: 0.001
13 + weight_decay: 0.01
14 + alpha: 1.0
15 + beta: 0.5
16 + gamma: 0.1
17 + huber_delta: 0.1
18 + log_every: 100
19 + eval_every: 250
20 + seq_len: 320
21 + lambda_grid:
22 + - 0.0
23 + - 0.01
24 + - 0.03
25 + - 0.1
26 + - 0.3
27 + kappa_grid:
28 + - 0.0
29 + - 0.5
30 + sweep_batches: 10
31 + max_budget: 8
32 + selection_rule: max_loss_reduction_per_ops
33 + model_overrides:
34 + ctrl_input: h
35 + ctrl_use_entropy: true
36 +model:
37 + name: tiny_p4v2
38 + vocab_size: 128
39 + d_model: 256
40 + n_layers: 6
41 + n_heads: 8
42 + n_kv_heads: 4
43 + ffn_width: 768
44 + max_seq_len: 320
45 + rope_theta: 10000.0
46 + norm_eps: 1.0e-05
47 + tie_embeddings: true
48 + init_std: 0.02
49 + use_workspace: true
50 + w: 64
51 + ffn_w: 192
52 + n_slots: 16
53 + n_editor_blocks: 2
54 + n_editor_heads: 2
55 + r_max: 8
56 + eta_max: 0.25
57 + clip_c: 1.0
58 + clip_eps: 1.0e-06
59 + readout_zero_init: false
60 + readout_init_std: 0.2
61 + readout_scale: 39.13882827758789
62 + controller_hidden: 0
63 + ctrl_input: h
64 + ctrl_use_entropy: true
65 + lambda_cost: 0.0
66 + kappa_risk: 0.0
67 + dense_writes: false
added experiments/runs/controller_v2s5_seed01_20261030004719/manifest.json +151 −0
Large diff (152 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/controller_v2s5_seed01_20261030004719/metrics.jsonl +30 −0
Large diff (30 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_mini_seed01_20261030005150/config.yaml +58 −0
Large diff (58 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_mini_seed01_20261030005150/manifest.json +40 −0
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_mini_seed01_20261030005150/metrics.jsonl +0 −0
Large diff (0 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s2_seed01_20261030004543/config.yaml +58 −0
Large diff (58 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s2_seed01_20261030004543/manifest.json +40 −0
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s2_seed01_20261030004543/metrics.jsonl +0 −0
Large diff (0 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s3_seed01_20261030004544/config.yaml +58 −0
Large diff (58 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s3_seed01_20261030004544/manifest.json +40 −0
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s3_seed01_20261030004544/metrics.jsonl +0 −0
Large diff (0 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s4_seed01_20261030004544/config.yaml +58 −0
Large diff (58 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s4_seed01_20261030004544/manifest.json +40 −0
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s4_seed01_20261030004544/metrics.jsonl +0 −0
Large diff (0 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s5_seed01_20261030004544/config.yaml +58 −0
Large diff (58 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s5_seed01_20261030004544/manifest.json +40 −0
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/labels_v2s5_seed01_20261030004544/metrics.jsonl +0 −0
Large diff (0 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_backbone_seed01_20261030005114/checkpoint/checkpoint.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_backbone_seed01_20261030005114/config.yaml +88 −0
Large diff (88 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_backbone_seed01_20261030005114/em_dev_iid_b0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_backbone_seed01_20261030005114/manifest.json +42 −0
Large diff (43 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_backbone_seed01_20261030005114/metrics.jsonl +6 −0
Large diff (6 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_editor_seed01_20261030005118/checkpoint/checkpoint.json +39 −0
Large diff (40 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_editor_seed01_20261030005118/config.yaml +106 −0
Large diff (106 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_editor_seed01_20261030005118/em_dev_iid_b0.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_editor_seed01_20261030005118/em_dev_iid_b2.json +1 −0
Large diff (2 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_editor_seed01_20261030005118/manifest.json +55 −0
Large diff (56 lines) is not loaded by default to keep the page fast.Load diff
added experiments/runs/mini_editor_seed01_20261030005118/metrics.jsonl +6 −0
Large diff (6 lines) is not loaded by default to keep the page fast.Load diff
modified paper/main.log +90 −78
Large diff (317 lines) is not loaded by default to keep the page fast.Load diff
modified paper/main.pdf +0 −0

Binary file not shown.

modified paper/sections/abstract.tex +20 −1
Large diff (21 lines) is not loaded by default to keep the page fast.Load diff
modified paper/sections/conclusion.tex +16 −1
Large diff (17 lines) is not loaded by default to keep the page fast.Load diff
modified paper/sections/limitations.tex +22 −1
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
modified paper/sections/mechanistic.tex +44 −1
Large diff (45 lines) is not loaded by default to keep the page fast.Load diff
modified paper/sections/reproducibility.tex +11 −1
Large diff (12 lines) is not loaded by default to keep the page fast.Load diff
modified paper/sections/results.tex +127 −1
Large diff (128 lines) is not loaded by default to keep the page fast.Load diff
added paper/sections/results_pilot.tex +1 −0
Large diff (1 lines) is not loaded by default to keep the page fast.Load diff
modified paper/tables/numbers.tex +74 −16
Large diff (101 lines) is not loaded by default to keep the page fast.Load diff
added paper/tables/routing_seeds.tex +14 −0
Large diff (14 lines) is not loaded by default to keep the page fast.Load diff
added reproducibility/data_card.md +24 −0
Large diff (24 lines) is not loaded by default to keep the page fast.Load diff
added reproducibility/model_card.md +12 −0
Large diff (12 lines) is not loaded by default to keep the page fast.Load diff
added reproducibility/run_index.md +39 −0
Large diff (39 lines) is not loaded by default to keep the page fast.Load diff
added results/analyzed/routing_seeds_v2.json +66 −0
Large diff (67 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/routing_mini.json +130 −0
Large diff (131 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/routing_v2s2.json +290 −0
Large diff (291 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/routing_v2s3.json +290 −0
Large diff (291 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/routing_v2s4.json +290 −0
Large diff (291 lines) is not loaded by default to keep the page fast.Load diff
added results/raw/routing_v2s5.json +290 −0
Large diff (291 lines) is not loaded by default to keep the page fast.Load diff
modified scripts/make_results_tables.py +95 −5
Large diff (132 lines) is not loaded by default to keep the page fast.Load diff
added scripts/reproduce_pilot.sh +65 −0
Large diff (65 lines) is not loaded by default to keep the page fast.Load diff
added scripts/run_index.py +28 −0
Large diff (29 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere