SPB Git
spboucher.ai

spb/wpai0001_uqo

Public

Sprints 3-6: transplant families, controls, sweeps, mechanism, Tier-0 confirmatory run; Tier-1 MLX harness; paper draft

- alignment maps (identity/Procrustes/ridge/CCA/sham), six graft families, control battery, supervised/distillation references
- model matrix (17 toy models), runs graft_tier0 v1/v2 and confirmatory (prereg v1.0 frozen, D10), persistent fresh-process check
- annihilation proposition (regression-fitted maps remove recipient-unpredictable directions) with proof
- MLX layer wrapping with bitwise no-op test; Tier-1 task-vector transfer script (dev run in progress)
- figures/tables generated from results; paper sections intro/theory/design/results/mechanism/robustness/discussion/limitations

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 9, 2026) parent 3c8aa79 Browse files

13 changed files +19,989 −0

13 changed files
Load all 13 files Unified Split
modified docs/DECISION_LOG.md +19 −0
@@ -21,3 +21,22 @@ All entries are dated; "before freeze" refers to the confirmatory freeze (PREREG
21 token restores nothing; it is added to the next sweep. The pilot's localization set (length-3 programs, 21 token restores nothing; it is added to the next sweep. The pilot's localization set (length-3 programs,
22 Ω at index 1, corrupt = `zib`) is a *development* set; confirmatory localization will use new seeds and 22 Ω at index 1, corrupt = `zib`) is a *development* set; confirmatory localization will use new seeds and
23 other corrupt operators. 23 other corrupt operators.
24+- **D7 (2026-11-09, before freeze).** Exploratory run `graft_tier0_f50a28ca1bf7` showed that affine maps fitted to
25+ predict the recipient's activations on Ω-containing rows (ridge, CCA) *remove* the Ω-specific component (the
26+ recipient has none to predict), while orthogonal maps (Procrustes) and maps fitted on task-free rows preserve
27+ it (Procrustes on task-free rows: 0.99 same-init / 0.96 different-init vs ridge-on-Ω-rows 0.17). The reference
28+ map for the secondary families (steering, subspace, adapter, H5, mismatch, dose, layer grid) is therefore
29+ **Procrustes fitted on task-free alignment rows**; ridge variants are kept as comparisons. Protocol tag
30+ `v2-unrelated-procrustes-reference`; the v1 run is retained in the registry.
31+- **D8 (2026-11-09, before freeze).** Pairs are classified by initialization: donor_s*i* and recipient_s*i* share
32+ the initialization seed (identical initial weights, different data); off-diagonal pairs do not. Reported
33+ separately because the identity graft works only for same-initialization pairs (0.87 vs 0.10).
34+- **D9 (2026-11-09, before freeze).** Tier-1 alignment prompts are *format-matched and task-free* (``Input:/Output:``
35+ frames with random content, D9) after the quick run showed that sentence-final states are off the manifold of
36+ ``Output:``-position states; held-out R² is now computed on rows never used for fitting; a donor-in-the-loop
37+ replace graft (donor few-shot state mapped into the recipient) and its controls (donor zero-shot state, shuffled
38+ query, sham map) are added; a declared norm-rescaled variant of the mapped vector is reported.
39+- **D10 (2026-11-09, FREEZE).** PREREGISTRATION v1.0 (Tier 0) frozen after `graft_tier0_30e3644aa801`; primary
40+ method = activation graft with task-free Procrustes at L0→L0, α = 1; confirmatory seed base 90 000. SHA-256 of
41+ `docs/PREREGISTRATION.md` at freeze recorded below.
42+ PREREGISTRATION.md sha256 at freeze: fe06d326bbbd27916d573a617e94d4443546f179587209e2e04a6893aad25d3e
modified docs/EXPERIMENT_LOG.md +42 −0
@@ -41,3 +41,45 @@ criteria of EXPERIMENT_PLAN §4 are met.
41 41
42 **Pilot variability for power (PREREGISTRATION §5).** Seed sd of recipient target accuracy 0.053; cluster 42 **Pilot variability for power (PREREGISTRATION §5).** Seed sd of recipient target accuracy 0.053; cluster
43 bootstrap half-width of the paired gap ≈ 0.022 with 1 000 items / ≈ 900 programs. 43 bootstrap half-width of the paired gap ≈ 0.022 with 1 000 items / ≈ 900 programs.
44+
45+| 2026-11-09 | Model matrix `artifacts/models/matrix_v1` (17 toy models: donor/recipient/unrelated × 3 seeds, w96 ×2, L6 ×2, stride-2 donor/recipient ×2) | completed | `artifacts/models/matrix_v1/index.json` | ≈ 45 s each on MPS; all dev accuracies 1.000 |
46+| 2026-11-09 | `graft_tier0_quick_7d4278be3e5a` (CPU, 1 seed, n = 300) | completed (55 s) | `experiments/summaries/graft_tier0_quick_7d4278be3e5a/` | pipeline check; revealed the map-suppression effect (D7) |
47+| 2026-11-09 | `graft_tier0_f50a28ca1bf7` (MPS, 9 pairs, n = 1 000, protocol v1) | completed (140 s) | `experiments/summaries/graft_tier0_f50a28ca1bf7/` | exploratory; see §Transplant results v1 |
48+| 2026-11-09 | `eval_persistent` on `graft_tier0_f50a28ca1bf7` (fresh process, seed base 70 000) | completed | `experiments/summaries/graft_tier0_f50a28ca1bf7/persistent/fresh_process_eval.json` | saved ridge adapter: target 0.143 vs base 0.118 (no acquisition); control 1.000 |
49+| 2026-11-09 | `tier1_taskvector_quick_b89f92479155` (MLX, qwen3b→qwen1.5b, 12 queries/task) | completed (150 s) | `experiments/summaries/tier1_taskvector_quick_b89f92479155/` | within-model vectors: antonym 0.75 (L16), capital 1.00 (L20) in Qwen2.5-1.5B; cross-model ≤ 0.25 = unrelated-task control; led to D9 |
50+
51+## Transplant results v1 — `graft_tier0_f50a28ca1bf7` (exploratory; superseded by the v2 protocol run)
52+
53+Target-stratum accuracy, mean over pairs (same-init n = 3 / different-init n = 6), control in parentheses:
54+activation graft identity 0.873 (0.99) / 0.101 (0.10); Procrustes on Ω rows 0.785 (0.985) / 0.697 (0.943);
55+**Procrustes on task-free rows 0.993 (0.996) / 0.956 (0.989)**; ridge on task-free rows 0.814 (1.0) / 0.795 (1.0);
56+ridge on Ω rows 0.170 / 0.175; CCA 0.08 / 0.089; sham map 0.10 (0.085); norm-matched random 0.089 (0.097);
57+shuffled token 0.10 (0.10); shuffled layer 0.187 (1.0); unrelated-skill donor 0.076 (1.0); steering α ≤ 4: ≤ 0.082;
58+subspace k ≤ 128 (ridge reference): ≤ 0.172; adapter graft 0.095; supervised adapter 1.0; distillation 1.0;
59+weight grafts: L0 MLP 0.262 (0.962), L0 block 0.642 (0.726), L0–L2 blocks + Ω embedding 0.774 (0.749), shuffled
60+layer 0.083 (0.089), unrelated donor 0.094 (0.933). Recipient baselines 0.111 / 0.047 / 0.067 (chance ≈ 0.10).
61+Mismatch (Procrustes on Ω rows): width 128→96 0.638 (0.981); depth 4→6 L1→L1 0.988 (1.0), L0→L0 0.358 (0.75).
62+Stride-2 (ridge): L3 0.604 (0.993). Mechanism: donor attn_out.0 ablated at read positions 0.086 (0.102);
63+rescue from intact donor_s1 1.000 (1.0); sham rescue 0.112; reverse graft recipient→donor 0.965 (1.0).
64+Rule-vs-cache (ridge fitted on Ω inputs {0..5}): seen 0.472, unseen 0.740 (consistent with suppression of seen
65+rows); adapter: seen 0.259, unseen 0.0. Dose (ridge): monotone to 0.34 at α = 1.5, control collapses at α = 2.
66+Layer grid (ridge): 0.25–0.42, best at recipient L3.
67+
68+| 2026-11-09 | `graft_tier0_30e3644aa801` (MPS, protocol v2: task-free Procrustes reference map) | completed (176 s) | `experiments/summaries/graft_tier0_30e3644aa801/` | exploratory; see §Transplant results v2 |
69+| 2026-11-09 | `eval_persistent` on `graft_tier0_30e3644aa801` (fresh process, seed base 70 000) | completed | `.../persistent/fresh_process_eval.json` | saved donor-free adapter: target 0.414 (Ω-step 0.455) vs base 0.140; control 0.981 |
70+
71+## Transplant results v2 — `graft_tier0_30e3644aa801` (exploratory; basis of the v1.0 freeze)
72+
73+Same as v1 for the activation-graft families and controls (identical numbers: same models, same splits).
74+With the task-free Procrustes reference map: subspace grafts k = 1/2/4/8/16/32 → 0.230/0.296/0.650/**0.936**/
75+0.970/0.983 (random subspace of equal k: 0.076/0.079/0.088/0.085/0.127/0.173; control 1.0 throughout);
76+adapter graft (donor-free) 0.581 (control 0.998); steering α ≤ 4 ≤ 0.096; layer grid: donor L1–L3 → any
77+recipient layer 1.00, L0→L0 0.98, L0→L1–3 0.75–0.82; dose: 0.44 (α 0.5), 0.90 (0.75), **0.98 (1.0)**, 0.92 (1.25),
78+0.75 (1.5), 0.39 (2.0; control 0.36); mismatch with task-free Procrustes: width 96 L0→L0 0.886, **L1→L1 1.000**;
79+depth 6 L0→L0 0.765, **L1→L1 1.000**, L1→L2 1.000, L2→L2 1.000 (sham 0.07–0.12 with control destroyed);
80+stride-2: L0 0.370, L1 0.747, **L2 1.000, L3 1.000** (ridge: 0.13–0.60); rule-vs-cache: Procrustes fitted on
81+seen Ω inputs → seen 0.966 / unseen 0.831 (ratio 0.86); donor-free adapter → seen 0.765 / unseen 0.000 (cache);
82+mechanism unchanged (ablation 0.086 → rescue 1.000, sham 0.115; reverse 0.965).
83+
84+| 2026-11-09 | `graft_tier0_confirmatory_0c2257f6803e` (MPS, PREREGISTRATION v1.0, seed base 90 000) | **completed** | `experiments/summaries/graft_tier0_confirmatory_0c2257f6803e/{results.json,CONFIRMATORY.md,confirmatory.json}` | primary: 9/9 pairs lower bound > SESOI, mean Δ_T^adj 0.875 (sd 0.042, min 0.776) → H2 SUPPORTED; H4 NOT SUPPORTED (one pair control 0.946 < 0.95; others ≥ 0.985); Holm: subspace k8 vs random +0.850, adapter vs base +0.472, weight L0 block vs shuffled +0.556 (retention 0.720), all rejected at Holm; H5 seen 0.941 / unseen 0.768 (ratio 0.816 ≥ 0.8) SUPPORTED; adapter cache 0.702 / 0.000 |
85+| 2026-11-09 | `eval_persistent` on the confirmatory run (fresh process, seed base 95 000) | completed | `.../persistent/fresh_process_eval.json` | saved adapter: target 0.344 [0.317, 0.372] vs base 0.115; control 0.979 |
modified docs/PREREGISTRATION.md +47 −0
@@ -65,3 +65,50 @@ the run manifest and in `EXPERIMENT_LOG.md`. No run is excluded for its outcome.
65 Split hashes are stored in manifests; evaluation splits are generated with seeds disjoint from training 65 Split hashes are stored in manifests; evaluation splits are generated with seeds disjoint from training
66 and alignment seeds; `assert_disjoint` is enforced at build time; alignment data for mappings never 66 and alignment seeds; `assert_disjoint` is enforced at build time; alignment data for mappings never
67 include evaluation items (checked by item hash). 67 include evaluation items (checked by item hash).
68+
69+---
70+
71+# Preregistration v1.0 — CONFIRMATORY FREEZE (Tier 0), 2026-11-09
72+
73+Frozen after exploratory runs `graft_tier0_f50a28ca1bf7` (protocol v1) and `graft_tier0_30e3644aa801`
74+(protocol v2) and **before** any confirmatory split was generated. Confirmatory evaluation and alignment
75+items come from seed base 90 000 (never used before); the trained models (`artifacts/models/matrix_v1`) are
76+unchanged; the analysis code (`scripts/run_transplant.py` at the commit recorded in the confirmatory manifest)
77+is unchanged.
78+
79+## Primary method (fixed)
80+Activation graft, donor in the loop, at the recipient's `resid_post.0` at every read position (untargeted),
81+α = 1, with the alignment map = **orthogonal Procrustes with isotropic scale fitted on task-free alignment rows**
82+(3 000 items without Ω, disjoint from evaluation items). Model pairs: the nine donor_s{i} → recipient_s{j}
83+pairs (3 same-initialization, 6 different-initialization).
84+
85+## Primary outcome and contrast
86+Δ_T^adj on `target_iid` = [Q_T(graft) − Q_T(recipient)] − [Q_T(sham map) − Q_T(recipient)], paired on items,
87+cluster bootstrap over programs (2 000 draws), reported per pair and summarized over pairs (mean, sd, min).
88+**Decision rule:** H2 (sufficiency) is supported if the lower 95 % bound of Δ_T^adj exceeds SESOI = 0.25 in at
89+least 8 of the 9 pairs and the mean over pairs exceeds 0.5; retention H4 is supported if `control_iid` accuracy
90+with the graft is ≥ 0.95 in all nine pairs.
91+
92+## Secondary pre-declared contrasts (Holm correction over these four families, α = 0.05)
93+1. Subspace graft k = 8 (Ω-contrast PCA through the primary map) vs random subspace k = 8, matched seeds.
94+2. Adapter graft (donor-free ridge interface to the primary map's targets) vs unmodified recipient; and the
95+ same adapter evaluated in a fresh process on new splits (persistence; `scripts/eval_persistent.py`).
96+3. Weight graft L0 block (attention + MLP + LN) vs shuffled-layer weight control, matched seeds; retention
97+ reported as a cost.
98+4. Rule-vs-cache (H5): primary-map graft fitted on Ω inputs {0,…,5} (Procrustes on seen rows) evaluated on
99+ unseen-input items vs seen-input items; supported if unseen ≥ 0.8 × seen. The donor-free adapter is
100+ predicted to FAIL this test (cache), and the failure is reported as such.
101+
102+## Mechanism tests (fixed, matched seed 0)
103+Donor ablation (attn_out.0 zeroed at read positions) → rescue from intact donor_s1 through the task-free
104+Procrustes map vs sham rescue; reverse graft recipient → donor on control items (no harm expected).
105+
106+## Portability (fixed)
107+Width mismatch (d 128 → 96) and depth mismatch (4 → 6 layers) with the task-free Procrustes map at L1 → L1
108+(the layer pair selected in exploration) and L0 → L0; stride-2 pair at L2 → L2 and L3 → L3.
109+
110+## Exploratory (not confirmatory)
111+Everything else produced by the script (layer grid, dose, other maps, steering, other k) is reported as
112+exploratory and labelled so in the paper.
113+
114+## Tier 1 (to be frozen in a separate section after the development run completes)
added experiments/manifests/graft_tier0_30e3644aa801.json +90 −0
@@ -0,0 +1,90 @@
1+{
2+ "config": {
3+ "models": "matrix_v1",
4+ "protocol": "v2-unrelated-procrustes-reference",
5+ "quick": false,
6+ "seed_base": 30000,
7+ "tag": "main"
8+ },
9+ "created_utc": "2026-11-09T07:36:09+00:00",
10+ "data_hashes": {
11+ "align_target": {
12+ "hash": "2aadb9513aeabf54",
13+ "n": 3000,
14+ "seed": 30001
15+ },
16+ "align_unrelated": {
17+ "hash": "19a475dcc593cb0d",
18+ "n": 3000,
19+ "seed": 30002
20+ },
21+ "comp_heldout": {
22+ "hash": "9c6da34a8ad49288",
23+ "n": 1000,
24+ "seed": 30012
25+ },
26+ "control_iid": {
27+ "hash": "98a88deb1a923d81",
28+ "n": 1000,
29+ "seed": 30011
30+ },
31+ "len_ood": {
32+ "hash": "1e5385df81ec873b",
33+ "n": 500,
34+ "seed": 30013
35+ },
36+ "target_iid": {
37+ "hash": "9a69f681d0b62bbb",
38+ "n": 1000,
39+ "seed": 30010
40+ }
41+ },
42+ "experiment_id": "graft_tier0",
43+ "git": {
44+ "branch": "main",
45+ "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
46+ "dirty": true
47+ },
48+ "hardware": {
49+ "chip": "Apple M5 Max",
50+ "cpu_count": 18,
51+ "hostname_hash": "9ad7358c662e",
52+ "machine": "arm64",
53+ "memory_gb": 48.0,
54+ "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
55+ "torch_cuda": false,
56+ "torch_mps": true
57+ },
58+ "history": [
59+ {
60+ "note": "",
61+ "status": "running",
62+ "utc": "2026-11-09T07:36:11+00:00"
63+ },
64+ {
65+ "note": "",
66+ "status": "completed",
67+ "utc": "2026-11-09T07:39:17+00:00"
68+ }
69+ ],
70+ "interpretation": "",
71+ "libraries": {
72+ "mlx": "?",
73+ "mlx_lm": "0.31.3",
74+ "numpy": "2.4.4",
75+ "pydantic": "2.13.4",
76+ "python": "3.14.4",
77+ "safetensors": "0.7.0",
78+ "scipy": "1.17.1",
79+ "torch": "2.12.0",
80+ "transformers": "5.8.1"
81+ },
82+ "outputs": {
83+ "items": "artifacts/runs/graft_tier0_30e3644aa801",
84+ "results": "experiments/summaries/graft_tier0_30e3644aa801/results.json"
85+ },
86+ "run_id": "graft_tier0_30e3644aa801",
87+ "seeds": {},
88+ "status": "completed",
89+ "warnings": []
90+}
\ No newline at end of file
added experiments/manifests/graft_tier0_confirmatory_0c2257f6803e.json +90 −0
@@ -0,0 +1,90 @@
1+{
2+ "config": {
3+ "models": "matrix_v1",
4+ "protocol": "v2-unrelated-procrustes-reference",
5+ "quick": false,
6+ "seed_base": 90000,
7+ "tag": "confirmatory"
8+ },
9+ "created_utc": "2026-11-09T07:41:03+00:00",
10+ "data_hashes": {
11+ "align_target": {
12+ "hash": "3afb3d9322bd4e85",
13+ "n": 3000,
14+ "seed": 90001
15+ },
16+ "align_unrelated": {
17+ "hash": "ce28064949d5c649",
18+ "n": 3000,
19+ "seed": 90002
20+ },
21+ "comp_heldout": {
22+ "hash": "89f1988aff8d8535",
23+ "n": 1000,
24+ "seed": 90012
25+ },
26+ "control_iid": {
27+ "hash": "ad4f0fde325b0378",
28+ "n": 1000,
29+ "seed": 90011
30+ },
31+ "len_ood": {
32+ "hash": "c1179ed20c26701a",
33+ "n": 500,
34+ "seed": 90013
35+ },
36+ "target_iid": {
37+ "hash": "c104c5c2174007f6",
38+ "n": 1000,
39+ "seed": 90010
40+ }
41+ },
42+ "experiment_id": "graft_tier0_confirmatory",
43+ "git": {
44+ "branch": "main",
45+ "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
46+ "dirty": true
47+ },
48+ "hardware": {
49+ "chip": "Apple M5 Max",
50+ "cpu_count": 18,
51+ "hostname_hash": "9ad7358c662e",
52+ "machine": "arm64",
53+ "memory_gb": 48.0,
54+ "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
55+ "torch_cuda": false,
56+ "torch_mps": true
57+ },
58+ "history": [
59+ {
60+ "note": "",
61+ "status": "running",
62+ "utc": "2026-11-09T07:41:05+00:00"
63+ },
64+ {
65+ "note": "",
66+ "status": "completed",
67+ "utc": "2026-11-09T07:44:03+00:00"
68+ }
69+ ],
70+ "interpretation": "",
71+ "libraries": {
72+ "mlx": "?",
73+ "mlx_lm": "0.31.3",
74+ "numpy": "2.4.4",
75+ "pydantic": "2.13.4",
76+ "python": "3.14.4",
77+ "safetensors": "0.7.0",
78+ "scipy": "1.17.1",
79+ "torch": "2.12.0",
80+ "transformers": "5.8.1"
81+ },
82+ "outputs": {
83+ "items": "artifacts/runs/graft_tier0_confirmatory_0c2257f6803e",
84+ "results": "experiments/summaries/graft_tier0_confirmatory_0c2257f6803e/results.json"
85+ },
86+ "run_id": "graft_tier0_confirmatory_0c2257f6803e",
87+ "seeds": {},
88+ "status": "completed",
89+ "warnings": []
90+}
\ No newline at end of file
added experiments/manifests/graft_tier0_f50a28ca1bf7.json +89 −0
@@ -0,0 +1,89 @@
1+{
2+ "config": {
3+ "models": "matrix_v1",
4+ "quick": false,
5+ "seed_base": 30000,
6+ "tag": "main"
7+ },
8+ "created_utc": "2026-11-09T07:31:00+00:00",
9+ "data_hashes": {
10+ "align_target": {
11+ "hash": "2aadb9513aeabf54",
12+ "n": 3000,
13+ "seed": 30001
14+ },
15+ "align_unrelated": {
16+ "hash": "19a475dcc593cb0d",
17+ "n": 3000,
18+ "seed": 30002
19+ },
20+ "comp_heldout": {
21+ "hash": "9c6da34a8ad49288",
22+ "n": 1000,
23+ "seed": 30012
24+ },
25+ "control_iid": {
26+ "hash": "98a88deb1a923d81",
27+ "n": 1000,
28+ "seed": 30011
29+ },
30+ "len_ood": {
31+ "hash": "1e5385df81ec873b",
32+ "n": 500,
33+ "seed": 30013
34+ },
35+ "target_iid": {
36+ "hash": "9a69f681d0b62bbb",
37+ "n": 1000,
38+ "seed": 30010
39+ }
40+ },
41+ "experiment_id": "graft_tier0",
42+ "git": {
43+ "branch": "main",
44+ "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
45+ "dirty": true
46+ },
47+ "hardware": {
48+ "chip": "Apple M5 Max",
49+ "cpu_count": 18,
50+ "hostname_hash": "9ad7358c662e",
51+ "machine": "arm64",
52+ "memory_gb": 48.0,
53+ "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
54+ "torch_cuda": false,
55+ "torch_mps": true
56+ },
57+ "history": [
58+ {
59+ "note": "",
60+ "status": "running",
61+ "utc": "2026-11-09T07:31:02+00:00"
62+ },
63+ {
64+ "note": "",
65+ "status": "completed",
66+ "utc": "2026-11-09T07:33:33+00:00"
67+ }
68+ ],
69+ "interpretation": "",
70+ "libraries": {
71+ "mlx": "?",
72+ "mlx_lm": "0.31.3",
73+ "numpy": "2.4.4",
74+ "pydantic": "2.13.4",
75+ "python": "3.14.4",
76+ "safetensors": "0.7.0",
77+ "scipy": "1.17.1",
78+ "torch": "2.12.0",
79+ "transformers": "5.8.1"
80+ },
81+ "outputs": {
82+ "items": "artifacts/runs/graft_tier0_f50a28ca1bf7",
83+ "results": "experiments/summaries/graft_tier0_f50a28ca1bf7/results.json"
84+ },
85+ "run_id": "graft_tier0_f50a28ca1bf7",
86+ "seeds": {},
87+ "status": "completed",
88+ "warnings": []
89+}
\ No newline at end of file
added experiments/manifests/graft_tier0_quick_7d4278be3e5a.json +89 −0
@@ -0,0 +1,89 @@
1+{
2+ "config": {
3+ "models": "matrix_v1",
4+ "quick": true,
5+ "seed_base": 30000,
6+ "tag": "main"
7+ },
8+ "created_utc": "2026-11-09T07:25:13+00:00",
9+ "data_hashes": {
10+ "align_target": {
11+ "hash": "2e9e93027c216aa2",
12+ "n": 600,
13+ "seed": 30001
14+ },
15+ "align_unrelated": {
16+ "hash": "4467a0296d6dbcc4",
17+ "n": 600,
18+ "seed": 30002
19+ },
20+ "comp_heldout": {
21+ "hash": "51ad8603009a9dd2",
22+ "n": 300,
23+ "seed": 30012
24+ },
25+ "control_iid": {
26+ "hash": "b8916fa4b7243e6f",
27+ "n": 300,
28+ "seed": 30011
29+ },
30+ "len_ood": {
31+ "hash": "e21b8c770ad0b017",
32+ "n": 200,
33+ "seed": 30013
34+ },
35+ "target_iid": {
36+ "hash": "50bc867931acc1be",
37+ "n": 300,
38+ "seed": 30010
39+ }
40+ },
41+ "experiment_id": "graft_tier0_quick",
42+ "git": {
43+ "branch": "main",
44+ "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
45+ "dirty": true
46+ },
47+ "hardware": {
48+ "chip": "Apple M5 Max",
49+ "cpu_count": 18,
50+ "hostname_hash": "9ad7358c662e",
51+ "machine": "arm64",
52+ "memory_gb": 48.0,
53+ "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
54+ "torch_cuda": false,
55+ "torch_mps": true
56+ },
57+ "history": [
58+ {
59+ "note": "",
60+ "status": "running",
61+ "utc": "2026-11-09T07:25:15+00:00"
62+ },
63+ {
64+ "note": "",
65+ "status": "completed",
66+ "utc": "2026-11-09T07:26:11+00:00"
67+ }
68+ ],
69+ "interpretation": "",
70+ "libraries": {
71+ "mlx": "?",
72+ "mlx_lm": "0.31.3",
73+ "numpy": "2.4.4",
74+ "pydantic": "2.13.4",
75+ "python": "3.14.4",
76+ "safetensors": "0.7.0",
77+ "scipy": "1.17.1",
78+ "torch": "2.12.0",
79+ "transformers": "5.8.1"
80+ },
81+ "outputs": {
82+ "items": "artifacts/runs/graft_tier0_quick_7d4278be3e5a",
83+ "results": "experiments/summaries/graft_tier0_quick_7d4278be3e5a/results.json"
84+ },
85+ "run_id": "graft_tier0_quick_7d4278be3e5a",
86+ "seeds": {},
87+ "status": "completed",
88+ "warnings": []
89+}
\ No newline at end of file
added experiments/manifests/tier1_taskvector_e105e3d91386.json +56 −0
@@ -0,0 +1,56 @@
1+{
2+ "config": {
3+ "k": 8,
4+ "pairs": [
5+ "qwen3b:qwen1.5b",
6+ "qwen1.5b:qwen3b",
7+ "llama3b:qwen3b",
8+ "qwen3b:llama3b"
9+ ],
10+ "quick": false,
11+ "split": "dev",
12+ "task_seed": 0
13+ },
14+ "created_utc": "2026-11-09T07:46:30+00:00",
15+ "data_hashes": {},
16+ "experiment_id": "tier1_taskvector",
17+ "git": {
18+ "branch": "main",
19+ "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
20+ "dirty": true
21+ },
22+ "hardware": {
23+ "chip": "Apple M5 Max",
24+ "cpu_count": 18,
25+ "hostname_hash": "9ad7358c662e",
26+ "machine": "arm64",
27+ "memory_gb": 48.0,
28+ "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
29+ "torch_cuda": false,
30+ "torch_mps": true
31+ },
32+ "history": [
33+ {
34+ "note": "",
35+ "status": "running",
36+ "utc": "2026-11-09T07:46:33+00:00"
37+ }
38+ ],
39+ "interpretation": "",
40+ "libraries": {
41+ "mlx": "?",
42+ "mlx_lm": "0.31.3",
43+ "numpy": "2.4.4",
44+ "pydantic": "2.13.4",
45+ "python": "3.14.4",
46+ "safetensors": "0.7.0",
47+ "scipy": "1.17.1",
48+ "torch": "2.12.0",
49+ "transformers": "5.8.1"
50+ },
51+ "outputs": {},
52+ "run_id": "tier1_taskvector_e105e3d91386",
53+ "seeds": {},
54+ "status": "running",
55+ "warnings": []
56+}
\ No newline at end of file
added experiments/manifests/tier1_taskvector_quick_b89f92479155.json +77 −0
@@ -0,0 +1,77 @@
1+{
2+ "config": {
3+ "k": 8,
4+ "pairs": [
5+ "qwen3b:qwen1.5b"
6+ ],
7+ "quick": true,
8+ "split": "dev",
9+ "task_seed": 0
10+ },
11+ "created_utc": "2026-11-09T07:29:41+00:00",
12+ "data_hashes": {
13+ "addthree": {
14+ "hash": "63af249180d379b8",
15+ "n_queries": 12
16+ },
17+ "antonym": {
18+ "hash": "22992056ec21621c",
19+ "n_queries": 12
20+ },
21+ "capital": {
22+ "hash": "0ce7fe08025fd9c5",
23+ "n_queries": 12
24+ },
25+ "morph": {
26+ "hash": "313569abbc843a4f",
27+ "n_queries": 12
28+ }
29+ },
30+ "experiment_id": "tier1_taskvector_quick",
31+ "git": {
32+ "branch": "main",
33+ "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
34+ "dirty": true
35+ },
36+ "hardware": {
37+ "chip": "Apple M5 Max",
38+ "cpu_count": 18,
39+ "hostname_hash": "9ad7358c662e",
40+ "machine": "arm64",
41+ "memory_gb": 48.0,
42+ "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
43+ "torch_cuda": false,
44+ "torch_mps": true
45+ },
46+ "history": [
47+ {
48+ "note": "",
49+ "status": "running",
50+ "utc": "2026-11-09T07:29:44+00:00"
51+ },
52+ {
53+ "note": "",
54+ "status": "completed",
55+ "utc": "2026-11-09T07:32:14+00:00"
56+ }
57+ ],
58+ "interpretation": "",
59+ "libraries": {
60+ "mlx": "?",
61+ "mlx_lm": "0.31.3",
62+ "numpy": "2.4.4",
63+ "pydantic": "2.13.4",
64+ "python": "3.14.4",
65+ "safetensors": "0.7.0",
66+ "scipy": "1.17.1",
67+ "torch": "2.12.0",
68+ "transformers": "5.8.1"
69+ },
70+ "outputs": {
71+ "results": "experiments/summaries/tier1_taskvector_quick_b89f92479155/results.json"
72+ },
73+ "run_id": "tier1_taskvector_quick_b89f92479155",
74+ "seeds": {},
75+ "status": "completed",
76+ "warnings": []
77+}
\ No newline at end of file
modified experiments/registry/runs.jsonl +19 −0
@@ -4,3 +4,22 @@
4 {"config_hash": "451ef5087182", "experiment_id": "pilot_tier0_opseq", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "planned", "utc": "2026-11-09T06:49:37+00:00"} 4 {"config_hash": "451ef5087182", "experiment_id": "pilot_tier0_opseq", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "planned", "utc": "2026-11-09T06:49:37+00:00"}
5 {"experiment_id": "pilot_tier0_opseq", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "running", "utc": "2026-11-09T06:49:38+00:00"} 5 {"experiment_id": "pilot_tier0_opseq", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "running", "utc": "2026-11-09T06:49:38+00:00"}
6 {"experiment_id": "pilot_tier0_opseq", "results": "experiments/summaries/pilot_tier0_opseq_451ef5087182/results.json", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "completed", "utc": "2026-11-09T06:54:13+00:00"} 6 {"experiment_id": "pilot_tier0_opseq", "results": "experiments/summaries/pilot_tier0_opseq_451ef5087182/results.json", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "completed", "utc": "2026-11-09T06:54:13+00:00"}
7+{"config_hash": "7d4278be3e5a", "experiment_id": "graft_tier0_quick", "run_id": "graft_tier0_quick_7d4278be3e5a", "status": "planned", "utc": "2026-11-09T07:25:15+00:00"}
8+{"experiment_id": "graft_tier0_quick", "run_id": "graft_tier0_quick_7d4278be3e5a", "status": "running", "utc": "2026-11-09T07:25:15+00:00"}
9+{"experiment_id": "graft_tier0_quick", "results": "experiments/summaries/graft_tier0_quick_7d4278be3e5a/results.json", "run_id": "graft_tier0_quick_7d4278be3e5a", "status": "completed", "utc": "2026-11-09T07:26:11+00:00"}
10+{"config_hash": "b89f92479155", "experiment_id": "tier1_taskvector_quick", "run_id": "tier1_taskvector_quick_b89f92479155", "status": "planned", "utc": "2026-11-09T07:29:44+00:00"}
11+{"config_hash": "f50a28ca1bf7", "experiment_id": "graft_tier0", "run_id": "graft_tier0_f50a28ca1bf7", "status": "planned", "utc": "2026-11-09T07:31:02+00:00"}
12+{"experiment_id": "graft_tier0", "run_id": "graft_tier0_f50a28ca1bf7", "status": "running", "utc": "2026-11-09T07:31:02+00:00"}
13+{"experiment_id": "tier1_taskvector_quick", "run_id": "tier1_taskvector_quick_b89f92479155", "status": "completed", "utc": "2026-11-09T07:32:14+00:00"}
14+{"experiment_id": "graft_tier0", "results": "experiments/summaries/graft_tier0_f50a28ca1bf7/results.json", "run_id": "graft_tier0_f50a28ca1bf7", "status": "completed", "utc": "2026-11-09T07:33:33+00:00"}
15+{"config_hash": "e105e3d91386", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "planned", "utc": "2026-11-09T07:34:21+00:00"}
16+{"error": "AttributeError(\"'WrappedBlock' object has no attribute 'use_sliding'\")", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "failed", "utc": "2026-11-09T07:34:22+00:00"}
17+{"config_hash": "30e3644aa801", "experiment_id": "graft_tier0", "run_id": "graft_tier0_30e3644aa801", "status": "planned", "utc": "2026-11-09T07:36:11+00:00"}
18+{"experiment_id": "graft_tier0", "run_id": "graft_tier0_30e3644aa801", "status": "running", "utc": "2026-11-09T07:36:11+00:00"}
19+{"config_hash": "e105e3d91386", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "planned", "utc": "2026-11-09T07:36:38+00:00"}
20+{"experiment_id": "graft_tier0", "results": "experiments/summaries/graft_tier0_30e3644aa801/results.json", "run_id": "graft_tier0_30e3644aa801", "status": "completed", "utc": "2026-11-09T07:39:17+00:00"}
21+{"config_hash": "0c2257f6803e", "experiment_id": "graft_tier0_confirmatory", "run_id": "graft_tier0_confirmatory_0c2257f6803e", "status": "planned", "utc": "2026-11-09T07:41:05+00:00"}
22+{"experiment_id": "graft_tier0_confirmatory", "run_id": "graft_tier0_confirmatory_0c2257f6803e", "status": "running", "utc": "2026-11-09T07:41:05+00:00"}
23+{"experiment_id": "graft_tier0_confirmatory", "results": "experiments/summaries/graft_tier0_confirmatory_0c2257f6803e/results.json", "run_id": "graft_tier0_confirmatory_0c2257f6803e", "status": "completed", "utc": "2026-11-09T07:44:03+00:00"}
24+{"error": "ValueError(\"invalid literal for int() with base 10: 'rocrustes'\")", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "failed", "utc": "2026-11-09T07:45:09+00:00"}
25+{"config_hash": "e105e3d91386", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "planned", "utc": "2026-11-09T07:46:33+00:00"}
added experiments/summaries/graft_tier0_30e3644aa801/persistent/adapter_graft.npz +0 −0

Binary file not shown.

added experiments/summaries/graft_tier0_30e3644aa801/persistent/fresh_process_eval.json +41 −0
@@ -0,0 +1,41 @@
1+{
2+ "utc": "2026-11-09T07:40:13+00:00",
3+ "run": "graft_tier0_30e3644aa801",
4+ "recipient": "recipient_s0",
5+ "hook": "resid_post.0",
6+ "fresh_process": true,
7+ "splits": {
8+ "target_iid": "e1cb4467208e4c8c",
9+ "control_iid": "ddb8b86e4022dbbb"
10+ },
11+ "target_iid": {
12+ "base": 0.14,
13+ "with_saved_adapter": 0.414,
14+ "ci95_with_saved_adapter": {
15+ "estimate": 0.414,
16+ "low": 0.38347969488028666,
17+ "high": 0.4449116790506519,
18+ "level": 0.95,
19+ "n_items": 1000,
20+ "n_clusters": 762,
21+ "n_boot": 2000
22+ },
23+ "omega_step_base": 0.118,
24+ "omega_step_with_saved_adapter": 0.455
25+ },
26+ "control_iid": {
27+ "base": 1.0,
28+ "with_saved_adapter": 0.981,
29+ "ci95_with_saved_adapter": {
30+ "estimate": 0.981,
31+ "low": 0.972,
32+ "high": 0.9890219560878244,
33+ "level": 0.95,
34+ "n_items": 1000,
35+ "n_clusters": 965,
36+ "n_boot": 2000
37+ },
38+ "omega_step_base": null,
39+ "omega_step_with_saved_adapter": null
40+ }
41+}
\ No newline at end of file
added experiments/summaries/graft_tier0_30e3644aa801/results.json +19330 −0
Large diff (19,330 lines) is not loaded by default to keep the page fast.Load diff

Diff truncated — file too large.

Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere