SPB Git
spboucher.ai

spb/wpai0001_uqo

Public

Sprints 3-6: transplant families, controls, sweeps, mechanism, Tier-0 confirmatory run; Tier-1 MLX harness; paper draft

- alignment maps (identity/Procrustes/ridge/CCA/sham), six graft families, control battery, supervised/distillation references
- model matrix (17 toy models), runs graft_tier0 v1/v2 and confirmatory (prereg v1.0 frozen, D10), persistent fresh-process check
- annihilation proposition (regression-fitted maps remove recipient-unpredictable directions) with proof
- MLX layer wrapping with bitwise no-op test; Tier-1 task-vector transfer script (dev run in progress)
- figures/tables generated from results; paper sections intro/theory/design/results/mechanism/robustness/discussion/limitations

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 9, 2026) parent 3c8aa79 Browse files

13 changed files +19,989 −0

13 changed files
Load all 13 files Unified Split
modified docs/DECISION_LOG.md +19 −0
@@ -21,3 +21,22 @@ All entries are dated; "before freeze" refers to the confirmatory freeze (PREREG
21 21 token restores nothing; it is added to the next sweep. The pilot's localization set (length-3 programs,
22 22 Ω at index 1, corrupt = `zib`) is a *development* set; confirmatory localization will use new seeds and
23 23 other corrupt operators.
24 +- **D7 (2026-11-09, before freeze).** Exploratory run `graft_tier0_f50a28ca1bf7` showed that affine maps fitted to
25 + predict the recipient's activations on Ω-containing rows (ridge, CCA) *remove* the Ω-specific component (the
26 + recipient has none to predict), while orthogonal maps (Procrustes) and maps fitted on task-free rows preserve
27 + it (Procrustes on task-free rows: 0.99 same-init / 0.96 different-init vs ridge-on-Ω-rows 0.17). The reference
28 + map for the secondary families (steering, subspace, adapter, H5, mismatch, dose, layer grid) is therefore
29 + **Procrustes fitted on task-free alignment rows**; ridge variants are kept as comparisons. Protocol tag
30 + `v2-unrelated-procrustes-reference`; the v1 run is retained in the registry.
31 +- **D8 (2026-11-09, before freeze).** Pairs are classified by initialization: donor_s*i* and recipient_s*i* share
32 + the initialization seed (identical initial weights, different data); off-diagonal pairs do not. Reported
33 + separately because the identity graft works only for same-initialization pairs (0.87 vs 0.10).
34 +- **D9 (2026-11-09, before freeze).** Tier-1 alignment prompts are *format-matched and task-free* (``Input:/Output:``
35 + frames with random content, D9) after the quick run showed that sentence-final states are off the manifold of
36 + ``Output:``-position states; held-out R² is now computed on rows never used for fitting; a donor-in-the-loop
37 + replace graft (donor few-shot state mapped into the recipient) and its controls (donor zero-shot state, shuffled
38 + query, sham map) are added; a declared norm-rescaled variant of the mapped vector is reported.
39 +- **D10 (2026-11-09, FREEZE).** PREREGISTRATION v1.0 (Tier 0) frozen after `graft_tier0_30e3644aa801`; primary
40 + method = activation graft with task-free Procrustes at L0→L0, α = 1; confirmatory seed base 90 000. SHA-256 of
41 + `docs/PREREGISTRATION.md` at freeze recorded below.
42 + PREREGISTRATION.md sha256 at freeze: fe06d326bbbd27916d573a617e94d4443546f179587209e2e04a6893aad25d3e
modified docs/EXPERIMENT_LOG.md +42 −0
@@ -41,3 +41,45 @@ criteria of EXPERIMENT_PLAN §4 are met.
41 41
42 42 **Pilot variability for power (PREREGISTRATION §5).** Seed sd of recipient target accuracy 0.053; cluster
43 43 bootstrap half-width of the paired gap ≈ 0.022 with 1 000 items / ≈ 900 programs.
44 +
45 +| 2026-11-09 | Model matrix `artifacts/models/matrix_v1` (17 toy models: donor/recipient/unrelated × 3 seeds, w96 ×2, L6 ×2, stride-2 donor/recipient ×2) | completed | `artifacts/models/matrix_v1/index.json` | ≈ 45 s each on MPS; all dev accuracies 1.000 |
46 +| 2026-11-09 | `graft_tier0_quick_7d4278be3e5a` (CPU, 1 seed, n = 300) | completed (55 s) | `experiments/summaries/graft_tier0_quick_7d4278be3e5a/` | pipeline check; revealed the map-suppression effect (D7) |
47 +| 2026-11-09 | `graft_tier0_f50a28ca1bf7` (MPS, 9 pairs, n = 1 000, protocol v1) | completed (140 s) | `experiments/summaries/graft_tier0_f50a28ca1bf7/` | exploratory; see §Transplant results v1 |
48 +| 2026-11-09 | `eval_persistent` on `graft_tier0_f50a28ca1bf7` (fresh process, seed base 70 000) | completed | `experiments/summaries/graft_tier0_f50a28ca1bf7/persistent/fresh_process_eval.json` | saved ridge adapter: target 0.143 vs base 0.118 (no acquisition); control 1.000 |
49 +| 2026-11-09 | `tier1_taskvector_quick_b89f92479155` (MLX, qwen3b→qwen1.5b, 12 queries/task) | completed (150 s) | `experiments/summaries/tier1_taskvector_quick_b89f92479155/` | within-model vectors: antonym 0.75 (L16), capital 1.00 (L20) in Qwen2.5-1.5B; cross-model ≤ 0.25 = unrelated-task control; led to D9 |
50 +
51 +## Transplant results v1 — `graft_tier0_f50a28ca1bf7` (exploratory; superseded by the v2 protocol run)
52 +
53 +Target-stratum accuracy, mean over pairs (same-init n = 3 / different-init n = 6), control in parentheses:
54 +activation graft identity 0.873 (0.99) / 0.101 (0.10); Procrustes on Ω rows 0.785 (0.985) / 0.697 (0.943);
55 +**Procrustes on task-free rows 0.993 (0.996) / 0.956 (0.989)**; ridge on task-free rows 0.814 (1.0) / 0.795 (1.0);
56 +ridge on Ω rows 0.170 / 0.175; CCA 0.08 / 0.089; sham map 0.10 (0.085); norm-matched random 0.089 (0.097);
57 +shuffled token 0.10 (0.10); shuffled layer 0.187 (1.0); unrelated-skill donor 0.076 (1.0); steering α ≤ 4: ≤ 0.082;
58 +subspace k ≤ 128 (ridge reference): ≤ 0.172; adapter graft 0.095; supervised adapter 1.0; distillation 1.0;
59 +weight grafts: L0 MLP 0.262 (0.962), L0 block 0.642 (0.726), L0–L2 blocks + Ω embedding 0.774 (0.749), shuffled
60 +layer 0.083 (0.089), unrelated donor 0.094 (0.933). Recipient baselines 0.111 / 0.047 / 0.067 (chance ≈ 0.10).
61 +Mismatch (Procrustes on Ω rows): width 128→96 0.638 (0.981); depth 4→6 L1→L1 0.988 (1.0), L0→L0 0.358 (0.75).
62 +Stride-2 (ridge): L3 0.604 (0.993). Mechanism: donor attn_out.0 ablated at read positions 0.086 (0.102);
63 +rescue from intact donor_s1 1.000 (1.0); sham rescue 0.112; reverse graft recipient→donor 0.965 (1.0).
64 +Rule-vs-cache (ridge fitted on Ω inputs {0..5}): seen 0.472, unseen 0.740 (consistent with suppression of seen
65 +rows); adapter: seen 0.259, unseen 0.0. Dose (ridge): monotone to 0.34 at α = 1.5, control collapses at α = 2.
66 +Layer grid (ridge): 0.25–0.42, best at recipient L3.
67 +
68 +| 2026-11-09 | `graft_tier0_30e3644aa801` (MPS, protocol v2: task-free Procrustes reference map) | completed (176 s) | `experiments/summaries/graft_tier0_30e3644aa801/` | exploratory; see §Transplant results v2 |
69 +| 2026-11-09 | `eval_persistent` on `graft_tier0_30e3644aa801` (fresh process, seed base 70 000) | completed | `.../persistent/fresh_process_eval.json` | saved donor-free adapter: target 0.414 (Ω-step 0.455) vs base 0.140; control 0.981 |
70 +
71 +## Transplant results v2 — `graft_tier0_30e3644aa801` (exploratory; basis of the v1.0 freeze)
72 +
73 +Same as v1 for the activation-graft families and controls (identical numbers: same models, same splits).
74 +With the task-free Procrustes reference map: subspace grafts k = 1/2/4/8/16/32 → 0.230/0.296/0.650/**0.936**/
75 +0.970/0.983 (random subspace of equal k: 0.076/0.079/0.088/0.085/0.127/0.173; control 1.0 throughout);
76 +adapter graft (donor-free) 0.581 (control 0.998); steering α ≤ 4 ≤ 0.096; layer grid: donor L1–L3 → any
77 +recipient layer 1.00, L0→L0 0.98, L0→L1–3 0.75–0.82; dose: 0.44 (α 0.5), 0.90 (0.75), **0.98 (1.0)**, 0.92 (1.25),
78 +0.75 (1.5), 0.39 (2.0; control 0.36); mismatch with task-free Procrustes: width 96 L0→L0 0.886, **L1→L1 1.000**;
79 +depth 6 L0→L0 0.765, **L1→L1 1.000**, L1→L2 1.000, L2→L2 1.000 (sham 0.07–0.12 with control destroyed);
80 +stride-2: L0 0.370, L1 0.747, **L2 1.000, L3 1.000** (ridge: 0.13–0.60); rule-vs-cache: Procrustes fitted on
81 +seen Ω inputs → seen 0.966 / unseen 0.831 (ratio 0.86); donor-free adapter → seen 0.765 / unseen 0.000 (cache);
82 +mechanism unchanged (ablation 0.086 → rescue 1.000, sham 0.115; reverse 0.965).
83 +
84 +| 2026-11-09 | `graft_tier0_confirmatory_0c2257f6803e` (MPS, PREREGISTRATION v1.0, seed base 90 000) | **completed** | `experiments/summaries/graft_tier0_confirmatory_0c2257f6803e/{results.json,CONFIRMATORY.md,confirmatory.json}` | primary: 9/9 pairs lower bound > SESOI, mean Δ_T^adj 0.875 (sd 0.042, min 0.776) → H2 SUPPORTED; H4 NOT SUPPORTED (one pair control 0.946 < 0.95; others ≥ 0.985); Holm: subspace k8 vs random +0.850, adapter vs base +0.472, weight L0 block vs shuffled +0.556 (retention 0.720), all rejected at Holm; H5 seen 0.941 / unseen 0.768 (ratio 0.816 ≥ 0.8) SUPPORTED; adapter cache 0.702 / 0.000 |
85 +| 2026-11-09 | `eval_persistent` on the confirmatory run (fresh process, seed base 95 000) | completed | `.../persistent/fresh_process_eval.json` | saved adapter: target 0.344 [0.317, 0.372] vs base 0.115; control 0.979 |
modified docs/PREREGISTRATION.md +47 −0
@@ -65,3 +65,50 @@ the run manifest and in `EXPERIMENT_LOG.md`. No run is excluded for its outcome.
65 65 Split hashes are stored in manifests; evaluation splits are generated with seeds disjoint from training
66 66 and alignment seeds; `assert_disjoint` is enforced at build time; alignment data for mappings never
67 67 include evaluation items (checked by item hash).
68 +
69 +---
70 +
71 +# Preregistration v1.0 — CONFIRMATORY FREEZE (Tier 0), 2026-11-09
72 +
73 +Frozen after exploratory runs `graft_tier0_f50a28ca1bf7` (protocol v1) and `graft_tier0_30e3644aa801`
74 +(protocol v2) and **before** any confirmatory split was generated. Confirmatory evaluation and alignment
75 +items come from seed base 90 000 (never used before); the trained models (`artifacts/models/matrix_v1`) are
76 +unchanged; the analysis code (`scripts/run_transplant.py` at the commit recorded in the confirmatory manifest)
77 +is unchanged.
78 +
79 +## Primary method (fixed)
80 +Activation graft, donor in the loop, at the recipient's `resid_post.0` at every read position (untargeted),
81 +α = 1, with the alignment map = **orthogonal Procrustes with isotropic scale fitted on task-free alignment rows**
82 +(3 000 items without Ω, disjoint from evaluation items). Model pairs: the nine donor_s{i} → recipient_s{j}
83 +pairs (3 same-initialization, 6 different-initialization).
84 +
85 +## Primary outcome and contrast
86 +Δ_T^adj on `target_iid` = [Q_T(graft) − Q_T(recipient)] − [Q_T(sham map) − Q_T(recipient)], paired on items,
87 +cluster bootstrap over programs (2 000 draws), reported per pair and summarized over pairs (mean, sd, min).
88 +**Decision rule:** H2 (sufficiency) is supported if the lower 95 % bound of Δ_T^adj exceeds SESOI = 0.25 in at
89 +least 8 of the 9 pairs and the mean over pairs exceeds 0.5; retention H4 is supported if `control_iid` accuracy
90 +with the graft is ≥ 0.95 in all nine pairs.
91 +
92 +## Secondary pre-declared contrasts (Holm correction over these four families, α = 0.05)
93 +1. Subspace graft k = 8 (Ω-contrast PCA through the primary map) vs random subspace k = 8, matched seeds.
94 +2. Adapter graft (donor-free ridge interface to the primary map's targets) vs unmodified recipient; and the
95 + same adapter evaluated in a fresh process on new splits (persistence; `scripts/eval_persistent.py`).
96 +3. Weight graft L0 block (attention + MLP + LN) vs shuffled-layer weight control, matched seeds; retention
97 + reported as a cost.
98 +4. Rule-vs-cache (H5): primary-map graft fitted on Ω inputs {0,…,5} (Procrustes on seen rows) evaluated on
99 + unseen-input items vs seen-input items; supported if unseen ≥ 0.8 × seen. The donor-free adapter is
100 + predicted to FAIL this test (cache), and the failure is reported as such.
101 +
102 +## Mechanism tests (fixed, matched seed 0)
103 +Donor ablation (attn_out.0 zeroed at read positions) → rescue from intact donor_s1 through the task-free
104 +Procrustes map vs sham rescue; reverse graft recipient → donor on control items (no harm expected).
105 +
106 +## Portability (fixed)
107 +Width mismatch (d 128 → 96) and depth mismatch (4 → 6 layers) with the task-free Procrustes map at L1 → L1
108 +(the layer pair selected in exploration) and L0 → L0; stride-2 pair at L2 → L2 and L3 → L3.
109 +
110 +## Exploratory (not confirmatory)
111 +Everything else produced by the script (layer grid, dose, other maps, steering, other k) is reported as
112 +exploratory and labelled so in the paper.
113 +
114 +## Tier 1 (to be frozen in a separate section after the development run completes)
added experiments/manifests/graft_tier0_30e3644aa801.json +90 −0
@@ -0,0 +1,90 @@
1 +{
2 + "config": {
3 + "models": "matrix_v1",
4 + "protocol": "v2-unrelated-procrustes-reference",
5 + "quick": false,
6 + "seed_base": 30000,
7 + "tag": "main"
8 + },
9 + "created_utc": "2026-11-09T07:36:09+00:00",
10 + "data_hashes": {
11 + "align_target": {
12 + "hash": "2aadb9513aeabf54",
13 + "n": 3000,
14 + "seed": 30001
15 + },
16 + "align_unrelated": {
17 + "hash": "19a475dcc593cb0d",
18 + "n": 3000,
19 + "seed": 30002
20 + },
21 + "comp_heldout": {
22 + "hash": "9c6da34a8ad49288",
23 + "n": 1000,
24 + "seed": 30012
25 + },
26 + "control_iid": {
27 + "hash": "98a88deb1a923d81",
28 + "n": 1000,
29 + "seed": 30011
30 + },
31 + "len_ood": {
32 + "hash": "1e5385df81ec873b",
33 + "n": 500,
34 + "seed": 30013
35 + },
36 + "target_iid": {
37 + "hash": "9a69f681d0b62bbb",
38 + "n": 1000,
39 + "seed": 30010
40 + }
41 + },
42 + "experiment_id": "graft_tier0",
43 + "git": {
44 + "branch": "main",
45 + "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
46 + "dirty": true
47 + },
48 + "hardware": {
49 + "chip": "Apple M5 Max",
50 + "cpu_count": 18,
51 + "hostname_hash": "9ad7358c662e",
52 + "machine": "arm64",
53 + "memory_gb": 48.0,
54 + "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
55 + "torch_cuda": false,
56 + "torch_mps": true
57 + },
58 + "history": [
59 + {
60 + "note": "",
61 + "status": "running",
62 + "utc": "2026-11-09T07:36:11+00:00"
63 + },
64 + {
65 + "note": "",
66 + "status": "completed",
67 + "utc": "2026-11-09T07:39:17+00:00"
68 + }
69 + ],
70 + "interpretation": "",
71 + "libraries": {
72 + "mlx": "?",
73 + "mlx_lm": "0.31.3",
74 + "numpy": "2.4.4",
75 + "pydantic": "2.13.4",
76 + "python": "3.14.4",
77 + "safetensors": "0.7.0",
78 + "scipy": "1.17.1",
79 + "torch": "2.12.0",
80 + "transformers": "5.8.1"
81 + },
82 + "outputs": {
83 + "items": "artifacts/runs/graft_tier0_30e3644aa801",
84 + "results": "experiments/summaries/graft_tier0_30e3644aa801/results.json"
85 + },
86 + "run_id": "graft_tier0_30e3644aa801",
87 + "seeds": {},
88 + "status": "completed",
89 + "warnings": []
90 +}
\ No newline at end of file
added experiments/manifests/graft_tier0_confirmatory_0c2257f6803e.json +90 −0
@@ -0,0 +1,90 @@
1 +{
2 + "config": {
3 + "models": "matrix_v1",
4 + "protocol": "v2-unrelated-procrustes-reference",
5 + "quick": false,
6 + "seed_base": 90000,
7 + "tag": "confirmatory"
8 + },
9 + "created_utc": "2026-11-09T07:41:03+00:00",
10 + "data_hashes": {
11 + "align_target": {
12 + "hash": "3afb3d9322bd4e85",
13 + "n": 3000,
14 + "seed": 90001
15 + },
16 + "align_unrelated": {
17 + "hash": "ce28064949d5c649",
18 + "n": 3000,
19 + "seed": 90002
20 + },
21 + "comp_heldout": {
22 + "hash": "89f1988aff8d8535",
23 + "n": 1000,
24 + "seed": 90012
25 + },
26 + "control_iid": {
27 + "hash": "ad4f0fde325b0378",
28 + "n": 1000,
29 + "seed": 90011
30 + },
31 + "len_ood": {
32 + "hash": "c1179ed20c26701a",
33 + "n": 500,
34 + "seed": 90013
35 + },
36 + "target_iid": {
37 + "hash": "c104c5c2174007f6",
38 + "n": 1000,
39 + "seed": 90010
40 + }
41 + },
42 + "experiment_id": "graft_tier0_confirmatory",
43 + "git": {
44 + "branch": "main",
45 + "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
46 + "dirty": true
47 + },
48 + "hardware": {
49 + "chip": "Apple M5 Max",
50 + "cpu_count": 18,
51 + "hostname_hash": "9ad7358c662e",
52 + "machine": "arm64",
53 + "memory_gb": 48.0,
54 + "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
55 + "torch_cuda": false,
56 + "torch_mps": true
57 + },
58 + "history": [
59 + {
60 + "note": "",
61 + "status": "running",
62 + "utc": "2026-11-09T07:41:05+00:00"
63 + },
64 + {
65 + "note": "",
66 + "status": "completed",
67 + "utc": "2026-11-09T07:44:03+00:00"
68 + }
69 + ],
70 + "interpretation": "",
71 + "libraries": {
72 + "mlx": "?",
73 + "mlx_lm": "0.31.3",
74 + "numpy": "2.4.4",
75 + "pydantic": "2.13.4",
76 + "python": "3.14.4",
77 + "safetensors": "0.7.0",
78 + "scipy": "1.17.1",
79 + "torch": "2.12.0",
80 + "transformers": "5.8.1"
81 + },
82 + "outputs": {
83 + "items": "artifacts/runs/graft_tier0_confirmatory_0c2257f6803e",
84 + "results": "experiments/summaries/graft_tier0_confirmatory_0c2257f6803e/results.json"
85 + },
86 + "run_id": "graft_tier0_confirmatory_0c2257f6803e",
87 + "seeds": {},
88 + "status": "completed",
89 + "warnings": []
90 +}
\ No newline at end of file
added experiments/manifests/graft_tier0_f50a28ca1bf7.json +89 −0
@@ -0,0 +1,89 @@
1 +{
2 + "config": {
3 + "models": "matrix_v1",
4 + "quick": false,
5 + "seed_base": 30000,
6 + "tag": "main"
7 + },
8 + "created_utc": "2026-11-09T07:31:00+00:00",
9 + "data_hashes": {
10 + "align_target": {
11 + "hash": "2aadb9513aeabf54",
12 + "n": 3000,
13 + "seed": 30001
14 + },
15 + "align_unrelated": {
16 + "hash": "19a475dcc593cb0d",
17 + "n": 3000,
18 + "seed": 30002
19 + },
20 + "comp_heldout": {
21 + "hash": "9c6da34a8ad49288",
22 + "n": 1000,
23 + "seed": 30012
24 + },
25 + "control_iid": {
26 + "hash": "98a88deb1a923d81",
27 + "n": 1000,
28 + "seed": 30011
29 + },
30 + "len_ood": {
31 + "hash": "1e5385df81ec873b",
32 + "n": 500,
33 + "seed": 30013
34 + },
35 + "target_iid": {
36 + "hash": "9a69f681d0b62bbb",
37 + "n": 1000,
38 + "seed": 30010
39 + }
40 + },
41 + "experiment_id": "graft_tier0",
42 + "git": {
43 + "branch": "main",
44 + "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
45 + "dirty": true
46 + },
47 + "hardware": {
48 + "chip": "Apple M5 Max",
49 + "cpu_count": 18,
50 + "hostname_hash": "9ad7358c662e",
51 + "machine": "arm64",
52 + "memory_gb": 48.0,
53 + "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
54 + "torch_cuda": false,
55 + "torch_mps": true
56 + },
57 + "history": [
58 + {
59 + "note": "",
60 + "status": "running",
61 + "utc": "2026-11-09T07:31:02+00:00"
62 + },
63 + {
64 + "note": "",
65 + "status": "completed",
66 + "utc": "2026-11-09T07:33:33+00:00"
67 + }
68 + ],
69 + "interpretation": "",
70 + "libraries": {
71 + "mlx": "?",
72 + "mlx_lm": "0.31.3",
73 + "numpy": "2.4.4",
74 + "pydantic": "2.13.4",
75 + "python": "3.14.4",
76 + "safetensors": "0.7.0",
77 + "scipy": "1.17.1",
78 + "torch": "2.12.0",
79 + "transformers": "5.8.1"
80 + },
81 + "outputs": {
82 + "items": "artifacts/runs/graft_tier0_f50a28ca1bf7",
83 + "results": "experiments/summaries/graft_tier0_f50a28ca1bf7/results.json"
84 + },
85 + "run_id": "graft_tier0_f50a28ca1bf7",
86 + "seeds": {},
87 + "status": "completed",
88 + "warnings": []
89 +}
\ No newline at end of file
added experiments/manifests/graft_tier0_quick_7d4278be3e5a.json +89 −0
@@ -0,0 +1,89 @@
1 +{
2 + "config": {
3 + "models": "matrix_v1",
4 + "quick": true,
5 + "seed_base": 30000,
6 + "tag": "main"
7 + },
8 + "created_utc": "2026-11-09T07:25:13+00:00",
9 + "data_hashes": {
10 + "align_target": {
11 + "hash": "2e9e93027c216aa2",
12 + "n": 600,
13 + "seed": 30001
14 + },
15 + "align_unrelated": {
16 + "hash": "4467a0296d6dbcc4",
17 + "n": 600,
18 + "seed": 30002
19 + },
20 + "comp_heldout": {
21 + "hash": "51ad8603009a9dd2",
22 + "n": 300,
23 + "seed": 30012
24 + },
25 + "control_iid": {
26 + "hash": "b8916fa4b7243e6f",
27 + "n": 300,
28 + "seed": 30011
29 + },
30 + "len_ood": {
31 + "hash": "e21b8c770ad0b017",
32 + "n": 200,
33 + "seed": 30013
34 + },
35 + "target_iid": {
36 + "hash": "50bc867931acc1be",
37 + "n": 300,
38 + "seed": 30010
39 + }
40 + },
41 + "experiment_id": "graft_tier0_quick",
42 + "git": {
43 + "branch": "main",
44 + "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
45 + "dirty": true
46 + },
47 + "hardware": {
48 + "chip": "Apple M5 Max",
49 + "cpu_count": 18,
50 + "hostname_hash": "9ad7358c662e",
51 + "machine": "arm64",
52 + "memory_gb": 48.0,
53 + "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
54 + "torch_cuda": false,
55 + "torch_mps": true
56 + },
57 + "history": [
58 + {
59 + "note": "",
60 + "status": "running",
61 + "utc": "2026-11-09T07:25:15+00:00"
62 + },
63 + {
64 + "note": "",
65 + "status": "completed",
66 + "utc": "2026-11-09T07:26:11+00:00"
67 + }
68 + ],
69 + "interpretation": "",
70 + "libraries": {
71 + "mlx": "?",
72 + "mlx_lm": "0.31.3",
73 + "numpy": "2.4.4",
74 + "pydantic": "2.13.4",
75 + "python": "3.14.4",
76 + "safetensors": "0.7.0",
77 + "scipy": "1.17.1",
78 + "torch": "2.12.0",
79 + "transformers": "5.8.1"
80 + },
81 + "outputs": {
82 + "items": "artifacts/runs/graft_tier0_quick_7d4278be3e5a",
83 + "results": "experiments/summaries/graft_tier0_quick_7d4278be3e5a/results.json"
84 + },
85 + "run_id": "graft_tier0_quick_7d4278be3e5a",
86 + "seeds": {},
87 + "status": "completed",
88 + "warnings": []
89 +}
\ No newline at end of file
added experiments/manifests/tier1_taskvector_e105e3d91386.json +56 −0
@@ -0,0 +1,56 @@
1 +{
2 + "config": {
3 + "k": 8,
4 + "pairs": [
5 + "qwen3b:qwen1.5b",
6 + "qwen1.5b:qwen3b",
7 + "llama3b:qwen3b",
8 + "qwen3b:llama3b"
9 + ],
10 + "quick": false,
11 + "split": "dev",
12 + "task_seed": 0
13 + },
14 + "created_utc": "2026-11-09T07:46:30+00:00",
15 + "data_hashes": {},
16 + "experiment_id": "tier1_taskvector",
17 + "git": {
18 + "branch": "main",
19 + "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
20 + "dirty": true
21 + },
22 + "hardware": {
23 + "chip": "Apple M5 Max",
24 + "cpu_count": 18,
25 + "hostname_hash": "9ad7358c662e",
26 + "machine": "arm64",
27 + "memory_gb": 48.0,
28 + "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
29 + "torch_cuda": false,
30 + "torch_mps": true
31 + },
32 + "history": [
33 + {
34 + "note": "",
35 + "status": "running",
36 + "utc": "2026-11-09T07:46:33+00:00"
37 + }
38 + ],
39 + "interpretation": "",
40 + "libraries": {
41 + "mlx": "?",
42 + "mlx_lm": "0.31.3",
43 + "numpy": "2.4.4",
44 + "pydantic": "2.13.4",
45 + "python": "3.14.4",
46 + "safetensors": "0.7.0",
47 + "scipy": "1.17.1",
48 + "torch": "2.12.0",
49 + "transformers": "5.8.1"
50 + },
51 + "outputs": {},
52 + "run_id": "tier1_taskvector_e105e3d91386",
53 + "seeds": {},
54 + "status": "running",
55 + "warnings": []
56 +}
\ No newline at end of file
added experiments/manifests/tier1_taskvector_quick_b89f92479155.json +77 −0
@@ -0,0 +1,77 @@
1 +{
2 + "config": {
3 + "k": 8,
4 + "pairs": [
5 + "qwen3b:qwen1.5b"
6 + ],
7 + "quick": true,
8 + "split": "dev",
9 + "task_seed": 0
10 + },
11 + "created_utc": "2026-11-09T07:29:41+00:00",
12 + "data_hashes": {
13 + "addthree": {
14 + "hash": "63af249180d379b8",
15 + "n_queries": 12
16 + },
17 + "antonym": {
18 + "hash": "22992056ec21621c",
19 + "n_queries": 12
20 + },
21 + "capital": {
22 + "hash": "0ce7fe08025fd9c5",
23 + "n_queries": 12
24 + },
25 + "morph": {
26 + "hash": "313569abbc843a4f",
27 + "n_queries": 12
28 + }
29 + },
30 + "experiment_id": "tier1_taskvector_quick",
31 + "git": {
32 + "branch": "main",
33 + "commit": "3c8aa79b74d2cc497e2fb506375ad8b3d2c30882",
34 + "dirty": true
35 + },
36 + "hardware": {
37 + "chip": "Apple M5 Max",
38 + "cpu_count": 18,
39 + "hostname_hash": "9ad7358c662e",
40 + "machine": "arm64",
41 + "memory_gb": 48.0,
42 + "platform": "macOS-27.0-arm64-arm-64bit-Mach-O",
43 + "torch_cuda": false,
44 + "torch_mps": true
45 + },
46 + "history": [
47 + {
48 + "note": "",
49 + "status": "running",
50 + "utc": "2026-11-09T07:29:44+00:00"
51 + },
52 + {
53 + "note": "",
54 + "status": "completed",
55 + "utc": "2026-11-09T07:32:14+00:00"
56 + }
57 + ],
58 + "interpretation": "",
59 + "libraries": {
60 + "mlx": "?",
61 + "mlx_lm": "0.31.3",
62 + "numpy": "2.4.4",
63 + "pydantic": "2.13.4",
64 + "python": "3.14.4",
65 + "safetensors": "0.7.0",
66 + "scipy": "1.17.1",
67 + "torch": "2.12.0",
68 + "transformers": "5.8.1"
69 + },
70 + "outputs": {
71 + "results": "experiments/summaries/tier1_taskvector_quick_b89f92479155/results.json"
72 + },
73 + "run_id": "tier1_taskvector_quick_b89f92479155",
74 + "seeds": {},
75 + "status": "completed",
76 + "warnings": []
77 +}
\ No newline at end of file
modified experiments/registry/runs.jsonl +19 −0
@@ -4,3 +4,22 @@
4 4 {"config_hash": "451ef5087182", "experiment_id": "pilot_tier0_opseq", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "planned", "utc": "2026-11-09T06:49:37+00:00"}
5 5 {"experiment_id": "pilot_tier0_opseq", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "running", "utc": "2026-11-09T06:49:38+00:00"}
6 6 {"experiment_id": "pilot_tier0_opseq", "results": "experiments/summaries/pilot_tier0_opseq_451ef5087182/results.json", "run_id": "pilot_tier0_opseq_451ef5087182", "status": "completed", "utc": "2026-11-09T06:54:13+00:00"}
7 +{"config_hash": "7d4278be3e5a", "experiment_id": "graft_tier0_quick", "run_id": "graft_tier0_quick_7d4278be3e5a", "status": "planned", "utc": "2026-11-09T07:25:15+00:00"}
8 +{"experiment_id": "graft_tier0_quick", "run_id": "graft_tier0_quick_7d4278be3e5a", "status": "running", "utc": "2026-11-09T07:25:15+00:00"}
9 +{"experiment_id": "graft_tier0_quick", "results": "experiments/summaries/graft_tier0_quick_7d4278be3e5a/results.json", "run_id": "graft_tier0_quick_7d4278be3e5a", "status": "completed", "utc": "2026-11-09T07:26:11+00:00"}
10 +{"config_hash": "b89f92479155", "experiment_id": "tier1_taskvector_quick", "run_id": "tier1_taskvector_quick_b89f92479155", "status": "planned", "utc": "2026-11-09T07:29:44+00:00"}
11 +{"config_hash": "f50a28ca1bf7", "experiment_id": "graft_tier0", "run_id": "graft_tier0_f50a28ca1bf7", "status": "planned", "utc": "2026-11-09T07:31:02+00:00"}
12 +{"experiment_id": "graft_tier0", "run_id": "graft_tier0_f50a28ca1bf7", "status": "running", "utc": "2026-11-09T07:31:02+00:00"}
13 +{"experiment_id": "tier1_taskvector_quick", "run_id": "tier1_taskvector_quick_b89f92479155", "status": "completed", "utc": "2026-11-09T07:32:14+00:00"}
14 +{"experiment_id": "graft_tier0", "results": "experiments/summaries/graft_tier0_f50a28ca1bf7/results.json", "run_id": "graft_tier0_f50a28ca1bf7", "status": "completed", "utc": "2026-11-09T07:33:33+00:00"}
15 +{"config_hash": "e105e3d91386", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "planned", "utc": "2026-11-09T07:34:21+00:00"}
16 +{"error": "AttributeError(\"'WrappedBlock' object has no attribute 'use_sliding'\")", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "failed", "utc": "2026-11-09T07:34:22+00:00"}
17 +{"config_hash": "30e3644aa801", "experiment_id": "graft_tier0", "run_id": "graft_tier0_30e3644aa801", "status": "planned", "utc": "2026-11-09T07:36:11+00:00"}
18 +{"experiment_id": "graft_tier0", "run_id": "graft_tier0_30e3644aa801", "status": "running", "utc": "2026-11-09T07:36:11+00:00"}
19 +{"config_hash": "e105e3d91386", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "planned", "utc": "2026-11-09T07:36:38+00:00"}
20 +{"experiment_id": "graft_tier0", "results": "experiments/summaries/graft_tier0_30e3644aa801/results.json", "run_id": "graft_tier0_30e3644aa801", "status": "completed", "utc": "2026-11-09T07:39:17+00:00"}
21 +{"config_hash": "0c2257f6803e", "experiment_id": "graft_tier0_confirmatory", "run_id": "graft_tier0_confirmatory_0c2257f6803e", "status": "planned", "utc": "2026-11-09T07:41:05+00:00"}
22 +{"experiment_id": "graft_tier0_confirmatory", "run_id": "graft_tier0_confirmatory_0c2257f6803e", "status": "running", "utc": "2026-11-09T07:41:05+00:00"}
23 +{"experiment_id": "graft_tier0_confirmatory", "results": "experiments/summaries/graft_tier0_confirmatory_0c2257f6803e/results.json", "run_id": "graft_tier0_confirmatory_0c2257f6803e", "status": "completed", "utc": "2026-11-09T07:44:03+00:00"}
24 +{"error": "ValueError(\"invalid literal for int() with base 10: 'rocrustes'\")", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "failed", "utc": "2026-11-09T07:45:09+00:00"}
25 +{"config_hash": "e105e3d91386", "experiment_id": "tier1_taskvector", "run_id": "tier1_taskvector_e105e3d91386", "status": "planned", "utc": "2026-11-09T07:46:33+00:00"}
added experiments/summaries/graft_tier0_30e3644aa801/persistent/adapter_graft.npz +0 −0

Binary file not shown.

added experiments/summaries/graft_tier0_30e3644aa801/persistent/fresh_process_eval.json +41 −0
@@ -0,0 +1,41 @@
1 +{
2 + "utc": "2026-11-09T07:40:13+00:00",
3 + "run": "graft_tier0_30e3644aa801",
4 + "recipient": "recipient_s0",
5 + "hook": "resid_post.0",
6 + "fresh_process": true,
7 + "splits": {
8 + "target_iid": "e1cb4467208e4c8c",
9 + "control_iid": "ddb8b86e4022dbbb"
10 + },
11 + "target_iid": {
12 + "base": 0.14,
13 + "with_saved_adapter": 0.414,
14 + "ci95_with_saved_adapter": {
15 + "estimate": 0.414,
16 + "low": 0.38347969488028666,
17 + "high": 0.4449116790506519,
18 + "level": 0.95,
19 + "n_items": 1000,
20 + "n_clusters": 762,
21 + "n_boot": 2000
22 + },
23 + "omega_step_base": 0.118,
24 + "omega_step_with_saved_adapter": 0.455
25 + },
26 + "control_iid": {
27 + "base": 1.0,
28 + "with_saved_adapter": 0.981,
29 + "ci95_with_saved_adapter": {
30 + "estimate": 0.981,
31 + "low": 0.972,
32 + "high": 0.9890219560878244,
33 + "level": 0.95,
34 + "n_items": 1000,
35 + "n_clusters": 965,
36 + "n_boot": 2000
37 + },
38 + "omega_step_base": null,
39 + "omega_step_with_saved_adapter": null
40 + }
41 +}
\ No newline at end of file
added experiments/summaries/graft_tier0_30e3644aa801/results.json +19330 −0
Large diff (19,330 lines) is not loaded by default to keep the page fast.Load diff

Diff truncated — file too large.

Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere