SPB Git

spb/wpai0001_uqo

Public
9.0 KB

# 🧠 The Neural Organ Transplant Experiment

UQO Working Paper No. AI-0001 — Causal Transfer of Cognitive Capabilities Across Local Large Language Models

Paper Python Tests Reproducible Institution Author

TL;DR — Can a bounded, causally identified subsystem that implements a measurable capability in a donor language model be transplanted into a recipient that demonstrably lacks it, without full-model training? This repository holds the preregistered protocol, the experiment harness (toy transformers with exact hook semantics; MLX/Transformers backends planned), the registry of runs and the paper. Result (Tier 0, preregistered confirmatory run, 2026-11-09): a donor-in-the-loop activation graft — the recipient's post-layer-0 residual row at every read position replaced by the donor's row through an orthogonal Procrustes map fitted on task-free rows — lifts the recipient from chance (0.06–0.10) to 0.966 on held-out Ω programs (control-adjusted effect +0.875, lower bound above 0.25 in 9/9 pairs; control stratum 0.992), carries the rule to unseen operands (0.77 vs 0.94), needs eight of 128 dimensions, survives width, depth and stride-2 mismatches, and is anchored by ablation → rescue (0.11 → 1.00; sham 0.11). Regression-fitted maps (ridge, CCA) annihilate the capability (Proposition 1); donor-free adapters cache; conventional learning on the same items is perfect. Tier 1 (Qwen2.5-1.5B/3B, Llama-3.2-3B) and Tier 2 (Qwen2.5-7B) runs: see status.


# 📌 Status

Item Value
Version 0.1 protocol draft (November 2026)
SSRN not yet submitted
Journal to be assessed after results (ML / mechanistic interpretability venue)
Code & data synthetic data regenerated from seeds; every run has a manifest in experiments/manifests/
Gates A ✔ · B ✔ · C ✔ · D ✔ · E ✔ (Tier 0 confirmatory) · F in progress

# 🎯 What this paper does

Prior work maps whole residual streams or concept directions between models (model stitching, activation buses, cross-model steering) and the one controlled attempt at activation-mediated capability transfer reports a null. We (i) causally localize a capability in the donor by interchange interventions, (ii) manufacture or verify the recipient's lack of it, and (iii) compare each transplant family (activation, subspace, circuit, weight, adapter, persistent) against matched sham, random and unrelated-donor controls on held-out, leakage-controlled tasks. Estimands:

text
Δ_T      = Q_T(R') − Q_T(R)                         transfer gain
Δ_T^adj  = Δ_T − [Q_T(R_sham) − Q_T(R)]             control-adjusted transfer (primary)
ρ_c      = Q_c(R') / Q_c(R)                         retention on control strata
N_D      = Q_T(D) − Q_T(D \ S_D)                    donor necessity

# 🏆 Headline results

Object Estimate (run)
Donor vs recipient, target stratum 1.000 vs 0.092; Ω-step 1.000 vs 0.062; control 1.000 both (pilot_tier0_opseq_451ef5087182)
Localization of Ω in the donor residual row at the read position restores 1.00 at every layer; L0 attention 1.00; L0–L2 MLPs 1.00/0.90/0.83; sham 0.00 (pilot)
Primary graft (confirmatory) target 0.966 (min 0.889), control 0.992 (min 0.946); Δ_T^adj +0.875, lower bound > 0.25 in 9/9 pairs → H2 supported; H4 (≥ 0.95 retention everywhere) not supported (graft_tier0_confirmatory_0c2257f6803e)
Controls (confirmatory) sham map 0.091 (control 0.100), norm-matched random 0.091, shuffled item 0.103, shuffled layer 0.186, unrelated-skill donor 0.076
Subspace k = 8 vs random k = 8 0.927 vs 0.077 (+0.850 [+0.837, +0.862])
Rule vs cache graft 0.941 seen / 0.768 unseen (ratio 0.82, H5 supported); donor-free adapter 0.702 / 0.000
Ablation → rescue 0.113 → 1.000 (sham 0.106)
Conventional learning on the same 3 000 items supervised adapter 1.000; distillation 1.000
Tier 1 / Tier 2 runs in progress (tier1_taskvector_*)

# 🗂 Repository layout

text
wp-ai-0001/
├── CLAUDE.md · docs/MANDATE.md       # binding rules and the verbatim research mandate
├── src/wp_ai_0001/                   # datasets (OPSEQ/FSA/MORPH), models + hooks, training, localization,
│                                     # evaluation, statistics, instrumentation, orchestration, visualization
├── tests/{unit,causal_sanity,...}    # generators, bootstrap, hook no-op and intervention algebra
├── scripts/                          # audit_environment, run_pilot, build_figures, build_paper
├── experiments/{registry,manifests,summaries}
├── artifacts/                        # models, per-item outputs, figures (git-ignored)
├── docs/                             # RESEARCH_GAP, PREREGISTRATION, EXPERIMENT_PLAN, COMPUTE_BUDGET,
│                                     # RISK_REGISTER, REPRODUCIBILITY, EXPERIMENT_LOG, DECISION_LOG, TASKS
├── literature/                       # verified arXiv records + BibTeX; unverified candidates listed apart
└── paper/                            # main.tex + sections/ + references.bib → main.pdf

# 🚀 Quick start

bash
python3 -m pip install -e ".[dev]"
python3 -m pytest tests -q
python3 scripts/audit_environment.py --cluster
python3 scripts/run_pilot.py --quick --device cpu --seeds 0 1 --n-boot 200   # ≈ 25 s CPU
python3 scripts/run_pilot.py --seeds 0 1 2 --steps 3000                        # ≈ 5 min on an M5 Max (MPS)
python3 scripts/build_figures.py --run <run_id> && scripts/build_paper.sh

# 🗃 Data

Synthetic micro-worlds generated from seeds (src/wp_ai_0001/datasets/), validated by exact recomputation, hashed and checked for item-level disjointness across splits; see data/README.md. Tier-1 checkpoints (Qwen2.5-1.5B/3B-Instruct, Llama-3.2-3B-Instruct, bf16) are cached locally and not redistributed.

# 🔬 Methodology

  1. Gap qualification (Phase 1) — scripts/run_pilot.py, training/train_toy.py, evaluation/metrics.py.
  2. Causal localization (Phase 2) — localization/patching.py (interchange, ablation, heads).
  3. Alignment (Phase 3) — alignment/ (to implement: Procrustes, ridge, CCA, random, unrelated-data).
  4. Transplant families and controls (Phases 4–5) — transplantation/ (to implement).
  5. Robustness, mechanism, confirmatory (Phases 6–8) — planned, see docs/EXPERIMENT_PLAN.md.
  6. Statistics — statistics/bootstrap.py (cluster bootstrap over programs; seed-level summaries).

# 🖼 Figures & tables inventory

Figure Content Source run
fig_gap_tier0.pdf donor/recipient accuracy per stratum, seeds, chance pilot_tier0_opseq_451ef5087182
fig_localization_patching_tier0.pdf layer × position restored-answer rate, 3 families idem
fig_localization_ablation_tier0.pdf layer × position accuracy after zero-ablation idem
fig_localization_heads_tier0.pdf layer × head patching at Ω token and read position idem

# ⚠️ Limitations

  • Tier-0 capability is a per-step map once the trace is visible (risk R1); multi-step claims need trace_stride > 1 and Tier ≥ 1 evidence.
  • No transplant, alignment or control battery has been executed; the paper contains placeholders.
  • Length generalization fails for both roles (absolute positions); it is a stratum, not a target.

# 📝 Citation

bibtex
@techreport{boucher2026organtransplant,
  author      = {Boucher, Simon-Pierre},
  title       = {The Neural Organ Transplant Experiment: Causal Transfer of Cognitive Capabilities Across Local Large Language Models},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {AI-0001},
  year        = {2026},
  month       = {November}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

  • Numbering: project id wp-ai-0001 (series "AI", number 0001) as requested in the mandate; repository wpai0001_uqo.git on spbgit, following the FIRC-0001 precedent rather than the wpN sequence.
  • Deviation from the template layout: src/wp_ai_0001/training/ added (Tier-0 models are trained here); experiment artifacts live under experiments/ and artifacts/ per the mandate instead of results/.
Files · 7697ca59b784d7876f137086bc6c7bcdc2ef4600
Menu

Theme