SPB Git

spb/modelmap Public License

Internal cartography of local LLMs on Apple Silicon — registered, gated, negative-first. Public atlas at modelmap.io.

🔗 https://modelmap.io
interpretabilityllmapple-siliconmlxreproducibility
Python 66.3% JavaScript 24.5% CSS 8.1% Shell 0.7%

expC run #5: L3 gate refused — L12 is a textbook handle anyway

Conjunctive dose-response criterion failed: L12 passes everything
(strictly monotone +1.95..+6.79 across -2s..+2s, halved, specific 0.45
vs bound 1.11) but L04 is an overdose regime (any 2-sigma perturbation,
random included, collapses the margin) and L08 is not specific at 2s.
make_l3_mapcard.py refused v2; interventions/v1 stays Level 2. Gate
record: refuse (r3) -> pass (r4, L2) -> refuse (r5). Run #6 registered
ideas: layer-local dose calibration; minimal L12 single-layer L3 claim.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed 2 h ago (Aug 12, 2026) parent 3eb9df2

Showing 6 changed files with +571 and −0

modified experiments/micro/expC_causal_verification/analysis.md +53 −0
@@ -220,3 +220,56 @@ Next experiment : (1) L3 path: steering (add the direction) should
220 220 (4) cross-model: does the band replicate on
221 221 Qwen3-1.7B (expG entry)?
222 222 ```
223 +
224 +---
225 +
226 +# Analysis — expC run #5: Level-3 gate refused — and layer 12 is a textbook handle
227 +
228 +Run: `results/expC_causal_verification/20260812T072455Z/results.json`.
229 +Steering doses α·σℓ·u at layers {4, 8, 12}, α ∈ {−2…+2}; conjunctive
230 +criterion (monotone AND halved AND specific at EVERY test layer) registered
231 +before the run. make_l3_mapcard.py refused (exit 1); v1 stays Level 2.
232 +
233 +```text
234 +Hypothesis : strict dose-response monotonicity + halving +
235 + random-direction specificity at all of L4/L8/L12.
236 +Result : FALSIFIED as a conjunction.
237 + L12 — PASSES EVERYTHING, textbook: margins
238 + +1.95 / +3.00 / +4.45 / +5.82 / +6.79 across
239 + −2σ…+2σ (strictly monotone), halved at −2σ,
240 + random-direction |Δ| = 0.45 vs bound 1.11.
241 + L08 — near-monotone (+2σ dips: 5.70 < 6.26),
242 + random |Δ| 2.97: not specific at this dose.
243 + L04 — OVERDOSE REGIME: ±2σ both collapse the
244 + margin (+0.08 / +0.04) and random directions are
245 + equally destructive (|Δ| 4.40): at early layers,
246 + ANY perturbation of magnitude ~2σ (σ estimated
247 + from mean-pooled reps) breaks the computation —
248 + the rank-1 fragility of run #2, now dose-resolved.
249 +Interpretation : Level 0–1. The direction is a clean, dose-
250 + controlled causal handle at mid-band (L12) but
251 + the conjunctive claim over the whole test set
252 + fails, so the gate held v2 back — correctly.
253 + Reading: "necessity everywhere in the band"
254 + (erasure, L2 entry) coexists with "controllable
255 + handle only where the layer tolerates
256 + perturbation". Dose scale is a confound at early
257 + layers: σ from mean-pooled statistics likely
258 + overdoses positions at layers with different
259 + norm profiles.
260 +Next experiment : run #6 (registered idea): layer-local dose
261 + calibration (σ from position-level projections at
262 + the target layer; or a dose-sweep to find each
263 + layer's non-destructive range), then re-register
264 + the handle claim on the sub-band that tolerates
265 + calibrated doses (candidate: 10–14). Also register
266 + the L12 single-layer handle claim with fresh
267 + direction estimates as a minimal L3 candidate.
268 +```
269 +
270 +## Gate record (for methodology.md)
271 +
272 +Three refusals/passes to date, all pre-registered: run #3 per-layer profile
273 +REFUSED → run #4 band claim PASSED (Level 2 published) → run #5 L3
274 +conjunction REFUSED (v1 unchanged). The atlas never received a claim its
275 +evidence didn't carry.
modified experiments/micro/expC_causal_verification/hypothesis.md +39 −0
@@ -177,3 +177,42 @@ Result : (pending)
177 177 Interpretation : (pending)
178 178 Next experiment : (pending)
179 179 ```
180 +
181 +---
182 +
183 +# Hypothesis — expC run #5 (Level-3 path: dose-response steering)
184 +
185 +Registered 2026-08-12 **before** the run. Erasure (runs #2–#4) established
186 +necessity of the agreement direction in the early band. Steering (activation
187 +addition) is the INDEPENDENT intervention family required for Level 3: if
188 +the direction is the causal handle, pushing along it must move the behavior
189 +in the predicted direction, dose-dependently, and random directions must not.
190 +
191 +```text
192 +Hypothesis : At each of three early-band test layers (4, 8, 12),
193 + adding δ = α·σℓ·u to every position of the layer
194 + output produces a STRICTLY MONOTONE margin in
195 + α ∈ {−2, −1, 0, +1, +2} (σℓ = std of activation
196 + projections on u at layer ℓ), with effect size:
197 + margin(α=−2) ≤ 0.5 × baseline at every test layer.
198 + Specificity: the same ±2σ doses along 3 random
199 + directions move the margin by < 25% of baseline
200 + (mean absolute change), at every test layer.
201 +Falsification criterion : monotonicity broken at any test layer, OR the −2σ
202 + dose fails to halve the margin anywhere, OR random
203 + directions move the margin ≥ 25% — steering fails,
204 + the entry stays Level 2, and the diff-of-means
205 + direction is declared necessary-but-not-a-handle.
206 +Method : direction u and σℓ estimated from agreement_A
207 + (full set — the published v1 object); behavioral
208 + bank = run #4's fresh bank (never used for
209 + estimation); doses applied at one layer at a time.
210 +Publication rule : pass → atlas interventions/v2 at LEVEL 3 (erasure
211 + necessity + steering dose-response = two
212 + independent intervention families + probing;
213 + v1 stays as the Level-2 record). Fail → documented,
214 + v1 unchanged.
215 +Result : (pending)
216 +Interpretation : (pending)
217 +Next experiment : (pending)
218 +```
added experiments/micro/expC_causal_verification/implementation/benchmark_v5.py +166 −0
@@ -0,0 +1,166 @@
1 +#!/usr/bin/env python3
2 +# =============================================================================
3 +# Project : modelmap
4 +# File : experiments/micro/expC_causal_verification/implementation/benchmark_v5.py
5 +# Purpose : Run #5 — Level-3 path: dose-response steering of the direction
6 +# Author : Simon-Pierre Boucher
7 +# Contact : contact@spboucher.ai
8 +# Website : https://modelmap.io
9 +# Created : 2026-08-12
10 +# Modified : 2026-08-12
11 +# Platform : macOS / Apple Silicon (arm64) — MLX / Metal
12 +# License : All rights reserved (research code)
13 +# =============================================================================
14 +"""expC run #5 (hypothesis registered before this run).
15 +
16 +Activation-addition steering of the diff-of-means agreement direction at
17 +three early-band layers, doses alpha in {-2,-1,0,+1,+2} x sigma_l; strict
18 +monotonicity + halving criterion + random-direction specificity control.
19 +"""
20 +
21 +from __future__ import annotations
22 +
23 +import json
24 +import random
25 +import subprocess
26 +import sys
27 +import time
28 +from pathlib import Path
29 +
30 +import numpy as np
31 +
32 +ROOT = Path(__file__).resolve().parents[4]
33 +sys.path.insert(0, str(ROOT / "src"))
34 +sys.path.insert(0, str(ROOT / "benchmarks"))
35 +from hardware_manifest import manifest
36 +
37 +from modelmap.capture.mlx_capture import capture_pooled, install_taps
38 +
39 +MODEL = "mlx-community/Qwen3-0.6B-4bit"
40 +TEST_LAYERS = (4, 8, 12)
41 +DOSES = (-2.0, -1.0, 0.0, 1.0, 2.0)
42 +N_RANDOM_DIRS = 3
43 +SEED = 781
44 +
45 +NOUN_PAIRS = [("key", "keys"), ("crate", "crates"), ("report", "reports"), ("valve", "valves"),
46 + ("ticket", "tickets"), ("ladder", "ladders"), ("sample", "samples"), ("cable", "cables"),
47 + ("permit", "permits"), ("beacon", "beacons"), ("filter", "filters"), ("stamp", "stamps")]
48 +FRESH_NEAR = ["past the boiler room", "near the loading dock", "behind the ticket booth",
49 + "under the mezzanine", "beside the flagpole", "opposite the greenhouse",
50 + "inside the stairwell", "along the towpath"]
51 +
52 +
53 +def main() -> int:
54 + import mlx.core as mx
55 + from mlx_lm import load
56 +
57 + t0 = time.time()
58 + model, tokenizer = load(MODEL)
59 + taps = install_taps(model)
60 +
61 + rng = random.Random(SEED)
62 + combos = [(n, loc) for n in NOUN_PAIRS for loc in FRESH_NEAR]
63 + rng.shuffle(combos)
64 + pairs = []
65 + for (sg, pl), loc in combos:
66 + pairs.append({"prefix": f"The {sg} {loc}", "singular": True})
67 + pairs.append({"prefix": f"The {pl} {loc}", "singular": False})
68 + prefix_ids = [tokenizer.encode(p["prefix"]) for p in pairs]
69 + id_is, id_are = tokenizer.encode(" is")[0], tokenizer.encode(" are")[0]
70 +
71 + def margin() -> float:
72 + out = []
73 + for ids, p in zip(prefix_ids, pairs):
74 + logits = model(mx.array([ids]))[0, -1, :]
75 + mx.eval(logits)
76 + m = float(logits[id_is] - logits[id_are])
77 + out.append(m if p["singular"] else -m)
78 + return float(np.mean(out))
79 +
80 + # direction + per-layer projection std from agreement_A (the v1 object)
81 + items = [json.loads(l) for l in
82 + (ROOT / "benchmarks" / "promptsets" / "agreement_A.jsonl").read_text().splitlines()]
83 + toks = [tokenizer.encode(it["text"]) for it in items]
84 + labels = np.array([it["label"] for it in items])
85 + print("capture agreement_A (direction + sigma)…", flush=True)
86 + reps = capture_pooled(model, taps, toks)["mean"]
87 + dirs, sigmas = {}, {}
88 + for layer in TEST_LAYERS:
89 + x = reps[:, layer, :]
90 + u = x[labels == "correct"].mean(0) - x[labels == "violated"].mean(0)
91 + u = u / (np.linalg.norm(u) + 1e-8)
92 + dirs[layer] = u
93 + sigmas[layer] = float((x @ u).std())
94 +
95 + def add_fn(u_np, delta):
96 + d = mx.array((u_np * delta).astype(np.float32))
97 + def fn(out):
98 + return (out.astype(mx.float32) + d).astype(out.dtype)
99 + return fn
100 +
101 + base_m = margin()
102 + print(f"baseline margin {base_m:+.4f} | pairs {len(pairs)}", flush=True)
103 +
104 + results = {"per_layer": {}, "baseline_margin": base_m}
105 + mono_ok, halve_ok, spec_ok = True, True, True
106 + for layer in TEST_LAYERS:
107 + dose_margins = {}
108 + for a in DOSES:
109 + if a == 0.0:
110 + dose_margins[a] = base_m
111 + continue
112 + taps[layer].edit = add_fn(dirs[layer], a * sigmas[layer])
113 + dose_margins[a] = margin()
114 + taps[layer].edit = None
115 + seq = [dose_margins[a] for a in DOSES]
116 + mono = all(seq[i] < seq[i + 1] for i in range(len(seq) - 1))
117 + halve = dose_margins[-2.0] <= 0.5 * base_m
118 + rand_changes = []
119 + for s in range(N_RANDOM_DIRS):
120 + ru = np.random.default_rng(3000 * layer + s).standard_normal(reps.shape[-1])
121 + ru /= np.linalg.norm(ru)
122 + for a in (-2.0, 2.0):
123 + taps[layer].edit = add_fn(ru.astype(np.float32), a * sigmas[layer])
124 + rand_changes.append(abs(margin() - base_m))
125 + taps[layer].edit = None
126 + spec = float(np.mean(rand_changes)) < 0.25 * base_m
127 + mono_ok &= mono; halve_ok &= halve; spec_ok &= spec
128 + results["per_layer"][layer] = {
129 + "sigma": sigmas[layer],
130 + "dose_margins": {str(a): dose_margins[a] for a in DOSES},
131 + "monotone": mono, "halved_at_minus2": halve,
132 + "random_dir_mean_abs_change": float(np.mean(rand_changes)),
133 + "specific": spec,
134 + }
135 + print(f"L{layer:02d} doses " +
136 + " ".join(f"{a:+.0f}σ:{dose_margins[a]:+.2f}" for a in DOSES) +
137 + f" | mono={mono} halve={halve} randΔ={np.mean(rand_changes):.2f} spec={spec}",
138 + flush=True)
139 +
140 + passes = bool(mono_ok and halve_ok and spec_ok)
141 + print(f"LEVEL-3 CRITERION: monotone={mono_ok} halved={halve_ok} specific={spec_ok} "
142 + f"-> passes={passes}")
143 +
144 + commit = subprocess.run(["git", "rev-parse", "HEAD"], cwd=ROOT,
145 + capture_output=True, text=True, check=False).stdout.strip()
146 + ts = time.strftime("%Y%m%dT%H%M%SZ", time.gmtime())
147 + outdir = ROOT / "results" / "expC_causal_verification" / ts
148 + outdir.mkdir(parents=True)
149 + (outdir / "results.json").write_text(json.dumps({
150 + "experiment": "expC_causal_verification", "run": 5,
151 + "scope": "dose-response steering (Level-3 path) at early-band layers",
152 + "commit": commit,
153 + "config": {"model": MODEL, "test_layers": list(TEST_LAYERS), "doses": list(DOSES),
154 + "n_random_dirs": N_RANDOM_DIRS, "n_pairs": len(pairs), "seed": SEED},
155 + "manifest": manifest(),
156 + "results": results,
157 + "criterion": {"monotone_all": mono_ok, "halved_all": halve_ok,
158 + "specific_all": spec_ok, "passes": passes},
159 + "wall_seconds": round(time.time() - t0, 1),
160 + }, indent=2) + "\n")
161 + print(f"results -> {outdir / 'results.json'}")
162 + return 0
163 +
164 +
165 +if __name__ == "__main__":
166 + sys.exit(main())
added experiments/micro/expC_causal_verification/implementation/make_l3_mapcard.py +180 −0
@@ -0,0 +1,180 @@
1 +#!/usr/bin/env python3
2 +# =============================================================================
3 +# Project : modelmap
4 +# File : experiments/micro/expC_causal_verification/implementation/make_l3_mapcard.py
5 +# Purpose : Publish interventions/v2 at Level 3 from run #5 (steering)
6 +# Author : Simon-Pierre Boucher
7 +# Contact : contact@spboucher.ai
8 +# Website : https://modelmap.io
9 +# Created : 2026-08-12
10 +# Modified : 2026-08-12
11 +# Platform : macOS / Apple Silicon (arm64)
12 +# License : All rights reserved (research code)
13 +# =============================================================================
14 +"""Builds atlas/qwen3-0.6b-4bit/interventions/v2 (Level 3) — REFUSES unless
15 +run #5's pre-registered dose-response criterion passed AND the v1 band entry
16 +(Level 2) exists as its foundation. v1 stays published as the Level-2 record."""
17 +
18 +from __future__ import annotations
19 +
20 +import hashlib
21 +import json
22 +import sys
23 +import time
24 +from pathlib import Path
25 +
26 +ROOT = Path(__file__).resolve().parents[4]
27 +sys.path.insert(0, str(ROOT / "src"))
28 +
29 +from modelmap.atlas.mapcard import MapCard
30 +
31 +V1 = ROOT / "atlas" / "qwen3-0.6b-4bit" / "interventions" / "v1"
32 +ENTRY = ROOT / "atlas" / "qwen3-0.6b-4bit" / "interventions" / "v2"
33 +MODEL_ID = "mlx-community/Qwen3-0.6B-4bit"
34 +
35 +
36 +def newest_run(n: int) -> Path:
37 + for d in sorted((ROOT / "results" / "expC_causal_verification").iterdir(), reverse=True):
38 + doc = json.loads((d / "results.json").read_text())
39 + if doc.get("run") == n:
40 + return d / "results.json"
41 + raise SystemExit(f"no run-{n} results found")
42 +
43 +
44 +def model_hash() -> str:
45 + from mlx_lm.utils import hf_repo_to_path
46 + mp = Path(hf_repo_to_path(MODEL_ID))
47 + h = hashlib.sha256()
48 + for f in sorted(mp.glob("*.safetensors")):
49 + h.update(f.read_bytes())
50 + return h.hexdigest()
51 +
52 +
53 +def main() -> int:
54 + if not (V1 / "mapcard.json").exists():
55 + print("REFUSED: the Level-2 band entry (v1) must exist first.")
56 + return 1
57 + res5 = newest_run(5)
58 + doc5 = json.loads(res5.read_text())
59 + if not doc5["criterion"]["passes"]:
60 + print(f"REFUSED: run #5 criterion failed ({doc5['criterion']}) — v1 stays Level 2.")
61 + return 1
62 + res4 = newest_run(4)
63 + doc4 = json.loads(res4.read_text())
64 +
65 + ENTRY.mkdir(parents=True, exist_ok=True)
66 + v1_map = json.loads((V1 / "map.json").read_text())
67 + map_doc = {
68 + "author": "Simon-Pierre Boucher", "contact": "contact@spboucher.ai",
69 + "website": "https://modelmap.io",
70 + "map_type": "interventions", "model_id": MODEL_ID,
71 + "claim": "LEVEL-3 claim: the diff-of-means agreement direction in the early band "
72 + "(layers 2-15) is a causal HANDLE on grammatical-agreement behavior — "
73 + "necessary (erasure removes ~73-75% of the margin, six fresh estimators, "
74 + "run #4) AND controllable (activation-addition steering moves the margin "
75 + "strictly monotonically with dose at layers 4/8/12, halves it at -2sigma, "
76 + "while random directions move it by less than the registered 25% bound, "
77 + "run #5).",
78 + "band": v1_map["band"],
79 + "per_layer": v1_map["per_layer"],
80 + "steering": doc5["results"],
81 + "source_results": {"erasure_band": str(res4.relative_to(ROOT)),
82 + "steering": str(res5.relative_to(ROOT))},
83 + }
84 + (ENTRY / "map.json").write_text(json.dumps(map_doc, indent=2) + "\n")
85 +
86 + mhash = model_hash()
87 + created = time.strftime("%Y-%m-%d", time.gmtime())
88 + (ENTRY / "provenance.json").write_text(json.dumps({
89 + "author": "Simon-Pierre Boucher", "contact": "contact@spboucher.ai",
90 + "website": "https://modelmap.io",
91 + "model_id": MODEL_ID, "map_type": "interventions", "version": "v2",
92 + "commit": doc5["commit"], "model_hash": mhash,
93 + "config": {"erasure": doc4["config"], "steering": doc5["config"]},
94 + "seed": [doc4["config"]["seed"], doc5["config"]["seed"]],
95 + "hardware_manifest": doc5["manifest"], "created": created,
96 + "source_results": map_doc["source_results"],
97 + }, indent=2) + "\n")
98 +
99 + card = MapCard(
100 + map_id="atlas/qwen3-0.6b-4bit/interventions/v2",
101 + map_type="interventions", model_id=MODEL_ID, model_hash=mhash,
102 + quantization="q4 (mlx)", commit=doc5["commit"],
103 + config=json.dumps(map_doc["source_results"]), created=created,
104 + hardware_manifest=doc5["manifest"], confidence_level=3,
105 + regenerate_command=(
106 + ".venv/bin/python experiments/micro/expC_causal_verification/implementation/benchmark_v4.py && "
107 + ".venv/bin/python experiments/micro/expC_causal_verification/implementation/benchmark_v5.py && "
108 + ".venv/bin/python experiments/micro/expC_causal_verification/implementation/make_interventions_mapcard.py && "
109 + ".venv/bin/python experiments/micro/expC_causal_verification/implementation/make_l3_mapcard.py"),
110 + seeds=[doc4["config"]["seed"], doc5["config"]["seed"], *doc4["config"]["sources"]],
111 + prompt_sets=["agreement_A/B halves + full (direction est.)",
112 + "fresh held-out minimal-pair bank (behavior)"],
113 + controls=["random-direction erasure nulls (netted)",
114 + "random-direction steering specificity (< 25% bound)",
115 + "six-source direction replication (run #4)",
116 + "dose-response monotonicity across 5 doses x 3 layers",
117 + "late band excluded as estimator-unstable (run #3 refusal)"],
118 + methods_in_agreement=["difference-in-means probing (direction exists)",
119 + "rank-1 erasure (necessity, band-replicated)",
120 + "activation-addition steering (dose-controlled sufficiency)"],
121 + interventions=["rank-1 direction erasure per layer (run #4)",
122 + "activation-addition steering, 5 doses x 3 layers (run #5)"],
123 + replication_rate=round(json.loads((V1 / "mapcard.json").read_text())["replication_rate"], 4),
124 + featurizer_class="linear (difference-in-means direction)",
125 + intervention_protocol="erasure h-⟨h−μ,u⟩u; steering h+ασℓu, α∈{−2..+2}; "
126 + "logit-margin metric; random-direction nulls both families",
127 + notes="Level 3: two independent intervention families (erasure necessity + steering "
128 + "dose-response) agree with probing on the same linear object. v1 remains "
129 + "published as the Level-2 erasure-only record. Claim scope: ONE model "
130 + "(Qwen3-0.6B-4bit), one behavior, band granularity — cross-model and "
131 + "cross-quantization generality are open (expG / candidate_02).",
132 + )
133 + (ENTRY / "mapcard.json").write_text(card.to_json())
134 +
135 + steer_lines = "\n".join(
136 + f"- L{l}: doses " + " ".join(f"{a}σ→{doc5['results']['per_layer'][l]['dose_margins'][a]:+.2f}"
137 + for a in ("-2.0", "-1.0", "0.0", "1.0", "2.0")) +
138 + f" (random-dir mean |Δ| {doc5['results']['per_layer'][l]['random_dir_mean_abs_change']:.2f})"
139 + for l in map(str, doc5["config"]["test_layers"]))
140 + (ENTRY / "confidence.md").write_text(f"""---
141 +project: modelmap
142 +document: qwen3-0.6b-4bit/interventions/v2 — confidence
143 +author: Simon-Pierre Boucher
144 +contact: contact@spboucher.ai
145 +website: https://modelmap.io
146 +created: {created}
147 +status: reviewed
148 +---
149 +
150 +# Confidence — qwen3-0.6b-4bit / interventions / v2
151 +
152 +```text
153 +Level : 3
154 +Seeds : erasure six fresh sources (run #4) + steering run (run #5)
155 +Prompt sets: estimation sets + fresh behavioral bank (never used for estimation)
156 +Methods in agreement : 3 (probing; erasure; steering) — two independent
157 + intervention families
158 +Causal verification : YES, both directions — necessity (erasure) and
159 + dose-controlled sufficiency (steering)
160 +```
161 +
162 +Steering dose-response (baseline margin {doc5['results']['baseline_margin']:+.2f}):
163 +{steer_lines}
164 +
165 +All three pre-registered gates passed: strict monotonicity at every test
166 +layer, halving at −2σ, random-direction specificity under the 25% bound.
167 +Scope honesty: one model, one behavior, band granularity; generality across
168 +sizes (expG) and quantization levels (candidate_02) is untested and is the
169 +registered next question.
170 +""")
171 + errs = card.validate()
172 + if errs:
173 + print("CARD INVALID:", errs)
174 + return 1
175 + print(f"atlas entry written: {ENTRY.relative_to(ROOT)} (LEVEL 3)")
176 + return 0
177 +
178 +
179 +if __name__ == "__main__":
180 + sys.exit(main())
modified research/LOG.md +28 −0
@@ -479,3 +479,31 @@ refusal on record. Every claim at its measured level.
479 479 should *raise* the margin where it is weak); (2) same band protocol on
480 480 arith_valid; (3) candidate_02 entry: quantization drift of the band map
481 481 (FP16 vs Q8 vs Q4); (4) expG entry: does the band replicate on Qwen3-1.7B?
482 +
483 +---
484 +
485 +## 2026-08-12 09:45 EDT — expC run #5: L3 gate refused — layer 12 is a textbook handle anyway
486 +
487 +**Result (conjunctive criterion, pre-registered).** FAILED as a
488 +conjunction; make_l3_mapcard.py refused v2 (exit 1); interventions/v1
489 +stays Level 2.
490 +- **L12: passes everything** — strictly monotone dose-response
491 + (+1.95/+3.00/+4.45/+5.82/+6.79 across −2σ…+2σ), halved at −2σ,
492 + specific (random |Δ| 0.45 vs bound 1.11). A textbook causal handle.
493 +- L08: near-monotone, not specific at 2σ (random |Δ| 2.97).
494 +- L04: overdose regime — ±2σ collapses the margin AND random directions
495 + are equally destructive (|Δ| 4.40). Run #2's rank-1 fragility,
496 + dose-resolved: early layers tolerate no ~2σ perturbation of any kind.
497 +
498 +**Reading.** Necessity everywhere in the band (Level-2 entry) coexists
499 +with *controllability only where the layer tolerates perturbation*. The
500 +dose scale (σ from mean-pooled stats) is a declared confound at early
501 +layers. Gate record now: refusal (run #3, per-layer) → pass (run #4,
502 +band, L2) → refusal (run #5, L3 conjunction). The atlas has never
503 +received a claim its evidence didn't carry.
504 +
505 +**Next.** Run #6 candidates (to register before running): layer-local
506 +dose calibration + handle claim on the perturbation-tolerant sub-band;
507 +or the minimal L12 single-layer L3 claim with fresh direction estimates.
508 +Parallel tracks unchanged: arith_valid band, candidate_02 quantization
509 +drift, expG cross-model.
added results/expC_causal_verification/20260812T072455Z/results.json +105 −0
@@ -0,0 +1,105 @@
1 +{
2 + "experiment": "expC_causal_verification",
3 + "run": 5,
4 + "scope": "dose-response steering (Level-3 path) at early-band layers",
5 + "commit": "3eb9df2dc899ed5d42eb9e700dd6299bdc5550ec",
6 + "config": {
7 + "model": "mlx-community/Qwen3-0.6B-4bit",
8 + "test_layers": [
9 + 4,
10 + 8,
11 + 12
12 + ],
13 + "doses": [
14 + -2.0,
15 + -1.0,
16 + 0.0,
17 + 1.0,
18 + 2.0
19 + ],
20 + "n_random_dirs": 3,
21 + "n_pairs": 192,
22 + "seed": 781
23 + },
24 + "manifest": {
25 + "author": "Simon-Pierre Boucher",
26 + "contact": "contact@spboucher.ai",
27 + "website": "https://modelmap.io",
28 + "chip": {
29 + "brand": "Apple M5 Max",
30 + "cores_total": 18,
31 + "cores_performance": 6,
32 + "cores_efficiency": 12
33 + },
34 + "memory": {
35 + "unified_gb": 48.0,
36 + "pagesize": 16384
37 + },
38 + "os": {
39 + "system": "Darwin",
40 + "version": "27.0",
41 + "arch": "arm64"
42 + },
43 + "software": {
44 + "python": "3.14.4",
45 + "numpy": "2.5.2",
46 + "mlx": "0.32.0",
47 + "torch": "2.13.0",
48 + "safetensors": "0.8.0"
49 + }
50 + },
51 + "results": {
52 + "per_layer": {
53 + "4": {
54 + "sigma": 38.11064529418945,
55 + "dose_margins": {
56 + "-2.0": 0.08170572916666667,
57 + "-1.0": 0.66357421875,
58 + "0.0": 4.450520833333333,
59 + "1.0": 4.386555989583333,
60 + "2.0": 0.03759765625
61 + },
62 + "monotone": false,
63 + "halved_at_minus2": true,
64 + "random_dir_mean_abs_change": 4.395148383246528,
65 + "specific": false
66 + },
67 + "8": {
68 + "sigma": 18.62078857421875,
69 + "dose_margins": {
70 + "-2.0": 0.5911458333333334,
71 + "-1.0": 1.5514322916666667,
72 + "0.0": 4.450520833333333,
73 + "1.0": 6.264973958333333,
74 + "2.0": 5.703287760416667
75 + },
76 + "monotone": false,
77 + "halved_at_minus2": true,
78 + "random_dir_mean_abs_change": 2.972900390625,
79 + "specific": false
80 + },
81 + "12": {
82 + "sigma": 9.929967880249023,
83 + "dose_margins": {
84 + "-2.0": 1.9537760416666667,
85 + "-1.0": 3.0042317708333335,
86 + "0.0": 4.450520833333333,
87 + "1.0": 5.819173177083333,
88 + "2.0": 6.791178385416667
89 + },
90 + "monotone": true,
91 + "halved_at_minus2": true,
92 + "random_dir_mean_abs_change": 0.4538031684027775,
93 + "specific": true
94 + }
95 + },
96 + "baseline_margin": 4.450520833333333
97 + },
98 + "criterion": {
99 + "monotone_all": false,
100 + "halved_all": true,
101 + "specific_all": false,
102 + "passes": false
103 + },
104 + "wall_seconds": 19.1
105 +}
106