spb/modelmap Public License

1---2project: modelmap3document: qwen3-0.6b-4bit/probes/v1 — confidence4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6website: https://modelmap.io7created: 2026-08-128status: reviewed9---1011# Confidence — qwen3-0.6b-4bit / probes / v11213```text14Level      : 115Seeds      : 516Prompt sets: 6 (2 disjoint template families per property)17Methods in agreement : 1 (linear probes only — Level 2 requires a second method)18Causal verification  : none (observational; Level 3 requires intervention)19```2021Per-property evidence:22- lang_id: maxAcc A/B = 1.000/1.000, seed SD 0.0000, dataset shift 0.0000, twin max selectivity 0.808, replication(top-5) 1.00/1.0023- code_prose: maxAcc A/B = 1.000/1.000, seed SD 0.0000, dataset shift 0.0000, twin max selectivity 0.883, replication(top-5) 1.00/1.0024- arith: maxAcc A/B = 1.000/1.000, seed SD 0.0000, dataset shift 0.0000, twin max selectivity 0.558, replication(top-5) 1.00/1.002526**This is a published NEGATIVE result (Level 1 for the negative claim).**27The random-init architecture twin matches the trained model at ceiling28(accuracy 1.00, 28/28 layers FDR-significant, for the twin as for the real29model; mean real-minus-twin selectivity within +/-0.06). By the validity30criterion registered in hypothesis.md BEFORE the run (twin selectivity must31stay < 0.05), this probing harness is INVALID for localization claims on32these promptsets: it measures the tokenizer + architecture prior, not33learned computation. The negative claim itself is controlled and replicated34(5 seeds, 2 disjoint promptsets, 3 properties) - hence Level 1.3536Consequences adopted: (1) probe maps are only publishable as REAL-MINUS-TWIN37differentials; (2) promptsets v2 must remove lexical separability (shared38vocabulary across classes); (3) the seed-vs-dataset variance hypothesis is39untestable at ceiling and moves to run #2.40