SPB Git

spb/modelmap Public License

Internal cartography of local LLMs on Apple Silicon — registered, gated, negative-first. Public atlas at modelmap.io.

Python 66.3% JavaScript 24.5% CSS 8.1% Shell 0.7%

History of experiments/micro/expA_probe_reliability/analysis.md · clear filter

  1. expA run #2: first positive maps (agreement, arith_valid) + real noise floor
    - v2 promptsets: structure-borne, token-balanced (overlap certificates);
      capture_pooled returns mean+last-token reps in one pass
    - Strict twin gate falsified again (word_order twin acc 0.96 — subword
      statistics); gate retired, differential (real-twin) is the map
    - agreement: 25/28 signal layers (max dSel +0.379); arith_valid: real
      0.86/0.90 vs twin 0.56, strongest last-token; word_order null-dominated
    - First real noise floor: top-5 replication ~0.54 off ceiling; dataset
      shift > seed SD on only 3-8/28 layers (registered 1/3 not met)
    - atlas probes/v2 published (Level 1, per-property verdicts); site charts
      generalized to any probes entry; home leads with agreement map
    
    Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 5 h ago (Aug 12, 2026) · 1 file changed +62
  2. expA run #1: validity gate FAILED — first atlas entry is a negative result
    - src/modelmap/capture: MLX Tap layer + random-init architecture twin
    - promptsets v1 (6x240, sha256 manifest) — now the positive-control corpus
    - Result: ceiling everywhere (acc 1.000); twin ALSO at 1.00 on all 28
      layers (max sel 0.56-0.88 vs registered 0.05 gate) — probe map reads
      tokenizer+architecture prior, zero trained-model signal. Shuffled-label
      control would not have caught it; only the twin null did.
    - Published atlas/qwen3-0.6b-4bit/probes/v1 (negative_result=true,
      Level 1, replication 1.00) via publish.py gate
    - Doctrine: probe maps only as REAL-TWIN differentials from now on
    
    Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    Simon-Pierre Boucher committed 5 h ago (Aug 12, 2026) · 1 file changed +73