[![modelmap — contour-line logo](https://www.modelmap.io/static/logo.svg)](https://modelmap.io)

# modelmap — Internal Cartography of Local Large Language Models

![status](https://img.shields.io/badge/status-active_research-6d4fc4)
![platform](https://img.shields.io/badge/platform-macOS_·_Apple_Silicon-0b0b0b)
![compute](https://img.shields.io/badge/compute-MLX_·_PyTorch--MPS-eb6834)
![atlas](https://img.shields.io/badge/atlas_entries-3-53389e)
![max confidence](https://img.shields.io/badge/max_confidence-Level_2_·_method--robust-0e9f4e)
![causal survival](https://img.shields.io/badge/correlational→causal_survival-0%2F2-9a6a00)
![registered runs](https://img.shields.io/badge/registered_runs-11-53389e)
![gate record](https://img.shields.io/badge/publication_gate-3_refusals_·_1_pass-8b8798)
![tests](https://img.shields.io/badge/tests-5_passing-0e9f4e)
![license](https://img.shields.io/badge/license-all_rights_reserved-555)

**modelmap** discovers, measures, and maps the internal structure of
pretrained open-weight LLMs **entirely on consumer Apple Silicon** — and
publishes the results as a reproducible, confidence-labeled public atlas at
**[modelmap.io](https://modelmap.io)**. Every map is versioned, provenanced
(commit + config + model hash + hardware manifest), regenerable by one
command, and labeled with the evidence level it actually earned. Negative
results and publication-gate refusals are first-class citizens.

> *"we think the model does X"* → *"here is the map, its evidence level,
> and the script that rebuilds it"*

## Headline results (2026-08-12 · Qwen3-0.6B-4bit · full report: [TR-2026-01](https://modelmap.io/publications/TR-2026-01))

| Finding | Evidence |
|---|---|
| **Activation capture is nearly free under MLX** — 1.004× plain prefill on a real 4-bit checkpoint (first Python capture from an actually-quantized local model) | expH, 3 registered runs |
| **Activation-store format rankings invert warm→cold** — mmap 3–11× ahead warm, zarr 4.4× ahead cold; IO granularity governs, not the container | expH run #2 (falsified own hypothesis) |
| **Probes on separable classes measure the tokenizer, not the model** — a random-init architecture twin matches the trained model at ceiling; shuffled-label controls miss it | expA run #1 (published negative, atlas probes/v1) |
| **Differential (real−twin) maps recover trained signal** on structure-borne properties: agreement 25/28 signal layers, equation-validity 0.90 vs twin 0.58 | expA run #2 (Level 1, atlas probes/v2) |
| **Decodability ≠ causal support** — the probe map's layer ranking failed two intervention tests (survival ledger **0/2**) | expC runs #1–#2 |
| **A single agreement direction is causally necessary across layers 2–15** — rank-1 erasure removes 73–75% of the behavior, replicated across six fresh estimators | expC run #4 (**Level 2**, atlas interventions/v1) |
| **Two Level-3 attempts refused by the publication gate** — incl. one that would have shipped a false claim without fresh re-registration | expC runs #5–#6 |

## How it works

- **Registered hypotheses** — every run is preceded by a written hypothesis
  with an explicit falsification criterion (`experiments/*/hypothesis.md`).
- **Mandatory nulls** — shuffled labels, random-init architecture twins,
  random-direction controls, FDR correction across unit scans.
- **Machine-enforced publication gates** — `tools/publish.py` refuses any
  atlas entry whose map card doesn't validate; per-run gates refuse claims
  that fail replication (`make_*_mapcard.py`).
- **Confidence taxonomy** on every artifact: L0 anecdotal · L1 correlational
  · L2 method-robust · L3 causal.
- **Everything local** — MLX / PyTorch-MPS on 16–64 GB Macs; quantized
  checkpoints are studied in the form people actually run.

## Repository layout

```text
research/       charter-driven paper trail: log, state of the art, 24 gaps, ranking
src/modelmap/   capture (MLX taps, quantized models), probes, stats, atlas schema
experiments/    micro-experiments A–H + candidates (hypothesis → run → analysis)
atlas/          versioned map artifacts: map.json + provenance + confidence + map card
publications/   official technical reports (rendered with live figures on the site)
site/           modelmap.io (Express, server-rendered SVG maps, mobile-first)
benchmarks/     harness, hardware manifests, checksummed promptsets
tools/          check_headers, new_experiment, new_map, publish (the gate)
```

## Quickstart (Apple Silicon, macOS 14+)

```bash
make setup          # venv + deps (numpy, mlx, torch, zarr, safetensors)
make test           # 5 correctness tests (probes vs planted structure, FDR, map cards)
make lint headers   # ruff + mandatory author-header check
make site-run       # build and preview modelmap.io locally on :8140
# regenerate any published map: see its mapcard.json "regenerate_command"
```

## Author

**Simon-Pierre Boucher** · [contact@spboucher.ai](mailto:contact@spboucher.ai) · [modelmap.io](https://modelmap.io)
Sister project: [localvm-research](https://www.localvm.dev) (out-of-core LLM execution on consumer Macs).

## Citation

```text
Boucher, S.-P. (2026). modelmap — Internal Cartography of Local Large
Language Models. https://modelmap.io (see also Technical Report TR-2026-01).
```

*All rights reserved (research code). © 2026 Simon-Pierre Boucher.*