# Canonicalization — `aia canonicalize`

The canonical upgrade (2026-09) turns the flat "everything is a model" corpus into one hierarchy and one vocabulary:

```
model_family   Llama 4 · Qwen3.6 · Claude · GPT 5.4
  └ model      Llama 4 Maverick · Qwen3.6-35B-A3B · Claude Fable 5.1          ← the unit counted as "a model"
      └ artifact   unsloth/Qwen3.6-35B-A3B-GGUF (quantization) · zai-org/GLM-5-FP8 (conversion) · mlx-community/… (packaging)
          └ deployment  OpenRouter qwen/qwen3.6-35b-a3b · Together …             ← `prices` rows, never entities
```

Evaluation-effort variants (`gpt-5-4-mini-medium`, `claude-opus-5-xhigh`, `deepseek-v4-pro-non-reasoning`) are **result configurations**
(`reasoning_effort`, `reasoning`, `thinking_budget`) of the canonical model, never entities of their own.

Two layers implement it:

* **at write time** — `FactWriter` / `Resolver` (src/aiatlas/sdk) apply the ontology to every incoming fact, so new data is born canonical;
* **on the existing corpus** — `aia canonicalize` (src/aiatlas/services/canonical.py) replays the same rules over what is already stored.

## Running it

```bash
.venv/bin/aia canonicalize                 # dry-run: plan + counts + examples, writes nothing
.venv/bin/aia canonicalize --apply         # execute the plan (one transaction per step)
.venv/bin/aia canonicalize --step results --step events --apply
.venv/bin/aia canonicalize --json          # machine-readable report
.venv/bin/aia anomalies [--severity critical] [--refresh]
.venv/bin/aia quarantine list | release <id> | discard <id>
```

Steps always run in this order (later steps depend on earlier ones):
`duplicates → variants → artifacts → families → licenses → taxonomy → results → events → anomalies`.

**Idempotence is a hard requirement**: a second `--apply` immediately after the first must report `0 change(s)`. Every step computes its plan
from reads, and `apply` executes exactly that plan; nothing depends on "what changed last time". Re-run it after every connector wave.

## What is never done

* **No row is ever deleted.** Merged duplicates and folded variants stay as rows with `status='merged'`, `merged_into=<survivor>`; their
  slugs and ids keep resolving (API follows `merged_into`, web 301s). Artifacts keep their own row, slug, aliases, identifiers, claims,
  results, prices and events.
* **Raw snapshots and the archive are never touched.** Canonical values live in `claims.value` / `entities.attributes`; the source label is
  kept next to them (`claims.value_raw`, `attributes.<prop>_raw`) whenever it differed.
* **No value is guessed.** Unknown licence / modality / status strings are kept as-is and recorded in `taxonomy_mappings` with
  `canonical = NULL` (the backlog is visible; nothing is dropped). Ambiguous identities go to `review_queue`, never to a merge.
* **Derived facts never outrank sourced facts.** Everything the engine infers is written by a tier-2 `derived` writer (source
  `ai-atlas.registry`): the ordinary writer rules apply, so a tier-1 explicit statement (`openness` from a lab page) is kept and the derived
  disagreement is stored once as `conflicting` (without a review item — derived disagreements are expected and inspectable).
* **Observation truth is preserved.** `first_seen_at` is when *we* first saw an entity; a release date earlier than that is a fact about the
  model (`attributes.release_date`), not about our observation. `/diff`, `/changes` and the counters stay observation-based; timelines use
  `occurred_at = coalesce(effective_at, observed_at)`.

## Steps

### duplicates
Exact normalised-name duplicates (`ids.normalize_alias`) within a type — models, providers, benchmarks, datasets, frameworks, libraries,
hardware — and across the organisation group (`company | organization | lab | university`, also joined by a shared `hf_org` / `github_org`).
* identifiers **do not conflict** (no scheme with two different values) → merge the weaker row into the survivor. Survivor = curated
  org type (`company/lab/university` beat the hub's generic `organization`), then most current claims, then oldest `first_seen_at`.
  Recorded as a `merge` decision in `resolution_decisions` and in `admin_audit_log`.
* identifiers **conflict** (`gemini-2-5-flash-lite` AA vs `gemini-2.5-flash-lite` Google id + a *different* AA slug) → `review_queue`
  `merge_candidate` only.
* same name but two organisations (non-org types) → review only. Pairs with a `keep_separate` decision are skipped.
* single-letter organisations (`w`) → `junk_entity` review item; `org_kind` defaults from the entity type (`ORG_TYPE_DEFAULT_KIND`).

### variants
Models whose name carries an evaluator effort suffix (`ontology.models.analyze_model_name().is_effort_variant`: `-xhigh/-high/-medium/-low/
-minimal`, `-thinking/-reasoning`, `-non-reasoning/-no-think`, `-32k-thinking`, `-high-effort`, up to three chained) **and** that are known
only through evaluator identifiers (`artificial_analysis`, `livebench_model_id`…), have no `hf_repo` and no tier-1 claim. Anything with an
official/hub/provider identifier is a real model that happens to end in "Thinking" (Qwen3-VL-8B-Thinking has its own weights) and is left alone.
`-max`, `-fast`, `-instant` are model tiers (Qwen3-Max, GPT-5.1-Codex-Max, Grok 4.1 Fast), never effort suffixes.

Base resolution (`Resolver.resolve_variant`): same `artificial_analysis` identifier / alias / slug as the base name, or the same
`variant_key` through the precomputed index; never onto another effort variant; several bases → prefer the variant's organisation, else review.
* resolved → every `benchmark_results.config` of the variant gains the effort dict + `aa_variant_slug`, then `merge_entities(variant →
  canonical)` moves results (dedupe keys recomputed, collisions close the older row), prices, relations, events, aliases and identifiers. The
  variant's AA identifier now points at the canonical model, so the next AA run resolves it directly; the writer adds the effort dict to
  the config of any result whose model ref is an effort variant (`effort_config`), so folded results stay distinguishable and comparable
  (`reasoning_effort` is a *condition* key: same `config_key`, partially comparable).
* unresolved → `identity_confidence='medium'`, `attributes.evaluation_variant_of_hint`, `variant_candidate` review item.

### artifacts
A model row is an **artifact** when its name / `hf_repo` carries a quantisation or precision token (`GGUF`, `FP8`, `MXFP4`, `AWQ`, `BF16`,
`MLX`, `ONNX`…), when its repository lives under a converter organisation (`ontology.models.CONVERTER_ORGS`: unsloth, bartowski,
mlx-community…), or when its hub repo attributes say `is_quantized` / `quant_format` (repo attributes only count with an `hf_repo`: a
provider's serving precision — OpenRouter `fp8` — says nothing about identity).

* **Official checkpoints stay models.** A repo under the developer's own organisation (`FAMILY_ORGS` publisher or the entity's own org,
  never a converter org) is the model itself even in native FP8 (`deepseek-ai/DeepSeek-R1`, `nvidia/…-Nemotron-3-Ultra-…-BF16`). It only
  becomes an artifact when the same organisation *also* has the plain model entity **and** nobody else knows the tagged repo under its own
  identity; when evaluators/providers do, the pair is a `merge_candidate` for review. No checkpoint entities are created.
* `artifact_kind`: `quantization` (bit-width / GGUF / AWQ…), `conversion` (BF16/FP8 repack, MLX, ONNX), else `packaging`.
* `canonical_id`: the `quantized_from` object (never `derived_from` / `fine_tuned_from` — a fine-tune is a new model), else the unique model with
  the same `variant_key` (same organisation preferred, then the family's official publisher); none → artifact with `canonical_id = NULL`,
  `identity_confidence='low'`, `unresolved_artifact` review item.
* Writes: `entity_type='artifact'`, `artifact_kind`, `canonical_id`, relation `artifact_of`, a `variant_of` decision. Slug, aliases,
  identifiers, claims, results, prices, events untouched. `model` and `artifact` are lookup-compatible types in the resolver, so the same
  hub repo keeps resolving to the same row.

### families
For every model with a family hint: family = versioned label when a version follows the family word (`Llama 3.1`, `Qwen3.6`, `Gemini 2.5`,
`Claude 4`, `GPT 5.4` — hyphenated slug versions `gpt-5-4` count; `38B`/`235B` are sizes, not versions), else the base family (`Claude`,
`Kimi`). One level only; the family entity carries `attributes.family_root` (`Llama`) for grouping.
Slug = `slugify(label)`; on collision with any other entity (the model `gpt-5.5` itself) the organisation slug is prefixed, then `family-`.
Organisation = the family's official publisher when known, else the model's. Writes: `entities.family_id`, relation `member_of_family`,
`attributes.family` only when the model had none (a sourced `family` claim is never overwritten).

### licenses
`normalize_license(attributes.license)` → canonical key (SPDX id when one exists: `Apache-2.0`, `MIT`, `CC-BY-NC-4.0`; AI Atlas key
otherwise: `Llama-3.1-Community`, `Gemma-Terms`). Creates one `license` entity per observed key (slug = key lower-cased, attributes =
`LicenseInfo.as_dict()`: commercial use, redistribution, derivatives, hosting restrictions, acceptable-use policy, OSI approval…), relation
`uses_license`, `attributes.license_key`. Unclassified strings stay in `license` (mapping recorded with `canonical = NULL`).

Openness 2.0 as **derived claims** (tier 2, `extractor='derived'`): `weights_available` (true with an `hf_repo` / model card / weights URL
or an open-ish openness label; false when proprietary; unknown → nothing written), `commercial_use_allowed`, `redistribution_allowed`,
`derivatives_allowed` from the licence, and the category `openness` = `derive_openness(dims)` (`open-source | open-weights | restricted-weights
| proprietary`). HF "gated" is a distribution mechanism, not a licence property: a gated Apache-2.0 repo is `open-weights`. A tier-1
`openness` statement that disagrees is kept current; the derived value is stored once as `conflicting`.

### taxonomy
Current claims (and claim-less attributes) for `license`, `openness`, `modalities`, `modalities_input`, `modalities_output`, `status`, hardware
and framework `kind`, `org_kind` are **re-encoded in place**: same claim row, same source, same tier — only the encoding changes
(`apache-2.0` → `Apache-2.0`, `restricted` → `restricted-weights`, `["Text","pdf"]` → `["document","text"]`, `computer` → `system`). The
source label goes to `claims.value_raw` / `attributes.<prop>_raw`. This is *not* a new assertion, so no change event is emitted. Every
raw → canonical pair (and every unknown raw value) lands in `taxonomy_mappings`.

The writer applies the same normalisation to incoming facts; when a source re-states the same licence in another spelling the current claim is
re-encoded and confirmed — never a `LICENSE_CHANGED` event.

### results
Back-fills `config_key` (comparability hash of the *task* keys + metric — variant, board, harness, evaluator, shots, scaffold, system…),
`trust_level` (`ontology.benchmarks.trust_level(source_key, config)`: official-benchmark / independent-evaluator / official-model-card /
community / peer-reviewed / unverified), `variant`, `run_group` (release, index version, dataset revision, run date) and `extractor`.
Then **one current row per (model, benchmark, metric, config_key)**: rows from an older run group than the latest observed one get
`is_current = false`, `valid_to = <newer observation>`, so leaderboards show the latest LiveBench release / aider run / AA index version
instead of piling up every run. Run keys are *not* part of `config_key` (two releases of the same task are partially comparable); condition keys
(`reasoning_effort`, temperature, judge…) are not either, so folded effort variants share a key and are compared as conditions.
Scores outside the metric bounds are stored with `confidence='low'` and flagged (`score_above_max` / `score_below_min`).

### events
Three clocks: `occurred_at = coalesce(effective_at, observed_at)` (when it happened), `observed_at` (when a connector saw it), `recorded_at`
(when the row was written). `is_backfill = true` when the event was observed before the connector's second successful run (the initial
corpus is history, not news), when `effective_at` is more than 3 days before `observed_at`, or for `NEW_*` events whose entity's
`release_date` / `published_at` predates observation by more than 3 days. `group_key = release:<entity>:<yyyy-mm>` for `RELEASE`,
`ANNOUNCEMENT`, `NEW_MODEL`, `VERSION_RELEASED` so one release seen in several documents groups. Live counters ("+N · 24 h", `/changes`,
pulse) count non-backfill events only; timelines use everything. The step prints live-event counts for the last 24 h before/after.
Importance is deterministic (`services.events.importance_for`): frontier or open-weight releases 3, price moves ≥ 50 % 3 / ≥ 20 % 2 / else 1,
context ≥ 5× 3, deprecations 3, leader changes 2, metadata corrections 0, artifact events 0, family events 1; tier > 2 sources lose one point.

### anomalies
Runs `ontology.anomalies` over live models/artifacts, hardware, live prices and current results; upserts `anomalies` rows by `dedupe_key`
(reopened when a check fires again; operator states `ignored` / `fixed` are kept), and auto-resolves open flags whose check no longer fires.
`aia anomalies` lists them. Flags never change the data.

## Write-time counterparts (SDK)

* `FactWriter(run_id=…)`: every claim, relation, price, result and event carries the connector run id (batch inspection/rollback).
* Same-source loophole closed: a claim supersedes only when `tier <= current.tier` **or** same source **and** same extractor — an LLM claim
  never overwrites a deterministic one from the same URL (it is stored as `conflicting`, once).
* `Resolver`: `keep_separate` decisions block alias resolution/merges; aliases whose normalisation collapses digit separators
  (`Qwen3-8B` ≡ `Qwen 38B` → `qwen38b`) additionally require the same `variant_key`; `first_seen_hint` back-dates `first_seen_at`
  (`min(now, hint)`); hierarchy hints (`EntityRef.family` / `.canonical` / `.artifact_kind` / `.identity_confidence`) are materialised.
* Quarantine: tier ≥ 2 connectors (hubs, leaderboards, registries) hold their facts until the run ends and compare the run with the
  connector's rolling baseline (median of the last 5 full-extraction runs): new entities > max(50, 1.3× baseline), price rows > 3×, or — on a
  full re-extraction — entity references < 0.8× or results < 0.5× → the whole run goes to `quarantined_runs` (`status='pending'`, serialised
  Facts), `connector_runs.status='quarantined'`, a `quarantine` review item; nothing is written. `aia quarantine release <id>` writes the
  facts exactly as the connector would have; `discard` drops them (snapshots stay archived). A connector's first run is never quarantined —
  it sets the baseline.
* SSRF guard (`sdk/fetch.py`): every URL and every redirect hop (max 5) is resolved and refused when it targets a non-http(s) scheme,
  `localhost`/`.local`/`.internal`, RFC1918, loopback, link-local (cloud metadata), CGNAT, IPv6 loopback/link-local/ULA or unspecified addresses.

## Counting rules (for the API / UI)

* **Models** = `entity_type = 'model' and merged_into is null` (artifacts and folded variants excluded; `include=artifacts` restores the old
  universe). **Artifacts** = `entity_type = 'artifact'`. **Families** = `entity_type = 'model_family'`.
* **Current benchmark results** = `valid_to is null and is_current`. **Live events** = `is_backfill = false`.