# Canonicalization — aia canonicalize

The canonical upgrade (2026-09) turns the flat "everything is a model" corpus into one hierarchy and one vocabulary:

model_family   Llama 4 · Qwen3.6 · Claude · GPT 5.4
  └ model      Llama 4 Maverick · Qwen3.6-35B-A3B · Claude Fable 5.1          ← the unit counted as "a model"
      └ artifact   unsloth/Qwen3.6-35B-A3B-GGUF (quantization) · zai-org/GLM-5-FP8 (conversion) · mlx-community/… (packaging)
          └ deployment  OpenRouter qwen/qwen3.6-35b-a3b · Together …             ← `prices` rows, never entities

Evaluation-effort variants (gpt-5-4-mini-medium, claude-opus-5-xhigh, deepseek-v4-pro-non-reasoning) are result configurations (reasoning_effort, reasoning, thinking_budget) of the canonical model, never entities of their own.

Two layers implement it:

• at write time — FactWriter / Resolver (src/aiatlas/sdk) apply the ontology to every incoming fact, so new data is born canonical;
• on the existing corpus — aia canonicalize (src/aiatlas/services/canonical.py) replays the same rules over what is already stored.

# Running it

.venv/bin/aia canonicalize                 # dry-run: plan + counts + examples, writes nothing
.venv/bin/aia canonicalize --apply         # execute the plan (one transaction per step)
.venv/bin/aia canonicalize --step results --step events --apply
.venv/bin/aia canonicalize --json          # machine-readable report
.venv/bin/aia anomalies [--severity critical] [--refresh]
.venv/bin/aia quarantine list | release <id> | discard <id>

Steps always run in this order (later steps depend on earlier ones): duplicates → variants → artifacts → families → licenses → taxonomy → results → events → anomalies.

Idempotence is a hard requirement: a second --apply immediately after the first must report 0 change(s). Every step computes its plan from reads, and apply executes exactly that plan; nothing depends on "what changed last time". Re-run it after every connector wave.

# What is never done

• No row is ever deleted. Merged duplicates and folded variants stay as rows with status='merged', merged_into=<survivor>; their slugs and ids keep resolving (API follows merged_into, web 301s). Artifacts keep their own row, slug, aliases, identifiers, claims, results, prices and events.
• Raw snapshots and the archive are never touched. Canonical values live in claims.value / entities.attributes; the source label is kept next to them (claims.value_raw, attributes.<prop>_raw) whenever it differed.
• No value is guessed. Unknown licence / modality / status strings are kept as-is and recorded in taxonomy_mappings with canonical = NULL (the backlog is visible; nothing is dropped). Ambiguous identities go to review_queue, never to a merge.
• Derived facts never outrank sourced facts. Everything the engine infers is written by a tier-2 derived writer (source ai-atlas.registry): the ordinary writer rules apply, so a tier-1 explicit statement (openness from a lab page) is kept and the derived disagreement is stored once as conflicting (without a review item — derived disagreements are expected and inspectable).
• Observation truth is preserved. first_seen_at is when we first saw an entity; a release date earlier than that is a fact about the model (attributes.release_date), not about our observation. /diff, /changes and the counters stay observation-based; timelines use occurred_at = coalesce(effective_at, observed_at).

# Steps

# duplicates

Exact normalised-name duplicates (ids.normalize_alias) within a type — models, providers, benchmarks, datasets, frameworks, libraries, hardware — and across the organisation group (company | organization | lab | university, also joined by a shared hf_org / github_org).

• identifiers do not conflict (no scheme with two different values) → merge the weaker row into the survivor. Survivor = curated org type (company/lab/university beat the hub's generic organization), then most current claims, then oldest first_seen_at. Recorded as a merge decision in resolution_decisions and in admin_audit_log.
• identifiers conflict (gemini-2-5-flash-lite AA vs gemini-2.5-flash-lite Google id + a different AA slug) → review_queue merge_candidate only.
• same name but two organisations (non-org types) → review only. Pairs with a keep_separate decision are skipped.
• single-letter organisations (w) → junk_entity review item; org_kind defaults from the entity type (ORG_TYPE_DEFAULT_KIND).

# variants

Models whose name carries an evaluator effort suffix (ontology.models.analyze_model_name().is_effort_variant: -xhigh/-high/-medium/-low/ -minimal, -thinking/-reasoning, -non-reasoning/-no-think, -32k-thinking, -high-effort, up to three chained) and that are known only through evaluator identifiers (artificial_analysis, livebench_model_id…), have no hf_repo and no tier-1 claim. Anything with an official/hub/provider identifier is a real model that happens to end in "Thinking" (Qwen3-VL-8B-Thinking has its own weights) and is left alone. -max, -fast, -instant are model tiers (Qwen3-Max, GPT-5.1-Codex-Max, Grok 4.1 Fast), never effort suffixes.

Base resolution (Resolver.resolve_variant): same artificial_analysis identifier / alias / slug as the base name, or the same variant_key through the precomputed index; never onto another effort variant; several bases → prefer the variant's organisation, else review.

• resolved → every benchmark_results.config of the variant gains the effort dict + aa_variant_slug, then merge_entities(variant → canonical) moves results (dedupe keys recomputed, collisions close the older row), prices, relations, events, aliases and identifiers. The variant's AA identifier now points at the canonical model, so the next AA run resolves it directly; the writer adds the effort dict to the config of any result whose model ref is an effort variant (effort_config), so folded results stay distinguishable and comparable (reasoning_effort is a condition key: same config_key, partially comparable).
• unresolved → identity_confidence='medium', attributes.evaluation_variant_of_hint, variant_candidate review item.

# artifacts

A model row is an artifact when its name / hf_repo carries a quantisation or precision token (GGUF, FP8, MXFP4, AWQ, BF16, MLX, ONNX…), when its repository lives under a converter organisation (ontology.models.CONVERTER_ORGS: unsloth, bartowski, mlx-community…), or when its hub repo attributes say is_quantized / quant_format (repo attributes only count with an hf_repo: a provider's serving precision — OpenRouter fp8 — says nothing about identity).

• Official checkpoints stay models. A repo under the developer's own organisation (FAMILY_ORGS publisher or the entity's own org, never a converter org) is the model itself even in native FP8 (deepseek-ai/DeepSeek-R1, nvidia/…-Nemotron-3-Ultra-…-BF16). It only becomes an artifact when the same organisation also has the plain model entity and nobody else knows the tagged repo under its own identity; when evaluators/providers do, the pair is a merge_candidate for review. No checkpoint entities are created.
• artifact_kind: quantization (bit-width / GGUF / AWQ…), conversion (BF16/FP8 repack, MLX, ONNX), else packaging.
• canonical_id: the quantized_from object (never derived_from / fine_tuned_from — a fine-tune is a new model), else the unique model with the same variant_key (same organisation preferred, then the family's official publisher); none → artifact with canonical_id = NULL, identity_confidence='low', unresolved_artifact review item.
• Writes: entity_type='artifact', artifact_kind, canonical_id, relation artifact_of, a variant_of decision. Slug, aliases, identifiers, claims, results, prices, events untouched. model and artifact are lookup-compatible types in the resolver, so the same hub repo keeps resolving to the same row.

# families

For every model with a family hint: family = versioned label when a version follows the family word (Llama 3.1, Qwen3.6, Gemini 2.5, Claude 4, GPT 5.4 — hyphenated slug versions gpt-5-4 count; 38B/235B are sizes, not versions), else the base family (Claude, Kimi). One level only; the family entity carries attributes.family_root (Llama) for grouping. Slug = slugify(label); on collision with any other entity (the model gpt-5.5 itself) the organisation slug is prefixed, then family-. Organisation = the family's official publisher when known, else the model's. Writes: entities.family_id, relation member_of_family, attributes.family only when the model had none (a sourced family claim is never overwritten).

# licenses

normalize_license(attributes.license) → canonical key (SPDX id when one exists: Apache-2.0, MIT, CC-BY-NC-4.0; AI Atlas key otherwise: Llama-3.1-Community, Gemma-Terms). Creates one license entity per observed key (slug = key lower-cased, attributes = LicenseInfo.as_dict(): commercial use, redistribution, derivatives, hosting restrictions, acceptable-use policy, OSI approval…), relation uses_license, attributes.license_key. Unclassified strings stay in license (mapping recorded with canonical = NULL).

Openness 2.0 as derived claims (tier 2, extractor='derived'): weights_available (true with an hf_repo / model card / weights URL or an open-ish openness label; false when proprietary; unknown → nothing written), commercial_use_allowed, redistribution_allowed, derivatives_allowed from the licence, and the category openness = derive_openness(dims) (open-source | open-weights | restricted-weights | proprietary). HF "gated" is a distribution mechanism, not a licence property: a gated Apache-2.0 repo is open-weights. A tier-1 openness statement that disagrees is kept current; the derived value is stored once as conflicting.

# taxonomy

Current claims (and claim-less attributes) for license, openness, modalities, modalities_input, modalities_output, status, hardware and framework kind, org_kind are re-encoded in place: same claim row, same source, same tier — only the encoding changes (apache-2.0 → Apache-2.0, restricted → restricted-weights, ["Text","pdf"] → ["document","text"], computer → system). The source label goes to claims.value_raw / attributes.<prop>_raw. This is not a new assertion, so no change event is emitted. Every raw → canonical pair (and every unknown raw value) lands in taxonomy_mappings.

The writer applies the same normalisation to incoming facts; when a source re-states the same licence in another spelling the current claim is re-encoded and confirmed — never a LICENSE_CHANGED event.

# results

Back-fills config_key (comparability hash of the task keys + metric — variant, board, harness, evaluator, shots, scaffold, system…), trust_level (ontology.benchmarks.trust_level(source_key, config): official-benchmark / independent-evaluator / official-model-card / community / peer-reviewed / unverified), variant, run_group (release, index version, dataset revision, run date) and extractor. Then one current row per (model, benchmark, metric, config_key): rows from an older run group than the latest observed one get is_current = false, valid_to = <newer observation>, so leaderboards show the latest LiveBench release / aider run / AA index version instead of piling up every run. Run keys are not part of config_key (two releases of the same task are partially comparable); condition keys (reasoning_effort, temperature, judge…) are not either, so folded effort variants share a key and are compared as conditions. Scores outside the metric bounds are stored with confidence='low' and flagged (score_above_max / score_below_min).

# events

Three clocks: occurred_at = coalesce(effective_at, observed_at) (when it happened), observed_at (when a connector saw it), recorded_at (when the row was written). is_backfill = true when the event was observed before the connector's second successful run (the initial corpus is history, not news), when effective_at is more than 3 days before observed_at, or for NEW_* events whose entity's release_date / published_at predates observation by more than 3 days. group_key = release:<entity>:<yyyy-mm> for RELEASE, ANNOUNCEMENT, NEW_MODEL, VERSION_RELEASED so one release seen in several documents groups. Live counters ("+N · 24 h", /changes, pulse) count non-backfill events only; timelines use everything. The step prints live-event counts for the last 24 h before/after. Importance is deterministic (services.events.importance_for): frontier or open-weight releases 3, price moves ≥ 50 % 3 / ≥ 20 % 2 / else 1, context ≥ 5× 3, deprecations 3, leader changes 2, metadata corrections 0, artifact events 0, family events 1; tier > 2 sources lose one point.

# anomalies

Runs ontology.anomalies over live models/artifacts, hardware, live prices and current results; upserts anomalies rows by dedupe_key (reopened when a check fires again; operator states ignored / fixed are kept), and auto-resolves open flags whose check no longer fires. aia anomalies lists them. Flags never change the data.

# Write-time counterparts (SDK)

• FactWriter(run_id=…): every claim, relation, price, result and event carries the connector run id (batch inspection/rollback).
• Same-source loophole closed: a claim supersedes only when tier <= current.tier or same source and same extractor — an LLM claim never overwrites a deterministic one from the same URL (it is stored as conflicting, once).
• Resolver: keep_separate decisions block alias resolution/merges; aliases whose normalisation collapses digit separators (Qwen3-8B ≡ Qwen 38B → qwen38b) additionally require the same variant_key; first_seen_hint back-dates first_seen_at (min(now, hint)); hierarchy hints (EntityRef.family / .canonical / .artifact_kind / .identity_confidence) are materialised.
• Quarantine: tier ≥ 2 connectors (hubs, leaderboards, registries) hold their facts until the run ends and compare the run with the connector's rolling baseline (median of the last 5 full-extraction runs): new entities > max(50, 1.3× baseline), price rows > 3×, or — on a full re-extraction — entity references < 0.8× or results < 0.5× → the whole run goes to quarantined_runs (status='pending', serialised Facts), connector_runs.status='quarantined', a quarantine review item; nothing is written. aia quarantine release <id> writes the facts exactly as the connector would have; discard drops them (snapshots stay archived). A connector's first run is never quarantined — it sets the baseline.
• SSRF guard (sdk/fetch.py): every URL and every redirect hop (max 5) is resolved and refused when it targets a non-http(s) scheme, localhost/.local/.internal, RFC1918, loopback, link-local (cloud metadata), CGNAT, IPv6 loopback/link-local/ULA or unspecified addresses.

# Counting rules (for the API / UI)

• Models = entity_type = 'model' and merged_into is null (artifacts and folded variants excluded; include=artifacts restores the old universe). Artifacts = entity_type = 'artifact'. Families = entity_type = 'model_family'.
• Current benchmark results = valid_to is null and is_current. Live events = is_backfill = false.