spb/ai-atlas

Sat, Sep 12, 2026 1

1. ontology: official family orgs, effort-config helpers, chained/budget effort suffixes, hyphenated family versions, property-level normalisation, run keys split from task keys

   …

   - models: FAMILY_ORGS / official_orgs / is_official_org; effort_config() and base_name() shared by the writer and the canonicalizer;
     effort suffixes chain ("-thinking-64k-high-effort"), LiveBench budgets ("32k thinking", "(no thinking)"); "max"/"fast"/"instant" are model
     tiers (Qwen3-Max, GPT-5.1-Codex-Max, Grok 4.1 Fast), no longer effort markers; family_release_hint reads "gpt-5-4" as 5.4 and never a size (38B)
   - taxonomy: normalize_property(entity_type, prop, value) → (canonical, value_raw, mappings); unknown values kept, mapped to NULL
   - benchmarks: RUN_KEYS (release, index_version, version, dataset_revision) leave config_key so a newer run replaces the older one; they are
     condition keys for comparability (partially comparable)
   - tests/test_ontology.py: licence table, openness derivation, taxonomy, 22 model names, comparability trio, metric bounds/trust
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher committed 12 days ago · 1 file +162
   e74d6fc