SPB Git forge

spb/ai-atlas

Public
41commits 1branches 0releases
4.6 MBsize
maindefault branch
12 days agolast push
HTML 77.2% TypeScript 10.5% Python 9.6% JavaScript 2.5%

History of tests/test_huggingface.py · clear filter

  1. Sat, Sep 12, 2026 2

    1. OpenRouter routers are products, not models; official-org quantisations are artifacts with a same-org canonical (None when unsure)
      …
      `openrouter/*` ids (auto, auto-beta, pareto-code, free, fusion…) → `product` entities (kind router) operated by OpenRouter, price booked
      against the product when not dynamic. Hugging Face: a quant/precision token in the repo name makes the repo an artifact even under the
      developer's own org (nvidia/Gemma-4-31B-IT-NVFP4, tencent/HY-MT1.5-1.8B-FP8, LiquidAI/LFM2.5-230M-GGUF, black-forest-labs/FLUX.2-klein-4b-fp8);
      canonical = the model built from the stripped name under the family's developer (medium confidence), or None when nothing can be stripped
      or the developer is unknown — never an invented model.
      
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 12 days ago · 1 file +26
    2. Connectors + registry on the canonical ontology (Stream B): artifacts, effort variants as configurations, benchmark families, identity helper
      …
      Hugging Face: repositories are models or artifacts (quantization / conversion / packaging) with `canonical` → base model (base_model tag or
      analysed name, family org), `family` hints, canonical licence (+license_raw), `access: gated|open` instead of openness=restricted,
      weights_available + derived openness, modalities from pipeline tags, parameter_count kept on artifacts only; converters `publish`, never `develop`.
      Leaderboards: Artificial Analysis / LiveBench / aider / SWE-bench fold effort variants ("(xhigh)", "-thinking-64k-high-effort", non-reasoning)
      into the base model with the setting in config; LiveBench categories become `livebench-<category>` benchmarks; aider well-formed rate →
      `aider-polyglot-well-formed`; AA GPQA → gpqa-diamond, τ² → Telecom; trust_level / variant / run_group on every result; bookkeeping out of config;
      free-text tags are aliases, identifiers only for trusted pinned vendor ids. New shared `connectors/_identity.py` (provider prefixes, first-party
      patterns, effort stripping with size-tier guard, family hints).
      Registry: benchmarks gain family / variant / version / family_head / canonical metric / bounds / higher_is_better / harness / comparability_note +
      14 entries (gpqa-diamond, aime-2024, swe-bench-pro, arc-agi-2, terminal-bench-2, mmteb, livebench ×7, aider-polyglot-well-formed); seed writes them
      and `variant_of` relations, org_kind from `kind`. Organisations: kind + country fields, upstage / stepfun / kwaipilot / kuaishou / thinking-machines /
      lg-ai-research / xiaomi / inclusionai entries with hub aliases. Repositories: canonical kinds, agents (agent_kind) and tools get their own entity types.
      OpenRouter: prices booked on the OpenRouter provider only (features.upstream_provider), `created` → openrouter_listed_at (no release_date),
      canonical modalities, identity helper. Provider pricing: fireworks_model_id (+ legacy scheme), org from the id, family hints.
      Labs: no hard-coded Anthropic modalities/tool constants (table-driven), Google pdf → document + google_model_id, canonical licence/status/
      capabilities/api_aliases helpers in labs/_common.py, family_model hints everywhere. arXiv/OpenReview: researchers only with an identifier
      (ORCID / OpenReview profile). GitHub/PyPI: canonical kind (+kind_raw), licence keys. docs/CONNECTORS.md rewritten on the canonical vocabulary.
      
      Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
      Simon-Pierre Boucher committed 12 days ago · 1 file +42 −10
  2. Fri, Sep 11, 2026 1