HTML 77.2%
TypeScript 10.5%
Python 9.6%
JavaScript 2.5%
1# Writing a connector23A connector is a Python class under `src/aiatlas/connectors/<group>/<name>.py`, listed in that module's `CONNECTORS = [Cls]`.4The registry auto-discovers it; `aia seed` registers it in the database; `aia run <name>` runs it; the scheduler runs it on its interval.56## Skeleton78```python9from aiatlas.registry import org_ref, provider_ref10from aiatlas.sdk.connector import BaseConnector, Parsed, RunContext11from aiatlas.sdk.facts import Facts, Target12from aiatlas.sdk.fetch import FetchResult1314class ExampleConnector(BaseConnector):15 name = "example" # unique, snake_case, stable (used in documents/claims/events)16 label = "Example Lab — models & news"17 description = "Official docs and blog of Example Lab."18 source_key = "example.com" # must exist in registry/sources.yaml (tier, rate limit, org)19 version = "1"; parser_version = "1" # bump parser_version when extraction improves → `aia reprocess example`20 interval_seconds = 3600; min_interval_seconds = 1800; max_interval_seconds = 8640021 rate_per_min = 15; tier = 1; priority = 0 | 1 | 222 expected_min_records = 5 # breakage detection: fewer entities/prices/results ⇒ run "suspect", nothing deleted23 concurrency = 22425 async def discover(self, ctx: RunContext) -> list[Target]:26 return [Target(url="https://example.com/models", doc_type="model_docs", key="models", min_bytes=2000),27 Target(url="https://example.com/blog/rss.xml", doc_type="feed", key="feed")]2829 async def extract(self, ctx: RunContext, target: Target, res: FetchResult, parsed: Parsed) -> Facts:30 facts = Facts()31 org = org_ref("example") # registry organization → deterministic identifiers32 facts.entities.append(org)33 if target.key == "models" and parsed.html:34 for row in parsed.html.tables[0]["rows"]: ...35 model = model_ref(facts, name, org, api_id=api_id, provider_key="example") # from connectors/labs/_common.py: identity + family hint36 facts.claim(model, "context_length", 128000, unit="tokens")37 facts.price(model=model, provider=provider_ref("example"), input_per_mtok=1.0, output_per_mtok=4.0)38 facts.follow(detail_url, doc_type="model_page", entity=model, needs_llm=True, meta={"llm_task": "model_passport"})39 elif target.key == "feed" and parsed.kind == "feed":40 announcement_events(facts, org, parsed.feed_items, source_name="example.com/blog") # from connectors/labs/_common.py41 return facts4243CONNECTORS = [ExampleConnector]44```4546## What `Parsed` gives you4748| `parsed.kind` | fields |49|---|---|50| `html` | `parsed.html: HtmlDoc` — `title, canonical, description, meta, og, json_ld, embedded_json (__NEXT_DATA__, data-props…), headings, tables [{headers, rows}], links [(href, text)], text`, `css()`, `links_matching()` ; `find_in_json(obj, key)` helper |51| `markdown` | `parsed.markdown: MarkdownDoc` — `front_matter, headings, tables, links, text, section(pattern)` |52| `feed` | `parsed.feed_items: list[FeedItem]` (`url, title, summary, published_at, categories, authors`) |53| `json` | `parsed.json` |54| `pdf` / `text` / `xml` | `parsed.text` |5556Helpers: `aiatlas.sdk.extract.numbers` (`parse_param_count("70B")`, `parse_active_params("235B-A22B")`, `parse_context_length("128K")`,57`parse_money_per_mtok("$3 / 1M tokens")`, `parse_percent`), `aiatlas.sdk.extract.dates` (`parse_datetime`, `parse_date` with precision),58`connectors/labs/_common.py` (`announcement_events`, `transpose_feature_table`, `kv_tables`, `clean_cell`, `money`, `tokens`, `month_year`, `parse_retirement`,59`model_ref`, `claim_license`, `claim_status`, `claim_modalities`, `claim_api_aliases`, `normalize_capabilities`),60`connectors/_identity.py` (see below).6162## Rules63641. **Direct mode first.** Use official pages, feeds, sitemaps, Markdown/raw files, embedded JSON. Public JSON files (e.g. `openrouter.ai/api/v1/models`,65 `pypi.org/pypi/<pkg>/json`, arXiv Atom) are fine — they are public documents, not commercial APIs. Never require an API key.66 Set `Target(escalate=True)` only for pages known to block bots *and* only if a key is configured; otherwise let the document be `blocked`.672. **Never fabricate.** Only emit a claim when the page states it. Unknown → no claim. Don't infer parameter counts from names unless the68 name literally contains them (`Qwen3-235B-A22B` → 235e9 / 22e9 is fine; "Large" is not). No hard-coded "all models of this lab do X" constants:69 read the comparison table, else omit.703. **Identifiers make resolution deterministic.** Give models the provider's API id (`{provider}_model_id`), HF repo (`hf_repo`), arXiv id71 (`arxiv`), GitHub repo (`github_repo`), PyPI name (`pypi`). Organizations always come from `org_ref(<registry key>)` (add missing orgs to72 `registry/organizations.yaml` with a `source_url`) — **never guess a developer from a display name**; the `_identity` helper only maps73 first-party prefixes and family words (`claude-` → Anthropic, `gemini-` → Google, `openrouter/x-ai/…` → xAI).744. **Properties are shared vocabulary** (see below) and **values are canonical**: run licences through `ontology.normalize_license`, statuses through75 `normalize_status`, modalities through `normalize_modalities`, capabilities through `_common.normalize_capabilities`; keep the source label in76 `<property>_raw` when it differed. Prefix metrics with `metric.` (downloads, likes, stars) so they never generate events.775. **Events**: NEW_*, PRICE_CHANGED, CONTEXT_CHANGED… are emitted automatically by the writer. Emit `ANNOUNCEMENT`/`RELEASE` events yourself for78 feed items (`announcement_events`) with `effective_at` = publication date and a `dedupe_key` (URL).796. **Follow-ups** (`facts.follow`) let a listing discover detail pages; keep `max_targets` reasonable (`ctx.max_targets`, default 2000).807. **Every connector has a fixture test**: save real responses under `tests/fixtures/<connector>/…` and assert extracted facts81 (see `tests/test_anthropic.py`). Run connectors with `--file key=path` to use fixtures instead of the network.828. Respect `rate_per_min` from `registry/sources.yaml`; arXiv ≤ 4/min; Hugging Face ≤ 30/min; GitHub ≤ 20/min.839. **Results carry provenance**: every `ResultObs` sets `trust_level` (`ontology.benchmarks.trust_level(source_key, config)`), `variant`84 (GPQA Diamond, SWE-bench Verified, a LiveBench category…) and `run_group` (the evaluation run the row belongs to: LiveBench release,85 aider run date, AA index version). Bookkeeping (cost, wall time, command lines, harness versions) stays out of `config`.8687## Identity: model families, models, artifacts, configurations8889The canonical hierarchy is **model_family → model → artifact → deployment** (`aiatlas.ontology.models`). Connectors describe it with hints on90`EntityRef`; the writer materialises `entities.family_id`, `canonical_id`, `artifact_kind`, `identity_confidence` and the relations91`member_of_family` / `artifact_of`.9293| hint | meaning | who sets it |94|---|---|---|95| `family=EntityRef("model_family", "Qwen3.6", organization=org)` | the versioned family the model belongs to (`_identity.family_ref(name, org)` or the lab's own label via `model_ref(family=…)`) | every model ref with a detectable family |96| `canonical=EntityRef("model", …)` | for an **artifact**: the model it packages; for an alias entity: the canonical entity | Hugging Face (quantisations, conversions, mirrors) |97| `artifact_kind` | `checkpoint` \| `quantization` \| `conversion` \| `packaging` | Hugging Face |98| `identity_confidence` | `high` (vendor id / official repo) · `medium` (leaderboard label, analysed base name) · `low` | everyone |99100**Entity types**: `model` (a release: Qwen3-8B, Claude Opus 5), `model_family` (Qwen3, Claude), `artifact` (`bartowski/Qwen3.8-27B-GGUF`,101`zai-org/GLM-5-FP8`, `mlx-community/Kimi-K2.5` — never a model of its own; keeps the full repo id as name, `parameter_count` = the packaged size),102`agent` (coding/browser agents from the repositories registry, attribute `agent_kind`), `tool` (applications, MCP servers), `framework`103(libraries, SDKs, inference engines… attribute `kind` canonical), `repository` (model-code drops), `researcher` (**only with an identifier**:104ORCID or OpenReview profile id — never from a bare author name; authors stay a claim on the paper).105106**Effort variants are configurations, not models.** `claude-opus-5-xhigh`, "GPT-5.5 (xhigh)", `deepseek-v3-1-reasoning`,107`…-thinking-64k-high-effort`, "Mistral Small 4 (Non-reasoning)" all point at the base model; the setting goes into `ResultObs.config`108(`reasoning_effort`, `reasoning: on|off|adaptive`, `thinking_budget`) and the evaluator's slug into `config.aa_slug` / `livebench_model_id`.109Size tiers and official products are not efforts: `mistral-medium`, `qwen3.7-max`, `sonar-reasoning`, `kimi-k2-thinking`, `grok-4-1-fast` stay models110(`_identity.strip_effort` only strips an ambiguous suffix when the stem still carries a version digit).111112### `connectors/_identity.py`113114`model_identity(api_id, trusted=False)` analyses an API id from a third-party source (`anthropic/claude-3-7-sonnet-20250219`,115`gemini/gemini-2.5-pro-preview-05-06`, `openrouter/x-ai/grok-4`, `gpt-4o-2024-08-06`, `Qwen/Qwen2.5-Coder-32B-Instruct`,116`fireworks_ai/accounts/fireworks/models/qwq-32b`) and returns the developer organisation key, resolver-friendly aliases (raw id, id without117provider prefix, id without effort suffixes), the effort configuration and — **only when `trusted=True`** (the string really is the vendor's own118API id, e.g. the `--model` argument aider passed) — the vendor identifier (`anthropic_model_id`, `gemini_model_id`, `openai_model_id`,119`xai_model_id`, `deepseek_model_id`, `mistral_model_id`, `cohere_model_id`, `hf_repo`, `openrouter`, `fireworks_model_id`). Free-text tags120(SWE-bench "Model:" tags, LiveBench ids, OpenRouter slugs) never become identifiers: the resolver refuses an alias match when the entity already121carries a *different* value for the same scheme, so a wrong id would split an entity instead of linking it. Rolling ids122(`deepseek/deepseek-chat`, `codestral-latest`) are aliases only even when trusted — they name whichever snapshot served that day.123`model_ref_from_api_id(facts, api_id, name=…, trusted=…)` builds the deduplicated model `EntityRef` (+ family hint) and returns the effort config.124`family_ref`, `split_effort_label`, `strip_effort`, `org_key_for_vendor`, `org_ref_in` are the building blocks.125126## Property vocabulary (entities.attributes)127128Canonical enums live in `src/aiatlas/ontology/` — connectors write canonical values and keep the source label in `<property>_raw`.129130**model**: `family` (label of the `model_family` hint), `version, release_date (ISO, may be YYYY-MM), status` (`ontology.taxonomy.MODEL_STATUSES`:131`announced | preview | active | limited-availability | deprecated | retired | archived | unknown`), `openness` (`ontology.openness.OPENNESS_CATEGORIES`:132`open-source | open-weights | restricted-weights | proprietary | unknown` — **derived** with `derive_openness` from `weights_available` + the licence,133asserted directly only when the source literally states it), `weights_available` (bool), `access` (`gated | open`, Hugging Face gating — access,134not a licence property), `gated_mode`, `license` (`ontology.licenses` key: `Apache-2.0`, `MIT`, `Llama-3.1-Community`, `Mistral-Research`,135`Gemma-Terms`… — SPDX id when one exists) + `license_raw`, `architecture, parameter_count (int), active_parameter_count, is_moe, modalities /136modalities_input / modalities_output` (`ontology.taxonomy.MODALITIES`: `text | image | audio | video | document | code | embedding | 3d | structured | action`,137sorted), `context_length (tokens), max_output_tokens, knowledge_cutoff (YYYY-MM), training_data_cutoff, languages, capabilities` (canonical slugs:138`function_calling, structured_output, reasoning, vision, audio_input, audio_output, image_generation, video_generation, code_execution,139search_grounding, caching, batch, fine_tuning, streaming, live_api, computer_use, file_search, url_context, mcp…` via `_common.normalize_capabilities`)140+ `capabilities_raw`, `tool_calling, structured_output, reasoning, vision, audio, fine_tuning_available, tokenizer, api_model_id, api_aliases` (always a141list; `api_alias` = the first one, kept for compatibility), `official_url, model_card_url, paper_url, repository_url, hf_repo, pipeline_tag, base_model,142quantization, quant_format (gguf|mlx|awq|gptq|fp8|…), artifact_kind, is_quantized, quantized_by, file_size_gb, deprecation_date, retirement_date,143retirement_tentative, openrouter_listed_at` (listing date — never `release_date`), `aa_release_date / aa_openness / aa_context_window` (second-hand144facts from Artificial Analysis), `metric.downloads, metric.likes`145146**company / organization / lab**: `country (ISO-2), headquarters, founded, website, domains, hf_org, github_org, org_kind`147(`ontology.taxonomy.ORG_KINDS`: `company | lab | university | nonprofit | government | community | consortium | individual`), `legal_name, founders, leadership, employee_count`148149**provider**: `website, pricing_url, docs_url, regions, features` · **paper**: `authors, published_at, updated_at, abstract, arxiv_id, doi, categories, primary_category, pdf_url, code_url, venue`150· **researcher**: `openreview_profile_url` (+ identifiers `orcid` / `openreview_profile`)151· **benchmark** (registry `registry/benchmarks.yaml` + `.d/`, see below): `family, variant, version, family_head, category, task, metric` (canonical:152`accuracy | pass@1 | pass^1 | resolved | pass_rate_2 | percent_cases_well_formed | global_average | average score | mean score | index | elo …`),153`metric_label, metric_raw, unit, metric_min, metric_max, higher_is_better, harness, comparability_note, creator, website, paper, known_limitations, methodology`154· **hardware**: `kind` (`ontology.taxonomy.HARDWARE_KINDS`), `architecture, release_date, memory_gb, memory_type, memory_bandwidth_gbs, tdp_watts, runtimes, manufacturer, spec_url,155price_usd, compute_fp16_tflops` · **framework / tool / agent / repository**: `kind` (`ontology.taxonomy.FRAMEWORK_KINDS`: `training-framework | inference-engine |156serving-engine | library | runtime | agent-framework | orchestration | evaluation-harness | sdk | tool | application | agent | mcp-server | vector-database |157observability | data-tooling`) + `kind_raw`, `agent_kind` (`coding | browser | research`), `repository_url, latest_version, latest_release_at, license (+ license_raw),158language, description, topics, pypi, metric.stars, metric.forks` · **dataset**: `license, modality, size, publisher, task, hf_repo`159160Relations: `develops, published_by` (artifact → its publishing org; converters never `develop`), `owns, operates, available_through, evaluated_on,161described_by, derived_from, fine_tuned_from, quantized_from, distilled_from, merged_from, superseded_by, variant_of` (benchmark variant → family head),162`runs_on, uses, manufactures, funded_by, acquired, authored, works_at, uses_dataset, evaluates_on, integrates`. The writer adds `member_of_family` and163`artifact_of` from the hints.164165## Prices166167`PriceObs(model, provider, provider_model_id, …)` — one row per model × provider × provider id. Aggregator prices belong to the aggregator:168OpenRouter rows are booked on `provider_ref("openrouter")` with `features.upstream_provider` = the routed lab's registry provider key (never on169the lab's own provider entity, which would create two "current" prices). Variants (`:free`, `:thinking`, `:nitro`) are separate rows of the same170model (`features.variant`). Identifier schemes for provider catalogues: `openrouter`, `groq_model_id`, `together_ai_model_slug`,171`fireworks_model_id` (the historical `fireworks-ai_model_id` is still emitted alongside).172173## Benchmark registry (`registry/benchmarks.yaml` + `registry/benchmarks.d/*.yaml`)174175Every entry declares `family`, `variant`, optional `version`, `family_head: true` on the representative member, canonical `metric`176(`ontology.benchmarks.normalize_metric`) with `metric_min` / `metric_max` / `higher_is_better`, `harness` and `comparability_note`, plus the177aliases evaluators use ("GPQA Diamond", "HLE", "SWE-Bench Verified", "τ²-Bench Telecom"…). `aia seed` writes them as claims and a `variant_of`178relation from each member to its family head. One benchmark entity per measured thing: LiveBench categories are `livebench-<category>` entities179(family `livebench`), aider's well-formed rate is `aider-polyglot-well-formed`, Artificial Analysis' GPQA results land on `gpqa-diamond`.180Connectors address benchmarks with `benchmark_ref(key)` (identifier `registry_benchmark`, `slug_hint` = key). Slugs are only hints: when an181organisation already owns the slug (`livebench` the org), the resolver appends a collision suffix (`livebench-2` in production) — identity is182the `registry_benchmark` identifier, and existing production slugs never change.183