SPB Git

spb/ai-atlas

Public
16.8 KB

# AI Atlas API — contract (v1)

Base: /api/v1 (FastAPI, src/aiatlas/api). JSON, UTF-8, ISO-8601 UTC timestamps, numbers as numbers (Postgres aggregates may arrive as strings — the web layer normalises). Public routes are cached in Redis (aia:api:*, 60–600 s) and rate-limited per IP on search. Admin routes require header x-aia-admin-token: <AIA_ADMIN_TOKEN>. OpenAPI at /api/v1/docs. Errors: {"detail": "..."} with 400/404/429/503. Never expose credentials, internal hostnames, raw archive content or raw_paths.

# Shared shapes

ts
type Num = number | string | null
type Org = { id: string; slug: string; name: string } | null
type EntitySummary = { id: string; entity_type: string; slug: string; name: string; description: string | null; status: string;
  organization: Org; attributes: Record<string, unknown>; quality: { score?: number; completeness?: number; primary_source_ratio?: number;
  freshness?: number; source_count?: number; conflicts?: number }; counts: { relations?: number; events?: number; claims?: number };
  first_seen_at: string; last_seen_at: string; updated_at: string }
type Page<T> = { items: T[]; total: number; limit: number; offset: number }
type Provenance = Record<string, { source_id: string | null; source_name?: string; url: string | null; observed_at: string; tier: number;
  confidence: string; extractor: string; unit?: string }>
type ChangeEvent = { id: string; event_type: string; category: string; property: string | null; old_value: unknown; new_value: unknown;
  summary: string; importance: 0 | 1 | 2 | 3; observed_at: string; effective_at: string | null; source_url: string | null; connector_name: string | null;
  entity: EntitySummary | null; meta: Record<string, unknown> }
type Price = { id: string; model: EntitySummary; provider: EntitySummary; provider_model_id: string | null; input_per_mtok: Num; output_per_mtok: Num;
  cached_input_per_mtok: Num; cache_write_per_mtok: Num; batch_input_per_mtok: Num; batch_output_per_mtok: Num; per_image: Num; currency: string;
  context_length: Num; max_output_tokens: Num; features: Record<string, unknown>; observed_at: string; valid_from: string; valid_to: string | null; source_url: string | null; tier: number }
type BenchmarkResult = { id: string; model: EntitySummary; benchmark: EntitySummary; score: number; metric: string | null; unit: string | null;
  higher_is_better: boolean; config: Record<string, unknown>; evaluated_at: string | null; observed_at: string; source_url: string | null; tier: number; confidence: string }
type Claim = { id: string; property: string; value: unknown; unit: string | null; tier: number; confidence: string; status: string; extractor: string;
  observed_at: string; effective_at: string | null; valid_from: string; valid_to: string | null; source_url: string | null; source_name: string | null }
type SourceRef = { source_id: string | null; source_name: string | null; domain: string | null; tier: number | null; url: string; doc_type: string; last_observed_at: string | null; snapshots: number }

# Public routes

route returns
GET /health `{ status: "ok"
GET /stats { entities: Record<type, number>, entities_total, sources, connectors, connectors_enabled, documents, snapshots, claims, claims_current, relations, change_events, change_events_24h, change_events_7d, benchmark_results, prices_current, prices_total, review_pending, llm_jobs, llm_tokens, last_snapshot_at, last_event_at, first_entity_at, archive: { raw_bytes, raw_files, text_bytes, text_files }, computed_at } — always live from the DB
GET /stats/history?days=90 { items: { day: string; counts: object }[] }
GET /search?q=&type=&limit=30&offset=0 { query: CompiledQuery, items: (EntitySummary & { rank: number })[], total: number } — uses services.search.compile_query + search_entities (+ embedding when the gateway is up)
GET /search/suggest?q= { items: { id, entity_type, slug, name, organization_name }[] } (≤ 8, prefix, fast)
GET /entities/{slug_or_id} EntityDetail (below). Also mounted as /models/{slug}, /companies/{slug}, /papers/{slug}, /providers/{slug}, /benchmarks/{slug}, /hardware/{slug}, /frameworks/{slug}, /datasets/{slug}, /tools/{slug} (404 if the type doesn't match)
GET /entities/{slug}/timeline?limit=50&before= { items: ChangeEvent[] } (entity's own events + events of entities it develops for companies)
GET /entities/{slug}/history?property= { items: Claim[] } — full claim history (all statuses), newest first
GET /entities/{slug}/asof?date=YYYY-MM-DD { existed: bool, first_seen_at, date, attributes: Record<string, unknown>, claims: Claim[] } — state as known at that date
GET /entities/{slug}/graph?depth=1&limit=80 { nodes: { id, slug, name, entity_type, organization_name? }[], edges: { source, target, predicate }[] } (depth ≤ 2, cap nodes)
GET /entities/{slug}/sources { items: SourceRef[] }
GET /entities/{slug}/related?limit= { items: EntitySummary[] } (same org / same family / shared relations)
`GET /models?q=&org=&family=&openness=&modality=&status=&min_params=&max_params=&min_context=&year_from=&year_to=&license=&sort=(updated name
`GET /companies?q=&country=&kind=&sort=(models name
`GET /papers?q=&category=&org=&since=&until=&sort=(published updated)&limit=&offset=`
GET /providers { items: (EntitySummary & { model_count: number; price_count: number; min_input_per_mtok: Num; min_output_per_mtok: Num })[] }
`GET /prices?model=&provider=&sort=(input output
GET /prices/history?model=&provider= { items: Price[] } (all rows incl. closed, oldest first)
GET /prices/index?days=180 { series: { day: string; median_input: Num; median_output: Num; min_input: Num; models: number }[]; movers: ChangeEvent[] } computed from prices history
GET /benchmarks `{ items: (EntitySummary & { result_count: number; model_count: number; top: { model: EntitySummary; score: number }
GET /benchmarks/{slug}/results?limit=&offset=&config= Page<BenchmarkResult> sorted by score (respecting higher_is_better), current rows only unless history=1
GET /benchmarks/{slug}/history?model= { items: BenchmarkResult[] } over time
GET /hardware?kind=&manufacturer=&min_memory=&sort= Page<EntitySummary>
`GET /hardware/fit?memory_gb=&quant=(4bit 8bit
GET /explore/types { items: { entity_type, count, label }[] }
GET /explore/{type}?q=&org=&sort=&limit=&offset= Page<EntitySummary> generic listing for any type (datasets, frameworks, tools, repositories, regulation…)
GET /changes?category=&type=&entity_type=&importance_min=&since=&until=&q=&limit=50&before=<observed_at cursor> Page<ChangeEvent> newest first; type = comma list of event types; excludes DOCUMENT_CHANGED unless include_documents=1
GET /changes/daily?date=YYYY-MM-DD { date, counts: Record<category, number>, sections: { category, label, items: ChangeEvent[] }[], new_models: EntitySummary[] } ("What changed in AI today", DB-generated only)
GET /changes/categories?days=7 { items: { category, event_type, count }[] }
GET /timeline?entity=&year=&category=&limit= { items: { month: string; events: ChangeEvent[] }[] } grouped by month; global when no entity
GET /compare?ids=slug,slug,… (2–6) `{ entity_type, dimensions: { key, label, unit?, kind: "number"
`GET /diff?a=YYYY-MM-DD&b=YYYY-MM-DD&scope=(all models
GET /sources { items: { key, name, domain, tier, kind, category, organization: Org, enabled, documents, last_crawled_at, connectors: { name, label, health, last_success_at, interval_seconds }[] }[] } (public transparency page)
GET /methodology { metrics: metric_definitions[], confidence_levels, tiers, event_types, extractors }
GET /trending?days=7&limit=12 { items: (EntitySummary & { views: number })[] } from page_views
POST /views { path } { ok: true } (beacon; 1 req/s/IP)
GET /sitemap?type=&limit=5000&offset=0 { items: { slug, entity_type, updated_at }[], total }
GET /api-keys/me (header x-api-key) { label, plan, rate_per_min, usage_count } — developer keys (later); public routes work without a key

# EntityDetail

ts
EntitySummary & {
  attributes: Record<string, unknown>            // full
  provenance: Provenance
  aliases: string[]; identifiers: { scheme: string; value: string }[]
  relations: { predicate: string; direction: "out" | "in"; items: EntitySummary[]; total: number }[]   // grouped, ≤ 24 per group
  sources: SourceRef[]                           // documents describing this entity (deduped by URL)
  timeline: ChangeEvent[]                         // latest 30
  quality: {...}; counts: {...}
  // type-specific blocks (present only when relevant)
  prices?: Price[]                                // current rows across providers (model) or for this provider (provider)
  price_history?: Price[]                         // closed + current, oldest first (model)
  results?: BenchmarkResult[]                     // model: its results · benchmark: leaderboard top 100
  lineage?: { ancestors: EntitySummary[]; descendants: EntitySummary[]; quantizations: EntitySummary[] }   // model
  providers?: EntitySummary[]                     // model
  hardware_fit?: { hardware: EntitySummary; quantization: string; estimated_memory_gb: number; fits: boolean }[]   // model, ESTIMATED
  models?: Page<EntitySummary>                    // company / provider / hardware(runnable)
  papers?: EntitySummary[]                        // company / model
  repositories?: EntitySummary[]                  // company / model / framework
}

# Admin routes (x-aia-admin-token)

route returns / action
GET /admin/overview { stats, queue: Record<kind, Record<status, n>>, heartbeats, connectors: { ok, degraded, failing, disabled }, review_pending, recent_errors: n, llm: { jobs_24h, tokens_24h, by_stage } , archive }
GET /admin/connectors full connectors rows + last run + docs/snapshots counts + source tier
POST /admin/connectors/{name}/run { force?: bool } enqueue an immediate run (Redis flag aia:run-now:<name> consumed by the scheduler tick) → { queued: true }
PATCH /admin/connectors/{name} { enabled?, interval_seconds?, priority? } update
GET /admin/runs?connector=&limit= · GET /admin/errors?connector=&limit= rows
GET /admin/documents?connector=&status=&q=&limit=&offset= · GET /admin/documents/{id} documents (+ snapshots list); GET /admin/snapshots/{id} → metadata + structured + diff + first 20 kB of cleaned text (never raw HTML)
GET /admin/jobs?status=&kind= · POST /admin/jobs/{id}/retry · POST /admin/jobs/requeue-dead queue
GET /admin/llm-jobs?limit= · GET /admin/llm/health LLM factory accounting
GET /admin/review?status=pending&kind= · POST /admin/review/{id} `{ action: "approve" "reject"
GET /admin/entities/duplicates?type=&limit= candidate pairs by normalized name similarity (pg_trgm) within a type
POST /admin/entities/merge { source_id, target_id } · POST /admin/entities/{id}/claims/{claim_id}/retract curation
POST /admin/reprocess { connector, url? } enqueue reprocess_snapshot
POST /admin/llm/enqueue { limit?, task? } queue llm_extract for pending snapshots
GET /admin/infrastructure heartbeats, archive size, DB size (pg_database_size), table sizes, node hostname, uptime
POST /admin/cache/flush { flushed: n }
POST /admin/stats/recompute · POST /admin/quality/recompute maintenance

# Implementation notes (2026-09-11, src/aiatlas/api)

Deviations and additions relative to the table above — everything else is implemented as written.

  • Validation errors: parameters rejected by FastAPI/pydantic typing or bounds (limit=999, non-integer depth…) return 422 with { detail: "<field>: <message>", errors: [...] }; semantic errors raised by the routes (bad sort, bad date, wrong scope, ids count) return 400.
  • /companies (listing, alias, /compare type company) covers the organization-like types company | organization | lab | university. /frameworks/{slug} also accepts library | runtime; /tools/{slug} accepts tool | agent | application | product | mcp_server.
  • /explore/{type} accepts plural forms (/explore/models, /explore/companies) and adds entity_type + label to the page.
  • EntitySummary.description is truncated at 280 characters in summaries (full text in EntityDetail).
  • Cursor feeds (/changes, /entities/{slug}/timeline) add next_before (the observed_at of the last item, or null at the end). /changes also honours offset and entity= for convenience; total is capped at 10 000.
  • /changes/daily adds total, labels, previous_day, next_day; sections are capped at per_section (default 30) items each.
  • /timeline month groups carry count; the response includes total (events returned).
  • /search: query carries semantic: bool (embedding used or not). When the embedding endpoint times out (2 s) the API backs off to FTS-only for 5 minutes (aia:api:search:embed-degraded) so the LLM never adds latency to consecutive queries.
  • hardware_fit (model detail) is omitted when parameter_count is unknown (nothing is estimated from thin air); when present the detail also carries hardware_fit_assumptions: string[]. /hardware/fit adds estimated: true, counts, optional openness= filter.
  • /prices/index series rows add max_input and offers; response adds days and note. /prices/history requires model or provider.
  • /benchmarks/{slug}/results and /history include the benchmark summary; result rows include valid_to.
  • /diff adds scope (resolved description) and extra counts (events, price_rows_opened/closed, claims_superseded, entities_at_a/b).
  • /sources items add snapshots, claims, base_url, robots_policy, priority, notes; response adds tiers legend.
  • /methodology adds quality_version, expected_fields, principles; event_types are live counts from change_events.
  • /trending accepts type=; /sitemap limit ≤ 5000 (the one listing not capped at 200).
  • Rate limits are in-process sliding windows (search 60/min, views 1/s, admin 240/min per IP) — fine for the single uvicorn worker behind Next.js.
  • Admin: GET /admin/review?status=all lists every status; approving a conflict with resolution.keep_claim_id promotes that claim; PATCH /admin/connectors/{name} also resets health/circuit_open_until when toggling; /admin/connectors flags run_now_pending and connectors present in code but not in the table; /admin/snapshots/{id} never returns raw_path/text_path (has_raw/has_text booleans instead).
  • Migration 0002_api_indexes adds partial indexes for the read paths (type + release_date/family/openness/updated/first_seen, importance-ordered events, current leaderboards/prices, page views, review kinds).
Files · afabc36216d003b383d172abe2f5ecd612a1e4ec
Menu

Theme