spb/ai-atlas

1# AI Atlas API — contract (v1)23Base: `/api/v1` (FastAPI, `src/aiatlas/api`). JSON, UTF-8, ISO-8601 UTC timestamps, numbers as numbers (Postgres aggregates may arrive as4strings — the web layer normalises). Public routes are cached in Redis (`aia:api:*`, 60–600 s) and rate-limited per IP on search.5Admin routes require header `x-aia-admin-token: <AIA_ADMIN_TOKEN>`. OpenAPI at `/api/v1/docs`. Errors: `{"detail": "..."}` with 400/404/429/503.6Never expose credentials, internal hostnames, raw archive content or `raw_path`s.78## Shared shapes910```ts11type Num = number | string | null12type Org = { id: string; slug: string; name: string } | null13type EntitySummary = { id: string; entity_type: string; slug: string; name: string; description: string | null; status: string;14  organization: Org; attributes: Record<string, unknown>; quality: { score?: number; completeness?: number; primary_source_ratio?: number;15  freshness?: number; source_count?: number; conflicts?: number }; counts: { relations?: number; events?: number; claims?: number };16  first_seen_at: string; last_seen_at: string; updated_at: string }17type Page<T> = { items: T[]; total: number; limit: number; offset: number }18type Provenance = Record<string, { source_id: string | null; source_name?: string; url: string | null; observed_at: string; tier: number;19  confidence: string; extractor: string; unit?: string }>20type ChangeEvent = { id: string; event_type: string; category: string; property: string | null; old_value: unknown; new_value: unknown;21  summary: string; importance: 0 | 1 | 2 | 3; observed_at: string; effective_at: string | null; source_url: string | null; connector_name: string | null;22  entity: EntitySummary | null; meta: Record<string, unknown> }23type Price = { id: string; model: EntitySummary; provider: EntitySummary; provider_model_id: string | null; input_per_mtok: Num; output_per_mtok: Num;24  cached_input_per_mtok: Num; cache_write_per_mtok: Num; batch_input_per_mtok: Num; batch_output_per_mtok: Num; per_image: Num; currency: string;25  context_length: Num; max_output_tokens: Num; features: Record<string, unknown>; observed_at: string; valid_from: string; valid_to: string | null; source_url: string | null; tier: number }26type BenchmarkResult = { id: string; model: EntitySummary; benchmark: EntitySummary; score: number; metric: string | null; unit: string | null;27  higher_is_better: boolean; config: Record<string, unknown>; evaluated_at: string | null; observed_at: string; source_url: string | null; tier: number; confidence: string }28type Claim = { id: string; property: string; value: unknown; unit: string | null; tier: number; confidence: string; status: string; extractor: string;29  observed_at: string; effective_at: string | null; valid_from: string; valid_to: string | null; source_url: string | null; source_name: string | null }30type SourceRef = { source_id: string | null; source_name: string | null; domain: string | null; tier: number | null; url: string; doc_type: string; last_observed_at: string | null; snapshots: number }31```3233## Public routes3435| route | returns |36|---|---|37| `GET /health` | `{ status: "ok"|"degraded", version, db: bool, redis: bool, llm: { available, reachable? }, time }` |38| `GET /stats` | `{ entities: Record<type, number>, entities_total, sources, connectors, connectors_enabled, documents, snapshots, claims, claims_current, relations, change_events, change_events_24h, change_events_7d, benchmark_results, prices_current, prices_total, review_pending, llm_jobs, llm_tokens, last_snapshot_at, last_event_at, first_entity_at, archive: { raw_bytes, raw_files, text_bytes, text_files }, computed_at }` — **always live from the DB** |39| `GET /stats/history?days=90` | `{ items: { day: string; counts: object }[] }` |40| `GET /search?q=&type=&limit=30&offset=0` | `{ query: CompiledQuery, items: (EntitySummary & { rank: number })[], total: number }` — uses `services.search.compile_query` + `search_entities` (+ embedding when the gateway is up) |41| `GET /search/suggest?q=` | `{ items: { id, entity_type, slug, name, organization_name }[] }` (≤ 8, prefix, fast) |42| `GET /entities/{slug_or_id}` | `EntityDetail` (below). Also mounted as `/models/{slug}`, `/companies/{slug}`, `/papers/{slug}`, `/providers/{slug}`, `/benchmarks/{slug}`, `/hardware/{slug}`, `/frameworks/{slug}`, `/datasets/{slug}`, `/tools/{slug}` (404 if the type doesn't match) |43| `GET /entities/{slug}/timeline?limit=50&before=` | `{ items: ChangeEvent[] }` (entity's own events + events of entities it develops for companies) |44| `GET /entities/{slug}/history?property=` | `{ items: Claim[] }` — full claim history (all statuses), newest first |45| `GET /entities/{slug}/asof?date=YYYY-MM-DD` | `{ existed: bool, first_seen_at, date, attributes: Record<string, unknown>, claims: Claim[] }` — state as known at that date |46| `GET /entities/{slug}/graph?depth=1&limit=80` | `{ nodes: { id, slug, name, entity_type, organization_name? }[], edges: { source, target, predicate }[] }` (depth ≤ 2, cap nodes) |47| `GET /entities/{slug}/sources` | `{ items: SourceRef[] }` |48| `GET /entities/{slug}/related?limit=` | `{ items: EntitySummary[] }` (same org / same family / shared relations) |49| `GET /models?q=&org=&family=&openness=&modality=&status=&min_params=&max_params=&min_context=&year_from=&year_to=&license=&sort=(updated|name|params|context|release|quality|downloads)&order=&limit=&offset=` | `Page<EntitySummary>` + `facets` when `facets=1`: `{ organizations: {slug,name,count}[], openness: {value,count}[], modalities, families, years, licenses, status }` |50| `GET /companies?q=&country=&kind=&sort=(models|name|updated|quality)&limit=&offset=` | `Page<EntitySummary & { model_count: number; paper_count: number }>` (+`facets`: countries, kinds) |51| `GET /papers?q=&category=&org=&since=&until=&sort=(published|updated)&limit=&offset=` | `Page<EntitySummary>` |52| `GET /providers` | `{ items: (EntitySummary & { model_count: number; price_count: number; min_input_per_mtok: Num; min_output_per_mtok: Num })[] }` |53| `GET /prices?model=&provider=&sort=(input|output|model|provider|observed)&limit=&offset=&current=1` | `Page<Price>` |54| `GET /prices/history?model=&provider=` | `{ items: Price[] }` (all rows incl. closed, oldest first) |55| `GET /prices/index?days=180` | `{ series: { day: string; median_input: Num; median_output: Num; min_input: Num; models: number }[]; movers: ChangeEvent[] }` computed from `prices` history |56| `GET /benchmarks` | `{ items: (EntitySummary & { result_count: number; model_count: number; top: { model: EntitySummary; score: number } | null })[] }` |57| `GET /benchmarks/{slug}/results?limit=&offset=&config=` | `Page<BenchmarkResult>` sorted by score (respecting `higher_is_better`), current rows only unless `history=1` |58| `GET /benchmarks/{slug}/history?model=` | `{ items: BenchmarkResult[] }` over time |59| `GET /hardware?kind=&manufacturer=&min_memory=&sort=` | `Page<EntitySummary>` |60| `GET /hardware/fit?memory_gb=&quant=(4bit|8bit|fp16)&context=8192&limit=` | `{ inputs, assumptions: string[], items: { model: EntitySummary; parameter_count: number; estimated_memory_gb: number; fits: boolean; headroom_gb: number; quantization: string; note: string }[] }` — **ESTIMATED**: bytes/param (4bit 0.5+overhead 1.15, 8bit 1.0, fp16 2.0) + KV cache estimate; label as estimated |61| `GET /explore/types` | `{ items: { entity_type, count, label }[] }` |62| `GET /explore/{type}?q=&org=&sort=&limit=&offset=` | `Page<EntitySummary>` generic listing for any type (datasets, frameworks, tools, repositories, regulation…) |63| `GET /changes?category=&type=&entity_type=&importance_min=&since=&until=&q=&limit=50&before=<observed_at cursor>` | `Page<ChangeEvent>` newest first; `type` = comma list of event types; excludes `DOCUMENT_CHANGED` unless `include_documents=1` |64| `GET /changes/daily?date=YYYY-MM-DD` | `{ date, counts: Record<category, number>, sections: { category, label, items: ChangeEvent[] }[], new_models: EntitySummary[] }` ("What changed in AI today", DB-generated only) |65| `GET /changes/categories?days=7` | `{ items: { category, event_type, count }[] }` |66| `GET /timeline?entity=&year=&category=&limit=` | `{ items: { month: string; events: ChangeEvent[] }[] }` grouped by month; global when no entity |67| `GET /compare?ids=slug,slug,…` (2–6) | `{ entity_type, dimensions: { key, label, unit?, kind: "number"|"text"|"list"|"bool"|"date" }[], items: { entity: EntitySummary; values: Record<key, unknown>; provenance: Provenance; prices?: Price[]; results?: BenchmarkResult[] }[] }` — dimensions per type: model (params, active params, context, max output, openness, license, modalities, release date, knowledge cutoff, best price in/out, benchmark scores shared by all), provider (models, min prices, features), hardware (memory, bandwidth, tdp, runtimes), framework (version, license, stars), company (country, founded, models, papers) |68| `GET /diff?a=YYYY-MM-DD&b=YYYY-MM-DD&scope=(all|models|org:<slug>|family:<name>)` | `{ a, b, new_entities: EntitySummary[], gone_entities: EntitySummary[], property_changes: ChangeEvent[], price_changes: ChangeEvent[], benchmark_changes: ChangeEvent[], counts }` |69| `GET /sources` | `{ items: { key, name, domain, tier, kind, category, organization: Org, enabled, documents, last_crawled_at, connectors: { name, label, health, last_success_at, interval_seconds }[] }[] }` (public transparency page) |70| `GET /methodology` | `{ metrics: metric_definitions[], confidence_levels, tiers, event_types, extractors }` |71| `GET /trending?days=7&limit=12` | `{ items: (EntitySummary & { views: number })[] }` from `page_views` |72| `POST /views` `{ path }` | `{ ok: true }` (beacon; 1 req/s/IP) |73| `GET /sitemap?type=&limit=5000&offset=0` | `{ items: { slug, entity_type, updated_at }[], total }` |74| `GET /api-keys/me` (header `x-api-key`) | `{ label, plan, rate_per_min, usage_count }` — developer keys (later); public routes work without a key |7576### `EntityDetail`7778```ts79EntitySummary & {80  attributes: Record<string, unknown>            // full81  provenance: Provenance82  aliases: string[]; identifiers: { scheme: string; value: string }[]83  relations: { predicate: string; direction: "out" | "in"; items: EntitySummary[]; total: number }[]   // grouped, ≤ 24 per group84  sources: SourceRef[]                           // documents describing this entity (deduped by URL)85  timeline: ChangeEvent[]                         // latest 3086  quality: {...}; counts: {...}87  // type-specific blocks (present only when relevant)88  prices?: Price[]                                // current rows across providers (model) or for this provider (provider)89  price_history?: Price[]                         // closed + current, oldest first (model)90  results?: BenchmarkResult[]                     // model: its results · benchmark: leaderboard top 10091  lineage?: { ancestors: EntitySummary[]; descendants: EntitySummary[]; quantizations: EntitySummary[] }   // model92  providers?: EntitySummary[]                     // model93  hardware_fit?: { hardware: EntitySummary; quantization: string; estimated_memory_gb: number; fits: boolean }[]   // model, ESTIMATED94  models?: Page<EntitySummary>                    // company / provider / hardware(runnable)95  papers?: EntitySummary[]                        // company / model96  repositories?: EntitySummary[]                  // company / model / framework97}98```99100## Admin routes (`x-aia-admin-token`)101102| route | returns / action |103|---|---|104| `GET /admin/overview` | `{ stats, queue: Record<kind, Record<status, n>>, heartbeats, connectors: { ok, degraded, failing, disabled }, review_pending, recent_errors: n, llm: { jobs_24h, tokens_24h, by_stage } , archive }` |105| `GET /admin/connectors` | full `connectors` rows + last run + docs/snapshots counts + source tier |106| `POST /admin/connectors/{name}/run` `{ force?: bool }` | enqueue an immediate run (Redis flag `aia:run-now:<name>` consumed by the scheduler tick) → `{ queued: true }` |107| `PATCH /admin/connectors/{name}` `{ enabled?, interval_seconds?, priority? }` | update |108| `GET /admin/runs?connector=&limit=` · `GET /admin/errors?connector=&limit=` | rows |109| `GET /admin/documents?connector=&status=&q=&limit=&offset=` · `GET /admin/documents/{id}` | documents (+ snapshots list); `GET /admin/snapshots/{id}` → metadata + `structured` + `diff` + first 20 kB of cleaned text (never raw HTML) |110| `GET /admin/jobs?status=&kind=` · `POST /admin/jobs/{id}/retry` · `POST /admin/jobs/requeue-dead` | queue |111| `GET /admin/llm-jobs?limit=` · `GET /admin/llm/health` | LLM factory accounting |112| `GET /admin/review?status=pending&kind=` · `POST /admin/review/{id}` `{ action: "approve"|"reject"|"edit", resolution?: object }` | review queue; approving a `merge_candidate` calls `merge_entities(source, target)` (aliases/identifiers/claims/relations/events moved, `merged_into` set) |113| `GET /admin/entities/duplicates?type=&limit=` | candidate pairs by normalized name similarity (`pg_trgm`) within a type |114| `POST /admin/entities/merge` `{ source_id, target_id }` · `POST /admin/entities/{id}/claims/{claim_id}/retract` | curation |115| `POST /admin/reprocess` `{ connector, url? }` | enqueue `reprocess_snapshot` |116| `POST /admin/llm/enqueue` `{ limit?, task? }` | queue `llm_extract` for pending snapshots |117| `GET /admin/infrastructure` | heartbeats, archive size, DB size (`pg_database_size`), table sizes, node hostname, uptime |118| `POST /admin/cache/flush` | `{ flushed: n }` |119| `POST /admin/stats/recompute` · `POST /admin/quality/recompute` | maintenance |120