spb/company-atlas

1# Company Atlas API — contract (v1)23Base: `/api/v1` · JSON (orjson) · UTC ISO-8601 timestamps (`Z`) · immutable ids (`co_…`, `sen_…`, `evt_…`) · slugs for public URLs.4Public GET endpoints need no auth. Owner endpoints (watchlists, alerts) take `X-CA-Owner-Token` (client-generated random string ≥ 24 chars, stored hashed).5Admin endpoints take `X-CA-Admin-Token`. Errors: `{ "detail": string }` with 4xx/5xx. Pagination: `?page=1&per_page=25` (max 200) →6`{ items, page, per_page, total, pages }`. Lists accept `sort` where documented; unknown params are ignored.78Interpretive language is careful by design (spec §167–168): events say *"no longer listed"*, never *"fired"*; anything inferred carries a9`confidence` (0–1) and a `confidence_label` (`VERIFIED | HIGH_CONFIDENCE | LIKELY | INFERRED | LOW_CONFIDENCE`).1011## Shapes1213```ts14type CompanyCard = {15  id: string; slug: string; display_name: string; legal_name: string | null; canonical_domain: string; website: string;16  description: string | null; industries: string[]; industry_primary: string | null; country: string | null; hq_city: string | null;17  hq_region: string | null; public_company: boolean; ticker: string | null; exchange: string | null; founded_year: number | null;18  employees_band: string | null; logo_url: string | null; status: string; onboarding_status: string; importance: number; tier: 1|2|3|4;19  metrics: Partial<Record<Metric, number>>;              // activity_score, hiring_momentum_30d, product_velocity, ai_adoption, corporate_change_index, open_jobs …20  counts: { sensors: number; observations: number; changes: number; events: number; jobs_open: number };21  last_event_at: string | null; last_observed_at: string | null;22  sparkline?: number[];                                  // 30 daily activity values when `?sparkline=1` (or on detail)23};24type Metric = 'activity_score'|'hiring_momentum_7d'|'hiring_momentum_30d'|'hiring_momentum_90d'|'open_jobs'|'ai_adoption'|'product_velocity'|25  'geo_expansion'|'developer_momentum'|'communication_activity'|'pricing_activity'|'leadership_activity'|'corporate_change_index'|'anomaly_score'|'historical_coverage';2627type Event = {28  id: string; company: { id: string; slug: string; display_name: string; canonical_domain: string; country: string|null; logo_url: string|null };29  event_type: string; event_subtype: string; importance: number; confidence: number; confidence_label: string;30  title: string; summary: string | null; old_value: string | null; new_value: string | null;31  payload: Record<string, unknown>; entities: Record<string, unknown>; tags: string[];32  detected_at: string; effective_at: string | null; published_at: string | null;33  source_url: string | null; surface: string | null; sensor_id: string | null; change_id: string | null; cluster_id: string | null;34  origin: 'deterministic'|'llm'|'hybrid'|'backfill'; model_name: string | null; prompt_version: string | null; status: 'active'|'retracted'|'duplicate'|'review';35  sources?: { source_url: string; surface: string | null; detected_at: string; kind: string; sensor_id: string | null }[];   // detail only36};3738type Sensor = {39  id: string; company_id: string; surface: string; connector_id: string; url: string; canonical_url: string; domain: string;40  status: string; tier: 'A'|'B'|'C'|'D'|'E'; quality_score: number; discovery_confidence: number; discovery_method: string | null;41  current_interval_s: number; next_run_at: string; last_run_at: string | null; last_success_at: string | null; last_change_at: string | null;42  last_status: number | null; last_failure_class: string | null; consecutive_failures: number;43  observation_count: number; snapshot_count: number; change_count: number; meaningful_change_count: number; event_count: number; created_at: string;44};4546type Snapshot = { id: string; sensor_id: string; version_no: number; fetched_at: string; title: string | null; language: string | null;47  text_length: number | null; block_count: number | null; extracted_summary: Record<string, number>; content_hash: string; previous_snapshot_id: string | null };4849type Change = { id: string; sensor_id: string; surface: string; company_id: string; detected_at: string; significance: number; kind: string;50  blocks_added: number; blocks_removed: number; blocks_modified: number; text_delta_ratio: number; similarity: number | null;51  snapshot_before: string | null; snapshot_after: string; diff?: DiffPayload; structured_delta?: Record<string, unknown> };5253type DiffPayload = { added: BlockDelta[]; removed: BlockDelta[]; modified: BlockDelta[]; moved: string[]; counts: Record<string, number>;54  text_delta_ratio: number; similarity: number; reasons: string[] };55type BlockDelta = { key: string; kind: string; path: string; before: string | null; after: string | null; weight: number; similarity: number | null };5657type Job = { id: string; title: string; department: string | null; location_text: string | null; city: string | null; country: string | null;58  remote: boolean | null; employment_type: string | null; seniority: string | null; url: string | null; posted_at: string | null;59  first_seen_at: string; last_seen_at: string; removed_at: string | null; status: 'open'|'no_longer_listed'; is_ai: boolean };6061type Person = { id: string; name: string; title: string | null; role_category: string | null; is_executive: boolean; first_seen_at: string; last_seen_at: string; removed_at: string | null; status: string; source_url: string | null };62type Product = { id: string; name: string; category: string | null; description: string | null; url: string | null; first_seen_at: string; last_seen_at: string; removed_at: string | null; status: string };63type Plan = { id: string; plan_name: string; price: number | null; price_text: string | null; currency: string | null; billing_period: string | null; unit: string | null; features: string[]; contact_sales: boolean; version_no: number; valid_from: string; valid_to: string | null; status: string; source_url: string | null };64type Location = { id: string; kind: string; name: string | null; city: string | null; region: string | null; country: string | null; lat: number | null; lon: number | null; first_seen_at: string; last_seen_at: string; removed_at: string | null; status: string; source_url: string | null };65type NewsItem = { id: string; title: string; url: string; summary: string | null; category: string | null; published_at: string | null; first_seen_at: string; language: string | null };66type MetricPoint = { day: string; value: number; confidence: number };67type Signal = { id: string; company_id: string | null; scope: string; scope_key: string | null; kind: string; strength: number; confidence: number; title: string; explanation: string | null; evidence: Record<string, unknown>; window_days: number; detected_at: string; status: string };68```6970## Endpoints7172### Platform73| Method | Path | Notes |74|---|---|---|75| GET | `/health`, `/ready` | service health (also at `/api/v1/health`) |76| GET | `/stats` | `{ companies, companies_active, sensors, sensors_active, observations, snapshots, changes, meaningful_changes, events, jobs_open, countries, industries, observations_today, changes_today, events_today, dataset_started_at, dataset_age_days, oldest_history_days, last_observation_at, archive: {objects, bytes} }` (cached 60 s) |77| GET | `/stats/history?days=90` | `{ items: GlobalDaily[] }` — `{ day, companies_active, sensors_active, observations, changes, meaningful_changes, events, events_by_type, jobs_open, jobs_new, jobs_removed, activity_index }` |78| GET | `/system` | public aggregate health only: `{ sensors_online, sensors_failing, observations_today, events_today, countries_covered, queue_lag_s, scheduler_last_tick_at, fetch_per_min, success_rate_24h }` |79| GET | `/pulse` | homepage aggregate: `{ stats, live: Event[12], movers: CompanyCard[10], hiring: CompanyCard[8], launches: Event[8], pricing: Event[8], ai: CompanyCard[8], industries: IndustryRow[12], countries: CountryRow[12], trending: TrendRow[10], activity_index: {value, delta_7d, series: MetricPoint[30]}, map: MapBucket[] }` (cached 60 s) |80| GET | `/live?limit=50&since=<iso>&event_type=&min_importance=` | latest active events (no cache); `since` returns only newer than that time |81| GET | `/live/stream` | **SSE**: `event: event` with an `Event` JSON per message (poll-based, ~5 s), `event: heartbeat` every 20 s, `?since=` supported |8283### Companies84| Method | Path | Notes |85|---|---|---|86| GET | `/companies` | filters: `q, country, industry, tier, public (bool), status, has_events (bool), sort=activity\|events\|hiring\|name\|importance\|recent, sparkline=1` → page of `CompanyCard` |87| GET | `/companies/{slug_or_id}` | `CompanyCard & { aliases: string[], domains: {domain, kind}[], relationships: {kind, company: {slug, display_name} \| null, to_name, valid_from, valid_to, confidence}[], metrics_detail: {metric, value, confidence, computed_at, inputs}[], sensors_by_surface: Record<string, number>, coverage: {historical_coverage, first_observed_at, days_observed, sensor_uptime}, signals: Signal[], sparklines: {activity_30d: number[], hiring_90d: number[]} }` |88| GET | `/companies/{slug}/events` | filters `event_type, event_subtype, since, until, min_importance, surface`; `sort=recent\|importance` → page of `Event` |89| GET | `/companies/{slug}/timeline?filter=all\|products\|jobs\|pricing\|leadership\|locations\|legal\|news\|developer&limit=200` | `{ items: (Event & {day: string})[], days: {day, count}[] }` grouped for the timeline UI |90| GET | `/companies/{slug}/metrics?metric=activity_score&days=90` | `{ current: {metric, value, confidence, computed_at, formula_version, inputs}[], series: Record<Metric, MetricPoint[]> }` |91| GET | `/companies/{slug}/jobs?status=open\|removed\|all&q=&country=&ai=1` | page of `Job` + `{ summary: {open, new_7d, removed_7d, ai_open, by_country: {country,n}[], by_department: {department,n}[], remote_ratio} }` as `meta` |92| GET | `/companies/{slug}/people` | `{ listed: Person[], no_longer_listed: Person[] }` |93| GET | `/companies/{slug}/products` | `{ listed: Product[], removed: Product[] }` |94| GET | `/companies/{slug}/pricing` | `{ current: Plan[], history: Plan[] }` |95| GET | `/companies/{slug}/locations` | `{ items: Location[], countries: string[] }` |96| GET | `/companies/{slug}/news?limit=50` | `{ items: NewsItem[] }` |97| GET | `/companies/{slug}/sensors` | `{ items: Sensor[] }` |98| GET | `/companies/{slug}/history` | `{ sensors: (Sensor & { versions: Snapshot[] })[] }` — historical page viewer index (max 20 versions per sensor) |99| GET | `/companies/{slug}/similar?limit=8` | `{ items: CompanyCard[] }` (same industry/country, closest importance) |100| GET | `/companies/compare?companies=stripe,adyen,block` | `{ companies: CompanyCard[], metrics: Record<Metric, Record<slug, number>>, series: Record<slug, MetricPoint[]>, events_30d: Record<slug, Record<event_type, number>>, jobs: Record<slug, {open, ai_open, new_30d}>, locations: Record<slug, number> }` (2–6 companies) |101102### Sensors, snapshots, changes (provenance)103| Method | Path | Notes |104|---|---|---|105| GET | `/sensors/{id}` | `Sensor & { company: CompanyRef, latest_snapshot: Snapshot \| null }` |106| GET | `/sensors/{id}/snapshots?limit=50` | `{ items: Snapshot[] }` |107| GET | `/sensors/{id}/changes?limit=50` | `{ items: Change[] }` |108| GET | `/snapshots/{id}` | `Snapshot & { text: string (≤ 200 kB), blocks: Block[], extracted: object }` |109| GET | `/snapshots/{id}/diff/{other_id}` | `{ before: Snapshot, after: Snapshot, diff: DiffPayload }` (computed on demand) |110| GET | `/changes/{id}` | `Change` with `diff` and `structured_delta`, plus `events: Event[]` |111| GET | `/events/{id}` | `Event` detail with `sources`, `change` (Change summary), `company` |112113### Events114| Method | Path | Notes |115|---|---|---|116| GET | `/events` | filters `event_type, event_subtype, country, industry, since, until, min_importance, min_confidence, q, surface, origin, company`; `sort=recent\|importance` → page of `Event` |117| GET | `/events/types` | `{ types: {event_type, subtypes: {event_subtype, count_30d}[], count_30d}[] }` |118| GET | `/events/summary?days=7&group=type\|industry\|country` | `{ items: {key, count, delta_pct}[] }` |119120### Rankings, industries, countries, signals, trends121| Method | Path | Notes |122|---|---|---|123| GET | `/rankings?kind=most_active\|hiring_growth\|hiring_decline\|product_velocity\|ai_active\|geo_expansion\|developer_momentum\|pricing_changes\|unusual_activity&window=24h\|7d\|30d\|90d\|1y&country=&industry=&limit=50` | `{ kind, window, items: (CompanyCard & { rank: number; value: number; delta: number \| null })[] }` |124| GET | `/industries` | `{ items: IndustryRow[] }` — `{ slug, name, parent_slug, companies, events_7d, events_30d, hiring_momentum_30d, activity_score, ai_adoption, top_event_types: string[] }` |125| GET | `/industries/{slug}` | `IndustryRow & { description, companies: CompanyCard[24] (most active), events: Event[20], hiring: {open, new_30d, removed_30d, momentum_30d}, series: MetricPoint[90] (activity), countries: {country, companies}[], trending: TrendRow[] }` |126| GET | `/countries` | `{ items: CountryRow[] }` — `{ code, name, region, companies, events_7d, events_30d, hiring_momentum_30d, activity_score, industry_mix: {industry, companies}[], lat, lon }` |127| GET | `/countries/{code}` | `CountryRow & { companies: CompanyCard[24], events: Event[20], movers: CompanyCard[10], new_entrants: CompanyCard[10], series: MetricPoint[90], industries: IndustryRow[] }` |128| GET | `/signals?kind=&scope=company\|industry\|country\|global&limit=50` | `{ items: Signal[] }` |129| GET | `/trends?window=7d\|30d\|90d&limit=30` | `{ items: TrendRow[] }` — `{ term, mentions, companies, momentum, series: number[] }` |130| GET | `/map?metric=events_30d\|companies\|hiring` | `{ buckets: MapBucket[] }` — `{ lat, lon, country, city: string \| null, companies: number, events_30d: number, jobs_open: number, top: {slug, display_name}[] }` (clustered by city/country, ≤ 600 buckets) |131| GET | `/index` | Global Corporate Activity Index: `{ value, baseline: 100, delta_7d, delta_30d, series: MetricPoint[365], by_type: Record<string, number>, by_country: {key, value}[], by_industry: {key, value}[], formula_version }` |132133### Search134| Method | Path | Notes |135|---|---|---|136| GET | `/search?q=&types=companies,events,industries,countries,people,products&limit=10` | `{ query, companies: CompanyCard[], events: Event[], industries: IndustryRow[], countries: CountryRow[], people: (Person & {company: CompanyRef})[], products: (Product & {company: CompanyRef})[], took_ms }` |137| GET | `/search/suggest?q=` | `{ items: { kind: 'company'\|'industry'\|'country'\|'event_type', label, sublabel, href }[] }` (≤ 10, < 50 ms) |138| GET | `/ask?q=` | natural-language routing (deterministic parser → structured query; LLM optional): `{ interpretation: {filters…}, answer: string, companies: CompanyCard[], events: Event[], sources: string[] }` |139140### Watchlists & alerts (owner token)141| Method | Path | Notes |142|---|---|---|143| GET/POST | `/watchlist` | GET → `{ items: CompanyCard[], events: Event[30] (for the watched companies) }`; POST `{ company: slug }` adds; DELETE `/watchlist/{slug}` removes |144| GET/POST | `/alerts` | POST `{ name, company?: slug, condition: {event_types?: string[], min_importance?: number, metrics?: {activity_score?: {gt: number}}}, channel: 'web'\|'webhook', target?: url }`; DELETE `/alerts/{id}` |145| GET | `/alerts/deliveries?limit=50` | recent deliveries for this owner |146147### Exports & docs148| Method | Path | Notes |149|---|---|---|150| GET | `/export/events.{json,ndjson,csv}?since=&event_type=&country=&limit=10000` | streamed export |151| GET | `/export/companies.{json,ndjson,csv}?country=&industry=` | streamed export |152| GET | `/export/jobs.ndjson?company=&since=` | |153| GET | `/sitemap?kind=companies\|industries\|countries&page=` | `{ items: {slug, updated_at}[], pages }` — only companies with `indexed = true` |154| GET | `/methodology` | `{ metrics: {metric, formula_version, description, inputs: string[]}[], significance_bands, event_types: string[], confidence_labels }` |155156### Admin (`X-CA-Admin-Token`)157| Method | Path | Notes |158|---|---|---|159| GET | `/admin/overview` | `{ companies_by_status, sensors_by_status, sensors_by_tier, queue: {pending, running, dead, oldest_pending_s}, llm: {pending, done_today, failed_today, budget_left}, failures_24h_by_class, fetch_rate_1h, change_rate_1h, meaningful_rate_1h, storage: {objects, bytes}, workers: {name, last_seen_at, inflight}[], cost_today: {fetch, browser, llm} }` |160| GET | `/admin/connectors` | `{ items: {id, name, version, category, enabled, sensors_active, sensors_failing, success_rate_24h, avg_latency_ms, change_rate_24h, errors_24h, last_run_at}[] }` |161| GET | `/admin/sensors?status=&domain=&connector=&company=&filter=healthy\|failing\|stale\|blocked\|redirected\|low_quality\|high_activity&page=` | page of `Sensor & {company: CompanyRef}` |162| POST | `/admin/sensors/{id}/{action}` | action ∈ `pause, resume, retry, rediscover, retire, run_now`; body `{ interval_s?, connector_id? }` for `set_interval`, `set_connector` |163| GET | `/admin/companies?onboarding_status=&page=` · POST `/admin/companies` `{ website, display_name?, country?, industries? }` (create + queue discovery) · POST `/admin/companies/{slug}/rediscover` |164| GET | `/admin/failures?class=&since=&page=` · GET `/admin/queue?kind=&status=` · POST `/admin/queue/requeue-dead` |165| GET | `/admin/llm?status=&page=` · GET `/admin/reviews?kind=&status=open` · POST `/admin/reviews/{id}` `{ resolution: 'accepted'\|'rejected', note? }` |166| POST | `/admin/events/{id}/retract` `{ reason }` · POST `/admin/events/{id}/restore` |167| GET | `/admin/quality` | `{ coverage: {companies_active_pct, sensors_active_pct}, freshness: {sensors_checked_24h_pct, stale}, duplicate_rate, event_confidence_avg, unknown_surfaces, failed_sensors, calibration: {correct, duplicate, noise, misclassified} }` |168| GET | `/admin/costs?days=30` | `{ items: {day, dimension, key, units, cost_estimate}[], per_1000_companies, per_million_observations, per_meaningful_event }` |169| POST | `/admin/cache/clear` | |170171## Conventions for implementers172- Every list endpoint is bounded (`per_page ≤ 200`, `limit ≤ 500`), uses indexed predicates, and returns `Cache-Control: public, max-age=60` for public aggregates (`/pulse`, `/stats`, `/rankings`, `/industries`, `/countries`) and `no-store` for `/live*`, owner and admin routes.173- Company lookups accept slug **or** id. Unknown → 404 `{detail: "company not found"}`.174- Text search: Postgres FTS (`companies.search`, `events.search`) + trigram fallback for short/partial queries; never LIKE on unindexed columns.175- Numbers: metrics are 0–100 floats rounded to 1 decimal except `hiring_momentum_*` (percentage, may be negative) and `open_jobs` (int).176- Never fabricate: when a metric has no inputs, omit it (or `null`) rather than returning 0 as if measured.177