@dci/api — DataCenterIndex REST API
Fastify 5 service exposing the public read API (/api/v1/*) consumed by apps/web, and a token-protected admin
API (/api/admin/*) for connector operations, document forensics, reconciliation and curation. Response types
are the contract in packages/core/src/api-types.ts.
bash
pnpm --filter @dci/api dev # tsx watch, port API_PORT (8311)
pnpm --filter @dci/api start
pnpm --filter @dci/api typecheck
pnpm --filter @dci/api test # vitest: query interpretation + integration against the local Postgres (.env)Docs: GET /api/v1/docs (Swagger UI), GET /api/v1/openapi.json.
Conventions
- Envelope on every
/api/v1response:{ data, meta, sources }.metacarriestotal/page/perPage,generatedAtand, where a figure is derived,methodology.sources(detail endpoints) lists theSourceRefs behind the provenance / events shown. - Errors:
{ error, statusCode, details? }— 400 with zod issues for bad query/body, 404 for unknown entities and routes, 401 for admin, 429 for rate limits. - Caching: in-process LRU + Redis (
dci:api:cache:*), key = route + normalized query, TTL per route family (dashboard 60 s, lists 120 s, details 300 s, map 300 s, events/news 30 s, search 60 s, sitemap 600 s). Headers:Cache-Control: public, max-age, s-maxage=<ttl>, stale-while-revalidate, weakETag(independent ofgeneratedAt;If-None-Match→ 304),X-Cache: HIT|MISS. Disable withDCI_API_CACHE=0. Admin writes bust the affected prefixes (/datacenters,/dashboard,/map,/events). - Ids or slugs are accepted everywhere a
:slugappears; a facility whosemerged_intois set redirects to its survivor in the detail endpoint and is hidden from lists. - MW methodology:
mw = COALESCE(it_capacity_mw, total_power_mw, planned_power_mw).knownMwsums operational/expansion/partially_operational facilities,constructionMwunder_construction,plannedMwrumored/proposed/announced/permitting/approved/delayed (planned figure first). Per-facility estimates keep theirmwIsEstimateflag; nothing is invented. - Rate limit: 240 req/min per IP on public routes (
X-RateLimit-*headers), 60/min on admin. Client IP is taken fromX-Forwarded-For/X-Real-IPonly when the peer is a trusted proxy (DCI_TRUSTED_PROXIES, default 127/8, 10/8, 172.16/12, ::1). - Logging: pino JSON on stdout. Every request is also batched into ClickHouse
api_requests(never fatal;DCI_API_CH_LOG=0disables)./api/metricsexposes Prometheus text (requests by route/status, latency histogram, cache hits/misses, memory cache size, pool max).
Public routes (/api/v1)
| Route | Returns | Notes |
|---|---|---|
GET /datacenters |
FacilitySummary[] |
FacilityFilters from api-types: `q, country (csv iso2), region, metro, operator, status (csv), type (csv), min_mw, max_mw, opened_from/to, planned_from/to (pipeline statuses by expected year), cloud (provider slug: tenant role cloud or cloud region in same metro), hyperscale, colocation, ai, renewable, has_mw, has_geo, confidence (csv), sort (name |
GET /datacenters/:idOrSlug |
FacilityDetail |
aliases, owner, campus, tenants (carriers / cloud), IXPs, cloud regions in the metro (12), nearby ≤ 25 km (8, haversine), linked projects (facility_id or same operator + metro), current provenance, last 50 events, sourceHistory, externalIds. |
GET /operators |
OperatorSummary[] |
`q, kind, country, sort (facilities |
GET /operators/:slug |
OperatorDetail |
facilities paginated with ?fPage (50). |
GET /countries |
CountrySummary[] |
countries with ≥ 1 facility or cloud region; ?all=1 for every row; ?region=. |
GET /countries/:slugOrIso2 |
CountryDetail |
top operators, metros, cloud regions, recent projects/events, growth by opened year, status/type breakdowns, announced investment, rankings positions, facilities (?fPage). |
GET /metros |
MetroSummary[] |
?country=&q=. |
GET /metros/:slug |
MetroDetail |
operators, cloud providers/regions, IXPs, facilities (?fPage), projects, events, constraints (news mentions with power/land/water/grid/moratorium keywords; [] when none), growth, rankings. |
GET /cloud-regions |
CloudRegionSummary[] |
?provider=&country=&status=. |
GET /cloud-regions/:slug |
summary + metro, sibling regions, facilities in metro | |
GET /ixps, GET /ixps/:slug |
IxpSummary[] / summary + facilities |
?country=&q=. |
GET /projects |
ProjectSummary[] |
`status (csv), country, operator, metro, min_mw, ai, q, sort (updated |
GET /projects/pipeline |
aggregates | by status, by expected year, top 15 countries, totals. |
GET /projects/:slug |
ProjectDetail |
timeline sorted by partialDateSortKey, statusHistory from project_status_changed events. |
GET /events |
EventDTO[] |
type (csv), country, operator (slug), metro, project, entity_type, entity_id, min_significance, since, page, per_page ≤ 100; entity {slug,name} resolved per entity type; rejected events hidden. |
GET /events/:id |
EventDTO |
|
GET /news |
news items | country, operator, since, q, page. |
GET /map |
MapResponse |
zoom, bbox=w,s,e,n, status, type, operator, country, min_mw, max_mw, ai, hyperscale, cloud_regions=1. zoom < 5 → country clusters (centroid from countries or facility mean), 5–8 → grid clusters (gridCell computed in SQL), ≥ 9 → points capped at 5 000 (else zoom-8 clusters, mode: "clusters"). Only facilities with coordinates; p = geo precision. |
GET /search?q= |
SearchResponse |
interprets country / operator (trigram) / status / NN MW / type words; hits across facilities (tsvector + trigram), operators, countries, metros, cities (href=/datacenters?q=<city>), cloud regions, projects, IXPs (≤ 30); facilities block (20 + total) when the query has text or filters. |
GET /rankings, GET /rankings/:key?limit= |
ranking list / Ranking |
current rows from the rankings table (computed by the worker). |
GET /dashboard |
Dashboard |
live counts + MW by status, events 24h/7d, sources/documents, last crawl, capacity over time (daily_metrics → fallback opened year), pipeline by year, AI expansion, latest events, new projects, recently verified; top lists from rankings with live fallback. |
GET /stats/timeseries?metric=&dim=global&days=365 |
{day, value}[] |
without metric: available metrics. |
GET /sources, GET /sources/:id |
SourceRef + counts |
documents, provenance rows, facilities, breakdowns. |
GET /sitemap/:kind?page= |
{slug, updatedAt}[] |
kinds facilities, operators, countries, metros, projects, cloud-regions; 5 000 per page. |
System: GET /api/health (no I/O), GET /api/ready (Postgres ping, Redis reported), GET /api/metrics.
Admin routes (/api/admin, header x-dci-admin-token: $DCI_ADMIN_TOKEN)
Constant-time token compare; 401 otherwise, 503 when the token is not configured. All responses no-store.
| Route | Purpose |
|---|---|
GET /connectors |
ConnectorHealthDTO[] from connectors + last run + document aggregates + ClickHouse crawl_log (avg ms, cost by fetcher, 7 days; run stats fallback). |
GET /connectors/:id |
config, last 30 runs, documents by page type, error samples. |
POST /connectors/:id/run `{task: crawl |
discover |
POST /connectors/:id/pause / resume |
toggles connectors.paused. |
PATCH /connectors/:id/schedule {group: interval} |
validates intervals, merges into connectors.schedule and rewrites the schedule section of config/connectors/<id>.yaml (backup <id>.yaml.bak-<ts>). |
GET /runs?connector=&limit=, GET /runs/:id |
runs (with log). |
| `GET /documents?connector=&pageType=&status=error | changed |
GET /documents/:id |
row + versions + provenance referencing it + resolved entity_refs. |
GET /documents/:id/raw?version= |
decompressed body from MinIO (zstd, gzip fallback), original content-type, capped 5 MB (X-Truncated). |
GET /documents/:id/versions/:vid/diff |
diff summary + detected changes (+ previous version). |
POST /documents/:id/reprocess / refetch / quarantine {quarantined} |
enqueue (urls + force in job data) / toggle. |
GET /matches?status=pending |
entity_matches with candidate + matched facility summaries. |
POST /matches/:id/approve |
points the candidate key at the matched facility; if the candidate had created a separate facility it is merged (aliases, provenance, tenants, IXPs, keys, projects, events, document refs moved; merged_into set). |
POST /matches/:id/reject |
keeps the separate facility. |
PATCH /facilities/:id |
curation of name, status, facility_type, lat, lng, geo_precision, it_capacity_mw, total_power_mw, planned_power_mw, mw_is_estimate, opened_on, operator_id, is_ai, is_hyperscale, description (+note); writes src_manual provenance rows (source auto-created, kind registry) and a status_changed / capacity_changed / facility_updated event. Hand-entered coordinates default to geo_precision = approximate. |
POST /facilities/:id/merge {into} |
merge duplicate. |
PATCH /events/:id {reviewStatus} |
review. |
POST /devtool/fetch {url, level?, maxLevel?, renderJs?, country?, waitForSelector?, userAgent?} |
fetch through the escalation stack (SSRF policy enforced by the fetchers): status, fetcher, credits, html (≤ 1.5 MB), text (≤ 200 kB), markdown, title, JSON-LD, embedded JSON, classification, geo, address, 50 links. |
| `POST /devtool/preview {url | html, extractor, defaults?, connectorId?}` |
POST /devtool/validate-config {yaml} |
connectorConfigSchema result + warnings. |
POST /devtool/save-config {id, yaml, version?} |
writes the YAML (backup), bumps parserVersion, upserts connectors. |
POST /devtool/run-sample {yaml, urls[] ≤ 10} |
per-URL fetch + extract + normalize + validate with a test context; field coverage matrix. |
GET /devtool/configs, GET /devtool/configs/:id |
YAML inventory / raw text. |
GET /ops |
worker heartbeat (dci:worker:status), queue counts, premium budgets (Redis keys or ClickHouse credits today), DB size + table estimates, ClickHouse/MinIO/Redis health, open system_alerts, last 20 runs, API cache stats. |
POST /maintenance/:task (`rankings |
metrics |
POST /alerts/:id/resolve, POST /cache/invalidate {prefix} |
housekeeping. |
Environment
| Variable | Default | Role |
|---|---|---|
API_PORT / API_HOST |
8311 / 0.0.0.0 |
listen address |
DATABASE_URL |
postgres://dci:dci@127.0.0.1:5432/dci |
Postgres (via @dci/db; PG_POOL_MAX 16) |
REDIS_URL |
redis://127.0.0.1:6379/0 |
cache + BullMQ producers + worker status |
CLICKHOUSE_URL / CLICKHOUSE_DB |
http://127.0.0.1:8123 / dci |
request log, connector cost stats (optional) |
S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY, S3_SECRET_KEY, S3_REGION |
MinIO dev values | raw document bodies |
DCI_ADMIN_TOKEN |
— | admin API token (admin disabled when missing) |
DCI_CONFIG_DIR |
<repo>/config/connectors |
connector YAML directory |
DCI_WORKER_CONNECTORS_DIR |
<repo>/apps/worker/src/connectors |
optional: registers worker parsers for the dev tool |
DCI_API_CACHE, DCI_API_CACHE_ENTRIES |
1, 800 |
response cache toggle / LRU size |
DCI_API_RATE_LIMIT, DCI_ADMIN_RATE_LIMIT |
240, 60 |
per-minute per IP |
DCI_TRUSTED_PROXIES |
127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,::1/128 |
proxies allowed to set X-Forwarded-For |
DCI_API_CH_LOG |
1 |
ClickHouse request logging |
DCI_LOG_LEVEL |
info |
pino level |
NEXT_PUBLIC_SITE_URL |
https://www.datacenterindex.io |
OpenAPI server URL |
In development the repo-root .env is loaded automatically (process.loadEnvFile, never overriding set variables).
Layout
text
src/main.ts entry (env, graceful shutdown)
src/app.ts Fastify app: plugins, error handling, hooks, route registration
src/env.ts typed env
src/cache.ts / redis.ts LRU + Redis response cache; ioredis clients
src/queues.ts BullMQ producers (crawl / maintenance)
src/storage.ts MinIO reader (zstd / gzip)
src/metrics.ts Prometheus registry src/telemetry.ts ClickHouse api_requests batch
src/lib/ http (envelope, ETag, errors), params (zod), route (publicGet), sql (fragments), rows (coercion), dto (mappers), resolve (slug/id, entity refs), source-history
src/repositories/ facilities, operators, countries, metros, cloud-regions, ixps, projects, events, map, search, rankings, dashboard, misc (stats/sources/news/sitemap), admin/{connectors,merge}
src/search/interpret.ts query interpretation (pure, tested)
src/routes/ public/{facilities,graph,activity,discovery}, admin/{index,connectors,documents,matches,curation,devtool,ops}, system