spb/countryatlas

1# CountryAtlas — Architecture (contract for all contributors)23> **www.countryatlas.co — Understand the world, one country at a time.**4> The country is the primary unit of navigation. Every visible number is traceable to its source.56This document is the binding contract between the data platform (Python), the API (FastAPI) and the7web app (Next.js). Change it first, then the code.89## 1. Deployment topology (MacLustr cluster)1011```12Internet ─ DNS A www.countryatlas.co → 51.161.112.61 (BHS64, OVH Beauharnois)13        └─ Caddy (TLS auto) ─ WireGuard wg1 ─→ M2M32 (Mac Studio M2, 12c/32GB)  10.67.0.x14                                                 ├─ countryatlas-web        Next.js 16   :8290  (public upstream)15                                                 ├─ countryatlas-api        FastAPI      :8291  (loopback only; reached via Next rewrites /api/v1/*)16                                                 └─ countryatlas-scheduler  Python loop  (daily refresh, writes snapshots)17                                                 ~/countryatlas-data/  (raw/, staging/, build/, atlas.duckdb, exports/, logs/)18```1920* Orchestrated by `mld` from the gateway M1M32 (manifest `deploy/countryatlas.mld.json`, PM2 processes).21* No database port is ever public. The API listens on 127.0.0.1:8291 only.22* The web app listens on 0.0.0.0:8290 (reached through the WireGuard tunnel only; the Mac has no public IP).23* Apex `countryatlas.co` → 301 to `www` (Caddy redirect on BHS64; requires an A record for the apex).2425## 2. Storage: DuckDB snapshots (no Postgres/ClickHouse/Redis for the MVP)2627Volumes: ~220 countries × ~250 indicators × ≤65 years ≈ 3–4 M observations. DuckDB handles this in28milliseconds, exports Parquet/CSV natively and needs zero operations. The design keeps the door open to29Postgres/ClickHouse later (same logical schema).3031**Snapshot-swap rule (critical):**32331. The pipeline builds a brand-new file `~/countryatlas-data/build/atlas-<run_id>.duckdb`.342. Derived tables are computed inside it (latest, rankings, changes, events, similarity, insights, coverage).353. Integrity checks pass → `os.replace()` it onto `~/countryatlas-data/atlas.duckdb` (atomic rename) and copy36   to `snapshots/atlas-<run_id>.duckdb` (keep last 7).374. The API opens the DB **read-only**; on every request it compares the file's `st_ino`/`st_mtime` with the open38   one and reopens when they differ. Readers never block the writer and vice-versa.3940Raw source payloads are kept forever (gzip JSON/CSV) under `raw/<connector>/<dataset>/<YYYY-MM-DD>/…`.41Nothing is ever fetched without also being stored raw.4243### 2.1 Logical schema (DuckDB)4445```sql46countries(id TEXT PK /* = iso3 */, iso2, iso3, iso_numeric, slug, short_name, official_name, capital,47          continent, region_wb /* WB region id */, region_wb_name, subregion, income_group /* HIC/UMC/LMC/LIC */,48          currency_code, currency_name, area_km2 DOUBLE, latitude, longitude, flag_emoji,49          un_member BOOLEAN, status TEXT /* country|territory|historical */, independent BOOLEAN,50          landlocked BOOLEAN, borders TEXT[] , languages TEXT[], demonym, kind TEXT /* country|aggregate */)51groups(id TEXT PK /* world|oecd|g7|g20|eu|brics|nac|ecs|hic|... */, slug, name, kind /* world|region|income|org|custom */,52       description, wb_code /* WLD, OED, EUU, HIC ... when the source has an aggregate */)53group_members(group_id, country_id)54sources(id TEXT PK /* worldbank|imf|oecd|eurostat|who|fred|owid|bis|ilo */, name, organization, url, licence,55        attribution, api_base, last_success_at, notes)56indicators(id TEXT PK /* = slug */, slug, name, short_name, description, topic, subtopic, unit, unit_short,57           frequency /* A|Q|M */, precision INT, aggregation /* sum|mean|weighted_mean|none */,58           higher_is_better BOOLEAN NULL, ranking_eligible BOOLEAN, per_capita_of TEXT NULL,59           format TEXT /* number|percent|currency|index|years|per_1000|... */, scale TEXT /* raw|thousands|millions|billions */,60           source_priority TEXT[] , methodology TEXT, tags TEXT[], featured BOOLEAN)61indicator_sources(indicator_id, source_id, dataset, series_code, params JSON, priority INT, transform TEXT NULL, notes)62observations(country_id, indicator_id, period DATE /* first day of period */, year INT, frequency,63             value DOUBLE, unit, source_id, source_dataset, source_series_code,64             is_estimate BOOL, is_forecast BOOL, revision INT, retrieved_at TIMESTAMP, source_updated_at TIMESTAMP,65             status TEXT /* verified|imported|warning|stale|quarantined */, metadata JSON)66             -- PK (country_id, indicator_id, period, frequency). Exactly ONE source per WHOLE SERIES (country, indicator,67             -- frequency): the highest-priority source having non-forecast data for that country, unless a lower-priority68             -- source is > 3 years fresher (metadata.merge_reason = "priority" | "fresher"). Sources are never spliced69             -- inside a series. Every other source's complete series is kept in observations_alt.70observations_alt(same columns)      -- complete alternative series (other sources), for provenance / alternative views71observation_revisions(country_id, indicator_id, period, frequency, old_value, new_value, old_source_id, new_source_id,72             changed_at TIMESTAMP, run_id)   -- never silently overwrite: carry forward from the previous snapshot73latest(country_id, indicator_id, period, year, value, prev_period, prev_value, change_abs, change_pct,74       rank_world INT, n_world INT, rank_region INT, n_region INT, rank_income INT, n_income INT,75       source_id, is_forecast, is_estimate, status)       -- latest NON-forecast observation per country×indicator76rankings(indicator_id, year, country_id, value, rank INT, n INT, pct_rank DOUBLE)   -- countries only (kind='country'), non-forecast77changes(id, country_id, indicator_id, kind /* yoy_drop|yoy_jump|record_high|record_low|n_year_high|n_year_low|sign_flip|accelerating|decelerating */,78        period, year, value, ref_value, delta, delta_pct, window_years INT, severity DOUBLE /* 0-1 */, headline TEXT, detail JSON, detected_at)79        -- "What changed" = recent (latest period) ; events = whole history80events(id, country_id, indicator_id, kind, period, year, value, ref_value, delta, delta_pct, severity, headline, detail JSON)81similarity(country_id, mode /* overall|economic|demographic|energy|social */, peer_id, score DOUBLE /* 0-100 */, rank INT,82           contributions JSON /* {indicator: {z_a, z_b, weight, contribution}} */)83insights(id, country_id, template_id, text, values JSON, indicators TEXT[], computed_at)84country_dna(country_id, dims JSON /* {income:0-100, demographics:..., urbanization, trade, energy, emissions, innovation, education, public_spending} */, year_ref)85coverage(country_id, n_indicators INT, n_observations INT, latest_year INT, coverage_pct DOUBLE, updated_at)86import_runs(run_id, connector, dataset, started_at, finished_at, status /* ok|failed|partial */, rows_raw, rows_norm, rows_valid,87            warnings INT, errors INT, message, raw_path)88validation_issues(run_id, connector, indicator_id, country_id, period, severity /* info|warning|error */, code, message)89meta(key, value)      -- build_run_id, built_at, schema_version, indicator_count, observation_count, ...90```9192## 3. Registries (YAML, versioned in git, loaded by both pipeline and API)9394* `registry/countries.yaml` — canonical registry generated by `scripts/build_country_registry.py` from the World Bank95  country list (region, income group, capital, lat/long) merged with the mledoze/countries dataset (ISO numeric, official96  name, subregion, currency, area, UN membership, flag, borders, languages). Hand-edited fields survive regeneration97  (`overrides:` block). Aggregates (WLD, OED, EUU, HIC …) are **not** countries; they live in `groups.yaml`.98* `registry/groups.yaml` — World, WB regions (7), income groups (4), organisations (OECD, EU, G7, G20, BRICS, ASEAN,99  African Union, Eurozone, Nordic, NAFTA/USMCA, Commonwealth…) with explicit member lists (ISO3).100* `registry/indicators.yaml` — the canonical indicator registry (see §4). Every indicator maps to one or more101  external series (connector + code + params + priority).102* `registry/topics.yaml` — topics (economy, government, population, …) with order, blurb, icon, and the ordered list of103  indicator slugs to show on the country topic page, plus `headline` indicators for the country overview.104105**Rule:** the code never identifies a country by its name. Always `iso3` (id) or `slug`.106URLs use slugs (`/countries/canada`); the API accepts both iso3 and slug.107108## 4. Indicator conventions109110* `slug` is kebab-case, stable, and is the URL: `/indicators/gdp-per-capita`.111* `unit` is human (`current US$`, `% of GDP`, `years`, `people`, `% of population`), `unit_short` for axes (`US$`, `%`, `yrs`).112* `format` drives display: `currency` (scale automatically: 1.2T / 45.3B), `percent` (1 decimal), `number` (thousand separators,113  scale for large), `years`, `index`, `per_1000`, `per_100k`, `ratio`, `celsius`, `tonnes`.114* `higher_is_better` is only set when it is unambiguous (life expectancy = true, infant mortality = false). Otherwise null115  (inflation, population, exchange rate) — rankings then sort descending by value and say "highest".116* `ranking_eligible` false for series that make no sense to rank (exchange rate, policy rate in local terms, indexes).117* Period → `period` is the first day: annual `YYYY-01-01`, quarterly `YYYY-{01,04,07,10}-01`, monthly `YYYY-MM-01`.118* Forecast observations (IMF WEO projections) are stored with `is_forecast=true` and never enter `latest`, `rankings`,119  `changes` or `events`; they are shown dashed on charts.120* A "topic" belongs to the fixed set: `economy, government, population, labor, income, housing, health, education, trade,121  energy, climate, environment, infrastructure, digital, innovation, agriculture, tourism, security, quality-of-life`.122123## 5. Pipeline (`ca` CLI, package `countryatlas`)124125```126ca registry validate                # YAML sanity (unique slugs, known connectors, topics)127ca fetch [--connector X] [--indicator Y]   # raw payloads → raw/…  (retries, backoff, rate limit, ETag/If-Modified-Since when available)128ca normalize [--connector X]        # raw → staging/<connector>/<dataset>.parquet in the NormalizedObservation schema129ca validate                         # rules (§6) → statuses + validation_issues130ca build                            # merge staging by source priority → new DuckDB → derived tables → integrity → atomic swap131ca refresh [--connector X]          # fetch + normalize + validate + build (idempotent; safe to run any time)132ca schedule                         # long-running loop: refresh daily at 03:15 America/Toronto; also refresh on SIGUSR1133ca status                           # last runs, row counts, freshness, stale sources134ca export indicator <slug> | country <iso3>   # CSV/JSON/Parquet into exports/ (also served by the API on demand)135```136137Connector interface (`countryatlas/connectors/base.py`):138139```python140class Connector(Protocol):141    id: str                                   # "worldbank"142    def discover(self) -> list[DatasetDescriptor]143    def fetch(self, spec: IndicatorSourceSpec, ctx: FetchContext) -> RawPayload      # one HTTP "unit" per spec; stored raw144    def normalize(self, raw: RawPayload, spec: IndicatorSourceSpec) -> list[NormalizedObservation]145    def validate(self, rows: list[NormalizedObservation]) -> ValidationReport          # source-specific checks146```147148Error isolation: each (connector, dataset) runs in its own try/except and its own `import_runs` row. A failed fetch149**never** removes previously good data: `build` merges the newest successful staging file per dataset. A connector150that fails validation with `error` severity is quarantined (its staging file is ignored and the previous one kept).151152## 6. Validation rules (deterministic)153154* duplicates (same key twice in one dataset) → error (dataset quarantined)155* impossible values: negative for `nonnegative` indicators, percentages outside [-5, 105] for `percent_share`156  indicators, life expectancy outside [20, 100], etc. (bounds in registry `bounds: [min, max]`) → row `quarantined`157* unit change vs registry unit → dataset quarantined158* extreme jump: |Δ| > `jump_threshold` × robust std of the country series (MAD) → row `warning` (kept, flagged)159* partial download: rows < 30 % of the previous run for the same dataset → dataset quarantined (previous kept)160* stale: `source_updated_at` / latest period older than `stale_after_days` (annual: 800 d, quarterly: 200 d, monthly: 75 d) → `stale`161* mapping errors: unknown country code → logged, row dropped (aggregates are dropped unless mapped to a group)162163Unusual values are never deleted, only flagged.164165## 7. Derived computations166167* **latest**: last non-forecast observation per country×indicator; previous = previous period of the same frequency; ranks168  among `kind='country'` for the same *year* (if a country's latest year is older than the global max year by > 2,169  its rank is still computed within that year but flagged `rank_year != max_year` in the API response).170* **rankings**: for every `ranking_eligible` indicator and every year with ≥ 20 countries.171* **changes / events**: per country×indicator series (annual or higher), deterministic detectors:172  YoY change beyond ±(2 × MAD of yearly diffs) and beyond an indicator-specific absolute floor (e.g. inflation ±2 pts,173  unemployment ±1 pt, GDP growth ±3 pts, population growth ±0.5 pt), record high/low over the whole series, N-year high/low174  (N ∈ {10, 20, 30}), sign flip (growth → contraction), acceleration (3 consecutive increases of the diff).175  `severity` = min(1, |z| / 4) blended with the floor ratio. Headlines are template strings with computed numbers —176  **no LLM**.177* **similarity**: features per mode (see `registry/similarity.yaml`): log-transform heavy-tailed features (GDP pc, population,178  area), z-score across countries (latest values, only countries with ≥ 70 % of the mode's features), weighted Euclidean179  distance → score = 100 × exp(−d / d₀); store the top 12 peers with per-feature contributions (explainable).180* **insights**: templates in `registry/insights.yaml` (e.g. "{country}'s population grew {pct}% since {y0}."),181  each fully computed from data; numbers are never generated by a model.182* **country_dna**: 9 dimensions in [0, 100] = percentile rank of the country for a representative indicator (or the mean of183  2-3 indicators) among all countries: income (GDP pc PPP), demographics (median age ↔ fertility), urbanization,184  trade (trade % GDP), energy (energy use pc), emissions (CO₂ pc), innovation (R&D % GDP, patents pc), education185  (tertiary enrolment, expected years), public spending (gov. expenditure % GDP). Descriptive, not a score.186187### 7.1 Pipeline implementation notes and deviations (as built, 2026-09-11 — see docs/PIPELINE.md)188189No table or column of `schema.sql` was changed. The following precisions/deviations from §2, §5–§7 are binding for the API:190191* **Staging granularity** is one parquet per *source spec* `(connector, dataset, code, indicator)` —192  `staging/<connector>/<dataset>__<code>__<indicator>.parquet` — not one per dataset. Error isolation, quarantine and the193  "keep the previous file" rule apply per spec. `import_runs` holds the **latest attempt per spec** (not the full history);194  `import_runs.dataset` is `"<dataset>:<code>→<indicator>"`; `rows_raw` is the raw payload size in bytes.195* **Series-level source selection** (`build.py::_merge_observations`): for each `(country_id, indicator_id, frequency)`196  the whole series comes from one source — the highest-priority source with any non-forecast data for that country; if a197  lower-priority source's latest non-forecast year is more than 3 years more recent, the freshest source wins instead.198  The decision is stored per row in `metadata.merge_reason` (`"priority"` | `"fresher"`); `meta`/build counts report199  `series_fresher_source`. Consequence: forecast rows appear in `observations` only when the chosen source itself publishes200  them (IMF-only indicators, or IMF chosen as fresher); the complete IMF series (history + forecasts) is always available in201  `observations_alt` for alternative views. Series with forecast-only data fall back to plain priority.202* **Canonical frequency only in derived tables**: `latest`, `rankings`, `changes`, `events`, `similarity`, `insights`,203  `country_dna` and `coverage` use only observations whose `frequency` equals the indicator's registry `frequency`204  (inflation A, policy-rate M, real-house-price-index Q…). Higher-frequency series (e.g. FRED monthly CPI for the USA) stay in205  `observations` for the series/chart endpoints only. Integrity emits warnings when a `latest` row has a non-canonical206  frequency or a headline indicator is ranked in a pool of fewer than 50 countries.207* **Quarantined rows stay in `observations`** (never deleted) but are excluded from every derived table (`latest`,208  `rankings`, `changes`, `events`, `similarity`, `insights`, `country_dna`). A lower-priority source is *not* promoted when209  a row is quarantined (the value is flagged, not replaced). `observations_alt` = every row of every non-chosen source,210  whatever its status.211* **Stale** is evaluated on the *end* of the period (annual 2024 → 2024-12-31) of each country's latest observation, not on212  `source_updated_at` (the WDI vintage date says nothing about a country whose series stops in 2019); only that latest row is213  flagged `stale`. With 800 days, an annual series ending in 2023 is stale in September 2026, one ending in 2024 is not.214* **Extreme jump**: positive level series (formats currency/number/tonnes/kwh/per_*/km/ha with lower bound ≥ 0) are215  compared on log-differences; others on absolute differences. Threshold = `jump_threshold` × 1.4826·MAD with a floor216  (10 % relative, or 2 % of the country's series range) to avoid flagging smooth series; ≥ 5 points required. ~5 % of WDI217  rows carry `warning`.218* **`latest` ranks** are computed among *all* `kind='country'` values of the same indicator within one **year**: the most219  recent year ≤ the country's latest year in which the country has a value AND at least 20 countries are ranked220  (`MIN_COUNTRIES_FOR_RANKING`). `latest.rank_year` = that year — it may be older than `latest.year` when only a handful of221  countries already report the newest year (e.g. CHN internet-users: value 2025, ranks from 2024); ranks are NULL when no222  such year exists. Q/M series use the last period of each year. `rank_income` is NULL when the country has no income group.223  `change_10y_*` compares with the observation exactly 10 years earlier (same frequency).224* **Ranking direction**: rank 1 = lowest value when `higher_is_better = false`, otherwise the highest value — i.e. "best"225  when `higher_is_better` is set, "highest" when it is null. `rankings.pct_rank = 1 − (rank − 1)/(n − 1)` (1.0 = rank 1).226  `rankings` includes every year with ≥ 20 countries for `ranking_eligible` indicators.227* **Changes / events**: working scale = points for percent-like indicators, log-differences (reported as % change) for228  positive level series, absolute otherwise. z = (Δ − median)/max(1.4826·MAD, 0.25 × floor); a move must also clear the229  floor (`change_floor` in the indicator's unit; default 5 % relative, or 2 % of the series range with a 0.5-point minimum230  for shares/rates). YoY severity = `0.3·min(1,|z|/4) + 0.7·min(1,|Δ|/(3·floor))` when the registry defines `change_floor`231  (real-world magnitude dominates), else `0.6·z-part + 0.4·min(1,|Δ|/(2·floor))`; records `0.5 + 0.25·min(1,n/100) +232  0.25·(exceedance in floor units)`; N-year highs/lows 0.4–0.6; sign flips 0.7; acceleration 0.4. The stored severity is233  multiplied by an **importance weight** (headline indicators ×1.0, `featured` ×0.9, others ×0.7; `detail.weight`,234  `detail.raw_severity`). **`changes` are recent by construction**: a detection is kept only if the series' latest year is235  within 2 years of the indicator's max year in the snapshot AND within 3 years of today; older detections exist only in236  `events`. Record/N-year detectors are skipped for series that are monotone over their whole history, and indicators tagged237  `cumulative` (e.g. `cumulative-co2`) run no detector at all. `events` (whole history) contain YoY jumps/drops, records238  reached after a ≥ 5-year gap since the previous record and sign flips, at most 30 per series; `changes` add N-year239  highs/lows (N ∈ {10, 20, 30}) and acceleration/deceleration (3 consecutive increases/decreases of the difference) at the240  latest period only. `id = sha1("changes|"+country+indicator+kind+period)[:16]` (`"events|"` for events). Headlines are241  English templates. **Noise rules** (changes and events): Q/M series are annualised before detection (last value of the242  calendar year for indices/shares/rates/ratios, calendar-year mean otherwise; `period` = Jan 1 of that year) so an indicator243  yields at most one detection per year; indicators with `format: index` or `ranking_eligible: false` only report YoY moves244  with |Δ%| ≥ 10 %; `events` drop rows with severity < 0.35 and keep at most the 3 most severe per (country, indicator, year).245* **OWID freshness**: raw.githubusercontent.com sends no `Last-Modified`; `source_updated_at` for the co2/energy files is the246  date of the last GitHub commit touching the file (`api.github.com/repos/owid/<repo>/commits?path=…`), grapher charts use247  `lastUpdated` from their metadata.248* **Similarity**: features/weights/transforms in `registry/similarity.yaml`; z-scores from `latest` (mixed years allowed);249  a pair needs ≥ 50 % of the mode's weight in common (distance rescaled to full weight); `d0` = median pairwise distance of250  the mode; `contributions[feature].contribution` = share of the squared distance. `country_dna` dimensions are defined in251  the `dna:` section of the same file (percentile rank 0–100, `invert` for fertility); `year_ref` = max year of the inputs.252* **Insights**: `registry/insights.yaml` (16 templates, kinds `change_since` / `rank_in_group` / `vs_median` / `avg_growth`),253  English text, `values` JSON keeps the raw numbers. `insights.id = sha1(country|template_id)[:16]`.254* **Forecasts**: OWID grapher charts with a `*projected*` column (UN WPP) contribute `is_forecast=true` rows for years255  without an estimate, capped at current year + 6.256* **World Bank**: `source=<id>` is added automatically when the indicator metadata says the series lives outside WDI257  (e.g. WGI = source 3, codes `GOV_WGI_*`); codes moved to "WDI Database Archives" (source 57) fail with a RETIRED hint and258  must be remapped in the registry. `lastupdated` (WDI vintage) → `source_updated_at`; `obs_status` E/F → estimate/forecast.259* **Integrity**: the headline-coverage check (≥ 100 countries in `latest`) applies only to headline indicators that have260  staging data; `--no-strict` turns integrity failures into warnings (stored in `meta.integrity_warnings`).261* **`meta` keys**: `schema_version`, `build_run_id`, `built_at`, `observation_count`, `observations_alt_count`,262  `indicator_count`, `country_count`, `latest_count`, `rankings_count`, `changes_count`, `events_count`, `insights_count`,263  `similarity_count`, `staging_files`, `connectors`, `build_duration_s`, `integrity_warnings`.264* **`validation_issues`** are capped at 2 000 rows per rule per spec (5 000 per spec file) to stay small; counts are exact in265  `import_runs.warnings/errors`.266* `ca sql "<query>"` was added to the CLI (read-only DuckDB); `ca normalize` re-reads the newest raw files per spec.267* **Change detection 2.0 (2026-09-12)**: three more `kind` values in `changes`/`events` — `structural_break`, `trend_reversal`,268  `volatility_spike` — computed with `countryatlas.stats` (docs/PIPELINE.md §Build step 6). Same row shape; `detail` records the269  parameters (`gain`, `shift_ratio`, `run`, `ratio`…). Similarity `contributions[feature]` gain `value_a` / `value_b` (raw values).270* **Reliability (2026-09-12)**: validation adds `impossible_year` (row quarantined), `schema_change` (dataset quarantined),271  `null_spike` and `vintage_shift` (warnings vs the previous staging file); the build refuses to publish a snapshot with < 90 % of272  the previous snapshot's observations, records `meta.previous_observation_count` and `meta.source_health`, and writes273  `logs/build-<run_id>.json`. HTTP retries are deterministic (2/4/8/16 s, 5 attempts, `Retry-After` honoured).274275## 8. API (FastAPI, `/api/v1`, OpenAPI at `/api/v1/openapi.json`, docs at `/api/v1/docs`)276277Every response carries `meta: {built_at, run_id, generated_at}`; every value object carries provenance:278279```json280{"value": 53372.1, "period": "2024-01-01", "year": 2024, "unit": "current US$", "is_estimate": false, "is_forecast": false,281 "status": "verified",282 "provenance": {"source": "worldbank", "source_name": "World Bank", "dataset": "WDI", "series_code": "NY.GDP.PCAP.CD",283               "retrieved_at": "2026-09-11T03:20:11Z", "source_updated_at": "2026-07-01", "url": "https://data.worldbank.org/indicator/NY.GDP.PCAP.CD?locations=CA",284               "transform": null, "licence": "CC BY 4.0"}}285```286287Endpoints (all GET, JSON, cached in-process with the snapshot run_id as cache key):288289```290/health                                   {status, run_id, built_at, observations}291/countries?region=&income=&q=             list (id, slug, name, flag, region, income, population_latest, gdp_latest, coverage_pct)292/countries/{id}                           header + headline metrics (topics.yaml `headline`) with latest+rank+change293/countries/{id}/topics/{topic}            all indicators of the topic: latest + sparkline (last 30 pts) + rank294/countries/{id}/series/{indicator}?from=&to=&freq=     full history (values[], with per-observation provenance summary)295/countries/{id}/changes                   recent changes (sorted by severity)296/countries/{id}/events?limit=             timeline297/countries/{id}/similar?mode=overall      peers with contributions298/countries/{id}/insights299/countries/{id}/dna300/countries/{id}/download.{csv|json}       full country dataset301/indicators?topic=&q=                     list302/indicators/{slug}                        definition + sources + coverage + world/regions latest + freshness303/indicators/{slug}/map?year=              {year, values: {ISO3: value}, legend: {min,max,breaks[]}, n}304/indicators/{slug}/trend?group=world      median/aggregate trend (WB aggregate when available)305/indicators/{slug}/download.{csv|json}306/series?country=CAN,FRA&indicator=gdp-per-capita&from=1990&to=2026&freq=A307/rankings                                 list of rankable indicators (featured first)308/rankings/{indicator}?year=&group=&sort=asc|desc&limit=&offset=     rows: rank, country, value, change_1y, change_10y, sparkline309/rankings/{indicator}/history?countries=CAN,USA    rank by year310/compare?countries=CAN,USA,FRA&indicators=…&from=&to=&mode=absolute|per-capita|index100|pct      series bundle311/compare/snapshot?countries=…&topic=      table of latest values for the topic's indicators312/regions | /regions/{slug}                group page data (members, aggregate metrics, member rankings)313/search?q=&limit=                         [{type: country|indicator|topic|region|source, id, slug, name, hint, score}]314/home                                     global snapshot + curated lists (largest economies, fastest growth, …) + recent changes + recently updated315/changes?limit=&kind=                     global recent changes feed316/sources | /sources/{id}                  source metadata + datasets + import runs + freshness317/methodology                              static from registry (units, priorities, validation rules) — for the page318/admin/*   (header X-Admin-Token)         connectors health, import_runs, validation_issues, coverage matrix, trigger refresh (SIGUSR1)319```320321Errors: RFC 7807 problem+json. Unknown country/indicator → 404. Rate limit 120 req/min/IP (in-process token bucket; requests that322carry `X-CountryAtlas-Internal: <CA_ADMIN_TOKEN>` or arrive from loopback without any forwarding header — server-side renders, the323build prerender, the pipeline — are exempt). Public GET responses carry a weak `ETag` derived from the snapshot run id (304 on324`If-None-Match`) and `Cache-Control: public, max-age=300, stale-while-revalidate=3600`. API 1.1 analytics endpoints: see docs/API.md.325326## 9. Web app (Next.js 16, React 19, TypeScript, Tailwind v4, App Router)327328* Server components fetch the API at `process.env.API_URL` (`http://127.0.0.1:8291`). Browser-side fetches go to the329  same origin `/api/v1/*` (Next `rewrites`).330* Routes (2.0, 2026-09-12): `/` (hero map + time machine, world pulse, movers), `/explore` (World Explorer, full viewport),331  `/trajectories`, `/scatter`, `/finder`, `/extremes`, `/peers`, `/countries`, `/countries/[slug]` (story, timeline 2.0, DNA reference,332  similar 2.0), `/countries/[slug]/[topic]`, `/compare`, `/compare/[...slugs]` (head-to-head for two countries, percentile / change modes),333  `/rankings`, `/rankings/[indicator]` (filters, table/bars/map, rank race), `/indicators`, `/indicators/[slug]` (frames map, distribution,334  related), `/regions`, `/regions/[slug]`, `/regions/compare`, `/changes`, `/stories`, `/stories/[slug]`, `/download` (`/data` → 308),335  `/updates`, `/sources`, `/sources/[id]`, `/methodology`, `/api` (interactive explorer), `/admin`, `/sitemap/<shard>.xml`, `/robots.txt`,336  OG images per country / indicator / ranking / compare. Full-bleed routes (`/explore`, `/trajectories`) are declared in337  `components/layout/main-frame.tsx`.338* Charts: our own SVG chart kit in `apps/web/src/components/charts/` (server-renderable, touch, accessible summaries):339  LineChart, AreaChart, StackedArea, RankedBars, Scatter, BubbleChart (animated, region colours, fit line, trails), Histogram340  (markers), RankRace, Sparkline, SlopeChart, SmallMultiples, PopulationPyramid, Choropleth (d3-geo + world-atlas 110m, SVG;341  zoom/pan canvas in `components/explorer/map-canvas.tsx`), DNA radial (reference polygon). Shared controls in342  `components/controls/` (YearSlider with play, IndicatorSelect, Segmented); URL state via `lib/url-state.ts` (client) /343  `lib/url-params.ts` (server-safe).344* i18n: all UI strings through `apps/web/src/i18n/en.ts` (`t('key')`), no hard-coded English in components.345* Design tokens in `apps/web/src/app/globals.css` (@theme). Editorial, calm, data-dense. No card soup.346* Navigation: Explore · Countries · Compare · Rankings · Indicators · Changes · More ▾ (Regions, Trajectories, Scatter, Finder,347  Extremes, Above/below expected, Stories, Sources, API, Downloads, Data updates, Methodology). Mobile tab bar: Explore · Countries ·348  Compare · Rank · Search; bottom sheets for filters and map details; no horizontal overflow at 320–1920 px (`qa/final-sweep.mjs`).349* Direction semantics: `--inc`/`--dec` (teal/sienna) and the diverging ramp `--div-1..7` colour *increase/decrease*; `--up`/`--down`350  (green/red) are used only when an indicator declares `higher_is_better` (ChangeChip). Quality badges: fresh · historical · sparse ·351  limited-coverage · stale · flagged · forecast (`components/data/quality-badge.tsx`, vocabulary shared with the API).352* Provenance drawer: any value component (`<Metric>`, chart point) opens `<ProvenanceSheet>` with the provenance object.353354## 10. Ports & env355356| Process | Port | Env |357|---|---|---|358| countryatlas-web | 8290 | `API_URL=http://127.0.0.1:8291`, `NEXT_PUBLIC_SITE_URL=https://www.countryatlas.co` |359| countryatlas-api | 8291 (127.0.0.1) | `CA_DATA_DIR=~/countryatlas-data`, `CA_ADMIN_TOKEN`, `FRED_API_KEY` |360| countryatlas-scheduler | — | same as API |361362Secrets (FRED key, admin token) live only in the mld manifest on M1M32 and in local `.env` (git-ignored).363