# Architecture

```
 8 probes (Go, launchd/systemd)            RIPE RIS Live (wss)            public status pages (optional)
 Québec · Montréal · Atlanta · Gravelines   23 collectors, ~10k prefixes/s   Statuspage / GCP / AWS RSS / status.io
 Dublin · Istanbul · Uşak · Ayia Napa              │                                  │
        │ signed gzip batches (HTTPS)               │                                  │
        ▼                                           ▼                                  ▼
 ┌──────────────── BHS64b (OVH Beauharnois, Docker Compose) ──────────────────────────────────────┐
 │ edge Caddy :8350 (bound to wg1 10.67.0.61) ── /ingest,/api → api :8352 ── / → web :8351         │
 │ api (FastAPI)  ── ingest → ClickHouse writer + Redis live counters                               │
 │ bgp (RIS Live) ── 10 s per-collector aggregates, per-origin minutes, sampled raw                 │
 │ corroboration  ── vendor_status (Postgres)                                                       │
 │ engine (10 s)  ── baselines (5 min) → robust z → components → scopes → global → events → fronts  │
 │                   → Redis live state + pub/sub → SSE ;  ClickHouse pressure_history/provenance   │
 │ Postgres 17 (registry, config, events)  ClickHouse 25.8 (telemetry, history)  Redis 7 (live)     │
 └──────────────────────────────────────────────────────────────────────────────────────────────────┘
        ▲ WireGuard 10.67.0.0/24
 BHS64 gateway: Caddy TLS  https://www.internetpressure.io → 10.67.0.61:8350   (DNS A → 51.161.112.61)
```

## Decisions

- **Python (FastAPI) backend, Go probe.** The spec mandates Go for the agent; the backend language is free. Python matches the
  other MacLustr platforms (AI Atlas, SatelliteIndex, CountryAtlas) and keeps the scoring code readable/testable. All stores are
  spoken to over their wire protocols (asyncpg, ClickHouse HTTP, redis) — no ORM.
- **One backend image, four processes** (`ip api`, `ip engine`, `ip bgp`, `ip corroboration`). They share nothing but the stores.
- **ClickHouse HTTP interface** via httpx (JSONEachRow) instead of a driver: async, dependency-free, trivially inspectable.
- **Baselines from raw measurements** (180 d TTL) recomputed every 5 minutes; MAD estimated from the IQR (0.7413 × IQR) in a
  single pass; hour-of-day seasonality (±1 h) once ≥ 3 days of history exist. `measurements_1m` (AggregatingMergeTree MV, 1 y)
  serves long-range charts.
- **Self-exclusion** (`engine/health.py`): < 2 fresh usable probes, every probe failing at once, or a store down ⇒
  `internal_status = degraded`, score frozen (`stale: true`), no history rows, no incidents. A probe failing ≥ 80 % of its HTTP
  targets while the median probe is fine is excluded (its uplink, not the Internet).
- **Calibration**: until at least one signal has ≥ 20 % of `baseline_min_samples` (12 five-minute buckets), the instrument
  publishes `pressure: null`, `level: calibrating`. Nothing is invented.
- **Config-driven scoring**: `packages/config/pressure.yaml`, optionally overridden by the Postgres `config` row (admin UI); the
  engine re-reads it every cycle. Weights must sum to 1 (validated).
- **Explainability**: every cycle stores `signal_features` (component, signal, scope, current, baseline, MAD, z, samples, stress,
  contribution) and the live payload carries `explain[]` (attribution = component contribution × share of stress mass by region /
  signal / service) and `components[].drivers[]`.

## Scoring pipeline (engine/compute.py, engine/scoring.py)

1. **Pair signals**: per (probe, target, kind, resolver) in the last 120 s: latency z (ttfb, tcp, rtt), loss excess, failure-rate
   excess, 5xx/TLS/reset rates, DNS failure & latency; per target: corroborated unavailability (≥ 2 probes in ≥ 2 regions) and
   resolver disagreement (one resolver answers, another SERVFAILs); per (probe, target) traceroute: route fingerprint changed vs
   the 7-day dominant route, hop-count z, latency shift on changed paths.
2. **Pair stress** 0..1: `z_to_stress` ramps from z = 1 to z = 6; rate signals use `rate_stress` (excess / max(scale, 3 × baseline));
   route churn uses `ratio_stress`.
3. **Signal stress** = importance-weighted mean of pair stresses; the weight is damped by baseline coverage
   (`coverage_factor`), so thin baselines cannot move the index.
4. **Component score** = `saturate(Σ signal_weight × signal_stress)`, `saturate(x) = 100·(1−e^{−kx})/(1−e^{−k})`, k = 1.2.
5. **Routing** (global only, ASN level via `bgp_origin_1m`): weighted-median robust z across collectors of per-minute
   withdrawals / announcements / origin changes vs their 7-day (seasonal) baseline; collector disagreement when a minority of
   collectors spike.
6. **Corroboration**: Σ impact(indicator) × service importance / 4, capped at 1 — weight 0.05, optional by design.
7. **Global** = Σ weight × component over available components (weights renormalised). Regions/countries/services/ASNs reuse
   the same machinery over their signal subsets (routing null where not attributable).
8. **Velocity / acceleration / volatility** from the last hour of `pressure_history`.
9. **Events** (`engine/events.py`): scope score ≥ 45 with confidence ≥ 0.45 for 2 cycles ⇒ `detected`; → `developing` → `active`
   (6 cycles); < 30 ⇒ `recovering`; 10 min under 30 ⇒ `resolved`. Rule-based hypotheses with evidence.
10. **Pressure Fronts** (`engine/fronts.py`): ≥ 3 stressed (probe, target) pairs on a source-region → destination-region corridor
    with saturated corridor stress ≥ 35.

## Ethics & load

Per probe with ~27 700 targets (tiers 20 s / 45 s / 300 s / 900 s; the long tail is HTTP-only): ~60 HTTP/s, ~26 DNS/s (4 resolvers / 600 s, infrastructure only), ~20 ICMP runs/s (every 120 s, infrastructure only), ≤ 1 traceroute in flight (every 15 min
for the 30 `tr: true` targets). Keep-alive disabled (to measure TCP/TLS), body read capped at 16 KiB, no retries, no parallel
checks to one host, intervals never below 10 s. Total ≈ 105 measurements/s/probe → ~70 M rows/day for 8 probes (~3–4 GB/day
compressed in ClickHouse).

## Replay

`ip replay <from> <to> --weights '{…}'` (and `POST /api/admin/replay`) recomputes the global index from stored component
history with alternative weights. Re-deriving components from raw measurements with alternative engine parameters is an
offline job to add when enough history exists (raw is kept 180 days for exactly that purpose).

## Retention — we never erase (founder decision, 2026-09-12, migration `0002_keep_forever.sql`)

Every table is kept indefinitely: measurements, measurements_1m, traceroutes, probe_health, bgp_stats_10s, bgp_origin_1m,
pressure_history, signal_features, engine_runs. The single exception is `bgp_events`, the *raw* RIS Live stream (≈ 10 k
prefixes/s at the source, already sampled 1/25 for announcements): 90 days. Nightly backups (`deploy/bin/backup.sh`) copy
Postgres and the irreplaceable ClickHouse tables off-node to BHS128. Disk growth ≈ 3–4 GB/day on BHS64b's 878 GB RAID-1;
plan a second ClickHouse volume (or a cold tier on BHS128) within a year.

## Registry at scale (2026-09-13)

`data/targets/targets.yaml` (2 128 hand-curated) + `data/targets/fragments/{government,finance,infrastructure,developer,media,commerce}.yaml`
(26 700 more, every host verified by DNS + HTTP with the probe UA at authoring time). `ip seed` loads the main file then the
fragments, skipping duplicate ids or (host, url, ip) — first occurrence wins. Tier 4 (900 s) carries the long tail; the probes
receive the whole list (≈ 8 MB JSON, cached per config version) and schedule ≈ 90 000 jobs each with 48-way concurrency.
Engine: ≈ 35 000 (probe, target) pairs in each 120 s window, ≈ 100 000 signals, ~2 s per 15 s cycle. The sampling boost is
delivered inside batch responses and never bumps the config version (a bump makes all probes re-download the registry).