SPB Git forge
spboucher.ai ↗

spb/internetpressure

Public
15commits 1branches 0releases
29.7 MBsize
maindefault branch
10 days agolast push
TypeScript 36.3% Python 31.8% Go 18% JavaScript 9.8% Shell 1.9% SQL 1.4% CSS 0.5%
19.1 KB
View source Raw Blame History

# InternetPressure public API (v1) — contract

Served by apps/api (FastAPI) on :8352; the Next app (apps/web, :8351) reaches it through the edge on the same origin (/api/* → api, /ingest/* → api, everything else → web). Browser code therefore always calls relative URLs (/api/v1/...). All timestamps are UTC ISO-8601 with Z. All numbers are plain JSON numbers (never strings). Every response carries Cache-Control: no-store unless stated. Unknown scope → 404 {"error":"not_found"}.

Rate limit (public tier): 300 req/min per IP (internal SSR calls exempt); SSE connections: 4 per IP. 429 with Retry-After.

Pressure levels (from packages/config/pressure.yaml): calm ≤10, normal ≤25, elevated ≤40, stressed ≤55, high ≤70, severe ≤85, extreme ≤100. Every level field is one of these ids; level_label is its label.

Internal status (self-exclusion, §57 of the spec): internal_status is "ok", "degraded" (too few fresh probes / BGP stale / stores unhealthy — score is frozen at its last value and stale: true) or "stale" (engine hasn't run). The UI must show a visible "instrument degraded" state instead of interpreting a frozen number.

# GET /api/v1/status

json
{ "ok": true, "ts": "…", "internal_status": "ok", "engine": { "last_run": "…", "cycle_ms": 412, "cycle_seconds": 10 },
  "ingest": { "last_batch": "…", "batches_5m": 312, "measurements_5m": 41200 },
  "probes": { "fresh": 8, "total": 8, "excluded": [] },
  "bgp": { "fresh": true, "last_message": "…", "collectors": 12 },
  "stores": { "clickhouse": true, "postgres": true, "redis": true }, "version": "0.1.0" }

# GET /api/v1/pressure/global

json
{
  "ts": "…", "pressure": 42.7, "level": "stressed", "level_label": "Stressed",
  "delta_1h": 6.3, "delta_24h": -2.1, "velocity_per_h": 7.2, "acceleration_per_h2": 3.1, "volatility_1h": 2.4,
  "trend": "rising",                       // rising | falling | stable
  "confidence": 0.86, "stale": false, "internal_status": "ok",
  "coverage": { "probes_active": 8, "probes_total": 8, "probe_regions": 5, "targets": 212,
                "measurements_5m": 41200, "bgp_collectors": 12, "baseline_days": 6.2 },
  "components": [
    { "id": "routing", "label": "Routing", "score": 57.1, "weight": 0.25, "contribution": 14.3,
      "trend": "rising", "delta_1h": 11.0, "confidence": 0.9,
      "drivers": [ { "label": "BGP withdrawals/s 4.8× baseline", "points": 9.1, "scope_type": "bgp", "scope_id": "withdrawals" } ] },
    { "id": "latency", … }, { "id": "dns", … }, { "id": "availability", … }, { "id": "http_tls", … },
    { "id": "path", … }, { "id": "corroboration", … }
  ],
  "explain": [
    { "text": "+14.3 points from elevated BGP route churn", "points": 14.3, "component": "routing", "scope_type": "global", "scope_id": null },
    { "text": "+11.0 from North America East packet loss", "points": 11.0, "component": "latency", "scope_type": "region", "scope_id": "na-east" },
    { "text": "−3.1 because Western Europe remains stable", "points": -3.1, "component": "latency", "scope_type": "region", "scope_id": "eu-west" }
  ],
  "sparkline_1h": [ 36.1, 36.4, … ]        // 60 points, 1 per minute, oldest first (null where missing)
}

# GET /api/v1/pressure/history?scope_type=global|region|country|asn|service|component&scope_id=…&range=1h|6h|24h|7d|30d|1y

Server-side aggregation; step is chosen by the server (1h→10s, 6h→1m, 24h→1m, 7d→5m, 30d→1h, 1y→1d).

json
{ "scope_type": "global", "scope_id": null, "range": "24h", "step_seconds": 60,
  "points": [ { "ts": "…", "pressure": 41.2, "components": { "routing": 55.0, "latency": 40.1, "dns": 12.0,
                "availability": 30.3, "http_tls": 22.0, "path": 47.9, "corroboration": 0.0 }, "confidence": 0.85 } ],
  "summary": { "min": 21.0, "max": 61.4, "avg": 33.7, "max_ts": "…" } }

Cache-Control: public, max-age=30 for ranges ≥ 24h.

# GET /api/v1/pressure/regions

json
{ "ts": "…", "regions": [
  { "id": "na-east", "name": "North America East", "continent": "North America", "lat": 43, "lon": -76,
    "pressure": 51.3, "level": "stressed", "level_label": "Stressed", "delta_1h": 12.1, "trend": "rising", "confidence": 0.8,
    "components": { "routing": null, "latency": 63.2, "dns": 10.1, "availability": 35.0, "http_tls": 20.0, "path": 58.0 },
    "probes": 3, "targets": 41, "incidents": 1, "coverage_ok": true, "role": "both" } ] }   // role: probe | target | both

Regional pressure = stress observed from probes in the region (source view) blended with stress observed toward targets anchored in the region (destination view); routing is null for regions without ASN attribution.

# GET /api/v1/pressure/region/{id}

Region object above plus: history_24h ({step_seconds, points:[{ts,pressure}]}), baseline_7d ({median, p90}), incidents (list, see incidents), top_asns ([{asn, name, pressure}]), top_services ([{slug, name, pressure, observed_availability_24h}]), probes (probe list objects), matrix (latency matrix rows for this region, see /latency).

# GET /api/v1/pressure/countries

json
{ "ts": "…", "countries": [ { "cc": "CA", "name": "Canada", "region": "na-east", "lat": 56.1, "lon": -106.3,
   "pressure": 34.0, "level": "elevated", "level_label": "Elevated", "delta_1h": 2.0, "trend": "stable",
   "components": { … }, "probes": 2, "targets": 18, "role": "both", "coverage_ok": true } ] }

Only countries where we have at least one probe or one anchored target appear. The map colours these; every other country is drawn neutral (we do not pretend to observe it).

# GET /api/v1/pressure/country/{cc}

Country object plus history_24h, baseline_7d, incidents, asns, services, probes, targets ([{target_id, name, category, pressure, ok_ratio_1h, ttfb_ms_median}]).

# GET /api/v1/asns

{ "ts": …, "asns": [ { "asn": 13335, "name": "Cloudflare", "country": "US", "pressure": 18.2, "level": "normal", "routing": 12.0, "latency": 22.1, "availability": 9.0, "targets": 6, "prefixes_observed": 1200, "importance": 5 } ] }

# GET /api/v1/pressure/asn/{asn}

json
{ "asn": 13335, "name": "Cloudflare, Inc.", "country": "US", "importance": 5, "ts": "…",
  "pressure": 18.2, "level": "normal", "level_label": "Normal", "delta_1h": -1.0, "trend": "falling", "confidence": 0.7,
  "components": { "routing": 12.0, "latency": 22.1, "availability": 9.0, "dns": 4.0, "http_tls": 6.0, "path": 15.0 },
  "bgp": { "prefixes_observed_24h": 1200, "announcements_1h": 340, "withdrawals_1h": 12, "churn_ratio": 1.1,
           "origin_changes_1h": 0, "path_stability": 0.97, "series_24h": [ { "ts": "…", "announcements": 5, "withdrawals": 0 } ] },
  "regions_observed": [ "na-east", "eu-west", "eu-east-med" ],
  "targets": [ { "target_id": "cloudflare-www", "name": "Cloudflare", "pressure": 12.0, "ok_ratio_1h": 1.0, "ttfb_ms_median": 71.0 } ],
  "history_24h": { "step_seconds": 60, "points": [ … ] }, "incidents": [ … ] }

# GET /api/v1/services

{ "services": [ { "slug": "cloudflare", "name": "Cloudflare", "category": "cdn", "pressure": 12.0, "level": "normal", "observed_availability_24h": 0.9994, "targets": 6, "affected_regions": [], "vendor_status": { "indicator": "none", "incidents": 0, "source": "status.cloudflare.com", "checked_at": "…" } } ] } vendor_status is null when we have no connector for that provider.

# GET /api/v1/service/{slug}

Service object plus:

json
{ "observed": { "availability_24h": 0.9994, "availability_1h": 1.0, "ttfb_ms_median_1h": 71.0, "ttfb_ms_baseline": 68.0,
                "tls_ms_median_1h": 30.1, "failures_1h": 2 },
  "affected_regions": [ { "id": "na-east", "name": "…", "observation": "Elevated TLS latency from 2 probes" } ],
  "vendor_status": { "indicator": "minor", "incidents": 1, "titles": ["…"], "source": "…", "url": "…", "checked_at": "…" },
  "discrepancy": "Vendor reports no incident; we observe elevated TLS latency from 4 probe regions.",  // or null
  "matrix": [ { "probe_id": "ca-qc-01", "probe_region": "na-east", "targets": [ { "target_id": "…", "ok": true, "ttfb_ms": 70.0, "z": 0.4, "ts": "…" } ] } ],
  "targets": [ … ], "history_24h": { … }, "incidents": [ … ] }

# GET /api/v1/targets · GET /api/v1/target/{id}

List: { "targets": [ { "target_id", "name", "hostname", "category", "provider", "service_id", "country", "region", "importance", "tier", "pressure", "ok_ratio_1h", "ttfb_ms_median_1h" } ] }. Detail adds latest_by_probe ([{probe_id, kind, ts, ok, error, dns_ms, tcp_ms, tls_ms, ttfb_ms, http_status, resolved_ip, packet_loss, rtt_avg_ms, z}]), series_24h ({step_seconds, points:[{ts, ttfb_ms_p50, ok_ratio}]}) and dns ({ resolvers: [{resolver, rcode, answers, ms}], disagreement: false }).

# GET /api/v1/probes

json
{ "probes": [ { "probe_id": "ca-qc-01", "name": "Québec City (Bell)", "region": "na-east", "country": "CA", "city": "Québec",
  "provider": "Bell Canada", "asn": 577, "lat": 46.8, "lon": -71.2, "status": "online",   // online | stale | offline | excluded
  "last_seen": "…", "version": "0.1.0", "measurements_1h": 5120, "uptime_24h": 0.998, "clock_offset_ms": -14,
  "capabilities": ["http","dns","ping","traceroute"] } ] }

# GET /api/v1/incidents?status=active|resolved|all&limit=50&offset=0

json
{ "total": 3, "incidents": [ {
  "event_id": "evt_01J…", "slug": "2026-09-12-north-america-east-latency-anomaly",
  "type": "regional_latency",  // regional_latency | dns_disruption | routing_instability | service_degradation | availability_loss | path_instability | global_pressure
  "title": "North America East latency anomaly", "summary": "Elevated latency and packet loss observed from 3 probes toward 41 targets.",
  "status": "active",          // detected | developing | active | recovering | resolved
  "scope_type": "region", "scope_id": "na-east", "scope_label": "North America East",
  "started_at": "…", "updated_at": "…", "ended_at": null, "duration_s": 1260,
  "peak_pressure": 76.0, "current_pressure": 71.2, "confidence": 0.93,
  "affected_probes": 3, "affected_targets": 41, "affected_asns": [577, 16276], "affected_services": ["aws", "github"],
  "hypotheses": [ { "text": "Possible upstream transit issue", "confidence": 0.6,
                    "evidence": ["Route fingerprints changed on 62 % of paths", "Latency rose on paths crossing AS6453"] } ]
} ] }

# GET /api/v1/incident/{slug}

Incident object plus timeline ([{ts, status, pressure, note}]), evidence ([{signal_id, label, scope_type, scope_id, current, baseline, robust_z, samples, ts}]), series ({step_seconds, points:[{ts, pressure, global_pressure}]} from 30 min before start to now/end), probes ([{probe_id, region, observation}]), targets ([{target_id, name, service_id, observation}]), bgp ({withdrawals_ratio, announcements_ratio, origin_changes} or null), annotations ([{ts, author, text}]).

# GET /api/v1/fronts

json
{ "ts": "…", "fronts": [ { "id": "front_na-east_eu-west", "name": "North Atlantic Pressure Front", "status": "developing",
  "intensity": 74.0, "confidence": 0.89, "direction": "east", "since": "…",
  "from": { "region": "na-east", "name": "North America East", "lat": 43, "lon": -76 },
  "to":   { "region": "eu-west", "name": "Western Europe", "lat": 49, "lon": 3 },
  "observed": { "latency_pct": 43.0, "churn_x": 4.8, "loss_pct": 3.1, "pairs": 17, "targets": 17, "route_changes": 9 } } ] }

# GET /api/v1/bgp/stats

json
{ "ts": "…", "fresh": true, "updates_per_s": 812.4, "announcements_per_s": 760.0, "withdrawals_per_s": 52.4,
  "baseline": { "announcements_per_s": 640.0, "withdrawals_per_s": 11.0 }, "ratio": { "announcements": 1.19, "withdrawals": 4.76 },
  "unique_prefixes_1m": 14211, "unique_origins_1m": 2210, "origin_changes_1m": 3, "peers": 1450,
  "collectors": [ { "id": "rrc00", "location": "Amsterdam", "announcements_per_s": 120.1, "withdrawals_per_s": 8.0, "peers": 210, "last_message": "…", "fresh": true } ],
  "series_1h": [ { "ts": "…", "announcements": 7600, "withdrawals": 520 } ],           // per minute
  "top_origins_1h": [ { "asn": 13335, "name": "Cloudflare", "announcements": 340, "withdrawals": 12 } ] }

# GET /api/v1/latency

json
{ "ts": "…", "global": { "rtt_ms_median": 41.2, "rtt_ms_baseline": 39.0, "ttfb_ms_median": 118.0, "ttfb_ms_baseline": 110.0, "packet_loss_pct": 0.3 },
  "matrix": [ { "from": "na-east", "to": "eu-west", "rtt_ms": 92.1, "rtt_ms_baseline": 88.0, "ttfb_ms": 160.0, "loss_pct": 0.0, "z": 0.6, "pairs": 37 } ],
  "by_probe": [ { "probe_id": "ca-qc-01", "rtt_ms_median": 30.1, "ttfb_ms_median": 90.0, "loss_pct": 0.0, "z": 0.2 } ] }

# GET /api/v1/ticker

json
{ "ts": "…", "bgp_updates_per_s": 812.4, "bgp_withdrawals_per_s": 52.4, "bgp_updates_per_min": 48744,
  "probes_active": 8, "probes_total": 8, "measurements_per_s": 12.3, "measurements_per_min": 738,
  "targets_degraded": 4, "targets_total": 212, "regions_elevated": 3, "regions_normal": 9, "regions_severe": 0,
  "dns_failures_per_min": 2, "median_global_rtt_ms": 41.2, "route_changes_per_min": 1.2, "active_incidents": 1,
  "internal_status": "ok" }

# GET /api/v1/routes?probe={probe_id}&target={target_id}

json
{ "probe": { "probe_id": "…", "name": "…", "asn": 577 }, "target": { "target_id": "…", "name": "…", "hostname": "…", "asn": 13335 },
  "current":  { "ts": "…", "route_hash": "…", "reached": true, "total_ms": 30.2,
                "hops": [ { "n": 1, "ip": "192.168.2.1", "asn": null, "asn_name": null, "rtt_ms": 1.2, "private": true } ] },
  "baseline": { "route_hash": "…", "share_7d": 0.82, "first_seen": "…", "last_seen": "…", "hops": [ … ] },
  "diff": { "changed": true, "added": [ { "n": 6, "ip": "…", "asn": 6453 } ], "removed": [ … ],
            "asn_path_current": [577, 6453, 13335], "asn_path_baseline": [577, 577, 13335], "latency_shift_ms": 12.4, "hop_delta": 1 },
  "history_24h": [ { "ts": "…", "route_hash": "…", "hop_count": 11, "total_ms": 30.2 } ],
  "route_share_7d": [ { "route_hash": "…", "share": 0.82, "asn_path": [ … ] } ] }

GET /api/v1/routes/pairs → { "pairs": [ { "probe_id", "target_id", "changed_24h": 3, "current_route_hash", "stable": false } ] }.

# GET /api/v1/history/summary?year=2026&month=9

json
{ "year": 2026, "month": 9, "days": [ { "date": "2026-09-12", "min": 18.0, "max": 61.4, "avg": 30.2, "events": 2 } ],
  "top_events": [ incident objects, by peak_pressure ], "top_asns": [ { "asn", "name", "events", "max_pressure" } ],
  "top_regions": [ { "id", "name", "events", "max_pressure", "hours_elevated" } ],
  "largest": { "pressure": {…incident}, "routing": {…}, "dns": {…}, "latency": {…} }, "available_months": ["2026-09"] }

Without month → per-month rows in months instead of days. Without year → all years.

# GET /api/v1/explain

Deep explainability: { "ts", "pressure", "components": [ { "id", "score", "weight", "contribution", "signals": [ { "signal_id": "ttfb_z", "label": "…", "scope_type": "region", "scope_id": "na-east", "current": 161.0, "baseline_median": 110.0, "mad": 9.0, "robust_z": 5.6, "samples": 412, "stress": 0.71, "contribution": 6.2 } ] } ], "excluded_probes": [], "notes": [ "…" ] }

# GET /api/v1/methodology

Public copy of the scoring config: { "weights": {…}, "levels": [ … ], "engine": { "cycle_seconds", "baseline_days", "z_anomaly", … }, "version": 1, "updated_at": "…" }.

# GET /api/v1/search?q=

{ "results": [ { "type": "country|asn|service|region|target|incident", "id": "…", "label": "…", "href": "/asn/13335", "pressure": 12.0 } ] }

# GET /api/v1/live — Server-Sent Events

Headers Content-Type: text/event-stream, Cache-Control: no-store, X-Accel-Buffering: no. Sends retry: 5000, an id: per message, a : ping comment every 15 s, and on connect an immediate snapshot event.

event data
snapshot { "global": <GET /pressure/global>, "ticker": <GET /ticker>, "regions": <regions[]>, "fronts": <fronts[]>, "incidents": <active incidents[]> }
global_pressure_update same shape as GET /pressure/global (sent every engine cycle, i.e. only when the engine actually ran)
regional_pressure_update { "ts", "regions": [ … ], "countries": [ { "cc", "pressure", "level", "delta_1h" } ] }
ticker <GET /ticker> (every 5 s, from real counters)
bgp_stats { "ts", "updates_per_s", "announcements_per_s", "withdrawals_per_s", "ratio": {…}, "fresh" } (every 5 s)
probe_stats { "ts", "probes_active", "probes_total", "measurements_per_s", "excluded": [] }
incident_created / incident_updated incident object
service_degradation { "ts", "slug", "name", "pressure", "regions": [ … ], "observation": "…" }
front_update <GET /fronts>
internal_status { "ts", "internal_status", "reason" }

Nothing is ever emitted without a corresponding backend computation — if the engine pauses, the stream only pings.

# Admin API (/api/admin/*, header X-IP-Admin-Token: <token>)

  • GET /api/admin/overview → { "probes": [ { …probe, "health": { "uptime_24h", "clock_offset_ms", "missing_ratio_1h", "error_rate_1h", "buffered", "spool_bytes", "version", "last_health" } } ], "ingest": { "batches_per_min", "measurements_per_min", "rejected_per_min", "last_batch" }, "stores": { "clickhouse": { "ok", "inserts_per_s", "tables": [ { "name", "rows", "bytes", "oldest", "newest" } ] }, "postgres": { "ok", "size_bytes" }, "redis": { "ok", "used_memory_bytes", "keys" } }, "bgp": { "collectors": [ … ], "messages_per_s", "fresh", "reconnects_24h" }, "engine": { "last_run", "cycle_ms_p50", "cycle_ms_max", "runs_1h", "errors_1h", "internal_status", "excluded_probes" }, "corroboration": [ { "id", "name", "ok", "last_fetch", "incidents" } ] }
  • GET /api/admin/targets · POST /api/admin/targets (target object) · PATCH /api/admin/targets/{id} · DELETE /api/admin/targets/{id}
  • GET /api/admin/probes · POST /api/admin/probes {probe_id, name, region, country, city, provider, asn, lat, lon} → { …probe, "key": "<hex, shown once>" } · PATCH /api/admin/probes/{id} (enabled, metadata) · POST /api/admin/probes/{id}/rotate-key → new key
  • GET /api/admin/config → full pressure.yaml as JSON · PUT /api/admin/config (validates: weights sum to 1 ± 0.001) → stored in Postgres config and applied next cycle
  • GET /api/admin/baselines?signal_id=&scope_type=&scope_id= → { "signal_id", "points": [ { "ts", "value", "median", "mad", "z" } ], "samples", "baseline_days" }
  • GET /api/admin/raw?table=measurements|traceroutes|bgp_events|bgp_stats|pressure_history|signal_features|probe_health&probe_id=&target_id=&limit=200 → { "columns": [ … ], "rows": [ [ … ] ] }
  • GET /api/admin/incidents?status= · PATCH /api/admin/incidents/{id} { "review": "confirmed|dismissed|unreviewed", "note": "…" }
  • POST /api/admin/annotations { "ts", "scope_type", "scope_id", "text" } · GET /api/admin/annotations
  • POST /api/admin/replay { "from": "…", "to": "…", "weights": {…} } → { "step_seconds", "points": [ { "ts", "pressure_original", "pressure_replayed" } ] }
  • POST /api/admin/boost { "targets": [ … ], "factor": 0.5, "seconds": 900 } — manual sampling boost pushed to probes.

Errors: 401 {"error":"unauthorized"}, 422 {"error":"validation", "detail": …}.