| 65 |
65 |
|
| 66 |
66 |
## Ethics & load |
| 67 |
67 |
|
| 68 |
|
−Per probe with 326 targets: ~7 HTTP/s, ~11 DNS/s (4 resolvers / 120 s), ~5 ICMP runs/s, ≤ 1 traceroute in flight (every 15 min |
|
68 |
+Per probe with 2 128 targets (tiers 20 s / 45 s / 300 s): ~18 HTTP/s, ~14 DNS/s (4 resolvers / 600 s), ~18 ICMP runs/s (every 120 s), ≤ 1 traceroute in flight (every 15 min |
| 69 |
69 |
for the 30 `tr: true` targets). Keep-alive disabled (to measure TCP/TLS), body read capped at 16 KiB, no retries, no parallel |
| 70 |
|
−checks to one host, intervals never below 10 s. Total ≈ 25 measurements/s/probe → ~17 M rows/day for 8 probes (~1 GB/day |
|
70 |
+checks to one host, intervals never below 10 s. Total ≈ 50 measurements/s/probe → ~35 M rows/day for 8 probes (~1–2 GB/day |
| 71 |
71 |
compressed in ClickHouse). |
| 72 |
72 |
|
| 73 |
73 |
## Replay |
| 76 |
76 |
history with alternative weights. Re-deriving components from raw measurements with alternative engine parameters is an |
| 77 |
77 |
offline job to add when enough history exists (raw is kept 180 days for exactly that purpose). |
| 78 |
78 |
|
| 79 |
|
−## Retention |
|
79 |
+## Retention — we never erase (founder decision, 2026-09-12, migration `0002_keep_forever.sql`) |
| 80 |
80 |
|
| 81 |
|
−measurements 180 d · measurements_1m 1 y · traceroutes 1 y · bgp_events 3 d (1/25 sampled announcements, all withdrawals) · |
| 82 |
|
−bgp_stats_10s ∞ · bgp_origin_1m 30 d · pressure_history ∞ · signal_features 1 y · probe_health 180 d · engine_runs 90 d. |
|
81 |
+Every table is kept indefinitely: measurements, measurements_1m, traceroutes, probe_health, bgp_stats_10s, bgp_origin_1m, |
|
82 |
+pressure_history, signal_features, engine_runs. The single exception is `bgp_events`, the *raw* RIS Live stream (≈ 10 k |
|
83 |
+prefixes/s at the source, already sampled 1/25 for announcements): 90 days. Nightly backups (`deploy/bin/backup.sh`) copy |
|
84 |
+Postgres and the irreplaceable ClickHouse tables off-node to BHS128. Disk growth ≈ 1–2 GB/day on BHS64b's 878 GB RAID-1; |
|
85 |
+plan a second ClickHouse volume (or a cold tier on BHS128) within a year. |
| 83 |
86 |
|