SPB Git forge
spboucher.ai ↗

spb/satelliteindex

Public
3commits 1branches 0releases
417.0 KBsize
maindefault branch
10 days agolast push
TypeScript 66.5% Python 30.9% JavaScript 1.4% CSS 0.7%
5.0 KB
View source Raw Blame History

# SatelliteIndex.io — project guide (condensed from the founding spec, see docs/SPEC-summary.md)

Mission: the definitive public intelligence layer for everything operating, launched, proposed, licensed, decaying or changing in Earth orbit — Bloomberg Terminal × FlightRadar24 × Crunchbase for orbital infrastructure. Not a simple satellite tracker.

# Architecture (never couple the website to third-party sources)

text
External sources → connector workers → raw snapshots (SI_DATA_DIR/raw, gzip + raw_records) → normalization → entity resolution
→ canonical Postgres → derived analytics (matviews, stats_snapshots, search_index, events) → FastAPI /api/v1 → Next.js
  • Backend src/satelliteindex/ (Python 3.12, FastAPI, SQLAlchemy Core + asyncpg, Alembic, sgp4, APScheduler, Redis).
    • connectors/ framework (BaseConnector.run() = bookkeeping, hashing, circuit breaker; execute() per connector) — CelesTrak GP (celestrak_gp), CelesTrak groups (celestrak_groups), SATCAT (celestrak_satcat), derived_analytics. Registry: connectors/__init__.py.
    • orbital/ SGP4 (vectorised SatrecArray), TEME→geodetic, orbit classification (documented metric orbit_class).
    • services/ entity resolution (SatIndex, NORAD → COSPAR → name; never auto-merge ambiguous), classification (curated registry/constellations.yaml), positions service (in-memory propagator, 30 s cache), events (dedupe keys), Redis cache/locks.
    • api/routers/ satellites, search, stats/rankings/density, orbit positions, constellations, operators, countries, launches/sites/debris/reentries, events, misc (sources, status, methodology, sitemap, view beacon), admin (x-si-admin-token).
    • CLI si: migrate | seed | run <connector> [--file group=path] | status | stats | position <norad> | schedule | api | backup.
  • Frontend apps/web/ (Next 16, Tailwind v4, Three/R3F globe). Guide: docs/FRONTEND-GUIDE.md.
  • Database: migrations/versions/0001_initial.py (plain SQL). Internal ids are prefixed ULIDs (sat_…); NORAD/COSPAR are source identifiers. Orbital history is append-only (orbital_elements), orbital_state = latest per object. Materialized views refresh hourly.

# Hard rules (from the spec)

  • No hardcoded satellite counts, source results, timestamps, fake analytics or placeholder charts. Missing → "Unavailable".
  • Never overwrite orbital history, discard provenance, merge ambiguous satellites silently, or delete entities because a source went quiet (empty/short upstream responses are suspicious, not "no data").
  • Never expose credentials (Space-Track, DB, Scrapfly…) or internal cluster addresses. Admin behind token. Rate limits on search/positions.
  • Derived analytics (orbit class, mission type, constellation membership, activity score, orbital density) are labelled derived with a documented, versioned methodology (metric_definitions, /methodology). Never present them as safety metrics. Never invent collision probabilities or precise reentry locations.
  • Mobile-first (390/430/768/1440), no horizontal overflow, ≥ 44 px targets, DOM order = visual order. Design: premium, scientific, dark, data-dense — no crypto-dashboard look, no wall of rounded cards.
  • Units: km, km/s, kg, degrees, minutes, UTC. ISO 3166 country codes.
  • Every feature: real data, error + loading states, mobile + desktop, source attribution, performance, accessibility, logging, tests, deployment.

# Local development

text
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -e ".[dev]"   # backend deps
cp .env.example .env            # local Postgres `satelliteindex`, Redis db 4
.venv/bin/si migrate && .venv/bin/si seed
.venv/bin/si run celestrak_satcat && .venv/bin/si run celestrak_gp && .venv/bin/si run celestrak_groups && .venv/bin/si run derived_analytics
.venv/bin/si api                # http://127.0.0.1:8311 (docs at /api/v1/docs)
pnpm install && pnpm dev:web    # http://localhost:8310
pytest                          # tests/ (fixtures in tests/fixtures, no live network by default)

CelesTrak blocks a second download of the same GP group within 2 h (HTTP 403 "has not updated") — the connector treats it as unchanged; si run celestrak_gp --file active=/path/active.json ingests a local snapshot.

# Production

Since 2026-09-14: OVH server BHS128b (ssh BHS128b, Ubuntu 24.04), outside mld: PM2 under systemd (~/apps/pm2.satelliteindex.config.cjs, generated from the mld manifest kept in ~/apps/.manifests/) — satelliteindex-api (uvicorn 127.0.0.1:8311), satelliteindex-scheduler (si schedule), satelliteindex-web (Next :8310). Postgres 17 + Redis native; code ~/apps/satelliteindex (uv Python 3.12 venv), data in ~/satelliteindex-data/. Public route: MacLustr Tunnel (BHS64 Caddy) https://www.satelliteindex.io → wg1 10.67.0.62:8310, apex redirected. Release = rsync + uv pip install -e . + si migrate + web build + pm2 restart (exact command in docs/DEPLOY.md). Former home: M2M32b via mld (manifest deploy/satelliteindex.mld.json), kept as a cold copy.