SPB Git forge
3commits 1branches 0releases
417.0 KBsize
maindefault branch
10 days agolast push
TypeScript 66.5% Python 30.9% JavaScript 1.4% CSS 0.7%
5.3 KB

# Deploying SatelliteIndex on MacLustr

Since 2026-09-14 production runs on the OVH server BHS128b (ssh BHS128b, ubuntu@51.161.112.85, Ubuntu 24.04, 16 threads / 128 GB), outside mld: code in ~/apps/satelliteindex (venv .venv = uv Python 3.12, uv pip install -e .), Postgres 17 native (db/role satelliteindex/satelliteindex, pg_trgm + uuid-ossp, en_US.UTF-8), Redis native db 4 (facet cache, rebuilt on the fly), data in ~/satelliteindex-data, PM2 under systemd (pm2-ubuntu) with ~/apps/pm2.satelliteindex.config.cjs generated from the mld manifest by ~/apps/.manifests/gen-pm2.py (secrets, 0600). Tunnel route: mlt add www.satelliteindex.io BHS128b:8310 (BHS64 Caddy → wg1 10.67.0.62); firewall ufw allow in on wg1 from 10.67.0.1 to any port 8310 proto tcp. Release on BHS128b: rsync -az --exclude .venv --exclude node_modules --exclude .git --exclude 'apps/web/.next' --exclude data . BHS128b:apps/satelliteindex/ then ssh BHS128b 'cd apps/satelliteindex && ~/.local/bin/uv pip install --python .venv/bin/python -e . && DATABASE_URL=postgresql+asyncpg://satelliteindex:satelliteindex@127.0.0.1:5432/satelliteindex SI_DATA_DIR=$HOME/satelliteindex-data .venv/bin/si migrate && pnpm install --frozen-lockfile && (cd apps/web && API_URL=http://127.0.0.1:8311 NEXT_PUBLIC_SITE_URL=https://www.satelliteindex.io pnpm build) && pm2 restart satelliteindex-api satelliteindex-scheduler satelliteindex-web && pm2 save'. The M2M32b copy (code, Postgres satelliteindex, ~/satelliteindex-data) is kept as a cold copy, retired from mld (--keep-dir).

Production formerly ran on M2M32b (Mac Studio M2 Max, 12 c / 32 GB, dedicated) behind the MacLustr Tunnel (BHS64 Caddy → wg1 10.67.0.19). Everything goes through the gateway M1M32 and mld (see ~/Desktop/cluster-skill/mld).

# Topology

text
Internet → GoDaddy DNS (A www.satelliteindex.io → 51.161.112.61)
        → BHS64 Caddy (TLS Let's Encrypt, HTTP→HTTPS)  →  WireGuard wg1  →  M2M32b:8310 (Next.js)
                                                                              └─ rewrite /api/v1/* → 127.0.0.1:8311 (FastAPI, 2 uvicorn workers)
                                                                              └─ satelliteindex-scheduler (si schedule: connectors + backups)
                                                                              └─ Postgres 17 (Homebrew, db satelliteindex) · Redis db 4

Node prerequisites (done 2026-09-11 by mld prepare M2M32b + Homebrew): node 25, pnpm, pm2 (LaunchAgent), uv + Python 3.12, postgresql@17 and redis as brew services, role/db satelliteindex (password satelliteindex, localhost only), extensions pg_trgm, uuid-ossp.

# Release procedure (from the laptop)

bash
cd ~/Desktop/Projets/apps-web/satelliteindex
pytest -q && (cd apps/web && pnpm typecheck)                       # 1. tests + typecheck
deploy/render-manifest.sh --push                                     # 2. manifest with the real SI_ADMIN_TOKEN → M1M32:~/dispatch/apps/satelliteindex.json
~/Desktop/cluster-skill/mld stage ~/Desktop/Projets/apps-web/satelliteindex satelliteindex   # 3. laptop → gateway staging (respects sync_excludes)
~/Desktop/cluster-skill/mld deploy satelliteindex --node M2M32b     # 4. rsync → node, post_sync hooks (venv, migrate, seed, pnpm build), PM2, health, tunnel route, registry
ssh M2M32b 'cd ~/apps/satelliteindex && bash deploy/first-run.sh'   # 5. first time only: initial ingestion (SATCAT, GP, groups, analytics)
curl -sI https://www.satelliteindex.io | head -1                     # 6. public check

mld deploy re-points the Caddy route https://www.satelliteindex.io → M2M32b:8310 on BHS64 automatically and runs the public health check (/api/v1/health). mld heal (every 5 min on M1M32) restarts anything missing after a power cut.

Secrets: only SI_ADMIN_TOKEN (laptop copy deploy/.admin-token, git-ignored; rendered manifest in deploy/rendered/, git-ignored). Never commit .env.

# Operations

bash
~/Desktop/cluster-skill/mld status | grep satelliteindex             # where / online
~/Desktop/cluster-skill/mld logs satelliteindex                      # PM2 logs
ssh M2M32b 'cd ~/apps/satelliteindex && SI_LOG_JSON=0 .venv/bin/si status'          # connector table
ssh M2M32b 'cd ~/apps/satelliteindex && .venv/bin/si run celestrak_gp --force'      # manual run (or /admin → Run now)
ssh M2M32b 'cd ~/apps/satelliteindex && .venv/bin/si backup'         # pg_dump → ~/satelliteindex-data/backups (also nightly 04:40 by the scheduler)

Restore: pg_restore -d satelliteindex_restore --clean --if-exists <dump> into a fresh database, then point DATABASE_URL at it. Off-node copies of the nightly dumps: scripts/backup-offnode.sh (rsync to M1M32:~/backups/satelliteindex/).

# Apex domain

satelliteindex.io has no A record yet. When A @ → 51.161.112.61 exists at GoDaddy, set "redirects": ["satelliteindex.io"] in the manifest tunnel block, deploy/render-manifest.sh --push, then mld tunnel route satelliteindex — Caddy will redirect the apex to www.

# Scaling later

  • A second node can run satelliteindex-scheduler / workers: connector runs are guarded by Redis locks (si:lock:connector:<name>).
  • Workers on Linux (OVH) are possible: the backend has no macOS dependency; Docker files are not provided yet (PM2 is the MacLustr standard).