# Deploying Market Atlas on MacLustr

Since 2026-09-13 production runs on the OVH server BHS128 (ssh BHS128, ubuntu@51.161.112.69, Ubuntu 24.04, 12 threads / 128 GB), outside mld: code in ~/apps/market-atlas, Postgres 17 native (db/role market_atlas, en_US.UTF-8), data in ~/market-atlas-data, PM2 under systemd (pm2-ubuntu, LimitNOFILE 1 048 576) with ~/apps/pm2.market-atlas.config.cjs generated from the mld manifest by ~/apps/.manifests/gen-pm2.py (secrets, 0600). Tunnel route: mlt add www.market-atlas.co BHS128:8380 --websocket (BHS64 Caddy → wg1 10.67.0.60). Firewall: ufw allow in on wg1 from 10.67.0.1 to any port 8380 proto tcp. The apps were moved because the LAN uplink (Bell) could not sustain the cluster's outbound connection count. Release on BHS128: rsync -az --exclude node_modules --exclude .git --exclude 'apps/web/.next' --exclude 'apps/api/dist' --exclude data . BHS128:apps/market-atlas/ then ssh BHS128 'cd apps/market-atlas && pnpm install --frozen-lockfile && pnpm --filter @market-atlas/api build && (cd apps/web && API_URL=http://127.0.0.1:8380 NEXT_PUBLIC_SITE_URL=https://www.market-atlas.co pnpm build) && pm2 restart market-atlas-core market-atlas-web && pm2 save'. pnpm ≥ 10 needs onlyBuiltDependencies (esbuild) in pnpm-workspace.yaml, otherwise ERR_PNPM_IGNORED_BUILDS on a fresh install. The M2U64 copy (~/apps/market-atlas, ~/market-atlas-data, Postgres market_atlas) was deleted on 2026-09-14: BHS128 holds the only production data — schedule an off-host copy of ~/market-atlas-data/backups and archive/. The mld procedure below is kept for reference (manifest backup: laptop ~/Desktop/Cluster/secrets/mld-manifests-retired-20260914/market-atlas.json).

Production formerly ran on M2U64 (Mac Studio M2 Ultra, 24 c / 64 GB) behind the MacLustr Tunnel (BHS64 Caddy → WireGuard wg1 10.67.0.12). Everything goes through the gateway M1M32 and mld (~/Desktop/cluster-skill/mld).

Internet → GoDaddy DNS (A www.market-atlas.co → 51.161.112.61)
        → BHS64 Caddy (TLS, WebSocket passthrough) → wg1 → M2U64:8380  market-atlas-core (Fastify edge: /v1 REST + WS + SSE, workers, connectors)
                                                                    └─ proxies every non-/v1 path → 127.0.0.1:8382  market-atlas-web (Next.js)
                                                                    └─ Postgres 17 Homebrew (db/role market_atlas) · ~/market-atlas-data (raw, archive, backups, logs)

Node prerequisites (present on M2U64 on 2026-09-12): node 25, pnpm 9, pm2, postgresql@17 (brew service), role/db market_atlas (password market_atlas, localhost only). Data dirs: ~/market-atlas-data/{raw,archive,backups,logs,cache}.

# Release procedure (from the laptop)

cd ~/Desktop/Projets/apps-web/market-atlas
pnpm typecheck && pnpm test                                          # 1. types + 58+ unit/connector/consensus/calendar tests
deploy/render-manifest.sh --push                                     # 2. manifest with MA_ADMIN_TOKEN + HFMD_API_KEY → M1M32:~/dispatch/apps/market-atlas.json
~/Desktop/cluster-skill/mld stage ~/Desktop/Projets/apps-web/market-atlas market-atlas   # 3. laptop → gateway staging (sync_excludes)
~/Desktop/cluster-skill/mld deploy market-atlas --node M2U64        # 4. rsync → node, hooks (pnpm install, esbuild, migrate+seed, next build), PM2, health, tunnel route, registry
curl -s https://www.market-atlas.co/v1/health                        # 5. public check

The post_sync hooks build the API bundle (apps/api/dist), run migrations + seeds and build the web app. main.js also migrates and seeds at start (idempotent). mld heal (every 5 min on M1M32) restarts anything missing after a power cut and keeps the Caddy route pointing at the current node.

# Operations

~/Desktop/cluster-skill/mld status | grep market-atlas
ssh M2U64 'pm2 logs market-atlas-core --lines 100 --nostream'
ssh M2U64 'cd ~/apps/market-atlas && export PATH=/opt/homebrew/opt/postgresql@17/bin:/opt/homebrew/bin:$PATH DATABASE_URL=postgres://market_atlas:market_atlas@127.0.0.1:5432/market_atlas MA_DATA_DIR=$HOME/market-atlas-data && node apps/api/dist/cli.js status'
ssh M2U64 '… node apps/api/dist/cli.js run cboe-delayed-quotes'      # one poll, printed
ssh M2U64 '… node apps/api/dist/cli.js replay 2026-09-12'            # re-normalize archived raw payloads offline
ssh M2U64 '… node apps/api/dist/cli.js backup'                       # pg_dump of metadata → ~/market-atlas-data/backups (also nightly ~04:00)
curl -s https://www.market-atlas.co/v1/stats | python3 -m json.tool  # live counters

Admin console: https://www.market-atlas.co/admin (token = deploy/.admin-token). Connector actions (pause/resume/restart/test), schema-change acknowledgements, storage/partitions, discovery form. Environment variables are listed in .env.example.

Retention: observations partitions older than MA_OBSERVATION_RETENTION_DAYS (45) are exported to ~/market-atlas-data/archive/<year>/…ndjson.gz then dropped (index in observation_archives). Bars, events, filings, quotes are permanent. Off-node copies of backups/ and archive/ should be scheduled to M1M32 (rsync -a M2U64:~/market-atlas-data/backups/ M1M32:~/backups/market-atlas/).

Apex domain: add A market-atlas.co → 51.161.112.61 at GoDaddy, then set "redirects": ["market-atlas.co"] in the manifest and redeploy.