SPB Git

spb/ai-atlas

Public
3.8 KB

# Deploying AI Atlas on MacLustr

Production runs on M2M32c (Mac Studio M2, 12 c / 32 GB, dedicated) behind the MacLustr Tunnel (BHS64 Caddy → wg1 10.67.0.20). Everything goes through the gateway M1M32 and mld (~/Desktop/cluster-skill/mld).

# Topology

text
Internet → GoDaddy DNS (A www.ai-atlas.co + A ai-atlas.co → 51.161.112.61)
        → BHS64 Caddy (TLS Let's Encrypt, apex → www redirect)  →  WireGuard wg1  →  M2M32c:8320 (Next.js)
                                                                                      └─ rewrite /api/v1/* → 127.0.0.1:8321 (FastAPI, 2 uvicorn workers)
                                                                                      └─ ai-atlas-scheduler (aia schedule: connectors + jobs worker + backups)
                                                                                      └─ Postgres 17 + pgvector (db aiatlas) · Redis db 5 · ~/ai-atlas-data
        LLM factory: https://www.llm-api.io/v1 (MacLustr llm-api on M1M64, key "ai-atlas") — optional, extraction degrades gracefully

Node prerequisites (done 2026-09-11 by mld bootstrap + mld prepare M2M32c + Homebrew): node 26, pnpm, pm2 (LaunchAgent), uv + Python 3.12, postgresql@17, pgvector, redis as brew services, role/db aiatlas (password aiatlas, localhost only), extensions vector, pg_trgm, uuid-ossp.

# Release procedure (from the laptop)

bash
cd ~/Desktop/Projets/apps-web/ai-atlas
.venv/bin/pytest -q && (cd apps/web && pnpm typecheck)              # 1. tests + typecheck
deploy/render-manifest.sh --push                                     # 2. manifest with AIA_ADMIN_TOKEN + AIA_LLM_API_KEY → M1M32:~/dispatch/apps/ai-atlas.json
~/Desktop/cluster-skill/mld stage ~/Desktop/Projets/apps-web/ai-atlas ai-atlas   # 3. laptop → gateway staging (sync_excludes)
~/Desktop/cluster-skill/mld deploy ai-atlas --node M2M32c            # 4. rsync → node, hooks (venv, migrate, seed, pnpm build), PM2, health, tunnel route, registry
ssh M2M32c 'cd ~/apps/ai-atlas && nohup bash deploy/first-run.sh > ~/ai-atlas-data/logs/first-run.log 2>&1 &'   # 5. first time: initial corpus
curl -sI https://www.ai-atlas.co | head -1                           # 6. public check

mld deploy re-points https://www.ai-atlas.co → M2M32c:8320 on BHS64 and runs the public health check. mld heal (every 5 min on M1M32) restarts anything missing after a power cut. Secrets: deploy/.admin-token, deploy/.llm-key (git-ignored); rendered manifest in deploy/rendered/.

# Operations

bash
~/Desktop/cluster-skill/mld status | grep ai-atlas
~/Desktop/cluster-skill/mld logs ai-atlas
ssh M2M32c 'cd ~/apps/ai-atlas && AIA_LOG_JSON=0 .venv/bin/aia status'         # connector table + queue depth
ssh M2M32c 'cd ~/apps/ai-atlas && AIA_LOG_JSON=0 .venv/bin/aia run huggingface --force'
ssh M2M32c 'cd ~/apps/ai-atlas && .venv/bin/aia backup'                         # pg_dump → ~/ai-atlas-data/backups (nightly 04:40 too)
scripts/backup-offnode.sh                                                       # copies dumps + raw archive to M1M32:~/backups/ai-atlas
curl -s https://www.ai-atlas.co/api/v1/stats | python3 -m json.tool | head      # live counters

Admin dashboard: https://www.ai-atlas.co/admin (token = deploy/.admin-token). Environment variables the processes read are listed in .env.example.

# Scaling

  • More extraction throughput: run aia worker on other nodes with DATABASE_URL pointing at M2M32c (Postgres must then listen on the LAN — not enabled by default) or run additional aia worker --kind llm_extract processes on M2M32c.
  • The LLM factory can point at any OpenAI-compatible server (AIA_LLM_BASE_URL), e.g. a second llm-api node.
  • Raw archive grows with the number of changed snapshots (gzip, content-addressed). Disk: 423 GB free on M2M32c at launch.
Files · e6b6b4aeb02f00c4c6493363aecc51e3a804d968
Menu

Theme