# Deployment

# Production topology (2026-09-10)

text

GoDaddy DNS  llm-api.io  A www → 51.161.112.61 (BHS64)
BHS64 (OVH Beauharnois)  Caddy TLS  https://www.llm-api.io → 10.67.0.x:8300 over WireGuard
M1M64 (Mac Studio M1 Max 64 GB, rented, public IP 45.74.241.221, user simon)
   wg1 10.67.0.x
   PM2 (LaunchDaemon)  llm-api-server  → .venv/bin/llm-api serve      127.0.0.1:8300
                        llm-api-web     → next start -p 8301 -H 127.0.0.1
   ~/llm-api/            models/ (SSD library)  data/ (SQLite, .secret, .env)  logs/
   ~/apps/llm-api/       code (rsync'd by mld: server/ + web/)

The whole app is deployed and routed by the cluster orchestrator mld (gateway M1M32):

bash

mld stage ~/Desktop/Cluster/llm-api llm-api        # laptop → M1M32 staging (excludes node_modules, .next, .venv)
mld deploy llm-api --node M1M64                     # rsync → post_sync hooks (uv venv + pip, pnpm build) → PM2 → health → tunnel route → registry
mld status --live | grep llm-api
mld logs llm-api

Manifest: M1M32:~/dispatch/apps/llm-api.json (dir ~/apps/llm-api, port 8300, health /health, tunnel www.llm-api.io on BHS64, pinned to M1M64). Secrets (ADMIN_PASSWORD, HF_TOKEN) live in the manifest's env on M1M32 only and are written into the PM2 environment; the app also reads ~/llm-api/.env.

M1M64 is a reserved node: always deploy with --node M1M64.

# WireGuard relay on M1M64 (managed firewall)

The API binds 127.0.0.1:8300, but Caddy on BHS64 reaches the node at its WireGuard address 10.67.0.40:8300. On this rented Macly Mac the macOS application firewall is MDM-managed (socketfilterfw refuses changes) and blocks Homebrew binaries such as socat from accepting connections on the WireGuard interface (the cluster's usual wg-forward-setup.sh relay therefore fails silently: it listens, but every connection is dropped). The relay is instead an Apple-signed /usr/bin/ssh -L 10.67.0.40:8300:127.0.0.1:8300 simon@127.0.0.1 (key ~/.ssh/id_ed25519_wgfwd, authorized on the node itself), run as the PM2 process llm-api-wgfwd and declared as the third process of the manifest. Verify with ssh BHS64 curl -s http://10.67.0.40:8300/health. It has no graphical session guaranteed at boot, so PM2 runs as a system LaunchDaemon (mld prepare does this).

# Manual (without mld)

bash

./scripts/install.sh
cd ~/llm-api && /path/to/server/.venv/bin/llm-api serve                 # API
cd /path/to/web && pnpm start -p 8301 -H 127.0.0.1                     # console

PM2:

bash

pm2 start /path/to/server/.venv/bin/llm-api --name llm-api-server --cwd ~/llm-api -- serve
pm2 start node --name llm-api-web --cwd /path/to/web -- node_modules/next/dist/bin/next start -p 8301 -H 127.0.0.1
pm2 save && pm2 startup

launchd alternative: a LaunchDaemon plist running .venv/bin/llm-api serve with WorkingDirectory=~/llm-api, KeepAlive=true, RunAtLoad=true (see mld render for the pattern).

# Reverse proxy

Any TLS terminator works (Caddy, nginx). Forward everything to 127.0.0.1:8300; the API serves /v1, /api, /health, /openapi itself and proxies the rest to Next. SSE needs buffering off (X-Accel-Buffering: no is already set). api.llm-api.io can point at the same upstream if a separate API host is wanted.

# Updating

bash

mld stage ~/Desktop/Cluster/llm-api llm-api && mld deploy llm-api --node M1M64

Deploy = rsync + rebuild + pm2 restart; the manager unloads the loaded model on shutdown and clears stale state on start.