# Deployment

# Production topology (2026-09-10)

text

GoDaddy DNS  llm-api.io  A www → 51.161.112.61 (BHS64)
BHS64 (OVH Beauharnois)  Caddy TLS  https://www.llm-api.io → 10.67.0.x:8300 over WireGuard
M1M64 (Mac Studio M1 Max 64 GB, rented, public IP 45.74.241.221, user simon)
   wg1 10.67.0.x
   PM2 (LaunchDaemon)  llm-api-server  → .venv/bin/llm-api serve      127.0.0.1:8300
                        llm-api-web     → next start -p 8301 -H 127.0.0.1
   ~/llm-api/            models/ (SSD library)  data/ (SQLite, .secret, .env)  logs/
   ~/apps/llm-api/       code (rsync'd by mld: server/ + web/)

The whole app is deployed and routed by the cluster orchestrator mld (gateway M1M32):

bash

mld stage ~/Desktop/Cluster/llm-api llm-api        # laptop → M1M32 staging (excludes node_modules, .next, .venv)
mld deploy llm-api --node M1M64                     # rsync → post_sync hooks (uv venv + pip, pnpm build) → PM2 → health → tunnel route → registry
mld status --live | grep llm-api
mld logs llm-api

Manifest: M1M32:~/dispatch/apps/llm-api.json (dir ~/apps/llm-api, port 8300, health /health, tunnel www.llm-api.io on BHS64, pinned to M1M64). Secrets (ADMIN_PASSWORD, HF_TOKEN) live in the manifest's env on M1M32 only and are written into the PM2 environment; the app also reads ~/llm-api/.env.

M1M64 is a reserved node: always deploy with --node M1M64. It has no graphical session guaranteed at boot, so PM2 runs as a system LaunchDaemon (mld prepare does this).

# Manual (without mld)

bash

./scripts/install.sh
cd ~/llm-api && /path/to/server/.venv/bin/llm-api serve                 # API
cd /path/to/web && pnpm start -p 8301 -H 127.0.0.1                     # console

PM2:

bash

pm2 start /path/to/server/.venv/bin/llm-api --name llm-api-server --cwd ~/llm-api -- serve
pm2 start node --name llm-api-web --cwd /path/to/web -- node_modules/next/dist/bin/next start -p 8301 -H 127.0.0.1
pm2 save && pm2 startup

launchd alternative: a LaunchDaemon plist running .venv/bin/llm-api serve with WorkingDirectory=~/llm-api, KeepAlive=true, RunAtLoad=true (see mld render for the pattern).

# Reverse proxy

Any TLS terminator works (Caddy, nginx). Forward everything to 127.0.0.1:8300; the API serves /v1, /api, /health, /openapi itself and proxies the rest to Next. SSE needs buffering off (X-Accel-Buffering: no is already set). api.llm-api.io can point at the same upstream if a separate API host is wanted.

# Updating

bash

mld stage ~/Desktop/Cluster/llm-api llm-api && mld deploy llm-api --node M1M64

Deploy = rsync + rebuild + pm2 restart; the manager unloads the loaded model on shutdown and clears stale state on start.

spb/llm-api

# Deployment

# Production topology (2026-09-10)

# Manual (without mld)

# Reverse proxy

# Updating

Theme