# Deploying WebSensor on MacLustr

WebSensor runs as three PM2 processes on one node (M4M64b), behind ngrok (www.websensor.io), orchestrated by mld from the M1M32 gateway.

Internet → www.websensor.io → ngrok → websensor-api (:8260, Fastify)
                                         ├── /api/v1/*  REST + WebSocket /api/v1/live
                                         └── /*         reverse proxy → websensor-web (Next.js :8261, loopback)
websensor-engine (scheduler + pipeline, Prometheus :8262) → PostgreSQL 17 `websensor` + Redis (loopback)
Blob store: ~/websensor-data/blobs (content-addressed, zstd) — never synced by mld (excluded `data/`).

# First deployment

1. Copy deploy/websensor.mld.json.example to M1M32:~/dispatch/apps/websensor.json and fill the two secrets (ANTHROPIC_API_KEY, SCRAPFLY_API_KEY). The manifest is 0600 on the gateway; a gitignored local copy may live in deploy/websensor.mld.json.
2. ~/Desktop/cluster-skill/mld stage ~/Desktop/Projets/apps-web/websensor websensor
3. ~/Desktop/cluster-skill/mld deploy websensor --node M4M64b Post-sync hooks: pnpm install, createdb + migrations, registry sync, next build. Health: /api/ready (database + engine activity in the last 15 minutes), public https://www.websensor.io/api/ready.
4. Verify: mld status --live, curl https://www.websensor.io/api/v1/stats, and a WebSocket handshake (npx wscat -c wss://www.websensor.io/api/v1/live).

# DNS / ngrok

• www.websensor.io is a reserved ngrok domain; the CNAME at GoDaddy points to the ngrok *.ngrok-cname.com target. The apex websensor.io should redirect to www (GoDaddy forwarding, or an A record to ngrok + the gateway's 301 on the apex host).
• WebSocket upgrades traverse ngrok without configuration; the API sends a heartbeat every 25 s so idle connections survive ngrok's timeouts.

# Operations

• Logs: mld logs websensor or ssh M4M64b pm2 logs websensor-engine.
• Registry edits: change config/sources.yaml, redeploy (sync runs in post-sync) or ssh M4M64b "cd ~/apps/websensor && node node_modules/tsx/dist/cli.mjs apps/engine/src/cli.ts sync". Seed sensors removed from the YAML are disabled automatically; discovery-created sensors are kept.
• Discovery: cli.ts discover [sourceId…]; cli.ts probe <domain> for a dry run.
• Metrics: engine http://127.0.0.1:8262/metrics, API /api/metrics (Prometheus text). Public health dashboard: /health on the site, /api/v1/health/connectors.
• Budgets: WS_LLM_DAILY_CALL_BUDGET (Claude calls/day), WS_SCRAPFLY_DAILY_BUDGET (Scrapfly calls/day, ~30 credits each with anti-bot). Both counters reset at UTC midnight and are visible in metrics_daily.
• Backups: pg_dump websensor + ~/websensor-data/blobs (content-addressed; safe to rsync incrementally).
• Retention: sensor_runs unchanged/304 rows are pruned after 14 days, all runs after 60 days; snapshots, changes and events are kept.