# Deploying WebSensor on MacLustr

WebSensor runs as three PM2 processes on one node (M4M64b), behind ngrok (`www.websensor.io`), orchestrated by
`mld` from the M1M32 gateway.

```
Internet → www.websensor.io → ngrok → websensor-api (:8260, Fastify)
                                         ├── /api/v1/*  REST + WebSocket /api/v1/live
                                         └── /*         reverse proxy → websensor-web (Next.js :8261, loopback)
websensor-engine (scheduler + pipeline, Prometheus :8262) → PostgreSQL 17 `websensor` + Redis (loopback)
Blob store: ~/websensor-data/blobs (content-addressed, zstd) — never synced by mld (excluded `data/`).
```

## First deployment
1. Copy `deploy/websensor.mld.json.example` to `M1M32:~/dispatch/apps/websensor.json` and fill the two secrets
   (`ANTHROPIC_API_KEY`, `SCRAPFLY_API_KEY`). The manifest is `0600` on the gateway; a gitignored local copy
   may live in `deploy/websensor.mld.json`.
2. `~/Desktop/cluster-skill/mld stage ~/Desktop/Projets/apps-web/websensor websensor`
3. `~/Desktop/cluster-skill/mld deploy websensor --node M4M64b`
   Post-sync hooks: `pnpm install`, `createdb` + migrations, registry sync, `next build`. Health: `/api/ready`
   (database + engine activity in the last 15 minutes), public `https://www.websensor.io/api/ready`.
4. Verify: `mld status --live`, `curl https://www.websensor.io/api/v1/stats`, and a WebSocket handshake
   (`npx wscat -c wss://www.websensor.io/api/v1/live`).

## DNS / ngrok
- `www.websensor.io` is a reserved ngrok domain; the CNAME at GoDaddy points to the ngrok `*.ngrok-cname.com`
  target. The apex `websensor.io` should redirect to `www` (GoDaddy forwarding, or an A record to ngrok + the
  gateway's 301 on the apex host).
- WebSocket upgrades traverse ngrok without configuration; the API sends a heartbeat every 25 s so idle
  connections survive ngrok's timeouts.

## Operations
- Logs: `mld logs websensor` or `ssh M4M64b pm2 logs websensor-engine`.
- Registry edits: change `config/sources.yaml`, redeploy (sync runs in post-sync) or
  `ssh M4M64b "cd ~/apps/websensor && node node_modules/tsx/dist/cli.mjs apps/engine/src/cli.ts sync"`.
  Seed sensors removed from the YAML are disabled automatically; discovery-created sensors are kept.
- Discovery: `cli.ts discover [sourceId…]`; `cli.ts probe <domain>` for a dry run.
- Metrics: engine `http://127.0.0.1:8262/metrics`, API `/api/metrics` (Prometheus text). Public health dashboard:
  `/health` on the site, `/api/v1/health/connectors`.
- Budgets: `WS_LLM_DAILY_CALL_BUDGET` (Claude calls/day), `WS_SCRAPFLY_DAILY_BUDGET` (Scrapfly calls/day, ~30
  credits each with anti-bot). Both counters reset at UTC midnight and are visible in `metrics_daily`.
- Backups: `pg_dump websensor` + `~/websensor-data/blobs` (content-addressed; safe to rsync incrementally).
- Retention: `sensor_runs` unchanged/304 rows are pruned after 14 days, all runs after 60 days; snapshots, changes
  and events are kept.