# Operator runbook

# Daily checks

GET /api/health → db: ok, redis: ok, stream.connected: true, stream.last_msg_age < 10.
/settings → Kraken status online, clock drift < 2 s; /market → data-quality badges green (lag_bars 0–1).
pm2 ls on the node: crypto-trade-bot-api|web|worker online, restarts not climbing.

# Common operations

Task	Command
Apply migrations	`ktrade migrate`
One-shot OHLC pull	`ktrade backfill`
Probe key permissions (never trades)	`ktrade preflight` or Settings → Preflight
Change owner password	`ktrade set-owner-password` → put hash in env → restart api
Verify audit chain	`ktrade audit-verify`
Kill switch	Risk Center → toggle (audited) — blocks all new orders; cancels eligible open orders in Phase 4/5
Add a pair to the universe	edit `KTRADE_UNIVERSE`, restart worker (pairs refresh immediately)

# Failure modes

WS disconnected / checksum mismatches climbing: worker reconnects with backoff; if persistent, restart the worker; check Kraken status page. Backtests are unaffected (REST data).
Stale candles (lag_bars > 2): REST backfill failing → see worker logs ingest.backfill.error; rate-limit waits are logged as kraken.ratelimit.wait.
Postgres down: API health reports db: error; gateway defaults to deny; restart postgresql@17.
Exchange outage: kill switch still blocks new orders locally; cancellation of resting exchange orders cannot be guaranteed while the exchange is unreachable (Phase 5 note).

# Backups & retention

Postgres: nightly pg_dump ktrade (add to PM2 cron on the node); candles are re-accumulable only forward in time — back up candles to keep history.
Retention: trades 30 d, book samples 14 d (worker retention_loop).

# Deployment (MacLustr)

Manifest M1M32:~/dispatch/apps/crypto-trade-bot.json; mld deploy crypto-trade-bot (post_sync builds venv + web); route https://crypto-trade-bot.app → <node>:8471 on BHS64. Remote-first: edit on the node, git push to spbgit.