# Operations runbook

## Where
- Node **M3U96b**, directory `~/apps/wp-spboucher-ai` (git clone, origin spbgit `wp-spboucher-ai.git`). PM2: `wp-web` (Next, :8560) and `wp-worker`.
- Data: `~/data/wp-spboucher-ai` (files/previews/inbox/tmp). Backups: `~/backups/wp-spboucher-ai` (+ rsync `M3U96a:backups/wp-spboucher-ai`).
- Secrets: `~/apps/wp-spboucher-ai/.env` (mode 600); laptop copy `~/Desktop/Cluster/secrets/wp-spboucher-ai/.env`; owner initial password in `owner-initial-password.txt` (must be changed at first login).
- Registry / route: `mld status`, `mld tunnel route wp-spboucher-ai`, manifest `M1M32:~/dispatch/apps/wp-spboucher-ai.json` (copy in `infra/mld/`).

## Everyday
```bash
ssh M3U96b
cd ~/apps/wp-spboucher-ai && pm2 logs wp-web --lines 100 ; pm2 logs wp-worker --lines 100
curl -s localhost:8560/api/health
```
Edit on the node → `pnpm build && pm2 restart wp-web wp-worker` → `git commit && git push` → laptop `git pull`.

## Publishing a new working paper
1. Put the PDF in `~/data/wp-spboucher-ai/inbox/` (or any directory listed in Settings → Ingestion) and click **Scan directories now** in `/admin/ingest` (or `pnpm ingest:scan`).
2. Open the candidate: check integrity, duplicates, proposed title/authors/abstract/number/date; **Create draft paper** (or **Attach version** to an existing paper).
3. Complete the record in `/admin/papers/<id>` (abstract paragraphs, findings, contributions, methodology, data, reproducibility, artifacts, keywords, JEL, topics, methods, datasets, primary domain).
4. **Publish** from the status bar. The worker extracts the full text, builds previews, embeddings and relations. Verify the public page and `/api/v1/papers/<slug>`.

Revisions: upload a new version on the paper page (kept immutable; previous versions remain downloadable unless withdrawn with a reason).

## Schedules (worker, node local time)
00:10 daily statistics · 03:00 relations rebuild · 04:40 encrypted backup (pg_dump + manuscripts; 14-day retention; rsync to M3U96a). Manual: `/admin/system` or `pnpm backup`. Restore drill: `pnpm restore:drill`.

## Restore
```bash
openssl enc -d -aes-256-cbc -pbkdf2 -in ~/backups/wp-spboucher-ai/wp-<stamp>.dump.enc -out /tmp/wp.dump -pass env:BK   # BK = BACKUP_PASSPHRASE
createdb wp_spboucher_restored && pg_restore --no-owner -d wp_spboucher_restored /tmp/wp.dump
openssl enc -d -aes-256-cbc -pbkdf2 -in ~/backups/wp-spboucher-ai/wp-files-<stamp>.tar.enc -out /tmp/files.tar -pass env:BK && tar -xf /tmp/files.tar -C ~/data/wp-spboucher-ai
```

## Rollback
`mld deploy` keeps the previous release; `mld move`/`deploy` roll back automatically on failed health checks. Manual: `git checkout <sha> && pnpm build && pm2 restart wp-web wp-worker` on the node. Migrations are additive; never edit an applied migration (checksums are verified).

## Security notes
Admin: Argon2id passwords (policy ≥ 14 chars), TOTP, `__Host-` session cookie (HttpOnly/Strict/Secure), rotation every 12 h, idle 7 d / absolute 30 d, CSRF double-submit on every mutation, same-origin enforcement, login limiter (5 failures / 15 min, exponential lockout), audit log of every write. Public API: read-only role, 120 req/min per IP (60 for search/files). Uploads: PDF magic + poppler integrity, 150 MB max, content-addressed.

## Known limits
- Semantic search/relations depend on the optional OpenAI key; without it the platform falls back to lexical signals (labelled).
- Rate limiting is in-process (single web process).
- No official UQO logo is used (`site.institution.logo_authorized = false` until permission is confirmed).