1# Operations — WP4123## Nodes4- **Compute: M3U96b** (`~/wp41_uqo`, venv `.venv` Python 3.12, torch 2.15 MPS). Cache `~/wp41-cache/{bars,instruments,grids,manifests}`.5- **Data: M3U96a** FinData OS API, reached over the LAN: `FINDATA_BASE_URL=http://192.168.2.80:8600/api/v1`.6- **Laptop**: mirror of the repository (`~/Desktop/UQO/Recherche/Recherche-UQO/finance/wp41_uqo`), tests, LaTeX.78## Environment on the node9```bash10export FINDATA_API_KEY_FILE=~/findata-secrets/api-key-wp41.txt # mode 600, dedicated key (scopes read/query/export)11export FINDATA_BASE_URL=http://192.168.2.80:8600/api/v112export WP41_RATE=8 # client-side req/s (server limit 600/min)13export WP41_CACHE=~/wp41-cache14```1516## Sync and run17```bash18# laptop -> node (code only; caches and results stay on the node, metrics are rsynced back)19rsync -a --exclude .venv --exclude .git --exclude results/logs ./ M3U96b:~/wp41_uqo/20ssh M3U96b 'cd ~/wp41_uqo && uv pip install -q -p .venv/bin/python -e . && .venv/bin/python -m pytest -q'21ssh M3U96b 'cd ~/wp41_uqo && nohup .venv/bin/python scripts/download_pilot.py --config configs/main.yaml --symbols-from all --workers 3 > results/logs/download_main.out 2>&1 &'22ssh M3U96b 'cd ~/wp41_uqo && nohup .venv/bin/python scripts/run_experiment.py --config configs/main.yaml > results/logs/main.out 2>&1 &'23# node -> laptop (aggregate results only)24rsync -a --exclude 'origins_*.csv' --exclude '*.npz' --exclude '*.pt' M3U96b:~/wp41_uqo/results/ results/25```2627## Logs and provenance28- `results/logs/provenance_*.jsonl`: one line per API call (method, path, status, request id, dataset id/version, ms).29- `~/wp41-cache/manifests/bars_manifest.jsonl`: one line per export (dataset version, sha256, rows, request id, time).30- `results/metrics/<exp>/<model>/train_log.jsonl`: training curves, throughput, memory; `last.pt` resumes a crashed run.3132## Safety33- Never print the key; the client redacts `fd_…` tokens in exceptions and logs.34- Downloads stop when free disk < 40 GB (`min_free_gb`). Exports are bounded (≤ 500 symbols, ≤ 12 years per request).35- No production app is touched; the node's PM2 apps (toveli, autoscholar, wp-web) keep running.3637## Gotchas38- **Segfault when LightGBM is imported after torch** (two copies of libomp on macOS): every entry point imports `lightgbm` first (`scripts/run_experiment.py`, `chronos_x/experiment.py`). Verified on M3U96b with torch 2.15 / lightgbm 4.x.39- Detached runs: use `scripts/node_run.sh <name> <cmd>` (nohup + `< /dev/null` + disown); a plain `ssh host 'nohup … &'` was killed when the SSH channel closed.40