# Operations — WP41

# Nodes

• Compute: M3U96b (~/wp41_uqo, venv .venv Python 3.12, torch 2.15 MPS). Cache ~/wp41-cache/{bars,instruments,grids,manifests}.
• Data: M3U96a FinData OS API, reached over the LAN: FINDATA_BASE_URL=http://192.168.2.80:8600/api/v1.
• Laptop: mirror of the repository (~/Desktop/UQO/Recherche/Recherche-UQO/finance/wp41_uqo), tests, LaTeX.

# Environment on the node

export FINDATA_API_KEY_FILE=~/findata-secrets/api-key-wp41.txt      # mode 600, dedicated key (scopes read/query/export)
export FINDATA_BASE_URL=http://192.168.2.80:8600/api/v1
export WP41_RATE=8                                                  # client-side req/s (server limit 600/min)
export WP41_CACHE=~/wp41-cache

# Sync and run

# laptop -> node (code only; caches and results stay on the node, metrics are rsynced back)
rsync -a --exclude .venv --exclude .git --exclude results/logs ./ M3U96b:~/wp41_uqo/
ssh M3U96b 'cd ~/wp41_uqo && uv pip install -q -p .venv/bin/python -e . && .venv/bin/python -m pytest -q'
ssh M3U96b 'cd ~/wp41_uqo && nohup .venv/bin/python scripts/download_pilot.py --config configs/main.yaml --symbols-from all --workers 3 > results/logs/download_main.out 2>&1 &'
ssh M3U96b 'cd ~/wp41_uqo && nohup .venv/bin/python scripts/run_experiment.py --config configs/main.yaml > results/logs/main.out 2>&1 &'
# node -> laptop (aggregate results only)
rsync -a --exclude 'origins_*.csv' --exclude '*.npz' --exclude '*.pt' M3U96b:~/wp41_uqo/results/ results/

# Logs and provenance

• results/logs/provenance_*.jsonl: one line per API call (method, path, status, request id, dataset id/version, ms).
• ~/wp41-cache/manifests/bars_manifest.jsonl: one line per export (dataset version, sha256, rows, request id, time).
• results/metrics/<exp>/<model>/train_log.jsonl: training curves, throughput, memory; last.pt resumes a crashed run.

# Safety

• Never print the key; the client redacts fd_… tokens in exceptions and logs.
• Downloads stop when free disk < 40 GB (min_free_gb). Exports are bounded (≤ 500 symbols, ≤ 12 years per request).
• No production app is touched; the node's PM2 apps (toveli, autoscholar, wp-web) keep running.

# Gotchas

• Segfault when LightGBM is imported after torch (two copies of libomp on macOS): every entry point imports lightgbm first (scripts/run_experiment.py, chronos_x/experiment.py). Verified on M3U96b with torch 2.15 / lightgbm 4.x.
• Detached runs: use scripts/node_run.sh <name> <cmd> (nohup + < /dev/null + disown); a plain ssh host 'nohup … &' was killed when the SSH channel closed.