1# API_INTEGRATION.md — how WP39 talks to FinData OS23## Authorization4- Header `Authorization: Bearer <key>`; the key is read from `FINDATA_API_KEY` or the file `FINDATA_API_KEY_FILE`5 (default `~/findata-secrets/api-key-wp39.txt`, mode 600). The client never logs headers, never puts the key in a URL,6 and redacts any string starting with `fd_` in exceptions (`src/wp39/api_client.py::_redact`).7- Dedicated key `wp39_uqo` (scopes `read:market, read:fundamentals, query, export`; **no admin**), created 2026-11-02 with8 the node CLI `findata create-key` because `POST /admin/keys` returned HTTP 500 for API-key callers (bug logged in9 DATA_AUDIT §8).10- Compute runs on **M3U96a** against `http://127.0.0.1:8600/api/v1` (same host as the lake; the public URL11 `https://findata.spboucher.ai/api/v1` is used from the laptop for probes only).1213## Client (`src/wp39/api_client.py`)14- `httpx.Client` with 60 s timeout, retry with bounded exponential backoff (5 attempts, 0.5 → 8 s, jitter) on 429/5xx and15 network errors; idempotent GETs only are retried automatically; export creation is retried only when no export id was16 received.17- Every response's `X-Request-ID`, `X-Dataset-Id`, `X-Dataset-Version` are recorded in the provenance log18 (`artifacts/logs/api_calls.jsonl`, one line per call, no payloads).19- Cursor iteration for `/market/bars` and `/coverage/symbols`; export polling (`/exports/{id}`) every 1 s up to 10 min.20- Arrow/Parquet schema validation against `GET /catalog/schema/bars` (schema v1) before any file is accepted.2122## Calls used by the pipeline23| Step | Call | Scope | Volume |24|---|---|---|---|25| 00 probe | `/health`, `/catalog/datasets`, `/catalog/schema/bars`, `/openapi.json` | read | 4 |26| 01 universe | `/coverage/symbols?dataset_id=market_bars.stock.1m.raw` (cursor, 5,000/page), `/instruments/{symbol}` | read | ≈ 4 + N |27| 02 PIT panel | `/fundamentals/{entity}?period_type=q&limit=20000` (all facts, local as-of join), `/fundamentals/{entity}/periods` | read:fundamentals | 2 × N entities |28| 03 features | `POST /exports` (1m raw, ≤ 12 y, 1 symbol) ×2 per symbol, `/exports/{id}`, `/download`; SPY from `market_bars.etf.1m.raw` | export | 2 × N + 2 |29| checks | `POST /query` (cross-section counts, spot checks ≤ 100 k rows) | query | few |3031Exception (documented): earnings-release rows are extracted once on M3U96a with32`psql -d findata -c "\copy (select ... from earnings_events) to '…'"` (read-only). The extract's SHA-256, row count and33timestamp are in `artifacts/manifests/earnings_events.json`.3435## Idempotent partitioning and manifests36- Exports are cached per `(symbol, chunk)` as `$WP39_CACHE/bars/<symbol>__<start>__<end>.parquet` and deleted after the37 feature rows are written; `artifacts/manifests/bars_manifest.jsonl` keeps `symbol, start, end, dataset_id,38 dataset_version, row_count, bytes, sha256, exported_at, request_id`.39- Fundamentals extracts: `$WP39_CACHE/fundamentals/<entity>.parquet` with the same manifest fields.40- Re-running a script skips completed partitions (manifest lookup) and never overwrites a partition with a different41 dataset version without logging a `version_drift` warning.42