SPB Git

spb/findata

Public
5.0 KB · 59 lines markdown
1# Using FinData OS from a Claude Code session (or any shell)23**Key**: `~/Desktop/Cluster/secrets/findata/api-key-claude-code.txt` (scopes: read:market, read:fundamentals, query, export, admin;4created 2026-11-02 for the owner's Claude Code sessions). Never paste it into code, docs or chat; the helper reads it.5**Base URL**: https://findata.spboucher.ai (API prefix `/api/v1`, Swagger at `/api/docs`). Auth header: `Authorization: Bearer <key>`.67## Fastest path: the `fdq` helper8```bash9FDQ=~/Desktop/Cluster/projets/findata/scripts/fdq          # laptop mirror; also at M3U96a:~/apps/findata/scripts/fdq10$FDQ health                                               # status, alerts, lake free space, worker, backup age11$FDQ datasets                                             # 64 datasets: id, status, rows, first/last session12$FDQ bars AAPL 1m 2024-03-01 2024-03-31                   # JSON page (<= 10 000 rows/page, 1m <= 31 days per call); raw|split|split_div as 5th arg13$FDQ sql "SELECT session_date, min(low) lo, max(high) hi, sum(volume) v FROM market_bars_stock_1m_raw WHERE symbol='AAPL' AND year=2024 GROUP BY 1 ORDER BY 1"14$FDQ sql "SELECT trade_date, strike, call_put, bid, ask, open_interest, delta FROM options_eod WHERE underlying='SPY' AND trade_date=DATE '2024-03-15' AND expiry=DATE '2024-03-28' ORDER BY strike" 50015$FDQ fund MSFT 2015-01-01T00:00:00Z rev,net_inc,eps_d     # point-in-time: only facts available at as_of16$FDQ export "AAPL,MSFT,NVDA" 2024-01-01 2024-12-31 market_bars.stock.1m.raw /tmp/x.parquet   # large ranges -> async Parquet17$FDQ jobs ; $FDQ enqueue '{"kind":"incremental","params":{"type":"stock","period":"day"}}'  # admin (ingestion queue)18```1920## Python (SDK)21```bash22cd ~/Desktop/Cluster/projets/findata23FINDATA_API_KEY=$(cat ~/Desktop/Cluster/secrets/findata/api-key-claude-code.txt) .venv/bin/python my_script.py24```25```python26import os27from findata_sdk import FinData28fd = FinData("https://findata.spboucher.ai", api_key=os.environ["FINDATA_API_KEY"])29bars = fd.bars("AAPL", "1m", "2024-03-01", "2024-03-31")            # pyarrow.Table (all pages, Arrow IPC transport)30df = bars.to_pandas()31tbl = fd.sql("SELECT symbol, count(*) n FROM market_bars_stock_1m_split_div WHERE symbol IN ('AAPL','MSFT') AND year=2020 GROUP BY 1")32facts = fd.fundamentals("AAPL", as_of="2020-01-01T00:00:00Z", metrics=["rev", "net_inc"], period_type="q")33path = fd.export(["AAPL", "MSFT"], "2015-01-01", "2024-12-31", dest="/tmp/aapl_msft_1m.parquet")   # ~2 s per 1 M rows34```3536## What is there (SQL view names = dataset id with dots -> underscores)37| View | Content | Partition columns for the fast path |38|---|---|---|39| `market_bars_stock_1m_raw` (+ `_split`, `_split_div`) | US stocks 1-minute, 16 195 symbols incl. `-DELISTED`, 2000 -> today | `symbol`, `year`, `session_date` |40| `market_bars_stock_{5m,30m,1h,1d}_{raw,split,split_div}` | coarser bars (5m raw is derived from 1m; vendor 5m/30m/1h exist only adjusted) | same |41| `market_bars_etf_*` | same structure for 5 161 ETFs | same |42| `market_bars_index_{1m,5m,30m,1h,1d}_raw` | 122 indices (no volume) | same |43| `market_bars_fx_*_raw`, `market_bars_crypto_*_raw` | FX majors (volume = tick count), crypto (UTC) | same |44| `market_bars_futures_{tf}_continuous_{raw,ratio,absolute}` | continuous series per root (ES, NQ, CL...) | `symbol`, `year` |45| `market_bars_futures_contracts_{tf}_raw` | individual contracts, symbol = `ROOT_MYY` (e.g. `ES_H25`) | `symbol`, `year` |46| `options_eod` | EOD chains 2010 -> today, 11 582 underlyings, IV + Greeks | `underlying`, `period` (`2024q1`, `latest`), `trade_date` |4748Bars columns: `instrument_id, symbol_as_reported, asset_class, interval, timestamp_utc (bar OPEN, UTC), session_date, open, high, low, close, volume, currency, trade_count, vwap, adjustment_type, source_id, ingested_at, quality_flags, revision_id`.49Fundamentals live in Postgres (not in SQL views): use `/api/v1/fundamentals/{ticker}` (as_of semantics) — 5 451 entities, 76 metrics (`rev, net_inc, eps_d, assets_tot, cash_op, shares, roe` ...).5051## Performance rules (measured on the full lake)52- Put `symbol='X'` / `symbol IN (...)` (or `underlying='X'`) and ideally `year=` or a date range in every SQL statement -> 30-300 ms. Without them the view is a directory scan of up to 320 k files (30-60 s, may hit the 30 s timeout).53- Cross-sections: use `market_bars_stock_1d_raw WHERE session_date = DATE '...'` (about 1.3 s) or a `session_date =` predicate on 1m (about 3 s).54- REST bars endpoint: 1m <= 31 days per call, 5m <= 186 days, 1d <= 10 years; pages of <= 10 000 rows with `next_cursor`; larger ranges -> `POST /exports` (Parquet, dataset version pinned).55- Rate limit 600 req/min per key. Responses carry `X-Request-ID` and `X-Dataset-Version`.5657## Operations (admin scope)58`fdq jobs`, `fdq enqueue '{...}'` (kinds: incremental, backfill, derive, compaction, fundamentals, options, backup, quality, gc), `fdq get /api/v1/admin/storage`, `fdq get /api/v1/quality`. Full operator notes: `docs/OPERATIONS.md`. License: FirstRate Data private use — never republish raw bars/tables.59
Files · f85ef07ccaee074d0f21ee15a0887b71bc6ad458
Menu

Theme