# Raw data (external — not shipped) The extraction step (`scripts/01_extract_data.py`) and a few raw-dependent analysis sections read four DuckDB stores that are **too large to live in this repository** (~3.83 billion option records and ~11.5 billion intraday OHLCV bars): | File | Contents | |---|---| | `options.duckdb` | End-of-day option chains (`option_chain` table): 11,077 underlyings, 2010–2025, quotes, IVs, Greeks, volume, OI | | `stock_5min.duckdb` | 5-minute OHLCV bars, US equities (`ohlcv` table) | | `etf_5min.duckdb` | 5-minute OHLCV bars, ETFs (`ohlcv` table) | | `index_5min.duckdb` | 5-minute OHLCV bars, indices incl. SPX/VIX (`ohlcv` table) | Place them in this directory — or point the `WP7_RAW_DATA_DIR` environment variable at the directory that contains them: ```bash export WP7_RAW_DATA_DIR=/path/to/duckdb/stores ``` **Without the raw stores** the pipeline still reproduces every analysis that feeds the paper's tables from the derived datasets in `data/processed/` (shipped with the repository). Raw-dependent steps detect the missing stores and skip themselves with an explanatory message. See `AUDIT.md` §6.1 for the exact reproducibility map. > ⚠️ As of 2026-08-05 the original stores no longer exist on the author's > machine; `data/processed/` is the authoritative surviving copy of the data.