# Raw data (external — not shipped)

The extraction step (scripts/01_extract_data.py) and a few raw-dependent analysis sections read four DuckDB stores that are too large to live in this repository (~3.83 billion option records and ~11.5 billion intraday OHLCV bars):

File	Contents
`options.duckdb`	End-of-day option chains (`option_chain` table): 11,077 underlyings, 2010–2025, quotes, IVs, Greeks, volume, OI
`stock_5min.duckdb`	5-minute OHLCV bars, US equities (`ohlcv` table)
`etf_5min.duckdb`	5-minute OHLCV bars, ETFs (`ohlcv` table)
`index_5min.duckdb`	5-minute OHLCV bars, indices incl. SPX/VIX (`ohlcv` table)

Place them in this directory — or point the WP7_RAW_DATA_DIR environment variable at the directory that contains them:

bash

export WP7_RAW_DATA_DIR=/path/to/duckdb/stores

Without the raw stores the pipeline still reproduces every analysis that feeds the paper's tables from the derived datasets in data/processed/ (shipped with the repository). Raw-dependent steps detect the missing stores and skip themselves with an explanatory message. See AUDIT.md §6.1 for the exact reproducibility map.

⚠️ As of 2026-08-05 the original stores no longer exist on the author's machine; data/processed/ is the authoritative surviving copy of the data.