SPB Git

spb/wp7_uqo

Public

UQO Working Paper No. 7 — Options-implied information for cross-asset return and volatility prediction: evidence from 3.8B option contracts.

18.2 KB

# 📈 The Options-Implied Information Content for Cross-Asset Return and Volatility Prediction

UQO Working Paper No. 7 — Evidence from 3.8 Billion Option Contracts

Paper Python Data Reproducible Institution Author

TL;DR — A daily panel of option-implied features and 5-minute realized variances for 69 liquid US underlyings (49 large stocks, 17 ETFs, 3 indices; 264,383 ticker-days, 2010–2025) is built from 3.83 billion option records. Option-implied features predict weekly returns in a statistically detectable but economically marginal way (R² 0.3 % for stocks, 0.6 % ETFs, 1.0 % indices; only the 25δ skew survives two-way clustering; one strategy of fourteen is profitable after 10 bps, with t = 2.2). The IV surface raises the HAR-RV R² from 0.415 to 0.457 (1-day) and 0.421 to 0.515 (5-day) on a common sample, in every subperiod; out of sample the gain holds under MSE but not under QLIKE. The implied/realized correlation ratio predicts stress at 5–20 days (t 2.3–4.1); ATM IV Granger-causes RV in 100 % of tickers but explains 20–29 % of its 20-day FEVD. Price magnets rejected; tree-based ML does not beat HAR-RV.

⚠️ Version 1.2 (2026-10-17) corrects a target-alignment error in v1.1: the 5-day "forward" targets were trailing windows (r_{t−3}+…+r_{t+1}). The v1.1 headline results (weekly R² 5–19 %, kurtosis Sharpe 2.33, R²_OOS 0.99) are withdrawn. See docs/CHANGELOG-REVISION.md.


# 📌 Status

Item Value
Version 1.2 (October 2026) — revised 2026-10-17 (editorial-board review; docs/REVIEW-REPORT.md, docs/CHANGELOG-REVISION.md)
SSRN 7626643 — uploaded 2026-10-17 with the v1.1 results; must be replaced by paper/main.pdf (v1.2)
Journal target Journal of Futures Markets (double-anonymous; paper/main_anon.pdf); fallback JBF / JFM. Not yet submitted
Code & data parquet-only pipeline reproducible in < 2 min (make pipeline); raw DuckDB stores no longer available

# 🎯 What this paper does

Five research questions on the information content of equity option markets, answered on a panel of 69 liquid underlyings:

RQ Question Result (v1.2)
RQ1 Do implied moments predict cross-sectional returns, and is it tradable? Weekly R² 0.3 % (stocks) → 1.0 % (indices); daily ≈ 0; only Skew₂₅δ robust (t_DC 2.91, positive sign); best net strategy (ATM IV sort) Sharpe 0.34 at 10 bps
RQ2 IV surface vs HAR-RV for RV forecasting? In-sample +9.9 % (1-day) / +22.3 % (5-day) on a common sample, positive in 9/9 subperiods; OOS: better under MSE (14/30 tickers), worse under QLIKE (15/30); IV level alone best at 5 days
RQ3 Implied − realized correlation → market stress? Ratio predicts stress at 5/10/20 d (t_NW 3.27 / 4.13 / 2.33), Feb 2016–Dec 2025
RQ4 Greeks by DTE; max-OI "price magnets"? RV fit rises with DTE (3.8 % → 9.5 %); magnet share 47.0 % < 50 % (frozen output)
RQ5 ML on the SPX surface vs the VIX? HAR-RV (R²_OOS 0.437) beats VIX (0.336), RF (0.143), GBM (0.095); 2-week ATM IV = 50.8 % of RF importance (frozen output)

# 🏆 Headline results

Object Estimate
Option records / underlyings / period 3.83 B / 11,077 / 2010–2025
Analysis panel 264,383 ticker-days, 69 tickers (119,081 with all ten predictors)
5-day return R², stocks / ETFs / indices / all 0.29 % / 0.61 % / 1.01 % / 0.31 %
Only predictor significant under two-way clustering (5-day) Skew₂₅δ: +19 bps per SD, t_DC = 2.91
Best long-short sort (5-day, calendar time) ATM IV: 13.6 %/yr, Sharpe 0.54 gross (t_NW 2.19), 0.34 net of 10 bps, break-even 27 bps
HAR-RV → HAR-RV + IV surface, 1-day / 5-day R² (common sample) 0.415 → 0.457 (+9.9 %) / 0.421 → 0.515 (+22.3 %)
OOS (30 tickers): median MSE ratio / QLIKE ratio 0.967 (DM>1.96: 14, <−1.96: 0) / 1.171 (DM>1.96: 0, <−1.96: 15)
Implied/realized correlation ratio → stress, t_NW (5/10/20 d) 3.27 / 4.13 / 2.33
IV → RV Granger causality 100 % of tickers (mean F = 62.4)
RV FEVD explained by IV shocks at 20 d (RV first / IV first) 20.2 % / 28.5 %
Placebo R² (within-ticker permutation) 0.0004 vs 0.0031 actual

# 🗂 Repository layout

text
options-implied-cross-asset/
├── data/
│   ├── raw/                  # external DuckDB stores (not shipped — see data/raw/README.md)
│   └── processed/            # 5 derived parquets (unchanged; forward targets re-derived at load time)
├── src/wp7/                  # analysis library (config, data_io, econometrics, forecasting, portfolio)
├── scripts/                  # numbered entry points 01–14 (table below)
├── results/                  # CSV result tables (every paper table derives from these)
│   └── _pre-revision-20261017/   # v1.1 CSVs kept for audit
├── figures/                  # supplementary figures (the paper itself is tables-only)
├── docs/                     # REVIEW-REPORT.md, CHANGELOG-REVISION.md, AUDIT.md, CHANGES.md, verify/
├── Makefile                  # make pipeline | figures | paper | all
└── paper/                    # main.tex + preamble.tex + sections/ + tables/ (generated) + appendix/ + references.bib → main.pdf

# 🚀 Quick start

bash
python3 -m pip install -r requirements.txt   # scripts bootstrap src/ themselves
make pipeline        # scripts 02–13 (raw-dependent steps skip themselves gracefully)
python3 scripts/14_make_paper_tables.py      # regenerate paper/tables/*.tex from results/
make figures         # supplementary figures from the result CSVs
cd paper && make     # latexmk → main.pdf   (latexmk -pdf main_anon.tex for the anonymous version)
Script Question Needs raw stores?
01_extract_data.py raw → processed parquets yes (entirely)
02_rq1_return_predictability.py RQ1 panel + Fama–MacBeth (NW) no
03_rq2_rv_forecasting.py RQ2 HAR-RV vs IV surface, OOS, DM (30 tickers legacy; 67 tickers default) no
04_rq3_correlation_divergence.py RQ3 correlation divergence (3 specifications, NW) stage 1 only
05_rq4_greeks_decay_magnets.py RQ4 Greeks decay + magnets part A; part B has a parquet fallback
06_rq5_ml_rv_forecast.py RQ5 ML vs VIX (frozen rows; --lake re-runs from the FinData OS lake) raw stores, or the lake with --lake
07_descriptive_stats.py descriptive panels A–H panel H only
08_subperiod_regime.py subperiods, VIX regimes, rolling R² no (VIX from vix_daily.parquet, fallback RQ3 parquet)
09_portfolio_sorts.py sorts, double sorts, calendar-time strategies + costs no
10_robustness.py HC1 / NW(5,22) / double-clustered SE / controls / quantile / ticker R² no
11_granger_var.py Granger, VAR, orthogonalized IRF / FEVD no
12_make_figures.py supplementary figures no
13_extended_robustness.py winsorization / NW lags / Spearman ICs / deciles / leave-one-year-out / placebo no
14_make_paper_tables.py all LaTeX table fragments (paper/tables/) no

Environment variables: WP7_RAW_DATA_DIR (raw DuckDB stores, default data/raw/), WP7_RESULTS_DIR (default results/), WP7_LEGACY_TARGETS=1 (reproduce the mis-aligned v1.1 targets, audit only), WP7_LEGACY_VIX=1 (use the 3,794-day VIX fallback instead of vix_daily.parquet), FDQ_BIN (path of the FinData OS fdq helper for the batch-N scripts).

# Revision 2026-11-04 (Stage D review): legacy mode, v2 outputs, new scripts

Every analysis script above keeps a --legacy switch that regenerates the committed CSVs of HEAD 3e1fc1c byte-for-byte (make test runs them all into _verify/results/, compares with docs/verify/compare_results.py and runs tests/test_inference.py); the default mode writes the revision outputs to new files (results/*_v2.csv or new names) and never overwrites a legacy CSV. The four raw-only CSVs (rq4_greeks_decay, rq5_model_comparison, rq5_feature_importance, descriptive_options_quality) are frozen outputs of commit c54a8e2 and carry code_version / status = frozen-unverified columns; the withdrawn 5-/22-day and degenerate OLS rows were purged from rq5_model_comparison.csv (full copy in results/_pre-revision-20261017/).

Script Package What it adds Runtime (laptop, M-series, 4 threads)
03 default L1-B1/B2 training-window winsorization, all tickers (≥600 complete rows), ticker FE, CW/GW/MCS/panel DM, Campbell–Thompson R²_OOS, stacked forecasts (results/rq2_oos_forecasts.parquet, regenerable) 3 min
04 default L1-A1/A2 RQ3 series rebuilt 2010–2025 from the parquets (29 names), stress-definition × transform × spec × controls grid, logit HAC, episode bootstrap, Holm/Romano–Wolf 45 s
05 default L1-D1 permutation null and placebo for the price-magnet share 2 s
07, 08 default N-E5 complete VIX series, single denominator in the regime tables 20 s / 2 min
09, 13 default L1-C1/C5/E2 simple-return basis, DK columns, IC NW(4), circular-shift placebo 2 min / 50 s
10 default L1-C5 DK(4)/DK(22) t and 95% intervals (section D skipped; see 22) 35 s
11 default L1-E7/E8 HAC Granger Wald, ADF/KPSS, eigenvalues, residual-bootstrap IRF/FEVD bands, log VAR, 68 tickers 4.5 min
15_factor_alphas.py L1-C3 FF5+MOM alphas of the 14 strategies (Ken French files, sha256 in the data appendix; not committed) 17 s
16_skew_measures.py L1-C4 four skew measures, daily orthogonalization on the IV level 3 min
17_specification_curve.py L1-C5 400-specification curve for the 25δ skew, figures/fig_spec_curve.pdf 2 min
18_multiple_testing.py L1-E1 Holm/BH/Romano–Wolf over the RQ1, strategy, RQ3 and DM families; deflated Sharpe ~10 min
19_fetch_vix_daily.py N-E5 complete VIX close from the FinData OS lake (16 read-only statements) 5 s
20_rq2_subperiod_oos.py, 23_rq2_null_mc.py L1-B3/E6, HV-B4 LOYO/subperiod adj. R², DK Wald, OOS loss ratios, RV-window robustness; nested-null Monte Carlo 1 min / 2 min
21_coverage_reduced_model.py L1-E3/E12 coverage, 119,081 vs 119,093 reconciliation, reduced model, lagged features 20 s
22_quantile_bootstrap.py L1-E9 exact quantile regression with 22-day date-block bootstrap SEs 9 min (3 workers)
24_data_inventory.py L1-F1 checksums, corrected Table 1 with the lake inventory, data availability statement 15 s
25–29, 06 --lake N-H1, N-C1x, N-D1x, N-A3, N-D3, N-D2 lake re-extractions (read-only fdq sql, one underlying and one quarter/year per statement; logs in reproducibility/WP-07/logs/) 17 min, 2 min, 10 min, 3 min, 10 min, ~25 min
30_intervals_mde.py, 31_inline_numbers.py L1-E4, L1-F2 intervals/MDEs of the headline estimates; results/inline_numbers.json + paper/macros.tex 1 min / 5 s
tables_rq1/rq2/rq3/rq4/ts/e.py all new table fragments (paper/tables/*_v2.tex and new names; the 28 legacy fragments are untouched) seconds

Environment used for the revision: Python 3.14.4 (macOS 27.0, Apple silicon, 18 cores), numpy 2.4.4, pandas 3.0.2, scipy 1.17.1, statsmodels 0.14.6, arch 8.0.0, scikit-learn 1.6.1; OMP_NUM_THREADS=OPENBLAS_NUM_THREADS=MKL_NUM_THREADS=VECLIB_MAXIMUM_THREADS=4, seeds fixed at 42 (bootstraps) and 0–9 (tree learners).

# 🗃 Data

Options chains with Greeks (3.83 B contract-days in the 2026-08 stores, 11,077 underlyings, 2010–2025) and 1-minute/5-minute OHLCV bars (1,761 M bars in the stores used; the "≈11,500" of version 1.2 was a typo — the Panel B rows sum to 1,761 M) licensed from FirstRate Data and served by the FinData OS lake (M3U96a; catalog of 2026-11-04: options.eod 4,232,091,733 rows, market_bars.stock.5m.raw 2,060,507,511, etf.5m.split 330,637,732, index.5m.raw 41,162,094). The five derived parquets in data/processed/ are versioned in git (analysis data, registry: data_gitignored = false) and are sufficient for every table except the frozen RQ4-A and RQ5 outputs; the original DuckDB stores were lost on 2026-08-05 (docs/AUDIT.md §6.1). Licence: private use, no redistribution of raw bars or option records. Of the lake's records, 275,448,124 enter the 69-name feature panel (results/data_records_used.csv, package N-H1).

File Rows × columns md5 sha256
merged_options_rv.parquet 264,383 × 31 ba40a3b07edad98caddd81b6725f11d4 a760969177b35394d14b66e6a154c07095dac51b5ef1dedf5af939ecf9653d50
options_features.parquet 266,772 × 16 890ebb3f4753a080968306f1c1b43bb9 14df277c2d098a9c3b02cc375159a0b06c1828093158a47d994dfb5e8d3b3b79
realized_vol.parquet 419,934 × 17 8142da7b16b5a22facf860e565221ecd 16e43b9c622311a0be4593309864ba8bd43363967f077cec191961aec723f7a3
correlation_divergence.parquet 3,794 × 18 8e9f9abcfb31cac68cf647d7038f96e8 99ca0dc7d8676b8d2e57cc2d8f9a44d9d867899b9ce736ef0db9d287710ee9fd
price_magnet_data.parquet 10,480 × 6 abcae5f7eee7c6277bc0af415c1138e0 d9b338918d49d2170d69205eef9f8ba7617e5cfe7d3077f39e7ac77747a88517

Regenerable, not committed (gitignored): data/processed/vix_daily.parquet (scripts/19), correlation_divergence_v2.parquet (scripts/04), the batch-N parquets (scripts/26–29, 06 --lake), results/*.parquet, external_data/ff/ (Ken French daily FF5 and momentum factors, downloaded 2026-11-04T02:44Z, sha256 e58577223032aa45071a46686f7068900c371b516c9eb04e28e974c34264f84f and e56d5bc845544e2993e77353b12c611cfce3149ef754f9b59fc67904ddadd369). The analysis universe is a fixed, ex-post list of liquid names (survivorship bias acknowledged in the paper).

# 🔬 Methodology

  1. Implied features (ATM IV 30/90 d, term slope, 25δ skew, smile-based skewness and kurtosis proxies, put–call ratios, net gamma) per ticker-day (src/wp7/data_io.py, scripts/01).
  2. Forward targets: strictly forward sums Σ_{j=1..5} x_{t+j} (data_io.fix_forward_targets).
  3. Return predictability: pooled OLS (HC1, NW, two-way clustering), Fama–MacBeth with NW, quintile/decile sorts with NW t-stats, calendar-time overlapping-cohort strategies with measured turnover and net-of-cost Sharpe ratios (econometrics.py, portfolio.py).
  4. RV forecasting: HAR-RV (RV_t, RV^(w), RV^(m)) vs IV models on a common sample; rolling OOS (500/250), MSE, QLIKE, R²_OOS, DM with NW variance (forecasting.py).
  5. Correlation divergence: equal-weight implied correlation (CBOE/Skintzi–Refenes analogue), 22-day realized correlation, LPM with NW(h).
  6. Greeks decay & price magnets: by DTE bucket; max-OI attraction test.
  7. Granger / VAR: per-ticker F-tests; bivariate VAR(5) with Cholesky-orthogonalized IRF/FEVD under both orderings.
  8. Robustness: subperiods, VIX regimes, controls, quantiles, ticker R², winsorization, NW lags, Spearman ICs, deciles, leave-one-year-out, placebo.

# 🖼 Figures & tables inventory

The paper is tables-only: 27 table fragments generated by scripts/14_make_paper_tables.py into paper/tables/ (plus the frozen raw-data inventory table typed in sections/data.tex). Supplementary figures from scripts/12_make_figures.py are in figures/.

# ⚠️ Limitations

  • Fixed ex-post universe of 69 liquid names (survivorship bias); results do not speak to small caps, where the literature locates most option-implied return predictability.
  • "Implied skewness/kurtosis" are smile-based proxies, not model-free risk-neutral moments; results are sensitive to tail trimming.
  • Long-short strategies are equal-weighted, not factor-adjusted; costs are proportional assumptions.
  • RQ4-A (Greeks by DTE) and RQ5 (ML) are frozen outputs of the original run (raw stores unavailable); only their 1-day components are reported.
  • RQ3 stress indicator is defined from the VIX, which is also a regressor; realized correlation available from Feb 2016 only.

# 📝 Citation

bibtex
@techreport{boucher2026options,
  author      = {Boucher, Simon-Pierre},
  title       = {The Options-Implied Information Content for Cross-Asset Return and Volatility Prediction: Evidence from 3.8 Billion Option Contracts},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {7},
  year        = {2026},
  month       = {October},
  note        = {Version 1.2}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

  • Revision 2026-10-17 (v1.2): full editorial-board review — docs/REVIEW-REPORT.md (diagnosis, methodology verdicts, literature, maturity) and docs/CHANGELOG-REVISION.md (every corrected number, scripts re-run, items not re-executable). Pre-revision state: git tag pre-revision-20261017, docs/main_pre-revision-20261017.pdf, results/_pre-revision-20261017/. Bibliography 45 → 97 entries, all verified on Crossref (docs/verify/crossref/).
  • Reproduction audit of v1.0 (2026-08-05): docs/AUDIT.md, docs/CHANGES.md, evidence in docs/verify/ (v1.1 logs and CSVs).
  • The May 2026 flat version is archived in Recherche-UQO/_archives/versions-mai-2026/options-implied-cross-asset--_old/; the original project is in …/options-implied-cross-asset--hf-wp7-20260519/.
  • Harmonisation 2026-10-17: canonical title page, bibliography style aer, pinned requirements.txt.
Files · review-20261103
Menu

Theme