AUDIT — WP7 "Options-Implied Information Content" (source: UQO/UQO_WP/hf-wp7-spb20260519)
Audit performed on 2026-08-05, before any restructuring. The full original tree is preserved
verbatim in _old/.
1. Project overview
Working Paper No. 7 (UQO): The Options-Implied Information Content for Cross-Asset Return and
Volatility Prediction: Evidence from 3.8 Billion Option Contracts. Five research questions (RQ1–RQ5)
answered with 11 Python scripts, 5 derived Parquet datasets, 33 CSV result tables, and a sectioned
LaTeX paper. The paper contains no figures — it is tables-only by design (the only
\includegraphics is the UQO logo on the title page). The original figures/ directory was empty.
2. Scripts (original scripts/, 2,814 lines total)
| Script | Purpose | Inputs | Outputs |
|---|---|---|---|
01_extract_data.py |
Extract option-implied features + realized vol from raw DuckDB stores, merge | options.duckdb, stock_5min.duckdb, etf_5min.duckdb, index_5min.duckdb (external) |
options_features.parquet, realized_vol.parquet, merged_options_rv.parquet |
02_rq1_return_predictability.py |
RQ1: pooled OLS (HC1) + Fama-MacBeth panel regressions | merged_options_rv.parquet |
rq1_regression_results.csv, rq1_meta.csv |
03_rq2_rv_forecasting.py |
RQ2: HAR-RV vs IV-surface models, rolling OOS, Diebold-Mariano | merged_options_rv.parquet |
rq2_insample.csv, rq2_oos_results.csv, rq2_diebold_mariano.csv |
04_rq3_correlation_divergence.py |
RQ3: implied vs realized correlation, stress prediction, crisis windows | options.duckdb, stock_5min.duckdb, index_5min.duckdb, realized_vol.parquet |
correlation_divergence.parquet, rq3_stress_prediction.csv, rq3_crisis_analysis.csv |
05_rq4_greeks_decay_magnets.py |
RQ4: Greeks info decay by DTE + max-OI price-magnet test | options.duckdb, stock_5min.duckdb, realized_vol.parquet |
rq4_greeks_decay.csv, rq4_price_magnet.csv, price_magnet_data.parquet |
06_rq5_ml_rv_forecast.py |
RQ5: RF/GBM on SPX surface vs VIX vs HAR-RV | options.duckdb, index_5min.duckdb |
rq5_model_comparison.csv, rq5_feature_importance.csv |
07_descriptive_stats.py |
Descriptive statistics (Panels A–H) | merged_options_rv.parquet, index_5min.duckdb (VIX), options.duckdb (quality) |
8 descriptive_*.csv |
08_subperiod_regime.py |
Subperiods, VIX regimes, rolling R², pre/post COVID | merged_options_rv.parquet, index_5min.duckdb (VIX) |
subperiod_results.csv, regime_results.csv, rolling_r2.csv, pre_post_covid.csv |
09_portfolio_sorts.py |
Quintile sorts, double sorts, transaction-cost analysis | merged_options_rv.parquet |
portfolio_sort_results.csv, double_sort_iv_skew.csv, transaction_cost_analysis.csv |
10_robustness.py |
Newey-West, double-clustered SE, controls, quantile reg., ticker R² | merged_options_rv.parquet, index_5min.duckdb (VIX, section E) |
robustness_newey_west.csv, robustness_double_clustered.csv, robustness_with_controls.csv, (robustness_quantile_regression.csv, robustness_ticker_r2.csv — missing, see §6.2) |
11_granger_var.py |
Granger causality, bivariate VAR(5), IRF, FEVD | merged_options_rv.parquet |
granger_causality.csv, var_results.csv, irf_results.csv, fevd_results.csv |
Code-quality observations (fixed in the refactor, without changing any computation)
- Heavy duplication:
winsorize()re-defined in 6 scripts; the OLS + HC1 t-stat block copy-pasted in 7 places; standardize-then-lstsqpattern repeated ~15 times. - Hard-coded ticker lists duplicated across scripts (stocks/ETFs/indices exclusion lists appear in 02, 07, 09 with identical content).
- Paths built with
Path(__file__).parent.parent.parent— the raw-database location is implicit and non-configurable. warnings.filterwarnings('ignore')everywhere; bareexcept:clauses in several hot loops.- The p-value column in
02(rq1_regression_results.csv) uses an ad-hoc normal-tail approximation (0.5·exp(−t²/2)·√(2/π)), not an exact two-sided normal/t p-value. Kept as-is (results preservation) but documented in the code.
3. Data
| File | Size | Status |
|---|---|---|
data/merged_options_rv.parquet |
56 MB | Derived (script 01). Master analysis panel: 264,383 ticker-days × 69 tickers, 2010–2025 |
data/options_features.parquet |
24 MB | Derived (script 01, intermediate) |
data/realized_vol.parquet |
49 MB | Derived (script 01, intermediate; consumed by 04, 05) |
data/correlation_divergence.parquet |
351 KB | Derived (script 04). Contains VIX close series as a column |
data/price_magnet_data.parquet |
319 KB | Derived (script 05) |
Raw data: the four raw DuckDB stores (options.duckdb ≈ 3.83 B rows, stock_5min.duckdb,
etf_5min.duckdb, index_5min.duckdb) lived outside the project (two directories up) and
no longer exist on this machine. Consequence: scripts 01, 04, 05, 06 and parts of 07, 08, 10
cannot be re-executed end-to-end. All derived datasets and result CSVs survive, so every
parquet-only analysis remains reproducible (see §6.1).
4. Results (33 CSVs)
All 33 CSVs in results/ are outputs of the scripts above (mapping in §2). No orphan results.
Two CSVs that 10_robustness.py is coded to write are absent (see §6.2).
5. LaTeX
wp7/— current paper:main.tex(clean preamble, metadata macros) +sections/titlepage, introduction, literature, data, methodology, results, robustness, discussion, conclusion, references+appendix/appendix.tex+Makefile/.latexmkrc+uq_logo.jpg. Compiles with latexmk (last built 2026-06-13,main.pdf300 KB). Bibliography is a manualthebibliography(37 entries; exactly matches the 37 cited keys — no missing/unused refs).WP7_Options_Implied_Information_Content.tex(root, 1,066 lines) — older monolithic draft of the same paper (identical section structure). Superseded bywp7/. → kept only in_old/, treated as dead file.- Build artifacts (
main.aux/.log/.out/.toc/.fls/.fdb_latexmk) — regenerable, not migrated. .DS_Store,.claude/settings.local.json— noise/session config, not migrated.
6. Flags — items requiring the author's attention
6.1 Raw data gone → partial reproducibility (blocking for full pipeline)
The raw DuckDB stores are not on this machine. The pipeline is therefore reproducible from the
processed parquets onward only. In the new repo, every raw-dependent script checks for the raw
stores under a configurable WP7_RAW_DATA_DIR and exits with a clear message when absent.
Re-runnable today: RQ1 (02), RQ2 (03), portfolio sorts (09), Granger/VAR (11), robustness A–D & F
(10), descriptives A–E & G (07), subperiods A/C/D (08), and the regression stages of RQ3 (3E–3F,
from correlation_divergence.parquet).
6.2 10_robustness.py — memory bug; two result files missing; paper cites them
- Section D (quantile regression) builds
np.diag(weights)on the full sample (~110k obs → a 110k×110k dense matrix ≈ 97 GB). The original run almost certainly died there (MemoryError), which explains whyrobustness_quantile_regression.csvandrobustness_ticker_r2.csv(written later in the script) are missing fromresults/. - Yet
sections/robustness.texcites those results (quantile-regression significance; ticker-level R²: mean 5.9 %, median 4.7 %, IQR [2.0 %, 8.2 %]). These numbers had no surviving CSV. - In the refactor the weighting was rewritten with broadcasting (
X * w[:,None]) — mathematically identical, memory-safe. Sections D and F were re-run to regenerate the two missing CSVs. - Verification outcome — DISCREPANCY (author review required):
- Ticker-level R² (robustness.tex §4.4): the paper quotes mean 5.9 %, median 4.7 %, IQR [2.0 %, 8.2 %]. Regenerating section F from the surviving panel with the script's exact specification yields mean 14.0 %, median 13.2 %, IQR [10.0 %, 17.6 %] (69 tickers). The paper's figures cannot be reproduced from the surviving data; they may come from an earlier data vintage or a different specification. The paper text was left unchanged. (Qualitatively the claim survives either way — panel results are not driven by outliers.)
- Quantile regressions (robustness.tex §4.4): the paper states the predictors "are significant across the return distribution", but the quantile-regression code computes coefficients only — no standard errors or t-statistics exist in any output. The regenerated coefficients are consistently signed across τ ∈ [0.10, 0.90] (implied kurtosis positive, PC volume ratio negative), which supports stability but not a significance claim. Paper text left unchanged; flagged for the author.
- Rolling-window stats (robustness.tex §4.4): verified exact — paper (mean 8.0 %, std 4.5 %,
range [2.1 %, 17.9 %] for 5-day returns; 41.2 %, std 10.7 % for 1-day RV) matches
rolling_r2.csvto the digit.
6.3 Paper claims not fully backed by a surviving results file
- Robustness §"VIX < 30" subsample (script 10, section E) prints to console only — no CSV was ever written. Not quoted numerically in the paper (prose only), so nothing to reconcile.
- Diebold-Mariano: the paper quotes 4 tickers (AAPL, AMD, CAT, COST);
rq2_diebold_mariano.csvcontains the full set — verified consistent.
6.4 Minor inconsistencies (documentation-level, no action taken on results)
04computes aconst_returnsDuckDB query whose result is never used (dead code; realized correlations actually come from daily returns inrealized_vol.parquet). Removed in refactor — no output touched.- Sample-size figures in the paper (e.g., N = 264,383) match
merged_options_rv.parquet(verified by row count) anddescriptive_by_group.csv.
7. Verification of reproduction (Phase 2c results)
Method: the refactored scripts were re-run with WP7_RESULTS_DIR pointed at a scratch directory
(_verify/results/, kept in the repo for audit), then compared against the originals with
_verify/compare_results.py (byte comparison first, then numeric with rtol=1e-9).
Environment: Python 3.13 / numpy 2.4.4 / pandas 3.0.2 (Apple Silicon).
Outcome — 28 files compared, all conform; 1 explained deviation; 2 files newly generated:
| Status | Files |
|---|---|
| Byte-identical | 10 (all descriptives A–E/G, portfolio sorts, double sort, transaction costs, rq1_meta, rq3_crisis, rq4_price_magnet) |
| Numerically equal (max rel. diff ≤ 2×10⁻¹⁰, i.e. floating-point noise) | 15 (rq1 regressions, rq2 in-sample/OOS/DM, granger, VAR/IRF/FEVD, subperiods, rolling R², pre/post-COVID, NW, double-clustered, controls) |
| Deviation, explained | rq3_stress_prediction.csv — only the t_stat column. Root cause: the design matrix contains an exact linear dependency (corr_divergence = implied_corr − realized_corr), so X'X is singular and HC1 t-stats are numerically degenerate. The economically meaningful corr_ratio t-stats move from (4.07, 8.30, 3.91) to (4.07, 8.22, 3.73) across BLAS/numpy versions — same significance everywhere; the near-zero t-stats (≈10⁻⁷) on the collinear regressors are pure noise. Original CSV kept as authoritative. |
| Newly generated (missing from original archive) | robustness_quantile_regression.csv, robustness_ticker_r2.csv — see §6.2 |
Raw-dependent outputs (rq4_greeks_decay, rq5_*, descriptive_vix_regimes,
descriptive_options_quality, regime_results) could not be re-run (§6.1); the original CSVs are
shipped unchanged in results/.
No original CSV and no number in the paper was modified.