SPB Git

spb/wp39_uqo

Public
9.1 KB

# RESEARCH_DESIGN.md — preregistered specification (internal)

Frozen 2026-11-02 14:40 ET, before the first test-period observation was opened. Hash of this file at freeze is recorded in artifacts/manifests/design_freeze.json by scripts/02_build_pit_panel.py. Later changes are amendments in DECISIONS.md (dated, with reason) and never alter frozen primary endpoints.

# 1. Question

Does one-minute OHLCV activity contain incremental information about a firm's forthcoming adverse accounting outcome, beyond (i) point-in-time fundamentals, (ii) daily OHLCV, (iii) coarser intraday aggregates of the same bars, and (iv) calendar/industry/size controls? Anticipation (prices move before the release) is not the claim; the claim is about the marginal forecasting value of sampling frequency holding the information set, calendar coverage, universe, cutoff, horizon and training budget fixed.

# 2. Hypotheses — null, alternative, primary endpoint, decision rule

Null H0 Alternative H1 Primary endpoint Decision rule
H1 incremental information OOS log loss of B5 (fundamentals + daily + 1m) ≥ that of B2 (fundamentals + daily) B5 < B2 Δ log loss (B5 − B2) on the untouched test period, 2021-01 → 2026-10, horizon h = 20 95 % block-bootstrap CI (blocks = calendar quarters, 1,000 draws) excludes 0 and Giacomini–White conditional test p < 0.05 (two-sided)
H2 temporal resolution Δ(B5 − B2) ≤ Δ(B4 − B2) and ≤ Δ(B3 − B2) 1m gain strictly exceeds 5m and 30m gains paired Δ log loss B5 − B4 and B5 − B3 both CIs exclude 0 after Holm correction over the 3 frequency contrasts
H3 heterogeneity the B5 − B2 gain is equal across terciles of size, illiquidity, leverage gain larger in small, illiquid, levered firms interaction coefficient β_HF×tercile in the panel regression (§5) Holm-corrected p < 0.05 for the pre-specified ordering
H4 timing the B5 − B2 gain does not increase as h shrinks (40→20→10→5→1) monotone increase slope of Δ log loss on −h across horizons sign test on the five ordered horizons + placebo cutoffs (−45 td) showing no gain
H5 pricing the LFDS does not predict the announcement-window abnormal return nor post-cutoff factor-adjusted returns it does (a) regression of CAR[c+1, A_q+1] on LFDS with controls; (b) decile long-short FF5+MOM alpha with costs CI excludes 0; economic value reported with turnover and costs regardless of significance

Primary endpoint for the headline sentence: H1 at h = 20 on the binary primary target with log loss. Brier, AUC, PR-AUC, calibration slope/intercept, MAE/RMSE/OOS-R² (continuous target) are secondary metrics reported for every model. The headline sentence will read: "At a 20-trading-day horizon, on a 2012–2026 panel of U.S. listed firms with a strictly point-in-time information set, one-minute OHLCV features change the out-of-sample log loss of forecasts of an adverse operating-income surprise by [Δ, 95 % CI] relative to a daily-market-and-accounting benchmark; the gain is [consistent/inconsistent] with [mechanisms] and has [measured/absent] economic value."

# 3. Sample

Universe: all FinData fundamentals entities (5,451; survivor-conditioned, see LIMITATIONS) with ≥ 8 quarterly periods and 1-minute bars; firm-quarters 2012-Q1 → 2026-Q2 with reveal dates ≤ 2026-10-30 and admissible cutoffs (PIT_PROTOCOL §3). Sector = SEC SIC division (2-digit), size = log market cap at c (raw close × latest available shares).

Expected sample (to be reported, not assumed): ~4,300 firms × ~40 quarters ≈ 150–170 k firm-quarters before exclusions.

# 4. Variables

Targets — configs/research.yaml. Primary binary D_q = 1[S_q ≤ −0.5 % of assets], primary continuous S_q (winsorised at 1/99 % on the training period).

PIT fundamentals (B1) — latest available quarter at c: rev growth (yoy), gross margin gp/rev, operating margin op_inc/rev, roa, net_inc/assets_tot, cash_op/assets_tot, accruals (net_inc − cash_op)/assets_tot, cash/assets_tot, leverage debt/assets_tot, interest coverage intr_cover (clipped), current ratio assets_c/liab_c, net issuance (Δshares yoy), capex/assets, R&D/rev, S_{j*} (last surprise), last-four-quarter surprise mean, loss indicator, log assets, log market cap, information age c − A_{j*}, fiscal-quarter dummies, sector dummies, year-quarter calendar dummies (in-sample only; for prediction, calendar effects are replaced by the trailing cross-sectional base rate).

Daily OHLCV (B2) — from 1m regular-session bars aggregated to daily: 20- and 60-day cumulative split-adjusted returns, daily return volatility (20d), Parkinson range volatility, skewness and kurtosis of daily returns (60d), daily volume relative to 60-day median (20d mean and last 5d), Amihud illiquidity (60d), share of zero-return days, overnight (close-to-open) vs intraday (open-to-close) return decomposition (20d sums), max drawdown (60d), beta and residual volatility against SPY (60d daily).

Intraday families (B3 = 30m, B4 = 5m, B5 = 1m) — the same families computed on bars of the given length, each aggregated per session then averaged (or summed) over the 20-day window: realized variance (RV), bipower variation (BPV), relative jump max(0, RV − BPV)/RV, negative semivariance share RS⁻/RV, signed-jump variation (RS⁺ − RS⁻)/RV, realized skewness and kurtosis, maximum absolute bar-return share of RV (concentration), first-order autocorrelation of bar returns, variance ratio RV_freq / RV_daily, intraday drift (close − open)/open, time-of-day volume profile (share of volume in the first and last 30 minutes), volume dispersion across bars (Gini), down-bar volume share (volume on bars with negative return / total — a bar statistic, not order flow), volume surprise relative to the firm's 60-day time-of-day seasonal profile, fraction of bars present (activity), RV on market-down bars vs market-up bars (asymmetry vs SPY), realized beta to SPY at that frequency, vol-of-vol (dispersion of daily RV within the window). At 30m the within-day counts are small (13 bars), so skewness/kurtosis and autocorrelation are computed on the pooled window.

Note on frequency comparison: B3/B4/B5 have the same number of feature columns (one per family); dimensionality does not grow with frequency. A supplementary experiment (Phase 4) concatenates all three to test whether 1m dominates when coarser aggregates are also present.

# 5. Models and inference

Baseline ladder B0–B6 (configs/models.yaml). Logistic ridge for B1–B5 with C tuned on the validation period only; LightGBM with an identical random-search budget (24 trials, early stopping on validation) for every information set. B7/B8 (temporal neural, text) are not run (owner instruction to minimise AI use; declared in DECISIONS D-006).

Panel regression for interpretability and H3: S_{i,q} = α_i + δ_{s(i),t(q)} + β·LFDS_{i,c} + γ'X_{i,c} + ε_{i,q}, where LFDS is the B5 out-of-fold predicted probability (walk-forward, so it is never fitted on the same quarter), with firm fixed effects and sector×year-quarter effects, two-way clustered SEs (firm, calendar quarter). H3 interactions use ex-ante terciles formed at c.

Comparisons: Diebold–Mariano / Giacomini–White on per-quarter mean loss differentials with HAC; stationary block bootstrap over calendar quarters for paired CIs; Holm within each hypothesis family; model confidence set over the ladder as a supplementary exhibit.

# 6. Validation protocol

  • Chronological: train 2012-01 → 2018-12, validation 2019-01 → 2020-12, test 2021-01 → 2026-10 (never opened until Phase 3 gate). Purge: training observations whose A_q falls after the validation start minus one quarter are dropped (embargo = 1 calendar quarter) and likewise at the test boundary.
  • Walk-forward: expanding window, annual refits, first test year 2016, 1-quarter embargo; this provides the out-of-fold LFDS for the panel regressions and the H4/H5 exercises.
  • Firm hold-out: 20 % of firms (hash of entity id) never seen in training, evaluated separately from the time hold-out.
  • Train-only transforms: winsorisation bounds, medians for imputation, standardisation, sector-time demeaning, tercile cut-points — all computed on the training rows of the current fold.

# 7. Power

With ≈ 40 k test firm-quarters, base rate ≈ 20–30 %, and quarter-level clustering (23 test quarters), the minimum detectable paired Δ log loss is roughly 0.002–0.004 (to be computed from the validation-period variance of the loss differential and reported in artifacts/evaluation/power.json before the test set is opened).

# 8. What would make us abandon or pivot

If, on the validation period, B2 already captures all of B5 (Δ within ±0.001 and CI covering 0), H1 is likely null; we proceed to the test anyway (a precise null is a result) and devote Phase 5 to characterising where intraday information does and does not help. If the earnings-release matching proves unreliable (> 10 % irregular lags after cleaning), the event-time design is demoted and the fixed-calendar design becomes primary (amendment required).

Files · main
Menu

Theme