SPB Git

Repository Public Research preview

methodology_decisions.md 5.4 KB 23 lines markdown

1# Methodology decisions log (WP46)23Each decision is dated, motivated, and tied to evidence. "Before freeze" = taken before the pre-registration tag.45| ID | Date | Decision | Motivation / evidence |6|---|---|---|---|7| D-01 | 2026-11-09 | Returns in **percent** log units; variances in percent². | Numerical conditioning of the GARCH likelihoods (arch rescales otherwise); explicit units everywhere. |8| D-02 | 2026-11-09 | Session windows: equities 09:30–16:00 ET (XNYS calendar, early closes), futures CME trading day 18:00→17:00 ET, FX 17:00→17:00 ET, crypto UTC day. | `docs/findata_audit.md` §4: vendor `session_date` for futures/FX is an ET calendar day, not the trading day; the CME/NY convention aligns the daily return and the realized measure on the same window. |9| D-03 | 2026-11-09 | Realized measures: 1-min RV, 5-min calendar RV, subsampled 5-min RV (primary), bipower, semivariances, medRV; previous-tick sampling. | Liu, Patton & Sheppard (2015): 5-min RV is hard to beat; subsampling reduces grid dependence. |10| D-04 | 2026-11-09 | Primary proxy = subsampled 5-min RV + squared overnight return (full-day variance); secondary: r², RV1, RV5, BV, medRV, Hansen–Lunde scaled RV. | The models forecast the close-to-close variance; audit shows mean(RV5ss + r_on²)/mean(r²) ∈ [0.93, 1.15] for equities/futures/FX. |11| D-05 | 2026-11-09 | Session validity: ≥ 60 % of the instrument's **modal** 1-minute bar count (equities: 391/211 per calendar). | Grains (ZC) trade ~730 min/day, NG ~1,100, FX 1,440; a fixed 1,380 threshold invalidated 74 % of ZC days. |12| D-06 | 2026-11-09 | Crypto realized measures are invalid before 2022-01-01 (daily returns kept). | Audit: BTC 1-min RV is 10–100× the squared daily return in 2013–2019 (12,810 one-minute moves > 5 % in 2017 alone: cross-venue bounce in the vendor aggregate); in 2020–2021 the ratio is still 1.3–3 (ETH 2021: 92.7 vs 32.5 percent²); from 2022 the ratio is ≈ 1.0–1.3. |13| D-07 | 2026-11-09 | Tier B input = primary proxy of day t−1 (observed at the close of t−1); invalid days imputed by r² inside the recursion only (flagged). | Keeps the recursion defined; the imputation never enters the evaluation target. |14| D-08 | 2026-11-09 | Rolling window 1,000 days, refit every 5 days, daily filtering with fixed parameters; expanding and 2,000-day windows as robustness. | Compute budget (≈ 90 specs × 42 instruments × 3,700 origins); weekly re-estimation changes parameters negligibly relative to daily. |15| D-09 | 2026-11-09 | Multi-step forecasts: exact analytic recursions (GARCH, GJR with κ = E[z²1{z<0}], NAGARCH, QGARCH, CGARCH, IGARCH, GAS, HEAVY, GARCH-X, MS-GARCH Haas recursion, FIGARCH via arch); simulation with 2,000 paths and a fixed seed for EGARCH, APARCH, TARCH, AVGARCH, Realized GARCH; HAR direct projections per horizon. | The brief forbids naive extrapolation; each model card states the recursion. |16| D-10 | 2026-11-09 | Failure policy: bounded retries (3 starting values + Nelder–Mead polish for custom models; 2 restarts for arch), then keep the last successful parameters (flag `stale_fit`), else NaN. No replacement by another model. | Brief §"Computation, failures and fairness". |17| D-11 | 2026-11-09 | Unsupported architectures (FIEGARCH, FIAPARCH, HYGARCH, smooth-transition GARCH, GARCH-MIDAS, SV, Realized EGARCH) are listed in the registry as `unsupported` and not scored. | No vetted implementation in the version-locked libraries (arch 8.0.0, rugarch 1.5-6) or outside the pre-registered information set; a custom implementation without an authoritative cross-check would violate the verification rule. |18| D-12 | 2026-11-09 | Innovation distributions: normal, Student-t, GED, Hansen skew-t for arch and custom GARCH-type models; normal/t only for MS-GARCH, Realized GARCH, HEAVY, GARCH-X; t only for GAS-t. | Formal validity and implementation availability; recorded as `unsupported` combinations otherwise. |19| D-13 | 2026-11-09 | Validation period 2012-01-03 → 2015-12-31 (design checks, pilot, failure accounting); test period 2016-01-04 → 2026-10-30, untouched until the protocol freeze. | FX starts in 2010 (+1,000-day window → 2014), crypto later: validation checks use the instruments available. |20| D-14 | 2026-11-09 | Cross-check policy: every arch-backed and custom model is compared with rugarch 1.5-6 on a shared fixture (`results/metrics/rugarch_crosscheck.json`); FIGARCH differs by 1.47 log-likelihood units (different truncation/constraints between arch and rugarch) and is reported as an implementation caveat. | Verified 2026-11-09. |21| D-15 | 2026-11-09 | Smoke tests of the forecasting machinery were run on SPY 2024 forecasts (values only, no losses) before the freeze; the pilot uses the validation period only. | Disclosure of what was seen before the pre-registration. |22| D-16 | 2026-11-09 (evening, post-freeze) | Seven architectures listed as `unsupported` in D-11 (ST-GARCH, GARCH-MIDAS, HYGARCH, FIAPARCH, FIEGARCH, Realized EGARCH) plus Beta-t-EGARCH are implemented as **post-registration extensions** (status `extension`, exploratory): same protocol, same losses, reported in a separate section and never in the confirmatory tables. | Author's instruction that the paper be a synthesis of the whole GARCH family; D-11 is superseded for these seven; verification = nesting tests and the HYGARCH(amp = 1)–FIGARCH identity, no external package cross-check (`model_cards/_discrepancies.md` §Extensions). |23
Files main
Menu

Theme