project: anomaly-atlas document: expH_oos_stability/hypothesis author: Simon-Pierre Boucher contact: contact@spboucher.ai data_source: hfmarketdata.io created: 2026-08-12 modified: 2026-08-12 status: final
Hypothesis — expH_oos_stability
Pre-specified 2026-08-12 before any validation data was touched. The
VALIDATION split (2016-01-01 → 2022-01-01, frozen in data/universe.py
before the first scan) is opened HERE for the first time. The HOLDOUT
(2022→) stays sealed — it is touched once, only to confirm whatever
survives this experiment.
text
Hypothesis
Four pre-registered questions on the untouched validation split:
(Q-a) CKX 1day — the lone kappa=0.25 cost survivor — does NOT replicate
(skeptical prior: sparse-name estimation artifact).
(Q-b) The NEGATIVE finding replicates: zero intraday rules from the
expG pool have positive net mean at kappa=0.25 on validation.
(Q-c) ES->SPY basis effect (fresh corr(SPY_{t-1}, ES_t) = -0.032 in
train) replicates in sign with a small day-block bootstrap p,
and remains splice-invariant.
(Q-d) The daily 2008-2015 mega-cap reversal family was crisis-regime:
its VR30 excess and excess AC1 SHRINK toward zero on validation.
Falsification criterion
(Q-a) falsified (CKX real) if net@0.25 > 0 with bootstrap CI excluding
0 on validation AND in both validation sub-periods. (Q-b) falsified if
ANY intraday rule has net@0.25 CI above 0. (Q-c) falsified if the sign
flips or splices disagree. (Q-d) falsified if the family's median VR30
excess is as negative or more negative than 2008-2015 (regime story
wrong -> persistent structure, escalate to holdout).
Artifact null(s)
Inherited from the scans: EDGE-based costs (expG model), MA(1)-consistent
VR null (expC), both-fresh synchronization (expD). No new nulls.
Method (pre-declared)
Window: VALIDATION 2016-2021; sub-periods 2016-2018 / 2019-2021.
(a,b) The expG pool (31 rules, rebuilt mechanically) re-evaluated with
identical machinery: gross, kappa*, net@{0.25, 1.0} with 21-day-block
bootstrap CIs (n=300, seed 42).
(c) expD machinery on 2016-2021: ES(3 splices)->SPY and SPX->SPY fresh
xcorr +/-3, day-block bootstrap p (n=200, seed 42).
(d) Liquid-12 daily cells on validation: AC1, VR30, MA(1) excess, EDGE
excess AC1 — side by side with their 2008-2015 train values.
Everything through the cached client; results sanitized JSON.
Result
Run 20260812T072518Z (418 requests, 18.8M rows — the validation window's
first and only download). (Q-b) REPLICATES: zero rules net-positive with
CI>0 at kappa=1.0; at kappa=0.25 only CKX. (Q-a) CKX fails its
pre-registered robustness clause: validation overall net@0.25 = +16.3
bp/day CI [1.4, 26.5], BUT 2019-2021 sub-period CI covers zero
([-19.5, +22.0]) and kappa* decays 0.445 -> 0.263 within validation —
NOT confirmed, NOT escalated to the holdout. (Q-c) ES->SPY FAILS to
replicate: fresh corr = +0.003 (p=0.68), sign flipped, all three splices
agree. (Q-d) regime story CONFIRMED: the daily family's median VR30
excess goes -0.143 (train 2008-15) -> +0.023 (validation); the crisis
reversal vanished OOS (a few single names retain small negative excess
AC1: MSFT, NVDA, QQQ — noted for the record, not promoted).
Interpretation
The full ladder closes with the charter's anticipated honest outcome:
NOTHING in the searched universe survives artifact nulls + search
correction + costs + out-of-sample validation. The negative finding
("short-horizon anomalies in open bar data do not survive costs, and the
gross survivors do not replicate") is itself corrected, OOS-confirmed
and robust -> Level 2; it becomes the atlas's first entry, together with
the SPX->SPY artifact demonstration (present in train +0.132 AND
validation +0.080 — the artifact replicates even though the "anomalies"
do not, which is exactly the point). The sealed holdout (2022->) is NOT
spent on a dead pool; it remains sealed for any future Level-2 claim.
Next experiment
Atlas entries (via tools/new_finding.py) for whatever this settles;
anything that PASSES here goes to the sealed holdout exactly once.