project: anomaly-atlas document: expH_oos_stability/analysis author: Simon-Pierre Boucher contact: contact@spboucher.ai data_source: hfmarketdata.io created: 2026-08-12 modified: 2026-08-12 status: reviewed
Analysis — expH_oos_stability
Run: results/expH_oos_stability/20260812T072518Z/results.json. The
validation split (2016-2021), frozen before the first scan, was opened here
for the first time (418 requests, 18.8M rows). The holdout (2022→) remains
sealed. All four pre-registered questions resolved:
| Question | Train | Validation | Verdict |
|---|---|---|---|
| Q-b nothing survives κ=1 | 0 survivors | 0 survivors | negative finding replicates |
| Q-a CKX 1day net@0.25 | κ*=0.28 | +16.3 bp/d CI [1.4, 26.5], but 2019-21 CI [−19.5, +22.0] | fails robustness clause — not confirmed |
| Q-c ES→SPY fresh corr | −0.032 (FDR, splice-invariant) | +0.003 (p=0.68) | does not replicate |
| Q-d daily reversal family | VR30 excess −0.143 | +0.023 | crisis-regime story confirmed |
Reading
- The ladder closes on the honest outcome. Artifact nulls → search correction → costs → OOS: nothing survives all four. This is the charter's anticipated result ("a well-documented 'it's all artifacts' is a success"), now demonstrated rather than assumed — and it becomes the atlas's first Level-2 entry.
- CKX was exactly what it looked like. The lone cost survivor decays within validation (κ* 0.445 → 0.263), fails its pre-registered two-sub-period clause, and lives in a name whose capacity rounds to zero. Not escalated; the sealed holdout is not spent on it.
- ES→SPY is a lesson in regime-fragility: a splice-invariant, FDR-significant train effect that flips sign OOS. Splice-invariance guards against one artifact — it says nothing about stationarity.
- The artifact replicates even though the anomalies do not. SPY still leads SPX on validation (+0.080 vs +0.132 in train) — content staleness is structural. That contrast (artifacts stable, "alphas" not) is the project's cleanest single sentence.
Hand-off
Two atlas entries created via tools/new_finding.py (F001 negative
finding, Level 2; F002 SPX artifact demonstration, Level 2). Candidate 02
(turn-of-month) and candidate 01 (intraday momentum) keep their protocols
for future work; the holdout stays sealed until a claim earns it.