1# PHASE 2 — Baselines and immutable splits (2026-11-02)23## Completed4- Chronological session splits with warm-up (21 sessions) and a one-session embargo at the start of every evaluation5 block (`pipeline.session_split`, `build_indices`); train origins = `origins_per_session` random eligible slots,6 evaluation origins = fixed stride. Main protocol frozen in `configs/main.yaml` / `configs/final.yaml`:7 train ≤ 2022-12-31, validation 2023, development test 2024, untouched final 2025; 60 held-out instruments.8- Single causal pass producing targets and all baseline inputs per origin (`baselines/features.py`, parallel).9- Baselines (`configs/baselines.yaml`): random walk (trailing RMS), seasonal volatility, EWMA-t, HAR-RV (minute scale),10 GARCH(1,1)-t on 5-minute returns (analytic term structure, own filter), ridge, elastic net, LightGBM (two-part11 location / log-variance models with a Student-t assembled from residual kurtosis); MLP and TCN with the Student-t head.12- Scoring (`eval/scores.py`, `pipeline.score_horizon/summarise/compare`): CRPS (sample + closed form), WIS, quantile13 loss, PIT/KS, coverage, sharpness, QLIKE, tail Brier, directional accuracy, day-cluster bootstrap, HAC DM, Holm/BH.14- Known constraint: LightGBM and torch cannot share a process on macOS (libomp) → baselines and neural stages are15 separate subprocesses (`scripts/run_experiment.py --stage`).1617## Pilot evidence (20 instruments, 2022-09-01 → 2022-12-30; test = 2022-12-16 → 12-30, 4 659 origins, 10 sessions)18Scaled CRPS of the cumulative return (95 % coverage in parentheses) and relative CRPS change vs the random walk with19HAC Diebold–Mariano t (H = 5 / 15 / 30 / 60):2021| Model | H=5 | H=15 | H=30 | H=60 | Δ vs random walk |22|---|---|---|---|---|---|23| naive_rw | 0.4803 (0.96) | 0.4874 (0.96) | 0.4952 (0.96) | 0.4819 (0.97) |24| seasonal_vol | 0.4746 (0.94) | 0.4793 (0.94) | 0.4870 (0.93) | 0.4767 (0.93) | -1.98% (t=-5.7) / -2.51% (t=-6.6) / -2.69% (t=-5.8) / -2.12% (t=-3.4) |25| ewma | 0.4699 (0.97) | 0.4775 (0.96) | 0.4868 (0.96) | 0.4773 (0.96) | -2.83% (t=-8.1) / -2.79% (t=-6.7) / -2.62% (t=-5.2) / -1.77% (t=-2.3) |26| har_rv | 0.4833 (0.99) | 0.4761 (0.96) | 0.4845 (0.95) | 0.4728 (0.95) | -0.24% (t=-0.7) / -3.20% (t=-8.1) / -3.25% (t=-7.0) / -3.04% (t=-4.6) |27| garch_t | 0.4714 (0.97) | 0.4783 (0.98) | 0.4876 (0.98) | 0.4789 (0.98) | -2.58% (t=-8.9) / -2.63% (t=-8.0) / -2.45% (t=-7.2) / -1.60% (t=-3.7) |28| ridge | 0.4734 (0.91) | 0.4765 (0.94) | 0.4842 (0.94) | 0.4723 (0.94) | -2.06% (t=-3.7) / -3.07% (t=-5.7) / -3.29% (t=-5.5) / -2.96% (t=-4.1) |29| elastic_net | 0.4732 (0.91) | 0.4764 (0.94) | 0.4841 (0.94) | 0.4725 (0.94) | -2.14% (t=-3.8) / -3.10% (t=-5.8) / -3.31% (t=-5.6) / -2.91% (t=-3.9) |30| lightgbm | 0.5090 (0.89) | 0.5034 (0.92) | 0.5116 (0.92) | 0.5019 (0.92) | +3.20% (t=3.7) / +1.62% (t=2.0) / +1.59% (t=1.5) / +2.49% (t=1.8) |31| mlp | 0.4689 (0.95) | 0.4769 (0.95) | 0.4847 (0.94) | 0.4743 (0.94) | -3.08% (t=-9.1) / -2.96% (t=-8.3) / -3.01% (t=-7.2) / -2.47% (t=-4.2) |32| tcn | 0.4687 (0.96) | 0.4767 (0.96) | 0.4838 (0.95) | 0.4743 (0.95) | -3.11% (t=-9.8) / -2.94% (t=-7.8) / -3.16% (t=-7.3) / -2.50% (t=-4.0) |33| mrpt_small_1m_only | 0.4693 (0.96) | 0.4765 (0.96) | 0.4842 (0.95) | 0.4745 (0.95) | -3.00% (t=-8.9) / -2.99% (t=-7.8) / -3.16% (t=-6.8) / -2.42% (t=-3.9) |34| mrpt_small | 0.4697 (0.96) | 0.4763 (0.96) | 0.4831 (0.95) | 0.4736 (0.95) | -2.96% (t=-8.8) / -3.07% (t=-7.9) / -3.36% (t=-7.2) / -2.62% (t=-4.2) |35Reading (pilot only, not a finding): every volatility-aware model improves on the random walk by 2–3 % in CRPS;36the random walk is over-dispersed (95 % coverage 0.96–0.97, 50 % coverage 0.61); LightGBM over-fits the location37on 4 942 training origins and is worse than the random walk; HAR-RV at H = 5 is miscalibrated (99 % coverage).38Baselines wall-clock: 22 s for all eight on the pilot (origin frames 4 s). Tables: `results/metrics/pilot/leaderboard_test.csv`,39`compare_{crps,wis,qlike}_vs_naive_rw_test.csv`.4041## Go / no-go42GO. Splits are immutable in the configs; baselines produce calibrated-looking distributions and score through the43same code path as the neural models (identical origins, identical realised targets, asserted at run time).44