Verification checks (P4)
Run on 2026-10-30. Script: out/state/verify.py (independent re-implementation, positional slicing of raw data ≤ t).
| check | outcome |
|---|---|
| forecasts.csv has 180 rows, 60 per model, horizon=1, target=origin+1 month, origins 2021-09-01..2026-08-01 | PASS |
| independent re-implementation (numpy, raw data sliced to ≤ t, explicit inverse) reproduces all ridge/mean/naive forecasts and actuals | PASS (max abs diff 4.96e-11) |
| RMSE/MAE recomputed from forecasts.csv equal results.json (tol 1e-9; CSV has 10 decimals) | PASS (first attempt used 1e-12 tolerance and failed only because of CSV rounding; tolerance corrected) |
| RMSEs positive, RMSE ≥ MAE for each model | PASS |
| rel_improvement identities 1 − rmse_ridge/rmse_base | PASS |
| exclusion ledger: 417 − 2 − 2 = 413 = n_analysed; first-origin training rows 354 (1992-03..2021-08) | PASS |
| Leakage test: all values dated after 2023-06 replaced by 999 in a temp copy; forecasts for origins ≤ 2023-06 unchanged (max diff 0.0), later origins changed | PASS |
| Clean re-run (deleted outputs, re-ran analysis.py): results.json and forecasts.csv byte-identical | PASS |
| Chronology first_origin < last_origin; all training label dates ≤ origin (assert in analysis.py) | PASS |
| Magnitudes vs baseline: ridge RMSE 0.561 between mean 0.544 and naive 0.883; plausible for monthly IP growth (sd ≈ 1.04 full sample incl. COVID) | PASS |