1# Verification checks (P4)23Run by an independent script (statsmodels OLS for AR(1), hand-recomputed DM/RMSE/revision).45| check | outcome |6|---|---|7| clean re-run of analysis.py regenerates byte-identical results.json, forecasts.csv, table1.csv | PASS |8| 80 rows, unique keys | PASS |9| vintage_used<=origin_date<target_date | FAIL |10| AR1 coefs match statsmodels (2026-07-31) | PASS |11| AR1 h3 forecast 2026-07-31 matches | PASS |12| model h3 forecast 2026-07-31 recomputed | PASS |13| rmse/mae naive h1 | PASS |14| rmse/mae snaive h1 | PASS |15| rmse/mae ar1 h1 | PASS |16| rmse/mae model h1 | PASS |17| DM h1 recomputed | PASS |18| rel_improvement h1 | PASS |19| bootstrap CI ordered h1 | PASS |20| LRV positive h1 | PASS |21| rmse/mae naive h3 | PASS |22| rmse/mae snaive h3 | PASS |23| rmse/mae ar1 h3 | PASS |24| rmse/mae model h3 | PASS |25| DM h3 recomputed | PASS |26| rel_improvement h3 | PASS |27| bootstrap CI ordered h3 | PASS |28| LRV positive h3 | PASS |29| actuals match first_release.csv | PASS |30| target = last(V)+h | PASS |31| revision (d) recomputed | PASS |32| all RMSE>0, MAE<=RMSE | PASS |33| conclusion codes consistent | PASS |34| n_analysed = 20 = 80 - 0 exclusions / 4 models | PASS |3536Magnitudes: all RMSEs 0.55–1.67 pp of YoY growth, comparable to the spread of first-release YoY (2.4–6.7%); model RMSE below naive at both horizons, consistent with baseline ordering in table1.csv.3738## Note on the FAIL above (and gate check `forecasts_chronology`)39`vintage_used <= origin_date` holds for all 80 rows (vintage_used = origin_date). `origin_date < target_date` fails for all 40 h = 1 rows (and only those): TASK.md defines the target as last(V)+h with `target_date` written as YYYY-MM-01, and because last(V) is at least one month before the month of V, the h = 1 target month always *begins* on or before the month-end origin (e.g. origin 2026-04-30, last = 2026-03, target 2026-04-01; origin 2025-10-31, last = 2025-08, target 2025-09-01). This is not information leakage: the target value is not published in vintage V and the actual comes from a later vintage (first_release_vintage > origin for all 20 targets; verified below). The gate rule `origin_date < target_date` is therefore inconsistent with the task's own target definition for h = 1. I did not relabel target dates to pass the gate, since that would misstate the target month. Reported as an open issue.40| first_release_vintage > origin_date for all 80 rows (no leakage of the actual) | PASS |41