# Verification checks (P4) — 2026-10-30

| # | check | outcome |
|---|---|---|
| 1 | Clean re-run (deleted results.json, forecasts.csv, table1.csv, figure1.png) regenerates byte-identical results.json/forecasts.csv/table1.csv (SHA-256 compared), done twice (before and after adding HLN sensitivity keys) | PASS |
| 2 | forecasts.csv: 80 rows, unique (origin,horizon,model); vintage_used == origin_date | PASS |
| 3 | Information rule: last(V) = target − h precedes origin date for every row; each forecast reads only RSAFS vintage V (code inspection: only `vintages/RSAFS_vintage_{V}` enters FUNCS); RSAFS_current used only in estimand (d) block | PASS |
| 4 | actual_first_release equals first_release.csv yoy for every target | PASS |
| 5 | naive, snaive recomputed independently; AR(1) recomputed with statsmodels OLS (sample 1993-02…last(V)) and iterated — matches to 1e-5 | PASS |
| 6 | RMSE/MAE recomputed from forecasts.csv for all 8 (model,h); RMSE ≥ MAE in all cases | PASS |
| 7 | DM statistic and p-value recomputed independently (Bartlett weights 2/3,1/3 at h=3; lag 0 at h=1); LRV > 0 at both horizons (0.809, 2.119) | PASS |
| 8 | rel_improvement = 1 − RMSE_model/RMSE_naive recomputed | PASS |
| 9 | Bootstrap 95% CIs of RMSE difference contain the point estimate (h1, h3) | PASS |
| 10 | Estimand (d) recomputed independently from RSAFS_current.csv | PASS |
| 11 | Counts: n_raw pairs 80 − 0 removed = 80 analysed; n_analysed (model) = 20 | PASS |
| 12 | n_input_rows equal `wc -l` − 1 for all 23 files | PASS |
| 13 | Magnitudes vs baseline: model RMSE (0.55 h1, 1.14 h3) of same order as baselines (0.94–1.67); errors in percent; h3 errors > h1 errors for naive/AR1/model as expected | PASS |
| 14 | conclusion codes consistent with RMSE ordering and p<0.05 rule (h1: p=0.0151 → significant; h3: p=0.122 → not significant) | PASS |