# Verification checks (P4) — 2026-10-30

| # | check | outcome |
|---|---|---|
| 1 | Delete results.json, forecasts.csv, table1.csv, figure1.png, exclusions.csv; re-run analysis.py; SHA-256 of results.json/forecasts.csv/table1.csv identical to previous run | PASS (IDENTICAL) |
| 2 | forecasts.csv: 80 rows, exactly one per (origin, horizon, model), 10 origins | PASS |
| 3 | vintage_used == origin_date for every row (only vintage V used at origin V) | PASS |
| 4 | Every target month's first-release vintage is strictly after the origin (actual not observable at the origin) | PASS |
| 5 | Naive h=1 forecast equals the first-release YoY of last(V) for all 10 origins (independent cross-check vintage vs first_release.csv) | PASS |
| 6 | RMSE/MAE recomputed from forecasts.csv match results.json (1e-9), RMSE ≥ MAE > 0 for all 8 cells | PASS |
| 7 | DM statistic reproduced independently with statsmodels OLS-on-constant HAC (Bartlett, maxlags=h−1, no small-sample correction; HC0 at h=1): 2.4290 (h1), 1.5466 (h3) | PASS |
| 8 | Long-run variances positive (0.809 at h1, 2.119 at h3) | PASS |
| 9 | rel_improvement = 1 − RMSE_model/RMSE_naive identity | PASS |
| 10 | conclusion codes consistent with RMSE ordering and p<0.05 | PASS |
| 11 | Bootstrap 95% intervals contain their point estimates (h1, h3) | PASS |
| 12 | n_analysed = 20 = 10 origins × 2 horizons; exclusions ledger: 20 − 0 = 20 | PASS |
| 13 | n_input_rows has 23 entries (10 RSAFS + 10 PAYEMS vintages + UMCSENT + RSAFS_current + first_release) matching README row counts | PASS (initial assertion "24" in my check script was my own miscount; 23 is the correct number of CSVs) |
| 14 | mean_abs_revision recomputed from per-target revisions | PASS |
| 15 | Magnitude: model RMSE between 0.3× and 2× naive; all errors in percent units, comparable to YoY range 2.4–6.7% | PASS |
| 16 | Code inspection: RSAFS_current.csv read only after all forecasts are written, used only for estimand (d); PAYEMS/UMCSENT not used in forecasts | PASS |
| 17 | Model specification fixed (log.md entry) before any out-of-sample error was computed; single run, no tuning | PASS (documented) |