| 1 |
Delete results.json, forecasts.csv, table1.csv, figure1.png, exclusions.csv; re-run analysis.py; SHA-256 of results.json/forecasts.csv/table1.csv identical to previous run |
PASS (IDENTICAL) |
| 2 |
forecasts.csv: 80 rows, exactly one per (origin, horizon, model), 10 origins |
PASS |
| 3 |
vintage_used == origin_date for every row (only vintage V used at origin V) |
PASS |
| 4 |
Every target month's first-release vintage is strictly after the origin (actual not observable at the origin) |
PASS |
| 5 |
Naive h=1 forecast equals the first-release YoY of last(V) for all 10 origins (independent cross-check vintage vs first_release.csv) |
PASS |
| 6 |
RMSE/MAE recomputed from forecasts.csv match results.json (1e-9), RMSE ≥ MAE > 0 for all 8 cells |
PASS |
| 7 |
DM statistic reproduced independently with statsmodels OLS-on-constant HAC (Bartlett, maxlags=h−1, no small-sample correction; HC0 at h=1): 2.4290 (h1), 1.5466 (h3) |
PASS |
| 8 |
Long-run variances positive (0.809 at h1, 2.119 at h3) |
PASS |
| 9 |
rel_improvement = 1 − RMSE_model/RMSE_naive identity |
PASS |
| 10 |
conclusion codes consistent with RMSE ordering and p<0.05 |
PASS |
| 11 |
Bootstrap 95% intervals contain their point estimates (h1, h3) |
PASS |
| 12 |
n_analysed = 20 = 10 origins × 2 horizons; exclusions ledger: 20 − 0 = 20 |
PASS |
| 13 |
n_input_rows has 23 entries (10 RSAFS + 10 PAYEMS vintages + UMCSENT + RSAFS_current + first_release) matching README row counts |
PASS (initial assertion "24" in my check script was my own miscount; 23 is the correct number of CSVs) |
| 14 |
mean_abs_revision recomputed from per-target revisions |
PASS |
| 15 |
Magnitude: model RMSE between 0.3× and 2× naive; all errors in percent units, comparable to YoY range 2.4–6.7% |
PASS |
| 16 |
Code inspection: RSAFS_current.csv read only after all forecasts are written, used only for estimand (d); PAYEMS/UMCSENT not used in forecasts |
PASS |
| 17 |
Model specification fixed (log.md entry) before any out-of-sample error was computed; single run, no tuning |
PASS (documented) |