# Verification checks (P4) — 2026-10-30

# Independent recomputation (script /tmp/verify.py, statsmodels OLS/HAC used as independent implementation)

[PASS] forecasts.csv has 80 rows, unique (origin,horizon,model)
[FAIL] vintage_used <= origin_date < target_date for all rows
[PASS] first_release yoy = 100 ln(level/prev) (max abs diff <1e-5)
[PASS] naive at 2026-04-30 recomputed independently
[PASS] AR(1) h=3 at 2026-04-30 matches statsmodels OLS
[PASS] target of 2026-04-30 h=3 is 2026-06-01 (last=2026-03)
[PASS] first_release levels match shipped vintages where first_release_vintage is a shipped vintage
[PASS] rmse/mae naive h1 recomputed
[PASS] rmse/mae naive h3 recomputed
[PASS] rmse/mae snaive h1 recomputed
[PASS] rmse/mae snaive h3 recomputed
[PASS] rmse/mae ar1 h1 recomputed
[PASS] rmse/mae ar1 h3 recomputed
[PASS] rmse/mae model h1 recomputed
[PASS] rmse/mae model h3 recomputed
[PASS] DM h1 equals statsmodels HAC t-stat (2.428978)
[PASS] rel_improvement h1 identity
[PASS] conclusion code h1 consistent
[PASS] DM h3 equals statsmodels HAC t-stat (1.546577)
[PASS] rel_improvement h3 identity
[PASS] conclusion code h3 consistent
[PASS] mean_abs_revision recomputed (0.548220)
[PASS] n_analysed = 20 = exclusions ledger final n_after
[PASS] n_input_rows match README counts (RSAFS 2026-07-31=414, UMCSENT=887, current=417, first_release=28)
[PASS] magnitudes: all RMSE between 0.3 and 3 pp, same order as naive baseline

Note on the FAIL above: the check "vintage_used <= origin_date < target_date" fails for the 40 h=1 rows. This is not an information leak: by TASK.md, the target is last(V)+h with target_date written as YYYY-MM-01, and because RSAFS is published with a 1-2 month lag, every h=1 target month starts on or before the month-end origin date (e.g. origin 2025-10-31, last(V)=2025-08, h=1 target 2025-09-01). The substantive real-time condition was checked instead (below) and passes.

# Real-time availability

[PASS] vintage_used == origin_date for all 80 rows (only the vintage dated V feeds origin V).
[PASS] 0 of 80 target months are present in the vintage used (no target value was published at the origin).
[PASS] analysis.py asserts that no vintage contains a month after its origin date.
[PASS] RSAFS_current.csv is read only after all forecasts are produced, in the revision block (code inspection).
[PASS] UMCSENT / PAYEMS not used in any forecast (code inspection; read only for row counts).

# Reproducibility

[PASS] Deleted results.json, forecasts.csv, table1.csv, figure1.png, aux_diagnostics.json; re-ran analysis.py; results.json byte-identical (cmp), forecasts.csv and table1.csv SHA-256 identical.

# Internal consistency

[PASS] RMSE >= MAE > 0 for every model/horizon; DM long-run variances positive (h1 0.809, h3 2.119; aux_diagnostics.json).
[PASS] exclusions: 20 − 0 = 20 = n_analysed.
[PASS] magnitudes: model RMSEs (0.55, 1.14) are below but of the same order as the naive baseline (1.00, 1.42); h=3 RMSE > h=1 RMSE for naive and model, as expected.

# Gate

Gate run at end of P4: all checks ok except report/claims (not yet written, P5) and forecasts_chronology (spec conflict above, cannot be fixed without misdating targets in violation of TASK.md).

spb/wp29_uqo