Verification checks (P4)
| # | check | outcome |
|---|---|---|
| 1 | Re-run out/analysis.py twice in workspace; cmp results.json |
PASS (byte-identical) |
| 2 | Clean-state rerun in /tmp/clean (outputs deleted first); cmp results.json | PASS (byte-identical) |
| 3 | forecasts.csv has 80 rows, 10 per (model,horizon) | PASS |
| 4 | actual_first_release equals first_release.csv yoy for each target_date | PASS |
| 5 | vintage_used <= origin_date for all rows; vintage_used == origin_date | PASS |
| 6 | Target month > last(V) (asserted in script: last(V) < month(V), target = last(V)+h) | PASS |
| 7 | origin_date < target_date (gate rule, comparing month-START date of target) | FAIL for 40/40 h=1 rows — see note below |
| 8 | Independent recomputation of DM statistics/p-values (h=1: 2.428978/0.015141; h=3: 1.546577/0.121965) | PASS (match to 6 dp) |
| 9 | AR(1) params at origin 2026-07-31 re-estimated with statsmodels OLS (0.72680, 0.83987, n=401) | PASS |
| 10 | mean_abs_revision_h1_targets recomputed independently (0.548220) | PASS |
| 11 | All RMSE > 0; MAE <= RMSE for every model/horizon | PASS |
| 12 | Bootstrap 95% intervals for RMSE_naive − RMSE_model contain the point estimate | PASS |
| 13 | Exclusion ledger: 80 − 0 = 80 = 4 × n_analysed(20) | PASS |
| 14 | Magnitudes: model RMSE vs naive baseline plausible (0.55 vs 1.00 at h=1; 1.14 vs 1.42 at h=3), AR(1) close to naive as expected given phi≈0.84 | PASS |
| 15 | Gate run 1 (20261030T053408.762973Z): all key/type/positivity/reproducibility checks ok; failures = report/claims/phase ledger not yet written (expected at this stage) + forecasts_chronology (40 rows) | partial |
Note on check 7 (unresolvable conflict between TASK.md and the gate rule)
TASK.md requires target_date in YYYY-MM-01 format and defines the target as last(V)+h, where last(V) is the last month in the vintage dated V. Because a month-end vintage always contains at most the month before V, the h=1 target month is at latest the month containing V, so its first day (YYYY-MM-01) is always on or before the month-end origin date (e.g. origin 2026-05-31, last 2026-04, target 2026-05-01). The gate's literal rule origin_date < target_date therefore cannot hold for any h=1 row under the TASK.md definitions. The substantive information rule — the target month's value is not yet published in the vintage used — holds for all 80 rows (check 6). I keep the TASK.md format (authoritative) rather than relabelling target dates to make the gate pass.
| 16 | Gate run 2 (20261030T053512.664445Z): 64/65 checks ok; only forecasts_chronology fails (see note) | partial |