SPB Git

spb/wp29_uqo

Public
2.6 KB

# Task card — F3-conf-04

  • Family / split: F3 Forecasting and temporal information / confirmatory
  • Question: rolling-origin forecasts of Canadian quarterly annualised CPI inflation (NSA) at h = 1, 2, 4 over 40 origins (2015Q4 … 2025Q3); seasonal-naive baseline vs a model of the executor's choice; estimands = RMSE/MAE per horizon, relative improvements, and at h = 4 the mean squared-error loss differential with a dependence-robust (HAC) standard error and a Diebold–Mariano statistic.
  • Data type: observed (Bank of Canada Valet CPI, monthly as served + derived quarterly file), 381 + 127 rows.
  • Scope / prerequisites: expanding-window multi-horizon evaluation, seasonal-naive definition with overlapping targets, Newey–West/HAC long-run variance, DM test.
  • Allowed information: quarters dated ≤ t at origin t (estimation, selection, transformations).
  • Expected artifacts: out/forecasts.csv (240 rows), results.json (32 keys incl. loss_diff_mean_h4, loss_diff_se_h4, hac_lags_h4, hac_kernel_h4, dm_stat_h4, dm_pvalue_h4), report.md, analysis.py, claims.json.
  • Rubric summary: required criteria — exact seasonal-naive errors at the three horizons; n_origins; ledger chronology (targets, seasonal-naive rows, actuals); model errors consistent with the ledger; relative improvements consistent; mean loss differential consistent with the ledger; HAC SE consistent with the reported kernel and lag (lag in [3, 8]); DM statistic and p-value consistent. Critical checks — invalid_inference (i.i.d. SE; lag below h − 1), leakage (RMSE bound at h = 1; seasonal naive using post-origin values; perfect foresight), fabrication (row count, actuals, n_input_rows), false_completion. Baseline = rmse_snaive_h1.
  • Oracle provenance: independent_code (~/wp29-oracles/F3-conf-04/build.py; HAC references recomputed from the executor's own loss differentials); designer = WP29 F3 authoring agent; no human check.
  • Difficulty rationale: moderate-hard; the forecasting part is routine, the inference part requires recognising the MA(h − 1) overlap and implementing a HAC variance with an admissible lag, which the convenient i.i.d. t-test ignores.
  • Budget class: benchmark (4.00 USD cap, 25 min wall, 80 turns per session).
  • Anticipated failure modes: i.i.d. standard error of the mean loss differential; HAC lag 1 or automatic bandwidth below 3; seasonal naive defined relative to the origin (y_{t−3}) instead of the target (y_{t+h−4}); mixing up the sign of the loss differential; p-value from a one-sided test.
  • Contamination risk: low (derived quarterly series and 2015–2025 origin set).
Files · 4f59ed568641ceda6f39552d5cc3a441aa812088
Menu

Theme