Task card — F3-conf-01
- Family / split: F3 Forecasting and temporal information / confirmatory
- Question: rolling-origin forecasts (h = 1, 3; 48 month-end origins 2022-05 … 2026-04) of Canadian monthly real GDP year-over-year growth using only information published at each origin: GDP is published with a 2-month lag, LFS employment with a ~1-week lag. Naive (last published GDP growth) vs a model of the executor's choice; estimands = RMSE/MAE and relative improvement.
- Data type: observed (Statistics Canada WDS, current vintage), one CSV with GDP and employment levels and y/y growth, 345 monthly rows 1998-01 … 2026-09.
- Scope / prerequisites: expanding-window evaluation with an explicit information set (ragged edge), exact naive baseline, forecast ledger with declared information dates.
- Allowed information: at origin t: GDP columns for months ≤ t − 2, employment columns for months ≤ t − 1 (estimation, selection, transformations, plugged-in values and training targets alike).
- Expected artifacts: out/forecasts.csv (192 rows; columns incl. last_target_date_used, last_predictor_date_used), results.json (26 keys incl. last_target_date_available / last_predictor_date_available), report.md, analysis.py, claims.json.
- Rubric summary: required criteria — exact naive RMSE/MAE at h = 1, 3; n_origins; ledger chronology (targets, naive values, actuals, information dates within the rules); model errors consistent with the ledger; relative improvement consistent; availability dates at the last origin correct. Critical checks — leakage (declared dates beyond the limits; naive using the origin month; model RMSE at h = 1 below a bound calibrated on lag-respecting reference models), fabrication (row count, actuals, n_input_rows), false_completion. Baseline = rmse_naive_h1.
- Oracle provenance: independent_code (
~/wp29-oracles/F3-conf-01/build.py, pure Python/numpy); designer = WP29 F3 authoring agent; no human check. - Difficulty rationale: moderate; the default pandas alignment on observation dates silently violates the rule and halves the naive RMSE — the task tests whether the information set is implemented, not modelling skill.
- Budget class: benchmark (4.00 USD cap, 25 min wall, 80 turns per session).
- Anticipated failure modes: using gdp_yoy(t) as the naive; training on targets not yet published at the origin; plugging emp_yoy(t); misreporting the information dates; treating the empty 2026-08/09 GDP cells as a data error to impute.
- Contamination risk: low (series transformation, 2026 window and origin set are specific to this task).