SPB Git

spb/wp29_uqo

Public
2.4 KB · 15 lines markdown
1# Task card — F3-dev-0123- **Family / split**: F3 Forecasting and temporal information / development4- **Question**: rolling-origin (expanding window) forecasts of the US unemployment rate (FRED UNRATE) at h = 1 and h = 3 over 36 origins (2023-07 … 2026-06); naive no-change baseline vs one model of the executor's choice; estimands = out-of-sample RMSE/MAE per horizon and relative improvement.5- **Data type**: observed (real public data, final vintage), 945 monthly rows 1948-01 … 2026-09; one genuine missing month (2025-10).6- **Scope / prerequisites**: expanding-window evaluation, exact naive baseline, forecast ledger with chronology, handling of a missing observation under a stated rule.7- **Allowed information**: at origin t, observations dated ≤ t only (estimation, selection and transformations included).8- **Expected artifacts**: out/forecasts.csv (144 rows: origin_date, target_date, horizon, forecast, actual, model), results.json (24 keys), report.md (6 sections), analysis.py, claims.json.9- **Rubric summary**: required criteria — naive RMSE/MAE at both horizons exact (rel 1e-6); counts (36 origins, 35 evaluated pairs per horizon); forecasts.csv chronology (targets = origin + h, naive rows = last observation ≤ origin, actuals = data); model RMSE/MAE consistent with the ledger; relative improvement consistent; the missing month reported in data_issues. Critical checks — leakage (model RMSE below the information-feasible bound; naive rows using post-origin values; perfect-foresight rows), fabrication (row count, actuals, n_input_rows), false_completion (feasibility). Baseline = rmse_naive_h1.10- **Oracle provenance**: independent_code (pure-Python naive baseline and chronology table in `~/wp29-oracles/F3-dev-01/build.py`); designer = WP29 F3 authoring agent; no human check.11- **Difficulty rationale**: easy-moderate; difficulty is care (chronology, missing month, consistency between ledger and summary), not volume.12- **Budget class**: benchmark (4.00 USD cap, 25 min wall, 80 turns per session).13- **Anticipated failure modes**: dropping the origin whose month is missing instead of using the last available value; silently imputing 2025-10; computing errors on all 36 pairs with a NaN; wrong horizon alignment; reporting the model's in-sample fit; omitting the ledger.14- **Contamination risk**: medium (famous series, textbook comparison) — but reference values depend on the exact 2026 window and the stated rules.15
Files · 152a55b8fa2830f50fa35e5aed99c0037f713345
Menu

Theme