Task card — F3-conf-02
- Family / split: F3 Forecasting and temporal information / confirmatory
- Question: real-time forecasts of US real GDP growth (SAAR %) at h = 1 and h = 2 quarters from 8 vintage-dated origins (2024-01-31 … 2025-12-31); at each origin only the ALFRED vintage dated at that origin may be used; actuals are first releases. Naive (last published value) vs a model of the executor's choice; estimands = real-time RMSE/MAE, relative improvement, and the mean absolute revision of the h = 1 targets.
- Data type: observed (ALFRED vintages = genuine real-time data; FRED current vintage; derived first-release table), 10 small CSV files.
- Scope / prerequisites: vintage discipline (training set per origin from the right file), naive baseline, evaluation against first releases, forecast ledger documenting the training data used.
- Allowed information: at origin d, the vintage file dated d only;
current_vintage.csvonly for the revision statistic;first_release.csvfor actuals. - Expected artifacts: out/forecasts.csv (32 rows with vintage_date, train_last_date, train_last_value, n_train), results.json (23 keys), report.md, analysis.py, claims.json.
- Rubric summary: required criteria — exact naive RMSE/MAE at h = 1, 2; exact mean absolute revision; n_origins; ledger chronology and vintage consistency; model errors consistent with the ledger; relative improvement consistent. Critical checks — leakage (training data not restricted to the origin vintage: revised values, later observations, excess rows; naive from the current vintage; perfect foresight), fabrication (row count, actuals vs first releases, n_input_rows), false_completion. Baseline = rmse_naive_h1.
- Oracle provenance: independent_code (
~/wp29-oracles/F3-conf-02/build.py); designer = WP29 F3 authoring agent; no human check. - Difficulty rationale: moderate; little computation, but the convenient current-vintage file and the irregular origin spacing (shutdown delay) invite shortcuts that the ledger exposes.
- Budget class: benchmark (4.00 USD cap, 25 min wall, 80 turns per session).
- Anticipated failure modes: estimating on the current vintage truncated at the origin (revised values); evaluating against current values instead of first releases; mis-dating targets after the irregular 2025-12-31 origin; reporting n_train from the wrong file.
- Contamination risk: low (specific vintage grid; 2024–2026 window).