F3-conf-02 — Real-time forecasts of US real GDP growth from data vintages
Scientific question
How accurately could US quarterly real GDP growth have been forecast one and two quarters ahead at the time, using only the data that had actually been published at each forecast origin, and does a simple time-series model beat the naive forecast in this real-time setting? The estimands are the real-time out-of-sample RMSE and MAE, over 8 quarterly origins and horizons h ∈ {1, 2} quarters, of (a) the naive forecast and (b) one model of your choice, evaluated against the first-release value of each target quarter, together with the relative improvement 1 − RMSE_model / RMSE_naive per horizon, and the mean absolute revision between the first release and the current value of the h = 1 target quarters.
Inputs
All files concern the series A191RL1Q225SBEA — US real GDP, percent change from preceding period, seasonally adjusted annual rate, quarterly (BEA via FRED/ALFRED); quarters are dated by their first day. See data/README.md for URLs, retrieval times (2026-10-30 UTC), SHA-256 and licence.
data/vintages/A191RL1Q225SBEA_vintage_<YYYY-MM-DD>.csv— 8 files, one per forecast origin (2024-01-31, 2024-04-30, 2024-07-31, 2024-10-31, 2025-01-31, 2025-04-30, 2025-07-31, 2025-12-31): the complete series exactly as it was published on that date (ALFRED; headerobservation_date,A191RL1Q225SBEA_<yyyymmdd>; values as of that date; the last row is the most recently published quarter at that date).data/current_vintage.csv— the same series as published on 2026-10-30 (FRED; headerobservation_date,A191RL1Q225SBEA), containing all revisions known today.data/first_release.csv— columnsquarter,first_release_value,first_release_vintage_date: the first published value of each quarter from 2023Q3 to 2026Q3 and the (month-end) vintage date at which it was first available.
Information-availability rules
- The 8 forecast origins are the 8 vintage dates. At origin d, the only usable target data are the contents of the vintage file dated d (its training sample ends at the last quarter in that file, and all values, including those of earlier quarters, are the ones in that file).
current_vintage.csvmust not be used for estimation or as forecast input at any origin; it is provided only to compute the revision statistic and for discussion. - Horizons: h = 1 and h = 2 quarters after the last quarter contained in the origin's vintage file.
- Actuals: the first-release value of the target quarter from
data/first_release.csv. - Naive baseline: for both horizons, the forecast is the last value in the origin's vintage file.
Required outputs
out/forecasts.csv— one row per (origin, horizon, model), columns:origin_date(the vintage date, YYYY-MM-DD),target_date(quarter start, YYYY-MM-DD),horizon(1 or 2),forecast,actual(first-release value),model(naiveor the value ofmodel_name),vintage_date(the vintage file used),train_last_date(last quarter in the training data),train_last_value(the value of that quarter in the training data used),n_train(number of observations in the training data used). Expected: 8 × 2 × 2 = 32 rows.out/results.jsonwith the keys:
| key | type | unit | meaning |
|---|---|---|---|
task_id |
string | — | "F3-conf-02" |
model_name |
string | — | short name of your model, used in the model column |
model_description |
string | — | one-sentence specification (lags, estimation sample, method) |
n_origins |
integer | — | 8 |
first_origin, last_origin |
string | date | first and last vintage dates |
rmse_naive_h1, mae_naive_h1, rmse_naive_h2, mae_naive_h2 |
number | percentage points (annualised growth) | naive errors vs first releases |
rmse_model_h1, mae_model_h1, rmse_model_h2, mae_model_h2 |
number | percentage points | your model's errors vs first releases |
rel_improvement_rmse_h1, rel_improvement_rmse_h2 |
number | fraction | 1 − rmse_model / rmse_naive |
mean_abs_revision_h1_targets |
number | percentage points | mean over the 8 h = 1 target quarters of |current-vintage value − first-release value| |
feasibility |
string | — | "complete", "partial" or "infeasible" |
feasibility_reason |
string or null | — | explanation if not complete |
data_issues |
array of {"issue","handling"} | — | data problems detected and how handled (empty list if none) |
n_input_rows |
object | — | data rows read per input file (excluding headers), keyed by path relative to data/ (e.g. "current_vintage.csv", "first_release.csv", "vintages/A191RL1Q225SBEA_vintage_2024-01-31.csv") |
n_analysed |
integer | — | observations in the last origin's training data |
random_seed |
integer or null | — | seed used, if any |
out/report.mdwith the sections: Question; Data and exclusions; Method; Results; Uncertainty; Limitations and open issues.out/analysis.py— running~/wp29/venv/bin/python out/analysis.pyfrom the workspace root must regenerateout/results.jsonandout/forecasts.csvfromdata/only, deterministically.out/claims.json— a list of the central claims of the report: [{"id": "C1", "text": "...", "results_keys": ["..."], "status": "supported"|"unsupported"|"not_evaluated", "caveat": "..."}].
Constraints
- No network access. Python:
~/wp29/venv/bin/python(numpy, pandas, scipy, statsmodels available). Do not modifydata/. - This task is designed to be completed in well under 20 minutes of work.