F3-conf-02 — Real-time forecasts of US real GDP growth
Question
How accurately could US quarterly real GDP growth (A191RL1Q225SBEA: % change from the previous quarter, seasonally adjusted annual rate) have been forecast 1 and 2 quarters ahead, using only the data published at each of 8 origins (first_origin = 2024-01-31 to last_origin = 2025-12-31, n_origins = 8)? And does a simple time-series model beat the naive forecast? Forecasts are scored against the first-release (advance) value of each target quarter.
Data and exclusions
- Inputs: 8 ALFRED vintage files (307–314 rows each),
current_vintage.csv(318 rows) andfirst_release.csv(13 rows). Row counts are inn_input_rows. Every SHA-256 matchesdata/README.md. No values are missing, no dates are duplicated, and there are no gaps in the quarterly series (out/state/data_audit.md). - Information rule: at each origin the only data used are the contents of that origin's vintage file.
current_vintage.csvis used only for the revision statistic.first_release.csvis used only for the actuals, and every target's first release was published after its origin (checked in code). - Exclusions apply only to the model's estimation sample (
out/exclusions.csv, shown for the last origin). There are 314 rows. Forming lagged pairs drops 1 row. Restricting to t ≥ 1985Q1 drops 150. Dropping t ∈ {2020Q2, 2020Q3, 2020Q4} drops 3, because a pandemic outlier (≈ −28% and ≈ +35%) appears there as either y_t or y_{t−1}. That leaves 160 regression observations (n_analysed). The naive forecast uses no exclusions. - Data issues and how each was handled are listed in
data_issues. They are the pandemic outliers, the 8th origin being 2025-12-31 because the 2025Q3 advance estimate was delayed by the shutdown, and the fact that the current vintage contains look-ahead revisions.
Method
- Naive: for both horizons the forecast is the last value in the origin's vintage file.
- Model (
model_name=ar1_post1985_excovid, seemodel_description): an AR(1) with intercept, estimated by OLS separately on each origin's vintage. The regression sample runs from t = 1985Q1 to the last quarter in that vintage, excluding the pandemic observations. The h = 2 forecast iterates the h = 1 forecast. I fixed this specification before computing any forecast error, and I did not try other specifications. The estimates were stable across origins (intercept ≈ 1.73–1.78, AR coefficient ≈ 0.335–0.343; these are printed byanalysis.pyand are not results keys). - Targets: the last quarter in the vintage plus h quarters, so 2024Q1/2024Q2 for the first origin through 2025Q4/2026Q1 for the last. Each origin has 8 errors per horizon. RMSE and MAE are computed over these. The revision statistic is the mean over the 8 h = 1 target quarters of |current value − first release|.
- No randomness is used (
random_seed= null).out/analysis.pyregenerates every output deterministically.
Results
| h = 1 | h = 2 | |
|---|---|---|
| Naive RMSE (pp) | 2.010 (rmse_naive_h1) |
2.265 (rmse_naive_h2) |
| Naive MAE (pp) | 1.687 (mae_naive_h1) |
1.812 (mae_naive_h2) |
| AR(1) RMSE (pp) | 1.485 (rmse_model_h1) |
1.400 (rmse_model_h2) |
| AR(1) MAE (pp) | 1.234 (mae_model_h1) |
1.030 (mae_model_h2) |
| Relative RMSE improvement | 0.261 (rel_improvement_rmse_h1) |
0.382 (rel_improvement_rmse_h2) |
- At both horizons, the AR(1) had lower real-time RMSE and MAE than the naive forecast. The RMSE reduction was 26% at h = 1 and 38% at h = 2.
- Real-time 1-quarter-ahead forecast errors are large: RMSE ≈ 1.5–2.0 pp of annualised growth.
- The mean absolute revision from first release to the 2026-10-30 vintage, over the h = 1 targets, is 0.5625 pp (
mean_abs_revision_h1_targets). That is about a third of the AR(1)'s h = 1 RMSE. So scoring against current values instead of first releases would give materially different error figures.
Uncertainty
There are only 8 forecast errors per horizon, and they come from a single 2-year period, so these statistics are very imprecise. I ran a supplementary Diebold–Mariano test on squared loss, with the Harvey–Leybourne–Newbold small-sample correction and a t(7) reference distribution. It gives p = 0.18 at h = 1 (supp_dm_hln_pvalue_two_sided_h1, statistic supp_dm_hln_stat_h1 = 1.49) and p = 0.11 at h = 2 (supp_dm_hln_pvalue_two_sided_h2, statistic 1.84). Neither difference is statistically significant at the 5% level. The AR(1) had the smaller squared error at 5 of 8 origins for h = 1 (supp_n_model_wins_sqloss_h1) and at 8 of 8 for h = 2 (supp_n_model_wins_sqloss_h2). With a sample this small, the DM test's asymptotic justification is weak.
Limitations and open issues
- The evaluation window (2024–2026Q1) is short and unusual. It includes a large swing (2025Q1 −0.3%, 2025Q3 +4.3%) driven by trade and inventory effects, which penalises the naive forecast heavily. The ranking may not hold in other periods.
- The model is effectively close to forecasting the unconditional mean (≈ 2.6%). Its advantage reflects mean reversion in a volatile period, not real predictive content.
- The choices of estimation start (1985) and pandemic exclusion were made a priori and not tuned. Other reasonable choices would give somewhat different numbers, which I did not evaluate.
- "First release" means the value at the first month-end vintage containing the quarter, as defined in
data/README.md. - The task was completed in full (
feasibility= "complete").