F3-conf-02 — Real-time forecasts of US real GDP growth
Question
How accurately could US quarterly real GDP growth (A191RL1Q225SBEA, % change q/q at a seasonally adjusted annual rate) have been forecast 1 and 2 quarters ahead at the time? The forecasts use only the ALFRED vintage published at each of 8 origins (first_origin = 2024-01-31 … last_origin = 2025-12-31, n_origins = 8). They are scored against first-release values. The question is also whether a simple AR model beats the naive last-value forecast.
Data and exclusions
- Inputs: 8 vintage files (307–314 rows each),
current_vintage.csv(318 rows) andfirst_release.csv(13 rows). Row counts are inn_input_rows. All SHA-256 hashes matchdata/README.md. No missing values, no duplicate dates, no gaps (seeout/state/data_audit.md). - Real-time rule: at each origin only the vintage file dated at that origin is read for estimation and forecast input.
current_vintage.csvis used only formean_abs_revision_h1_targets. The script asserts that each target's first-release date falls after its origin. - Exclusions (model only; the naive forecast uses all rows): keep quarters from 1985Q1 onward; form lag pairs; drop pairs that involve the pandemic quarters 2020Q2–2020Q3. At the last origin this gives 314 rows → 163 → 162 → 159 regression pairs (
n_analysed= 314 rows in the last origin's training vintage;n_model_estimation_pairs_last_origin= 159). The full ledger for every origin is inout/exclusions.csv. - Data issues are listed in
data_issues. Origins are unevenly spaced because the 2025 shutdown delayed the advance estimate of 2025Q3. As a result, 2025Q4 is the h=2 target of origin 7 and also the h=1 target of origin 8. - Process note: a working-directory slip briefly created a folder of my own state files under
data/out/. I moved it toout/state/. The data files themselves were never written, and their hashes were re-verified afterwards.
Method
- Naive: the forecast for both horizons is the last value in the origin vintage.
- Model
ar1_1985(model_name,model_description): an AR(1) with intercept, estimated by OLS at each origin on that vintage only. The sample runs from 1985Q1 to the last published quarter, minus the pandemic pairs. The h=2 forecast is iterated from the h=1 forecast. I chose this specification before computing any forecast and did not tune it on the evaluation results. Across origins the estimates are stable: φ ≈ 0.34 and intercept ≈ 1.75 (printed by the script). - Evaluation: errors are forecast − first release, summarised by RMSE and MAE over 8 origins per horizon. No randomness is used (
random_seed= null).
Results
| h=1 | h=2 | |
|---|---|---|
| Naive RMSE / MAE (pp) | 2.010 (rmse_naive_h1) / 1.688 (mae_naive_h1) |
2.265 (rmse_naive_h2) / 1.813 (mae_naive_h2) |
| AR(1) RMSE / MAE (pp) | 1.484 (rmse_model_h1) / 1.233 (mae_model_h1) |
1.398 (rmse_model_h2) / 1.030 (mae_model_h2) |
| 1 − RMSE_model/RMSE_naive | 0.262 (rel_improvement_rmse_h1) |
0.383 (rel_improvement_rmse_h2) |
- In this sample the AR(1) has lower RMSE and MAE than the naive forecast at both horizons. It does better because it pulls forecasts toward the post-1985 mean, which helps around volatile quarters such as the 2025Q1 contraction (−0.3) and the following rebound.
- The mean absolute revision between the first release and the 2026-10-30 value of the 8 h=1 target quarters is 0.5625 pp (
mean_abs_revision_h1_targets). That is about a third of the AR(1) h=1 MAE, so the choice of "truth" vintage matters at the margin.
Uncertainty
Each horizon has only 8 forecast errors, and the h=1 and h=2 samples overlap in their target quarters, so these estimates are very imprecise. Two descriptive tests of the squared-error loss differential (naive² − model²):
- h=1: the model is better at 5 of 8 origins (
n_origins_model_better_sqerr_h1); t = 1.49, two-sided p = 0.18 (dm_tstat_sqerr_h1,dm_pvalue_two_sided_h1). - h=2: the model is better at 8 of 8 origins (
n_origins_model_better_sqerr_h2); t = 1.55, p = 0.17 (dm_tstat_sqerr_h2,dm_pvalue_two_sided_h2).
These tests use simple t-statistics without HAC correction and assume approximate normality, which n = 8 cannot support well. The model's advantage is therefore not statistically significant at conventional levels. Parameter-estimation uncertainty is not propagated into the forecasts.
Limitations and open issues
- The evaluation window is short (2024–2025, 8 origins), so the results may not generalise to other periods, such as recessions.
- Excluding the pandemic quarters and starting the sample in 1985 are judgement calls (made a priori). Other choices would change the AR estimates somewhat.
- Advance estimates are noisy (see the revision statistic). Scoring against later vintages could change the ranking; this was not evaluated.
- The task is complete (
feasibility= "complete"). No required value is null.