SPB Git

spb/wp29_uqo

Public

# F3-conf-02 — Real-time forecasts of US real GDP growth

# Question

How accurately could US quarterly real GDP growth (A191RL1Q225SBEA, % change q/q at a seasonally adjusted annual rate) have been forecast 1 and 2 quarters ahead at the time? The forecasts use only the ALFRED vintage published at each of 8 origins (first_origin = 2024-01-31 … last_origin = 2025-12-31, n_origins = 8). They are scored against first-release values. The question is also whether a simple AR model beats the naive last-value forecast.

# Data and exclusions

  • Inputs: 8 vintage files (307–314 rows each), current_vintage.csv (318 rows) and first_release.csv (13 rows). Row counts are in n_input_rows. All SHA-256 hashes match data/README.md. No missing values, no duplicate dates, no gaps (see out/state/data_audit.md).
  • Real-time rule: at each origin only the vintage file dated at that origin is read for estimation and forecast input. current_vintage.csv is used only for mean_abs_revision_h1_targets. The script asserts that each target's first-release date falls after its origin.
  • Exclusions (model only; the naive forecast uses all rows): keep quarters from 1985Q1 onward; form lag pairs; drop pairs that involve the pandemic quarters 2020Q2–2020Q3. At the last origin this gives 314 rows → 163 → 162 → 159 regression pairs (n_analysed = 314 rows in the last origin's training vintage; n_model_estimation_pairs_last_origin = 159). The full ledger for every origin is in out/exclusions.csv.
  • Data issues are listed in data_issues. Origins are unevenly spaced because the 2025 shutdown delayed the advance estimate of 2025Q3. As a result, 2025Q4 is the h=2 target of origin 7 and also the h=1 target of origin 8.
  • Process note: a working-directory slip briefly created a folder of my own state files under data/out/. I moved it to out/state/. The data files themselves were never written, and their hashes were re-verified afterwards.

# Method

  • Naive: the forecast for both horizons is the last value in the origin vintage.
  • Model ar1_1985 (model_name, model_description): an AR(1) with intercept, estimated by OLS at each origin on that vintage only. The sample runs from 1985Q1 to the last published quarter, minus the pandemic pairs. The h=2 forecast is iterated from the h=1 forecast. I chose this specification before computing any forecast and did not tune it on the evaluation results. Across origins the estimates are stable: φ ≈ 0.34 and intercept ≈ 1.75 (printed by the script).
  • Evaluation: errors are forecast − first release, summarised by RMSE and MAE over 8 origins per horizon. No randomness is used (random_seed = null).

# Results

h=1 h=2
Naive RMSE / MAE (pp) 2.010 (rmse_naive_h1) / 1.688 (mae_naive_h1) 2.265 (rmse_naive_h2) / 1.813 (mae_naive_h2)
AR(1) RMSE / MAE (pp) 1.484 (rmse_model_h1) / 1.233 (mae_model_h1) 1.398 (rmse_model_h2) / 1.030 (mae_model_h2)
1 − RMSE_model/RMSE_naive 0.262 (rel_improvement_rmse_h1) 0.383 (rel_improvement_rmse_h2)
  • In this sample the AR(1) has lower RMSE and MAE than the naive forecast at both horizons. It does better because it pulls forecasts toward the post-1985 mean, which helps around volatile quarters such as the 2025Q1 contraction (−0.3) and the following rebound.
  • The mean absolute revision between the first release and the 2026-10-30 value of the 8 h=1 target quarters is 0.5625 pp (mean_abs_revision_h1_targets). That is about a third of the AR(1) h=1 MAE, so the choice of "truth" vintage matters at the margin.

# Uncertainty

Each horizon has only 8 forecast errors, and the h=1 and h=2 samples overlap in their target quarters, so these estimates are very imprecise. Two descriptive tests of the squared-error loss differential (naive² − model²):

  • h=1: the model is better at 5 of 8 origins (n_origins_model_better_sqerr_h1); t = 1.49, two-sided p = 0.18 (dm_tstat_sqerr_h1, dm_pvalue_two_sided_h1).
  • h=2: the model is better at 8 of 8 origins (n_origins_model_better_sqerr_h2); t = 1.55, p = 0.17 (dm_tstat_sqerr_h2, dm_pvalue_two_sided_h2).

These tests use simple t-statistics without HAC correction and assume approximate normality, which n = 8 cannot support well. The model's advantage is therefore not statistically significant at conventional levels. Parameter-estimation uncertainty is not propagated into the forecasts.

# Limitations and open issues

  • The evaluation window is short (2024–2025, 8 origins), so the results may not generalise to other periods, such as recessions.
  • Excluding the pandemic quarters and starting the sample in 1985 are judgement calls (made a priori). Other choices would change the AR estimates somewhat.
  • Advance estimates are noisy (see the revision statistic). Scoring against later vintages could change the ranking; this was not evaluated.
  • The task is complete (feasibility = "complete"). No required value is null.
Files · main
Menu

Theme