SPB Git

spb/wp29_uqo

Public

# F3-conf-02 — Real-time forecasts of US real GDP growth

# Question

How accurately could US quarterly real GDP growth (A191RL1Q225SBEA: % change from the previous quarter, seasonally adjusted annual rate) have been forecast 1 and 2 quarters ahead, using only the data published at each of 8 origins (first_origin = 2024-01-31 to last_origin = 2025-12-31, n_origins = 8)? And does a simple time-series model beat the naive forecast? Forecasts are scored against the first-release (advance) value of each target quarter.

# Data and exclusions

  • Inputs: 8 ALFRED vintage files (307–314 rows each), current_vintage.csv (318 rows) and first_release.csv (13 rows). Row counts are in n_input_rows. Every SHA-256 matches data/README.md. No values are missing, no dates are duplicated, and there are no gaps in the quarterly series (out/state/data_audit.md).
  • Information rule: at each origin the only data used are the contents of that origin's vintage file. current_vintage.csv is used only for the revision statistic. first_release.csv is used only for the actuals, and every target's first release was published after its origin (checked in code).
  • Exclusions apply only to the model's estimation sample (out/exclusions.csv, shown for the last origin). There are 314 rows. Forming lagged pairs drops 1 row. Restricting to t ≥ 1985Q1 drops 150. Dropping t ∈ {2020Q2, 2020Q3, 2020Q4} drops 3, because a pandemic outlier (≈ −28% and ≈ +35%) appears there as either y_t or y_{t−1}. That leaves 160 regression observations (n_analysed). The naive forecast uses no exclusions.
  • Data issues and how each was handled are listed in data_issues. They are the pandemic outliers, the 8th origin being 2025-12-31 because the 2025Q3 advance estimate was delayed by the shutdown, and the fact that the current vintage contains look-ahead revisions.

# Method

  • Naive: for both horizons the forecast is the last value in the origin's vintage file.
  • Model (model_name = ar1_post1985_excovid, see model_description): an AR(1) with intercept, estimated by OLS separately on each origin's vintage. The regression sample runs from t = 1985Q1 to the last quarter in that vintage, excluding the pandemic observations. The h = 2 forecast iterates the h = 1 forecast. I fixed this specification before computing any forecast error, and I did not try other specifications. The estimates were stable across origins (intercept ≈ 1.73–1.78, AR coefficient ≈ 0.335–0.343; these are printed by analysis.py and are not results keys).
  • Targets: the last quarter in the vintage plus h quarters, so 2024Q1/2024Q2 for the first origin through 2025Q4/2026Q1 for the last. Each origin has 8 errors per horizon. RMSE and MAE are computed over these. The revision statistic is the mean over the 8 h = 1 target quarters of |current value − first release|.
  • No randomness is used (random_seed = null). out/analysis.py regenerates every output deterministically.

# Results

h = 1 h = 2
Naive RMSE (pp) 2.010 (rmse_naive_h1) 2.265 (rmse_naive_h2)
Naive MAE (pp) 1.687 (mae_naive_h1) 1.812 (mae_naive_h2)
AR(1) RMSE (pp) 1.485 (rmse_model_h1) 1.400 (rmse_model_h2)
AR(1) MAE (pp) 1.234 (mae_model_h1) 1.030 (mae_model_h2)
Relative RMSE improvement 0.261 (rel_improvement_rmse_h1) 0.382 (rel_improvement_rmse_h2)
  • At both horizons, the AR(1) had lower real-time RMSE and MAE than the naive forecast. The RMSE reduction was 26% at h = 1 and 38% at h = 2.
  • Real-time 1-quarter-ahead forecast errors are large: RMSE ≈ 1.5–2.0 pp of annualised growth.
  • The mean absolute revision from first release to the 2026-10-30 vintage, over the h = 1 targets, is 0.5625 pp (mean_abs_revision_h1_targets). That is about a third of the AR(1)'s h = 1 RMSE. So scoring against current values instead of first releases would give materially different error figures.

# Uncertainty

There are only 8 forecast errors per horizon, and they come from a single 2-year period, so these statistics are very imprecise. I ran a supplementary Diebold–Mariano test on squared loss, with the Harvey–Leybourne–Newbold small-sample correction and a t(7) reference distribution. It gives p = 0.18 at h = 1 (supp_dm_hln_pvalue_two_sided_h1, statistic supp_dm_hln_stat_h1 = 1.49) and p = 0.11 at h = 2 (supp_dm_hln_pvalue_two_sided_h2, statistic 1.84). Neither difference is statistically significant at the 5% level. The AR(1) had the smaller squared error at 5 of 8 origins for h = 1 (supp_n_model_wins_sqloss_h1) and at 8 of 8 for h = 2 (supp_n_model_wins_sqloss_h2). With a sample this small, the DM test's asymptotic justification is weak.

# Limitations and open issues

  • The evaluation window (2024–2026Q1) is short and unusual. It includes a large swing (2025Q1 −0.3%, 2025Q3 +4.3%) driven by trade and inventory effects, which penalises the naive forecast heavily. The ranking may not hold in other periods.
  • The model is effectively close to forecasting the unconditional mean (≈ 2.6%). Its advantage reflects mean reversion in a volatile period, not real predictive content.
  • The choices of estimation start (1985) and pandemic exclusion were made a priori and not tuned. Other reasonable choices would give somewhat different numbers, which I did not evaluate.
  • "First release" means the value at the first month-end vintage containing the quarter, as defined in data/README.md.
  • The task was completed in full (feasibility = "complete").
Files · main
Menu

Theme