# F3-dev-01 — Rolling-origin forecasts of the US unemployment rate

# Question

Over 36 monthly forecast origins (2023-07-01 … 2026-06-01), does a simple autoregressive model beat the naive (no-change) forecast of the US civilian unemployment rate (FRED UNRATE, SA, %) at horizons h = 1 and h = 3 months? Measured by out-of-sample RMSE and MAE, plus the relative improvement 1 − RMSE_model/RMSE_naive.

# Data and exclusions

• data/UNRATE.csv: 945 data rows, 1948-01-01 to 2026-09-01, a full monthly grid (no duplicates, no skipped dates, all values numeric).
• One missing value: 2025-10-01 (empty field). Two (origin, horizon) pairs have that month as their target (origin 2025-09 at h=1, origin 2025-07 at h=3). They stay in forecasts.csv with an empty actual but are left out of all error statistics. So 35 pairs are evaluated at each horizon.
• How the gap is handled in the model: monthly changes that involve 2025-10 are dropped from estimation. At origin 2025-10 the naive forecast is the 2025-09 value, and the model treats 2025-10 as an extra forecast step. At later origins, the lag values that need 2025-10 use a linear interpolation between 2025-09 and 2025-11, both of which are dated ≤ origin.
• Every training window contains the COVID-19 jump (2020-04: +10.4 pp; 2020-05: −1.6 pp). It is handled with impulse dummies (see Method).
• The data are the final vintage (revised seasonal factors). The errors measure how forecastable the revised series is. They are not real-time forecast performance.

# Method

• Naive: for every horizon, the forecast is the latest non-missing observation dated ≤ origin.
• Model AR_BIC_diff: an AR(p) model on monthly first differences, with an intercept and impulse dummies for the 2020-04 and 2020-05 changes, estimated by OLS.
  • At each origin, p ∈ {0,…,6} is re-selected by BIC on a common sample (starting after 6 lags).
  • The window expands from 1948 to the origin.
  • Multi-step forecasts are iterated and then summed back to levels.
  • Selection, estimation and differencing use only observations dated ≤ origin.
  • BIC chose p = 1 at all 36 origins.
• I fixed this specification before looking at any evaluation-period errors. I did not search across alternative models for a better score.
• out/analysis.py regenerates results.json, forecasts.csv and supplementary.json (the extra statistics below). It is deterministic and uses no random numbers.

# Results

(Units are percentage points.) The AR model on differences does not beat the naive forecast in this period. Its RMSE is about 12% higher at h=1 and 14% higher at h=3, and its MAE is also higher at both horizons. The estimated AR(1) coefficient on monthly changes is positive (momentum). In 2023–2026, though, monthly changes were small, rounded to 0.1 pp, and tended to reverse, so extrapolating the last change hurt accuracy.

# Uncertainty

• Diebold–Mariano test (squared-error loss, Harvey–Leybourne–Newbold small-sample correction, rectangular HAC with h−1 lags, t(n−1) reference). A positive statistic means the model is worse.
  • h=1: statistic 2.86, two-sided p = 0.007.
  • h=3: statistic 2.46, p = 0.019.
• How much to trust these tests:
  • The sample is only 35 evaluated pairs from one stretch of the labour market.
  • The h=3 errors overlap, and the missing target breaks the error sequence, so the HAC estimate is rough.
  • Errors from forecast origins close together are correlated.
  • The p-values should therefore be read as indicative only.
• Sensitivity check (not the main specification): the same model without the COVID dummies gives RMSE 0.11711 at h=1 and 0.15860 at h=3. That is a relative improvement of about −0.00001 and −0.0002, so it is essentially identical to naive. The outlier pushes the AR coefficient almost to zero. The result therefore depends on how the COVID outlier is handled: with the dummies the model is worse than naive, and without them it is tied with naive. In neither case does it beat naive.

# Limitations and open issues

• I evaluated a single model specification. Other simple models (levels AR, a short estimation window, AR models with mean reversion) were not tested, so this result does not show that no simple model can beat naive.
• The data are the final vintage, so this is not a real-time evaluation.
• The evaluation is short (36 origins in a calm period) and has low power. The conclusion may not carry over to recessions, when momentum in unemployment changes is strong.
• Imputing the 2025-10 gap by linear interpolation for the lag values is a choice I made. It affects only forecasts made at origins 2025-11 onward, and only through the lagged change.