# F3-dev-01 — Rolling-origin forecasts of the US unemployment rate: AR(2) on differences vs naive

# Question

Over the 36 expanding-window forecast origins 2023-07-01 … 2026-06-01 (n_origins, first_origin, last_origin), does a simple autoregressive model forecast the US civilian unemployment rate (FRED UNRATE, SA, %) better than the naive no-change forecast at horizons of 1 and 3 months? Estimands: out-of-sample RMSE and MAE (percentage points) of both forecasts and the relative RMSE improvement 1 − RMSE_model/RMSE_naive per horizon.

# Data and exclusions

• data/UNRATE.csv, 945 data rows (n_input_rows), 1948-01 to 2026-09, SHA-256 matching the README; no duplicate or skipped months (see out/state/data_audit.md).
• One missing value: 2025-10-01 (listed in data_issues). How it was handled: (i) at origin 2025-10-01 the naive forecast is the 2025-09 value, and the AR forecast is iterated from 2025-09; (ii) the pairs (origin 2025-09, h=1) and (origin 2025-07, h=3) have no actual. They stay in out/forecasts.csv with an empty actual and are excluded from the error statistics. So n_evaluated_h1 = 35 and n_evaluated_h3 = 35; (iii) in later training windows, regression rows that touch the missing month are dropped. The forecasting state uses linear interpolation between observations dated ≤ origin.
• Exclusion ledger (out/exclusions.csv, last training window): 945 rows → drop 1 missing → drop 3 observations dated after the last origin (used only as actuals) → 941 = n_analysed. The AR regression in the last window uses 935 rows (n_regression_rows_last_window), because differencing, the two lags and the missing month each cost rows.
• The COVID spike (2020-04: 14.8%) is kept as genuine data in every training window.

# Method

• Naive: the forecast at every horizon is the latest observation dated ≤ origin.
• Model AR2_diff (model_name, model_description): AR(2) with intercept on monthly first differences, estimated by OLS on an expanding window that starts in 1948-01 and is re-estimated at every origin. Level forecasts come from iterating the difference recursion h steps ahead (plus one extra step at the origin with the missing value). I fixed the specification before looking at any evaluation errors. No model selection was done, so selection cannot leak test information.
• Information rule: at every origin the code keeps only observations dated ≤ origin, before any estimation or transformation. A leakage test supports this: I set all values after 2024-12 to 9.9 in a temporary copy of the data, and the forecasts for origins ≤ 2024-12 did not change (out/state/checks.md).
• Errors: actual − forecast on the final-vintage data, over pairs with a non-missing actual. The method is deterministic, so no random seed is used (random_seed = null).

# Results

The AR(2)-on-differences model does not beat the naive forecast at either horizon. Its RMSE is about 2.3% higher at h=1 and 0.8% higher at h=3, and its MAE is also slightly higher at both horizons. In the last window the estimated coefficients are close to zero (intercept ≈ 0.0003, lag coefficients ≈ 0.04 and −0.04; recorded in out/state/log.md, not a results key). So the model behaves almost like a random walk, and its forecasts differ from the naive ones by only a few hundredths of a point.

# Uncertainty

• Diebold–Mariano test of equal squared-error loss (model minus naive), with the Harvey–Leybourne–Newbold small-sample correction, a rectangular HAC variance with h−1 lags, and t(n−1) reference:
  • h=1: mean loss difference 0.00065 (mean_loss_diff_h1), DM = 2.50 (dm_stat_h1), p = 0.017 (dm_pvalue_h1). At the 5% level the model is significantly worse than naive, but the difference in magnitude is tiny.
  • h=3: mean loss difference 0.00038 (mean_loss_diff_h3), DM = 1.36 (dm_stat_h3), p = 0.18 (dm_pvalue_h3). The difference is not significant.
• With only 35 evaluated pairs per horizon, overlapping h=3 errors and a calm sample period (UNRATE stayed between 3.5% and 4.5% from 2023-07 to 2026-09; descriptive check recorded in out/state/checks.md, not a results key), these comparisons have little power and depend on the period chosen. They should not be generalised to other periods.

# Limitations and open issues

• Final vintage: the errors are computed on revised data. This measures how forecastable the revised series is, not real-time accuracy, and no real-time validity is claimed.
• I evaluated only one model specification, chosen a priori. Other specifications (e.g. a shorter estimation window, or treating the COVID observations as outliers) might do differently. They were deliberately not tried, to avoid choosing a specification on the test errors. The 2020 spike probably pushes the full-sample AR coefficients towards zero.
• The handling of the 2025-10 gap (interpolating the forecasting state, dropping regression rows) is a documented choice. It affects only a few origins.
• The DM test with n = 35 and a HAC variance is approximate.