# F3-dev-01 — Rolling-origin forecasts of the US unemployment rate

# Question

Over 36 expanding-window forecast origins (first_origin = 2023-07-01 to last_origin = 2026-06-01, n_origins = 36), does a simple autoregressive-type model forecast the US civilian unemployment rate (FRED UNRATE, SA, %) better than the naive no-change forecast at horizons of 1 and 3 months? We measure this with out-of-sample RMSE and MAE, and with the relative improvement 1 − RMSE_model/RMSE_naive.

# Data and exclusions

• Input: data/UNRATE.csv, final vintage retrieved 2026-10-30. It has 945 data rows covering 1948-01 to 2026-09 (n_input_rows), and its SHA-256 matches the README. There are no duplicate or missing dates.
• One missing value is listed in data_issues: 2025-10-01 has an empty field. Handling: at origin 2025-10-01 the naive forecast uses the 2025-09 value. The model's Kalman-filter likelihood skips the missing month. The two pairs whose target is 2025-10-01 (origin 2025-09 at h=1, origin 2025-07 at h=3) stay in out/forecasts.csv with an empty actual and are left out of the error statistics. This leaves 35 evaluated pairs per horizon (n_evaluated_h1, n_evaluated_h3).
• Exclusion ledger (out/exclusions.csv): 945 rows → drop 1 missing value → 944 → drop 3 observations dated after the last origin (used only as actuals) → 941 = n_analysed. This is the number of observations in the last training window.

# Method

• Naive: every horizon is forecast with the latest non-missing observation dated ≤ origin.
• Model (model_name = ARIMA210_nc): see model_description. The model is AR(2) on first differences with no constant. It is estimated by Gaussian maximum likelihood (statsmodels SARIMAX, Kalman filter) on an expanding window from 1948-01 up to and including the origin, and re-estimated at each origin. Forecasts for 3 months are built by iterating the 1-month forecast. I fixed the specification before looking at any evaluation errors and did not try other models. There is no randomness, so random_seed is null.
• Information rule: at each origin the series is cut at the origin date (s.loc[:t]) before any estimation or transformation.
• Evaluation: errors are actual − forecast on the final-vintage data. A Diebold–Mariano test with squared-error loss, HAC variance using h−1 lags and the Harvey–Leybourne–Newbold small-sample correction is reported as a supplement.

# Results

All errors are in percentage points. The AR model does not beat the naive forecast at either horizon. Its RMSE is about 2.3% higher at h=1 and about 0.5% higher at h=3, and its MAE is also slightly higher at both horizons. The model's forecasts stay very close to the no-change forecast, always within 0.2 pp (see out/state/checks.md).

# Uncertainty

• h=1: DM statistic = −2.46 (dm_stat_h1), p = 0.019 (dm_pvalue_h1). The model's squared-error loss is higher than the naive loss by a statistically detectable amount, but the difference in RMSE is small (rel_improvement_rmse_h1).
• h=3: DM statistic = −0.96 (dm_stat_h3), p = 0.34 (dm_pvalue_h3). There is no detectable difference.
• With only 35 evaluated pairs, overlapping errors at h=3, and a single 3-year window, these results are fragile. The DM test also does not account for the fact that the model nests the random walk. That issue matters most at h=1, where the DM p-value should be treated as approximate.

# Limitations and open issues

• Final-vintage data are used, so this is a pseudo-out-of-sample exercise. No real-time validity is claimed, because revisions to seasonal factors were not known at the origins.
• Every training window contains the 2020 COVID spike (e.g. 14.8% in 2020-04). This may distort the Gaussian-MLE AR coefficients. I kept the full sample on purpose to avoid choosing the sample after seeing the evaluation results. A model fitted on a shorter or robust sample might do differently, but that was not evaluated.
• Only one model was assessed, so the results say nothing about AR-type models in general or about other specifications.
• The 2025-10 missing value lowers the evaluated count to 35 per horizon. The task is otherwise complete (feasibility = "complete").