1# F3-dev-01 — Rolling-origin forecasts of the US unemployment rate23## Question4Over 36 expanding-window forecast origins (`first_origin` = 2023-07-01 to `last_origin` = 2026-06-01, `n_origins` = 36), does a simple autoregressive-type model forecast the US civilian unemployment rate (FRED UNRATE, SA, %) better than the naive no-change forecast at horizons of 1 and 3 months? We measure this with out-of-sample RMSE and MAE, and with the relative improvement 1 − RMSE_model/RMSE_naive.56## Data and exclusions7- Input: `data/UNRATE.csv`, final vintage retrieved 2026-10-30. It has 945 data rows covering 1948-01 to 2026-09 (`n_input_rows`), and its SHA-256 matches the README. There are no duplicate or missing dates.8- One missing value is listed in `data_issues`: 2025-10-01 has an empty field. Handling: at origin 2025-10-01 the naive forecast uses the 2025-09 value. The model's Kalman-filter likelihood skips the missing month. The two pairs whose target is 2025-10-01 (origin 2025-09 at h=1, origin 2025-07 at h=3) stay in `out/forecasts.csv` with an empty `actual` and are left out of the error statistics. This leaves 35 evaluated pairs per horizon (`n_evaluated_h1`, `n_evaluated_h3`).9- Exclusion ledger (`out/exclusions.csv`): 945 rows → drop 1 missing value → 944 → drop 3 observations dated after the last origin (used only as actuals) → 941 = `n_analysed`. This is the number of observations in the last training window.1011## Method12- **Naive:** every horizon is forecast with the latest non-missing observation dated ≤ origin.13- **Model (`model_name` = ARIMA210_nc):** see `model_description`. The model is AR(2) on first differences with no constant. It is estimated by Gaussian maximum likelihood (statsmodels SARIMAX, Kalman filter) on an expanding window from 1948-01 up to and including the origin, and re-estimated at each origin. Forecasts for 3 months are built by iterating the 1-month forecast. I fixed the specification before looking at any evaluation errors and did not try other models. There is no randomness, so `random_seed` is null.14- **Information rule:** at each origin the series is cut at the origin date (`s.loc[:t]`) before any estimation or transformation.15- **Evaluation:** errors are actual − forecast on the final-vintage data. A Diebold–Mariano test with squared-error loss, HAC variance using h−1 lags and the Harvey–Leybourne–Newbold small-sample correction is reported as a supplement.1617## Results18| horizon | RMSE naive | MAE naive | RMSE model | MAE model | 1 − RMSE_model/RMSE_naive |19|---|---|---|---|---|---|20| h=1 | 0.1171 (`rmse_naive_h1`) | 0.0914 (`mae_naive_h1`) | 0.1198 (`rmse_model_h1`) | 0.0956 (`mae_model_h1`) | −0.0229 (`rel_improvement_rmse_h1`) |21| h=3 | 0.1586 (`rmse_naive_h3`) | 0.1200 (`mae_naive_h3`) | 0.1594 (`rmse_model_h3`) | 0.1209 (`mae_model_h3`) | −0.0051 (`rel_improvement_rmse_h3`) |2223All errors are in percentage points. The AR model does **not** beat the naive forecast at either horizon. Its RMSE is about 2.3% higher at h=1 and about 0.5% higher at h=3, and its MAE is also slightly higher at both horizons. The model's forecasts stay very close to the no-change forecast, always within 0.2 pp (see `out/state/checks.md`).2425## Uncertainty26- **h=1:** DM statistic = −2.46 (`dm_stat_h1`), p = 0.019 (`dm_pvalue_h1`). The model's squared-error loss is higher than the naive loss by a statistically detectable amount, but the difference in RMSE is small (`rel_improvement_rmse_h1`).27- **h=3:** DM statistic = −0.96 (`dm_stat_h3`), p = 0.34 (`dm_pvalue_h3`). There is no detectable difference.28- With only 35 evaluated pairs, overlapping errors at h=3, and a single 3-year window, these results are fragile. The DM test also does not account for the fact that the model nests the random walk. That issue matters most at h=1, where the DM p-value should be treated as approximate.2930## Limitations and open issues31- Final-vintage data are used, so this is a pseudo-out-of-sample exercise. No real-time validity is claimed, because revisions to seasonal factors were not known at the origins.32- Every training window contains the 2020 COVID spike (e.g. 14.8% in 2020-04). This may distort the Gaussian-MLE AR coefficients. I kept the full sample on purpose to avoid choosing the sample after seeing the evaluation results. A model fitted on a shorter or robust sample might do differently, but that was not evaluated.33- Only one model was assessed, so the results say nothing about AR-type models in general or about other specifications.34- The 2025-10 missing value lowers the evaluated count to 35 per horizon. The task is otherwise complete (`feasibility` = "complete").35