# F3-dev-01 — Rolling-origin forecasts of the US unemployment rate: AR(2) on differences vs naive

## Question
Over the 36 expanding-window forecast origins 2023-07-01 … 2026-06-01 (`n_origins`, `first_origin`, `last_origin`), does a simple autoregressive model forecast the US civilian unemployment rate (FRED UNRATE, SA, %) better than the naive no-change forecast at horizons of 1 and 3 months? Estimands: out-of-sample RMSE and MAE (percentage points) of both forecasts and the relative RMSE improvement 1 − RMSE_model/RMSE_naive per horizon.

## Data and exclusions
- `data/UNRATE.csv`, 945 data rows (`n_input_rows`), 1948-01 to 2026-09, SHA-256 matching the README; no duplicate or skipped months (see `out/state/data_audit.md`).
- One missing value: 2025-10-01 (listed in `data_issues`). How it was handled: (i) at origin 2025-10-01 the naive forecast is the 2025-09 value, and the AR forecast is iterated from 2025-09; (ii) the pairs (origin 2025-09, h=1) and (origin 2025-07, h=3) have no actual. They stay in `out/forecasts.csv` with an empty `actual` and are excluded from the error statistics. So `n_evaluated_h1` = 35 and `n_evaluated_h3` = 35; (iii) in later training windows, regression rows that touch the missing month are dropped. The forecasting state uses linear interpolation between observations dated ≤ origin.
- Exclusion ledger (`out/exclusions.csv`, last training window): 945 rows → drop 1 missing → drop 3 observations dated after the last origin (used only as actuals) → 941 = `n_analysed`. The AR regression in the last window uses 935 rows (`n_regression_rows_last_window`), because differencing, the two lags and the missing month each cost rows.
- The COVID spike (2020-04: 14.8%) is kept as genuine data in every training window.

## Method
- **Naive**: the forecast at every horizon is the latest observation dated ≤ origin.
- **Model `AR2_diff`** (`model_name`, `model_description`): AR(2) with intercept on monthly first differences, estimated by OLS on an expanding window that starts in 1948-01 and is re-estimated at every origin. Level forecasts come from iterating the difference recursion h steps ahead (plus one extra step at the origin with the missing value). I fixed the specification before looking at any evaluation errors. No model selection was done, so selection cannot leak test information.
- **Information rule**: at every origin the code keeps only observations dated ≤ origin, before any estimation or transformation. A leakage test supports this: I set all values after 2024-12 to 9.9 in a temporary copy of the data, and the forecasts for origins ≤ 2024-12 did not change (`out/state/checks.md`).
- **Errors**: actual − forecast on the final-vintage data, over pairs with a non-missing actual. The method is deterministic, so no random seed is used (`random_seed` = null).

## Results
| horizon | n evaluated | RMSE naive | RMSE AR2_diff | MAE naive | MAE AR2_diff | 1 − RMSE ratio |
|---|---|---|---|---|---|---|
| h=1 | 35 (`n_evaluated_h1`) | 0.1171 (`rmse_naive_h1`) | 0.1199 (`rmse_model_h1`) | 0.0914 (`mae_naive_h1`) | 0.0957 (`mae_model_h1`) | −0.0235 (`rel_improvement_rmse_h1`) |
| h=3 | 35 (`n_evaluated_h3`) | 0.1586 (`rmse_naive_h3`) | 0.1598 (`rmse_model_h3`) | 0.1200 (`mae_naive_h3`) | 0.1213 (`mae_model_h3`) | −0.0076 (`rel_improvement_rmse_h3`) |

The AR(2)-on-differences model does **not** beat the naive forecast at either horizon. Its RMSE is about 2.3% higher at h=1 and 0.8% higher at h=3, and its MAE is also slightly higher at both horizons. In the last window the estimated coefficients are close to zero (intercept ≈ 0.0003, lag coefficients ≈ 0.04 and −0.04; recorded in `out/state/log.md`, not a results key). So the model behaves almost like a random walk, and its forecasts differ from the naive ones by only a few hundredths of a point.

## Uncertainty
- Diebold–Mariano test of equal squared-error loss (model minus naive), with the Harvey–Leybourne–Newbold small-sample correction, a rectangular HAC variance with h−1 lags, and t(n−1) reference:
  - h=1: mean loss difference 0.00065 (`mean_loss_diff_h1`), DM = 2.50 (`dm_stat_h1`), p = 0.017 (`dm_pvalue_h1`). At the 5% level the model is significantly *worse* than naive, but the difference in magnitude is tiny.
  - h=3: mean loss difference 0.00038 (`mean_loss_diff_h3`), DM = 1.36 (`dm_stat_h3`), p = 0.18 (`dm_pvalue_h3`). The difference is not significant.
- With only 35 evaluated pairs per horizon, overlapping h=3 errors and a calm sample period (UNRATE stayed between 3.5% and 4.5% from 2023-07 to 2026-09; descriptive check recorded in `out/state/checks.md`, not a results key), these comparisons have little power and depend on the period chosen. They should not be generalised to other periods.

## Limitations and open issues
- **Final vintage**: the errors are computed on revised data. This measures how forecastable the revised series is, not real-time accuracy, and no real-time validity is claimed.
- I evaluated only one model specification, chosen a priori. Other specifications (e.g. a shorter estimation window, or treating the COVID observations as outliers) might do differently. They were deliberately not tried, to avoid choosing a specification on the test errors. The 2020 spike probably pushes the full-sample AR coefficients towards zero.
- The handling of the 2025-10 gap (interpolating the forecasting state, dropping regression rows) is a documented choice. It affects only a few origins.
- The DM test with n = 35 and a HAC variance is approximate.