# F3-dev-01 — Rolling-origin forecasts of US unemployment: AR(2) on differences vs naive

## Question
Over 36 expanding-window forecast origins from 2023-07-01 to 2026-06-01 (`n_origins`, `first_origin`, `last_origin`), does a simple autoregressive model forecast the US unemployment rate (UNRATE, SA, percent) better than the naive no-change forecast at horizons of 1 and 3 months? The measures are RMSE, MAE and 1 − RMSE_model/RMSE_naive.

## Data and exclusions
- `data/UNRATE.csv` (FRED, final vintage), 945 data rows, 1948-01 to 2026-09 (`n_input_rows`). The SHA-256 matches the README. Dates are contiguous monthly with no duplicates. See `out/state/data_audit.md`.
- One value is missing: 2025-10-01 (`data_issues`). Two pairs target that month (origin 2025-09 at h=1 and origin 2025-07 at h=3). Both stay in `out/forecasts.csv` with an empty `actual` and are left out of all error statistics. That gives 35 evaluated pairs per horizon (`n_evaluated_h1`, `n_evaluated_h3`).
- Exclusion ledger (`out/exclusions.csv`): 945 rows − 1 missing − 3 months after the last origin (2026-07 to 2026-09, used only as targets) = 941 observed months in the last training window (`n_analysed`).

## Method
- **Naive:** the forecast is the latest non-missing observation dated ≤ origin. At origin 2025-10 this is the 2025-09 value.
- **Model `AR2_diff`** (`model_name`, `model_description`): an AR(2) on first differences with an intercept. It is fitted by OLS on the expanding window from 1948-01 to the origin and re-estimated at every origin. Level forecasts come from iterating the difference recursion and adding up the steps. The lag order and sample start were fixed before looking at any results, with no selection step, so the evaluation window was never used for tuning.
- **Information rules:** at each origin the series is cut to dates ≤ origin before any transformation or estimation. An interior gap in the training window (2025-10, at origins ≥ 2025-11) is filled by linear interpolation that uses only data ≤ origin. At origin 2025-10, where the origin month itself is missing, the model forecasts from 2025-09 using 2 steps (h=1) and 4 steps (h=3).
- Nothing is random (`random_seed` = null). `out/analysis.py` regenerates every output.

## Results
| horizon | n | RMSE naive | RMSE AR2_diff | MAE naive | MAE AR2_diff | 1 − RMSE ratio |
|---|---|---|---|---|---|---|
| h=1 | 35 | 0.1171 | 0.1199 | 0.0914 | 0.0957 | −0.0235 |
| h=3 | 35 | 0.1586 | 0.1597 | 0.1200 | 0.1212 | −0.0074 |

Keys: `rmse_naive_h1`, `rmse_model_h1`, `mae_naive_h1`, `mae_model_h1`, `rel_improvement_rmse_h1`, and the matching `_h3` keys. All values are in percentage points except the last column, which is a fraction.

The AR(2)-on-differences model did **not** beat the naive forecast. Its RMSE was about 2.3% higher at h=1 (`rel_improvement_rmse_h1` = −0.023) and about 0.7% higher at h=3 (`rel_improvement_rmse_h3` = −0.007). MAE gives the same ranking. The model's forecasts never differ from the naive ones by more than 0.017 pp (`out/state/checks.md`), so in practice it acts as a slightly perturbed random walk.

## Uncertainty
A Diebold–Mariano test was run on the squared-error loss differential (model minus naive). It uses a Bartlett HAC variance with h−1 lags, the Harvey–Leybourne–Newbold small-sample correction and t(n−1) reference values.
- At h=1, DM = 2.50 and p = 0.017 (`dm_stat_h1`, `dm_pvalue_h1`): the model is slightly but consistently worse.
- At h=3, DM = 1.57 and p = 0.125 (`dm_stat_h3`, `dm_pvalue_h3`): no significant difference.

Each horizon has only 35 evaluated pairs. The DM test with HAC variance is only approximate in samples this small, and the overlapping h=3 errors are autocorrelated. The size of the RMSE differences (a few thousandths of a point) is negligible compared with UNRATE's 0.1-pp reporting precision.

## Limitations and open issues
- The data are the final vintage, so the errors measure forecastability of the revised series. They do **not** show how the model would have performed in real time.
- Only one pre-specified model was tested. Other AR-type choices could rank differently, for example a shorter estimation window, excluding 2020 (the COVID spike strongly affects OLS on differences), or an AR in levels. They were not explored, to avoid tuning on the evaluation window.
- The 2025-10 gap is handled by linear interpolation for the model's lag inputs. This touches only a few origins.
- The window is short (3 years) and quiet (unemployment between about 3.4 and 4.6%), so the results may not carry over to recessions.