# F3-dev-01 — Rolling-origin forecasts of the US unemployment rate: AR model vs naive

## Question
Using 36 expanding-window forecast origins (2023-07 through 2026-06), does a simple autoregressive model forecast the US unemployment rate (FRED UNRATE) better than the naive no-change forecast at horizons of 1 and 3 months? I measure this with out-of-sample RMSE and MAE, and with the relative RMSE improvement 1 − RMSE_model/RMSE_naive.

## Data and exclusions
- `data/UNRATE.csv`: 945 data rows (1948-01 to 2026-09), monthly, seasonally adjusted, final vintage. There are no duplicate dates and no gaps in the monthly calendar.
- **Missing value:** 2025-10-01 has no value. It is the target for (origin 2025-09, h=1) and (origin 2025-07, h=3). Both pairs appear in `forecasts.csv` with an empty `actual` and are left out of the error statistics. That leaves **35 evaluated pairs per horizon**.
- **COVID outliers:** UNRATE hit 14.8 in 2020-04, with monthly changes of up to +10.4 pp. In the primary model, regression rows whose dependent or lagged difference is dated 2020-04 to 2020-09 are left out of estimation. This rule was fixed using only pre-2023 information, so it is the same at every origin. A sensitivity check without the exclusion is reported below.
- The 1949-10 value (7.9, a strike-related spike) is kept as published.

## Method
- **Naive:** the forecast is the latest observation dated ≤ origin. At origin 2025-10, whose own value is missing, that means 2025-09.
- **Model `AR2_diff`:** an AR(2) with intercept on first differences, Δu_t = c + φ1 Δu_{t−1} + φ2 Δu_{t−2} + e_t. It is estimated by OLS at each origin, using all regression rows from 1948 up to the origin (expanding window), minus the COVID rows and any rows involving the missing 2025-10 value. The fitted differences are iterated recursively and accumulated into levels for h = 1, 3. The lag order of 2 was fixed in advance; there was no data-driven selection and no tuning on the evaluation period.
- **Missing value inside the model:** once 2025-11 is observed, 2025-10 is linearly interpolated, using only data ≤ origin, so that the forecast has a starting state. It is never used in estimation. At origin 2025-10 the model forecasts from 2025-09, iterating h+1 steps.
- Every estimation step and transformation uses only data dated ≤ origin. Errors are computed against the final-vintage values in the file. There are 927 regression rows in the last training window (origin 2026-06).
- Replication: `~/wp29/venv/bin/python out/analysis.py` regenerates `results.json`, `forecasts.csv` and `supplementary.json`, deterministically (bootstrap seed 12345).

## Results
Errors are in percentage points. n = 35 per horizon.

| Horizon | RMSE naive | RMSE AR2_diff | MAE naive | MAE AR2_diff | 1 − RMSE ratio |
|---|---|---|---|---|---|
| h = 1 | 0.1171 | 0.1183 | 0.0914 | 0.0976 | −0.010 |
| h = 3 | 0.1586 | 0.1742 | 0.1200 | 0.1402 | −0.099 |

The AR(2)-in-differences model does **not** beat the naive forecast on either metric at either horizon. At h=1 it is about 1% worse in RMSE; at h=3 it is about 10% worse.

Sensitivity, from `supplementary.json`: if the 2020 rows are kept in estimation, the relative improvement is −0.023 (h=1) and −0.008 (h=3). So the model is still slightly worse than naive. The h=3 shortfall is smaller here because the estimated coefficients pick up less momentum. The estimated momentum (φ2 ≈ 0.28 at the last origin, intercept ≈ 0) pushed the forecasts upward during the 2024–2025 drift, and they then overshot when unemployment flattened and fell in 2026.

## Uncertainty
- Diebold–Mariano test (squared loss, HAC variance with h−1 lags, Harvey–Leybourne–Newbold small-sample correction, t(34) reference):
  - h=1: statistic −0.29, p = 0.78. No detectable difference.
  - h=3: statistic −2.19, p = 0.036. The naive forecast is nominally better.
- Moving-block bootstrap over origins (block 6, 2000 replications), 90% interval for 1 − RMSE ratio:
  - h=1: [−0.104, 0.050]
  - h=3: [−0.191, −0.044]
- Caveats on these inferences:
  - With only 35 overlapping evaluations in a calm, low-volatility period, they are fragile.
  - At h=3 the overlapping errors make the HAC variance and the bootstrap imprecise.
  - The p = 0.036 at h=3 should not be read as strong evidence. The sensitivity specification, which differs only in the COVID treatment, shrinks the gap to about −1%. That means the h=3 result depends on a specification choice.

## Limitations and open issues
- Final-vintage data, so the results say nothing about real-time performance. Seasonal-factor revisions are applied retroactively.
- Errors are reported to sub-0.1 pp precision, but the data are rounded to 0.1 pp. Many naive errors are exactly 0 or ±0.1, so differences between models are small relative to that granularity.
- Only one model specification was evaluated as primary (plus one sensitivity). Other models (ARMA, levels AR with mean reversion, models with covariates) were not tried, and the conclusion applies only to this simple class.
- The evaluation window is short (3 years) and covers a single regime, a slow drift up from 3.5% to about 4.4% followed by a plateau. It does not include a turning point such as a recession.
- The COVID-exclusion window (2020-04..2020-09) is a judgement call. The results are sensitive to it, especially at h=3 (see Results).