# F3-dev-01 — Can AR(2) on differences beat the naive forecast of US unemployment?

## Question
Over 36 monthly forecast origins from 2023-07-01 to 2026-06-01 (`first_origin`, `last_origin`, `n_origins`), does a simple autoregressive model forecast the US civilian unemployment rate (FRED UNRATE, %, SA) better than the naive no-change forecast at 1- and 3-month horizons? Each window grows with the origin (expanding window). I measure accuracy with RMSE and MAE in percentage points, and the relative improvement 1 − RMSE_model/RMSE_naive.

## Data and exclusions
- `data/UNRATE.csv` (SHA-256 621ff6fa…a06c, which matches the README) has 945 monthly rows from 1948-01 to 2026-09 (`n_input_rows`). There are no gaps or duplicate dates. Full audit: `out/state/data_audit.md`.
- One value is missing: 2025-10-01 (`data_issues`). Two (origin, horizon) pairs target that month: (2025-09, h=1) and (2025-07, h=3). Both stay in `out/forecasts.csv` with an empty `actual` and are left out of the error statistics. That leaves 35 evaluated pairs per horizon (`n_evaluated_h1`, `n_evaluated_h3`).
- Exclusion ledger (`out/exclusions.csv`): 945 raw rows, minus 1 missing value, minus 3 observations dated after the last origin, gives 941 observations in the last training window (`n_analysed`). That window yields 935 usable regression rows (`n_regression_rows_last_window`).
- The 2020 COVID observations (14.8% in 2020-04) are real data points. I kept them, a choice made before any forecasts were evaluated.

## Method
- **Naive:** the forecast is the latest non-missing observation dated ≤ origin. For origin 2025-10, whose own value is missing, this is the 2025-09 value.
- **Model `AR2_diff`** (`model_name`, `model_description`): an AR(2) with intercept on monthly first differences, fitted by OLS. At each origin it is re-estimated on all observations from 1948-01 up to that origin. Regression rows that involve the missing month are dropped. Forecasts are iterated h steps ahead and the predicted differences are added to the last observed level. For origin 2025-10, the model starts from 2025-09 and runs h+1 steps. For origins 2025-11 and 2025-12, the missing 2025-10 level needed for the lag state is filled by linear interpolation between 2025-09 and 2025-11. Both of those dates are ≤ the origin, so no future information is used.
- I fixed the specification in `out/state/log.md` before computing any test-period errors and did no model selection on the test window. The method uses no randomness (`random_seed` = null).
- Every transformation and estimate uses only data dated ≤ origin. A perturbation test confirmed this: setting all post-origin data to 99 left the forecasts unchanged (`out/state/checks.md`).

## Results
| horizon | n | RMSE naive | RMSE AR2_diff | MAE naive | MAE AR2_diff | 1 − RMSE ratio |
|---|---|---|---|---|---|---|
| 1 | 35 | 0.1171 (`rmse_naive_h1`) | 0.1199 (`rmse_model_h1`) | 0.0914 (`mae_naive_h1`) | 0.0957 (`mae_model_h1`) | −0.0235 (`rel_improvement_rmse_h1`) |
| 3 | 35 | 0.1586 (`rmse_naive_h3`) | 0.1598 (`rmse_model_h3`) | 0.1200 (`mae_naive_h3`) | 0.1213 (`mae_model_h3`) | −0.0076 (`rel_improvement_rmse_h3`) |

- The AR(2) model did **not** beat the naive forecast. Its RMSE was about 2.3% higher at h=1 and 0.8% higher at h=3 (`rel_improvement_rmse_h1`, `rel_improvement_rmse_h3`), and its MAE was also higher at both horizons.
- The two forecasts are almost identical: they never differ by more than 0.016 pp (`out/state/checks.md`). In the last window the AR coefficients are close to zero (φ1 = 0.037, φ2 = −0.042, `ar_coefficients_last_window`), so the fitted model is essentially a random walk.
- Both methods tended to under-forecast slightly over this period, when unemployment was drifting up. Mean error (actual − forecast) was about +0.017 pp at h=1 and +0.04 pp at h=3 for both (`mean_error_naive_h1`, `mean_error_model_h1`, `mean_error_naive_h3`, `mean_error_model_h3`).

## Uncertainty
- I ran a Diebold–Mariano test on squared-error loss, with HAC variance using h−1 lags and the Harvey–Leybourne–Newbold small-sample correction (t reference with n−1 df).
  - h=1: statistic = 2.50, two-sided p = 0.017 (`dm_stat_h1`, `dm_pvalue_two_sided_h1`). The model's loss is higher, but the RMSE difference is tiny (0.003 pp).
  - h=3: statistic = 1.36, p = 0.18 (`dm_stat_h3`, `dm_pvalue_two_sided_h3`).
- These p-values are only indicative, for three reasons:
  - The AR model nests the random walk, so the standard DM test is not correctly sized. A Clark–West-type adjustment would be more suitable, and I did not run it.
  - There are only 35 evaluation pairs, and the h=3 errors overlap.
  - The pairs on either side of the missing month were treated as consecutive.
- The data support "no improvement over naive in this window". They do not support any claim about a practically meaningful difference.

## Limitations and open issues
- **Final-vintage data:** the errors are pseudo-out-of-sample on revised data. They are not real-time performance, and I make no claim about real-time validity.
- **Short window:** the evaluation covers only three years (35 pairs per horizon) of fairly calm, slowly rising unemployment. The result may not carry over to recessions or turning points.
- **COVID observations:** these values were kept in the estimation sample. A probable reason the AR coefficients are near zero is that OLS is dominated by the 2020 swings, which would flatten the model into a random walk. I did not test this; a sensitivity check (a later sample start or outlier dummies) was not run because the task asks for one pre-specified model.
- **Missing 2025-10 value:** two pairs are unevaluated. The interpolated lag state for origins 2025-11 and 2025-12 is an assumption, but it affects the model forecast at most marginally.
- Only one model was evaluated, as the task specifies. The conclusion is about this AR(2)-on-differences specification, not about all time-series models.