# F3-dev-01 — Can AR(2) on differences beat the naive forecast of US unemployment?

# Question

Over 36 monthly forecast origins from 2023-07-01 to 2026-06-01 (first_origin, last_origin, n_origins), does a simple autoregressive model forecast the US civilian unemployment rate (FRED UNRATE, %, SA) better than the naive no-change forecast at 1- and 3-month horizons? Each window grows with the origin (expanding window). I measure accuracy with RMSE and MAE in percentage points, and the relative improvement 1 − RMSE_model/RMSE_naive.

# Data and exclusions

• data/UNRATE.csv (SHA-256 621ff6fa…a06c, which matches the README) has 945 monthly rows from 1948-01 to 2026-09 (n_input_rows). There are no gaps or duplicate dates. Full audit: out/state/data_audit.md.
• One value is missing: 2025-10-01 (data_issues). Two (origin, horizon) pairs target that month: (2025-09, h=1) and (2025-07, h=3). Both stay in out/forecasts.csv with an empty actual and are left out of the error statistics. That leaves 35 evaluated pairs per horizon (n_evaluated_h1, n_evaluated_h3).
• Exclusion ledger (out/exclusions.csv): 945 raw rows, minus 1 missing value, minus 3 observations dated after the last origin, gives 941 observations in the last training window (n_analysed). That window yields 935 usable regression rows (n_regression_rows_last_window).
• The 2020 COVID observations (14.8% in 2020-04) are real data points. I kept them, a choice made before any forecasts were evaluated.

# Method

• Naive: the forecast is the latest non-missing observation dated ≤ origin. For origin 2025-10, whose own value is missing, this is the 2025-09 value.
• Model AR2_diff (model_name, model_description): an AR(2) with intercept on monthly first differences, fitted by OLS. At each origin it is re-estimated on all observations from 1948-01 up to that origin. Regression rows that involve the missing month are dropped. Forecasts are iterated h steps ahead and the predicted differences are added to the last observed level. For origin 2025-10, the model starts from 2025-09 and runs h+1 steps. For origins 2025-11 and 2025-12, the missing 2025-10 level needed for the lag state is filled by linear interpolation between 2025-09 and 2025-11. Both of those dates are ≤ the origin, so no future information is used.
• I fixed the specification in out/state/log.md before computing any test-period errors and did no model selection on the test window. The method uses no randomness (random_seed = null).
• Every transformation and estimate uses only data dated ≤ origin. A perturbation test confirmed this: setting all post-origin data to 99 left the forecasts unchanged (out/state/checks.md).

# Results

• The AR(2) model did not beat the naive forecast. Its RMSE was about 2.3% higher at h=1 and 0.8% higher at h=3 (rel_improvement_rmse_h1, rel_improvement_rmse_h3), and its MAE was also higher at both horizons.
• The two forecasts are almost identical: they never differ by more than 0.016 pp (out/state/checks.md). In the last window the AR coefficients are close to zero (φ1 = 0.037, φ2 = −0.042, ar_coefficients_last_window), so the fitted model is essentially a random walk.
• Both methods tended to under-forecast slightly over this period, when unemployment was drifting up. Mean error (actual − forecast) was about +0.017 pp at h=1 and +0.04 pp at h=3 for both (mean_error_naive_h1, mean_error_model_h1, mean_error_naive_h3, mean_error_model_h3).

# Uncertainty

• I ran a Diebold–Mariano test on squared-error loss, with HAC variance using h−1 lags and the Harvey–Leybourne–Newbold small-sample correction (t reference with n−1 df).
  • h=1: statistic = 2.50, two-sided p = 0.017 (dm_stat_h1, dm_pvalue_two_sided_h1). The model's loss is higher, but the RMSE difference is tiny (0.003 pp).
  • h=3: statistic = 1.36, p = 0.18 (dm_stat_h3, dm_pvalue_two_sided_h3).
• These p-values are only indicative, for three reasons:
  • The AR model nests the random walk, so the standard DM test is not correctly sized. A Clark–West-type adjustment would be more suitable, and I did not run it.
  • There are only 35 evaluation pairs, and the h=3 errors overlap.
  • The pairs on either side of the missing month were treated as consecutive.
• The data support "no improvement over naive in this window". They do not support any claim about a practically meaningful difference.

# Limitations and open issues

• Final-vintage data: the errors are pseudo-out-of-sample on revised data. They are not real-time performance, and I make no claim about real-time validity.
• Short window: the evaluation covers only three years (35 pairs per horizon) of fairly calm, slowly rising unemployment. The result may not carry over to recessions or turning points.
• COVID observations: these values were kept in the estimation sample. A probable reason the AR coefficients are near zero is that OLS is dominated by the 2020 swings, which would flatten the model into a random walk. I did not test this; a sensitivity check (a later sample start or outlier dummies) was not run because the task asks for one pre-specified model.
• Missing 2025-10 value: two pairs are unevaluated. The interpolated lag state for origins 2025-11 and 2025-12 is an assumption, but it affects the model forecast at most marginally.
• Only one model was evaluated, as the task specifies. The conclusion is about this AR(2)-on-differences specification, not about all time-series models.