F3-conf-04 — Multi-horizon forecasts of Canadian quarterly CPI inflation
Question
Over 40 expanding-window origins (first_origin = 2015-10-01 to last_origin = 2025-07-01, n_origins = 40), does a seasonal AR model forecast annualised, non-seasonally-adjusted quarterly CPI inflation better than the seasonal-naive forecast at h = 1, 2, 4, and is the h = 4 squared-error loss differential significantly different from zero when its serial correlation (from overlapping 4-step errors) is accounted for?
Data and exclusions
Input data/cpi_quarterly.csv (127 rows; n_input_rows) — SHA-256 matches the README; data/cpi_monthly.csv (381 rows) was used only for an audit (quarterly averages and inflation reproduced to rounding precision, see out/state/data_audit.md). Issues (data_issues): 1995Q1 inflation is undefined and dropped; infl_annualised used as given; the 2020 COVID and 2021–23 inflation-surge observations are kept. Exclusion ledger (out/exclusions.csv): 127 → 126 (1995Q1) → 118 observations in the last estimation sample (n_analysed = 118; 4 initial lags and the 4 post-origin quarters 2025Q4–2026Q3, which serve only as targets).
Method
- Baseline
snaive: forecast of quarter t+h = inflation of quarter t+h−4. - Model
ar_seasdum(model_name,model_description): OLS regression of inflation on four quarterly seasonal intercepts plus p own lags, p ∈ {0,…,4} chosen by AIC at each origin on a common sample; estimated on all quarters ≤ origin (expanding window from 1995Q2); multi-step forecasts are iterated. AIC picked p = 1 at 15 origins and p = 2 at 25 (selected_ar_order_counts). No randomness (random_seed= null). - Evaluation: RMSE and MAE over the 40 origins per horizon; relative improvement 1 − RMSE_model/RMSE_snaive.
- h = 4 test: d_t = e²_snaive,t − e²_model,t; standard error of d̄ from a Newey–West (Bartlett kernel,
hac_kernel_h4) long-run variance with truncation laghac_lags_h4= 3 (= h − 1, fixed in advance); DM statistic compared with N(0,1), two-sided.
Results
| h | RMSE snaive | RMSE model | MAE snaive | MAE model | 1 − RMSE ratio |
|---|---|---|---|---|---|
| 1 | 2.849 (rmse_snaive_h1) |
2.192 (rmse_model_h1) |
2.137 (mae_snaive_h1) |
1.519 (mae_model_h1) |
0.231 (rel_improvement_rmse_h1) |
| 2 | 2.847 (rmse_snaive_h2) |
2.460 (rmse_model_h2) |
2.126 (mae_snaive_h2) |
1.550 (mae_model_h2) |
0.136 (rel_improvement_rmse_h2) |
| 4 | 2.882 (rmse_snaive_h4) |
2.340 (rmse_model_h4) |
2.181 (mae_snaive_h4) |
1.594 (mae_model_h4) |
0.188 (rel_improvement_rmse_h4) |
Units: percentage points of annualised inflation. The seasonal AR has lower RMSE and MAE than the seasonal-naive forecast at all three horizons (RMSE gains of 14–23 %).
At h = 4: d̄ = 2.834 squared pp (loss_diff_mean_h4), HAC SE = 2.082 (loss_diff_se_h4), DM = 1.36 (dm_stat_h4), two-sided p = 0.174 (dm_pvalue_h4). With the pre-specified test, the h = 4 gain is not statistically distinguishable from zero at conventional levels. The model had the smaller squared error at 29 of 40 origins (sensitivity_h4.n_origins_model_better_sqerr_h4).
Uncertainty
- Sensitivity of the h = 4 test (reported in
sensitivity_h4, not the primary result): Harvey–Leybourne–Newbold small-sample correction with t(39) gives p = 0.222 (hln_pvalue_t39). The sample autocorrelations of d_t at lags 3 and 4 are negative (−0.28, −0.41;d_autocorr_lag1_to_4), so longer Bartlett truncations shrink the SE: p = 0.125 at lag 4, 0.039 at lag 6, 0.018 at lag 8 (bartlett_L4_pvalue,bartlett_L6_pvalue,bartlett_L8_pvalue). The verdict is therefore sensitive to the bandwidth; the primary lag 3 was fixed from the h − 1 rule before results were seen, and we do not switch to a lag that delivers significance. With 40 overlapping observations the long-run variance is poorly estimated whichever lag is used. - The relative-improvement figures at h = 1 and h = 2 have no formal test here (not required); they are point estimates from 40 forecasts each.
Limitations and open issues
- Errors are computed on the current vintage; no real-time validity is claimed.
- The evaluation window contains the 2020 pandemic shock and the 2021–23 inflation surge; d_t is unlikely to be covariance-stationary, which the DM asymptotics assume, and a few large errors dominate squared-error loss.
- Only one model class was tried (no search across models on the evaluation sample), which avoids data snooping but means the result is specific to this specification.
- The DM test treats forecasts as given (Diebold–Mariano), not as a test of population model superiority with estimation uncertainty (West/Giacomini–White).