F3-conf-04 — Multi-horizon forecasts of Canadian quarterly CPI inflation
Question
Across 40 rolling origins from 2015Q4 to 2025Q3, does a simple time-series model forecast Canadian quarterly CPI inflation (400·Δln of the quarterly-average CPI, not seasonally adjusted) better than the seasonal-naive forecast at h = 1, 2 and 4? And at h = 4, is the mean squared-error loss differential distinguishable from zero once we use a HAC standard error that allows for the serial correlation caused by overlapping forecast errors?
Data and exclusions
data/cpi_quarterly.csvhas 127 rows read (1995Q1–2026Q3).data/cpi_monthly.csvhas 381 rows read and was used only for checks: every quarter has 3 months,cpi_avgmatches the monthly mean (max abs diff 3e-7), andinfl_annualisedmatches 400·Δln(cpi_avg) (max abs diff 3e-6). I found no duplicate quarters and no gaps.- 1995Q1 has no inflation value, so it is left out of estimation. Nothing else is excluded.
- The evaluation window includes the 2020 COVID shock and the 2021–23 inflation surge. I did not winsorise these quarters.
- The data are a current vintage. I make no claim of real-time validity.
Method
- Seasonal naive. The forecast for quarter t+h is the observed value of quarter t+h−4.
- Model (
ar_sdum_aic). OLS of inflation on four quarter-of-year intercept dummies plus p own lags.- The window expands from 1995Q2 to the origin, so each forecast uses only data dated ≤ origin.
- At each origin, p ∈ {0,…,4} is chosen by AIC on a common sample (starting after the first 4 observations). The chosen model is then re-estimated on the full window.
- Multi-step forecasts are built by iterating the model forward. AIC chose p = 2 up to 2021Q4 and p = 1 from 2022Q1 on.
- The last estimation sample has 121 observations.
- Forecast errors. e = actual − forecast. RMSE and MAE are computed over 40 forecasts per horizon.
- h = 4 test. d_t = e²_snaive − e²_model.
- The standard error of d̄ uses a Bartlett (Newey–West) long-run variance with truncation lag 4. This gives non-zero weight to autocovariances 1–4, which covers the required h−1 = 3.
- I chose this specification before looking at the test results.
- The p-value is two-sided and uses a Student t with 39 df.
- No randomness is involved, so no seed was set.
Results
| h | RMSE snaive | RMSE model | MAE snaive | MAE model | 1 − RMSE ratio |
|---|---|---|---|---|---|
| 1 | 2.849 | 2.193 | 2.137 | 1.519 | 0.230 |
| 2 | 2.847 | 2.463 | 2.126 | 1.553 | 0.135 |
| 4 | 2.882 | 2.343 | 2.181 | 1.597 | 0.187 |
(Errors are in percentage points, annualised.) At every horizon the model has lower RMSE and lower MAE than the seasonal-naive forecast. The relative RMSE improvement is 13–23%.
At h = 4, d̄ = 2.818 squared pp, with HAC SE 1.851 (Bartlett, lag 4). This gives DM = 1.52 and p = 0.136. At the 5% level the difference is not statistically distinguishable from zero.
Uncertainty
The DM result changes with the truncation lag chosen for the long-run variance. All variants below are in results.json → supplementary.dm_sensitivity:
| Variant | DM | p |
|---|---|---|
| Bartlett, lag 3 | 1.35 | 0.18 |
| Rectangular, lag 3 | 1.66 | 0.11 |
| Rectangular lag 3 with Harvey–Leybourne–Newbold correction | 1.51 | 0.14 |
| Bartlett, lag 6 | 2.04 | 0.048 |
| Bartlett, lag 8 | 2.35 | 0.024 |
- Longer lags shrink the SE, which suggests negative higher-order autocorrelation in d_t.
- With only 40 overlapping observations, a long-run variance estimate is imprecise. I keep the pre-specified lag-4 result as primary rather than picking the lag that gives significance.
- The fair summary is weak, fragile evidence in favour of the model at h = 4.
Supplementary DM tests at the other horizons are also not significant at 5%:
- h = 1: d̄ = 3.31, p = 0.105 (Bartlett lag 1).
- h = 2: d̄ = 2.04, p = 0.447 (Bartlett lag 2).
If targets in 2020–2023 are dropped, the h = 4 mean loss differential falls to 1.27 (n = 24). So part of the advantage comes from the volatile period.
Limitations and open issues
- There are only 40 origins and their errors overlap. As a result, DM inference is low-powered and depends on the lag choice.
- The evaluation window has a structural break (pandemic and the inflation surge). Squared-error differentials are dominated by a few quarters.
- The model is deliberately simple: no exogenous predictors, constant seasonality, and an expanding window. Other model choices could rank differently. I selected only one model and did no model search on evaluation data.
- The data are a current vintage, not real-time.
- I did not apply the Giacomini–White or Clark–West adjustments. The models are not nested in the DM sense, since seasonal naive is a restricted random walk on the 4th lag with no estimation.