F3-conf-04 — Multi-horizon forecasts of Canadian quarterly CPI inflation
Question
Over 40 rolling origins (2015Q4–2025Q3), does a time-series model forecast Canadian quarterly CPI inflation (400·Δln of the quarterly average CPI, not seasonally adjusted) better than the seasonal-naive forecast at h = 1, 2 and 4? And at h = 4, is the mean squared-error loss differential distinguishable from zero once we allow for the serial correlation that overlapping multi-step errors create?
Data and exclusions
- Input:
data/cpi_quarterly.csv, 127 rows (1995Q1–2026Q3).data/cpi_monthly.csv(381 rows) was used only as a check. I recomputed the quarterly averages and inflation from the monthly file. The largest differences were 3.3e-7 forcpi_avgand 3.3e-6 for inflation, which is rounding only. - 1995Q1 has no inflation value because there is no earlier quarter, so I dropped it. The inflation series therefore runs from 1995Q2. There are no other gaps and no missing values.
- Targets run from 2016Q1 to 2026Q3. Quarters after 2026Q3 are not needed. The last estimation sample (origin 2025Q3, AR(1)) uses 121 observations.
Method
- Seasonal naive: the forecast for quarter t+h is the value of quarter t+h−4.
- Model (
ar_seas_aic): an autoregression with four quarterly seasonal intercepts (no common constant), estimated by OLS. At each origin:- the lag order p ∈ {0,…,4} is chosen by AIC, comparing all orders on the same sample;
- the chosen model is re-estimated on the full expanding window from 1995Q2 up to the origin;
- forecasts for h = 2 and h = 4 are built by iterating the 1-step forecast forward.
- Each forecast uses only quarters dated ≤ origin. This covers estimation and lag selection. No transformations are applied.
- AIC chose p = 2 at origins 2015Q4–2021Q4 and p = 1 from 2022Q1 onward (see
out/diagnostics.json). - h = 4 test: d_t = e²_snaive,t − e²_model,t over the 40 origins, so a positive value means the model did better. The standard error of the mean d̄ uses a Bartlett (Newey–West) long-run variance with truncation lag 4. This is above the required minimum of h−1 = 3: with a Bartlett kernel, a lag of 3 gives only weight 0.25 to autocorrelation at lag 3. I fixed the lag before looking at the test result. DM = d̄/SE, with a two-sided p-value from the standard normal.
Results
RMSE and MAE are in annualised percentage points.
| h | RMSE snaive | RMSE model | MAE snaive | MAE model | 1 − RMSE ratio |
|---|---|---|---|---|---|
| 1 | 2.849 | 2.193 | 2.137 | 1.519 | 0.230 |
| 2 | 2.847 | 2.463 | 2.126 | 1.553 | 0.135 |
| 4 | 2.882 | 2.343 | 2.181 | 1.597 | 0.187 |
- The model has lower RMSE and lower MAE than the seasonal naive at all three horizons. The RMSE improvement ranges from 13% to 23%.
- At h = 4: d̄ = 2.818 squared pp, HAC SE = 1.851 (Bartlett, lag 4), DM = 1.52, two-sided p = 0.128.
Uncertainty
The h = 4 improvement is not statistically distinguishable from zero at the 5% level with the chosen HAC estimator. The conclusion depends on the HAC settings, as the sensitivity checks in out/diagnostics.json show:
| HAC setting | DM | p-value (normal) | Notes |
|---|---|---|---|
| Bartlett, lag 3 | 1.35 | 0.18 | |
| Bartlett, lag 4 (main) | 1.52 | 0.13 | t(39) p = 0.14; HLN-corrected p = 0.17 |
| Rectangular, lag 3 | 1.66 | 0.10 | |
| Bartlett, lag 6 | 2.04 | 0.04 | HLN-corrected t(39) p = 0.07 |
- Only the larger-lag setting gets below 5%, and the small-sample (HLN) correction removes even that. I therefore treat the evidence as weak, not conclusive.
- With 40 origins, d_t is heavy-tailed and driven by a few quarters, especially the volatile 2021–2023 inflation period. HAC inference at this sample size is imprecise.
- As supplementary information, the same test at the other horizons gives DM ≈ 1.66 (p ≈ 0.10) at h = 1 and DM ≈ 0.80 (p ≈ 0.42) at h = 2, both using Bartlett lag 1.
Limitations and open issues
- The data are the current vintage, not the real-time vintages. CPI is essentially unrevised, but I make no claim that these results reflect real-time performance.
- I specified one model. I did not search over models, which would raise multiple-testing concerns. A different model could do better or worse.
- The DM test treats the forecasts as given. It does not account for parameter-estimation uncertainty or for the fact that the model nests a seasonal-mean structure (cf. Clark–West). Its asymptotic justification is weak with 40 observations.
- I chose the HAC lag (4, Bartlett) myself. As shown above, the inference depends on that choice.