F3-conf-04 — Multi-horizon forecasts of Canadian quarterly CPI inflation
Question
Across 40 rolling origins (2015Q4–2025Q3), does a time-series model forecast Canadian quarterly CPI inflation (400·Δln of the quarterly-average NSA CPI) better than the seasonal-naive forecast at h = 1, 2, 4? And at h = 4, is the mean squared-error loss differential distinguishable from zero once the serial correlation caused by overlapping forecast errors is accounted for (a Diebold–Mariano test with a HAC standard error)?
Data and exclusions
data/cpi_quarterly.csv: 127 rows (1995Q1–2026Q3);data/cpi_monthly.csv: 381 rows (provenance only). I checked that the quarterlycpi_avgequals the mean of the monthly values, and thatinfl_annualisedequals 400·Δlncpi_avg. Both checks passed (tolerance 1e-5).- The only exclusion is 1995Q1, which has no inflation value because it is the first difference. That leaves 126 inflation observations. All 120 targets (40 origins × 3 horizons, last target 2026Q3) are observed.
- The information rule is enforced in code: each forecast uses only quarters ≤ origin. The current data vintage is used, so no real-time validity is claimed.
Method
- Baseline (
snaive): the forecast for quarter t+h is the inflation value of quarter t+h−4. - Model (
ar_sd_aic): OLS regression of inflation on an intercept, three quarterly seasonal dummies and p own lags.- p ∈ {0,…,4} is chosen by AIC at every origin. All candidate p are compared on a common sample that starts at 1996Q2.
- The window expands up to the origin. Multi-step forecasts are iterated, and the dummies are set for the target quarter.
- AIC chose p = 2 for the origins 2015Q4–2021Q4 and p = 1 for 2022Q1 onward.
- The last estimation sample (origin 2025Q3) has 118 observations.
- h = 4 test: d_t = e²_snaive − e²_model over the 40 origins. The standard error of d̄ uses a Newey–West long-run variance (Bartlett kernel, truncation lag 3 = h−1, chosen before seeing results). The p-value is two-sided and normal-based.
- The procedure is deterministic, with no random numbers.
Results
| h | RMSE snaive | RMSE model | MAE snaive | MAE model | 1 − RMSE ratio |
|---|---|---|---|---|---|
| 1 | 2.849 | 2.192 | 2.137 | 1.519 | 0.231 |
| 2 | 2.847 | 2.460 | 2.126 | 1.550 | 0.136 |
| 4 | 2.882 | 2.340 | 2.181 | 1.594 | 0.188 |
(Units: percentage points, annualised.) The model has lower RMSE and lower MAE than the seasonal-naive forecast at every horizon, with RMSE gains of 14–23%.
At h = 4, d̄ = 2.834 squared pp. The HAC standard error is 2.082 (Bartlett, 3 lags), giving DM = 1.36 and p = 0.174. At conventional levels, the h = 4 difference cannot be distinguished from zero.
Uncertainty
Sensitivity checks, printed by analysis.py and not stored in results.json:
- Small-sample correction: with the Harvey–Leybourne–Newbold correction and a t(39) reference, the statistic is 1.24 (p ≈ 0.22).
- Alternative HAC choices: a rectangular kernel with 3 lags gives DM = 1.67. Bartlett with 4 lags gives 1.53.
- Longer Bartlett lags (6 and 8) give 2.06 and 2.37. This is because of negative higher-order autocorrelations in d_t. These lags were not pre-specified and I do not use them for inference. With only 40 overlapping observations, the conclusion depends on the HAC choice.
- Shorter horizons (descriptive only): Bartlett HAC with 3 lags gives DM ≈ 1.75 at h = 1 and ≈ 0.80 at h = 2.
- Excluding the 2021–2023 inflation surge (targets dated 2021Q1–2023Q4, n = 28 at h = 4): RMSE is 2.03 for snaive and 1.52 for the model. The ranking holds outside the surge period.
Limitations and open issues
- Only one model specification was evaluated, and no search over models was done. Forty origins give low power, and the surge years strongly influence the losses.
- The DM test is applied to forecasts from an expanding-window estimated model. It is interpreted as a test of equal forecast accuracy for these specific procedures, not of population models.
- The data are the current vintage, with occasional basket and chain updates. Results are not real-time.
- The normal reference distribution and the HAC choice matter in this small sample (see Uncertainty).