1# F3-conf-04 — Multi-horizon forecasts of Canadian quarterly CPI inflation23## Question4Over 40 rolling quarterly origins (2015Q4–2025Q3; `n_origins`, `first_origin`, `last_origin`), does a time-series model beat the seasonal-naive forecast of Canadian quarterly CPI inflation (400·Δln of the quarterly-average NSA CPI) at horizons 1, 2 and 4 quarters? And at h = 4, is the mean squared-error loss differential different from zero once we allow for the serial correlation that overlapping 4-step errors create?56## Data and exclusions7- `data/cpi_quarterly.csv` (127 rows) is the analysis input. `data/cpi_monthly.csv` (381 rows) was used only to check provenance (`n_input_rows`). Both SHA-256 hashes match `data/README.md`. Recomputing the quarterly averages and inflation from the monthly file reproduces the quarterly file up to rounding (`out/state/data_audit.md`).8- Exclusions (`out/exclusions.csv`): (1) 1995Q1 has no inflation value because there is no prior quarter (127 → 126). (2) The last estimation sample keeps only quarters ≤ the last origin 2025Q3 (126 → 122 = `n_analysed`). Later quarters are used only as forecast targets.9- Data issues (`data_issues`): the missing 1995Q1 value, and some large but genuine outliers (2008Q4, 2020Q2, 2022Q1–Q2). The outliers were kept as observed.1011## Method12- **Seasonal-naive:** the forecast for quarter t+h is the observed inflation of quarter t+h−4.13- **Model (`model_name` = `dsar_bic`, see `model_description`):** a direct multi-step OLS regression, one per horizon. It regresses inflation at s+h on four seasonal dummies for the target quarter and on p lags (infl_s, …, infl_{s−p+1}). The lag order p ∈ {0,…,4} is chosen by BIC on a common sample. At every origin the model is re-estimated and p is re-selected on an expanding window that starts in 1995Q2 and uses only quarters ≤ the origin. BIC chose p ∈ {0,1} at h = 1 and h = 2. At h = 4 it chose p = 0 at all 40 origins (`supplementary.selected_p_counts`), so at that horizon the model reduces to expanding-window seasonal means.14- **Metrics:** RMSE and MAE over the 40 origins for each horizon, and the relative improvement 1 − RMSE_model/RMSE_snaive.15- **h = 4 test:** d_t = e²_snaive,t − e²_model,t. The standard error of d̄ uses a Newey–West (Bartlett kernel) long-run variance with truncation lag 3 = h−1 (`hac_lags_h4`, `hac_kernel_h4`). DM = d̄/SE, with a two-sided p-value from N(0,1).16- The script is deterministic and uses no random numbers (`random_seed` = null). `out/analysis.py` regenerates `out/results.json`, `out/forecasts.csv`, `out/exclusions.csv` and `out/sensitivity_dm.csv`.1718## Results19| h | RMSE snaive | MAE snaive | RMSE model | MAE model | 1 − RMSE ratio |20|---|---|---|---|---|---|21| 1 | 2.849 (`rmse_snaive_h1`) | 2.138 (`mae_snaive_h1`) | 2.197 (`rmse_model_h1`) | 1.561 (`mae_model_h1`) | 0.229 (`rel_improvement_rmse_h1`) |22| 2 | 2.847 (`rmse_snaive_h2`) | 2.126 (`mae_snaive_h2`) | 2.361 (`rmse_model_h2`) | 1.533 (`mae_model_h2`) | 0.171 (`rel_improvement_rmse_h2`) |23| 4 | 2.882 (`rmse_snaive_h4`) | 2.181 (`mae_snaive_h4`) | 2.355 (`rmse_model_h4`) | 1.602 (`mae_model_h4`) | 0.183 (`rel_improvement_rmse_h4`) |2425Units are annualised percentage points. On both RMSE and MAE, the model beats the seasonal-naive forecast at all three horizons. RMSE falls by about 17–23%.2627At h = 4: d̄ = 2.761 squared pp (`loss_diff_mean_h4`), HAC SE = 2.148 (`loss_diff_se_h4`), DM = 1.286 (`dm_stat_h4`), two-sided p = 0.199 (`dm_pvalue_h4`). At conventional levels, the h = 4 squared-loss advantage cannot be distinguished from zero.2829## Uncertainty30- The p-value of the h = 4 test depends on the HAC choice (`supplementary.dm_sensitivity_h4`; also `out/sensitivity_dm.csv`):3132 | kernel | lag | SE | DM | p (normal) |33 |---|---|---|---|---|34 | Bartlett | 3 (primary) | 2.15 | 1.29 | 0.20 |35 | Bartlett | 4 | 1.91 | 1.45 | 0.15 |36 | Bartlett | 6 | 1.42 | 1.95 | 0.05 |37 | Rectangular | 3 | 1.75 | 1.58 | 0.11 |3839 The rectangular lag-3 version with the Harvey–Leybourne–Newbold small-sample correction and t(39) gives p ≈ 0.16. None of these settings reaches the 5% level. The Bartlett lag-6 variant comes closest (p ≈ 0.052).40- With T = 40 overlapping errors, the HAC variance is estimated imprecisely and the normal approximation is rough.41- The loss differential is dominated by a few quarters, especially around the 2021–2022 inflation surge. As a descriptive check, dropping targets dated 2021–2022 gives a mean differential of 3.70 (`supplementary.loss_diff_mean_h4_excl_2021_2022_targets`). So the model's average advantage is not driven only by the surge, but it remains noisy.42- No significance tests were run at h = 1 and h = 2. The task does not ask for them, so the improvements there are point estimates only. The mean loss differentials are 3.29 at h = 1 and 2.53 at h = 2 (`supplementary.mean_loss_diff_h1`, `supplementary.mean_loss_diff_h2`).4344## Limitations and open issues45- Errors are computed on the current data vintage, not on real-time vintages. Each quarter is assumed known at the end of that quarter, while actual publication comes about three weeks later. No claim of real-time validity is made.46- The model is a single, deliberately simple specification and was not tuned on the evaluation period. Its h = 4 version is just seasonal means. Other models (SARIMA, models using monthly data) were not tried.47- The DM test treats the model's forecasts as given and ignores estimation uncertainty in the parameters; it is a test of these specific forecasts (Giacomini–White view). The primary HAC choice (Bartlett, lag 3) meets the minimum lag the task requires. That kernel puts reduced weight on the MA(3) autocovariances, and the alternative choices give somewhat smaller p-values, as shown above.48- `n_analysed` (122) counts the quarterly observations up to the last origin. The last regressions use 118, 117 and 115 observations for h = 1, 2 and 4, because of the lags and the horizon offset (`supplementary.n_reg_last_origin`).49