F3-conf-01 — Forecasting Canadian monthly GDP yoy growth under publication lags
Question
At the end of each month t (48 origins, 2022-05 to 2026-04), how accurately can year-over-year growth of Canadian monthly real GDP be forecast for months t+1 and t+3, using only data published by then? And does a model that adds LFS employment growth, which comes out sooner, beat the naive forecast (the last published GDP yoy, i.e. month t−2)?
Data and exclusions
data/canada_gdp_employment.csv: 345 rows, 1998-01 to 2026-09. GDP is missing for 2026-08 and 2026-09 because it has not been published yet. Current vintage only.- Checks: months are unique and consecutive. From 1999-01 onward,
gdp_yoyandemp_yoymatch their levels exactly at 4 decimals. The 1998 yoy values are filled in, although the README says they should be empty, and they cannot be verified because the file has no 1997 levels. They were used as given. - Information set at origin t: GDP up to month t−2, employment up to month t−1. Every training target, regressor and plug-in value obeys this. The ledger records the latest month used for each series. At the last origin (2026-04) the latest available data are GDP for 2026-02 and employment for 2026-03.
- COVID exclusion: I chose this before computing any out-of-sample errors. Training pairs whose target or regressor month falls in 2020-03..2021-12 are dropped from the model's estimation sample, because yoy growth ranges from −16% to +18.5% in that period. At the forecast step, the model still plugs in the latest available values.
Method
- Naive: the forecast is
gdp_yoy(t−2)for both horizons. - Model
ardl_direct_emp: a direct h-step OLS regression, y(s+h) = a + b·y(s−2) + c·e(s−1).- y is GDP yoy and e is employment yoy.
- It is re-estimated at every origin and for each horizon, on an expanding window starting in 1998.
- Only pseudo-origins s with s+h ≤ t−2 are used, so every training target is already published at the origin.
- There is no model selection and no tuning. The single specification was fixed in advance.
- At the last origin the estimation sample has 310 observations for h=1 and 306 for h=3;
n_analysedreports the h=1 figure.
- Errors: forecast − actual against the current-vintage
gdp_yoy, giving RMSE and MAE over 48 origins per horizon.
Results (percentage points)
| RMSE naive | RMSE model | MAE naive | MAE model | 1 − RMSE ratio | |
|---|---|---|---|---|---|
| h=1 | 0.814 | 0.743 | 0.642 | 0.604 | 0.087 |
| h=3 | 1.140 | 0.902 | 0.872 | 0.753 | 0.208 |
The employment-augmented direct model beats the naive forecast at both horizons, on both RMSE and MAE. The gain is larger at h=3, where the naive forecast is effectively 5 months stale.
Uncertainty
- These are supplementary Diebold–Mariano tests on squared-error loss, using a HAC variance with h−1 lags, the Harvey small-sample correction, and t(47) reference values. They are reported in
results.jsonundersupplementary_dm_test.- h=1: DM = −2.03, p ≈ 0.048.
- h=3: DM = −1.65, p ≈ 0.105.
- With only 48 overlapping origins, the h=1 gain is borderline significant and the h=3 gain is not significant at 5%, despite being the larger gain.
- Sensitivity check, run with
out/analysis.py --no-covid-exclusion, which prints results but writes nothing: the outcome depends on the COVID handling.- Without the exclusion, the relative RMSE improvement is 0.012 at h=1 (RMSE 0.804) and 0.304 at h=3 (RMSE 0.794).
- The model's ranking against naive does not change, but how large the gain is depends on this choice.
Limitations and open issues
- I used current-vintage data and simulated only the publication timing. The results are not real-time forecast accuracy, because GDP and LFS revisions are ignored.
- The evaluation window, 2022-05 to 2026-07, is short and dominated by the post-COVID normalisation of growth. The errors overlap across origins (serial correlation), especially at h=3.
- The COVID exclusion window is a judgement call; see the sensitivity check.
- I evaluated one model only. There was no search over specifications, so there is no selection bias, but there is also no claim that this model is the best one.