F3-conf-01 — Forecasting Canadian monthly GDP YoY growth under publication lags
Question
At the end of each month t (48 origins, first_origin = 2022-05-01 to last_origin = 2026-04-01; n_origins = 48), how accurately can year-over-year growth of Canadian monthly real GDP for months t+1 and t+3 be forecast using only data published by then? And does a model that adds the timelier LFS employment growth beat the naive forecast (latest published GDP growth, month t−2)?
Data and exclusions
- One input,
data/canada_gdp_employment.csv(SHA-256 matches the README): 345 rows (n_input_rows), 1998-01 to 2026-09, no gaps or duplicates. GDP is empty for 2026-08/09; I never need those months. The YoY columns match a recomputation from the levels exactly. Full audit:out/state/data_audit.md. - Information rule, applied per origin in code: GDP months ≤ t−2, employment months ≤ t−1, training targets ≤ t−2. At the last origin this means GDP up to
last_target_date_available= 2026-02-01 and employment up tolast_predictor_date_available= 2026-03-01. A perturbation test (randomising every value not yet available at origin 2024-01) left that origin's forecasts unchanged (out/state/checks.md). - Exclusions (
out/exclusions.csv, last origin): 345 rows → 338 with target ≤ t−2 → 335 (h=1) / 333 (h=3) with lags inside the file → 310 (h=1,n_analysed) / 306 (h=3,n_analysed_h3) after dropping training rows whose target or regressor months fall in the COVID window 2020-03..2021-12. I chose this exclusion before evaluating, and did not tune it on test errors. The evaluation targets (2022-06..2026-07) are not affected by it. - Data issues (
data_issues): the README says the YoY columns are empty for the first 12 months, but the file has values for 1998; I used them as given.
Method
- Naive: forecast =
gdp_yoy(t−2) for both horizons. - Model (
model_name=ardl_emp_direct): a direct OLS regression for each horizon: gdp_yoy(t+h) on a constant, gdp_yoy(t−2), emp_yoy(t−1) and emp_yoy(t−2). It is re-estimated at every origin on an expanding window running from 1998 to the latest observed target (t−2). I specified it once in advance; I did no model search and compared no alternatives on the test errors. There is no randomness (random_seed= null). - Scoring: errors = forecast − actual over the 48 origins, on the current vintage. The ledger is
out/forecasts.csv(192 rows).out/analysis.pyregenerates everything, and a clean re-run produced byte-identical output.
Results
| RMSE naive | MAE naive | RMSE model | MAE model | 1 − RMSE ratio | |
|---|---|---|---|---|---|
| h=1 | 0.814 (rmse_naive_h1) |
0.642 (mae_naive_h1) |
0.666 (rmse_model_h1) |
0.554 (mae_model_h1) |
0.181 (rel_improvement_rmse_h1) |
| h=3 | 1.140 (rmse_naive_h3) |
0.872 (mae_naive_h3) |
0.851 (rmse_model_h3) |
0.690 (mae_model_h3) |
0.253 (rel_improvement_rmse_h3) |
Units are percentage points of YoY growth. The model has lower RMSE and MAE than the naive forecast at both horizons: 18% lower RMSE at h=1 and 25% lower at h=3. The naive error grows with the horizon, because the information gap is h+2 months.
Uncertainty
I ran a Diebold–Mariano test on squared loss (HLN small-sample correction, Newey–West variance with h+1 lags, t(47) reference). It gives dm_stat_h1 = −1.56 (p = 0.126, dm_pvalue_two_sided_h1) and dm_stat_h3 = −1.24 (p = 0.221, dm_pvalue_two_sided_h3). The point estimates favour the model, but the difference is not statistically significant at 5% with only 48 overlapping, autocorrelated errors.
Limitations and open issues
- Current-vintage data: this simulates publication timing only, not first-release values. The results are not a real-time validation, and true real-time errors would likely be larger.
- No claim about attribution: I did not separate how much of the gain comes from employment and how much from the regression's mean reversion/intercept relative to the naive forecast. I estimated no employment-free benchmark.
- The evaluation window (2022–2026) is a single post-pandemic disinflation/slowdown period, so the results may not generalise.
- Excluding the COVID window is a judgement call. Other choices (e.g., dummies, or using no exclusion) could change the model errors.
- The 1998 YoY values come from levels that are not in the file. They are used unverified, but the recomputed values from 1999 on match exactly.