1# F3-conf-01 — Forecasting Canadian monthly GDP y/y growth under publication lags23## Question4At the end of each month t (48 origins, `first_origin` 2022-05-01 to `last_origin` 2026-04-01), how accurately can the year-over-year growth of Canadian monthly real GDP for months t+1 and t+3 be forecast using only data published by then? And does a model that also uses Labour Force Survey employment growth beat the naive forecast? At origin t, GDP is known only up to t−2 and employment only up to t−1. At the last origin this gives `last_target_date_available` = 2026-02-01 and `last_predictor_date_available` = 2026-03-01.56## Data and exclusions7- Input: `data/canada_gdp_employment.csv`, 345 rows (`n_input_rows`). The SHA-256 matches the README. Full audit in `out/state/data_audit.md`.8- Every year-over-year value that can be recomputed from the levels in the file matches exactly. GDP is missing for 2026-08 and 2026-09 (not yet published). 1998 `gdp_yoy` values are present even though the README says they should be empty. They are kept as given (`data_issues`).9- The file contains data published after each origin. The code removes it at each origin: training pairs need a target month ≤ t−2, a GDP regressor month ≤ t−2 and an employment month ≤ t−1. An independent check that blanks out these later data and re-estimates gives the same forecasts (`out/state/checks.md`, check 5).10- Exclusions for the last estimation sample (origin 2026-04-01, h=1), from `out/exclusions.csv`:11 - 345 candidate rows.12 - Minus 8 rows not available at the origin.13 - Minus 2 rows missing a lag or target.14 - Minus 25 rows that touch the COVID window 2020-03..2021-12.15 - Leaves 310 rows (`n_analysed`). The h=3 sample has 306 rows (`n_analysed_last_h3`).16- I fixed the COVID exclusion before computing any out-of-sample error. It affects only model training, not the naive forecast.1718## Method19- **Naive forecast**: `gdp_yoy[t−2]` for both horizons.20- **Model `ardl_direct_emp`** (`model_name`, `model_description`): a separate OLS regression for each horizon h, `gdp_yoy[s+h] = a + b·gdp_yoy[s−2] + c·emp_yoy[s−1]`.21 - It is re-estimated at every origin on an expanding window starting in 1998, using only pairs that were observable at that origin.22 - The forecast plugs in `gdp_yoy[t−2]` and `emp_yoy[t−1]`.23 - No model selection was done and nothing is random (`random_seed` = null).24- **Errors**: actual minus forecast on the current-vintage data, summarised as RMSE and MAE over the 48 origins. The full forecast ledger is `out/forecasts.csv` (192 rows).25- **Diagnostic (not the submitted model)**: the same regression without employment, estimated on the same sample (`diag_rmse_ar_only_h*`, `diag_mae_ar_only_h*`). It tells us whether any gain comes from employment.2627## Results28| horizon | RMSE naive | MAE naive | RMSE model | MAE model | 1 − RMSE ratio | RMSE AR-only (diagnostic) |29|---|---|---|---|---|---|---|30| h=1 | 0.814 (`rmse_naive_h1`) | 0.642 (`mae_naive_h1`) | 0.743 (`rmse_model_h1`) | 0.604 (`mae_model_h1`) | 0.087 (`rel_improvement_rmse_h1`) | 0.715 (`diag_rmse_ar_only_h1`) |31| h=3 | 1.140 (`rmse_naive_h3`) | 0.872 (`mae_naive_h3`) | 0.902 (`rmse_model_h3`) | 0.753 (`mae_model_h3`) | 0.208 (`rel_improvement_rmse_h3`) | 0.864 (`diag_rmse_ar_only_h3`) |3233- In this sample the employment model has a lower RMSE and MAE than the naive forecast at both horizons: about 9% lower RMSE at h=1 and 21% at h=3.34- **The gain does not come from employment.** Without employment, the regression is more accurate still: RMSE 0.715 vs 0.743 at h=1 and 0.864 vs 0.902 at h=3.35- In the final estimation the employment coefficient is small: 0.058 at h=1 and 0.032 at h=3 (`last_coefficients_h1`, `last_coefficients_h3`). The GDP coefficient is below one (0.82 and 0.66), with a positive constant.36- So the improvement over the naive forecast comes from the GDP growth series drifting back towards its average, which the naive forecast ignores. Adding employment growth does not help out of sample.37- The answer to "does the model with employment beat the naive forecast?" is yes on point estimates, but not significantly (see Uncertainty), and not because of employment.3839## Uncertainty40- I used Diebold–Mariano tests of equal squared-error loss, model vs naive, with HAC variance and the Harvey–Leybourne–Newbold small-sample correction:41 - h=1: statistic 1.39, two-sided p = 0.17 (`dm_stat_h1`, `dm_pvalue_two_sided_h1`).42 - h=3: statistic 1.44, p = 0.16 (`dm_stat_h3`, `dm_pvalue_two_sided_h3`).43- HAC lags were 2 at h=1 and 4 at h=3 (`dm_hac_lags_h1`, `dm_hac_lags_h3`). I chose these myself because year-over-year targets overlap. This choice is a judgement call.44- With only 48 overlapping origins, neither improvement is statistically significant at conventional levels.45- I did not run a significance test for the model vs the AR-only diagnostic.4647## Limitations and open issues48- **Not real-time.** The data are the current vintage. The exercise reproduces publication timing only, not the first-release values forecasters actually saw. Revisions to GDP and LFS data are ignored, so the errors are not real-time errors.49- **One evaluation window.** The 48 origins (2022–2026) cover the post-COVID slowdown. With a different window the ranking could change.50- **COVID exclusion.** Training rows touching 2020-03..2021-12 were excluded; this was decided in advance, not tuned. I did not report a variant that keeps those rows.51- **Diagnostic after the fact.** The AR-only comparison was added after seeing the employment coefficients. It is reported as a diagnostic and does not replace the submitted model.52- **Unverifiable 1998 values.** The 1998 `gdp_yoy` values cannot be checked against levels in the file.53- **Workflow incident.** The analysis script was first written by mistake to `data/out/`. It was moved to `out/` and the stray folder removed. The data file's SHA-256 was re-checked and is unchanged.54