# F3-conf-03 — Rolling-origin ridge forecasts of US industrial production growth

## Question
Does a fixed-penalty ridge regression (λ = 50, key `lambda`) on 8 standardised monthly predictors (key `n_features`) forecast next month's US industrial production growth (100×Δln INDPRO) better than the naive last-value forecast and the expanding historical mean? Evaluation uses `n_origins` = 60 rolling origins from `first_origin` (2021-09-01) to `last_origin` (2026-08-01), horizon h = 1.

## Data and exclusions
- `data/ip_predictors.csv` (`n_input_rows`: 417 rows, 1992-01 to 2026-09; SHA-256 matches `data/README.md`). Full audit in `out/state/data_audit.md`.
- Missing values: only `neworder_chg` for 1992-01/02; those months are never ridge feature rows. The README says every change series is empty in 1992-01, but `ip_growth`, `oil_chg` and `permit_chg` do have 1992-01 values. I used them as given. Extreme 2020 (COVID) and 2008 values were kept. All of this is listed under `data_issues`.
- Exclusion ledger (`out/exclusions.csv`): 417 raw rows, minus 2 (before the 1992-03 training start), minus 2 (rows whose target falls after the last origin) = 413 training rows at the last origin (`n_analysed`).

## Method
- Naive forecast: `ip_growth` of month t. Expanding mean: mean of `ip_growth` from 1992-04 to t.
- Ridge: at each origin t, the training rows are s = 1992-03 … t−1, with target `ip_growth`_{s+1}. The features are ip_growth_s, its lags 1 and 2, oil_chg_s, fedfunds_s, permit_chg_s, neworder_chg_s and tcu_s. Each feature is standardised with the mean and sample SD (ddof = 1) of that origin's training rows, recomputed at every origin (`normalisation_window` = "expanding"). The target is centred on its training mean ȳ. Then b = (Z′Z + 50 I)⁻¹ Z′(y − ȳ), and the forecast is ȳ + z̃_t·b. No values dated after t are used. A test in which all data after a given origin were corrupted left the forecasts at that origin unchanged (`out/state/checks.md`).
- Metrics: RMSE and MAE of actual − forecast over the 60 origins. Relative improvement = 1 − RMSE_ridge/RMSE_baseline.
- Uncertainty: two-sided Diebold–Mariano test on squared-error loss differentials (h = 1, Harvey–Leybourne–Newbold small-sample correction, t₅₉ reference). There is no randomness (`random_seed` = null).
- Replication: `~/wp29/venv/bin/python out/analysis.py`. It writes `out/results.json`, `out/forecasts.csv` (180 rows) and `out/exclusions.csv`.

## Results
| model | RMSE (%/month) | MAE (%/month) |
|---|---|---|
| naive | 0.883 (`rmse_naive`) | 0.699 (`mae_naive`) |
| expanding mean | 0.544 (`rmse_mean`) | 0.429 (`mae_mean`) |
| ridge λ=50 | 0.561 (`rmse_ridge`) | 0.451 (`mae_ridge`) |

- Ridge vs naive: relative RMSE improvement +0.365 (`rel_improvement_vs_naive`). Ridge's RMSE is about 36% lower.
- Ridge vs expanding mean: relative RMSE improvement −0.031 (`rel_improvement_vs_mean`). Ridge's RMSE is about 3% higher than the mean's, and its MAE is also higher (`mae_ridge` > `mae_mean`).
- Answer: ridge beats the naive forecast but not the expanding historical mean over 2021-09 to 2026-08.

## Uncertainty
- Ridge vs naive: DM statistic −3.36 (`dm_stat_ridge_vs_naive`), p = 0.0014 (`dm_pvalue_ridge_vs_naive`). The ridge advantage over naive is statistically significant at conventional levels.
- Ridge vs mean: DM statistic +0.54 (`dm_stat_ridge_vs_mean`), p = 0.59 (`dm_pvalue_ridge_vs_mean`). The two forecasts are statistically indistinguishable. The small ridge disadvantage is well within sampling noise.
- With 60 evaluation points, RMSE differences of a few percent are not reliably detectable.

## Limitations and open issues
- The data are the current (revised) vintage and publication lags are ignored. These results describe revised-data predictability under a stylised timing assumption, not real-time forecasting performance.
- λ = 50 is fixed by the task and was not tuned. Other penalties or feature sets could give different results; none were explored, by design.
- The training windows include the extreme 2020 observations. Under ridge's squared loss these may distort the coefficients. Following the task, no outlier treatment was applied.
- The DM test assumes a covariance-stationary loss differential. The evaluation window is a single, partly post-COVID regime.
- The README/data discrepancy about 1992-01 values is noted but unresolved. It affects only the earliest training row's lag-2 feature.
- `feasibility` = "complete".