# Real-time forecasting of US retail-sales YoY growth from data vintages (T1-forecast)

## Question
Could the year-over-year growth of US retail and food services sales (RSAFS, g_m = 100·ln(L_m/L_{m−12}), percent) have been forecast in real time at 10 month-end origins (`first_origin` = 2025-10-31 to `last_origin` = 2026-07-31, `n_origins` = 10), h = 1 and h = 3 months beyond the last published month last(V)? And does a model of my own design beat the no-change (naive) forecast when errors are measured against the first-release YoY value of each target month?

## Data and exclusions
- Inputs (row counts in `n_input_rows`): 10 RSAFS ALFRED vintages and 10 PAYEMS vintages (one per origin), UMCSENT (current), RSAFS current vintage, and `first_release.csv` (28 target months, 2024-06 to 2026-09). All 23 SHA-256 hashes match `data/README.md` (`out/state/data_audit.md`).
- **Real-time design.** The forecast at origin V uses only the RSAFS vintage dated V (`vintage_used` = `origin_date` in all 80 rows of `out/forecasts.csv`). The target is last(V)+h. Publication delays: for the first six origins (2025-10-31 to 2026-03-31) last(V) is two months before the origin month (e.g. V = 2025-10-31 ends at 2025-08, so its h=1 target is 2025-09 and its h=3 target is 2025-11). From 2026-04-30 on it is one month before (the 2026-04-30 vintage added 2026-02 and 2026-03 at once). Targets therefore run from 2025-09 to 2026-07 (h=1) and from 2025-11 to 2026-09 (h=3). All 20 targets are in `first_release.csv`, and none of them is in the vintage used for its forecast.
- **Actuals.** `yoy_first_release` is used as given. Because of the irregular calendar, 2025-09 first appears in the 2025-11-30 vintage, and 2026-02 and 2026-03 both first appear in 2026-04-30. Their "first release" is defined at month-end granularity (see `data_issues`).
- **Revisions.** For 2025–2026, the current RSAFS file has levels about 4.7–4.9% below those in the 2026-07-31 vintage. YoY growth is always computed within one vintage, so levels are never mixed across vintages. `RSAFS_current.csv` is used only for the revision estimand (d).
- **Exclusions** (`out/exclusions.csv`): 20 candidate model forecast–actual pairs, 0 removed, `n_analysed` = 20 (10 per horizon). UMCSENT has 210 empty values, all before 1978. It is not used, and neither is PAYEMS.

## Method
- **Baselines** (from vintage V only): naive, where the forecast is g_{last(V)} for both horizons; seasonal naive, where the forecast is g_{T−12}; and AR(1), fitted by OLS of g_m on a constant and g_{m−1} for 1993-02 ≤ m ≤ last(V), then iterated h steps.
- **My model** (`model_name` = rw_drift_base_effect; full text in `model_description`) treats the log level as a random walk with drift. The drift is the mean monthly log change over the last 12 published months: d = (x_l − x_{l−12})/12. The forecast is g_{l+h} = 100·(x_l + h·d − x_{l+h−12}). Unlike the naive forecast, it accounts for the monthly changes that leave the 12-month window ("base effects"), all of which are already published at the origin. It uses only the RSAFS vintage dated V. I fixed the specification before computing any forecast error, and nothing was tuned on the evaluation targets.
- **Evaluation:** RMSE and MAE against `yoy_first_release`. The Diebold–Mariano statistic for the model against naive uses d_t = e²_naive − e²_model and a Bartlett (Newey–West) long-run variance with truncation lag h−1, with no HLN factor and a two-sided normal p-value, as specified in TASK.md.
- **Revision (d):** the mean over the 10 h=1 targets of |g^current_T − yoy_first_release_T|.
- `out/analysis.py` regenerates everything deterministically (`random_seed` = null, since no randomness is used).

## Results
Table 1 (`out/table1.csv`; percentage points; n = 10 per cell):

| model | RMSE h=1 | MAE h=1 | RMSE h=3 | MAE h=3 |
|---|---|---|---|---|
| naive | 0.997 (`rmse_naive_h1`) | 0.849 (`mae_naive_h1`) | 1.420 (`rmse_naive_h3`) | 1.200 (`mae_naive_h3`) |
| seasonal naive | 1.675 (`rmse_snaive_h1`) | 1.307 (`mae_snaive_h1`) | 1.316 (`rmse_snaive_h3`) | 1.066 (`mae_snaive_h3`) |
| AR(1) | 0.941 (`rmse_ar1_h1`) | 0.798 (`mae_ar1_h1`) | 1.187 (`rmse_ar1_h3`) | 0.953 (`mae_ar1_h3`) |
| model | 0.550 (`rmse_model_h1`) | 0.486 (`mae_model_h1`) | 1.142 (`rmse_model_h3`) | 0.934 (`mae_model_h3`) |

- At h=1 the model has the lowest RMSE and MAE of the four methods. Its RMSE is 44.8% below the naive RMSE (`rel_improvement_rmse_h1` = 0.448). DM = 2.43 (`dm_stat_h1`), p = 0.015 (`dm_pvalue_h1`), so `conclusion_code_h1` = model_better_significant.
- At h=3 the model again has the lowest RMSE and MAE. Its RMSE is 19.6% below naive (`rel_improvement_rmse_h3` = 0.196). DM = 1.55 (`dm_stat_h3`), p = 0.12 (`dm_pvalue_h3`), so `conclusion_code_h3` = model_better_not_significant.
- Among the baselines, AR(1) is slightly better than naive at both horizons. Seasonal naive is the worst at h=1 but beats naive at h=3.
- Revision: the mean absolute gap between first-release and current-vintage YoY growth for the h=1 targets is 0.548 pp (`mean_abs_revision_h1_targets`). That is about the same size as the model's h=1 RMSE.
- `out/figure1.png` shows the first-release actuals with the naive, AR(1) and model forecasts for each horizon.

## Uncertainty
- With only n = 10 origins, the DM statistic is compared with a normal distribution far outside its asymptotic regime. Its p-values should be read as rough indications.
- **h=1:** the model's squared error is below the naive one at 8 of 10 origins (aux file `out/state/aux_diagnostics.json`, `n_origins_model_sq_error_below_naive`). However, a single origin (2026-05-31) contributes a loss differential of 3.11, about 45% of the total (`loss_diff_by_origin`).
- **HLN sensitivity:** applying the Harvey–Leybourne–Newbold factor and t(9) critical values gives a p-value of 0.047 at h=1 and 0.28 at h=3 (`sensitivity_dm_hln_pvalue_t_nminus1`). These are not the required estimand and are reported only as a sensitivity check. The h=1 significance is therefore borderline once small-sample corrections are applied.
- **h=3:** the forecasts overlap, so errors are serially correlated. With lag-2 Bartlett weights, a long-run variance estimated from 10 points is very imprecise. The h=3 improvement is not distinguishable from zero.
- No bootstrap was run. A resampling interval built from 10 dependent observations would add little reliability.

## Limitations and open issues
- **Small sample.** Ten origins in a single 2025–2026 episode, which includes a large jump in YoY growth in spring 2026, cannot establish that the model would generally beat the naive forecast. The h=1 result depends heavily on a few origins.
- **What the model exploits.** Its advantage is mechanical. It uses the base effects already visible in the vintage, so it is a sensible real-time benchmark rather than a behavioural model. PAYEMS and UMCSENT were deliberately not used, so their value as predictors was not evaluated.
- **First-release actuals.** These include values that are not advance estimates (2026-02) and are themselves revised later, by a mean of 0.548 pp (`mean_abs_revision_h1_targets`). Rankings evaluated against the current vintage could differ; this was not evaluated.
- **Gate conflict (open issue).** The executable gate requires origin_date < target_date in `out/forecasts.csv`. Under TASK.md's own definitions (target = last(V)+h, target_date = YYYY-MM-01, month-end origins, and 1–2 month publication lags), every h=1 target month starts on or before its origin date. For example, origin 2025-10-31 has its h=1 target at 2025-09-01. The 40 h=1 rows therefore fail this gate check by construction. Satisfying it would mean misdating the targets, so I did not change them. Instead I verified the substantive information rule: each forecast uses only vintage V, and no target value is present in that vintage (`out/state/checks.md`). As a result, phases P4/P5 are recorded as "blocked" by this check only. All other gate checks were run and recorded.