# Real-time forecasting of US retail-sales YoY growth from ALFRED vintages

## Question
Could the year-over-year growth of US retail and food services sales (RSAFS, g_m = 100·ln(L_m/L_{m−12})) have been forecast in real time, 1 and 3 months beyond the last published month, better than the no-change (naive) forecast, when forecast errors are measured against the first published value of each target month? We evaluate 10 month-end origins from 2025-10-31 (`first_origin`) to 2026-07-31 (`last_origin`), `n_origins` = 10.

## Data and exclusions
- Inputs (row counts in `n_input_rows`): 10 RSAFS and 10 PAYEMS ALFRED vintages, the current UMCSENT and RSAFS files, and `first_release.csv`. All SHA-256 hashes match `data/README.md` (see `out/state/data_audit.md`).
- **Real-time design.** The forecast made at origin V uses only `RSAFS_vintage_V` (`vintage_used` = `origin_date` in every row of `out/forecasts.csv`). The target is last(V)+h, where last(V) is the last month in that vintage. Observed publication delays: last(V) = 2025-08 at V = 2025-10-31 and 2026-01 at V = 2026-03-31 (two months before the vintage date); the 2026-04-30 vintage adds two months (2026-02 and 2026-03) at once. For every other origin last(V) is the month before V. As a result, the h=1 targets are 2025-09, 2025-10, 2025-11, 2025-12, 2026-01, 2026-02, 2026-04, 2026-05, 2026-06 and 2026-07, and the h=3 targets run from 2025-11 to 2026-09. 2026-03 is not an h=1 target, and 2026-05 is not an h=3 target.
- Actuals are `yoy_first_release` from `first_release.csv`. The h=3 actuals for 2026-08 and 2026-09 come after a large level revision of RSAFS (current levels are about 4–5% below the vintage levels). YoY is always computed within a single vintage, so the actuals stay internally consistent.
- PAYEMS and UMCSENT were audited but are not used by any forecast. `RSAFS_current.csv` is used only for the revision estimand.
- Exclusions (`out/exclusions.csv`): 80 forecast–actual pairs were planned and 80 were analysed, with 0 removed. `n_analysed` = 20 pairs for my model (10 per horizon).
- Data problems and how each was handled are listed in `data_issues`.

## Method
- **Baselines** (computed from the vintage dated V): naive (g_last for both horizons); seasonal naive (g_{T−12} from the vintage); AR(1) (OLS of g_m on a constant and g_{m−1} over 1993-02 ≤ m ≤ last(V), with iterated forecasts; φ̂ ≈ 0.84 at every origin, see `out/supplementary.json`).
- **My model** (`model_name`, `model_description`): a base-effect random walk with drift in the log level. The forecast is ln L̂_T = ln L_last + h·g_last/1200, and the YoY forecast is 100·(ln L̂_T − ln L_{T−12}), with L_{T−12} taken from the same vintage. In words, it is the naive forecast corrected for the known change in the year-earlier comparison level (base effects) plus a drift. The model has no estimated parameters. I fixed the specification before seeing any evaluation result and tried no alternatives on the evaluation sample, so there is no specification search to correct for.
- **Evaluation:** RMSE and MAE over the 10 origins for each horizon; 1 − RMSE_model/RMSE_naive; and a Diebold–Mariano test on d_t = e²_naive − e²_model. The DM test uses a Bartlett (Newey–West) long-run variance with truncation lag h−1, no HLN correction, and a two-sided normal p-value.
- **Supplementary uncertainty measure:** a moving-block bootstrap (block length max(h,2), B = 5000, seed `random_seed`) of RMSE_naive − RMSE_model.
- **Revision estimand:** the mean over the 10 h=1 targets of |g_current − yoy_first_release|.

## Results
Table 1 (`out/table1.csv`; RMSE / MAE in percentage points; n = 10 per cell):

| model | h=1 RMSE | h=1 MAE | h=3 RMSE | h=3 MAE |
|---|---|---|---|---|
| naive | 0.997 (`rmse_naive_h1`) | 0.849 (`mae_naive_h1`) | 1.420 (`rmse_naive_h3`) | 1.200 (`mae_naive_h3`) |
| seasonal naive | 1.675 (`rmse_snaive_h1`) | 1.307 (`mae_snaive_h1`) | 1.316 (`rmse_snaive_h3`) | 1.066 (`mae_snaive_h3`) |
| AR(1) | 0.941 (`rmse_ar1_h1`) | 0.798 (`mae_ar1_h1`) | 1.187 (`rmse_ar1_h3`) | 0.953 (`mae_ar1_h3`) |
| BE-RWD (model) | 0.550 (`rmse_model_h1`) | 0.486 (`mae_model_h1`) | 1.142 (`rmse_model_h3`) | 0.934 (`mae_model_h3`) |

- **h = 1:** my model lowers RMSE by 44.8% relative to naive (`rel_improvement_rmse_h1` = 0.448). The DM statistic is 2.43 (`dm_stat_h1`), with p = 0.015 (`dm_pvalue_h1`), giving `conclusion_code_h1` = model_better_significant.
- **h = 3:** the improvement is 19.6% (`rel_improvement_rmse_h3` = 0.196). The DM statistic is 1.55 (`dm_stat_h3`), with p = 0.122 (`dm_pvalue_h3`), giving `conclusion_code_h3` = model_better_not_significant.
- At h = 3 every alternative has a lower RMSE than naive, but my model's edge over AR(1) is small: 1.142 against 1.187.
- **Revisions:** first-release YoY of the h=1 targets differs from today's published YoY by 0.548 pp on average (`mean_abs_revision_h1_targets`). That is the same order as the model's h=1 RMSE. All 10 revisions are negative (`out/supplementary.json`), consistent with the large level revision after mid-2026.
- Figure 1 (`out/figure1.png`) plots the first-release actuals and the naive, AR(1) and model forecasts for each horizon.

## Uncertainty
- **Small sample.** With only 10 origins, the DM statistic relies on a normal approximation that is unreliable at this sample size. The long-run variance is estimated from 10 loss differentials, and at h=3 from overlapping errors with only two lags. With n = 10 the omitted HLN correction would shrink the statistic and Student-t critical values would be wider. The h=1 p-value of 0.015 (`dm_pvalue_h1`) should therefore be read as suggestive evidence, not a firm rejection.
- **h = 1 evidence.** Two other measures point the same way. The model has the smaller squared error at 8 of 10 origins (`share_origins_model_smaller_sq_error_h1` = 0.8). The block-bootstrap 95% interval for RMSE_naive − RMSE_model is [0.26, 0.70] pp (`boot_rmse_diff_ci95_lo_h1`, `boot_rmse_diff_ci95_hi_h1`) and excludes zero. Even so, a bootstrap on 10 points is itself fragile.
- **h = 3 evidence.** The bootstrap interval is [−0.14, 0.63] pp (`boot_rmse_diff_ci95_lo_h3`, `boot_rmse_diff_ci95_hi_h3`) and includes zero. The model wins at 6 of 10 origins (`share_origins_model_smaller_sq_error_h3`). The data cannot distinguish the model from naive at this horizon.
- **Why the h=1 gain is plausible.** It has a mechanical explanation that should carry over to other periods: the year-earlier comparison level is already known at the origin, and the naive forecast ignores it.

## Limitations and open issues
- **Short, unusual sample.** There are only 10 origins, the period covers irregular publication timing and a large benchmark revision, and the h=3 errors overlap. Results may not generalise.
- **Mixed-vintage actuals.** First-release actuals for targets 2026-08 and 2026-09 (h=3, last two origins) come from vintages after a big level revision. Forecasts made from pre-revision vintages are compared with actuals from post-revision vintages; because the comparison is in YoY terms the effect is muted but not removed.
- **Predictors not used.** PAYEMS and UMCSENT were not used. I made no attempt to show whether they add information, and nothing is claimed about them.
- **Unresolved conflict with the gate.** The provided gate checks `origin_date < target_date` in `out/forecasts.csv`. TASK.md requires `target_date` to be the first day of the target month (YYYY-MM-01). Because a month-end vintage contains at most the month before V, every h=1 target month begins before its month-end origin date (e.g. origin 2026-05-31, target 2026-05-01), so 40 of 80 rows fail that literal rule. The substantive availability rule — the target month is not yet published in the vintage used — holds for all 80 rows, and this is asserted in `out/analysis.py`. I kept the format TASK.md specifies instead of relabelling dates to pass the gate. Phase P4 is therefore recorded as "blocked" by this rule only (see `out/state/checks.md`). All required estimands were produced (`feasibility` = complete).