# Real-time forecasting of US retail-sales YoY growth from ALFRED vintages (T1-forecast)

## Question
Could the year-over-year log growth of US retail and food services sales (RSAFS), g_m = 100·ln(L_m/L_{m−12}), have been forecast in real time, using only the data published at each of 10 month-end origins (2025-10-31 … 2026-07-31; `first_origin`, `last_origin`, `n_origins`), one and three months beyond the last published month? And does a model of my own design beat the no-change (naive) forecast, when errors are measured against the first-release value of each target month?

## Data and exclusions
Inputs (hashes verified against `data/README.md`; row counts in `n_input_rows`; full audit in `out/state/data_audit.md`): 10 ALFRED vintages of RSAFS and of PAYEMS, current UMCSENT, current RSAFS, and `first_release.csv`.

**Real-time design.** The forecast at origin V uses only the RSAFS vintage dated V (`vintage_used` = `origin_date` in every row of `out/forecasts.csv`). The target is last(V)+h, where last(V) is the last month present in that vintage (`last_month_by_origin`):

| origin V | last(V) | h=1 target | h=3 target |
|---|---|---|---|
| 2025-10-31 | 2025-08 | 2025-09 | 2025-11 |
| 2025-11-30 | 2025-09 | 2025-10 | 2025-12 |
| 2025-12-31 | 2025-10 | 2025-11 | 2026-01 |
| 2026-01-31 | 2025-11 | 2025-12 | 2026-02 |
| 2026-02-28 | 2025-12 | 2026-01 | 2026-03 |
| 2026-03-31 | 2026-01 | 2026-02 | 2026-04 |
| 2026-04-30 | 2026-03 | 2026-04 | 2026-06 |
| 2026-05-31 | 2026-04 | 2026-05 | 2026-07 |
| 2026-06-30 | 2026-05 | 2026-06 | 2026-08 |
| 2026-07-31 | 2026-06 | 2026-07 | 2026-09 |

**Publication delays observed.** The first six vintages (2025-10-31 to 2026-03-31) end two months before the vintage date rather than one; the 2026-04-30 vintage adds two months at once (2026-02 and 2026-03). So the target months are unevenly spaced (h=1 never targets 2026-03, h=3 never targets 2026-05). The actual for every target is `yoy_first_release` from `first_release.csv`. Its first-release vintage is always later than the origin. Other data issues are listed in `data_issues`: revisions within the vintage grid, a large (~3.5–4 %) downward level revision between the 2026-08-31 and 2026-09-30 vintages that also appears in the current file, PAYEMS delays, and pre-1978 gaps in UMCSENT. `RSAFS_current.csv` is used only for estimand (d).

**Exclusions.** None: 80 forecast–actual pairs out of 80, of which 20 belong to my model (`n_analysed`). See `out/exclusions.csv`.

## Method
All four forecasts are computed from the RSAFS vintage dated V only:
- **naive**: g_{last(V)} for both horizons;
- **seasonal naive (snaive)**: g_{T−12} from the vintage;
- **AR(1)**: OLS of g_m on a constant and g_{m−1}, 1993-02 ≤ m ≤ last(V), with iterated forecasts (coefficients in `ar1_coef_by_origin`);
- **my model, "base-effect drift" (BED)** (`model_name`, `model_description`): the log level is extrapolated as ln L̂_T = ln L_last + h·μ, with μ = (ln L_last − ln L_{last−12})/12 (the latest YoY growth spread evenly across months). The forecast is ĝ_T = 100·(ln L̂_T − ln L_{T−12}), where the base-period level L_{T−12} is already published in vintage V. In words, the naive forecast is corrected for the known *base effect*, i.e. the monthly changes from a year ago that drop out of the YoY window, and h months of drift are added. The model estimates no parameters. Its specification was fixed before any out-of-sample error was computed, and no alternative variant was tried on the evaluation sample. PAYEMS and UMCSENT are not used.

Evaluation: RMSE and MAE over the 10 origins per horizon against first-release YoY. Relative improvement = 1 − RMSE_model/RMSE_naive. Diebold–Mariano test of model vs naive: d_t = e²_naive − e²_model, Bartlett (Newey–West) long-run variance with truncation lag h−1, no HLN correction, two-sided normal p-value. Supplementary measure: a circular moving-block bootstrap (block length 3, B = 9999, seed `random_seed` = 20261030) of the paired errors, giving a 95 % percentile interval for RMSE_model − RMSE_naive. Estimand (d) is the mean over the 10 h=1 targets of |g_current − yoy_first_release|. Everything is produced by `out/analysis.py`.

## Results
RMSE / MAE in percentage points of YoY growth (`out/table1.csv`, n = 10 each):

| model | RMSE h=1 | MAE h=1 | RMSE h=3 | MAE h=3 |
|---|---|---|---|---|
| naive | 0.997 (`rmse_naive_h1`) | 0.849 (`mae_naive_h1`) | 1.420 (`rmse_naive_h3`) | 1.200 (`mae_naive_h3`) |
| seasonal naive | 1.675 (`rmse_snaive_h1`) | 1.307 (`mae_snaive_h1`) | 1.316 (`rmse_snaive_h3`) | 1.066 (`mae_snaive_h3`) |
| AR(1) | 0.941 (`rmse_ar1_h1`) | 0.798 (`mae_ar1_h1`) | 1.187 (`rmse_ar1_h3`) | 0.953 (`mae_ar1_h3`) |
| BED model | 0.550 (`rmse_model_h1`) | 0.486 (`mae_model_h1`) | 1.142 (`rmse_model_h3`) | 0.934 (`mae_model_h3`) |

- **h = 1:** the BED model has the lowest RMSE and MAE of the four methods. Its RMSE is 44.8 % below naive (`rel_improvement_rmse_h1` = 0.448), and it has the smaller squared error at 8 of 10 origins (`n_model_wins_h1`). DM = 2.43, p = 0.015 (`dm_stat_h1`, `dm_pvalue_h1`), so `conclusion_code_h1` = `model_better_significant`.
- **h = 3:** the BED model again has the lowest RMSE and MAE, but its margin over AR(1) is small (1.142 vs 1.187). Its RMSE is 19.6 % below naive (`rel_improvement_rmse_h3` = 0.196), with 6 wins out of 10 (`n_model_wins_h3`). DM = 1.55, p = 0.122 (`dm_stat_h3`, `dm_pvalue_h3`), so `conclusion_code_h3` = `model_better_not_significant`.
- Seasonal naive is the worst method at h = 1 but beats naive at h = 3. AR(1) beats naive at both horizons.
- **Revisions (d):** the first-release YoY growth of the 10 h=1 target months differs from the currently published growth by 0.548 pp on average (`mean_abs_revision_h1_targets`). The revisions are all downward (`mean_signed_revision_h1_targets` = −0.548; maximum 0.976 pp, `max_abs_revision_h1_targets`). That average revision is the same size as the BED model's h=1 RMSE.
- Figure: `out/figure1.png` shows the first-release actuals with the naive and BED forecasts, one panel per horizon.

## Uncertainty
There are only 10 origins, the evaluation window is a single year, and the h = 3 errors overlap. The DM statistic is compared with the standard normal, which is an asymptotic approximation. With n = 10 the Newey–West variance estimate is itself very noisy, and the test is known to over-reject in small samples (this is why the Harvey–Leybourne–Newbold correction exists; as the task specifies, it is not applied here). So the h = 1 p-value of 0.015 (`dm_pvalue_h1`) should be read as suggestive evidence, not decisive. With the HLN factor and t(9) critical values it would be less extreme; that variant was not computed for the reported keys. Two things support the h = 1 result: the model wins at 8 of 10 origins, and every block-bootstrap resample gives the model the lower RMSE. The 95 % interval for RMSE_model − RMSE_naive is [−0.649, −0.279] pp (`boot_rmse_diff_lo_h1`, `boot_rmse_diff_hi_h1`, `boot_share_model_better_h1`). This bootstrap is crude at n = 10 and should not be over-interpreted. At h = 3 the DM test does not reject (p = 0.122, `dm_pvalue_h3`), and the bootstrap interval [−0.688, 0.082] pp includes zero (`boot_rmse_diff_lo_h3`, `boot_rmse_diff_hi_h3`). An improvement over naive at h = 3 is plausible but not established.

## Limitations and open issues
- **Gate chronology check conflicts with the task definition.** The gate requires `origin_date < target_date` in `out/forecasts.csv`, which fails for all 40 h = 1 rows. TASK.md defines the target as last(V)+h, written as YYYY-MM-01. Since last(V) is always at least one month before the month of V, the h = 1 target month always begins on or before the month-end origin (e.g. origin 2026-04-30, target 2026-04-01). The forecast is still made before the target's value exists in any data: the target value is absent from vintage V, and its first-release vintage is later than the origin in all 80 rows (`out/state/checks.md`). I kept the target dates as TASK.md defines them rather than relabelling them to pass the gate. So the gate reports this one check as failed, and phases P4/P5 are marked "blocked" for this reason only. All other gate checks, including reproducibility, pass.
- Small sample (10 origins, one year, a period with shutdown-related publication disruption). The ranking of methods could change in other periods.
- The model was designed by one analyst, and its design reflects general knowledge of how YoY base effects work. I did not search over variants on these data. Still, I cannot fully exclude awareness of the general behaviour of 2025–26 data as a source of bias.
- The h = 3 actuals for the last two origins (targets 2026-08 and 2026-09) come from vintages released after a large level revision. Within-vintage YoY growth is less affected than levels, but not entirely unaffected.
- The large downward revision behind estimand (d) is mostly a single annual/benchmark revision. It is a statement about this particular window, not about typical revision sizes.
- PAYEMS and UMCSENT were not used. Whether they would add information was not evaluated.
- **Controller resume (gate re-run).** The controller re-ran the gate and reported the same single failure (`forecasts_chronology`); I re-ran it again and got the same result (receipt `20261030T060507.995429Z`). For 6 of the 10 h = 1 rows (origins 2025-10-31 to 2026-03-31) the whole target month lies before the origin date. For the other 4 (2026-04-30 to 2026-07-31) the target month is the origin's own month. So no date inside the target month satisfies `origin_date < target_date`. To pass the check I would have to write a date outside the target month, for example the first-release date, into a column that TASK.md defines as the target month. That would falsify the deliverable, so I did not do it. Resolving this requires changing either the gate rule or the task's target definition.