# Real-time forecasts of US retail-sales YoY growth: does a base-effect model beat no-change?

## Question
Using only the data published at each of 10 month-end forecast origins (2025-10-31 … 2026-07-31; `first_origin`, `last_origin`, `n_origins`), how well can the year-over-year log growth g = 100·ln(L_m/L_{m−12}) of US Retail Trade and Food Services sales (RSAFS) be forecast 1 and 3 months beyond the last published month, judged against the first-release value of each target month? Does a model of my own design beat the naive (no-change) forecast?

## Data and exclusions
- **Inputs** (FRED/ALFRED, retrieved 2026-10-30; all SHA-256 hashes match `data/README.md`, see `out/state/data_audit.md`; row counts in `n_input_rows`): 10 RSAFS vintages and 10 PAYEMS vintages dated at the origins, UMCSENT (current), RSAFS current vintage, and `first_release.csv` (actuals).
- **Real-time design.** The forecast at origin V uses only `RSAFS_vintage_V.csv` (column `vintage_used` in `forecasts.csv` equals `origin_date` on every row). Targets are last(V)+h, where last(V) is the last month in that vintage. **Publication delays observed:** for the six origins 2025-10-31 … 2026-03-31 the last published month is two calendar months before the origin (last = 2025-08, 2025-09, 2025-10, 2025-11, 2025-12, 2026-01) instead of the usual one; the 2026-04-30 vintage adds 2026-02 and 2026-03 at once, and the four origins 2026-04-30 … 2026-07-31 end one month before (last = 2026-03 … 2026-06). Consequently the h=1 targets are 2025-09 … 2026-02 and 2026-04 … 2026-07 (2026-03 is never an h=1 target), and the h=3 targets are 2025-11 … 2026-04 and 2026-06 … 2026-09 (13 distinct target months overall). Origin information is in `results.json → supplementary.origins`.
- **Actuals**: `yoy_first_release` from `first_release.csv`; each target's first-release vintage is strictly after its origin (check 4 in `out/state/checks.md`).
- **Not used in forecasts**: `RSAFS_current.csv` (used only for the revision estimand), PAYEMS and UMCSENT (read for auditing only; the chosen model does not use them).
- **Exclusions** (`out/exclusions.csv`): no forecast–actual pair was dropped; 20 model pairs evaluated (`n_analysed`). Only the 15 rows of `first_release.csv` that are not targets of any origin are unused.
- **Data issues** (full list in `data_issues`): irregular publication timing (above); an annual/benchmark revision of RSAFS between the 2026-08-31 and 2026-09-30 vintages lowered levels by ≈4% and current YoY growth sits below first release for every h=1 target; it changes the base of the first-release actuals for h=3 targets 2026-08 and 2026-09. No adjustment was made, since the task defines the actual as the first release.

## Method
- **Baselines** (from vintage V only): naive g_{last(V)}; seasonal naive g_{T−12}; AR(1) estimated by OLS of g_m on a constant and g_{m−1} over 1993-02 … last(V), re-estimated at each origin, with iterated forecasts.
- **Own model, “BED (base-effect drift)”** (`model_name`, `model_description`). YoY growth of month T is the sum of the 12 monthly log growths T−11 … T, and 12−h of them are already published at the origin. BED keeps those known terms and replaces only the h unknown months with the trailing 12-month average monthly growth: ĝ_T = 100·ln(L_last/L_{T−12}) + h·g_last/12, with both levels taken from vintage V. Nothing is estimated and there is nothing to tune. In comparison, the naive forecast implicitly fills the unknown months with the h oldest months of the last 12-month window, which brings in last year's month-specific noise (the "base effect"). I fixed this specification in `out/state/log.md` before computing any out-of-sample error and evaluated no other specification. I did not try PAYEMS or UMCSENT variants, because searching over models on 10 origins would mean selecting on the evaluation sample.
- **Evaluation**: RMSE and MAE against the first release; relative improvement 1 − RMSE_model/RMSE_naive; Diebold–Mariano test on d_t = e²_naive − e²_model with a Bartlett long-run variance at truncation lag h−1 (no HLN factor) and a two-sided N(0,1) p-value. As a supplement, a moving-block bootstrap of RMSE_model − RMSE_naive (block length 2 at h=1, 3 at h=3, 9 999 draws, seed `random_seed` = 20261030).
- **Revision estimand**: mean over the 10 h=1 targets of |g^current_T − yoy_first_release_T|.
- Everything is produced by `out/analysis.py`.

## Results
Table 1 (`out/table1.csv`), percentage points of YoY growth, n = 10 per cell:

| model | RMSE h=1 | MAE h=1 | RMSE h=3 | MAE h=3 |
|---|---|---|---|---|
| naive | 0.997 (`rmse_naive_h1`) | 0.849 (`mae_naive_h1`) | 1.420 (`rmse_naive_h3`) | 1.200 (`mae_naive_h3`) |
| seasonal naive | 1.675 (`rmse_snaive_h1`) | 1.307 (`mae_snaive_h1`) | 1.316 (`rmse_snaive_h3`) | 1.066 (`mae_snaive_h3`) |
| AR(1) | 0.941 (`rmse_ar1_h1`) | 0.798 (`mae_ar1_h1`) | 1.187 (`rmse_ar1_h3`) | 0.953 (`mae_ar1_h3`) |
| BED model | 0.550 (`rmse_model_h1`) | 0.486 (`mae_model_h1`) | 1.142 (`rmse_model_h3`) | 0.934 (`mae_model_h3`) |

- At h=1, BED reduces RMSE relative to naive by 44.8% (`rel_improvement_rmse_h1` = 0.448). DM = 2.43, p = 0.015 (`dm_stat_h1`, `dm_pvalue_h1`), so `conclusion_code_h1` = `model_better_significant`. BED has a smaller squared error than naive at 8 of 10 origins (`supplementary.bootstrap_rmse_diff.h1.n_origins_model_smaller_sq_error`).
- At h=3, BED reduces RMSE relative to naive by 19.6% (`rel_improvement_rmse_h3` = 0.196). DM = 1.55, p = 0.122 (`dm_stat_h3`, `dm_pvalue_h3`), so `conclusion_code_h3` = `model_better_not_significant`. BED wins at 6 of 10 origins. At this horizon its RMSE is only slightly below AR(1)'s (1.142 vs 1.187), and I did not test that difference.
- At h=1, AR(1) is only marginally better than naive, and seasonal naive is the worst baseline. At h=3, all three alternatives (seasonal naive, AR(1), BED) have lower RMSE than naive.
- Figure 1 (`out/figure1.png`) shows the misses common to all methods: the jump to ≈6.6% YoY in 2026-05 and the drop to 4.9% in 2026-07.
- **Revisions**: the mean absolute revision between first-release and current YoY growth of the h=1 targets is 0.548 pp (`mean_abs_revision_h1_targets`). Every revision is negative (−0.17 to −0.98 pp; `supplementary.revisions_h1_targets`), which reflects the benchmark revision. The revision is about as large as BED's h=1 RMSE, so the ranking could differ if the forecasts were scored against current data.

## Uncertainty
- There are only 10 origins per horizon. The DM statistic relies on asymptotic normality, which is a rough approximation at n = 10, and without the Harvey–Leybourne–Newbold correction the test tends to over-reject in small samples. The h=1 p-value of 0.015 (`dm_pvalue_h1`) should therefore be read as suggestive evidence rather than a precise significance level. A Student-t reference distribution or the HLN factor would give a larger p-value; I did not compute this.
- At h=1 the evidence holds up under a second check: the moving-block bootstrap 95% interval for RMSE_model − RMSE_naive is [−0.70, −0.26] pp and lies entirely below zero (`supplementary.bootstrap_rmse_diff.h1`). This bootstrap is itself crude with n = 10.
- At h=3 the long-run variance uses only two lags on overlapping 3-step errors. The DM test cannot reject equal accuracy (`dm_pvalue_h3` = 0.12), and the bootstrap interval [−0.64, +0.14] pp contains zero (`supplementary.bootstrap_rmse_diff.h3`).
- The 10 origins span a single 9-month period with unusual publication delays and a sharp acceleration in mid-2026, so the results may not carry over to other periods.

## Limitations and open issues
- The sample is small and covers a single episode. The h=1 result is significant at 5% under the prescribed DM test but has not been confirmed on other periods.
- The actual is the first release, which here sometimes comes from a vintage that already includes a large level rebasing (2026-08 and 2026-09 targets). With a different definition of "truth" (current vintage) the errors would change by about 0.5 pp on average (`mean_abs_revision_h1_targets`).
- PAYEMS and UMCSENT were deliberately left out (see Method), so the task's question about the value of these predictors is not addressed (claim C5: not evaluated).
- `n_input_rows` lists the 23 CSV files read by the script. `data/README.md` was read for provenance only and has no data rows.
- Feasibility: complete (`feasibility`). No required value is null.