# Real-time forecasting of US retail-sales YoY growth from ALFRED vintages (T1-forecast)

## Question
Using only the data published at each of 10 month-end forecast origins (2025-10-31 … 2026-07-31; `first_origin`, `last_origin`, `n_origins`), how well can the year-over-year log growth of US retail and food services sales (RSAFS), g_m = 100·ln(L_m/L_{m−12}), be forecast 1 and 3 months beyond the last published month, and does a model of my own beat the naive no-change forecast when errors are measured against the first-release YoY of each target month?

## Data and exclusions
- Inputs: 10 RSAFS and 10 PAYEMS ALFRED vintages (one per origin), UMCSENT (current vintage), RSAFS_current, first_release.csv. Row counts are in `n_input_rows`; every SHA-256 matched `data/README.md` (out/state/data_audit.md).
- **Real-time design.** The forecast at origin V uses only the RSAFS vintage dated V (`vintage_used` = `origin_date` for all 80 rows of out/forecasts.csv). Targets are last(V)+h, where last(V) is the last month in that vintage. Publication delays seen in the vintages: at the six origins 2025-10-31 … 2026-03-31 the last published month is two months before the origin month (last(V) = 2025-08 … 2026-01). The 2026-04-30 vintage adds two months at once (2026-02, 2026-03), and the last four origins have the usual one-month delay (last(V) = 2026-03 … 2026-06). Per-origin last(V) and delay are stored in `real_time_design`. Because of this, month 2026-03 is never an h=1 target, and for the first six origins the h=1 "forecast" is really a backcast of a month that had already ended but was not yet published.
- Actuals: `yoy_first_release` from first_release.csv. Where the first-release vintage is one of the shipped vintages, its values match that vintage exactly (check C4).
- RSAFS_current is used only for the revision estimand. PAYEMS and UMCSENT are read (for row counts) but are not used by any forecast.
- Exclusions (out/exclusions.csv): none. All 80 forecast–actual pairs (10 origins × 2 horizons × 4 models) have an actual. `n_analysed` = 20 model pairs (10 per horizon).
- Data issues found, with how each was handled, are listed in `data_issues`: irregular publication timing; PAYEMS gaps and a benchmark revision; pre-1978 UMCSENT gaps; a benchmark revision in RSAFS_current that also affects the first-release levels of 2026-08 and 2026-09; and routine monthly revisions.

## Method
- Baselines, computed from vintage V as TASK.md specifies: **naive** = g_{last(V)}; **seasonal naive** = g_{T−12}; **AR(1)** = OLS of g_m on a constant and g_{m−1} for 1993-02 ≤ m ≤ last(V), iterated h steps.
- **Own model, `model_name` = rw_drift_yoy** (see `model_description`). This is a base-effect-aware random walk with drift on log levels. The drift μ is the mean monthly log change over the 36 months ending at last(V) in vintage V. The forecast is ĝ_T = 100·(ln L_last + h·μ − ln L_{T−12}), where the base-month level L_{T−12} is already published in vintage V. The point of the model is that YoY growth is a 12-month sum: 12−h of its monthly components are already known at the origin, and only h months need forecasting. The naive forecast instead implicitly assumes that the month dropping out of the 12-month window equals the month coming in. I fixed the specification and the 36-month window before computing any forecast error (out/state/log.md). Nothing was tuned on the evaluation sample, and no alternatives were tried.
- Evaluation: RMSE and MAE over the 10 origins per horizon. Diebold–Mariano test of model vs naive: d_t = e²_naive − e²_model, with Bartlett long-run variance at truncation lag h−1 (h=3 weights 2/3, 1/3), no HLN factor, two-sided normal p-value. As an additional distribution-free measure, I report an exact two-sided sign test on the sign of d_t (`dm_details`).
- Revision estimand: the mean over the 10 h=1 target months of |g^current_T − yoy_first_release_T|.
- No randomness is used (`random_seed` = null). out/analysis.py regenerates every output deterministically.

## Results
RMSE / MAE in percentage points of YoY growth (out/table1.csv):

| model | h=1 RMSE | h=1 MAE | h=3 RMSE | h=3 MAE |
|---|---|---|---|---|
| naive | 0.997 (`rmse_naive_h1`) | 0.849 (`mae_naive_h1`) | 1.420 (`rmse_naive_h3`) | 1.200 (`mae_naive_h3`) |
| seasonal naive | 1.675 (`rmse_snaive_h1`) | 1.307 (`mae_snaive_h1`) | 1.316 (`rmse_snaive_h3`) | 1.066 (`mae_snaive_h3`) |
| AR(1) | 0.941 (`rmse_ar1_h1`) | 0.798 (`mae_ar1_h1`) | 1.187 (`rmse_ar1_h3`) | 0.953 (`mae_ar1_h3`) |
| rw_drift_yoy | 0.552 (`rmse_model_h1`) | 0.497 (`mae_model_h1`) | 1.054 (`rmse_model_h3`) | 0.747 (`mae_model_h3`) |

- The model has the lowest RMSE and MAE of the four methods at both horizons.
- Its RMSE is 44.6% lower than the naive RMSE at h=1 (`rel_improvement_rmse_h1` = 0.446) and 25.8% lower at h=3 (`rel_improvement_rmse_h3` = 0.258).
- AR(1) beats naive by about 6% (h=1) and 16% (h=3) in RMSE (ratios of the `rmse_ar1_h*` and `rmse_naive_h*` keys). Seasonal naive is the worst method at h=1, but it beats naive at h=3.
- Diebold–Mariano, model vs naive:
  - h=1: DM = 2.30, p = 0.021 (`dm_stat_h1`, `dm_pvalue_h1`), giving `conclusion_code_h1` = model_better_significant.
  - h=3: DM = 1.86, p = 0.062 (`dm_stat_h3`, `dm_pvalue_h3`), giving `conclusion_code_h3` = model_better_not_significant.
- Revisions: the first-release YoY of the 10 h=1 target months differs from today's published YoY by 0.548 pp on average (`mean_abs_revision_h1_targets`; per-month values in `abs_revision_h1_by_target`, range 0.17–0.98). This is the same order of magnitude as the model's h=1 RMSE, so a substantial part of what a "forecast error" against first release measures is measurement noise in the first release relative to today's data.
- Figure: out/figure1.png shows first-release actuals and the naive, AR(1) and model forecasts by target month, one panel per horizon.

## Uncertainty
- There are only n = 10 origins per horizon, and at h=3 the errors from neighbouring origins overlap. The DM statistic relies on asymptotic normality and on a long-run variance estimated from 10 points. With samples this small, DM tests without the HLN correction are known to over-reject, so the h=1 p-value of 0.021 (`dm_pvalue_h1`) should be read as indicative, not conclusive.
- A distribution-free check is weaker. The model had a lower squared error than naive at 8 of 10 origins at each horizon, and the exact two-sided sign-test p-value is 0.109 for both (`dm_details`). This is not significant at 5%.
- At h=3 the DM p-value of 0.062 (`dm_pvalue_h3`) is above 5%.
- The evaluation window covers one short period (target months 2025-09 to 2026-09). It includes an acceleration of YoY growth to 6.5–6.7% in 2026-05 and 2026-06 (first release), and it contains publication disruptions. The RMSE ranking could differ in other periods.
- I computed no bootstrap interval, since 10 dependent observations would make one unreliable.

## Limitations and open issues
- **Gate conflict (structural, reported, not fixed).** The output-contract gate's `forecasts_chronology` rule (origin_date < target_date) fails for all 40 h=1 rows. TASK.md defines the target as last(V)+h and formats `target_date` as YYYY-MM-01. With month-end origins, an h=1 target month always starts before the origin date, and for the six two-month-delay origins it has ended entirely before it. No date format that respects TASK.md can satisfy this rule, so I kept the TASK.md definition (TASK.md is authoritative). Information validity is unaffected: no target month is present in the vintage used, and vintage_used ≤ origin_date for every row (out/state/checks.md, C3, C10, C13). The latest gate receipt therefore reports passed = false on this single check only.
- With n = 10 origins, all significance statements are fragile (see Uncertainty).
- The model is univariate. PAYEMS and UMCSENT were not used, so I cannot say whether they would help. I deliberately ran no specification search on 10 origins.
- The actuals mix pre- and post-benchmark vintages (2026-08 and 2026-09 come from post-benchmark vintages). YoY ratios within a vintage largely cancel level shifts, but this is a possible source of extra error at the last h=3 targets.
- The two-month publication delays at six origins mean the h=1 results partly measure backcasting, not forecasting, ability.
- Workflow incident: a shell-directory slip briefly created a folder `data/out/` (containing only my own state notes). I removed it at once. No input file was modified, and all input SHA-256 values were re-verified afterwards (out/state/log.md).