# Real-time forecasting of US retail-sales YoY growth from data vintages (T1-forecast)

# Question

Could the year-over-year growth of US retail and food services sales (RSAFS, g_m = 100·ln(L_m/L_{m−12}), percent) have been forecast in real time at 10 month-end origins (first_origin = 2025-10-31 to last_origin = 2026-07-31, n_origins = 10), h = 1 and h = 3 months beyond the last published month last(V)? And does a model of my own design beat the no-change (naive) forecast when errors are measured against the first-release YoY value of each target month?

# Data and exclusions

• Inputs (row counts in n_input_rows): 10 RSAFS ALFRED vintages and 10 PAYEMS vintages (one per origin), UMCSENT (current), RSAFS current vintage, and first_release.csv (28 target months, 2024-06 to 2026-09). All 23 SHA-256 hashes match data/README.md (out/state/data_audit.md).
• Real-time design. The forecast at origin V uses only the RSAFS vintage dated V (vintage_used = origin_date in all 80 rows of out/forecasts.csv). The target is last(V)+h. Publication delays: for the first six origins (2025-10-31 to 2026-03-31) last(V) is two months before the origin month (e.g. V = 2025-10-31 ends at 2025-08, so its h=1 target is 2025-09 and its h=3 target is 2025-11). From 2026-04-30 on it is one month before (the 2026-04-30 vintage added 2026-02 and 2026-03 at once). Targets therefore run from 2025-09 to 2026-07 (h=1) and from 2025-11 to 2026-09 (h=3). All 20 targets are in first_release.csv, and none of them is in the vintage used for its forecast.
• Actuals. yoy_first_release is used as given. Because of the irregular calendar, 2025-09 first appears in the 2025-11-30 vintage, and 2026-02 and 2026-03 both first appear in 2026-04-30. Their "first release" is defined at month-end granularity (see data_issues).
• Revisions. For 2025–2026, the current RSAFS file has levels about 4.7–4.9% below those in the 2026-07-31 vintage. YoY growth is always computed within one vintage, so levels are never mixed across vintages. RSAFS_current.csv is used only for the revision estimand (d).
• Exclusions (out/exclusions.csv): 20 candidate model forecast–actual pairs, 0 removed, n_analysed = 20 (10 per horizon). UMCSENT has 210 empty values, all before 1978. It is not used, and neither is PAYEMS.

# Method

• Baselines (from vintage V only): naive, where the forecast is g_{last(V)} for both horizons; seasonal naive, where the forecast is g_{T−12}; and AR(1), fitted by OLS of g_m on a constant and g_{m−1} for 1993-02 ≤ m ≤ last(V), then iterated h steps.
• My model (model_name = rw_drift_base_effect; full text in model_description) treats the log level as a random walk with drift. The drift is the mean monthly log change over the last 12 published months: d = (x_l − x_{l−12})/12. The forecast is g_{l+h} = 100·(x_l + h·d − x_{l+h−12}). Unlike the naive forecast, it accounts for the monthly changes that leave the 12-month window ("base effects"), all of which are already published at the origin. It uses only the RSAFS vintage dated V. I fixed the specification before computing any forecast error, and nothing was tuned on the evaluation targets.
• Evaluation: RMSE and MAE against yoy_first_release. The Diebold–Mariano statistic for the model against naive uses d_t = e²_naive − e²_model and a Bartlett (Newey–West) long-run variance with truncation lag h−1, with no HLN factor and a two-sided normal p-value, as specified in TASK.md.
• Revision (d): the mean over the 10 h=1 targets of |g^current_T − yoy_first_release_T|.
• out/analysis.py regenerates everything deterministically (random_seed = null, since no randomness is used).

# Results

Table 1 (out/table1.csv; percentage points; n = 10 per cell):

• At h=1 the model has the lowest RMSE and MAE of the four methods. Its RMSE is 44.8% below the naive RMSE (rel_improvement_rmse_h1 = 0.448). DM = 2.43 (dm_stat_h1), p = 0.015 (dm_pvalue_h1), so conclusion_code_h1 = model_better_significant.
• At h=3 the model again has the lowest RMSE and MAE. Its RMSE is 19.6% below naive (rel_improvement_rmse_h3 = 0.196). DM = 1.55 (dm_stat_h3), p = 0.12 (dm_pvalue_h3), so conclusion_code_h3 = model_better_not_significant.
• Among the baselines, AR(1) is slightly better than naive at both horizons. Seasonal naive is the worst at h=1 but beats naive at h=3.
• Revision: the mean absolute gap between first-release and current-vintage YoY growth for the h=1 targets is 0.548 pp (mean_abs_revision_h1_targets). That is about the same size as the model's h=1 RMSE.
• out/figure1.png shows the first-release actuals with the naive, AR(1) and model forecasts for each horizon.

# Uncertainty

• With only n = 10 origins, the DM statistic is compared with a normal distribution far outside its asymptotic regime. Its p-values should be read as rough indications.
• h=1: the model's squared error is below the naive one at 8 of 10 origins (aux file out/state/aux_diagnostics.json, n_origins_model_sq_error_below_naive). However, a single origin (2026-05-31) contributes a loss differential of 3.11, about 45% of the total (loss_diff_by_origin).
• HLN sensitivity: applying the Harvey–Leybourne–Newbold factor and t(9) critical values gives a p-value of 0.047 at h=1 and 0.28 at h=3 (sensitivity_dm_hln_pvalue_t_nminus1). These are not the required estimand and are reported only as a sensitivity check. The h=1 significance is therefore borderline once small-sample corrections are applied.
• h=3: the forecasts overlap, so errors are serially correlated. With lag-2 Bartlett weights, a long-run variance estimated from 10 points is very imprecise. The h=3 improvement is not distinguishable from zero.
• No bootstrap was run. A resampling interval built from 10 dependent observations would add little reliability.

# Limitations and open issues

• Small sample. Ten origins in a single 2025–2026 episode, which includes a large jump in YoY growth in spring 2026, cannot establish that the model would generally beat the naive forecast. The h=1 result depends heavily on a few origins.
• What the model exploits. Its advantage is mechanical. It uses the base effects already visible in the vintage, so it is a sensible real-time benchmark rather than a behavioural model. PAYEMS and UMCSENT were deliberately not used, so their value as predictors was not evaluated.
• First-release actuals. These include values that are not advance estimates (2026-02) and are themselves revised later, by a mean of 0.548 pp (mean_abs_revision_h1_targets). Rankings evaluated against the current vintage could differ; this was not evaluated.
• Gate conflict (open issue). The executable gate requires origin_date < target_date in out/forecasts.csv. Under TASK.md's own definitions (target = last(V)+h, target_date = YYYY-MM-01, month-end origins, and 1–2 month publication lags), every h=1 target month starts on or before its origin date. For example, origin 2025-10-31 has its h=1 target at 2025-09-01. The 40 h=1 rows therefore fail this gate check by construction. Satisfying it would mean misdating the targets, so I did not change them. Instead I verified the substantive information rule: each forecast uses only vintage V, and no target value is present in that vintage (out/state/checks.md). As a result, phases P4/P5 are recorded as "blocked" by this check only. All other gate checks were run and recorded.