# Real-time forecasting of US retail-sales YoY growth from ALFRED vintages (T1-forecast)

# Question

Using only the data published at each of 10 month-end forecast origins (2025-10-31 … 2026-07-31; first_origin, last_origin, n_origins), how well can the year-over-year log growth of US retail and food services sales (RSAFS), g_m = 100·ln(L_m/L_{m−12}), be forecast 1 and 3 months beyond the last published month, and does a model of my own beat the naive no-change forecast when errors are measured against the first-release YoY of each target month?

# Data and exclusions

• Inputs: 10 RSAFS and 10 PAYEMS ALFRED vintages (one per origin), UMCSENT (current vintage), RSAFS_current, first_release.csv. Row counts are in n_input_rows; every SHA-256 matched data/README.md (out/state/data_audit.md).
• Real-time design. The forecast at origin V uses only the RSAFS vintage dated V (vintage_used = origin_date for all 80 rows of out/forecasts.csv). Targets are last(V)+h, where last(V) is the last month in that vintage. Publication delays seen in the vintages: at the six origins 2025-10-31 … 2026-03-31 the last published month is two months before the origin month (last(V) = 2025-08 … 2026-01). The 2026-04-30 vintage adds two months at once (2026-02, 2026-03), and the last four origins have the usual one-month delay (last(V) = 2026-03 … 2026-06). Per-origin last(V) and delay are stored in real_time_design. Because of this, month 2026-03 is never an h=1 target, and for the first six origins the h=1 "forecast" is really a backcast of a month that had already ended but was not yet published.
• Actuals: yoy_first_release from first_release.csv. Where the first-release vintage is one of the shipped vintages, its values match that vintage exactly (check C4).
• RSAFS_current is used only for the revision estimand. PAYEMS and UMCSENT are read (for row counts) but are not used by any forecast.
• Exclusions (out/exclusions.csv): none. All 80 forecast–actual pairs (10 origins × 2 horizons × 4 models) have an actual. n_analysed = 20 model pairs (10 per horizon).
• Data issues found, with how each was handled, are listed in data_issues: irregular publication timing; PAYEMS gaps and a benchmark revision; pre-1978 UMCSENT gaps; a benchmark revision in RSAFS_current that also affects the first-release levels of 2026-08 and 2026-09; and routine monthly revisions.

# Method

• Baselines, computed from vintage V as TASK.md specifies: naive = g_{last(V)}; seasonal naive = g_{T−12}; AR(1) = OLS of g_m on a constant and g_{m−1} for 1993-02 ≤ m ≤ last(V), iterated h steps.
• Own model, model_name = rw_drift_yoy (see model_description). This is a base-effect-aware random walk with drift on log levels. The drift μ is the mean monthly log change over the 36 months ending at last(V) in vintage V. The forecast is ĝ_T = 100·(ln L_last + h·μ − ln L_{T−12}), where the base-month level L_{T−12} is already published in vintage V. The point of the model is that YoY growth is a 12-month sum: 12−h of its monthly components are already known at the origin, and only h months need forecasting. The naive forecast instead implicitly assumes that the month dropping out of the 12-month window equals the month coming in. I fixed the specification and the 36-month window before computing any forecast error (out/state/log.md). Nothing was tuned on the evaluation sample, and no alternatives were tried.
• Evaluation: RMSE and MAE over the 10 origins per horizon. Diebold–Mariano test of model vs naive: d_t = e²_naive − e²_model, with Bartlett long-run variance at truncation lag h−1 (h=3 weights 2/3, 1/3), no HLN factor, two-sided normal p-value. As an additional distribution-free measure, I report an exact two-sided sign test on the sign of d_t (dm_details).
• Revision estimand: the mean over the 10 h=1 target months of |g^current_T − yoy_first_release_T|.
• No randomness is used (random_seed = null). out/analysis.py regenerates every output deterministically.

# Results

RMSE / MAE in percentage points of YoY growth (out/table1.csv):

• The model has the lowest RMSE and MAE of the four methods at both horizons.
• Its RMSE is 44.6% lower than the naive RMSE at h=1 (rel_improvement_rmse_h1 = 0.446) and 25.8% lower at h=3 (rel_improvement_rmse_h3 = 0.258).
• AR(1) beats naive by about 6% (h=1) and 16% (h=3) in RMSE (ratios of the rmse_ar1_h* and rmse_naive_h* keys). Seasonal naive is the worst method at h=1, but it beats naive at h=3.
• Diebold–Mariano, model vs naive:
  • h=1: DM = 2.30, p = 0.021 (dm_stat_h1, dm_pvalue_h1), giving conclusion_code_h1 = model_better_significant.
  • h=3: DM = 1.86, p = 0.062 (dm_stat_h3, dm_pvalue_h3), giving conclusion_code_h3 = model_better_not_significant.
• Revisions: the first-release YoY of the 10 h=1 target months differs from today's published YoY by 0.548 pp on average (mean_abs_revision_h1_targets; per-month values in abs_revision_h1_by_target, range 0.17–0.98). This is the same order of magnitude as the model's h=1 RMSE, so a substantial part of what a "forecast error" against first release measures is measurement noise in the first release relative to today's data.
• Figure: out/figure1.png shows first-release actuals and the naive, AR(1) and model forecasts by target month, one panel per horizon.

# Uncertainty

• There are only n = 10 origins per horizon, and at h=3 the errors from neighbouring origins overlap. The DM statistic relies on asymptotic normality and on a long-run variance estimated from 10 points. With samples this small, DM tests without the HLN correction are known to over-reject, so the h=1 p-value of 0.021 (dm_pvalue_h1) should be read as indicative, not conclusive.
• A distribution-free check is weaker. The model had a lower squared error than naive at 8 of 10 origins at each horizon, and the exact two-sided sign-test p-value is 0.109 for both (dm_details). This is not significant at 5%.
• At h=3 the DM p-value of 0.062 (dm_pvalue_h3) is above 5%.
• The evaluation window covers one short period (target months 2025-09 to 2026-09). It includes an acceleration of YoY growth to 6.5–6.7% in 2026-05 and 2026-06 (first release), and it contains publication disruptions. The RMSE ranking could differ in other periods.
• I computed no bootstrap interval, since 10 dependent observations would make one unreliable.

# Limitations and open issues

• Gate conflict (structural, reported, not fixed). The output-contract gate's forecasts_chronology rule (origin_date < target_date) fails for all 40 h=1 rows. TASK.md defines the target as last(V)+h and formats target_date as YYYY-MM-01. With month-end origins, an h=1 target month always starts before the origin date, and for the six two-month-delay origins it has ended entirely before it. No date format that respects TASK.md can satisfy this rule, so I kept the TASK.md definition (TASK.md is authoritative). Information validity is unaffected: no target month is present in the vintage used, and vintage_used ≤ origin_date for every row (out/state/checks.md, C3, C10, C13). The latest gate receipt therefore reports passed = false on this single check only.
• With n = 10 origins, all significance statements are fragile (see Uncertainty).
• The model is univariate. PAYEMS and UMCSENT were not used, so I cannot say whether they would help. I deliberately ran no specification search on 10 origins.
• The actuals mix pre- and post-benchmark vintages (2026-08 and 2026-09 come from post-benchmark vintages). YoY ratios within a vintage largely cancel level shifts, but this is a possible source of extra error at the last h=3 targets.
• The two-month publication delays at six origins mean the h=1 results partly measure backcasting, not forecasting, ability.
• Workflow incident: a shell-directory slip briefly created a folder data/out/ (containing only my own state notes). I removed it at once. No input file was modified, and all input SHA-256 values were re-verified afterwards (out/state/log.md).