# T1-forecast — Real-time forecasting of US retail-sales growth from data vintages: a complete mini-project

## Scientific question
How well could the year-over-year growth of US retail and food services sales have been forecast *in real time* — using only the data actually published at each forecast origin — one and three months beyond the last published month, and does a model of your own design beat the no-change (naive) forecast once the forecast errors are measured against the first published value of each target month? This is an end-to-end forecasting project: question → data audit → real-time baselines → your model → rolling-origin evaluation → uncertainty → figure → short paper. The estimands are, for horizons h ∈ {1, 3} months and over the 10 forecast origins defined below: (a) the out-of-sample RMSE and MAE of three baselines (naive, seasonal naive, AR(1)) and of your model, all evaluated against the first-release value of the target; (b) the relative improvement of your model over the naive forecast, 1 − RMSE_model / RMSE_naive; (c) the Diebold–Mariano statistic of your model against the naive forecast under squared-error loss, with a long-run variance that is robust to the serial correlation of multi-step forecast errors; (d) the mean absolute revision between the first-release growth of the h = 1 target months and their currently published growth. Report what the data show, including a negative result if your model does not beat the naive forecast.

## Inputs
All series are monthly, seasonally adjusted, dated by the first day of the month, and come from FRED/ALFRED (Federal Reserve Bank of St. Louis); full provenance, retrieval times (2026-10-30 UTC), SHA-256 and terms are in `data/README.md`.
- `data/vintages/RSAFS_vintage_<YYYY-MM-DD>.csv` — 10 files, one per forecast origin (2025-10-31, 2025-11-30, 2025-12-31, 2026-01-31, 2026-02-28, 2026-03-31, 2026-04-30, 2026-05-31, 2026-06-30, 2026-07-31): the series RSAFS — Advance Retail Sales: Retail Trade and Food Services, total, millions of dollars, SA (US Census Bureau) — exactly as it was published on that date (ALFRED vintage; header `observation_date,RSAFS_<yyyymmdd>`; first observation 1992-01; the last row is the most recent month published at that date). The target variable is the year-over-year growth g_m = 100 · ln(L_m / L_{m−12}) in percent, where L is the level in the relevant vintage.
- `data/vintages/PAYEMS_vintage_<YYYY-MM-DD>.csv` — 10 files, same dates: the series PAYEMS — All Employees, Total Nonfarm, thousands of persons, SA (US Bureau of Labor Statistics) — as published on that date (ALFRED vintage; first observation 1939-01). A candidate predictor.
- `data/UMCSENT.csv` — the series UMCSENT — University of Michigan Index of Consumer Sentiment (1966Q1 = 100) — current vintage as served by FRED on 2026-10-30 (header `observation_date,UMCSENT`; 1952-11 to 2026-09; the field is empty for the months before 1978 in which the survey was not conducted). A candidate predictor; see the availability rule below.
- `data/RSAFS_current.csv` — RSAFS as served by FRED on 2026-10-30 (header `observation_date,RSAFS`; 1992-01 to 2026-09), containing all revisions known today. **Provided only for the revision analysis (estimand (d)); it may not enter any forecast.**
- `data/first_release.csv` — one row per target month from 2024-06 to 2026-09: `target_month`, `first_release_vintage` (the first month-end ALFRED vintage in the designer's grid 2024-06-30 … 2026-10-29 in which the month appears), `level_first_release` (the RSAFS level of the month in that vintage), `level_prev_year_same_vintage` (the level of the month twelve months earlier in the same vintage) and `yoy_first_release` = 100 · ln(level_first_release / level_prev_year_same_vintage), 6 decimals. This is the "actual" against which every forecast is evaluated.
- `data/README.md` — provenance.

## Information-availability rules
- Forecast origins are the 10 vintage dates listed above. A forecast made at origin V may use **only** the RSAFS vintage file dated V, the PAYEMS vintage file dated V, and the UMCSENT values of months on or before the month of V (the final sentiment reading of a month is published before the end of that month and is not revised afterwards; the current UMCSENT file is used with this rule). Estimation samples, model selection, transformations and the values plugged into a forecast must all respect this rule at every origin.
- Let last(V) be the most recent month present in the RSAFS vintage dated V (it is not always the month before V: publication delays occurred). The target of origin V at horizon h is the month last(V) + h; the actual is `yoy_first_release` of that month in `data/first_release.csv`.
- Baselines (all three required, computed from the RSAFS vintage dated V):
  - **naive**: the forecast for both horizons is g_{last(V)}, the last available growth value in the vintage;
  - **seasonal naive**: the forecast for target month T is g_{T−12} from the vintage;
  - **AR(1)**: ordinary least squares of g_m on a constant and g_{m−1} over all consecutive pairs with 1993-02 ≤ m ≤ last(V) in the vintage, then iterated (plug-in) forecasts h steps beyond last(V).
- Your model is free (it may use the vintage dynamics, PAYEMS and/or UMCSENT), but it must obey the availability rule and must be described in `model_description`.
- Diebold–Mariano test (model vs naive, per horizon): loss differential d_t = e²_naive,t − e²_model,t over the 10 origins; `dm_stat_h` = d̄ / sqrt(σ̂²_LR / n) where σ̂²_LR is the Bartlett-kernel (Newey–West) long-run variance of d_t with truncation lag h − 1 (lag 0 only at h = 1; lags 1 and 2 with weights 2/3 and 1/3 at h = 3), without the Harvey–Leybourne–Newbold small-sample factor; `dm_pvalue_h` is the two-sided p-value from the standard normal. You may add other uncertainty measures (e.g. a bootstrap interval for the RMSE difference) in the report.
- The series `RSAFS_current.csv` is used only for estimand (d): mean over the 10 h = 1 target months of |g^current_T − yoy_first_release_T|, where g^current is the growth computed from the current file.

## Required outputs
- `out/forecasts.csv` — exactly one row per (origin, horizon, model): 10 origins × 2 horizons × 4 models = 80 rows, columns `origin_date` (vintage date, YYYY-MM-DD), `target_date` (YYYY-MM-01), `horizon` (1 or 3), `forecast` (percent), `actual_first_release` (percent, from `first_release.csv`), `model` (one of `naive`, `snaive`, `ar1`, `model`), `vintage_used` (the vintage date of the RSAFS file used, YYYY-MM-DD).
- `out/results.json` with the keys:

| key | type | unit | meaning |
|---|---|---|---|
| `task_id` | string | — | "T1-forecast" |
| `model_name` | string | — | short name of your model |
| `model_description` | string | — | specification, predictors, estimation window, alignment of predictors with the availability rule |
| `n_origins` | integer | — | 10 |
| `first_origin`, `last_origin` | string | — | first and last origin dates (YYYY-MM-DD) |
| `rmse_naive_h1`, `mae_naive_h1`, `rmse_naive_h3`, `mae_naive_h3` | number | percent | naive baseline errors against first-release actuals |
| `rmse_snaive_h1`, `mae_snaive_h1`, `rmse_snaive_h3`, `mae_snaive_h3` | number | percent | seasonal-naive baseline errors |
| `rmse_ar1_h1`, `mae_ar1_h1`, `rmse_ar1_h3`, `mae_ar1_h3` | number | percent | AR(1) baseline errors |
| `rmse_model_h1`, `mae_model_h1`, `rmse_model_h3`, `mae_model_h3` | number | percent | your model's errors |
| `rel_improvement_rmse_h1`, `rel_improvement_rmse_h3` | number | — | 1 − RMSE_model / RMSE_naive |
| `dm_stat_h1`, `dm_pvalue_h1`, `dm_stat_h3`, `dm_pvalue_h3` | number | — | Diebold–Mariano statistic and two-sided p-value, model vs naive, as defined above |
| `conclusion_code_h1`, `conclusion_code_h3` | string | — | one of `model_better_significant`, `model_better_not_significant`, `naive_better_not_significant`, `naive_better_significant` ("better" = lower RMSE; "significant" = `dm_pvalue_h` < 0.05) |
| `mean_abs_revision_h1_targets` | number | percent | estimand (d) |
| `feasibility` | string | — | "complete", "partial" or "infeasible" |
| `feasibility_reason` | string or null | — | explanation if not complete |
| `data_issues` | array of {"issue","handling"} | — | every data problem detected (irregular publication timing, missing values, revisions that affect the design, …) and how it was handled; empty list if none |
| `n_input_rows` | object | rows | {file name: number of data rows (lines after the header)} for every file under `data/` that you read |
| `n_analysed` | integer or null | — | number of forecast–actual pairs evaluated for your model (both horizons) |
| `random_seed` | integer or null | — | seed used, if any (e.g. bootstrap) |

- `out/table1.csv` — the RMSE/MAE table: one row per (model, horizon) with columns `model`, `horizon`, `rmse`, `mae`, `n`.
- `out/figure1.png` — first-release actuals and the forecasts of the naive baseline and of your model over the target months, one panel per horizon (or an equivalent single figure).
- `out/report.md` — a short paper with the sections: Question; Data and exclusions; Method; Results; Uncertainty; Limitations and open issues. The "Data and exclusions" section must document the real-time design (which vintage feeds which forecast, publication delays observed) and the "Uncertainty" section must interpret the Diebold–Mariano results given the small number of origins.
- `out/analysis.py` — running `~/wp29/venv/bin/python out/analysis.py` from the workspace root must regenerate `out/results.json`, `out/forecasts.csv`, `out/table1.csv` and `out/figure1.png` from `data/` only, deterministically.
- `out/claims.json` — a list of the central claims of the report: [{"id": "C1", "text": "...", "results_keys": ["..."], "status": "supported"|"unsupported"|"not_evaluated", "caveat": "..."}].

## Constraints
- No network access. Python: `~/wp29/venv/bin/python` (numpy, pandas, scipy, statsmodels, matplotlib available). Do not modify `data/`.
- Budget class: transfer mini-project — 12.00 USD cap, 60 minutes wall, 150 turns per session. The project is designed to be completed in well under 45 minutes of work.