F3-dev-01 — Rolling-origin forecasts of the US unemployment rate: can a simple time-series model beat the naive forecast?
Scientific question
Over the last three years, does a simple autoregressive-type model forecast the US civilian unemployment rate better than the naive (no-change) forecast at one- and three-month horizons? The estimands are the out-of-sample root-mean-squared error (RMSE) and mean absolute error (MAE) of (a) the naive forecast and (b) one model of your choice, computed over a fixed set of rolling (expanding-window) forecast origins, and the relative improvement of the model over the naive forecast, 1 − RMSE_model / RMSE_naive, at each horizon.
Inputs
data/UNRATE.csv— observed data: US unemployment rate (percent, seasonally adjusted, monthly), FRED series UNRATE (source BLS), retrieved 2026-10-30 (UTC) from https://fred.stlouisfed.org/graph/fredgraph.csv?id=UNRATE, exactly as served (headerobservation_date,UNRATE, one row per month dated YYYY-MM-01, 1948-01 to 2026-09). Missing observations appear as an empty value field. This is the current (final) vintage; seedata/README.mdfor provenance, SHA-256 and terms.
Information-availability rules
- Forecast origins: the 36 months t = 2023-07-01, 2023-08-01, …, 2026-06-01 (the
observation_dateof the origin month). Horizons h ∈ {1, 3} months; the target of origin t at horizon h is the month t + h. - A forecast made at origin t may use only observations dated ≤ t (expanding window: the training sample grows with t). Model selection, parameter estimation and any transformation (e.g. differencing, scaling) must use only observations dated ≤ t.
- Naive baseline: the forecast for every horizon is the most recent available observation dated ≤ t (if the origin month itself is missing, use the latest earlier observation).
- If the actual value at a target month is missing in the data, that (origin, horizon) pair still appears in the forecast ledger with an empty
actual, but is excluded from all error statistics; report the number of evaluated pairs per horizon. - The evaluation errors are computed on the data as given (final vintage); do not claim real-time validity.
Required outputs
out/forecasts.csv— the forecast ledger, one row per (origin, horizon, model), with columnsorigin_date(YYYY-MM-DD),target_date(YYYY-MM-DD),horizon(integer),forecast(number),actual(number, empty if missing),model(string:naivefor the baseline, and the value ofmodel_namefor your model). Expected: 36 origins × 2 horizons × 2 models = 144 rows.out/results.jsonwith the keys:
| key | type | unit | meaning |
|---|---|---|---|
task_id |
string | — | "F3-dev-01" |
model_name |
string | — | short name of your model (e.g. "AR2_diff"), used in the model column |
model_description |
string | — | one-sentence specification (lags, transformation, estimation sample start, estimation method) |
n_origins |
integer | — | number of forecast origins (36) |
first_origin, last_origin |
string | date | first and last origin dates |
n_evaluated_h1, n_evaluated_h3 |
integer | — | (origin, horizon) pairs with a non-missing actual, per horizon |
rmse_naive_h1, mae_naive_h1, rmse_naive_h3, mae_naive_h3 |
number | percentage points | naive-forecast errors per horizon |
rmse_model_h1, mae_model_h1, rmse_model_h3, mae_model_h3 |
number | percentage points | your model's errors per horizon |
rel_improvement_rmse_h1, rel_improvement_rmse_h3 |
number | fraction | 1 − rmse_model / rmse_naive (negative if the model is worse) |
feasibility |
string | — | "complete", "partial" or "infeasible" |
feasibility_reason |
string or null | — | explanation if not complete |
data_issues |
array of {"issue","handling"} | — | data problems detected and how handled (empty list if none) |
n_input_rows |
object | — | {"UNRATE.csv": number of data rows read, excluding the header} |
n_analysed |
integer | — | observations used in the last training window |
random_seed |
integer or null | — | seed used, if any |
out/report.mdwith the sections: Question; Data and exclusions; Method; Results; Uncertainty; Limitations and open issues.out/analysis.py— running~/wp29/venv/bin/python out/analysis.pyfrom the workspace root must regenerateout/results.jsonandout/forecasts.csvfromdata/only, deterministically.out/claims.json— a list of the central claims of the report: [{"id": "C1", "text": "...", "results_keys": ["..."], "status": "supported"|"unsupported"|"not_evaluated", "caveat": "..."}].
Constraints
- No network access. Python:
~/wp29/venv/bin/python(numpy, pandas, scipy, statsmodels available). Do not modifydata/. - This task is designed to be completed in well under 20 minutes of work.