Contract — F3-dev-01
Question. Over 36 expanding-window forecast origins (2023-07-01 … 2026-06-01), does a simple autoregressive-type model forecast the US unemployment rate (UNRATE, %, SA, monthly, final vintage) better than the naive no-change forecast at horizons h=1 and h=3 months?
Estimands. Per horizon: out-of-sample RMSE and MAE (percentage points) of naive and of one model; relative improvement 1 − RMSE_model/RMSE_naive. Meaningful as pseudo-out-of-sample accuracy on the revised (final-vintage) series, over this specific 36-origin window; not real-time accuracy. With 36 (overlapping for h=3) errors, sampling uncertainty is large.
Outputs. out/forecasts.csv (144 rows: origin_date,target_date,horizon,forecast,actual,model); out/results.json (keys task_id, model_name, model_description, n_origins, first_origin, last_origin, n_evaluated_h1, n_evaluated_h3, rmse/mae naive & model h1/h3, rel_improvement_rmse_h1/h3, feasibility, feasibility_reason, data_issues, n_input_rows, n_analysed, random_seed); out/report.md (Question; Data and exclusions; Method; Results; Uncertainty; Limitations and open issues); out/analysis.py; out/claims.json.
Information rules. Forecast at origin t uses only observations dated ≤ t, for estimation, model selection and transformations. Naive = latest non-missing observation ≤ t. Target = t+h; missing actual → row kept with empty actual, excluded from statistics; report evaluated counts.
Invalidity / impossibility. Using data > t; choosing the model by looking at out-of-sample errors; missing data making targets unevaluable (handled by exclusion, not imputation of actuals). Task is infeasible only if data unreadable; not the case.