SPB Git

spb/wp46_uqo

Public

WP46: wide scoring, analysis pipeline (DM/MCS/SPA/pooled bootstrap/regimes/tail risk/accounting)

Simon-Pierre Boucher committed (Nov 9, 2026) parent 76b91fc Browse files

16 changed files +3,516 −6

added model_cards/arch5.md +39 −0
@@ -0,0 +1,39 @@
1 +# Model card — `arch5` (ARCH(5))
2 +
3 +## 1. Identity
4 +- **model_id:** `arch5` · **family:** classical · **tier:** A · **status:** implemented · **impl:** arch 8.0.0 (`ARCH(p=5)`)
5 +- **Label:** ARCH(5) · **distributions:** normal, t, ged, skewt
6 +- **Seminal citation:** `engle1982autoregressive`
7 +- **Other references:** `bollerslev1986generalized` (GARCH as parsimonious ARCH(∞)); `bollerslev1992arch`, `engle2001garch` (surveys); innovation laws `bollerslev1987conditionally` (t), `nelson1991conditional` (GED), `hansen1994autoregressive` (skew-t).
8 +
9 +## 2. Definition
10 +$$r_t = \mu + \varepsilon_t,\quad \varepsilon_t = \sigma_t z_t,\qquad \sigma_t^2 = \omega + \sum_{i=1}^{5}\alpha_i\,\varepsilon_{t-i}^2 .$$
11 +Pre-sample squared residuals ($t - i < 0$) are replaced by the backcast. Innovation laws (arch parameterisations, all with unit variance): normal; Student-t (ν); GED (ν); Hansen skew-t (η, λ).
12 +
13 +## 3. Parameter space and admissibility
14 +- Parameters: `mu, omega, alpha[1..5]` (+ `nu` / `eta, lambda`).
15 +- arch bounds: $\omega \in [10^{-8}v, 10v]$ with $v = \overline{\varepsilon^2}$; $\alpha_i \in [0, 1]$; constraints $\omega > 0$, $\alpha_i \ge 0$, $\sum_i\alpha_i < 1$ (linear inequalities passed to SLSQP). Distribution bounds: ν ∈ [2.05, 500] (t), ν ∈ [1.01, 500] (GED), η ∈ [2.05, 300], λ ∈ [−1, 1].
16 +- Admissibility flag (`ArchModel.fit`): $\sum_i \alpha_i < 1$ (second-order stationarity). Positivity $\omega > 0$, $\alpha_i \ge 0$ is sufficient (and necessary for pure ARCH).
17 +
18 +## 4. Nesting and relations
19 +Nests `constvar` (α = 0). Not nested in GARCH(1,1) (which is ARCH(∞) with geometrically decaying weights); the registry keeps ARCH(5) as the pre-specified "pure ARCH" order variant. The HAR regressors (daily/weekly/monthly averages) are a restricted long ARCH on realized measures.
20 +
21 +## 5. Estimation
22 +arch `ConstantMean` + `ARCH(5)` + distribution, `fit(disp="off", options={"maxiter": 1000})`: SLSQP on the exact log-likelihood $\sum_t[\log f_z(\varepsilon_t/\sigma_t) - \tfrac12\log\sigma_t^2]$ with the bounds/constraints of §3. Starting values: arch grid over $\alpha \in \{0.10, 0.15, …, 0.90\}$ split equally across the 5 lags, $\omega = (1-\alpha)\operatorname{var}(\varepsilon)$, best Gaussian log-likelihood kept. Initial variance: backcast $= \sum_{i<\min(75,T)} w_i\varepsilon_i^2$, $w_i \propto 0.94^i$. Retries: ≤ 2 restarts from the previous estimate scaled by $0.9 + 0.2(i+1)/3$ with maxiter 2000 if `convergence_flag ≠ 0`. Warnings (ConvergenceWarning, DataScaleWarning — arch warns but does not rescale) stored in `notes`. Only point estimates are used in the rolling benchmark.
23 +
24 +## 6. Forecasting
25 +arch `GARCH._analytic_forecast` (ARCH is `GARCH(p=5, o=0, q=0)`): $\hat\sigma^2_{t+k|t} = \omega + \sum_{i=1}^{5}\alpha_i\,\widetilde{\varepsilon^2}_{t+k-i}$ where $\widetilde{\varepsilon^2}_{s} = \varepsilon_s^2$ for $s \le t$ and $= \hat\sigma^2_{s|t}$ for $s > t$. Exact under $E[z^2] = 1$. Cumulative $h$-day variance = sum of the per-step forecasts. Method: `analytic`.
26 +
27 +## 7. Implementation
28 +- `src/wp46/models/arch_models.py` (`SPECS["arch5"]`), arch 8.0.0 (`sheppard2025arch`).
29 +- Cross-check: not in the rugarch fixture (`results/metrics/rugarch_crosscheck.json` covers garch11, gjr11, egarch11, aparch11, figarch11, igarch11, cgarch11, nagarch11, garch11-skewt); the recursion is arch's `garch_recursion`, which the custom GARCH(1,1) test matches to 1e-9 (`tests/unit/test_custom_models.py::test_custom_garch_equals_arch_loglik_and_forecast`).
30 +- Tests: none dedicated; distributions checked in `tests/unit/test_distributions.py::test_loglik_matches_arch`.
31 +
32 +## 8. Known limitations / failure modes
33 +Short memory (5 days): under-reacts to persistent volatility; estimates frequently hit the $\alpha_i = 0$ bound or the $\sum\alpha_i < 1$ constraint on persistent series (boundary optima, SLSQP `convergence_flag ≠ 0`); forecasts revert to $\omega/(1-\sum\alpha_i)$ within a few days. Fourth-moment conditions for QMLE inference (`he1999properties`) are not checked (only point estimates are used).
34 +
35 +## 9. Computational footprint
36 +Analytic forecasts; 7–9 parameters; the 5-lag recursion is numba-compiled in arch. No pilot timing (not in the pilot). Expected cost between `garch11` and `aparch11`.
37 +
38 +## 10. Literature expectations
39 +`hansen2005forecast` report ARCH(q) specifications among the worst of their 330 models on both DM/$ and IBM; `bollerslev1986generalized` motivated GARCH precisely by the long ARCH orders needed. Expectation: dominated by GARCH(1,1) at every horizon; included as the pre-registered pure-ARCH order variant (protocol §4).
added model_cards/constvar.md +39 −0
@@ -0,0 +1,39 @@
1 +# Model card — `constvar` (constant variance, window mean)
2 +
3 +## 1. Identity
4 +- **model_id:** `constvar` · **family:** baseline · **tier:** A · **status:** implemented · **impl:** arch 8.0.0 (`ConstantVariance`)
5 +- **Label (registry):** Constant variance (window mean) · **distributions (registry):** normal
6 +- **Seminal citation:** none (registry `ref: null`); the constant-variance model is the degenerate GARCH(1,1) with α = β = 0 (`bollerslev1986generalized`).
7 +- **Other references:** `engle1982autoregressive` (ARCH as the alternative to constant variance); `hansen2005forecast` (uses the same kind of benchmarks).
8 +
9 +## 2. Definition
10 +Conditional mean and variance (returns in percent, variances in percent²):
11 +$$r_t = \mu + \varepsilon_t,\qquad \varepsilon_t = \sigma_t z_t,\qquad \sigma_t^2 = \sigma^2 \ \ \text{for all } t .$$
12 +Innovation law: $z_t \sim$ i.i.d. N(0,1) only (registry).
13 +
14 +## 3. Parameter space and admissibility
15 +- Parameters: `mu`, `sigma2`; $\sigma^2 > 0$. arch bounds: $\sigma^2 \in [v/10^5,\ 10(v + \bar r^2)]$ with $v$ the sample variance of the residuals; constraint $\sigma^2 \ge 0$.
16 +- No stationarity condition; the admissibility flag is always `True` in `ArchModel.fit` (no model-specific check).
17 +
18 +## 4. Nesting and relations
19 +Nested in GARCH(1,1) ($\alpha = \beta = 0$), in ARCH(5) ($\alpha_i = 0$) and in every GARCH-type model of the registry. It is the "no dynamics" floor of the benchmark; `histvar252` is its rolling-window analogue without a mean equation.
20 +
21 +## 5. Estimation
22 +arch 8.0.0 `ConstantMean` + `ConstantVariance` + `Normal`: closed form (arch's `_fit_no_arch_normal_errors`: $\hat\mu$ = window mean, $\hat\sigma^2$ = mean squared residual over the 1,000-day window) — no numerical optimisation, no backcast needed, no retries. Wrapper: `src/wp46/models/arch_models.py::ArchModel.fit`. Standard errors are not used: only point estimates enter the rolling benchmark (`src/wp46/estimation/rolling.py`).
23 +
24 +## 6. Forecasting
25 +`res.forecast(horizon=22, method="analytic")`: $\hat\sigma^2_{t+k|t} = \hat\sigma^2$ for every $k$ (flat term structure); cumulative $h$-day variance $= h\,\hat\sigma^2$ (`Forecast.cumulative`). Method recorded: `analytic`.
26 +
27 +## 7. Implementation
28 +- `src/wp46/models/arch_models.py` (`SPECS["constvar"]`, `ArchModel`), library arch 8.0.0 (`sheppard2025arch`).
29 +- Cross-check: not part of the rugarch fixture (`results/metrics/rugarch_crosscheck.json` has no `constvar` entry); the closed-form estimate needs no numerical cross-check.
30 +- Tests: no dedicated unit test; exercised only through the driver machinery (`tests/integration/test_rolling_driver.py` uses `garch11`, `har_rv`, `gjr11`, `heavy11`, not `constvar`).
31 +
32 +## 8. Known limitations / failure modes
33 +No dynamics, so it is mis-specified for every instrument in the universe; its only role is a floor. No convergence failures possible. With rolling re-estimation it reacts to volatility changes with the 1,000-day window lag.
34 +
35 +## 9. Computational footprint
36 +Closed form (fastest model of the registry together with `histvar*`); analytic forecasts. No pilot timing is recorded for `constvar` in `results/metrics/pilot/pilot_report.json` (pilot models: garch11, egarch11, gjr11, aparch11, nagarch11, realgarch11, har_rv, histvar22).
37 +
38 +## 10. Literature expectations
39 +Hansen & Lunde (`hansen2005forecast`) and every later comparison find constant-variance / no-dynamics benchmarks dominated by any ARCH-type model at the 1-day horizon; the literature matrix (`docs/literature_matrix.md`, §a.1) lists it as a baseline only. Expected to be excluded from the MCS at short horizons; at h = 22 the gap to GARCH(1,1) is expected to narrow because mean reversion brings GARCH forecasts toward the unconditional variance (`christoffersen2000how`).
added model_cards/ewma094.md +38 −0
@@ -0,0 +1,38 @@
1 +# Model card — `ewma094` (EWMA / RiskMetrics 1994, λ = 0.94)
2 +
3 +## 1. Identity
4 +- **model_id:** `ewma094` · **family:** baseline · **tier:** A · **status:** implemented · **impl:** arch 8.0.0 (`EWMAVariance(lam=0.94)`, λ fixed)
5 +- **Label:** EWMA (RiskMetrics 1994, lambda = 0.94) · **distributions:** normal
6 +- **Seminal citation:** `jpmorgan1996riskmetrics` (RiskMetrics Technical Document, 4th ed.)
7 +- **Other references:** `engle1986modelling` (IGARCH, of which EWMA is the zero-intercept case; registry `nested_in: igarch11`); `zumbach2009riskmetrics` (RM2006 successor).
8 +
9 +## 2. Definition
10 +$$r_t = \mu + \varepsilon_t,\qquad \sigma_t^2 = \lambda\,\sigma_{t-1}^2 + (1-\lambda)\,\varepsilon_{t-1}^2,\qquad \lambda = 0.94 .$$
11 +arch's `ewma_recursion` starts from $\sigma_0^2$ = backcast. Innovation law: N(0,1) (only μ is estimated; the Gaussian likelihood is used to estimate μ).
12 +
13 +## 3. Parameter space and admissibility
14 +- Estimated: `mu` only. λ is fixed at 0.94 (`parameter_names()` is empty for the volatility process).
15 +- $0 < \lambda < 1$ enforced at construction. No stationarity: the process is integrated ($\lambda + (1-\lambda) = 1$) and has no unconditional variance. Admissibility flag always `True` (no check in `ArchModel.fit`).
16 +
17 +## 4. Nesting and relations
18 +EWMA = IGARCH(1,1) with $\omega = 0$, $\alpha = 1-\lambda = 0.06$, $\beta = \lambda$ (`igarch11` card); = GARCH(1,1) on the unit-persistence boundary with zero intercept. `rm2006` is a weighted mixture of 14 EWMAs with different decays.
19 +
20 +## 5. Estimation
21 +arch `ConstantMean` + `EWMAVariance(0.94)` + `Normal`, `fit(disp="off", options={"maxiter": 1000})`: SLSQP over the single parameter μ (Gaussian log-likelihood with the EWMA variance path); starting value = sample mean; backcast = exponentially weighted average of the first min(75, T) squared residuals (weights ∝ 0.94ⁱ). Restarts (≤ 2) as for all arch-backed models (`src/wp46/models/arch_models.py::ArchModel.fit`). Only point estimates are used in the rolling benchmark; filtering with `am.fix(params)`.
22 +
23 +## 6. Forecasting
24 +arch `_analytic_forecast` for EWMA: $\hat\sigma^2_{t+1|t} = \lambda\sigma_t^2 + (1-\lambda)\varepsilon_t^2$ and $\hat\sigma^2_{t+k|t} = \hat\sigma^2_{t+1|t}$ for $k = 2..22$ (flat; a martingale variance). Cumulative $h$-day variance $= h\,\hat\sigma^2_{t+1|t}$. Method: `analytic`.
25 +
26 +## 7. Implementation
27 +- `src/wp46/models/arch_models.py` (`SPECS["ewma094"]`, `fixed=True`), arch 8.0.0 (`sheppard2025arch`).
28 +- Cross-check: none in `results/metrics/rugarch_crosscheck.json` (parameter-free recursion; rugarch has no fixed-λ EWMA entry in the fixture script).
29 +- Tests: no dedicated test; not in the pilot.
30 +
31 +## 8. Known limitations / failure modes
32 +Fixed decay calibrated on 1990s daily data; integrated dynamics give a flat term structure (no mean reversion), which the literature identifies as its main weakness at h = 5, 22. Cannot fail to converge in the variance recursion (only μ is optimised).
33 +
34 +## 9. Computational footprint
35 +One-parameter SLSQP, analytic forecasts: among the cheapest estimated models. No pilot timing.
36 +
37 +## 10. Literature expectations
38 +`hansen2005forecast` include RiskMetrics-type EWMA among the dominated benchmarks on the DM/$ and IBM series; `jpmorgan1996riskmetrics` is a risk-management standard rather than a forecasting contender. The literature matrix (§a.1) lists it as an "industry baseline; flat term structure" — expectation: close to GARCH(1,1) at h = 1 in calm periods, inferior at h = 22 and in regimes where mean reversion matters.
added model_cards/garch11.md +42 −0
@@ -0,0 +1,42 @@
1 +# Model card — `garch11` (GARCH(1,1), the benchmark)
2 +
3 +## 1. Identity
4 +- **model_id:** `garch11` · **family:** classical · **tier:** A · **status:** implemented · **benchmark: true** · **impl:** arch 8.0.0 (`GARCH(p=1, o=0, q=1)`)
5 +- **Label:** GARCH(1,1) · **distributions:** normal, t, ged, skewt
6 +- **Seminal citation:** `bollerslev1986generalized`
7 +- **Other references:** `engle1982autoregressive` (ARCH); `nelson1990stationarity` (strict stationarity); `bollerslev1992quasi`, `lee1994asymptotic`, `lumsdaine1996consistency` (QMLE theory); `hansen2005forecast` ("does anything beat a GARCH(1,1)?"); `engle2001garch`, `bollerslev2010glossary` (nomenclature); distributions `bollerslev1987conditionally`, `nelson1991conditional`, `hansen1994autoregressive`.
8 +
9 +## 2. Definition
10 +$$r_t = \mu + \varepsilon_t,\quad \varepsilon_t = \sigma_t z_t,\qquad \sigma_t^2 = \omega + \alpha\,\varepsilon_{t-1}^2 + \beta\,\sigma_{t-1}^2 ,$$
11 +with $\sigma_0^2$ initialised as $\omega + (\alpha+\beta)\cdot\text{backcast}$ (arch `garch_recursion`: both pre-sample terms use the backcast). Innovations $z_t$ i.i.d. with unit variance: normal; Student-t(ν), ν > 2; GED(ν); Hansen skew-t(η, λ). Order convention: arch's `p` = ARCH order, `q` = GARCH order (Bollerslev's (p,q) is the reverse; irrelevant for (1,1)).
12 +
13 +## 3. Parameter space and admissibility
14 +- Parameters: `mu, omega, alpha[1], beta[1]` (+ `nu` or `eta, lambda`).
15 +- arch bounds: $\omega \in [10^{-8}v, 10v]$ ($v = \overline{\varepsilon^2}$), $\alpha, \beta \in [0, 1]$; linear constraints $\omega > 0$, $\alpha \ge 0$, $\beta \ge 0$, $\alpha + \beta < 1$ (SLSQP inequality constraints; no reparameterisation). Distribution bounds: ν ∈ [2.05, 500] (t), ν ∈ [1.01, 500] (GED), η ∈ [2.05, 300], λ ∈ [−1, 1].
16 +- Admissibility flag: $\alpha + \beta < 1$ (covariance stationarity, unconditional variance $\omega/(1-\alpha-\beta)$). The strict-stationarity condition $E\log(\alpha z^2 + \beta) < 0$ (`nelson1990stationarity`) is weaker and not used as the flag.
17 +- (The custom cross-check class `garch11_custom` in `src/wp46/models/custom.py` uses the reparameterisation $\omega = e^{\theta_0}$, $s = \alpha+\beta = \sigma(\theta_1)$, $\alpha = s\,\sigma(\theta_2)$; it is not the registry implementation.)
18 +
19 +## 4. Nesting and relations
20 +Nests `constvar` (α = β = 0) and ARCH(1) (β = 0). Nested in `garch21`, `garch12`, `gjr11` (γ = 0), `nagarch11` (θ = 0), `qgarch11` (γ = 0), `aparch11` (δ = 2, γ = 0), `avgarch11`/`tarch11` only through the power change (δ = 1 vs 2, not nested), `cgarch11` (φ = 0, ρ = 0 collapse), `figarch11` (d = 0), `garchx_rv11` (γ = 0), `msgarch2` (identical regimes), and is the ν → ∞ limit of `gas_t11` up to reparameterisation. `igarch11` is the boundary α + β = 1; `ewma094` additionally sets ω = 0.
21 +
22 +## 5. Estimation
23 +arch `ConstantMean` + `GARCH(1,0,1)` + distribution; `fit(disp="off", options={"maxiter": 1000})` → SLSQP on the exact log-likelihood $\sum_t[\log f_z(\varepsilon_t/\sigma_t) - \tfrac12\log\sigma_t^2]$ with bounds/constraints of §3 (`src/wp46/models/arch_models.py::ArchModel.fit`). Starting values: arch grid α ∈ {0.01, 0.05, 0.1, 0.2} × persistence ∈ {0.5, 0.7, 0.9, 0.98}, $\omega = (1-\text{pers})\,\overline{\varepsilon^2}$, best Gaussian log-likelihood; distribution start ν from arch. Initial variance: arch backcast (exponentially weighted mean of the first min(75, T) squared residuals, weights ∝ 0.94ⁱ). Retries: ≤ 2 restarts (previous estimate × (0.9 + 0.2(i+1)/3), maxiter 2000) if `convergence_flag ≠ 0`; the better of the fits is kept. arch warns (DataScaleWarning) but does not rescale when var(ε) ∉ [0.1, 10 000) (percent units keep equities/FX inside the range; very low-volatility instruments may warn). Standard errors: arch's robust covariance is computed by arch but **only point estimates are used** in the rolling benchmark (`rolling.py`: refit every 5 days, daily filtering with `am.fix(params)`).
24 +
25 +## 6. Forecasting
26 +arch `GARCH._analytic_forecast`:
27 +$$\hat\sigma^2_{t+1|t} = \omega + \alpha\varepsilon_t^2 + \beta\sigma_t^2,\qquad \hat\sigma^2_{t+k|t} = \omega + (\alpha + \beta)\,\hat\sigma^2_{t+k-1|t},\ k = 2..22,$$
28 +exact under $E[z^2] = 1$ for every supported innovation law. Cumulative $h$-day variance $=\sum_{k\le h}\hat\sigma^2_{t+k|t}$. Method: `analytic`. The custom class reproduces arch's 22-step forecasts to relative 1e-10 (`test_custom_garch_equals_arch_loglik_and_forecast`).
29 +
30 +## 7. Implementation
31 +- `src/wp46/models/arch_models.py` (`SPECS["garch11"]`), arch 8.0.0 (`sheppard2025arch`).
32 +- Cross-check (`results/metrics/rugarch_crosscheck.json`, rugarch 1.5-6, fixture `tests/replication/fixture_returns.csv`: T = 3,000 GJR-t data): `garch11_t` log-likelihood −3379.4618 (ours) vs −3379.5028 (rugarch), **difference +0.0410**; parameters agree (α 0.07667 vs 0.07664, β 0.89790 vs 0.89793, ν 6.916 vs 6.923). `garch11_skewt`: −3379.2756 vs −3379.3148, **+0.0392** — note rugarch `sstd` is Fernández–Steel skewing while arch is Hansen's skew-t (`fernandez1998bayesian` vs `hansen1994autoregressive`): the two likelihoods are not the same density, only the orders of magnitude are compared.
33 +- Tests: `tests/replication/test_rugarch.py::test_rugarch_crosscheck` (asserts |Δℓ| < 0.5 and |Δα|, |Δβ| < 5e-3); `tests/unit/test_custom_models.py::test_custom_garch_equals_arch_loglik_and_forecast` (recursion identical to arch's `compute_variance` to 1e-9, forecasts to 1e-10); `tests/integration/test_rolling_driver.py::test_driver_refit_cadence_and_outputs` (driver with `garch11`, normal); `tests/unit/test_distributions.py` (densities equal arch's to 1e-12).
34 +
35 +## 8. Known limitations / failure modes
36 +Near-integrated estimates ($\alpha + \beta \to 1$) on long windows or after breaks (`lamoureux1990persistence`, `hillebrand2005neglecting`) produce boundary optima and the flag `admissible = False` when SLSQP ends with $\alpha + \beta \ge 1$ (constraint is strict only up to solver tolerance); in the pilot, `garch11__t` on SPY had 26/202 refits flagged inadmissible in an earlier version of `pilot_report.json` (the current file holds CL and EURUSD: 207/207 `ok`). ω is poorly identified when persistence is high (small ω, large unconditional variance). Symmetric response to news (no leverage) — the main mis-specification for equities.
37 +
38 +## 9. Computational footprint
39 +Analytic recursion and forecasts; 4–6 parameters. Pilot fit time per refit (`results/metrics/pilot/pilot_report.json`, laptop, 1,000-obs window, median s): normal 0.0042 (CL) / 0.0049 (EURUSD); t 0.0065 / 0.0074. The cheapest estimated dynamic model of the registry.
40 +
41 +## 10. Literature expectations
42 +`hansen2005forecast`: nothing beats GARCH(1,1) on DM/$ (FX), while leverage models do on IBM (equities). `awartani2005predicting`, `brownlees2011practical`: asymmetric models beat GARCH(1,1) on equity indices, more so in crises. `marcucci2005forecasting`: regime models win at short horizons only. The literature matrix (§G) frames GARCH(1,1) as the benchmark that the MCS/SPA machinery tests against; the pre-registered hypotheses H1–H6 (`docs/preregistered_protocol.md`) are all stated relative to this model.
added model_cards/garch12.md +40 −0
@@ -0,0 +1,40 @@
1 +# Model card — `garch12` (GARCH with two variance lags: arch `GARCH(p=1, o=0, q=2)`)
2 +
3 +## 1. Identity
4 +- **model_id:** `garch12` · **family:** classical · **tier:** A · **status:** implemented · **impl:** arch 8.0.0 (`GARCH(p=1, o=0, q=2)`)
5 +- **Label:** GARCH(1,2) · **distributions:** normal, t, ged, skewt
6 +- **Seminal citation:** `bollerslev1986generalized`
7 +- **Other references:** `nelson1992inequality`, `bougerol1992stationarity`, `francq2004maximum`.
8 +- **Naming caveat:** arch's `q = 2` means two lagged conditional variances; in Bollerslev's convention this is GARCH(2,1). See `_discrepancies.md`.
9 +
10 +## 2. Definition
11 +$$\sigma_t^2 = \omega + \alpha_1\varepsilon_{t-1}^2 + \beta_1\sigma_{t-1}^2 + \beta_2\sigma_{t-2}^2,\qquad r_t = \mu + \sigma_t z_t ,$$
12 +pre-sample terms replaced by the backcast. Innovations: normal, t, GED, Hansen skew-t.
13 +
14 +## 3. Parameter space and admissibility
15 +- Parameters: `mu, omega, alpha[1], beta[1], beta[2]` (+ distribution).
16 +- arch bounds/constraints: $\omega \in [10^{-8}v, 10v]$, $\alpha_1, \beta_j \in [0,1]$, $\omega > 0$, $\alpha_1 \ge 0$, $\beta_j \ge 0$, $\alpha_1 + \beta_1 + \beta_2 < 1$. Non-negativity of $\beta_2$ is sufficient, not necessary (`nelson1992inequality`: with p = 1, q = 2 the necessary and sufficient conditions allow $\beta_2 < 0$ under restrictions) — arch's constraint set is the sufficient one.
17 +- Admissibility flag: $\alpha_1 + \beta_1 + \beta_2 < 1$.
18 +
19 +## 4. Nesting and relations
20 +Nests `garch11` ($\beta_2 = 0$). Pre-registered order variant. Not nested in `garch21`.
21 +
22 +## 5. Estimation
23 +As `garch11` (arch SLSQP, grid starting values with the GARCH mass split equally across the two variance lags, backcast initialisation, ≤ 2 restarts). Only point estimates are used in the rolling benchmark.
24 +
25 +## 6. Forecasting
26 +arch `GARCH._analytic_forecast`: $\hat\sigma^2_{t+k|t} = \omega + \alpha_1\widetilde{\varepsilon^2}_{t+k-1} + \beta_1\tilde\sigma^2_{t+k-1} + \beta_2\tilde\sigma^2_{t+k-2}$ (observed values for $s \le t$, forecasts for $s > t$). Cumulative = sum over k ≤ h. Method: `analytic`.
27 +
28 +## 7. Implementation
29 +- `src/wp46/models/arch_models.py` (`SPECS["garch12"]`), arch 8.0.0.
30 +- Cross-check: not in the rugarch fixture; same `garch_recursion` as the validated (1,1) case.
31 +- Tests: none dedicated.
32 +
33 +## 8. Known limitations / failure modes
34 +$\beta_1$ and $\beta_2$ are poorly separated (near-collinear lagged variances) → flat likelihood, boundary optima at $\beta_2 = 0$, more SLSQP non-convergence than (1,1); the roots of $1 - \beta_1 L - \beta_2 L^2$ can be close to the unit circle.
35 +
36 +## 9. Computational footprint
37 +Analytic; one extra parameter vs `garch11`. No pilot timing.
38 +
39 +## 10. Literature expectations
40 +Same as `garch21`: `hansen2005forecast` and the surveys report no systematic out-of-sample gain from higher GARCH orders on daily data. Expectation: statistically indistinguishable from `garch11`, with a higher failure/inadmissibility count (H5).
added model_cards/garch21.md +40 −0
@@ -0,0 +1,40 @@
1 +# Model card — `garch21` (GARCH with two ARCH lags: arch `GARCH(p=2, o=0, q=1)`)
2 +
3 +## 1. Identity
4 +- **model_id:** `garch21` · **family:** classical · **tier:** A · **status:** implemented · **impl:** arch 8.0.0 (`GARCH(p=2, o=0, q=1)`)
5 +- **Label:** GARCH(2,1) · **distributions:** normal, t, ged, skewt
6 +- **Seminal citation:** `bollerslev1986generalized`
7 +- **Other references:** `nelson1992inequality` (positivity for higher orders), `bougerol1992stationarity` (strict stationarity of GARCH(p,q)), `francq2004maximum` (QMLE for general orders), `bollerslev2010glossary`.
8 +- **Naming caveat:** in arch, `p` is the number of lagged squared residuals and `q` the number of lagged variances, so this is $\sigma_t^2 = \omega + \alpha_1\varepsilon_{t-1}^2 + \alpha_2\varepsilon_{t-2}^2 + \beta_1\sigma_{t-1}^2$. In Bollerslev's (1986) convention (p = GARCH lags, q = ARCH lags) the same model is written GARCH(1,2). The registry label follows arch's convention (recorded in `_discrepancies.md`).
9 +
10 +## 2. Definition
11 +$$r_t = \mu + \varepsilon_t,\ \varepsilon_t = \sigma_t z_t,\qquad \sigma_t^2 = \omega + \alpha_1\varepsilon_{t-1}^2 + \alpha_2\varepsilon_{t-2}^2 + \beta_1\sigma_{t-1}^2 ,$$
12 +pre-sample $\varepsilon^2$ and $\sigma^2$ replaced by the backcast. Innovations: normal, t(ν), GED(ν), Hansen skew-t(η, λ).
13 +
14 +## 3. Parameter space and admissibility
15 +- Parameters: `mu, omega, alpha[1], alpha[2], beta[1]` (+ distribution).
16 +- arch bounds/constraints: $\omega \in [10^{-8}v, 10v]$, $\alpha_i, \beta_1 \in [0,1]$, $\omega > 0$, $\alpha_i \ge 0$, $\beta_1 \ge 0$, $\alpha_1 + \alpha_2 + \beta_1 < 1$. The non-negativity constraints are sufficient but not necessary for positivity when p > 1 (`nelson1992inequality`): arch excludes admissible negative-$\alpha_2$ solutions.
17 +- Admissibility flag: $\alpha_1 + \alpha_2 + \beta_1 < 1$.
18 +
19 +## 4. Nesting and relations
20 +Nests `garch11` ($\alpha_2 = 0$) and ARCH(2). Pre-registered order variant (protocol §4: "(1,1) everywhere, plus GARCH(2,1)/(1,2) and ARCH(5)"). Not nested in `garch12`.
21 +
22 +## 5. Estimation
23 +As `garch11` (arch SLSQP, grid starting values with $\alpha$ split equally across the two ARCH lags, backcast initialisation, ≤ 2 restarts; `src/wp46/models/arch_models.py::ArchModel.fit`). Only point estimates are used in the rolling benchmark.
24 +
25 +## 6. Forecasting
26 +arch `GARCH._analytic_forecast`: $\hat\sigma^2_{t+k|t} = \omega + \alpha_1\widetilde{\varepsilon^2}_{t+k-1} + \alpha_2\widetilde{\varepsilon^2}_{t+k-2} + \beta_1\tilde\sigma^2_{t+k-1}$ with observed $\varepsilon^2_s$, $\sigma^2_s$ for $s \le t$ and their forecasts for $s > t$ ($E[z^2] = 1$). Cumulative = sum over k ≤ h. Method: `analytic`.
27 +
28 +## 7. Implementation
29 +- `src/wp46/models/arch_models.py` (`SPECS["garch21"]`), arch 8.0.0.
30 +- Cross-check: not in the rugarch fixture (only the (1,1) orders are cross-checked); the recursion is the same `garch_recursion` validated for (1,1) in `tests/unit/test_custom_models.py`.
31 +- Tests: none dedicated.
32 +
33 +## 8. Known limitations / failure modes
34 +Weak identification of $\alpha_2$ (often at the zero bound → SLSQP boundary convergence); near-integrated persistence as for GARCH(1,1); extra parameter rarely improves out-of-sample (expectation §10).
35 +
36 +## 9. Computational footprint
37 +Analytic; one more parameter than `garch11`. No pilot timing.
38 +
39 +## 10. Literature expectations
40 +`hansen2005forecast` found higher-order GARCH specifications no better than (1,1) out of sample; `bollerslev1986generalized` and the surveys (`bollerslev1992arch`) note that (1,1) is adequate for most daily series. Expectation: no gain over `garch11`, slightly more failures.
added model_cards/histvar22.md +37 −0
@@ -0,0 +1,37 @@
1 +# Model card — `histvar22` (historical variance, 22 days)
2 +
3 +## 1. Identity
4 +- **model_id:** `histvar22` · **family:** baseline · **tier:** A · **status:** implemented · **impl:** ols (parameter-free, `src/wp46/models/baselines.py::HistVar`)
5 +- **Label:** Historical variance, 22 days · **distributions (registry):** empirical (standardised in-window returns for VaR)
6 +- **Seminal citation:** none (registry `ref: null`). Rolling sample variance is the classical "historical volatility" benchmark used e.g. in `poon2003forecasting` (survey) and `hansen2005forecast`.
7 +- Siblings: `histvar66`, `histvar252` (same code, window 66 / 252).
8 +
9 +## 2. Definition
10 +$$\hat\sigma^2_{t+1|t} = \frac{1}{W}\sum_{i=0}^{W-1}\big(r_{t-i} - \bar r_{t,W}\big)^2,\qquad W = 22,\quad \bar r_{t,W} = \tfrac1W\sum_{i=0}^{W-1} r_{t-i},$$
11 +i.e. `np.var(r[-W:], ddof=0)` (population variance around the window mean). No conditional-mean model beyond the window mean reported as `mu`; no innovation law (empirical standardised returns are used for VaR/ES).
12 +
13 +## 3. Parameter space and admissibility
14 +No estimated parameters; always "converged" and "admissible" (`FitResult(..., loglik=nan, converged=True)`). Requires at least 2 observations in the window (filter returns NaN otherwise).
15 +
16 +## 4. Nesting and relations
17 +Equal-weight moving average of squared demeaned returns — the limit of EWMA with uniform weights; distinct from `constvar` only by the window length (22 vs the 1,000-day estimation window) and the absence of a mean equation. `histvar66`/`histvar252` differ only by W.
18 +
19 +## 5. Estimation
20 +None. `HistVar.fit` records the window mean of the 1,000-day estimation window as `mu` (time ≈ 0). `filter` returns the rolling W-day variance of the past returns (day t uses $r_{t-W..t-1}$, first value copied from the second).
21 +
22 +## 6. Forecasting
23 +`forecast`: $\hat\sigma^2_{t+k|t} = \operatorname{var}(r_{t-W+1..t})$ for all $k = 1..22$ (flat); cumulative $h$-day variance $= h \cdot \hat\sigma^2$. Mean forecast = window mean. Method recorded: `direct`.
24 +
25 +## 7. Implementation
26 +- `src/wp46/models/baselines.py::HistVar`; dispatched by `make_model("histvar22")` in `src/wp46/estimation/rolling.py`.
27 +- Cross-check: none needed (deterministic sample moment); not in `results/metrics/rugarch_crosscheck.json`.
28 +- Tests: `histvar22` ran in the pilot (`results/metrics/pilot/pilot_report.json`: 1,032/1,032 origins `ok` on CL); no dedicated unit test.
29 +
30 +## 8. Known limitations / failure modes
31 +Cannot fail numerically. Short window → noisy forecasts; squared-return proxy noise enters directly; no mean reversion at long horizons.
32 +
33 +## 9. Computational footprint
34 +Negligible (pilot median fit time 0.000 s per refit on CL and EURUSD; forecast is one `np.var`).
35 +
36 +## 10. Literature expectations
37 +Hansen & Lunde (`hansen2005forecast`) and `poon2003forecasting` report historical-variance benchmarks dominated by GARCH-type models at the daily horizon; `brownlees2011practical` show the same in calm and crisis periods. Short rolling windows are closer to recent volatility and can be competitive at h = 1 in stressed regimes relative to long windows, but the literature matrix lists all `histvar*` as floors, not contenders.
added model_cards/histvar252.md +37 −0
@@ -0,0 +1,37 @@
1 +# Model card — `histvar252` (historical variance, 252 days)
2 +
3 +## 1. Identity
4 +- **model_id:** `histvar252` · **family:** baseline · **tier:** A · **status:** implemented · **impl:** ols (parameter-free, `src/wp46/models/baselines.py::HistVar`)
5 +- **Label:** Historical variance, 252 days · **distributions (registry):** empirical (standardised in-window returns for VaR)
6 +- **Seminal citation:** none (registry `ref: null`). Rolling sample variance is the classical "historical volatility" benchmark used e.g. in `poon2003forecasting` (survey) and `hansen2005forecast`.
7 +- Siblings: `histvar66`, `histvar252` (same code, window 22 / 66).
8 +
9 +## 2. Definition
10 +$$\hat\sigma^2_{t+1|t} = \frac{1}{W}\sum_{i=0}^{W-1}\big(r_{t-i} - \bar r_{t,W}\big)^2,\qquad W = 252,\quad \bar r_{t,W} = \tfrac1W\sum_{i=0}^{W-1} r_{t-i},$$
11 +i.e. `np.var(r[-W:], ddof=0)` (population variance around the window mean). No conditional-mean model beyond the window mean reported as `mu`; no innovation law (empirical standardised returns are used for VaR/ES).
12 +
13 +## 3. Parameter space and admissibility
14 +No estimated parameters; always "converged" and "admissible" (`FitResult(..., loglik=nan, converged=True)`). Requires at least 2 observations in the window (filter returns NaN otherwise).
15 +
16 +## 4. Nesting and relations
17 +Equal-weight moving average of squared demeaned returns — the limit of EWMA with uniform weights; distinct from `constvar` only by the window length (252 vs the 1,000-day estimation window) and the absence of a mean equation. `histvar22`/`histvar66` differ only by W.
18 +
19 +## 5. Estimation
20 +None. `HistVar.fit` records the window mean of the 1,000-day estimation window as `mu` (time ≈ 0). `filter` returns the rolling W-day variance of the past returns (day t uses $r_{t-W..t-1}$, first value copied from the second).
21 +
22 +## 6. Forecasting
23 +`forecast`: $\hat\sigma^2_{t+k|t} = \operatorname{var}(r_{t-W+1..t})$ for all $k = 1..22$ (flat); cumulative $h$-day variance $= h \cdot \hat\sigma^2$. Mean forecast = window mean. Method recorded: `direct`.
24 +
25 +## 7. Implementation
26 +- `src/wp46/models/baselines.py::HistVar`; dispatched by `make_model("histvar252")` in `src/wp46/estimation/rolling.py`.
27 +- Cross-check: none needed (deterministic sample moment); not in `results/metrics/rugarch_crosscheck.json`.
28 +- Tests: not run in the pilot (only `histvar22` was); no dedicated unit test.
29 +
30 +## 8. Known limitations / failure modes
31 +Cannot fail numerically. One-year window: smooth but slow; close to `constvar` in spirit (window 252 vs 1,000); no reaction to recent shocks beyond the equal weight 1/252.
32 +
33 +## 9. Computational footprint
34 +Negligible (same code as `histvar22`, whose pilot median fit time is 0.000 s per refit).
35 +
36 +## 10. Literature expectations
37 +Hansen & Lunde (`hansen2005forecast`) and `poon2003forecasting` report historical-variance benchmarks dominated by GARCH-type models at the daily horizon; `brownlees2011practical` show the same in calm and crisis periods. Long windows are expected to be the weakest at h = 1 and relatively better at h = 22 (`christoffersen2000how` on horizon decay); the literature matrix lists all `histvar*` as floors.
added model_cards/histvar66.md +37 −0
@@ -0,0 +1,37 @@
1 +# Model card — `histvar66` (historical variance, 66 days)
2 +
3 +## 1. Identity
4 +- **model_id:** `histvar66` · **family:** baseline · **tier:** A · **status:** implemented · **impl:** ols (parameter-free, `src/wp46/models/baselines.py::HistVar`)
5 +- **Label:** Historical variance, 66 days · **distributions (registry):** empirical (standardised in-window returns for VaR)
6 +- **Seminal citation:** none (registry `ref: null`). Rolling sample variance is the classical "historical volatility" benchmark used e.g. in `poon2003forecasting` (survey) and `hansen2005forecast`.
7 +- Siblings: `histvar66`, `histvar252` (same code, window 22 / 252).
8 +
9 +## 2. Definition
10 +$$\hat\sigma^2_{t+1|t} = \frac{1}{W}\sum_{i=0}^{W-1}\big(r_{t-i} - \bar r_{t,W}\big)^2,\qquad W = 66,\quad \bar r_{t,W} = \tfrac1W\sum_{i=0}^{W-1} r_{t-i},$$
11 +i.e. `np.var(r[-W:], ddof=0)` (population variance around the window mean). No conditional-mean model beyond the window mean reported as `mu`; no innovation law (empirical standardised returns are used for VaR/ES).
12 +
13 +## 3. Parameter space and admissibility
14 +No estimated parameters; always "converged" and "admissible" (`FitResult(..., loglik=nan, converged=True)`). Requires at least 2 observations in the window (filter returns NaN otherwise).
15 +
16 +## 4. Nesting and relations
17 +Equal-weight moving average of squared demeaned returns — the limit of EWMA with uniform weights; distinct from `constvar` only by the window length (66 vs the 1,000-day estimation window) and the absence of a mean equation. `histvar22`/`histvar252` differ only by W.
18 +
19 +## 5. Estimation
20 +None. `HistVar.fit` records the window mean of the 1,000-day estimation window as `mu` (time ≈ 0). `filter` returns the rolling W-day variance of the past returns (day t uses $r_{t-W..t-1}$, first value copied from the second).
21 +
22 +## 6. Forecasting
23 +`forecast`: $\hat\sigma^2_{t+k|t} = \operatorname{var}(r_{t-W+1..t})$ for all $k = 1..22$ (flat); cumulative $h$-day variance $= h \cdot \hat\sigma^2$. Mean forecast = window mean. Method recorded: `direct`.
24 +
25 +## 7. Implementation
26 +- `src/wp46/models/baselines.py::HistVar`; dispatched by `make_model("histvar66")` in `src/wp46/estimation/rolling.py`.
27 +- Cross-check: none needed (deterministic sample moment); not in `results/metrics/rugarch_crosscheck.json`.
28 +- Tests: not run in the pilot (only `histvar22` was); no dedicated unit test.
29 +
30 +## 8. Known limitations / failure modes
31 +Cannot fail numerically. Intermediate window (≈ one quarter): less noisy than 22 days, slower to react; no mean reversion at long horizons.
32 +
33 +## 9. Computational footprint
34 +Negligible (same code as `histvar22`, whose pilot median fit time is 0.000 s per refit).
35 +
36 +## 10. Literature expectations
37 +Hansen & Lunde (`hansen2005forecast`) and `poon2003forecasting` report historical-variance benchmarks dominated by GARCH-type models at the daily horizon; `brownlees2011practical` show the same in calm and crisis periods. The literature matrix lists all `histvar*` as floors, not contenders.
added model_cards/rm2006.md +38 −0
@@ -0,0 +1,38 @@
1 +# Model card — `rm2006` (RiskMetrics 2006)
2 +
3 +## 1. Identity
4 +- **model_id:** `rm2006` · **family:** baseline · **tier:** A · **status:** implemented · **impl:** arch 8.0.0 (`RiskMetrics2006()`, default τ₀ = 1560, τ₁ = 4, k_max = 14, ρ = √2)
5 +- **Label:** RiskMetrics 2006 · **distributions:** normal
6 +- **Seminal citation:** registry gives `zumbach2007riskmetrics`, **which is not a key of `paper/references.bib`**; the verified key is `zumbach2009riskmetrics` (SSRN posting of the 2007 RiskMetrics Group document) — see `_discrepancies.md`.
7 +- **Other references:** `jpmorgan1996riskmetrics` (predecessor), `ding1996modeling` (long-memory motivation).
8 +
9 +## 2. Definition (as implemented by arch)
10 +$$\sigma_t^2 = \sum_{k=1}^{14} w_k\,\sigma_{k,t}^2,\qquad \sigma_{k,t}^2 = \mu_k\,\sigma_{k,t-1}^2 + (1-\mu_k)\,\varepsilon_{t-1}^2,$$
11 +$$\tau_k = \tau_1\rho^{k-1},\quad \mu_k = e^{-1/\tau_k},\quad w_k \propto 1 - \frac{\ln\tau_k}{\ln\tau_0},\ \ \sum_k w_k = 1,\qquad \tau_0 = 1560,\ \tau_1 = 4,\ \rho = \sqrt2 .$$
12 +Each component is initialised with its own backcast (exponentially weighted mean of the first squared residuals, weights $\mu_k^i$ up to the lag where $\mu_k^i < 0.01$). Mean: $r_t = \mu + \varepsilon_t$; innovations N(0,1).
13 +
14 +## 3. Parameter space and admissibility
15 +No estimated volatility parameter (only `mu`). The multi-scale kernel has unit total persistence ($\sum_k w_k[\mu_k + (1-\mu_k)] = 1$): integrated, no unconditional variance. Admissibility flag always `True`.
16 +
17 +## 4. Nesting and relations
18 +Weighted sum of 14 EWMA (IGARCH(1,1), ω = 0) components with decays from τ = 4 days to τ = 4·√2¹³ ≈ 362 days; `ewma094` is the single-component special case (λ = 0.94 ↔ τ ≈ 16.2 days). Long-memory-like kernel without estimation (literature matrix §a.1), related in spirit to FIGARCH/HAR hyperbolic weights.
19 +
20 +## 5. Estimation
21 +arch `ConstantMean` + `RiskMetrics2006()` + `Normal`; SLSQP over μ only (`fit(disp="off", options={"maxiter": 1000})`, ≤ 2 restarts, see `ArchModel.fit`); per-component backcasts as in §2. Only point estimates are used; filtering with `am.fix(params)`.
22 +
23 +## 6. Forecasting
24 +arch `_analytic_forecast` for RiskMetrics2006 repeats the one-step forecast at all horizons: $\hat\sigma^2_{t+k|t} = \hat\sigma^2_{t+1|t} = \sum_k w_k\,\sigma^2_{k,t+1}$, $k = 1..22$ (flat); cumulative $h$-day variance $= h\,\hat\sigma^2_{t+1|t}$. Method: `analytic`. Note: the original RM2006 methodology (`zumbach2009riskmetrics`) derives horizon-dependent forecasts from the multi-scale kernel; arch 8.0.0 does not implement them (recorded in `_discrepancies.md`).
25 +
26 +## 7. Implementation
27 +- `src/wp46/models/arch_models.py` (`SPECS["rm2006"]`, `fixed=True`), arch 8.0.0 (`sheppard2025arch`).
28 +- Cross-check: none (not in the rugarch fixture; rugarch has no RM2006).
29 +- Tests: none dedicated; not in the pilot.
30 +
31 +## 8. Known limitations / failure modes
32 +No estimation risk, but fixed weights calibrated by RiskMetrics Group on pre-2007 data; integrated (flat) term structure in this implementation; sensitive to the backcast in short samples (components with τ up to 362 days need long histories — the 1,000-day window covers ≈ 2.8 τ_max).
33 +
34 +## 9. Computational footprint
35 +14 EWMA recursions per likelihood evaluation, one optimised parameter; analytic forecasts. Cheap. No pilot timing.
36 +
37 +## 10. Literature expectations
38 +Not covered by `hansen2005forecast` (2005 predates RM2006). The literature matrix lists RM2006 as an optional multi-horizon baseline whose long-memory kernel should behave like a non-estimated FIGARCH/HAR; expectation: competitive with GARCH(1,1) at h = 1, with the flat term structure limiting h = 22 performance.
added results/metrics/analysis_pilot_validation/per_symbol/SPY.json +2667 −0
Large diff (2,668 lines) is not loaded by default to keep the page fast.Load diff
added results/metrics/analysis_pilot_validation/pooled.csv +55 −0
Large diff (55 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/pooled_pilot_validation.csv +55 −0
Large diff (55 lines) is not loaded by default to keep the page fast.Load diff
added scripts/analyze.py +288 −0
Large diff (288 lines) is not loaded by default to keep the page fast.Load diff
modified scripts/compute_losses.py +10 −6
Large diff (34 lines) is not loaded by default to keep the page fast.Load diff
modified src/wp46/scoring/evaluate.py +54 −0
Large diff (58 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme