SPB Git

Repository Public Research preview

_discrepancies.md 18.6 KB

# Discrepancies noticed between the registry, the code and the literature definitions (model-card pass, 2026-11-09)

Factual observations from reading configs/model_registry.yaml, src/wp46/models/*.py, src/wp46/estimation/rolling.py, arch 8.0.0 internals (.venv/lib/python3.12/site-packages/arch/univariate/{volatility,recursions_python,base,mean,distribution}.py), docs/methodology_decisions.md, docs/literature_matrix.md, results/metrics/rugarch_crosscheck.json, results/metrics/pilot/pilot_report.json and tests/. Nothing here was modified; each item says where it matters.

# A. Registry ↔ references.bib

  1. Three registry bibkeys do not exist in paper/references.bib: zumbach2007riskmetrics (rm2006; verified key zumbach2009riskmetrics), gonzalez1998smooth (stgarch; verified key gonzalezrivera1998smooth), hansen2016realized (realegarch11; verified key hansen2016exponential). The cards cite the verified keys; the registry ref: fields should be corrected (the registry is frozen with the protocol — a documented erratum rather than a silent edit).

# B. Registry / decisions log ↔ code

  1. GJR multi-step and admissibility use κ = ½ for every distribution in the scored implementation. D-09 states "GJR with κ = E[z²1{z<0}]", and the task brief repeats it, but the registry's gjr11 is arch-backed: arch's _analytic_forecast sets the future asymmetric term to 0.5 × σ² (_asym_resids[h+m] = sqrt(0.5 * forecasts)), its optimizer constraint is α + ½γ + β < 1, and ArchModel.fit flags admissibility with 0.5·γ. The κ-aware version exists only in the custom twin gjr11_custom (cross-check/simulation tests, not in the registry). Consequence: for the skew-t (λ ≠ 0) the arch GJR multi-step forecasts and the admissibility flag are approximations (exact weight κ = neg_half_moment, > 0.5 for λ < 0). Either score gjr11_custom for the skew-t or amend D-09.

  2. Simulation forecasts for arch-backed models are unseeded for non-normal innovations. ArchModel.forecast passes rng=np.random.default_rng(seed).standard_normal only when fit.dist == "normal"; otherwise rng=None, and arch falls back to self._distribution.simulate(dp) whose generator was created with seed=None (Distribution.__init__ → default_rng()). Affects h > 1 forecasts of egarch11, aparch11, tarch11, avgarch11 with t/GED/skew-t (D-09/protocol: "fixed seed", forecast_protocol.yaml: seed 46). Custom simulation models (realgarch11) are seeded correctly.

  3. No admissibility flag for tarch11, avgarch11, aparch11 (also constvar, ewma094, rm2006): ArchModel.fit only computes a model-specific flag for garch11/21/12, arch5, gjr11, figarch11, egarch11; the others get admissible = True unconditionally, so only the SLSQP constraints bound them. For APARCH/TARCH the arch constraint α + β < 1 (resp. α + ½γ + β < 1) is not the Ding–Granger–Engle / Zakoïan stationarity condition (which involves E[(|z| − γz)^δ]); inadmissible power-model optima will be recorded as ok.

  4. Registry impl: custom comment says "validated vs arch/rugarch", but three scored custom architectures have no library counterpart and no cross-check: qgarch11, gas_t11, garchx_rv11 (and heavy11, msgarch2 have no library cross-check either; they have simulation-recovery tests). D-11 justifies unsupported status by "a custom implementation without an authoritative cross-check would violate the verification rule" — the same argument applies to these scored models unless the simulation-recovery test is accepted as the authoritative check (then qgarch11, gas_t11, garchx_rv11 lack even that).

  5. MS-GARCH multi-step recursion: MSGARCH2.forecast computes E[ε²_{t+j}|F_t] as the marginal-probability mixture Σ_k P(S_{t+j}=k|F_t)·E[σ²_{k,t+j}] and feeds it to both regime recursions. This treats the future regime indicator and the regime variances as uncorrelated given F_t. The class docstring calls it "the exact recursion (HMP 2004)"; the exactness could not be verified here against haas2004new (the paper's multi-step formula should be checked; if it conditions jointly on the regime path, the code is an approximation).

  6. garchm11 has no test and no cross-check. GARCHM11 is not imported by any test module (tests/unit/test_custom_models.py parametrises GJR11, NAGARCH11, QGARCH11, CGARCH11, IGARCH11, GAST11; realized models separately), and rugarch archm was not included in tests/replication/rugarch_crosscheck.R. CLAUDE.md rule 6 requires unit tests + cross-check + simulation recovery for every accepted implementation.

  7. TGARCH cross-check incomplete: rugarch_crosscheck.R fits fGARCH(TGARCH)-t (rugarch.tgarch11_t, log-lik −3378.2657, converged) but tests/replication/test_rugarch.py never fits ArchModel("tarch11"), so comparison has no tgarch11_t entry. One line in the test would close it.

  8. Realized GARCH cross-check absent: rugarch realGARCH did not converge on the fixture (comparison.realgarch11_n.diff = null); the registry/D-14 claim "every arch-backed and custom model is compared with rugarch" does not hold for realgarch11 (nor for the models of item 5). Only the seeded recovery test supports it.

  9. FIGARCH −1.47 log-likelihood gap vs rugarch (D-14, progress.md open issue) is attributed to truncation/constraints but not resolved; arch's constraints (0 ≤ φ ≤ (1−d)/2, 0 ≤ β ≤ d + φ) are a sufficient set, and arch's recursion uses the backcast for all lags beyond the sample (within a 1,000-day window and 1,000-lag truncation the oldest lags are always backcast).

  10. Order-naming convention: garch21 is arch GARCH(p=2, o=0, q=1) = two ARCH lags + one GARCH lag; garch12 = one ARCH lag + two GARCH lags. In Bollerslev's (1986) (p, q) = (GARCH lags, ARCH lags) convention the labels are swapped (docs/literature_matrix.md §a.1 caveat). The registry labels follow arch; the paper must state the convention.

  11. GAS-t "inverse-information scaling" is implemented in the variance (level) parameterisation, equivalent to Harvey's Beta-t-GARCH; the registry label "GAS-t(1,1) / Beta-t-GARCH" is consistent, but note that the ν → ∞ limit maps to GARCH with β_G = β − α (not β), which matters when comparing persistence across models.

  12. HEAVY estimation is joint, not separate: the docstring says each equation "is estimated separately by quasi-maximum likelihood"; the code maximises the sum of the two QLs in a single L-BFGS-B run. Equivalent at the optimum (variation-free parameters) but the optimizer path, convergence flag and n_iter are joint. Also the RM-equation initial value μ₀ is a 75-day plain mean of RM, not the exponentially weighted backcast used for h₀.

  13. Realized GARCH docstring vs code: the literature matrix says "multi-step by the VAR-type recursion"; the code simulates (2,000 paths, clip ±30 on log σ²). D-09 lists Realized GARCH under simulation — consistent with the code, inconsistent with the matrix wording.

  14. EGARCH centring constant: arch uses √(2/π) = E|z| for the normal in all distributions; for t/GED/skew-t the difference is absorbed by ω (observationally equivalent), so omega is not comparable across distributions and is not Nelson's ω. arch also restricts β ∈ [0, 1) (Nelson: |β| < 1). The cross-check confirms the α/γ label swap relative to rugarch.

  15. Distribution bounds differ between arch and custom models: arch bounds ν ∈ [2.05, 500] (t), ν ∈ [1.01, 500] (GED), η ∈ [2.05, 300], λ ∈ [−1, 1]; custom transforms give ν ∈ (2.05, ∞) (t), ν ∈ (0.3, ∞) for the GED, λ ∈ (−0.99, 0.99). DIST_BOUNDS in distributions.py (GED (0.5, 20)) is imported by custom.py but never used. A GED shape below 1.01 is admissible in custom models but not in arch — minor, but the same distribution has different admissible ranges across implementations.

  16. arch does not rescale but warns: rescale=None (default in ConstantMean) only emits DataScaleWarning when var(ε) ∉ [0.1, 10 000); percent units keep most instruments inside, but low-volatility instruments (e.g. short-maturity Treasury futures, var ≪ 0.1 percent²) will trigger the warning and SLSQP may be ill-conditioned (ω bounds scale with the data, tolerances do not). Warnings are recorded in notes.

  17. Pilot report state: results/metrics/pilot/pilot_report.json currently contains only CL and EURUSD (the last run_pilot.py call overwrote SPY/AAPL); it shows leakage_check_passed: false for CL and true for EURUSD, whereas progress.md records "leakage check passed". The cards only quote the fit timings and status counts from this file.

  18. Task brief vs code on the HEAVY RM equation: the brief says the HEAVY RM equation "uses the multiplicative-error QL" — confirmed (−Σ(log μ_t + RM_t/μ_t)); and that garchx_rv11 "RM dynamics from the HEAVY-RM equation" — confirmed; but engle2002new does not specify the auxiliary RM forecast model, so the GARCH-X multi-step forecasts depend on an auxiliary equation that is the code's choice.

  19. RM2006 forecasts are flat in arch (one-step repeated at all horizons); the original RiskMetrics 2006 methodology (zumbach2009riskmetrics) specifies horizon-dependent forecasts from the multi-scale kernel. Expectation sections at h = 5, 22 should treat rm2006 as "EWMA-like flat forecast with a long-memory one-step kernel".

  20. CGARCH on the cross-check fixture collapses to GARCH(1,1) (log-likelihood −3379.4617 vs −3379.4618 for garch11-t): the fixture is GJR-t data without a component structure, so the cross-check validates the likelihood code path but not the identification of the components; rugarch's csGARCH optimum is 0.30 lower.

  21. MS-GARCH reported loglik may include the identification penalty (50·(ln u₁ − ln u₂) when u₁ > u₂ at the optimum); the fits table will then store a penalised value.

  22. CustomModel.fit start-vector loop: max_restarts = 3 but only three starting vectors are defined and the loop breaks after the second start if it succeeds; n_restarts records the index of the last start tried (0–2), not the number of restarts. Harmless, but the fits-table column semantics differ from the arch path (n_restarts = number of extra fits).

  23. HAR _har_X uses the 22-day average ending at t (inclusive) and the h-step target is the average daily variance over t+1..t+h (then multiplied by h in the forecast) — consistent with Corsi's normalisation; the per-step allocation (even split within 2–5 and 6–22) is a WP46 convention needed only because the driver stores per-step variances; it does not affect the h = 1, 5, 22 cumulative forecasts.

# Extensions (E-01)

Observations from reading src/wp46/models/extensions.py, tests/unit/test_extensions.py, the extension entries of configs/model_registry.yaml and src/wp46/estimation/rolling.py (model-card pass for the seven post-registration architectures, 2026-11-09). Registry keys gonzalezrivera1998smooth and hansen2016exponential are now correct (item A.1 is resolved for stgarch11 and realegarch11; zumbach2007riskmetrics remains). The stubs stgarch.md (id renamed to stgarch11) and the unsupported garch_midas.md, hygarch11.md, fiaparch11.md, fiegarch11.md, realegarch11.md were replaced.

  1. "Exact re-implementation" vs test tolerance. The module docstring states that HYGARCH(amp = 1) and FIAPARCH(δ = 2, γ = 0) are "exact re-implementations of FIGARCH", but the only identity test (test_hygarch_amp1_matches_arch_figarch_likelihood) accepts |Δ log-likelihood| < 5.0 at arch's optimum, and FIAPARCH has only the nesting inequality (fit ≥ arch FIGARCH − 1.0), no evaluation at the FIGARCH point. With the same weights, truncation and backcast the difference should be ≈ 0; the tolerance should be tightened (or the residual gap explained) before the identity is cited as a cross-check. None of the other six extensions has any external check (CLAUDE.md rule 6 and D-11 are not satisfied — consistent with the extension label, but D-11's text still calls these architectures unsupported).

  2. ST-GARCH has an exact analytic multi-step recursion for symmetric laws that the code replaces by simulation. Since $F(e) + F(-e) = 1$, $E[F(\varepsilon)\varepsilon^2 \mid \sigma^2] = \sigma^2/2$ for symmetric $z$, hence $E_t[\sigma^2_{t+k}] = \omega + (\alpha_1 + \alpha_2/2 + \beta)E_t[\sigma^2_{t+k-1}]$ exactly; the docstring's justification ("the transition is nonlinear in eps") does not apply to the normal/t pairs that are scored, and the 2,000-path simulation only adds Monte Carlo noise at k ≥ 2. Also: no location parameter c (the former stub card and part of the literature allow one); γ is in percent⁻¹ (scale-dependent) and unidentified under α₂ = 0; the recursion initialises $\sigma_0^2 = s_0$ whereas garch11_custom uses $\omega + (\alpha+\beta)s_0$, so the GARCH(1,1) nest differs in the first observation.

  3. Beta-t-EGARCH score and parameter mapping. The docstring calls $u_t$ "the score of the t density w.r.t. log sigma2"; it is twice that score (the score w.r.t. $\log\sigma$, Harvey's λ), and $u_t \in [-1, \nu)$ rather than "(−1, ν]". Relative to harvey2008beta (unstandardised $t_\nu$, log-scale λ): β = φ, α = 2κ, γ = 2κ*, ω = 2δ + (1−φ) log(ν/(ν−2)). α > 0 is imposed (the literature leaves κ free); β ∈ (−1, 1) is allowed (the literature uses 0 < φ < 1). Closed-form multi-step moments exist (harvey2013dynamic) but simulation is used. The ν → ∞ limit is not GARCH(1,1), yet the A-05 floor (GARCH(1,1)-t − 50) is applied to this model.

  4. Realized EGARCH clipping asymmetry: in-sample $\log\sigma^2$ is clipped at ±50, the multi-step simulation at ±30 (same inconsistency as realgarch11, item 14). Single realized measure (the paper allows several); joint likelihood excluded from the A-05 floor (correct) and not comparable with Tier A fits.

  5. GARCH-MIDAS deviations from engle2013stock: (a) monthly RV is the sum of squared demeaned returns (window mean), the paper uses raw squared returns; (b) $g_t$ divides $\varepsilon_{t-1}^2$ by $\tau_{t-1}$ (previous day) rather than the paper's $\tau_t$ — identical except on the first trading day of each month; (c) $g_0 = 1$ and the backcast $s_0$ is computed but unused; (d) short-history rules at the window start: months 6–23 use renormalised weights over the available months, months 0–5 use the plain mean of available monthly RVs, and month 0 uses its own contemporaneous RV (within-window look-ahead confined to the first month of in-sample filtering; forecasts, issued ≥ 47 months into the 1,000-day window, are unaffected and test_no_lookahead_extensions does not cover garch_midas); (e) θ > 0 and w > 1 by construction (the paper leaves θ's sign free), so GARCH(1,1) is a limit (θ → 0) where w is unidentified; (f) $\varphi_K = 0$ exactly for w > 1, so 23 of the 24 lags carry weight; (g) τ is frozen over the 22-day horizon. The former registry/D-11 objection ("needs a low-frequency series outside the information set") no longer applies: the long-run component is built from the window's own daily returns (Tier A).

  6. HYGARCH admissibility and identification: the flag checks ω > 0 and the 1,000 truncated weights ≥ −1e-10 (numerical, not Conrad–Haag/Conrad analytic conditions) and ignores Davidson's covariance-stationarity condition amp < 1 (explosive amp > 1 optima are recorded as ok); inadmissible proposals return the 1e10 penalty inside the likelihood (a discontinuous wall for L-BFGS-B); amp = e^θ excludes amp = 0 (GARCH) and makes amp = 1 (FIGARCH) an interior point reached only numerically; the discarded tail mass of the 1,000-lag truncation ($O(1000^{-d})$) is observationally similar to amp < 1 inside a 1,000-day window (amp, d and ω/(1−β) trade off). step_forecast pads lags older than the window with $\sigma_0^2$ (first in-sample variance) whereas the in-sample recursion uses the backcast $s_0$ — a difference only for windows shorter than 1,000 days. arch's sufficient constraints (φ ≤ (1−d)/2, β ≤ d + φ) are not imposed, so HYGARCH(amp = 1) can reach FIGARCH parameter sets that figarch11 cannot.

  7. FIAPARCH: δ ∈ (0.5, 4) and |γ| < 0.99 by construction (literature: δ > 0, |γ| < 1); d, φ, β ∈ (0, 1) exclude the APARCH (d = 0) limit; the pre-sample value $s_0^{\delta/2}$ assumes $E(|z| - \gamma z)^\delta = 1$ (arch's APARCH convention, not exact); no stationarity flag; the FIGARCH identity is not tested at the (δ = 2, γ = 0) point (item 25).

  8. FIEGARCH: (a) rec_fiegarch receives a backcast argument ls0 that it never uses — pre-sample $g = 0$, so $\log\sigma_0^2 = \omega$ (the unconditional mean), contrary to the base-class "backcast" contract and the fit.aux["backcast"] plumbing; (b) $d = 0.499,\sigma(\theta) \in (0, 0.499)$ by construction, so the admissibility flag (d < 0.5, |φ|, |ψ| < 1) is vacuous, d ≥ ½ cannot be estimated (the literature only requires d < ½ for covariance stationarity of $\log\sigma^2$, not at estimation) and the d = 0 EGARCH point is excluded; (c) γ = e^θ > 0 is imposed (not in bollerslev1996modeling); (d) $E|z|$ is the exact value for the fitted law (unlike arch's EGARCH, item 15), so omega is not comparable with egarch11's; (e) $\log\sigma^2$ clipped at ±50 in-sample and in simulation; (f) no test of fiegarch_weights against an ARFIMA reference.

  9. Registry dists narrower than the class supports: stgarch11, garch_midas, hygarch11, fiaparch11, fiegarch11 inherit supports = (normal, t, ged, skewt) but the registry lists [normal, t], so GED/skew-t pairs are neither scored nor reported as unsupported; betategarch11 (t) and realegarch11 (normal, t) are consistent. For the fractional models the admissibility/persistence logic does not depend on the law, so the restriction is a scope decision, not a technical one; for stgarch11 the persistence condition is only exact for symmetric laws.

  10. Rolling driver special case: rolling.py passes the calendar months only when job.model_id == "garch_midas" (string match); any future MIDAS variant must repeat the special case, and the unit tests use 21-day blocks instead of calendar months (the two calendars give different monthly RV series and therefore different fits).

  11. D-11 wording is stale: docs/methodology_decisions.md D-11 still lists FIEGARCH, FIAPARCH, HYGARCH, smooth-transition GARCH, GARCH-MIDAS and Realized EGARCH as unsupported/not scored; the registry now labels them extension and scores them (progress.md, E-01). A decision entry recording the post-registration status change (and its separate reporting) is missing from the decisions log.

Files main
Menu

Theme