# docs/novelty_matrix.md — what exists, what RAID adds, what is *not* new (Phase 0, 2026-11-02)

Legend — **E** engineering novelty (new implementation strategy, measured), **M** methodological novelty (new estimator
or inference result), **=** equivalent to existing software (RAID must match it), **×** not offered. Claims marked E are
only valid once the corresponding experiment (E1–E7) produces the number; nothing in this table is a result.

| Capability | R highfrequency 1.0.3 | R rugarch 1.5.6 | Python arch 8.0 | statsmodels 0.15 | Polars / DuckDB | RAID (planned) | Novelty class |
|---|---|---|---|---|---|---|---|
| RV, BPV, semivariance, quarticity, rCov from bars | yes (`rCov`, `rBPCov`, `rSVar`, `rQuar`) | × | × | × | by hand (SQL/expr) | CPU FP64 + Metal FP32 batched over thousands of series | **E** (batched GPU, fused rolling) ; **=** definitions |
| Calendar-aware aggregation, early closes, DST, 16:00 print | partial (`aggregateTS`, user calendars) | × | × | × | manual | explicit MarketCalendar from XNYS rules, FinData sessions as truth | **E** (correctness tooling), not M |
| HAR-RV / log-HAR / HAR-X / HARQ | `HARmodel` (HAR-RV, HARQ, HAR-J/CJ) | × | OLS by hand | OLS + HAC | × | batched rolling HAR with HAC/HC inference, GPU sufficient statistics | **E** (batched, GPU), **=** definitions (Corsi 2009; Bollerslev–Patton–Quaedvlieg 2016) |
| OLS/WLS/ridge, HC0–3, HAC (Newey–West), cluster-robust | × | × | OLS + HAC in `arch.unitroot`/`covariance` | yes | × | QR/SVD CPU reference; GPU batched X'X/X'y accumulation | **=** inference; **E** batched accumulation |
| GARCH(1,1), GJR, EGARCH, Gaussian / Student-t | × | yes (`ugarchfit`) | yes (`arch_model`) | × | × | CPU reference optimizer; GPU batched likelihood evaluation across series / parameter proposals | **E** (batched likelihood on Metal), **=** models |
| Realized GARCH, HEAVY, DCC, GARCH-MIDAS | HEAVY (`HEAVYmodel`) | realized GARCH, rmgarch DCC | HEAVY ×, DCC × | × | × | Tier B after Tier A | **=** (if done) |
| Diebold–Yilmaz connectedness, generalized FEVD | × | × | × | VAR + FEVD (orthogonalized; generalized by hand) | × | Tier B batched rolling VAR + GFEVD | **E** (rolling batched), **=** method (WP 6 precedent) |
| Stationary / circular block bootstrap | × | × | yes (`arch.bootstrap`) | × | × | GPU replica generation + batched statistic evaluation | **E** |
| Pre-averaging, realized kernels, multi-scale RV | yes (`rKernelCov`, `rAVGCov`, `rTSCov`) | × | × | × | × | Tier C, only if validated against highfrequency; 1-min bars limit their relevance | **=** at best |
| Arrow/Parquet → estimator zero-copy | × (xts) | × | × (pandas) | × | native columnar | Arrow C++ views over FinData exports feeding CPU/GPU without copies where layout allows | **E** (measured copies / RSS, E6) |
| Python bindings with GIL release, Arrow/NumPy interop | — | — | — | — | — | nanobind | engineering, standard practice |
| Device discovery, CPU fallback, precision policy, provenance in every estimate | — | — | — | — | — | yes | **E** (reproducibility tooling) |

## What the paper can legitimately claim (if the experiments support it)
1. A measured break-even analysis Metal vs multicore CPU for realized measures, rolling regressions, batched GARCH
   likelihoods and bootstrap replicas on Apple Silicon, including shader compilation, synchronization and transfer, with
   explicit CPU-wins regions (RQ1).
2. Evidence that batching across series preserves estimator accuracy and inference (coverage, convergence rates
   identical to the FP64 reference within tolerance) while raising throughput (RQ2).
3. Evidence on copies, peak RSS and end-to-end latency of an Arrow/Parquet-to-estimator pipeline vs
   pandas/Polars/DuckDB baselines (RQ3).
4. A validated, open C++20 library whose Tier A estimators reproduce highfrequency / rugarch / arch / statsmodels to
   documented tolerances (reference tables in the appendix).

## What the paper must not claim
- New econometric estimators or new asymptotic results (none planned; **M = none**).
- Microstructure-noise robustness from 1-minute OHLCV bars (no trades/quotes; realized kernels/pre-averaging on 1-min
  bars are a sampling-frequency exercise, not noise correction at tick level).
- Order-flow or order-book content (not inferable from bars).
- Energy savings unless `powermetrics` (root) or an external meter is actually used; `powermetrics` requires sudo and
  is optional in E7 (documented as "measured" only when run).
- GPU "speedups" without the equivalent CPU multicore implementation and the same convergence tolerance.

## Prior work positioning (verified DOIs in `docs/literature/crossref_resolved.json`)
Realized measures: Andersen, Bollerslev, Diebold & Labys (2003, Econometrica); Barndorff-Nielsen & Shephard (2004, 2006,
JFEC); Barndorff-Nielsen, Kinnebrock & Shephard (2010, OUP volume); Andersen, Bollerslev & Diebold (2007, REStat);
Huang & Tauchen (2005, JFEC); Liu, Patton & Sheppard (2015, J. Econometrics). Noise-robust: Zhang, Mykland &
Aït-Sahalia (2005, JASA); Barndorff-Nielsen, Hansen, Lunde & Shephard (2008, Econometrica); Jacod et al. (2009, SPA).
HAR: Corsi (2009, JFEC); Bollerslev, Patton & Quaedvlieg (2016, J. Econometrics); Patton & Sheppard (2015, REStat).
GARCH: Engle (1982); Bollerslev (1986); Nelson (1991); Glosten, Jagannathan & Runkle (1993); Bollerslev & Wooldridge
(1992, Econometric Reviews); Fiorentini, Calzolari & Panattoni (1996, JAE, analytic derivatives); Hansen & Lunde (2005,
JAE); Hansen, Huang & Shek (2012, JAE, Realized GARCH); Noureldin, Shephard & Sheppard (2012, JAE, HEAVY); Engle (2002,
JBES, DCC); Engle, Ghysels & Sohn (2013, REStat, GARCH-MIDAS). Inference: White (1980); Newey & West (1987); Cameron,
Gelbach & Miller (2011, JBES); Politis & Romano (1994, JASA); Diebold & Mariano (1995/2002, JBES); Hansen, Lunde & Nason
(2003 OBES; 2011 Econometrica MCS — the 2011 DOI is still to be resolved); Patton (2011, J. Econometrics). Connectedness:
Diebold & Yilmaz (2012, IJF; 2014, J. Econometrics); Pesaran & Shin (1998, Econ. Letters). Software: Boudt, Kleen &
Sjørup (2022, JSS); Ghalanos (rugarch, CRAN). Computing: Aldrich, Fernández-Villaverde, Gallant & Rubio-Ramírez (2011,
JEDC, GPU in economics); Kahan (1965, CACM); Blackford et al. (2002, ACM TOMS, BLAS). Systems references still to add
with verified sources: Apache Arrow/Parquet documentation, Apple Metal documentation, nanobind, Eigen.