RESEARCH_STATUS.md — RAID (UQO WP 40)
Last update: 2026-11-02 (Phases 0–8 passed). Legend: ✅ verified by tests/artifacts · 🧪 implemented, validation pending · 🧭 experimental · ⛔ blocked · ⏳ planned · ✖ not implemented (deliberately).
Phases
| Phase | Gate | Status | Evidence |
|---|---|---|---|
| 0 Audit | scope & feasibility memo | ✅ passed 2026-11-02 | docs/audit.md, docs/capabilities.md, docs/novelty_matrix.md, docs/risks.md, docs/PROGRESS.md |
| 1 Foundation | clean build + tests + docs | ✅ passed 2026-11-02 | ctest 26/26 (314 assertions), pytest 7/7, docs/PROGRESS.md Phase 1 |
| 2 Data | offline synthetic tests + opt-in live tests | ✅ passed 2026-11-02 | offline [findata] 4/4, live [live] 2/2 (60 assertions), docs/operations/findata.md |
| 3 Numerical primitives | accuracy + measured break-even | ✅ passed 2026-11-02 | [metal] 6 cases / 6 411 assertions; experiments/results/bench_primitives_summary.json (Metal wins: fused realized ≥ 39 k elements up to 3.4×, GARCH NLL P=64 up to 7.9×; CPU wins: rolling sums, GARCH P=1, small panels) |
| 4 Econometrics (Tier A) | references + Monte Carlo | ✅ passed 2026-11-02 | R (highfrequency/rugarch) + statsmodels/arch references, tests/simulation Monte Carlo, docs/methods/registry.md |
| 5 Extensions (Tier B) | per-feature specs + tests | ✅ passed 2026-11-02 (VAR/GFEVD/DY/LW only) | tests/reference/test_var_reference.cpp 184 assertions vs statsmodels |
| 6 Benchmarking E1–E7 | every claim traceable | ✅ passed 2026-11-02 | experiments/results/raw/*.jsonl + manifests; key_numbers.json; docs/PROGRESS.md Phase 6 |
| 7 Manuscript | reproducible build, no placeholders | ✅ passed 2026-11-02 | paper/main.pdf 30 p., 0 undefined refs, tables/figures generated from artifacts |
| 8 Independent review | clean-checkout reproduction | ✅ passed 2026-11-02 | clean clone: cpu-only 49/49, Metal 55/55, paper 30 p./0 undefined, tables byte-identical; .gitignore defect found and fixed; docs/RELEASE-CHECKLIST.md |
Methods registry (summary; full entries in docs/methods/registry.md from Phase 4)
| Method | Tier | CPU FP64 | Metal FP32 | Reference software | Status |
|---|---|---|---|---|---|
| returns (log/arith), lag/lead, rolling windows, calendar-aware aggregation | A | ✅ | ⏳ (Phase 3) | numpy; exchange_calendars (calendar exact 2000–2026) | ✅ CPU |
| RV, BPV, RSV±, RQ, jump proxy, TriPV, rCov/rCorr | A | ✅ (numpy truth 1e−12) | ✅ fused kernel 1e−6 vs FP64 | highfrequency 1.0.3 (R comparison Phase 4) | ✅ CPU |
| OLS/WLS/ridge, HC0–3, HAC, cluster, rolling OLS | A | ✅ (statsmodels 1e−9) | ✅ sufficient statistics 1e−6 (k ≤ 8) | statsmodels 0.15, scipy 1.18 | ✅ CPU |
| HAR-RV, log-HAR, HAR-X, HARQ, rolling, batched | A | ✅ (highfrequency 1e−8, statsmodels 1e−9) | ✅ batched sufficient statistics | highfrequency 1.0.3, statsmodels 0.15 | ✅ |
| GARCH(1,1), GJR, EGARCH (N, t), QMLE/Hessian/OPG SE, forecasts | A | ✅ (arch 8.0: loglik 5e−4, params 5 %) | ✅ batched NLL 2e−5 (grids) | arch 8.0 ✅, rugarch 1.5.6 (Phase 4) | ✅ CPU |
| Bootstrap (stationary / circular / iid), Monte Carlo harness | A | ✅ (theory + MC coverage) | ⏳ GPU replicas (E5) | theory (Politis–Romano) | ✅ |
| VAR(p), FEVD, GFEVD, Diebold–Yilmaz (static, rolling), Ledoit–Wolf | B | ✅ (statsmodels 1e−9) | ⏳ (E4: batched covariance inputs) | statsmodels 0.15 | ✅ |
| Realized GARCH, HEAVY, DCC, GARCH-MIDAS, FIGARCH, BVAR/VECM, Kalman, change-point | B | ✖ not implemented (deliberate: no experiment in E1–E7 needs them) | ✖ | — | ✖ |
| Pre-averaging, realized kernels, MSRV, BNS test | C | ✖ (1-min bars) | highfrequency | 🧭 only if validated | |
| Hawkes, order-flow, LOB methods | C | ✖ | ✖ | — | ✖ no admissible data (bars only) |
Blockers
- None at Phase 0. Seven cluster nodes unreachable (m2m16, M2M32, M2M32b, M4M36, m4ma, m4mb, m4mc): E7 reports them as not tested unless they come back.
Headline measured results (laptop M5 Max unless stated; see experiments/results/key_numbers.json)
- Metal vs CPU FP64 accuracy: realized measures ≤ 1.6e−7 relative on 233 k real session segments; bitwise reproducible.
- Fused realized kernel: Metal wins from ≈ 4e4 elements, 2.0–3.4× at 4e6–4e7 (kernel level); 1.07–1.41× at the estimator level on 40 M real returns (Amdahl: FP64 post-processing and conversion).
- Batched GARCH likelihood grids: 5.1× (E3, 888 × 256) to 7.9× (P = 64 microbenchmark); P = 1 → CPU wins.
- Rolling sums: CPU wins everywhere (negative result). Small panels (≤ 4e5 elements): ties or CPU.
- Pipeline (E6): RAID 20 M bars/s vs Polars/DuckDB 14 M, pandas 0.9 M; GPU irrelevant at that stage.
- E7: GPU gains largest on weakest CPUs (M2 4.0×) and smallest on M5 (1.1×).
Next safe action (owner decisions)
Read paper/main.pdf; decide on SSRN deposit and the target journal (JFEC / JFDS / JCF); optionally run the Linux
CPU-only build and pip wheel .; optionally create a least-privilege FinData key for benchmark reruns.