WP40 RAID: Phase 8 — clean-checkout reproduction passed, number audit 32/32, release checklist; status final
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
4 changed files +37 −6
modified
RESEARCH_STATUS.md
+5 −5
@@ -1,6 +1,6 @@ | ||
| 1 | 1 | # RESEARCH_STATUS.md — RAID (UQO WP 40) |
| 2 | 2 | |
| 3 | −Last update: 2026-11-02 (Phase 7 passed; Phase 8 review in progress). Legend: ✅ verified by tests/artifacts · 🧪 implemented, validation pending · | |
| 3 | +Last update: 2026-11-02 (Phases 0–8 passed). Legend: ✅ verified by tests/artifacts · 🧪 implemented, validation pending · | |
| 4 | 4 | 🧭 experimental · ⛔ blocked · ⏳ planned · ✖ not implemented (deliberately). |
| 5 | 5 | |
| 6 | 6 | ## Phases |
@@ -14,7 +14,7 @@ Last update: 2026-11-02 (Phase 7 passed; Phase 8 review in progress). Legend: | ||
| 14 | 14 | | 5 Extensions (Tier B) | per-feature specs + tests | ✅ passed 2026-11-02 (VAR/GFEVD/DY/LW only) | `tests/reference/test_var_reference.cpp` 184 assertions vs statsmodels | |
| 15 | 15 | | 6 Benchmarking E1–E7 | every claim traceable | ✅ passed 2026-11-02 | `experiments/results/raw/*.jsonl` + manifests; `key_numbers.json`; `docs/PROGRESS.md` Phase 6 | |
| 16 | 16 | | 7 Manuscript | reproducible build, no placeholders | ✅ passed 2026-11-02 | `paper/main.pdf` 30 p., 0 undefined refs, tables/figures generated from artifacts | |
| 17 | −| 8 Independent review | clean-checkout reproduction | ⏳ | — | | |
| 17 | +| 8 Independent review | clean-checkout reproduction | ✅ passed 2026-11-02 | clean clone: cpu-only 49/49, Metal 55/55, paper 30 p./0 undefined, tables byte-identical; `.gitignore` defect found and fixed; `docs/RELEASE-CHECKLIST.md` | | |
| 18 | 18 | |
| 19 | 19 | ## Methods registry (summary; full entries in `docs/methods/registry.md` from Phase 4) |
| 20 | 20 | | Method | Tier | CPU FP64 | Metal FP32 | Reference software | Status | |
@@ -43,6 +43,6 @@ Last update: 2026-11-02 (Phase 7 passed; Phase 8 review in progress). Legend: | ||
| 43 | 43 | - Pipeline (E6): RAID 20 M bars/s vs Polars/DuckDB 14 M, pandas 0.9 M; GPU irrelevant at that stage. |
| 44 | 44 | - E7: GPU gains largest on weakest CPUs (M2 4.0×) and smallest on M5 (1.1×). |
| 45 | 45 | |
| 46 | −## Next safe action | |
| 47 | −Phase 8: independent review — clean-checkout build and test (CPU-only preset and Metal preset), numerical audit of the | |
| 48 | −manuscript numbers against `key_numbers.json`, license/secret scan, release checklist. | |
| 46 | +## Next safe action (owner decisions) | |
| 47 | +Read `paper/main.pdf`; decide on SSRN deposit and the target journal (JFEC / JFDS / JCF); optionally run the Linux | |
| 48 | +CPU-only build and `pip wheel .`; optionally create a least-privilege FinData key for benchmark reruns. | |
modified
docs/PROGRESS.md
+13 −0
@@ -197,3 +197,16 @@ load, thermal, seeds, input hashes); summaries and every paper table/figure prod | ||
| 197 | 197 | negotiation) or hand-verified; `latexmk` exit 0, **0 undefined citations/references**; all tables `\input` from |
| 198 | 198 | `paper/tables/` and figures from `paper/figures/`, both generated by `make_paper_assets.py` from tracked artifacts. |
| 199 | 199 | Known cosmetic overfull boxes in two listings; no placeholder, no hand-typed number. |
| 200 | + | |
| 201 | +## Phase 8 — Independent review (2026-11-02, 21:30–22:10 ET) — GATE PASSED | |
| 202 | +- Clean clone of `wp40_uqo.git` → `cmake --preset cpu-only` (Arrow/Metal off) builds, `ctest` 49/49; `macos-release` | |
| 203 | + builds, `ctest` 55/55; `raid estimate realized … --backend metal` runs on the clone (fp32, 5 sessions); `latexmk` | |
| 204 | + exit 0, 30 pages, 0 undefined; `make_paper_assets.py` regenerates the 9 tables byte-identical (the 3 PDF figures | |
| 205 | + differ only by matplotlib's embedded creation timestamp). | |
| 206 | +- **Defect found and fixed by this phase**: the `.gitignore` pattern `data/` silently excluded `include/raid/data/` and | |
| 207 | + `src/data/` from every earlier commit (the working tree built, the repository did not); now `/data/`. The raw | |
| 208 | + benchmark records (`experiments/results/raw/`, 9.6 MB) were also ignored and are now tracked, as the charter requires. | |
| 209 | +- Numerical audit: `scripts/audit_manuscript_numbers.py` — 32/32 quantitative claims of the prose found verbatim from | |
| 210 | + `key_numbers.json`. Secret/data scan: `scripts/check_no_secrets.sh` clean; only synthetic fixtures tracked. | |
| 211 | +- Release checklist: `docs/RELEASE-CHECKLIST.md` (gaps stated: Linux build not executed, pip wheel not built from a | |
| 212 | + clean environment, E1–E6 need a FinData key). Tag `v0.1.0-wp40`. | |
added
docs/RELEASE-CHECKLIST.md
+18 −0
@@ -0,0 +1,18 @@ | ||
| 1 | +# Release checklist (charter §12) — status 2026-11-02 | |
| 2 | + | |
| 3 | +| Criterion | Status | Evidence | | |
| 4 | +|---|---|---| | |
| 5 | +| No secrets in git | ✅ | `scripts/check_no_secrets.sh` (CI step); keys only via env/file; manifests store a SHA-256 fingerprint | | |
| 6 | +| No redistribution of raw FirstRate data | ✅ | `datasets/cache/` and `/data/` ignored; only synthetic fixtures tracked; paper/README report aggregates only | | |
| 7 | +| Tier A methods pass reference tests | ✅ | `tests/reference/*` vs statsmodels 0.15 / arch 8.0 / scipy 1.18 / exchange_calendars 4.13 / highfrequency 1.0.3 / rugarch 1.5.6 (registry tolerances) | | |
| 8 | +| Metal unsupported cases fall back safely | ✅ | `backend::resolve` + `[backend]` test; CPU-only preset builds; E7 bundle on a node without developer tools | | |
| 9 | +| Performance claims reproduced | ✅ (on the laptop and 11 nodes) | raw records + manifests tracked; `scripts/audit_manuscript_numbers.py` 32/32 claims verbatim from `key_numbers.json` | | |
| 10 | +| All paper tables/figures from tracked artifacts | ✅ | `experiments/scripts/make_paper_assets.py` + `experiments/manifests/paper_assets.json` | | |
| 11 | +| Clean environment builds and runs | ✅ after fix | Phase 8 clean clone exposed an over-broad `.gitignore` (`data/` hid the data layer): fixed; cpu-only and Metal presets build, 55 tests pass, manuscript compiles | | |
| 12 | +| Python wheel | 🧪 | extension builds and passes pytest (10 tests) from the build tree; `pip wheel .` via scikit-build-core not yet exercised in a clean environment | | |
| 13 | +| Linux CPU-only build | ⏳ untested | preset exists (`cpu-only`), CI workflow written (`.github/workflows/ci.yml`) but the repository is hosted on spbgit (no GitHub Actions run) | | |
| 14 | +| Versioned release | ✅ | library 0.1.0, manuscript 1.0, `CITATION.cff`, tag `v0.1.0-wp40` | | |
| 15 | +| Energy claims | n/a | none made (no instrument) | | |
| 16 | + | |
| 17 | +Not done / honest gaps: Linux build never executed; Python wheel not built from sdist; E1–E6 require a FinData key | |
| 18 | +(private data); the laptop was shared during the benchmarks (recorded in manifests). | |
modified
scripts/audit_manuscript_numbers.py
+1 −1
@@ -32,7 +32,7 @@ chk("E3: grid cpu ms", E3["grid"]["cpuMT_ms"], "{:.0f}\\,ms on 18 CPU threads") | ||
| 32 | 32 | chk("E3: grid speedup", E3["grid"]["cpuMT_ms"] / E3["grid"]["metal_ms"], "({:.1f}$\\times$)") |
| 33 | 33 | chk("E4: cond N=30", E4["30"]["cond_median"], "{:.0f} ($N = 30$)") |
| 34 | 34 | chk("E4: dy ms N=100", E4["100"]["dy_ms"], "and {:.0f}\\,ms") |
| 35 | −chk("E5: beta bias n500", E5["garch_n500"]["beta"]["bias"], "({:+.3f})".replace("+", "")) | |
| 35 | +chk("E5: beta bias n500", E5["garch_n500"]["beta"]["bias"], "(${:.3f}$)") | |
| 36 | 36 | chk("E5: repro metal vs cpu", E5["repro_realized"]["metal_vs_cpu_max_rel_rv"], "$3.7\\times10^{{-8}}$") |
| 37 | 37 | chk("E6: RAID cpu s", E6["RAID cpu"]["median_s"], "{:.3f}\\,s") |
| 38 | 38 | chk("E6: polars s", E6["polars"]["median_s"], "{:.3f}\\,s") |
| 39 | 39 | |