| 1 |
Clean re-run: delete results.json, re-run analysis.py; SHA-256 identical (3cdddb23…725c) |
PASS |
| 2 |
unverifiable_references equals ids failing >=1 boolean check |
PASS |
| 3 |
n_verified (4) + len(unverifiable) (1) = 5 = n_analysed = number of references |
PASS |
| 4 |
n_input_rows match wc -l (README 17, sources 134/58/107/66/91) and JSON item count 5 |
PASS |
| 5 |
Every evidence string used for a finding decision exists verbatim in its extract (script asserts; 26 strings) |
PASS |
| 6 |
SE rows follow coefficient rows for R4 (script asserts) |
PASS |
| 7 |
Arithmetic: exp(-0.431)-1=-0.3501 vs -35.0%; 0.431/0.210=2.05>1.96 (also 0.431/0.218=1.98); -0.268/0.366=-0.7322 vs -0.732; 0.0962/0.261=0.3686 vs 36.85%/-0.3685 |
PASS (last within rounding) |
| 8 |
Header title equals body title line for every extract (script asserts) |
PASS |
| 9 |
arXiv number in each entry matches source file id (script asserts) |
PASS |
| 10 |
Year rule applied with both first-submission and reproduced-version years (R4: 2025 first, 2026 v3) |
PASS |
| 11 |
Baseline (naive number match) vs full method: baseline finds all cited numbers in all 5 extracts, so it would miss R3's timing misattribution; full method flags R3 (also flagged by title/year) |
consistent |
| 12 |
No data/ file modified (hashes unchanged vs README) |
PASS |
| 13 |
Information rule: only extract text used; no external sources |
PASS (by construction) |