Data audit (P1)
All SHA-256 values match data/README.md and bundle.json.
| file | SHA-256 | lines (wc -l) | content |
|---|---|---|---|
| data/README.md | c7d07d4b…0128 | (computed by analysis.py) | provenance table |
| data/studies.json | 124d1f3e…be1 | 5 items | id, source, short, extract |
| sources/arxiv-2410.12944v3.txt | d5270b44…803b | 112 | Paradis et al. 2024 |
| sources/arxiv-2507.09089v2.txt | b6af8a2b…ed2d | 132 | Becker et al. 2025 |
| sources/arxiv-2411.01067v2.txt | 46a1ee4e…053a | 109 | Bono & Xu 2024 |
| sources/arxiv-2511.13860v1.txt | 8fb2520f…dfa5 | 162 | Bono 2025 |
| sources/arxiv-2304.09339v1.txt | 1db48b1c…6e | 108 | Kreitmeir & Raschky 2023 |
"Rows/columns": the inputs are text extracts, not tabular data; the unit of analysis is the study (5 items, 0 duplicates, no missing ids).
Per-source findings / anomalies
- S1 (Paradis) Table III line 76-78: ExpCon Model 2 = -0.24. (SE 0.14); Sec VI-C line 58: 95% CI [-0.51, 0.03], p=0.086. Consistency: -0.24±1.96·0.14 = [-0.514, 0.034]; z=1.714 → p≈0.087. CI-implied SE = 0.54/3.92 = 0.1378 (vs printed 0.14, rounding). Anomalies: (a) line 48 Mean LogToT AI=4.6 vs no-AI=4.3 contradicts AI faster (96 vs 114 min) — apparently swapped/typo; not used. (b) Table III says n=96, VI-B gives N=47+46=93 for the time comparison. (c) Table III row labels rendered at end of lines ("R^{2} … Adj.", "p value") — rendering artefact. (d) Abstract "significantly shortened" refers to the t-test (p=.038); the extracted covariate-adjusted Model 2 is not significant.
- S2 (Becker/METR) line 34/62: "+19%" more time (regression estimate, S = exp(β)−1). No SE/CI printed in the text; the CI is only in Figure 1 (omitted per header line 8). Raw ratio 34% (line 72) is a different estimator, not used.
- S3 (Bono & Xu) Table 3 line 55: All Content | 53.84 | 44.29 | 63.10 | -29.79%***. Uncertainty only by stars (editorial note: *** 0<p<=0.01). Check: 44.29/63.10−1 = −29.81% vs printed −29.79% (rounding). Header row lacks a leading label cell (rendering). Durations are holding accuracy constant (predicted control duration at treatment accuracy, Appendix B) — an adjusted ratio.
- S4 (Bono 2025) Table 3 lines 90-96: Full Corpus Aware 50.5 s, Control 56.9 s, % Time Savings 11.1, p=0.471. Check: 1−50.5/56.9 = 11.25% vs printed 11.1. Anomalies in other columns: Malicious/Non-Malicious "time per email (s)" of 1.8–2.8 are implausibly small relative to Full Corpus ≈50 s (unit ambiguity, possibly minutes), and the malicious % (−52.9) does not match 2.8/1.9 (−47.4%). Arm Ns (52/56/59 sum to 167) appear to be pre-filter; 13 responses filtered (3 Aware, 4 Control, 6 Blind). Aware arm combines agent queue ordering + verdict display, compared to Control without AI.
- S5 (Kreitmeir & Raschky) Table 1 panel A col (3): Output Treat×Post −0.278 (0.295), N 18332. Outcome is an activity count measure around a ban — not a completion-time comparison on the same tasks; not randomised. Notes print "** p<0.5" (presumably 0.05; typo). Row label rendered as "\times TreatPost". Date line "August 24, 2026" is a rendering artefact (header line 8).
Information-availability decisions
No input violates the rules per se; the binding constraint is that S2 and S3 uncertainty is not printed in the extract → those two are harmonisable but not poolable. No remembered values substituted.