Data audit (P1)
All SHA-256 hashes recomputed and match data/README.md and bundle.json (rechecked after the workspace incident logged in log.md: data/ unchanged).
| file | SHA-256 (prefix…suffix) | lines (wc -l) | bytes | content |
|---|---|---|---|---|
| data/README.md | c7d07d4b…f0128 | 17 | — | provenance table |
| data/studies.json | 124d1f3e…babbe1 | 12 (5 items) | — | 5 study items id/source/short/extract |
| sources/arxiv-2410.12944v3.txt (S1) | d5270b44…fb803b | 112 | 9014 | Paradis et al. 2024 |
| sources/arxiv-2507.09089v2.txt (S2) | b6af8a2b…0aed2d | 132 | 13302 | Becker et al. 2025 |
| sources/arxiv-2411.01067v2.txt (S3) | 46a1ee4e…053a | 109 | 11502 | Bono & Xu 2024 |
| sources/arxiv-2511.13860v1.txt (S4) | 8fb2520f…dfa5 | 162 | 13272 | Bono 2025 |
| sources/arxiv-2304.09339v1.txt (S5) | 1db48b1c…ec06e | 108 | 11504 | Kreitmeir & Raschky 2023 |
Inputs are text extracts (not tabular data): "schema" = provenance header + text; missing values / duplicates apply only at item level: 5 items, 5 distinct ids, no duplicates, every item has its requested quantity printed.
Per-study extraction and anomalies
S1 Table III, ExpCon row, Model 2: -0.24. with SE (0.14); Section VI-C: 95% CI [-0.51, 0.03], p=0.086. Units: log minutes. Checks: CI-derived SE = 0.54/3.92 = 0.1378 ≈ 0.14 (consistent); Wald p from -0.24/0.14 = 0.0865 ≈ 0.086 (consistent); 1−exp(−0.24)=21.3% ≈ printed 21%.
Anomalies: (a) VI-B prints Mean LogToT = 4.6 for AI and 4.3 for no-AI while saying AI was faster (96 vs 114 min) — internally inconsistent (values appear swapped); not used. (b) Table III caption says n = 96 while VI-B reports N=47 + N=46 = 93 — sample discrepancy; reported as printed. (c) stray "p<"/"p=" lines are rendering artefacts per README.
S2 Section 3 / abstract: "19% more time" (regression-based S = exp(β)−1, D.1–D.3). 136 AI-allowed + 110 disallowed issues, 16 developers, 246 tasks. CI shown only in Figure 1/15 (not reproduced) → no SE obtainable. Raw ratio 34% in footnote 12 (not the headline estimator; not used).
S3 Table 3 "Aggregate Results, All Content": Overall 53.84, Treatment 44.29, Control 63.10, Improvement -29.79%*** (minutes, holding accuracy constant). 181 subjects. Stars (*** 0<p≤0.01) are the only uncertainty info → not poolable. Anomaly: 44.29/63.10 − 1 = −29.81% vs printed −29.79% (rounding-level discrepancy). θ from printed % = ln(0.7021) = −0.35368; from durations −0.35396. Use printed % (TASK convention example uses exactly −29.79%). Star convention non-standard (* = 0.05<p≤0.1).
S4 Table 3 "Full Corpus": Aware 50.5 s, Control 56.9 s, % Time Savings 11.1, p-value 0.471. Arms: Control N=52, Aware N=56 (Blind N=59 not used: no AI references, so not AI-vs-no-AI). 167 analysts total after filtering 13. Anomalies: (a) 50.5/56.9 → 11.25% saving vs printed 11.1 (minor discrepancy). (b) Malicious/Non-Malicious columns (2.8/1.8 and 1.9/2.4 "s") cannot average to 50.5/56.9 s — units in those columns ambiguous/inconsistent; not used. (c) Aware arm also gets an agent-prioritised queue order: comparison is the AI-agent package vs no AI on the same email pool. θ = ln(1 − 0.111) = −0.11766; approximate SE = 0.11766 / z(1−0.471/2) = 0.11766/0.72085 = 0.1632.
S5 Table 1 Panel A col (3) Output: Treat×Post -0.278 (0.295), N 18332. Outcome = output event counts (whether logged is ambiguous in the extract), DiD around withdrawal of access (ban), not randomised, not task completion time → not harmonisable. Anomalies: notes print "** p<0.5" (presumably 0.05); row label rendered "\times TreatPost"; date line "August 24, 2026" is a rendering artefact (per header); panel B text says April 11 excluded but panel label lists 11.04.
Information-availability
No input violates the rules. The METR CI (Figure 1) and Security Copilot SEs are not in the extracts → treated as unavailable (S2, S3 not poolable). No external values used.