# Data audit (P1)

All SHA-256 values computed with `shasum -a 256` match those listed in data/README.md.

| file | SHA-256 | lines (wc -l) | content |
|---|---|---|---|
| data/studies.json | 124d1f3e3671e9770b91963bc6b26d63281086ef9112fd711cc4607c89babbe1 | 5 items (fields id, source, short, extract) | task design file |
| data/README.md | c7d07d4bd3e0afd9da948c0b74fd76657084044df7d28c08e4399dc27f3b0128 | 17 | provenance |
| sources/arxiv-2410.12944v3.txt (S1 Paradis) | d5270b44…803b | 112 | text extract, Table III |
| sources/arxiv-2507.09089v2.txt (S2 Becker/METR) | b6af8a2b…ed2d | 132 | text extract |
| sources/arxiv-2411.01067v2.txt (S3 Bono & Xu) | 46a1ee4e…053a | 109 | text extract, Table 3 (minutes) |
| sources/arxiv-2511.13860v1.txt (S4 Bono) | 8fb2520f…a5 | 162 | text extract, Tables 1, 3 (seconds as labelled) |
| sources/arxiv-2304.09339v1.txt (S5 Kreitmeir & Raschky) | 1db48b1c…6e | 108 | text extract, Table 1 |

The inputs are text, not tabular data, so "rows/columns" means lines and the table cells extracted. Nothing is missing and there are no duplicate items (S1–S5 are unique; there are 5 distinct sources).

## Extracted quantities and anomalies
- **S1**: Table III, ExpCon row, Model 2 = `-0.24.` with SE `(0.14)`; Section VI-C gives 95% CI [-0.51, 0.03] and p=0.086; n = 96 (Table III caption).
  - Anomaly A1: Section VI-B gives N=47 (AI) + N=46 (no-AI) = 93, but Table III says n = 96. The difference is not explained in the extract.
  - Anomaly A2: Section VI-B gives "Mean LogToT=4.6" for the AI group and 4.3 for no-AI, yet it says the AI group was faster (96 vs 114 min). The two log means appear to be swapped, or are internally inconsistent (ln 96≈4.56, ln 114≈4.74). They are not used.
  - A3: The SE implied by the CI is (0.03+0.51)/3.92 = 0.1378 and the SE implied by the p-value is 0.1398; the printed SE is 0.14. These agree up to rounding. **Decision:** use the printed SE 0.14 (the direct convention) and report the SE from the CI as a sensitivity check.
  - A4: Table III is collapsed (all four models on one line, per the editorial note), so Model 2 is taken as the second value on the line. The row labels of the Adj. R² and p-value lines are also displaced.
- **S2**: "+19%" ("increases completion time by 19%"; "19% more time on average"). The 95% CI appears only in Figure 1/Figure 15, which are not reproduced, and the extract prints no SE or p-value. Sample: 16 developers, 246 tasks (136 AI-allowed, 110 AI-disallowed).
- **S3**: Table 3, Aggregate Results / All Content: Overall 53.84, Treatment 44.29, Control 63.10, Improvement -29.79%*** (minutes, holding accuracy constant). Uncertainty is given by stars only (*** = 0<p<=0.01 by the paper's definition). N = 181 subjects.
  - A5: 44.29/63.10 − 1 = −29.81%, but −29.79% is printed. That is slightly outside the range that rounding of the printed means can explain (−29.80% to −29.82%). It is negligible (Δθ ≈ 0.0003). **Decision:** use the printed −29.79% as the extract instructs.
  - A6: The paper's star convention is unusual (* means 0.05<p<=0.1).
- **S4**: Table 3, Full Corpus: Aware 50.5 s, Control 56.9 s, % time savings 11.1, p-value 0.471. Arms: Control N=52, Aware N=56, Blind N=59 (167 analysts in total).
  - A7: Ambiguous units. The Malicious and Non-Malicious columns (2.8/1.9, 1.8/2.4 "s") cannot be averaged to the Full Corpus values (50.5/56.9 s) under any unit. This suggests a unit or rendering inconsistency in the source table. Only the Full Corpus column is used, and θ depends only on the ratio, so it is unit-free.
  - A8: 50.5/56.9 − 1 = −11.25% vs the printed 11.1% saving. This is within the range rounding can explain (11.08–11.41%). Use the printed 11.1.
  - A9: "Aware" bundles agent queue ordering with verdict display. Aware vs Control is the AI-vs-no-AI contrast (Blind is not used).
- **S5**: Table 1 panel A, col (3) Output: Treat×Post −0.278 (0.295), N 18332.
  - The outcome is GitHub output counts (an "aggregate Output measure", described as log in 3.1), and the design is a DiD around the withdrawal of access. It is not task completion time → **not harmonisable**.
  - A10: The table note prints "** p<0.5", which is presumably a typo for p<0.05. The row label is rendered as "\times TreatPost".
  - A11: The header for panel B says "Post 05/06/11.04", but the text says the 11th was excluded. This is an internal inconsistency that does not affect panel A.
  - The date line "August 24, 2026" is a rendering artefact (as the extract header notes).
- Stray "p<"/"p=" lines and footnote numbers are rendering artefacts (README).

## Information-availability
No input violates the rules. The one risk is S2's confidence interval, which exists only in an unreproduced figure, so it is treated as unavailable: S2 is not poolable.

## Exclusion ledger
See out/exclusions.csv: 5 assessed → 4 harmonisable (S5 removed) → 2 poolable (S2 and S3 removed, no SE).