F4-conf-02 — Are the six assertions supported by their cited sources?
Question
For each of six assertions in data/assertions.json, decide from the cited extract alone whether it is supported, not_supported or not_assessable, giving one locator and a one-sentence reason (keys verdicts, locators, reasons). A1–A5 cite Armendariz (2026), arXiv:2609.27732v1 (drug consumption rooms, Switzerland); A6 cites Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1 (Rhode Island RESEA trial).
Data and exclusions
Four input files were read (n_input_rows): assertions.json (6 items), README.md (14 lines), arxiv-2609.27732v1.txt (134 lines), arxiv-2606.14621v1.txt (58 lines). Their SHA-256 hashes match the README and bundle (out/state/data_audit.md). No assertion was excluded (out/exclusions.csv: 6 in, 0 removed), so n_analysed = 6. The only evidence used was the extracts. I found seven issues inside the sources and recorded them in data_issues. They are: a Table 6 vs Table A19 standard-error mismatch for the same 5 km CS-DR estimate (0.210 vs 0.218); 608 vs 640 control regions; the abstract's "35%" fall in fatal hospitalisations vs −0.354 log points (≈ −29.8%); narrative text calling the HIV estimates "close to zero"; a 0.05 pp rounding gap in the 3 km "% change"; an unsigned confidence interval in the RESEA text; and a rendering defect ("p ¡ 0.001"). None of these decides a verdict.
Method
out/analysis.py finds the relevant table rows by regex, reads the printed cells, prints the source lines it uses, and applies the task conventions. Percentage change is 100·(exp(b) − 1). A 95% CI is b ± 1.96·SE, with 2.0 as a sensitivity check. A relative change is ATE divided by the control mean. Each verdict rule is written explicitly in the script. A5 is checked by searching the cited extract for any sex or gender term (none were found). For a simpler comparison, a naive baseline asks only whether every number in an assertion appears verbatim in the source (baseline_numeric_verbatim_match). This baseline misses the sign error in A2 and the missing subgroup in A5, so it is not used for any verdict.
Results
(verdicts; counts n_supported = 2, n_not_supported = 3, n_not_assessable = 1)
| id | verdict | locator | decisive element |
|---|---|---|---|
| A1 | supported | Table 6, row "ATT", col "5 km" | Table 6 prints −0.431** with SE (0.210) and −35.0%. exp(−0.431) − 1 = −35.0% (computation_details.A1) |
| A2 | not_supported | Table 7, row "HIV", col "1 km" | The printed ATT is +0.063 (SE 0.801), so the sign is positive, not negative |
| A3 | not_supported | Table 6, row "% change", col "3 km" | −0.283 log points is a −24.7% change (computed −24.6%), not −28.3%. The assertion treats log points as percent |
| A4 | not_supported | Table 7, row "Hepatitis", col "5 km" | −0.406 ± 1.96·0.250 = [−0.896, 0.084], which includes zero (the estimate has no star). The asserted interval is ±1 SE |
| A5 | not_assessable | not reported in arxiv-2609.27732v1 | The extract has no estimates by sex |
| A6 | supported | RESEA Table 2, row "Weeks on UI", col "ATE" | Table 2 prints −1.99 weeks, CI [−2.24, −1.75], and a control mean of 17.3. −1.99/17.3 = −11.5% (computation_details.A6) |
The full one-sentence reasons are in reasons.
Uncertainty
The verdicts are categorical and involve no sampling randomness (random_seed = null). The places where a convention matters are listed below.
- A1, critical value: z = −0.431/0.210 = −2.05 (
computation_details.A1.z), which is significant at 5% with either 1.96 or 2.0, and the source prints**. If the Table A19 SE of 0.218 were used, z = −1.977 (computation_details.A1.z_with_A19_se). That is still significant at 1.96, but just short of 2.0. A1 itself quotes SE 0.210 from Table 6, and the source's own stars mark the estimate as significant at 5%, so the verdict stands. It would only become debatable if one replaced the source's SE with the appendix SE and used z = 2.0. - A1 and A3, log-to-percent conversion: exp(−0.431) − 1 = −35.01%, which matches the assertion. For A3, the 28.3% figure equals the log-point coefficient, while the exact conversion gives −24.6% to −24.7%. No reasonable conversion yields 28.3%, so A3 stays not_supported.
- A4, critical value: the CI includes zero with 1.96 ([−0.896, 0.084]) and with 2.0 ([−0.906, 0.094]) (
computation_details.A4). The asserted [−0.66, −0.16] matches only a ±1·SE band. The verdict is the same under either convention. - A6: the rounding of 11.50% does not affect the verdict. No convention is involved.
- A2 and A5 do not depend on any convention.
Limitations and open issues
- The verdicts rely only on the extracts. Omitted sections of the Swiss paper might contain sex-specific results, but by the task rules A5 must still be not_assessable, not refuted.
- A1 rests on Table 6. The inconsistent SE in Table A19 for the same specification is an internal inconsistency in the source (
data_issues), and I cannot resolve it from the extract. - The verdict rules in
analysis.pyencode my reading of the tables. They check the printed values programmatically, but the classification of each element (sign, magnitude, interval, significance, population) was my judgement. - The task is complete (
feasibility= "complete").