F4-conf-02 — Citation-support check of six assertions
Question
For each of six literature-review assertions, decide from the cited extract alone whether it is supported, not_supported or not_assessable, with a locator and a one-sentence reason.
Data and exclusions
data/assertions.json(6 assertions),data/README.md(14 lines),data/sources/arxiv-2609.27732v1.txt(Armendariz 2026, 134 lines),data/sources/arxiv-2606.14621v1.txt(Li, Pearson-Merkowitz & Yokum 2026, 58 lines).- Only the extracts were used as evidence; nothing was excluded. No outside knowledge of the papers was used.
Method
out/analysis.py parses the relevant table rows from the extracts (printing each source line it relies on), checks every element of each assertion (sign, magnitude, SE/CI, significance, population), and applies the task conventions: % change = 100·(exp(b) − 1); 95% CI = b ± 1.96·SE (± 2.0 checked as an alternative). A claim was judged not_assessable only when the extract contains no estimate for the relevant population (checked by searching the extract for sex/gender terms: 0 hits).
Results
| id | verdict | locator | decisive element |
|---|---|---|---|
| A1 | supported | arxiv-2609.27732v1 | Table 6 | row "ATT" | column "5 km" | ATT −0.431** (0.210), "% change" −35.0% (100·(exp(−0.431)−1) = −35.01%), text: significant at 5% |
| A2 | not_supported | arxiv-2609.27732v1 | Table 7 | row "HIV" | column "1 km" | printed coefficient is +0.063 (SE 0.801), not −0.063; no fall is shown |
| A3 | not_supported | arxiv-2609.27732v1 | Table 6 | row "% change" | column "3 km" | −0.283 log points = −24.7% (printed; 100·(exp(−0.283)−1) = −24.65%), not −28.3%; also not significant (SE 0.279) |
| A4 | not_supported | arxiv-2609.27732v1 | Table 7 | row "Hepatitis" | column "5 km" | −0.406 ± 1.96·0.250 = [−0.896, 0.084] includes zero; the asserted [−0.66, −0.16] is ±1 SE |
| A5 | not_assessable | not reported in arxiv-2609.27732v1 | extract reports no estimates by sex |
| A6 | supported | arxiv-2606.14621v1 | Table 2 | row "Weeks on UI" | column "Average treatment effect estimate" | ATE −1.99 weeks, 95% CI [−2.24, −1.75], control mean 17.3; 1.99/17.3 = 11.50% (text also says 11.5%) |
Counts: 2 supported, 3 not_supported, 1 not_assessable.
Uncertainty
- A1 relies on the log-to-percent conversion (printed −35.0% agrees with 100·(exp(b)−1)) and on the significance stars. Under the alternative SE printed in Table A19 for the same estimate (0.218), |t| = 1.98 > 1.96, still significant at 5%; verdict unchanged. The assertion quotes SE 0.210, which matches Table 6.
- A3 depends on the log-to-percent convention: under the exp conversion the change is −24.7%. Only a naive "log points = percent" reading gives 28.3%, which is not the source's own conversion (the source prints −24.7%), so the verdict is robust.
- A4 depends on the critical value: with 1.96 the CI is [−0.896, 0.084]; with 2.0 it is [−0.906, 0.094]. Both include zero, and the table prints no star (p ≥ 0.10). Verdict unchanged under any conventional critical value.
- A2, A5, A6 do not depend on conventions (A6's 11.5% is simple arithmetic, also printed in the source text).
Limitations and open issues
- Internal inconsistencies in the Armendariz extract (recorded in
data_issues, none decisive): SE of the 5 km CS-DR ATT is 0.210 in Table 6 but 0.218 in Table A19; control regions 608 (Tables 6–7) vs 640 (A19 note); abstract says fatal hospitalisations fall 35% whereas Table 7 (−0.354) implies −29.8% and Section 6.1 says ~30%; the text calls HIV estimates "close to zero" though the 5 km HIV ATT is −0.340. - In Li et al., the results text reports the weeks-on-UI CI as positive magnitudes [1.75, 2.24] while Table 2 prints [−2.24, −1.75]; treated as equivalent for a reduction.
- A5's
not_assessableverdict refers only to the extract; the omitted sections of the paper might contain sex-specific results, but those are inadmissible here.