SPB Git

spb/wp29_uqo

Public

# F4-conf-02 — Are the six assertions supported by their cited sources?

# Question

For each of six literature-review assertions (A1–A5 cite Armendariz 2026, arXiv:2609.27732v1; A6 cites Li, Pearson-Merkowitz & Yokum 2026, arXiv:2606.14621v1), decide from the cited extract alone whether it is supported, not_supported or not_assessable, with one locator and a one-sentence reason.

# Data and exclusions

Inputs read: data/assertions.json (6 items), data/README.md (14 lines), data/sources/arxiv-2609.27732v1.txt (134 lines), data/sources/arxiv-2606.14621v1.txt (58 lines). Only the extracts were used as evidence; nothing was excluded. Data issues found (all also listed in results.json):

  • DCR paper: Table 6 gives the 5 km CS-DR ATT −0.431** an SE of 0.210, but Appendix Table A19 gives the same estimate an SE of 0.218.
  • DCR paper: the Table A19 note says there are 640 control regions, but the tables report 608.
  • DCR paper: the abstract says fatal hospitalisations fall by 35%. The Table 7 fatal ATT of −0.354 implies −29.8%, and Section 6.1 says "approximately 30%".
  • DCR paper: two of the percentages in the Table A19 "% change" row are off by 0.1 percentage point from 100·(exp(b)−1). This is probably rounding.
  • DCR paper: the text calls the HIV estimates "close to zero", but the 5 km HIV ATT is −0.340 (not significant).
  • RESEA paper: the results text gives the CIs as unsigned magnitudes ([1.75, 2.24]), while Table 2 gives them signed ([−2.24, −1.75]). The text also contains "p ¡ 0.001", which is a rendering error for "p < 0.001". None of these issues changes a verdict.

# Method

out/analysis.py parses the printed table cells, prints the source lines it uses, and applies the task conventions:

  • % change = 100·(exp(b)−1);
  • 95% CI = b ± 1.96·SE, with 2.0 also checked;
  • an assertion is supported only if every element holds. For A5 the script searches the extract for any sex or gender subgroup (men/women/male/female/sex/gender) and finds none.

# Results

id verdict locator decisive element
A1 supported arxiv-2609.27732v1 | Table 6 | row "ATT" | column "5 km" Printed −0.431** (0.210), p<0.05, % change −35.0%; 100·(exp(−0.431)−1) = −35.01%
A2 not_supported arxiv-2609.27732v1 | Table 7 | row "HIV" | column "1 km" Printed ATT is +0.063 (SE 0.801), not −0.063, so the sign is wrong
A3 not_supported arxiv-2609.27732v1 | Table 6 | row "% change" | column "3 km" −0.283 log points = −24.65% (printed −24.7%), not 28.3%; the assertion treats log points as percent
A4 not_supported arxiv-2609.27732v1 | Table 7 | row "Hepatitis" | column "5 km" −0.406 ± 1.96·0.250 = [−0.896, 0.084], which includes 0 (no stars); the asserted [−0.66, −0.16] is ±1 SE
A5 not_assessable not reported in arxiv-2609.27732v1 The extract has no estimates by sex
A6 supported arxiv-2606.14621v1 | Table 2 | row "Weeks on UI" | column "Average treatment effect estimate" −1.99 weeks, 95% CI [−2.24, −1.75], control mean 17.3; 1.99/17.3 = 11.5% (stated in the text too)

Counts: 2 supported, 3 not supported, 1 not assessable.

# Uncertainty

No sampling uncertainty is involved: the verdicts are deterministic readings of the printed values. Some verdicts depend on a convention:

  • A1:
    • The 35.0% figure relies on the log-to-percent conversion. 100·(exp(−0.431)−1) = −35.01%, and the source prints the same −35.0%.
    • The 5% significance claim relies on the printed stars. As a check, |t| = 2.05 with Table 6's SE but 1.98 with Table A19's SE (0.218), so the claim holds narrowly under the 1.96 critical value. With 2.0 as the critical value it would fail on the A19 SE (1.98 < 2.0). Both tables print ** (p<0.05), so the verdict stays supported.
  • A3: This depends on the conversion. Under exp(b)−1 the effect is −24.65%. The assertion's 28.3% comes only from the cruder approximation that treats log points as percent. The source itself prints −24.7%, and the task requires the exp conversion, so the verdict stays not_supported.
  • A4: This depends on the critical value. With 1.96 the CI is [−0.896, 0.084]; with 2.0 it is [−0.906, 0.094]. Both include zero. The asserted interval matches only a ±1·SE band (a 68% interval), which no reasonable 95% convention produces. The estimate is also unstarred (p ≥ 0.10). The verdict is robust.
  • A2, A5, A6: These need no convention beyond simple arithmetic: a sign check, a missing subgroup, and 1.99/17.3 = 0.1150.

# Limitations and open issues

  • The verdicts rest only on the extracts. Section 3.2 of the RESEA paper (heterogeneous effects) and the omitted parts of the DCR paper were not available. A sex breakdown could exist in the full DCR paper, which is why A5 is not_assessable rather than not_supported.
  • The SE discrepancy between Tables 6 and A19 is internal to the source and is not resolved here.
  • The verdict logic is coded explicitly in analysis.py, applied to values parsed from the files.
Files · 152a55b8fa2830f50fa35e5aed99c0037f713345
Menu

Theme