SPB Git

spb/wp29_uqo

Public

# F4-conf-02 — Are the six assertions supported by their cited sources?

# Question

A draft literature review makes six assertions (A1–A6), five citing Armendariz (2026), arXiv:2609.27732v1 (drug consumption rooms, DCR) and one citing Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1 (Rhode Island RESEA RCT). For each one, using only the cited extract, I classify it as supported, not_supported or not_assessable, and give one locator and a one-sentence reason (verdicts, locators, reasons).

# Data and exclusions

Inputs: data/assertions.json (6 items), data/README.md (14 lines), data/sources/arxiv-2609.27732v1.txt (134 lines), data/sources/arxiv-2606.14621v1.txt (58 lines) (n_input_rows). The SHA-256 hashes match the README and bundle (out/state/data_audit.md). No assertion was excluded (out/exclusions.csv), so all 6 were assessed (n_analysed = 6). Six data issues were found and logged (data_issues). None of them changes a verdict:

  • Table A19 prints SE 0.218 for the CS-DR estimate −0.431, while Table 6 prints 0.210.
  • The Table A19 note gives 640 control regions; everywhere else the extract gives 608.
  • The abstract says fatal hospitalisations fall by "35%", but Table 7 (−0.354) implies −29.8%.
  • The text says there are "five" sub-categories, but the Figure 7 note says "four".
  • The RESEA text contains a rendering defect, "p ¡ 0.001".
  • The RESEA text prints the confidence interval (CI) as magnitudes of the reduction, [1.75, 2.24], while the tables print it with signs, [−2.24, −1.75].

# Method

out/analysis.py reads the extracts, parses the relevant table rows, prints the source lines it relies on, and applies the task's conventions: percentage change = 100·(exp(b) − 1); 95% CI = b ± 1.96·SE, with 2.0 checked as an alternative (conventions). The verdict logic is written into the script explicitly, and intermediate quantities are stored in computation_details. As a naive baseline, I checked whether every decimal number in each assertion appears verbatim in the cited source (baseline_verdicts_number_match). That baseline flags A3 and A4. It misses A2, where the digits match but the sign is wrong, and it cannot detect A5.

# Results

Counts: 2 supported, 3 not_supported, 1 not_assessable (n_supported, n_not_supported, n_not_assessable).

id verdict (verdicts) locator (locators) decisive element (reasons, computation_details)
A1 supported arxiv-2609.27732v1, Table 6, row "ATT", col "5 km" ATT −0.431**, SE (0.210), % change −35.0%; 100(exp(−0.431)−1) = −35.01%
A2 not_supported arxiv-2609.27732v1, Table 7, row "HIV", col "1 km" printed ATT is +0.063 (SE 0.801), not −0.063, so there is no fall
A3 not_supported arxiv-2609.27732v1, Table 6, row "% change", col "3 km" −0.283 implies −24.7% (printed and recomputed −24.65%), not 28.3%
A4 not_supported arxiv-2609.27732v1, Table 7, row "Hepatitis", col "5 km" −0.406 ± 1.96·0.250 = [−0.896, 0.084], which contains zero; the asserted interval is ±1 SE
A5 not_assessable not reported in arxiv-2609.27732v1 the extract has no results by sex (0 regex hits for men/women/male/female/sex/gender)
A6 supported arxiv-2606.14621v1, Table 2, row "Weeks on UI", col "Average treatment effect estimate" ATE −1.99 weeks, 95% CI [−2.24, −1.75], control mean 17.3; 1.99/17.3 = 11.50%

# Uncertainty

The verdicts are classifications, not estimates, so they carry no sampling uncertainty. Three verdicts depend on a convention:

  • A1 (critical value / significance). The assertion's "significant at 5%" matches the printed **. As a cross-check: |b|/SE = 2.05 with SE 0.210 (computation_details.A1.z), which is above both 1.96 and 2.0. If the Table A19 SE of 0.218 were used instead, the ratio would be 1.98. That is still above 1.96 but just below 2.0. Table A19 still prints **, and the assertion describes the Table 6 numbers, so the verdict does not change.
  • A3 (log-to-percent conversion). Converting gives −24.7% and the source prints −24.7%. The figure 28.3% would follow only from reading the log coefficient directly as a percentage, which the task's conventions rule out. Even that reading gives a 28.3% reduction only as a first-order approximation, and the source prints −24.7%. The verdict holds.
  • A4 (critical value). With 1.96 the CI is [−0.896, 0.084]; with 2.0 it is [−0.906, 0.094] (computation_details.A4). The CI would exclude zero only with a critical value below 1.624, i.e. a confidence level under about 90%. The verdict holds under any reasonable 95% convention.

A2, A5 and A6 depend on no convention. A6 involves only printed values and a single division.

# Limitations and open issues

  • The verdicts rest solely on the extracts, as the task requires. In particular, the A5 verdict of not_assessable is relative to the extract: the omitted sections of the paper might report results by sex, but they are inadmissible here.
  • The internal inconsistencies of the DCR source (SE 0.210 vs 0.218; 608 vs 640 controls; the abstract's 35% vs the table-implied ~30% for fatal outcomes) are reported in data_issues but not resolved. None of them decides a verdict.
  • The verdict logic is written into out/analysis.py (the task allows this), so the script documents and checks the judgements rather than deriving them automatically.
  • Feasibility: complete (feasibility).
Files · main
Menu

Theme