SPB Git

spb/wp29_uqo

Public

# F4-conf-02 — Are the six assertions supported by their cited sources?

# Question

For six literature-review assertions (A1–A5 cite Armendariz 2026, arXiv:2609.27732v1; A6 cites Li, Pearson-Merkowitz & Yokum 2026, arXiv:2606.14621v1), decide from the cited extract alone whether each is supported, not_supported or not_assessable, with a locator and a one-sentence reason (verdicts, locators, reasons).

# Data and exclusions

Inputs: data/assertions.json (6 items), data/README.md (14 lines), the DCR extract (134 lines) and the RESEA extract (58 lines) (n_input_rows). SHA-256 of both extracts match the README. No exclusions: all 6 assertions were assessed (n_analysed = 6; out/exclusions.csv). Only the extracts were used as evidence.

Source issues found (data_issues): (i) DCR Table A19 prints SE 0.218 for the CS-DR ATT −0.431 that Table 6 prints with SE 0.210; (ii) the A19 note says 640 control regions vs 608 elsewhere; (iii) the DCR abstract says fatal hospitalisations fall by 35%, but the 5 km fatal ATT −0.354 converts to −29.8%; (iv) the text calls HIV estimates "close to zero" although the 5 km ATT is −0.340; (v) RESEA prose prints the reduction CIs as positive magnitudes, and has a rendering defect "p ¡ 0.001". None of these changes a verdict.

# Method

out/analysis.py finds the deciding table lines, prints them, and checks each element of each assertion against the printed cells. It uses the task conventions: % change = 100·(exp(b) − 1) and 95% CI = b ± 1.96·SE, with ±2.0·SE as a sensitivity check. As a baseline, it also checks whether every decimal number in an assertion appears verbatim in the cited source (baseline_naive_number_match_verdicts). The baseline agrees with the final verdicts on 4 of the 6 assertions (baseline_agreement_with_final). It misses the sign error in A2 and the subgroup that is never reported in A5.

# Results

Counts: 2 supported, 3 not_supported, 1 not_assessable (n_supported, n_not_supported, n_not_assessable).

id verdict (verdicts) locator (locators) decisive element (reasons, derived)
A1 supported DCR Table 6, row ATT, col 5 km −0.431** (0.210), % change −35.0%; 100(exp(−0.431)−1) = −35.01% (derived.A1_pct_from_logpoints)
A2 not_supported DCR Table 7, row HIV, col 1 km printed ATT is +0.063 (SE 0.801): the assertion gets the sign wrong
A3 not_supported DCR Table 6, row % change, col 3 km −0.283 log points = −24.7% (−24.65%, derived.A3_pct_from_logpoints), not 28.3%
A4 not_supported DCR Table 7, row Hepatitis, col 5 km −0.406 (0.250) gives a 95% CI of [−0.896, 0.084], which includes 0 (derived.A4_ci95_1.96); the asserted [−0.66, −0.16] is ±1 SE (derived.A4_ci_1se)
A5 not_assessable not reported in arxiv-2609.27732v1 the extract gives no results by sex
A6 supported RESEA Table 2, row Weeks on UI, col ATE −1.99 weeks, 95% CI [−2.24, −1.75], control mean 17.3; 1.99/17.3 = 11.5% (derived.A6_relative_reduction_pct)

# Uncertainty

These verdicts are deterministic judgements, not statistical estimates. Here is which verdicts depend on a convention:

  • A1 depends on the significance claim. With the Table 6 SE, t = −2.05 (derived.A1_t_stat_table6). With the Table A19 SE of 0.218, |t| = 1.98, which is still above 1.96 and carries ** in both tables. The verdict therefore holds under both SEs (data_issues). Under a strict 2.0 critical value, however, |t| = 1.98 with the A19 SE would just miss. This is the one place where a convention matters close to the threshold. The assertion quotes SE 0.210 from Table 6, and with that SE it passes under either critical value.
  • A3 depends on the log-to-percent conversion. The source itself prints −24.7%. The only way to get 28.3% is to read log points directly as percent, and both the task convention and the printed row rule that out, so the verdict is robust.
  • A4 depends on the critical value. The CI includes zero under 1.96 (derived.A4_ci95_1.96) and under 2.0 (derived.A4_ci95_2.0). The interval would exclude zero only at about ±1 SE (roughly a 68% interval), which is not a 95% CI. The verdict is robust.
  • A6 involves only the rounded ratio 11.50%, which is also printed in the source text. A2 and A5 depend on no convention.

# Limitations and open issues

  • The verdicts rest on the extracts only. A5 is not_assessable because the extract contains no results by sex. The full paper might contain such results, but that would be outside the admissible evidence.
  • The DCR source contradicts itself in places: the SE for the main ATT (0.210 vs 0.218), the number of control regions (608 vs 640), and the fatal-outcome percentage in the abstract (35% vs −29.8% by conversion). I decided verdicts on the table cells the assertions quote.
  • In A6, "selection into RESEA" matches the source's own wording. Table 3 calls the same estimates ITT, which is consistent with "selection".
  • No randomness is involved (random_seed = null). The task is complete (feasibility).
Files · 76f9a44a3a85fb2bc2b0b370a4ae4eed9b95e579
Menu

Theme