SPB Git

spb/wp29_uqo

Public
5.4 KB · 44 lines markdown
1# F4-conf-02 — Are the six assertions supported by their cited sources?23## Question4A draft literature review makes six assertions (A1–A6), five citing Armendariz (2026), arXiv:2609.27732v1 (drug consumption rooms, DCR) and one citing Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1 (Rhode Island RESEA RCT). For each one, using only the cited extract, I classify it as supported, not_supported or not_assessable, and give one locator and a one-sentence reason (`verdicts`, `locators`, `reasons`).56## Data and exclusions7Inputs: `data/assertions.json` (6 items), `data/README.md` (14 lines), `data/sources/arxiv-2609.27732v1.txt` (134 lines), `data/sources/arxiv-2606.14621v1.txt` (58 lines) (`n_input_rows`). The SHA-256 hashes match the README and bundle (`out/state/data_audit.md`). No assertion was excluded (`out/exclusions.csv`), so all 6 were assessed (`n_analysed` = 6).8Six data issues were found and logged (`data_issues`). None of them changes a verdict:9- Table A19 prints SE 0.218 for the CS-DR estimate −0.431, while Table 6 prints 0.210.10- The Table A19 note gives 640 control regions; everywhere else the extract gives 608.11- The abstract says fatal hospitalisations fall by "35%", but Table 7 (−0.354) implies −29.8%.12- The text says there are "five" sub-categories, but the Figure 7 note says "four".13- The RESEA text contains a rendering defect, "p ¡ 0.001".14- The RESEA text prints the confidence interval (CI) as magnitudes of the reduction, [1.75, 2.24], while the tables print it with signs, [−2.24, −1.75].1516## Method17`out/analysis.py` reads the extracts, parses the relevant table rows, prints the source lines it relies on, and applies the task's conventions: percentage change = 100·(exp(b) − 1); 95% CI = b ± 1.96·SE, with 2.0 checked as an alternative (`conventions`). The verdict logic is written into the script explicitly, and intermediate quantities are stored in `computation_details`. As a naive baseline, I checked whether every decimal number in each assertion appears verbatim in the cited source (`baseline_verdicts_number_match`). That baseline flags A3 and A4. It misses A2, where the digits match but the sign is wrong, and it cannot detect A5.1819## Results20Counts: 2 supported, 3 not_supported, 1 not_assessable (`n_supported`, `n_not_supported`, `n_not_assessable`).2122| id | verdict (`verdicts`) | locator (`locators`) | decisive element (`reasons`, `computation_details`) |23|---|---|---|---|24| A1 | supported | arxiv-2609.27732v1, Table 6, row "ATT", col "5 km" | ATT −0.431**, SE (0.210), % change −35.0%; 100(exp(−0.431)−1) = −35.01% |25| A2 | not_supported | arxiv-2609.27732v1, Table 7, row "HIV", col "1 km" | printed ATT is +0.063 (SE 0.801), not −0.063, so there is no fall |26| A3 | not_supported | arxiv-2609.27732v1, Table 6, row "% change", col "3 km" | −0.283 implies −24.7% (printed and recomputed −24.65%), not 28.3% |27| A4 | not_supported | arxiv-2609.27732v1, Table 7, row "Hepatitis", col "5 km" | −0.406 ± 1.96·0.250 = [−0.896, 0.084], which contains zero; the asserted interval is ±1 SE |28| A5 | not_assessable | not reported in arxiv-2609.27732v1 | the extract has no results by sex (0 regex hits for men/women/male/female/sex/gender) |29| A6 | supported | arxiv-2606.14621v1, Table 2, row "Weeks on UI", col "Average treatment effect estimate" | ATE −1.99 weeks, 95% CI [−2.24, −1.75], control mean 17.3; 1.99/17.3 = 11.50% |3031## Uncertainty32The verdicts are classifications, not estimates, so they carry no sampling uncertainty. Three verdicts depend on a convention:33- **A1 (critical value / significance).** The assertion's "significant at 5%" matches the printed `**`. As a cross-check: |b|/SE = 2.05 with SE 0.210 (`computation_details.A1.z`), which is above both 1.96 and 2.0. If the Table A19 SE of 0.218 were used instead, the ratio would be 1.98. That is still above 1.96 but just below 2.0. Table A19 still prints `**`, and the assertion describes the Table 6 numbers, so the verdict does not change.34- **A3 (log-to-percent conversion).** Converting gives −24.7% and the source prints −24.7%. The figure 28.3% would follow only from reading the log coefficient directly as a percentage, which the task's conventions rule out. Even that reading gives a 28.3% reduction only as a first-order approximation, and the source prints −24.7%. The verdict holds.35- **A4 (critical value).** With 1.96 the CI is [−0.896, 0.084]; with 2.0 it is [−0.906, 0.094] (`computation_details.A4`). The CI would exclude zero only with a critical value below 1.624, i.e. a confidence level under about 90%. The verdict holds under any reasonable 95% convention.3637A2, A5 and A6 depend on no convention. A6 involves only printed values and a single division.3839## Limitations and open issues40- The verdicts rest solely on the extracts, as the task requires. In particular, the A5 verdict of not_assessable is relative to the extract: the omitted sections of the paper might report results by sex, but they are inadmissible here.41- The internal inconsistencies of the DCR source (SE 0.210 vs 0.218; 608 vs 640 controls; the abstract's 35% vs the table-implied ~30% for fatal outcomes) are reported in `data_issues` but not resolved. None of them decides a verdict.42- The verdict logic is written into `out/analysis.py` (the task allows this), so the script documents and checks the judgements rather than deriving them automatically.43- Feasibility: complete (`feasibility`).44
Files · main
Menu

Theme