SPB Git

spb/wp29_uqo

Public
3.6 KB · 62 lines json
1{2  "task_id": "F4-conf-02",3  "verdicts": {4    "A1": "supported",5    "A2": "not_supported",6    "A3": "not_supported",7    "A4": "not_supported",8    "A5": "not_assessable",9    "A6": "supported"10  },11  "locators": {12    "A1": "arxiv-2609.27732v1 | Table 6 | row \"ATT\" | column \"5 km\"",13    "A2": "arxiv-2609.27732v1 | Table 7 | row \"HIV\" | column \"1 km\"",14    "A3": "arxiv-2609.27732v1 | Table 6 | row \"% change\" | column \"3 km\"",15    "A4": "arxiv-2609.27732v1 | Table 7 | row \"Hepatitis\" | column \"5 km\"",16    "A5": "not reported in arxiv-2609.27732v1",17    "A6": "arxiv-2606.14621v1 | Table 2 | row \"Weeks on UI\" | column \"Average treatment effect estimate\""18  },19  "reasons": {20    "A1": "Table 6 prints ATT -0.431** with SE (0.210) at 5 km and a -35.0% change (100*(exp(-0.431)-1) = -35.0%), and the text states significance at the 5% level, so every element matches.",21    "A2": "The 1 km HIV ATT is printed as +0.063 (SE 0.801), positive and far from significant, so the claimed negative sign (-0.063) and the claimed fall in HIV hospitalisations are contradicted.",22    "A3": "The source converts the 3 km ATT of -0.283 to a -24.7% change (100*(exp(-0.283)-1) = -24.65%), not -28.3%, and the estimate is not statistically significant (SE 0.279), so reading log points as percent is wrong.",23    "A4": "With ATT -0.406 and SE (0.250) the 95% CI is [-0.896, 0.084], which includes zero (no star; text calls it insignificant); the claimed [-0.66, -0.16] corresponds to +/-1 SE only.",24    "A5": "The extract reports no estimates by sex (Tables 6, 7, A19 and Section 6.1 are pooled over the population), so a men-versus-women comparison can be neither confirmed nor refuted.",25    "A6": "Table 2 prints the ATE as -1.99 weeks with 95% CI [-2.24, -1.75] and a control mean of 17.3 weeks; 1.99/17.3 = 11.5%, matching the 11.5% stated in the results text."26  },27  "n_supported": 2,28  "n_not_supported": 3,29  "n_not_assessable": 1,30  "feasibility": "complete",31  "feasibility_reason": null,32  "data_issues": [33    {34      "issue": "Armendariz: the CS-DR 5 km ATT -0.431** has SE (0.210) in Table 6 but (0.218) in Table A19 for the same estimator, buffer and sample (17,250 obs).",35      "handling": "A1 judged against Table 6 (the table the assertion paraphrases); significance at 5% holds under either SE (|t| = 0.431/0.210 = 2.05 or 0.431/0.218 = 1.98, both > 1.96; starred ** in both tables), so the verdict does not change."36    },37    {38      "issue": "Armendariz: Table 6/7 notes state 608 never-treated control regions; Table A19 note states 640.",39      "handling": "Recorded; not decisive for any assertion."40    },41    {42      "issue": "Armendariz: abstract says fatal hospitalisations fall by 35%, but Table 7 5 km fatal ATT -0.354 implies -29.8%, matching Section 6.1's 'approximately 30%'.",43      "handling": "Recorded; not decisive for any assertion (none concerns fatal outcomes)."44    },45    {46      "issue": "Armendariz: text says HIV estimates are 'close to zero', while the 5 km HIV ATT is -0.340 (imprecise, SE 0.294).",47      "handling": "Recorded; A2 is decided by the printed 1 km coefficient (+0.063), not by the prose."48    },49    {50      "issue": "Li et al.: results text gives the weeks-on-UI CIs as positive magnitudes ([1.75, 2.24]) while Table 2 prints signed values [-2.24, -1.75]; a rendering artefact 'p ¡ 0.001' appears in the short-term paragraph.",51      "handling": "Treated as equivalent sign conventions for a reduction; Table 2 used as the locator for A6."52    }53  ],54  "n_input_rows": {55    "assertions.json": 6,56    "README.md": 14,57    "arxiv-2609.27732v1.txt": 134,58    "arxiv-2606.14621v1.txt": 5859  },60  "n_analysed": 6,61  "random_seed": null62}
Files · 6df43c338e49477c769c36e7f7d249af2563db7f
Menu

Theme