F4-conf-01: Treatment effects and their uncertainty taken from two published results tables
Question
The task is to copy specified point estimates, uncertainty measures and counts exactly as printed from two arXiv extracts, and to give each one a locator:
- DCR study (arxiv-2609.27732v1): staggered DiD on Swiss drug consumption room openings, Tables 6 and 7.
- RESEA study (arxiv-2606.14621v1): randomised trial in Rhode Island, Table 2.
Two quantities that the sources do not print are derived, with the convention stated:
- a 95% CI for the Table 6 5 km ATT, computed from its SE;
- an SE for the RESEA wage ATE, computed from its CI.
The task also asks us to identify which requested quantity is not reported at all. Nothing is re-estimated.
Data and exclusions
We read data/README.md (10 lines), arxiv-2609.27732v1.txt (134 lines) and arxiv-2606.14621v1.txt (58 lines). Only these extracts were used. Table A19 (DCR) and Table 3 (RESEA) were read for consistency checks only, because every key names Table 6, 7 or 2.
Method
out/analysis.py finds the table rows by their labels and the columns by their headers, then parses the printed numbers and prints each line it matched.
- Stars are stripped from estimates and recorded separately.
- DCR standard errors come from the parenthesised line under each coefficient row.
- RESEA ATE cells are split into the estimate, the p-value and the CI bounds.
- Formatting of printed numbers. Thousands separators are dropped (17,250 → 17250; $2,141 → 2141). JSON cannot store trailing zeros, so printed
0.210appears as0.21and-0.340as-0.34. The numeric values are identical to the printed ones.
Results
| key | value | source |
|---|---|---|
| dcr_att_5km / se | -0.431** / (0.210) | Table 6, ATT, 5 km |
| dcr_att_3km / se | -0.283 / (0.279) | Table 6, ATT, 3 km |
| dcr_obs_5km | 17,250 | Table 6, Observations, 5 km |
| dcr_narcotics_att_3km / se | 0.549*** / (0.187) | Table 7, Narcotics, 3 km |
| dcr_fatal_att_1km / se | -1.207*** / (0.137) | Table 7, Drug-related (fatal), 1 km |
| dcr_hiv_att_5km / se | -0.340 / (0.294) | Table 7, HIV, 5 km |
| dcr_hiv_att_5km_p_value | null: not reported (stars only) | Table 7 |
| dcr_hiv_att_5km_significant_10pct | false (no star) | Table 7 |
| dcr_att_5km CI (derived) | [-0.8426, -0.0194] | from Table 6 |
| ri_wage_ate | $1,153; 95% CI [166, 2,141] | Table 2, Wage, ATE |
| ri_wage_ate_se_reported | false | Table 2 |
| ri_wage_ate_se_derived | 503.83 | from Table 2 CI |
| ri_control_mean_wage | $33,043 | Table 2, Wage, Control |
| ri_reemployment_ate_pp | 1.47 pp | Table 2, Reemployment, ATE |
| ri_weeks_ui_ate | -1.99 weeks; 95% CI [-2.24, -1.75] | Table 2, Weeks on UI, ATE |
The full locator strings are in results.json under locators.
Uncertainty
- DCR study (Tables 6 and 7). The uncertainty is printed as standard errors (clustered at the MedStat region level, in parentheses) plus significance stars. No exact p-values or CIs are printed.
- RESEA study (Table 2). The uncertainty is printed as 95% confidence intervals and two-sided p-values. No standard errors are printed.
How the derived quantities were computed:
- DCR 95% CI. -0.431 ± 1.96 × 0.210 = [-0.8426, -0.0194].
- This uses a normal approximation with critical value 1.96.
- The extract does not say which critical value the paper uses (it could be a bootstrap or t critical value). Using 2.0 instead would give [-0.851, -0.011].
- The interval excludes zero, which matches the ** (p<0.05) star.
- RESEA wage SE. (2141 − 166) / (2 × 1.96) = 503.83.
- This assumes the printed interval is a symmetric Wald interval with a normal critical value of 1.96.
- The assumption is supported in two ways. The half-widths are 987 and 988. The implied z of 2.288 gives a normal two-sided p of 0.0221, against a printed p of 0.022.
- Using 2.0 instead would give an SE of about 493.75.
Limitations and open issues
- Two conflicting SEs for the 5 km ATT. Table A19 prints an SE of (0.218) for the same CS-DR 5 km ATT of -0.431, whereas Table 6 prints (0.210). We used Table 6, as the key names it.
- Control-region count does not match the observation count. Tables 6 and 7 say there are 608 never-treated control regions; the A19 note says 640. The printed Observations (16,175 / 16,875 / 17,250) equal (treated + 640) × 25 exactly. With 608 controls the counts would be 15,375 / 16,075 / 16,450. The observation counts are reported as printed.
- Narrative text does not match the tables in places:
- The abstract says fatal hospitalisations fall by 35%. Table 7's -0.354 at 5 km implies a fall of 29.8%, and Section 6.1 says about 30%.
- The text calls the HIV estimates "close to zero", but the 5 km point estimate is -0.340. It is insignificant, not near zero.
- Weeks-on-UI CI sign. The RESEA prose gives this CI as magnitudes ([1.75, 2.24] weeks). We used the signed Table 2 bounds [-2.24, -1.75].
- Rendering and layout. The RESEA extract has a rendering defect ("p ¡ 0.001"). Its Tables 2 and 3 were re-set by the extract editor, so we rely on the editor's statement that the values are verbatim. We could not check them against the original PDF.
- Missing p-value. The exact p-value for the HIV 5 km ATT is not printed, so it is reported as null. A normal-approximation p could be computed from the estimate and SE, but that would be a derived value, not a printed one.