# F4-conf-01 — Treatment effects and uncertainty extracted from two published tables

# Question

I extracted printed point estimates, standard errors, confidence-interval bounds and sample counts from two source extracts:

• Armendariz (2026), arXiv:2609.27732v1: drug consumption rooms (DCRs) in Switzerland, staggered DiD (Callaway–Sant'Anna), Tables 6 and 7.
• Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1: Rhode Island RESEA randomised trial, Table 2.

I also derived two quantities the sources do not print, and identified one requested quantity that is not reported anywhere. Nothing was re-estimated.

# Data and exclusions

I read three files: data/README.md (10 lines), arxiv-2609.27732v1.txt (134 lines) and arxiv-2606.14621v1.txt (58 lines) (n_input_rows). Their SHA-256 hashes match the README and bundle (input_sha256; see out/state/data_audit.md). No items were excluded: all 25 requested items were assessed (n_analysed, out/exclusions.csv). The inputs have some internal inconsistencies, listed in data_issues. The most important is that the 5 km CS-DR ATT has SE (0.210) in Table 6 but (0.218) in Appendix Table A19. Per the key definitions, Table 6 is used.

# Method

out/analysis.py parses the table lines in the extracts and prints every source line it matched. Parsing rules:

• DCR tables: the standard error is on the line after the coefficient row, and significance comes from the stars.
• RESEA Table 2: each cell has the form "estimate; p; 95% CI: [lo, hi]".
• Numbers: "$" and thousands separators are stripped and signs kept, so "17,250" becomes 17250 and "$1,153" becomes 1153.

Each value carries a locator in locators. The two derived quantities use the normal critical value 1.96 (derivation_notes):

• the CI of the DCR 5 km ATT is estimate ± 1.96·SE;
• the SE of the RI wage ATE is (upper − lower)/(2·1.96).

There is no randomness (random_seed = null). Because this is an extraction task, the baseline is simply the verbatim reading of each named cell. The derivations are the only computations.

# Results

DCR study (log points):

• Table 6, all specifications:
  • 5 km ATT: −0.431 (dcr_att_5km), SE 0.210 (dcr_att_5km_se), starred ** in the source.
  • 3 km ATT: −0.283 (dcr_att_3km), SE 0.279 (dcr_att_3km_se).
  • Observations at 5 km: 17250 (dcr_obs_5km).
• Table 7, sub-outcomes:
  • Narcotics at 3 km: 0.549 (dcr_narcotics_att_3km), SE 0.187 (dcr_narcotics_att_3km_se).
  • Drug-related (fatal) at 1 km: −1.207 (dcr_fatal_att_1km), SE 0.137 (dcr_fatal_att_1km_se).
  • HIV at 5 km: −0.340 (dcr_hiv_att_5km), SE 0.294 (dcr_hiv_att_5km_se). It has no star, so the source does not mark it as significant at 10% (dcr_hiv_att_5km_significant_10pct = false).
  • The exact p-value for HIV at 5 km is not reported: Table 7 prints only stars (dcr_hiv_att_5km_p_value = null).
• Derived: the 95% CI of the Table 6 5 km ATT is [−0.8426, −0.0194] (dcr_att_5km_ci_low, dcr_att_5km_ci_high).

RESEA study (Table 2):

• Wage ATE: $1153 per year (ri_wage_ate), printed 95% CI [166, 2141] (ri_wage_ate_ci_low, ri_wage_ate_ci_high).
  • No standard error is printed (ri_wage_ate_se_reported = false).
  • Derived SE: 503.83 (ri_wage_ate_se_derived).
• Control-group mean wage: $33043 (ri_control_mean_wage).
• Reemployment ATE: 1.47 pp (ri_reemployment_ate_pp).
• Weeks on UI ATE: −1.99 weeks (ri_weeks_ui_ate), printed 95% CI [−2.24, −1.75] (ri_weeks_ui_ate_ci_low, ri_weeks_ui_ate_ci_high).

# Uncertainty

• DCR (Tables 6 and 7): uncertainty is printed as standard errors clustered at the MedStat region level, plus significance stars. No CIs or exact p-values are printed for these cells.
• RESEA (Table 2): uncertainty is printed as two-sided p-values and 95% confidence intervals, with no standard errors.

What the derived quantities assume:

• DCR 5 km CI (dcr_att_5km_ci_low, dcr_att_5km_ci_high): the ATT estimator is asymptotically normal, and the CI is a Wald interval with z = 1.96.
  • Using 2.0 instead gives [−0.851, −0.011] (sensitivity.dcr_att_5km_ci_z2.0).
  • Using the A19 SE gives [−0.8583, −0.0037] (sensitivity.dcr_att_5km_ci_using_tableA19_se).
  • In all three versions the interval excludes zero, which matches the ** star.
• RI wage SE (ri_wage_ate_se_derived): the printed CI is a symmetric normal-approximation interval. A t critical value would differ negligibly at N = 23,549.
  • The CI is close to symmetric around 1153: the half-widths imply SEs of 503.57 and 504.08 (sensitivity.ri_wage_ate_se_from_lower_half, sensitivity.ri_wage_ate_se_from_upper_half).
  • The normal p-value implied by the derived SE is 0.0221 (sensitivity.ri_wage_normal_p_from_derived_se), which matches the printed p=0.022.
  • With 2.0 the SE would be 493.75 (sensitivity.ri_wage_ate_se_z2.0).
• Stars check: for all 22 starred-or-unstarred DCR coefficient/SE pairs in Tables 6, 7 and A19, the printed stars agree with normal-approximation p-values computed from the printed SEs (sensitivity.significance_star_mismatches is empty).

# Limitations and open issues

• Decimals in JSON: JSON numbers cannot keep trailing zeros, so the printed SE "0.210" is stored as 0.21 (dcr_att_5km_se). The printed form appears in the locators/source lines that the script prints.
• Exact p-value: the HIV 5 km p-value cannot be obtained from the source. A normal-approximation value (sensitivity.dcr_hiv_att_5km_normal_p_approx, 0.2475) is derived, not printed, and is deliberately not used for the key.
• Inconsistencies inside the sources (data_issues), none of which affect the keys:
  • the Table 6 vs A19 SE for the same 5 km estimate (0.210 vs 0.218);
  • 608 vs 640 control regions;
  • the abstract's "35%" fall in fatal hospitalisations vs Section 6.1's "approximately 30%";
  • the RESEA text printing the weeks-on-UI CI as positive magnitudes;
  • an encoding defect ("p ¡ 0.001").
• Feasibility: the task is complete (feasibility = "complete"). The one missing quantity was designed to be recognised as not reported.