# F4-conf-01 — Extracted treatment effects and uncertainty

# Question

This task pulls printed effect estimates, uncertainty measures and counts from two open-access papers, exactly as printed. The first is Armendariz (2026), arXiv:2609.27732v1, which studies drug consumption rooms (DCR) with a staggered difference-in-differences design. The second is Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1, a randomised trial of Rhode Island's RESEA programme. The task also derives one CI from an SE and one SE from a CI, and flags a quantity the source does not report. Nothing is re-estimated.

# Data and exclusions

• Inputs: two source extracts and data/README.md. The SHA-256 hashes match the README and bundle. Line counts are in n_input_rows: 134, 58 and 10 lines.
• No items were excluded (out/exclusions.csv). There were 25 requested items, all assessed (n_analysed).
• Each value comes from the table named in its key, even when a different table prints a conflicting value (see Limitations).

# Method

out/analysis.py parses the " | "-separated tables, converts printed strings to numbers without rounding, and prints the matched source lines. It writes out/results.json deterministically; no randomness is involved (random_seed = null). Each key has a locator in locators.

Derivation conventions (from derivation_notes):

• CI from SE: estimate ± 1.96 × SE (normal approximation).
• SE from CI: (upper − lower)/(2 × 1.96).

# Results

DCR study, Table 6 (log points; clustered SE in parentheses):

• 5 km ATT: −0.431** with SE 0.210 (dcr_att_5km, dcr_att_5km_se), based on 17,250 observations (dcr_obs_5km).
• 3 km ATT: −0.283 with SE 0.279 (dcr_att_3km, dcr_att_3km_se).
• Derived 95% CI for the 5 km ATT: [−0.8426, −0.0194] (dcr_att_5km_ci_low, dcr_att_5km_ci_high).

DCR study, Table 7:

• Narcotics, 3 km: 0.549*** with SE 0.187 (dcr_narcotics_att_3km, dcr_narcotics_att_3km_se).
• Drug-related (fatal), 1 km: −1.207*** with SE 0.137 (dcr_fatal_att_1km, dcr_fatal_att_1km_se).
• HIV, 5 km: −0.340 with SE 0.294 and no star, so it is not significant at 10% (dcr_hiv_att_5km, dcr_hiv_att_5km_se, dcr_hiv_att_5km_significant_10pct = false).
• The exact p-value for HIV at 5 km is not reported. The extract prints only stars, so dcr_hiv_att_5km_p_value = null.

RESEA study, Table 2:

• Wage ATE: $1,153 per year, 95% CI [166, 2,141] (ri_wage_ate, ri_wage_ate_ci_low, ri_wage_ate_ci_high). Control-group mean wage: $33,043 (ri_control_mean_wage).
• The source prints no SE for the wage ATE (ri_wage_ate_se_reported = false). The derived SE is 503.83 (ri_wage_ate_se_derived).
• Reemployment ATE: 1.47 pp (ri_reemployment_ate_pp).
• Weeks on UI ATE: −1.99, 95% CI [−2.24, −1.75] (ri_weeks_ui_ate, ri_weeks_ui_ate_ci_low, ri_weeks_ui_ate_ci_high).

JSON drops trailing zeros: 0.210 is stored as 0.21 and −0.340 as −0.34. The numeric values are unchanged, and the printed forms are given above.

# Uncertainty

What each source reports:

• DCR, Tables 6 and 7: standard errors clustered at the MedStat region level, plus stars for p<0.01, p<0.05 and p<0.10. The extract contains no confidence intervals and no exact p-values for these cells.
• RESEA, Table 2: two-sided p-values and 95% confidence intervals from the authors' pre-registered methodology. No standard errors are printed.

What the derived values assume:

• DCR 5 km CI: the estimator is asymptotically normal, the printed SE (rounded to 3 decimals) is the correct one, and z = 1.96. A t critical value or 2.0 would widen the interval slightly.
• RESEA wage SE: the printed CI is a symmetric Wald interval with z = 1.96. The midpoint is 1153.5, matching the estimate up to rounding. With N = 23,549 a t critical value would make no practical difference. Converting back, this SE gives p ≈ 0.022, the same as the printed p (out/state/checks.md).
• Rounding: the derived values carry the rounding error of the printed inputs.

# Limitations and open issues

• Requested value not printed: the exact p-value for the HIV 5 km ATT is not printed, so it is null.
• Problems found in the sources (all listed in data_issues):
  1. The same 5 km CS-DR estimate has SE 0.210 in Table 6 but 0.218 in Table A19. I used Table 6, as the key requires.
  2. The Table A19 note gives 640 control regions; the tables give 608.
  3. The abstract says fatal hospitalisations fall by "35%", but the Table 7 5 km value of −0.354 log points is about −29.8% (Section 6.1 says ~30%).
  4. The text calls the HIV estimates "close to zero", but the 5 km value is −0.340.
  5. The RESEA text gives the weeks-on-UI CI as unsigned magnitudes; Table 2 is signed.
  6. The RESEA text contains a rendering defect, "p ¡ 0.001".
  7. The RESEA tables were re-set by the extract editor. I relied on the editor's statement that the values are verbatim; Table 3's first column matches Table 2.
• Overall: feasibility is "complete" (feasibility). The null p-value is the expected answer for a quantity the source does not report, not a failure.