SPB Git

spb/wp29_uqo

Public

# F4-conf-01 — Treatment effects and uncertainty extracted from two published tables

# Question

Extract, exactly as printed, specified treatment-effect estimates, standard errors (SEs), confidence-interval (CI) bounds and counts from (i) Armendariz (2026), arXiv:2609.27732v1, a staggered DiD study of drug consumption room (DCR) openings in Switzerland (Tables 6 and 7), and (ii) Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1, an RCT of Rhode Island's RESEA programme (Table 2). Attach a locator to each value, derive a 95% CI from a printed SE and an SE from a printed CI, and flag the requested quantity that the source does not print. Nothing is re-estimated.

# Data and exclusions

Inputs: the two source extracts and data/README.md, all SHA-256-verified against the README/bundle (out/state/data_audit.md). Line counts: n_input_rows (README.md 10, arxiv-2609.27732v1.txt 134, arxiv-2606.14621v1.txt 58). I assessed 25 requested items (n_analysed) and excluded none (out/exclusions.csv). I logged eight input problems in data_issues. The ones that matter most:

  • The 5 km CS-DR ATT has SE 0.210 in Table 6 but 0.218 in Table A19. I used Table 6 because the key names it.
  • Table A19's note says there are 640 never-treated regions; Tables 6 and 7 say 608.
  • For the fatal 5 km effect, the abstract says 35% and Section 6.1 says about 30%.
  • The prose gives the weeks-on-UI CI as positive magnitudes, while Table 2 prints negative bounds.
  • One paragraph has a rendering defect: "p ¡ 0.001".

# Method

out/analysis.py parses the " | "-delimited table rows. In the DCR tables, the SE is the parenthesised value on the row after each coefficient. The script removes "$", thousands separators, units and significance stars, keeps the printed sign, and prints every matched source line. Locators follow <source> | Table n | row "…" | column "…". Significance at 10% is read from the stars (* means p<0.10). The script searches the extract for an exact p-value and finds none. Derivations use the normal critical value 1.96 (derivation_notes). There is no randomness (random_seed = null). The baseline is simply the direct extraction, since the estimands are the printed numbers.

# Results

Note that JSON drops trailing zeros (0.210 is stored as 0.21). The verbatim strings are kept in printed_strings.

DCR (arxiv-2609.27732v1)

  • Table 6, overall ATT:
    • 5 km: −0.431** with SE 0.210 (dcr_att_5km, dcr_att_5km_se).
    • 3 km: −0.283 with SE 0.279 (dcr_att_3km, dcr_att_3km_se).
    • Observations at 5 km: 17,250 (dcr_obs_5km).
  • Table 7, by outcome:
    • Narcotics at 3 km: 0.549*** with SE 0.187 (dcr_narcotics_att_3km, dcr_narcotics_att_3km_se).
    • Drug-related (fatal) at 1 km: −1.207*** with SE 0.137 (dcr_fatal_att_1km, dcr_fatal_att_1km_se).
    • HIV at 5 km: −0.340 with SE 0.294 and no star (dcr_hiv_att_5km, dcr_hiv_att_5km_se). It is therefore not marked significant at 10% (dcr_hiv_att_5km_significant_10pct = false). The source prints no exact p-value, so dcr_hiv_att_5km_p_value = null.
  • Derived 95% CI for the Table 6 5 km ATT: [−0.8426, −0.0194] (dcr_att_5km_ci_low, dcr_att_5km_ci_high).

RESEA (arxiv-2606.14621v1, Table 2)

  • Wage ATE: $1,153 per year, 95% CI [166, 2,141] (ri_wage_ate, ri_wage_ate_ci_low, ri_wage_ate_ci_high).
    • No SE is printed (ri_wage_ate_se_reported = false).
    • The SE implied by the CI is 503.83 (ri_wage_ate_se_derived).
  • Control-group mean wage: $33,043 (ri_control_mean_wage).
  • Reemployment ATE: 1.47 pp (ri_reemployment_ate_pp).
  • Weeks on UI ATE: −1.99, 95% CI [−2.24, −1.75] (ri_weeks_ui_ate, ri_weeks_ui_ate_ci_low, ri_weeks_ui_ate_ci_high).

# Uncertainty

  • SEs: all DCR uncertainties (Tables 6 and 7) are SEs clustered at the MedStat region level, printed in parentheses (dcr_*_se). The stars use the source's thresholds: *** p<0.01, ** p<0.05, * p<0.10.
  • CIs: the RESEA uncertainties are two-sided 95% CIs, printed with p-values. They come from the "methodology described in the main text", which is not part of the extract.
  • Derived DCR CI: dcr_att_5km_ci_low/_high = ATT ± 1.96·SE. This assumes the estimator is approximately normal with the printed clustered SE. With a t critical value or 2.0 the interval would be slightly wider. The interval excludes 0, which agrees with the ** marking (|t| ≈ 2.05; out/state/checks.md).
  • Derived RESEA SE: ri_wage_ate_se_derived = (2,141 − 166)/(2·1.96). This assumes a symmetric normal-approximation CI. The printed CI is symmetric to within 1 USD of rounding. Plugging the derived SE back in gives a two-sided p of about 0.022, matching the printed p=0.022 (checks.md).
  • Inputs are printed to only 3 decimals (DCR) or whole dollars (RESEA), so the derived values carry that rounding error.

# Limitations and open issues

  • The source does not print the exact p-value of the HIV 5 km ATT. It cannot be obtained without a distributional assumption, so I left it null. Its absence is the expected "not reported" item.
  • The task is otherwise complete (feasibility = "complete").
  • Using Table A19's SE (0.218) instead of Table 6's would give a different derived CI. The source does not explain the discrepancy, and I did not resolve it.
  • The method behind the RESEA CI is not in the extract, so the derived SE depends on the normality and symmetry assumption.
  • I found the internal inconsistencies listed under data_issues but did not adjudicate them. None of them changes a requested key.
Files · 452964ee0fbbaa40fbbed2c87a54ca798ffd6a85
Menu

Theme