# F4-conf-01 — Extracting treatment effects and uncertainty from two published tables

## Question
Extract, exactly as printed, specified point estimates, standard errors (SEs), 95% confidence-interval (CI) bounds and sample counts from (a) Armendariz (2026), arXiv:2609.27732v1 (Swiss drug consumption rooms, staggered DiD, Tables 6–7) and (b) Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1 (Rhode Island RESEA RCT, Table 2). Attach a locator to each value. Derive one 95% CI from an SE and one SE from a CI. Identify any requested quantity that the source does not report. Nothing is re-estimated.

## Data and exclusions
Inputs read: `data/README.md` (10 lines), `data/sources/arxiv-2609.27732v1.txt` (134 lines) and `data/sources/arxiv-2606.14621v1.txt` (58 lines). These two extracts are the only evidence used. Values come from the tables named in each key: DCR Table 6 and Table 7, and RESEA Table 2. DCR Appendix Table A19 and RESEA Table 3 were read only for cross-checks and supplied no reported values.

## Method
`out/analysis.py` parses the pipe-delimited tables. It locates each cell by row label and column header, and it prints every matched source line. The standard error for a coefficient is taken from the parenthesised line directly below that coefficient. Significance stars are removed from the number and kept for the significance flag. Thousands separators are removed (e.g. "17,250" → 17250, "2,141" → 2141). Decimals are kept as printed. One limitation of the format: JSON number literals drop trailing zeros, so the printed 0.210 is stored as 0.21 and −0.340 as −0.34. The printed strings are shown in the table below and in the script output.

The script also runs consistency checks. It confirms that each star is consistent with a normal z-test using coef/SE, and that the derived wage SE reproduces the printed p-value. It compares duplicate reports of the same estimate across tables and compares the running text with the tables.

## Results
| key | value (as printed) | locator |
|---|---|---|
| dcr_att_5km | -0.431 (\*\*) | 2609.27732v1, Table 6, row "ATT", col "5 km" |
| dcr_att_5km_se | 0.210 | same cell, SE line |
| dcr_att_3km | -0.283 | Table 6, "ATT", "3 km" |
| dcr_att_3km_se | 0.279 | SE line |
| dcr_obs_5km | 17,250 → 17250 | Table 6, "Observations", "5 km" |
| dcr_narcotics_att_3km | 0.549 (\*\*\*) | Table 7, "Narcotics", "3 km" |
| dcr_narcotics_att_3km_se | 0.187 | SE line |
| dcr_fatal_att_1km | -1.207 (\*\*\*) | Table 7, "Drug-related (fatal)", "1 km" |
| dcr_fatal_att_1km_se | 0.137 | SE line |
| dcr_hiv_att_5km | -0.340 (no star) | Table 7, "HIV", "5 km" |
| dcr_hiv_att_5km_se | 0.294 | SE line |
| dcr_hiv_att_5km_p_value | **null — not reported** (only stars are printed) | — |
| dcr_hiv_att_5km_significant_10pct | false (no star) | Table 7 |
| dcr_att_5km_ci_low / high | **derived**: −0.843 / −0.019 | Table 6 (derived) |
| ri_wage_ate | $1,153 → 1153 | 2606.14621v1, Table 2, "Wage", "Average treatment effect estimate" |
| ri_wage_ate_ci_low / high | 166 / 2,141 → 2141 | same cell |
| ri_wage_ate_se_reported | false | Table 2 prints p and 95% CI only |
| ri_wage_ate_se_derived | **derived**: 503.8 | Table 2 (derived) |
| ri_control_mean_wage | $33,043 → 33043 | Table 2, "Wage", "Control" |
| ri_reemployment_ate_pp | 1.47 | Table 2, "Reemployment", ATE |
| ri_weeks_ui_ate | -1.99 | Table 2, "Weeks on UI", ATE |
| ri_weeks_ui_ate_ci_low / high | -2.24 / -1.75 | same cell |

Full locator strings are in `out/results.json`. In total 26 reported or derived items were assessed (`n_analysed`). Feasibility: **complete**. The single requested quantity that the source does not print, the HIV exact p-value, is correctly `null`.

## Uncertainty
- **Standard errors:** all DCR values (Tables 6, 7 and A19) are SEs clustered at the MedStat-region level, printed in parentheses. Significance is shown only by stars (*** p<0.01, ** p<0.05, * p<0.10), and no exact p-values are printed.
- **Confidence intervals:** all RESEA ATE values are given as two-sided p-values plus 95% CIs, with no SEs. According to the Table 2 caption, the ATE CIs follow "the methodology described in the main text", which is not part of the extract.
- **Derived DCR CI:** −0.431 ± 1.96 × 0.210 = [−0.843, −0.019]. This assumes the clustered-SE ATT is asymptotically normal and uses no small-sample or t correction.
  - With 50 treated regions and 608 controls, a t critical value would widen the interval only slightly.
  - Using 2.0 instead of 1.96 would give [−0.851, −0.011].
  - The interval excludes 0, which matches the printed \*\* (z-based p ≈ 0.040).
- **Derived RESEA SE:** (2141 − 166)/(2 × 1.96) = 503.8 USD. This assumes a symmetric Wald interval with a normal critical value; with N = 23,549 a t value is effectively the same.
  - The printed CI is almost symmetric around the estimate: its midpoint is 1153.5 against the printed 1153.
  - The implied p-value (0.022) equals the printed p=0.022, which supports the assumption.
- A normal-approximation p-value for the HIV 5 km ATT can be computed (≈0.25). It is **not** reported as a result, because the source does not print it and computing it would need a distributional assumption.

## Limitations and open issues
Data issues detected in the sources (also listed in `results.json` → `data_issues`):
1. **SE conflict for the same estimate:** the 5 km CS-DR ATT −0.431 has SE 0.210 in Table 6 but 0.218 in Appendix Table A19. Table 6 was used, as the keys specify. The derived CI would be [−0.858, −0.004] with 0.218.
2. **Control-group count conflict:** Tables 6–7 give 608 never-treated control regions, while the Table A19 note gives 640.
3. **Abstract vs. Table 7:** the DCR abstract says fatal hospitalisations "fall by 35%". Table 7 (5 km) gives −0.354 log points, which is about −29.8% (Section 6.1 itself says "approximately 30%"). The abstract appears to treat log points as percent.
4. **Text vs. Table 7 on HIV:** the DCR text calls the HIV estimates "close to zero", but the 5 km estimate is −0.340, which is insignificant but not small.
5. **Sign convention in RESEA text:** the running text gives the weeks-on-UI CI as positive magnitudes [1.75, 2.24]. Table 2's signed interval [−2.24, −1.75] was used.
6. **Rendering defect:** "p ¡ 0.001" appears in the RESEA text. RESEA Tables 2–3 were re-set by the extract editor, and their values are stated to be verbatim.

All star markers checked were consistent with coef/SE under a normal approximation. No values were taken from memory of the papers.