# F4-conf-01 — Treatment effects and uncertainty extracted from two published tables

## Question
I extracted printed point estimates, standard errors, confidence-interval bounds and sample counts from two source extracts:
- Armendariz (2026), arXiv:2609.27732v1: drug consumption rooms (DCRs) in Switzerland, staggered DiD (Callaway–Sant'Anna), Tables 6 and 7.
- Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1: Rhode Island RESEA randomised trial, Table 2.

I also derived two quantities the sources do not print, and identified one requested quantity that is not reported anywhere. Nothing was re-estimated.

## Data and exclusions
I read three files: `data/README.md` (10 lines), `arxiv-2609.27732v1.txt` (134 lines) and `arxiv-2606.14621v1.txt` (58 lines) (`n_input_rows`). Their SHA-256 hashes match the README and bundle (`input_sha256`; see `out/state/data_audit.md`). No items were excluded: all 25 requested items were assessed (`n_analysed`, `out/exclusions.csv`). The inputs have some internal inconsistencies, listed in `data_issues`. The most important is that the 5 km CS-DR ATT has SE (0.210) in Table 6 but (0.218) in Appendix Table A19. Per the key definitions, Table 6 is used.

## Method
`out/analysis.py` parses the table lines in the extracts and prints every source line it matched. Parsing rules:
- DCR tables: the standard error is on the line after the coefficient row, and significance comes from the stars.
- RESEA Table 2: each cell has the form "estimate; p; 95% CI: [lo, hi]".
- Numbers: "$" and thousands separators are stripped and signs kept, so "17,250" becomes 17250 and "$1,153" becomes 1153.

Each value carries a locator in `locators`. The two derived quantities use the normal critical value 1.96 (`derivation_notes`):
- the CI of the DCR 5 km ATT is estimate ± 1.96·SE;
- the SE of the RI wage ATE is (upper − lower)/(2·1.96).

There is no randomness (`random_seed` = null). Because this is an extraction task, the baseline is simply the verbatim reading of each named cell. The derivations are the only computations.

## Results
DCR study (log points):
- **Table 6, all specifications:**
  - 5 km ATT: −0.431 (`dcr_att_5km`), SE 0.210 (`dcr_att_5km_se`), starred ** in the source.
  - 3 km ATT: −0.283 (`dcr_att_3km`), SE 0.279 (`dcr_att_3km_se`).
  - Observations at 5 km: 17250 (`dcr_obs_5km`).
- **Table 7, sub-outcomes:**
  - Narcotics at 3 km: 0.549 (`dcr_narcotics_att_3km`), SE 0.187 (`dcr_narcotics_att_3km_se`).
  - Drug-related (fatal) at 1 km: −1.207 (`dcr_fatal_att_1km`), SE 0.137 (`dcr_fatal_att_1km_se`).
  - HIV at 5 km: −0.340 (`dcr_hiv_att_5km`), SE 0.294 (`dcr_hiv_att_5km_se`). It has no star, so the source does not mark it as significant at 10% (`dcr_hiv_att_5km_significant_10pct` = false).
  - The exact p-value for HIV at 5 km is **not reported**: Table 7 prints only stars (`dcr_hiv_att_5km_p_value` = null).
- **Derived:** the 95% CI of the Table 6 5 km ATT is [−0.8426, −0.0194] (`dcr_att_5km_ci_low`, `dcr_att_5km_ci_high`).

RESEA study (Table 2):
- **Wage ATE:** $1153 per year (`ri_wage_ate`), printed 95% CI [166, 2141] (`ri_wage_ate_ci_low`, `ri_wage_ate_ci_high`).
  - No standard error is printed (`ri_wage_ate_se_reported` = false).
  - Derived SE: 503.83 (`ri_wage_ate_se_derived`).
- **Control-group mean wage:** $33043 (`ri_control_mean_wage`).
- **Reemployment ATE:** 1.47 pp (`ri_reemployment_ate_pp`).
- **Weeks on UI ATE:** −1.99 weeks (`ri_weeks_ui_ate`), printed 95% CI [−2.24, −1.75] (`ri_weeks_ui_ate_ci_low`, `ri_weeks_ui_ate_ci_high`).

## Uncertainty
- **DCR (Tables 6 and 7):** uncertainty is printed as **standard errors** clustered at the MedStat region level, plus significance stars. No CIs or exact p-values are printed for these cells.
- **RESEA (Table 2):** uncertainty is printed as **two-sided p-values and 95% confidence intervals**, with no standard errors.

What the derived quantities assume:
- **DCR 5 km CI** (`dcr_att_5km_ci_low`, `dcr_att_5km_ci_high`): the ATT estimator is asymptotically normal, and the CI is a Wald interval with z = 1.96.
  - Using 2.0 instead gives [−0.851, −0.011] (`sensitivity.dcr_att_5km_ci_z2.0`).
  - Using the A19 SE gives [−0.8583, −0.0037] (`sensitivity.dcr_att_5km_ci_using_tableA19_se`).
  - In all three versions the interval excludes zero, which matches the ** star.
- **RI wage SE** (`ri_wage_ate_se_derived`): the printed CI is a symmetric normal-approximation interval. A t critical value would differ negligibly at N = 23,549.
  - The CI is close to symmetric around 1153: the half-widths imply SEs of 503.57 and 504.08 (`sensitivity.ri_wage_ate_se_from_lower_half`, `sensitivity.ri_wage_ate_se_from_upper_half`).
  - The normal p-value implied by the derived SE is 0.0221 (`sensitivity.ri_wage_normal_p_from_derived_se`), which matches the printed p=0.022.
  - With 2.0 the SE would be 493.75 (`sensitivity.ri_wage_ate_se_z2.0`).
- **Stars check:** for all 22 starred-or-unstarred DCR coefficient/SE pairs in Tables 6, 7 and A19, the printed stars agree with normal-approximation p-values computed from the printed SEs (`sensitivity.significance_star_mismatches` is empty).

## Limitations and open issues
- **Decimals in JSON:** JSON numbers cannot keep trailing zeros, so the printed SE "0.210" is stored as 0.21 (`dcr_att_5km_se`). The printed form appears in the locators/source lines that the script prints.
- **Exact p-value:** the HIV 5 km p-value cannot be obtained from the source. A normal-approximation value (`sensitivity.dcr_hiv_att_5km_normal_p_approx`, 0.2475) is derived, not printed, and is deliberately not used for the key.
- **Inconsistencies inside the sources** (`data_issues`), none of which affect the keys:
  - the Table 6 vs A19 SE for the same 5 km estimate (0.210 vs 0.218);
  - 608 vs 640 control regions;
  - the abstract's "35%" fall in fatal hospitalisations vs Section 6.1's "approximately 30%";
  - the RESEA text printing the weeks-on-UI CI as positive magnitudes;
  - an encoding defect ("p ¡ 0.001").
- **Feasibility:** the task is complete (`feasibility` = "complete"). The one missing quantity was designed to be recognised as not reported.