# F4-conf-01 — Extracting treatment effects and their uncertainty from two published tables

## Question
The task is to pull specified treatment effects, standard errors (SEs), 95% confidence-interval (CI) bounds and counts out of two arXiv extracts, copying them exactly as printed. The sources are Armendariz (2026), arXiv:2609.27732v1, a staggered DiD study of Swiss drug consumption rooms (DCRs), using Tables 6 and 7; and Li, Pearson-Merkowitz & Yokum (2026), arXiv:2606.14621v1, an RCT of Rhode Island's RESEA programme, using Table 2. Each value gets a locator. I also derive two quantities the sources do not print, state the convention used for each, and flag requested quantities that are not reported. Nothing is re-estimated.

## Data and exclusions
- Inputs: `data/sources/arxiv-2609.27732v1.txt` (134 lines), `data/sources/arxiv-2606.14621v1.txt` (58 lines) and `data/README.md` (10 lines) (`n_input_rows`). The SHA-256 of each extract matches the one listed in the README (`out/state/data_audit.md`).
- Nothing was filtered out. All 25 requested value items were found and assessed (`n_analysed` = 25; `out/exclusions.csv` shows 0 removed).
- Information rules: only the two extracts were used. Two other tables, DCR Table A19 and RESEA Table 3, also print some of the same numbers. I used them only for cross-checks, never as the source of a key.
- Problems found in the inputs (`data_issues`, 6 entries):
  1. Table 6 gives the 5 km CS-DR ATT (-0.431**) an SE of 0.210. Table A19 prints the same estimate with SE 0.218 (`check_dcr_a19_se_5km`). The key names Table 6, so I used Table 6.
  2. The number of control regions is inconsistent: 608 in Tables 6 and 7, but 640 in the Table A19 note. The printed observation counts are exactly 25 × (treated + 640) and are not whole multiples of (treated + 608). I report the observation count as printed.
  3. The DCR narrative contradicts itself. The abstract says fatal hospitalisations fall by 35%, while Section 6.1 says about 30% at 5 km. The text also calls the HIV estimates "close to zero", although the 5 km estimate is -0.340.
  4. The RESEA narrative gives the weeks-on-UI CI as unsigned magnitudes. I used the signed values from Table 2.
  5. There is a rendering defect, "p ¡ 0.001", in the RESEA text.
  6. The RESEA tables label the same numbers differently: "ATE" in Table 2 and ITT in Table 3. The extract does not describe how the CIs were built.

## Method
`out/analysis.py` parses the " | "-separated table lines and attaches each SE line in parentheses to the coefficient row above it. It removes `$`, thousands separators, stars and units, keeps the sign, and writes each printed literal into the JSON so that trailing zeros survive (for example 0.210 and -0.340). It also prints every source line it matched. Significance at the 10% level is read from the stars: any of `*`, `**` or `***` means p < 0.10.

The two derived quantities use the normal critical value 1.96 (`derivation_notes`):
- the DCR 95% CI is ATT ± 1.96 × SE;
- the RESEA wage SE is (upper − lower)/(2 × 1.96).

As a sensitivity check I repeated both with a critical value of 2.0 (`baseline_*` keys). The script involves no randomness (`random_seed` = null).

## Results
**DCR study (arxiv-2609.27732v1), log points**

| Quantity | Estimate | SE | Keys |
|---|---|---|---|
| Table 6 ATT, 5 km | -0.431 (\*\*) | 0.210 | `dcr_att_5km`, `dcr_att_5km_se` |
| Table 6 ATT, 3 km | -0.283 | 0.279 | `dcr_att_3km`, `dcr_att_3km_se` |
| Table 7 Narcotics, 3 km | 0.549 (\*\*\*) | 0.187 | `dcr_narcotics_att_3km`, `dcr_narcotics_att_3km_se` |
| Table 7 Drug-related (fatal), 1 km | -1.207 (\*\*\*) | 0.137 | `dcr_fatal_att_1km`, `dcr_fatal_att_1km_se` |
| Table 7 HIV, 5 km | -0.340 (no star) | 0.294 | `dcr_hiv_att_5km`, `dcr_hiv_att_5km_se` |

- Table 6 reports 17250 observations for the 5 km buffer (`dcr_obs_5km`).
- The source does not mark the HIV 5 km ATT as significant at the 10% level (`dcr_hiv_att_5km_significant_10pct` = false).
- Its exact p-value is not printed, so it is reported as null (`dcr_hiv_att_5km_p_value`). The tables give significance stars only.
- Derived 95% CI for the 5 km ATT: [-0.8426, -0.0194] (`dcr_att_5km_ci_low`, `dcr_att_5km_ci_high`).

**RESEA study (arxiv-2606.14621v1), Table 2**
- Wage ATE: $1153 per year, 95% CI [166, 2141] (`ri_wage_ate`, `ri_wage_ate_ci_low`, `ri_wage_ate_ci_high`).
- No SE is printed for the wage ATE (`ri_wage_ate_se_reported` = false). The SE implied by the CI is 503.8265 (`ri_wage_ate_se_derived`).
- Control-group mean wage: $33043 (`ri_control_mean_wage`).
- Reemployment ATE: 1.47 pp (`ri_reemployment_ate_pp`).
- Weeks-on-UI ATE: -1.99 weeks, 95% CI [-2.24, -1.75] (`ri_weeks_ui_ate`, `ri_weeks_ui_ate_ci_low`, `ri_weeks_ui_ate_ci_high`).

Every locator is in `locators`. Feasibility is "complete" (`feasibility`).

## Uncertainty
- **DCR values are standard errors.** In Tables 6 and 7 the numbers in parentheses are SEs clustered at the MedStat region level. The stars mark p < 0.01, 0.05 and 0.10.
- **RESEA values are confidence intervals and p-values.** Table 2 prints two-sided p-values and 95% CIs, and no SEs.
- **The derived DCR CI assumes normality.** It assumes the ATT is approximately normally distributed with the printed SE, and uses a critical value of 1.96. It is consistent with the source: it excludes 0, and the implied normal p is 0.0401 (`check_dcr_att_5km_p_implied_normal`), which fits the printed `**`. With a critical value of 2.0 the CI would be [-0.851, -0.011] (`baseline_dcr_att_5km_ci_low_z2`, `baseline_dcr_att_5km_ci_high_z2`).
- **The derived wage SE assumes a symmetric normal-approximation CI.** It assumes the printed CI is estimate ± 1.96 × SE. Two checks support this:
  - the CI midpoint is 1153.5 (`check_ri_wage_ci_midpoint`);
  - the p-value implied by the derived SE is 0.0221 (`check_ri_wage_p_implied_by_derived_se`), close to the printed p = 0.022.

  With a critical value of 2.0 the SE would be 493.75 (`baseline_ri_wage_ate_se_derived_z2`). The extract does not describe how the CI was actually built. Because the CI bounds are rounded to whole dollars, the derived SE is only accurate to about ±0.3.
- **The HIV 5 km estimate is not significant.** Its |ATT/SE| is 1.1565 (`check_dcr_hiv_5km_abs_z`), below 1.645, which matches the absence of a star.

## Limitations and open issues
- **The SE of the 5 km ATT is ambiguous.** Tables 6 and A19 print different SEs (0.210 vs 0.218) for the same estimate. I followed the key's definition and used Table 6, but the source itself is inconsistent. With 0.218 the derived CI would be wider.
- **The number of control regions is uncertain.** The count is printed as 608, but 640 fits the observation counts. Which one is right is unresolved in the extract.
- **The exact HIV p-value cannot be given.** It is not reported, so it is null. Recovering it from the SE would require a distributional assumption, and the key asks only for a printed value.
- **The derived quantities rest on convention.** Both use the normal 1.96 critical value. A t critical value could not be used because the degrees of freedom are not reported (for DCR, the clusters are not stated consistently).
- **The 3.5% wage figure was a check, not an extracted value.** The ratio of the wage ATE to the control mean matches the narrative's 3.5% (`out/state/checks.md`, check 15).