1# F4-conf-03 — Harmonising effect sizes reported under incompatible definitions23## Question4Five effects from four arXiv papers are reported under different definitions: log-point ATTs, a percentage-point ATE, an elasticity, and a linear-probability coefficient. For each one I re-express the printed point estimate on the target definition given in `data/effects.json`, using only the quantities printed in the cited extract. Where an item cannot be converted, I name the quantity that is missing.56## Data and exclusions7Inputs: `data/effects.json` (5 items) and the four extracts under `data/sources/`: Armendariz 2026 (134 lines), Li et al. 2026 (58 lines), Kanayama et al. 2025/26 (66 lines) and Tossou 2025 (91 lines), plus `data/README.md` (16 lines). Only the printed quantities are used; I added nothing from memory or from the full papers. Nothing was excluded.89## Method10I applied the conventions in TASK.md. `out/analysis.py` finds each value by pattern-matching its table row in the extract and prints the source line it uses.1112| id | printed input(s) (locator) | transformation | harmonised value |13|---|---|---|---|14| E1 | ATT = -0.431 (arxiv-2609.27732v1, Table 6, row ATT, col 5 km) | 100·(exp(-0.431)−1) | **-35.01 %** of the counterfactual rate (matches printed "% change" -35.0%) |15| E2 | ATE = 1.47 pp; control mean = 75.0% (arxiv-2606.14621v1, Table 2, row Reemployment, cols "Average treatment effect estimate" and "Control") | 100·1.47/75.0 | **+1.96 %** of the control mean |16| E3 | η = -0.388; % MW changes = 0.047 (arxiv-2505.04555v3, Table 2, rows "Employment elasticity w.r.t MW" and "% MW changes") | 100·(-0.388)·0.047 | **-1.82 %** employment (unrounded -1.8236%) |17| E4 | ATT = -0.354 (arxiv-2609.27732v1, Table 7, row "Drug-related (fatal)", col 5 km) | R0·(exp(-0.354)−1), where R0 is the counterfactual level | **not harmonisable** (null) |18| E5 | β = -0.0962 (arxiv-2510.03658v1, Table 3, row Treatment, col (4)) | 100·(-0.0962) | **-9.62 pp** |1920E4: the target is an absolute change in hospitalisations per 100,000 per year. Converting a log-point effect to that unit needs the baseline (counterfactual or pre-treatment) level of the fatal drug-related hospitalisation rate in the treated 5 km regions. The extract does not print that level anywhere; a regex search of the extract finds no numeric rate per 100,000. **Missing quantity: the baseline fatal drug-related hospitalisation rate per 100,000 inhabitants per year in the treated regions.** For reference only, the relative change implied by -0.354 is 100·(exp(-0.354)−1) = -29.81%. That number is not the target definition.2122## Results23- Harmonised: 4 of 5 (E1 -35.01%, E2 +1.96%, E3 -1.82%, E5 -9.62 pp). Not harmonisable: 1 (E4).24- Feasibility: **complete**, in the sense defined in TASK.md: every item has either a value or a justified statement that it cannot be converted.25- Consistency with the authors' own statements: E1 matches the printed -35.0%. E3 (-1.8%) agrees with the abstract's "2% employment decline". E2 (1.96%) is the relative counterpart of the 1.47 pp effect; the paper does not print this figure.2627Problems found in the sources (also listed in `data_issues` in results.json):281. **Armendariz abstract vs Table 7.** The abstract says fatal hospitalisations "fall by 35%". The Table 7 5 km coefficient (-0.354) implies -29.8%, and Section 6.1 itself says "approximately 30%".292. **Armendariz Table 6 vs Table A19.** The same ATT, -0.431, has SE 0.210 in Table 6 and 0.218 in Table A19. The control count is 608 regions in Tables 6/7 but 640 in the A19 note. The text also says "five" sub-categories, while the Figure 7 note says four.303. **Kanayama Table 2.** The note says "about 4%" minimum-wage increase, but the table prints 0.047. I used 0.047, as the target definition requires. Using 0.04 would give -1.55%.314. **Tossou.** The headline -0.0855 / 0.455 refers to a Table 2 that is not in the extract. The printed "% Change" of -0.3685 differs from -0.0962/0.261 = -0.3686 in the last digit (rounding).325. **Li et al.** The abstract rounds the effect to 1.5 pp, versus 1.47 pp in Table 2.3334## Uncertainty35No standard errors or confidence intervals were propagated; all values are point estimates only.36- **Exact conversions:**37 - E1: exp(b)−1 is the exact relative change implied by a log-point coefficient.38 - E5: a pure change of units, 0–1 scale to pp.39 - E2: exact under the stated definition. Note, though, that the denominator is the raw control-group mean (75.0%), not a block-adjusted counterfactual.40- **Approximate conversion:** E3. Multiplying the elasticity by the proportional change is a first-order (linear) approximation. The 0.047 is itself an average across 47 prefecture-level changes.41- The printed inputs are rounded (e.g. 75.0%, 0.047), which limits the precision of the harmonised values to about 2–3 significant digits.4243## Limitations and open issues44- E4 cannot be computed from the extract. Even with the baseline rate, converting via R0·(exp(b)−1) would need the level of the rate, not the mean of the log rate. These differ (Jensen's inequality), so the baseline would have to be the counterfactual mean rate in the treated regions.45- E2 uses the unadjusted control mean as the denominator. The ATE is block-adjusted, so the ratio mixes an adjusted numerator with an unadjusted denominator; this follows the stated target definition.46- E3 depends on which minimum-wage change is used: 0.047 as printed in the table, or "about 4%" as in the note.47- The source problems listed above were flagged but not resolved; I cannot resolve them from the extracts alone.48