# F4-conf-03 — Harmonising effect sizes reported under incompatible definitions

# Question

Re-express five printed treatment effects (data/effects.json, E1–E5) on each item's target definition, using only quantities printed in the cited extract and an explicit transformation. Where the target cannot be reached, say so and name the missing quantity.

# Data and exclusions

Inputs: data/effects.json (5 items), data/README.md (16 lines), and four source extracts (arxiv-2609.27732v1: 134 lines; arxiv-2606.14621v1: 58; arxiv-2505.04555v3: 66; arxiv-2510.03658v1: 91). Only the extracts were used as evidence. Numbers that the text attributes to tables not in the extract (for example Tossou's "Table 2", −0.0855 with mean 0.455) were not used. Nothing was excluded otherwise.

# Method

out/analysis.py finds each required table row with a regex, prints the source line, parses the printed values, and applies the convention given in TASK.md:

log points b → 100·(exp(b) − 1) %;
pp effect Δ with control level p → 100·Δ/p %;
elasticity η with proportional treatment change g → 100·η·g %;
an LPM coefficient on a 0–1 scale → ×100 percentage points.

The script also runs consistency checks against the "% change" rows the authors printed. Values are rounded to 2 decimals; the transformation strings keep 4 decimals.

# Results

id	printed input(s) (locator)	transformation	harmonised value
E1	ATT = −0.431** (arxiv-2609.27732v1, Table 6, row ATT, col 5 km)	100·(exp(−0.431) − 1)	−35.01 % (matches printed "% change" −35.0%)
E2	ATE = 1.47 pp; control mean = 75.0% (arxiv-2606.14621v1, Table 2, row Reemployment, cols "Average treatment effect estimate" and "Control")	100·1.47/75.0	+1.96 % relative to the control mean
E3	η = −0.388 (Table 2, row "Employment elasticity w.r.t MW"); g = 0.047 (Table 2, row "% MW changes"), arxiv-2505.04555v3	100·(−0.388)·0.047	−1.82 % employment
E4	ATT = −0.354** (arxiv-2609.27732v1, Table 7, row "Drug-related (fatal)", col 5 km)	would need Y0·(exp(−0.354) − 1)	not harmonisable (null)
E5	β = −0.0962 (arxiv-2510.03658v1, Table 3, row Treatment, col (4))	100·(−0.0962)	−9.62 pp

4 items harmonised; 1 not harmonisable. The task is complete in the sense TASK.md defines: every item has either a value or a justified statement of impossibility.

E4: why it cannot be harmonised. An absolute change in hospitalisations per 100,000 inhabitants requires the baseline (counterfactual or pre-treatment) level of the fatal drug-related hospitalisation rate in the treated 5 km regions. The extract does not print that level, and it does not state the time unit of the rate (per year). The only quantity that can be computed is the relative change, 100·(exp(−0.354) − 1) = −29.81 %, which is not the target.

Data issues detected (full list in results.json → data_issues):

Fatal hospitalisations (DCR abstract). The abstract says fatal hospitalisations "fall by 35%". Table 7 implies −29.8%, and Section 6.1 says "approximately 30%". The 35% appears to be the overall-outcome figure.
DCR Tables 6 and A19. The standard error for the same −0.431 ATT differs between Table 6 (0.210) and Table A19 (0.218). The A19 note says 640 control regions, while the tables say 608. Neither affects any point estimate used here.
Minimum-wage size (Kanayama et al.). The Table 2 note says "about 4%", but the table row prints 0.047. I used 0.047 as the target definition instructs. With g = 0.04 the result would be −1.55 %.
Kanayama et al. internal check (consistent). (missing + excess)/g = (−0.03 + 0.012)/0.047 = −0.383, which agrees with the printed −0.388 up to rounding.
Tossou. Table 3 col (4) prints a "% Change" of −0.3685, against −0.0962/0.261 = −0.3686; the difference is rounding of the printed mean. The headline 0.0855 / 18.79% figures refer to a table not in the extract.
RESEA control mean. The control mean (75.0%) is a raw group mean, not a block-adjusted counterfactual (see Uncertainty).
README. data/README.md mentions a "London Table 3 header" that matches no extract in this task. It was ignored.

# Uncertainty

No standard errors or confidence intervals were propagated. All values are point-estimate transformations.

Exact rescalings:
- E5: ×100.
- E1: exp transform. It is exact given the convention, but exp(E[b]) ≠ E[exp(b)], so it is the transform of the point estimate, not an unbiased estimate of the percent effect.
Approximate:
- E3: the elasticity is applied linearly to a 4.7% change, which is a first-order approximation. The input g is itself printed only to 3 decimals, and the note's "about 4%" disagrees with it.
- E2: the denominator is the raw control-group mean (75.0%, printed to 1 decimal). The ATE is block-adjusted, but no block-adjusted counterfactual mean is printed, so the ratio mixes an adjusted numerator with an unadjusted denominator.
Rounding of the printed inputs limits precision to about ±0.01 in the last reported decimal for E1, E2 and E3.

# Limitations and open issues

E4 cannot be harmonised without the baseline fatal hospitalisation rate per 100,000 (and its time unit), which is not in the extract.
The harmonised values inherit any errors in the printed tables. Several internal inconsistencies were found and documented, but they cannot be resolved from the extracts alone.
The effects differ in nature (ATT in log points, ITT in pp, a six-month elasticity, an LPM DiD coefficient), so a common unit does not make them substantively comparable.

spb/wp29_uqo