# F4-conf-03 — Harmonising effect sizes reported under incompatible definitions

# Question

For the five effects in data/effects.json, re-express each printed point estimate on its target definition, using only quantities printed in the cited arXiv extract and an explicit transformation (TASK.md conventions), or state that it cannot be done and name the quantity that is missing.

# Data and exclusions

Inputs: data/effects.json (5 items), data/README.md and the four source extracts under data/sources/. Line counts are in n_input_rows. The SHA-256 of every extract matches the value in data/README.md (out/state/data_audit.md). All 5 items were assessed (n_analysed = 5). No item was excluded (out/exclusions.csv: 5 − 0 = 5). Problems found in the sources are listed in data_issues and summarised under Limitations.

# Method

Every printed input is read from the extract by out/analysis.py, which also prints the source line it comes from. Transformations (exact formulas in harmonised.<id>.transformation):

• E1 (log-point ATT → relative % change): 100·(exp(b) − 1), with b = −0.431 (Table 6, ATT, 5 km).
• E2 (pp → % of control mean): 100·Δ/p, with Δ = 1.47 pp (Table 2 ATE) and p = 75.0% (Table 2 control mean).
• E3 (elasticity → % employment change for the observed MW rise): 100·η·g, with η = −0.388 and g = 0.047 (Table 2, row "% MW changes").
• E4 (log-point ATT → absolute change per 100,000 per year): this needs R0·(exp(b) − 1), where R0 is the counterfactual level of the fatal rate. R0 is not printed in the extract.
• E5 (LPM coefficient on a 0–1 scale → percentage points): 100·β, with β = −0.0962 (Table 3, Treatment, column (4)).

As a baseline check, each conversion was compared with the conversions the authors themselves print (out/state/checks.md).

# Results

• E1 matches the printed "% change" row (−35.0%) to within rounding (harmonised.E1.transformation).
• E3 is consistent with the table's missing + excess jobs (−0.03 + 0.012 = −0.018) and with the abstract's "2% employment decline" (harmonised.E3.value).
• E4 cannot be computed. The missing quantity is the counterfactual, pre-treatment or control-group mean level of the fatal drug-related hospitalisation rate per 100,000 inhabitants per year in the treated 5 km regions (harmonised.E4.missing_quantity). For information only, the relative change implied by b = −0.354 is exp(−0.354) − 1 = −0.2981 (stated in harmonised.E4.transformation). No external value was used in its place.
• Counts: n_harmonised = 4 and n_not_harmonisable = 1. Every item has either a value or a justified statement that it cannot be harmonised, so feasibility = "complete" (feasibility_reason = null).

# Uncertainty

No standard errors or confidence intervals were propagated. Only point estimates were harmonised, as the task specifies.

• Exact conversions (given the stated conventions): E1, because 100(exp(b) − 1) is the exact mapping from log points to percent, and E5, because ×100 is a pure rescaling.
• E2 is exact arithmetic, but its meaning is approximate. It treats the raw aggregate control mean (75.0%) as the counterfactual level, whereas the ATE is adjusted for randomisation week.
• E3 is a first-order (linear) approximation, η·g. It also inherits whatever approximation the authors used to build η.
• Printed inputs are rounded (for example η to 3 decimals and g to 3 decimals). The extra digits in E3 (−1.8236) are therefore arithmetic, not precision.

# Limitations and open issues

• E4 cannot be harmonised from the extract. No level of the fatal hospitalisation rate is printed anywhere in it (regex search plus a full reading). An absolute effect would require that baseline level.

• Source inconsistencies (DCR paper):

  • The abstract says fatal hospitalisations "fall by 35%". Section 6.1 says about 30%, and Table 7 implies −29.8%.
  • The CS-DR standard error is 0.210 in Table 6 but 0.218 in Table A19.
  • The number of control regions is 608 in the tables but 640 in the A19 note.

  None of these affects the harmonised values (data_issues).

• MW paper: the table note says "about 4%" minimum-wage increase while the table row prints 0.047. Following the target definition, I used the table value. Using 4% would give −1.55% instead (data_issues).

• RESEA paper: the control mean is unadjusted, whereas the ATE is block-adjusted (see Uncertainty).

• Benin paper: the text cites a "table 2" estimate that is not in the extract. The printed % Change (−0.3685) differs slightly from −0.0962/0.261 = −0.3686 because of rounding. Neither affects E5.