# F4-conf-03 — Harmonising effect sizes reported under incompatible definitions

# Question

Five treatment effects from four arXiv extracts are printed under different definitions: log points, percentage points, an elasticity, and a linear-probability coefficient. For each one, I re-express the printed point estimate on the target definition given in data/effects.json. I use only quantities printed in the cited extract and an explicit transformation. Where the target cannot be reached, I say so and name the quantity that is missing.

# Data and exclusions

• Inputs (n_input_rows): effects.json (5 items), README.md (16 lines), and four source extracts: arxiv-2609.27732v1 (134 lines), arxiv-2606.14621v1 (58), arxiv-2505.04555v3 (66) and arxiv-2510.03658v1 (91). All SHA-256 hashes match data/README.md (see out/state/data_audit.md).
• No exclusions: all 5 items were assessed (n_analysed = 5; out/exclusions.csv: 5 → 5).
• Problems found in the inputs are listed in data_issues (7 entries). The ones that matter here are:
  • The Armendariz abstract says fatal hospitalisations "fall by 35%". Table 7 implies −29.8%, and Section 6.1 says "approximately 30%".
  • Kanayama et al.'s note to Table 2 says "about 4%", but the table row prints 0.047.
  • The Armendariz extract prints no level of the fatal hospitalisation rate.
  • Two smaller Armendariz discrepancies have no effect on the results: the standard errors and control-region counts in Table A19 differ from those in Table 6, and A19's % change row disagrees with exp(b) − 1 by 0.1 in the last digit.

# Method

I applied the conventions stated in TASK.md. Each item's formula and the printed inputs are recorded in harmonised.<id>.transformation, and line-level locators are in harmonised.<id>.locator. out/analysis.py parses the numbers directly from the table rows and prints every source line it uses.

• E1: 100·(exp(b) − 1), with b = −0.431 (Table 6, ATT, 5 km).
• E2: 100·Δ/p, with Δ = 1.47 pp (Table 2, Reemployment, Average treatment effect estimate) and p = 75.0% (Table 2, Control).
• E3: 100·η·g, with η = −0.388 (Table 2, Employment elasticity w.r.t MW) and g = 0.047 (Table 2, "% MW changes"; read as a proportion, consistent with the note's "about 4%").
• E4: the target would be Y0·(exp(b) − 1), with b = −0.354 (Table 7, Drug-related (fatal), 5 km). Y0 is the counterfactual fatal rate per 100,000 per year.
• E5: 100·β, with β = −0.0962 (Table 3, Treatment, column (4)).

As a baseline, the untransformed printed values in their original units are kept under baseline_reported.

# Results

• Four items were harmonised (n_harmonised = 4) and one was not (n_not_harmonisable = 1).
• E4 cannot be expressed as an absolute change. The extract prints only log-point ATTs, standard errors and region counts; it gives no mean, baseline or counterfactual level of the fatal drug-related hospitalisation rate per 100,000. The missing quantity is that level for the 5 km treated regions (harmonised.E4.missing_quantity). The relative change that can be computed, −29.8133% (baseline_reported.E4.computable_relative_change_percent), is not the target definition.
• The task as a whole is feasibility = "complete": every item has either a value or a justified statement of impossibility.

# Uncertainty

• No standard errors or confidence intervals were propagated (none were required). The values are point estimates only.
• E5 is exact: it is a pure rescaling (×100).
• E1 is exact given the printed b. However, b is rounded to 3 decimals, so only about 3 significant digits are meaningful (≈ −35.0%). It is also the exp-transform of an average log effect, not necessarily the percent change in the mean rate.
• E2 is an exact ratio of printed numbers, but it is approximate as an estimand. The denominator is the raw aggregate control mean (75.0%, one decimal), while the ATE is adjusted for randomisation week. The printed numbers do not give an exact counterfactual level.
• E3 is a first-order (linear) approximation, η·g. It also depends on which MW change is used: with the note's "about 4%", the result would be about −1.55% instead of −1.82%. I used the table's 0.047, as the task instructs.

# Limitations and open issues

• E4 is blocked by a missing quantity (see Results). I did not substitute any external or remembered baseline rate. The time unit ("per year") is also not stated explicitly in the extract.
• The extracts contain internal inconsistencies (data_issues), most importantly the 35% vs ≈30% figure for the fatal outcome in the Armendariz abstract. I used the table values throughout.
• The harmonised effects come from different populations, estimators and estimands (ATT, ITT/ATE, elasticity, DiD-LPM). Putting them on a common scale does not make them directly comparable as causal quantities.