# F4-conf-03 — Harmonising effect sizes reported under incompatible definitions

# Question

Five treatment effects from four open-access arXiv extracts are reported under different definitions (log-point ATT, percentage-point ATE, elasticity, linear-probability coefficient). The task is to re-express each printed point estimate on the target definition given in data/effects.json, using only numbers printed in the cited extract and a stated transformation. Where that is impossible, the report says so and names the quantity that is missing.

# Data and exclusions

Inputs: data/effects.json (5 items), data/README.md and the four extracts under data/sources/. Line counts are in n_input_rows. All SHA-256 hashes match the README (see out/state/data_audit.md). All 5 items were assessed (n_analysed = 5) and none were excluded (out/exclusions.csv). Problems found in the inputs are listed in data_issues. The main ones:

• the DCR abstract's "35%" fall in fatal hospitalisations disagrees with Table 7 (−29.8%) and with Section 6.1 ("approximately 30%");
• the MW Table 2 note says "about 4%" but the table prints 0.047;
• small rounding mismatches in the DCR Table A19 and Benin Table 3 "% change" rows;
• the Benin text refers to a "table 2" estimate that is not in the extract;
• the RESEA control mean is a raw mean, while the ATE is adjusted for randomisation week.

# Method

out/analysis.py finds each required table cell by regular expression in the extract, prints the source line it uses, and applies the conventions in TASK.md:

• E1: log points to relative %, 100·(exp(b)−1), with b = −0.431 (DCR Table 6, ATT, 5 km).
• E2: pp to relative % of the control mean, 100·Δ/p, with Δ = 1.47 pp and p = 75.0% (RESEA Table 2, Reemployment).
• E3: elasticity times the printed proportional MW change, 100·η·g, with η = −0.388 and g = 0.047 (MW Table 2).
• E4: an absolute change per 100,000 would be R0·(exp(b)−1) with b = −0.354 (DCR Table 7, fatal, 5 km). This needs the baseline rate R0.
• E5: 0–1 probability to percentage points, 100·β, with β = −0.0962 (Benin Table 3, col. 4).

No randomness is involved (random_seed = null). The printed estimates are recorded under baseline_reported_estimates, which serves as the "no transformation" baseline.

# Results

• E1 (harmonised.E1.value): −35.0141% change in the drug-related hospitalisation rate. This agrees with the paper's printed "% change" of −35.0%.
• E2 (harmonised.E2.value): +1.96% relative to the control-group reemployment rate.
• E3 (harmonised.E3.value): −1.8236% change in employment for the 4.7% minimum-wage rise printed in the table.
• E4 (harmonised.E4): not harmonisable (value null). The extract never prints a baseline level (counterfactual or pre-opening mean) of the fatal drug-related hospitalisation rate per 100,000 per year, so the absolute change cannot be computed (harmonised.E4.missing_quantity). From the printed numbers, only the relative change of −29.8% can be reached; this appears in harmonised.E4.transformation.
• E5 (harmonised.E5.value): −9.62 percentage points.
• Counts: 4 items harmonised (n_harmonised) and 1 not harmonisable (n_not_harmonisable). Every item has either a value or a justified statement of impossibility, so feasibility = "complete".

# Uncertainty

No standard errors or confidence intervals were propagated; all values are point estimates only. How exact each conversion is:

• E5 is an exact rescaling.
• E1 is exact under the log-point convention, given that the coefficient is printed to 3 decimals. Rounding of the printed b to ±0.0005 moves the result by about ±0.03 pp.
• E2 is exact arithmetic. However, it is an approximation to a counterfactual relative effect, because the raw control mean (75.0%) stands in for the counterfactual level of the week-adjusted ATE.
• E3 is a first-order (linear-elasticity) approximation. It also depends on using g = 0.047 rather than the note's "about 4%".
• E4 has no value.

# Limitations and open issues

• E4 cannot be expressed in absolute units without the baseline fatal hospitalisation rate per 100,000 per year for the 5 km treated regions, which this extract does not print. No external or remembered value was substituted.
• The source inconsistencies listed in data_issues were recorded but do not change any harmonised value, because each item uses the specific table cell named in effects.json.
• Values are kept to 4 decimals. The printed inputs keep their original precision.