# F4-conf-03 — Harmonising effect sizes reported under incompatible definitions

# Question

The task gives five treatment effects from four arXiv extracts, each printed under its own definition (log points, percentage points, an elasticity, a linear-probability coefficient). For each one, I re-express the printed point estimate on the target definition named in data/effects.json. I use only numbers printed in the cited extract, together with a stated transformation. Where an item cannot be converted, I say so and name the quantity that is missing.

# Data and exclusions

• Inputs read: data/effects.json (5 items), data/README.md and the four source extracts. Line counts are in n_input_rows. All SHA-256 hashes match data/README.md (see out/state/data_audit.md).

• No item was excluded (out/exclusions.csv). All 5 items were assessed (n_analysed = 5).

• data_issues lists inconsistencies inside the sources and rendering defects. The main ones:

  • DCR paper. The abstract says fatal hospitalisations fall by 35%. Section 6.1 says about 30%, and the Table 7 coefficient implies −29.8%.
  • DCR paper. The standard error of the 5 km ATT is printed as 0.210 in Table 6 but 0.218 in Table A19. The A19 note gives 640 control regions, while the tables give 608.
  • DCR paper. Three "% change" cells are off by 0.1 from 100·(exp(b)−1).
  • Minimum-wage paper. Table 2 prints "% MW changes" = 0.047, but the table note says "about 4%".
  • Benin paper. The abstract's 0.0855 refers to a Table 2 that is not in the extract.
  • RESEA paper. The control mean in Table 2 is a raw group mean and is not adjusted for the randomisation blocks.

  None of these issues changes a harmonised value.

# Method

out/analysis.py finds each table row in the extract by its text, checks that the parsed cells match the printed strings exactly, prints the source lines it uses, and applies the conventions from TASK.md:

• E1 (DCR, Table 6, ATT, 5 km, b = −0.431 log points): 100·(exp(b)−1).
• E2 (RESEA, Table 2, Reemployment: ATE Δ = 1.47 pp, control mean p = 75.0%): 100·Δ/p.
• E3 (minimum wage, Table 2: η = −0.388, g = 0.047 from row "% MW changes"): 100·η·g.
• E4 (DCR, Table 7, fatal, 5 km, b = −0.354): the absolute change is R0·(exp(b)−1). This needs the baseline (counterfactual) fatal hospitalisation rate R0 per 100,000.
• E5 (Benin, Table 3, Treatment, column (4), b = −0.0962 on a 0–1 scale): 100·b gives percentage points.

There is no randomness (random_seed = null). The printed inputs are stored under baseline_printed_estimates, which serves as the baseline.

# Results

• Four items were harmonised (n_harmonised = 4) and one could not be (n_not_harmonisable = 1). The task is "complete" in the sense TASK.md defines (feasibility): every item has either a value or a justified statement of impossibility.
• E1. The result matches the paper's own printed "% change" of −35.0% (consistency_checks.E1_vs_printed_pct_change).
• E4. Not harmonisable: the extract prints no level of the fatal hospitalisation rate, or of any hospitalisation rate (harmonised.E4.missing_quantity). For information only, the implied relative change would be −29.81% (consistency_checks.E4_relative_pct_only). That figure is not the target definition.
• E5. The paper also reports the relative figure b/mean = −36.85%. That is a different definition, and I do not use it as the harmonised value (consistency_checks.E5_pct_change_consistency).

# Uncertainty

• No standard errors or confidence intervals were propagated, as the task allows. The harmonised values are point estimates only.
• Exact conversions:
  • E5: rescaling by 100.
  • E1: the log-point to percent mapping is an exact identity for the point estimate. Note that 100·(exp(b̂)−1) is not an unbiased estimate of the percent effect when b̂ is noisy.
• Approximate conversions:
  • E2: the raw control mean (75.0%) stands in for the counterfactual level, while the ATE is adjusted for the randomisation blocks.
  • E3: the elasticity is applied linearly to a 4.7% change. The value depends on reading g = 0.047 as a proportion. With the note's "about 4%" instead, the result would be −1.55% (consistency_checks.E3_alt_with_note_4pct).
• All inputs are rounded as printed (3–4 decimals). The values are reported to 2 decimals.

# Limitations and open issues

• E4 cannot be converted from the admissible evidence. It would need the baseline or counterfactual fatal drug-related hospitalisation rate per 100,000 per year in the treated 5 km regions. I did not substitute any external or remembered value.
• E3 interpretation. The row labelled "% MW changes" prints 0.047, and I read it as a proportion (4.7%). The note's "about 4%" conflicts slightly with this.
• Source inconsistencies. These are listed in data_issues and only recorded; none affects the inputs used.