# F4-conf-03: Harmonising effect sizes reported under incompatible definitions

# Question

Five effects from four open-access arXiv papers (listed in data/effects.json) are reported under different definitions: log-point ATTs, a percentage-point effect, an elasticity, and a linear-probability coefficient. The task is to restate each printed point estimate on the target definition given for that item. Only quantities printed in the cited extract may be used, and every transformation must be stated explicitly. Where the target cannot be reached, the report must say so and name the quantity that is missing.

# Data and exclusions

• The inputs are the four extracts under data/sources/, plus data/effects.json (5 items) and data/README.md. Line counts are in n_input_rows. The SHA-256 hashes of all four extracts match the values in the README (see out/state/data_audit.md).
• No item was excluded. All 5 items were assessed (n_analysed = 5; out/exclusions.csv: 5 − 0 = 5).
• Problems found in the inputs are listed in data_issues, together with how each was handled. In summary:
  • The DCR abstract says fatal outcomes "fall by 35%". Table 7, however, implies about −29.8%, and Section 6.1 says "approximately 30%".
  • For the same −0.431 estimate, the DCR paper prints a standard error of 0.210 in Table 6 but 0.218 in Table A19. The Table A19 note says 640 control regions, while the tables say 608.
  • Two "% change" values in DCR Table A19 differ in the last digit from 100·(exp(ATT) − 1).
  • In the minimum-wage paper, the row labelled "% MW changes" prints 0.047, while the table note says "about 4%".
  • Tossou's "% Change" row differs in the last digit from the coefficient divided by the mean. Tossou's abstract cites a Table 2 that is not in the extract.
  • The RESEA control mean is a raw mean, whereas the treatment effect is adjusted for randomisation blocks.
  • There are minor rendering defects. None of them affects a number used here.

# Method

The conventions are those set out in TASK.md. out/analysis.py reads each value directly from its table row and prints the source lines it uses. The printed, untransformed estimates are the baseline (baseline_printed_values, baseline_printed_units).

• E1 (log points → relative %): 100·(exp(b) − 1), with b = −0.431 (DCR Table 6, ATT row, 5 km column).
• E2 (pp → % of the control mean): 100·Δ/p, with Δ = 1.47 pp (RESEA Table 2, Reemployment row, ATE column) and p = 75.0% (Control column).
• E3 (elasticity → implied % change): 100·η·g, with η = −0.388 and g = 0.047 (MW Table 2, "Employment elasticity w.r.t MW" and "% MW changes" rows). I read g as a proportion (4.7%), the only reading that fits the note's "about 4%".
• E4 (log points → absolute change per 100,000): this would need R0·(exp(b) − 1), with b = −0.354 (DCR Table 7, fatal row, 5 km column). The baseline level R0 is not printed anywhere in the extract. The script searches the extract for a printed level and finds none.
• E5 (LPM coefficient → pp): 100·β, with β = −0.0962 (Tossou Table 3, Treatment row, column (4)).

The full formula and locator for each item are stored in harmonised.<id>.transformation and harmonised.<id>.locator. No randomness is involved (random_seed = null).

# Results

• 4 items were harmonised (n_harmonised) and 1 was not (n_not_harmonisable).
• Feasibility is "complete" (feasibility): every item has either a value or a justified statement that it cannot be converted.
• E1 agrees with the paper's own "% change" row (−35.0%).
• E4 can only be given in relative terms: −29.81%, as recorded in harmonised.E4.transformation. An absolute change requires the baseline (counterfactual) fatal drug-related hospitalisation rate per 100,000 in the 5 km treated regions (harmonised.E4.missing_quantity).

# Uncertainty

No standard errors or confidence intervals were propagated; all values are point estimates.

• Exact conversions: E1 (exact under the log-point convention), E5 (a pure rescaling) and E2 (an exact ratio of printed numbers).
• E2 caveat: the denominator is the raw aggregate control mean (75.0%), not a block-adjusted counterfactual level. The printed inputs are also rounded (1.47; 75.0).
• Approximate conversion: E3. The formula η·g is a linear (first-order) approximation. The input g = 0.047 is printed to only 3 decimals, and its unit is ambiguous given the label "%" and the note's "about 4%".
• E4: no value is reported, so no uncertainty applies.

# Limitations and open issues

• E4 cannot be converted to an absolute rate from the extract. I did not guess or substitute an outside baseline.
• E3 depends on reading 0.047 as a proportion. The source's own note ("about 4%") does not match it exactly. See data_issues.
• E2 uses a raw control mean as its denominator, while the effect is block-adjusted. This follows the stated target definition, but it is not a model-based counterfactual level.
• Source inconsistencies (see data_issues): the DCR abstract's fatal −35% versus Table 7's −29.8%; differing standard errors and control-region counts between DCR Tables 6 and A19; rounding discrepancies. None of these changes the point estimates used here.
• Values are rounded to 2 decimals. The unrounded results are given in each transformation string.