F4-conf-03 — Harmonising effect sizes reported under incompatible definitions
Scientific question
A synthesis compares treatment effects from four open-access studies that report their estimates under different definitions: a semi-elasticity (coefficient on a log outcome), a percentage-point effect on a probability, an elasticity, and a linear-probability coefficient. For each of the five effects listed in data/effects.json, re-express the printed point estimate on the target definition stated for that item, using only quantities printed in the cited extract and an explicitly stated transformation; where the target definition cannot be reached from the printed quantities, say so and name the additional quantity that would be required. The estimands are the harmonised point estimates (no uncertainty propagation is required). Each item carries a locator for every printed quantity used.
Inputs
data/effects.json— the five items: {"id", "source", "effect" (what and where), "reported_definition", "target_definition"}.data/sources/arxiv-2609.27732v1.txt— Armendariz (2026), Do Drug Consumption Rooms Reduce Drug-Related Hospitalizations? Evidence from Switzerland, arXiv:2609.27732v1, CC BY 4.0: abstract, Section 6 (Tables 6 and 7, Section 6.1), Appendix Table A19. Outcomes are log hospitalisation rates per 100,000 inhabitants; ATTs in log points; the tables also print a "% change" row for the overall effect.data/sources/arxiv-2606.14621v1.txt— Li, Pearson-Merkowitz & Yokum (2026), Evaluating the Impact of Rhode Island's Self-Sustaining RESEA Program on Employment Outcomes, arXiv:2606.14621v1, CC BY 4.0: abstract, Section 3 (Table 2: treatment/control means, differences, ATEs with p-values and 95% CIs), Table 3. Reemployment is in percent (means) and percentage points (effects).data/sources/arxiv-2505.04555v3.txt— Kanayama, Miyaji & Otani (2025/2026), Just After Minimum Wage Hikes: Short-Run Labor-Demand Response and Reallocation, arXiv:2505.04555v3, CC BY 4.0: abstract and Appendix A.1 (Table 2: elasticities and the size of the minimum-wage change, with note).data/sources/arxiv-2510.03658v1.txt— Tossou (2025), Who benefits the most? Direct and indirect effects of a free cesarean section policy in Benin, arXiv:2510.03658v1, CC BY 4.0: abstract, Section 4.1 and Table 3 (linear probability model of infant death or stillbirth; "Mean Dep. Var." row).data/README.md— provenance, retrieval times, licences, SHA-256 of the extracts.
Information-availability rules
- The extracts are the only admissible evidence; a quantity that is not printed in the cited extract is not available, even if you believe it is in the full paper or elsewhere.
- Conventions: a log-point coefficient b corresponds to a relative change of 100·(exp(b) − 1) percent; a percentage-point effect Δ on a probability with counterfactual (control) level p corresponds to a relative change of 100·Δ/p percent; an elasticity η applied to a proportional change g of the treatment variable implies a relative change of 100·η·g percent. State the transformation you apply in
transformation. harmonisableisfalseonly when the target definition requires a quantity that the cited extract does not print; thenvalueisnullandmissing_quantitynames the quantity. Do not substitute a guessed or externally remembered value.feasibilityrefers to the task as a whole: the task is complete when every item has either a harmonised value or an explicit, justified statement of impossibility.
Required outputs
out/results.jsonwith the keys:
| key | type | unit | meaning |
|---|---|---|---|
harmonised |
object | — | {"E1": {"value": number or null, "unit": "", "transformation": "", "harmonisable": true/false, "missing_quantity": string or null, "locator": "<source id |
n_harmonised |
integer | — | number of items with a non-null value |
n_not_harmonisable |
integer | — | number of items with harmonisable = false |
task_id |
string | — | "F4-conf-03" |
feasibility |
string | — | "complete", "partial" or "infeasible" — refers to the task as a whole |
feasibility_reason |
string or null | — | explanation if not complete |
data_issues |
array of {"issue","handling"} | — | problems detected in the inputs (rendering defects, internal inconsistencies of a source, etc.) and how they were handled; empty list if none |
n_input_rows |
object | lines / items | {file name: number of lines} for every file under data/ that you read (text files: number of newline-terminated lines as reported by wc -l; JSON files: number of top-level items) |
n_analysed |
integer or null | — | number of reported items (values, assertions, effects or references) you actually assessed |
random_seed |
integer or null | — | null (no randomness is involved) |
out/report.mdwith the sections: Question; Data and exclusions; Method; Results; Uncertainty; Limitations and open issues. "Uncertainty" must note that no standard errors were propagated and which conversions are exact versus approximate.out/analysis.py— running~/wp29/venv/bin/python out/analysis.pyfrom the workspace root must regenerateout/results.jsonfromdata/only, deterministically (the script must print the source lines it relies on).out/claims.json— a list of the central claims of the report: [{"id": "C1", "text": "...", "results_keys": ["..."], "status": "supported"|"unsupported"|"not_evaluated", "caveat": "..."}].
Constraints
- No network access. Python:
~/wp29/venv/bin/python(numpy, pandas, scipy, statsmodels available; the standard library is sufficient for this task). Do not modifydata/. - The source extracts under
data/sources/are the only admissible evidence. Do not rely on memory of the papers or on other versions of them; if a quantity is not in the extract, it is not available. - Report numbers exactly as printed in the source (same sign, same number of decimals); do not round or rescale printed values unless the key definition asks for a transformation.
- This task is designed to be completed in well under 20 minutes of work.