Contract (P0) — T3-synthesis
Question. Does access to a generative-AI assistant change the time workers need to complete their tasks? Synthesise five primary studies reported under heterogeneous effect definitions.
Estimands.
- The five printed quantities, verbatim (same sign/decimals), each with a locator.
- For each study, θ = ln(T_AI / T_noAI) (same tasks, randomised AI vs no-AI comparison; negative = faster with AI) and SE(θ), when obtainable from printed quantities under the TASK conventions:
- log-time regression coefficient b → θ = b, SE = SE(b) (or SE=(U−L)/3.92 from a 95% CI);
- percentage change c in time or ratio of means → θ = ln(1+c);
- only a two-sided p-value → SE ≈ |θ|/z_{1−p/2} (approximate; flagged);
- only stars / thresholds / figure-only CI → harmonisable, not poolable (SE null);
- not a completion-time AI-vs-no-AI comparison (e.g. output counts, DiD on output) → not harmonisable.
- Inverse-variance FE and DerSimonian–Laird RE pooled θ over poolable studies (95% CI with 1.96), Q (df=k−1), I² = max(0,(Q−df)/Q)·100, τ² = max(0,(Q−df)/C); heterogeneity label (<25 low, 25–50 moderate, ≥50 substantial); significance = RE CI excludes 0; pooled % change = 100(exp(RE)−1). Meaningful under: approximate normality of θ̂, independent studies, comparable estimand (ratio of mean times), and that the exp(E log T) vs E T distinction is ignored as the conventions prescribe.
Outputs. out/table1.csv (12 columns, 5 rows); out/figure1.png (forest plot, harmonisable studies, points w/o CI when SE null, RE diamond); out/results.json (keys: task_id, target_definition, items{S1..S5 with 11 fields}, k_extracted, k_harmonised, k_pooled, fe_, re_, q_stat, i2, tau2, heterogeneity_interpretation, pooled_significant_5pct, pooled_pct_change_re, feasibility, feasibility_reason, data_issues, n_input_rows, n_analysed, random_seed); out/report.md (6 sections); out/analysis.py (regenerates results/table/figure from data/, deterministic, prints source lines relied on); out/claims.json; plus protocol files (state/, exclusions.csv).
Information rules. Only the extracts are admissible; nothing from memory, figures or omitted sections. Printed numbers reported exactly as printed.
Invalidity / impossibility. Using non-printed quantities (e.g. METR CI from Figure 1); pooling a study with no legitimate SE; pooling a non-harmonisable study; fewer than 1 poolable study would make pooling impossible (not the case expected here).