SPB Git

spb/wp29_uqo

Public
2.5 KB · 21 lines markdown
1# Contract (P0) — T3-synthesis23**Question.** Does access to a generative-AI assistant change the time workers need to complete their tasks? Synthesise five primary studies reported under heterogeneous effect definitions.45**Estimands.**61. The five printed quantities, verbatim (same sign/decimals), each with a locator.72. For each study, θ = ln(T_AI / T_noAI) (same tasks, randomised AI vs no-AI comparison; negative = faster with AI) and SE(θ), when obtainable from printed quantities under the TASK conventions:8   - log-time regression coefficient b → θ = b, SE = SE(b) (or SE=(U−L)/3.92 from a 95% CI);9   - percentage change c in time or ratio of means → θ = ln(1+c);10   - only a two-sided p-value → SE ≈ |θ|/z_{1−p/2} (approximate; flagged);11   - only stars / thresholds / figure-only CI → harmonisable, not poolable (SE null);12   - not a completion-time AI-vs-no-AI comparison (e.g. output counts, DiD on output) → not harmonisable.133. Inverse-variance FE and DerSimonian–Laird RE pooled θ over poolable studies (95% CI with 1.96), Q (df=k−1), I² = max(0,(Q−df)/Q)·100, τ² = max(0,(Q−df)/C); heterogeneity label (<25 low, 25–50 moderate, ≥50 substantial); significance = RE CI excludes 0; pooled % change = 100(exp(RE)−1).14   Meaningful under: approximate normality of θ̂, independent studies, comparable estimand (ratio of mean times), and that the exp(E log T) vs E T distinction is ignored as the conventions prescribe.1516**Outputs.** out/table1.csv (12 columns, 5 rows); out/figure1.png (forest plot, harmonisable studies, points w/o CI when SE null, RE diamond); out/results.json (keys: task_id, target_definition, items{S1..S5 with 11 fields}, k_extracted, k_harmonised, k_pooled, fe_*, re_*, q_stat, i2, tau2, heterogeneity_interpretation, pooled_significant_5pct, pooled_pct_change_re, feasibility, feasibility_reason, data_issues, n_input_rows, n_analysed, random_seed); out/report.md (6 sections); out/analysis.py (regenerates results/table/figure from data/, deterministic, prints source lines relied on); out/claims.json; plus protocol files (state/, exclusions.csv).1718**Information rules.** Only the extracts are admissible; nothing from memory, figures or omitted sections. Printed numbers reported exactly as printed.1920**Invalidity / impossibility.** Using non-printed quantities (e.g. METR CI from Figure 1); pooling a study with no legitimate SE; pooling a non-harmonisable study; fewer than 1 poolable study would make pooling impossible (not the case expected here).21
Files · 76f9a44a3a85fb2bc2b0b370a4ae4eed9b95e579
Menu

Theme