SPB Git

spb/wp29_uqo

Public
2.0 KB · 10 lines json
1[2 {"id":"C1","text":"Four of five studies can be harmonised to theta = ln(T_AI/T_noAI); only S1 and S4 have obtainable SEs and are pooled.","results_keys":["k_extracted","k_harmonised","k_pooled","pooled_studies"],"status":"supported","caveat":"S2 and S3 are excluded from pooling because uncertainty is only in a figure or given as stars, not for quality reasons."},3 {"id":"C2","text":"S5 (ChatGPT-ban DiD on GitHub output counts) is not harmonisable.","results_keys":["items"],"status":"supported","caveat":"Outcome is output counts in a non-randomised design, not task time."},4 {"id":"C3","text":"The pooled fixed- and random-effects estimate is theta = -0.188 (about -17% task time), 95% CI [-0.396, 0.020].","results_keys":["fe_estimate","fe_ci_low","fe_ci_high","re_estimate","re_se","re_ci_low","re_ci_high","pooled_pct_change_re"],"status":"supported","caveat":"k = 2; S4 SE approximate (from a p-value)."},5 {"id":"C4","text":"The pooled effect is not statistically significant at the 5% level.","results_keys":["pooled_significant_5pct","re_ci_low","re_ci_high"],"status":"supported","caveat":"Low power with two studies."},6 {"id":"C5","text":"Measured heterogeneity among the pooled studies is low (Q = 0.32, I2 = 0, tau2 = 0), but this does not show homogeneity.","results_keys":["q_stat","i2","tau2","heterogeneity_interpretation"],"status":"supported","caveat":"Q has 1 df; the unpooled S2 (+0.174) and S3 (-0.354) point in opposite directions."},7 {"id":"C6","text":"The evidence is inconclusive and does not support a general claim that generative AI shortens task-completion time.","results_keys":["re_ci_low","re_ci_high","items","k_pooled"],"status":"supported","caveat":"Interpretive conclusion based on the non-significant pooled interval and conflicting unpooled studies."},8 {"id":"C7","text":"Pooled estimate is robust to using the CI-derived SE for S1 and the times-based theta for S4.","results_keys":["sensitivity_fe_with_S1_ci_se","sensitivity_fe_with_S4_from_times","fe_estimate"],"status":"supported","caveat":"Only rounding-level alternatives were examined."}9]10
Files · main
Menu

Theme