SPB Git

spb/wp29_uqo

Public
2.5 KB · 10 lines json
1[2 {"id":"C1","text":"Four of five studies can be harmonised to theta=ln(T_AI/T_noAI); only two (S1, S4) have an obtainable SE and are pooled; S5 is not harmonisable (output counts around a ban).","results_keys":["k_extracted","k_harmonised","k_pooled","pooled_studies","items"],"status":"supported","caveat":"Poolability is determined by reporting (what uncertainty is printed in the extracts), not by study quality."},3 {"id":"C2","text":"The inverse-variance fixed-effect and DerSimonian-Laird random-effects pooled theta over S1 and S4 is -0.188 (about 17% shorter time), 95% CI -0.396 to 0.020.","results_keys":["fe_estimate","fe_ci_low","fe_ci_high","re_estimate","re_se","re_ci_low","re_ci_high","pooled_pct_change_re","pooled_pct_change_re_ci"],"status":"supported","caveat":"k=2; S4 SE is approximate (back-calculated from p=0.471, two-sided assumed)."},4 {"id":"C3","text":"The pooled effect is not statistically significant at the 5% level (random-effects CI includes 0).","results_keys":["pooled_significant_5pct","re_ci_low","re_ci_high"],"status":"supported","caveat":"Low power with two studies."},5 {"id":"C4","text":"Heterogeneity among the two pooled studies is low (Q=0.32 on 1 df, I2=0, tau2=0), but this cannot capture the disagreement among the four harmonised studies, which differ in sign (S2 +0.174 vs S3 -0.354).","results_keys":["q_stat","q_df","i2","tau2","heterogeneity_interpretation","items"],"status":"supported","caveat":"tau2 essentially unidentified with k=2."},6 {"id":"C5","text":"The available evidence is mixed and setting-dependent and does not support a general claim that generative AI shortens task-completion time.","results_keys":["pooled_significant_5pct","items","k_pooled"],"status":"supported","caveat":"Interpretive synthesis; based on the pooled CI and the opposite-sign unpooled estimates S2 and S3."},7 {"id":"C6","text":"The pooled result is robust to using the CI-implied SE for S1 and to computing S4 theta from the ratio of printed times.","results_keys":["sensitivity_fe_s1_se_from_ci","sensitivity_fe_s4_theta_from_time_ratio","s1_se_from_ci"],"status":"supported","caveat":"Only these two perturbations were examined."},8 {"id":"C7","text":"Several internal inconsistencies exist in the sources (S1 Mean LogToT labels and n=96 vs 93; S3 printed % vs time ratio; S4 implausible unit labels) and do not change the harmonised values materially.","results_keys":["data_issues"],"status":"supported","caveat":"Units for S4 remain ambiguous; theta is unit-free."}9]10
Files · main
Menu

Theme