SPB Git

spb/wp29_uqo

Public
3.8 KB · 7 lines csv
1study,estimate_as_printed,uncertainty_as_printed,sample_as_printed,definition,locator,harmonised_estimate,harmonised_se,harmonisable,poolable,transformation,reason_if_not2S1,"ExpCon (Model 2) = -0.24 (marker '.', p<.1) on LogToT","SE (0.14) [Table III, Model 2]; 95%CI = [-0.51,0.03], p=0.086 [Section VI-C]","n = 96 (Table III caption); N=47 AI, N=46 no-AI in Section VI-B","OLS coefficient on experimental condition (AI vs no-AI) with natural-log time on task as dependent variable, controlling for developer-level factors (Model 2)","arxiv-2410.12944v3, Table III (line 76: ExpCon row, 2nd value = Model 2; line 78: SE row), Section VI-C (line 58)",-0.24,0.14,True,True,theta = b = -0.24; SE(theta) = SE(b) = 0.14 (printed). Cross-check from CI: (0.03 - (-0.51))/(2*1.96) = 0.1378,3S2,"19% more time (issue completion time, AI allowed vs disallowed; regression-based)","none printed in extract (95% CI using HC3 SEs is shown only in Figure 1, not reproduced)","16 developers; 246 tasks (136 issues AI-allowed, 110 AI-disallowed)","S = E[time AI allowed]/E[time AI disallowed] - 1, estimated as exp(beta)-1 from log-linear OLS of completion time on AI indicator and log forecast time","arxiv-2507.09089v2, Section 3 Results (line 62); Abstract (line 34); estimator Sections 2.3, D.1-D.3; CI method D.2 (line 97)",0.17395330712343798,,True,False,theta = ln(1 + c) = ln(1 + 0.19) = 0.1740,"Not poolable: no SE, CI or p-value for the 19% estimate is printed in the extract; the confidence interval appears only in Figure 1 (not reproduced), so SE(theta) cannot be obtained."4S3,"All Content (aggregate): Overall 53.84, Treatment 44.29, Control 63.10 minutes; Improvement -29.79%***","significance stars only (*** = 0<p<=0.01, per editorial note); no SE or CI printed",181 subjects completed the task (Section 3); treatment/control split not printed,"Percentage difference in mean task duration (minutes), Copilot vs control, holding accuracy constant (control duration predicted at treatment accuracy; Appendix B); bootstrap significance","arxiv-2411.01067v2, Table 3 (line 55: 'Aggregate Results / All Content' row; columns Treatment, Control, Improvement)",-0.353679434958934,,True,False,theta = ln(1 + c) = ln(1 + (-0.2979)) = -0.3537 (cross-check ln(44.29/63.10) = -0.3540),"Not poolable: uncertainty is reported only as significance stars (***, p<=0.01); a p-value threshold does not identify SE(theta)."5S4,Full Corpus: Aware Time per Email (s) 50.5; Control Time per Email (s) 56.9; % Time Savings 11.1,p-value 0.471,"Aware N=56, Control N=52 (Blind N=59 not used); 25-email queue per subject","Percentage time saving in mean processing time per email (full queue), Aware arm (agent verdicts + rationale + priority ordering) vs Control (no AI)","arxiv-2511.13860v1, Section 3.3, Table 3 (lines 90-96; column 'Full Corpus')",-0.11765804346823246,0.1632205919720678,True,True,c = -11.1/100 = -0.111; theta = ln(1 + c) = ln(0.889) = -0.1177 (cross-check ln(50.5/56.9) = -0.1193); APPROXIMATE SE = |theta|/z_(1-p/2) = 0.1177/0.7209 = 0.1632 from printed p = 0.471,6S5,"Treat x Post (Output, column 3, panel A) = -0.278",SE (0.295) (clustered by user); no significance star,"N = 18332 (user-days, users with affiliation, panel A)","Difference-in-differences coefficient on an aggregate GitHub output count (Push, PRs, comments, Create, Issues) for Italian vs Austrian/French users, around the ChatGPT ban (withdrawal of access)","arxiv-2304.09339v1, Section 4, Table 1, panel A, row 'Treat x Post' (line 80), SE line 82, N line 88, column (3) Output",,,False,False,none (not harmonisable),"Not harmonisable: the outcome is a count of GitHub output events per user-day (activity volume), not task-completion time on the same tasks; the contrast is country-level withdrawal of access (DiD), not a randomised AI-vs-no-AI comparison of time on identical tasks."7
Files · main
Menu

Theme