model  n_runs  acceptance_taskw  det_completion  critical_error_rate  quality_mean  cost_mean  cost_per_accepted  wall_median_s  tool_calls_mean  timeout_rate
0   fable      20              0.85            0.95                 0.05        86.938      2.805              3.300         397.60            21.90           0.0
1    opus      30              0.90            1.00                 0.00        85.694      0.906              1.006         218.80            17.20           0.0
2  sonnet      20              0.85            1.00                 0.00        80.896      0.341              0.401         131.05            13.45           0.0
      contrast  outcome  estimate  ci_lo  ci_hi  perm_p   J
0  sonnet-opus        S    -0.050 -0.150  0.000   1.000  10
1  sonnet-opus        D     0.000  0.000  0.000   1.000  10
2  sonnet-opus  quality    -4.799 -6.702 -2.965   0.002  10
3  sonnet-opus        C    -0.564 -0.678 -0.465   0.001  10
4   fable-opus        S    -0.050 -0.150  0.000   1.000  10
5   fable-opus        D    -0.050 -0.150  0.000   1.000  10
6   fable-opus  quality     1.243 -1.375  3.840   0.404  10
7   fable-opus        C     1.899  1.563  2.306   0.002  10