spb/wpai0002_uqo

1\begin{tabular}{@{}lrllllccrr@{}}2\toprule3System & Params & $\Delta$ val & $\Delta$ test [95\% CI] & $\Delta$ domain & $\Delta$ period & Num. & Plac. & Pangram & Compute \\4 & (M) & (3\,000) & (15\,649) & (2\,493) & (2\,574) & & & Human & (h) \\5\midrule6B1 generic paraphrase & -- & $-2.75 \pm 0.09$ & $-2.68$ [-2.78, -2.59] & $-2.75 \pm 0.10$ & $-2.04 \pm 0.07$ & 1.000 & 1.000 & -- & -- \\7HTR-Edit tagger (23.6 M) & 23.6 & $-2.27 \pm 0.12$ & $-2.20$ [-2.36, -2.05] & -- & -- & 1.000 & 0.999 & -- & 0.3$^{\dagger}$ \\8HTR-Tiny $M^\ast$ (33.5 M), confirmatory & 33.5 & $+0.42 \pm 0.14$ & $+0.44$ [+0.30, +0.58] & $-0.10 \pm 0.13$ & $-0.52 \pm 0.12$ & 0.991 & 0.992 & 54/54 & 2.7 \\9HTR-Small (130.8 M), 15 k-step denoiser & 130.8 & $+0.20 \pm 0.13$ & $+0.11$ [-0.03, +0.24] & $-0.35 \pm 0.14$ & $-0.67 \pm 0.11$ & 0.990 & 0.991 & 54/54 & 1.2 \\10HTR-Small v2, 22.5 k-step denoiser & 130.8 & $-0.13 \pm 0.14$ & $-0.09$ [-0.22, +0.04] & -- & -- & 0.988 & 0.989 & -- & 2.9 \\11HTR-Base (334.8 M), MPS denoiser (under-trained) & 334.8 & $-0.21 \pm 0.13$ & $-0.26$ [-0.39, -0.12] & -- & -- & 0.988 & 0.990 & -- & 1.8 \\12HTR-Base (334.8 M), H100 & 334.8 & $+0.83 \pm 0.13$ & $+0.69$ [+0.56, +0.83] & $+0.29 \pm 0.13$ & $-0.12 \pm 0.10$ & 0.992 & 0.993 & 53/54 & 2.3 \\13HTR-Base, H100, seed 2 & 334.8 & $+0.67 \pm 0.14$ & $+0.63$ [+0.49, +0.76] & $+0.32 \pm 0.14$ & $-0.07 \pm 0.10$ & 0.991 & 0.992 & -- & 2.3 \\14HTR-Base, H100, fine-tune lr 2e-5 & 334.8 & $+0.68 \pm 0.13$ & $+0.55$ [+0.41, +0.69] & $+0.25 \pm 0.13$ & $-0.23 \pm 0.10$ & 0.991 & 0.992 & -- & 0.3 \\15HTR-Large (1.07 B), stage-2 denoiser & 1\,068.6 & $-0.64 \pm 0.12$ & $-0.71$ [-0.84, -0.58] & $-1.15 \pm 0.13$ & $-1.62 \pm 0.12$ & 0.988 & 0.987 & -- & 2.4$^{\dagger}$ \\16HTR-Large, stage-3 denoiser, lr 1.5e-5 & 1\,068.6 & $-0.60 \pm 0.10$ & $-0.79$ [-0.89, -0.69] & $-1.05 \pm 0.11$ & $-1.10 \pm 0.10$ & 0.988 & 0.989 & -- & 1.8 \\17HTR-Large, stage-3 denoiser, lr 6e-5 (diverged) & 1\,068.6 & $-1.11 \pm 0.11$ & $-1.25$ [-1.35, -1.15] & $-1.55 \pm 0.12$ & $-1.20 \pm 0.10$ & 0.983 & 0.984 & -- & 0.4 \\18B2 T5-small (60.5 M) & 60.5 & $-0.04 \pm 0.14$ & $-0.08$ [-0.26, +0.08] & -- & -- & 0.986 & 0.733 & 50/54 & 0.1 \\19flan-T5-large (783 M) & 783.2 & $+2.04 \pm 0.15$ & $+1.79$ [+1.62, +1.96] & $+1.27 \pm 0.18$ & $+1.09 \pm 0.14$ & 0.987 & 0.992 & 54/54 & 0.6 \\20flan-T5-xl (2.85 B) & 2\,849.8 & $+2.56 \pm 0.16$ & $+2.39$ [+2.20, +2.57] & $+1.78 \pm 0.18$ & $+2.12 \pm 0.15$ & 0.986 & 0.992 & 54/54 & 1.2 \\21Qwen2.5-1.5B SFT & $\approx$1\,500 & $+2.19 \pm 0.12$ & $+1.97$ [+1.82, +2.10] & $+1.78 \pm 0.13$ & $+1.30 \pm 0.14$ & 0.998 & 0.998 & 48/54 & 0.9$^{\dagger}$ \\22Qwen2.5-1.5B SFT, seed 2 & $\approx$1\,500 & $+2.17 \pm 0.12$ & $+1.97$ [+1.83, +2.11] & $+1.75 \pm 0.13$ & $+1.28 \pm 0.14$ & 0.998 & 0.998 & -- & 0.9$^{\dagger}$ \\23Qwen2.5-3B SFT & $\approx$3\,000 & $+2.42 \pm 0.13$ & $+2.27$ [+2.13, +2.40] & $+2.05 \pm 0.13$ & $+1.91 \pm 0.14$ & 0.998 & 0.999 & 49/54 & 1.5$^{\dagger}$ \\24\bottomrule25\end{tabular}26