1% ---------------------------------------------------------------------------2% Appendix — Additional robustness results (Agent H; every number from3% analysis/tables or analysis/derived; sources in comments)4% ---------------------------------------------------------------------------5\section{Additional robustness results}\label{app:robustness}67This appendix collects the full tables behind Sections~\ref{sec:results} and~\ref{sec:reliability} and the sensitivity analyses that the amendments of Section~\ref{sec:data-missing} call for: the complete occupation-by-arm table, the per-scenario model comparison, the quality-threshold sensitivity, the weighting sensitivity, the recomputation of every contrast without the eight re-judged runs of amendment A-008, the three reruns of amendment A-010 with their first attempts, the executor-node check, the judges' severity by model and arm, and the human-cost assumption behind Figure~\ref{fig:economic}.89\subsection{Outcomes by occupation and arm}\label{app:robustness-occupation}1011Table~\ref{tab:occupation-full} gives, for each of the fifty occupation-by-arm cells of the pilot, the acceptance rate, the deterministic completion rate, the mean quality, the critical-error rate, the mean marginal cost and the mean number of tool calls over the three repeats.12% source: analysis/tables/table3_occupation_arm_pilot.csv1314\begin{table}[htbp]15\centering16\caption{Outcomes by occupation and arm, 150 pilot runs (3 repeats per cell)}17\label{tab:occupation-full}18\scriptsize19\setlength{\tabcolsep}{3.5pt}20\begin{tabular}{@{}llrrrrrr@{}}21\toprule22Occupation (scenario) & Arm & $S$ & $D$ & $Q$ & $E$ & Cost (USD) & Tool calls \\23\midrule24Academic research (RES-01) & A0 & 0.000 & 0.000 & 23.8 & 0.000 & 0.394 & 15.7 \\25 & A1 & 1.000 & 1.000 & 82.2 & 0.000 & 0.627 & 15.7 \\26 & A2 & 1.000 & 1.000 & 86.0 & 0.000 & 0.622 & 16.3 \\27 & A3 & 1.000 & 1.000 & 81.9 & 0.000 & 0.685 & 18.0 \\28 & A4 & 1.000 & 1.000 & 86.4 & 0.000 & 0.732 & 19.0 \\29\addlinespace30Accounting (ACC-01) & A0 & 0.000 & 0.000 & 27.1 & 0.000 & 0.744 & 34.3 \\31 & A1 & 1.000 & 1.000 & 95.3 & 0.000 & 1.129 & 21.7 \\32 & A2 & 1.000 & 1.000 & 89.3 & 0.000 & 1.039 & 19.3 \\33 & A3 & 1.000 & 1.000 & 98.9 & 0.000 & 1.185 & 24.3 \\34 & A4 & 1.000 & 1.000 & 93.8 & 0.000 & 1.202 & 26.3 \\35\addlinespace36Banking (BANK-01) & A0 & 0.000 & 0.000 & 16.7 & 0.000 & 0.472 & 20.0 \\37 & A1 & 1.000 & 1.000 & 96.7 & 0.000 & 0.948 & 9.3 \\38 & A2 & 1.000 & 1.000 & 94.3 & 0.000 & 0.800 & 7.3 \\39 & A3 & 1.000 & 1.000 & 91.3 & 0.000 & 0.842 & 12.0 \\40 & A4 & 1.000 & 1.000 & 91.5 & 0.000 & 0.880 & 15.3 \\41\addlinespace42Business intelligence (BI-01) & A0 & 0.000 & 0.000 & 5.4 & 0.000 & 0.865 & 13.3 \\43 & A1 & 0.667 & 0.667 & 50.0 & 0.333 & 0.878 & 14.0 \\44 & A2 & 1.000 & 1.000 & 76.9 & 0.000 & 0.936 & 15.3 \\45 & A3 & 1.000 & 1.000 & 78.1 & 0.000 & 1.212 & 22.7 \\46 & A4 & 0.667 & 1.000 & 72.4 & 0.000 & 1.161 & 24.0 \\47\addlinespace48Corporate finance (FIN-01) & A0 & 0.000 & 0.000 & 19.4 & 0.000 & 0.309 & 15.0 \\49 & A1 & 1.000 & 1.000 & 85.6 & 0.000 & 0.795 & 12.3 \\50 & A2 & 1.000 & 1.000 & 82.5 & 0.000 & 0.730 & 10.3 \\51 & A3 & 1.000 & 1.000 & 87.8 & 0.000 & 0.693 & 12.3 \\52 & A4 & 1.000 & 1.000 & 86.3 & 0.000 & 0.726 & 14.3 \\53\addlinespace54Executive administration (ADM-01) & A0 & 0.000 & 0.000 & 51.4 & 0.000 & 1.569 & 78.0 \\55 & A1 & 0.000 & 1.000 & 62.8 & 0.000 & 1.227 & 14.7 \\56 & A2 & 0.333 & 1.000 & 66.7 & 0.000 & 1.083 & 12.7 \\57 & A3 & 0.000 & 1.000 & 62.2 & 0.000 & 1.385 & 17.7 \\58 & A4 & 0.000 & 1.000 & 64.9 & 0.000 & 1.282 & 18.3 \\59\addlinespace60Human resources (HR-01) & A0 & 0.000 & 0.000 & 35.4 & 0.000 & 0.836 & 42.0 \\61 & A1 & 1.000 & 1.000 & 73.6 & 0.000 & 0.748 & 10.0 \\62 & A2 & 1.000 & 1.000 & 76.1 & 0.000 & 0.788 & 10.3 \\63 & A3 & 1.000 & 1.000 & 76.3 & 0.000 & 0.855 & 14.3 \\64 & A4 & 1.000 & 1.000 & 74.3 & 0.000 & 0.915 & 16.7 \\65\addlinespace66Legal services (LEGAL-01) & A0 & 0.000 & 0.000 & 12.6 & 1.000 & 0.238 & 15.3 \\67 & A1 & 1.000 & 1.000 & 92.9 & 0.000 & 0.712 & 9.7 \\68 & A2 & 1.000 & 1.000 & 90.7 & 0.000 & 0.683 & 9.0 \\69 & A3 & 1.000 & 1.000 & 91.4 & 0.000 & 0.783 & 15.3 \\70 & A4 & 1.000 & 1.000 & 92.6 & 0.000 & 0.930 & 17.3 \\71\addlinespace72Procurement (PROC-01) & A0 & 0.000 & 0.000 & 62.9 & 0.000 & 1.154 & 18.7 \\73 & A1 & 1.000 & 1.000 & 91.0 & 0.000 & 0.600 & 7.7 \\74 & A2 & 1.000 & 1.000 & 89.2 & 0.000 & 0.536 & 8.3 \\75 & A3 & 1.000 & 1.000 & 90.4 & 0.000 & 0.669 & 13.0 \\76 & A4 & 1.000 & 1.000 & 89.2 & 0.000 & 0.695 & 14.0 \\77\addlinespace78Real-estate appraisal (RE-01) & A0 & 0.000 & 0.000 & 19.2 & 1.000 & 0.514 & 42.0 \\79 & A1 & 1.000 & 1.000 & 93.5 & 0.000 & 0.567 & 13.7 \\80 & A2 & 0.667 & 0.667 & 98.1 & 0.000 & 0.617 & 14.7 \\81 & A3 & 1.000 & 1.000 & 98.8 & 0.000 & 0.746 & 22.3 \\82 & A4 & 0.667 & 0.667 & 96.7 & 0.000 & 0.735 & 22.7 \\83\bottomrule84\end{tabular}85\begin{tablenotes}86\footnotesize87\item Source: \texttt{analysis/tables/table3\_occupation\_arm\_pilot.csv} (script \texttt{analysis/econometrics/estimate.py}). $S$ acceptance, $D$ deterministic completion, $Q$ mean quality, $E$ critical-error rate; cost is the mean marginal API cost per run.88\end{tablenotes}89\end{table}9091\subsection{Model sensitivity by scenario}\label{app:robustness-models}9293Table~\ref{tab:models-scenario} gives the acceptance, mean quality and mean cost of arm A3 on each scenario under the three worker models. The three models agree on acceptance in nine scenarios out of ten---all three fail ADM-01 in every run and solve the other eight in every run---and differ on BI-01, where Opus is accepted in three of three runs and Sonnet and Fable in one of two each. Fable's quality exceeds Opus's in six scenarios and Sonnet's is below Opus's in nine; Fable's cost exceeds Opus's in every scenario by a factor between 2.6 (RE-01) and 3.6 (ACC-01), and Sonnet's is below Opus's by a factor between 2.3 (BANK-01) and 4.1 (BI-01).94% source: analysis/tables/table9c_model_by_scenario.csv9596\begin{table}[htbp]97\centering98\caption{Arm A3 by scenario and worker model: acceptance, mean quality and mean cost}99\label{tab:models-scenario}100\footnotesize101\setlength{\tabcolsep}{4pt}102\begin{adjustbox}{max width=\textwidth}103\begin{tabular}{@{}lrrrrrrrrr@{}}104\toprule105 & \multicolumn{3}{c}{Acceptance} & \multicolumn{3}{c}{Quality} & \multicolumn{3}{c}{Cost (USD)} \\106\cmidrule(lr){2-4}\cmidrule(lr){5-7}\cmidrule(lr){8-10}107Scenario & Sonnet & Opus & Fable & Sonnet & Opus & Fable & Sonnet & Opus & Fable \\108\midrule109ACC-01 & 1.0 & 1.0 & 1.0 & 92.7 & 98.9 & 98.1 & 0.469 & 1.185 & 4.286 \\110ADM-01 & 0.0 & 0.0 & 0.0 & 61.9 & 62.2 & 66.7 & 0.580 & 1.385 & 4.320 \\111BANK-01 & 1.0 & 1.0 & 1.0 & 89.4 & 91.3 & 98.1 & 0.354 & 0.842 & 2.709 \\112BI-01 & 0.5 & 1.0 & 0.5 & 68.3 & 78.1 & 71.7 & 0.296 & 1.212 & 3.045 \\113FIN-01 & 1.0 & 1.0 & 1.0 & 79.6 & 87.8 & 85.2 & 0.263 & 0.693 & 2.448 \\114HR-01 & 1.0 & 1.0 & 1.0 & 73.1 & 76.3 & 75.2 & 0.383 & 0.855 & 2.739 \\115LEGAL-01 & 1.0 & 1.0 & 1.0 & 90.2 & 91.4 & 93.1 & 0.336 & 0.783 & 2.397 \\116PROC-01 & 1.0 & 1.0 & 1.0 & 86.0 & 90.4 & 94.6 & 0.230 & 0.669 & 2.136 \\117RE-01 & 1.0 & 1.0 & 1.0 & 90.8 & 98.8 & 97.1 & 0.283 & 0.746 & 1.954 \\118RES-01 & 1.0 & 1.0 & 1.0 & 76.9 & 81.9 & 89.6 & 0.215 & 0.685 & 2.012 \\119\bottomrule120\end{tabular}121\end{adjustbox}122\begin{tablenotes}123\footnotesize124\item Source: \texttt{analysis/tables/table9c\_model\_by\_scenario.csv} (script \texttt{analysis/robustness/sensitivity\_models.py}). Sonnet and Fable: 2 repeats per scenario (stage \texttt{sensitivity}); Opus: 3 repeats (pilot, arm A3). Fable's ADM-01 first repeat carries a critical violation (compliance with the injection) and fails deterministic completion.125\end{tablenotes}126\end{table}127128\subsection{Quality threshold and task weights}\label{app:robustness-weights}129130Table~\ref{tab:q-sensitivity} in the main text reports acceptance by arm for $q$ from 50 to 90; it is not repeated here. Table~\ref{tab:weights} reports the four preregistered acceptance contrasts under the two weightings of the preregistration: equal scenario weights (the primary analysis) and weights proportional to the scenario authors' professional-minutes estimates of Table~\ref{tab:pilot}, which range from 150 minutes (LEGAL-01, HR-01, PROC-01) to 420 (ACC-01). The minutes weighting raises the weight of the deepest scenarios, on which the shell arms succeed, and lowers that of the executive-administration scenario (180 minutes), on which they fail; every contrast keeps its sign, the shell premium rises from 0.867 to 0.885, the computer-use premium from 0.033 to 0.041, the self-repair premium falls from $-0.067$ to $-0.082$, and the macOS-layer contrast falls from 0.033 to 0.025. The bound of Proposition~\ref{prop:alf}(vi) is loose here because the per-scenario acceptance probabilities span the full unit interval across occupations; the realised change is at most 0.018 in any contrast.131% source: analysis/tables/table2_contrasts_pilot.csv (estimate, estimate_minutes_weighted); table1 (acceptance_taskw, acceptance_minw); benchmark minutes from sections/benchmark.tex tab:pilot132133\begin{table}[htbp]134\centering135\caption{Acceptance contrasts and arm acceptance rates under equal and professional-minutes scenario weights}136\label{tab:weights}137\small138\begin{tabular}{@{}lrr@{}}139\toprule140 & Equal weights & Minutes weights \\141\midrule142A3 $-$ A1 (primary) & $+0.033$ & $+0.041$ \\143A1 $-$ A0 & $+0.867$ & $+0.885$ \\144A4 $-$ A3 & $-0.067$ & $-0.082$ \\145A2 $-$ A1 & $+0.033$ & $+0.025$ \\146\midrule147Acceptance, A0 & 0.000 & 0.000 \\148Acceptance, A1 & 0.867 & 0.885 \\149Acceptance, A2 & 0.900 & 0.909 \\150Acceptance, A3 & 0.900 & 0.926 \\151Acceptance, A4 & 0.833 & 0.844 \\152\bottomrule153\end{tabular}154\begin{tablenotes}155\footnotesize156\item Sources: \texttt{analysis/tables/table2\_contrasts\_pilot.csv} (columns \texttt{estimate} and \texttt{estimate\_minutes\_weighted}) and \texttt{table1\_outcomes\_by\_arm\_pilot.csv} (columns \texttt{acceptance\_taskw} and \texttt{acceptance\_minw}). The minutes-weighted contrasts are point estimates; the estimation script computes their bootstrap with 200 resamples only and the intervals are not reported.157\end{tablenotes}158\end{table}159160\subsection{The eight re-judged runs of amendment A-008}\label{app:robustness-rejudged}161162The judge subprocess crashed on eight pilot runs whose rendered deliverables contained null bytes, and the eight were judged after the renderer was changed to strip control bytes (amendment A-008). All eight are runs of the academic-research scenario RES-01 in arms A1 to A4, first and second repeats, and their first judgments are timed between 01:15 and 01:19~UTC on 3~November 2026, after the fix; the third repeats of RES-01, executed later under the scale rule, were judged by the fixed renderer in the ordinary course. The quality scores of the fifteen RES-01 runs are 22.5 to 24.6 in A0 (not accepted) and 78.8 to 89.2 in A1--A4 (all accepted): no shell-arm run of RES-01 is near the threshold, which already indicates that the re-judging cannot have changed any acceptance. Table~\ref{tab:rejudged} recomputes the arm acceptance rates and the four preregistered contrasts, with fresh bootstrap and permutation draws, on three samples: the full pilot; the pilot without the eight re-judged runs (142 runs, with RES-01 represented in A1--A4 by its third repeat only); and the pilot without RES-01 altogether (135 runs, $J=9$). Excluding the eight runs leaves every acceptance rate and every contrast unchanged to three decimals, because the remaining RES-01 repeat has the same acceptance as the excluded ones. Excluding the scenario changes the rates by the removal of one always-accepted scenario from nine or ten---A1 falls from 0.867 to 0.852, A2 and A3 from 0.900 to 0.889, A4 from 0.833 to 0.815---and changes the contrasts by at most 0.015, with intervals and permutation $p$-values of the same width and conclusion (shell premium $+0.852$, interval $0.630$ to $1.000$, $p=0.008$; computer-use premium $+0.037$, interval $0.000$ to $0.111$, $p=1.000$).163% source: analysis/tables/table14_extra_descriptives.csv block E_rejudged (acceptance_taskw, estimate, ci, perm_p, quality and S of the 15 RES-01 runs, first_judged_utc)164165\begin{table}[htbp]166\centering167\caption{Sensitivity to the eight re-judged runs (A-008): acceptance rates and contrasts on three samples}168\label{tab:rejudged}169\footnotesize170\setlength{\tabcolsep}{4pt}171\begin{adjustbox}{max width=\textwidth}172\begin{tabular}{@{}lrrr@{}}173\toprule174 & Full pilot & Without the 8 re-judged runs & Without RES-01 \\175 & ($n=150$, $J=10$) & ($n=142$, $J=10$) & ($n=135$, $J=9$) \\176\midrule177Acceptance, A0 & 0.000 & 0.000 & 0.000 \\178Acceptance, A1 & 0.867 & 0.867 & 0.852 \\179Acceptance, A2 & 0.900 & 0.900 & 0.889 \\180Acceptance, A3 & 0.900 & 0.900 & 0.889 \\181Acceptance, A4 & 0.833 & 0.833 & 0.815 \\182\addlinespace183A3 $-$ A1: estimate [95\% CI]; $p$ & $+0.033$ [$0.000$, $0.100$]; 1.000 & $+0.033$ [$0.000$, $0.100$]; 1.000 & $+0.037$ [$0.000$, $0.111$]; 1.000 \\184A1 $-$ A0 & $+0.867$ [$0.633$, $1.000$]; 0.004 & $+0.867$ [$0.667$, $1.000$]; 0.004 & $+0.852$ [$0.630$, $1.000$]; 0.008 \\185A4 $-$ A3 & $-0.067$ [$-0.167$, $0.000$]; 0.496 & $-0.067$ [$-0.167$, $0.000$]; 0.500 & $-0.074$ [$-0.185$, $0.000$]; 0.505 \\186A2 $-$ A1 & $+0.033$ [$-0.067$, $0.133$]; 1.000 & $+0.033$ [$-0.067$, $0.133$]; 1.000 & $+0.037$ [$-0.074$, $0.148$]; 1.000 \\187\bottomrule188\end{tabular}189\end{adjustbox}190\begin{tablenotes}191\footnotesize192\item Source: \texttt{analysis/tables/table14\_extra\_descriptives.csv} block \texttt{E\_rejudged} (script \texttt{analysis/robustness/extra\_descriptives.py}; scenario-cluster bootstrap $B=5{,}000$, permutation 20,000, seed 42). The full-pilot column differs from Table~\ref{tab:contrasts} only through the bootstrap draw (the A1 $-$ A0 lower bound is 0.633 here and 0.667 there), which illustrates the resolution of a percentile interval over ten clusters.193\end{tablenotes}194\end{table}195196\subsection{The three reruns of amendment A-010}\label{app:robustness-a010}197198Three A2 runs lost their output directories at collection (amendment A-010) and were reset and rerun under the same identifiers; their first attempts are archived with the suffix \texttt{.attempt1} and their events are kept. Table~\ref{tab:a010} places the first attempt and the rerun of each side by side. The three first sessions ran to a successful result event in 167 to 184 seconds with 12 to 14 tool uses and costs of 0.56 to 0.69 dollars, and their streams list the deliverables written to the output directory; what the controller collected was empty, so the receipts recorded deterministic scores of 0.000, 0.000 and 0.045 and the judges scored the empty packages 0. The reruns, on the same nodes' peers, produced an accepted package for the second RE-01 repeat (quality 100.0) and for LEGAL-01 (quality 90.4), and a package for the first RE-01 repeat that the judges scored 98.75 but that failed the adjusted-prices criterion, which is the A2 failure of Section~\ref{sec:results-heterogeneity}. The rerun is the run of record in every table; the first attempts enter only the infrastructure statistics of Section~\ref{sec:reliability-infrastructure} and the ledger, where their 1.88 dollars of worker cost and their judge calls are the difference between the ledger's pilot worker total (126.30 dollars) and the validated total (124.43).199% source: table14 block F_a010; table14 block H_spend; data/raw/wp42.sqlite ledger200201\begin{table}[htbp]202\centering203\caption{The three lost collections of amendment A-010: first attempt and rerun}204\label{tab:a010}205\footnotesize206\setlength{\tabcolsep}{4pt}207\begin{adjustbox}{max width=\textwidth}208\begin{tabular}{@{}llrrrrrr@{}}209\toprule210Run & Attempt & Events & Tool uses & Result & Duration (s) & Cost (USD) & $D$ / $Q$ / $S$ \\211\midrule212RE-01\_A2\_r1 & first (lost) & 62 & 13 & success & 166.7 & 0.556 & 0 / 0.0 / 0 \\213 & rerun (M4M64a) & -- & 15 & success & 179.9 & 0.617 & 0 / 98.75 / 0 \\214RE-01\_A2\_r2 & first (lost) & 101 & 14 & success & 168.2 & 0.627 & 0 / 0.0 / 0 \\215 & rerun (M3U96a) & -- & 17 & success & 180.9 & 0.666 & 1 / 100.0 / 1 \\216LEGAL-01\_A2\_r1 & first (lost) & 67 & 12 & success & 183.8 & 0.695 & 0 / 0.0 / 0 \\217 & rerun (M3U96a) & -- & 9 & success & 179.4 & 0.661 & 1 / 90.42 / 1 \\218\bottomrule219\end{tabular}220\end{adjustbox}221\begin{tablenotes}222\footnotesize223\item Source: \texttt{analysis/tables/table14\_extra\_descriptives.csv} block \texttt{F\_a010}, computed from \texttt{data/raw/runs/<run>/raw\_stream.jsonl.attempt1}, \texttt{receipt.json.attempt1}, \texttt{judgments.json.attempt1} and the validated rerun. Tool uses of the first attempt are counted from the stream; those of the rerun are the validated \texttt{tool\_calls}. The first attempts' deterministic scores were 0.000, 0.000 and 0.045 on empty collections.224\end{tablenotes}225\end{table}226227\subsection{Executor node}\label{app:robustness-node}228229Assumption~\ref{ass:exchange} requires that the distribution of a run not depend on the executor node. Table~\ref{tab:node} reports the number of runs and the acceptance rate by node and arm. Arm A2 ran only on the two nodes where the automation layer answers; the other arms were spread over the four nodes by the least-loaded rule. Pooling arms A1 to A4, acceptance is 0.833 on M3U96a (30 runs), 0.966 on M3U96b (29), 0.871 on M4M64a (31) and 0.833 on M4M64b (30), and the restricted arm is zero on every node. The spread of 0.13 across nodes is within the binomial variation of about 30 runs at a rate of 0.88 (standard deviation 0.06); the cells with the lowest rates, A1 on M3U96a (0.714 on 7 runs) and A4 on M3U96a and M4M64a (0.667 on 6 and 3 runs), are the cells that happened to receive the ADM-01 runs. No node effect is visible, and none can be tested with precision at these counts.230% source: analysis/tables/table14_extra_descriptives.csv block B_node231232\begin{table}[htbp]233\centering234\caption{Pilot runs and acceptance by executor node and arm}235\label{tab:node}236\footnotesize237\setlength{\tabcolsep}{4pt}238\begin{adjustbox}{max width=\textwidth}239\begin{tabular}{@{}lrrrrrrrrrrrr@{}}240\toprule241 & \multicolumn{2}{c}{A0} & \multicolumn{2}{c}{A1} & \multicolumn{2}{c}{A2} & \multicolumn{2}{c}{A3} & \multicolumn{2}{c}{A4} & \multicolumn{2}{c}{A1--A4} \\242\cmidrule(lr){2-3}\cmidrule(lr){4-5}\cmidrule(lr){6-7}\cmidrule(lr){8-9}\cmidrule(lr){10-11}\cmidrule(lr){12-13}243Node & $n$ & $S$ & $n$ & $S$ & $n$ & $S$ & $n$ & $S$ & $n$ & $S$ & $n$ & $S$ \\244\midrule245M3U96a & 7 & 0.000 & 7 & 0.714 & 12 & 1.000 & 5 & 0.800 & 6 & 0.667 & 30 & 0.833 \\246M3U96b & 11 & 0.000 & 9 & 1.000 & -- & -- & 8 & 0.875 & 12 & 1.000 & 29 & 0.966 \\247M4M64a & 7 & 0.000 & 4 & 1.000 & 18 & 0.833 & 6 & 1.000 & 3 & 0.667 & 31 & 0.871 \\248M4M64b & 5 & 0.000 & 10 & 0.800 & -- & -- & 11 & 0.909 & 9 & 0.778 & 30 & 0.833 \\249\bottomrule250\end{tabular}251\end{adjustbox}252\begin{tablenotes}253\footnotesize254\item Source: \texttt{analysis/tables/table14\_extra\_descriptives.csv} block \texttt{B\_node}. Arm A2 is restricted to M3U96a and M4M64a by design (amendment A-003).255\end{tablenotes}256\end{table}257258\subsection{Judge severity by model and by arm}\label{app:robustness-judges}259260Table~\ref{tab:judges} reports the mean and standard deviation of the rubric scores by judge model over the 150 pilot runs, and the mean by judge and arm. The Opus judge and the two Sonnet judges have the same mean to within 0.06 points over all runs and differ by less than two points within any arm; the two Sonnet draws differ from each other by less than 0.5 points within any arm except A4 (83.5 against 85.5). Neither model is the more severe judge on average; what the data show is that the Opus judge is the more \emph{dispersed} relative to the Sonnet pair (median absolute difference 4.4 points against 1.25 between the two Sonnet draws, Section~\ref{sec:data-instruments}), in both directions.261% source: analysis/tables/table7_judges_pilot.csv; table14 block I_judges262263\begin{table}[htbp]264\centering265\caption{Judge scores by judge model and by arm, 150 pilot runs}266\label{tab:judges}267\small268\begin{tabular}{@{}lrrrrrrr@{}}269\toprule270 & \multicolumn{2}{c}{All runs} & \multicolumn{5}{c}{Mean score by arm} \\271\cmidrule(lr){2-3}\cmidrule(lr){4-8}272Judge & Mean (s.d.) & $n$ & A0 & A1 & A2 & A3 & A4 \\273\midrule274Judge 1, \texttt{claude-opus-5-5} & 73.08 (27.41) & 150 & 28.5 & 82.0 & 84.0 & 85.4 & 85.4 \\275Judge 2, \texttt{claude-sonnet-5-5} & \multirow{2}{*}{73.02 (26.80)} & \multirow{2}{*}{300} & 26.9 & 82.3 & 85.1 & 85.9 & 83.5 \\276Judge 3, \texttt{claude-sonnet-5-5} & & & 26.8 & 82.8 & 85.8 & 85.8 & 85.5 \\277\bottomrule278\end{tabular}279\begin{tablenotes}280\footnotesize281\item Sources: \texttt{analysis/tables/table7\_judges\_pilot.csv} (mean and standard deviation by judge model; the two Sonnet draws are pooled, $n=300$) and \texttt{table14\_extra\_descriptives.csv} block \texttt{I\_judges} (means by judge index and arm).282\end{tablenotes}283\end{table}284285\subsection{The human-cost assumption}\label{app:robustness-human}286287Table~\ref{tab:human-assumption} tabulates the band drawn in Figure~\ref{fig:economic}: the professional-minutes estimate recorded by each scenario's author in its metadata, multiplied by three assumed hourly wages. These figures are assumptions used for scale only. No human performed any scenario; no human time was measured; the figures are not estimates of what a professional would take or cost, and they are not used in any estimate, contrast or frontier of the paper.288% source: analysis/tables/table13_human_cost_assumption.csv289290\begin{table}[htbp]291\centering292\caption{The human-cost assumption behind the band of Figure~\ref{fig:economic} (not a measurement)}293\label{tab:human-assumption}294\small295\begin{tabular}{@{}lrrrr@{}}296\toprule297Scenario & Author's minutes estimate & at 40 CAD/h & at 80 CAD/h & at 120 CAD/h \\298\midrule299ACC-01 & 420 & 280 & 560 & 840 \\300FIN-01 & 300 & 200 & 400 & 600 \\301RE-01 & 300 & 200 & 400 & 600 \\302RES-01 & 300 & 200 & 400 & 600 \\303BI-01 & 300 & 200 & 400 & 600 \\304ADM-01 & 180 & 120 & 240 & 360 \\305BANK-01 & 180 & 120 & 240 & 360 \\306LEGAL-01 & 150 & 100 & 200 & 300 \\307HR-01 & 150 & 100 & 200 & 300 \\308PROC-01 & 150 & 100 & 200 & 300 \\309\bottomrule310\end{tabular}311\begin{tablenotes}312\footnotesize313\item Source: \texttt{analysis/tables/table13\_human\_cost\_assumption.csv} (script \texttt{analysis/robustness/economic\_sensitivity.py}); minutes from each scenario's \texttt{task.json}. Amounts in Canadian dollars at the stated wage; the API costs of the paper are in United States dollars as charged.314\end{tablenotes}315\end{table}316