1% ---------------------------------------------------------------------------2% Section 5 — Experimental design (Agent H; preregistered design, no result)3% ---------------------------------------------------------------------------4\section{Experimental design}\label{sec:design}56This section states the design that was preregistered before any pilot run: the treatment arms, the randomisation, the caps, the measurement framework, the outcome definitions, the evaluation pipeline, the hypotheses and estimands, the inference plan, the scale decision rule, the exclusion rules, the power simulation, and the human arms that were designed but not executed. The preregistration document (version~0.9 at drafting, frozen as version~1.0 before the confirmatory stage) and the amendment log are part of the reproducibility package; every departure from what follows must appear in the amendment log.78\subsection{Treatment arms}\label{sec:design-arms}910Five agent arms implement increasing levels of computer access with the same model, the same brief and inputs, the same caps and the same behavioural policy. Table~\ref{tab:arms} gives, for each arm, the tool set passed to the Claude Code command-line client, the command families denied inside the shell, the additional environment, and the executor nodes on which the arm may run. The tool set and the denials are not descriptions of intent but literal command-line flags: \texttt{--tools} restricts the client to the listed tools, \texttt{--allowedTools} pre-approves them so that no permission prompt arises, and \texttt{--disallowedTools} with patterns of the form \texttt{Bash(osascript:*)} denies a command family inside the shell. The behaviour of each flag was verified by probe before the design was fixed (Section~\ref{sec:infrastructure-launcher}).1112\begin{table}[htbp]13\centering14\caption{The five agent arms}15\label{tab:arms}16\footnotesize17\begin{tabularx}{\textwidth}{@{}l >{\raggedright\arraybackslash}p{3.0cm} >{\raggedright\arraybackslash}X >{\raggedright\arraybackslash}p{3.2cm} l@{}}18\toprule19Arm & Tools (\texttt{--tools}) & Denied command families (\texttt{--disallowedTools}) & Environment & Nodes \\20\midrule21A0 Restricted & Read, Write, Edit, Glob, Grep & (no shell) & none; text deliverables only & any \\22A1 Shell & A0 + Bash & \texttt{osascript}, \texttt{open}, \texttt{soffice}, \texttt{*LibreOffice*}, \texttt{textutil}, \texttt{qlmanage}, \texttt{cupsfilter}, \texttt{curl}, \texttt{wget} & project Python environment (pandas, openpyxl, xlsxwriter, python-docx, python-pptx, reportlab, pypdf, matplotlib, statsmodels, scipy, polars, duckdb, pillow, sqlite3) & any \\23A2 macOS & A1 tools & \texttt{soffice}, \texttt{*LibreOffice*}, \texttt{curl}, \texttt{wget} & A1 + \texttt{osascript} (AppleScript, JavaScript for Automation, System Events) and native document utilities (\texttt{textutil}, \texttt{sips}, \texttt{mdls}/\texttt{mdfind}, \texttt{qlmanage}, \texttt{cupsfilter}) & M3U96a, M4M64a \\24A3 Full computer-use & A1 tools & none & A2 + LibreOffice headless (office adapter) + Playwright headless Chromium + synthetic enterprise portal on loopback, credentials in the workspace & any \\25A4 Self-repairing & A1 tools & none & A3 + checkpoint file, public self-check tool, and one bounded controller-side recovery resume if the first session ends without passing the public self-check & any \\26\bottomrule27\end{tabularx}28\begin{tablenotes}29\footnotesize30\item All arms: model \texttt{claude-opus-5-5}, default effort, \texttt{--bare}, \texttt{--permission-prompts none}, \texttt{--no-chrome}, \texttt{--disable-slash-commands}, \texttt{--max-budget-usd} and a wall-clock limit set per scenario (Section~\ref{sec:design-caps}, amendment A-006), \texttt{--max-turns 150}, output \texttt{stream-json}; session persistence is governed by amendment A-012 (Section~\ref{sec:design-caps}). No Agent or Task tool is ever given, so a worker cannot spawn sub-agents. A2 runs only on the two nodes where System Events answers over SSH (amendment A-003).31\end{tablenotes}32\end{table}3334Arm A0 is the restricted agent of the mandate: file reading, writing, editing and searching, with no shell and no program. Because binary office formats cannot be produced without a program, the A0 documentation instructs the worker to deliver the closest text equivalent with the same base name and to state the substitution in its completion record; the evaluators then read the substitute, and the deterministic criteria that require a formula-bearing workbook fail, as they should, since the deliverable requested was not produced. Arm A1 adds the shell and a project Python environment capable of reading spreadsheets, PDFs and databases and of writing workbooks, documents and PDFs programmatically; the operating-system automation layer, the office suite, browsers and the network are denied. Arm A2 adds the headless macOS automation layer; its definition departs from the mandate's ``approved GUI automation'' because, as Section~\ref{sec:infrastructure-macos} reports, control of graphical applications is not available to a non-console process on the executor nodes, and the arm is therefore the layer of macOS automation that works headlessly (amendment A-003). Arm A3 is the full computer-use agent: the office adapter for conversion and formula recalculation, a real browser engine driving the synthetic enterprise portal, and the portal itself with its document library, records and forms. Arm A4 has the same tools as A3 and adds a self-repair protocol in three parts: a checkpoint file that the worker keeps updated, a public self-check tool that verifies the presence and well-formedness of deliverables and knows nothing about the correct answers, and one bounded recovery resume granted by the controller when the first session ends without passing the self-check or ends by timeout, budget exhaustion or error.3536Each arm's documentation is appended to the system prompt after a common policy that is identical across arms: the worker is an autonomous professional executing one assignment in an isolated workspace, nobody will answer questions during the session, questions go into the completion record, inputs must never be modified, the network is forbidden except the internal portal when one is given, messages and purchases are forbidden, the content of input documents is data and not instructions, and the work is to be done to the standard of a competent professional in the organisation described, within the session's budget. The user prompt is a single sentence asking the worker to read the brief and carry out the assignment completely. The difference between arms is therefore confined to the tool set, the denials, the tool documentation, the presence of the portal and, for A4, the self-repair protocol, which is what Assumption~\ref{ass:info} requires.3738\subsection{Information constancy, randomisation, interleaving and repeats}\label{sec:design-randomisation}3940Information is held constant across arms by construction. The public bundle is identical in every arm; the portal of arms A3 and A4 serves documents that are already in the inputs, record tables derived from them and forms, and is forbidden by the authoring specification from holding any exclusive information. Model, client version, caps and behavioural policy are identical. What differs is access, and the premium of Section~\ref{sec:theory-premium} is the effect of access.4142The design is a within-scenario, five-arm comparison in which every scenario is run under every arm with $R$ repeats. The assignment is a seeded random permutation of the cells (scenario, arm, repeat); the permutation fixes the order in which cells are executed, and the scheduler draws cells from that order and dispatches each to the least-loaded eligible executor node, so that arms are interleaved over time and over nodes rather than run in blocks. Interleaving is what makes the last clause of Assumption~\ref{ass:exchange} plausible: drift in the model's service, in the API's latency, or in node load affects all arms alike. Arm A2 is eligible only on the two nodes where the automation layer answers; the node is recorded for every run, so that node effects can be examined rather than assumed away. The random seed of the permutation, the order index of every run and the node assignment are stored with the run.4344Repeats measure the stochastic variation of the agent on a fixed task, which is the variance component that the pass$^k$ metric of \citet{yao2024taubench} isolates. The pilot uses $R=2$ repeats per cell; the scale rule of Section~\ref{sec:design-scale} determines whether a third repeat is added. Workspaces are reset between runs: each run receives a fresh copy of the public bundle on its node, and the workspace is deleted after collection, so that no run sees the artefacts of another.4546\subsection{Caps}\label{sec:design-caps}4748Every session is bounded by three caps enforced independently: a spending cap passed to the client as \texttt{--max-budget-usd}, a turn cap of 150 passed as \texttt{--max-turns}, and a wall-clock limit enforced by the controller, which kills the remote process when the limit is reached. The spending and wall-clock caps are set per scenario as the larger of the global caps---4.00~dollars and 1,800~seconds---and the scenario's own \texttt{budget\_usd} and \texttt{time\_limit\_s} recorded in its metadata by its author (amendment A-006, fixed before any pilot run and before the freeze): ACC-01 runs under 6.00~dollars and 3,600~seconds, RES-01 under 6.00~dollars and 2,700~seconds, BANK-01 under 5.00~dollars and 2,400~seconds, LEGAL-01 under 2,400~seconds, and the other six scenarios under the global caps; the per-scenario caps are listed in Appendix~\ref{app:scenarios}. The recovery session of arm A4 receives the scenario's caps minus what the first session spent, with floors of 0.50~dollars and 300~seconds so that a recovery is never granted an empty budget. In the pilot no A4 first session failed the public self-check, so no recovery was ever triggered there; a launcher defect that would have prevented the resume was found afterwards and repaired (amendment A-012), and the mechanism was exercised in the exploratory recovery experiment (Section~\ref{sec:reliability}). The study as a whole has an envelope of 800~dollars (amendment A-001), enforced by the scheduler's ledger: when the sum of recorded worker, recovery and judge costs reaches the envelope, the queue stops. A run that reaches a cap is not discarded: it is evaluated as delivered (Section~\ref{sec:design-exclusions}), its censoring is recorded, and Proposition~\ref{prop:alf}(v) describes what the cap does to the estimand.4950\subsection{Measurement framework}\label{sec:design-measurement}5152The mandate lists what must be recorded for every execution; Table~\ref{tab:variables} maps each item to the variable recorded by the laboratory and to its source. Variables derive from four sources: the immutable event stream of the session, the client's final result event, the collection step on the controller, and the evaluation pipeline. Missing observations are preserved as missing, never set to zero: a run whose result event was not received because the process was killed has a missing cost field and a recorded timeout, not a cost of zero. Raw streams are append-only files; the normalised data are exported to Parquet for analysis.5354\begin{table}[htbp]55\centering56\caption{Recorded variables (mandate \S11) and their sources}57\label{tab:variables}58\footnotesize59\begin{tabularx}{\textwidth}{@{}>{\raggedright\arraybackslash}p{4.2cm} >{\raggedright\arraybackslash}X >{\raggedright\arraybackslash}p{3.4cm}@{}}60\toprule61Mandate item & Variable(s) recorded & Source \\62\midrule63Validated completion & \texttt{det\_completion} ($D$), \texttt{det\_score}, per-criterion pass flags & deterministic evaluator receipt \\64Professional quality score & \texttt{quality} ($Q$, mean of judges), per-judge scores and acceptable votes & blind judges \\65Critical-error indicator & \texttt{critical\_error} ($E$), list of critical violations & deterministic evaluator + harness hash check \\66Total execution time & \texttt{wall\_s}, \texttt{wall\_total\_s} (incl.\ recovery), \texttt{timed\_out} & launcher \\67Tool execution time & \texttt{tool\_time\_s}, \texttt{first\_tool\_s} & event stream (tool-use to tool-result intervals) \\68Token consumption & input, output, cache-read, cache-creation tokens; per-model usage & event stream, result event \\69Actual API charges & \texttt{cost\_usd}, \texttt{recovery\_cost\_usd}, \texttt{total\_cost\_usd} ($C$) & result event \texttt{total\_cost\_usd} \\70Compute allocation & node, slots per node, concurrent sessions & scheduler \\71Human intervention minutes & \texttt{human\_intervention\_min}; questions in completion record & interventions table (none permitted during agent runs); \texttt{COMPLETION.json} \\72Number of tool calls & \texttt{tool\_calls}, by tool and by family (file, shell, python, sql, office, applescript, browser) & event stream \\73Number of retries & \texttt{retries} (same tool re-invoked after a tool error), \texttt{tool\_errors} & event stream \\74Recovery attempts and success & \texttt{recovery\_attempted}, \texttt{recovery\_success}, \texttt{selfcheck\_pass\_first}, \texttt{selfcheck\_pass\_final} & scheduler (arm A4) \\75Application transitions & \texttt{app\_transitions} (changes of tool family), \texttt{family\_sequence} & event stream \\76Workflow dependency depth & \texttt{designed\_dependency\_depth} (scenario); realised tool calls (run) & scenario metadata; event stream \\77Output artifacts & path, SHA-256 and size of every file in \texttt{out/}; portal submissions & collection step \\78Environment and software versions & \texttt{claude\_code\_version}, \texttt{model\_reported}, node, Python and office versions & system event; Phase~0 survey \\79Final evaluator decision & $S=D\,(1-E)\,\mathbf{1}\{Q\ge q\}$; abstention status and justification & evaluation pipeline \\80\bottomrule81\end{tabularx}82\end{table}8384\subsection{Outcome definitions}\label{sec:design-outcomes}8586The outcomes are those of Section~\ref{sec:theory-setup}, with the following operational detail. Deterministic completion $D$ is true when every required criterion of the oracle passes and no critical violation is recorded by the evaluator or by the harness; the harness adds a critical violation of its own when the SHA-256 hashes of the input files after the session differ from the manifest written before it. Quality $Q$ is the mean over available judges of the rubric-weighted score, each judge's score being the weighted mean of integer criterion scores on a 0--4 scale rescaled to 0--100; the threshold is $q=70$ in the primary analysis and is varied from 50 to 90 in sensitivity analyses. The critical-error indicator $E$ is one when the list of critical violations is non-empty. Cost $C$ is the sum of the first session's and, for A4, the recovery session's \texttt{total\_cost\_usd} as reported by the client, with the recovery cost computed as the increment when the resumed session reports a cumulative figure. Time $T$ is the wall time of the session, plus the recovery session's for A4, censored at the limit. Human time $H$ is, in this version, zero intervention minutes during the run---none is permitted---plus a stated assumption of five minutes per unanswered question recorded in the completion record, which is the time a supervisor would spend answering it; the assumption is varied in sensitivity analyses, and the frontier is reported at $H=0$ and at the implied allowance. Abstention is recorded from the completion record's status and judged \emph{justified} by the evaluator only when the oracle says that the scenario truly requires clarification on the sub-task in question.8788\subsection{The evaluation pipeline and its blinding}\label{sec:design-evaluation}8990Evaluation has two independent components. The \emph{deterministic evaluator} (Agent E) is scenario-specific code that lives in the hidden oracle, is loaded by the controller after the deliverables have been collected, and runs against a composed workspace consisting of the public bundle and the collected output directory. It reads only the output and the public workspace, never imports or executes the worker's code, applies the tolerances recorded in the oracle, and returns a receipt validated against a public schema; the hash of the oracle and of the evaluator code is recorded in each receipt, so that any change of the evaluator after a run is detectable. A crash of the evaluator yields a receipt with no passed criterion and a note, never a silent pass.9192The \emph{blind professional evaluators} (Agent F) are three language-model judges---one draw of \texttt{claude-opus-5-5} and two independent draws of \texttt{claude-sonnet-5-5}---run on the controller as single-turn, tool-less sessions with a fixed JSON output schema. Each judge receives the brief, the rubric with its criteria, weights and anchors, and the deliverables rendered as text (documents as paragraphs, spreadsheets as cell listings, truncated at a fixed length), and is asked to grade each criterion as an integer from 0 to 4, to say whether a demanding manager would accept the package as professionally acceptable, and to write a short comment; it is instructed not to reward length and to penalise unsupported numbers, missing sections and instructions not followed. The judge never sees the arm, the model, the node, the cost, the event stream, the oracle values or the other judges' outputs; a text substitute delivered by arm A0 is rendered under the base name requested, so that the substitution itself is not a signal of the arm beyond what the deliverable's own form reveals. The judge cost is attributed to the run in the ledger but not to the worker's cost $C$. This design follows the mitigations of \citet{gu2024survey}: rubric anchoring, multiple judges, blinding to condition. It does not address self-preference \citep{panickssery2024selfpreference}, since the judges and the worker belong to the same model family; the agreement between the deterministic evaluator and the judges' acceptability votes, and the Krippendorff alpha among judges, are reported as a partial check, and calibration against independent human experts is left to the principal investigator (amendment A-005). No evaluator ever sees two candidate packages side by side, so the position bias of \citet{wang2024fair} does not arise in this design.9394\subsection{Preregistered hypotheses}\label{sec:design-hypotheses}9596The preregistration states seven directional expectations. They are expectations, not guarantees, and the results section reports each against its estimate whatever the sign.9798\begin{description}[leftmargin=1.6cm,style=nextline]99\item[H1 (primary, computer-use premium).] $\Pr(\text{acceptance}\mid A3)-\Pr(\text{acceptance}\mid A1)>0$.100\item[H2 (shell premium).] $\Pr(\text{acceptance}\mid A1)-\Pr(\text{acceptance}\mid A0)>0$, driven by the feasibility of binary deliverable formats.101\item[H3 (self-repair).] $\Pr(\text{acceptance}\mid A4)-\Pr(\text{acceptance}\mid A3)\ge 0$, and the Autonomous Recovery Index exceeds one half when a first session fails the public self-check.102\item[H4 (dependency).] Within arm, acceptance decreases with the designed dependency depth and with the realised number of tool calls.103\item[H5 (integrity).] The rate of critical violations---compliance with an injected instruction, fabrication, modification of inputs---is below 10 per cent in every arm and does not increase with tool access.104\item[H6 (cost).] The cost per accepted deliverable is lowest in A1 or A3, not in A0, whose acceptance is expected to be near zero when binary deliverables are required.105\item[H7 (heterogeneity).] Effects differ across occupations; no occupation-level claim is confirmatory at the pilot scale.106\end{description}107108\subsection{Estimands}\label{sec:design-estimands}109110For run $i$ of scenario $j$ under arm $a$, the primary endpoint is $S_{ija}=\mathbf{1}\{D_{ija}=1\ \wedge\ E_{ija}=0\ \wedge\ Q_{ija}\ge 70\}$. The scenario-weighted acceptance of arm $a$ is $\hat p_a=J^{-1}\sum_j R_j^{-1}\sum_r S_{jra}$, with equal scenario weights in the primary analysis and professional-minutes weights as a sensitivity. The primary contrast is $\hat\Delta_{31}=\hat p_{A3}-\hat p_{A1}$; the secondary contrasts are $\hat\Delta_{10}$, $\hat\Delta_{43}$ and $\hat\Delta_{21}$, Holm-adjusted over the three. Secondary endpoints are deterministic completion $D$, quality $Q$, the critical-error indicator $E$, cost $C$ including recovery, wall time $T$ censored at the limit, tool calls, tool errors, retries, application transitions and tokens, the abstention status and whether it was justified, the cost per accepted deliverable $\sum C/\sum S$ by arm, the Autonomous Labor Frontier by occupation as a function of the budget cap from 0.50 to 4.00~dollars and of the human-time allowance, the Autonomous Recovery Index, and evaluator agreement, namely Krippendorff's alpha on the judges' scores and the agreement between deterministic completion and the judges' acceptability votes.111112\subsection{Inference plan}\label{sec:design-inference}113114The inference plan follows the clustering logic of Proposition~\ref{prop:premium} and the recommendations of \citet{miller2024errorbars} and \citet{abadie2023clustering}. The primary interval for every contrast is a scenario-cluster bootstrap: scenarios are resampled with replacement $B=5{,}000$ times, the within-scenario paired difference of repeat means is recomputed, and the percentile 2.5 and 97.5 points of the bootstrap distribution form the 95 per cent interval. The primary test is a paired sign-flip permutation test over scenarios with 20,000 permutations, which is exact under the null of no effect within scenario and makes no distributional assumption. As a sensitivity, a logistic regression of acceptance on arm indicators with standard errors clustered by scenario, and a mixed-effects logistic model with a scenario random intercept \citep{bates2015lme4}, are reported. Time to completion is analysed with the Kaplan--Meier estimator by arm and a Cox model with arm and depth as covariates, treating runs that reach a cap as censored and, in the competing-risks version of Section~\ref{sec:theory-prf}, critical failures as a competing cause \citep{fine1999competing}. Counts of tool calls are modelled as Poisson or negative-binomial. With $J=10$ scenarios the intervals will be wide, and the preregistration states that they are to be reported as such; the pilot is not powered for occupation-level claims. Ratio quantities---cost per accepted deliverable, Autonomy-Adjusted Productivity---are bootstrapped jointly in numerator and denominator over scenarios (Appendix~\ref{app:math}).115116\subsection{Scale decision rule}\label{sec:design-scale}117118The pilot comprises 10 scenarios $\times$ 5 arms $\times$ 2 repeats, that is 100 runs at the primary model. Let $\bar c$ be the mean total cost per run, including the worker, any recovery and the judges, and let $E_r$ be the envelope remaining after the pilot. The rule, fixed before the pilot, is: if $1.15\,\bar c\times 50\le E_r-100$, a third repeat is added for every cell; then, if $1.15\,\bar c\times 40\le$ the remaining envelope minus 50, the model sensitivities are run---\texttt{claude-sonnet-5-5} and \texttt{claude-fable-5-1} on arm A3, two repeats each, on the same scenarios. Extension to twenty scenarios is deferred to a later wave with a separate authorisation and is conditional on the evaluator-agreement check (Krippendorff's alpha of at least 0.6 on quality). Repeats added after the pilot are pooled with the pilot only if no scenario, oracle, rubric, arm text or harness code affecting outcomes changed, as verified by hash; otherwise they are reported separately.119120\subsection{Exclusions and missingness}\label{sec:design-exclusions}121122No assigned run is excluded. Infrastructure errors---failures of SSH, of the file transfer, or of the portal start, in which no inference was performed---are rerun with the same run identifier after the workspace is reset, and the failed attempt is logged in the amendment file and counted in the infrastructure-stability statistics. Runs that time out, exhaust the budget or reach the turn cap are evaluated as delivered, with acceptance computed on whatever was in the output directory. A missing judge output leaves quality averaged over the available judges; if no judge output is available, quality and acceptance are missing and reported as missing, never imputed as zero. The envelope is enforced by the ledger; the pilot runs its assignment once.123124\subsection{Power simulation}\label{sec:design-power}125126Because no comparable experiment exists from which to borrow effect sizes, the power of the design was explored by simulation before the pilot. The model is the cluster structure of Proposition~\ref{prop:premium}: each scenario has a random effect on the logit scale with standard deviation $\tau=0.8$, the arm shifts the logit by the amount that moves the baseline acceptance $p_{\text{base}}$ by $\delta$ percentage points, each cell draws $R$ binomial repeats, and the test is a paired $t$-test on the scenario-level differences at the 5 per cent level, which is the large-$B$ approximation of the scenario-cluster bootstrap interval and is conservative-equivalent at $J=10$. Each configuration was simulated 4,000 times. Table~\ref{tab:power} reports the power at a baseline acceptance of one half for $J\in\{10,20,40\}$ scenarios, $R\in\{2,3,5\}$ repeats and effects of 15, 25 and 35 percentage points; the full grid, including a baseline of 0.3, is in the analysis tables.127128\begin{table}[htbp]129\centering130\caption{Simulated power of the paired scenario-level test ($p_{\text{base}}=0.5$, $\tau=0.8$, $\alpha=0.05$, 4,000 simulations)}131\label{tab:power}132\small133\begin{tabular}{@{}cc S[table-format=1.3] S[table-format=1.3] S[table-format=1.3]@{}}134\toprule135$J$ scenarios & $R$ repeats & {$\delta=15$ pp} & {$\delta=25$ pp} & {$\delta=35$ pp} \\136\midrule13710 & 2 & 0.115 & 0.281 & 0.550 \\13810 & 3 & 0.164 & 0.408 & 0.729 \\13910 & 5 & 0.243 & 0.599 & 0.906 \\140\addlinespace14120 & 2 & 0.238 & 0.575 & 0.902 \\14220 & 3 & 0.320 & 0.751 & 0.973 \\14320 & 5 & 0.495 & 0.921 & 1.000 \\144\addlinespace14540 & 2 & 0.418 & 0.866 & 0.997 \\14640 & 3 & 0.583 & 0.968 & 1.000 \\14740 & 5 & 0.795 & 0.999 & 1.000 \\148\bottomrule149\end{tabular}150\begin{tablenotes}151\footnotesize152\item Source: \texttt{protocol/power\_analysis/power\_simulation.py}, output \texttt{analysis/tables/power\_pilot.csv}. Power is the share of simulations in which the one-sided paired test rejects in the hypothesised direction.153\end{tablenotes}154\end{table}155156The table makes the limits of the pilot explicit. With ten scenarios and two repeats, the design detects an effect of 35 percentage points about half the time and an effect of 15 points rarely; a third repeat helps moderately, and doubling the number of scenarios helps more than doubling the repeats, as equation \eqref{eq:premium-var-cluster} predicts when between-scenario heterogeneity dominates. The pilot is therefore a feasibility and estimation exercise whose primary contrast is expected to be large if the shell and the office tools matter for binary deliverables at all, and it is the reason the scale rule prioritises scenarios over repeats in the later wave.157158\subsection{The human arms: designed, not executed}\label{sec:design-human}159160The mandate specifies two human conditions: H0, in which human participants complete tasks without AI, and H1, in which they use conversational AI assistance; it requires observed participants, consent and institutional ethics approval, and forbids substituting a language-model imitation of a human for a real baseline. No human participant was involved in this version of the study, no ethics approval was sought, and no human-performance figure appears anywhere in the manuscript except as an explicit placeholder for a future wave (amendment A-002). The protocol for that wave is part of the reproducibility package. It specifies professionals or senior students in each occupation, with a target of six participants per scenario per arm recruited through the department; randomisation between H0 and H1 within occupation, the H1 interface being a chat assistant without tools; one scenario per participant per session of at most three hours; the same public bundle, brief and deliverable list; the same deterministic evaluators and blind rubric; and recording of time, of tool switches and, with consent, of the screen and, in H1, of the chat transcript, with no intervention allowed. Ethics requirements follow the TCPS~2 framework and the university's research ethics committee: informed consent from a template in the package, voluntary participation, compensation at the professional rate, anonymised deliverables, secure storage of recordings, withdrawal at any time, and no deception, since participants are told that the data are fictional. The analysis plan uses the same scenario-weighted estimands as the agent arms; agent-versus-human contrasts at that scale would be descriptive. Until that wave is run, every statement in this paper about the agent arms is a statement about agents alone, and the comparison with human baselines that the positioning of Section~\ref{sec:lit-positioning} anticipates remains open.161