% --------------------------------------------------------------------------- % Section 2 — Related literature (Agent H, written before any experimental run; % contains no WP42 result) % --------------------------------------------------------------------------- \section{Related literature}\label{sec:literature} The question of how much white-collar work an autonomous computer-using agent can perform to a professional standard sits at the intersection of six literatures that rarely cite one another: the task-based economics of automation, the randomised evidence on generative-AI productivity, the engineering literature on agent benchmarks and computer-use evaluation, the economics of delegation and supervision, the empirical study of agent reliability and of model-based evaluation, and the methodology of preregistered experiments with clustered data. This section reviews each stream with an eye to what it already establishes and what it leaves unmeasured, and closes with a positioning statement that names, for every construct introduced in Section~\ref{sec:theory}, its closest verified precedent. Every reference cited here was verified against Crossref, the arXiv abstract page or the official publisher page before being admitted to the bibliography; where a preprint later appeared at a conference, we cite the preprint because that is the record that was checked. \subsection{Task-based models of automation and occupational exposure}\label{sec:lit-tasks} The task framework is the natural home for a measurement exercise that insists on separating tasks from occupations. \citet{autor2003skill} introduced the distinction between routine and non-routine tasks and showed that computer capital substitutes for routine cognitive and manual tasks while complementing non-routine analytic and interactive ones; the unit of analysis in that paper is the task, not the job, which is precisely the distinction that the mandate of the present study asks us to preserve. \citet{acemoglu2011skills} generalised the approach into a Ricardian assignment model in which technologies and skill groups compete for tasks, and \citet{autor2015jobs} used it to explain why automation has historically created roughly as many jobs as it destroyed. \citet{acemoglu2018race} and \citet{acemoglu2019automation} formalised the displacement, productivity and reinstatement effects: automation extends the set of tasks that capital performs, and the labour-market consequences depend on whether new labour-intensive tasks are created in the same movement. The empirical study of industrial robots by \citet{acemoglu2020robots} remains the template for estimating the local labour-market effects of a task-replacing technology. The most direct bridge between that framework and the present study is the accounting formula of \citet{acemoglu2025macroeconomics}, in which the aggregate productivity gain from generative AI equals the share of tasks exposed, times the share of exposed tasks actually automated, times the average cost saving per automated task. Each factor in that product is a task-level parameter---a success probability, a cost, a quality condition---that the formula takes as given. The Autonomous Labor Frontier of Section~\ref{sec:theory} can be read as an attempt to measure those parameters experimentally rather than to assume them. Two theoretical extensions are relevant to the interpretation of such measurements. \citet{restrepo2025missed} pushes the task model to the limit of artificial general intelligence and distinguishes ``bottleneck'' from ``supplementary'' work, so that the value of automating a task depends on what remains unautomated around it; \citet{ide2025knowledge} models AI as a worker inside a knowledge hierarchy, changing who solves which problems within the firm rather than merely which tasks are performed. Occupational exposure indices are the empirical descendants of the task framework. \citet{frey2017future} assigned a probability of computerisation to 702 occupations using expert labels and engineering bottlenecks; \citet{arntz2017revisiting} showed that accounting for task-level heterogeneity within occupations reduces the share of United States jobs ``at risk'' from 38 to about 9 per cent, a correction that is itself an argument for measuring tasks rather than occupations. \citet{brynjolfsson2017workforce} and \citet{brynjolfsson2018machines} built the Suitability for Machine Learning rubric and found that most occupations contain some suitable tasks but almost none is fully exposed, so that reorganisation rather than wholesale replacement is the expected margin of adjustment. \citet{felten2021exposure} linked AI capability benchmarks to occupational abilities, and \citet{felten2023language} re-weighted the resulting index toward language modelling. The most-cited generative-AI exposure study, \citet{eloundou2024gpts}, had humans and GPT-4 rate O*NET tasks for whether a large language model, alone or embedded in software, could halve the time required at equal quality; roughly 80 per cent of workers were found to have at least a tenth of their tasks exposed. International institutions replicated the approach at global scale---\citet{gmyrek2023generative} and the refined index of \citet{gmyrek2025refined} at the ILO, \citet{pizzinelli2023exposure} and \citet{cazzaniga2024genai} at the IMF---and added a complementarity dimension that splits exposed tasks between those likely to be automated and those likely to be augmented. \citet{hampole2025labor} construct firm-level, time-varying exposure from job postings and find muted net employment effects because workers shift effort toward unexposed tasks. All of these indices share the limitation that motivates the present study: exposure is an \emph{ex ante} judgement about what a model could do, elicited from experts or from a model, not a measurement of what an autonomous agent actually does when handed the files. Two newer strands partially close that gap. The first uses revealed usage. The Anthropic Economic Index of \citet{handa2025economic} maps millions of conversations to O*NET tasks and finds usage concentrated in software and writing, with a majority of conversations classified as augmentation rather than automation; the follow-up of \citet{appel2025uneven} documents a rise in directive, delegation-style requests and heterogeneous enterprise adoption, and \citet{tomlinson2025working} compute an analogous applicability score from Copilot conversations. Early labour-market evidence remains mixed: \citet{brynjolfsson2025canaries} find relative employment declines for the youngest workers in the most exposed occupations in payroll data, while \citet{humlum2025still}, linking Danish adoption surveys to administrative registers, find rapid adoption but small earnings and hours effects to date. The second strand measures capability directly on economically valuable deliverables; it is the subject of Section~\ref{sec:lit-benchmarks} and the immediate precursor of our design. Two features of the task framework carry over directly into the measurement conventions of Section~\ref{sec:theory}. The first is the weight attached to a task within an occupation. In the exposure literature the weight is the share of the occupation's time or value that the task represents, obtained from O*NET importance ratings or from the time-use judgements of raters \citep{eloundou2024gpts,felten2021exposure}; in the accounting formula of \citet{acemoglu2025macroeconomics} it is the share of the occupation's labour cost. Neither source is available for synthetic scenarios, which is why the mandate requires weights to be either defensible or explicitly presented as sensitivity assumptions, and why the frontier is reported under more than one weighting. The second feature is the distinction between automation and augmentation that \citet{cazzaniga2024genai}, \citet{handa2025economic} and \citet{appel2025uneven} draw in different ways: an exposed task may be one the model performs alone or one in which it assists a worker, and the economic consequences differ. The present study is deliberately placed on the automation side of that distinction---the agent owns the task from brief to deliverable and no human intervenes during a run---so that what is measured is autonomous completion, and the human-time parameter of the frontier records what autonomy would cost in supervision rather than what assistance would save in effort. \subsection{Field and laboratory experiments on generative-AI productivity}\label{sec:lit-experiments} The randomised evidence on \emph{assistive} generative AI is now substantial and consistent in the sign of first-order effects. \citet{noy2023experimental} randomised access to ChatGPT among college-educated professionals performing mid-level writing tasks and found a 40 per cent reduction in time together with an 18 per cent increase in rated quality, with the largest gains for initially weaker writers. \citet{brynjolfsson2025generative} studied the staggered rollout of a generative-AI assistant to roughly five thousand customer-support agents and estimated a 15 per cent average increase in issues resolved per hour, concentrated among less experienced agents, with improved customer sentiment and lower attrition. In software, \citet{peng2023copilot} randomised GitHub Copilot in a controlled programming task and found 55.8 per cent faster completion, and \citet{cui2024developers} pooled three field experiments inside large firms to estimate a 26 per cent increase in completed tasks. \citet{dellacqua2023jagged} gave consultants access to GPT-4 and documented a ``jagged frontier'': inside the frontier, consultants worked faster and produced higher-rated output, but on a task deliberately chosen to lie outside it, AI users were 19 percentage points less likely to be correct. \citet{dellacqua2025cybernetic} extend the design to teams and find that an individual with AI matched a two-person team without it. \citet{otis2024entrepreneurial} find that an AI mentor helped high-performing entrepreneurs and hurt low performers, a reminder that heterogeneity by baseline skill can flip the sign of an average effect; \citet{choi2023lawyering} report speed gains with uneven quality effects on legal drafting; and the meta-analysis of \citet{vaccaro2024combinations}, covering 106 experiments, finds that human--AI combinations perform on average \emph{worse} than the better of human or AI alone except in content-creation tasks, a result that bears directly on how much human oversight of an agent adds or subtracts. Two findings in this stream are warnings for the design of any study of agent productivity. The randomised trial of \citet{becker2025metr} gave experienced open-source developers early-2025 AI tools on their own repositories: participants expected a 24 per cent speed-up and believed \emph{ex post} that they had been 20 per cent faster, yet were measured to be 19 per cent slower. It is, to our knowledge, the only randomised trial in which perceived and measured effects diverge in sign, and it establishes that self-reported productivity cannot substitute for timed, independently verified outcomes. In a radiology experiment, \citet{agarwal2023radiology} show that providing AI predictions to human readers does not improve accuracy on average because readers mis-weight the signal, so that the efficient design is a task allocation between human and machine rather than side-by-side assistance. Together with the decision-authority model of \citet{athey2020allocation} and the algorithm-aversion evidence of \citet{dietvorst2015aversion}, these results justify measuring the performance of an \emph{autonomous} agent separately from the performance of a human assisted by AI rather than conflating the two, which is why the present design treats agent arms (A0--A4) and human arms (H0--H1) as distinct treatments rather than as points on one scale. The design lessons of this stream are as important as its findings. Every credible study in it randomises access rather than comparing adopters with non-adopters; every one times the task and grades its output blind to condition, usually with several graders; and the better ones report heterogeneity by baseline skill, by task type and by the position of the task relative to the model's frontier. The present design borrows each of these elements for an autonomous agent: tool access is randomised within task, time and cost are measured by the harness rather than reported by the agent, quality is graded blind by multiple evaluators against a rubric fixed before the runs, and heterogeneity is examined by occupation, difficulty and designed depth. What it cannot borrow in this version is the human comparison group, because the human arms require participants and an ethics approval that do not yet exist (Section~\ref{sec:design-human}); the agent-only estimates are therefore absolute rates and within-agent contrasts, not relative-to-human measures of the kind that \citet{patwardhan2025gdpval} and \citet{wijk2024rebench} report. What this stream lacks is randomised evidence on fully autonomous, multi-application agents producing complete professional deliverables. Every study cited above evaluates a human using a chat assistant or an inline completion tool; none measures the completion rate, the cost, or the supervision burden of an agent that owns a task from the brief to the delivered files. \subsection{Agent benchmarks and computer-use evaluations}\label{sec:lit-benchmarks} Benchmarks for autonomous agents have moved in three years from single-function code generation to multi-application, long-horizon work. In software engineering, SWE-bench \citep{jimenez2024swebench} poses real GitHub issues with hidden unit tests; agent scaffolds such as SWE-agent \citep{yang2024sweagent} and the OpenHands platform \citep{wang2024openhands} defined the agent--computer interface paradigm---a file viewer, an editor and a shell---that Claude Code also instantiates and that \citet{liu2026claudecode} dissect in the first academic treatment of its tool loop, context management and permission model. Terminal-Bench \citep{merrill2026terminalbench} extends the paradigm to hard, expert-written command-line tasks and audits itself with the Agentic Benchmark Checklist of \citet{zhu2025rigorous}, which catalogues the outcome-validity and task-validity failures of existing agent benchmarks and which we treat as a design standard. SWE-Lancer \citep{miserendino2025swelancer} is particularly relevant economically: it prices real freelance software tasks worth one million dollars in total and measures how many dollars an agent earns. MLE-bench \citep{chan2024mlebench} and RE-Bench \citep{wijk2024rebench} compare agents with human experts on machine-learning and research-engineering tasks under explicit time budgets and find agents ahead at short budgets but humans ahead at long ones---an early quantitative statement of the horizon effect that Section~\ref{sec:lit-reliability} returns to. General computer use is measured by OSWorld \citep{xie2024osworld}, whose 369 tasks run in real Ubuntu, Windows and macOS desktops with execution-based checks and on which early models solved about 12 per cent of tasks against 72 per cent for humans; by its long-horizon extension \citep{yuan2026osworld2}; by WebArena \citep{zhou2024webarena}, which self-hosts e-commerce, forum, content-management and map sites; by AgentBench \citep{liu2024agentbench}, with eight environments including an operating system, a database and web shopping; and by GAIA \citep{mialon2023gaia}, whose questions are easy for humans and hard for assistants because they require tool use and multi-step browsing. Enterprise-flavoured benchmarks are the closest analogues of the Virtual White-Collar Economy described in Section~\ref{sec:benchmark}: WorkArena \citep{drouin2024workarena} on ServiceNow knowledge-work interfaces; TheAgentCompany \citep{xu2024agentcompany}, a simulated software company with version control, chat, a wiki and simulated colleagues, on which the best agent completed about a quarter of tasks end to end; $\tau$-bench \citep{yao2024taubench}, whose retail and airline customer-service tasks come with policy constraints and the pass$^k$ reliability metric; AppWorld \citep{trivedi2024appworld}, with nine simulated applications and state-based evaluation; OfficeBench \citep{wang2024officebench} on cross-application office workflows; SpreadsheetBench \citep{ma2024spreadsheetbench} on real spreadsheet manipulation; LegalBench \citep{guha2023legalbench} on legal reasoning tasks built with lawyers; and Vending-Bench \citep{backlund2025vendingbench}, a months-long simulated business whose coherence collapses unpredictably over long horizons. The computer-use capability of the Claude models was introduced in \citet{anthropic2024computeruse}, and the accompanying system card \citep{anthropic2025opus4card} documents agentic safety evaluations, including resistance to prompt injection in computer use and behaviour on impossible tasks in Claude Code, two behaviours that our scenarios deliberately probe. Two benchmarks come closest to the economic question we pose. GDPval \citep{patwardhan2025gdpval} builds tasks from the actual work of professionals with fourteen years' average experience across 44 occupations in the nine largest sectors of the United States economy, with real files as inputs and documents, spreadsheets, slides and diagrams as deliverables, graded blind by experts against the human product; frontier models won or tied roughly half of pairwise comparisons at the time of publication, and the paper reports speed and cost multiples relative to experts. METR's time-horizon construct \citep{kwa2025horizon} defines the length of task, measured in human-expert time, that a model completes with 50 per cent probability, finds it doubling roughly every seven months on software tasks drawn from HCAST \citep{rein2025hcast} and RE-Bench, and models the success--length relationship as a logistic curve. \citet{kapoor2024matter} argue that agent evaluations must report cost alongside accuracy and must hold out tasks to prevent overfitting of scaffolds, concerns that \citet{zhu2025rigorous} systematise. The gap that remains is specific. GDPval evaluates a single deliverable produced from a prompt and files, graded against a human reference, without measuring the agent's interaction with a live computer environment, the number of dependent operations, the human interventions required, or the probability of recovery from an error. OSWorld, WorkArena and TheAgentCompany measure live computer use, but in software or IT-service settings, without occupational weighting and without grading against a professional standard. The METR horizon is one-dimensional and software-only. None of these studies randomises tool access---text only, shell, operating-system automation, browser and office applications---on the same tasks, and none evaluates against a hidden oracle that the worker cannot see while also recording the worker's cost and the supervision it would require. The design of Section~\ref{sec:design} is built to occupy that gap. Table~\ref{tab:benchmarks} summarises the comparison with the benchmarks closest to the present design, restricted to the attributes that the review could verify from each paper. \begin{table}[htbp] \centering \caption{Closest benchmark precedents and what the present design adds to each} \label{tab:benchmarks} \footnotesize \begin{tabularx}{\textwidth}{@{}>{\raggedright\arraybackslash}p{2.6cm} >{\raggedright\arraybackslash}X >{\raggedright\arraybackslash}X@{}} \toprule Benchmark & What it measures & What the present design adds \\ \midrule GDPval \citep{patwardhan2025gdpval} & Deliverables from real files in 44 occupations, graded blind by experts against the human product; cost and speed multiples & Live computer environment with observed process; hidden oracle with planted errors and traps; randomised tool access; human-time and recovery measured \\ \addlinespace METR time horizon \citep{kwa2025horizon,rein2025hcast} & Task length in human-expert time at which success is 50 per cent, on software tasks & Designed dependency depth varied within and across non-software occupations; cost and critical errors measured jointly with success \\ \addlinespace METR developer RCT \citep{becker2025metr} & Measured versus perceived productivity of humans using AI tools on their own repositories & Autonomous rather than assisted work; the agent's cost and time are harness-measured, not self-reported \\ \addlinespace TheAgentCompany \citep{xu2024agentcompany} & End-to-end tasks in a simulated software company with chat, wiki and simulated colleagues & Occupations outside software and IT; professional-standard grading with a quality threshold; occupational weighting \\ \addlinespace OSWorld \citep{xie2024osworld,yuan2026osworld2} & Execution-checked tasks in real desktops; observation-space ablations; human reference rate & Professional deliverables rather than state changes; deterministic and blind grading combined; tool bundles randomised on the same tasks \\ \addlinespace WorkArena \citep{drouin2024workarena}; AppWorld \citep{trivedi2024appworld} & Knowledge-work interface tasks on an enterprise platform; day-to-day tasks across simulated applications with state-based evaluation & Complete professional assignments with planted errors and injection; critical-error and abstention outcomes \\ \addlinespace $\tau$-bench \citep{yao2024taubench} & Customer-service tasks with policy constraints; pass$^k$ reliability across repeated trials & Repeated runs within scenario used for a variance decomposition and a recovery index, across occupations \\ \addlinespace SWE-Lancer \citep{miserendino2025swelancer} & Dollar value of real freelance software tasks earned by an agent & Cost per accepted deliverable net of failures and supervision, for non-software occupations \\ \addlinespace OfficeBench \citep{wang2024officebench}; SpreadsheetBench \citep{ma2024spreadsheetbench} & Cross-application office workflows; real spreadsheet manipulation & Office work embedded in occupational assignments with an oracle, a portal, and a self-repair arm \\ \bottomrule \end{tabularx} \end{table} \subsection{The economics of AI agents, delegation and supervision}\label{sec:lit-delegation} Delegating work to an agent whose effort and output cannot be perfectly observed is the canonical principal--agent problem. \citet{holmstrom1979moral} shows that any informative signal about unobservable effort should enter the contract, and \citet{holmstrom1991multitask} shows that when some dimensions of output are easier to measure than others, high-powered incentives distort effort toward the measurable dimension. The second result is an exact description of an agent that passes deterministic checks while neglecting unmeasured quality, and it motivates the separation, in our evaluation pipeline, between the deterministic evaluator and the blind professional judges, as well as the explicit measurement of critical errors that a pass/fail test would miss. The economics-of-AI literature reframes these ideas for machine agents. \citet{agrawal2019prediction} model AI as a fall in the cost of prediction, so that the complementary input---judgement about payoffs---becomes the scarce human contribution; \citet{athey2020allocation} model the choice of whether a human or an AI holds decision authority and show that it can be optimal to make the AI \emph{less} accurate so that the human remains engaged. \citet{korinek2023generative} catalogues the uses of language models in economic research and offers a taxonomy of delegation with verification; \citet{horton2023silicus} shows that language models can serve as simulated economic agents in experiments, a method that the mandate of the present study forbids for human baselines but that is admissible for synthetic counterparties inside the virtual economy. \citet{immorlica2024agents} ask what changes in markets when generative AI acts as an economic agent; \citet{hadfield2025economy} sketch the institutions an economy of AI agents would require; \citet{kolt2025governing} applies agency law to AI agents and argues that monitoring and loyalty are the central governance problems. Two recent contracting models are closest to the delegation framing of Section~\ref{sec:theory-pa}: \citet{sreenivas2026contracting} write down a model of language-model delegation in which the principal chooses a verification cost and the agent chooses a model and an effort level, deriving the agency cost of imperfect verification; and \citet{catalini2026agi} argue that as the cost of execution collapses, the binding constraint becomes human verification, defining a ``measurability gap'' between what an AI can do and what a human can economically check. Three further strands feed the supervision construct. In AI alignment, \citet{bowman2022oversight} define scalable oversight experimentally by asking whether a human supervising a model can outperform both alone, which is the alignment-community counterpart of the supervision question posed here in economic terms. \citet{svanberg2024costeffective} provide the clearest cost-effectiveness template: for computer-vision tasks they compute the share of exposed wage value whose automation is cost-effective at current system prices and show how that share moves with cost, which is exactly the role played by the budget parameter in the frontier of Section~\ref{sec:theory}. Finally, the revealed-usage evidence that users increasingly delegate complete tasks rather than ask for help with pieces of them \citep{appel2025uneven} shows that the supervision question has moved from the hypothetical to the operational. The link between this stream and the frontier of Section~\ref{sec:theory} is the human-time parameter. In \citet{sreenivas2026contracting} the principal's verification cost is a choice variable that trades off against the agency cost of unverified delegation; in \citet{catalini2026agi} the measurability gap is the set of outputs that an AI can produce but a human cannot economically check. The frontier's condition that human time be below an allowance is the empirical counterpart of both: it asks what share of accepted work could have been delegated with at most a given amount of checking and answering, and Section~\ref{sec:theory-pa} shows that the allowance enters the delegation decision in the same way as the agent's price. What this literature does not provide, and what no observational study can, is a measurement of how much verification a given deliverable \emph{would} have required; the present design approximates it, in this version, by the questions the agent itself records as unanswered, valued at a stated assumption, and by the deterministic and blind evaluation that a supervising professional would otherwise have to perform. \subsection{Reliability, long-horizon failure, self-repair and judge validity}\label{sec:lit-reliability} A consistent finding across benchmarks is that agent success falls steeply with task length and with the number of dependent steps. \citet{dziri2023faith} show that transformer accuracy on compositional tasks decays rapidly with the depth of the computation graph and that apparent competence often reflects pattern matching on shallow sub-graphs. \citet{sinha2025horizon} isolate execution from planning and show that small differences in per-step accuracy compound into large differences in the length of task a model can execute, and that models display a self-conditioning effect in which earlier errors raise the probability of later ones. \citet{kwa2025horizon} fit a logistic success--length curve; \citet{yao2024taubench} show that pass$^k$, the probability of succeeding on all $k$ independent trials, lies far below pass@1, so that reliability rather than peak capability is the binding constraint on deployment; and \citet{backlund2025vendingbench} document qualitatively similar collapses over very long horizons. \citet{cemri2025fail} derive a taxonomy of fourteen failure modes in multi-agent language-model systems from more than 150 traces---specification problems, inter-agent misalignment and task-verification failures---which supplies a coding frame for our own failure analysis. \citet{stroebl2024flaws} show that resampling with an imperfect verifier has a hard ceiling: beyond a point, additional samples raise the rate of false positives rather than the number of correct solutions, a result that applies directly to any self-repairing treatment whose checker is itself a model or a shallow test. On self-repair, Reflexion \citep{shinn2023reflexion} and Self-Refine \citep{madaan2023selfrefine} show large gains from verbal self-reflection and iterative self-feedback on coding and reasoning tasks when an external signal such as a test result is available, and \citet{chen2023selfdebug} show that models can debug their own code by explaining execution results. But \citet{huang2023selfcorrect} show that without external feedback, intrinsic self-correction of reasoning does not improve accuracy and can worsen it. The implication for the self-repairing arm of our design is that recovery must be defined relative to the \emph{type} of error signal available---a deterministic test, an environment error, or none---and that a recovery index must distinguish detection from correction; Section~\ref{sec:theory-wdf} adopts exactly that decomposition. Two further lessons of this stream shape the design. The pass$^k$ finding of \citet{yao2024taubench}---that the probability of succeeding on every one of $k$ independent trials falls steeply in $k$---implies that a single run per task measures capability, not reliability, and that any claim about deployment must rest on repeated runs of the same task; the within-scenario repeats of Section~\ref{sec:design-randomisation} exist for that reason, and the variance decomposition of Proposition~\ref{prop:premium} separates the run-to-run component that pass$^k$ captures from the between-task component that it does not. The self-conditioning result of \citet{sinha2025horizon} implies that errors are not independent across the steps of a workflow, so that the simple compounding model of Section~\ref{sec:theory-wdf} is a benchmark shape rather than a structural claim, and the design records the realised number of tool calls, tool errors and retries so that departures from it can be examined. The failure taxonomy of \citet{cemri2025fail} and the long-horizon observations of \citet{backlund2025vendingbench} supply the categories under which failed runs are coded after the fact. For evaluation validity, \citet{zheng2023judge} show, with MT-Bench and Chatbot Arena, that strong language-model judges agree with humans about as often as humans agree with one another, but document position bias, verbosity bias and self-enhancement bias. \citet{wang2024fair} show that swapping the order of two candidate responses can reverse a judge's verdict; \citet{panickssery2024selfpreference} show that language-model evaluators recognise and favour their own outputs; and \citet{gu2024survey} survey the field and its mitigations, namely rubric anchoring, multiple judges, blinding and calibration against human labels. The language-model judges of our evaluation pipeline are therefore blinded to the experimental condition, anchored to a rubric fixed before any run, drawn from more than one model, and reported with the agreement statistics of the next subsection; calibration against independent human experts is a requirement that this version of the study does not meet (amendment A-005) and that Section~\ref{sec:design} states explicitly. \subsection{Experimental methodology}\label{sec:lit-methods} Preregistration and pre-analysis plans are now expected in experimental economics. \citet{nosek2018preregistration} separate prediction from postdiction and describe registration workflows; \citet{olken2015preanalysis} discusses the trade-off between credibility and flexibility; \citet{simmons2011falsepositive} quantify how analytic degrees of freedom inflate false-positive rates. With many outcomes, occupations and treatments, multiple-testing adjustment is required, and \citet{list2019multiple} adapt step-down procedures to experimental economics. For design and estimation, \citet{athey2016econometrics} is the reference on randomisation-based inference, stratification and power in experiments. Our data are clustered: repeated runs within scenario, scenarios within occupation, and all agents within a model version. The number of clusters at the occupation level is small. \citet{cameron2015practitioner} is the standard guide to cluster-robust inference; \citet{cameron2008bootstrap} introduce the wild cluster bootstrap for few clusters; and \citet{abadie2023clustering} clarify that clustering should follow the sampling or assignment design rather than mere correlation, which in our case means clustering at the level at which scenarios are sampled and arms are assigned within them. Mixed-effects models are the natural way to pool across scenarios, with \citet{bates2015lme4} as the frequentist reference. Benchmarks are tests, and psychometrics applies: \citet{vania2021irt} fit item-response models to NLP test sets to compare their informativeness; \citet{polo2024tinybenchmarks} show that item-response selection of about one hundred items can estimate benchmark performance within two points, which is the basis for calibrating scenario difficulty and for scaling from a small pilot; and \citet{zhou2025lost} apply item-response theory to language-model benchmarks and find many items uninformative or mis-keyed. \citet{miller2024errorbars} provide the simplest correct treatment of uncertainty in model evaluations---question-level clustering, paired differences and power analysis---and every success-rate comparison in this study follows that treatment. Completion times are right-censored when an agent reaches a budget, a time limit or an emergency stop; survival analysis is the appropriate framework. \citet{kaplan1958nonparametric} provides the non-parametric completion curve, \citet{cox1972regression} the proportional-hazards regression on treatment and scenario covariates, and \citet{fine1999competing} the competing-risks model in which a critical failure is a terminal event competing with acceptance rather than a form of missingness; Section~\ref{sec:theory-prf} builds the Professional Reliability Frontier on that model. Inter-rater agreement among blind evaluators should be reported with chance-corrected statistics appropriate to ordinal scores: \citet{krippendorff2004reliability} and \citet{hayes2007reliability} define and justify Krippendorff's alpha for multiple raters with missing data, and \citet{gwet2008interrater} proposes the AC1 and AC2 coefficients as a remedy for the paradox of low kappa under high agreement, which is likely whenever most deliverables pass. For cost-effectiveness, the incremental cost-effectiveness ratio of the economic-evaluation literature has a direct analogue in the incremental cost per additional professionally acceptable deliverable, with uncertainty summarised by bootstrapping the joint distribution of cost and effect; within AI evaluation, \citet{kapoor2024matter} propose reporting the Pareto frontier of accuracy against dollar cost, \citet{svanberg2024costeffective} convert task-level feasibility into the share of wage value that is cost-effective to automate, and \citet{patwardhan2025gdpval} report cost and speed multiples relative to experts. Following these precedents, every headline quantity in this study is presented as a frontier over the budget and the quality threshold rather than as a single number. \subsection{Positioning: precedents and incremental contribution}\label{sec:lit-positioning} The mandate of this study instructs us not to assert novelty where a precedent exists. The honest summary of the review above is that none of the six constructs introduced in Section~\ref{sec:theory} is new as an \emph{idea}; the contribution of the study lies in their joint, experimental, occupation-weighted measurement under randomised tool access with hidden-oracle evaluation. Table~\ref{tab:precedents} lists, for each construct, the closest verified precedents and the incremental element. \begin{table}[htbp] \centering \caption{Constructs of the study, their closest verified precedents and the incremental element} \label{tab:precedents} \small \begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}p{3.1cm} >{\raggedright\arraybackslash}X >{\raggedright\arraybackslash}X} \toprule Construct & Closest precedents & Incremental element \\ \midrule Autonomous Labor Frontier & Task weighting from exposure indices \citep{eloundou2024gpts,felten2021exposure,gmyrek2025refined}; share-of-tasks-automated term of \citet{acemoglu2025macroeconomics}; cost-effectiveness share of \citet{svanberg2024costeffective}; deliverable-level grading of \citet{patwardhan2025gdpval}; task-length horizon of \citet{kwa2025horizon} & Joint probability of validated success, cost within budget and human time within a cap, with explicit thresholds $(B,H,q)$; estimation from randomised runs on live computer environments with independent validation; sensitivity over task weights \\ \addlinespace Workflow Dependency Frontier & Compositional depth \citep{dziri2023faith}; execution length and self-conditioning \citep{sinha2025horizon}; pass$^k$ \citep{yao2024taubench}; long-horizon collapse \citep{backlund2025vendingbench}; multi-application tasks \citep{xu2024agentcompany,wang2024officebench} & Dependency depth, number of applications and decision points treated as \emph{designed} factors in office workflows across occupations; per-step hazard estimated with survival models \\ \addlinespace Computer-Use Tool Premium & Observation-space and interface ablations \citep{xie2024osworld,trivedi2024appworld,yang2024sweagent}; scaffold ablations \citep{patwardhan2025gdpval} & Tool access randomised across the same scenarios and occupations; premium reported as a treatment effect with scenario-clustered inference \citep{abadie2023clustering,cameron2008bootstrap}; described as an ablation, not a new concept \\ \addlinespace Autonomous Recovery Index & Reflexion, Self-Refine, self-debugging \citep{shinn2023reflexion,madaan2023selfrefine,chen2023selfdebug}; limits of intrinsic correction \citep{huang2023selfcorrect} and of imperfect verifiers \citep{stroebl2024flaws}; failure taxonomies \citep{cemri2025fail} & Decomposition of recovery into detection and correction, conditional on the error signal available, compared across a checkpointed arm and the same tools without recovery \\ \addlinespace Autonomy-Adjusted Productivity & Joint time-and-quality measurement in assistive experiments \citep{noy2023experimental,brynjolfsson2025generative,dellacqua2023jagged}; perceived-versus-measured divergence \citep{becker2025metr}; cost-controlled evaluation \citep{kapoor2024matter}; verification-cost models \citep{catalini2026agi,sreenivas2026contracting} & Accepted deliverables per unit of operating cost plus wage-valued human time, measured inside an autonomous-agent experiment and reported as a function of the supervision cap \\ \addlinespace Professional Reliability Frontier & Accuracy-against-cost reporting \citep{kapoor2024matter,patwardhan2025gdpval,miserendino2025swelancer}; validity standards \citep{zhu2025rigorous}; jagged frontier \citep{dellacqua2023jagged}; multitask incentives \citep{holmstrom1991multitask} & Critical error as a competing terminal event \citep{fine1999competing} graded against a professional standard, traced over quality thresholds; proposed as a reporting standard \\ \bottomrule \end{tabularx} \end{table} Three precedents deserve an explicit comparison because a reader may reasonably ask what the present design adds to them. Relative to GDPval \citep{patwardhan2025gdpval}, our scenarios are synthetic rather than drawn from the work of real professionals, which costs realism but buys three things that GDPval cannot offer: a hidden oracle with planted, countable errors and designed traps, so that correctness is evaluated deterministically and not only by pairwise preference; a live environment with a shell, an operating-system automation layer, a browser and an enterprise portal, so that the \emph{process} of the work---tool calls, application transitions, retries, recovery---is observed and not only its product; and a randomised assignment of tool sets, so that the contribution of each layer of computer access is a treatment effect rather than a description. Relative to TheAgentCompany \citep{xu2024agentcompany} and OSWorld \citep{xie2024osworld}, our environment is less rich as software but spans occupations outside software and information technology, weights tasks by their place in an occupation, and grades deliverables against a professional standard with a quality threshold rather than against a binary state check. Relative to the METR horizon \citep{kwa2025horizon} and the METR developer trial \citep{becker2025metr}, we replace human-time length with a designed dependency depth that can be varied within and across occupations, and we measure cost and the supervision burden jointly with success rather than success alone. Two limitations of the positioning must be stated now rather than after the results. First, the human arms H0 and H1 of the mandate are designed and documented (Section~\ref{sec:design-human}) but not executed in this version of the study; the comparison of agents with observed human baselines under the same protocol, which would be the fourth element of the design, is therefore deferred, and no human-performance figure is imputed or simulated in its place. Second, whether any of the six named constructs survives as a distinct concept is to be decided after the pilot: the names are labels for measurement conventions, and the mandate requires that they be dropped if the evidence does not justify them (Section~\ref{sec:results}).