1% ---------------------------------------------------------------------------2% Section 1 — Introduction (Agent H; written after the experiments, from3% verified results only). Every number carries a comment naming its source file.4% ---------------------------------------------------------------------------5\section{Introduction}\label{sec:introduction}67Large language models have become agents. A model that once answered questions in a chat window can now be given a shell, a file system, a browser and a set of office applications, told to complete an assignment, and left alone until it reports back. Claude Code, the command-line client studied in this paper, is one such agent: a loop in which the model reads files, runs programs, inspects their output, edits documents and decides what to do next, with no human in the loop unless the configuration requires one \citep{liu2026claudecode,anthropic2024computeruse}. The economic question this raises is not whether such a system can \emph{assist} a professional---the randomised evidence on assistive generative AI is by now consistent on the direction of first-order effects \citep{noy2023experimental,brynjolfsson2025generative,peng2023copilot,dellacqua2023jagged}---but whether it can \emph{own} a piece of white-collar work from brief to deliverable: receive the file, do the reconciliation, build the model, write the memorandum, submit the form, and hand back something a demanding manager would accept without redoing it. That question is the subject of this paper.89\subsection{Motivation: from exposure to measurement}\label{sec:intro-motivation}1011The economics of automation has, since \citet{autor2003skill}, treated the task rather than the occupation as the unit at which technology substitutes for labour, and the task-based models of \citet{acemoglu2011skills}, \citet{acemoglu2018race} and \citet{acemoglu2019automation} make the consequences for employment depend on which tasks a technology can perform, at what cost, and whether new tasks are created in their place. \citet{acemoglu2025macroeconomics} writes the aggregate productivity effect of generative AI as the product of three task-level quantities---the share of tasks exposed, the share actually automated, and the average cost saving per automated task---and observes that the second and third are, at present, assumed rather than measured. The occupational exposure literature that followed \citet{frey2017future}, from the suitability-for-machine-learning rubric of \citet{brynjolfsson2018machines} to the GPT-exposure ratings of \citet{eloundou2024gpts} and the international indices of \citet{gmyrek2025refined} and \citet{cazzaniga2024genai}, estimates the first quantity by asking humans, or models, which tasks a model \emph{could} perform. \citet{arntz2017revisiting} showed how much that answer depends on looking inside occupations rather than across them. None of these indices observes an agent actually doing the work; the usage-based measures of \citet{handa2025economic} and \citet{tomlinson2025working} observe what people ask models to do, not whether the result was professionally usable.1213Two developments make direct measurement possible and necessary. First, agent benchmarks have moved from code generation to complete deliverables and live environments: GDPval grades documents, spreadsheets and presentations produced from real professional files against the human product \citep{patwardhan2025gdpval}; OSWorld, WebArena, TheAgentCompany and AppWorld place agents in real or simulated desktops, web applications and companies and check the resulting state \citep{xie2024osworld,zhou2024webarena,xu2024agentcompany,trivedi2024appworld}; METR's time horizon measures the length of task, in expert human time, that a model completes half the time \citep{kwa2025horizon}. Second, the delegation of whole tasks has become ordinary practice: the share of directive requests in a large sample of model conversations rose from 27 to 39 per cent in eight months \citep{appel2025uneven}, and the economics of delegation to a model whose work must be verified has acquired its own theory \citep{sreenivas2026contracting,catalini2026agi}. What is still missing is an experiment that treats the agent's access to the computer as a treatment, its output as a deliverable graded against a professional standard, its cost and its supervision burden as outcomes, and its tasks as a weighted sample of an occupation. This paper reports the pilot of such an experiment.1415Three features of its design deserve emphasis because they distinguish it from the benchmarks above. The first is the \emph{hidden oracle}. Every scenario is generated with a reference solution, a set of planted errors with known locations, a prompt-injection document and, where the specification calls for it, a missing-information trap that cannot be resolved without asking; the worker never sees any of this, the deterministic evaluator sees all of it, and the professional judges see only the brief, the rubric and the deliverables. Correctness is therefore measured against facts the worker could not have been told, and not only against a preference between two outputs. The second is \emph{randomised tool access}. The same scenario, model, brief, inputs, caps and behavioural policy are run under five tool configurations, from file access without a shell to full computer use with a browser, office tools and an enterprise portal, with bounded self-repair as a fifth arm; the order of runs is a seeded permutation interleaved over nodes. The contribution of each layer of access is thus a within-scenario treatment effect with scenario-clustered inference \citep{abadie2023clustering,miller2024errorbars}, not a description of what one configuration achieved. The third is the separation between \emph{task} and \emph{occupation}. Each scenario is a complete assignment in a named occupation; the frontier we estimate is a task-weighted probability, the weights are stated and varied, and the paper nowhere converts a task-level success rate into a statement about jobs. The theoretical section states this restriction formally (Section~\ref{sec:theory-not}); the mandate of the study makes it a rule.1617A fourth feature is methodological rather than substantive: the study is run as an experiment, not as a leaderboard. The hypotheses, estimands, inference plan, caps and scale rule were written down before any run \citep{nosek2018preregistration,olken2015preanalysis}; every departure is logged as an amendment; the unit of inference is the scenario, because repeats of the same scenario are not independent evidence about the population of tasks \citep{miller2024errorbars}; costs are read from the client's own accounting and reported alongside accuracy, as \citet{kapoor2024matter} ask; and the benchmark's validity threats---answer leakage, vacuous criteria, evaluator defects---were audited against the checklist discipline of \citet{zhu2025rigorous} before the freeze. The consequence is that the paper reports what the pilot can and cannot say with the same care: a null on the preregistered primary contrast is reported as a null, an exploratory result is labelled as such, and the heterogeneity across occupations is described, not tested. We take this discipline to be part of the contribution, because the economic use of agent measurements depends on knowing their uncertainty, and because the field's own audits have found that agent benchmarks often lack it.1819\subsection{The research question}\label{sec:intro-question}2021The question the study was commissioned to answer is: what proportion of economically valuable white-collar work can an autonomous computer-using agent complete to a professionally acceptable standard, and how do its performance, cost, reliability and supervision requirements vary with the task and with the computing environment it is given? We make the question operational in four steps. A \emph{run} is one agent session on one scenario under one tool configuration. A run is \emph{accepted} when the deliverables pass every required deterministic criterion of the hidden oracle, contain no critical violation---no fabricated figure, no modified input, no compliance with an injected instruction, no attempted external communication---and receive a mean blind quality score of at least 70 on 100 from three judges grading against the scenario's rubric. The \emph{Autonomous Labor Frontier} of an occupation is the task-weighted probability that a run is accepted \emph{and} costs no more than a budget $B$ \emph{and} requires no more than $H$ minutes of human time (Definition~\ref{def:alf}). The \emph{tool premium} between two configurations is the scenario-weighted difference in acceptance probability (Definition~\ref{def:premium}). Four companion measures---the Workflow Dependency Frontier, the Autonomous Recovery Index, Autonomy-Adjusted Productivity and the Professional Reliability Frontier---are defined in Section~\ref{sec:theory}. Following the positioning of the literature review (Section~\ref{sec:lit-positioning}), none is claimed to be new as an idea; each is a measurement convention whose precedents are listed in Table~\ref{tab:precedents}.2223The preregistered primary hypothesis was that full computer use would raise acceptance relative to the shell alone; the secondary hypotheses concerned the shell premium, the self-repair premium, the dependency gradient, the integrity of the arms, the cost ranking and heterogeneity across occupations (Section~\ref{sec:design-hypotheses}). The preregistration, drafted before any pilot run, is part of the reproducibility package together with the amendment log that records every subsequent decision.2425\subsection{The laboratory}\label{sec:intro-laboratory}2627The laboratory is Claude Code calling Claude Code. A controller session on a laptop holds the hidden oracles, the experiment database and the ledger; it draws the assignment, composes an isolated workspace for each run, launches a bare-mode Claude Code session over SSH on one of four Mac Studio nodes of an authorised cluster, streams and stores every event of the session, enforces a wall-clock limit, collects and hashes the deliverables, deletes the workspace, runs the scenario's deterministic evaluator and three blind judges, and renders a private dashboard (Section~\ref{sec:infrastructure}); the client's own architecture is described by \citet{liu2026claudecode}. The worker is a separate operating-system process on a separate machine, started in a mode that ignores the controller's settings and instruction files, with a tool set fixed by command-line flags whose effect was verified by probe, with no ability to spawn sub-agents, and with no oracle, rubric or evaluator code anywhere on its node. The five arms differ only in those flags, in the tool documentation appended to the system prompt, and, for the two richest arms, in the presence of a synthetic enterprise portal served on the loopback interface and of a self-repair protocol (Table~\ref{tab:arms}). The benchmark consists of ten scenarios, one in each of ten occupations---accounting, corporate finance, real estate appraisal, academic research, executive administration, banking, legal services, human resources, procurement and business intelligence---set in a synthetic economy of fictional organisations with coherent records, generated by one agent and independently validated by another who solved the task blind before opening the oracle (Section~\ref{sec:benchmark}). Each scenario carries a designed dependency depth of 21 to 44 operations, three to nine planted errors, an injection document and, in four cases, a clarification trap (Table~\ref{tab:pilot}).2829The five arms are meant to correspond to recognisable working environments rather than to arbitrary ablations. A0, with file reading, writing, editing and search but no program execution, is the agent as a document assistant: it can read everything and write prose, but cannot compute, convert or produce a binary office file. A1 adds the shell and a project Python environment and is the agent as a programmer's workstation: anything that can be scripted can be done, but the operating system's automation layer, the office suite, browsers and the network are denied. A2 adds the headless macOS automation layer and native document utilities and is the agent at a Mac without a graphical session. A3 adds LibreOffice for conversion and recalculation, a real browser engine, and the enterprise portal with its document library, record tables and forms, and is the agent at a professional's desk inside an organisation's systems. A4 keeps A3's tools and adds a checkpoint file, a public self-check that knows nothing about the answers, and one controller-granted resume if the first session ends without passing the self-check; it is the agent with a supervisor's safety net but without the supervisor. The economic content of the contrasts follows: the shell premium measures the value of computation, the macOS premium the value of the operating system's automation, the computer-use premium the value of applications and enterprise access, and the self-repair premium the value of a bounded second chance.3031\subsection{What the pilot finds}\label{sec:intro-findings}3233The pilot executed 150 runs---ten scenarios, five arms, three repeats, all at \texttt{claude-opus-5-5} on Claude Code 2.1.314, on 2--3 November 2026---plus 40 model-sensitivity runs and 40 runs of an exploratory recovery experiment, for a total API spend of 261.89 dollars against an envelope of 800.34% docs/PHASE_5_AUDIT.md §0 (scope, spend); protocol/amendments.csv A-009, A-01135No run timed out, no infrastructure error was reported by the launcher, and every uninterrupted session ended with a success result.36% docs/PHASE_5_AUDIT.md §337The findings are as follows.3839\paragraph{Without a shell, nothing is accepted; with one, almost everything is.} Scenario-weighted acceptance is 0.000 in the restricted arm A0, 0.867 in the shell arm A1, 0.900 in the macOS arm A2, 0.900 in the full computer-use arm A3 and 0.833 in the self-repairing arm A4.40% analysis/tables/table1_outcomes_by_arm_pilot.csv, column acceptance_taskw41The shell premium is $+0.867$ with a scenario-cluster bootstrap interval of $[0.667, 1.000]$ and a sign-flip permutation $p$ of 0.004; it is the only confirmatory effect in the study.42% analysis/tables/table2_contrasts_pilot.csv, row A1-A0 / S43Mean blind quality rises from 27.4 to 82.3 between A0 and A1 and is 85.0, 85.7 and 84.8 in A2--A4.44% table1, column quality_mean45The mechanism is the one the preregistration anticipated: the deliverables are workbooks with live formulas and formatted documents, which cannot be produced without a program, and A0 delivers text substitutes that fail the deterministic criteria whatever their content.4647\paragraph{The primary contrast is a precise null.} Full computer use minus shell is $+0.033$ $[0.000, 0.100]$, $p=1.0$, on acceptance, and $+3.3$ points $[-0.8, 9.6]$, $p=0.30$, on quality.48% table2_contrasts_pilot.csv, rows A3-A1 / S and quality49The macOS layer adds $+0.033$ $[-0.067, 0.133]$ and self-repair subtracts $0.067$ $[-0.167, 0.000]$, neither distinguishable from zero.50% table2, rows A2-A1 / S and A4-A3 / S51Deterministic completion is 0.967 to 1.000 in every arm with a shell,52% table1, column det_completion53so the pilot's scenarios, solvable by design with a shell and a Python environment, leave no headroom for the browser, the office adapter or the portal to improve on. The null is informative in one direction only: at this level of difficulty, the richer environment neither helps nor hurts, and the experiment does not test whether it would help on tasks that cannot be completed from the shell alone.5455\paragraph{Failures are concentrated and legible.} Seven of 150 runs contain a critical violation. Six are in A0: three runs of the appraisal scenario complied with the injected instruction and echoed the planted conclusion, and three runs of the contract-review scenario reported fewer issues than the minimum; one A1 run deleted its input database while exploring and then reported honestly that it could not deliver. No violation occurred in A2, A3 or A4, and no run attempted an external communication.56% docs/PHASE_5_AUDIT.md §2; analysis/tables/audit_violations_pilot.csv57The critical-error rate is 0.200 in A0, 0.033 in A1 and 0.000 elsewhere.58% table1, column critical_error_rate59The remaining non-acceptances are of two kinds. The executive-administration scenario passes its deterministic checks in every run of A1--A4 but fails the quality gate in all but one of them: the judges rate the triage between 62 and 67, below the threshold of 70.60% analysis/tables/table3_occupation_arm_pilot.csv (executive administration: det 1.0, acceptance 0.0/0.33/0.0/0.0); docs/PHASE_5_AUDIT.md §261Two other scenarios---business intelligence and appraisal---carry the few remaining shell-arm failures.62% analysis/tables/table3_occupation_arm_pilot.csv63The two instruments disagree in 9 per cent of runs, in both directions, which is why the acceptance definition requires both.64% docs/PHASE_5_AUDIT.md §26566\paragraph{Heterogeneity is a property of scenarios, and depth is confounded with it.} Nine of the ten scenarios are accepted in every run of the full computer-use arm; the exception is the triage, for the quality reason above. Arm A1 loses one business-intelligence run (the deleted database), A2 one appraisal run, and A4 one appraisal run and one business-intelligence run; the appraisal losses are a single adjusted-price criterion failed at a quality near 98, and the A4 business-intelligence loss is a quality-gate failure with every deterministic check passed.67% analysis/tables/table3_occupation_arm_pilot.csv (acceptance vs det by occupation and arm); docs/PHASE_5_AUDIT.md §2 (RE-01 A2/A4 runs)68Acceptance is 1.000 at the four shallowest designed depths (21 to 24 operations) in every arm with a shell and 0.500 to 0.667 at the deepest (44), which is the direction the dependency hypothesis predicted; but depth 44 is two scenarios, one of them the triage, so the gradient cannot be separated from the scenario effect at this scale and the Workflow Dependency Frontier is reported as descriptive.69% analysis/tables/table4_dependency_pilot.csv70Within arm, the realised number of tool calls is 12 to 19 in the shell arms against 29 in A0, whose calls are mostly failed attempts to do without a program.71% table1, columns tool_calls_mean and tool_errors_mean7273\paragraph{Cost is about one dollar per accepted deliverable and scales with the model, not with the tools.} Mean marginal API cost per run is 0.71 dollars in A0 and 0.78 to 0.93 in the other arms; the cost per accepted deliverable, which keeps failed runs in the denominator, is 0.95 in A1, 0.87 in A2, 1.01 in A3 and 1.11 in A4.74% table1, columns cost_mean and cost_per_accepted75Median wall time is 162 seconds in A0 and 214 to 230 seconds in A1--A4, and every run finished within 600 seconds.76% table1, column wall_median_s; analysis/derived/estimates_pilot.json, survival.p_done_60077On the same ten scenarios in arm A3, \texttt{claude-sonnet-5-5} reaches 0.85 acceptance at 0.40 dollars per accepted deliverable with quality 4.8 points lower than Opus ($[-6.7, -3.0]$, $p=0.002$), and \texttt{claude-fable-5-1} reaches 0.85 at 3.30 dollars with quality not distinguishable from Opus ($+1.2$ $[-1.4, 3.8]$).78% analysis/tables/table9_model_sensitivity.csv; table9b_model_contrasts.csv79Across the three models, acceptance moves by 0.05 and cost by a factor of eight.8081\paragraph{The supervision margin, not the budget, separates the frontier from the ceiling.} Workers were forbidden to ask questions during a run and instructed to record them; valued at five minutes each, the recorded questions imply 13.5 to 18.8 minutes of supervisor time per run in the shell arms.82% analysis/tables/table12_supervision_pilot.csv, column mean_H_min83At a supervision allowance of zero the Autonomous Labor Frontier of arm A3, averaged over the ten occupations with equal weights, is 0.167 at any budget of one dollar or more; at an allowance of fifteen minutes it is 0.500 at one dollar and 0.667 at 1.50 dollars and above, against a marginal acceptance ceiling of 0.900.84% computed from analysis/tables/table5_alf_pilot.csv (mean over the 10 occupations, weights = equal and equal_H15, arm A3)85The budget ceases to bind at about 1.50 dollars; the supervision allowance never does within the range examined. This is the pilot's clearest economic message, and it is a message about the agent's propensity to leave questions open rather than about its ability to do the work.8687\paragraph{Bounded recovery works when something interrupts the agent.} In the pilot, no A4 first session failed the public self-check, so the self-repair arm was in effect the full computer-use arm plus a checkpoint and a self-check, and the Autonomous Recovery Index was unidentified.88% analysis/derived/estimates_pilot.json, recovery (n_failed_first_selfcheck 0); protocol/amendments.csv A-011, A-01289An exploratory experiment, added after the pilot and labelled as such, interrupted sessions at 120 seconds: without recovery (A3) acceptance was 0.05; with the bounded resume (A4) it was 0.85, a difference of $+0.80$ $[0.55, 0.95]$, $p=0.004$, and 20 of 20 resumed sessions passed the public self-check.90% analysis/tables/table10_recovery.csv; analysis/derived/recovery_experiment.json91The recovered runs cost 0.86 dollars on average (the client's cumulative total over both sessions), 1.08 per accepted deliverable, about the same as an uninterrupted A4 run (1.11) and close to an uninterrupted A3 run (1.01).92% table10_recovery.csv, A4 interrupted@120s (cost_mean 0.863, cost_per_accepted 1.079); analysis/derived/recovery_experiment.json9394\paragraph{The judges agree with one another and, mostly, with the oracle.} Krippendorff's alpha across the three blind judges is 0.973 over the 150 pilot runs, with pairwise correlations of 0.96 to 0.99; the two draws of the same judge model are near-identical, which overstates independence.95% analysis/derived/estimates_pilot.json, agreement; docs/PHASE_5_AUDIT.md §296The judges and the worker belong to the same model family, and the self-preference bias documented by \citet{panickssery2024selfpreference} is not addressed by the design; calibration against human experts is left to a later wave.9798\subsection{Contributions}\label{sec:intro-contributions}99100The paper makes four contributions, each of which we state with its limits.101102First, it \emph{operationalises} the Autonomous Labor Frontier and its companions as estimable quantities with stated thresholds, decomposes the frontier into deterministic completion, a quality gate, an integrity factor and budget feasibility (Proposition~\ref{prop:decomp}), gives the premium a clustered variance that explains why scenarios matter more than repeats (Proposition~\ref{prop:premium}), connects the dependency frontier to a compounding-error benchmark (Proposition~\ref{prop:compound}) and the recovery index to a detection-and-correction decomposition (Proposition~\ref{prop:recovery}), and reads all of them through a delegation-with-verification model (Proposition~\ref{prop:delegation}). The constructs are presented as measurement conventions that operationalise the task model of \citet{acemoglu2018race} and the cost-effectiveness logic of \citet{svanberg2024costeffective}; the names are provisional, and Section~\ref{sec:implications} says which of them the evidence supports.103104Second, it delivers a \emph{benchmark} of ten validated scenarios with hidden oracles, deterministic evaluators and professional rubrics in ten occupations, authored to a common specification inside a coherent synthetic economy, with designed dependency depth, planted errors, injection documents and clarification traps, and a generation-and-validation protocol in which the validator solves the task blind before seeing the oracle and found and fixed between one and seven evaluator defects per scenario before the freeze.105% docs/PHASE_5_AUDIT.md §1106The benchmark is self-authored by agents, which Section~\ref{sec:implications} discusses as a circularity; it is also, for that reason, extensible at low cost.107108Third, it delivers an \emph{infrastructure} that was built rather than described: a controller that launches, bounds, logs, hashes, resets and evaluates independent Claude Code sessions on a cluster with role separation that is operational rather than nominal, and that ran 230 sessions without an unplanned timeout or an unrecovered infrastructure error.109% docs/PHASE_5_AUDIT.md §3110Three defects were found during the study and are recorded as amendments: a judge crash on deliverables containing null bytes, a silent loss of three runs' deliverables at collection, and a launcher flag that made the A4 resume impossible until it was needed (Section~\ref{sec:reliability}).111112Fourth, it reports \emph{two results that were not foreseen}: that the computer-use premium is a precise null at the difficulty of the pilot, because the shell alone saturates it, and that bounded end-of-run recovery converts almost every interrupted session into an accepted deliverable when the interruption is induced. The first says where the next benchmark tier must be built; the second says that the self-repair mechanism is worth measuring on failures that occur naturally, which the pilot did not produce.113114\subsection{What this paper is not}\label{sec:intro-not}115116The paper is not an estimate of job displacement, of the share of any occupation that can be automated, or of the effect of agents on wages or employment. It measures the probability that one family of agents completes ten designed tasks to a stated standard under stated constraints; the step from that probability to adoption and from adoption to labour-market outcomes runs through complementarities, reorganisation and demand that the experiment does not observe \citep{acemoglu2019automation,brynjolfsson2018machines,hampole2025labor,humlum2025still}. It contains no human baseline: the mandate's human arms were designed, with a consent protocol and an ethics plan, but no participant was recruited and no approval was sought, and no human-performance figure appears anywhere except as an explicitly labelled assumption in a sensitivity analysis (amendment A-002; Section~\ref{sec:design-human}). The scenarios are synthetic, were written and validated by language-model agents, and are solvable by design; the judges are language models; the model family, the client version and the date are single; the macOS layer is the subset of automation that works without a console session; no interface-perturbation condition was executed; the recovery experiment is exploratory; and the benchmark has ten scenarios, so that every interval in the paper is wide and every occupation-level statement is descriptive. These limits are developed in Section~\ref{sec:implications}, where they are the premises of the research agenda.117118Nor is the paper a product evaluation or a safety evaluation. It reports the behaviour of one client and three models of one vendor under one protocol, chosen because the client exposes the flags that make randomised tool restriction verifiable and because the cost of each session is reported by the client itself; it makes no comparison with other vendors' agents, which the agenda places first among the extensions. The integrity results---seven violations in 150 runs, none in the three richest arms---concern designed traps of modest sophistication inside a loopback environment with no real counterpart to harm, and they are reported as reliability statistics of the arms, not as evidence about the safety of the agent in deployment, which the vendor's own evaluations address under conditions this study does not reproduce \citep{anthropic2025opus4card}.119120\subsection{Roadmap}\label{sec:intro-roadmap}121122Section~\ref{sec:literature} reviews the task-based, experimental, benchmark, delegation, reliability and methodological literatures and positions the study against its closest precedents. Section~\ref{sec:theory} states the measurement framework. Section~\ref{sec:benchmark} describes the occupational taxonomy, the synthetic economy, the anatomy of a scenario, the generation-and-validation protocol and the ten pilot scenarios. Section~\ref{sec:design} states the preregistered design: arms, randomisation, caps, outcomes, evaluation, hypotheses, estimands, inference, the scale rule and the human arms. Section~\ref{sec:infrastructure} describes the laboratory as built and the measured environment that constrained it. Section~\ref{sec:data} documents the data and the measurement pipeline. Section~\ref{sec:results} reports the main results against each preregistered hypothesis, and Section~\ref{sec:reliability} the reliability, cost, heterogeneity, model-sensitivity and recovery analyses together with the defects found during the study. Section~\ref{sec:implications} draws the economic implications, states the limitations and sets out the agenda; Section~\ref{sec:conclusion} concludes. A disclosure statement records how the study was produced. Appendices contain the proofs, the scenario and rubric documentation, the infrastructure and reproducibility records and the robustness tables.123