1% ---------------------------------------------------------------------------2% Section 1 — Introduction (journal version; condensed from the monograph's3% Section 1). Every number carries a comment naming its source file.4% ---------------------------------------------------------------------------5\section{Introduction}\label{sec:introduction}67Large language models have become agents. A model that once answered questions in a chat window can now be given a shell, a file system, a browser and a set of office applications, told to complete an assignment, and left alone until it reports back. Claude Code, the command-line client studied here, is one such agent: a loop in which the model reads files, runs programs, inspects their output, edits documents and decides what to do next, with no human in the loop unless the configuration requires one \citep{liu2026claudecode,anthropic2024computeruse}. The economic question this raises is not whether such a system can \emph{assist} a professional---the randomised evidence on assistive generative AI is consistent in the sign of first-order effects \citep{noy2023experimental,brynjolfsson2025generative,peng2023copilot,dellacqua2023jagged}---but whether it can \emph{own} a piece of white-collar work from brief to deliverable: receive the files, do the reconciliation, build the model, write the memorandum, submit the form, and hand back something a demanding manager would accept without redoing it.89The economics of automation has, since \citet{autor2003skill}, treated the task rather than the occupation as the unit at which technology substitutes for labour, and the task-based models of \citet{acemoglu2011skills}, \citet{acemoglu2018race} and \citet{acemoglu2019automation} make the employment consequences depend on which tasks a technology can perform, at what cost, and whether new tasks are created in their place. \citet{acemoglu2025macroeconomics} writes the aggregate productivity effect of generative AI as the product of the share of tasks exposed, the share actually automated and the average cost saving per automated task, and observes that the second and third factors are assumed rather than measured. The exposure literature from \citet{frey2017future} through \citet{brynjolfsson2018machines}, \citet{eloundou2024gpts}, \citet{gmyrek2025refined} and \citet{cazzaniga2024genai} estimates the first factor by asking humans or models which tasks a model \emph{could} perform; \citet{arntz2017revisiting} showed how much the answer depends on looking inside occupations. Usage-based measures \citep{handa2025economic,tomlinson2025working,appel2025uneven} observe what people ask models to do, not whether the result was professionally usable. None of these observes an agent actually doing the work.1011Two developments make direct measurement possible. Agent benchmarks have moved from code generation to complete deliverables and live environments: GDPval grades documents, spreadsheets and presentations produced from real professional files against the human product \citep{patwardhan2025gdpval}; OSWorld, WebArena, TheAgentCompany and AppWorld place agents in real or simulated desktops, web applications and companies and check the resulting state \citep{xie2024osworld,zhou2024webarena,xu2024agentcompany,trivedi2024appworld}; METR's time horizon measures the length of task, in expert human time, that a model completes half the time \citep{kwa2025horizon}. And the delegation of whole tasks has become ordinary practice, with its own contracting theory \citep{sreenivas2026contracting,catalini2026agi}. What is still missing is an experiment that treats the agent's access to the computer as a randomised treatment, its output as a deliverable graded against a professional standard and a hidden ground truth, its cost and its supervision burden as outcomes, and its tasks as a weighted sample of an occupation. This paper reports the pilot of such an experiment.1213\paragraph{Design.} We built a laboratory in which Claude Code launches Claude Code. A controller session holds the hidden oracles, the experiment database and the ledger; it composes an isolated workspace for each run, launches a bare-mode Claude Code session over SSH on one of four Mac Studio nodes, streams and stores every event, enforces a wall-clock limit, collects and hashes the deliverables, deletes the workspace, and runs the scenario's deterministic evaluator and three blind language-model judges (Section~\ref{sec:benchmark}). The benchmark consists of ten scenarios, one in each of ten occupations---accounting, corporate finance, real-estate appraisal, academic research, executive administration, banking, legal services, human resources, procurement and business intelligence---set in a synthetic economy of fictional organisations, each generated by one agent and independently validated by another who solved the task blind before opening the oracle. Every scenario carries a reference solution, three to nine planted errors with known locations, a prompt-injection document and, in four cases, a clarification trap; the worker never sees any of this.14% ../manuscript/sections/benchmark.tex Table tab:pilot (planted errors 3–9; clarification traps in 4 scenarios)1516Each scenario is run under five tool configurations that correspond to recognisable working environments: A0, file reading and writing without a shell (the agent as document assistant); A1, the shell and a Python environment (a programmer's workstation); A2, the shell plus the headless macOS automation layer; A3, full computer use with LibreOffice, a browser engine and a synthetic enterprise portal (a professional's desk inside an organisation's systems); and A4, the same tools with a checkpoint file, a public self-check and one controller-granted recovery resume (the agent with a safety net but no supervisor). The order of runs is a seeded permutation interleaved over nodes, so that each contrast is a within-scenario treatment effect with scenario-clustered inference \citep{abadie2023clustering,miller2024errorbars}. A run is \emph{accepted} when it passes every required criterion of the hidden oracle, contains no critical violation---no fabricated figure, no modified input, no compliance with an injected instruction, no attempted external communication---and receives a mean blind quality score of at least 70 on 100. The hypotheses, estimands, inference plan, caps and scale rule were preregistered before any run, and every departure is logged as an amendment (A-001 to A-013) \citep{nosek2018preregistration,olken2015preanalysis}.1718\paragraph{Findings.} The pilot executed 150 runs on 2--3 November 2026---ten scenarios, five arms, three repeats, all on Claude Code 2.1.314 with \texttt{claude-opus-5-5}---plus 40 model-sensitivity runs and 40 runs of an exploratory recovery experiment, for a total API spend of 261.89 dollars against an envelope of 800.19% ../docs/PHASE_5_AUDIT.md §0 (scope, spend); ../protocol/amendments.csv A-001, A-009, A-01120Four results organise the paper. First, without a shell nothing is accepted and with one almost everything is: scenario-weighted acceptance is 0.000 in A0 and 0.867, 0.900, 0.900 and 0.833 in A1--A4, and the shell premium of $+0.867$ [0.667, 1.000] (permutation $p=0.004$) is the only confirmatory effect in the study.21% ../analysis/tables/table1_outcomes_by_arm_pilot.csv (acceptance_taskw); table2_contrasts_pilot.csv (A1-A0, S)22The mechanism is the one the preregistration anticipated: the deliverables are workbooks with live formulas and formatted documents, which cannot be produced without a program. Second, the preregistered primary contrast---full computer use minus shell---is a precise null, $+0.033$ [0.000, 0.100], $p=1.0$: at this level of difficulty every arm with a shell sits at a ceiling, and the browser, the office adapter and the portal had nothing left to add on scenarios that were, by their authoring specification, solvable from the shell.23% ../analysis/tables/table2_contrasts_pilot.csv (A3-A1, S)24Third, failures are concentrated and legible: seven of 150 runs contain a critical violation, six of them in A0, and no run in the three richest arms; the remaining non-acceptances are almost all one scenario, the executive triage, which passes every deterministic check and fails the quality gate.25% ../docs/PHASE_5_AUDIT.md §2; ../analysis/tables/table3_occupation_arm_pilot.csv26Fourth, cost is about one dollar of marginal API charges per accepted deliverable and scales with the model, not with the tools: \texttt{claude-sonnet-5-5} reaches 0.85 acceptance at 0.40 dollars and \texttt{claude-fable-5-1} 0.85 at 3.30 dollars.27% ../analysis/tables/table1_outcomes_by_arm_pilot.csv (cost_per_accepted); table9_model_sensitivity.csv2829Two further results bear on the economics of delegation. Because workers were forbidden to ask questions during a run and instructed to record them, the completion records yield a supervision proxy; valued at five minutes per question, the Autonomous Labor Frontier of arm A3---the task-weighted probability that a run is accepted, costs no more than a budget $B$ and requires no more than $H$ minutes of human time---is 0.167 at a supervision allowance of zero and 0.667 at fifteen minutes, against a marginal acceptance ceiling of 0.900, while the budget ceases to bind at about 1.50 dollars.30% computed from ../analysis/tables/table5_alf_pilot.csv (mean over 10 occupations, arm A3, weights equal and equal_H15)31The supervision margin, not the budget, is what separates the frontier from the ceiling. And in an exploratory experiment that interrupts sessions at 120 seconds, bounded recovery raises acceptance from 0.05 to 0.85 ($+0.80$ [0.55, 0.95], $p=0.004$) at essentially the cost of an uninterrupted run; the mechanism was never triggered in the pilot because no first session failed its self-check.32% ../analysis/tables/table10_recovery.csv; ../analysis/derived/recovery_experiment.json3334\paragraph{Contribution and limits.} The paper contributes an operational measurement framework (the Autonomous Labor Frontier and four companion measures, presented as conventions with stated precedents rather than as new ideas), a benchmark of ten validated scenarios with hidden oracles, a laboratory that was built rather than described and ran 230 sessions without an unplanned timeout, and two results that were not foreseen: a precise null on the primary contrast that locates where the next benchmark tier must be built, and a recovery mechanism that works when something interrupts the agent. Its limits are stated with equal care, because they are the premises of every interpretation. The human arms of the mandate were designed but not executed; no participant was recruited and no human-performance figure appears except as a labelled assumption (amendment A-002). The benchmark is self-authored by language-model agents and validated by language-model agents, which is a circularity the hidden oracle mitigates but does not remove. The judges are models of the workers' family. There are $J=10$ scenarios, one per occupation, so that every interval is wide and every occupation-level statement is descriptive. The recovery experiment was added after the pilot and is exploratory. And nothing in the paper is an estimate of job displacement: it measures the probability that one family of agents completes ten designed tasks to a stated standard under stated constraints, in a laboratory, on one date.35% ../protocol/amendments.csv A-002, A-005, A-011; ../docs/PHASE_5_AUDIT.md §5–63637Section~\ref{sec:framework} states the framework. Section~\ref{sec:benchmark} describes the benchmark and the laboratory. Section~\ref{sec:design} states the preregistered design. Section~\ref{sec:results} reports the results. Section~\ref{sec:discussion} discusses their economic reading and limitations, and Section~\ref{sec:conclusion} concludes. The monograph of which this is the journal version contains the proofs, the scenario and rubric documentation, the infrastructure and reproducibility records and the robustness tables (Appendix~\ref{app:pointer}).38