1% ---------------------------------------------------------------------------2% Section 10 — Economic implications and limitations (Agent H; written from3% verified results only; every number carries a comment naming its source).4% ---------------------------------------------------------------------------5\section{Economic implications and limitations}\label{sec:implications}67The pilot produced one confirmatory effect, one precise null, one exploratory result and a set of descriptive quantities. This section asks what they mean for the economics of autonomous white-collar work, what they do not mean, and what has to be done next for the frontier to become a measurement that policy and firms can use. We proceed from the ceiling to its caveats, then to cost, to the frontier as a device, to the delegation reading, and to the agenda.89\subsection{What a ceiling at 0.87--0.90 means}\label{sec:impl-ceiling}1011Every arm that had a shell accepted between 0.833 and 0.900 of its runs, with deterministic completion of 0.967 to 1.000 and a critical-error rate of 0.033 or zero.12% analysis/tables/table1_outcomes_by_arm_pilot.csv13Three things follow, each limited to the setting that produced it.1415The first is that, for the kind of assignment in the pilot---a complete professional deliverable with a workbook of live formulas and a formatted report, built from a bundle of documents, statements, exports and policies of realistic size---the binding constraint on an autonomous agent in late 2026 is not the richness of its computing environment but the mere ability to execute programs. The step from A0 to A1 is the whole story of the pilot: acceptance rises by 0.867, quality by 55 points, and the critical-error rate falls from 0.200 to 0.033.16% analysis/tables/table2_contrasts_pilot.csv, rows A1-A0 (S, quality); table1 critical_error_rate17A0's failures are not failures of reasoning: its text substitutes often contain the right numbers, and its mean quality of 27 reflects both the missing formats and the fact that, without a program, it could not build the appraisal grid and in three runs out of three echoed the injected conclusion instead.18% docs/PHASE_5_AUDIT.md §219The browser, the office adapter and the enterprise portal added nothing measurable on top of the shell, and the headless macOS layer added nothing either. The arms were used: A3 and A4 submitted the portal form in every run and made nine application transitions on average against three in A1,20% docs/PHASE_5_AUDIT.md §3 (portal 100 %); table1 app_transitions_mean21so the null is not an artefact of tools left idle; it is the result of tools that were not needed.2223The second is that the null is informative about the benchmark, not about the tools. The scenarios were authored to a specification that requires every deliverable to be producible from the public bundle with a shell and a Python environment, and the validators confirmed this by solving each scenario before opening its oracle.24% docs/PHASE_5_AUDIT.md §125A task that can be completed from the shell cannot show a computer-use premium, whatever the agent. The preregistered power simulation had already said that, with ten scenarios and three repeats, an effect of 25 percentage points would be detected four times in ten;26% analysis/tables/power_pilot.csv (J=10, R=3, p_base 0.5, delta 25: 0.408)27the pilot found an effect of three points with an interval that excludes anything above ten, which is a sharper statement than the design promised, and it is sharp because the arms are at a ceiling where the between-scenario variance of the treatment effect is small. The right reading is that the computer-use premium must be measured on tasks that are infeasible from the shell: tasks whose inputs live only behind a portal or a graphical application, whose deliverables are states of a system rather than files, or whose length exceeds what one session can plan. None of these is in the pilot; the agenda below puts them there.2829The third is what the ceiling says positively. In 120 runs with a shell, one agent destroyed an input and said so; none fabricated a figure that the oracle could detect, none obeyed an injected instruction, none attempted to communicate outside its workspace.30% docs/PHASE_5_AUDIT.md §2; analysis/tables/audit_violations_pilot.csv31The integrity hypothesis H5 is therefore supported in its second clause---violations do not increase with tool access; they fall---and refuted in its first for A0, whose rate of 0.200 exceeds the 10 per cent bound.32% table1 critical_error_rate; preregistration §3 H533That the richest arms were the cleanest is consistent with the mechanism of the A0 violations: an agent that cannot do the work is the one that reaches for the shortcut the injection offers. The result does not generalise to adversarial inputs designed against this model, and a single injection document per scenario is a thin test; it does say that, on the designed traps of this benchmark, access did not buy misbehaviour.3435\subsection{What the ceiling does not mean}\label{sec:impl-limits}3637The limitations below are the premises of every interpretation in this section. They are listed once, with what each implies.3839\paragraph{The scenarios are synthetic and were authored by language-model agents.} Organisations, documents, figures and errors were generated to be coherent and realistic; their realism is the validators' judgement, and the validators were also language-model agents, launched as fresh sessions but drawn from the same family as the workers. A scenario written by a model and solved by a model may share assumptions about what a professional deliverable looks like that a practising accountant or appraiser would not share. The frontier is measured on designed difficulty, and the design was done by the kind of system being measured. This is the benchmark's central circularity; it is partly mitigated by the hidden oracle, which fixes facts that neither author nor worker can negotiate, and by the deterministic criteria, which the validators tightened in every scenario before the freeze,40% docs/PHASE_5_AUDIT.md §1 (1–7 evaluator defects fixed per scenario)41and it is removed only by scenarios drawn from real professional work and judged by real professionals.4243\paragraph{The scenarios are solvable by design.} The authoring specification requires solvability from the public bundle; the clarification traps are the only designed occasions for abstention. The frontier therefore excludes, by construction, the part of white-collar work that is ill-specified, that requires negotiation, or that cannot be completed from the artefacts a worker is handed, and the ceiling is a ceiling on designed-solvable work.4445\paragraph{Ten scenarios, one per occupation.} With $J=10$ every scenario-cluster interval on an acceptance difference is at least ten points wide, no occupation-level statement is confirmatory, and the heterogeneity we report is driven by two scenarios: the executive-administration triage, which passes the deterministic checks and fails the quality gate in eleven of twelve shell-arm runs, and the two scenarios, business intelligence and appraisal, that carry the four remaining shell-arm failures.46% analysis/tables/table3_occupation_arm_pilot.csv (A1 BI 0.667; A2 RE 0.667; A4 BI 0.667 and RE 0.667)47The dependency gradient of H4 is in the hypothesised direction---acceptance is 1.000 at the four shallowest depths in every shell arm and 0.500 to 0.667 at depth 44---but depth 44 is two scenarios, one of which is the triage, so the gradient and the triage are the same observation.48% analysis/tables/table4_dependency_pilot.csv4950\paragraph{One model family, one client version, one date.} The primary arms use one model; the sensitivities use two others from the same vendor; the client is a single version on all nodes; the runs span two days. The frontier is dated, and the model-sensitivity result---acceptance moving by 0.05 across models whose cost differs eightfold---describes three points of one vendor's line.51% analysis/tables/table9_model_sensitivity.csv5253\paragraph{The judges are language models of the same family as the workers.} Agreement among judges is high, but two of the three are draws of the same model and the third is the workers' model; self-preference \citep{panickssery2024selfpreference} and shared blind spots are not excluded by blinding to condition. The quality gate, which is the sole reason the triage scenario is not accepted, rests on these judges. The deterministic component does not, and the 9 per cent disagreement between the two instruments is the only internal check available.54% docs/PHASE_5_AUDIT.md §25556\paragraph{No human arm.} The human arms were designed and not executed (amendment A-002). Every wage, every professional-minutes figure and every human-cost band in the paper is an assumption labelled as such; nothing in the data compares an agent with a person.5758\paragraph{The macOS layer is the headless subset.} Control of graphical applications was not available to a non-console process on the executor nodes (amendment A-003), so arm A2 is the automation that works without a console session, and the mandate's GUI condition remains to be run on a dedicated desktop.5960\paragraph{No interface perturbation was executed.} The mandate asks for interface perturbations; none was built into the pilot, and the robustness figure the mandate lists for them does not exist in this version.6162\paragraph{The recovery experiment is exploratory.} It was added after the pilot, when the A4 arm turned out never to trigger a recovery (amendments A-011 and A-012); its interruption is induced at a fixed time rather than arising from the agent's own failure, and its result---an Autonomous Recovery Index of 1.0 on twenty interruptions---is a statement about resuming a session that was cut off, not about detecting and correcting a substantive error.63% analysis/derived/recovery_experiment.json6465\paragraph{Caps and concurrency.} Runs were bounded at four to six dollars and thirty to sixty minutes, with twelve concurrent sessions; no run reached a cap, so the caps did not censor, but the frontier is flat beyond them by Proposition~\ref{prop:alf}(v), and the three deliverables lost at collection under concurrency (amendment A-010) are a reminder that the laboratory's stability was itself under test.6667\subsection{The cost structure and Autonomy-Adjusted Productivity}\label{sec:impl-cost}6869The marginal API cost of an accepted deliverable is 0.87 to 1.11 dollars in the shell arms, and the arms are ordered A2 $<$ A1 $<$ A3 $<$ A4, the richer environments paying for tokens spent on tools that did not raise acceptance.70% analysis/tables/table1_outcomes_by_arm_pilot.csv, cost_per_accepted71Under the study's loading convention, which adds the judges' cost and a nominal compute allowance per node-hour to the API charge, the figures are 1.15 to 1.41 dollars;72% analysis/tables/table11_economic_sensitivity_pilot.csv, cost_per_accepted_loaded at price_multiplier 1.073at four times current token prices they are 3.48 to 4.44 marginal.74% table11, price_multiplier 4.075Autonomy-Adjusted Productivity, the number of accepted deliverables per hundred dollars once the supervisor's time implied by the recorded questions is valued at a wage, is 8.4 for A3 at 40 Canadian dollars an hour, 4.4 at 80 and 3.0 at 120; A4 is indistinguishable, A2 is 7.5, 3.9 and 2.6, and A1 is 6.5, 3.3 and 2.3.76% table11, accepted_per_100usd_aap77Two features of this structure matter economically. The first is that, once a wage is attached to the supervision implied by open questions, the supervisor's time dominates the API charge at every wage examined: at 80 dollars an hour, fifteen minutes of question-answering costs twenty dollars against one dollar of tokens. The economically relevant cost of the agent, in this pilot, is the cost of the human who closes its questions, which is the conclusion of \citet{catalini2026agi} reached from the other direction. The second is that the model, not the tool set, is the cost lever: Sonnet at 0.40 dollars per accepted deliverable and Fable at 3.30 bracket Opus at 1.01 with acceptance within 0.05 of it, and the quality difference---Sonnet five points lower, Fable not distinguishable---is the only thing the extra spending buys.78% analysis/tables/table9_model_sensitivity.csv; table9b_model_contrasts.csv7980The human-cost band used in the sensitivity figure is the scenario authors' estimate of professional minutes, 150 to 420 per scenario, multiplied by an assumed wage: 100 to 280 Canadian dollars at 40 an hour, 300 to 840 at 120.81% analysis/tables/table13_human_cost_assumption.csv82Against it, one dollar of marginal cost per accepted deliverable is two orders of magnitude lower. We decline to call this a productivity comparison, for three reasons already stated: the human figure is not measured, the acceptance is judged by models, and the agent's deliverable arrives with questions attached. What the band does establish is the order of magnitude of the verification cost that the delegation model of Section~\ref{sec:theory-pa} can absorb, to which we now turn.8384\subsection{The frontier as a measurement device}\label{sec:impl-alf}8586The Autonomous Labor Frontier was defined with three thresholds because the economic question has three margins. The pilot shows that the margins are not interchangeable. Marginal acceptance in A3 is 0.900. At a supervision allowance of zero---no open question---the frontier averaged over the ten occupations is 0.167 at any budget of one dollar or more. At an allowance of fifteen minutes, the frontier is 0.267 at 0.75 dollars, 0.500 at one dollar, and 0.667 at 1.50 dollars and above; the budget stops binding between one and one and a half dollars, and the supervision allowance never stops binding in the range examined. The other shell arms behave alike: at fifteen minutes and four dollars, A1 and A2 are at 0.500 and A4 at 0.600.87% computed from analysis/tables/table5_alf_pilot.csv (mean over the 10 occupations, equal weights; weights = equal for H=0 and equal_H15 for H=15)88The gap between 0.900 and 0.667 is the share of accepted work that arrived with more than three open questions; the gap between 0.900 and 0.167 is the share that arrived with any question at all.8990This has two uses. For policy, the exposure indices of \citet{eloundou2024gpts}, \citet{felten2023language} and \citet{gmyrek2025refined} answer the question ``could a model do this task?''; the frontier answers ``did an agent do this task, at this cost, with this much help?'', and the pilot shows that the answer to the second question depends more on the help than on the cost. An exposure index that ignores the supervision margin will overstate the autonomous share by the width of that gap, which here is between a quarter and three quarters of the ceiling depending on the allowance. The frontier is also the quantity that \citet{acemoglu2025macroeconomics} needs for the second and third terms of his accounting---the share actually automated and the cost saving---and the pilot's contribution to that accounting is to show that the share is a function of $H$ before it is a function of $B$.9192For firms, the frontier is a delegation map. A manager deciding whether to hand an assignment to an agent needs, for that kind of assignment, the acceptance probability, the cost, the critical-error rate and the expected number of questions; the frontier at the firm's own $(B,H,q)$ is the probability that the delegation succeeds without further intervention. The pilot's values say that, for designed-solvable assignments of this kind, an agent with a shell delivers an acceptable package nine times in ten for about a dollar, that it will record about three questions on the way,93% analysis/tables/table12_supervision_pilot.csv, mean_questions A3 2.994that it will formally declare a need for clarification in 0.133 of runs, each of them on a scenario whose oracle required it, although a clarification trap was present in four of the ten scenarios, so that most traps produced a recorded question rather than an abstention,95% table12, abstention_rate and justified_abstention_rate (A3: 0.133 and 0.133); manuscript Table tab:pilot (Clar. column: 4 scenarios)96and that the supervisor's time is the cost to manage. Whether the questions are a defect of the agent or a feature of a careful professional is a judgement the firm must make; what the frontier does is make the quantity visible.9798The six named constructs were to be kept or dropped on the evidence. The ALF and the tool premium earned their place: the first because its margins behaved differently, the second because it was identified and estimated with a usable interval. Autonomy-Adjusted Productivity is a useful accounting convention and nothing more, as Section~\ref{sec:theory-aap} said. The Professional Reliability Frontier reduced, in a pilot with no timeouts and almost no critical errors, to the acceptance rate; it is kept as a reporting standard for benchmarks where the other two coordinates move. The Workflow Dependency Frontier could not be estimated with designed variation in depth, since depth and scenario are confounded at $J=10$. The Autonomous Recovery Index was unidentified in the main design and estimated only under induced interruption; it is kept for the mechanism it revealed, with the qualification that it measures resumption, not correction.99100\subsection{Delegation with verification}\label{sec:impl-delegation}101102Proposition~\ref{prop:delegation} says that delegation with verification beats in-house production when $p\ge(c_A+\kappa_V)/c_H$, and that verification is worth its cost when $\kappa_V\le(1-p)(\ell-c_H)+eL$. The pilot supplies $p$ and $e$ and the order of magnitude of $c_A$; it supplies nothing about $c_H$, $\kappa_V$, $\ell$ or $L$ except the labelled assumptions of the sensitivity analysis. Within those assumptions, the first inequality is slack: with $p=0.9$, $c_A$ about one dollar and $c_H$ between 100 and 840 Canadian dollars, delegation pays as long as verifying the agent's package costs less than about nine tenths of producing it in-house.103% p from table1 (A3 acceptance 0.9); c_A from table1 cost_per_accepted; c_H band from table13104The second inequality is the one that bites. In 90 runs of arms A2--A4, no critical violation was recorded, so the point estimate of $e$ is zero and a conventional upper bound on it is about 0.033;105% zero events in 90 runs; rule-of-three upper bound 3/90 computed by the writer106but $L$, the loss from a critical error that passes unnoticed, is unbounded in the professions represented here---a fabricated reconciliation total, an appraisal that echoes a lender's number, a payroll letter with the wrong amount---and the product $eL$ does not vanish because $e$ is small. The pilot's delegation reading is therefore the one \citet{catalini2026agi} and \citet{sreenivas2026contracting} reach in theory: when execution is cheap and accurate, the decision turns on the cost of checking, and the cost of checking is set by the task's measurability. The hidden oracle of this study is a laboratory device for making tasks measurable; its analogue inside a firm is a deterministic check written before the work is delegated, which is to say a test. The pilot shows that such tests are cheap to write for the kinds of deliverables here---every scenario had one---and that they disagree with a professional's judgement in one run in eleven,107% docs/PHASE_5_AUDIT.md §2 (9 % disagreement)108which is the residual that the judgement must cover.109110The multitask logic of \citet{holmstrom1991multitask} appears in the pilot in a specific form. The triage scenario is the one where the deterministic criteria are satisfied and the professional judgement is not; an agent optimising against the test alone would have been ``done'' on every run, and only the second instrument records that the output was not what the executive would have wanted. An organisation that verifies by test alone will delegate this task; one that verifies by judgement will not; the frontier at $q=70$ records the second decision and the frontier at a lower threshold the first. Reporting the frontier over $q$ is how that choice is left to the reader rather than made for them.111112\subsection{An agenda}\label{sec:impl-agenda}113114The limitations above define the next wave, which the mandate anticipates and the preregistration defers. We state it concretely.115116\paragraph{A human wave.} The human arms H0 and H1 require recruited professionals or senior students, consent, compensation and institutional ethics approval under the protocol of Section~\ref{sec:design-human}; six participants per scenario per arm on the same bundles, with the same deterministic evaluators and rubric, would turn the human-cost band into a measurement and the acceptance ceiling into a comparison. Until then, the comparison that the positioning of Section~\ref{sec:lit-positioning} anticipates is not made.117118\paragraph{A harder tier.} The benchmark should grow to twenty and then forty scenarios, adding the ten uncovered occupations of the taxonomy and, within occupations, the short and the workflow classes that the pilot lacks. The workflow class must contain the 30-to-60-operation assignments the mandate requires, with inputs that exist only behind the portal or a graphical application, deliverables that are states of a system, and multi-session length, so that the computer-use premium is identified on tasks where the shell alone cannot finish. Interface perturbations---renamed fields, moved menus, a portal that times out---should be a designed factor with its own arm. The power table of Section~\ref{sec:design-power} says that forty scenarios with three repeats detect a 25-point effect nineteen times in twenty; the pilot's ceiling says the effect must be given room to exist.119% analysis/tables/power_pilot.csv (J=40, R=3, p_base 0.5, delta 25: 0.968)120121\paragraph{Cross-vendor agents and multiple versions.} The frontier should be estimated for at least one agent from another vendor under the same protocol, with the same flags where they exist and documented substitutes where they do not, and repeated at intervals so that the date dependence becomes a time series rather than a caveat.122123\paragraph{Human expert judges.} A sample of deliverables should be graded by practising professionals in each occupation, blind to condition, with Krippendorff's alpha reported between human and model judges \citep{krippendorff2004reliability,hayes2007reliability}; the model judges stay for scale, calibrated to the human sample.124125\paragraph{Holdout scenarios and external authorship.} A set of scenarios should be authored or adapted by people outside the project, held out from every development run, and used once, so that the circularity of self-authorship is bounded rather than argued away; the agentic-benchmark checklist of \citet{zhu2025rigorous} and the holdout discipline of \citet{kapoor2024matter} are the standards to meet.126127\paragraph{Natural failures for the recovery index.} The ARI should be estimated where failures arise from the agent rather than from an interruption, which requires the harder tier; and the self-check should be extended, with care, toward substantive checks that remain ignorant of the answers, so that detection and correction can be separated as Proposition~\ref{prop:recovery} intends and the imperfect-verifier ceiling of \citet{stroebl2024flaws} can be measured rather than assumed.128129\paragraph{A console session for the GUI arm.} The mandate's A2 needs a dedicated desktop session on which the automation consent prompts can be answered; with it, the graphical layer becomes an arm rather than an infrastructure limitation.130131Each item is a cost. The pilot used a third of its envelope;132% docs/PHASE_5_AUDIT.md §4 (261.89 of 800 USD)133the next wave, at the pilot's mean cost per run, would run forty scenarios under five arms with three repeats for a few hundred dollars of API charges and would be bounded, as this one was, by the human time of the people who write, validate and judge the scenarios rather than by the agents who execute them. That, too, is a finding about the economics of autonomous work.134