1\section{Limitations, reproducibility and implications}2\label{sec:limitations}34\subsection{What the study cannot establish}56\paragraph{Designer dependence.} Tasks, oracles and treatments were authored within one organisation by sub-sessions of the same client family that executed them. Fresh contexts prevent executors from seeing answers; they do not make the benchmark externally independent. Reference values rest on analytical identities, independently written code and source documents with locators, and every evaluator was validated on seeded artefacts, but no human verified a single reference value. A reader who wants an externally independent estimate of what Claude Code can do should treat our task suite as one laboratory's instrument, not as a standard.78\paragraph{Scope of the estimand.} The primary estimand is the average paired difference in verified completion over \emph{this} distribution of \nConfTasks{} bounded tasks with one model, one client version, one envelope and one week of provider behaviour. Nothing here licenses statements about scientific research in general, about other models, or about the same model next month. With $J=\nConfTasks$ task clusters, the confidence intervals are wide by construction; we report them rather than the point estimates alone, and we do not add runs after looking at results.910\paragraph{Identification of mechanisms.} The adjacent-arm contrasts attribute differences to structure, gates and review, but each treatment differs from its neighbour in more than one way: arm B is longer than arm A; arm C receives feedback that arm B does not; arm D spends part of its envelope on a second session. The length-matched control addresses the first confound only. We therefore describe B$-$A, C$-$B and D$-$C as mechanism \emph{evidence}, not as identified effects of structure, verification and review.1112\paragraph{Evaluation.} Critical violations are detected by deterministic rules on deliverables; the rules see outputs, not intent, and they are task-specific. A false claim that happens not to trip a rule is scored as a failed criterion, not as a violation; a legitimate but unusual implementation that trips a tolerance is scored as a failure. The pilot found one harness-level false rejection and it was corrected before the freeze; the adversarial audit after the results found four oracle-level false rejections in the confirmatory set (Section~\ref{sec:results}), which is why the frozen result and the rubric-corrected result are both reported and the per-criterion failure table (Appendix~\ref{app:stats}) is provided. A corollary is that the study's confirmatory stage had no discriminating power at the top: with every arm at or near the ceiling, differences of a few percentage points are differences in oracle behaviour, and the preregistered contrasts measure the absence of a protocol effect, not its size. No human reviewer scored any artefact; cells that would require one are unavailable.1314\paragraph{Prompt delivery.} The one-word user prompt of amendment A-006 means that the executors were steered entirely by the system-prompt text and the task file, which is a stronger test of the written contract than intended, and that the controller-enforced feedback loop, the review-repair stage and the resumption instruction were not delivered in any frozen run. The frozen comparison is therefore a comparison of protocol texts; the evidence about controlled feedback and repair comes from the post-fix verification runs only (seven runs). We did not rerun the study after the fix because the defect is symmetric, the controller gate never failed in the benchmark, and the post-fix verification runs behaved as the originals did; a full replication with the corrected launcher is the obvious next step.1516\paragraph{Isolation.} Executors share a user account on each node and could read sibling workspaces or the transcript of a reviewer; the gate program is world-readable. We audited rather than prevented these behaviours: after correcting a path-normalisation error in the audit found by the systems review, \nAuditOutside{} of \nAllRuns{} runs referenced a path outside their workspace (one listed the names of sibling run directories without reading them; one read its own client transcript; two referenced the home directory in a shell idiom), and no run issued a network command, read another run's files or invoked the client; this is weaker than an operating-system boundary per run, and the nodes were not offline, contrary to the wording of the common preamble, only without web tools.1718\paragraph{Autonomy.} The executor sessions were intervention-free, but the study as a whole was not autonomous: a human wrote the mandate, chose the question, named the families and hypotheses, and reviewed the final manuscript. We measured what we could (Section~\ref{sec:reflexive}) and report the rest as unknown.1920\subsection{Reproducibility}2122Three modes are distinguished and all three were exercised. \emph{Infrastructure tests} (\texttt{make test}) and the seeded evaluator validations need no provider access and were run before the freeze. \emph{Artefact replay} (\texttt{make analyze}, \texttt{make paper}, \texttt{release/validate\_release.py}) regenerates every estimate, table, figure and the manuscript from the committed state store; the release validation re-runs the headline analysis in a clean copy and compares the regenerated estimate files byte for byte with the committed ones. \emph{Fresh live replication} requires an API key, executor hosts and the hidden oracles; the exact commands, flags, versions and seeds are recorded, but hosted-model stochasticity is not seedable, so a fresh replication reproduces the \emph{procedure}, not the realised outcomes, and should be compared at the task-cluster level with the intervals reported here. Costs are the client's list-price estimates, which Anthropic documents as estimates; billing-level figures were not available to the session.2324\subsection{Implications for scientific practice}2526Three practical conclusions follow from the design independently of the direction of the results. First, the cost of \emph{verifying} delegated research is not in the agent's inference budget: it is in writing tasks whose answers are known, oracles that cannot be read by the executor, and validation sets that catch evaluator errors, all of which took more builder effort than the benchmark runs cost. Second, most of the scientific-integrity failures that the literature attributes to autonomous research systems (leakage, invalid inference, fabricated counts, false completion) are detectable from deliverables by deterministic rules if the deliverable contract is specified in advance, which argues for contracts over prose instructions. Third, the harness-level failures we encountered (an evaluator invoked on the wrong workspace, a shell quoting error that silently prevented a workspace from being populated) would have produced confident, wrong conclusions had they not been caught by checks that do not trust process exit codes; an executor's declaration of success is the least informative signal in the pipeline.27