# CLAUDE.md — UQO Working Paper 29

Protocol-Guided Autonomous Research with Claude Code

A Reproducible Experimental Study of Scientific Execution, Verification, and Recovery

Principal researcher: Simon-Pierre Boucher, Université du Québec en Outaouais. Project identity: UQO Working Paper No. 29 / WP29; follow the series' verified numbering conventions. Language: English throughout the paper, code documentation, captions, tables, research logs and appendices. Mandate prepared: 29 October 2026. Refresh technical documentation and literature at execution time. Deliverable: a genuine research article supported by an implemented method, live Claude Code experiments, auditable evidence, authentic screenshots, statistical analysis and a reproducibility package.

This file is the complete research execution mandate. It is intentionally more detailed than a routine repository instruction file. Read it in full in manageable sections, build an indexed phase plan, and preserve an immutable copy. If the installed client's context limits make a modular layout necessary, preserve this complete mandate under protocol/WP29_FULL_MANDATE.md, use a concise root CLAUDE.md, and explicitly load and record the sections needed for each phase. Do not discard requirements while reorganizing them.

Provenance note (added by the executor, 2026-10-29): this file is a verbatim preservation of the mandate pasted by the principal researcher into the Claude Code session that started WP29 on 2026-10-29 (local time, Gatineau). Two obvious paste artefacts in §2.1 ("mon WP18–'WP26", "byte-identical l '' L to") are preserved as received. SHA-256 of this file is recorded in protocol/amendments.csv at freeze time.

# 0. Execute the research, not merely a description of the research

You are the research executor for WP29. Your job is to implement, run, measure, challenge and document a method for autonomous scientific research using Claude Code.

The central requirement is real experimentation with Claude Code as an experimental subject. Launch genuine, authorized Claude Code sessions on scientific tasks; collect their actual outputs; evaluate their artifacts; capture the real execution interface or genuine evidence dashboards; measure outcomes; and write the paper from those observations. A plausible account of what an agent would do is not an experiment. Mock providers and replayed responses may test infrastructure but must never be presented as fresh live-Claude observations.

Complete phases in dependency order and continue autonomously through feasible work. Do not stop after a plan, a repository scaffold, a literature summary, a demonstration screenshot or a draft introduction. Make normal reversible technical decisions within the authorized scope, document them, and proceed. Ask only when an indispensable external input, access right, resource limit or scope-changing decision is genuinely missing. Existing authorization remains valid; do not repeatedly request it.

Scientific integrity overrides the desired direction of results:

• Never invent experiments, participants, prompts, screenshots, model identifiers, outputs, tokens, prices, elapsed times, citations, effect sizes or significance.
• Do not assume the proposed method outperforms simpler instructions. A credible null or negative result is acceptable.
• Do not describe the paper as fully autonomous without measuring human preparation, intervention and review.
• Do not conflate a role instruction with an independently running agent, or a different session with independent scientific judgment.
• Do not treat a successful process exit, a compiled PDF or an agent's declaration of success as scientific validation.
• Do not claim experiments were performed on the researcher's infrastructure unless the actual permitted machines and runs were recorded.
• Do not submit, publish, contact reviewers or make external commitments as an implicit part of preparing the manuscript.

# 0.1 WP29 identity and collision handling

Inspect the existing UQO research index and working directory first. The latest user instruction assigns this autonomous-research-method paper to WP29. An earlier idea may also have used WP29. Do not overwrite or delete unrelated work and do not silently renumber this project. If a collision exists, preserve the prior material, document the conflict and create an isolated branch or working directory for the currently requested WP29. Resolve the index transparently before final packaging.

# 0.2 Authorship and disclosure

Use Simon-Pierre Boucher as the named researcher. Verify the affiliation format from current UQO working-paper sources. Do not invent coauthors, grants, funding, approvals, institutional endorsements or human reviewers. Do not automatically apply funding language from another project.

Include a substantive AI-use statement specifying who prepared the mandate, which agent designed the harness, which agents executed tasks, which tools evaluated outputs, and what human review actually occurred. Treat model identities as provenance rather than accountable human authorship. Check the chosen venue's current disclosure requirements before preparing a submission version.

# 1. Scientific question and intended contribution

# 1.1 Main research question

Under controlled information, tool access and resource conditions, does an explicit research protocol, combined with executable verification and persistent evidence records, improve Claude Code's ability to complete scientifically valid research tasks autonomously, and at what cost in compute, time and human effort?

# 1.2 Secondary questions

1. Which benefits come from structured instructions, executable gates, durable state or separate review?
2. Does the method reduce unsupported claims, data leakage, invalid statistical inference, incomplete replication and false declarations of completion?
3. Can execution resume after interruption without losing provenance, duplicating expensive jobs or silently changing the protocol?
4. When do stronger controls reduce throughput or cause unnecessary blocking?
5. Does observed improvement survive task, model-version, budget and domain variation?
6. How much human work is shifted into specification, evaluator construction and final review?
7. Which elements transfer from short scientific tasks to bounded end-to-end research projects?
8. What can screenshots and traces verify, and what requires independent artifact or scientific evaluation?

# 1.3 Candidate contribution

Develop and evaluate a method called Protocol-Guided Autonomous Research, abbreviated PGAR as a working label only. Check prior usage before treating the label as distinctive.

The candidate contribution has four components:

• A scientific delegation contract: objectives, estimands, evidence requirements, permitted adaptation and stopping rules.
• A reference execution system: phase scheduling, durable state, provenance, bounded recovery and executable controls.
• An experimental evaluation: randomized or carefully blocked comparisons using actual Claude Code executions.
• An evidence package: frozen task definitions, traces, results, authentic visual records, evaluation code and a reproducible paper build.

These are proposed contributions, not established achievements. Determine the closest prior work before claiming novelty. Be willing to position the paper as a careful systems-and-methodology evaluation rather than a new general intelligence architecture.

# 1.4 Working titles

Primary working title:

Protocol-Guided Autonomous Research with Claude Code: Scientific Verification, Reproducibility, and the Cost of Delegation

Alternative, if the experiment supports the emphasis:

From Research Instructions to Verified Scientific Execution: A Controlled Study of Claude Code

Choose the final title from the demonstrated scope. Avoid titles implying universal autonomy, guaranteed discovery or superiority that the study has not established.

# 2. Source material and corpus audit

# 2.1 Available methodological starting point

Locate these existing artifacts if supplied in the research environment:

• Methode_recherche_autonome_CLAUDE.pdf: methodological guide derived from the researcher's mandates.
• Kit_recherche_autonome_CLAUDE.zip: reusable protocol and registry templates.
• CLAUDE_WP_UQO_2026-10-29.zip: the collected source mandates.

These files are starting specifications, not a dataset of completed autonomous experiments. Their presence does not establish that any previous working paper was executed without human assistance.

If the files are not mounted, retrieve the accessible primary mandates using actual endpoints and preserve content hashes:

• WP15: https://git.spboucher.ai/raw/wp15_uqo/main/docs/CLAUDE.md
• WP18–WP26: https://git.spboucher.ai/raw/wp{N}_uqo/main/CLAUDE.md
• WP28: https://git.spboucher.ai/raw/wp28_uqo/main/CLAUDE.md
• WP27 supplementary brief: https://git.spboucher.ai/raw/wp27_uqo/main/docs/DESIGN-BRIEF.md

The previously collected corpus contained eleven distinct files named CLAUDE.md: WP15, mon WP18–'WP26 and WP28. WP15's PROMPT_INITIAL_CLAUDE.md was byte-identical l '' L to its CLAUDE.md in that snapshot; check rather than count it twice. WP27's design brief is not an additional CLAUDE.md. Current remote contents may differ from the collected snapshot.

# 2.2 Descriptive analysis of the mandates

Create a reproducible content-coding protocol. For each document record:

• source URL, retrieval time, content hash and available revision metadata;
• research domain and type;
• explicit question, estimand, assumptions and falsification conditions;
• literature and novelty requirements;
• data-access and temporal-information controls;
• baseline requirements;
• phases, dependencies, gates, stopping rules and recovery instructions;
• artifact requirements, provenance, review and disclosure;
• instruction length and contradictory or non-operational requirements;
• statements added during implementation when explicitly identified.

Separate verbatim observations from analyst interpretation. State whether coding was automated, manually reviewed or both. If no human second coder exists, do not invent intercoder reliability. Use this small, researcher-selected corpus to motivate design patterns, not to estimate the prevalence of practices across all research.

Do not treat existing source mandates as hidden evaluation tasks if the executor has already read their solutions or outputs. Derive new, clearly documented task instances for experimental evaluation and separate development from final tasks.

# 3. Literature and technical reconnaissance

# 3.1 Literature streams

Conduct and record a current primary-source review covering:

1. Autonomous scientific discovery and agentic research systems.
2. Language agents for data analysis and scientific programming.
3. Scientific-workflow systems, provenance and computational reproducibility.
4. Preregistration, registered reports and specification discipline.
5. Agent evaluation, benchmark contamination and evaluator validity.
6. Human–AI delegation, automation bias and intervention measurement.
7. Durable execution, failure recovery, observability and resource accounting.
8. Evidence-grounded generation and verification of scientific claims.

Verified starting identifiers, to read in full and check for subsequent versions:

• The AI Scientist-v2: https://arxiv.org/abs/2504.08066
• Agent Laboratory: https://arxiv.org/abs/2501.04227
• ScienceAgentBench: https://arxiv.org/abs/2410.05080

Expand from these sources and their references. Do not copy reported headline results into the manuscript without checking design, units, uncertainty and comparison conditions. Do not label a preprint as peer-reviewed without verification.

Create research/literature_matrix.csv, research/novelty_matrix.csv, research/source_notes.md and verified BibTeX. Compare PGAR to the closest systems on inputs, workflow, verification, human involvement, task scope, resources and evaluated outcomes. Explain which ideas are established and where the present implementation or evidence adds something.

# 3.2 Verify the installed Claude Code environment

Inspect current official documentation and the actual installed client:

• https://code.claude.com/docs/en/memory
• https://code.claude.com/docs/en/headless
• https://code.claude.com/docs/en/cli-reference
• https://code.claude.com/docs/en/hooks
• https://code.claude.com/docs/en/permissions

Record retrieval dates and relevant versions. Run harmless capability probes such as claude --version and claude --help, and inspect the installed authentication/status mechanism without printing secrets.

The documentation consulted when preparing this mandate describes non-interactive execution through claude -p, structured output and session resumption. It also distinguishes instruction loading from enforced permissions and notes that bare mode skips automatic CLAUDE.md discovery. Verify these behaviors on the installed version before constructing experimental arms. Do not assume that passing a filename or mentioning it in prose caused the intended instructions to load.

Use only supported flags and authentication flows. Preserve the exact command, settings and effective loaded context for every run. Do not assume a temperature parameter, random-seed flag, pricing model or model identifier exists. Unsupported controls must be reported as unavailable, not silently simulated.

# 4. Formalize the research method

# 4.1 Research contract

Represent a protocol as:

P = (Q, Θ, D, A, G, V, B, R, L),

where:

• Q: scientific questions and scope;
• Θ: estimands, hypotheses, assumptions and admissible interpretations;
• D: data, documents, versions and information-availability restrictions;
• A: authorized actions and tools;
• G: phase dependency graph;
• V: verification criteria and evaluator definitions;
• B: resource budgets and stopping constraints;
• R: recovery, amendment and escalation rules;
• L: required evidence and provenance records.

Use this notation only if it clarifies actual implementation. Explain how every component is serialized and used. Do not present decorative notation as a theorem or as evidence of rigor.

# 4.2 State and transitions

Define states pending, running, passed, failed, blocked, stale, and skipped_optional. A phase becomes eligible only when its required dependencies are satisfied. An artifact's validity is conditional on the input, code, configuration and criterion versions used to produce and evaluate it.

Specify invariants that the harness can actually enforce:

1. A required phase cannot be passed without its required check receipts.
2. A changed upstream artifact invalidates affected downstream results.
3. A run cannot consume more than the externally enforced resource envelope without being stopped or explicitly amended.
4. A claim cannot be marked verified without a corresponding evidence record and appropriate evaluator outcome.
5. An optional skipped phase cannot be used to support a claim that requires its results.

These are workflow invariants, not guarantees that every scientific conclusion is true. State the limits of the checkers. If presenting propositions about state consistency, give precise assumptions and a short proof or exhaustive property tests as appropriate; never imply those properties establish scientific correctness.

# 4.3 Observable autonomy

Distinguish:

• workflow autonomy: phases executed without operational human guidance;
• scientific completion: artifacts satisfy the independent task rubric;
• intervention-free verified completion: both conditions hold;
• autonomous execution with incorrect output: no intervention, but scientific failure;
• correct assisted completion: success after substantive human intervention.

The method must preserve these distinctions in its database, plots, tables and claims.

# 5. Build a reference implementation

# 5.1 Components

Implement a real, runnable system with:

• a task registry and immutable task bundles;
• treatment-specific instruction assembly;
• isolated executor workspaces;
• a bounded process/session launcher;
• an event collector and durable state store;
• deterministic verification adapters;
• a separate scientific-evaluation layer;
• a resource and cost accountant;
• failure-injection and recovery machinery;
• an evidence browser and screenshot capture mechanism;
• statistical analysis and manuscript-generation pipelines.

Prefer a small auditable Python implementation using standard libraries and justified dependencies. SQLite is adequate for structured state when concurrency and transactions are handled correctly. Export analysis-ready CSV/Parquet files. Avoid building a large web platform before the core experiment runs.

# 5.2 Separation of roles and trust

Distinguish the builder, executor, runtime verifier, final evaluator, reviewer and reporting pipeline.

The builder may be Claude Code developing this study. The executors must be fresh actual Claude Code sessions under the assigned treatment. The final evaluator must not rely exclusively on the executor's self-assessment.

Keep hidden labels, expected results, held-out instances, frozen rubrics and authoritative evaluator logs inaccessible to executors through real filesystem/process/access isolation. A prompt saying "do not read the answers" is insufficient. A Git worktree with read access to its parent or sibling directories is not a security boundary.

Use supported containers, separate OS identities, mounted read-only bundles or another verifiable isolation mechanism. Preserve legitimate authentication without exposing credentials. Test the boundary using harmless probes. Do not weaken permission systems or defeat nested-session protections to make an experiment run. If the supported client disallows nested invocation, use an authorized external supervisor or documented session launch mechanism, and record that architecture.

The builder's involvement in task design is a limitation. Fresh executor contexts do not make the whole benchmark externally independent. Prefer reference answers from analytical identities, independently implemented deterministic solutions, external benchmarks or an evaluator-controlled task generator. Document who could access answers at each stage.

# 5.3 Instruction and memory contamination

Experimental workspaces must not inherit this WP29 mandate or the full PGAR treatment when assigned to a weaker arm. Audit parent CLAUDE.md files, user-level instructions, auto memory, rules, plugins, hooks, MCP servers, cached state and conversation history.

Apply a verified, supported isolation configuration consistently across arms. If bare mode is used, explicitly supply the intended instructions and needed capabilities for every arm and document any departures from ordinary interactive usage. Do not accidentally compare different instruction loaders, tool inventories or authentication behaviors as if only protocol structure differed.

Record both intended and effective configuration. Test isolation with innocuous sentinel instructions before final experiments, then remove sentinels from the scientific tasks. Failed isolation invalidates the affected comparison and requires a documented rerun policy.

# 5.4 Durable execution

Persist state atomically; log jobs before submission; capture process/session/job IDs; reconcile external jobs before resuming. Make retry policy idempotent where possible. Separate transient service retries from additional scientific reasoning attempts and charge both to the resource accounting.

The controller uses bounded work units and explicit termination conditions. It must not recursively spawn unbounded copies of itself. Apply an allowlisted executor entry point and a configured maximum process/session depth. Experimental workers solve assigned tasks; they do not launch another WP29 study.

# 6. A genuine self-experiment without circular validation

"Claude experimenting with itself" means that Claude Code helps implement a study in which actual Claude Code instances are subjects under controlled conditions. It does not mean introspection, self-reported competence or a dialogue in which one role declares another role successful.

For each observation preserve:

1. Frozen task, inputs and treatment assignment.
2. Fresh execution identity and effective environment.
3. Actual invocation, live events and tool outputs.
4. Submitted artifacts and terminal status.
5. Independent evaluation receipts.
6. Resource usage and intervention history.
7. Screenshots or other visual evidence tied to the run.

The principal experimental observations must use real Claude Code inference. A simulator, a provider mock, copied responses, or a non-Claude model behind a compatibility endpoint cannot be counted as a live Claude Code observation. Such runs may appear in a separate engineering validation section with accurate labels.

Do not request or invent hidden chain-of-thought. Analyze observable actions, visible messages, tool calls, files, explicit decisions and errors. Describe this as behavioral and artifact evidence, not privileged access to the model's internal reasoning.

If live execution is genuinely unavailable, complete infrastructure and clearly labeled mock tests, but report the empirical study as blocked. Do not submit a fictional results paper to satisfy the mandate. The requested completion target still requires live experiments once the missing access is supplied.

# 7. Research hypotheses and primary endpoints

Formulate and freeze hypotheses before confirmatory runs. Suggested hypotheses:

• H1: a structured protocol improves intervention-free, science-verified completion relative to an information-matched open instruction.
• H2: executable gates reduce critical integrity failures relative to a protocol without mandatory runtime gates.
• H3: separate review improves claim validity, with measurable additional cost.
• H4: persistent state and recovery rules improve valid continuation after standardized interruptions.
• H5: benefits vary by task family and task complexity.
• H6: human preparation and review costs materially change the apparent efficiency gains.

These hypotheses are not guaranteed directions of effect. Report uncertainty and counterevidence.

# 7.1 Primary endpoint

Use intervention-free science-verified completion within the assigned budget as the primary binary endpoint, unless the pilot reveals a defensible reason to amend before confirmatory evaluation.

For task instance i, arm a, repeat r:

Y_iar = 1{required scientific criteria satisfied, no critical integrity violation, no substantive human intervention, within resource envelope}.

The criteria must be task-specific, defined in advance and evaluated outside the executor. A complete-looking report that is wrong scores zero. A correct result after assistance is recorded as assisted success, not unassisted success. Appropriate recognition of an intentionally infeasible task is evaluated under that task's predeclared rubric and reported separately from completion of feasible tasks.

Do not make a verbose refusal or an empty output score highly by avoiding claims. Fixed required scientific outputs remain in the denominator.

# 7.2 Secondary endpoints

• Scientifically correct completion, with and without assistance.
• Fixed-rubric scientific-quality score and its component scores.
• Critical integrity violations: leakage, fabrication, invalid inference, unreported data substitution, false completion or corrupted provenance.
• Unsupported central claims and errors per required claim set.
• Additional unsupported claims introduced voluntarily, reported separately.
• Fresh-environment reproducibility success and numerical agreement.
• Correct detection of blocked or invalid tasks; false blocking on feasible tasks.
• Recovery success, duplicate jobs, lost artifacts and stale-result reuse.
• Latency, total inference usage, actual known costs and human minutes.
• Cost per assigned task and cost per verified completion, with undefined ratios reported when no task succeeds.

Report instrument coverage and missing measurements. Do not quietly infer missing usage or exact cost from a rough wall-clock estimate.

# 8. Experimental treatments and fairness

# 8.1 Four main arms

All arms receive the same substantive scientific question, accessible input data, relevant task definitions, output obligations and common safety/integrity constraints. Do not construct A by deleting information necessary to solve the task. The experimental intervention changes how the work is organized and checked.

Provide all arms with the same common final outcome evaluation after submission. That evaluator is distinct from the runtime gate feedback offered only to the relevant arms. Its hidden answers are never returned to workers. An executor in A or B may voluntarily write and run tests; record this rather than forbidding competent behavior.

# 8.2 Shared resource envelope

Match model identity, available tools, scientific input information, wall-time ceiling, total inference budget and computational allowance as closely as the platform permits. In D, reviewer calls consume the same total envelope rather than free additional compute. Keep an explicit accounting of executor and reviewer costs.

Report both equal-budget comparisons and, when feasible, quality-cost frontiers across prespecified budgets. A more complex method may spend more on instructions or checks; that overhead is part of the treatment cost, not something to hide.

# 8.3 Length and information controls

Add a secondary long-form, unstructured instruction control or a matched-content comparison to distinguish structure from extra scientific information and instruction length. Do not add meaningless padding merely to match token counts. Publish arm prompts, a content-equivalence audit and measured initial-context sizes.

Treat this as a separate ablation if including it would exceed the primary budget. Do not imply the four main arms fully identify a "pure effect of structure" when content, feedback and computational allocation also differ.

# 8.4 Randomization and temporal blocking

Use paired task instances across arms. Randomize arm order and block by task, execution window and supported model version. Avoid running all baseline tasks first and all full-method tasks after a provider update. Independent repeats use fresh sessions and clean workspaces; changing only a directory name is not enough.

Fixed seeds can control task generation and deterministic tools. They may not control hosted model stochasticity. Preserve actual repeat outcomes and document the limits of deterministic replication.

# 9. Benchmark suite with scientifically meaningful tasks

Build six families with explicit scoring oracles and artifact requirements. Favor bounded tasks that require scientific reasoning, data handling and evidence, rather than arbitrary coding puzzles.

F1 Data provenance and audit

Examples: duplicate entities, inconsistent units, unavailable-at-origin variables, a changing sample universe, timestamp shifts, and a corrupted input file. Tasks should require a documented analytic dataset and an exclusion ledger, not only bug discovery.

Reference evaluation: independently computed counts, unit conversions, temporal constraints and permitted exclusion rules. Preserve legitimate variation in implementation.

F2 Estimation and statistical inference

Examples: reproduce a regression with specified fixed effects; select uncertainty estimates under a known dependence design; detect a deliberately inappropriate standard error; perform a Monte Carlo coverage analysis.

Use analytical or independently coded references where possible. Define numerical tolerances from mathematical or statistical reasoning. For stochastic quantities, compare with uncertainty rather than demanding identical last decimals.

F3 Forecasting and temporal information

Examples: rolling-origin evaluation on a public economic series; a feature with release lag; revised target data; a future-informed normalization step; forecasting at multiple horizons with overlapping targets.

Require a simple valid baseline and untouched evaluation information. Reference checks must verify chronology, not just score prediction accuracy. Do not claim historical real-time validity from a single final-vintage dataset.

F4 Source-grounded evidence synthesis

Use accessible full-text sources with verifiable tables or claims. Tasks can involve extracting coefficients and uncertainties, checking whether a cited source supports an assertion, and separating incompatible effect-size definitions.

Reference evaluation requires page/table locators and independently checked values. Respect source rights. Do not infer that a citation exists because a generated DOI looks plausible. If human annotation is unavailable, identify the alternative reference process and its uncertainty.

F5 Theory, mechanisms and numerical validation

Examples: a tractable equilibrium with a budget identity, a statistical estimator with a known bias property, a proposed statement admitting a counterexample, or a numerical solver with a known limiting solution.

Require explicit assumptions and correct scope. A large simulation does not prove a universal statement. Use exact small cases and deliberately broken identities to test validators. Formal verification is optional when feasible, not something to claim without an actual proof-checker run.

F6 Integrated scientific mini-projects

Require a complete bounded chain: question, data/source audit, benchmark, justified method, results, uncertainty, failure discussion, generated table/figure and short report. Include at least one genuine negative result or invalid-premise task designed before evaluation.

Evaluate the chain and the scientific interpretation. Do not let attractive prose compensate for wrong calculations. Keep this family sufficiently substantial to distinguish autonomous scientific execution from a collection of unit-test coding exercises.

# 9.1 Data sources and representativeness

Use real, publicly accessible research data and primary documents for a substantial part of the suite. At the target scale, aim for at least eight real-data tasks and four genuine source-document tasks; classify overlapping categories and report counts clearly. Select actual datasets after verifying access, provenance, license, schema and suitability. Possible families include official economic time series, open statistical datasets and openly reproducible published analyses. These are candidates, not assertions that particular endpoints are available.

Synthetic tasks with known truth remain scientifically useful for measurement and stress testing, but must be labeled. "Real experiment" refers to actual agent execution; distinguish that from whether the task uses observed or synthetic data.

Maintain tasks/task_cards/ with a card for each instance: question, data type, scope, prerequisites, allowed information, expected artifacts, rubric, oracle provenance, difficulty rationale, budget, failure modes and contamination risk.

# 9.2 Split and leakage controls

Use separate development, pilot and confirmatory task sets. Do not tune the protocol or validators against the final outcomes. Freeze task and oracle versions before the final evaluation. Executor sessions cannot access sibling outputs, previous attempts, scoring keys or the manuscript's results section.

For externally published tasks, report possible pretraining contamination. Newly generated parameters reduce direct answer copying but do not eliminate familiarity with the underlying problem. For internally authored tasks, report evaluator/designer dependence rather than claiming external independence.

# 10. Scale, budget and stopping design

# 10.1 Staged plan

Treat the following as planning configurations, not completed counts or automatic power justifications:

The target task benchmark comprises four tasks in each of six families. The transfer mini-projects are additional held-out projects, not relabeled repetitions of F6 tasks. A reviewer stage in D can require multiple sessions; distinguish task executions from session counts and API calls.

Before scaling, measure actual pilot costs and durations. Account for instruction tokens, retries, compilation, capture overhead, reviewer calls, evaluator time and human preparation. Estimate a resource envelope with a documented contingency allowance. Reuse existing explicit budgets. If indispensable paid capacity has not been authorized, obtain a precise budget decision rather than launching an uncapped run.

If resources support only a smaller study, define and freeze a reduced design before examining confirmatory outcomes. A possible exploratory configuration is 12 tasks × 4 arms × 2 repeats, with explicit limits. Do not describe this as automatically sufficient for confirmatory inference. Report reduced coverage and uncertainty.

# 10.2 Precision and power

Use development-only estimates or justified scenarios to assess detectable differences and interval widths at the task-cluster level. Increasing repeats of the same small task set does not create the same scientific coverage as more independent tasks.

Choose the final design based on feasible precision, cost and domain coverage. Preserve the design decision and simulation code. Do not continue adding runs until a preferred effect becomes significant.

# 10.3 Stopping and exclusions

Set time, cost, call and retry limits before a run. A scientific failure remains a failure. Infrastructure outages have a separate status and a predeclared policy for balanced reruns. Retain every assigned run in the registry and report intention-to-treat results alongside any justified sensitivity analysis.

Do not silently remove failed, slow or embarrassing executions. Report assigned tasks, launched runs, completed runs, evaluated runs, scientific successes and missing measurements separately.

# 11. End-to-end transfer study

Conduct real bounded research projects beyond the short benchmark. Suggested designs, to finalize after data access checks:

1. Empirical forecasting project: a public economic series with explicit availability dates, rolling evaluation, naive and statistical baselines, uncertainty and a short paper.
2. Simulation-methodology project: a finite-sample estimator comparison with known analytical properties, Monte Carlo uncertainty, failure regimes and an executable technical note.
3. Evidence-synthesis project: a small set of accessible primary studies requiring source extraction, careful harmonization and an appropriately limited synthesis.

Require a research report, actual calculations, provenance, generated plots/tables and a replication entry point. Set a bounded scope so all arms have a plausible opportunity within the shared resource envelope. Do not demand a novel top-journal discovery as a binary scoring criterion.

Use the same outcome rubric across arms within each mini-project. Report transfer results separately from the task benchmark. With only a few projects, emphasize case-level results and limited generalization; do not turn many generated files into fictitious independent sample units.

The fact that WP29 itself is produced under this mandate is an additional instrumented case study. It is not an independent confirmatory observation and must not be pooled with the randomized benchmark as such.

# 12. Stress tests and mechanism ablations

Implement standardized, harmless faults inside isolated research workspaces. Examples:

• Interrupt a worker after a known checkpoint, then resume it.
• Remove access to an explicitly required local fixture.
• Change a non-secret upstream data file and test stale-output invalidation.
• Introduce a duplicate job submission request and test idempotency.
• Supply a deliberately inconsistent unit or timestamp in a labeled evaluation fixture.
• Return a controlled transient error from a test service adapter.
• Make a previously plausible citation fail source verification.
• Present a scientific hypothesis for which the reference result is null or negative.

Never corrupt real user data or unrelated repositories. Fault injection is part of an experimental condition and must be logged outside the executor. If a service failure is simulated while Claude Code execution is live, state both facts precisely.

Predefine the intervention point, fault severity, allowable recovery actions and success criteria. Compare recovery against a matched no-fault run where feasible. Separate "recognized a blockage" from "completed the intended research despite the fault."

Mechanism ablations can remove one component from PGAR: persistent state, mandatory gates, claim ledger, explicit failure rules or review. Use matched information and budgets. Avoid an enormous factorial design that is impossible to interpret or afford. Select the most informative ablations before final results.

# 13. Evaluation validity and human involvement

# 13.1 Independent outcome evaluation

Use task-specific deterministic checks where scientific truth is available. For interpretive outputs, combine a frozen rubric with source verification and a clearly identified review process. Human expert review is desirable for a prespecified sample of central claims and proof tasks when actual reviewers are available and authorized.

Do not fabricate human scores, consent, credentials or recruitment. If no human reviewers participate, label those cells unavailable and disclose the resulting limitation. A simulated economist role is not a human economist.

Model-based grading may be a supplementary measure. Record the grading model, prompts, context, rubric and response. Test graders against seeded correct and incorrect artifacts; report false acceptance and false rejection. A same-family model grader may share systematic errors with the executor.

# 13.2 Blinding and rubric design

Use neutral run IDs and remove explicit treatment labels from scoring bundles when feasible. Some artifacts may reveal the treatment; document imperfect blinding. Do not claim a blinded evaluation if the grader can plainly see arm labels or prior results.

Grade required scientific outputs, not generic writing quality alone. Specify critical failures that cannot be offset by a polished report. A quality score can summarize components, but its weights are frozen before final scoring and its components are also reported.

# 13.3 Human intervention ledger

Record preparation, configuration, troubleshooting, scientific guidance, direct file edits, evaluator construction, screenshot curation and final review. Log actor, timestamp, phase/run, intervention category, duration if measured and scientific impact.

If historical preparation time is unknown, state that it is unknown. Do not reconstruct exact minutes from memory. Distinguish observed time, instrumented estimates and unmeasured effort. A one-time setup cost may be amortized under explicit scenarios, with the unamortized cost also shown.

# 14. Statistical analysis plan

# 14.1 Unit of analysis and estimands

The execution is indexed by task, arm and repeat, but repetitions within a task are dependent. Report task-level paired effects and family-level heterogeneity. The main estimand is the average difference in intervention-free verified completion between specified arms over the evaluated task distribution, not all possible scientific research.

Define the primary contrast in advance, for example D versus A. Treat B versus A, C versus B and D versus C as prespecified mechanism contrasts, with appropriate multiplicity handling or explicit secondary status.

For a task-weighted completion estimate, use:

p̂_a = (1/J) Σ_{j=1}^{J} ( (1/R_j) Σ_{r=1}^{R_j} Y_{jar} ), Δ̂_ab = p̂_a − p̂_b.

State the weights, treatment of unequal repeats and missingness. Do not allow a heavily repeated easy task to dominate by accident.

# 14.2 Uncertainty

Use task-cluster bootstrap or randomization/permutation procedures appropriate to the assignment mechanism and sample size. With few clusters, report their limitations and avoid relying on asymptotic normal approximations alone. Hierarchical models may be sensitivity analyses if their assumptions and priors are explicit.

Do not treat screenshots, paragraphs, tool calls or multiple rubric items from one run as independent experimental subjects. For claim-level error analyses, retain task/run dependence.

# 14.3 Cost, time and failure

Report distributions, not only averages. Provide medians, tails, success-conditioned and unconditional quantities where informative, with clear denominators. Report completion by the common deadline; treat unfinished task times as censored when using time-to-completion methods rather than recording timeout as successful completion.

Show costs per assigned run and total resources spent, including failed attempts. Separate setup costs, experimental inference, review, deterministic evaluation and documentation overhead. If billing is subscription-based or exact run cost cannot be attributed, use observed usage/time and transparent accounting scenarios instead of invented dollar amounts.

# 14.4 Robustness

At minimum assess sensitivity to task family, complexity, failure definitions, budget, uncertain rubric labels and alternative reasonable aggregation. Add model/version replication when feasible and label it exploratory if not preregistered.

Run negative-control analyses and data-quality assertions on the analysis dataset. Independently recalculate headline table cells. All manuscript numbers must come from saved outputs with identifiers and uncertainty.

# 15. Real screenshots and visual evidence are mandatory

# 15.1 What must be captured

Create authentic visual records of the actual experiment and the implemented system:

1. Environment/version verification in the research-only terminal or client.
2. An actual task's assigned protocol and effective treatment configuration.
3. A real Claude Code execution while it is operating on a task.
4. Actual generated code or data artifact tied to that run.
5. A real gate rejection or test failure, when one occurs or is deliberately induced in a labeled stress test.
6. A real correction and rerun, with distinct before/after event IDs.
7. A real interruption/recovery sequence in the dedicated recovery experiment.
8. The experiment-status dashboard showing actual registered runs.
9. A generated results table or plot backed by the frozen analysis data.
10. The compiled WP29 manuscript and evidence/replication inventory.

Do not fabricate an error to make the main experiment look dramatic. Controlled fault-injection evidence is welcome but must be labeled as such.

# 15.2 Acceptable capture modes

• Native live capture: an actual research terminal, Claude Code UI or research application captured with permitted OS/browser tools.
• Live evidence dashboard capture: a browser screenshot of a real local dashboard populated from the actual event store while runs execute.
• Post-run evidence viewer capture: a screenshot of a real viewer showing archived run data; explicitly label it post-run.
• Transcript replay: optional rendering of a recorded terminal session, explicitly labeled replay and not substituted for a live UI capture.

Use the capture tools available on the verified platform. On a headless environment, implement a simple browser evidence viewer over real event data and use a genuine browser screenshot. If a native Claude Code UI cannot be captured, report that limit and distinguish dashboard captures from native screenshots. Do not imply that a screenshot of an HTML imitation of a terminal is a native Claude session.

Do not use image-generation tools to create evidence. Do not manufacture terminal output, timestamps, charts, progress bars, chat bubbles or successful tests. Diagrams can be programmatically drawn, but label them as diagrams rather than observations.

# 15.3 Capture contract

Implement a capture mechanism with configured event triggers rather than relying exclusively on handpicked successes. For each image record:

screenshot_id; run_id and task_id; treatment and phase; event_id or event-range linkage; capture_timestamp_utc; capture_mode: native_live / dashboard_live / viewer_postrun / transcript_replay; capture_tool and version; source_surface or local URL; original_path and SHA-256; redacted_public_path and SHA-256, if applicable; redaction/crop description; selection_rule; caption and claim IDs illustrated.

Hashing shows file integrity relative to the recorded manifest; it does not independently prove when or how the image was captured. Triangulate screenshots with event timestamps, logs, process IDs and artifact hashes. Describe their evidentiary limits.

# 15.4 Privacy and presentation

Capture only the research workspace and necessary application regions. Exclude secrets, private messages, unrelated windows and personal information. Prefer preventing sensitive content from entering the capture at all. Keep authorized originals in restricted storage; create separately hashed public redactions when needed and document what was removed. Never alter scientific values or hide an error to improve appearance.

Produce a prespecified audit gallery. For example, capture fixed milestones for a stratified subset of task families and arms, plus all fault-injection milestones. Set storage bounds. Report missing captures and capture failures. Screenshots document events; they are not independent samples for inference.

Use selected readable screenshots in the main paper and a fuller gallery in an appendix or evidence supplement. Provide captions with run ID, treatment, event, capture mode and what the image does and does not establish.

# 16. Experiment records and evidence schemas

Use explicit schemas and schema validation. Prefer structured records to prose-only logs.

# 16.1 Run manifest

At minimum:

{
  "run_id": "generated_by_harness",
  "task_id": "frozen_task_identifier",
  "split": "development_or_pilot_or_confirmatory_or_transfer",
  "treatment": "A_or_B_or_C_or_D",
  "repeat_index": 0,
  "task_hash": null,
  "protocol_hash": null,
  "effective_context_hash": null,
  "code_commit": null,
  "environment_id": null,
  "client_version": null,
  "model_requested": null,
  "model_reported": null,
  "session_ids": [],
  "started_at_utc": null,
  "ended_at_utc": null,
  "status": "not_started",
  "budget": {},
  "usage": {},
  "human_intervention_ids": [],
  "artifact_manifest": null,
  "evaluation_receipt": null,
  "screenshot_ids": []
}

This is a schema illustration, not an example completed run. Replace null fields from actual observations. Validate required fields at each state transition; unavailable metadata must be explicitly classified rather than filled with invented values.

# 16.2 Event stream

Record unique event ID, run/session ID, UTC timestamp, monotonic elapsed time where available, event type, tool/phase, status, artifact references, duration, usage and error classification. Keep raw authorized provider output and normalized records separate. Never log authentication material.

# 16.3 Scientific claim ledger

Record claim ID, exact text, claim type, scope, hypothesis, task/run or proof/source IDs, evidence paths, evaluation status, assumptions, uncertainty, manuscript location and amendments. Separate the claims made by experimental agents from claims made by the WP29 authors about the experiment.

# 16.4 Dataset and transformation ledger

Record snapshot hashes, origin, retrieval time, units, schema, information-availability rule, license, parent artifacts, transformations and exclusions. Evaluation oracles have a separate protected ledger inaccessible to workers.

# 16.5 Intervention and amendment ledgers

Record every substantive human change and every alteration to a frozen design. Preserve previous versions and identify whether a change was made before or after seeing results. Frozen confirmatory results must not be overwritten by exploratory reruns.

# 17. Repository structure

Adapt to existing UQO conventions while preserving the following functional separation:

wp29-autonomous-research/
  CLAUDE.md
  README.md
  Makefile
  pyproject.toml
  environment.lock
  .env.example
  protocol/
    WP29_FULL_MANDATE.md
    preregistration.md
    scientific_contract.md
    treatment_definitions.md
    evaluation_plan.md
    screenshot_protocol.md
    amendments.csv
  research/
    corpus_manifest.csv
    corpus_coding.csv
    literature_matrix.csv
    novelty_matrix.csv
    source_notes.md
  configs/
    development.yaml
    pilot.yaml
    confirmatory.yaml
    transfer.yaml
    budgets.yaml
    capture.yaml
  tasks/
    registry.csv
    task_cards/
    public_bundles/
  src/pgar/
    contracts.py
    scheduler.py
    state.py
    launcher.py
    isolation.py
    events.py
    provenance.py
    budgets.py
    recovery.py
    runtime_gates.py
    capture.py
    evidence_viewer.py
    exports.py
  evaluators/
    README.md
    public_evaluation_interface/
  tests/
    unit/
    integration/
    isolation/
    invalid_artifacts/
  runs/
    development/
    pilot/
    confirmatory/
    transfer/
  evidence/
    manifests/
    public_screenshots/
    public_traces/
    galleries/
  analysis/
    build_analysis_dataset.py
    primary_analysis.py
    robustness.py
    figures.py
    tables.py
  results/
    analysis_data/
    estimates/
    figures/
    tables/
  reports/
    phase_status.md
    interventions.csv
    environment_audit.md
    isolation_audit.md
    evaluator_validation.md
    pilot_report.md
    experiment_accounting.md
    failure_analysis.md
    final_review.md
    reproducibility_report.md
  paper/
    main.tex
    references.bib
    sections/
    appendices/
    figures/
    tables/
  release/

Hidden evaluation material and restricted originals live outside all executor-accessible mounts. The repository tree alone does not enforce that boundary. Document the actual external evaluator storage and access controls without revealing secrets.

# 18. Figures required for the research paper

Generate each quantitative figure from the frozen analysis dataset. Include uncertainty and sample counts where relevant. Use vector output for charts and diagrams, and native-resolution raster images for screenshots. Do not substitute screenshots of plots for reproducible plotting scripts.

Target figure set, retaining only figures supported by completed work:

1. Method architecture: researcher contract, executor, runtime gates, persistent state, evaluator and evidence store; clearly show trust boundaries.
2. Experimental design: task splits, treatment assignment, repeated executions, budget matching and final evaluation.
3. Phase and evidence graph: actual artifact dependencies and invalidation paths in the implemented method.
4. Primary completion comparison: arm-level and task-paired effects with uncertainty.
5. Scientific-quality and integrity outcomes: component scores and critical failure rates.
6. Quality-cost frontier: actual usage/cost and scientific outcomes under declared budgets.
7. Task-family heterogeneity: paired effects or completion rates by family, with sample sizes.
8. Recovery experiment: observed event timeline and recovery outcomes under standardized faults.
9. Human-effort decomposition: preparation, execution assistance and review, with unknown amounts visible.
10. Authentic screenshot case study: protocol, live run, detected failure, correction and verified artifact, tied to event IDs.
11. Error taxonomy: rates of distinct scientifically meaningful errors; permit multiple errors per run without mislabeling the denominator.
12. Transfer mini-projects: case-level results and evidence coverage.
13. Ablations or sensitivity: selected component removals and alternative scoring assumptions.
14. WP29 reflexive execution trace: a descriptive account of how this paper was produced, clearly separated from confirmatory evidence.

Do not force fourteen figures if fewer experiments support fewer figures. A complete evidence gallery belongs in the supplement, not in an unreadable main-paper collage.

# 19. Tables required for the research paper

Produce executable tables with explicit denominators, units, statuses and notes:

1. Closest prior systems and the precise contribution relative to them.
2. Source mandate corpus and coded design features.
3. Scientific contract components, implementation and verification mechanisms.
4. Task suite, domains, data type, reference-answer provenance and difficulty.
5. Treatment differences, common information, tools, budgets and context sizes.
6. Assignment and execution accounting: planned, assigned, launched, completed, failed, blocked and evaluated.
7. Primary endpoint, effect sizes, uncertainty and task-cluster counts.
8. Scientific-quality components and integrity violations.
9. Runtime, usage, cost and success-conditioned efficiency with honest accounting.
10. Recovery and fault-injection outcomes, including false blocking.
11. Ablations, length/information controls and robustness analyses.
12. End-to-end mini-project results and verified artifacts.
13. Human interventions and preparation/review effort.
14. Limitations, unresolved risks and the scope of claims actually supported.

Save table source data and code alongside LaTeX outputs. No manual numeric editing in the manuscript. Missing values are labeled with a reason; zero, unavailable and not attempted are different states.

# 20. Manuscript requirements

Produce an English research article suitable for serious review in AI-for-science, agent evaluation, research methods or computational research infrastructure. It must be an argument supported by experiments, not a user manual padded into a paper.

Suggested structure:

1. Introduction: scientific motivation, question, contribution and actual findings.
2. Related work: autonomous research, agent evaluation, workflows and reproducibility.
3. Conceptual framework: delegation contract, autonomy and evidence.
4. Corpus-informed method design: what the prior mandates motivate and cannot establish.
5. PGAR implementation: contracts, phases, controls, state, isolation and evidence.
6. Experimental design: tasks, treatments, budgets, randomization and estimands.
7. Evaluation: scientific rubrics, protected oracles, intervention measurement and uncertainty.
8. Main results: completion, quality, integrity and resources.
9. Recovery, ablations and failure analysis.
10. End-to-end transfer and authentic visual case studies.
11. Reflexive analysis of the WP29 production process.
12. Limitations, reproducibility and implications for scientific practice.
13. Conclusion proportional to the evidence.

Appendices: complete prompts and protocol versions; task cards and rubrics; evaluator validation; statistical details; capture methodology and gallery; model/environment records; intervention ledger; extra results; reproduction instructions.

Aim for roughly 25–40 substantive main-text pages plus justified appendices if the work supports it. Page count is not an acceptance criterion. Do not inflate the text with repeated descriptions of autonomy. All figures must be discussed in the argument. Describe null results and failed tasks seriously.

Use the actual UQO title-page and working-paper conventions. Verify author metadata. Use clear notation, numbered equations where useful, booktabs-style tables, linked references and legible figures. Write the abstract and final introduction after the main findings are verified.

# 20.1 Claims that are not permitted without evidence

• "Fully autonomous science" from a few successful task runs.
• "Independent reviewer" solely because a role prompt uses that word.
• "Novel scientific discovery" when the task reproduces a known answer.
• "Reproducible" when only archived responses were replayed.
• "Lower cost" when setup and reviewer compute were omitted.
• "Real-time data" when only a revised final-vintage dataset was used.
• "Authentic native screenshot" for a fabricated or reconstructed interface.
• "Human-level" or "better than researchers" without a valid human comparison.
• "All checks passed" without executed receipts and an appropriate scientific scope.

# 21. Reproducibility and release modes

Implement and distinguish three modes:

1. Infrastructure tests: small fixtures and mocks that validate orchestration, schemas, gates and capture plumbing. They do not require provider access and do not generate scientific treatment-effect results.
2. Artifact replay: reconstruct the analysis, tables, figures and manuscript from recorded authorized outputs. This tests downstream reproducibility, not fresh model behavior.
3. Fresh live replication: launch new Claude Code sessions using the recorded task and treatment protocol, acknowledging provider stochasticity, model changes and access requirements.

Provide real commands matching the implemented code. A possible command interface is make test, make smoke-live, make pilot, make main, make analyze, make paper, make replay, make validate-release; these names are proposals until implemented. Never claim they ran before actual execution.

Pin the environment as far as practical; record operating system, architecture, client version, model metadata, tool versions and significant configuration. Store authorized raw traces and responses securely, and create a public subset where disclosure is permitted.

Build a release manifest with file hashes, provenance, licensing notes and exclusions. Validate a clean replay of the headline analysis. Inspect the final PDF page by page for unreadable screenshots, clipped formulas, broken tables and unresolved references. The PDF must be nonempty, searchable and consistent with the saved results.

# 22. Sequential execution phases and acceptance gates

Phase 00 — Inspect identity, sources and environment

Inspect the UQO index, WP29 collisions, existing files, source artifacts, current official documentation, Claude Code availability, auth status, resources and capture capabilities. Produce reports/environment_audit.md, a source inventory and a concrete gap list.

Gate: project identity is protected; real capabilities and missing inputs are distinguished; no credentials are exposed. Continue feasible setup while any external blocker is resolved.

Phase 01 — Literature and scientific blueprint

Read primary prior work, code the source mandates, define the candidate contribution and write the smallest executable study blueprint. Establish scientific endpoints and competing explanations.

Gate: closest prior work is identified; novelty claims are qualified; the study tests a method rather than assumes it works.

Phase 02 — Protocol, task suite and evaluator specification

Create treatment prompts, task cards, evaluator definitions, splits, budget plan, capture protocol, intervention categories and analysis plan. Design protected oracles and task-level assignment.

Gate: scientific information is matched across arms; success/failure definitions are frozen for the pilot; hidden answers have an explicit protection plan; negative and infeasible cases are covered.

Phase 03 — Implement the minimal harness

Build task loading, isolation, genuine session launching, event storage, resource limits, output collection and deterministic evaluation. Use mocks only for infrastructure tests. Test successful and deliberately invalid artifacts.

Gate: actual tests show the harness rejects known defects and preserves provenance; worker access to hidden evaluation material is prevented by tested controls.

Phase 04 — Capture system and evidence viewer

Implement actual screenshot capture and a small evidence viewer backed by the real database. Test event-triggered capture, hashes, redaction and caption metadata. Measure overhead consistently across arms.

Gate: at least one genuine capture is linked to an actual test event; synthetic UI imagery is not used; public evidence contains no secrets. Mark infrastructure-only captures accordingly.

Phase 05 — Real smoke experiment

Run the four-arm smoke configuration using actual Claude Code inference on one development task, subject to available authorization and limits. Capture real sessions/events and independently grade the artifacts.

Gate: four genuine task executions are accounted for, including failures; configuration isolation and treatment delivery are verified; the results are development observations, not confirmatory findings.

Phase 06 — Development pilot and design freeze

Run the development pilot, diagnose instrumentation and evaluator failures, measure costs and set the final affordable design. Perform task-cluster precision analysis. Freeze final tasks, prompts, thresholds, budgets, assignment, screenshot sampling and analysis code interfaces.

Gate: dated protocol and hashes exist; amendments are traceable; pilot tasks do not contaminate the final set; the chosen scale is justified rather than assumed sufficient.

Phase 07 — Main benchmark

Execute the frozen assignments with balanced order and actual live sessions. Preserve every run, event, failure, intervention, artifact and required capture. Monitor limits without tuning the method on outcomes.

Gate: the execution ledger reconciles with the frozen assignment; integrity and capture coverage are audited; completion counts are real and independently checked.

Phase 08 — Transfer, recovery and selected ablations

Run the separately specified mini-projects, controlled recovery tasks and prespecified ablations. Keep exploratory follow-ups separate. Capture before/failure/after evidence for recovery cases.

Gate: experiments support their own stated scope; no synthetic event is mislabeled natural; new cases have not inherited prior answers.

Phase 09 — Independent evaluation and statistical analysis

Freeze submissions, execute final evaluators, conduct available authorized reviews, build the analysis dataset and estimate effects/uncertainty. Verify headline results from raw records and preserve adverse findings.

Gate: scientific scoring is not mere self-report; uncertainty respects task dependence; exclusions and missingness are visible; tables and plots regenerate.

Phase 10 — Manuscript and visual evidence appendix

Write the complete article from the verified results. Include method algorithms, authentic screenshots with provenance, quantitative figures, tables, failure analysis and limitations. Finalize the title and abstract from what actually happened.

Gate: every central claim is supported; screenshots are correctly classified; prose matches evidence and scope; bibliography is verified.

Phase 11 — Adversarial scientific and systems review

Run distinct reviews of experimental validity, scientific task correctness, evaluator leakage, systems isolation, cost accounting and manuscript claims. Record the actual identities and dependence of reviewers. Resolve objections with evidence and rerun affected checks where permitted.

Gate: critical unresolved problems are corrected or the associated claims removed; review is not falsely labeled human or independent.

Phase 12 — Reproduction, final build and delivery

Replay the analysis in a clean documented environment; verify all deliverables; compile and inspect the PDF; package permitted artifacts and exact commands. Produce final status and limitations.

Gate: actual research paper, live experiment evidence, authentic captures, generated results and a functioning replay package are delivered. A proposal or mock-only demonstration does not satisfy this gate.

At every phase append to reports/phase_status.md: phase ID, timestamps, exact work performed, input versions, outputs, checks executed, failures, costs, interventions, unresolved scientific issues and next eligible action. An issue in one branch does not justify abandoning independent feasible work.

# 23. Final acceptance rubric

If the authorized environment cannot meet one criterion, report the exact blocked dimension and deliver the strongest honest partial package. Do not declare full completion. Do not hide a failed hypothesis: the method's limitations may be the most useful scientific result.

# 24. Final deliverables

Create names consistent with the verified UQO series, including equivalents of:

• WP29_Autonomous_Research_Claude_Code_Boucher.pdf
• full LaTeX source, bibliography and appendix;
• WP29_Method_and_Implementation.zip;
• frozen protocols, treatment prompts, task cards and assignment manifests;
• live-run registry and authorized traces;
• evaluator specification, validation and scoring outputs;
• WP29_Experiment_Results.csv and machine-readable detailed results;
• generated vector figures and LaTeX tables with producing scripts;
• WP29_Visual_Evidence_Appendix.pdf and a screenshot manifest;
• a shareable evidence gallery with authentic public-safe images;
• WP29_Reproducibility.md with actually tested replay commands;
• WP29_Limitations_and_Claims.md;
• WP29_Human_Interventions_and_Costs.csv;
• WP29_Final_Review.md;
• a concise research summary explaining the contribution, evidence and limitations.

The final report must state:

1. The research question and precise method contributed.
2. Which experiments were actually run, on which client/model versions.
3. Numbers of assigned tasks, live executions, completed evaluations and failures.
4. Main results with effect sizes, uncertainty and denominators.
5. What evidence supports or fails to support each principal hypothesis.
6. Human preparation/intervention/review and known resource costs.
7. Which screenshots are live native captures, live dashboard captures, archived viewers or replays.
8. What can be reproduced from saved artifacts and what requires fresh provider access.
9. Outstanding scientific limitations and the actual final validation status.
10. Exact paths to all deliverables.

# 25. Start now

Begin with the repository and environment audit, then build the smallest live, measurable experiment. Verify the source mandates and prior research. Implement the method and evaluation boundaries before spending resources on large runs. Carry out the experiments with actual Claude Code instances, preserve their real traces, take authentic screenshots, calculate the results, and write a rigorous English WP29 article.

The objective is not to demonstrate that Claude can produce a persuasive paper about itself. The objective is to establish, through controlled and auditable evidence, what protocol-guided autonomous research can accomplish, when it fails, and what it costs.