SPB Git

spb/wp29_uqo

Public
6.8 KB · 176 lines latex
1% ============================================================================2% UQO Working Paper Template — Economics / Finance3% ============================================================================4\documentclass[12pt,letterpaper]{article}56% --- Encoding & Language ---7\usepackage[utf8]{inputenc}8\usepackage[T1]{fontenc}9\DeclareUnicodeCharacter{2212}{--}10\DeclareUnicodeCharacter{2192}{$\rightarrow$}11\DeclareUnicodeCharacter{2265}{$\geq$}12\DeclareUnicodeCharacter{2264}{$\leq$}13\DeclareUnicodeCharacter{00D7}{$\times$}14\DeclareUnicodeCharacter{2026}{\ldots}15\DeclareUnicodeCharacter{2248}{$\approx$}16\DeclareUnicodeCharacter{03B2}{$\beta$}17\DeclareUnicodeCharacter{03C4}{$\tau$}18\DeclareUnicodeCharacter{0394}{$\Delta$}19\DeclareUnicodeCharacter{03B1}{$\alpha$}20\DeclareUnicodeCharacter{03C1}{$\rho$}21\DeclareUnicodeCharacter{03C3}{$\sigma$}22\DeclareUnicodeCharacter{0398}{$\Theta$}23\DeclareUnicodeCharacter{2208}{$\in$}24\DeclareUnicodeCharacter{2227}{$\wedge$}25\DeclareUnicodeCharacter{00B2}{$^2$}26\DeclareUnicodeCharacter{2713}{yes}27\DeclareUnicodeCharacter{2714}{yes}28\DeclareUnicodeCharacter{2011}{-}2930\usepackage[english]{babel}3132% --- Page Layout ---33\usepackage[34  letterpaper,35  top=1in,36  bottom=1in,37  left=1in,38  right=1in39]{geometry}4041% --- Typography ---42\usepackage{mathptmx}          % Times-like font (text + math)43\usepackage{microtype}         % Improved typographic refinements44\usepackage{setspace}45\onehalfspacing                % 1.5 line spacing4647% --- Math ---48\usepackage{amsmath,amssymb,amsthm}4950% --- Tables ---51\usepackage{booktabs}52\usepackage{tabularx}53\usepackage{multirow}54\usepackage{threeparttable}55\usepackage{siunitx}56\sisetup{57  input-symbols = {()},58  table-format  = -1.3,59  parse-numbers = false60}6162% --- Figures ---63\usepackage{graphicx}64\usepackage{subcaption}65\usepackage{algorithm}66\usepackage{algpseudocode}67\usepackage{longtable}68\usepackage{array}69\usepackage{adjustbox}70\usepackage{fvextra}71\usepackage{float}7273% --- Colors & Links ---74\usepackage[dvipsnames]{xcolor}75\usepackage[76  colorlinks  = true,77  linkcolor   = NavyBlue,78  citecolor   = NavyBlue,79  urlcolor    = NavyBlue,80  breaklinks  = true81]{hyperref}8283% --- Bibliography ---84\usepackage[authoryear,round,semicolon]{natbib}85\bibliographystyle{aer}8687% --- Captions ---88\usepackage[89  font       = small,90  labelfont  = bf,91  labelsep   = period,92  justification = justified,93  singlelinecheck = false94]{caption}9596% --- Headers & Footers ---97\usepackage{fancyhdr}98\pagestyle{fancy}99\fancyhf{}100\renewcommand{\headrulewidth}{0pt}101\fancyfoot[C]{\thepage}102103% --- Section Formatting ---104\usepackage{titlesec}105\titleformat{\section}{\large\bfseries}{\thesection.}{0.5em}{}106\titleformat{\subsection}{\normalsize\bfseries}{\thesubsection.}{0.5em}{}107\titleformat{\subsubsection}{\normalsize\itshape}{\thesubsubsection.}{0.5em}{}108109% --- Appendix Support ---110\usepackage[toc,page]{appendix}111112% --- Misc ---113\usepackage{enumitem}114\setlist{nosep,leftmargin=*}115116% ============================================================================117% METADATA — Edit these fields for each working paper118% ============================================================================119\newcommand{\WPnumber}{29}120\newcommand{\WPtitle}{Protocol-Guided Autonomous Research with Claude Code}121\newcommand{\WPsubtitle}{Scientific Verification, Reproducibility, and the Cost of Delegation: A Controlled Study}122\newcommand{\WPdate}{October 2026}123\newcommand{\WPversion}{1.0}124\newcommand{\WPabstract}{%125  Instruction files that steer coding agents through research tasks range from one sentence to several thousand words of phases, gates and integrity rules, but whether that structure changes what the agent delivers has not been measured with the agent held fixed. We ran \nMainRuns{} live Claude Code sessions (client 2.1.314, model claude-opus-5-5) on \nConfTasks{} bounded scientific tasks in six families, \nRepeats{} fresh sessions per task under each of four regimes: an open instruction; a structured protocol with phases, ledgers, stopping and recovery rules; the protocol plus a controller-enforced executable gate; and the same plus a separate review-and-repair session. Executors ran in bare mode on isolated hosts; deterministic hidden oracles, validated on seeded artefacts, scored every run on a separate host; the design, tasks and analysis were frozen with hashes before the confirmatory runs. Intervention-free verified completion was \pA{} under the open instruction and \pD{} under the full method; the paired task-level difference is \dDA{} (task-cluster bootstrap 95\% interval [\dDAlo, \dDAhi]), the mechanism contrasts are equally null, and critical integrity violations were rare in every arm. The protocol, gate and reviewer bought full compliance with the deliverable contract at two to three times the inference cost per run. The failures that remained in every arm were, on inspection, defects of our oracles rather than of the executors: twelve independent sessions reported an undefined month as undefined and excluded impossible values with a documented ledger, and a rubric-corrected sensitivity analysis leaves the null unchanged. Interruption-and-resumption, stale-input, duplicate-job and missing-fixture experiments, a length-matched control and three held-out end-to-end mini-projects complete the evidence, together with an instrumented account of how this paper was itself produced under a mandate of the same kind. For bounded tasks with known answers, the marginal value of protocol structure lies in auditability, not correctness, and the scarce resource is a trustworthy oracle.%126}\newcommand{\WPkeywords}{autonomous research agents; Claude Code; research protocols; executable verification; reproducibility; agent evaluation; delegation}127\newcommand{\WPjel}{C18, C63, C88, O33}128129% --- Author ---130\newcommand{\WPauthor}{Simon-Pierre Boucher}131\newcommand{\WPaffiliation}{%132  Département des sciences administratives\\133  Université du Québec en Outaouais%134}135\newcommand{\WPemail}{simon-pierre.boucher@uqo.ca}136\newcommand{\WPaddress}{%137  Gatineau -- Pavillon Alexandre-Taché\\138  283, boulevard Alexandre-Taché\\139  Gatineau, Québec, Canada J9A 1L8%140}141142% ============================================================================143% DOCUMENT144% ============================================================================145\input{tables/numbers}146\providecommand{\nRepeats}{3}147\begin{document}148149% --- Title Page ---150\input{sections/titlepage}151152% --- Main Body ---153\input{sections/introduction}154\input{sections/literature}155\input{sections/framework}156\input{sections/corpus}157\input{sections/implementation}158\input{sections/design}159\input{sections/evaluation}160\input{sections/results}161\input{sections/recovery}162\input{sections/transfer}163\input{sections/reflexive}164\input{sections/limitations}165\input{sections/conclusion}166\input{sections/disclosure}167168% --- References ---169\newpage170\bibliography{references}171172% --- Appendix ---173\input{appendices/appendix}174175\end{document}176
Files · 152a55b8fa2830f50fa35e5aed99c0037f713345
Menu

Theme