1% Author: Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca2\providecommand{\E}{\mathbb{E}}3\section{Experimental design and preregistration}4\label{sec:design}56The design has one purpose: to attribute any difference between a deliberating committee and its alternatives to a named channel, communication, role diversity, evidence partition, inference compute, verification or provider, under a fixed information set and a fixed budget. This section states the estimands, the arms and the contrast each pair identifies, the preregistered hypotheses, the metrics and inference procedures, the sample sizes and budget actually used in this version of the paper, the freeze, and the arms that are pending.78\subsection{Estimands}9\label{subsec:design_estimands}1011For a property $i$ sold at date $t_i$ with log price $p_i$, every arm produces an $\mathcal{I}_{t_i}$-measurable log estimate $\hat y_i$ and, for the arms with intervals, quantiles $q_{i,\tau}$ at $\tau\in\{0.05,0.25,0.50,0.75,0.95\}$. The operational target is the selected market value $\tilde v_i=\E[p_i\mid\mathcal{I}_{t_i},D_i=1]$ defined in Section~\ref{subsec:values}; by Lemma~\ref{lem:proxy} the expected squared distance to the price decomposes into the distance to $\tilde v_i$ and the irreducible transaction noise $\Var(p\mid\mathcal{I}_t,D=1)$, so that a ranking of arms by price-prediction error is a ranking by distance to $\tilde v$ and is silent about the structural fundamental value $V^{F}$. The per-record error is $e_i=\hat y_i-p_i$, the log of the ratio of the estimate to the price. The primary estimand is the population mean of $|e_i|$ over arm's-length residential sales in the evaluation window, and the primary contrast between two arms $A$ and $B$ is the mean of the paired difference $|e_i^{B}|-|e_i^{A}|$ over the same properties; a negative value favours $B$. Calibration estimands are the coverage probabilities $\Pr(q_{i,.05}\le P_i\le q_{i,.95})$ and $\Pr(q_{i,.25}\le P_i\le q_{i,.75})$, whose nominal values are 0.90 and 0.50. All estimands are defined on the test partition and, separately, on the 2026 holdout partition; the validation partition serves only to choose.1213\subsection{Arms and what each contrast identifies}14\label{subsec:design_arms}1516Table~\ref{tab:arms} lists the eleven arms. Four are non-agentic baselines fitted on the training partition and tuned on validation only: a hedonic regression of log price with municipality-by-quarter and grid fixed effects (A); a hierarchical hedonic model with municipality and grid random intercepts and a repeat-sales adjustment when a prior sale exists (B); a gradient-boosted model on the feature matrix of Section~\ref{subsec:data_infoset} (C); and an automated valuation model that adds comparable-based and spatial-temporal lag features to the boosted model and produces conformal intervals (D). Seven are agentic and share the packet of Section~\ref{sec:architecture}: the single generalist (E), the independent committee (F), the structured debate with one, two or four rounds (G1, G2, G4), the debate with a blinded skeptic and verification (H), the placebo committee (I), the full-view debate (G-fullview) and the cost-matched single agent (J). Two rows of the table describe the same protocol at different rounds because the rounds grid is itself a treatment.1718\begin{table}[tbp]19\centering20\begin{threeparttable}21\caption{Treatment arms and the contrast each pair identifies.}22\label{tab:arms}23\footnotesize24\begin{tabular}{@{}l>{\raggedright\arraybackslash}p{4.8cm}>{\raggedright\arraybackslash}p{9.1cm}@{}}25\toprule26Arm & Description & Contrast and channel identified \\27\midrule28A & Hedonic ridge regression, fixed effects & Reference level of a transparent statistical model \\29B & Hierarchical hedonic with repeat-sales adjustment & Value of the prior sale and of spatial pooling \\30C & Gradient boosting (LightGBM), tuned on validation & Reference level of tabular machine learning \citep{grinsztajn2022why} \\31D & AVM: boosting plus comparable and lag features, conformal intervals & H1 benchmark; the strongest non-agentic arm \\32E & Single generalist, full packet & Reference level of one model on the same evidence \\33F & Twelve roles, independent, aggregated & F vs E: ensembling and role diversity without communication \\34G1, G2, G4 & Structured debate, 1, 2 or 4 rounds & G vs F: communication at fixed roles and views; rounds 0/1/2/4: diminishing returns, convergence, anchoring \\35H & Debate with blinded skeptic and deterministic verification & H vs G2: verification and a blinded reviewer \\36I & Placebo committee: generic prompts, same views, same mechanics & G2 vs I: prompt--evidence complementarity at a fixed partition \\37G-fullview & Default prompts, full packet for every role & G2 vs G-fullview: value of the evidence partition \\38J & Generalist sampled twelve times plus self-refinement & G2 vs J: communication versus compute at matched tokens; F vs J: twelve roles versus one model resampled \\39\bottomrule40\end{tabular}41\begin{tablenotes}[flushleft]42\footnotesize43\item \textit{Notes.} Arm letters follow the preregistration and are used throughout the tables. All agentic arms receive the same evidence packet under the same information cutoff; the mid-tier model pair is \texttt{claude-sonnet-5-5} and \texttt{gpt-6.1-sol}, assigned to roles by a seeded balanced permutation in the mixed committee. Each contrast is paired on the same properties. Source: \texttt{configs/treatments.yaml}.44\end{tablenotes}45\end{threeparttable}46\end{table}4748The contrasts are read as follows. F against E isolates what twelve differently instructed and differently informed agents add over one agent given everything, without any communication; under Proposition~\ref{prop:exchangeable} the gain is bounded by the common component of their errors. G against F isolates communication, because the round-0 estimates of G are, in distribution, those of F, and Proposition~\ref{prop:degroot} states that no pool of later rounds can beat the best pool of round~0 unless evidence is exchanged. G against J separates communication from compute: J spends a comparable token budget on repeated reasoning of one model with one view, so that by equation~\eqref{eq:evidencemse} the contrast is positive only if provider diversification and evidence exchange outweigh the concentration of evidence weight that a partition induces; the paper does not treat test-time compute as a nuisance but as a rival explanation that must be matched \citep{snell2024scaling}. G against I holds the views, the mechanics and the budget fixed and changes only the mandates, so that by Proposition~\ref{prop:placebo} a non-zero contrast identifies complementarity between the prompt and the evidence, not the value of the partition; G against G-fullview holds the mandates fixed and removes the partition, so the two secondary contrasts together separate the two components of role diversity. H against G isolates the effect of flagging fabricated citations and out-of-band adjustments and of a reviewer who has not read the discussion. The rounds grid identifies the shape of the returns to deliberation and allows the convergence, anchoring and interquartile-range statistics of the transcripts to be plotted against the error. The three committee compositions, run for E, F and G2 on validation, identify provider dependence: the term $\sigma_g^{2}(1-\sum_m (n_m/K)^{2})$ of the theory predicts that the mixed committee has the lowest pooled error if provider-common errors are material, and no difference if they are not.4950\subsection{Preregistered hypotheses}51\label{subsec:design_hypotheses}5253Six hypotheses were written before any test-set evaluation and are reproduced from the preregistration document. H1 states that G2, the two-round debate of twelve roles in the mixed committee, has a lower mean absolute log error than the AVM (D) on the test subsample. H2 states that G2 has a lower mean absolute log error than F on the same properties. H3 states that G2 has a lower mean absolute log error than J. H4 states that G2 has a lower mean absolute log error and a lower 95th percentile of absolute log error than I. H5 states that the 90 percent interval of G2 under median-of-quantiles aggregation has a coverage closer to 0.90 than that of E and of F. H6 is descriptive: at least ninety percent of the evidence codes cited by G2 agents exist in their packets, and the adjusted-comparable consistency check passes for at least eighty percent of turns. The primary family comprises the four mean-absolute-log-error contrasts of H1 to H4; the family-wise error rate is controlled at five percent by the step-down procedure of \citet{holm1979simple}, and all tests are two-sided. The P95 component of H4, H5, the rounds grid, H against G2, G2 against G-fullview, the provider committees, the stratified errors, the spatial holdout, the cost frontier and the repeat-sales triangulation are secondary and are reported with confidence intervals and without claims of significance. The preregistration also states what would falsify the claim that deliberation adds information: H2 and H3 both not rejected, with confidence intervals excluding a two percent relative improvement in mean absolute log error. The mapping between each hypothesis and the propositions of Section~\ref{sec:theory} is given in Table~\ref{tab:hypmap}.5455\subsection{Metrics}56\label{subsec:design_metrics}5758The evaluation protocol of the repository fixes the definitions; we summarise them. The primary metric is the mean absolute log error $\frac{1}{n}\sum_i|e_i|$. Secondary accuracy metrics are the mean absolute error in CAD, the median absolute percentage error, the root mean squared log error, the 90th and 95th empirical percentiles of $|e_i|$, the mean log error as a bias measure, and the severe-mispricing rate, the share of properties with $|e_i|>\log 1.25$, reported separately as overvaluation and undervaluation in percent terms as a lender would read them. Interval metrics are the raw coverage of the 50 and 90 percent intervals that the five quantiles bracket directly; the coverage of 80 and 95 percent intervals, which the quantiles do not bracket and which are obtained by fitting a normal distribution in log space to the five quantiles on the probit grid, and are labelled as interpolated wherever they appear; the mean log width of the 90 and 50 percent intervals; the pinball loss averaged over the five quantiles \citep{koenker1978regression}; and a lower-biased continuous ranked probability score on the five-point grid \citep{gneiting2007strictly}. A probability-integral-transform histogram and a reliability diagram per quantile complete the calibration assessment, and a split-conformal rescaling of the intervals, with factors estimated on validation only, is reported as a remedy rather than as a result \citep{romano2019conformalized}. Each agentic arm also reports its mean estimated cost in dollars per property, its mean wall-clock latency and its mean token count from the ledger. Stratified results are reported by region group, urban class, property class, price tercile, decade of construction and local transaction density, and any cell with fewer than thirty properties is suppressed.5960\subsection{Inference}61\label{subsec:design_inference}6263All contrasts are paired on the same properties. The sampling unit for inference is the block defined by municipality and calendar month of the sale, which absorbs the dependence between sales in the same local market at the same time; a paired block bootstrap resamples blocks with replacement, 2,000 times with a fixed seed, applying the same block draw to both arms, and reports the percentile 95 percent interval of the contrast and a bootstrap $p$-value from the recentred distribution \citep{cameron2008bootstrap}. Holm's adjustment is applied within the primary family. As a complement, a Diebold--Mariano statistic on the loss differential ordered by sale date, with a Newey--West long-run variance and the small-sample correction of \citet{harvey1997testing}, is reported for the same pairs \citep{diebold1995comparing,newey1987simple}; its stationarity assumption is not guaranteed in a cross-section ordered by date, and the block bootstrap remains the primary inference tool. Effect sizes are reported as relative changes in the primary metric, so that a contrast can be practically negligible and statistically precise, or the reverse, and the text says which.6465\subsection{Budget, sample sizes and the nested design}66\label{subsec:design_budget}6768The preregistered subsamples are 240 validation, 600 test and 300 holdout properties. The experiments of this version of the paper are run under a self-imposed envelope of roughly one thousand US dollars at list prices and use nested prefixes of those subsamples. A pilot of twelve validation properties with E, F and G2 in the two single-provider committees, capped at sixty dollars, served to catch prompt errors and to calibrate the cost of a turn; the live sanity run measured a single-agent turn at the mid tier at about 5,500 input and 1,000 output tokens, which prices arm~E at roughly two cents per property, arm~F at about a quarter and the two-round debate at between 1.2 and 1.4 dollars. The validation tier uses forty properties for E, F and G2 in all three committees, for G1, H, I, G-fullview and J in the mixed committee, and twenty properties for G4, capped at five hundred dollars; it is the only tier on which anything is chosen, and it supplies the aggregation weights and the conformal factors. The test tier uses the first eighty test properties at the mid tier for the six main arms E, F, G2, H, I and J in the mixed committee, capped at five hundred dollars, and the first three hundred test properties at the small tier for E, F and G2, capped at one hundred and twenty dollars, which provides power for the central contrasts and a second point on the cost frontier. The holdout tier uses the first sixty 2026 properties for E, F and G2, capped at one hundred and twenty dollars. These sizes are those of the budget configuration of the repository; with eighty properties the primary contrasts are powered to detect only large differences, and the paper reports the width of the intervals rather than claiming precision it does not have.6970Two properties of the runner make the design extensible without compromising it. Every run is resumable, so that a run of size $n$ can be continued to $n'>n$ by valuing the additional properties only; and because the subsample order is fixed by the seed and the runner takes the first $n$ properties, the eighty-property test run is a prefix of the six-hundred-property preregistered sample, not a different sample. Extending the test tier to two hundred or to the full six hundred properties at the mid tier for the six main arms is estimated at roughly 1,100 and 3,300 dollars respectively at list prices before caching. A spend cap set in the environment stops any run when the ledger exceeds it, writes what exists and records the event in the manifest, so that a truncated run is never mistaken for a complete one.7172\subsection{Freeze procedure}73\label{subsec:design_freeze}7475Before the first test-set run, the freeze command records in a versioned file the hash of every role prompt, including the placebo prompts, of the four configuration files, of the schema module and of the agents, debate and provider code, together with the model identifiers of the cost-matched pairs, the package version, the Python version and the timestamp. The preregistration document is frozen at the same moment. The check command compares the working tree to the frozen hashes and lists every deviation; the paper reports any such deviation, with its reason, in the limitations section. No prompt, role, round count, model identifier, effort setting, packet construction rule or aggregation weight was changed after the freeze on the basis of a test-set number; the validation tier is where every such choice was made, and the test and holdout tiers were each evaluated once.7677\subsection{Pending arms and the prospective set}78\label{subsec:design_pending}7980A human-appraiser arm, in which licensed or trained appraisers would value the same packets under the same cutoff with blinded prices, randomised case order and recorded confidence intervals, is part of the research program but has not been performed: it requires a research-ethics certificate from the Université du Québec en Outaouais, which has not yet been obtained, and no human benchmark is reported or simulated in this paper. The arm is marked pending wherever a table could have contained it. A prospective evaluation set is defined by sales registered in the live database after the freeze timestamp, whose identifiers are absent from the snapshot and which therefore could not have influenced any choice; it is fetched by a dedicated script, valued under the frozen protocol, and reported as the final robustness check, together with the descriptive comparison of these sales with their listing history where one exists.81