1% ============================================================================2% Limitations (WP 26)3% Author: Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca4% Register: reports/limitations_register.md; numbers: results/evaluation*/contrasts.csv, repeat_variation.json5% ============================================================================6\section{Limitations}7\label{sec:limitations}89The limitations of the study are of three kinds: those of the sample on which the agentic arms were run, those of the evidence that any arm could see, and those of the agents and the protocol themselves. We take them in that order, state what each implies for the reading of the results, and record what has been done or could be done about it. The register from which this section is drawn is a living document in the repository.1011The first limitation is statistical power. The preregistered test subsample has six hundred properties; the version of the experiment reported here values the first eighty of them at the mid tier for the six main arms, the first three hundred at the small tier for three arms, the first sixty of the holdout subsample and forty prospective sales, under a self-imposed envelope of roughly one thousand US dollars at list prices. The paired design and the low variance of the within-property differences make the contrasts between agentic arms far more precise than their sample size suggests, but the precision has limits that the intervals state exactly. Against the independent committee the two-round debate's 95 percent interval of $[-0.006, 0.005]$ log points excludes relative improvements larger than about four percent of the mean absolute log error; against the placebo committee the interval of $[-0.005, 0.006]$ excludes about three percent; against the cost-matched single agent the interval of $[-0.017, 0.004]$ excludes about seven percent; and against the automated valuation model, whose errors are not correlated with the committee's in the way the agentic arms' errors are with one another, the interval of $[-0.033, 0.026]$ excludes only improvements of about twenty percent. The preregistration specified that the claim of deliberation adding information would be falsified if the contrasts against the independent committee and the compute-matched agent both failed to reject with intervals excluding a two percent relative improvement; the first interval reaches four percent and the second seven, so that the strict criterion is not met. What the data establish is a point estimate at zero with that resolution, repeated in the validation, holdout and prospective partitions with the same sign or the opposite one. Because the subsamples are nested, the first eighty properties of the six hundred in a fixed stratified order, the runs can be resumed to the full preregistered size without redrawing; at six hundred properties the interval against the independent committee would shrink by a factor of about 2.7, to roughly plus or minus 0.002 log points, which is the resolution the falsification criterion requires. The stratified results by region, property class and price tercile are reported only for cells with thirty or more properties and, for the agentic arms, exist for few of them; nothing in the paper should be read as a claim about thin markets or any other stratum for those arms.1213The second limitation is the target. All accuracy metrics measure the distance between an estimate and a realised transaction price, which Lemma~\ref{lem:proxy} shows to be the distance to the selected market value plus a common noise term. A price is not a value: it embeds the bargaining outcome, the seller's urgency, the marketing history and the appraisal that financed the mortgage, and it is observed only for properties that sold. The repeat-sales proxy of Section~\ref{sec:fundamental} is the only check on the selected target that the data allow, and it is available for the statistical baselines but not yet for the agentic arms, because the evaluated properties of 2025 and 2026 have had too little time to resell; six of the six hundred test properties and none of the holdout properties carry a usable resale, and the test is deferred rather than reported on that basis. The rent and user-cost diagnostics bound the gap between price and fundamental over the sample period but, for want of an area bridge between leases and sales, cannot close it. The title's question is therefore answered only in part, and the part that concerns fundamental value is answered for the tabular baselines alone.1415The third limitation is the evidence. No arm observes the condition of the dwelling: there is no inspection report, no photograph, no description of finishes or renovations, and the building-condition role is instructed to say so and to keep its interval wide. Every arm is therefore blind to the component of value that an appraiser visiting the property would see, and the comparison between arms is a comparison of reasoning on identical, incomplete evidence, not of valuation as practised. Listing data were audited and excluded by design, because their coverage begins after the test window and because asking prices would be observable for some subjects and not for others; the arms consequently see no asking price, no days on market and no listing text, information that the literature shows to be valuable and that a commercial deployment would have. The mortgage-rate field of the market context is null in this snapshot and the agents are told so. The roll's land and building split is an as-of approximation rescaled from the current roll, and the roll's reference date is inferred from the triennial cycle rather than recorded; both are disclosed to the agents with their status. More generally, every role's view is a subset of the same four sections of one packet, so the union of the views is small and the scope that Proposition~\ref{prop:exchange} leaves for evidence exchange to help is correspondingly small. The null on deliberation is a null for this information set; a richer or more specialised evidence base is the most plausible route to a different result.1617The fourth limitation concerns the providers. The two models of each tier are matched on their published list prices per token, not on the compute they consume, and identical prices are a vendor decision rather than a measurement; the token counts of every call are logged and the cost frontier is reported, but the compute matching of arm~J to arm~F, and of the debate to either, is approximate. Reasoning was left to each vendor's adaptive mode at a medium effort setting and no sampling parameters were sent, because the current models of one vendor reject them; the variation between repeated calls on which arm~J relies is therefore the vendor's default, not a controlled temperature. Hosted models are not deterministic and the paper does not claim bitwise reproducibility of any agentic result: the repeat of eight test properties under the frozen two-round protocol gave a mean absolute difference of 0.007 log points in the committee median, a maximum of 0.019, a correlation of 0.998 between the two runs' absolute errors and every repeated point inside the original 90 percent interval, which is small relative to the contrasts but is measured on eight properties. The models themselves are fixed by identifier at the freeze and will be deprecated by their vendors; the archived transcripts, not the live endpoints, are the reproducible object.1819The fifth limitation is the protocol. The role prompts, the placebo prompt, the packet layout, the anonymised committee table, the median anchor and the revision instruction were each written once, exercised on a pilot of twelve validation properties for the three main arms, and frozen; no sensitivity analysis to prompt wording, to the order of sections in the packet, to the number or identity of comparables shown, or to the choice of the median rather than the mean as the committee statistic was performed, and the paper cannot say how much of any arm's accuracy is attributable to these choices. The twenty-role extension and the shuffled-prompt placebo exist in the code but were not run live. The rounds grid stops at four, and the four-round arm was run on twenty validation properties only. The stopping rule on the interquartile range, the verification thresholds of arm~H and the trimming and robust-pool parameters are reasonable defaults rather than tuned values, and were in any case chosen on validation. Because the single design shares one information set across arms, the paper cannot separate what the agents would do with privileged evidence from what they do with the common packet; that is a feature of the identification strategy and a limit on its external validity.2021The sixth limitation is the human arm. No licensed or trained appraiser valued any packet; the arm requires a research-ethics certificate from the Université du Québec en Outaouais that has not yet been obtained, and no human benchmark is reported, approximated or simulated. The comparison of the committee with professional judgement on identical information, and the evaluation of a human-plus-committee workflow, remain to be done. Two further points belong in the record. The 2026 holdout and the prospective set are each evaluated once, as the protocol requires, but the prospective set covers one week of registrations and forty agentic subjects and is a consistency check rather than an endpoint; and the upward bias of every arm on that set, the tabular baselines included, may reflect the selection of deeds that register quickly rather than a property of the valuations. Finally, the welfare model of Section~\ref{sec:welfare} is illustrative and calibrated only in its loan-to-value cap; it yields comparative statics, not estimates, and the paper makes no causal welfare claim on the strength of the accuracy results.22