# 🏘️ Can Autonomous AI Agents Discover the Fundamental Value of Real Estate? **UQO Working Paper No. 26** — *Evidence from Quebec Transactions and Multi-Agent Deliberation* [](paper/main.pdf) [](pyproject.toml) [](data/raw/README.md) [](Makefile) [](https://uqo.ca) [](mailto:simon-pierre.boucher@uqo.ca) > **TL;DR** — We ask whether *communication* among heterogeneous autonomous valuation agents adds measurable information, as opposed to adding compute, correlated opinions or persuasive prose. Twelve role-specialised LLM agents (appraisal method, comparables, hedonic, market economist, building condition, geospatial risk, underwriting, buyer, seller, rental/user cost, data auditor, skeptic) each receive a least-privilege, timestamped evidence packet (subject attributes, assessment roll, up to twelve comparables sold before the valuation date, local market state, flood/transit/census context) and return a validated JSON estimate with quantiles, cited evidence ids and adjustments. We compare independent aggregation (F), structured debate (G, 1–4 rounds), debate with a blinded skeptic and deterministic evidence verification (H), placebo roles (I), a token-matched single agent (J), a single LLM (E) and strong tuned baselines (hedonic A, hierarchical/repeat-sales B, gradient boosting C, AVM with conformal intervals D) on 631,866 registered Quebec sales (2021–2026), with chronological, spatial and prospective holdouts and a preregistered contrast family. On the test subsample the committees reach a mean absolute log error of 0.157–0.159 against 0.166 for one LLM and 0.171 for the AVM, but **every preregistered paired contrast includes zero**: debate does not beat independent aggregation (Δ = −0.001, 95 % CI [−0.006, 0.005]), placebo roles do as well as specialised roles, and a token-matched single agent is within 0.006. Rounds halve the dispersion of the committee without improving accuracy (anchoring index 0.7–0.8), exactly as the information-aggregation theory of Section 3 predicts. LLM intervals are 30 % narrower than the AVM's and under-cover (0.86–0.89 at nominal 0.90; 0.78–0.81 for the small-model tier), and in 2026 the LLM arms over-value by 10 log points while the AVM is unbiased. A twelve-agent debate costs about $1.50 and 5.6 minutes per property; the AVM costs nothing at the margin. --- ## 📌 Status | Item | Value | |---|---| | Version | 1.0 (October 2026) | | SSRN | not yet submitted | | Journal | — (candidates: Real Estate Economics, JREFE; or an AI/finance venue) | | Code & data | fully reproducible with the mock provider and a labelled synthetic dataset (`make all`); real microdata (Vrai Prix snapshot) are private; live LLM runs are archived (`results/runs/`, ledgers) and resumable | | Preregistration | `docs/preregistration.md`; protocol freeze `results/freeze.json` (2026-10-28 01:05 UTC, digest `f0e9d6a9…`) | | Human appraiser arm | **pending** (ethics certificate required) — not performed, never simulated | ## 🎯 What this paper does 1. **Formalises valuation as decentralised information aggregation** (Section 3): equal-weight pools of correlated signals have an MSE floor that no number of agents removes; opinion-only revision toward the committee median contracts dispersion by $(1-\lambda)^r$ without reducing error; evidence exchange helps only for evidence outside the common view; permuting role prompts with fixed evidence views is distributionally vacuous; no linear deliberation beats the optimal round-0 pool. Eleven propositions with proofs (Appendix), each mapped to a preregistered contrast. 2. **Builds a leakage-defended, as-of-date pipeline** on the Vrai Prix registered-sales snapshot joined to the assessment roll (nine leakage checks, two development leaks caught and documented), with chronological splits, a spatial holdout, stratified LLM subsamples, and a prospective set of 1,182 sales registered after the freeze. 3. **Implements the multi-agent system** (12 default + 8 optional roles, least-privilege packets, JSON contracts, Anthropic and OpenAI adapters with structured outputs, cost-matched model pairs, mixed-provider committees, six aggregation rules, cost ledger, freeze) and six deliberation protocols (E–J plus a full-view variant). 4. **Evaluates once, after the freeze**, with paired block-bootstrap inference (municipality × month blocks), Holm correction, calibration at 50/80/90/95 %, faithfulness audits (cited ids, adjustment consistency, hull checks), run-to-run variation, cost and latency. 5. **Triangulates fundamental value** with resale-implied values (92,089 sales with a later resale), rental-board rents and a user-cost diagnostic, and keeps forecasting accuracy separate from the identification of fundamentals. 6. **Reports failure as rigorously as success**: the headline is a precise null on deliberation. ## 🏆 Headline results (test partition, 2025-04 → 2025-12; LLM arms on the first 80 properties of the stratified subsample, mid tier `claude-sonnet-5-5` / `gpt-6.1-sol`, mixed committee, median aggregation) | Arm | Mean \|log error\| | MdAPE | P95 \|log error\| | 90 % coverage | Cost / property | |---|---|---|---|---|---| | A hedonic ridge (full test, n = 96,935) | 0.191 | 0.126 | 0.641 | 0.909 | ≈ 0 | | B hierarchical + repeat sales | 0.182 | 0.115 | 0.645 | 0.913 | ≈ 0 | | C LightGBM (attributes + location) | 0.179 | 0.114 | 0.630 | — | ≈ 0 | | D AVM (C + as-of comparables, conformal quantiles) | 0.171 | 0.105 | 0.622 | 0.896 | ≈ 0 | | E single LLM | 0.166 | 0.107 | 0.508 | 0.888 | $0.02 | | F twelve independent roles | 0.159 | 0.091 | 0.499 | 0.888 | $0.23 | | G structured debate, 2 rounds | 0.158 | 0.087 | 0.493 | 0.888 | $1.52 | | H debate + blinded skeptic + verification | 0.158 | 0.092 | 0.493 | 0.875 | $1.35 | | I placebo roles | 0.157 | 0.093 | 0.497 | 0.888 | $1.40 | | J token-matched single agent | 0.163 | 0.118 | 0.497 | 0.862 | $0.26 | Preregistered paired contrasts vs G (block bootstrap, Holm): D − G = −0.005 [−0.033, 0.026]; F − G = −0.001 [−0.006, 0.005]; J − G = −0.006 [−0.017, 0.004]; I − G = 0.000 [−0.005, 0.006]; all Holm p = 1. Small tier (`claude-haiku-5-5` / `gpt-6-luna`, n = 300): 0.176–0.182 with 90 % coverage 0.78–0.81. Holdout 2026 (n = 60): LLM arms 0.186–0.195 with +0.10 log bias, AVM 0.180 with bias 0.002. Prospective (post-freeze registrations, n = 40 / 1,182): LLM 0.179–0.187, AVM 0.202. Run-to-run variation of G: mean |Δ log point| 0.007. Total live API spend ≈ 1,079 USD (1,098 USD including sanity and pilot calls). ## 🗂 Repository layout ``` ├── CLAUDE.md # research protocol (phases 0–8, arms A–J, standards) ├── README.md · pyproject.toml · Makefile · .env.example ├── configs/ # data windows & splits, model registry + list prices, treatments, budget tiers ├── src/agentic_valuation/ │ ├── data/ analytic schema, build (real & synthetic), splits, leakage audit, synthetic generator │ ├── features/ spatial index, as-of comparables, market context, geo layers, feature matrix │ ├── baselines/ hedonic ridge, hierarchical/repeat-sales, LightGBM/CatBoost/XGBoost, AVM + conformal │ ├── agents/ 20 role prompts + generalist + placebo, evidence packets & role views, turn runner │ ├── debate/ protocols (F/G/H/I/J), committees, aggregation (6 rules), stopping rule │ ├── experiments/ run_arm (resumable, ledger, spend cap), freeze, providers check │ ├── evaluation/ metrics, block bootstrap/Holm/DM, calibration & conformal, strata, faithfulness, LaTeX tables │ ├── fundamental/ repeat sales index, resale-implied values, rents & user cost │ ├── theory/ aggregation and welfare simulations │ └── providers/ Anthropic, OpenAI, mock adapters + cost ledger ├── scripts/ # 01–05 data, 10 baselines, 15 descriptives, 20–22 evaluation, 30–36 runs/freeze/weights, │ # 40–41 theory/welfare, 50–52 fundamental, 60–62 prospective ├── tests/ # 143 tests (synthetic data + mock provider; `--real` marker for private data) ├── docs/ # research plan, data-access audit, preregistration, architecture, data dictionary, │ # evaluation protocol, agents, literature map, fundamental value, prospective set ├── reports/ # phase_00 … phase_08, limitations register ├── data/{raw,processed,synthetic} # raw snapshot + external layers (not committed), analytic layers, synthetic set ├── results/{runs,ledger,evaluation*,tables*,fundamental,theory,data} ├── figures/ # all figures (PDF) used in the paper └── paper/ # main.tex, sections/, appendix/, tables/, figures/, references.bib, main.pdf ``` ## 🚀 Quick start ```bash python3 -m pip install -e ".[dev]" python3 -m pytest tests -q # 143 tests, synthetic data, mock provider make synth # synthetic analytic layer, splits, features AV_PROVIDER_MODE=mock make smoke # F and G2 on five synthetic properties, no keys # real data (needs the Vrai Prix snapshot layers in data/raw/, see data/raw/README.md) make data features baselines theory fundamental evaluate tables paper # live LLM runs (keys in .env; estimated spend is capped per tier, see configs/budgets.yaml) python3 scripts/33_run_tier.py validation --live --dry-run python3 scripts/33_run_tier.py pilot --live # then validation → make weights → final / final_small / holdout av providers-check # verifies that configured model ids exist at both vendors ``` Every LLM run is resumable and nested: raising `n` in `configs/budgets.yaml` and re-running a tier extends the same stratified subsample (test 600, holdout 300 drawn once). ## 🗃 Data Vrai Prix registered sales (public Quebec land register) joined to the municipal assessment roll — the WP 22 processed snapshot of 2026-10-21 (769,995 sales → 631,866 residential house/condo/plex with coordinates and a roll match). External layers: provincial flood maps, transit stops, census dissemination-area statistics, rental-board leases (from the Immo-Ka ancillary databases). Immo-Ka listings were audited and excluded from the arms (coverage starts after the test window; ask prices would be an information asymmetry). Agents never receive street addresses; coordinates are rounded to 3 decimals. The platform's own price index and AVM estimates are excluded from every packet (built with post-cutoff data). Microdata are not redistributed; checksums in `data/raw/SHA256SUMS`. ## 🔬 Methodology Chronological splits (train ≤ 2024-06, validation 2024-07→2025-03, test 2025-04→2025-12, holdout 2026), spatial holdout (three regions), prospective set (registered after the freeze). Information set: sale date − 1 day; comparables strictly earlier, same class, 1/3/10 km expanding radius, 540 days; prior sale of the parcel disclosed; roll in force with its reference date; market context from past sales only. Primary endpoint: mean absolute log error; paired block bootstrap (municipality × month, 2,000 draws); Holm over the four primary contrasts; coverage at 50/80/90/95 %; faithfulness audit of citations and adjustments. Aggregation weights estimated on validation only. Providers: official SDKs, structured outputs, adaptive reasoning, no sampling parameters, cost-matched pairs at identical list prices. ## 📊 Figures & tables inventory Figures (`figures/`): error distribution by arm, PIT histograms, coverage vs nominal, rounds curve, cost–accuracy frontier, stratified errors, architecture schematic, theory (MSE vs ρ; deliberation rounds), welfare (failure vs uncertainty; systemic loss vs correlation), fundamental (resale gap, price–rent, user cost). Tables (`paper/tables/`): descriptives, splits, variables, roles, arms, main results (test/validation/holdout/prospective), contrasts, coverage, strata, cost, faithfulness, resale sample, resale comparison, user-cost parameters, plus extra tables generated by `scripts/22_extra_tables.py`. ## ⚠️ Limitations LLM arms are evaluated on nested subsamples (80 at the mid tier, 300 at the small tier, 60 holdout, 40 prospective): the primary contrasts rule out improvements larger than about 0.5–3 log points but not smaller ones. Realised prices are noisy, selected proxies of value. No inspection/condition data exist for any arm. Provider pairs are matched on list prices, not on compute. Hosted LLMs are non-deterministic (measured). The human-appraiser arm is pending. See `reports/limitations_register.md` and Section 13. ## 📖 Citation ```bibtex @techreport{boucher2026agents, title = {Can Autonomous AI Agents Discover the Fundamental Value of Real Estate? Evidence from Quebec Transactions and Multi-Agent Deliberation}, author = {Boucher, Simon-Pierre}, institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives}, type = {UQO Working Paper}, number = {26}, year = {2026}, month = {October} } ``` ## 👤 Author & contact Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais — simon-pierre.boucher@uqo.ca ## 📝 Notes - Working Paper No. 25 does not exist in the Recherche-UQO index at the time of writing; number 26 was assigned as instructed. - The Anthropic API key supplied with the task was rejected by the API; the owner's key from a sibling project was used and should be rotated. - Transcripts and evidence packets of every live run (≈ 0.5 GB) are archived locally and not committed; predictions, manifests, ledgers and evaluation outputs are. - `results/runs/_*` are mock/synthetic/pilot/superseded runs excluded from every evaluation.