# 🏘️ Can Autonomous AI Agents Discover the Fundamental Value of Real Estate?

UQO Working Paper No. 26 — Evidence from Quebec Transactions and Multi-Agent Deliberation

TL;DR — We ask whether communication among heterogeneous autonomous valuation agents adds measurable information, as opposed to adding compute, correlated opinions or persuasive prose. Twelve role-specialised LLM agents (appraisal method, comparables, hedonic, market economist, building condition, geospatial risk, underwriting, buyer, seller, rental/user cost, data auditor, skeptic) each receive a least-privilege, timestamped evidence packet (subject attributes, assessment roll, up to twelve comparables sold before the valuation date, local market state, flood/transit/census context) and return a validated JSON estimate with quantiles, cited evidence ids and adjustments. We compare independent aggregation (F), structured debate (G, 1–4 rounds), debate with a blinded skeptic and deterministic evidence verification (H), placebo roles (I), a token-matched single agent (J), a single LLM (E) and strong tuned baselines (hedonic A, hierarchical/repeat-sales B, gradient boosting C, AVM with conformal intervals D) on 631,866 registered Quebec sales (2021–2026), with chronological, spatial and prospective holdouts and a preregistered contrast family. On the test subsample the committees reach a mean absolute log error of 0.157–0.159 against 0.166 for one LLM and 0.171 for the AVM, but every preregistered paired contrast includes zero: debate does not beat independent aggregation (Δ = −0.001, 95 % CI [−0.006, 0.005]), placebo roles do as well as specialised roles, and a token-matched single agent is within 0.006. Rounds halve the dispersion of the committee without improving accuracy (anchoring index 0.7–0.8), exactly as the information-aggregation theory of Section 3 predicts. LLM intervals are 30 % narrower than the AVM's and under-cover (0.86–0.89 at nominal 0.90; 0.78–0.81 for the small-model tier), and in 2026 the LLM arms over-value by 10 log points while the AVM is unbiased. A twelve-agent debate costs about $1.50 and 5.6 minutes per property; the AVM costs nothing at the margin.

# 📌 Status

# 🎯 What this paper does

1. Formalises valuation as decentralised information aggregation (Section 3): equal-weight pools of correlated signals have an MSE floor that no number of agents removes; opinion-only revision toward the committee median contracts dispersion by $(1-\lambda)^r$ without reducing error; evidence exchange helps only for evidence outside the common view; permuting role prompts with fixed evidence views is distributionally vacuous; no linear deliberation beats the optimal round-0 pool. Eleven propositions with proofs (Appendix), each mapped to a preregistered contrast.
2. Builds a leakage-defended, as-of-date pipeline on the Vrai Prix registered-sales snapshot joined to the assessment roll (nine leakage checks, two development leaks caught and documented), with chronological splits, a spatial holdout, stratified LLM subsamples, and a prospective set of 1,182 sales registered after the freeze.
3. Implements the multi-agent system (12 default + 8 optional roles, least-privilege packets, JSON contracts, Anthropic and OpenAI adapters with structured outputs, cost-matched model pairs, mixed-provider committees, six aggregation rules, cost ledger, freeze) and six deliberation protocols (E–J plus a full-view variant).
4. Evaluates once, after the freeze, with paired block-bootstrap inference (municipality × month blocks), Holm correction, calibration at 50/80/90/95 %, faithfulness audits (cited ids, adjustment consistency, hull checks), run-to-run variation, cost and latency.
5. Triangulates fundamental value with resale-implied values (92,089 sales with a later resale), rental-board rents and a user-cost diagnostic, and keeps forecasting accuracy separate from the identification of fundamentals.
6. Reports failure as rigorously as success: the headline is a precise null on deliberation.

# 🏆 Headline results (test partition, 2025-04 → 2025-12; LLM arms on the first 80 properties of the stratified subsample, mid tier claude-sonnet-5-5 / gpt-6.1-sol, mixed committee, median aggregation)

Preregistered paired contrasts vs G (block bootstrap, Holm): D − G = −0.005 [−0.033, 0.026]; F − G = −0.001 [−0.006, 0.005]; J − G = −0.006 [−0.017, 0.004]; I − G = 0.000 [−0.005, 0.006]; all Holm p = 1. Small tier (claude-haiku-5-5 / gpt-6-luna, n = 300): 0.176–0.182 with 90 % coverage 0.78–0.81. Holdout 2026 (n = 60): LLM arms 0.186–0.195 with +0.10 log bias, AVM 0.180 with bias 0.002. Prospective (post-freeze registrations, n = 40 / 1,182): LLM 0.179–0.187, AVM 0.202. Run-to-run variation of G: mean |Δ log point| 0.007. Total live API spend ≈ 1,079 USD (1,098 USD including sanity and pilot calls).

# 🗂 Repository layout

├── CLAUDE.md                       # research protocol (phases 0–8, arms A–J, standards)
├── README.md · pyproject.toml · Makefile · .env.example
├── configs/                        # data windows & splits, model registry + list prices, treatments, budget tiers
├── src/agentic_valuation/
│   ├── data/        analytic schema, build (real & synthetic), splits, leakage audit, synthetic generator
│   ├── features/    spatial index, as-of comparables, market context, geo layers, feature matrix
│   ├── baselines/   hedonic ridge, hierarchical/repeat-sales, LightGBM/CatBoost/XGBoost, AVM + conformal
│   ├── agents/      20 role prompts + generalist + placebo, evidence packets & role views, turn runner
│   ├── debate/      protocols (F/G/H/I/J), committees, aggregation (6 rules), stopping rule
│   ├── experiments/ run_arm (resumable, ledger, spend cap), freeze, providers check
│   ├── evaluation/  metrics, block bootstrap/Holm/DM, calibration & conformal, strata, faithfulness, LaTeX tables
│   ├── fundamental/ repeat sales index, resale-implied values, rents & user cost
│   ├── theory/      aggregation and welfare simulations
│   └── providers/   Anthropic, OpenAI, mock adapters + cost ledger
├── scripts/                        # 01–05 data, 10 baselines, 15 descriptives, 20–22 evaluation, 30–36 runs/freeze/weights,
│                                   # 40–41 theory/welfare, 50–52 fundamental, 60–62 prospective
├── tests/                          # 143 tests (synthetic data + mock provider; `--real` marker for private data)
├── docs/                           # research plan, data-access audit, preregistration, architecture, data dictionary,
│                                   # evaluation protocol, agents, literature map, fundamental value, prospective set
├── reports/                        # phase_00 … phase_08, limitations register
├── data/{raw,processed,synthetic}  # raw snapshot + external layers (not committed), analytic layers, synthetic set
├── results/{runs,ledger,evaluation*,tables*,fundamental,theory,data}
├── figures/                        # all figures (PDF) used in the paper
└── paper/                          # main.tex, sections/, appendix/, tables/, figures/, references.bib, main.pdf

# 🚀 Quick start

python3 -m pip install -e ".[dev]"
python3 -m pytest tests -q                      # 143 tests, synthetic data, mock provider
make synth                                      # synthetic analytic layer, splits, features
AV_PROVIDER_MODE=mock make smoke                # F and G2 on five synthetic properties, no keys
# real data (needs the Vrai Prix snapshot layers in data/raw/, see data/raw/README.md)
make data features baselines theory fundamental evaluate tables paper
# live LLM runs (keys in .env; estimated spend is capped per tier, see configs/budgets.yaml)
python3 scripts/33_run_tier.py validation --live --dry-run
python3 scripts/33_run_tier.py pilot --live     # then validation → make weights → final / final_small / holdout
av providers-check                              # verifies that configured model ids exist at both vendors

Every LLM run is resumable and nested: raising n in configs/budgets.yaml and re-running a tier extends the same stratified subsample (test 600, holdout 300 drawn once).

# 🗃 Data

Vrai Prix registered sales (public Quebec land register) joined to the municipal assessment roll — the WP 22 processed snapshot of 2026-10-21 (769,995 sales → 631,866 residential house/condo/plex with coordinates and a roll match). External layers: provincial flood maps, transit stops, census dissemination-area statistics, rental-board leases (from the Immo-Ka ancillary databases). Immo-Ka listings were audited and excluded from the arms (coverage starts after the test window; ask prices would be an information asymmetry). Agents never receive street addresses; coordinates are rounded to 3 decimals. The platform's own price index and AVM estimates are excluded from every packet (built with post-cutoff data). Microdata are not redistributed; checksums in data/raw/SHA256SUMS.

# 🔬 Methodology

Chronological splits (train ≤ 2024-06, validation 2024-07→2025-03, test 2025-04→2025-12, holdout 2026), spatial holdout (three regions), prospective set (registered after the freeze). Information set: sale date − 1 day; comparables strictly earlier, same class, 1/3/10 km expanding radius, 540 days; prior sale of the parcel disclosed; roll in force with its reference date; market context from past sales only. Primary endpoint: mean absolute log error; paired block bootstrap (municipality × month, 2,000 draws); Holm over the four primary contrasts; coverage at 50/80/90/95 %; faithfulness audit of citations and adjustments. Aggregation weights estimated on validation only. Providers: official SDKs, structured outputs, adaptive reasoning, no sampling parameters, cost-matched pairs at identical list prices.

# 📊 Figures & tables inventory

Figures (figures/): error distribution by arm, PIT histograms, coverage vs nominal, rounds curve, cost–accuracy frontier, stratified errors, architecture schematic, theory (MSE vs ρ; deliberation rounds), welfare (failure vs uncertainty; systemic loss vs correlation), fundamental (resale gap, price–rent, user cost). Tables (paper/tables/): descriptives, splits, variables, roles, arms, main results (test/validation/holdout/prospective), contrasts, coverage, strata, cost, faithfulness, resale sample, resale comparison, user-cost parameters, plus extra tables generated by scripts/22_extra_tables.py.

# ⚠️ Limitations

LLM arms are evaluated on nested subsamples (80 at the mid tier, 300 at the small tier, 60 holdout, 40 prospective): the primary contrasts rule out improvements larger than about 0.5–3 log points but not smaller ones. Realised prices are noisy, selected proxies of value. No inspection/condition data exist for any arm. Provider pairs are matched on list prices, not on compute. Hosted LLMs are non-deterministic (measured). The human-appraiser arm is pending. See reports/limitations_register.md and Section 13.

# 📖 Citation

@techreport{boucher2026agents,
  title       = {Can Autonomous AI Agents Discover the Fundamental Value of Real Estate? Evidence from Quebec Transactions and Multi-Agent Deliberation},
  author      = {Boucher, Simon-Pierre},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {UQO Working Paper},
  number      = {26},
  year        = {2026},
  month       = {October}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais — simon-pierre.boucher@uqo.ca

# 📝 Notes

• Working Paper No. 25 does not exist in the Recherche-UQO index at the time of writing; number 26 was assigned as instructed.
• The Anthropic API key supplied with the task was rejected by the API; the owner's key from a sibling project was used and should be rotated.
• Transcripts and evidence packets of every live run (≈ 0.5 GB) are archived locally and not committed; predictions, manifests, ledgers and evaluation outputs are.
• results/runs/_* are mock/synthetic/pilot/superseded runs excluded from every evaluation.