SPB Git

spb/wp26_uqo

Public
18.0 KB

# CLAUDE.md — Can Autonomous AI Agents Discover the Fundamental Value of Real Estate?

# Mission and non-negotiable scientific standard

Act as a multidisciplinary research team in real estate economics, appraisal, econometrics, machine learning, causal inference, multi-agent systems, and reproducible software engineering. Build an original, publication-quality research program and working codebase titled Can Autonomous AI Agents Discover the Fundamental Value of Real Estate? Evidence from Quebec Transactions and Multi-Agent Deliberation. The study must test, rather than assume, whether structured deliberation among autonomous AI agents improves valuation accuracy, uncertainty calibration, explanation faithfulness, and decision usefulness relative to strong statistical, machine-learning, single-LLM, and human baselines.

Never invent data, results, citations, human assessments, API capabilities, or model performance. Clearly distinguish a transaction price, an appraisal estimate, an investment value, and an unobservable latent fundamental value. Transaction prices are noisy proxies, not ground-truth fundamental values. State identification assumptions and develop multiple operational definitions of value. A negative finding is publishable if the design is rigorous.

# Research questions and hypotheses

RQ1: Do specialized multi-agent systems outperform single LLMs, hedonic models, repeat-sales approaches, gradient boosting, and conventional AVMs on genuinely unseen Quebec transactions? RQ2: Is any improvement due to deliberation, complementary information, greater inference compute, ensembling, or privileged access to data? RQ3: Does role diversity improve calibration and reduce large valuation errors, including in thin markets and under distribution shift? RQ4: Can agents provide faithful, auditable explanations grounded in property attributes, comparable sales, market trends, and uncertainty? RQ5: Does performance vary across regions, price tiers, building types, market cycles, and transaction liquidity? RQ6: Are agentic valuations economically useful for lending and appraisal after accounting for inference cost, latency, human review, and fairness?

Pre-register primary endpoints and hypotheses before running final test evaluations. Primary outcome: out-of-time absolute log-price error on arm's-length residential transactions. Co-primary or secondary: MAE in CAD, MdAPE, RMSE log, tail error (P90/P95), interval coverage, interval width, CRPS or pinball loss, cost and latency. Specify all definitions precisely.

Use authorized, documented access to Vrai-Prix actual Quebec sales transactions and Immo-Ka listing/property information where useful. These are potential sources, not automatically accessible. Begin with a data-access audit; inspect available schemas, endpoints, exports, documentation, usage rights, update frequency, geographic scope, and join keys. Ask for access instructions and credentials only when required; use environment variables and a local secrets manager, never put secrets in source control or manuscript. Do not bypass authentication, scrape restricted endpoints, or infer consent. Do not send personal identifiers, private listing remarks, occupant information, or confidential records to external LLM APIs without authorization and suitable safeguards.

Candidate variables: sale price, deed/closing date, listing date and ask price, location at permitted geographic precision, property type, land area, living area, construction year, renovations, bedrooms/bathrooms, condition, quality, energy features, garage, basement, municipal assessment, tax amounts, neighborhood amenities, transit, schools, flood/fire exposure, mortgage rates, regional housing supply, and local sales liquidity. Track missingness and provenance. Only use features observable as of the valuation timestamp; prevent post-sale information and target leakage. Listing data are optional and must not create unfair advantages across arms.

Implement ingestion adapters for CSV/Parquet/SQL and optional authenticated APIs. Store raw snapshots immutable with data dictionaries, versioned transformations, provenance, licenses, and record-level lineage. Deduplicate repeated transfers and listings; link sales/listings probabilistically with confidence and manual validation sample. Identify non-arm's-length, related-party, distressed, and anomalous transfers; analyze exclusion sensitivity. Respect Quebec privacy requirements and organizational research-ethics review where applicable. Publish only de-identified or synthetic examples if redistribution is not authorized.

# Theoretical contribution: valuation as decentralized information aggregation

Define a latent market-consistent value V*{i,t} conditional on an explicit information set and market state. Distinguish a structural fundamental-value construct (discounted expected housing services / net rents and resale proceeds, with user cost, risk, and liquidity) from a transaction-price prediction target. Formulate an information-aggregation model in which agents k receive partially overlapping signals s{ik}, have correlated errors, strategic or behavioral biases, and communicate through a constrained deliberation protocol. Derive testable conditions under which deliberation reduces or increases mean-squared error, particularly with correlated signals, anchoring, common-model dependence, and information cascades. Do not claim general existence/uniqueness theorems without proof. Propose falsifiable propositions about diversity, correlation, communication topology, and aggregation weights.

# Multi-agent architecture

Implement a modular 5–20-role architecture with default 12 agents:

  1. Certified-appraisal-method specialist (sales comparison, cost, income when relevant)
  2. Comparable-sales selection and adjustment specialist
  3. Hedonic/econometric valuation specialist
  4. Local housing-market economist
  5. Building-condition/inspection specialist (only verified observations; no invented inspection)
  6. Geospatial/accessibility and environmental-risk specialist
  7. Mortgage underwriting and collateral-risk specialist
  8. Buyer / household willingness-to-pay analyst
  9. Seller / listing-strategy analyst
  10. Rental-income / user-cost analyst
  11. Data-quality, leakage, and provenance auditor
  12. Independent skeptical reviewer / red-team agent

Add optional roles up to 20 (construction-cost estimator, urban planning/zoning, macro rates, energy efficiency, climate risk, regional expert, uncertainty auditor, fairness auditor). Agents are simulated analytic roles, not licensed Quebec appraisers or real lenders; never misrepresent professional certification.

Each agent receives a role-specific, least-privilege evidence packet, standardized valuation date, explicit information cutoff, comparable pool, permitted tools, and a structured JSON schema: point estimate, distribution/quantiles, assumptions, evidence IDs, adjustments, missing data, confidence rationale, counterarguments, and cost. Enforce machine-readable validation, bounded retries, and citations to internal evidence IDs. Missing inspection observations must remain missing.

# Providers and orchestration

Support Anthropic Claude API and OpenAI API through provider adapters with configurable model IDs (never assume latest model identifiers), structured-output validation, rate-limit handling, retry/backoff, token budgets, logging, and privacy-aware caching. Use official SDKs and documented capabilities. Keep provider configuration in .env.example with ANTHROPIC_API_KEY and OPENAI_API_KEY placeholders. Never log raw credentials. Record model IDs, API parameters, prompt hashes, temperature, tool calls, token usage, cost estimates, and latency. Design runs that compare all-Claude, all-OpenAI, and mixed-provider committees at matched cost and information budgets; randomize provider-to-role assignment where feasible.

Use Python 3.12+, typed Pydantic schemas, a provider-neutral orchestration layer, async concurrency with strict limits, deterministic non-LLM tools, and SQLite/PostgreSQL for experiment metadata. Optional UI: simple Streamlit or React dashboard for valuation dossiers and debates. No live autonomous trading, lending decisions, or real estate offers; research only.

# Deliberation protocols and experimental treatments

Implement and evaluate at least: A. Hedonic OLS / regularized regression (proper spatial/time controls) B. Repeat-sales or hierarchical hedonic where sample supports it C. CatBoost/XGBoost/LightGBM tuned on validation only D. Strong tabular AVM with spatial-temporal features and calibrated uncertainty E. Single LLM, evidence-matched F. Independent LLM agents aggregated without communication G. Structured debate: independent estimates → evidence exchange → critique → revised estimates → aggregation H. Debate with a blinded skeptic and explicit evidence verification I. Random/shuffled-role or placebo deliberation J. Cost-matched single-agent repeated reasoning and cost-matched ensemble

Use the same authorized information cutoff and comparable selection opportunity across treatments. Test arithmetic mean, median, trimmed mean, inverse-variance weighting (estimated out of sample), stacking, and robust Bayesian aggregation. No final-test-informed aggregation weights. Explicitly separate improvements from more tokens, more tools, more models, or more data. Limit debate to pre-specified rounds (e.g. 0, 1, 2, 4) and evaluate diminishing returns, convergence, anchoring, and correlated hallucinations.

# Experimental design and leakage defense

Use a chronological split with train/validation/test and a final untouched holdout; specify dates based on actual coverage. Add spatial holdouts (entire municipalities or regions), rolling-origin evaluation, and distribution-shift stress tests. Comparable retrieval must be strictly as-of-date; no same-property future transfers, post-sale listings, or near-duplicate records leaking across partitions. Group by property identity and assess neighborhood dependence. Build a leakage-audit suite and adversarial tests. If available, retain a separate prospective evaluation set gathered after model/prompt freeze.

Stratify results by Montreal/Quebec City/Outaouais/other regions as data permit, urban vs rural, condominium vs detached vs multiplex, price deciles, transaction density, and property age. Avoid presenting unsupported granular estimates. Compute paired confidence intervals via clustered/block bootstrap (property, municipality, time as justified), Diebold-Mariano-style comparisons only where assumptions hold, and multiple-testing adjustments. Report effect sizes, statistical uncertainty, and practical significance. No p-hacking or test-set prompt tuning.

# Human expert comparison

Where feasible, recruit licensed appraisal professionals and/or trained human appraisers under an approved protocol; do not fabricate a human benchmark. Give identical timestamped information packets, time budgets, and clear instructions. Randomize case order, blind sale prices, log confidence intervals, measure inter-rater reliability, and evaluate human-only, AI-only, and human+AI. Distinguish human appraiser judgement from retrospective transaction price. If human recruitment is unavailable, mark this arm as pending and do not claim it was performed.

# Explainability and calibration

Assess whether explanations cite genuine, timestamp-valid evidence and whether stated adjustments are numerically consistent with the predicted value. Evaluate factuality, provenance precision/recall, counterfactual stability, sensitivity to spurious variables, and human-rated usefulness using blinded rubrics. Do not equate plausible prose with faithful reasoning. Quantify interval coverage at 50%, 80%, 90%, and 95%, conditional coverage by subgroup, sharpness, conformal prediction where appropriate, and calibration under drift. Report severe overvaluation/undervaluation rates relevant to collateral risk.

# Fundamental-value identification extensions

Because realized price is not fundamental value, add triangulation: repeat transactions and subsequent resale (carefully accounting for intervening market movements), rental yields and discounted cash-flow where reliable rents exist, independent professional valuations if legally accessible, and price-to-rent/user-cost diagnostics. Explain selection bias and limits of each proxy. A model can forecast transaction prices well while failing to identify economic fundamentals; keep these conclusions separate.

# Economic welfare and market design

Construct a stylized buyer-seller-lender model in which valuation accuracy and uncertainty affect search, bargaining, credit approval, collateral haircuts, and transaction outcomes. Calibrate transparently where evidence permits; otherwise label simulations as illustrative. Study information asymmetry, bargaining surplus, adverse selection, risk of common-model correlated mispricing, and market concentration in AI valuation providers. Do not imply causal welfare gains from predictive performance alone.

# Software deliverables

Create a reproducible repository:

  • README.md, CLAUDE.md, pyproject.toml, .env.example, LICENSE (only where rights permit)
  • configs/ for data splits, model IDs, treatments, budgets, and seeds
  • src/data/ ingestion, schema validation, joins, as-of joins, leakage checks
  • src/features/ spatial/temporal features, comparable retrieval
  • src/baselines/ hedonic, gradient boosting, AVM, uncertainty
  • src/agents/ role prompts, evidence packets, provider adapters, JSON contracts
  • src/debate/ protocols, critique, aggregation, stopping rules
  • src/evaluation/ metrics, calibration, stratification, clustered inference
  • src/experiments/ reproducible runners, cost tracking, audit trails
  • tests/ unit/integration/leakage/mock-provider tests
  • notebooks/ exploratory analysis (not authoritative pipelines)
  • paper/ complete LaTeX manuscript, BibTeX, figures, tables, appendices
  • reports/ experiment cards, failure analyses, model cards, limitations

Implement mock LLM adapters and a clearly labeled synthetic dataset so the entire pipeline runs without keys or proprietary data. Include CLI commands for ingest, validate, train, run agents, evaluate, generate tables, and build PDF. Pin dependencies, set seeds, capture versions and hardware, and make outputs reproducible. Do not claim deterministic reproducibility for hosted LLMs; store response artifacts securely and measure run-to-run variation.

# Manuscript requirements

Write in polished academic English with a serious, restrained style. Target Real Estate Economics, Journal of Real Estate Finance and Economics, or an appropriate finance/AI journal depending on demonstrated contribution. Include abstract, introduction, literature with verified citations/DOIs, theory and propositions, Quebec institutional setting, data, identification, architecture, pre-registered experiments, main results, ablations, calibration, human comparison (if executed), economic implications, limitations, conclusion, references, and detailed reproducibility appendices. Never fabricate empirical tables: use placeholders clearly marked NOT YET ESTIMATED until generated by scripts. All figures/tables must come from versioned outputs. Produce main.tex, bibliography, compilation script, and publication-ready PDF when toolchain is available.

# Execution phases and acceptance gates

Phase 0: inventory actual files, access rights, API documentation, computing environment, and research constraints; produce an honest gap analysis. Phase 1: literature review and novelty map; identify close prior multi-agent real-estate/AVM papers and avoid unsupported priority claims. Phase 2: preregistration, formal estimands, theory and proof attempts, data dictionary, data-governance plan. Phase 3: build clean as-of-date data pipeline, leakage tests, and strong tuned non-LLM baselines. Phase 4: implement 5-agent minimal prototype, then 12-agent default and optional 20-agent extension; run synthetic smoke tests. Phase 5: run provider-balanced, information-matched, compute-matched randomized ablations on validation; freeze protocol. Phase 6: evaluate final temporal/spatial holdouts once, with clustered uncertainty and cost accounting. Phase 7: human benchmark if authorized, welfare model, robustness, interpretability audits. Phase 8: produce manuscript, supplementary appendix, reproducibility bundle, and limitations register.

After each phase, produce a concise reports/phase_XX.md with completed artifacts, failed checks, decisions, outstanding assumptions, and exact commands. Stop and ask targeted questions only for indispensable access or decisions; otherwise proceed with explicitly documented defaults. Never fabricate completion.

# Immediate first actions

  1. Inspect the repository and report what exists vs what is missing.
  2. Draft research_plan.md, data_access_checklist.md, preregistration.md, and architecture.md.
  3. Search and verify the relevant academic literature if browsing is available; otherwise flag bibliography for verification.
  4. Create runnable synthetic-data and mock-provider baselines before making paid API calls.
  5. Present an API-cost budget with small pilot, medium validation, and final experiment tiers, without inventing vendor prices.
  6. Ask for Vrai-Prix / Immo-Ka schemas or authorized credentials only when the integration stage is reached.
  7. Start implementation, tests, and LaTeX scaffolding; prioritize scientific validity over theatrical agent dialogue.

# Final guiding principle

The central result must isolate whether communication among heterogeneous autonomous valuation agents adds measurable information, rather than merely adding compute, correlated opinions, or persuasive narratives. Report failure as rigorously as success.


# Repository conventions (added by the implementation, 2026-10-27)

  • UQO Working Paper No. 26, folder Recherche-UQO/immobilier/agentic-valuation-quebec, git remote gitsrv:srv/git/wp26_uqo.git.
  • Python package: src/agentic_valuation/ (installed editable: pip install -e .). CLI: av <command> (see README).
  • Secrets only in .env (gitignored). Snapshot provenance in data/raw/README.md. Every result file is produced by a numbered script in scripts/.
Files · main
Menu

Theme