# Architecture (WP 26) ## 1. Layers and contracts ``` data/raw (immutable snapshot) scripts/01_build_analytic.py └─► data/processed/analytic.parquet ──────────► scripts/02_splits.py ──► data/processed/splits.parquet (tx_id, split) ANALYTIC_COLUMNS (src/agentic_valuation/data/analytic_schema.py) — identical for real and synthetic data │ ├─► src/agentic_valuation/features/ as-of features, comparable retrieval (strictly < valuation date) │ build_feature_matrix(analytic, splits, as_of=...) → X (polars), y = log(price) │ retrieve_comparables(analytic, tx_id, cutoff) → list[Comparable] │ market_context(analytic, tx, cutoff) → MarketContext │ geo_context(tx, external) → GeoContext | None ├─► src/agentic_valuation/baselines/ A hedonic, B hierarchical/repeat-sales, C GBM, D AVM(+intervals) │ fit(train, valid) / predict(X) → Prediction(point, q05..q95) ├─► src/agentic_valuation/agents/ packets.py (EvidencePacket builder + role views), roles.py (12+8 role prompts), │ runner.py (one agent turn → AgentTurn via providers.get_provider) ├─► src/agentic_valuation/debate/ protocols.py (F independent, G structured, H skeptic+verification, I placebo, J cost-matched), │ aggregation.py (mean, median, trimmed, inverse-variance, stacking, robust Bayes), stopping.py ├─► src/agentic_valuation/experiments/ run_arm.py (arm × committee × subsample → results/runs//predictions.parquet + transcripts/), │ sampling.py (stratified subsample), freeze.py (protocol hash) └─► src/agentic_valuation/evaluation/ metrics.py, calibration.py, stratify.py, inference.py (clustered bootstrap, DM), faithfulness.py → results/tables/*.csv|tex, figures/*.pdf → paper/tables, paper/figures ``` **Prediction record** (every arm writes the same parquet): `tx_id, split, arm, committee, aggregation, point, q05, q25, q50, q75, q95, cost_usd, latency_s, tokens, run_id`. Baselines write `committee="none"`, `aggregation="model"`. ## 2. Information set and leakage rules (apply to every arm) * Valuation date = sale date; information cutoff = valuation date − 1 day (sale not yet observed). * Comparables: sales with `date < cutoff` only; the subject's own parcel is excluded from its comparables, **except** that a strictly earlier sale of the same parcel is exposed separately as `prior_sale_*` (repeat-sales information, equal for all arms). * Market context is computed from sales with `date ≤ cutoff` only (no `market_index` table from the platform: it is built with later data). * Roll values: the roll in force at the sale date (`assessment_at_sale`, WP 22). The roll's reference date (≈ 18 months before entry in force) is stated to agents. * Model training uses only the `train` partition; hyper-parameters tuned on `validation` only; `test` evaluated once after the freeze; `holdout` (2026) evaluated once at the very end; `prospective` fetched after the freeze. * Group structure: parcel_id for repeated sales; `tests/test_leakage.py` asserts no parcel appears in both train and test *with the later sale in train*, and that no comparable post-dates its subject. ## 3. Agents * 12 default roles + 8 optional (`agents/roles.py`), each a system prompt with `ROLE_ID: ` and a declared **least-privilege view** of the packet (e.g. the building-condition specialist sees only subject attributes and roll data, and must report that no inspection is available). * One turn = one structured call returning `AgentEstimate` (estimate/revision) or `CritiqueSet` (critique). Schema validation with ≤ 2 corrective retries; failures are logged, never silently filled. * Evidence ids: `P`, `C01..C12`, `M1`, `G1`. Faithfulness audit checks (i) cited ids exist, (ii) adjusted comparable prices are within a plausible band of the raw price, (iii) the point estimate lies in the convex hull of adjusted comparables ± 25 % or the deviation is justified in `key_assumptions`. * Providers: `claude-sonnet-5-5` ↔ `gpt-6.1-sol` (cost-matched mid tier, $2/$10 per M tokens); small tier `claude-haiku-5-5` ↔ `gpt-6-luna` ($0.10/$0.50). Committees: all-Anthropic, all-OpenAI, mixed (role→provider assignment randomised per property with a seeded RNG, recorded). * No sampling parameters are sent (current Anthropic models reject them); run-to-run variation is measured by repeating 10 % of properties. ## 4. Debate protocol (G) Round 0: independent estimates from role views. Round r ≥ 1: every agent sees the anonymised table of current estimates/quantiles (`committee_median` included), the one-paragraph `reasoning_summary` of each, and any evidence ids others cited that it did not see (evidence exchange within its permitted view only — the least-privilege view is *expanded* only by ids, never by the subject's price). It returns a `CritiqueSet` then a revised `AgentEstimate`. Stop after the pre-specified number of rounds or when the inter-quartile range of point estimates changes by < 1 % between rounds (recorded, never used to pick the best round). H adds a blinded skeptic (sees only the final estimates and the packet, never the discussion) and a deterministic evidence-verification tool that flags cited ids absent from the packet or adjustments outside ±40 % of the raw comparable price; flagged agents must revise. I keeps the mechanics of G but permutes role prompts across agents (placebo roles). J: single generalist sampled k times (k matched to the token count of F) + a 2-step self-refine chain, cost-matched to G2. ## 5. Aggregation Rules on the final point estimates (and quantiles for intervals): mean, median, 20 % trimmed mean, inverse-variance (weights from each role's validation-set error variance), stacking (non-negative least squares on validation), robust Bayes (Student-t likelihood, scale from validation). Weights estimated on `validation`, frozen, applied to `test`/`holdout`. ## 6. Reproducibility `make all` runs: tests → analytic layer → splits → features → baselines → mock agent smoke → evaluation → tables/figures → paper. Live-API runs are explicit (`av run-arm ... --live`) and write to `results/runs//` with the ledger, model ids, prompt hashes and the git commit of the code. Hosted LLM outputs are not deterministic; transcripts are archived so every number in the paper traces to stored artefacts.