# Architecture (WP 26)

# 1. Layers and contracts

text

data/raw (immutable snapshot)                     scripts/01_build_analytic.py
  └─► data/processed/analytic.parquet  ──────────► scripts/02_splits.py ──► data/processed/splits.parquet (tx_id, split)
        ANALYTIC_COLUMNS (src/agentic_valuation/data/analytic_schema.py) — identical for real and synthetic data
              │
              ├─► src/agentic_valuation/features/        as-of features, comparable retrieval (strictly < valuation date)
              │       build_feature_matrix(analytic, splits, as_of=...)  → X (polars), y = log(price)
              │       retrieve_comparables(analytic, tx_id, cutoff)      → list[Comparable]
              │       market_context(analytic, tx, cutoff)               → MarketContext
              │       geo_context(tx, external)                          → GeoContext | None
              ├─► src/agentic_valuation/baselines/       A hedonic, B hierarchical/repeat-sales, C GBM, D AVM(+intervals)
              │       fit(train, valid) / predict(X) → Prediction(point, q05..q95)
              ├─► src/agentic_valuation/agents/          packets.py (EvidencePacket builder + role views), roles.py (12+8 role prompts),
              │                                           runner.py (one agent turn → AgentTurn via providers.get_provider)
              ├─► src/agentic_valuation/debate/          protocols.py (F independent, G structured, H skeptic+verification, I placebo, J cost-matched),
              │                                           aggregation.py (mean, median, trimmed, inverse-variance, stacking, robust Bayes), stopping.py
              ├─► src/agentic_valuation/experiments/     run_arm.py (arm × committee × subsample → results/runs/<run_id>/predictions.parquet + transcripts/),
              │                                           sampling.py (stratified subsample), freeze.py (protocol hash)
              └─► src/agentic_valuation/evaluation/      metrics.py, calibration.py, stratify.py, inference.py (clustered bootstrap, DM), faithfulness.py
                                                              → results/tables/*.csv|tex, figures/*.pdf → paper/tables, paper/figures

Prediction record (every arm writes the same parquet): tx_id, split, arm, committee, aggregation, point, q05, q25, q50, q75, q95, cost_usd, latency_s, tokens, run_id. Baselines write committee="none", aggregation="model".

# 2. Information set and leakage rules (apply to every arm)

Valuation date = sale date; information cutoff = valuation date − 1 day (sale not yet observed).
Comparables: sales with date < cutoff only; the subject's own parcel is excluded from its comparables, except that a strictly earlier sale of the same parcel is exposed separately as prior_sale_* (repeat-sales information, equal for all arms).
Market context is computed from sales with date ≤ cutoff only (no market_index table from the platform: it is built with later data).
Roll values: the roll in force at the sale date (assessment_at_sale, WP 22). The roll's reference date (≈ 18 months before entry in force) is stated to agents.
Model training uses only the train partition; hyper-parameters tuned on validation only; test evaluated once after the freeze; holdout (2026) evaluated once at the very end; prospective fetched after the freeze.
Group structure: parcel_id for repeated sales; tests/test_leakage.py asserts no parcel appears in both train and test with the later sale in train, and that no comparable post-dates its subject.

# 3. Agents

12 default roles + 8 optional (agents/roles.py), each a system prompt with ROLE_ID: <id> and a declared least-privilege view of the packet (e.g. the building-condition specialist sees only subject attributes and roll data, and must report that no inspection is available).
One turn = one structured call returning AgentEstimate (estimate/revision) or CritiqueSet (critique). Schema validation with ≤ 2 corrective retries; failures are logged, never silently filled.
Evidence ids: P, C01..C12, M1, G1. Faithfulness audit checks (i) cited ids exist, (ii) adjusted comparable prices are within a plausible band of the raw price, (iii) the point estimate lies in the convex hull of adjusted comparables ± 25 % or the deviation is justified in key_assumptions.
Providers: claude-sonnet-5-5 ↔ gpt-6.1-sol (cost-matched mid tier, $2/$10 per M tokens); small tier claude-haiku-5-5 ↔ gpt-6-luna ($0.10/$0.50). Committees: all-Anthropic, all-OpenAI, mixed (role→provider assignment randomised per property with a seeded RNG, recorded).
No sampling parameters are sent (current Anthropic models reject them); run-to-run variation is measured by repeating 10 % of properties.

# 4. Debate protocol (G)

Round 0: independent estimates from role views. Round r ≥ 1: every agent sees the anonymised table of current estimates/quantiles (committee_median included), the one-paragraph reasoning_summary of each, and any evidence ids others cited that it did not see (evidence exchange within its permitted view only — the least-privilege view is expanded only by ids, never by the subject's price). It returns a CritiqueSet then a revised AgentEstimate. Stop after the pre-specified number of rounds or when the inter-quartile range of point estimates changes by < 1 % between rounds (recorded, never used to pick the best round). H adds a blinded skeptic (sees only the final estimates and the packet, never the discussion) and a deterministic evidence-verification tool that flags cited ids absent from the packet or adjustments outside ±40 % of the raw comparable price; flagged agents must revise. I keeps the mechanics of G but permutes role prompts across agents (placebo roles). J: single generalist sampled k times (k matched to the token count of F) + a 2-step self-refine chain, cost-matched to G2.

# 5. Aggregation

Rules on the final point estimates (and quantiles for intervals): mean, median, 20 % trimmed mean, inverse-variance (weights from each role's validation-set error variance), stacking (non-negative least squares on validation), robust Bayes (Student-t likelihood, scale from validation). Weights estimated on validation, frozen, applied to test/holdout.

# 6. Reproducibility

make all runs: tests → analytic layer → splits → features → baselines → mock agent smoke → evaluation → tables/figures → paper. Live-API runs are explicit (av run-arm ... --live) and write to results/runs/<run_id>/ with the ledger, model ids, prompt hashes and the git commit of the code. Hosted LLM outputs are not deterministic; transcripts are archived so every number in the paper traces to stored artefacts.

spb/wp26_uqo