SPB Git

spb/wp26_uqo

Public
1.5 KB · 10 lines markdown
1# Phase 4 — agents, protocols, smoke tests, pilot (2026-10-27)23Artefacts: `src/agentic_valuation/agents/` (12 default + 8 optional roles, generalist, placebo roles with identical views, full-view variant), `debate/` (independent, structured debate with critique + revision turns, blinded skeptic + deterministic evidence verification, placebo, cost-matched single agent; aggregation rules; IQR early-stopping rule recorded), `experiments/run_arm.py` (resumable runs, manifests, ledger, spend cap), `scripts/33_run_tier.py` (budget tiers), `av` CLI, 131 tests passing (mock provider + synthetic data).45Smoke (mock, synthetic): F, G2, H, I, J, E all produce the 15-column prediction contract; the faithfulness audit consumes the transcripts.67Live sanity (real validation data, 2 properties, both providers): structured outputs validate on the first try; reasoning cites packet ids and performs explicit time/size/land adjustments. Pilot (2 properties × E/F/G2 × all-Anthropic/all-OpenAI): a G2 property costs ≈ $1.46 (528k tokens, 60 turns, ≈ 3 min with 4-way concurrency); F ≈ $0.18–0.26; E ≈ $0.015. Dispersion across roles fell from 178–345k (round 0) to 225–255k (round 2) on the first property while the realised price was 179k — an early illustration that consensus is not accuracy.89Decisions: budget envelope ≈ $1,000 (configs/budgets.yaml): validation n=40 (ablations + weights), test n=80 at the mid tier for six arms, 300 at the small tier for E/F/G2, holdout 60. Pilot runs archived under `results/runs/_pilot_*` (excluded from evaluation).10
Files · b780d9effbc0d688f848faf14e38219f6088143b
Menu

Theme