# Phase 6 — final evaluation: test, holdout, prospective (2026-10-28)

Runs evaluated once after the freeze: test (mid tier, mixed committee, first 80 of the 600-property subsample): E, F, G2, H, I, J; test small tier (claude-haiku-5-5 / gpt-6-luna, first 300): E, F, G2; holdout 2026 (first 60 of 300): E, F, G2; prospective (1,182 sales registered after the freeze; 40 for LLM arms): E, F, G2 + baselines refitted with identical hyper-parameters; repeat of G2 on 8 test properties. Baselines on the full partitions. Total live spend ≈ 1,079 USD (ledgers; 1,098 incl. sanity and pilot calls), 0 failed properties.

Primary results (test, mean absolute log error): A 0.191, B 0.182, C 0.179, D 0.171 (full partition); E 0.166, F 0.159, G2 0.158, H 0.158, I 0.157, J 0.163 (n = 80). Preregistered paired contrasts (block bootstrap mun × month, Holm): D−G2 −0.005 [−0.033, 0.026]; F−G2 −0.001 [−0.006, 0.005]; J−G2 −0.006 [−0.017, 0.004]; I−G2 0.000 [−0.005, 0.006]; I−G2 on P95 +0.018 [0.000, 0.028]. H1–H4 not rejected. Coverage at nominal 90 %: LLM mid tier 0.86–0.89, AVM 0.90; small tier 0.78–0.81. Holdout 2026: LLM arms 0.186–0.195 with +0.10 log bias; AVM 0.180, bias 0.002. Prospective: LLM 0.179–0.187, AVM 0.202, all biased upward. Run-to-run: mean |Δ log point| 0.007.

Outputs: results/evaluation*/, results/tables*/, paper/tables/, figures/, results/evaluation/repeat_variation.json, results/fundamental/eval_vs_resale*.csv.

Commands: scripts/33_run_tier.py {final,final_small,holdout} --live, scripts/30_run_arm.py … --data-dir data/processed/prospective_bundle, scripts/20_evaluate.py --split {test,holdout,prospective,validation} …, scripts/35_rounds_curve.py, scripts/36_repeat_variation.py, scripts/21_figures.py, scripts/51_eval_vs_resale.py.