# ⚖️ Does the Sales Comparison Approach Actually Work?

UQO Working Paper No. 16 — A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation — Evidence from 391,000 Single-Family Transactions in Quebec, 2021–2026

TL;DR — The sales comparison approach rests on professional convention: three comparables, a short window, a small radius, an adjustment grid, a reconciliation rule. We backtest 2,256 configurations of the approach, point in time, on 189,354 masked target sales drawn from 390,996 Quebec single-family transactions matched to the assessment roll (configurations chosen on 2024, evaluated on 2025–2026). Adjustments dominate (median absolute error of the five nearest sales: 18.1% time-only → 13.0% with a hedonic grid); similar beats near (selecting physically similar sales gives the same gain with no adjustment); the optimal number of comparables is ~20–30, not 3; windows should be wide, radius barely matters, weighting and reconciliation matter little. The best transparent rule reaches 11.1% out of sample versus 10–11% for a quarterly-retrained gradient-boosting model: the cost of transparency is small, and accuracy is governed by submarket liquidity.

# 📌 Status

# 🎯 What this paper does

For every target sale the price is masked and the approach is applied with only the information available on the valuation date: candidate comparables are selected (nearest / most similar / compromise, k ∈ {1 … 50}, radius d, window T), adjusted (none / time only / stylized conventional grid / hedonic grid estimated quarterly / assessment ratio), weighted and reconciled (mean, median, weighted). Error surfaces Error(k, d, T) are mapped, the best configuration is chosen on the 2024 validation year and evaluated out of sample on 2025–2026 against hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks. Accuracy is then decomposed by submarket liquidity, region, price stratum and property type.

# 🏆 Headline results

# 🗂 Repository layout

sales-comparison-validation/
├── pipeline/                 # 01_dataset → 02_index → 03_models → 04_backtest (+04b_gather) → 05_analyze → 06_report
├── run_all.sh                # full chain on the compute node (CV_VERSION=YYYY-MM-DD)
├── run_distributed.sh        # 15-shard backtest across 5 MacLustr nodes, gather on M2U64
├── make_pdf.sh               # pull out/<version>/ + docs/ from M2U64, compile docs/RAPPORT.pdf (French report)
├── docs/                     # PROTOCOLE.md (protocol), RAPPORT.md/.pdf (French results report), README-fr.md
├── ref/                      # MUN.csv — municipality → MRC → region (MAMH)
├── out/<version>/            # sales.parquet, index_asof.npy, models_*.json, est_E*.npy, tables/, fig/, results.json — not committed
├── data/                     # vraiprix-<date>.db snapshot (node only) — not committed
└── paper/                    # main.tex + sections/ + tables/ (generated) + figures/ → main.pdf

# 🚀 Quick start

rsync -a --exclude out --exclude data --exclude .git ./ M2U64:comparables-validation/
ssh M2U64 'cd ~/comparables-validation && CV_VERSION=2026-10-10 ./run_all.sh'   # ≈ 30 min (or ./run_distributed.sh)
./make_pdf.sh                                                                     # pull out/ + docs/, build RAPPORT.pdf
cd paper && python3 make_paper_tables.py && make                                  # tables + numbers.tex macros → main.pdf

Variables: CV_VERSION (output folder), CV_SAMPLE=3000 (target subsample for a dry run), CV_PROCS (processes, 20). Environment on M2U64: ~/uqo-eval-terrains/env (Python 3.14, pandas, numpy, scipy, lightgbm, matplotlib).

# 🗃 Data

Snapshot of the Vrai Prix production database (2026-10-10): 766,596 residential transactions and 3.75 M assessment-roll units. Single-family sales matched to the roll, cleaned by logged rules R0–R11 (out/<v>/exclusions.csv): 456,950 → 390,996 sales (2021-01-05 → 2026-09-29), split train 201,642 / validation 64,795 (2024) / test 124,559 (2025–2026). Median price $400,000. All numbers in the paper are macros in paper/tables/numbers.tex, regenerated by paper/make_paper_tables.py from out/<version>/.

# 🔬 Methodology

1. Point-in-time index and quarterly hedonic grids (02_index.py, 03_models.py) — nothing after the valuation date is used.
2. Backtest of 2,256 configurations × benchmarks on 189,354 targets (04_backtest.py, sharded; 04b_gather.py).
3. Selection on 2024, evaluation on 2025–2026; error surfaces, marginal effects of each rule, conditional rules by stratum (05_analyze.py).
4. Decomposition by liquidity, region, price, type; stability over time; prediction intervals (06_report.py, paper/make_paper_tables.py).

# 🖼 Figures & tables inventory

Tables (paper/tables/*.tex, 28 generated): descriptives, exclusions, selection, adjustments, grid, weighting, surfaces, competition, density, regions, strata, stability, intervals, conditional rules, hypotheses H2–H5.

# ⚠️ Limitations

• Adjustments simulated on observable characteristics (area, lot, age, storeys, attachment, assessed value); unobserved condition and quality adjustments made by appraisers are not modelled.
• Single-family houses in Quebec only; results for condos/plexes not covered.
• Registry prices may include non-arm's-length transfers surviving the screens.

# 📝 Citation

@techreport{boucher2026salescomparison,
  author      = {Boucher, Simon-Pierre},
  title       = {Does the Sales Comparison Approach Actually Work? A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {16},
  year        = {2026},
  month       = {October}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

• Numbering: issued on 2026-10-10 as "Working Paper No. 11" (number already used by Half a Million Prices, Twenty Models); renumbered 16 on 2026-10-17. The SSRN upload 7626720 predates the renumbering.
• Deviation from Recherche-UQO/HARMONISATION.md §2: the pipeline keeps pipeline/ (not scripts/) and out/ (not results/) because the same tree runs on M2U64 under ~/comparables-validation with these paths.
• French protocol and report in docs/ (PROTOCOLE.md, RAPPORT.md/.pdf); original French README in docs/README-fr.md.
• Per-node copies of the working tree pulled on 2026-10-10 (M*omparables-validation/) were removed from the repository and archived in Recherche-UQO/_archives/sales-comparison-node-copies-20261010/.