# ⚖️ Does the Sales Comparison Approach Actually Work? **UQO Working Paper No. 16** — *A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation — Evidence from 391,000 Single-Family Transactions in Quebec, 2021–2026* [-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) [](pipeline/) [](docs/PROTOCOLE.md) [](pipeline/) [](https://uqo.ca) [](mailto:simon-pierre.boucher@uqo.ca) > **TL;DR** — The sales comparison approach rests on professional convention: three comparables, a short window, a > small radius, an adjustment grid, a reconciliation rule. We backtest 2,256 configurations of the approach, point in > time, on 189,354 masked target sales drawn from 390,996 Quebec single-family transactions matched to the assessment > roll (configurations chosen on 2024, evaluated on 2025–2026). **Adjustments dominate** (median absolute error of the > five nearest sales: 18.1% time-only → 13.0% with a hedonic grid); **similar beats near** (selecting physically similar > sales gives the same gain with no adjustment); **the optimal number of comparables is ~20–30, not 3**; windows should > be wide, radius barely matters, weighting and reconciliation matter little. The best transparent rule reaches > **11.1%** out of sample versus 10–11% for a quarterly-retrained gradient-boosting model: the cost of transparency is > small, and accuracy is governed by submarket liquidity. --- ## 📌 Status | Item | Value | |---|---| | Version | 1.0 (October 2026) | | SSRN | abstract **7626720** (PRELIMINARY_UPLOAD, 2026-10-17) — uploaded PDF carries the old cover "Working Paper No. 11"; **re-upload `paper/main.pdf` (No. 16)** | | Journal | — | | Code & data | pipeline in `pipeline/`, run on the MacLustr node M2U64; the Vrai Prix database snapshot is not redistributed | ## 🎯 What this paper does For every target sale the price is masked and the approach is applied with only the information available on the valuation date: candidate comparables are selected (nearest / most similar / compromise, k ∈ {1 … 50}, radius d, window T), adjusted (none / time only / stylized conventional grid / hedonic grid estimated quarterly / assessment ratio), weighted and reconciled (mean, median, weighted). Error surfaces Error(k, d, T) are mapped, the best configuration is chosen on the 2024 validation year and evaluated out of sample on 2025–2026 against hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks. Accuracy is then decomposed by submarket liquidity, region, price stratum and property type. ## 🏆 Headline results | Object | Estimate (test 2025–2026, MdAPE) | |---|---| | 5 nearest sales, time adjustment only (M2) | 18.1% | | 5 nearest sales, hedonic adjustment grid (M7) | 13.0% | | 5 most similar sales, no physical adjustment | ≈ same gain as the grid | | Number of comparables with a grid, k = 1 → 3 → 5 → 10 → 20–30 | 16.5% → 13.8% → 13.0% → 12.4% → minimum | | Best transparent rule chosen in 2024 (M0: compromise, k = 30, ≤ 25 km, 24 months, hedonic grid) | **11.1%** | | Assessment-ratio comparable (M13) | 11.5% | | Quarterly-retrained GBM using assessed value (M11b) / hybrid (M12) | ≈ 10–11% | | Liquidity effect (M7): > 200 sales within 2 km / year vs < 20 | 10.2% vs 25.3% | ## 🗂 Repository layout ``` sales-comparison-validation/ ├── pipeline/ # 01_dataset → 02_index → 03_models → 04_backtest (+04b_gather) → 05_analyze → 06_report ├── run_all.sh # full chain on the compute node (CV_VERSION=YYYY-MM-DD) ├── run_distributed.sh # 15-shard backtest across 5 MacLustr nodes, gather on M2U64 ├── make_pdf.sh # pull out// + docs/ from M2U64, compile docs/RAPPORT.pdf (French report) ├── docs/ # PROTOCOLE.md (protocol), RAPPORT.md/.pdf (French results report), README-fr.md ├── ref/ # MUN.csv — municipality → MRC → region (MAMH) ├── out// # sales.parquet, index_asof.npy, models_*.json, est_E*.npy, tables/, fig/, results.json — not committed ├── data/ # vraiprix-.db snapshot (node only) — not committed └── paper/ # main.tex + sections/ + tables/ (generated) + figures/ → main.pdf ``` ## 🚀 Quick start ```bash rsync -a --exclude out --exclude data --exclude .git ./ M2U64:comparables-validation/ ssh M2U64 'cd ~/comparables-validation && CV_VERSION=2026-10-10 ./run_all.sh' # ≈ 30 min (or ./run_distributed.sh) ./make_pdf.sh # pull out/ + docs/, build RAPPORT.pdf cd paper && python3 make_paper_tables.py && make # tables + numbers.tex macros → main.pdf ``` Variables: `CV_VERSION` (output folder), `CV_SAMPLE=3000` (target subsample for a dry run), `CV_PROCS` (processes, 20). Environment on M2U64: `~/uqo-eval-terrains/env` (Python 3.14, pandas, numpy, scipy, lightgbm, matplotlib). ## 🗃 Data Snapshot of the Vrai Prix production database (2026-10-10): 766,596 residential transactions and 3.75 M assessment-roll units. Single-family sales matched to the roll, cleaned by logged rules R0–R11 (`out//exclusions.csv`): 456,950 → **390,996** sales (2021-01-05 → 2026-09-29), split train 201,642 / validation 64,795 (2024) / test 124,559 (2025–2026). Median price $400,000. All numbers in the paper are macros in `paper/tables/numbers.tex`, regenerated by `paper/make_paper_tables.py` from `out//`. ## 🔬 Methodology 1. **Point-in-time index and quarterly hedonic grids** (`02_index.py`, `03_models.py`) — nothing after the valuation date is used. 2. **Backtest of 2,256 configurations** × benchmarks on 189,354 targets (`04_backtest.py`, sharded; `04b_gather.py`). 3. **Selection on 2024, evaluation on 2025–2026**; error surfaces, marginal effects of each rule, conditional rules by stratum (`05_analyze.py`). 4. **Decomposition** by liquidity, region, price, type; stability over time; prediction intervals (`06_report.py`, `paper/make_paper_tables.py`). ## 🖼 Figures & tables inventory | Figure | Content | |---|---| | `f1_mdape_vs_k` | error vs number of comparables, by adjustment | | `f2_surface_d_T` | error surface over radius and window | | `f3_ajustements` | contribution of adjustment types | | `f4_competition` | comparables vs hedonic / GBM / ratio / hybrid | | `f5_densite`, `f6_k_par_densite` | accuracy and optimal k by submarket liquidity | | `f7_indice` | point-in-time price index | Tables (`paper/tables/*.tex`, 28 generated): descriptives, exclusions, selection, adjustments, grid, weighting, surfaces, competition, density, regions, strata, stability, intervals, conditional rules, hypotheses H2–H5. ## ⚠️ Limitations - Adjustments simulated on observable characteristics (area, lot, age, storeys, attachment, assessed value); unobserved condition and quality adjustments made by appraisers are not modelled. - Single-family houses in Quebec only; results for condos/plexes not covered. - Registry prices may include non-arm's-length transfers surviving the screens. ## 📝 Citation ```bibtex @techreport{boucher2026salescomparison, author = {Boucher, Simon-Pierre}, title = {Does the Sales Comparison Approach Actually Work? A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation}, institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives}, type = {Working Paper}, number = {16}, year = {2026}, month = {October} } ``` ## 👤 Author & contact **Simon-Pierre Boucher** — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 [simon-pierre.boucher@uqo.ca](mailto:simon-pierre.boucher@uqo.ca) ## 🗒 Notes - Numbering: issued on 2026-10-10 as "Working Paper No. 11" (number already used by *Half a Million Prices, Twenty Models*); renumbered **16** on 2026-10-17. The SSRN upload 7626720 predates the renumbering. - Deviation from `Recherche-UQO/HARMONISATION.md` §2: the pipeline keeps `pipeline/` (not `scripts/`) and `out/` (not `results/`) because the same tree runs on M2U64 under `~/comparables-validation` with these paths. - French protocol and report in `docs/` (`PROTOCOLE.md`, `RAPPORT.md/.pdf`); original French README in `docs/README-fr.md`. - Per-node copies of the working tree pulled on 2026-10-10 (`M*omparables-validation/`) were removed from the repository and archived in `Recherche-UQO/_archives/sales-comparison-node-copies-20261010/`.