SPB Git

spb/wp9_uqo Public

UQO Working Paper No. 9 — A grand hedonic model of the Canadian housing market: decomposing structure and location value.

TeX 60.1% Python 39.8%
ZIP
NameLast commitUpdated
data WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
figures WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
paper Scholarly upgrade v1.2: +32 verified references, new Discussion... 5 days ago
results WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
scripts WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
src WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
.gitignore WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
AUDIT.md WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
CHANGES.md WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
PAPER_REVIEW.md Scholarly upgrade v1.2: +32 verified references, new Discussion... 5 days ago
README.md WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
requirements.txt WP9 — A Grand Hedonic Model of the Canadian Housing Market 5 days ago
UPGRADE_REPORT.md Scholarly upgrade v1.2: +32 verified references, new Discussion... 5 days ago
README.md

# 🏘️ A Grand Hedonic Model of the Canadian Housing Market

UQO Working Paper No. 9Decomposing the Value of Structure and Location across 140,931 MLS Listings with High-Dimensional Neighbourhood Fixed Effects

Paper Python DuckDB Estimation Reproducible Tables verified Institution Author Contact

TL;DR — On 140,931 Canadian MLS listings, absorbing 1,153 neighbourhood (FSA) fixed effects lifts explained log-price variance from 46% → 77%. Location alone is worth ~30 percentage points of R² — more than every structural attribute combined. The model values held-out homes with a median absolute error of 15.8% (OOS R² = 0.764), competitive with commercial AVMs, while staying fully transparent.


# 📖 Table of contents

  1. What this paper does
  2. Headline results
  3. Repository layout
  4. Quick start
  5. The pipeline, step by step
  6. Data
  7. Methodology
  8. The two results tiers (reference vs. reproduced)
  9. Figures & tables inventory
  10. Reproduction verification
  11. Limitations
  12. Citation
  13. Author & contact

# 🎯 What this paper does

A dwelling is the archetypal heterogeneous good: no two houses are identical, and the most important attribute — where it stands — is unobservable as a scalar. This project estimates a semi-logarithmic hedonic price equation at national scale for Canada:

ln P_i = α + β·ln(Area_i) + x_i'γ + d_i'δ + μ_f(i) + ε_i

where μ_f(i) is a fixed effect for the Forward Sortation Area (FSA — the first three characters of the postal code) of listing i. With 1,153 absorbed FSA intercepts, every neighbourhood gets an arbitrary price level that soaks up schools, transit, coastline, employment density — observed or not — and the structural implicit prices (β, γ) are identified purely from within-neighbourhood variation. Estimation is by absorbing least squares (numerically identical to full-dummy OLS); inference is clustered by FSA.

Beyond the headline decomposition the paper delivers: out-of-sample valuation accuracy (80/20 split, Duan-smeared retransformation), robustness across six sample cuts, quantile hedonic regressions, a quadratic test of diminishing returns to floor space, an urban price gradient in distance to the nine major metros, Moran's I spatial diagnostics, leave-one-province-out transferability, and a ranked map of Canada's most and least expensive neighbourhoods net of structure.

# 🏆 Headline results

Quantity Value
Estimation sample 140,931 listings — 82,334 houses, 57,857 condos, 9 provinces
Absorbed neighbourhood effects 1,153 FSAs
R²: structural only (M1) 0.464
R²: + dwelling type & ownership (M2) 0.469
R²: + province FE (M3) 0.567
R²: grand model, FSA FE (M5) 0.767
Living-area elasticity (M5) 0.547 (cluster SE 0.009)
Full-bathroom premium 0.109 log points ≈ +11% per bathroom
Bedrooms, conditional on area ≈ 0 (the textbook hedonic result)
Out-of-sample R² (log price) 0.764
Median absolute valuation error 15.8% (59% of homes priced within ±20%)
Moran's I of residuals, structural → grand 0.46 → 0.08 (−82%)
Urban gradient −8.5% location premium per doubling of distance to metro
Neighbourhood premia span ×9 between the most and least expensive FSAs

The most expensive neighbourhoods net of structure are all in the City of Vancouver (V6S, V8E, V6T: +150–200% vs. the national median); the cheapest are in rural Saskatchewan, Manitoba and Newfoundland (−60 to −67%).

# 📁 Repository layout

wp9_uqo/
├── README.md                  ← you are here
├── AUDIT.md                   forensic audit of the original project + verification
├── CHANGES.md                 restructuring report (what moved, what was rewritten)
├── requirements.txt           pinned Python dependencies
├── .gitignore
├── data/
│   ├── raw/
│   │   ├── realtor_mls_unique.duckdb   ⚠ 747 MB — NOT in git (see Data section)
│   │   └── README.md
│   └── processed/
│       └── analysis.parquet   estimation sample, 142k rows (generated by step 01)
├── src/wp9/                   analysis package
│   ├── config.py              paths, constants, seeds, metro coordinates
│   ├── parsing.py             parsers for the semi-structured MLS fields
│   ├── sample.py              raw DuckDB → estimation sample
│   ├── models.py              design matrix, M1–M5 ladder, AbsorbingLS, FE recovery
│   └── plotstyle.py           shared matplotlib style
├── scripts/                   numbered pipeline entry points
│   ├── 01_build_sample.py
│   ├── 02_estimate_core.py
│   ├── 03_estimate_extended.py
│   ├── 04_make_figures.py     (--results reference|reproduced)
│   └── 05_make_tables.py      (--results reference|reproduced)
├── figures/                   all 17 paper figures (PNG, generated)
├── results/
│   ├── reference/             ORIGINAL published outputs — canonical paper numbers
│   ├── reproduced/            outputs regenerated end-to-end by this pipeline
│   └── tables/                the 6 LaTeX tables consumed by the paper
└── paper/
    ├── main.tex               preamble + metadata; \input's the sections
    ├── main.pdf               compiled paper (26 pages)
    ├── sections/              titlepage, introduction, literature, data,
    │                          methodology, results, robustness, conclusion
    ├── references.bib         32 entries, natbib author-year, aer style
    ├── Makefile / .latexmkrc  build config
    └── uq_logo.jpg

# 🚀 Quick start

git clone https://github.com/spboucher-ai/wp9_uqo.git
cd wp9_uqo
python3 -m pip install -r requirements.txt

# Full pipeline (needs data/raw/realtor_mls_unique.duckdb — see Data section):
python3 scripts/01_build_sample.py        # duckdb → data/processed/analysis.parquet
python3 scripts/02_estimate_core.py       # M1–M5 ladder → results/reproduced/
python3 scripts/03_estimate_extended.py   # OOS, robustness, quantile, Moran, LOPO…
python3 scripts/04_make_figures.py        # figures/ (17 PNG)
python3 scripts/05_make_tables.py         # results/tables/ (6 .tex)

# Paper:
cd paper && latexmk -pdf main.tex         # or `make`

Without the raw DuckDB you can still run steps 02→05: the committed data/processed/analysis.parquet (6 MB) is the estimation sample produced by step 01.

# 🔬 The pipeline, step by step

Step Script Input Output Runtime*
01 01_build_sample.py raw DuckDB (172,019 rows) analysis.parquet (142k rows, 21 cols) ~40 s
02 02_estimate_core.py parquet fit.json, coef_M{1,2,3,5}.csv, grand_model.parquet, fsa_premia.csv ~1 min
03 03_estimate_extended.py parquet + step 02 oos.json, robustness.csv, heterogeneity.csv, quantile.csv, lopo.csv, ext2.json, Moran arrays, gradient/nonlinearity bands ~6 min
04 04_make_figures.py results tier + parquet 17 PNG figures ~30 s
05 05_make_tables.py results tier 6 LaTeX tables ~5 s

* Apple Silicon, single process.

Step 01 in detail (documented in src/wp9/parsing.py / sample.py):

  • bedrooms "3 + 1" (main + basement) → summed;
  • living area: prefers the explicit floor-area measurement; banded entries ("1100-1500 sqft") map to the upper bound of the band; falls back to size_interior; sqft → m² at 0.0929;
  • lot size parsed from free text ("6000 sqft", "0.14 ac", "under 1/2 acre", "50 x 120" frontage×depth) with a missingness indicator; the top 1% of positive parsed lots (multi-acre rural acreage strings) treated as "no usable lot info";
  • dwelling type consolidated to 8 groups, ownership to 6 groups;
  • FSA from the postal code; FSAs with < 25 listings pooled into <PROV>_other;
  • filters: strictly positive price, non-missing living area / bedrooms / bathrooms, valid FSA; then the extreme 1% tails of price and living area are trimmed.

# 📊 Data

Source file: data/raw/realtor_mls_unique.duckdb — a de-duplicated snapshot of Canadian MLS "for-sale" listings: 172,019 rows × 81 columns (single table listings), with list price, geocoded coordinates, postal code, and semi-structured building/lot attributes.

⚠️ The raw DuckDB (747 MB) is not distributed in this repository (GitHub's 100 MB file limit, plus its content is scraped listing data). It lives on the author's machines; place it at data/raw/realtor_mls_unique.duckdb to run step 01. All downstream artifacts — including the committed estimation sample — derive from it programmatically.

Estimation-sample snapshot (Table 1 of the paper): median list price ≈ $639,888; median living area ≈ 135 m²; 3 bedrooms; 2 full bathrooms. Both price and price/m² are strongly right-skewed, motivating the log transformation. Ontario, Quebec, BC and Alberta dominate; PEI and the territories have no listings in this snapshot.

# 📐 Methodology

  • Specification ladder — M1 structural only → M2 + type/ownership → M3 + province FE (house subsample) → M4 houses + FSA FE → M5 grand model (all residential, FSA FE). The M3→M5 gap ≈ 20 pp of R² is the value of resolving location at neighbourhood scale.
  • Absorbing least squares (linearmodels.AbsorbingLS) sweeps out the 1,153 FSA intercepts without materialising dummies; slopes are numerically identical to full-dummy OLS. SEs clustered by FSA throughout.
  • Retransformation — predictions in levels use Duan's (1983) smearing estimator (no log-normality assumption).
  • Validation — random 80/20 split, evaluation restricted to FSAs observed in training (out-of-support FE are not identified); leave-one-province-out CV with province-specific intercepts; Moran's I with row-standardised k-NN weights (k = 10, 15,000-listing sample, 199 permutations).

# 🧭 The two results tiers (reference vs. reproduced)

The original upstream cleaning code (RE_DB_QC/hedonic) was lost before this repository was assembled; src/wp9/sample.py is a careful reconstruction from the paper's own data section, calibrated against every stored output (see AUDIT.md). It reproduces the published sample to +0.86% and all headline estimates closely — but not to the last digit. To keep the published record intact:

  • results/reference/ — the original outputs (verbatim files, plus values transcribed from the published tables). Default source for figures/tables, so the paper always shows exactly the published numbers.
  • results/reproduced/ — regenerated end-to-end by scripts 02–03 on the reconstructed sample.
python3 scripts/04_make_figures.py --results reproduced   # opt in to regenerated numbers
python3 scripts/05_make_tables.py  --results reproduced

# 🖼️ Figures & tables inventory

# Figure Shows Numbers from
1 fig_price_dist raw vs. log price distribution micro sample
2 fig_province_ppm2 median $/m² by province micro sample
3 fig_r2 R² across the M1–M5 ladder reference
4 fig_forest structural implicit prices, 95% CI (M3) reference
5 fig_size_gradient median price by size bin × dwelling type micro sample
6a/6b fig_map / fig_fsa_map listing map & FSA medians, coloured by ln $/m² micro sample
7 fig_premia top/bottom 12 FSA premia vs. national median reference
8 fig_decomp variance decomposition bar reference
9 fig_nonlinear marginal elasticity vs. size (quadratic model) reproduced band + reference anchor
10 fig_gradient location premium vs. distance to metro reproduced
11 fig_quantile quantile coefficients τ = 0.1…0.9 reference
12 fig_moran Moran scatterplots, structural vs. grand reproduced clouds, reference I
13 fig_heterogeneity size elasticity by province, 95% CI reference
14 fig_oos OOS error buckets (±10/±20%) reference
15/16 fig_fit / fig_resid predicted vs. actual; residual diagnostics reproduced

(The leave-one-province-out results appear as a table only.)

Tables (results/tables/): summary_stats, regression (M1/M2/M3/M5), robustness (6 sample cuts), quantile, lopo, oos — all six verified numerically identical to the originally published tables.

# ✅ Reproduction verification

Full side-by-side in AUDIT.md §5. Highlights (original → reproduced):

  • Sample: 140,931 → 142,146 (+0.86%); condos match to 3 listings.
  • R² ladder: 0.464/0.469/0.567/0.762/0.767 → 0.467/0.470/0.569/0.766/0.770.
  • OOS: R² 0.764 → 0.770; median APE 15.8% → 15.6%.
  • Moran's I: 0.459→0.082 vs. 0.494→0.072 (same −82~85% conclusion).
  • Known gaps (flagged, not hidden): M5 living elasticity 0.547 vs. 0.530; lot-field coefficients differ (the original free-text lot parser could not be fully recovered); LOPO mean 0.362 vs. 0.315 with identical province ranking.

# ⚠️ Limitations

List prices, not transactions; no construction year / renovation status / interior quality (absorbed into FSA effects and the residual); lot information sparse and noisy; a single cross-section — levels, not dynamics. See the paper's conclusion for the research agenda these imply.

# 📚 Citation

@techreport{boucher2026grandhedonic,
  author      = {Boucher, Simon-Pierre},
  title       = {A Grand Hedonic Model of the Canadian Housing Market:
                 Decomposing the Value of Structure and Location across
                 140,931 MLS Listings with High-Dimensional Neighbourhood
                 Fixed Effects},
  institution = {Universit\'e du Qu\'ebec en Outaouais,
                 D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {9},
  year        = {2026},
  month       = {May}
}

# 👤 Author & contact

Simon-Pierre Boucher Département des sciences administratives, Université du Québec en Outaouais Gatineau — Pavillon Alexandre-Taché, 283 boulevard Alexandre-Taché, Gatineau (QC) J9A 1L8

📧 contact@spboucher.ai · 🌐 spboucher.ai

All code files carry the header Author: Simon-Pierre Boucher — contact@spboucher.ai.

© 2026 Simon-Pierre Boucher. All rights reserved. The listing data snapshot is not redistributed; code and paper are shared for research reproducibility.