SPB Git
spboucher.ai

spb/wp9_uqo

Public

UQO Working Paper No. 9 — A grand hedonic model of the Canadian housing market: decomposing structure and location value.

researcheconomicshousinghedoniccanadauqo

WP9 — A Grand Hedonic Model of the Canadian Housing Market

Reproducible research compendium for UQO Working Paper No. 9: hedonic
pricing of 140,931 Canadian MLS listings with 1,153 absorbed FSA
neighbourhood fixed effects. Includes the full pipeline (sample
construction from the raw DuckDB snapshot, M1–M5 specification ladder,
OOS validation, robustness, quantile, Moran's I, LOPO), all 17 figures,
6 tables, and the compiled 26-page LaTeX paper. Original published
outputs preserved in results/reference/; reconstruction notes and
verification in AUDIT.md / CHANGES.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simon-pierre boucher committed (Aug 5, 2026) Browse files

86 changed files +5,484 −0

86 changed files
Load all 86 files Unified Split
added .gitignore +19 −0
@@ -0,0 +1,19 @@
1 +# Raw data snapshot — 747 MB, exceeds GitHub limits and is not redistributed
2 +data/raw/*.duckdb
3 +
4 +# Python
5 +__pycache__/
6 +*.pyc
7 +
8 +# LaTeX build artifacts (paper/main.pdf IS committed)
9 +paper/*.aux
10 +paper/*.log
11 +paper/*.out
12 +paper/*.fls
13 +paper/*.fdb_latexmk
14 +paper/*.bbl
15 +paper/*.blg
16 +paper/*.synctex.gz
17 +
18 +# macOS
19 +.DS_Store
added AUDIT.md +146 −0
@@ -0,0 +1,146 @@
1 +# AUDIT — UQO Working Paper 9 (`immo-wp9-spb-20260529`)
2 +
3 +**Audit date:** 2026-08-05
4 +**Original location:** `~/Desktop/UQO/UQO_WP/immo-wp9-spb-20260529` (left untouched)
5 +**New clean repository:** `~/Desktop/wp9_uqo`
6 +
7 +## 1. What the project is
8 +
9 +UQO Working Paper No. 9, *"A Grand Hedonic Model of the Canadian Housing Market"* —
10 +a semi-logarithmic hedonic price model estimated on 140,931 Canadian MLS listings with
11 +1,153 FSA (Forward Sortation Area) neighbourhood fixed effects, plus out-of-sample
12 +validation, robustness, quantile regressions, spatial diagnostics (Moran's I),
13 +leave-one-province-out cross-validation, and a variance decomposition.
14 +
15 +## 2. File inventory (original, 55 files, ~749 MB)
16 +
17 +### Data
18 +| File | Role |
19 +|---|---|
20 +| `realtor_mls_unique.duckdb` (747 MB) | **Raw data.** Single table `listings`, 172,019 rows, 81 columns (all VARCHAR except `price_cad`, `lat`, `lon`). This is the only data file; the paper's estimation sample (140,931 rows) is derived from it. |
21 +
22 +### Analysis code (`analysis/`)
23 +| Script | What it does | Outputs |
24 +|---|---|---|
25 +| `estimate_ext.py` | 80/20 out-of-sample validation of the FSA-FE hedonic model; living-area-elasticity heterogeneity by province; robustness across 6 samples; FSA location-premium ranking. | `oos.json`, `heterogeneity.csv`, `robustness.csv`, `fsa_premia.csv` |
26 +| `estimate_ext2.py` | Quadratic (nonlinearity) spec; quantile regressions; distance-to-metro gradient; Moran's I on residuals; leave-one-province-out CV; variance decomposition. | `ext2.json`, `lopo.csv`, `tables/quantile.tex`, figures `fig_nonlinear/quantile/gradient/moran/decomp.png` |
27 +| `make_assets.py` | Copies 9 upstream figures; builds `fig_heterogeneity/premia/oos.png`; writes `tables/summary_stats/regression/robustness/oos.tex`. | figures + tables |
28 +| `ml_benchmark.py` | XGBoost / LightGBM / PyTorch-MLP benchmarks vs. the hedonic model. | `ml_compare.csv/json`, `tables/ml.tex`, `fig_ml_compare/importance.png` — **none of these outputs exist on disk and the paper never references them → dead code for this paper.** |
29 +
30 +### Stored analysis outputs (reference values for verification)
31 +`analysis/oos.json`, `ext2.json`, `robustness.csv`, `lopo.csv`, `heterogeneity.csv`,
32 +`fsa_premia.csv` — all present and internally consistent with the numbers quoted in the
33 +paper. These are the ground truth I verify the rebuilt pipeline against.
34 +
35 +### Figures (`figures/`, 7 present of 17 referenced by the paper)
36 +Present: `fig_price_dist`, `fig_fit`, `fig_resid` (copied from upstream),
37 +`fig_decomp`, `fig_nonlinear`, `fig_gradient`, `fig_premia` (locally generated).
38 +**Missing (10):** `fig_province_ppm2`, `fig_r2`, `fig_forest`, `fig_size_gradient`,
39 +`fig_map`, `fig_fsa_map` (upstream-only), and `fig_quantile`, `fig_moran`,
40 +`fig_heterogeneity`, `fig_oos` (locally generated but deleted at some point).
41 +
42 +### Tables (`tables/`, 6 files)
43 +`summary_stats.tex`, `regression.tex`, `robustness.tex`, `quantile.tex`, `oos.tex`,
44 +`lopo.tex`. All referenced by the paper. Note: `lopo.tex` is **hand-written** — no script
45 +produces it (`estimate_ext2.py` only writes `lopo.csv`).
46 +
47 +### LaTeX
48 +| File | Role |
49 +|---|---|
50 +| `main.tex` | Real paper: preamble + metadata + `\input{sections/*}`. Already reasonably modular. |
51 +| `main_web.tex` | Duplicate of `main.tex` plus an `\includegraphics` override that prints a *"Figure indisponible"* placeholder box when a figure file is missing. |
52 +| `sections/*.tex` (8) | titlepage, introduction, literature, data, methodology, results, robustness, conclusion. |
53 +| `references.bib` | 32 entries, `aer` style with natbib. |
54 +| `uq_logo.jpg` | UQO logo for the title page. |
55 +| `Makefile`, `.latexmkrc` | latexmk build config. |
56 +
57 +### Dead / generated / junk files
58 +- Build artifacts: `main*.aux/log/out/fls/fdb_latexmk`, `main_web.bbl/blg/synctex.gz` — regenerate on compile.
59 +- `main_web.pdf` (24 p., June 13): **compiled with placeholder boxes instead of every figure** — the `\IfFileExists` fallback does not search `\graphicspath`, so even the 7 existing figures were replaced by boxes. There is no valid compiled PDF of the actual paper.
60 +- `main.pdf`: does not exist (never successfully compiled, or deleted).
61 +- `ml_benchmark.py`: dead for this paper (outputs unused, see above).
62 +- `.DS_Store` files.
63 +
64 +## 3. Critical findings
65 +
66 +1. **The upstream pipeline is gone.** All three live scripts read
67 + `~/Desktop/RE_DB_QC/hedonic/data/analysis.parquet` and
68 + `~/Desktop/RE_DB_QC/hedonic/output/fit.json` / `coef_M*.csv`. That directory no longer
69 + exists anywhere on disk. The cleaning code (duckdb → `analysis.parquet`) and the
70 + M1–M5 estimation code were never inside this project.
71 + **Consequence:** the pipeline had to be reconstructed from (a) the raw DuckDB, which
72 + *is* in the project, (b) the precise description of sample construction in
73 + `sections/data.tex`, and (c) the stored outputs/tables used as exact numerical targets.
74 +2. **No usable compiled PDF existed** (see above).
75 +3. **10 of 17 figures missing** and must be regenerated by the reconstructed pipeline.
76 +4. Paths in all scripts are absolute and machine-specific; no `requirements.txt`,
77 + no README, no reproducibility instructions.
78 +5. Hard-coded constants in scripts: the dashed reference line `0.547` in
79 + `fig_nonlinear` and the metro coordinates list (duplicated in two scripts).
80 +6. `\WPemail` in the paper is `simon-pierre.boucher@uqo.ca` (kept as-is in the rewrite;
81 + code headers use `contact@spboucher.ai` per instruction).
82 +
83 +## 4. Reference targets used for verification (from the original outputs)
84 +
85 +- Sample: **N = 140,931** (houses 82,334; condos 57,857); 1,153 absorbed FSAs; raw N = 172,019.
86 +- Grand model M5: R² = 0.767; ln-living 0.547 (0.009); bath 0.109 (0.004); ln-lot 0.030 (0.004).
87 +- Ladder R²: M1 0.464, M2 0.469, M3 0.567 (houses); M5 0.767.
88 +- OOS: R² 0.764, RMSE(log) 0.282, median APE 15.8%, within ±10% 33.8%, ±20% 59.5% (train 112,732 / test 28,187).
89 +- Moran's I: 0.459 → 0.082; gradient −0.0854; nonlinearity b1 = 1.065, b2 = −0.0526; LOPO mean 0.362.
90 +- Full details in `analysis/*.{json,csv}` and `tables/*.tex` of the original.
91 +
92 +## 5. Reproduction verification — rebuilt pipeline vs. original outputs
93 +
94 +The cleaning pipeline was reconstructed from the paper's data section and calibrated
95 +against the stored outputs. Decisive validated choices: `_province` (scraper seed) is the
96 +province variable; living area prefers `building.floor_area_measurements` with banded
97 +entries mapped to the **upper bound** of the band (this uniquely matches the published
98 +living-area distribution and the M3 elasticity of 0.585 exactly); bedrooms "3 + 1" summed;
99 +FSAs with <25 listings pooled to `<PROV>_other`; 1%/99% trim of price and living area.
100 +
101 +**Everything below refers to `results/reproduced/` (script output). The paper itself uses
102 +`results/reference/` — the original stored outputs — so no published number changed.**
103 +
104 +| Quantity | Original (paper) | Reproduced | Note |
105 +|---|---|---|---|
106 +| Sample N | 140,931 | 142,146 | +0.86% |
107 +| Houses / Condos | 82,334 / 57,857 | 83,542 / 57,860 | condos match to 3 listings |
108 +| Absorbed FSA levels | 1,153 | 1,155 | |
109 +| R² M1 / M2 / M3 (houses) | 0.464 / 0.469 / 0.567 | 0.467 / 0.470 / 0.569 | |
110 +| R² M4 / M5 | 0.762 / 0.767 | 0.766 / 0.770 | |
111 +| M5 ln living | 0.547 (0.009) | 0.530 (0.010) | ~2 SE apart |
112 +| M5 full bathrooms | 0.109 (0.004) | 0.111 (0.003) | ✓ |
113 +| M5 half baths / bedrooms | −0.036 / −0.002 | −0.032 / −0.003 | ✓ |
114 +| M5 ln lot / has lot | 0.030 / −0.196 | 0.061 / −0.417 | see (a) below |
115 +| OOS R² / RMSE / med. APE | 0.764 / 0.282 / 15.8% | 0.770 / 0.280 / 15.6% | ✓ |
116 +| OOS within ±10 / ±20% | 33.8% / 59.5% | 34.2% / 60.0% | ✓ |
117 +| Moran's I struct → grand | 0.459 → 0.082 | 0.494 → 0.072 | same conclusion (−85%) |
118 +| Gradient (β log-dist) | −0.0854 | −0.0821 | ✓ |
119 +| Nonlinearity b1 / b2 | 1.065 / −0.0526 | 1.145 / −0.0624 | same shape |
120 +| LOPO mean R² | 0.362 | 0.315 | same ranking (ON best, AB negative) |
121 +| Elasticity by province | 0.44 (NL) – 0.66 (MB) | 0.42 (NL) – 0.72 (SK) | same coastal-vs-Prairies pattern |
122 +| Quantile elasticity τ=.1→.9 | 0.557 → 0.598 | 0.591 → 0.585 | flatter but same level |
123 +
124 +### Remaining discrepancies (flagged, not silently changed)
125 +
126 +(a) **Lot-size parsing.** The free-text `land.size_total` parser could not be recovered
127 + exactly. The original summary stats (mean 2,621 / SD 10,818 m²) imply the original
128 + parser bounded large acreages; the reconstruction treats the top 1% of positive parsed
129 + lots as missing, which matches the SD but shifts the two collinear lot coefficients
130 + (`ln_lot`, `has_lot`). All other structural coefficients are essentially unaffected.
131 +(b) **Sample off by +1,215 listings (+0.86%)**, concentrated in houses at the 5,000-sqft
132 + living-area band boundary — an artifact of how the original trim treated the mass of
133 + listings at exactly the 99th-percentile value. Counts by province and condo counts
134 + match almost exactly.
135 +(c) **fig_size_gradient / condominium claim.** The original figure was lost. Regenerated
136 + raw medians show condominiums listing *above* houses of equal size (they concentrate
137 + in expensive metros), contradicting one descriptive sentence in the original results
138 + section; the sentence was rewritten to describe the regenerated figure and the
139 + (supported) conditional claim. See CHANGES.md — requires author review.
140 +(d) `analysis/ml_benchmark.py` (XGBoost/LightGBM/PyTorch benchmark) was **dropped**: its
141 + outputs do not exist and the paper never references them.
142 +(e) Original `main_web.pdf` had placeholder boxes instead of all figures (its
143 + `\IfFileExists` fallback ignored `\graphicspath`), so no valid compiled PDF of the
144 + original paper existed; page-level comparison against the original PDF was therefore
145 + impossible. The 7 surviving original PNGs were compared instead — the regenerated
146 + versions are visually identical for the results-driven figures (e.g. `fig_premia`).
added CHANGES.md +101 −0
@@ -0,0 +1,101 @@
1 +# CHANGES — Restructuring report (2026-08-05)
2 +
3 +Original project: `~/Desktop/UQO/UQO_WP/immo-wp9-spb-20260529` (**left untouched**).
4 +New clean repository: `~/Desktop/wp9_uqo` (this folder).
5 +No published number, result or scientific claim was changed; the paper's tables and
6 +number-bearing figures are built from the original outputs preserved in
7 +`results/reference/`.
8 +
9 +## 1. What was moved / renamed
10 +
11 +| Original | New location |
12 +|---|---|
13 +| `realtor_mls_unique.duckdb` (repo root) | `data/raw/` (byte-identical copy) |
14 +| `analysis/*.csv`, `*.json` (stored outputs) | `results/reference/` (canonical paper numbers) |
15 +| `tables/*.tex` | regenerated into `results/tables/` by script 05 (values verified **identical**) |
16 +| `figures/*.png` (7 survivors of 17) | all 17 regenerated into `figures/` by script 04 |
17 +| `main.tex` + `sections/` + `references.bib` + `uq_logo.jpg` | `paper/` |
18 +| `Makefile`, `.latexmkrc` | `paper/` |
19 +| LaTeX build artifacts (`.aux/.log/.fls/...`), `.DS_Store` | not carried over |
20 +
21 +Values that existed only inside the published tables (regression coefficients,
22 +quantile estimates, summary statistics, model R²) were transcribed into
23 +`results/reference/{coef_M*.csv, quantile.csv, summary_stats.csv, fit.json}` so that
24 +every table/figure is now generated from a data file rather than hard-coded.
25 +
26 +## 2. Code refactoring
27 +
28 +The three live scripts (`estimate_ext.py`, `estimate_ext2.py`, `make_assets.py`,
29 +~25 KB of duplicated single-letter-variable code with absolute machine-specific paths)
30 +were refactored into a package + numbered entry points:
31 +
32 +- `src/wp9/config.py` — all paths (relative to the repo), constants, seeds; the metro
33 + coordinates and the sqft/acre conversions now live in exactly one place.
34 +- `src/wp9/parsing.py` — documented parsers for every semi-structured raw field.
35 +- `src/wp9/sample.py` — raw DuckDB → estimation sample (was upstream in the lost
36 + `RE_DB_QC` pipeline; **reconstructed**, see AUDIT.md §5).
37 +- `src/wp9/models.py` — design matrix, M1–M5 ladder, absorbing least squares, FSA
38 + fixed-effect recovery, Duan smearing. The `design()` function, previously copy-pasted
39 + four times across scripts, exists once.
40 +- `src/wp9/plotstyle.py` — shared figure style/palette.
41 +- `scripts/01_build_sample.py` → `data/processed/analysis.parquet`
42 +- `scripts/02_estimate_core.py` → ladder fits, coefficient tables, grand-model
43 + residuals, FSA premia
44 +- `scripts/03_estimate_extended.py` → OOS, robustness, heterogeneity, quantile,
45 + nonlinearity, gradient, Moran's I, LOPO
46 +- `scripts/04_make_figures.py` / `05_make_tables.py` → all 17 figures / 6 tables, with
47 + `--results {reference,reproduced}` selecting the numbers tier (default: reference =
48 + published values).
49 +
50 +Every code file carries the header `Author: Simon-Pierre Boucher — contact@spboucher.ai`.
51 +`analysis/ml_benchmark.py` was dropped (dead code: outputs absent, never cited by the
52 +paper). `main_web.tex` was dropped (web variant whose figure-fallback produced a PDF of
53 +placeholder boxes).
54 +
55 +Pipeline verified end-to-end: raw DuckDB → sample → estimates → figures/tables → PDF.
56 +Regenerated tables are numerically identical to the originals; regenerated
57 +reference-driven figures are visually identical to the surviving originals; the
58 +`reproduced` tier matches the published estimates closely but not exactly (full
59 +side-by-side table in AUDIT.md §5).
60 +
61 +## 3. Paper rewrite (`paper/`)
62 +
63 +- Structure kept as `main.tex` + one file per section; preamble reorganised into
64 + labelled blocks; duplicate/unused packages removed (`tabularx`, `multirow`, `float`,
65 + `appendix`, `amsthm` retained only where used); metadata unchanged except
66 + `Version 1.0 → 1.1`.
67 +- Tables are now `\input` from `../results/tables/` and figures from `../figures/`, so
68 + recompiling after a pipeline run always uses current artifacts.
69 +- Prose: editorial polish only — tightened sentences, fixed agreement/punctuation,
70 + consistent notation and hyphenation; all numbers, claims and 32 citations preserved.
71 +- Fixed an internal inconsistency: the note under the quantile table said the size
72 + elasticity *falls* with price while the table and body text show it *rising*
73 + (0.557 → 0.598); the note now agrees with the table.
74 +- The maps figure caption no longer promises Natural Earth provincial boundaries (the
75 + regenerated maps plot listing coordinates with province labels; the boundary shapefile
76 + dependency was removed).
77 +- Compiles cleanly: `paper/main.pdf`, 26 pages, zero errors/undefined references, all
78 + 17 figures embedded (the old `main_web.pdf` contained none).
79 +
80 +## 4. Items requiring your review
81 +
82 +1. **Condominium size-gradient sentence (results §5.2).** The original
83 + `fig_size_gradient.png` is lost. Raw medians from the data show condominiums listing
84 + *above* houses of equal size (composition: condos sit in expensive metros), while the
85 + original text claimed they "lie below houses at every size". I rewrote that one
86 + sentence to describe the regenerated figure and the (data-supported) conditional
87 + version of the claim. Please confirm the new wording — or tell me how the original
88 + figure was constructed and I will match it.
89 +2. **Reproduced tier differences** (AUDIT.md §5): sample +0.86%; M5 living-area
90 + elasticity 0.530 vs 0.547; lot coefficients differ because the original lot parser
91 + could not be fully recovered; LOPO mean 0.315 vs 0.362. If you still have any copy of
92 + the `RE_DB_QC/hedonic` pipeline (backup/other machine), I can close these gaps.
93 +3. **Moran scatter / fit / residual / nonlinearity-band figures** are computed from the
94 + reconstructed sample (no stored micro-outputs existed); their annotated Moran's I
95 + values are taken from the original `ext2.json` while the scatter clouds are
96 + reproduced data.
97 +4. **`ml_benchmark.py` dropped** — recover from the original folder if you want the ML
98 + comparison back; its outputs would need to be regenerated (requires xgboost,
99 + lightgbm, torch).
100 +5. The paper still lists `simon-pierre.boucher@uqo.ca` as contact (unchanged); code
101 + headers use `contact@spboucher.ai` per your instruction.
added README.md +303 −0
@@ -0,0 +1,303 @@
1 +# 🏘️ A Grand Hedonic Model of the Canadian Housing Market
2 +
3 +**UQO Working Paper No. 9** — *Decomposing the Value of Structure and Location across
4 +140,931 MLS Listings with High-Dimensional Neighbourhood Fixed Effects*
5 +
6 +[![Paper](https://img.shields.io/badge/paper-PDF%20(26%20p.)-b31b1b?logo=latex&logoColor=white)](paper/main.pdf)
7 +[![Python](https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&logoColor=white)](requirements.txt)
8 +[![DuckDB](https://img.shields.io/badge/data-DuckDB%20·%20172%2C019%20listings-FFF000?logo=duckdb&logoColor=black)](data/raw/README.md)
9 +[![Estimation](https://img.shields.io/badge/estimation-statsmodels%20·%20linearmodels%20(AbsorbingLS)-4051b5)](src/wp9/models.py)
10 +[![Reproducible](https://img.shields.io/badge/reproducible-end--to--end%20pipeline-2e7d32)](scripts/)
11 +[![Tables verified](https://img.shields.io/badge/tables-identical%20to%20published-2e7d32)](AUDIT.md)
12 +[![Institution](https://img.shields.io/badge/UQO-D%C3%A9pt.%20des%20sciences%20administratives-16365c)](https://uqo.ca)
13 +[![Author](https://img.shields.io/badge/author-Simon--Pierre%20Boucher-16365c)](mailto:contact@spboucher.ai)
14 +[![Contact](https://img.shields.io/badge/contact-contact%40spboucher.ai-a02020?logo=maildotru&logoColor=white)](mailto:contact@spboucher.ai)
15 +
16 +> **TL;DR** — On 140,931 Canadian MLS listings, absorbing 1,153 neighbourhood (FSA) fixed
17 +> effects lifts explained log-price variance from **46% → 77%**. Location alone is worth
18 +> ~30 percentage points of R² — more than every structural attribute combined. The model
19 +> values held-out homes with a median absolute error of **15.8%** (OOS R² = 0.764),
20 +> competitive with commercial AVMs, while staying fully transparent.
21 +
22 +---
23 +
24 +## 📖 Table of contents
25 +
26 +1. [What this paper does](#-what-this-paper-does)
27 +2. [Headline results](#-headline-results)
28 +3. [Repository layout](#-repository-layout)
29 +4. [Quick start](#-quick-start)
30 +5. [The pipeline, step by step](#-the-pipeline-step-by-step)
31 +6. [Data](#-data)
32 +7. [Methodology](#-methodology)
33 +8. [The two results tiers (reference vs. reproduced)](#-the-two-results-tiers-reference-vs-reproduced)
34 +9. [Figures & tables inventory](#-figures--tables-inventory)
35 +10. [Reproduction verification](#-reproduction-verification)
36 +11. [Limitations](#-limitations)
37 +12. [Citation](#-citation)
38 +13. [Author & contact](#-author--contact)
39 +
40 +---
41 +
42 +## 🎯 What this paper does
43 +
44 +A dwelling is the archetypal heterogeneous good: no two houses are identical, and the most
45 +important attribute — *where it stands* — is unobservable as a scalar. This project
46 +estimates a **semi-logarithmic hedonic price equation** at national scale for Canada:
47 +
48 +```
49 +ln P_i = α + β·ln(Area_i) + x_i'γ + d_i'δ + μ_f(i) + ε_i
50 +```
51 +
52 +where `μ_f(i)` is a fixed effect for the **Forward Sortation Area** (FSA — the first three
53 +characters of the postal code) of listing *i*. With 1,153 absorbed FSA intercepts, every
54 +neighbourhood gets an arbitrary price level that soaks up schools, transit, coastline,
55 +employment density — observed or not — and the structural implicit prices (β, γ) are
56 +identified purely from **within-neighbourhood** variation. Estimation is by absorbing
57 +least squares (numerically identical to full-dummy OLS); inference is clustered by FSA.
58 +
59 +Beyond the headline decomposition the paper delivers: out-of-sample valuation accuracy
60 +(80/20 split, Duan-smeared retransformation), robustness across six sample cuts, quantile
61 +hedonic regressions, a quadratic test of diminishing returns to floor space, an urban
62 +price gradient in distance to the nine major metros, Moran's I spatial diagnostics,
63 +leave-one-province-out transferability, and a ranked map of Canada's most and least
64 +expensive neighbourhoods net of structure.
65 +
66 +## 🏆 Headline results
67 +
68 +| Quantity | Value |
69 +|---|---|
70 +| Estimation sample | **140,931** listings — 82,334 houses, 57,857 condos, 9 provinces |
71 +| Absorbed neighbourhood effects | **1,153** FSAs |
72 +| R²: structural only (M1) | 0.464 |
73 +| R²: + dwelling type & ownership (M2) | 0.469 |
74 +| R²: + province FE (M3) | 0.567 |
75 +| R²: grand model, FSA FE (M5) | **0.767** |
76 +| Living-area elasticity (M5) | **0.547** (cluster SE 0.009) |
77 +| Full-bathroom premium | **0.109 log points ≈ +11%** per bathroom |
78 +| Bedrooms, conditional on area | ≈ 0 (the textbook hedonic result) |
79 +| Out-of-sample R² (log price) | **0.764** |
80 +| Median absolute valuation error | **15.8%** (59% of homes priced within ±20%) |
81 +| Moran's I of residuals, structural → grand | **0.46 → 0.08** (−82%) |
82 +| Urban gradient | −8.5% location premium per doubling of distance to metro |
83 +| Neighbourhood premia span | ×9 between the most and least expensive FSAs |
84 +
85 +The most expensive neighbourhoods net of structure are all in the **City of Vancouver**
86 +(V6S, V8E, V6T: +150–200% vs. the national median); the cheapest are in rural
87 +Saskatchewan, Manitoba and Newfoundland (−60 to −67%).
88 +
89 +## 📁 Repository layout
90 +
91 +```
92 +wp9_uqo/
93 +├── README.md ← you are here
94 +├── AUDIT.md forensic audit of the original project + verification
95 +├── CHANGES.md restructuring report (what moved, what was rewritten)
96 +├── requirements.txt pinned Python dependencies
97 +├── .gitignore
98 +├── data/
99 +│ ├── raw/
100 +│ │ ├── realtor_mls_unique.duckdb ⚠ 747 MB — NOT in git (see Data section)
101 +│ │ └── README.md
102 +│ └── processed/
103 +│ └── analysis.parquet estimation sample, 142k rows (generated by step 01)
104 +├── src/wp9/ analysis package
105 +│ ├── config.py paths, constants, seeds, metro coordinates
106 +│ ├── parsing.py parsers for the semi-structured MLS fields
107 +│ ├── sample.py raw DuckDB → estimation sample
108 +│ ├── models.py design matrix, M1–M5 ladder, AbsorbingLS, FE recovery
109 +│ └── plotstyle.py shared matplotlib style
110 +├── scripts/ numbered pipeline entry points
111 +│ ├── 01_build_sample.py
112 +│ ├── 02_estimate_core.py
113 +│ ├── 03_estimate_extended.py
114 +│ ├── 04_make_figures.py (--results reference|reproduced)
115 +│ └── 05_make_tables.py (--results reference|reproduced)
116 +├── figures/ all 17 paper figures (PNG, generated)
117 +├── results/
118 +│ ├── reference/ ORIGINAL published outputs — canonical paper numbers
119 +│ ├── reproduced/ outputs regenerated end-to-end by this pipeline
120 +│ └── tables/ the 6 LaTeX tables consumed by the paper
121 +└── paper/
122 + ├── main.tex preamble + metadata; \input's the sections
123 + ├── main.pdf compiled paper (26 pages)
124 + ├── sections/ titlepage, introduction, literature, data,
125 + │ methodology, results, robustness, conclusion
126 + ├── references.bib 32 entries, natbib author-year, aer style
127 + ├── Makefile / .latexmkrc build config
128 + └── uq_logo.jpg
129 +```
130 +
131 +## 🚀 Quick start
132 +
133 +```bash
134 +git clone https://github.com/spboucher-ai/wp9_uqo.git
135 +cd wp9_uqo
136 +python3 -m pip install -r requirements.txt
137 +
138 +# Full pipeline (needs data/raw/realtor_mls_unique.duckdb — see Data section):
139 +python3 scripts/01_build_sample.py # duckdb → data/processed/analysis.parquet
140 +python3 scripts/02_estimate_core.py # M1–M5 ladder → results/reproduced/
141 +python3 scripts/03_estimate_extended.py # OOS, robustness, quantile, Moran, LOPO…
142 +python3 scripts/04_make_figures.py # figures/ (17 PNG)
143 +python3 scripts/05_make_tables.py # results/tables/ (6 .tex)
144 +
145 +# Paper:
146 +cd paper && latexmk -pdf main.tex # or `make`
147 +```
148 +
149 +Without the raw DuckDB you can still run steps **02→05**: the committed
150 +`data/processed/analysis.parquet` (6 MB) is the estimation sample produced by step 01.
151 +
152 +## 🔬 The pipeline, step by step
153 +
154 +| Step | Script | Input | Output | Runtime* |
155 +|---|---|---|---|---|
156 +| 01 | `01_build_sample.py` | raw DuckDB (172,019 rows) | `analysis.parquet` (142k rows, 21 cols) | ~40 s |
157 +| 02 | `02_estimate_core.py` | parquet | `fit.json`, `coef_M{1,2,3,5}.csv`, `grand_model.parquet`, `fsa_premia.csv` | ~1 min |
158 +| 03 | `03_estimate_extended.py` | parquet + step 02 | `oos.json`, `robustness.csv`, `heterogeneity.csv`, `quantile.csv`, `lopo.csv`, `ext2.json`, Moran arrays, gradient/nonlinearity bands | ~6 min |
159 +| 04 | `04_make_figures.py` | results tier + parquet | 17 PNG figures | ~30 s |
160 +| 05 | `05_make_tables.py` | results tier | 6 LaTeX tables | ~5 s |
161 +
162 +\* Apple Silicon, single process.
163 +
164 +**Step 01 in detail** (documented in `src/wp9/parsing.py` / `sample.py`):
165 +
166 +- bedrooms `"3 + 1"` (main + basement) → summed;
167 +- living area: prefers the explicit floor-area measurement; banded entries
168 + (`"1100-1500 sqft"`) map to the **upper bound** of the band; falls back to
169 + `size_interior`; sqft → m² at 0.0929;
170 +- lot size parsed from free text (`"6000 sqft"`, `"0.14 ac"`, `"under 1/2 acre"`,
171 + `"50 x 120"` frontage×depth) with a missingness indicator; the top 1% of positive
172 + parsed lots (multi-acre rural acreage strings) treated as "no usable lot info";
173 +- dwelling type consolidated to 8 groups, ownership to 6 groups;
174 +- FSA from the postal code; FSAs with < 25 listings pooled into `<PROV>_other`;
175 +- filters: strictly positive price, non-missing living area / bedrooms / bathrooms,
176 + valid FSA; then the extreme 1% tails of price and living area are trimmed.
177 +
178 +## 📊 Data
179 +
180 +**Source file:** `data/raw/realtor_mls_unique.duckdb` — a de-duplicated snapshot of
181 +Canadian MLS "for-sale" listings: **172,019 rows × 81 columns** (single table
182 +`listings`), with list price, geocoded coordinates, postal code, and semi-structured
183 +building/lot attributes.
184 +
185 +⚠️ **The raw DuckDB (747 MB) is not distributed in this repository** (GitHub's 100 MB
186 +file limit, plus its content is scraped listing data). It lives on the author's machines;
187 +place it at `data/raw/realtor_mls_unique.duckdb` to run step 01. All downstream artifacts
188 +— including the committed estimation sample — derive from it programmatically.
189 +
190 +Estimation-sample snapshot (Table 1 of the paper): median list price ≈ **$639,888**;
191 +median living area ≈ **135 m²**; 3 bedrooms; 2 full bathrooms. Both price and price/m²
192 +are strongly right-skewed, motivating the log transformation. Ontario, Quebec, BC and
193 +Alberta dominate; PEI and the territories have no listings in this snapshot.
194 +
195 +## 📐 Methodology
196 +
197 +- **Specification ladder** — M1 structural only → M2 + type/ownership → M3 + province FE
198 + (house subsample) → M4 houses + FSA FE → M5 **grand model** (all residential, FSA FE).
199 + The M3→M5 gap ≈ 20 pp of R² *is* the value of resolving location at neighbourhood scale.
200 +- **Absorbing least squares** (`linearmodels.AbsorbingLS`) sweeps out the 1,153 FSA
201 + intercepts without materialising dummies; slopes are numerically identical to
202 + full-dummy OLS. SEs clustered by FSA throughout.
203 +- **Retransformation** — predictions in levels use Duan's (1983) smearing estimator (no
204 + log-normality assumption).
205 +- **Validation** — random 80/20 split, evaluation restricted to FSAs observed in
206 + training (out-of-support FE are not identified); leave-one-province-out CV with
207 + province-specific intercepts; Moran's I with row-standardised k-NN weights (k = 10,
208 + 15,000-listing sample, 199 permutations).
209 +
210 +## 🧭 The two results tiers (reference vs. reproduced)
211 +
212 +The original upstream cleaning code (`RE_DB_QC/hedonic`) was **lost** before this
213 +repository was assembled; `src/wp9/sample.py` is a careful reconstruction from the
214 +paper's own data section, calibrated against every stored output (see `AUDIT.md`). It
215 +reproduces the published sample to **+0.86%** and all headline estimates closely — but
216 +not to the last digit. To keep the published record intact:
217 +
218 +- **`results/reference/`** — the original outputs (verbatim files, plus values
219 + transcribed from the published tables). **Default** source for figures/tables, so the
220 + paper always shows exactly the published numbers.
221 +- **`results/reproduced/`** — regenerated end-to-end by scripts 02–03 on the
222 + reconstructed sample.
223 +
224 +```bash
225 +python3 scripts/04_make_figures.py --results reproduced # opt in to regenerated numbers
226 +python3 scripts/05_make_tables.py --results reproduced
227 +```
228 +
229 +## 🖼️ Figures & tables inventory
230 +
231 +| # | Figure | Shows | Numbers from |
232 +|---|---|---|---|
233 +| 1 | `fig_price_dist` | raw vs. log price distribution | micro sample |
234 +| 2 | `fig_province_ppm2` | median $/m² by province | micro sample |
235 +| 3 | `fig_r2` | R² across the M1–M5 ladder | reference |
236 +| 4 | `fig_forest` | structural implicit prices, 95% CI (M3) | reference |
237 +| 5 | `fig_size_gradient` | median price by size bin × dwelling type | micro sample |
238 +| 6a/6b | `fig_map` / `fig_fsa_map` | listing map & FSA medians, coloured by ln $/m² | micro sample |
239 +| 7 | `fig_premia` | top/bottom 12 FSA premia vs. national median | reference |
240 +| 8 | `fig_decomp` | variance decomposition bar | reference |
241 +| 9 | `fig_nonlinear` | marginal elasticity vs. size (quadratic model) | reproduced band + reference anchor |
242 +| 10 | `fig_gradient` | location premium vs. distance to metro | reproduced |
243 +| 11 | `fig_quantile` | quantile coefficients τ = 0.1…0.9 | reference |
244 +| 12 | `fig_moran` | Moran scatterplots, structural vs. grand | reproduced clouds, reference I |
245 +| 13 | `fig_heterogeneity` | size elasticity by province, 95% CI | reference |
246 +| 14 | `fig_oos` | OOS error buckets (±10/±20%) | reference |
247 +| 15/16 | `fig_fit` / `fig_resid` | predicted vs. actual; residual diagnostics | reproduced |
248 +
249 +(The leave-one-province-out results appear as a table only.)
250 +
251 +Tables (`results/tables/`): `summary_stats`, `regression` (M1/M2/M3/M5),
252 +`robustness` (6 sample cuts), `quantile`, `lopo`, `oos` — all six verified
253 +**numerically identical** to the originally published tables.
254 +
255 +## ✅ Reproduction verification
256 +
257 +Full side-by-side in [`AUDIT.md`](AUDIT.md) §5. Highlights (original → reproduced):
258 +
259 +- Sample: 140,931 → 142,146 (+0.86%); condos match to 3 listings.
260 +- R² ladder: 0.464/0.469/0.567/0.762/0.767 → 0.467/0.470/0.569/0.766/0.770.
261 +- OOS: R² 0.764 → 0.770; median APE 15.8% → 15.6%.
262 +- Moran's I: 0.459→0.082 vs. 0.494→0.072 (same −82~85% conclusion).
263 +- Known gaps (flagged, not hidden): M5 living elasticity 0.547 vs. 0.530; lot-field
264 + coefficients differ (the original free-text lot parser could not be fully recovered);
265 + LOPO mean 0.362 vs. 0.315 with identical province ranking.
266 +
267 +## ⚠️ Limitations
268 +
269 +List prices, not transactions; no construction year / renovation status / interior
270 +quality (absorbed into FSA effects and the residual); lot information sparse and noisy;
271 +a single cross-section — levels, not dynamics. See the paper's conclusion for the
272 +research agenda these imply.
273 +
274 +## 📚 Citation
275 +
276 +```bibtex
277 +@techreport{boucher2026grandhedonic,
278 + author = {Boucher, Simon-Pierre},
279 + title = {A Grand Hedonic Model of the Canadian Housing Market:
280 + Decomposing the Value of Structure and Location across
281 + 140,931 MLS Listings with High-Dimensional Neighbourhood
282 + Fixed Effects},
283 + institution = {Universit\'e du Qu\'ebec en Outaouais,
284 + D\'epartement des sciences administratives},
285 + type = {Working Paper},
286 + number = {9},
287 + year = {2026},
288 + month = {May}
289 +}
290 +```
291 +
292 +## 👤 Author & contact
293 +
294 +**Simon-Pierre Boucher**
295 +Département des sciences administratives, Université du Québec en Outaouais
296 +Gatineau — Pavillon Alexandre-Taché, 283 boulevard Alexandre-Taché, Gatineau (QC) J9A 1L8
297 +
298 +📧 **contact@spboucher.ai** · 🌐 [spboucher.ai](https://www.spboucher.ai)
299 +
300 +All code files carry the header `Author: Simon-Pierre Boucher — contact@spboucher.ai`.
301 +
302 +© 2026 Simon-Pierre Boucher. All rights reserved. The listing data snapshot is not
303 +redistributed; code and paper are shared for research reproducibility.
added data/processed/analysis.parquet +0 −0

Binary file not shown.

added data/raw/README.md +8 −0
@@ -0,0 +1,8 @@
1 +# Raw data
2 +
3 +`realtor_mls_unique.duckdb` (747 MB) — de-duplicated snapshot of Canadian MLS
4 +"for-sale" listings (172,019 rows, table `listings`, 81 columns). This file is
5 +the untouched original input; nothing in the pipeline modifies it.
6 +
7 +Every downstream artifact (the estimation sample, all figures, all tables) is
8 +derived from this file by `scripts/01_build_sample.py` onward.
added figures/fig_decomp.png +0 −0

Binary file not shown.

added figures/fig_fit.png +0 −0

Binary file not shown.

added figures/fig_forest.png +0 −0

Binary file not shown.

added figures/fig_fsa_map.png +0 −0

Binary file not shown.

added figures/fig_gradient.png +0 −0

Binary file not shown.

added figures/fig_heterogeneity.png +0 −0

Binary file not shown.

added figures/fig_map.png +0 −0

Binary file not shown.

added figures/fig_moran.png +0 −0

Binary file not shown.

added figures/fig_nonlinear.png +0 −0

Binary file not shown.

added figures/fig_oos.png +0 −0

Binary file not shown.

added figures/fig_premia.png +0 −0

Binary file not shown.

added figures/fig_price_dist.png +0 −0

Binary file not shown.

added figures/fig_province_ppm2.png +0 −0

Binary file not shown.

added figures/fig_quantile.png +0 −0

Binary file not shown.

added figures/fig_r2.png +0 −0

Binary file not shown.

added figures/fig_resid.png +0 −0

Binary file not shown.

added figures/fig_size_gradient.png +0 −0

Binary file not shown.

added paper/.latexmkrc +6 −0
@@ -0,0 +1,6 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +$pdf_mode = 1;
3 +$pdflatex = 'pdflatex -interaction=nonstopmode -halt-on-error -synctex=1 %O %S';
4 +$bibtex_use = 2;
5 +$clean_ext = 'synctex.gz run.xml bbl bcf fdb_latexmk fls log aux out toc lof lot blg';
6 +@default_files = ('main.tex');
added paper/Makefile +9 −0
@@ -0,0 +1,9 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +all:
3 + latexmk -pdf main.tex
4 +
5 +clean:
6 + latexmk -c
7 +
8 +distclean:
9 + latexmk -C
added paper/main.pdf +0 −0

Binary file not shown.

added paper/main.tex +117 −0
@@ -0,0 +1,117 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +% UQO Working Paper No. 9
4 +% A Grand Hedonic Model of the Canadian Housing Market
5 +%
6 +% Build: latexmk -pdf main.tex (or `make` in this directory)
7 +% Figures are read from ../figures/, tables from ../results/tables/.
8 +% ============================================================================
9 +\documentclass[12pt,letterpaper]{article}
10 +
11 +% ---------------------------------------------------------------- encoding
12 +\usepackage[utf8]{inputenc}
13 +\usepackage[T1]{fontenc}
14 +\usepackage[english]{babel}
15 +
16 +% ---------------------------------------------------------------- layout
17 +\usepackage[letterpaper,margin=1in]{geometry}
18 +\usepackage{setspace}
19 +\onehalfspacing
20 +
21 +% ---------------------------------------------------------------- typography
22 +\usepackage{mathptmx}
23 +\usepackage{microtype}
24 +
25 +% ---------------------------------------------------------------- math
26 +\usepackage{amsmath,amssymb,amsthm}
27 +
28 +% ---------------------------------------------------------------- tables
29 +\usepackage{booktabs}
30 +\usepackage{threeparttable}
31 +\usepackage{makecell}
32 +
33 +% ---------------------------------------------------------------- figures
34 +\usepackage{graphicx}
35 +\usepackage{subcaption}
36 +\graphicspath{{../figures/}{./}}
37 +
38 +% ---------------------------------------------------------------- captions
39 +\usepackage[font=small,labelfont=bf,labelsep=period,justification=justified,singlelinecheck=false]{caption}
40 +
41 +% ---------------------------------------------------------------- colours & links
42 +\usepackage[dvipsnames]{xcolor}
43 +\usepackage[colorlinks=true,linkcolor=NavyBlue,citecolor=NavyBlue,urlcolor=NavyBlue,breaklinks=true]{hyperref}
44 +
45 +% ---------------------------------------------------------------- bibliography
46 +\usepackage[authoryear,round,semicolon]{natbib}
47 +\bibliographystyle{aer}
48 +
49 +% ---------------------------------------------------------------- headings & lists
50 +\usepackage{titlesec}
51 +\titleformat{\section}{\large\bfseries}{\thesection.}{0.5em}{}
52 +\titleformat{\subsection}{\normalsize\bfseries}{\thesubsection.}{0.5em}{}
53 +\titleformat{\subsubsection}{\normalsize\itshape}{\thesubsubsection.}{0.5em}{}
54 +\usepackage{fancyhdr}
55 +\pagestyle{fancy}\fancyhf{}
56 +\renewcommand{\headrulewidth}{0pt}
57 +\fancyfoot[C]{\thepage}
58 +\usepackage{enumitem}
59 +\setlist{nosep,leftmargin=*}
60 +
61 +% ============================================================================
62 +% METADATA
63 +% ============================================================================
64 +\newcommand{\WPnumber}{9}
65 +\newcommand{\WPtitle}{A Grand Hedonic Model of the Canadian Housing Market}
66 +\newcommand{\WPsubtitle}{Decomposing the Value of Structure and Location across
67 + 140{,}931 MLS Listings with High-Dimensional Neighbourhood Fixed Effects}
68 +\newcommand{\WPdate}{May 2026}
69 +\newcommand{\WPversion}{1.1}
70 +\newcommand{\WPkeywords}{Hedonic pricing, Housing markets, Canada, Fixed effects,
71 + Spatial heterogeneity, Automated valuation}
72 +\newcommand{\WPjel}{R31, R21, C21, C55}
73 +
74 +\newcommand{\WPauthor}{Simon-Pierre Boucher}
75 +\newcommand{\WPaffiliation}{D\'epartement des sciences administratives\\
76 + Universit\'e du Qu\'ebec en Outaouais}
77 +\newcommand{\WPemail}{simon-pierre.boucher@uqo.ca}
78 +\newcommand{\WPaddress}{Gatineau -- Pavillon Alexandre-Tach\'e\\
79 + 283, boulevard Alexandre-Tach\'e\\ Gatineau, Qu\'ebec, Canada J9A 1L8}
80 +
81 +\newcommand{\WPabstract}{%
82 +We estimate a large-scale hedonic price model for the Canadian residential real-estate
83 +market using a cross-section of 140{,}931 active MLS listings drawn from the nine provinces
84 +present in the data. A semi-logarithmic specification decomposes dwelling prices into
85 +structural attributes, dwelling type and ownership form, and 1{,}153 neighbourhood
86 +(Forward Sortation Area) fixed effects estimated by absorbing least squares. Moving from a
87 +purely structural model to the full specification raises the explained share of price
88 +variation from 46\% to 77\%, establishing that location is the dominant price determinant
89 +in Canada: neighbourhood effects alone account for roughly thirty percentage points of
90 +explanatory power. The living-area elasticity is 0.51--0.62, each full bathroom commands a
91 +premium of about 11--15\%, and---conditional on floor space---bedroom counts are
92 +economically negligible. The implicit prices are stable across alternative samples and
93 +trimming rules, and the model delivers strong out-of-sample valuation accuracy (held-out
94 +$R^2=0.76$; median absolute error of 16\%). The neighbourhood effects absorb the bulk of
95 +the spatial dependence in prices---Moran's~$I$ falls from 0.46 to 0.08---and we further
96 +document diminishing returns to floor space, an urban price gradient that decays with
97 +distance to major metros, and quantile and cross-province heterogeneity in the implicit
98 +prices. We map the resulting neighbourhood premia, which span a factor of roughly nine
99 +between the most and least expensive areas.}
100 +
101 +\begin{document}
102 +
103 +\input{sections/titlepage}
104 +
105 +\setcounter{page}{1}
106 +\input{sections/introduction}
107 +\input{sections/literature}
108 +\input{sections/data}
109 +\input{sections/methodology}
110 +\input{sections/results}
111 +\input{sections/robustness}
112 +\input{sections/conclusion}
113 +
114 +\newpage
115 +\bibliography{references}
116 +
117 +\end{document}
added paper/references.bib +309 −0
@@ -0,0 +1,309 @@
1 +% ============================================================================
2 +% References — Hedonic Pricing of the Canadian Housing Market
3 +% ============================================================================
4 +
5 +@article{rosen1974hedonic,
6 + author = {Rosen, Sherwin},
7 + title = {Hedonic Prices and Implicit Markets: Product Differentiation in Pure Competition},
8 + journal = {Journal of Political Economy},
9 + volume = {82},
10 + number = {1},
11 + pages = {34--55},
12 + year = {1974}
13 +}
14 +
15 +@article{lancaster1966new,
16 + author = {Lancaster, Kelvin J.},
17 + title = {A New Approach to Consumer Theory},
18 + journal = {Journal of Political Economy},
19 + volume = {74},
20 + number = {2},
21 + pages = {132--157},
22 + year = {1966}
23 +}
24 +
25 +@article{court1939hedonic,
26 + author = {Court, Andrew T.},
27 + title = {Hedonic Price Indexes with Automotive Examples},
28 + journal = {The Dynamics of Automobile Demand, General Motors},
29 + pages = {99--117},
30 + year = {1939}
31 +}
32 +
33 +@article{griliches1961hedonic,
34 + author = {Griliches, Zvi},
35 + title = {Hedonic Price Indexes for Automobiles: An Econometric Analysis of Quality Change},
36 + journal = {The Price Statistics of the Federal Government, NBER},
37 + pages = {173--196},
38 + year = {1961}
39 +}
40 +
41 +@article{halvorsen1980interpretation,
42 + author = {Halvorsen, Robert and Palmquist, Raymond},
43 + title = {The Interpretation of Dummy Variables in Semilogarithmic Equations},
44 + journal = {American Economic Review},
45 + volume = {70},
46 + number = {3},
47 + pages = {474--475},
48 + year = {1980}
49 +}
50 +
51 +@article{sirmans2005composition,
52 + author = {Sirmans, G. Stacy and Macpherson, David A. and Zietz, Emily N.},
53 + title = {The Composition of Hedonic Pricing Models},
54 + journal = {Journal of Real Estate Literature},
55 + volume = {13},
56 + number = {1},
57 + pages = {3--43},
58 + year = {2005}
59 +}
60 +
61 +@incollection{malpezzi2003hedonic,
62 + author = {Malpezzi, Stephen},
63 + title = {Hedonic Pricing Models: A Selective and Applied Review},
64 + booktitle = {Housing Economics and Public Policy},
65 + editor = {O'Sullivan, Tony and Gibb, Kenneth},
66 + publisher = {Blackwell},
67 + pages = {67--89},
68 + year = {2003}
69 +}
70 +
71 +@article{can1992specification,
72 + author = {Can, Ay\c{s}e},
73 + title = {Specification and Estimation of Hedonic Housing Price Models},
74 + journal = {Regional Science and Urban Economics},
75 + volume = {22},
76 + number = {3},
77 + pages = {453--474},
78 + year = {1992}
79 +}
80 +
81 +@book{anselin1988spatial,
82 + author = {Anselin, Luc},
83 + title = {Spatial Econometrics: Methods and Models},
84 + publisher = {Kluwer Academic Publishers},
85 + year = {1988}
86 +}
87 +
88 +@article{bourassa2007spatial,
89 + author = {Bourassa, Steven C. and Cantoni, Eva and Hoesli, Martin},
90 + title = {Spatial Dependence, Housing Submarkets, and House Price Prediction},
91 + journal = {Journal of Real Estate Finance and Economics},
92 + volume = {35},
93 + number = {2},
94 + pages = {143--160},
95 + year = {2007}
96 +}
97 +
98 +@article{case2004modeling,
99 + author = {Case, Bradford and Clapp, John and Dubin, Robin and Rodriguez, Mauricio},
100 + title = {Modeling Spatial and Temporal House Price Patterns: A Comparison of Four Models},
101 + journal = {Journal of Real Estate Finance and Economics},
102 + volume = {29},
103 + number = {2},
104 + pages = {167--191},
105 + year = {2004}
106 +}
107 +
108 +@article{kuminoff2010which,
109 + author = {Kuminoff, Nicolai V. and Parmeter, Christopher F. and Pope, Jaren C.},
110 + title = {Which Hedonic Models Can We Trust to Recover the Marginal Willingness to Pay for Environmental Amenities?},
111 + journal = {Journal of Environmental Economics and Management},
112 + volume = {60},
113 + number = {3},
114 + pages = {145--160},
115 + year = {2010}
116 +}
117 +
118 +@article{glaeser2005why,
119 + author = {Glaeser, Edward L. and Gyourko, Joseph and Saks, Raven},
120 + title = {Why Have Housing Prices Gone Up?},
121 + journal = {American Economic Review},
122 + volume = {95},
123 + number = {2},
124 + pages = {329--333},
125 + year = {2005}
126 +}
127 +
128 +@article{duan1983smearing,
129 + author = {Duan, Naihua},
130 + title = {Smearing Estimate: A Nonparametric Retransformation Method},
131 + journal = {Journal of the American Statistical Association},
132 + volume = {78},
133 + number = {383},
134 + pages = {605--610},
135 + year = {1983}
136 +}
137 +
138 +@article{cameron2015practitioner,
139 + author = {Cameron, A. Colin and Miller, Douglas L.},
140 + title = {A Practitioner's Guide to Cluster-Robust Inference},
141 + journal = {Journal of Human Resources},
142 + volume = {50},
143 + number = {2},
144 + pages = {317--372},
145 + year = {2015}
146 +}
147 +
148 +@article{correia2017reghdfe,
149 + author = {Correia, Sergio},
150 + title = {Linear Models with High-Dimensional Fixed Effects: An Efficient and Feasible Estimator},
151 + journal = {Working Paper},
152 + year = {2017}
153 +}
154 +
155 +@article{gabaix2016power,
156 + author = {Gabaix, Xavier},
157 + title = {Power Laws in Economics: An Introduction},
158 + journal = {Journal of Economic Perspectives},
159 + volume = {30},
160 + number = {1},
161 + pages = {185--206},
162 + year = {2016}
163 +}
164 +
165 +@article{mullainathan2017machine,
166 + author = {Mullainathan, Sendhil and Spiess, Jann},
167 + title = {Machine Learning: An Applied Econometric Approach},
168 + journal = {Journal of Economic Perspectives},
169 + volume = {31},
170 + number = {2},
171 + pages = {87--106},
172 + year = {2017}
173 +}
174 +
175 +@article{cmhc2018canadian,
176 + author = {Mok, Diana and Chan, Sheryl},
177 + title = {House Prices and the Geography of the Canadian Housing Market},
178 + journal = {Canadian Journal of Urban Research},
179 + volume = {27},
180 + number = {1},
181 + pages = {1--18},
182 + year = {2018}
183 +}
184 +
185 +@article{ekeland2004identification,
186 + author = {Ekeland, Ivar and Heckman, James J. and Nesheim, Lars},
187 + title = {Identification and Estimation of Hedonic Models},
188 + journal = {Journal of Political Economy},
189 + volume = {112},
190 + number = {S1},
191 + pages = {S60--S109},
192 + year = {2004}
193 +}
194 +
195 +@article{bajari2005hedonic,
196 + author = {Bajari, Patrick and Benkard, C. Lanier},
197 + title = {Demand Estimation with Heterogeneous Consumers and Unobserved Product Characteristics: A Hedonic Approach},
198 + journal = {Journal of Political Economy},
199 + volume = {113},
200 + number = {6},
201 + pages = {1239--1276},
202 + year = {2005}
203 +}
204 +
205 +@article{black1999better,
206 + author = {Black, Sandra E.},
207 + title = {Do Better Schools Matter? Parental Valuation of Elementary Education},
208 + journal = {Quarterly Journal of Economics},
209 + volume = {114},
210 + number = {2},
211 + pages = {577--599},
212 + year = {1999}
213 +}
214 +
215 +@article{chay2005does,
216 + author = {Chay, Kenneth Y. and Greenstone, Michael},
217 + title = {Does Air Quality Matter? Evidence from the Housing Market},
218 + journal = {Journal of Political Economy},
219 + volume = {113},
220 + number = {2},
221 + pages = {376--424},
222 + year = {2005}
223 +}
224 +
225 +@article{linden2008estimates,
226 + author = {Linden, Leigh and Rockoff, Jonah E.},
227 + title = {Estimates of the Impact of Crime Risk on Property Values from Megan's Laws},
228 + journal = {American Economic Review},
229 + volume = {98},
230 + number = {3},
231 + pages = {1103--1127},
232 + year = {2008}
233 +}
234 +
235 +@book{lesage2009introduction,
236 + author = {LeSage, James and Pace, R. Kelley},
237 + title = {Introduction to Spatial Econometrics},
238 + publisher = {Chapman and Hall/CRC},
239 + year = {2009}
240 +}
241 +
242 +@article{gibbons2012mostly,
243 + author = {Gibbons, Stephen and Overman, Henry G.},
244 + title = {Mostly Pointless Spatial Econometrics?},
245 + journal = {Journal of Regional Science},
246 + volume = {52},
247 + number = {2},
248 + pages = {172--191},
249 + year = {2012}
250 +}
251 +
252 +@article{combes2015empirics,
253 + author = {Combes, Pierre-Philippe and Gobillon, Laurent},
254 + title = {The Empirics of Agglomeration Economies},
255 + journal = {Handbook of Regional and Urban Economics},
256 + volume = {5},
257 + pages = {247--348},
258 + year = {2015}
259 +}
260 +
261 +@article{moran1950notes,
262 + author = {Moran, Patrick A. P.},
263 + title = {Notes on Continuous Stochastic Phenomena},
264 + journal = {Biometrika},
265 + volume = {37},
266 + number = {1/2},
267 + pages = {17--23},
268 + year = {1950}
269 +}
270 +
271 +@article{koenker1978regression,
272 + author = {Koenker, Roger and Bassett, Gilbert},
273 + title = {Regression Quantiles},
274 + journal = {Econometrica},
275 + volume = {46},
276 + number = {1},
277 + pages = {33--50},
278 + year = {1978}
279 +}
280 +
281 +@article{zietz2008determinants,
282 + author = {Zietz, Joachim and Zietz, Emily N. and Sirmans, G. Stacy},
283 + title = {Determinants of House Prices: A Quantile Regression Approach},
284 + journal = {Journal of Real Estate Finance and Economics},
285 + volume = {37},
286 + number = {4},
287 + pages = {317--333},
288 + year = {2008}
289 +}
290 +
291 +@article{mcmillen2010issues,
292 + author = {McMillen, Daniel P. and Redfearn, Christian L.},
293 + title = {Estimation and Hypothesis Testing for Nonparametric Hedonic House Price Functions},
294 + journal = {Journal of Regional Science},
295 + volume = {50},
296 + number = {3},
297 + pages = {712--733},
298 + year = {2010}
299 +}
300 +
301 +@article{pace1998spatiotemporal,
302 + author = {Pace, R. Kelley and Barry, Ronald and Clapp, John M. and Rodriquez, Mauricio},
303 + title = {Spatiotemporal Autoregressive Models of Neighborhood Effects},
304 + journal = {Journal of Real Estate Finance and Economics},
305 + volume = {17},
306 + number = {1},
307 + pages = {15--33},
308 + year = {1998}
309 +}
added paper/sections/conclusion.tex +49 −0
@@ -0,0 +1,49 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Conclusion}
4 +\label{sec:conclusion}
5 +% ============================================================================
6 +
7 +Using a nationwide cross-section of 140{,}931 MLS listings and 1{,}153 absorbed
8 +neighbourhood fixed effects, we estimate a grand hedonic model of the Canadian residential
9 +housing market. The exercise yields three robust conclusions. First, location dominates:
10 +resolving geography from the provincial to the neighbourhood scale raises explained price
11 +variation from 57\% to 77\%, and location as a whole accounts for roughly thirty
12 +percentage points of $R^2$---more than every structural attribute combined. Second, the
13 +structural implicit prices behave exactly as hedonic theory predicts and are strikingly
14 +stable across samples: living area carries an elasticity near 0.55, each full bathroom
15 +adds about 11--15\%, and bedroom counts are economically negligible once floor space is
16 +held fixed. Third, the neighbourhood premia are vast---a factor of roughly nine separates
17 +the most and least expensive FSAs---and the model translates into a credible valuation
18 +tool, predicting held-out prices with an out-of-sample $R^2$ of 0.76 and a median error of
19 +16\%. These conclusions survive an extensive battery of checks: the implicit prices are
20 +stable across subsamples and across the price distribution, the neighbourhood effects
21 +absorb 82\% of the spatial autocorrelation in residuals, floor space displays diminishing
22 +returns, value decays with distance to major metros, and the structural prices transfer
23 +across provinces in leave-one-province-out cross-validation.
24 +
25 +Several caveats temper these conclusions. The prices are \emph{list} prices rather than
26 +closing prices and may embed listing strategy and market tightness. The data lack year of
27 +construction, renovation status and interior quality, so the neighbourhood effects absorb
28 +some dwelling-level quality that is correlated with location; the within-FSA structural
29 +estimates are nonetheless purged of the cross-neighbourhood component of this confound.
30 +Lot information is sparse and noisily reported, limiting the precision of the land-value
31 +component. Finally, the analysis is a cross-section and characterises the spatial and
32 +structural \emph{level} of prices in the current market rather than their dynamics.
33 +
34 +These limitations chart a natural research agenda: linking listings to closing prices and
35 +time-on-market, enriching the attribute set with age and quality, decomposing the
36 +neighbourhood premia into capitalized amenities such as schools, transit and environmental
37 +quality \citep{black1999better,chay2005does}, modelling the modest residual spatial
38 +dependence explicitly \citep{lesage2009introduction}, and extending the cross-section to a
39 +panel to study price dynamics and the incidence of policy shocks. Even in its present
40 +form, the model provides a transparent, reproducible benchmark for automated valuation,
41 +market monitoring and the welfare analysis of local amenities across the Canadian
42 +residential market.
43 +
44 +\vspace{0.6em}
45 +\noindent\textbf{Reproducibility.} Data engineering was performed in \texttt{DuckDB} and
46 +\texttt{pandas}; estimation used \texttt{statsmodels} (cluster-robust OLS) and
47 +\texttt{linearmodels} (absorbing least squares); figures were produced in
48 +\texttt{matplotlib}. All tables and figures are generated programmatically from the source
49 +database by the numbered scripts in the accompanying repository.
added paper/sections/data.tex +77 −0
@@ -0,0 +1,77 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Data}
4 +\label{sec:data}
5 +% ============================================================================
6 +
7 +\subsection{Source and coverage}
8 +\label{subsec:data_sources}
9 +
10 +The analysis draws on a de-duplicated snapshot of the Canadian Multiple Listing Service
11 +(MLS), \texttt{realtor\_mls\_unique.duckdb}, containing 172{,}019 unique active
12 +``for-sale'' listings. Each record carries the list price, a structured set of building
13 +and lot attributes, ownership and dwelling-type fields, and---crucially for this
14 +study---a geocoded location (latitude, longitude and postal code). Geographic coverage is
15 +essentially complete: coordinates and postal codes are present for more than 99.9\% of
16 +records, so every listing can be mapped to its Forward Sortation Area (FSA), the first
17 +three characters of the Canadian postal code and our unit of neighbourhood.
18 +
19 +\subsection{Sample construction and variable parsing}
20 +\label{subsec:variables}
21 +
22 +The raw fields are semi-structured and require parsing. Bedroom counts reported as
23 +``3~+~1'' (main plus lower level) are summed. Living areas, reported inconsistently in
24 +square feet or square metres, are harmonised to square metres (preferring the explicit
25 +floor-area measurement and converting from square feet at
26 +$1\text{ ft}^2 = 0.0929\text{ m}^2$). Lot sizes are parsed from free-text fields into
27 +square metres where a numeric value can be recovered, with a missingness indicator
28 +retained. We restrict the sample to residential dwellings (houses, condominiums, plexes,
29 +townhouses and apartments) with a strictly positive list price and non-missing core
30 +structural fields, and we trim the extreme 1\% tails of price and living area to limit the
31 +influence of data-entry errors and ultra-luxury outliers. The resulting estimation sample
32 +contains \textbf{140{,}931 dwellings}, of which 82{,}334 are houses and 57{,}857 are
33 +condominiums.
34 +
35 +The dependent variable is the natural logarithm of the list price. Structural regressors
36 +are: $\ln$ living area; counts of bedrooms, full bathrooms and half bathrooms; parking
37 +spaces; storeys; and an indicator for the availability of lot information together with
38 +$\ln(1+\text{lot area})$. Categorical controls comprise the dwelling type (house,
39 +apartment/condo, row/townhouse, duplex, triplex, fourplex, manufactured home, other), the
40 +ownership form (freehold, condominium/strata, leasehold, etc.), and the broad listing
41 +category. The neighbourhood fixed effect is the FSA; FSAs with fewer than 25 listings are
42 +pooled into a province-level residual category so that each absorbed effect is estimated
43 +from a reasonable number of observations.
44 +
45 +\subsection{Descriptive statistics}
46 +\label{subsec:summary_stats}
47 +
48 +Table~\ref{tab:summary_stats} reports the descriptive statistics. The median dwelling
49 +lists for roughly \$565{,}000 and offers about 125~m$^2$ of living space, two full
50 +bathrooms and three bedrooms. Both price and price per square metre are strongly
51 +right-skewed---means exceed medians throughout---motivating the log transformation
52 +illustrated in Figure~\ref{fig:price}, whose right panel is approximately symmetric and
53 +underpins the semi-logarithmic hedonic specification.
54 +
55 +\input{../results/tables/summary_stats}
56 +
57 +\begin{figure}[t]\centering
58 +\includegraphics[width=\textwidth]{fig_price_dist.png}
59 +\caption{Distribution of list prices. Panel (a) shows the raw price (truncated at \$3M for
60 +readability) with the sample median marked; panel (b) shows log price, the dependent
61 +variable, which is close to symmetric.}
62 +\label{fig:price}
63 +\end{figure}
64 +
65 +The provincial composition is dominated by Ontario, Quebec, British Columbia and Alberta,
66 +which together account for the large majority of listings; the Atlantic provinces and the
67 +Prairies are present but thinner, and Prince Edward Island and the territories contain no
68 +listings in this snapshot. Median price per square metre ranges from roughly \$2{,}000 in
69 +Newfoundland and Labrador to over \$6{,}100 in British Columbia
70 +(Figure~\ref{fig:province}), foreshadowing the dominant role of location documented below.
71 +
72 +\begin{figure}[t]\centering
73 +\includegraphics[width=0.82\textwidth]{fig_province_ppm2.png}
74 +\caption{Median price per square metre of living area by province. Grey extensions show
75 +the mean--median gap, reflecting right-skew from high-value listings.}
76 +\label{fig:province}
77 +\end{figure}
added paper/sections/introduction.tex +71 −0
@@ -0,0 +1,71 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Introduction}
4 +\label{sec:introduction}
5 +% ============================================================================
6 +
7 +A dwelling is the archetypal heterogeneous good. No two houses are identical: they differ
8 +in floor space, in the number of bathrooms, in tenure, and---above all---in where they
9 +stand. The hedonic approach pioneered by \citet{lancaster1966new} and
10 +\citet{rosen1974hedonic} treats such a good as a bundle of attributes, each commanding an
11 +implicit price determined in equilibrium by the interaction of buyers' marginal
12 +willingness to pay and sellers' marginal cost of supply. Regressing the (log) price of a
13 +dwelling on its measurable characteristics recovers these implicit prices, and the
14 +resulting estimates provide the empirical backbone for house-price indices, property
15 +assessment, mortgage valuation, and the welfare analysis of local amenities.
16 +
17 +This paper estimates a hedonic price model for the Canadian residential market at national
18 +scale. Using a cross-section of \textbf{140{,}931} active MLS listings spanning the nine
19 +provinces present in the data---from British Columbia to Newfoundland and Labrador---we
20 +decompose dwelling prices into structural attributes, dwelling type and ownership form,
21 +and a rich set of \textbf{1{,}153 neighbourhood fixed effects} defined at the Forward
22 +Sortation Area (FSA) level. To our knowledge this is among the most geographically
23 +comprehensive single-equation hedonic exercises assembled for Canada. It is made feasible
24 +by an absorbing least-squares estimator that sweeps out the high-dimensional location
25 +effects without materialising thousands of dummy variables.
26 +
27 +The motivation is threefold. First, the relative importance of \emph{structure} versus
28 +\emph{location} in house prices is a first-order empirical question with direct
29 +implications for assessment and policy, yet credible national estimates for Canada are
30 +scarce. Second, the granularity of modern listing data---geocoded to the dwelling and
31 +covering the whole country---permits a clean within-neighbourhood identification of
32 +structural implicit prices that earlier, coarser data could not support. Third, a
33 +transparent, reproducible hedonic benchmark is a useful yardstick against which more
34 +complex machine-learning valuation models can be judged \citep{mullainathan2017machine}.
35 +
36 +Our main findings can be summarised as follows. The purely structural model explains 46\%
37 +of the variation in log prices; adding province fixed effects raises this to 57\%, and
38 +replacing them with FSA fixed effects lifts it to \textbf{77\%}. Neighbourhood location
39 +\emph{alone} therefore accounts for roughly thirty percentage points of explanatory
40 +power---more than all structural attributes combined. Conditional on location, the
41 +living-area elasticity is estimated at 0.51--0.62: a 10\% larger dwelling sells for about
42 +5--6\% more. Each additional full bathroom commands a premium of 11--15\%, whereas the
43 +number of bedrooms is economically negligible once floor space is held fixed---a classic
44 +hedonic result. The implicit prices are remarkably stable across alternative samples,
45 +trimming rules and dwelling types, and the model attains an out-of-sample $R^2$ of 0.76
46 +with a median absolute valuation error of 16\%, competitive with commercial automated
47 +valuation models. We map the estimated neighbourhood premia and show that the
48 +highest-valued FSAs---concentrated in the City of Vancouver and the Greater Toronto
49 +Area---trade at more than triple the national-median level net of structure. A battery of
50 +additional analyses sharpens the picture: the neighbourhood effects absorb 82\% of the
51 +spatial autocorrelation in raw residuals (Moran's~$I$ falls from 0.46 to 0.08); floor
52 +space exhibits clear diminishing returns; the location component of price decays with
53 +distance to the nearest major metropolis; and the implicit prices vary sensibly across the
54 +price distribution and across provinces while transferring well out-of-region.
55 +
56 +This paper relates to several strands of the literature. It builds directly on the hedonic
57 +tradition of \citet{rosen1974hedonic}, \citet{griliches1961hedonic} and the applied
58 +syntheses of \citet{sirmans2005composition} and \citet{malpezzi2003hedonic}. It connects
59 +to the spatial-econometric treatment of housing of \citet{can1992specification},
60 +\citet{anselin1988spatial} and \citet{bourassa2007spatial}, and to the literature on the
61 +spatial structure of house prices \citep{case2004modeling,glaeser2005why}.
62 +Methodologically it draws on high-dimensional fixed-effects estimation
63 +\citep{correia2017reghdfe}, cluster-robust inference \citep{cameron2015practitioner}, and
64 +retransformation from logs \citep{duan1983smearing}.
65 +
66 +The remainder of the paper is organised as follows. Section~\ref{sec:literature} reviews
67 +the related literature. Section~\ref{sec:data} describes the data and variable
68 +construction. Section~\ref{sec:methodology} presents the empirical strategy.
69 +Section~\ref{sec:results} reports the main results. Section~\ref{sec:robustness} provides
70 +robustness checks, heterogeneity analyses, and out-of-sample validation.
71 +Section~\ref{sec:conclusion} concludes.
added paper/sections/literature.tex +84 −0
@@ -0,0 +1,84 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Related Literature}
4 +\label{sec:literature}
5 +% ============================================================================
6 +
7 +\subsection{The hedonic framework}
8 +
9 +The hedonic method has two intellectual roots. \citet{lancaster1966new} recast consumer
10 +theory in terms of the characteristics embodied in goods rather than the goods themselves,
11 +while \citet{court1939hedonic} and \citet{griliches1961hedonic} developed the empirical
12 +machinery of quality-adjusted price indices. \citet{rosen1974hedonic} unified these ideas
13 +in an equilibrium model in which the observed price schedule traces out the envelope of
14 +buyers' bid functions and sellers' offer functions; its gradient with respect to a
15 +characteristic identifies, at the margin, the implicit price of that characteristic. The
16 +first-stage hedonic regression---the object of this paper---recovers these marginal
17 +implicit prices under weak assumptions and remains the workhorse for valuation, even where
18 +recovery of the deep structural demand parameters is contested. A subsequent literature
19 +has clarified exactly what the second stage can and cannot identify:
20 +\citet{ekeland2004identification} show that nonlinearity of the hedonic price function
21 +aids identification of preferences, \citet{bajari2005hedonic} develop a tractable demand
22 +estimator with unobserved product characteristics, and \citet{kuminoff2010which} document
23 +how sensitive welfare estimates are to specification. We remain at the first stage and
24 +target the implicit-price schedule itself, which is the relevant object for valuation and
25 +assessment.
26 +
27 +\subsection{Specification of housing hedonics}
28 +
29 +A large applied literature studies which attributes belong in a housing hedonic and what
30 +functional form to impose. \citet{sirmans2005composition} catalogue the regressors used in
31 +decades of published models and document the central role of living area, bathrooms, lot
32 +size, age and location. \citet{malpezzi2003hedonic} surveys functional-form choices and
33 +argues that the semi-logarithmic specification---log price on linear (or log)
34 +characteristics---is a robust default: it accommodates the right-skew of prices, yields
35 +coefficients interpretable as approximate percentage effects
36 +\citep{halvorsen1980interpretation}, and mitigates heteroskedasticity. We adopt this
37 +specification throughout. A complementary literature relaxes linearity:
38 +\citet{mcmillen2010issues} estimate fully nonparametric hedonic surfaces, and
39 +quantile-regression approaches \citep{koenker1978regression,zietz2008determinants} show
40 +that implicit prices differ systematically along the price distribution---a feature we
41 +document for the Canadian market in Section~\ref{subsec:quantile}.
42 +
43 +\subsection{Amenity capitalization}
44 +
45 +A large share of the value of location reflects the capitalization of local public goods
46 +and disamenities into prices. Quasi-experimental hedonic studies have measured the
47 +capitalization of school quality \citep{black1999better}, air quality \citep{chay2005does}
48 +and local crime risk \citep{linden2008estimates}. Our neighbourhood fixed effects
49 +deliberately bundle all such capitalized amenities into a single FSA-level premium rather
50 +than attempting to disentangle them; the premium therefore provides an upper envelope on
51 +the value of location that future work can decompose.
52 +
53 +\subsection{Space in housing models}
54 +
55 +Because housing is immobile, location is intrinsic to its value, and unobserved local
56 +amenities induce strong spatial dependence in prices. \citet{can1992specification} and
57 +\citet{anselin1988spatial} formalise spatial autocorrelation in hedonic errors, while
58 +\citet{bourassa2007spatial} and \citet{case2004modeling} show that submarket or
59 +spatial-fixed-effect controls substantially improve both fit and prediction. Our approach
60 +is deliberately non-parametric in space: rather than imposing a spatial weight matrix, we
61 +absorb a fixed effect for each of 1{,}153 FSA neighbourhoods, allowing the data to assign
62 +an arbitrary location premium to each area and identifying structural implicit prices from
63 +\emph{within-neighbourhood} variation. This is the housing analogue of the
64 +high-dimensional fixed-effects designs now standard in applied microeconomics
65 +\citep{correia2017reghdfe}, with inference clustered at the neighbourhood level to respect
66 +within-area correlation \citep{cameron2015practitioner}. The choice also speaks to a
67 +methodological debate: \citet{gibbons2012mostly} caution that parametric spatial-lag
68 +models are often hard to interpret causally and that flexible fixed effects are frequently
69 +preferable, while \citet{lesage2009introduction} and \citet{pace1998spatiotemporal}
70 +develop the spatial-autoregressive alternative. We side with the fixed-effects approach
71 +but validate it directly by testing for residual spatial autocorrelation with Moran's~$I$
72 +\citep{moran1950notes} in Section~\ref{subsec:spatial}. The dominance of location that we
73 +document is also consistent with the agglomeration literature, which links density and
74 +proximity to elevated land values \citep{combes2015empirics}.
75 +
76 +\subsection{House prices in Canada and valuation}
77 +
78 +The spatial dispersion of Canadian house prices---with Vancouver and Toronto at the
79 +expensive extreme---has attracted both academic and policy attention
80 +\citep{glaeser2005why,cmhc2018canadian}. We contribute a unified, nationwide hedonic
81 +benchmark estimated on a single consistent dataset. Finally, the rise of automated
82 +valuation models motivates pairing transparent hedonic estimates with predictive
83 +performance metrics \citep{mullainathan2017machine}; we therefore report out-of-sample
84 +accuracy alongside in-sample implicit prices.
added paper/sections/methodology.tex +87 −0
@@ -0,0 +1,87 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Empirical Strategy}
4 +\label{sec:methodology}
5 +% ============================================================================
6 +
7 +\subsection{The hedonic equation}
8 +\label{subsec:model}
9 +
10 +We estimate the semi-logarithmic hedonic price equation
11 +%
12 +\begin{equation}
13 + \ln P_{i}
14 + = \alpha
15 + + \beta\,\ln(\text{Area}_{i})
16 + + \mathbf{x}_{i}'\boldsymbol{\gamma}
17 + + \mathbf{d}_{i}'\boldsymbol{\delta}
18 + + \mu_{f(i)}
19 + + \varepsilon_{i},
20 + \label{eq:hedonic}
21 +\end{equation}
22 +%
23 +where $P_{i}$ is the list price of dwelling~$i$; $\text{Area}_{i}$ is living area in
24 +square metres; $\mathbf{x}_{i}$ collects the remaining structural attributes (bedrooms,
25 +full and half bathrooms, parking, storeys, the lot indicator and $\ln$ lot area);
26 +$\mathbf{d}_{i}$ is a vector of dwelling-type and ownership dummies; $\mu_{f(i)}$ is a
27 +fixed effect for the FSA neighbourhood $f$ to which dwelling~$i$ belongs; and
28 +$\varepsilon_{i}$ is an idiosyncratic error.
29 +
30 +Because $\beta$ multiplies a logged regressor, it is the \emph{elasticity} of price with
31 +respect to floor space. The elements of $\boldsymbol{\gamma}$ attached to count variables
32 +are semi-elasticities: following \citet{halvorsen1980interpretation}, a coefficient
33 +$\gamma$ implies an approximate proportional price change of $100\,(e^{\gamma}-1)\%$ for a
34 +one-unit increase, which we report whenever the distinction from the raw coefficient is
35 +material.
36 +
37 +\subsection{High-dimensional location effects}
38 +\label{subsec:identification}
39 +
40 +The neighbourhood fixed effects $\mu_{f}$ are the heart of the design. By including a
41 +separate intercept for each of the 1{,}153 FSAs, we allow every neighbourhood an arbitrary
42 +price level that absorbs all location-specific amenities---school quality, transit access,
43 +coastline, employment density---whether or not they are observed. The structural implicit
44 +prices in $\beta$ and $\boldsymbol{\gamma}$ are then identified purely from variation
45 +\emph{within} neighbourhoods, comparing dwellings that differ in their physical attributes
46 +but share a location. This is the housing counterpart of the within estimator, and it
47 +addresses the most pernicious source of omitted-variable bias in hedonic work: the
48 +correlation between structural quality and unobserved locational quality
49 +\citep{can1992specification,bourassa2007spatial}.
50 +
51 +Including more than a thousand dummies directly is numerically wasteful. We instead
52 +estimate Equation~\eqref{eq:hedonic} by \emph{absorbing least squares}, which partials the
53 +FSA effects out of both the dependent variable and the regressors before estimating the
54 +structural coefficients \citep{correia2017reghdfe}; the slope estimates are numerically
55 +identical to full-dummy OLS. All standard errors are clustered at the FSA level to allow
56 +for arbitrary within-neighbourhood correlation and heteroskedasticity
57 +\citep{cameron2015practitioner}.
58 +
59 +\subsection{Specification ladder}
60 +\label{subsec:ladder}
61 +
62 +We report a ladder of nested specifications that isolates the marginal contribution of
63 +each block of controls:
64 +%
65 +\begin{itemize}
66 + \item \textbf{M1 -- Structural:} structural attributes only.
67 + \item \textbf{M2 -- + Type/Ownership:} adds dwelling-type and ownership dummies.
68 + \item \textbf{M3 -- + Province:} adds province fixed effects.
69 + \item \textbf{M4 -- Houses + FSA:} replaces province with FSA fixed effects, houses only.
70 + \item \textbf{M5 -- Grand model:} FSA fixed effects over all residential dwellings.
71 +\end{itemize}
72 +%
73 +Columns M1--M3 are estimated on the house subsample to keep the structural interpretation
74 +clean; M5 is the preferred grand specification estimated over the full residential sample.
75 +The gap in $R^2$ between M3 and M5 measures the explanatory value of resolving location at
76 +the neighbourhood rather than the provincial scale.
77 +
78 +\subsection{Retransformation and out-of-sample evaluation}
79 +\label{subsec:retrans}
80 +
81 +Because the model is estimated in logs, predicted price levels require a retransformation
82 +correction. We use Duan's smearing estimator \citep{duan1983smearing}, multiplying
83 +$\exp(\widehat{\ln P})$ by the sample mean of $\exp(\widehat{\varepsilon})$, which is
84 +consistent without assuming log-normal errors. For predictive validation we randomly split
85 +the data 80/20, estimate the model on the training fold (restricting evaluation to
86 +neighbourhoods observed in training, since out-of-support FSA effects are not identified),
87 +and report held-out fit and percentage-error metrics on the test fold.
added paper/sections/results.tex +179 −0
@@ -0,0 +1,179 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Results}
4 +\label{sec:results}
5 +% ============================================================================
6 +
7 +\subsection{The value of location}
8 +\label{subsec:location}
9 +
10 +Figure~\ref{fig:r2} summarises the explanatory power of the specification ladder. The
11 +structural-only model (M1) accounts for 46.4\% of the variation in log prices. Adding
12 +dwelling-type and ownership controls (M2) barely moves the fit, but introducing province
13 +fixed effects (M3) raises $R^2$ to 56.8\%, and resolving location at the FSA scale lifts
14 +it to \textbf{76.2\%} for houses (M4) and \textbf{76.7\%} for the grand model (M5). The
15 +implication is stark: moving from province to neighbourhood resolution adds about twenty
16 +percentage points of explained variance, and \emph{location as a whole accounts for
17 +roughly thirty percentage points}---more than the entire structural bundle. This is the
18 +central result of the paper and a quantitative statement of the realtor's adage that what
19 +matters is ``location, location, location.''
20 +
21 +\begin{figure}[t]\centering
22 +\includegraphics[width=0.82\textwidth]{fig_r2.png}
23 +\caption{Share of log-price variation explained ($R^2$) across the five nested
24 +specifications. The jump from M3 (province) to M4/M5 (FSA) quantifies the value of
25 +resolving location at the neighbourhood scale.}
26 +\label{fig:r2}
27 +\end{figure}
28 +
29 +\subsection{Implicit prices of structural attributes}
30 +\label{subsec:implicit}
31 +
32 +Table~\ref{tab:regression} reports the regression estimates. The living-area elasticity is
33 +remarkably stable across specifications, at $0.66$ in the raw structural model and
34 +settling near $0.55$ once location is controlled for: a 10\% larger dwelling commands
35 +roughly a 5.5\% higher price. The slight decline as controls are added is consistent with
36 +larger homes being located in more expensive areas---a confound the FSA effects remove.
37 +
38 +Full bathrooms carry one of the strongest structural premia: about $0.11$ log points in
39 +the grand model, or roughly an \textbf{11\% price increase} per additional bathroom,
40 +rising to nearly 15\% in the province-FE model. Half bathrooms attract a small
41 +\emph{negative} conditional coefficient, which we read not as a disamenity but as a proxy
42 +for older or more compartmentalised floor plans once total area and full baths are held
43 +fixed. Bedrooms are economically negligible conditional on living area: holding floor
44 +space constant, subdividing it into more bedrooms does not raise value---a textbook
45 +hedonic finding that recurs across our samples. Lot information enters positively ($\ln$
46 +lot elasticity around $0.03$) but modestly, reflecting both the noisiness of the parsed
47 +lot field and the fact that, within a neighbourhood, lot variation is compressed.
48 +
49 +Figure~\ref{fig:forest} presents the structural implicit prices with 95\% cluster-robust
50 +confidence intervals, making visually plain the dominance of living area and bathrooms and
51 +the near-zero conditional effects of bedrooms, parking and storeys.
52 +
53 +\input{../results/tables/regression}
54 +
55 +\begin{figure}[t]\centering
56 +\includegraphics[width=0.82\textwidth]{fig_forest.png}
57 +\caption{Marginal implicit prices of structural attributes (houses, province-FE model M3)
58 +with 95\% cluster-robust confidence intervals, on the log-price scale.}
59 +\label{fig:forest}
60 +\end{figure}
61 +
62 +The price--size gradient by dwelling type (Figure~\ref{fig:size_gradient}) confirms the
63 +log--log structure: median price rises concavely with living area for both dwelling types.
64 +Unconditionally, condominiums list \emph{above} houses of the same size---a compositional
65 +effect of their concentration in the expensive metropolitan markets---whereas conditional
66 +on location and ownership the estimated dwelling-type effects show that houses command the
67 +premium, in line with Table~\ref{tab:regression}.
68 +
69 +\begin{figure}[t]\centering
70 +\includegraphics[width=0.78\textwidth]{fig_size_gradient.png}
71 +\caption{Median price by living-area bin and dwelling type. The concave gradient is
72 +linearised by the semi-log specification.}
73 +\label{fig:size_gradient}
74 +\end{figure}
75 +
76 +\subsection{The geography of housing value}
77 +\label{subsec:geography}
78 +
79 +Figure~\ref{fig:maps} maps the spatial structure of value directly. The familiar outline
80 +of populated Canada emerges from the listing coordinates. The Vancouver corridor and the
81 +Greater Toronto--Golden Horseshoe area sit at the top of the price-per-square-metre
82 +distribution, while the Prairies and Atlantic Canada anchor the bottom. The right panel
83 +aggregates to FSA medians, the geographic unit absorbed in the grand model.
84 +
85 +\begin{figure}[t]\centering
86 +\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{fig_map.png}
87 +\caption{All listings}\end{subfigure}\hfill
88 +\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{fig_fsa_map.png}
89 +\caption{FSA neighbourhood medians}\end{subfigure}
90 +\caption{Spatial distribution of housing value. Colour encodes log price per m$^2$; bubble
91 +area in panel~(b) is proportional to $\sqrt{\text{listings}}$. Labels mark the provinces
92 +with substantial samples; the data cover nine provinces, while the territories and Prince
93 +Edward Island contain no listings.}
94 +\label{fig:maps}
95 +\end{figure}
96 +
97 +To translate location into a clean dollar statement we recover each FSA's fixed effect
98 +from the grand model---its price premium net of structure, dwelling type and
99 +ownership---and express it relative to the national median (Figure~\ref{fig:premia}). The
100 +highest-valued neighbourhoods, all in the City of Vancouver, trade at \textbf{150--200\%
101 +above} the national-median neighbourhood for an otherwise identical dwelling; the lowest,
102 +in rural Saskatchewan, Manitoba and Newfoundland, sit \textbf{60--67\% below}. The full
103 +premium distribution thus spans a factor of roughly nine between the most and least
104 +expensive neighbourhoods, dwarfing the price range attributable to any single structural
105 +attribute.
106 +
107 +\begin{figure}[t]\centering
108 +\includegraphics[width=0.78\textwidth]{fig_premia.png}
109 +\caption{Highest- and lowest-valued neighbourhoods (FSAs) in Canada, measured as the
110 +estimated location premium relative to the national-median neighbourhood, net of
111 +structure, dwelling type and ownership. Only FSAs with at least 50 listings are shown.}
112 +\label{fig:premia}
113 +\end{figure}
114 +
115 +\subsection{Decomposing the variance of prices}
116 +\label{subsec:decomp}
117 +
118 +Figure~\ref{fig:decomp} casts the specification ladder as a decomposition of the variance
119 +of log prices into the share explained by each successive block of controls. Physical
120 +structure accounts for 46\% of the variance; dwelling type and ownership add a further
121 +0.4~points; province adds about 10~points; and resolving location to the neighbourhood
122 +adds a further 20~points, for a total location contribution near 30~points. Just under a
123 +quarter of the variance remains unexplained and is attributable to idiosyncratic pricing
124 +and unobserved dwelling quality. The visual makes the headline unmistakable: the single
125 +largest identified block of housing value in Canada is the neighbourhood.
126 +
127 +\begin{figure}[t]\centering
128 +\includegraphics[width=0.92\textwidth]{fig_decomp.png}
129 +\caption{Variance decomposition of Canadian log house prices into structure, dwelling
130 +type/ownership, province, neighbourhood (FSA) and the unexplained residual, from the
131 +nested specification ladder.}
132 +\label{fig:decomp}
133 +\end{figure}
134 +
135 +\subsection{Nonlinearity: diminishing returns to floor space}
136 +\label{subsec:nonlinear}
137 +
138 +The constant-elasticity assumption is convenient but restrictive. Re-estimating the grand
139 +model with a quadratic in log living area yields a positive linear term and a
140 +significantly negative quadratic term ($\widehat{\beta}_1=1.06$,
141 +$\widehat{\beta}_2=-0.053$), implying that the marginal elasticity of price with respect
142 +to floor space \emph{declines} with dwelling size. Figure~\ref{fig:nonlinear} traces the
143 +implied marginal elasticity: it falls from roughly $0.65$ for a compact 60~m$^2$ dwelling
144 +to about $0.45$ for a large 350~m$^2$ home. Economically, the first square metres of
145 +living space are valued most highly and additional space is subject to diminishing
146 +returns---consistent with the nonparametric hedonic surfaces of \citet{mcmillen2010issues}.
147 +The constant-elasticity estimate of $0.55$ is best read as an average over the size
148 +distribution.
149 +
150 +\begin{figure}[t]\centering
151 +\includegraphics[width=0.78\textwidth]{fig_nonlinear.png}
152 +\caption{Marginal elasticity of price with respect to living area as a function of
153 +dwelling size, from a grand model with a quadratic in log area (95\% cluster-robust band).
154 +The dashed line is the constant-elasticity estimate.}
155 +\label{fig:nonlinear}
156 +\end{figure}
157 +
158 +\subsection{The urban price gradient}
159 +\label{subsec:gradient}
160 +
161 +Classic urban theory predicts that, holding structure fixed, value declines with distance
162 +from employment centres. We compute the great-circle distance from each listing to the
163 +nearest of nine major Canadian metropolitan centres and relate it to the location
164 +component of price (the residual from a structure-only model). Figure~\ref{fig:gradient}
165 +confirms a pronounced gradient: dwellings within the metropolitan core carry location
166 +premia of tens of percent over the structure-only benchmark, and the premium decays
167 +steadily with distance, turning negative beyond roughly 80~km. A log-linear fit implies a
168 +semi-elasticity of $-0.085$ ($t>100$): a doubling of distance to the nearest metro is
169 +associated with an 8.5\% lower location premium. This agglomeration gradient is precisely
170 +the force that the FSA fixed effects absorb non-parametrically in the grand model
171 +\citep{combes2015empirics}.
172 +
173 +\begin{figure}[t]\centering
174 +\includegraphics[width=0.78\textwidth]{fig_gradient.png}
175 +\caption{Urban price gradient: the location component of price (relative to a
176 +structure-only benchmark) against distance to the nearest of nine major Canadian metros.
177 +The horizontal axis is on a symmetric-log scale.}
178 +\label{fig:gradient}
179 +\end{figure}
added paper/sections/robustness.tex +151 −0
Large diff (151 lines) is not loaded by default to keep the page fast.Load diff
added paper/sections/titlepage.tex +70 −0
Large diff (70 lines) is not loaded by default to keep the page fast.Load diff
added paper/uq_logo.jpg +0 −0

Binary file not shown.

added requirements.txt +10 −0
Large diff (10 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/coef_M1.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/coef_M2.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/coef_M3.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/coef_M5.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/ext2.json +21 −0
Large diff (22 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/fit.json +22 −0
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/fsa_premia.csv +868 −0
Large diff (868 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/heterogeneity.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/lopo.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/oos.json +11 −0
Large diff (12 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/quantile.csv +7 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/robustness.csv +7 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
added results/reference/summary_stats.csv +10 −0
Large diff (10 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/coef_M1.csv +10 −0
Large diff (10 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/coef_M2.csv +15 −0
Large diff (15 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/coef_M3.csv +23 −0
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/coef_M5.csv +26 −0
Large diff (26 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/ext2.json +21 −0
Large diff (22 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/fit.json +22 −0
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/fsa_premia.csv +870 −0
Large diff (870 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/gradient_bins.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/grand_model.parquet +0 −0

Binary file not shown.

added results/reproduced/heterogeneity.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/lopo.csv +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/moran_coords.npy +0 −0

Binary file not shown.

added results/reproduced/moran_resid_grand.npy +0 −0

Binary file not shown.

added results/reproduced/moran_resid_struct.npy +0 −0

Binary file not shown.

added results/reproduced/nonlinear_band.csv +61 −0
Large diff (61 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/oos.json +11 −0
Large diff (12 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/quantile.csv +7 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
added results/reproduced/robustness.csv +7 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/lopo.tex +16 −0
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/oos.tex +22 −0
Large diff (22 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/quantile.tex +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/regression.tex +30 −0
Large diff (30 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/robustness.tex +20 −0
Large diff (20 lines) is not loaded by default to keep the page fast.Load diff
added results/tables/summary_stats.tex +23 −0
Large diff (23 lines) is not loaded by default to keep the page fast.Load diff
added scripts/01_build_sample.py +29 −0
Large diff (29 lines) is not loaded by default to keep the page fast.Load diff
added scripts/02_estimate_core.py +62 −0
Large diff (62 lines) is not loaded by default to keep the page fast.Load diff
added scripts/03_estimate_extended.py +281 −0
Large diff (281 lines) is not loaded by default to keep the page fast.Load diff
added scripts/04_make_figures.py +379 −0
Large diff (379 lines) is not loaded by default to keep the page fast.Load diff
added scripts/05_make_tables.py +233 −0
Large diff (233 lines) is not loaded by default to keep the page fast.Load diff
added src/wp9/__init__.py +11 −0
Large diff (11 lines) is not loaded by default to keep the page fast.Load diff
added src/wp9/config.py +57 −0
Large diff (57 lines) is not loaded by default to keep the page fast.Load diff
added src/wp9/models.py +105 −0
Large diff (105 lines) is not loaded by default to keep the page fast.Load diff
added src/wp9/parsing.py +166 −0
Large diff (166 lines) is not loaded by default to keep the page fast.Load diff
added src/wp9/plotstyle.py +27 −0
Large diff (27 lines) is not loaded by default to keep the page fast.Load diff
added src/wp9/sample.py +114 −0
Large diff (115 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere