# 🏠 Hedonic Housing Price Models for the United States

UQO Working Paper No. 3 — A Multi-Method Comparison of Parametric, Quantile, and Machine Learning Approaches

TL;DR — Three hedonic frameworks are compared on 788,842 active Zillow listings covering all 50 U.S. states and DC. Semi-log OLS with 62 regressors gives R² = 0.634 (0.725 out of sample with 885 ZIP3 fixed effects; Moran's I = 0.27 on residuals); cluster-robust standard errors are 8–12× the HC3 errors and leave 33 of 63 coefficients significant. State-by-state estimation shows living-area elasticities from 0.06 to 1.15; a fully interacted model gains +6.1 pp over state fixed effects. Paired cluster-bootstrap Wald tests reject coefficient equality between τ = 0.10 and 0.90 for 10 of 13 key variables. XGBoost reaches R² = 0.833 under a random split but 0.425 when ten states are held out (median 0.653 across 51 leave-one-state-out models); once 0.6 % of corrupt records are removed, OLS is as accurate out of state (0.539) as the best tree ensemble. The gap is traced to memorised state price levels (within-state R² = 0.62), to "neighbourhood" variables that are state-median imputations for 93 % of listings, and to boosting capacity (held-out R² peaks at 75 rounds). A coding error (ln lot = 0 for 16.5 % of listings) is corrected: lot elasticity 0.095, not 0.018.

# 📌 Status

# 🎯 What this paper does

The paper compares three frameworks for hedonic housing valuation on a single large dataset:

ln P_i = α + x_i'β + μ_g(i) + ε_i      g = Census region → state → ZIP3

The core methodological contribution is a systematic study of spatial leakage in hedonic model evaluation: random train/test splits let geographically proximate listings appear on both sides of the split, inflating performance metrics. The revision quantifies the damage with a 10-state holdout, grouped 5-fold CV by state, leave-one-state-out, a size-matched random split and a within-state R², and identifies its mechanics: region dummies, state-median-imputed "neighbourhood" variables (93 % of school ratings) and boosting capacity (held-out R² peaks at 75 rounds, falls to 0.425 at 1,000). All estimates are listing-price gradients (asking prices), not causal willingness-to-pay parameters.

# 🏆 Headline results

• Geography does the heavy lifting in OLS: region dummies → state FE → 885 ZIP3 FE raises out-of-sample R² from 0.630 → 0.678 → 0.725; residual Moran's I = 0.2745 (SD 0.0076); tree-ensemble test residuals: 0.079 (XGBoost).
• Inference: ZIP3- and state-clustered SEs are 7.6× and 11.6× HC3 (medians); significant coefficients 57 → 35 → 33 of 63.
• One market or 51? State-by-state living-area elasticity 0.06–1.15; bedroom, pool, luxury and HOA coefficients change sign in 19–23 states; fully interacted OLS: +6.1 pp over state FE.
• Distributional heterogeneity: the garage gradient is 10× larger at τ = 0.10 than at τ = 0.90; the pool premium only emerges above the median; 10 of 13 inter-quantile differences survive a paired ZIP3-cluster bootstrap.
• Ablation cascade (XGBoost): structural 0.498 → +lot 0.546 → +amenities 0.638 → +neighbourhood 0.814 → +market 0.824 → full 0.833 — but the neighbourhood block is mostly a state encoding (school variables imputed for 93 %).
• Boosting rounds: held-out-state R² 0.550 at 75 rounds → 0.425 at 1,000 while within-training R² keeps rising.
• SHAP rankings are model-stable (Spearman ρ = 0.89–0.99) but not structural: the 3rd-ranked feature is a state identifier.
• Lot-size coding: ln(lot) was 0 for 16.5 % of listings; recoded elasticity 0.095 (not 0.018 / 0.059).

# 🗂 Repository layout

wp3_uqo/
├── README.md
├── requirements.txt               # pinned dependencies (validated 2026-08-05)
├── docs/                          # REVIEW-REPORT.md · CHANGELOG-REVISION.md (2026-10-17) · AUDIT.md · CHANGES.md · PAPER_REVIEW.md · UPGRADE_REPORT.md
├── data/                          # NOT tracked (3.2 GB)
│   ├── raw/us_housing.duckdb      #   raw Zillow extract (839,313 × 116 + 4 aux tables)
│   └── processed/                 #   analytical_sample.pkl · model_data.pkl · shap_data.pkl
├── src/wp3/                       # shared library
│   ├── config.py                  #   paths (WP3_ROOT-overridable), constants, feature blocks, holdout states, seeds
│   ├── data.py                    #   loaders + 62-regressor design-matrix builder
│   ├── models.py                  #   XGBoost training helper (dual validation schemes)
│   └── plotting.py                #   journal figure style + variable labels
├── scripts/                       # the pipeline, in order
│   ├── 01_spatial_robustness.py   #   ZIP3 FE OLS · Moran's I · XGB ± geography · ablation
│   ├── 02_qr_imputation.py        #   QR stability · inter-quantile Wald · imputation · winsorisation
│   ├── 03_make_figures.py         #   figures 1–11 + verification stats
│   ├── 04_export_tables.py        #   paper tables → results/tables/*.csv
│   ├── 05_revision_geo_validation.py   # trees curve · size-matched split · variants · OLS geo · GroupKFold · LOSO (~25 min)
│   ├── 06_revision_inference.py        # missingness audit · clustered SE · lot recoding · state-by-state · R² bootstrap · Moran (~6 min)
│   ├── 07_revision_qr_bootstrap.py     # paired (obs. + ZIP3-cluster) bootstrap for inter-quantile Wald tests (~25 min)
│   └── 08_revision_tables_figures.py   # paper/tables/*.tex fragments + figures 12–14
├── results/
│   ├── *.pkl                      #   NOT tracked (up to 437 MB)
│   ├── tables/                    #   13 CSVs mirroring the v1.1 tables
│   └── revision/                  #   18 CSVs feeding the v1.2 tables (scripts 05–07)
├── figures/                       # 14 paper figures (PNG, 300 dpi)
└── paper/                         # main.tex + sections/ + appendix/ + tables/ (generated) + references.bib → main.pdf

# 🚀 Quick start

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# place the artifacts (request them, see Data) in data/raw/, data/processed/, results/
export WP3_ROOT=/path/to/artifacts          # optional, defaults to the repo root

python scripts/01_spatial_robustness.py     # ~1–2 h (16 XGBoost models)
python scripts/02_qr_imputation.py          # ~1–3 h (12 quantile regressions on 150k obs)
python scripts/03_make_figures.py           # ~5 min, figures 1–11 (--figs fig2 fig4 for a subset)
python scripts/04_export_tables.py          # ~2 min, 13 CSV tables
python scripts/05_revision_geo_validation.py   # ~25 min (104 XGBoost fits)
python scripts/06_revision_inference.py        # ~6 min
python scripts/07_revision_qr_bootstrap.py     # ~25 min on 8 workers (--draws 200)
python scripts/08_revision_tables_figures.py   # <1 min, paper/tables/*.tex + figures 12–14

cd paper && make                            # latexmk → main.pdf (82 pages)

Apple Silicon, 32 GB RAM; scripts use all cores. Every script prints verification statistics comparing recomputed values with the numbers reported in the paper.

# 🗃 Data

• Source: Zillow active for-sale listings, 2025–2026 snapshot; 839,313 residential properties × 116 variables in a DuckDB database (properties, price_history, schools, tax_history, meta_columns).
• Analytical sample: 788,842 listings after five filters (price $10K–$10M, living area 200–20,000 sqft, 1–10 beds/baths, valid coordinates, US states + DC; 6.0 % attrition); 68 engineered columns (logs, age and age², ratios, amenity dummies, neighbourhood scores, market-status flags, 6 interactions, simplified categoricals).
• Not in git: us_housing.duckdb (1.6 GB), analytical_sample.pkl (457 MB), model_data.pkl (867 MB), ml_results.pkl (437 MB). Available on request (subject to Zillow Terms of Service): simon-pierre.boucher@uqo.ca.
• The scripts that built the analytical sample and estimated the primary models were lost before the 2026-08-05 restructuring; the stored pickles are the canonical record of those stages and every downstream number is validated against them (docs/AUDIT.md).

# 🔬 Methodology

1. OLS — 62 regressors, standardised and unstandardised variants, HC3 robust SE; fixed effects at region (4), state (50) and ZIP3 (886) levels (scripts/01).
2. Spatial diagnostics — Moran's I on OLS residuals, row-standardised KNN weights (k = 8), 3 × 5,000-obs subsamples, 999 permutations.
3. Quantile regression — statsmodels QuantReg, τ ∈ {0.10, …, 0.90}, 150,000-obs subsample, 10-seed stability check; inter-quantile Wald z-tests on β̂(0.10) − β̂(0.90) (scripts/02).
4. Machine learning — XGBoost (1,000 trees, depth 8, lr 0.05, early stopping 50), LightGBM, RF, Ridge, Lasso, Elastic Net; random 80/20 split (seed 42) and 10-state geographic holdout (CA NY TX FL OH CO NC WA IL GA, 438,315 test obs) (src/wp3/models.py).
5. Ablation — 6 nested feature sets: structural → +lot → +amenities → +neighbourhood → +market → full.
6. Interpretation — TreeSHAP on 10,000 test obs; cross-model Spearman stability.
7. Sensitivity — state-median-imputation drop-out, lot-size winsorisation (p99.5), Bike Score cap.
8. Revision (v1.2) — cluster-robust SE (ZIP3, state); state-by-state OLS and fully interacted prediction; paired cluster-bootstrap inter-quantile tests; size-matched split, within-state R², boosting-round curve, GroupKFold by state, leave-one-state-out; feature variants (no region, no school, missingness indicators); OLS geographic holdout on trimmed data; lot-size recoding; Moran's I on tree residuals; bootstrap CIs for R² (scripts/05–08).

# 🖼 Figures & tables inventory

Tables (results/tables/*.csv, 13 files): OLS coefficients, QR paths, inter-quantile Wald tests, stability CVs, Moran's I, imputation/winsorisation sensitivity, SHAP importance and cross-model correlations, ZIP3 FE, ablation. Revision tables (results/revision/*.csv, 18 files → paper/tables/*.tex): missingness audit, implausible values, list/Zestimate ratio, clustered SE, lot recoding, state heterogeneity, pooled vs interacted, R² bootstrap, Moran by model, designs, trees curve, variants, OLS geo holdout, grouped CV, LOSO, inter-quantile bootstrap.

# ⚠️ Limitations

• Listing (asking) prices, not transactions; gradients are conditional associations, not willingness to pay.
• A single 2025–2026 snapshot: no time dimension, no repeat-sales control.
• School variables are state medians for 93 % of listings; ln(lot) was coded 0 for 16.5 % (corrected for OLS in v1.2, not for QR/ablation/SHAP); 0.6 % of records carry corrupt secondary attributes.
• Inference is cluster-robust but rests on 51 unequal state clusters; no spatial econometric model is estimated.
• The upstream sample-construction and primary-estimation scripts are lost; those stages are reproducible only from the stored artifacts.

# 📝 Citation

@techreport{boucher2026hedonicus,
  author      = {Boucher, Simon-Pierre},
  title       = {Hedonic Housing Price Models for the United States: A Multi-Method
                 Comparison of Parametric, Quantile, and Machine Learning Approaches},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {3},
  year        = {2026},
  month       = {October},
  note        = {Version 1.2}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

• Format follows Recherche-UQO/HARMONISATION.md (2026-10-17): template title page, natbib + aer bibliography style, docs/ for the restructuring reports.
• The original flat May 2026 project (immo-wp3-spb-20260519, 3.2 GB) is archived in Recherche-UQO/_archives/versions-mai-2026/hedonic-models-united-states--immo-wp3-20260519/; docs/CHANGES.md lists the six manuscript inconsistencies corrected on 2026-08-05 (no scientific result altered).
• GitHub mirror: spboucher-ai/wp3-hedonic-housing-us (remote github); canonical remote is spbgit wp3_uqo.git.
• Scientific revision 2026-10-17 (v1.2): see docs/REVIEW-REPORT.md (editorial-board report) and docs/CHANGELOG-REVISION.md (audit trail of every correction, references added with DOIs, reanalyses run). The pre-revision state is tag pre-revision-20261017 and docs/main_pre-revision-20261017.pdf.
• Build verified 2026-10-17: 0 errors, 0 undefined references, 92 references rendered, 82 pages.
• Table fragments in paper/tables/ are generated by scripts/08_revision_tables_figures.py; their last row carries no \\ (supplied after \input in the section files) to avoid a Misplaced \noalign error.