# 🏠 Hedonic Housing Price Models for the United States

UQO Working Paper No. 3 — A Multi-Method Comparison of Parametric, Quantile, and Machine Learning Approaches

TL;DR — Three hedonic frameworks are compared on 788,842 active Zillow listings covering all 50 U.S. states and DC. Semi-log OLS with 62 regressors gives R² = 0.634 (0.725 with 886 ZIP3 fixed effects; Moran's I = 0.27 on residuals). Quantile regression rejects coefficient equality between τ = 0.10 and 0.90 for 11 of 13 key variables. XGBoost reaches R² = 0.833 under a random split but only 0.425–0.547 when ten entire states are held out, and removing geographic features improves geographic-holdout R² to 0.519 — random validation flatters machine learning through spatial leakage.

# 📌 Status

# 🎯 What this paper does

The paper compares three frameworks for hedonic housing valuation on a single large dataset:

ln P_i = α + x_i'β + μ_g(i) + ε_i      g = Census region → state → ZIP3

The core methodological contribution is a systematic study of spatial leakage in hedonic model evaluation: random train/test splits let geographically proximate listings appear on both sides of the split, inflating performance metrics. Three designs quantify the damage — a 10-state geographic holdout, a 6-stage feature-ablation cascade, and a latitude/longitude augmentation experiment. All estimates are listing-price gradients (asking prices), not causal willingness-to-pay parameters.

# 🏆 Headline results

• Geography does the heavy lifting in OLS: region dummies → state FE → 886 ZIP3 FE raises out-of-sample R² from 0.630 → 0.678 → 0.725.
• Residual spatial autocorrelation: Moran's I = 0.2745 (SD 0.0076) across three 5,000-listing subsamples, all p < 0.001.
• Distributional heterogeneity: the garage gradient is 10× larger at τ = 0.10 than at τ = 0.90 (z = 28.9); the pool premium only emerges above the median.
• Ablation cascade (XGBoost): structural 0.498 → +lot 0.546 → +amenities 0.638 → +neighbourhood 0.814 → +market 0.824 → full 0.833.
• SHAP rankings are model-stable (Spearman ρ = 0.89–0.99 across XGBoost / LightGBM / RF) but not structural.
• Imputation matters: the lot-size gradient triples (0.018 → 0.059) when state-median-imputed lot sizes are dropped.

# 🗂 Repository layout

wp3_uqo/
├── README.md
├── requirements.txt               # pinned dependencies (validated 2026-08-05)
├── docs/                          # AUDIT.md · CHANGES.md · PAPER_REVIEW.md · UPGRADE_REPORT.md
├── data/                          # NOT tracked (3.2 GB)
│   ├── raw/us_housing.duckdb      #   raw Zillow extract (839,313 × 116 + 4 aux tables)
│   └── processed/                 #   analytical_sample.pkl · model_data.pkl · shap_data.pkl
├── src/wp3/                       # shared library
│   ├── config.py                  #   paths (WP3_ROOT-overridable), constants, feature blocks, holdout states, seeds
│   ├── data.py                    #   loaders + 62-regressor design-matrix builder
│   ├── models.py                  #   XGBoost training helper (dual validation schemes)
│   └── plotting.py                #   journal figure style + variable labels
├── scripts/                       # the pipeline, in order
│   ├── 01_spatial_robustness.py   #   ZIP3 FE OLS · Moran's I · XGB ± geography · ablation
│   ├── 02_qr_imputation.py        #   QR stability · inter-quantile Wald · imputation · winsorisation
│   ├── 03_make_figures.py         #   all 11 figures + verification stats
│   └── 04_export_tables.py        #   paper tables → results/tables/*.csv
├── results/
│   ├── *.pkl                      #   NOT tracked (up to 437 MB)
│   └── tables/                    #   13 CSVs mirroring the paper's tables
├── figures/                       # 11 paper figures (PNG, 300 dpi)
└── paper/                         # main.tex + sections/ + appendix/ + references.bib → main.pdf

# 🚀 Quick start

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# place the artifacts (request them, see Data) in data/raw/, data/processed/, results/
export WP3_ROOT=/path/to/artifacts          # optional, defaults to the repo root

python scripts/01_spatial_robustness.py     # ~1–2 h (16 XGBoost models)
python scripts/02_qr_imputation.py          # ~1–3 h (12 quantile regressions on 150k obs)
python scripts/03_make_figures.py           # ~5 min, all 11 figures (--figs fig2 fig4 for a subset)
python scripts/04_export_tables.py          # ~2 min, 13 CSV tables

cd paper && make                            # latexmk → main.pdf (60 pages)

Apple Silicon, 32 GB RAM; scripts use all cores. Every script prints verification statistics comparing recomputed values with the numbers reported in the paper.

# 🗃 Data

• Source: Zillow active for-sale listings, 2025–2026 snapshot; 839,313 residential properties × 116 variables in a DuckDB database (properties, price_history, schools, tax_history, meta_columns).
• Analytical sample: 788,842 listings after five filters (price $10K–$10M, living area 200–20,000 sqft, 1–10 beds/baths, valid coordinates, US states + DC; 6.0 % attrition); 68 engineered columns (logs, age and age², ratios, amenity dummies, neighbourhood scores, market-status flags, 6 interactions, simplified categoricals).
• Not in git: us_housing.duckdb (1.6 GB), analytical_sample.pkl (457 MB), model_data.pkl (867 MB), ml_results.pkl (437 MB). Available on request (subject to Zillow Terms of Service): simon-pierre.boucher@uqo.ca.
• The scripts that built the analytical sample and estimated the primary models were lost before the 2026-08-05 restructuring; the stored pickles are the canonical record of those stages and every downstream number is validated against them (docs/AUDIT.md).

# 🔬 Methodology

1. OLS — 62 regressors, standardised and unstandardised variants, HC3 robust SE; fixed effects at region (4), state (50) and ZIP3 (886) levels (scripts/01).
2. Spatial diagnostics — Moran's I on OLS residuals, row-standardised KNN weights (k = 8), 3 × 5,000-obs subsamples, 999 permutations.
3. Quantile regression — statsmodels QuantReg, τ ∈ {0.10, …, 0.90}, 150,000-obs subsample, 10-seed stability check; inter-quantile Wald z-tests on β̂(0.10) − β̂(0.90) (scripts/02).
4. Machine learning — XGBoost (1,000 trees, depth 8, lr 0.05, early stopping 50), LightGBM, RF, Ridge, Lasso, Elastic Net; random 80/20 split (seed 42) and 10-state geographic holdout (CA NY TX FL OH CO NC WA IL GA, 438,315 test obs) (src/wp3/models.py).
5. Ablation — 6 nested feature sets: structural → +lot → +amenities → +neighbourhood → +market → full.
6. Interpretation — TreeSHAP on 10,000 test obs; cross-model Spearman stability.
7. Sensitivity — state-median-imputation drop-out, lot-size winsorisation (p99.5), Bike Score cap.

# 🖼 Figures & tables inventory

Tables (results/tables/*.csv, 13 files): OLS coefficients, QR paths, inter-quantile Wald tests, stability CVs, Moran's I, imputation/winsorisation sensitivity, SHAP importance and cross-model correlations, ZIP3 FE, ablation.

# ⚠️ Limitations

• Listing (asking) prices, not transactions; gradients are conditional associations, not willingness to pay.
• A single 2025–2026 snapshot: no time dimension, no repeat-sales control.
• Lot size is state-median-imputed for part of the sample; the gradient is sensitive to that choice.
• The upstream sample-construction and primary-estimation scripts are lost; those stages are reproducible only from the stored artifacts.

# 📝 Citation

@techreport{boucher2026hedonicus,
  author      = {Boucher, Simon-Pierre},
  title       = {Hedonic Housing Price Models for the United States: A Multi-Method
                 Comparison of Parametric, Quantile, and Machine Learning Approaches},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {3},
  year        = {2026},
  month       = {May}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

• Format follows Recherche-UQO/HARMONISATION.md (2026-10-17): template title page, natbib + aer bibliography style, docs/ for the restructuring reports.
• The original flat May 2026 project (immo-wp3-spb-20260519, 3.2 GB) is archived in Recherche-UQO/_archives/versions-mai-2026/hedonic-models-united-states--immo-wp3-20260519/; docs/CHANGES.md lists the six manuscript inconsistencies corrected on 2026-08-05 (no scientific result altered).
• GitHub mirror: spboucher-ai/wp3-hedonic-housing-us (remote github); canonical remote is spbgit wp3_uqo.git.
• Build verified 2026-10-17: 0 errors, 0 undefined references, 70 references rendered.