Stage F B1/B2 + N1..N14
Verification sign-off (Agent 11) corrections, numbers through macros/generators only. B1 SHAP rho macros renamed to the defined keys (d_shap_rho_xgb_lgb/_xgb_rf/_lgb_rf) + tests/test_inline_numbers.py (every \inum key used is defined, every numeric macro renders a number, committed inline_numbers.tex not stale). B2 make_inline_numbers.py: float() before round(), d() coerces numpy scalars, render guard rejects object representations (no np.float64 in the PDF). N1 one rounding rule: inline_numbers.json stores raw + digits, formatted macros/derived/README round once from raw (prose = tables: 0.637, 17.6, 0.767, 0.622, 0.523, 0.706, 0.8). N2 feasible correction on its own seed-42 baseline (0.635; share 82 -> 83 %), stated in 5.7.1. N3 trees-curve column "LOSO (mean of 51 state R2)" + legend/caption. N4 r2_ols_random_test_v12 refreshed via 10_stored_checks.py. N5 README written entirely by 08::t_readme. N6 title-page commit convention. N7 "monotonically"/"nothing out of state"/"under every rule" softened to the numbers. N8 SHAP beeswarm tercile markers. N9 cluster_se note -> tab:ols_results, 3.743 -> 3.7, \inumsci for the condition number. N10 eq:moran/iqr_test/mdape/within referenced. N11 abstract 149 words. N12 style flags removed, p. 66 widow gone. N13 run_meta.json restored from the M4BP48 log (5.67 h) with the re-run kept under previous_runs; 05 script no longer overwrites. N14 tab:contributions. Build: 89 pages, 0 errors, 0 undefined, 0 overfull > 5 pt, 0 float too large; 18 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
23 changed files +5,111 −1,832
modified
README.md
+99 −156
@@ -1,27 +1,18 @@ | ||
| 1 | 1 | <!-- Author: Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca --> |
| 2 | +<!-- generated by scripts/08_tables_figures_v2.py::t_readme from results/inline_numbers.json; do not edit by hand --> | |
| 2 | 3 | # 🏠 Hedonic Housing Price Models for the United States |
| 3 | 4 | |
| 4 | −**UQO Working Paper No. 3** — *A Multi-Method Comparison of Parametric, Quantile, and Machine Learning Approaches* | |
| 5 | +**UQO Working Paper No. 3** — *Validation Design, Market Price Levels and the Comparison of Parametric, Quantile and Machine-Learning Approaches* (version 2.0, November 2026) | |
| 5 | 6 | |
| 6 | −[-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) | |
| 7 | +[](paper/main.pdf) | |
| 7 | 8 | [](requirements.txt) |
| 8 | 9 | [](#-data) |
| 9 | −[](src/wp3/) | |
| 10 | −[](scripts/) | |
| 10 | +[](src/wp3/) | |
| 11 | +[](run_all.sh) | |
| 11 | 12 | [](https://uqo.ca) |
| 12 | 13 | [](mailto:simon-pierre.boucher@uqo.ca) |
| 13 | 14 | |
| 14 | −> **TL;DR** — Three hedonic frameworks are compared on **788,842 active Zillow listings** covering all 50 U.S. states and | |
| 15 | −> DC. Semi-log OLS with 62 regressors gives R² = 0.634 (0.725 out of sample with 885 ZIP3 fixed effects; Moran's *I* = 0.27 | |
| 16 | −> on residuals); cluster-robust standard errors are 8–12× the HC3 errors and leave **33 of 63** coefficients significant. | |
| 17 | −> State-by-state estimation shows living-area elasticities from 0.06 to 1.15; a fully interacted model gains +6.1 pp over | |
| 18 | −> state fixed effects. Paired cluster-bootstrap Wald tests reject coefficient equality between τ = 0.10 and 0.90 for | |
| 19 | −> **10 of 13** key variables. XGBoost reaches R² = 0.833 under a random split but **0.425** when ten states are held | |
| 20 | −> out (median **0.653** across 51 leave-one-state-out models); once 0.6 % of corrupt records are removed, **OLS is as | |
| 21 | −> accurate out of state (0.539)** as the best tree ensemble. The gap is traced to memorised state price levels (within-state | |
| 22 | −> R² = 0.62), to "neighbourhood" variables that are state-median imputations for **93 %** of listings, and to boosting | |
| 23 | −> capacity (held-out R² peaks at 75 rounds). A coding error (ln lot = 0 for 16.5 % of listings) is corrected: lot | |
| 24 | −> elasticity 0.095, not 0.018. | |
| 15 | +> **TL;DR** — Whether gradient-boosted trees price homes better than a hedonic regression depends on how the comparison is validated. On 783,872 Zillow asking prices (every U.S. state and DC; 788,842 listings before trimming 4,970 implausible records), semi-log OLS, quantile regression and boosted ensembles are estimated on one corrected design matrix and compared under seven validation designs with ZIP3-cluster bootstrap intervals. XGBoost leads OLS by 18.7 points of out-of-sample R² under a random split and by 13.2 under ZIP3 blocks; the lead shrinks to 4.8–6.3 points under state blocks and leave-one-state-out, and under a single ten-state holdout the two are indistinguishable (+2.6 pp [-0.1, +7.4]). Out-of-state error is dominated by market price levels (25.0 % of the ten-state test variance for the 1,000-round model): a level estimated from twenty local listings raises leave-one-state-out R² from 0.635 to 0.699 (oracle 0.712). Wild-cluster inference by state leaves 23 of 63 OLS slopes significant; 8 of 14 inter-quantile differences survive multiplicity control; SHAP rankings are stable across ensembles (Spearman ρ 0.905–0.987). Estimates are listing-price associations, not willingness to pay. | |
| 25 | 16 | |
| 26 | 17 | --- |
| 27 | 18 | |
@@ -29,34 +20,29 @@ | ||
| 29 | 20 | |
| 30 | 21 | | Item | Value | |
| 31 | 22 | |---|---| |
| 32 | −| Version | **1.2 (October 2026)** — revised 2026-10-17 by the editorial-board review (`docs/REVIEW-REPORT.md`, `docs/CHANGELOG-REVISION.md`); v1.1 (May 2026) frozen as git tag `pre-revision-20261017` and `docs/main_pre-revision-20261017.pdf` | | |
| 23 | +| Version | **2.0 (November 2026)** — rewritten on the Stage D evidence of the multi-agent review (branch `review-20261103`; v1.2 of October 2026 is tag `review-baseline-20261103`); every v1.2 statement withdrawn is listed in Appendix F of the paper | | |
| 33 | 24 | | SSRN | not yet submitted | |
| 34 | 25 | | Journal | not yet submitted | |
| 35 | −| Code & data | figures and tables reproducible from stored artifacts; raw Zillow extract (3.2 GB) available on request | | |
| 26 | +| Code & data | `run_all.sh` rebuilds the analytical sample from the raw DuckDB and every table, figure and inline number from the committed results; the raw Zillow extract is restricted by Zillow's terms and available from the author (`data/DATA_AVAILABILITY.md`) | | |
| 36 | 27 | |
| 37 | 28 | ## 🎯 What this paper does |
| 38 | 29 | |
| 39 | 30 | The paper compares three frameworks for hedonic housing valuation on a single large dataset: |
| 40 | 31 | |
| 41 | −<!-- generated by scripts/08_tables_figures_v2.py::t_readme from results/inline_numbers.json; do not edit by hand --> | |
| 42 | 32 | | Framework | Question it answers | Headline result (v2 design, trimmed sample) | |
| 43 | 33 | |---|---|---| |
| 44 | 34 | | **Semi-log OLS** (62 regressors + lot-missing indicator; HC3, ZIP3- and state-clustered SE, wild cluster bootstrap; state-by-state) | Average conditional association between an attribute and log asking price, and how much it hides | R² = 0.649; living-area elasticity 0.587 (p10–p90 across states 0.32–0.72); lot elasticity 0.097; 23 of 63 slopes significant at 5 % by wild cluster bootstrap | |
| 45 | 35 | | **Quantile regression** (τ ∈ {0.10, 0.25, 0.50, 0.75, 0.90}; paired ZIP3-cluster bootstrap, Holm / Romano–Wolf) | How attribute gradients vary across the price distribution | β(0.10) ≠ β(0.90) for 10 of 14 variables by percentile CI, 8 after Holm and Romano–Wolf | |
| 46 | 36 | | **Gradient boosting + SHAP** (XGBoost, LightGBM, RF; random / ZIP5 / ZIP3 / state blocks / LOSO / 10-state; nested round selection, 5 seeds) | How well prices can be predicted in seen and unseen markets, and which features drive predictions | OLS − XGBoost gap: -18.7 pp [-19.9, -17.5] (random), -6.3 pp [-7.6, -4.7] (LOSO), +2.6 pp [-0.1, +7.4] (10 states) | |
| 37 | + | |
| 38 | +``` | |
| 47 | 39 | ln P_i = α + x_i'β + μ_g(i) + ε_i g = Census region → state → ZIP3 |
| 48 | 40 | ``` |
| 49 | 41 | |
| 50 | −The core methodological contribution is a systematic study of **spatial leakage** in hedonic model evaluation: random | |
| 51 | −train/test splits let geographically proximate listings appear on both sides of the split, inflating performance | |
| 52 | −metrics. The revision quantifies the damage with a 10-state holdout, grouped 5-fold CV by state, leave-one-state-out, | |
| 53 | −a size-matched random split and a within-state R², and identifies its mechanics: region dummies, state-median-imputed | |
| 54 | −"neighbourhood" variables (93 % of school ratings) and boosting capacity (held-out R² peaks at 75 rounds, falls to 0.425 | |
| 55 | −at 1,000). All estimates are *listing-price* gradients (asking prices), not causal willingness-to-pay parameters. | |
| 42 | +The methodological contribution is the validation design. Random train/test splits let geographically proximate listings appear on both sides of the split; the paper reports the OLS–ensemble gap as a function of the spatial scale of the blocks (random, size-matched random, ZIP5, ZIP3 and state GroupKFold, leave-one-state-out, a ten-state holdout), decomposes out-of-market error into a level part and a within-market part, and reports the feasible level correction next to the oracle. The level error runs through the region dummies, through state-level encodings in the features (the school variables are state medians for 93 % of listings; the neighbourhood block adds 16.0 points of random-split R² but 3.8 out of state) and through boosting capacity: the held-out-state curve peaks after about 69 rounds and falls to 0.425 at 1,000, while the nested rule applied to the training states selects 175 rounds and reaches 0.515. All estimates are *listing-price* gradients (asking prices), not willingness-to-pay parameters. | |
| 56 | 43 | |
| 57 | 44 | ## 🏆 Headline results |
| 58 | 45 | |
| 59 | −<!-- generated by scripts/08_tables_figures_v2.py::t_readme from results/inline_numbers.json; do not edit by hand --> | |
| 60 | 46 | | Model | Random R² | ZIP3 blocks R² | State blocks R² | LOSO pooled R² | 10-state holdout R² | Within-state R² (10 states, oracle) | |
| 61 | 47 | |---|---:|---:|---:|---:|---:|---:| |
| 62 | 48 | | OLS (62 regressors + lot indicator) | 0.646 | 0.635 | 0.580 | 0.573 | 0.541 | 0.573 | |
@@ -65,162 +51,128 @@ at 1,000). All estimates are *listing-price* gradients (asking prices), not caus | ||
| 65 | 51 | | OLS, state-by-state (fully interacted) | 0.752 | — | — | — | — | — | |
| 66 | 52 | | Random forest (re-specified in v2) | 0.800 | — | — | — | 0.583 | 0.620 | |
| 67 | 53 | | LightGBM, nested rounds | 0.809 | 0.759 | 0.631 | 0.643 | 0.565 | 0.643 | |
| 68 | −| **XGBoost, nested rounds (5 seeds)** | **0.833** | **0.766** | **0.628** | **0.636** | **0.515** | **0.637** | | |
| 54 | +| **XGBoost, nested rounds (5 seeds)** | **0.833** | **0.767** | **0.628** | **0.637** | **0.515** | **0.637** | | |
| 69 | 55 | | XGBoost, 1,000 rounds (v1.1 comparator) | 0.833 | 0.768 | 0.615 | 0.623 | 0.425 | 0.628 | |
| 70 | −| XGBoost, 100 rounds | 0.770 | 0.736 | 0.634 | 0.639 | 0.538 | 0.631 | | |
| 71 | −| XGBoost, no region dummies (nested) | — | — | 0.623 | 0.625 | 0.527 | — | | |
| 72 | −| XGBoost, no school variables (nested) | — | — | 0.622 | 0.623 | 0.490 | — | | |
| 73 | −| XGBoost + 6 missingness indicators (nested) | 0.835 | 0.768 | 0.634 | 0.631 | 0.522 | — | | |
| 74 | − | |
| 75 | −OLS − XGBoost (nested) gap with 95 % ZIP3-cluster bootstrap CI: random -18.7 pp [-19.9, -17.5]; ZIP5 blocks -16.8 pp [-17.8, -15.8]; ZIP3 blocks -13.2 pp [-14.5, -11.9]; state blocks -4.8 pp [-6.2, -3.5]; LOSO -6.3 pp [-7.6, -4.7] (XGBoost better in 44 of 51 states); ten-state +2.6 pp [-0.1, +7.4]. LOSO median R²: OLS 0.551, XGBoost nested 0.634; feasible level correction with k = 20 listings raises LOSO XGBoost from 0.635 to 0.699 (oracle 0.712). | |
| 76 | − 0.630 → 0.678 → 0.725; residual Moran's *I* = 0.2745 (SD 0.0076); tree-ensemble test residuals: 0.079 (XGBoost). | |
| 77 | −- **Inference**: ZIP3- and state-clustered SEs are 7.6× and 11.6× HC3 (medians); significant coefficients 57 → 35 → 33 of 63. | |
| 78 | −- **One market or 51?** State-by-state living-area elasticity 0.06–1.15; bedroom, pool, luxury and HOA coefficients change | |
| 79 | − sign in 19–23 states; fully interacted OLS: +6.1 pp over state FE. | |
| 80 | −- **Distributional heterogeneity**: the garage gradient is 10× larger at τ = 0.10 than at τ = 0.90; the pool premium | |
| 81 | − only emerges above the median; 10 of 13 inter-quantile differences survive a paired ZIP3-cluster bootstrap. | |
| 82 | −- **Ablation cascade (XGBoost)**: structural 0.498 → +lot 0.546 → +amenities 0.638 → +neighbourhood **0.814** → +market | |
| 83 | − 0.824 → full 0.833 — but the neighbourhood block is mostly a state encoding (school variables imputed for 93 %). | |
| 84 | −- **Boosting rounds**: held-out-state R² 0.550 at 75 rounds → 0.425 at 1,000 while within-training R² keeps rising. | |
| 85 | −- **SHAP rankings are model-stable** (Spearman ρ = 0.89–0.99) but not structural: the 3rd-ranked feature is a state identifier. | |
| 86 | −- **Lot-size coding**: ln(lot) was 0 for 16.5 % of listings; recoded elasticity 0.095 (not 0.018 / 0.059). | |
| 56 | +| XGBoost, 100 rounds | 0.770 | 0.736 | 0.635 | 0.639 | 0.538 | 0.631 | | |
| 57 | +| XGBoost, no region dummies (nested) | — | — | 0.623 | 0.624 | 0.527 | — | | |
| 58 | +| XGBoost, no school variables (nested) | — | — | 0.622 | 0.622 | 0.490 | — | | |
| 59 | +| XGBoost + 6 missingness indicators (nested) | 0.835 | 0.768 | 0.634 | 0.632 | 0.523 | — | | |
| 60 | + | |
| 61 | +OLS − XGBoost (nested) gap with 95 % ZIP3-cluster bootstrap CI: random -18.7 pp [-19.9, -17.5]; ZIP5 blocks -16.8 pp [-17.8, -15.8]; ZIP3 blocks -13.2 pp [-14.5, -11.9]; state blocks -4.8 pp [-6.2, -3.5]; LOSO -6.3 pp [-7.6, -4.7] (XGBoost better in 44 of 51 states); ten-state +2.6 pp [-0.1, +7.4]. LOSO median R²: OLS 0.551, XGBoost nested 0.634; feasible level correction with k = 20 listings raises LOSO XGBoost from 0.635 (seed-42 fit) to 0.699 (oracle 0.712). | |
| 62 | + | |
| 63 | +## 🔑 Key results | |
| 64 | + | |
| 65 | +- **OLS** (trimmed v2 design, 783,872 listings): R² 0.649; living-area elasticity 0.587; lot elasticity 0.097 once ln(lot) = 0 for 16.5 % of listings is recoded (v1.1 coding: 0.018); residual Moran's *I* on the full random test set 0.471 (XGBoost residuals 0.236). | |
| 66 | +- **Inference**: ZIP3- and state-clustered SEs are 7.8× and 11.7× HC3 (medians); slopes significant at 5 %: 58 (HC3) → 40 (ZIP3) → 33 (state CR1) → 23 (wild cluster bootstrap by state) of 63. | |
| 67 | +- **One market or 51?** State-by-state living-area elasticity 0.06–1.13; a fully interacted model gains 6.1 pp of out-of-sample R² over state fixed effects. | |
| 68 | +- **Distributional heterogeneity** (QR on a 150,000-listing subsample): the garage gradient is 0.210 at τ = 0.10 and 0.018 at τ = 0.90; 10 of 14 inter-quantile differences are significant by the paired ZIP3-cluster bootstrap (999 draws), 8 after Romano–Wolf. | |
| 69 | +- **Validation design**: OLS − XGBoost (nested rounds) gap -18.7 pp [-19.9, -17.5] under the random split, -4.8 pp [-6.2, -3.5] under state blocks, -6.3 pp [-7.6, -4.7] under leave-one-state-out (XGBoost better in 44 of 51 states), +2.6 pp [-0.1, +7.4] under the ten-state holdout (1,000 ZIP3-cluster bootstrap draws). | |
| 70 | +- **Level, not structure**: 25.0 % of the ten-state test variance is state-level error for the 1,000-round model (17.6 % with nested rounds); twenty local listings raise LOSO R² from 0.635 to 0.699 (oracle 0.712). | |
| 71 | +- **Boosting rounds**: held-out-state R² peaks after about 69 rounds and falls to 0.425 at 1,000; the nested rule selects 175 rounds (0.515). | |
| 72 | +- **SHAP rankings are model-stable** (Spearman ρ 0.905–0.987 across XGBoost, LightGBM and the random forest) but not structural: the third-ranked feature is a state median for most listings. | |
| 87 | 73 | |
| 88 | 74 | ## 🗂 Repository layout |
| 89 | 75 | |
| 90 | 76 | ``` |
| 91 | −wp3_uqo/ | |
| 92 | −├── README.md | |
| 93 | −├── requirements.txt # pinned dependencies (validated 2026-08-05) | |
| 94 | −├── docs/ # REVIEW-REPORT.md · CHANGELOG-REVISION.md (2026-10-17) · AUDIT.md · CHANGES.md · PAPER_REVIEW.md · UPGRADE_REPORT.md | |
| 95 | −├── data/ # NOT tracked (3.2 GB) | |
| 96 | −│ ├── raw/us_housing.duckdb # raw Zillow extract (839,313 × 116 + 4 aux tables) | |
| 97 | −│ └── processed/ # analytical_sample.pkl · model_data.pkl · shap_data.pkl | |
| 98 | −├── src/wp3/ # shared library | |
| 99 | −│ ├── config.py # paths (WP3_ROOT-overridable), constants, feature blocks, holdout states, seeds | |
| 100 | −│ ├── data.py # loaders + 62-regressor design-matrix builder | |
| 101 | −│ ├── models.py # XGBoost training helper (dual validation schemes) | |
| 102 | −│ └── plotting.py # journal figure style + variable labels | |
| 103 | −├── scripts/ # the pipeline, in order | |
| 104 | −│ ├── 01_spatial_robustness.py # ZIP3 FE OLS · Moran's I · XGB ± geography · ablation | |
| 105 | −│ ├── 02_qr_imputation.py # QR stability · inter-quantile Wald · imputation · winsorisation | |
| 106 | −│ ├── 03_make_figures.py # figures 1–11 + verification stats | |
| 107 | −│ ├── 04_export_tables.py # paper tables → results/tables/*.csv | |
| 108 | −│ ├── 05_revision_geo_validation.py # trees curve · size-matched split · variants · OLS geo · GroupKFold · LOSO (~25 min) | |
| 109 | −│ ├── 06_revision_inference.py # missingness audit · clustered SE · lot recoding · state-by-state · R² bootstrap · Moran (~6 min) | |
| 110 | −│ ├── 07_revision_qr_bootstrap.py # paired (obs. + ZIP3-cluster) bootstrap for inter-quantile Wald tests (~25 min) | |
| 111 | −│ └── 08_revision_tables_figures.py # paper/tables/*.tex fragments + figures 12–14 | |
| 112 | −├── results/ | |
| 113 | −│ ├── *.pkl # NOT tracked (up to 437 MB) | |
| 114 | −│ ├── tables/ # 13 CSVs mirroring the v1.1 tables | |
| 115 | −│ └── revision/ # 18 CSVs feeding the v1.2 tables (scripts 05–07) | |
| 116 | −├── figures/ # 14 paper figures (PNG, 300 dpi) | |
| 117 | −└── paper/ # main.tex + sections/ + appendix/ + tables/ (generated) + references.bib → main.pdf | |
| 77 | +├── run_all.sh # the whole pipeline (--from-results skips the three node batches) | |
| 78 | +├── data/ | |
| 79 | +│ ├── DATA_AVAILABILITY.md # window, method, licence, SHA-256 of the raw extract | |
| 80 | +│ ├── raw/us_housing.duckdb # raw Zillow extract (not in git) | |
| 81 | +│ └── processed/ # analytical_sample_v2.parquet, design_v2.parquet (rebuilt by 00/00b; not in git) | |
| 82 | +├── src/wp3/ # shared library (config, build, design matrix, models, exhibits, plotting) | |
| 83 | +├── scripts/ # 00 build sample · 00b design matrix · 00c primary models · 04 legacy tables · 05 geographic designs | |
| 84 | +│ # 06 inference · 07 quantile regression · 08 tables + inline numbers · 08b figures · 09 geo inference · 10 stored checks | |
| 85 | +├── results/ # committed CSV/JSON outputs (build/, primary/, inference_v2/, qr/, geo/, geo_inference/, stored_checks/, inline_numbers.json) | |
| 86 | +├── figures/ # v2_fig_*.pdf (vector) + .png previews | |
| 87 | +├── tests/ # sample rebuild, design matrix, legacy OLS, fragment regeneration, inline-macro coverage | |
| 88 | +└── paper/ # main.tex + sections/ + appendix/ + tables/ (generated) + figures/ (wrappers) + inline_numbers.tex (generated) | |
| 118 | 89 | ``` |
| 119 | 90 | |
| 120 | 91 | ## 🚀 Quick start |
| 121 | 92 | |
| 122 | 93 | ```bash |
| 123 | 94 | python3 -m venv .venv && source .venv/bin/activate |
| 124 | −pip install -r requirements.txt | |
| 125 | −# place the artifacts (request them, see Data) in data/raw/, data/processed/, results/ | |
| 126 | −export WP3_ROOT=/path/to/artifacts # optional, defaults to the repo root | |
| 127 | − | |
| 128 | −python scripts/01_spatial_robustness.py # ~1–2 h (16 XGBoost models) | |
| 129 | −python scripts/02_qr_imputation.py # ~1–3 h (12 quantile regressions on 150k obs) | |
| 130 | −python scripts/03_make_figures.py # ~5 min, figures 1–11 (--figs fig2 fig4 for a subset) | |
| 131 | −python scripts/04_export_tables.py # ~2 min, 13 CSV tables | |
| 132 | −python scripts/05_revision_geo_validation.py # ~25 min (104 XGBoost fits) | |
| 133 | −python scripts/06_revision_inference.py # ~6 min | |
| 134 | −python scripts/07_revision_qr_bootstrap.py # ~25 min on 8 workers (--draws 200) | |
| 135 | −python scripts/08_revision_tables_figures.py # <1 min, paper/tables/*.tex + figures 12–14 | |
| 136 | − | |
| 137 | −cd paper && make # latexmk → main.pdf (82 pages) | |
| 95 | +pip install -r requirements.txt # environment/requirements.lock* pins the versions used | |
| 96 | +# place data/raw/us_housing.duckdb (request it, see Data) | |
| 97 | +./run_all.sh --from-results # rebuild sample, fast estimations, every exhibit, tests, PDF (~1 h) | |
| 98 | +./run_all.sh # also the three long batches (05 geo designs ~5.7 h, 00c --v2, 07 QR) | |
| 138 | 99 | ``` |
| 139 | 100 | |
| 140 | −Apple Silicon, 32 GB RAM; scripts use all cores. Every script prints verification statistics comparing recomputed values | |
| 141 | −with the numbers reported in the paper. | |
| 101 | +Scripts in pipeline order (`PY=.venv/bin/python`): | |
| 102 | + | |
| 103 | +```bash | |
| 104 | +$PY scripts/00_build_sample.py --check | |
| 105 | +$PY scripts/00b_design_matrix.py --check-legacy | |
| 106 | +$PY scripts/00c_primary_models.py --legacy --ols-only | |
| 107 | +$PY scripts/04_export_tables.py [--legacy-exhibits] | |
| 108 | +$PY scripts/10_stored_checks.py | |
| 109 | +$PY scripts/05_geo_designs.py --seeds 42 43 44 45 46 --resume --rf # node batch, ~5.7 h on 16 cores | |
| 110 | +$PY scripts/00c_primary_models.py --v2 # node batch | |
| 111 | +$PY scripts/07_qr_v2.py --draws 999 --legacy-check # node batch | |
| 112 | +$PY scripts/06_inference_v2.py | |
| 113 | +$PY scripts/09_geo_inference.py | |
| 114 | +$PY scripts/08_tables_figures_v2.py | |
| 115 | +$PY scripts/08b_figures_v2.py | |
| 116 | +$PY paper/make_inline_numbers.py | |
| 117 | +``` | |
| 142 | 118 | |
| 143 | 119 | ## 🗃 Data |
| 144 | 120 | |
| 145 | −- **Source**: Zillow active for-sale listings, 2025–2026 snapshot; 839,313 residential properties × 116 variables in a | |
| 146 | − DuckDB database (`properties`, `price_history`, `schools`, `tax_history`, `meta_columns`). | |
| 147 | −- **Analytical sample**: 788,842 listings after five filters (price $10K–$10M, living area 200–20,000 sqft, 1–10 | |
| 148 | − beds/baths, valid coordinates, US states + DC; 6.0 % attrition); 68 engineered columns (logs, age and age², ratios, | |
| 149 | − amenity dummies, neighbourhood scores, market-status flags, 6 interactions, simplified categoricals). | |
| 150 | −- **Not in git**: `us_housing.duckdb` (1.6 GB), `analytical_sample.pkl` (457 MB), `model_data.pkl` (867 MB), | |
| 151 | − `ml_results.pkl` (437 MB). Available on request (subject to Zillow Terms of Service): | |
| 152 | − simon-pierre.boucher@uqo.ca. | |
| 153 | −- The scripts that built the analytical sample and estimated the primary models were lost before the 2026-08-05 | |
| 154 | − restructuring; the stored pickles are the canonical record of those stages and every downstream number is validated | |
| 155 | − against them (`docs/AUDIT.md`). | |
| 121 | +- **Source**: Zillow active for-sale listings (single snapshot; window inferred from the listing dates, see `data/DATA_AVAILABILITY.md`); 839,313 raw residential properties. | |
| 122 | +- **Analytical sample**: 788,842 listings after the filters of the attrition table of the paper (6.0 % attrition); 783,872 after trimming 4,970 listings with implausible secondary attributes (0.6 %); 886 ZIP3 clusters; 6,787 rows in exact-duplicate groups (reported, kept). | |
| 123 | +- **Not in git**: the raw DuckDB and the processed parquet files; `scripts/00_build_sample.py` rebuilds the sample bit-identically from the raw extract (`tests/test_build.py`). | |
| 156 | 124 | |
| 157 | 125 | ## 🔬 Methodology |
| 158 | 126 | |
| 159 | −1. **OLS** — 62 regressors, standardised and unstandardised variants, HC3 robust SE; fixed effects at region (4), state | |
| 160 | − (50) and ZIP3 (886) levels (`scripts/01`). | |
| 161 | −2. **Spatial diagnostics** — Moran's *I* on OLS residuals, row-standardised KNN weights (k = 8), 3 × 5,000-obs | |
| 162 | − subsamples, 999 permutations. | |
| 163 | −3. **Quantile regression** — statsmodels `QuantReg`, τ ∈ {0.10, …, 0.90}, 150,000-obs subsample, 10-seed stability | |
| 164 | − check; inter-quantile Wald *z*-tests on β̂(0.10) − β̂(0.90) (`scripts/02`). | |
| 165 | −4. **Machine learning** — XGBoost (1,000 trees, depth 8, lr 0.05, early stopping 50), LightGBM, RF, Ridge, Lasso, | |
| 166 | − Elastic Net; random 80/20 split (seed 42) **and** 10-state geographic holdout (CA NY TX FL OH CO NC WA IL GA, | |
| 167 | − 438,315 test obs) (`src/wp3/models.py`). | |
| 168 | −5. **Ablation** — 6 nested feature sets: structural → +lot → +amenities → +neighbourhood → +market → full. | |
| 169 | −6. **Interpretation** — TreeSHAP on 10,000 test obs; cross-model Spearman stability. | |
| 170 | −7. **Sensitivity** — state-median-imputation drop-out, lot-size winsorisation (p99.5), Bike Score cap. | |
| 171 | −8. **Revision (v1.2)** — cluster-robust SE (ZIP3, state); state-by-state OLS and fully interacted prediction; paired | |
| 172 | − cluster-bootstrap inter-quantile tests; size-matched split, within-state R², boosting-round curve, GroupKFold by state, | |
| 173 | − leave-one-state-out; feature variants (no region, no school, missingness indicators); OLS geographic holdout on trimmed | |
| 174 | − data; lot-size recoding; Moran's *I* on tree residuals; bootstrap CIs for R² (`scripts/05`–`08`). | |
| 127 | +1. **OLS** — 62 regressors + lot-missing indicator on one corrected design matrix; HC3, ZIP3- and state-clustered (CR1) SEs, wild cluster bootstrap by state (Webb/Rademacher, 9,999 draws), Carter–Schnepel–Steigerwald effective clusters, Holm/BH; region, state and ZIP3 fixed effects; state-by-state regressions with ZIP3-clustered SEs (`00c`, `06`). | |
| 128 | +2. **Spatial diagnostics** — Moran's *I* on the full random test set (8-nearest-neighbour and distance-band weights, 999 permutations), Conley HAC check (`06`). | |
| 129 | +3. **Quantile regression** — τ ∈ {0.10, 0.25, 0.50, 0.75, 0.90} on a 150,000-listing subsample; paired ZIP3-cluster bootstrap (999 draws) for the inter-quantile tests, Holm / BH / Romano–Wolf; clustered SEs at τ = 0.10/0.90; legacy reproduction check (`07`). | |
| 130 | +4. **Machine learning** — XGBoost, LightGBM (rounds selected by nested GroupKFold on the training groups; fixed 100 and 1,000 as comparators; five seeds), re-specified random forest, Ridge/Lasso/Elastic Net; seven validation designs (`05`). | |
| 131 | +5. **Mechanisms** — exact SSE decomposition into level and within-state parts, feasible level correction with k ∈ {5, 20, 50, 100, 200} local listings, region × rounds factorial, 15-specification ablation with the neighbourhood block split (`05`, `09`). | |
| 132 | +6. **Interpretation** — TreeSHAP on the stored explanation rows (Saabas approximation for the RF, labelled); cross-model Spearman stability (`00c --v2`). | |
| 133 | +7. **Exhibits** — every table (`paper/tables/v2_*.tex`), figure (`figures/v2_fig_*.pdf`) and prose number (`paper/inline_numbers.tex`) is generated from the committed results (`08`, `08b`, `paper/make_inline_numbers.py`); `tests/` checks that they regenerate identically and that every macro used in the manuscript is defined. | |
| 175 | 134 | |
| 176 | 135 | ## 🖼 Figures & tables inventory |
| 177 | 136 | |
| 178 | −| Figure | Content | | |
| 179 | −|---|---| | |
| 180 | −| `fig1_price_distribution` | Price & log-price histograms with medians | | |
| 181 | −| `fig2_ols_diagnostics` | Residuals-vs-fitted, Q-Q, density, scale-location | | |
| 182 | −| `fig3_quantile_coefficients` | QR coefficient paths vs. OLS benchmark (8 panels) | | |
| 183 | −| `fig4_model_comparison` | Predicted vs. actual: OLS / XGBoost / LightGBM | | |
| 184 | −| `fig5_shap_summary` | SHAP beeswarm, top 20 features | | |
| 185 | −| `fig6_shap_importance` | Mean \|SHAP\| bar chart, top 20 | | |
| 186 | −| `fig7_geographic_prices` | 50,000-listing national price map | | |
| 187 | −| `fig8_regional_prices` | Log-price boxplots by Census region | | |
| 188 | −| `fig9_marginal_effects` | Bivariate scatters + binned means (6 attributes) | | |
| 189 | −| `fig10_shap_dependence` | SHAP dependence, 8 key features | | |
| 190 | −| `fig11_shap_interactions` | SHAP for interaction terms (waterfront × sqft, pool × South) | | |
| 191 | −| `fig12_geo_trees_curve` | Out-of-sample R² vs boosting rounds (random / early-stopping / 10 held-out states) | | |
| 192 | −| `fig13_loso_by_state` | Leave-one-state-out R² by held-out state | | |
| 193 | −| `fig14_state_heterogeneity` | State-by-state living-area elasticity and garage coefficient | | |
| 194 | − | |
| 195 | −Tables (`results/tables/*.csv`, 13 files): OLS coefficients, QR paths, inter-quantile Wald tests, stability CVs, | |
| 196 | −Moran's *I*, imputation/winsorisation sensitivity, SHAP importance and cross-model correlations, ZIP3 FE, ablation. | |
| 197 | −Revision tables (`results/revision/*.csv`, 18 files → `paper/tables/*.tex`): missingness audit, implausible values, | |
| 198 | −list/Zestimate ratio, clustered SE, lot recoding, state heterogeneity, pooled vs interacted, R² bootstrap, Moran by model, | |
| 199 | −designs, trees curve, variants, OLS geo holdout, grouped CV, LOSO, inter-quantile bootstrap. | |
| 137 | +| File | Label | Caption (first sentence) | | |
| 138 | +|---|---|---| | |
| 139 | +| `figures/v2_fig_ablation.pdf` | `fig:ablation` | Cumulative feature-ablation cascade for XGBoost (1,000 rounds with early stopping, trimmed v2 sample): mean out-of-sample R^2 over seeds 42-… | | |
| 140 | +| `figures/v2_fig_feasible_level.pdf` | `fig:feasible_level` | Feasible level correction of out-of-state predictions | | |
| 141 | +| `figures/v2_fig_gap_by_design.pdf` | `fig:gap_by_design` | The OLS-versus-ensemble gap by validation design | | |
| 142 | +| `figures/v2_fig_geographic_prices.pdf` | `fig:geographic` | Log asking prices of a 50,000-listing random subsample of the analytical sample by location (viridis scale; plate carr'ee projection, aspect… | | |
| 143 | +| `figures/v2_fig_loso_ols_vs_xgb.pdf` | `fig:loso` | Leave-one-state-out R^2 by held-out state, OLS versus XGBoost with nested round selection (trimmed v2 sample; each state is predicted by mod… | | |
| 144 | +| `figures/v2_fig_model_comparison.pdf` | `fig:model_comparison` | Predicted against actual log asking prices on the random-split test set (trimmed v2 sample, seed 42; 60,000-listing random subset per panel,… | | |
| 145 | +| `figures/v2_fig_ols_diagnostics.pdf` | `fig:diagnostics` | Diagnostics of the v2 OLS model on the 156,770 test listings of the random split (seed 42; out-of-sample residuals): (a) residuals against f… | | |
| 146 | +| `figures/v2_fig_price_distribution.pdf` | `fig:price_dist` | Distribution of asking prices in the analytical sample (N = 788,842 active listings; Table~reftab:descriptive): (a) levels, histogram of 100… | | |
| 147 | +| `figures/v2_fig_qr_coefficients.pdf` | `fig:qr_coefficients` | Quantile-regression coefficient paths on the v2 design for eight attributes (random subsample of 150,000 listings; continuous regressors sta… | | |
| 148 | +| `figures/v2_fig_shap_dependence.pdf` | `fig:shap_dependence` | SHAP dependence of the XGBoost model on six features (v2 design, 9,937 explanation listings): each point is one listing's contribution again… | | |
| 149 | +| `figures/v2_fig_shap_summary.pdf` | `fig:shap_summary` | SHAP summary plot of the XGBoost model on the v2 design (TreeSHAP on the 9,937 stored explanation listings of the random test set; the fifte… | | |
| 150 | +| `figures/v2_fig_state_heterogeneity_ci.pdf` | `fig:state_heterogeneity` | State-by-state OLS coefficients in natural units with inference (trimmed v2 sample, Table~reftab:state_heterogeneity): (a) the elasticity of… | | |
| 151 | +| `figures/v2_fig_trees_curve_seeds.pdf` | `fig:trees_curve` | Out-of-sample R^2 of the full XGBoost model as a function of the number of boosting rounds (trimmed v2 sample) | | |
| 152 | + | |
| 153 | +Tables: 43 generated fragments in `paper/tables/v2_*.tex`, one `\input` each in the manuscript (`scripts/08_tables_figures_v2.py`). | |
| 200 | 154 | |
| 201 | 155 | ## ⚠️ Limitations |
| 202 | 156 | |
| 203 | 157 | - Listing (asking) prices, not transactions; gradients are conditional associations, not willingness to pay. |
| 204 | −- A single 2025–2026 snapshot: no time dimension, no repeat-sales control. | |
| 205 | −- School variables are state medians for 93 % of listings; ln(lot) was coded 0 for 16.5 % (corrected for OLS in v1.2, not | |
| 206 | − for QR/ablation/SHAP); 0.6 % of records carry corrupt secondary attributes. | |
| 158 | +- A single snapshot: no time dimension, no repeat-sales control. | |
| 159 | +- School variables are state medians for 93 % of listings; ln(lot) was coded 0 for 16.5 % of listings (corrected for every model in v2); 0.6 % of records carry implausible secondary attributes and are trimmed. | |
| 207 | 160 | - Inference is cluster-robust but rests on 51 unequal state clusters; no spatial econometric model is estimated. |
| 208 | −- The upstream sample-construction and primary-estimation scripts are lost; those stages are reproducible only from the | |
| 209 | − stored artifacts. | |
| 161 | +- The raw extract cannot be redistributed; the three long batches were run once on a cluster node (seeds fixed, caches kept). | |
| 210 | 162 | |
| 211 | 163 | ## 📝 Citation |
| 212 | 164 | |
| 213 | 165 | ```bibtex |
| 214 | 166 | @techreport{boucher2026hedonicus, |
| 215 | 167 | author = {Boucher, Simon-Pierre}, |
| 216 | − title = {Hedonic Housing Price Models for the United States: A Multi-Method | |
| 217 | − Comparison of Parametric, Quantile, and Machine Learning Approaches}, | |
| 168 | + title = {Hedonic Housing Price Models for the United States: Validation Design, Market Price Levels | |
| 169 | + and the Comparison of Parametric, Quantile and Machine-Learning Approaches}, | |
| 218 | 170 | institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives}, |
| 219 | 171 | type = {Working Paper}, |
| 220 | 172 | number = {3}, |
| 221 | 173 | year = {2026}, |
| 222 | − month = {October}, | |
| 223 | − note = {Version 1.2} | |
| 174 | + month = {November}, | |
| 175 | + note = {Version 2.0} | |
| 224 | 176 | } |
| 225 | 177 | ``` |
| 226 | 178 | |
@@ -232,15 +184,6 @@ Université du Québec en Outaouais (UQO), Gatineau, QC. | ||
| 232 | 184 | |
| 233 | 185 | ## 🗒 Notes |
| 234 | 186 | |
| 235 | −- Format follows `Recherche-UQO/HARMONISATION.md` (2026-10-17): template title page, `natbib` + `aer` bibliography style, | |
| 236 | − `docs/` for the restructuring reports. | |
| 237 | −- The original flat May 2026 project (`immo-wp3-spb-20260519`, 3.2 GB) is archived in | |
| 238 | − `Recherche-UQO/_archives/versions-mai-2026/hedonic-models-united-states--immo-wp3-20260519/`; `docs/CHANGES.md` lists | |
| 239 | − the six manuscript inconsistencies corrected on 2026-08-05 (no scientific result altered). | |
| 240 | −- GitHub mirror: `spboucher-ai/wp3-hedonic-housing-us` (remote `github`); canonical remote is spbgit `wp3_uqo.git`. | |
| 241 | −- Scientific revision 2026-10-17 (v1.2): see `docs/REVIEW-REPORT.md` (editorial-board report) and | |
| 242 | − `docs/CHANGELOG-REVISION.md` (audit trail of every correction, references added with DOIs, reanalyses run). The pre-revision | |
| 243 | − state is tag `pre-revision-20261017` and `docs/main_pre-revision-20261017.pdf`. | |
| 244 | −- Build verified 2026-10-17: 0 errors, 0 undefined references, 92 references rendered, 82 pages. | |
| 245 | −- Table fragments in `paper/tables/` are generated by `scripts/08_revision_tables_figures.py`; their last row carries no `\\` | |
| 246 | − (supplied after `\input` in the section files) to avoid a `Misplaced \noalign` error. | |
| 187 | +- This README is written by `scripts/08_tables_figures_v2.py::t_readme` from `results/inline_numbers.json`; edit the generator, not the file. | |
| 188 | +- Canonical remote is spbgit `wp3_uqo.git`; GitHub mirror `spboucher-ai/wp3-hedonic-housing-us`. | |
| 189 | +- The review record (referee reports, change logs, verification) lives outside this repository in `UQO-WP-REVIEW/`; the v1.2 statements withdrawn in v2.0 are listed in Appendix F of the paper. | |
modified
figures/v2_fig_shap_summary.pdf
+0 −0
Binary file not shown.
modified
figures/v2_fig_shap_summary.png
+0 −0
Binary file not shown.
modified
figures/v2_fig_trees_curve_seeds.pdf
+0 −0
Binary file not shown.
modified
figures/v2_fig_trees_curve_seeds.png
+0 −0
Binary file not shown.
modified
paper/figures/v2_fig_shap_summary.tex
+1 −1
@@ -2,6 +2,6 @@ | ||
| 2 | 2 | \begin{figure}[htbp] |
| 3 | 3 | \centering |
| 4 | 4 | \includegraphics[width=\textwidth]{figures/v2_fig_shap_summary.pdf} |
| 5 | −\caption{SHAP summary plot of the XGBoost model on the v2 design (TreeSHAP on the 9,937 stored explanation listings of the random test set; the fifteen features with the largest mean $|\phi|$, Table~\ref{tab:shap_importance}). Each point is one listing; its horizontal position is the feature's contribution to the predicted log price and its colour the percentile rank of the feature value (viridis: dark = low, yellow = high). Contributions describe the fitted prediction function, not implicit prices; the school rating is a state median for most listings and acts as a state identifier.} | |
| 5 | +\caption{SHAP summary plot of the XGBoost model on the v2 design (TreeSHAP on the 9,937 stored explanation listings of the random test set; the fifteen features with the largest mean $|\phi|$, Table~\ref{tab:shap_importance}). Each point is one listing; its horizontal position is the feature's contribution to the predicted log price; the percentile rank of the feature value is encoded twice, by colour (viridis: dark = low, yellow = high) and by marker shape (triangle down = lowest tercile, circle = middle, triangle up = highest), so that the contrast is legible in grayscale. Contributions describe the fitted prediction function, not implicit prices; the school rating is a state median for most listings and acts as a state identifier.} | |
| 6 | 6 | \label{fig:shap_summary} |
| 7 | 7 | \end{figure} |
modified
paper/figures/v2_fig_trees_curve_seeds.tex
+1 −1
@@ -2,6 +2,6 @@ | ||
| 2 | 2 | \begin{figure}[htbp] |
| 3 | 3 | \centering |
| 4 | 4 | \includegraphics[width=\textwidth]{figures/v2_fig_trees_curve_seeds.pdf} |
| 5 | −\caption{Out-of-sample $R^2$ of the full XGBoost model as a function of the number of boosting rounds (trimmed v2 sample). Lines are means over seeds 42--46 and shaded bands the min--max range across seeds; the dotted vertical line marks the number of rounds (175) chosen by the inner GroupKFold(3) rule on the training states for the ten-state design. The held-out-state curve peaks between 65 and 70 rounds depending on the seed and declines to 0.425 at 1,000 rounds, while the early-stopping set drawn from the training states keeps rising; the peak is not observable without the test states (Table~\ref{tab:trees_curve}).} | |
| 5 | +\caption{Out-of-sample $R^2$ of the full XGBoost model as a function of the number of boosting rounds (trimmed v2 sample). Lines are means over seeds 42--46 (for leave-one-state-out, the unweighted mean of the 51 held-out-state $R^2$ values, not the pooled out-of-fold $R^2$) and shaded bands the min--max range across seeds; the dotted vertical line marks the number of rounds (175) chosen by the inner GroupKFold(3) rule on the training states for the ten-state design. The held-out-state curve peaks between 65 and 70 rounds depending on the seed and declines to 0.425 at 1,000 rounds, while the early-stopping set drawn from the training states keeps rising; the peak is not observable without the test states (Table~\ref{tab:trees_curve}).} | |
| 6 | 6 | \label{fig:trees_curve} |
| 7 | 7 | \end{figure} |
modified
paper/inline_numbers.tex
+2396 −798
Large diff (12,590 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/main.pdf
+0 −0
Binary file not shown.
modified
paper/main.tex
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/make_inline_numbers.py
+83 −42
Large diff (207 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/discussion.tex
+3 −2
Large diff (17 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/introduction.tex
+31 −3
Large diff (53 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/limitations.tex
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/literature.tex
+1 −0
Large diff (7 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/results.tex
+17 −11
Large diff (88 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/robustness.tex
+2 −2
Large diff (16 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/sections/titlepage.tex
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/tables/v2_cluster_se.tex
+2 −2
Large diff (15 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/tables/v2_corrigenda.tex
+1 −1
Large diff (8 lines) is not loaded by default to keep the page fast.Load diff
modified
paper/tables/v2_trees_curve.tex
+3 −3
Large diff (18 lines) is not loaded by default to keep the page fast.Load diff
modified
results/geo/run_meta.json
+58 −3
Large diff (70 lines) is not loaded by default to keep the page fast.Load diff
modified
results/inline_numbers.json
+2410 −804
Large diff (6,538 lines) is not loaded by default to keep the page fast.Load diff
Diff truncated — file too large.