spb/wp3_uqo Public

1% Author: Simon-Pierre Boucher — contact@spboucher.ai2%3% ═══════════════════════════════════════════════════════════════════════4% 7. DISCUSSION5% ═══════════════════════════════════════════════════════════════════════6\section{Discussion}7\label{sec:discussion}89The results support four main messages, each associated with a specific modeling10framework. For each, we interpret our estimates against the prior literature,11noting where they agree with published findings, where they diverge, and what the12divergences imply.1314\subsection{Message 1: OLS Remains Useful for Interpretable Associations}1516The semi-log OLS model explains 63.4\% of the variation in log listing prices17using 62 regressors---within the range typical of large cross-sectional hedonic18studies \citep{sirmans2005composition, malpezzi2003hedonic}---and its headline19gradients align well with the meta-analytic record. The living-area elasticity of200.63 is consistent with the concave size-price relationship documented across the21125 studies synthesized by \citet{sirmans2005composition}; the positive bathroom22and garage gradients and the negative age gradient likewise match the modal signs23in that synthesis. The negative conditional bedroom gradient---often treated as an24anomaly---is in fact the standard result once total living area is held constant,25reflecting a room-size trade-off rather than a distaste for bedrooms. The 33.4\%26foreclosure discount is larger than the roughly 27\% forced-sale discount27estimated from Massachusetts transactions by \citet{campbell2011forced}, as28expected given that our estimate reflects asking-price positioning of distressed29listings rather than realized sale prices. Regularized30linear benchmarks (Ridge, Lasso, Elastic Net) achieve essentially identical31performance ($R^2 = 0.628$--$0.630$), confirming that the parametric specification32is not overfit and that the 20-percentage-point gap relative to XGBoost reflects33genuine non-linearities and interactions, not estimation noise.3435The fixed-effects progression carries a sharper lesson. Moving from four Census36regions ($R^2 = 0.634$) to state fixed effects ($0.678$) to 886 ZIP3 fixed effects37($0.725$) shows that a large share of "unexplained" variation is spatial, echoing38the simulation-based advice of \citet{kuminoff2010which} that spatial fixed39effects are the single most consequential specification choice in hedonic work,40and the best-practice guidance of \citet{bishop2020best}. The ZIP3 model closes41roughly half the OLS-to-XGBoost gap, implying that fine geographic partitioning42---not flexible functional form---accounts for much of what tree ensembles add.43This dovetails with \citet{pace2020examining}, who find that machine-learning44methods extract predictive content from the residuals of hedonic models precisely45because those residuals retain spatial structure. The residual Moran's $I$ of460.2745 after regional controls---comparable in magnitude to the residual47autocorrelation documented in metropolitan samples by \citet{basu1998analysis}48---confirms that even our richest interpretable specification leaves the spatial49error structure that the spatial econometrics literature has long modeled50explicitly \citep{anselin1988spatial, lesage2009introduction, dubin1998spatial}.5152Two coefficient-level findings deserve confrontation with the capitalization53literature. The negative school-rating coefficient contradicts the positive54valuation of school quality identified by boundary-discontinuity designs55\citep{black1999better, bayer2007unified}. We read this divergence not as evidence56against school-quality capitalization but as a textbook illustration of why those57designs exist: in a national cross-section, school ratings are collinear with58property-tax rates (which enter separately), regional price levels, and unobserved59neighborhood composition, so the partial coefficient is uninformative about the60structural valuation. Analogously, the negative Walk Score coefficient---despite a61documented walkability premium \citep{pivo2011walkability}---reflects conditioning62simultaneously on bike and transit accessibility; the three scores are highly63correlated, and their premia cannot be attributed variable-by-variable. Both cases64reinforce the paper's interpretive stance: hedonic coefficients are conditional65associations whose structural content depends on identification that a national66listing cross-section does not provide \citep{kuminoff2013new}.6768Finally, the imputation sensitivity analysis shows the baseline lot-size gradient69(0.02) is attenuated roughly threefold by state-median imputation, with the70complete-case estimate (0.06) more plausible though subject to selection. The71broader point---data-quality audits are not optional in scraped-data hedonics---is72a practical extension of the measurement cautions in \citet{bishop2020best}.7374\subsection{Message 2: Quantile Regression Adds Distributional Insight}7576Quantile regression reveals that mean effects mask economically large77distributional heterogeneity, and the shape of that heterogeneity is consistent78with the housing literature. The declining living-area gradient (0.358 at79$\tau = 0.10$ versus 0.251 at $\tau = 0.90$) mirrors the pattern in80\citet{zietz2008determinants}, where square footage is valued relatively more in81lower-priced homes. The rising pool and lot-size gradients toward the top of the82distribution are consistent with amenity-driven luxury pricing documented at upper83quantiles in Hong Kong \citep{mak2010quantile} and Sydney \citep{waltl2019variation}.84The garage gradient---ten times larger at the bottom decile than the top85($z = 28.9$)---extends this logic: basic functional amenities differentiate86lower-priced properties where they are scarce, and become nearly universal (hence87unpriced) upstream. Such quantile-dependent pricing is one empirical face of88housing-market segmentation \citep{goodman1998andrew}: submarkets defined by89price tier value the same attribute bundle differently. Our inter-quantile Wald tests formalize what earlier housing90QR studies typically showed graphically: coefficient equality is rejected for 1191of 13 attributes, so the conditional listing-price distribution is differentially92stretched and compressed by attributes, not merely shifted. This is the93cross-sectional analogue of the finding in \citet{mcmillen2008changes} that house94price distributions move through coefficients rather than characteristics.9596For policy, the pattern implies that improvements to basic amenities are97associated with the largest proportional listing-price differences at the lower98end of the market, while luxury amenities differentiate the top---though we99emphasize, again, that these are associations in asking prices, filtered through100the seller-behavior mechanisms of \citet{genesove2001loss} and \citet{han2016role},101not causal renovation returns.102103The subsample stability analysis shows most quantile coefficients are highly104robust (CV $< 8\%$, 100\% sign stability), with age the notable exception105(CV = 78.6\%). Geographically heterogeneous vintage effects---historic premia in106some markets, obsolescence discounts in others---are the natural reading, and they107caution against interpreting any single national age gradient too literally.108109\subsection{Message 3: ML Performance Depends Critically on Validation Design}110111XGBoost improves on OLS by 20 percentage points of $R^2$ under random validation112(0.833 vs.\ 0.630)---squarely within the 15--25\% error-reduction range reported in113the ML-valuation literature \citep{bourassa2010predicting, park2015using,114chen2016xgboost}. Had we stopped there, the paper would read as one more115confirmation that boosting beats hedonics. The geographic holdout overturns that116reading: under state-level holdout XGBoost attains only $R^2 = 0.425$--$0.547$117depending on specification. The 29--41-point collapse is strikingly similar in118kind to what \citet{ploton2020spatial} document for ecological mapping models,119and it validates, in a housing context, the warnings of \citet{roberts2017cross}120and \citet{meyer2021predicting} about evaluating spatial predictions on randomly121held-out data.122123The ablation analysis identifies \emph{what} fails to transfer. Neighborhood124features contribute the largest random-validation gain ($+17.6$ pp) but only125$+3.5$ pp under geographic holdout---consistent with the conjecture that scores126like Walk Score and school ratings carry market-specific meaning. Validation127research on Walk Score itself finds that its correspondence with underlying128walkability constructs varies across metropolitan areas129\citep{duncan2011validation}, which supports the mechanism, though we do not130test it directly and flag it as plausible rather than established.131More striking, the features most obviously "about" geography are actively harmful132out-of-market: the full model with region dummies loses 7.6 pp of geographic133holdout $R^2$ relative to the market-status stage, and adding raw coordinates134---the most flexible location encoding---produces the largest interpolation gain135($+3.7$ pp random) alongside a large extrapolation loss. Region dummies and136coordinates let trees partition price levels by place, which is exactly what137\citet{pace2020examining} show residual-based ML exploits, and exactly what cannot138transfer to states never seen in training.139140We stress the scope of this conclusion, in light of the debate opened by141\citet{wadoux2021spatial}: for \emph{interpolation}---valuing a property in a142market represented in training data, the typical AVM production setting---random143validation is informative and the geographic features earn their keep. Our claim144concerns \emph{extrapolation} to unrepresented markets, where blocked validation145is the appropriate benchmark \citep{valavi2019blockcv, meyer2021predicting} and146where the AVM-evaluation literature already counsels reporting more than headline147accuracy \citep{steurer2021metrics}. The practical rule for deployment follows:148match the validation protocol to the deployment question, and if the question149involves new markets, prefer the leaner, geography-free specification---it costs1500.3 pp of interpolation accuracy and buys 9.4 pp of extrapolation accuracy.151152\subsection{Message 4: SHAP Interprets Prediction, Not Economics}153154SHAP values identify living area, bathrooms, school quality, lot size, and region155as the dominant predictive contributors, and this ranking is remarkably stable156across three tree ensembles ($\rho = 0.89$--$0.99$). The stability result addresses157a genuine concern in the explainability literature---that importance rankings are158artifacts of a particular fitted model \citep{ribeiro2016should}---and parallels159the motivation for model-agnostic explanation methods. But stability is not160structure. Three cautions from the literature apply directly. First, SHAP161decomposes predictions, not the data-generating process; a feature may earn a162large SHAP value by proxying unobservables \citep{lundberg2017unified,163lundberg2020local}. Second, correlated features share contributions in ways that164defeat attribute-level economic interpretation---our school-rating case is again165illustrative, ranking 3rd by SHAP while its OLS sign is negative and its credible166structural valuation \citep{black1999better, bayer2007unified} is positive.167Third, as \citet{rudin2019stop} argues, post-hoc explanation of a black box is not168a substitute for an interpretable model when stakes are high; in our framework,169the interpretable model (OLS/QR) and the black box answer different questions,170and SHAP does not convert the latter into the former. In Rosen's terms: SHAP171values are not implicit prices, and the stability we document should raise172confidence in SHAP as a description of \emph{this prediction technology}, not as173a measurement of \emph{market valuation}.174175\subsection{Synthesis}176177Across the four messages, a single theme recurs: each framework is reliable178precisely within the question it was built to answer. OLS with rich spatial179controls yields stable, literature-consistent conditional associations; quantile180regression reveals formally significant distributional structure; boosting181delivers real interpolation gains whose extrapolation content must be established182by design, not assumed; and SHAP describes the predictive machine without183licensing economic claims. The methodological corollary---that validation design184and feature choice interact, and that random-split comparisons overstate ML185advantages for out-of-market questions---is, we believe, the paper's most186transferable lesson for both the hedonic and the AVM literatures.187