SPB Git

spb/wp3_uqo Public

UQO Working Paper No. 3 — Hedonic housing price models for the US: parametric, quantile, and machine-learning approaches.

TeX 77.8% Python 22.1%
11.8 KB · 193 lines latex
Raw Blame History
1% Author: Simon-Pierre Boucher — contact@spboucher.ai2%3% ═══════════════════════════════════════════════════════════════════════4% 3. DATA5% ═══════════════════════════════════════════════════════════════════════6\section{Data and Variable Construction}7\label{sec:data}89\subsection{Data Source}1011The dataset is sourced from Zillow, the largest online real estate marketplace in the United States. The raw data contain 839,313 residential properties with 116 variables, representing active for-sale listings as of 2025--2026. The data include listing prices, structural characteristics, geocoordinates, neighborhood quality scores, and tax/transaction histories.1213\textbf{Important limitations of the data source.} The sample comprises active Zillow listings, not the universe of U.S.\ residential properties. Several sources of non-representativeness should be noted: (i) Zillow coverage varies by region, with some MLS-linked markets better represented than others; (ii) active listings capture supply at a point in time, not the full housing stock; (iii) the sample excludes off-market properties, FSBOs not listed on Zillow, and recently transacted properties that are no longer listed; (iv) Southern states are overrepresented (56.6\% of the sample vs.\ approximately 38\% of U.S.\ housing units), likely reflecting higher listing volumes in fast-growing Sun Belt markets. We do not claim national representativeness and caution against interpreting our estimates as population parameters for the entire U.S.\ housing market.1415\subsection{Listing Prices versus Transaction Prices}1617A critical limitation is that our dependent variable is the \textit{listing} (asking) price, not the realized transaction price. Listing prices are strategic objects: theory models them as commitment devices in seller search \citep{horowitz1992role} and as instruments that direct buyer attention \citep{han2016role}, and empirically they embed seller behavior---overpricing lengthens time-on-market and lowers eventual sale prices \citep{knight2002listing}, while loss-averse sellers systematically set higher asking prices \citep{genesove2001loss}. List-to-sale price ratios therefore vary by market condition, property type, and price tier: luxury properties are more frequently overpriced, distressed properties may be strategically underpriced, and regional norms for overbidding versus negotiation differ substantially. Throughout this paper, we interpret the estimated coefficients as \textbf{listing-price capitalization gradients}---conditional associations between attributes and asking prices---rather than transaction-price implicit prices. If listing premiums correlate systematically with property attributes (e.g., if waterfront properties are more frequently overpriced), the estimated gradients will reflect this pricing behavior in addition to underlying valuation differences.1819\subsection{Sample Construction}2021Table~\ref{tab:attrition} documents the sample construction.2223\begin{table}[H]24\centering25\caption{Sample Construction}26\label{tab:attrition}27\begin{threeparttable}28\begin{tabular}{lrrr}29\toprule30Step & $N$ remaining & Removed & Criterion \\31\midrule32Raw Zillow records & 839,313 & --- & --- \\33Valid price & 817,473 & 21,840 & $\$10{,}000 < P < \$10{,}000{,}000$ \\34Valid living area & 797,380 & 20,093 & $200 < \text{sqft} < 20{,}000$ \\35Valid bedrooms/bathrooms & 790,799 & 6,581 & $1 \leq \text{bed} \leq 10$, $1 \leq \text{bath} \leq 10$ \\36Valid geocoordinates & 789,199 & 1,600 & Non-missing lat/lon \\37U.S.\ states + DC only & 788,842 & 357 & Exclude PR, VI \\38\midrule39\textbf{Final analytical sample} & \textbf{788,842} & \textbf{50,471} & \textbf{(6.0\% removed)} \\40\bottomrule41\end{tabular}42\begin{tablenotes}43\small44\item \textit{Notes:} Filters applied sequentially. The 6.0\% removal rate suggests the raw data are reasonably clean, though the retained sample may still contain measurement error in secondary variables.45\end{tablenotes}46\end{threeparttable}47\end{table}4849\subsection{Variable Description}5051Our specification includes 62 regressors organized into six categories.5253\subsubsection{Structural Attributes (8 Variables)}5455The natural logarithm of living area ($\ln\text{sqft}$), number of bedrooms, number of bathrooms, property age ($2026 - \text{year\_built}$) and its square, number of stories, the bathroom-to-bedroom ratio, and square feet per bedroom. Living area enters in log form to accommodate the well-documented concavity of the size-price relationship.5657\subsubsection{Lot Characteristics (1 Variable)}5859The natural logarithm of lot size ($\ln\text{lot}$). Missing lot sizes (16.3\% of raw data) are imputed using state-level medians.6061\subsubsection{Amenity and Quality Indicators (11 Variables)}6263Binary indicators for swimming pool (39.8\% prevalence), spa (6.1\%), basement (24.3\%), fireplace (39.6\%), garage (67.2\%), waterfront location (8.9\%), central air conditioning (68.1\%), forced air heating (25.8\%), and hardwood flooring (26.7\%). The number of parking spaces and a composite luxury score (0--7) supplement these indicators.6465\subsubsection{Neighborhood and Location Variables (7 Variables)}6667Walk Score (0--100), Bike Score, Transit Score, average GreatSchools rating (1--10), school count, distance to nearest school, and local property tax rate. Walk Score is the accessibility measure for which a capitalization premium has been documented in the literature \citep{pivo2011walkability}; school ratings and tax rates proxy the local public goods whose capitalization into house prices is the subject of a large identification literature \citep{oates1969effects, black1999better, bayer2007unified}.6869\textbf{Data quality note.} Bike Score has a maximum value of 248 in our data, exceeding the expected 0--100 range. Only 38 observations (0.005\%) exceed 100, and these are retained without truncation. Results are robust to capping Bike Score at 100.7071\subsubsection{Market Status Variables (5 Variables)}7273Condominium indicator (14.6\%), HOA membership (37.1\%), log annualized HOA fees ($\ln(1 + \text{HOA}_\text{annual})$), new construction (5.9\%), and foreclosure status (0.5\%).7475\subsubsection{Interaction Terms (6 Variables)}7677\begin{itemize}[nosep]78    \item $\ln(\text{sqft}) \times \text{age}$: depreciation-size interaction;79    \item $\text{pool} \times \text{South}$: climate-dependent pool gradient;80    \item $\text{waterfront} \times \ln(\text{sqft})$: size-dependent waterfront gradient;81    \item $\text{basement} \times \text{North}$: region-dependent basement gradient;82    \item $\text{condo} \times \text{walk\_score}$: walkability gradient for condominiums;83    \item $\text{age} \times \text{luxury\_score}$: luxury mitigation of depreciation.84\end{itemize}8586\subsubsection{Categorical Controls (24 Dummy Variables)}8788Simplified indicators for roof type (6 dummies), construction material (8 dummies), foundation type (7 dummies), and Census region (3 dummies; Midwest reference).8990\subsection{Missing Data and Imputation}9192Table~\ref{tab:missing} reports missingness rates for key variables.9394\begin{table}[H]95\centering96\caption{Missing Data Rates and Imputation Methods}97\label{tab:missing}98\begin{threeparttable}99\small100\begin{tabular}{lrrl}101\toprule102Variable & Missing (\%) & Method & Notes \\103\midrule104Year built & 19.4 & State median & Affects age, age$^2$, interactions \\105Lot size & 16.3 & State median & Large variation \\106Walk Score & 3.1 & State median & \\107Bike Score & 6.1 & State median & Max = 248 (38 obs.) \\108Transit Score & 70.1 & Set to 0 & Suburban/rural properties \\109Garage & 35.1 & Set to False & Conservative \\110Climate risk factors & 100.0 & Excluded & Not available \\111\bottomrule112\end{tabular}113\begin{tablenotes}114\small115\item \textit{Notes:} State-level median imputation is used to preserve geographic variation. The high Transit Score missingness (70.1\%) likely reflects properties in areas without public transit infrastructure, for which zero is a reasonable value. Climate risk factors (flood, fire, heat, wind, air) are entirely missing and excluded from the analysis.116\end{tablenotes}117\end{threeparttable}118\end{table}119120\subsection{Data Quality Audit}121\label{sec:data_audit}122123We conduct three data quality checks to assess the sensitivity of key results to imputation and outliers.124125\textbf{Imputation prevalence.} Two variables---year built (19.4\% missing) and lot size (16.3\% missing)---account for the bulk of imputed values. To assess the consequences, we re-estimate the baseline OLS model on subsamples that exclude imputed observations (Section~\ref{sec:imputation_sensitivity}). The lot-size coefficient is the most sensitive: $\ln(\text{lot})$ triples from 0.018 to 0.059 when imputed lot sizes are dropped, suggesting that state-median imputation attenuates the lot-size gradient substantially.126127\textbf{Outlier prevalence.} We winsorize lot size at its 99.5th percentile and cap Bike Score at 100, then re-estimate OLS (Section~\ref{sec:winsorization}). The overall $R^2$ increases modestly from 0.634 to 0.640, and most coefficients are stable, with the exception of $\ln(\text{lot})$, which again increases substantially.128129\textbf{Geographic coverage.} The sample spans 886 unique three-digit ZIP code prefixes (ZIP3 codes) across 51 jurisdictions (50 states plus DC). Southern states account for 56.6\% of observations. We document the marginal explanatory power of progressively finer geographic controls in Section~\ref{sec:ols_fe}.130131\subsection{Descriptive Statistics}132133Table~\ref{tab:descriptive} presents summary statistics.134135\begin{table}[H]136\centering137\caption{Descriptive Statistics ($N = 788{,}842$)}138\label{tab:descriptive}139\begin{threeparttable}140\small141\begin{tabular}{lrrrrrr}142\toprule143Variable & Mean & Std.\ Dev. & Min & Median & Max \\144\midrule145\multicolumn{6}{l}{\textit{Continuous Variables}} \\146Listing Price (\$) & 621,737 & 817,965 & 10,300 & 399,900 & 9,999,999 \\147Living Area (sqft) & 2,093 & 1,181 & 208 & 1,820 & 19,600 \\148Bedrooms & 3.25 & 1.09 & 1 & 3 & 10 \\149Bathrooms & 2.52 & 1.16 & 1 & 2 & 10 \\150Property Age (years) & 42.2 & 31.5 & 0 & 36 & 200 \\151Walk Score (0--100) & 25.6 & 26.5 & 0 & 16 & 100 \\152Bike Score & 34.9 & 18.1 & 1 & 31 & 248 \\153Transit Score (0--100) & 8.9 & 18.3 & 0 & 0 & 100 \\154Avg.\ School Rating (1--10) & 6.02 & 0.90 & 1 & 6 & 10 \\155Property Tax Rate (\%) & 1.10 & 0.53 & 0 & 1.0 & 4.0 \\156Luxury Score (0--7) & 1.36 & 1.29 & 0 & 1 & 7 \\157\midrule158\multicolumn{6}{l}{\textit{Binary Variables (prevalence)}} \\159Swimming Pool & 39.8\% & & & & \\160Garage & 67.2\% & & & & \\161Fireplace & 39.6\% & & & & \\162Basement & 24.3\% & & & & \\163Waterfront & 8.9\% & & & & \\164HOA & 37.1\% & & & & \\165Central Air & 68.1\% & & & & \\166Hardwood Floors & 26.7\% & & & & \\167Condominium & 14.6\% & & & & \\168New Construction & 5.9\% & & & & \\169Foreclosure & 0.5\% & & & & \\170\midrule171\multicolumn{6}{l}{\textit{Regional Distribution}} \\172South & 56.6\% & & & & \\173West & 18.9\% & & & & \\174Midwest & 16.1\% & & & & \\175Northeast & 8.4\% & & & & \\176\bottomrule177\end{tabular}178\begin{tablenotes}179\small180\item \textit{Notes:} Listing prices, not transaction prices. Property age computed as $2026 - \text{year\_built}$, after imputation. Imputed values included in summary statistics.181\end{tablenotes}182\end{threeparttable}183\end{table}184185The median listing price is \$399,900, with substantial right-skewness (mean \$621,737). The log transformation substantially improves distributional symmetry (Figure~\ref{fig:price_dist}), supporting the semi-log specification.186187\begin{figure}[H]188    \centering189    \includegraphics[width=\textwidth]{figures/fig1_price_distribution.png}190    \caption{Distribution of Listing Prices: (a) Levels; (b) Natural Logarithm. The log transformation reduces right skewness and approximates normality.}191    \label{fig:price_dist}192\end{figure}193