SPB Git

spb/wp2_uqo Public

UQO Working Paper No. 2 — Decoding Real Estate Descriptions: text-based hedonic analysis of housing listings.

TeX 73.8% Python 26%
9.4 KB · 24 lines latex
Raw Blame History
1% Author: Simon-Pierre Boucher — contact@spboucher.ai2%3%======================================================================4\section{Introduction}5%======================================================================67Hedonic pricing models have been the dominant empirical framework for estimating the implicit prices of housing characteristics since the seminal contributions of \citet{rosen1974hedonic} and \citet{lancaster1966new}, building on a quality-adjustment tradition that dates back to the earliest hedonic price indexes \citep{griliches1971price}. The standard approach decomposes observed transaction prices into the marginal contributions of structural attributes (bedrooms, bathrooms, lot size), locational factors (neighborhood quality, proximity to amenities), and environmental characteristics \citep{sirmans2005composition}. Decades of empirical work have refined these models, establishing which structural and spatial variables explain the largest shares of price variation \citep{sirmans2006empirical, malpezzi2003hedonic}, clarifying the conditions under which hedonic estimates identify preference parameters \citep{ekeland2004identification, kuminoff2013new}, and embedding the framework in the operational infrastructure of property taxation, mass appraisal, and automated valuation models that price trillions of dollars of residential real estate \citep{kok2017big}.89Yet a persistent limitation of hedonic models is that they can only price attributes that the econometrician observes and measures. Property listing descriptions---the free-text narratives written by real estate agents to market properties---contain qualitative information that structured data fields fail to capture. These descriptions communicate the quality of finishes, the ambiance of a neighborhood, the motivation of the seller, the architectural character of a home, and the lifestyle a property affords. A listing that describes ``spa-like master bathroom with heated marble floors'' conveys quality information fundamentally different from one that states ``bathroom needs updating.'' Both properties may have the same number of bathrooms, yet the hedonic price contribution differs enormously. As \citet{pakes2003reconsideration} argued, standard hedonic regressions yield biased implicit price estimates when important characteristics are unobservable to the econometrician but observable to market participants---precisely the situation that listing text addresses. \citet{bajari2005demand} formalize this unobserved-characteristics problem in a structural hedonic setting; listing narratives offer a direct, scalable measurement of the missing quality dimensions.1011The opportunity to exploit this information has grown with the text-as-data movement in empirical economics \citep{gentzkow2019text, ash2023text}, which has turned unstructured text into a standard input for measurement in finance \citep{loughran2016textual}, monetary policy \citep{hansen2017transparency}, and labor markets \citep{marinescu2020opening}. Despite this momentum, the hedonic pricing literature has been slow to incorporate unstructured text, and the approaches used to date face a fundamental tradeoff. Keyword and vernacular methods \citep{haag2000real, nowak2017quality, goodwin2014broker} are interpretable but semantically shallow: they cannot recognize that ``quartz countertops'' and ``premium stone surfaces'' convey similar quality information. Sentiment and connotation measures \citep{goodwin2018connotation, zhu2023sentiment} reduce multidimensional quality descriptions to one or two polarity scores. Topic models \citep{blei2003latent} capture latent themes but produce factors that are difficult to interpret and unstable in short, formulaic listing texts. Most recently, machine learning pipelines built on rich text representations \citep{shen2020text, baur2023automated, devlin2019bert} extract substantially more price-relevant information, but the resulting features are opaque---effective for prediction but unsuitable for economic inference where the goal is to estimate the implicit prices of identifiable housing characteristics.1213This paper proposes a \textit{concept-projection} approach to text-augmented hedonic pricing. Rather than using transformer embeddings as black-box predictors, we project them onto economically interpretable semantic anchors. The framework proceeds in four stages. First, we encode each property listing's \textit{PublicRemarks} field into a dense 384-dimensional vector using the all-MiniLM-L6-v2 sentence transformer \citep{wang2020minilm}. Second, we define 20 reference descriptions, each carefully designed to embody a specific qualitative housing dimension---luxury finishes, renovation status, architectural style, waterfront access, seller motivation, and others. Third, we compute the cosine similarity between each listing's embedding and each reference embedding, yielding 20 interpretable scalar features per property. Fourth, we integrate these similarity features into a semi-logarithmic hedonic pricing model estimated by OLS with heteroskedasticity-robust standard errors. In the taxonomy of text-as-data methods, the approach is a semantic generalization of the dictionary method \citep{gentzkow2019text, loughran2016textual}: each reference description plays the role of a word list, but the matching is performed in embedding space, so paraphrases, synonyms, and cross-lingual equivalents load on the intended dimension. The objective is not to claim that changing listing words mechanically changes prices, but to measure whether semantic content embedded in listing narratives captures price-relevant information omitted from structured hedonic variables.1415Quebec provides a particularly interesting empirical context for this analysis. The province's real estate market exhibits substantial heterogeneity---from luxury urban properties in Montr\'eal and Qu\'ebec City to affordable family homes in suburban developments and waterfront properties in the Laurentians and Eastern Townships. Listing descriptions are predominantly in French with occasional English phrases, creating a bilingual corpus that tests the cross-lingual capabilities of the embedding model. The market spans urban, suburban, and peripheral segments with distinct price levels and qualitative characteristics, offering rich variation for semantic analysis. A well-established hedonic research tradition using Quebec microdata \citep{desrosiers2002landscaping, dube2014spatial} provides local benchmarks for the amenity values our text-based measures should---and, as we show, do---recover.1617Our contributions are threefold. First, we demonstrate that semantic features derived from listing text capture economically significant information beyond what structural variables measure. Adding the 20 cosine similarity variables to a standard hedonic specification increases the adjusted $R^2$ from 0.452 to 0.511 for 17,087 single-family homes ($F = 99.53$, $p < 0.001$). The semantic features contribute 9.3\% of the full model's explained variance---comparable to the gains from adding neighborhood controls in typical hedonic studies \citep{sirmans2006empirical}.1819Second, we estimate the implicit price gradients associated with qualitative housing dimensions that have been largely invisible to the hedonic literature. Listings semantically closer to modern/contemporary references are associated with 16.4\% higher prices per standard deviation; luxury-oriented language with 14.2\% premiums; and land/nature descriptions with 13.0\% premiums. Conversely, renovation-need language is associated with 7.9\% discounts and seller-urgency language with 8.5\% discounts. Quantile regressions reveal that these associations are heterogeneous across the price distribution: luxury premiums are amplified at upper quantiles, consistent both with quality complementarities and with the broader finding that implicit prices vary across the conditional price distribution \citep{zietz2007determinants, mcmillen2008changes}.2021Third, the reference-based approach offers a methodological contribution that resolves the depth-interpretability tension in text-augmented hedonic models. Unlike PCA-based or neural-network-based approaches that yield opaque features, each of our similarity measures corresponds to a named, researcher-defined qualitative dimension with a clear coefficient estimate---an ex ante interpretable design in the spirit advocated by \citet{rudin2019stop} for high-stakes applications. This makes the results directly useful for real estate appraisal, market analysis, and automated valuation models (AVMs), and the methodology is portable to any domain where free-text descriptions accompany structured economic data. In doing so, the paper speaks simultaneously to three audiences: housing economists seeking to price unobserved quality, text-as-data practitioners seeking interpretable feature construction, and the interpretable machine learning community seeking economically grounded applications.2223The remainder of the paper is organized as follows. Section~\ref{sec:literature} reviews the four literatures this work draws on. Section~\ref{sec:methodology} describes the methodological framework, including the identification strategy. Section~\ref{sec:data} presents the data and descriptive statistics. Section~\ref{sec:results} reports the empirical results. Section~\ref{sec:robustness} presents a comprehensive robustness analysis. Section~\ref{sec:discussion} discusses interpretation, implications, and limitations. Section~\ref{sec:conclusion} concludes with directions for future research.24