SPB Git forge

spb/wp14_uqo

Public
3commits 1branches 1releases
1.5 MBsize
maindefault branch
34 min agolast push
Python 52.4% TeX 47.6%
7.3 KB · 120 lines latex
Raw Blame History
1% ============================================================================2\section{Data}3\label{sec:data}4% ============================================================================56\subsection{Data Sources}7\label{subsec:data_sources}89We assemble a cross-section of residential listings from two complementary Quebec10platforms, both retrieved on 3~July~2026 with web-scraping actors. The first,11\textbf{Realtor.ca}, is the national Multiple Listing Service (MLS) portal and12provides broker-listed inventory (27{,}302 raw records). The second,13\textbf{DuProprio}, is the dominant for-sale-by-owner (FSBO) platform in Quebec14(7{,}693 raw records). Pooling the two sources broadens coverage across price15segments and sale channels---brokered and FSBO listings differ systematically in16seller composition and marketing \citep{hendel2009mls}---and a source indicator17absorbs any systematic level difference between them. The price we observe is the18\emph{asking} price; Section~\ref{subsec:limits} discusses the interpretation of19hedonic coefficients estimated on list rather than transaction prices20\citep{horowitz1992list,knight2002listing,han2016asking}.2122Each record is a structured JSON object containing a price, an address with a23postal code, a property-attribute block, a free-text description, a list of24features, and an array of image URLs. The two sources differ in their auxiliary25fields: Realtor.ca records include room-by-room dimensions but no geographic26coordinates, whereas DuProprio records include coordinates and captioned images but27no room block. We construct a unique key by concatenating the source and the listing28identifier. Because the data form a single-date cross-section, no time dimension is29modelled and no forward-in-time holdout is possible.3031\subsection{Sample Construction}32\label{subsec:sample}3334We restrict attention to \emph{houses}, defined as single-family dwellings and35townhouses. On Realtor.ca this corresponds to building types \textit{House} and36\textit{Row/Townhouse}; on DuProprio it corresponds to the property types37\textit{Home}, \textit{Bungalow}, \textit{Semi Detached}, \textit{Intergenerational},38\textit{Country Home}, \textit{Cottage}, \textit{Villa}, and \textit{Townhouse}.39Condominiums, income properties (duplex through quintuplex), vacant land, farms,40commercial properties, and mobile homes are excluded. After requiring a non-missing,41positive price, the pooled sample contains 19{,}273 houses.4243\subsection{Outlier Cleaning}44\label{subsec:cleaning}4546Scraped prices contain parsing artifacts spanning several orders of magnitude (the47raw minimum and maximum are \$899 and \$368~billion). We apply conservative,48transparent bounds rather than model-based trimming: prices are retained in the49\$50{,}000--\$15{,}000{,}000 range; living area in 300--20{,}000~ft$^2$; lot area in50200--5{,}000{,}000~ft$^2$; bedrooms in 0--15 and bathrooms in 0--12. When a single51attribute violates its bound it is set to missing (and later imputed) rather than52deleting the observation, preserving sample size. Finally, we trim the53price-per-square-foot distribution at its 0.5th and 99.5th percentiles54(approximately \$73--\$1{,}276/ft$^2$) among listings with a measured area, removing55residual price- or area-parsing errors. The cleaned sample contains56\textbf{19{,}123 houses}, of which 80.3\% originate from Realtor.ca57(Table~\ref{tab:app_flow} in the Appendix summarizes the flow).5859\subsection{Content and Language of the Descriptions}60\label{subsec:text_content}6162Because the language model reads the description verbatim, what the description63contains matters for the interpretation of the text layer. The two platforms differ64sharply. A Realtor.ca description is a free-text paragraph written by the listing65broker (mean length about 540 characters); only 2.5\% of these descriptions contain any66dollar amount, and fewer than 0.3\% mention the municipal assessment. A DuProprio67``description,'' as delivered by the crawler, concatenates the owner's comments with68the platform's structured property sheet rendered as text: room counts, living and69lot area, year of construction, exterior cladding, heating, inclusions, and---in70every one of the 3{,}767 DuProprio listings in the cleaned sample---the asking price71itself (``Asking Price \$499,999''), which equals the recorded list price in 3{,}76172cases; 46.6\% of DuProprio descriptions also mention the municipal assessment. The73language model was therefore shown the dependent variable for 19.7\% of the sample.74The extraction schema has no price field, and the prompt instructs the model to75report only what the text states about the property, so leakage can operate only76indirectly (a model that sees a high price may, for instance, lean towards77``like-new'' when the condition is ambiguous). We treat this as a first-order78threat to the text layer and address it in Sections~\ref{subsec:robust_leakage}79and~\ref{subsec:limits}: all text-layer gains are reported separately for80Realtor.ca, where the text contains no price, and for DuProprio. The vision model81receives only the image URL and never sees any text or price.8283Quebec is a bilingual listing market. Classifying each description from counts of84French and English stop words (a ratio rule of 1.5, as in \citealp{boucher2026decoding}),8544.6\% of descriptions are French, 43.1\% English, and 12.3\% ``mixed''---mostly86DuProprio descriptions that juxtapose an automated English translation and the87French original. Within Realtor.ca, 57.0\% of descriptions are French and 41.9\%88English, and the two groups are priced very differently: the median asking price of89English-language Realtor.ca listings is \$749{,}000 against \$524{,}900 for French90ones, a raw gap of 0.37 log points that falls to 0.21 within municipalities. English91descriptions concentrate in the Montreal West Island and other affluent anglophone92municipalities, so language proxies for location and for the clientele targeted by93the broker. Because the LLM processes both languages with the same schema, language94could also affect the extraction itself. We therefore add language indicators to the95hedonic specification and re-estimate the gains from both AI layers within each96language group (Section~\ref{subsec:robust_language}).9798\subsection{Summary Statistics}99\label{subsec:summary_stats}100101Table~\ref{tab:summary_stats} reports descriptive statistics for the key variables.102The median list price is \$599{,}900. The distribution of living area is right-skewed,103motivating the logarithmic transformation adopted in Section~\ref{sec:methodology}.104Living area is reported for only a minority of Realtor.ca listings; where it is105missing we reconstruct it from room-level dimensions (Section~\ref{subsec:structural}),106and where neither is available we rely on the area-free specification. The room107count is available only for Realtor.ca listings (the DuProprio room block is absent108and coded as zero rooms, with the source indicator absorbing the level).109110\input{tables/summary_stats}111112Table~\ref{tab:ai_summary} previews the machine-generated attributes described in113Section~\ref{sec:methodology}: roughly a quarter of listings are described as recently114renovated, the mean visual condition and curb-appeal scores are 7.4 and 7.2 on a11510-point scale, and 43\% of façades display a visual luxury cue. The text layer116covers all 19{,}123 listings and the vision layer all 19{,}123 photographs, of which11795.8\% were recognized by the model as exterior shots.118119\input{tables/ai_summary}120