spb/wp2_uqo

1% Author: Simon-Pierre Boucher — contact@spboucher.ai2%3%======================================================================4\section{Extended Specifications and Validation} \label{sec:extended}5%======================================================================67The analyses in this section were added in the present revision. They address, in turn, the language of the listings, unobserved location and clustered inference, measurement of lot and floor area, the attribution of effects to domains, out-of-sample performance, the sensitivity of the estimates to the phrasing of the references, and the choice of encoder. Table~\ref{tab:spec_summary} summarises the incremental explanatory power of the semantic block across all specifications; Table~\ref{tab:extended_specs} and Figure~\ref{fig:spec_stability} report the individual coefficients in the main ones.89\input{tables/tab_spec_summary}1011\subsection{The Language of the Listing} \label{sec:language}1213Section~\ref{sec:data_language} established that 40\% of the descriptions are in English and that English listings score about 0.23 lower on every French reference with the baseline encoder. Since English listings are also more expensive, every similarity is negatively correlated with price through language, and the regression coefficients of Model~D are identified partly from the comparison of French with English listings. Adding indicators for English and mixed-language descriptions (specification D-L in Table~\ref{tab:spec_summary}) raises the $R^2$ from 0.512 to 0.518, and the English indicator itself carries a coefficient of +19.5\% conditional on structure and length. More importantly, the incremental $R^2$ of the 20 similarities falls from 0.047 in Model~D to 0.017 once language is controlled: about 63\% of the apparent semantic gain of the first version is attributable to the language composition of the corpus. The block remains jointly significant ($F = 22.6$ with HC3 standard errors, 17.6 when clustered by FSA; $p < 0.001$ in both cases), and 16 of the 20 coefficients remain individually significant.1415The language subsamples confirm that the semantic information is not an artefact of the pooled comparison. Within French listings only ($n = 9{,}843$) and with FSA fixed effects, the block adds 0.029 to the $R^2$ and 15 coefficients are significant; within English listings only ($n = 6{,}779$) it adds 0.024, with 15 significant coefficients, even though the references are French and the encoder is monolingual. The signs of the core dimensions are the same in both languages (Table~\ref{tab:extended_specs}): Luxury and Land \& Nature are positive and Family-Friendly, Quiet \& Peaceful and Premium Location are negative in each subsample. Several other coefficients differ across languages, which is itself evidence that the monolingual projection does not measure the same thing in the two halves of the corpus.1617\subsection{Location Fixed Effects and Clustered Standard Errors}1819Specification D-L with 415 FSA fixed effects raises the $R^2$ to 0.667 (within $R^2$ 0.458), and municipality fixed effects (1,061 groups) raise it to 0.710. Location thus explains far more of the variation in asking prices than the semantic block does---a useful corrective to the first version's suggestion that the text features were comparable in importance to neighbourhood controls. Within FSAs, the semantic block still adds 0.020 to the $R^2$ (0.016 within municipalities), with $F = 31.4$ and 18 of the 20 coefficients significant when standard errors are clustered by FSA. The English indicator falls from +19.5\% to +9.1\% within FSAs: about half of the English premium is the location of anglophone neighbourhoods, and half persists within them.2021Clustering matters for inference but not for the conclusions. For the baseline Model~D, clustering by FSA lowers the robust block $F$ from 54.3 to 20.1 and the count of significant coefficients from 16 to 14; the standard errors of the semantic coefficients rise by 50\% on average (between 22\% and 107\%). This is the expected consequence of within-area correlation of listing prices \citep{cameron2015practitioner}, and we regard the clustered estimates as the appropriate ones for all specifications that use location \citep{abadie2023when}.2223\input{tables/tab_extended_specs}2425\begin{figure}[!htbp]26\centering27\includegraphics[width=0.9\textwidth]{fig12_specification_stability.pdf}28\caption{Semantic coefficients across specifications and encoders.}29\label{fig:spec_stability}30\begin{minipage}{0.9\textwidth}31\footnotesize32Notes: Price impact per standard deviation (Equation~\ref{eq:impact}) of each dimension in the baseline Model~D (HC3), in D-L with FSA fixed effects and FSA-clustered standard errors, and in the same specification with the multilingual encoder. Filled markers are significant at 5\%; hollow markers are not. Dimensions are ordered by their baseline coefficient.33\end{minipage}34\end{figure}3536Table~\ref{tab:extended_specs} shows how the individual coefficients respond. Five dimensions keep their sign and significance in every column: Luxury (+14.2\% in the baseline, +10.7\% within FSAs, +4.4\% and +7.6\% in the French and English subsamples), Land \& Nature (+13.0\%, +15.2\%, +10.2\%, +11.1\%), Family-Friendly ($-11.6\%$, $-6.4\%$, $-2.4\%$, $-4.1\%$), Quiet \& Peaceful ($-9.5\%$, $-11.7\%$, $-9.4\%$, $-5.3\%$) and Needs Renovation ($-7.9\%$, $-3.6\%$, insignificant in the French subsample, $-3.7\%$ in the English one). Modern/Contemporary and Waterfront remain positive and significant in all MiniLM specifications. Motivated Seller, by contrast, loses most of its baseline discount once language is controlled ($-8.5\%$ to $-2.9\%$, insignificant) and is only marginally significant within FSAs ($-4.0\%$): the urgency discount of the first version was in large part a language effect, Motivated Seller being the dimension with the largest French--English similarity gap (0.385 against 0.076). Panoramic View changes sign, and Premium Location becomes more negative within FSAs ($-12.6\%$), which is consistent with its reference text (schools, parks, family neighbourhoods) marking suburban segments rather than premium ones.3738\subsection{Measurement of Lot and Floor Area}3940Replacing the raw lot-size field by the logarithm of the unit-corrected lot area raises the $R^2$ of the FSA specification from 0.667 to 0.725 (within $R^2$ from 0.458 to 0.552): a one-standard-deviation increase in log lot area is associated with a 40\% higher asking price within an FSA, the largest structural coefficient in the model. The semantic block's increment falls to 0.013, and 14 coefficients remain significant. The Land \& Nature coefficient falls from +15.2\% to +4.2\%: much of what the ``large wooded lot'' reference captured was the lot area itself, which the raw field had failed to measure. Waterfront and Pool \& Landscaping, by contrast, keep or increase their coefficients (+10.9\% and +11.6\%), and Luxury (+10.4\%) and Modern/Contemporary (+9.6\%) are unchanged. On the 6,228 listings that report an interior floor area, adding its logarithm raises the $R^2$ to 0.773; the semantic block adds 0.011 and 10 coefficients remain significant. The semantic features therefore survive the inclusion of the two size variables that the baseline lacked or mismeasured, with a smaller incremental contribution.4142\subsection{Domain-Level Block Tests}4344Because the 20 dimensions are grouped into six domains (Table~\ref{tab:references}), Table~\ref{tab:block_tests} tests the joint significance of each domain's coefficients in specification D-L, with and without FSA fixed effects. Every domain is jointly significant in both specifications. Location \& Setting is by far the strongest ($F = 67.8$ within FSAs), followed by Quality \& Standing and Style \& Character; Market Signals (Income/Investment and Motivated Seller) is the weakest and is only marginally significant within FSAs ($F = 3.1$, $p = 0.046$). Domain-level inference is less sensitive to the arbitrary allocation of shared vocabulary among references than dimension-level inference, and we recommend it as the primary unit of interpretation for reference-based designs.4546\input{tables/tab_block_tests}4748\subsection{Out-of-Sample Validation}4950Table~\ref{tab:oos} reports five-fold cross-validation with random folds and with folds that hold out entire FSAs (Equations~\ref{eq:r2oos}--\ref{eq:mdape}). Two preliminary remarks. First, the raw lot-size field is not merely uninformative but harmful out of sample: with an extreme value in a held-out fold, the structural model's out-of-sample $R^2$ is $-1.04$; all other rows use the unit-corrected lot area, which also explains why the out-of-sample $R^2$ of the structural model (0.480) exceeds its in-sample counterpart in Table~\ref{tab:model_comparison}. Second, the sample is large relative to the number of regressors, so little shrinkage is expected and the in-sample gains should carry over.5152\input{tables/tab_oos}5354They do. Adding the 20 similarities to the structural-plus-length model raises the out-of-sample $R^2$ from 0.489 to 0.528 with random folds and from 0.484 to 0.522 with FSA-grouped folds, and lowers the median absolute percentage error from 27.1\% to 26.9\%. Adding language indicators to the structural-plus-length model achieves most of this on its own (0.523), and the similarities then add a further 0.011; the multilingual similarities add 0.028 over the language-augmented model and reach the best performance of all specifications (0.551 random, 0.545 spatial, MdAPE 26.4\%), slightly ahead of 20 principal components (0.544 and 0.537). The ordering of specifications is the same under spatial and random folds, and the spatial penalty is small (about 0.006 of $R^2$), so the predictive value of the text generalises to areas not seen in estimation. The gains in valuation accuracy are nevertheless modest in absolute terms---less than one percentage point of MdAPE---because the dominant source of error in these models is the absence of location and floor area, not the absence of text.5556\subsection{Sensitivity to the Phrasing of the References}5758The first version acknowledged that the 20 references embody one researcher's choices but did not quantify the consequence. Table~\ref{tab:reference_sensitivity} does so with three systematic perturbations: keeping only the first half of each reference's comma-separated phrases, only the second half, or reversing their order. The perturbed references are re-encoded and Model~D re-estimated.5960\input{tables/tab_reference_sensitivity}6162The similarity scores are very stable---the perturbed and baseline scores correlate at 0.93 to 0.99 on average across dimensions---and so is the block-level information: the incremental $R^2$ of the block lies between 0.042 and 0.054 against 0.048 in the baseline, and the joint test is always decisive. The individual coefficients are not stable. Reversing the order of the phrases, which changes the texts least, yields coefficients correlated at 0.81 with the baseline and preserves 88\% of the significant signs; keeping the first half yields a correlation of 0.63 and 81\% sign agreement; keeping the second half yields a correlation of 0.02 and 62\% sign agreement, with the Luxury coefficient turning negative and Land \& Nature turning negative. The explanation is the one anticipated in Section~\ref{sec:results}: with 20 highly collinear regressors, a small change in the shared variance among references redistributes the block's explanatory power among its members, so that the vector of coefficients---though not their sum---depends on wording choices that have no economic content. A reader who wishes to use the individual gradients of Table~\ref{tab:full_results} should regard them as one draw from a distribution over admissible phrasings, whose dispersion this table approximates.6364\subsection{A Multilingual Encoder}6566Finally, we replace all-MiniLM-L6-v2 by paraphrase-multilingual-MiniLM-L12-v2, a model distilled so that translations of a sentence receive similar embeddings \citep{reimers2020multilingual}, re-encode the references and the 17,087 descriptions, and recompute the 20 similarities (``ML'' specifications in Table~\ref{tab:spec_summary}). The multilingual similarities correlate only moderately with the baseline ones (0.33 to 0.59 across dimensions), and they largely remove the language gap (Table~\ref{tab:language}). Their information content is greater: the block adds 0.057 to the $R^2$ of the structural-plus-length model, 0.038 once language is controlled, 0.039 within FSAs and 0.028 with the corrected lot area---roughly twice the increments obtained with the baseline encoder in the corresponding specifications---with block $F$-statistics of 52 to 71. Out of sample the multilingual specification is the best of all (Table~\ref{tab:oos}).6768The individual coefficients, however, differ from the baseline in ways that bear directly on the interpretability claim (last column of Table~\ref{tab:extended_specs} and Figure~\ref{fig:spec_stability}). Luxury (+4.5\%), Land \& Nature (+22.1\%), Family-Friendly ($-7.4\%$), Quiet \& Peaceful ($-6.0\%$) and Needs Renovation ($-4.5\%$) keep their signs and significance---the same five dimensions that were stable across the MiniLM specifications. Entry-Level becomes negative ($-4.9\%$) and New Construction positive (+7.2\%), which are the signs one would expect a priori and the opposite of the baseline; Energy Efficient and Bright \& Spacious also change sign. Modern/Contemporary, Waterfront, Motivated Seller and Premium Location become insignificant, and Finished Basement and Pool \& Landscaping become strongly negative. With a different encoder, the same reference texts define different directions in a different space, and only those dimensions whose vocabulary is both distinctive and strongly price-related survive the change. The multilingual encoder is the appropriate choice for this bilingual corpus, and we regard the ML specifications with language indicators and FSA fixed effects as the preferred estimates of this paper; but the lesson of the comparison is general. Reference-based projection delivers a robust, interpretable statement at the level of the block and of a few well-separated dimensions, not twenty robust implicit prices.69