Python 52.4%
TeX 47.6%
1% ============================================================================2\section{Predictive Benchmark: From OLS to Deep Learning}3\label{sec:benchmark}4% ============================================================================56The hedonic OLS of Section~\ref{sec:results} is designed for interpretation: its7coefficients are implicit prices. It is not, however, the most accurate predictor of8price. To map the accuracy frontier---and to gauge how much curvature and interaction9the linear model leaves on the table---we benchmark a ladder of estimators of10increasing flexibility on the \emph{identical} feature set, under a common11five-fold cross-validation protocol. All rows of Table~\ref{tab:models} share the same12folds and random seed, so differences reflect the estimator, not the split.1314\subsection{The Estimator Ladder}15\label{subsec:ladder}1617\paragraph{(1) Ordinary least squares.} The linear hedonic of18Equation~\eqref{eq:hedonic}, with one-hot city, property-type, and source indicators19and within-property-type median imputation of missing continuous attributes. It is20the interpretable baseline and, mechanically, the least flexible: it imposes constant21marginal effects and additive separability.2223\paragraph{(2) Random forest.} A bagged ensemble of regression trees24\citep{breiman2001random,mullainathan2017machine}. Trees capture non-linearities and25low-order interactions automatically and handle missing values natively, but26averaging deep decorrelated trees tends to over-smooth sharp price thresholds.2728\paragraph{(3) Gradient boosting.} A histogram-based gradient-boosted tree ensemble29(600 boosting iterations, learning rate 0.05, up to 63 leaves, $L_2$ regularization).30Boosting fits trees sequentially to residuals, which lets it model both smooth31curvature (age, area) and abrupt thresholds; like the forest it ingests missing32values directly. It is the strongest single model in our comparison.3334\paragraph{(4) Neural networks.} A plain multilayer perceptron and a robust35residual network with entity embeddings, described in36Section~\ref{subsec:nn}. Neural networks are attractive when high-cardinality37categoricals (here, 195 municipalities) can be represented as dense learned vectors38rather than sparse indicators.3940\paragraph{(5) Blend.} An equally weighted average of the robust neural network and41the gradient-boosting predictions. Because the two models make partly decorrelated42errors---the network smooths through embeddings while the boosted trees cut sharp43thresholds---their average cancels idiosyncratic noise, a standard model-averaging44result. The weight of one half was fixed \emph{a priori}; Section~\ref{subsec:bench_results}45reports how the result varies with the weight and under a nested choice of the46weight.4748\subsection{Formal Definitions}49\label{subsec:formal}5051Let $y_i=\ln P_i$ and let $\mathbf{x}_i$ denote the feature vector of52Equation~\eqref{eq:blocks}. Each estimator approximates the conditional mean53$f(\mathbf{x})=\mathbb{E}[y\mid\mathbf{x}]$.5455\paragraph{Linear (OLS).} $f(\mathbf{x})=\mathbf{x}^{\!\top}\bm{\theta}$ with the56closed form of Section~\ref{subsec:econometrics}; categoricals enter as one-hot57indicators.5859\paragraph{Random forest \citep{breiman2001random}.} An average of $B$ regression60trees, each grown on a bootstrap resample with a random feature subset at every split:61%62\begin{equation}63 \hat f_{\text{RF}}(\mathbf{x})=\frac1B\sum_{b=1}^{B} T_b(\mathbf{x};\Theta_b),64 \label{eq:rf}65\end{equation}66%67where $\Theta_b$ denotes the randomness (resample and split subsets) of tree~$b$;68we use $B=400$ trees, half of the features at each split, and a minimum leaf size69of three.7071\paragraph{Gradient boosting \citep{friedman2001greedy}.} A forward stagewise additive72expansion built by fitting each new tree to the negative gradient of the loss:73%74\begin{equation}75 \resizebox{.94\textwidth}{!}{$\displaystyle76 F_M(\mathbf{x}) = F_0 + \nu\sum_{m=1}^{M} f_m(\mathbf{x}),77 \quad78 f_m = \arg\min_{f}\sum_{i=1}^{n}79 \big(\!-g_{mi}-f(\mathbf{x}_i)\big)^2,80 \quad81 g_{mi}=\frac{\partial L\big(y_i,F_{m-1}(\mathbf{x}_i)\big)}{\partial F_{m-1}(\mathbf{x}_i)}82 $}83 \label{eq:gbm}84\end{equation}85%86with learning rate $\nu$, squared-error loss $L$, and $L_2$-regularized histogram87trees $f_m$; missing values are routed to a learned default direction at each split.8889\paragraph{Neural network (robust).} High-cardinality categoricals are mapped to90learned embeddings \citep{guo2016entity}, concatenated with the batch-normalized91numeric block and projected to a $d$-dimensional trunk:92%93\begin{equation}94 \mathbf{h}_0 = \mathbf{W}_095 \Big[\, \textstyle\bigoplus_{c\in\{\text{city},\text{type},\text{src}\}}96 \mathbf{E}_c(\text{cat}_{ci})97 \;\Vert\; \mathrm{BN}(\mathbf{x}^{\text{num}}_i)\,\Big].98 \label{eq:nn_in}99\end{equation}100%101The trunk passes through $L=3$ pre-activation residual blocks102\citep{he2016deep,gorishniy2021revisiting},103%104\begin{equation}105 \mathbf{h}_{\ell}106 = \mathbf{h}_{\ell-1}107 + \mathrm{Drop}\!\Big(\mathbf{W}_2^{(\ell)}\,108 \mathrm{Drop}\!\big(\phi(\mathbf{W}_1^{(\ell)}\,109 \mathrm{BN}(\mathbf{h}_{\ell-1}))\big)\Big),110 \qquad \phi=\text{GELU},111 \label{eq:resblock}112\end{equation}113%114and a normalization head yields $\hat y_i = \mathbf{w}^{\!\top}\phi(\mathrm{BN}(\mathbf{h}_L))+b$.115Parameters minimize the Huber (smooth-$L_1$) loss, robust to price outliers,116%117\begin{equation}118 L_\delta(r)=119 \begin{cases}120 \tfrac12 r^2, & |r|\le \delta,\\[2pt]121 \delta\big(|r|-\tfrac12\delta\big), & |r|>\delta,122 \end{cases}123 \qquad r_i=\hat y_i-y_i,\;\delta=1,124 \label{eq:huber}125\end{equation}126%127on the standardized target, optimized with AdamW (decoupled weight decay)128\citep{loshchilov2019decoupled} under a cosine-annealed learning rate129$\eta_t=\tfrac12\eta_0\big(1+\cos(\pi t/T)\big)$, gradient-norm clipping, and early130stopping on a 10\% validation split of each training fold. Three seeds are averaged131per fold.132133\paragraph{Blend.} A convex combination of the two strongest models in log space,134%135\begin{equation}136 \widehat{\ln P}^{\,\text{blend}}137 = w\,\widehat{\ln P}^{\,\text{GBM}}138 + (1-w)\,\widehat{\ln P}^{\,\text{NN}},139 \qquad w=0.5.140 \label{eq:blend}141\end{equation}142143\paragraph{Evaluation.} With $\hat y_{-k(i)}$ the prediction for $i$ from the model144trained on the folds excluding $i$'s fold, we report out-of-sample fit and level145accuracy,146%147\begin{equation}148 R^2_{\text{OOS}} = 1-\frac{\sum_i\big(y_i-\hat y_{-k(i)}\big)^2}149 {\sum_i\big(y_i-\bar y\big)^2},150 \qquad151 \text{MdAPE}=\operatorname*{median}_i\Big|e^{\,\hat y_{-k(i)}-y_i}-1\Big|,152 \label{eq:cvmetrics}153\end{equation}154%155together with the share of listings predicted within $\pm15\%$ and $\pm25\%$ of the156asking price. The MdAPE is computed on the ratio of predicted to actual price and is157therefore unaffected by the retransformation constant of158Equation~\eqref{eq:smearing}.159160\subsection{Results}161\label{subsec:bench_results}162163\input{tables/models}164165Table~\ref{tab:models} reports the ladder in two panels. In Panel~A (structural,166amenity, room, listing, and text features) out-of-sample $R^2$ rises monotonically167with flexibility: from 0.628 for OLS, to 0.677 for the random forest, 0.711 for the168plain network, and a near-tie at the top between the robust network (0.741) and169gradient boosting (0.743); their blend attains $R^2_{\text{OOS}}=0.760$. Panel~B adds170the façade vision layer. Vision lifts every estimator---OLS gains 5.8 points171(0.628 $\to$ 0.686), the random forest 2.7 (0.677 $\to$ 0.704), the robust network1720.6 (0.741 $\to$ 0.747), and gradient boosting 2.7 (0.743 $\to$ 0.770)---and the173text\,$+$\,vision blend attains $R^2_{\text{OOS}}=0.779$ with a median absolute174percentage error of 14.0\%, the best model in the study.175176Three lessons follow. First, on the full feature set roughly nine points of177out-of-sample $R^2$ separate the interpretable linear model (0.686) from the178predictive frontier (0.779)---the price of interpretability, and a measure of the179curvature and interactions the hedonic form omits. Notably, the vision layer buys the180linear model more accuracy ($+5.8$ points) than switching from OLS to a random forest181does ($+4.9$ points in Panel~A), so better measurement rivals better estimation.182Second, on this tabular problem gradient boosting and a carefully regularized neural183network are statistically comparable, consistent with the broader evidence that tree184ensembles remain strong baselines on tabular data \citep{grinsztajn2022why}; the185gains come from \emph{combining} them. Third, the façade-vision layer is not redundant186with the text and structural fields: it is the more informative of the two AI layers187for every estimator.188189The equal-weight blend is not the best achievable combination, but it is close.190Varying the weight $w$ on gradient boosting in Equation~\eqref{eq:blend} over191$\{0.3,0.4,0.5,0.6,0.7\}$ gives out-of-sample $R^2$ of 0.771, 0.776, 0.779, 0.780,192and 0.780 for the text\,$+$\,vision models, so $w=0.5$ is within 0.002 of the best193weight on the grid; choosing $w$ in a nested fashion---on the four training folds,194then applied to the fifth---returns $R^2_{\text{OOS}}=0.775$, slightly below the195fixed equal weight, because the fold-specific optimal weights are noisy. For the196text-only models the equal weight is exactly the best on the grid (0.760). We197therefore keep the fixed equal weight and do not tune it on the test predictions.198199\begin{figure}[htbp]200 \centering201 \includegraphics[width=.72\textwidth]{figures/fig_ablation.pdf}202 \caption{Out-of-sample $R^2$ as feature layers are added (nested, left to right),203 random five-fold cross-validation. Each layer improves both the linear and boosted204 models; the façade-vision layer adds about $+5.8$ points for OLS and $+2.7$ for205 gradient boosting, the text layer $+0.9$ and $+1.4$.}206 \label{fig:ablation}207\end{figure}208209\subsection{Feature Ablation and Importance}210\label{subsec:ablation}211212Table~\ref{tab:ablation} and Figure~\ref{fig:ablation} decompose the accuracy gains213layer by layer. Reading down the table, the room-detail and listing-signal block adds214the largest increment of all (9.8 points for OLS, 3.7 for gradient boosting), which215reflects both the information in the Realtor.ca room block and the fact that this216block carries the platform's identity; the LLM text layer adds 0.9 and 1.4 points;217and the vision layer adds 5.8 and 2.7 points, the largest increment of the two AI218layers and the second largest overall. The pattern is monotone: no layer degrades219fit. Under spatially grouped folds (Table~\ref{tab:rob_spatial},220Section~\ref{subsec:robust_spatial}) all levels fall by three to five points, but the221increments from the text and vision layers are essentially unchanged.222223\input{tables/ablation}224225Figure~\ref{fig:importance} aggregates the gradient-boosting permutation importances226into feature blocks. Structural attributes and location fixed effects dominate, as227expected, but the vision block accounts for 13\% of total importance---more than the228rule-based amenities and the LLM text block combined---so the façade photograph is a229first-order, not marginal, source of price information in the boosted model.230Figure~\ref{fig:calibration} shows that the blend is well calibrated across the price231range, with an approximately symmetric error distribution centred near zero.232233\begin{figure}[htbp]234 \centering235 \includegraphics[width=.72\textwidth]{figures/fig_importance.pdf}236 \caption{Share of gradient-boosting permutation importance by feature block237 (text\,$+$\,vision model). The vision block accounts for 13\% of total importance,238 exceeding the rule-based amenity and LLM-text blocks.}239 \label{fig:importance}240\end{figure}241242\begin{figure}[htbp]243 \centering244 \includegraphics[width=\textwidth]{figures/fig_calibration.pdf}245 \caption{Out-of-sample calibration of the text\,$+$\,vision blend. Left: predicted246 versus actual price (log--log; dashed line is the 45$^\circ$ identity). Right:247 distribution of percentage prediction errors, with median absolute error of 14.0\%.}248 \label{fig:calibration}249\end{figure}250251\subsection{A Robust Tabular Neural Network}252\label{subsec:nn}253254Figure~\ref{fig:nn} details the robust network. High-cardinality categoricals enter255as learned \textbf{entity embeddings}: each municipality, property type, and source is256mapped to a dense vector (dimension $\approx 1.6\,C^{0.56}$ in cardinality $C$, so257about 30 dimensions for the 195 municipalities). The embeddings are concatenated with258the batch-normalized numeric block and projected to a 256-dimensional trunk, which259passes through three \textbf{residual blocks} (BatchNorm $\to$ Linear$_{512}$ $\to$260GELU $\to$ Dropout $\to$ Linear$_{256}$ $\to$ Dropout, with a skip connection),261followed by a normalization head that outputs $\ln(\text{Price})$.262263Robustness comes from the training recipe rather than raw capacity:264a \textbf{Huber (smooth-$L_1$) loss} that damps the influence of price outliers;265\textbf{AdamW} with decoupled weight decay and a \textbf{cosine-annealed} learning rate;266\textbf{gradient-norm clipping} at 5; \textbf{dropout} and batch normalization267throughout; \textbf{missingness indicators} appended to the numeric block so268the network can exploit the non-random pattern of missing fields; and269\textbf{early stopping} on a held-out 10\% of each training fold. Finally, three270networks with different random seeds are trained per fold and \textbf{averaged}, which271reduces prediction variance. The result (0.741 with text, 0.747 with text and vision)272is competitive with gradient boosting and, once blended with it, produces the best273model in the study.274275\begin{figure}[htbp]276 \centering277 \input{figures/nn_diagram}278 \caption{Architecture of the robust tabular neural network. High-cardinality279 categoricals enter as learned embeddings; numeric features are batch-normalized and280 augmented with missingness indicators. The concatenated representation is projected281 to a 256-dimensional trunk, refined by three residual blocks, and mapped to282 $\ln(\text{Price})$. Three seeds are ensembled per fold; the final model blends this283 network equally with gradient boosting.}284 \label{fig:nn}285\end{figure}286