% ============================================================================
\section{Predictive Benchmark: From OLS to Deep Learning}
\label{sec:benchmark}
% ============================================================================

The hedonic OLS of Section~\ref{sec:results} is designed for interpretation: its
coefficients are implicit prices. It is not, however, the most accurate predictor of
price. To map the accuracy frontier---and to gauge how much curvature and interaction
the linear model leaves on the table---we benchmark a ladder of estimators of
increasing flexibility on the \emph{identical} feature set, under a common
five-fold cross-validation protocol. All rows of Table~\ref{tab:models} share the same
folds and random seed, so differences reflect the estimator, not the split.

\subsection{The Estimator Ladder}
\label{subsec:ladder}

\paragraph{(1) Ordinary least squares.} The linear hedonic of
Equation~\eqref{eq:hedonic}, with one-hot city, property-type, and source indicators
and within-property-type median imputation of missing continuous attributes. It is
the interpretable baseline and, mechanically, the least flexible: it imposes constant
marginal effects and additive separability.

\paragraph{(2) Random forest.} A bagged ensemble of regression trees
\citep{breiman2001random,mullainathan2017machine}. Trees capture non-linearities and
low-order interactions automatically and handle missing values natively, but
averaging deep decorrelated trees tends to over-smooth sharp price thresholds.

\paragraph{(3) Gradient boosting.} A histogram-based gradient-boosted tree ensemble
(600 boosting iterations, learning rate 0.05, up to 63 leaves, $L_2$ regularization).
Boosting fits trees sequentially to residuals, which lets it model both smooth
curvature (age, area) and abrupt thresholds; like the forest it ingests missing
values directly. It is the strongest single model in our comparison.

\paragraph{(4) Neural networks.} A plain multilayer perceptron and a robust
residual network with entity embeddings, described in
Section~\ref{subsec:nn}. Neural networks are attractive when high-cardinality
categoricals (here, 195 municipalities) can be represented as dense learned vectors
rather than sparse indicators.

\paragraph{(5) Blend.} An equally weighted average of the robust neural network and
the gradient-boosting predictions. Because the two models make partly decorrelated
errors---the network smooths through embeddings while the boosted trees cut sharp
thresholds---their average cancels idiosyncratic noise, a standard model-averaging
result. The weight of one half was fixed \emph{a priori}; Section~\ref{subsec:bench_results}
reports how the result varies with the weight and under a nested choice of the
weight.

\subsection{Formal Definitions}
\label{subsec:formal}

Let $y_i=\ln P_i$ and let $\mathbf{x}_i$ denote the feature vector of
Equation~\eqref{eq:blocks}. Each estimator approximates the conditional mean
$f(\mathbf{x})=\mathbb{E}[y\mid\mathbf{x}]$.

\paragraph{Linear (OLS).} $f(\mathbf{x})=\mathbf{x}^{\!\top}\bm{\theta}$ with the
closed form of Section~\ref{subsec:econometrics}; categoricals enter as one-hot
indicators.

\paragraph{Random forest \citep{breiman2001random}.} An average of $B$ regression
trees, each grown on a bootstrap resample with a random feature subset at every split:
%
\begin{equation}
  \hat f_{\text{RF}}(\mathbf{x})=\frac1B\sum_{b=1}^{B} T_b(\mathbf{x};\Theta_b),
  \label{eq:rf}
\end{equation}
%
where $\Theta_b$ denotes the randomness (resample and split subsets) of tree~$b$;
we use $B=400$ trees, half of the features at each split, and a minimum leaf size
of three.

\paragraph{Gradient boosting \citep{friedman2001greedy}.} A forward stagewise additive
expansion built by fitting each new tree to the negative gradient of the loss:
%
\begin{equation}
  \resizebox{.94\textwidth}{!}{$\displaystyle
  F_M(\mathbf{x}) = F_0 + \nu\sum_{m=1}^{M} f_m(\mathbf{x}),
  \quad
  f_m = \arg\min_{f}\sum_{i=1}^{n}
  \big(\!-g_{mi}-f(\mathbf{x}_i)\big)^2,
  \quad
  g_{mi}=\frac{\partial L\big(y_i,F_{m-1}(\mathbf{x}_i)\big)}{\partial F_{m-1}(\mathbf{x}_i)}
  $}
  \label{eq:gbm}
\end{equation}
%
with learning rate $\nu$, squared-error loss $L$, and $L_2$-regularized histogram
trees $f_m$; missing values are routed to a learned default direction at each split.

\paragraph{Neural network (robust).} High-cardinality categoricals are mapped to
learned embeddings \citep{guo2016entity}, concatenated with the batch-normalized
numeric block and projected to a $d$-dimensional trunk:
%
\begin{equation}
  \mathbf{h}_0 = \mathbf{W}_0
  \Big[\, \textstyle\bigoplus_{c\in\{\text{city},\text{type},\text{src}\}}
  \mathbf{E}_c(\text{cat}_{ci})
  \;\Vert\; \mathrm{BN}(\mathbf{x}^{\text{num}}_i)\,\Big].
  \label{eq:nn_in}
\end{equation}
%
The trunk passes through $L=3$ pre-activation residual blocks
\citep{he2016deep,gorishniy2021revisiting},
%
\begin{equation}
  \mathbf{h}_{\ell}
  = \mathbf{h}_{\ell-1}
  + \mathrm{Drop}\!\Big(\mathbf{W}_2^{(\ell)}\,
    \mathrm{Drop}\!\big(\phi(\mathbf{W}_1^{(\ell)}\,
    \mathrm{BN}(\mathbf{h}_{\ell-1}))\big)\Big),
  \qquad \phi=\text{GELU},
  \label{eq:resblock}
\end{equation}
%
and a normalization head yields $\hat y_i = \mathbf{w}^{\!\top}\phi(\mathrm{BN}(\mathbf{h}_L))+b$.
Parameters minimize the Huber (smooth-$L_1$) loss, robust to price outliers,
%
\begin{equation}
  L_\delta(r)=
  \begin{cases}
    \tfrac12 r^2, & |r|\le \delta,\\[2pt]
    \delta\big(|r|-\tfrac12\delta\big), & |r|>\delta,
  \end{cases}
  \qquad r_i=\hat y_i-y_i,\;\delta=1,
  \label{eq:huber}
\end{equation}
%
on the standardized target, optimized with AdamW (decoupled weight decay)
\citep{loshchilov2019decoupled} under a cosine-annealed learning rate
$\eta_t=\tfrac12\eta_0\big(1+\cos(\pi t/T)\big)$, gradient-norm clipping, and early
stopping on a 10\% validation split of each training fold. Three seeds are averaged
per fold.

\paragraph{Blend.} A convex combination of the two strongest models in log space,
%
\begin{equation}
  \widehat{\ln P}^{\,\text{blend}}
  = w\,\widehat{\ln P}^{\,\text{GBM}}
  + (1-w)\,\widehat{\ln P}^{\,\text{NN}},
  \qquad w=0.5.
  \label{eq:blend}
\end{equation}

\paragraph{Evaluation.} With $\hat y_{-k(i)}$ the prediction for $i$ from the model
trained on the folds excluding $i$'s fold, we report out-of-sample fit and level
accuracy,
%
\begin{equation}
  R^2_{\text{OOS}} = 1-\frac{\sum_i\big(y_i-\hat y_{-k(i)}\big)^2}
  {\sum_i\big(y_i-\bar y\big)^2},
  \qquad
  \text{MdAPE}=\operatorname*{median}_i\Big|e^{\,\hat y_{-k(i)}-y_i}-1\Big|,
  \label{eq:cvmetrics}
\end{equation}
%
together with the share of listings predicted within $\pm15\%$ and $\pm25\%$ of the
asking price. The MdAPE is computed on the ratio of predicted to actual price and is
therefore unaffected by the retransformation constant of
Equation~\eqref{eq:smearing}.

\subsection{Results}
\label{subsec:bench_results}

\input{tables/models}

Table~\ref{tab:models} reports the ladder in two panels. In Panel~A (structural,
amenity, room, listing, and text features) out-of-sample $R^2$ rises monotonically
with flexibility: from 0.628 for OLS, to 0.677 for the random forest, 0.711 for the
plain network, and a near-tie at the top between the robust network (0.741) and
gradient boosting (0.743); their blend attains $R^2_{\text{OOS}}=0.760$. Panel~B adds
the façade vision layer. Vision lifts every estimator---OLS gains 5.8 points
(0.628 $\to$ 0.686), the random forest 2.7 (0.677 $\to$ 0.704), the robust network
0.6 (0.741 $\to$ 0.747), and gradient boosting 2.7 (0.743 $\to$ 0.770)---and the
text\,$+$\,vision blend attains $R^2_{\text{OOS}}=0.779$ with a median absolute
percentage error of 14.0\%, the best model in the study.

Three lessons follow. First, on the full feature set roughly nine points of
out-of-sample $R^2$ separate the interpretable linear model (0.686) from the
predictive frontier (0.779)---the price of interpretability, and a measure of the
curvature and interactions the hedonic form omits. Notably, the vision layer buys the
linear model more accuracy ($+5.8$ points) than switching from OLS to a random forest
does ($+4.9$ points in Panel~A), so better measurement rivals better estimation.
Second, on this tabular problem gradient boosting and a carefully regularized neural
network are statistically comparable, consistent with the broader evidence that tree
ensembles remain strong baselines on tabular data \citep{grinsztajn2022why}; the
gains come from \emph{combining} them. Third, the façade-vision layer is not redundant
with the text and structural fields: it is the more informative of the two AI layers
for every estimator.

The equal-weight blend is not the best achievable combination, but it is close.
Varying the weight $w$ on gradient boosting in Equation~\eqref{eq:blend} over
$\{0.3,0.4,0.5,0.6,0.7\}$ gives out-of-sample $R^2$ of 0.771, 0.776, 0.779, 0.780,
and 0.780 for the text\,$+$\,vision models, so $w=0.5$ is within 0.002 of the best
weight on the grid; choosing $w$ in a nested fashion---on the four training folds,
then applied to the fifth---returns $R^2_{\text{OOS}}=0.775$, slightly below the
fixed equal weight, because the fold-specific optimal weights are noisy. For the
text-only models the equal weight is exactly the best on the grid (0.760). We
therefore keep the fixed equal weight and do not tune it on the test predictions.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=.72\textwidth]{figures/fig_ablation.pdf}
  \caption{Out-of-sample $R^2$ as feature layers are added (nested, left to right),
  random five-fold cross-validation. Each layer improves both the linear and boosted
  models; the façade-vision layer adds about $+5.8$ points for OLS and $+2.7$ for
  gradient boosting, the text layer $+0.9$ and $+1.4$.}
  \label{fig:ablation}
\end{figure}

\subsection{Feature Ablation and Importance}
\label{subsec:ablation}

Table~\ref{tab:ablation} and Figure~\ref{fig:ablation} decompose the accuracy gains
layer by layer. Reading down the table, the room-detail and listing-signal block adds
the largest increment of all (9.8 points for OLS, 3.7 for gradient boosting), which
reflects both the information in the Realtor.ca room block and the fact that this
block carries the platform's identity; the LLM text layer adds 0.9 and 1.4 points;
and the vision layer adds 5.8 and 2.7 points, the largest increment of the two AI
layers and the second largest overall. The pattern is monotone: no layer degrades
fit. Under spatially grouped folds (Table~\ref{tab:rob_spatial},
Section~\ref{subsec:robust_spatial}) all levels fall by three to five points, but the
increments from the text and vision layers are essentially unchanged.

\input{tables/ablation}

Figure~\ref{fig:importance} aggregates the gradient-boosting permutation importances
into feature blocks. Structural attributes and location fixed effects dominate, as
expected, but the vision block accounts for 13\% of total importance---more than the
rule-based amenities and the LLM text block combined---so the façade photograph is a
first-order, not marginal, source of price information in the boosted model.
Figure~\ref{fig:calibration} shows that the blend is well calibrated across the price
range, with an approximately symmetric error distribution centred near zero.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=.72\textwidth]{figures/fig_importance.pdf}
  \caption{Share of gradient-boosting permutation importance by feature block
  (text\,$+$\,vision model). The vision block accounts for 13\% of total importance,
  exceeding the rule-based amenity and LLM-text blocks.}
  \label{fig:importance}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=\textwidth]{figures/fig_calibration.pdf}
  \caption{Out-of-sample calibration of the text\,$+$\,vision blend. Left: predicted
  versus actual price (log--log; dashed line is the 45$^\circ$ identity). Right:
  distribution of percentage prediction errors, with median absolute error of 14.0\%.}
  \label{fig:calibration}
\end{figure}

\subsection{A Robust Tabular Neural Network}
\label{subsec:nn}

Figure~\ref{fig:nn} details the robust network. High-cardinality categoricals enter
as learned \textbf{entity embeddings}: each municipality, property type, and source is
mapped to a dense vector (dimension $\approx 1.6\,C^{0.56}$ in cardinality $C$, so
about 30 dimensions for the 195 municipalities). The embeddings are concatenated with
the batch-normalized numeric block and projected to a 256-dimensional trunk, which
passes through three \textbf{residual blocks} (BatchNorm $\to$ Linear$_{512}$ $\to$
GELU $\to$ Dropout $\to$ Linear$_{256}$ $\to$ Dropout, with a skip connection),
followed by a normalization head that outputs $\ln(\text{Price})$.

Robustness comes from the training recipe rather than raw capacity:
a \textbf{Huber (smooth-$L_1$) loss} that damps the influence of price outliers;
\textbf{AdamW} with decoupled weight decay and a \textbf{cosine-annealed} learning rate;
\textbf{gradient-norm clipping} at 5; \textbf{dropout} and batch normalization
throughout; \textbf{missingness indicators} appended to the numeric block so
the network can exploit the non-random pattern of missing fields; and
\textbf{early stopping} on a held-out 10\% of each training fold. Finally, three
networks with different random seeds are trained per fold and \textbf{averaged}, which
reduces prediction variance. The result (0.741 with text, 0.747 with text and vision)
is competitive with gradient boosting and, once blended with it, produces the best
model in the study.

\begin{figure}[htbp]
  \centering
  \input{figures/nn_diagram}
  \caption{Architecture of the robust tabular neural network. High-cardinality
  categoricals enter as learned embeddings; numeric features are batch-normalized and
  augmented with missingness indicators. The concatenated representation is projected
  to a 256-dimensional trunk, refined by three residual blocks, and mapped to
  $\ln(\text{Price})$. Three seeds are ensembled per fold; the final model blends this
  network equally with gradient boosting.}
  \label{fig:nn}
\end{figure}