Python 52.4%
TeX 47.6%
1% ============================================================================2\section{Predictive Benchmark: From OLS to Deep Learning}3\label{sec:benchmark}4% ============================================================================56The hedonic OLS of Section~\ref{sec:results} is designed for interpretation: its7coefficients are implicit prices. It is not, however, the most accurate predictor of8price. To map the accuracy frontier---and to gauge how much curvature and interaction9the linear model leaves on the table---we benchmark a ladder of estimators of10increasing flexibility on the \emph{identical} feature set, under a common115-fold cross-validation protocol. All rows of Table~\ref{tab:models} share the same12folds and random seed, so differences reflect the estimator, not the split.1314\subsection{The Estimator Ladder}15\label{subsec:ladder}1617\paragraph{(1) Ordinary least squares.} The linear hedonic of18Equation~\eqref{eq:hedonic}, with one-hot city, property-type, and source indicators19and within-property-type median imputation of missing continuous attributes. It is20the interpretable baseline and, mechanically, the least flexible: it imposes constant21marginal effects and additive separability.2223\paragraph{(2) Random forest.} A bagged ensemble of regression trees24\citep{mullainathan2017machine}. Trees capture non-linearities and low-order25interactions automatically and handle missing values natively, but averaging26deep decorrelated trees tends to over-smooth sharp price thresholds.2728\paragraph{(3) Gradient boosting.} A histogram-based gradient-boosted tree ensemble29(600 boosting iterations, learning rate 0.05, up to 63 leaves, $L_2$ regularization).30Boosting fits trees sequentially to residuals, which lets it model both smooth31curvature (age, area) and abrupt thresholds; like the forest it ingests missing32values directly. It is the strongest single model in our comparison.3334\paragraph{(4) Neural networks.} A plain multilayer perceptron and a robust35residual network with entity embeddings, described in36Section~\ref{subsec:nn}. Neural networks excel when high-cardinality categoricals37(here, 195 municipalities) can be represented as dense learned vectors rather than38sparse indicators.3940\paragraph{(5) Blend.} A simple 50/50 average of the robust neural network and the41gradient-boosting predictions. Because the two models make partly decorrelated42errors---the network smooths through embeddings while the boosted trees cut sharp43thresholds---their average cancels idiosyncratic noise, a standard stacking result.4445\subsection{Formal Definitions}46\label{subsec:formal}4748Let $y_i=\ln P_i$ and let $\mathbf{x}_i$ denote the feature vector of49Equation~\eqref{eq:blocks}. Each estimator approximates the conditional mean50$f(\mathbf{x})=\mathbb{E}[y\mid\mathbf{x}]$.5152\paragraph{Linear (OLS).} $f(\mathbf{x})=\mathbf{x}^{\!\top}\bm{\theta}$ with the53closed form of Section~\ref{subsec:econometrics}; categoricals enter as one-hot54indicators.5556\paragraph{Random forest \citep{breiman2001random}.} An average of $B$ regression57trees, each grown on a bootstrap resample with a random feature subset at every split:58%59\begin{equation}60 \hat f_{\text{RF}}(\mathbf{x})=\frac1B\sum_{b=1}^{B} T_b(\mathbf{x};\Theta_b),61 \label{eq:rf}62\end{equation}63%64where $\Theta_b$ denotes the randomness (resample and split subsets) of tree~$b$.6566\paragraph{Gradient boosting \citep{friedman2001greedy}.} A forward stagewise additive67expansion built by fitting each new tree to the negative gradient of the loss:68%69\begin{equation}70 \resizebox{.94\textwidth}{!}{$\displaystyle71 F_M(\mathbf{x}) = F_0 + \nu\sum_{m=1}^{M} f_m(\mathbf{x}),72 \quad73 f_m = \arg\min_{f}\sum_{i=1}^{n}74 \big(\!-g_{mi}-f(\mathbf{x}_i)\big)^2,75 \quad76 g_{mi}=\frac{\partial L\big(y_i,F_{m-1}(\mathbf{x}_i)\big)}{\partial F_{m-1}(\mathbf{x}_i)}77 $}78 \label{eq:gbm}79\end{equation}80%81with learning rate $\nu$ and $L_2$-regularized histogram trees $f_m$; missing values82are routed to a learned default direction at each split.8384\paragraph{Neural network (robust).} High-cardinality categoricals are mapped to85learned embeddings \citep{guo2016entity}, concatenated with the batch-normalized86numeric block and projected to a $d$-dimensional trunk:87%88\begin{equation}89 \mathbf{h}_0 = \mathbf{W}_090 \Big[\, \textstyle\bigoplus_{c\in\{\text{city},\text{type},\text{src}\}}91 \mathbf{E}_c(\text{cat}_{ci})92 \;\Vert\; \mathrm{BN}(\mathbf{x}^{\text{num}}_i)\,\Big].93 \label{eq:nn_in}94\end{equation}95%96The trunk passes through $L=3$ pre-activation residual blocks97\citep{he2016deep,gorishniy2021revisiting},98%99\begin{equation}100 \mathbf{h}_{\ell}101 = \mathbf{h}_{\ell-1}102 + \mathrm{Drop}\!\Big(\mathbf{W}_2^{(\ell)}\,103 \mathrm{Drop}\!\big(\phi(\mathbf{W}_1^{(\ell)}\,104 \mathrm{BN}(\mathbf{h}_{\ell-1}))\big)\Big),105 \qquad \phi=\text{GELU},106 \label{eq:resblock}107\end{equation}108%109and a normalization head yields $\hat y_i = \mathbf{w}^{\!\top}\phi(\mathrm{BN}(\mathbf{h}_L))+b$.110Parameters minimize the Huber (smooth-$L_1$) loss, robust to price outliers,111%112\begin{equation}113 L_\delta(r)=114 \begin{cases}115 \tfrac12 r^2, & |r|\le \delta,\\[2pt]116 \delta\big(|r|-\tfrac12\delta\big), & |r|>\delta,117 \end{cases}118 \qquad r_i=\hat y_i-y_i,\;\delta=1,119 \label{eq:huber}120\end{equation}121%122optimized with AdamW (decoupled weight decay) \citep{loshchilov2019decoupled} under a123cosine-annealed learning rate $\eta_t=\tfrac12\eta_0\big(1+\cos(\pi t/T)\big)$,124gradient-norm clipping, and early stopping. Three seeds are averaged per fold.125126\paragraph{Blend.} A convex combination of the two strongest models in log space,127%128\begin{equation}129 \widehat{\ln P}^{\,\text{blend}}130 = w\,\widehat{\ln P}^{\,\text{GBM}}131 + (1-w)\,\widehat{\ln P}^{\,\text{NN}},132 \qquad w=0.5.133 \label{eq:blend}134\end{equation}135136\paragraph{Evaluation.} With $\hat y_{-k(i)}$ the prediction for $i$ from the model137trained on the folds excluding $i$'s fold, we report out-of-sample fit and level138accuracy,139%140\begin{equation}141 R^2_{\text{OOS}} = 1-\frac{\sum_i\big(y_i-\hat y_{-k(i)}\big)^2}142 {\sum_i\big(y_i-\bar y\big)^2},143 \qquad144 \text{MdAPE}=\operatorname*{median}_i\Big|e^{\,\hat y_{-k(i)}-y_i}-1\Big|.145 \label{eq:cvmetrics}146\end{equation}147148\subsection{Results}149\label{subsec:bench_results}150151\input{tables/models}152153Table~\ref{tab:models} reports the ladder in two panels. In Panel~A (structural,154amenity, and text features) out-of-sample $R^2$ rises monotonically with flexibility:155from 0.628 for OLS, to 0.677 for the random forest, 0.711 for the plain network, and a156near-tie at the top between the robust network (0.741) and gradient boosting (0.743);157their blend attains $R^2_{\text{OOS}}=0.760$. Panel~B adds the twelve façade vision158attributes. Vision lifts \emph{every} estimator---OLS gains 5.8 points159(0.628 $\to$ 0.686), the random forest 2.7 (0.677 $\to$ 0.704), and gradient boosting1602.7 (0.743 $\to$ 0.770), the single largest increment from any one feature layer at161the frontier---and the \textbf{text\,+\,vision blend attains162$R^2_{\text{OOS}}=0.779$ with a median absolute error of 14.0\%}, the best model in163the study.164165Three lessons follow. First, on the full feature set roughly nine points of166out-of-sample $R^2$ separate the interpretable linear model (0.686) from the167predictive frontier (0.779)---the price of interpretability, and a measure of the168curvature and interactions the hedonic form omits. Notably, the vision layer buys the169linear model more accuracy (+5.8 points) than switching from OLS to a random forest170does (+4.9), so better measurement rivals better estimation. Second, on this tabular171problem gradient boosting and a carefully regularized neural network are172statistically comparable, consistent with the broader evidence that tree ensembles173remain strong baselines on tabular data \citep{grinsztajn2022why}; the gains come174from \emph{combining} them. Third, the façade-vision layer is not redundant with the175text and structural fields: it moves the frontier more than any other block we add,176confirming that image-derived quality is genuinely orthogonal information.177178\begin{figure}[htbp]179 \centering180 \includegraphics[width=.72\textwidth]{figures/fig_ablation.pdf}181 \caption{Out-of-sample $R^2$ as feature layers are added (nested, left to right).182 Each layer improves both the linear and boosted models; the façade-vision layer183 delivers the largest single jump---about $+5.8$ points for OLS and $+2.7$ for184 gradient boosting.}185 \label{fig:ablation}186\end{figure}187188\subsection{Feature Ablation and Importance}189\label{subsec:ablation}190191Table~\ref{tab:ablation} and Figure~\ref{fig:ablation} decompose the accuracy gains192layer by layer. Reading down the table, room detail and listing signals add the first193large increment, the LLM text layer a smaller one, and the vision layer the single194largest jump for every estimator---raising OLS from 0.628 to 0.686 and gradient195boosting from 0.743 to 0.770 out of sample. The pattern is monotone: no layer degrades196fit, and the two AI layers together account for a substantial share of the distance197between the structural baseline and the frontier.198199\input{tables/ablation}200201Figure~\ref{fig:importance} aggregates the gradient-boosting permutation importances202into feature blocks. Structural attributes and location fixed effects dominate, as203expected, but the \textbf{vision block alone accounts for 13\% of total importance}---204more than the rule-based amenities and the LLM text block combined---reinforcing that205the façade photograph is a first-order, not marginal, source of price information.206Figure~\ref{fig:calibration} shows the blend is well calibrated across the price range,207with an approximately symmetric, tight error distribution centered near zero.208209\begin{figure}[htbp]210 \centering211 \includegraphics[width=.72\textwidth]{figures/fig_importance.pdf}212 \caption{Share of gradient-boosting permutation importance by feature block213 (text\,+\,vision model). The vision block accounts for 13\% of total importance,214 exceeding the rule-based amenity and LLM-text blocks.}215 \label{fig:importance}216\end{figure}217218\begin{figure}[htbp]219 \centering220 \includegraphics[width=\textwidth]{figures/fig_calibration.pdf}221 \caption{Out-of-sample calibration of the text\,+\,vision blend. Left: predicted222 versus actual price (log--log; dashed line is the 45$^\circ$ identity). Right:223 distribution of percentage prediction errors, with median absolute error of 14.0\%.}224 \label{fig:calibration}225\end{figure}226227\subsection{A Robust Tabular Neural Network}228\label{subsec:nn}229230Figure~\ref{fig:nn} details the robust network. High-cardinality categoricals enter231as learned \textbf{entity embeddings}: each municipality, property type, and source is232mapped to a dense vector (dimension $\approx 1.6\,C^{0.56}$ in cardinality $C$, so233about 29 dimensions for the 195 cities). The embeddings are concatenated with the234batch-normalized numeric block and projected to a 256-dimensional trunk, which passes235through three \textbf{residual blocks}236(BatchNorm $\to$ GELU $\to$ Linear$_{512}$ $\to$ Dropout $\to$ Linear$_{256}$, with a237skip connection), followed by a normalization head that outputs $\ln(\text{Price})$.238239Robustness comes from the training recipe rather than raw capacity:240a \textbf{Huber (smooth-$L_1$) loss} that damps the influence of price outliers;241\textbf{AdamW} with decoupled weight decay and \textbf{cosine-annealed} learning rate;242\textbf{gradient-norm clipping} at 5; \textbf{dropout} and batch normalization243throughout; \textbf{learned missingness indicators} appended to the numeric block so244the network can exploit the non-random pattern of missing fields; and245\textbf{early stopping} on a held-out 10\% of each training fold. Finally, three246networks with different random seeds are trained per fold and \textbf{averaged}, which247reduces prediction variance. The result (0.741) is competitive with gradient boosting248and, once blended with it, produces the best model in the study.249250\begin{figure}[htbp]251 \centering252 \input{figures/nn_diagram}253 \caption{Architecture of the robust tabular neural network. High-cardinality254 categoricals enter as learned embeddings; numeric features are batch-normalized and255 augmented with missingness indicators. The concatenated representation is projected256 to a 256-dimensional trunk, refined by three residual blocks, and mapped to257 $\ln(\text{Price})$. Three seeds are ensembled per fold; the final model blends this258 network 50/50 with gradient boosting.}259 \label{fig:nn}260\end{figure}261