spb/wp11_uqo

1% Author: Simon-Pierre Boucher — contact@spboucher.ai2% ============================================================================3\section{Robustness}4\label{sec:robust}56This section examines the five choices in the design that a sceptical7reader would most want varied: the rule that converts transformed-scale8predictions into price levels, the leakage channels of the random9holdout, the implementation of fine spatial effects, the resolution of10the Box--Cox grid and the hyper-parameters of the boosting machine.1112\subsection{The retransformation rule}1314Table~\ref{tab:retransformation} reports the scoreboard for the15transformed-scale models under the median rule used throughout16(eq.~\ref{eq:median}) and under Duan's mean rule17(eq.~\ref{eq:duan}). The smearing factors $\hat{s}$ lie between 1.0618and 1.07 for the least-squares models and between 1.03 and 1.05 for the19ensembles, whose in-sample residuals are shrunk by overfitting. Two20regularities stand out. First, the mean rule lowers the median error of21\emph{every} log-scale model under the random split, by about one point22for the linear models and 1.3 points for the ensembles. This is not what23a conditional-mean correction should do to a median-based metric; it24happens because the log residuals are left-skewed (skewness $-1.4$ to25$-1.8$), so that their median is $+0.06$ while their mean is zero, and26$\exp(\hat{z})$ under-prices the typical sale by about 6\%---which is,27by coincidence, the size of the smearing factor. Second, under the28forward-in-time split the mean rule lowers median errors by 2.5--329points for all models, because the market rose after the training30window and any upward factor offsets part of the staleness. Neither31effect changes a ranking when the rule is applied uniformly: boosting32leads the spline hedonic by 3.4 points under the random split and by330.7 points under the forward split with the mean rule, against 3.2 and340.5 with the median rule. What does change a ranking is applying the35rule to one family only. Version~1.0 of this paper smeared the linear36log models and not the ensembles; the result was a forward-split37comparison of 19.1\% for the smeared spline hedonic against 21.2\% for38the unsmeared boosting model, reported as a reversal. The comparable39numbers are 21.7\% against 21.2\% (median rule) or 19.1\% against4018.4\% (mean rule). The log-scale metrics are affected in the same41direction: smearing adds $\ln\hat{s} \approx 0.06$ to every log42prediction, which inflates RMSE$_{\ln}$ and depresses $R^2_{\ln}$ for43the smeared family only, so that the ``identical $R^2_{\ln}$'' of the44first version (0.52 for both) becomes 0.49 against 0.52 in favour of45boosting under the uniform rule.4647\begin{table}[tbp]48\centering49\begin{threeparttable}50\caption{Median rule versus Duan smearing}51\label{tab:retransformation}52\scriptsize53\setlength{\tabcolsep}{4pt}54\input{../results/tables/retransformation}55\begin{tablenotes}[flushleft]\footnotesize56\item \textit{Notes:} Median: transformed-scale prediction inverted as57is, eq.~\eqref{eq:median} (main scoreboard). Mean: Duan smearing,58eq.~\eqref{eq:duan}; $\hat{s}$ is the smearing factor estimated on59training residuals (random split), undefined for the Box--Cox model,60whose mean prediction averages the inverse transformation over 1{,}00061training residuals. $R^2_{\ln}$ under the mean rule is computed on the62smeared log prediction.63\end{tablenotes}64\end{threeparttable}65\end{table}6667\subsection{Leakage-free holdouts}6869Table~\ref{tab:group_splits} re-runs the full race under the two grouped70holdouts of Section~\ref{sec:holdouts}, and Table~\ref{tab:leakage}71decomposes the random holdout by whether the test sale's parcel, or its72exact coordinates, also appear in training. Repeat sales are irrelevant:73the parcel-grouped race reproduces the random race to within 0.1--0.474points for every model, and within the random holdout the boosting75model prices test sales whose parcel sold before in training (14.5\%)76no better than the others (14.4\%). Shared coordinates matter more, but77for a specific reason. Test sales that share coordinates with a training78sale are priced at 12.0\% by boosting against 15.3\% for the rest, and79at 16.7\% against 22.9\% by the $k$-NN rule, whose nearest neighbour is80then literally the same building; the linear models are unaffected81(17.8\% against 17.6\% for the splines). Condominiums account for most82of these co-located sales, and the boosting model prices them at 8.9\%83when their building is in training and 10.2\% when it is not, against8418.0\% and 20.3\% for the quadratic OLS. The condominium advantage of85Section~\ref{sec:ext} is therefore genuine micro-location, with a modest86recall component. The building-grouped race, in which no coordinate pair87straddles the split, confirms it: boosting posts 14.6\% against 17.8\%88for the spline hedonic, a 3.2-point gap identical to the random race,89and the only model that loses materially is the $k$-NN rule (22.3\%90against 21.2\%).9192\begin{table}[p]93\centering94\begin{threeparttable}95\caption{The race under random, parcel-grouped and building-grouped holdouts}96\label{tab:group_splits}97\small98\input{../results/tables/group_splits}99\begin{tablenotes}[flushleft]\footnotesize100\item \textit{Notes:} MdAPE in percent. Parcel-grouped: 20\% of101assessment-roll parcels assigned to the test set, so that repeat sales102of a property never straddle the split. Building-grouped: 20\% of exact103coordinate pairs assigned to the test set, so that units geocoded to the104same building or complex never straddle the split.105\end{tablenotes}106\end{threeparttable}107\end{table}108109\begin{table}[tbp]110\centering111\begin{threeparttable}112\caption{Random holdout: accuracy on test sales seen and unseen through their parcel or coordinates}113\label{tab:leakage}114\small115\input{../results/tables/leakage}116\begin{tablenotes}[flushleft]\footnotesize117\item \textit{Notes:} MdAPE (\%) on the random holdout. ``Same parcel in118training'': the parcel of the test sale sold at least once in the119training sample. ``Same coordinates in training'': at least one training120sale shares the exact geocoded coordinates of the test sale.121\end{tablenotes}122\end{threeparttable}123\end{table}124125\subsection{Nested spatial fixed effects}126127Section~\ref{sec:results} attributes the poor showing of the128$\sim$1.1~km grid to thin cells and to the arbitrary pricing of unseen129cells. Table~\ref{tab:nested_fe} adds two specifications in which the130grid effects enter \emph{together with} municipality effects. Because131the two sets of dummies are nearly collinear, the vanishing-penalty132ridge returns the minimum-norm least-squares solution, which assigns the133component common to a municipality to the municipality effect and only134the deviation to the cell effect---a crude form of partial pooling---and135a test sale in an unseen cell inherits its municipality's level. The136effect is large. Nested within municipalities, the 5.5~km grid improves137from 18.7\% to 17.7\% and the 1.1~km grid from 21.0\% to 17.3\%, the138lowest median error of any linear specification in the paper, below the139spline hedonic (17.6\%) with the same $R^2_{\ln}$ (0.59). Under the140forward-in-time split the nested fine grid posts 21.6\% against 23.3\%141for municipality effects alone and 25.3\% for the fine grid alone. The142conclusion for axis~C is therefore sharper than ``finer overfits'':143fine spatial effects are valuable when they are anchored to a coarser144level that supplies the prior for thin and unseen cells, and harmful145when they are not. A hierarchical or kriged location effect146\citep{case2004modeling, hill2018geospatial} would push the linear147frontier further; we leave its estimation to future work.148149\begin{table}[tbp]150\centering151\begin{threeparttable}152\caption{Nested spatial fixed effects}153\label{tab:nested_fe}154\small155\input{../results/tables/nested_fe}156\begin{tablenotes}[flushleft]\footnotesize157\item \textit{Notes:} Quadratic semi-log backbone with quarter fixed158effects. C3$'$ and C4$'$ add the grid effects to the municipality159effects; with a vanishing ridge penalty the estimator is the160minimum-norm least-squares solution and unseen cells fall back on their161municipality.162\end{tablenotes}163\end{threeparttable}164\end{table}165166\subsection{A finer Box--Cox grid}167168The main grid steps $\lambda$ by $0.25$. Refining it to steps of $0.05$169over $[-0.25, 0.75]$ moves the profile-likelihood maximum to170$\hat\lambda = 0.35$, with a profile that is flat between $0.25$ and171$0.55$ (differences of under one thousandth of the log-likelihood,172within the tolerance of the sparse iterative solver) and falls steeply173on either side. The Box--Cox model at $\hat\lambda = 0.35$ posts a174random-holdout MdAPE of 18.4\% against 18.6\% at $0.25$: the exponent is175imprecisely identified in a range that excludes both the linear and the176logarithmic case, and its exact value is immaterial for prediction.177178\subsection{Boosting hyper-parameters}179180Table~\ref{tab:hgb} re-estimates the boosting model under four181materially different configurations: shallow trees with more iterations,182deep trees, heavy regularisation (minimum leaf 200, $\ell_2 = 1$), and a183configuration whose number of iterations is chosen by early stopping on184the last 10\% of training sales in date order, the time-honest tuning185rule a careful practitioner would use for a forward-looking model.186Under the random holdout the four alternatives span 14.3--14.7\% MdAPE187against 14.4\% for the baseline, and under the forward-in-time holdout18821.2--21.6\% against 21.2\%; $R^2_{\ln}$ moves by at most 0.006. The189time-honest early-stopping rule selects 700 iterations on the random190training sample and 1{,}000 on the pre-2025 sample and reproduces the191baseline to within 0.1 point under both splits. The conclusions of Sections~\ref{sec:results}192and~\ref{sec:gap} do not depend on the particular configuration we193fixed a priori, and tuning on a time-ordered validation set does not194remove the level staleness that dominates the forward-split error,195because no choice of hyper-parameters lets a static tree extrapolate a196price level it has not seen.197198\begin{table}[tbp]199\centering200\begin{threeparttable}201\caption{Gradient-boosting hyper-parameter sensitivity}202\label{tab:hgb}203\footnotesize204\setlength{\tabcolsep}{4pt}205\input{../results/tables/hgb_sensitivity}206\begin{tablenotes}[flushleft]\footnotesize207\item \textit{Notes:} All configurations use the D4 feature set208(attributes, coordinates, month counter). Baseline: 600 iterations,209learning rate 0.08, 63 leaves, minimum leaf 20, $\ell_2 = 0.01$. Early210stopping: iterations added in blocks of 100 (learning rate 0.05) until211the squared log error on the last 10\% of training sales in date order212stops improving; the selected number of iterations is shown in brackets213for the random and forward splits.214\end{tablenotes}215\end{threeparttable}216\end{table}217