% ============================================================================ \section{Results} \label{sec:results} % ============================================================================ All results in this section are computed on the \nTargetsTest{} test targets of January 2025 to September 2026, with every configuration, weight and model frozen on the basis of the \nTargetsVal{} validation targets of 2024. Validation-period figures are reported alongside to document the stability of the rankings. Differences between methods quoted in the text are paired differences on the common targets, computed before rounding, so that they may differ by a tenth of a point from the difference of the rounded levels. \subsection{How many comparables, how far, how far back?} \label{sec:surface} Figure~\ref{fig:surface_k} plots the out-of-sample MdAPE against the number of comparables $k$ for each search radius, with a twelve-month window and selection by distance, without physical adjustment (left) and with the hedonic grid (right). Figure~\ref{fig:surface_dt} shows the best attainable MdAPE for each radius--window pair, and Table~\ref{tab:surface} reports the full surface for the hedonic grid. \begin{figure}[H] \centering \includegraphics[width=\textwidth]{f1_mdape_vs_k.pdf} \caption{Out-of-sample MdAPE (2025--2026) by number of comparables and search radius; twelve-month window, selection by distance, median reconciliation. Left: time adjustment only. Right: hedonic adjustment grid.} \label{fig:surface_k} \end{figure} \begin{figure}[H] \centering \includegraphics[width=0.9\textwidth]{f2_surface_d_T.pdf} \caption{Best attainable out-of-sample MdAPE (over $k$) for each search radius and look-back window; selection by distance.} \label{fig:surface_dt} \end{figure} \begin{table}[H] \centering \caption{Error surface $\mathrm{Error}(k,d)$: out-of-sample MdAPE (\%), selection by distance, hedonic grid, twelve-month window} \label{tab:surface} \small \input{tables/surface_geo_hedo} \begin{tablenotes}\footnotesize \item Note: MdAPE on the 2025--2026 test targets for which the configuration returns a value; the minimum of each row is in bold. Coverage falls below 90\% for radii of 1~km and less in low-density markets; see Table~\ref{tab:surface_cov} in the Appendix. \end{tablenotes} \end{table} Three regularities emerge. First, without physical adjustment, accuracy improves with $k$ up to about ten comparables (\noneKten{} at $k=10$ for a 10~km radius and twelve-month window) and then deteriorates (\noneKfifty{} at $k=50$): beyond the nearest few, additional sales are increasingly dissimilar and the median of their time-adjusted prices drifts away from the subject's value. This is the pattern that \citet{pace1996} predicts for a nearest-neighbour estimator whose bias grows with the number of neighbours. Second, with the hedonic grid the picture changes qualitatively. The error falls from \mdapeKone{} with a single comparable to \mdapeKthree{} with three, \mdapeKfive{} with five, \mdapeKten{} with ten, reaches \mdapeKopt{} at $k=\kOpt{}$ and stays there to $k=50$ (\mdapeKfifty{}). Adjustment converts dissimilar sales into usable evidence, so that the variance reduction from averaging more of them is no longer offset by their heterogeneity. Third, the window matters and the radius does not. The three-month window is dominated by the twelve- and twenty-four-month windows at every radius (Figure~\ref{fig:surface_dt}); but at the best $k$ the MdAPE with a 24-month window lies between 11.9 and 12.0\% for every radius from 0.5 to 25~km, because $k$ is a maximum and a wide radius only adds sales where the nearby ones run out. The conventional combination---three comparables, within a kilometre, within three months---yields an MdAPE of \narrowMdape{} with the hedonic grid and returns a value for only \narrowCov{} of targets; the best configuration of the surface chosen on the 2024 validation set (\bestSel{}, $k=\bestK$, $d\le\bestD$~km, $t=\bestT$ months, \bestAdj{}) yields \bestValMdape{} on validation and \bestTestMdape{} on the test set. Table~\ref{tab:h2} tests H2 by stratifying the $k$-curve by market density. The threshold beyond which additional comparables hurt exists only without physical adjustment (lower panel): with time adjustment alone the optimum is seven to ten comparables in the three densest strata, where the eleventh-nearest sale is already dissimilar, fifteen in the 5--19 stratum, and fifty in the sparsest one, where every sale within reach is needed. With the hedonic grid (upper panel) there is no threshold within the range studied: the error is minimised at twenty to fifty comparables in every density stratum and the curve is flat beyond twenty. H2 is therefore confirmed for the unadjusted method and rejected for the adjusted one---adjustment, not density, determines whether more comparables help. \begin{table}[H] \centering \caption{Optimal number of comparables by market density (H2): out-of-sample MdAPE (\%)} \label{tab:h2} \small \input{tables/h2} \begin{tablenotes}\footnotesize \item Note: selection by distance, $d\le 10$~km, twelve-month window, median. Density is the number of sample sales within 2~km in the twelve months preceding the target's date. The minimum of each row is in bold. \end{tablenotes} \end{table} \subsection{Which adjustments are warranted?} \label{sec:adjustments} Table~\ref{tab:adjustments} and Figure~\ref{fig:adjustments} compare the eight adjustment schemes at the $(5, 5\text{ km}, 12\text{ mo})$ anchor with selection by distance, equal weights and the median. The time adjustment alone is worth only \gainTime{} points of MdAPE relative to raw prices with a twelve-month window and \gainTimeLong{} points with a 24-month window: comparables are recent, and a monthly index estimated in real time is itself noisy. Any physical grid is worth a further \gainGridLo{} to \gainGridHi{} points. The stylised conventional grid---which uses no estimation whatsoever---recovers most of the gain of the hedonic grid (\convMdape{} against \mdapeMseven{}), consistent with the observation of \citet{colwell1983} that the grid is a regression with pre-set coefficients that need only be roughly right. The refinements add little: the regional grid is indistinguishable from the global one, the local grid is \localVsHedo{} (\localMdape{}), and the non-linear GBM adjustments improve on the hedonic grid by \gbmVsHedo{} points. The assessment-ratio adjustment stands apart: with \mdapeMthirteen{} it is the most accurate single scheme, but it returns a value for only \covMthirteen{} of targets because it requires comparables on the same assessment roll. \begin{table}[H] \centering \caption{Adjustment schemes at the $(5,\,5\text{ km},\,12\text{ months})$ anchor} \label{tab:adjustments} \small \input{tables/adjustments} \begin{tablenotes}\footnotesize \item Note: five nearest sales, equal weights, median. MdAPE, MAPE, bias (median signed error) and the share within $\pm10\%$ in per cent, on the 2025--2026 test targets unless indicated; coverage is the share of test targets for which the scheme returns a value. \end{tablenotes} \end{table} \begin{figure}[H] \centering \includegraphics[width=0.9\textwidth]{f3_ajustements.pdf} \caption{MdAPE by adjustment scheme, validation (2024) and test (2025--2026); five nearest sales, $\le 5$~km, $\le 12$ months, median.} \label{fig:adjustments} \end{figure} Table~\ref{tab:h3} examines H3 by crossing the adjustment schemes with market density at two anchors. The local grid tracks the hedonic grid closely where sales are abundant and falls behind where they are scarce, as the bias--variance argument predicts; at the tight $(3, 2\text{ km}, 6\text{ mo})$ anchor, where the local regression is estimated on the smallest pools, the gap widens. The non-linear GBM adjustments are the most accurate scheme in every stratum, but their advantage over the hedonic grid is largest in dense markets (about a point) and smallest in the sparsest one. H3 is therefore rejected in its first part---local re-estimation does not outperform the fixed grid even where sales are plentiful---and confirmed in its second. The practical implication is that adjustment grids should be estimated on the largest pool that is homogeneous in implicit prices---in these data, the province with municipality fixed effects---rather than on the handful of sales around the subject. \begin{table}[H] \centering \caption{Adjustment schemes by market density (H3): out-of-sample MdAPE (\%)} \label{tab:h3} \small \input{tables/h3} \begin{tablenotes}\footnotesize \item Note: selection by distance, equal weights, median. Upper panel: $(5, 5\text{ km}, 12\text{ mo})$; lower panel: $(3, 2\text{ km}, 6\text{ mo})$. The minimum of each row is in bold. \end{tablenotes} \end{table} The estimated hedonic grid itself is of independent interest (Table~\ref{tab:grid}). Within a municipality, the elasticity of price with respect to living area is \betaFloor{}, that with respect to lot area is \betaLand{}, and depreciation is \betaAge{} per year at the mean age with a convex profile, a magnitude in line with the single-family depreciation estimates of \citet{cannaday1986}; semi-detached and row dwellings sell at discounts of \betaSemi{} and \betaRow{} relative to detached dwellings of the same size. The within-municipality $R^2$ is \gridRtwo{}: the roll characteristics explain about a third of the price variation among dwellings of the same municipality, and the rest---location within the municipality, condition, quality---is what the comparables themselves must supply. These coefficients are stable across the eleven quarterly re-estimations to the second decimal. \begin{table}[H] \centering \caption{Hedonic adjustment grid (within-municipality coefficients, log price)} \label{tab:grid} \input{tables/grid} \begin{tablenotes}\footnotesize \item Note: coefficients of the within-municipality regression of the time-adjusted log price on characteristics, estimated on all sales before 2025 (grid in force in the first test quarter). Living and lot areas in logs; age in years; age$^2$/100. Omitted categories: detached, single-storey construction type. \end{tablenotes} \end{table} \subsection{How should comparables be selected? (H1)} \label{sec:selection} Table~\ref{tab:selection} compares the four selection criteria at the $(5,5,12)$ anchor with the hedonic grid. Selecting by physical similarity rather than by distance lowers the MdAPE by \gainSelHedo{} points when comparables are adjusted, and the compromise criterion, which uses both, is the best of the four. The ``least adjusted'' criterion, which embodies the professional advice to prefer comparables needing few adjustments, performs like the similarity criterion for which it is a proxy. \begin{table}[H] \centering \caption{Selection criteria at the $(5,\,5\text{ km},\,12\text{ months})$ anchor, hedonic grid} \label{tab:selection} \input{tables/selection} \end{table} The direct test of H1 is starker. Without any physical adjustment, the five nearest sales within one kilometre yield an MdAPE of \hOneGeoDone{} (coverage \hOneGeoDoneCov{}), and the five nearest within ten kilometres \hOneGeoDten{}; the five most similar sales within ten kilometres yield \hOneSimDten{}. Holding the radius fixed, the selection criterion alone is worth more than four points, as much as the hedonic grid is worth to the nearest sales: the nearby-but-dissimilar comparable is worse than the distant-but-similar one by a wide margin. The reason is that, in these data, the physical characteristics explain a larger share of the within-market variance of prices than micro-location does once the municipality is fixed. With the hedonic grid the two criteria converge (\hOneGeoDtenHedo{} against \hOneSimDtenHedo{}), because adjustment does for the nearest sales what selection does for the most similar; the compromise criterion, which uses both, is best (\hOneMixDtenHedo{}). H1 is confirmed. \subsection{Weighting and reconciliation (H4)} \label{sec:weighting} Table~\ref{tab:weighting} reports the ten weighting--reconciliation combinations at the $(5,5,12)$ anchor with the hedonic grid, and Table~\ref{tab:h4} stratifies seven of them by density. The range of the MdAPE across all ten combinations is \wrRange{} points. The median is more robust than the mean, as expected from the heavy tails of adjusted indications, and the mean carries a negative bias of about $-1.5$ points because averaging in dollars overweights the high indications in logs. The density profile does not support H4 in its stated form. The equal-weight median is the best or within a tenth of a point of the best in the three densest strata; weighting by gross adjustment improves on the equal-weight mean in dense markets (\hFourSimDense{} against \hFourEqualMeanDense{} where 200 or more sales are available) but not in sparse ones; distance weights are neutral everywhere; and the product of the three weights with the median is the worst combination in the two sparsest strata, because it concentrates the weight on one or two sales. Weighting thus matters, if anywhere, in liquid markets, which is the opposite of what H4 predicts, and the effects are an order of magnitude smaller than those of selection and adjustment. The marginal effects over the whole E2 design (Table~\ref{tab:marginal}) make the hierarchy explicit: the adjustment factor spans \margAdjLo{}--\margAdjHi{} points of MdAPE, selection \margSelLo{}--\margSelHi{}, weighting \margWLo{}--\margWHi{}, and reconciliation \margReconMedian{} (median) against \margReconMean{} (mean). \begin{table}[H] \centering \caption{Weighting and reconciliation at the $(5,\,5\text{ km},\,12\text{ months})$ anchor, hedonic grid} \label{tab:weighting} \small \input{tables/weighting} \begin{tablenotes}\footnotesize \item Note: ``1/gross adjustment'' weights each comparable by the inverse of the absolute hedonic adjustment it requires; ``product of the three'' multiplies the distance, recency and gross-adjustment weights. \end{tablenotes} \end{table} \begin{table}[H] \centering \caption{Weighting by market density (H4): out-of-sample MdAPE (\%)} \label{tab:h4} \footnotesize \input{tables/h4} \begin{tablenotes}\footnotesize \item Note: $(5, 5\text{ km}, 12\text{ mo})$ anchor, selection by distance, hedonic grid. Columns: weighting/reconciliation; ``1/adj.'' is the inverse gross-adjustment weight, ``prod.'' the product of the distance, recency and gross-adjustment weights. The minimum of each row is in bold. \end{tablenotes} \end{table} \begin{table}[H] \centering \caption{Marginal effects of the four rule factors in the E2 design} \label{tab:marginal} \input{tables/marginal} \begin{tablenotes}\footnotesize \item Note: mean out-of-sample MdAPE (\%) of all E2 configurations sharing the given level of the factor, averaged over the other factors and the three anchors. \end{tablenotes} \end{table} \subsection{The horse race} \label{sec:horserace} Table~\ref{tab:competition} and Figure~\ref{fig:competition} report the sixteen named models; Table~\ref{tab:competition_ci} in the Appendix adds cluster-bootstrap confidence intervals for each model's MdAPE and MAPE. The ordering is informative at every level. The traditional practices M1--M5---a few nearest sales, time-adjusted, weighted in one way or another---cluster at \mdapeMoneRange{} of MdAPE, with fewer than a third of estimates within $\pm10\%$ of the price. Adding a grid (M6, M7) or selecting by similarity (M3) brings the error into the \mdapeGridRange{} range. The assessment ratio (M13) reaches \mdapeMthirteen{} on the targets it covers. The best single transparent configuration frozen on 2024 (M0: \mZeroDesc) reaches \mdapeMzero{} (95\% interval \ciMzeroLo{}--\ciMzeroHi{}) with \pTenMzero{} of estimates within $\pm10\%$, and the optimised reconciliation M9 \mdapeMnine{}; M9 places weights of \wNineRole{} on the assessment-ratio indication, \wNineGbm{} on the GBM-adjusted one and \wNineNone{} on the time-only one, and nothing on the other grids. Among the external benchmarks, the direct hedonic model M10 is the least accurate (\mdapeMten{}): a global model with municipality and neighbourhood effects cannot match the local information of comparables. The gradient-boosting model without the assessed value (M11a, \mdapeMelevenA{}) is only \transparencyCostNoRole{} points better than the best frozen comparables configuration; with the assessed value (M11b) it reaches \mdapeMelevenB{} (\ciMelevenBLo{}--\ciMelevenBHi{}) and \pTenMelevenB{} within $\pm10\%$, and the hybrid M12 \mdapeMtwelve{}, with \wTwelveGbm{} of its weight on M11b and the rest on the GBM- and hedonic-adjusted comparables. \begin{table}[H] \centering \caption{Horse race: out-of-sample accuracy, 2025--2026} \label{tab:competition} \resizebox{\textwidth}{!}{\input{tables/competition}} \begin{tablenotes}\footnotesize \item Note: MdAPE, MAPE, median signed error and the shares within $\pm5$, $\pm10$ and $\pm20\%$ in per cent, on the test targets for which the model returns a value; coverage is the share of such targets. The validation column gives the MdAPE on the 2024 targets. Weights of M9 and M12 and the configuration of M0 are chosen on 2024 only. Bootstrap confidence intervals for each entry are in Table~\ref{tab:competition_ci}. \end{tablenotes} \end{table} \begin{figure}[H] \centering \includegraphics[width=0.95\textwidth]{f4_competition.pdf} \caption{Out-of-sample MdAPE by model, 2025--2026. Orange: configurations of the sales comparison approach; green: empirically optimised configurations; blue: external benchmarks.} \label{fig:competition} \end{figure} Table~\ref{tab:pairs} gives the paired differences with cluster-bootstrap confidence intervals and cluster-robust $t$ statistics for the comparisons that matter. The gain from the hedonic grid over time-only adjustment (M7 $-$ M2, $t = \tMsevenMtwo{}$), the gain from similarity selection (M3 $-$ M2), and the gain of M11b over M0 ($t = \tMelevenBMzero{}$) are all precisely estimated and far from zero. The difference between the conventional and hedonic grids (M7 $-$ M6) is small but significant, as is the loss from the local grid (M8a $-$ M7). The hybrid M12 improves on M11b by only six hundredths of a point of MdAPE, a difference that is statistically distinguishable from zero ($t = \tMtwelveMelevenB{}$) but economically negligible: a quarterly-retrained tree model with coordinates and the assessed value already extracts almost all of the local information that comparables carry. On the common support of the ratio method, M0 has a lower MdAPE than M13 but a higher MAPE, so that the two are not ranked by all measures. \begin{table}[H] \centering \caption{Paired differences in accuracy with cluster-bootstrap 95\% confidence intervals} \label{tab:pairs} \footnotesize \resizebox{\textwidth}{!}{\input{tables/pairs_b999}} \begin{tablenotes}\footnotesize \item Note: $\Delta$ is the difference $A-B$ in MdAPE and in MAPE (percentage points) on the common test targets; a negative value means $A$ is more accurate. Confidence intervals from 999 cluster-bootstrap replications over municipality--quarter cells; $t$ is the cluster-robust statistic of equation~\eqref{eq:dm} for the MAPE difference. ``$A$ better'' is the share of targets on which $A$ has the smaller error. \end{tablenotes} \end{table} Two findings deserve emphasis. The first is the \emph{cost of transparency}: \transparencyCost{} points of MdAPE separate the best frozen configuration of the sales comparison approach from the best opaque model with the same information plus the assessed value, and \transparencyCostNoRole{} points separate it from the same model without the assessed value. Every estimate of M0 is the median of \bestK{} identified sales adjusted by published coefficients; every estimate of M11b is the output of several hundred trees. The second is the \emph{value of the assessed value}. The ratio method M13 and the GBM with roll information M11b dominate their counterparts without it by \roleGainRatio{} and \roleGainGbm{} points respectively. The assessed value summarises characteristics that the roll extract does not expose to us---quality, condition, outbuildings---and a local price level that the assessor has already estimated; it is, in effect, a professional valuation available for every property, and the evidence says it should be used, as \citet{thibodeau2003} argued for marking assessed values to market. \subsection{Where does the method hold, and where does it fail?} \label{sec:strata} Table~\ref{tab:density} and Figure~\ref{fig:density} report accuracy by market density for the main models. The dependence is steep and common to all methods: the MdAPE of the hedonic-grid method M7 is \densHighMseven{} where two hundred or more sales occurred within 2~km in the past year and \densLowMseven{} where fewer than five did. The gap between the best transparent method and the best opaque one is widest in the sparsest markets, where the tree model's ability to borrow strength from a wider area is most valuable, and narrowest in the densest ones; the pattern is the comparable-sales counterpart of the rural accuracy penalty that \citet{bogin2020} document for both appraisers and AVMs. Tables~\ref{tab:regions}--\ref{tab:price} in the Appendix report the same comparison by region, price quintile, age and physical link; accuracy is highest in the dense suburban regions (Laval, Outaouais, Montérégie) and lowest in the remote regions (Gaspésie, Nord-du-Québec, Côte-Nord), in the lowest price quintile (\prixQoneMseven{} for M7, against \prixQfourMseven{} in the fourth), for dwellings older than seventy years and for integrated dwellings---the segments where condition and unobserved characteristics carry the largest share of value. The lowest quintile deserves a word of caution: because quintiles are defined on the realised price, the APE of a cheap sale is mechanically inflated by any given dollar error, so part of the gradient across quintiles is a denominator effect rather than a failure of the method. \begin{table}[H] \centering \caption{Out-of-sample MdAPE (\%) by market density} \label{tab:density} \footnotesize \input{tables/density} \begin{tablenotes}\footnotesize \item Note: $n$ is the number of test targets of the stratum covered by M7; in the sparsest stratum it is \densLowN{} of \densLowNall{} targets because five sales within 5~km and 12 months are not always available. Each model's MdAPE is computed on the targets it covers. \end{tablenotes} \end{table} \begin{figure}[H] \centering \includegraphics[width=0.85\textwidth]{f5_densite.pdf} \caption{Out-of-sample MdAPE by market density (sales within 2~km in the preceding twelve months), selected models.} \label{fig:density} \end{figure} Table~\ref{tab:h5} addresses H5 by splitting the test targets into terciles of submarket heterogeneity, measured by the dispersion of the adjusted indications of the ten nearest comparables within 10~km and 24 months---a quantity available at valuation time. In the homogeneous tercile the transparent configuration M0 is within \hFiveGapHomog{} points of M11b; in the heterogeneous tercile the gap is \hFiveGapHeter{} points. Simple, explainable rules rival machine learning precisely where the market is informative enough for a few well-chosen sales to pin down value; machine learning earns its keep where the market is thin or heterogeneous. H5 is confirmed in direction, although the gap never closes entirely. \begin{table}[H] \centering \caption{Accuracy by submarket heterogeneity (H5): out-of-sample MdAPE (\%)} \label{tab:h5} \footnotesize \input{tables/h5} \begin{tablenotes}\footnotesize \item Note: terciles of the standard deviation $\hat\sigma$ of the log adjusted indications of the ten nearest sales within 10~km and 24 months (hedonic grid), computed on the test targets. \end{tablenotes} \end{table} \subsection{Value intervals} \label{sec:intervals} The dispersion of the adjusted indications provides an interval of value. Table~\ref{tab:intervals} reports, for the time-only, hedonic and GBM adjustments at two anchors, the coverage of the interval $\hat P_i \exp(\pm z\,\hat\sigma_i)$ with $z=1.28$ (nominal 80\% under normality) and with $z$ calibrated on the validation targets as the 80th percentile of the standardised absolute errors $|\ln P_i - \ln \hat P_i|/\hat\sigma_i$---a split-conformal calibration in the sense of \citet{lei2018}---together with the median width of the calibrated interval. The nominal intervals under-cover (\ivCovLo{}--\ivCovHi{}\% for a nominal 80\%): the dispersion of five or ten adjusted sales understates the uncertainty of the estimate, because the comparables share unobserved characteristics with each other but not with the subject, which is the error-based versus model-based distinction drawn by \citet{krause2020}. Calibrated on 2024, the intervals achieve their target coverage in 2025--2026 to within half a point, but at a median width of \ivWidth{} of value: an honest 80\% interval around a comparables-based estimate spans roughly $\pm$\ivHalf{}. The dispersion of the comparables is a valid but coarse measure of uncertainty, and the breadth of the calibrated interval is itself a finding about the precision the method can claim. \begin{table}[H] \centering \caption{Coverage of value intervals derived from the dispersion of adjusted indications} \label{tab:intervals} \resizebox{\textwidth}{!}{\input{tables/intervals}} \begin{tablenotes}\footnotesize \item Note: selection by distance, equal weights, median. Coverage is the share of test targets whose price lies within the interval; width is $2z\hat\sigma$ in per cent of value. \end{tablenotes} \end{table} \subsection{Towards an empirical comparable-recommendation system} \label{sec:recommend} The steep dependence on density suggests that the best rule might be conditional on the market. Table~\ref{tab:cond} implements the simplest version of a recommendation system: for each density stratum, the configuration (among all \nConfigs{} that return a value for at least 95\% of the stratum's validation targets) with the lowest MdAPE on 2024 is selected and applied to the 2025--2026 targets of that stratum. The selected rules do differ---the assessment-ratio adjustment is chosen in the two densest strata, the hedonic grid elsewhere, and the compromise or least-adjusted selection everywhere---but together they achieve an out-of-sample MdAPE of \condMdape{} with \condCov{} coverage, no better than the \mdapeMzero{} of the best single rule at full coverage. The reason is that the best single rule is already self-adapting: retaining the thirty most similar sales within 25~km and 24 months uses nearby sales where they exist and reaches further only where they do not. A recommendation system therefore adds value through the choice of adjustment (assessment ratio where same-roll comparables are plentiful) rather than through stratum-specific counts and radii. \begin{table}[H] \centering \caption{Density-conditional rules chosen on 2024 and evaluated on 2025--2026} \label{tab:cond} \resizebox{\textwidth}{!}{\input{tables/cond_rules}} \end{table}