% ============================================================================
\section{Robustness}
\label{sec:robustness}
% ============================================================================

\subsection{Stability over time and seasonality}
\label{sec:stability}

Table~\ref{tab:stability} reports the MdAPE and bias of the main models by quarter of the test period. The rankings are stable quarter by quarter, and the validation-period (2024) and test-period (2025--2026) MdAPEs of every configuration in Tables~\ref{tab:adjustments}--\ref{tab:competition} differ by about a point in the same direction, which indicates that the rules selected on 2024 did not overfit it; the test period was simply slightly easier for every method. The comparable-based methods display a seasonal bias: pooling the two years by calendar quarter (Table~\ref{tab:seasonal}), the hedonic-grid method M7 is too low in the first quarter (\seasQoneMseven{}) and too high in the third (\seasQthreeMseven{}), and the best configuration M0 and the direct hedonic model M10 show the same pattern (\seasQoneMzero{} and \seasQthreeMzero{}; \seasQoneMten{} and \seasQthreeMten{}). This is the signature of the point-in-time constraint: every method works at the level of the previous month, Quebec prices rise into the spring and soften after the summer \citep{ngai2014}, and the price level of the valuation month is not knowable on the valuation date. The gradient-boosting benchmarks do not share the pattern (M11b: \seasQoneMelevenB{} and \seasQthreeMelevenB{}). The reason is not that they observe the month of sale---at prediction time their month feature is fixed at the last training month, and their level is carried forward by the same index as the comparables---and we have not identified it; it may reflect the composition of the comparable pool, which over-represents the busy spring and summer months when the target sells in winter. A seasonal correction estimated on prior years is the obvious remedy for the comparables method, and we flag it as an extension that we have not implemented.

\begin{table}[H]
\centering
\caption{Stability over the test period: MdAPE and bias (\%) by quarter}
\label{tab:stability}
\footnotesize
\input{tables/stability}
\end{table}

\begin{table}[H]
\centering
\caption{Median signed error (\%) by calendar quarter, test targets of 2025 and 2026 pooled}
\label{tab:seasonal}
\footnotesize
\input{tables/seasonal_bias}
\end{table}

\subsection{Selection stability and data snooping}
\label{sec:selection_stability}

The configuration M0 is the best of \nConfigsCov{} configurations with at least 97\% coverage on the 2024 validation targets, and its test-period accuracy is reported on sales that played no role in that choice. Two further checks establish that the choice is not fragile. First, the validation and test rankings of the \nConfigs{} configurations are almost identical: the Spearman rank correlation between their 2024 and 2025--2026 MdAPEs is \spearmanAll{} over all configurations and \spearmanCov{} among those eligible for selection. Second, M0 ranks \mZeroRankTest{}nd of \nConfigsCov{} in the test period, within a hundredth of a point of the configuration that would have been chosen with hindsight (\oracleTest{}), which replaces the hedonic grid by GBM adjustments and is not transparent. Table~\ref{tab:selection_top10} lists the ten best configurations on 2024 and their test ranks: all ten combine the compromise selection, a 24-month window and the hedonic grid, differ only in $k$ (20 to 50) and $d$ (2 to 25~km), and all ten rank among the twelve best out of sample. Table~\ref{tab:selection_near} shows that the \nearQuarterN{} configurations within a quarter of a point of the best validation MdAPE have test MdAPEs between \nearQuarterMin{} and \nearQuarterMax{}. The accuracy attributed to M0 is therefore a property of a broad plateau of wide-net, similarity-based, adjusted configurations rather than of one lucky draw, which is what protects the headline figure from the data-snooping bias of \citet{white2000}.

\begin{table}[H]
\centering
\caption{The ten best configurations on the 2024 validation targets and their out-of-sample rank}
\label{tab:selection_top10}
\footnotesize
\resizebox{\textwidth}{!}{\input{tables/selection_top10}}
\begin{tablenotes}\footnotesize
\item Note: configurations with at least 97\% coverage on 2024 (\nConfigsCov{} of \nConfigs{}). ``Rank 2025--26'' is the rank of the configuration's test MdAPE among the same \nConfigsCov{} configurations.
\end{tablenotes}
\end{table}

\begin{table}[H]
\centering
\caption{Near-optimal configurations on 2024 and the range of their out-of-sample MdAPE (\%)}
\label{tab:selection_near}
\footnotesize
\input{tables/selection_near}
\end{table}

\subsection{Coverage and the common-support comparison}

Configurations with narrow radii or short windows abstain on a substantial share of targets, and their MdAPE on the targets they cover is not comparable to that of configurations that cover all targets. Table~\ref{tab:surface_cov} in the Appendix reports the coverage of every point of the error surface. Our configuration selection enforces 97\% coverage; relaxing the constraint selects a narrower configuration (\mZeroUncDesc) with a lower MdAPE on its own support (\mZeroUncMdape{}) but \mZeroUncCov{} coverage. The conclusions of Section~\ref{sec:surface} hold on the common support of all configurations with at least 97\% coverage, and the paired comparisons of Table~\ref{tab:pairs} are always computed on the targets covered by both methods.

\subsection{Exclusion rules}
\label{sec:exclusion}

The exclusion of extreme price-to-assessment ratios (rule R8) is the only rule that removes a non-trivial share of sales on a criterion related to price. Its bounds affect the results through two channels: the composition of the target set, on which every method is evaluated, and the composition of the comparable pool. Table~\ref{tab:r8} examines the first channel with the existing estimates by tightening the bounds on the targets' ratio while leaving the pool unchanged. Restricting the test targets to ratios in $[0.5, 2]$ keeps \rEightTightShare{} of them and lowers every MdAPE by about half a point (M0 \rEightTightMzero{}, M11b \rEightTightMelevenB{}); the ordering of the models and the gaps between them are unchanged, and even the drastic $[0.67, 1.5]$ window, which keeps four targets in five, preserves them. The sales removed by tighter bounds are those with the largest errors for every method, as one would expect of transactions whose price departs from any market reference. The second channel---re-running the backtest with a different pool---requires the distributed computation and was not re-executed; because the baseline bounds are wide and remove only 1.8\% of the raw sales, and because an appraiser would in any case discard such sales from the comparable set, we do not expect it to alter the comparisons. The flip rule R9 removes 794 sales and cannot matter at the scale of the sample.

\begin{table}[H]
\centering
\caption{Sensitivity to the price-to-assessment bounds of rule R8, test targets: MdAPE (\%)}
\label{tab:r8}
\footnotesize
\resizebox{\textwidth}{!}{\input{tables/r8_sensitivity}}
\begin{tablenotes}\footnotesize
\item Note: the test targets are restricted to the stated interval of the ratio of sale price to assessed value; comparable pools, grids and models are unchanged. ``Share'' is the share of the \nTargetsTest{} test targets retained.
\end{tablenotes}
\end{table}

\subsection{Alternative accuracy measures and cluster definition}
\label{sec:clusters}

The MdAPE is our headline measure because the distribution of valuation errors is heavy-tailed and the median is the appraiser's natural loss. Table~\ref{tab:competition} shows that the MAPE and the shares within $\pm5$, $\pm10$ and $\pm20\%$ rank the models almost identically---the only inversion is between M9 and M11a, which the MAPE separates more clearly than the MdAPE---and the paired bootstrap on MAPE differences in Table~\ref{tab:pairs} yields the same conclusions as on MdAPE differences. The bias column shows that no comparable-based method is materially biased out of sample once the whole test period is pooled. Table~\ref{tab:cluster_sens} re-estimates the confidence intervals of four key paired differences under alternative cluster definitions, from municipality--quarter cells (the baseline, with several thousand clusters) to region--quarter cells (about a hundred). The intervals widen as clusters become coarser, as they should when dependence at wider scales is allowed for, but none of the conclusions changes: the smallest difference that we call significant, M12 $-$ M11b, remains distinguishable from zero even with region--quarter clusters.

\begin{table}[H]
\centering
\caption{Sensitivity of the bootstrap confidence intervals to the cluster definition: 95\% intervals of the MdAPE difference (points)}
\label{tab:cluster_sens}
\footnotesize
\resizebox{\textwidth}{!}{\input{tables/pairs_cluster_sens}}
\begin{tablenotes}\footnotesize
\item Note: 399 cluster-bootstrap replications for each alternative definition (999 for the baseline, Table~\ref{tab:pairs}).
\end{tablenotes}
\end{table}

\subsection{Index construction}

The point-in-time index uses an expanding window of all prior sales and municipality fixed effects. Because comparables are at most 24 months old and the time adjustment contributes only \gainTime{}--\gainTimeLong{} points of accuracy (Section~\ref{sec:adjustments}), the comparable-based results are insensitive to the details of the index. The global benchmarks are more exposed: their level in 2025--2026 is carried entirely by the index from the last training quarter to the valuation month, and we verified that an index without municipality fixed effects, which is contaminated by the shift of sales towards less expensive municipalities, lowers their level.

\subsection{Benchmark specification}
\label{sec:benchspec}

The gradient-boosting benchmarks are trained with an $L_1$ objective so that they estimate the conditional median, which is the quantity the MdAPE rewards; the same models trained with a squared-error objective have a median bias of about $-5\%$ in-sample and out of sample, because the distribution of log prices conditional on characteristics is left-skewed, and a correspondingly higher MdAPE; the direct hedonic model M10, estimated by least squares, is for the same reason recentred on the median of its validation residuals. The number of trees chosen by early stopping on the last 10\% of the training window is floored at 300 because the final, partial months of the sample make that slice unreliable. The hybrid weights of M12 and the reconciliation weights of M9 are convex combinations fitted on 2024 and applied unchanged; M9 places \wNineRole{} of its weight on the assessment-ratio indication and \wNineGbm{} on the GBM-adjusted one, and M12 places \wTwelveGbm{} on M11b. Neither combination was tuned beyond the choice of its components.