% ============================================================================ \section{Robustness, uncertainty, missingness and failure cases} \label{sec:robustness} % ============================================================================ \subsection{Sensitivity of the indication to schedule and depreciation assumptions} \label{sec:sensitivity} Table~\ref{tab:sensitivity} and Figure~\ref{fig:sensitivity} vary each assumption of the schedule one at a time around the B3 baseline. The ordering is instructive. The unit-cost anchor dominates (\sensAnchorAAltusMontralHardCostsDelta\% for anchor~A). Within anchor~B, the economic life (\sensEconomicLifeFiveZeroYDelta\% at 50 years, \sensEconomicLifeSevenFiveYDelta\% at 75), the regional level (\sensRegionalLevelZeroNineZeroDelta\% and \sensRegionalLevelOneOneZeroDelta\% for $\pm$10\%), the external-obsolescence scenario (\sensExternalObsolescenceOneZeroDelta\%), a flat storey multiplier (\sensStoreyMultiplierFlatOneZeroAllDelta\%), the strength of the condition adjustment (\sensConditionEffectiveAgeHalfStrengthDelta\% at half, \sensConditionEffectiveAgeOneFiveStrenDelta\% at one and a half) and whether the roll land value is indexed to the valuation date (\sensLandNotIndexedDelta\% when it is not) each move the median indication by five to seven percent; the grade band, the mark-ups and the add-ons move it by two to five percent. Removing depreciation altogether raises the indication by \sensNoDepreciationRcnPlusSPlusLDelta\% and takes its ratio to the asking price to \sensNoDepreciationRcnPlusSPlusLPriceRatio{}---replacement cost new plus land exceeds the asking price for the median house, which is the normal state of a market in which existing houses sell below reproduction, and which confirms that the question is the size of depreciation, not the existence of a gap. \begin{table}[H] \centering \caption{One-at-a-time sensitivity of the B3 indication (mid-size extractor)} \label{tab:sensitivity} \scriptsize \begin{adjustbox}{max width=\textwidth} \input{tables/sensitivity} \end{adjustbox} \end{table} \begin{figure}[H] \centering \includegraphics[width=0.95\textwidth]{f11_sensitivity.pdf} \caption{Change in the median B3 indication under each alternative assumption, relative to the anchor-B baseline.} \label{fig:sensitivity} \end{figure} \subsection{Counterfactual scenarios} \label{sec:scenarios} Table~\ref{tab:scenarios} reports labelled counterfactuals applied to every listing: a one-level change in the composite grade, a one-point change in the composite condition, extensive renovation evidence, a finished basement and an attached garage. Among listings whose indication changes, a one-level grade shift moves the indication by about \scenGradePlusOneLevelV\% up or \scenGradeMinusOneLevelV\% down, a one-point condition shift by \scenConditionPlusOneV\% up or \scenConditionMinusOneV\% down, and renovation evidence by \scenExtensiveRenovationEvidenceV\% for the \scenExtensiveRenovationEvidenceShare\% of listings not already read as extensively renovated; a finished basement adds \scenBasementFinishedV\% for the \scenBasementFinishedShare\% of listings whose basement is not already recorded as finished and a garage \scenAttachedGarageAddedV\% for the \scenAttachedGarageAddedShare\% without one. These magnitudes are the stakes of an extraction error on each attribute, and they are of the same order as the agreement gaps measured in Section~\ref{sec:results_extraction}: a kitchen misgraded by one level costs about as much as the whole image channel adds. \begin{table}[H] \centering \caption{Counterfactual scenarios on the B3 indication (labelled scenarios, not predictions)} \label{tab:scenarios} \footnotesize \begin{adjustbox}{max width=\textwidth} \input{tables/scenarios} \end{adjustbox} \end{table} \subsection{Monte Carlo propagation of attribute and parameter uncertainty} \label{sec:mc} The Monte Carlo layer draws, for each of \mcDraws{} replications, a grade and a condition around the extracted values with a spread that grows with the extractor's stated uncertainty and with the absence of an informative observation, together with unit costs within the published band, an economic life between 50 and 75 years, mark-ups and land indexing within their ranges and a regional level within $\pm$10\%. Figure~\ref{fig:mc} and Table~\ref{tab:mc} summarise the result. The 10--90\% interval has a median relative width of \mcRelWidth\% of the indication (interquartile range \mcRelWidthLo--\mcRelWidthHi\%), and one-at-a-time perturbations rank the sources of that width: grade (\oatGradePlusOne\% for a one-level shift), mark-ups (\oatSoftPlusEiHigh\%), condition (\oatConditionPlusOne\%), regional level (\oatRegionalOneOneZero\%), land indexing (\oatLandNotIndexed\%), unit-cost band (\oatUnitCostBand\%) and economic life (\oatEconomicLifeFiveZero\%). The intervals are honest about what they contain and silent about what they do not: they cover the asking price for \mcPriceCoverage\% of listings and the indexed assessor total for \mcAssessorCoverage\%, far below the nominal 80\%. The under-coverage is not a defect of the propagation but a measurement of what it omits---the dispersion of market prices around any cost indication, the error of the assessor's own land and building split, and the latent attributes---and it is the quantitative reason the paper describes the indication as a screening tool. \begin{figure}[H] \centering \includegraphics[width=\textwidth]{f13_monte_carlo.pdf} \caption{Left: distribution of the relative width of the 10--90\% Monte Carlo interval. Right: intervals, median indications and asking prices for a random subset of 250 listings sorted by indication.} \label{fig:mc} \end{figure} \begin{table}[H] \centering \caption{One-at-a-time contributions to the uncertainty of the indication (median absolute change, B4)} \label{tab:mc} \footnotesize \input{tables/mc_oat} \end{table} \subsection{Missing-not-at-random photographs: a bounding simulation} \label{sec:mnar_sim} Section~\ref{sec:mnar} documented the omissions sellers actually make. To bound the bias that \emph{additional} strategic omission would cause, we simulate sellers who drop kitchen, bathroom and basement photographs with a probability falling from 0.8 for rooms the extractor rates in the worst condition to 0.05 for rooms rated like new, re-aggregate the remaining photographs and recompute the B2 indication (\mnarSimDraws{} draws). The simulation removes \mnarSimDropped\% of photographs, lowers the share of listings with a composite grade from \mnarSimGradeAvailFull\% to \mnarSimGradeAvail\%, shifts the composite condition by \mnarSimCondShift{} points on average, moves the grade up for \mnarSimGradeUp\% of listings and down for \mnarSimGradeDown\%, and changes the median indication by \mnarSimVRatioPct\%. Because the rooms most likely to be hidden are those in the worst condition, omission flatters the remaining evidence; the effect on the indication is small in the median, because the condition factor acts on depreciation and the default grade absorbs missing kitchens, but it is one-directional, and no re-weighting can recover information that was never photographed. The appropriate remedy is institutional---mandatory room coverage in listings, or an inspection---rather than statistical. \subsection{Failure cases} \label{sec:failures} Table~\ref{tab:taxonomy} classifies the disagreements between the image channel and the broker's roofing field using the visual cues the extractor cited, and lists the most frequent cladding disagreements. Roof disagreements (\taxRoofDisagreements{} of \taxRoofCompared{} compared exterior images) come mainly from low-slope roofs whose membrane cannot be seen from the street and from metal-versus-shingle look-alikes, with smaller shares attributable to distance and to partial visibility; cladding disagreements (\taxCladdingDisagreements{} of \taxCladdingCompared{}) are dominated by stone read where the broker lists brick or vinyl and by wood read where the broker lists vinyl or brick---material pairs that look alike in photographs---and by brokers listing a material the image shows only on a side elevation. These are the error modes the ontology anticipated, and they argue for two design rules in deployment: attributes should be reported with the scene they were read from, and a broker's own material field should be preferred to the image when the two disagree and the image's confidence is not high. Beyond attribute errors, three systematic failure cases emerged. Split-level and one-and-a-half-storey houses are counted differently by the roll, the broker and the image, and the schedule's storey multiplier makes the choice consequential. Staged or virtually staged interiors, flagged by the extractors in a minority of listings, inflate finish grades and condition; the share is small but not negligible in the upper price quartile. And new construction advertised with renderings or model-home photographs is graded on an idealised building; the roll's year built catches most such cases, but the image channel does not know that it is looking at a drawing. \begin{table}[H] \centering \caption{Taxonomy of image--broker disagreements on roof covering and cladding (mid-size extractor)} \label{tab:taxonomy} \footnotesize \input{tables/error_taxonomy} \begin{tablenotes}\footnotesize \item Note: roof categories are assigned from the cue words in the extractor's one-line evidence statement, in the order listed; cladding rows give the most frequent (image, broker) pairs among disagreements. \end{tablenotes} \end{table} \subsection{Review load under confidence-based triage} \label{sec:review} A deployment that routes uncertain listings to a human must decide what ``uncertain'' means. Table~\ref{tab:review} shows the share of listings flagged by rules that combine a mean-confidence threshold with the absence of a finish grade or the presence of a visible defect. With the mid-size extractor, a threshold of 0.7 flags \reviewsonnetZeroSeven\% of listings and 0.8 flags \reviewsonnetZeroEight\%; the rules are driven by the defect and missing-grade components rather than by confidence, which barely varies. Since no human review was performed, the table quantifies the cost of review rules, not their benefit; the annotation protocol in Appendix~\ref{app:protocol} is designed to measure the benefit. \begin{table}[H] \centering \caption{Share of listings flagged for review under confidence-based rules} \label{tab:review} \footnotesize \input{tables/review_load} \end{table} \subsection{Replication across extractor tiers} \label{sec:tiers} Every cost result above was recomputed with the small extractor's attributes (Table~\ref{tab:levels_by_tier}). The level progression from B0 to B3 and the alignment statistics are close across tiers, which says that the cost indication is not sensitive to which of the two full-sample extractors specified the building, even though their judgment attributes agree only moderately: the schedule's coarse grades and the condition factor absorb much of the disagreement. The largest extractor was run on a 200-listing subsample for the cross-model table and is not used for cost results. \begin{table}[H] \centering \caption{Cost indications by extractor tier and information set (anchor B)} \label{tab:levels_by_tier} \footnotesize \begin{adjustbox}{max width=\textwidth} \input{tables/levels_by_tier} \end{adjustbox} \end{table}