révision scientifique complète (editorial board, 2026-10-17)
WP 16 v1.0 → v1.1 : indices régionaux et figure 1 corrigés, H3/H4 relus, mécanisme GBM retiré, équations ajoutées (erreurs, grille, ridge, bootstrap), bootstrap 999 rép. + t robuste, stabilité de la sélection (rang 2/1 370), sensibilité R8 et grappes, 64 références vérifiées (36 ajoutées, DOI), docs/REVIEW-REPORT.md et docs/CHANGELOG-REVISION.md, README v1.1, main_anon.pdf. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
50 changed files +2,604 −309
modified
README.md
+62 −38
@@ -4,7 +4,7 @@ | ||
| 4 | 4 | **UQO Working Paper No. 16** — *A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and |
| 5 | 5 | Value Reconciliation — Evidence from 391,000 Single-Family Transactions in Quebec, 2021–2026* |
| 6 | 6 | |
| 7 | −[-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) | |
| 7 | +[-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) | |
| 8 | 8 | [](pipeline/) |
| 9 | 9 | [](docs/PROTOCOLE.md) |
| 10 | 10 | [](pipeline/) |
@@ -14,12 +14,14 @@ Value Reconciliation — Evidence from 391,000 Single-Family Transactions in Que | ||
| 14 | 14 | > **TL;DR** — The sales comparison approach rests on professional convention: three comparables, a short window, a |
| 15 | 15 | > small radius, an adjustment grid, a reconciliation rule. We backtest 2,256 configurations of the approach, point in |
| 16 | 16 | > time, on 189,354 masked target sales drawn from 390,996 Quebec single-family transactions matched to the assessment |
| 17 | −> roll (configurations chosen on 2024, evaluated on 2025–2026). **Adjustments dominate** (median absolute error of the | |
| 18 | −> five nearest sales: 18.1% time-only → 13.0% with a hedonic grid); **similar beats near** (selecting physically similar | |
| 19 | −> sales gives the same gain with no adjustment); **the optimal number of comparables is ~20–30, not 3**; windows should | |
| 20 | −> be wide, radius barely matters, weighting and reconciliation matter little. The best transparent rule reaches | |
| 21 | −> **11.1%** out of sample versus 10–11% for a quarterly-retrained gradient-boosting model: the cost of transparency is | |
| 22 | −> small, and accuracy is governed by submarket liquidity. | |
| 17 | +> roll (configurations chosen on 2024, evaluated on 2025–2026; paired differences tested with a 999-replication cluster | |
| 18 | +> bootstrap over municipality × quarter cells). **Adjustments dominate** (median absolute error of the five nearest | |
| 19 | +> sales: 18.1% time-only → 13.0% with a hedonic grid); **similar beats near** (selecting physically similar sales gives | |
| 20 | +> the same gain with no adjustment); **the optimal number of comparables is ~20–30, not 3**; windows should be wide, | |
| 21 | +> radius barely matters, weighting and reconciliation matter little. The best transparent rule reaches **11.1%** out of | |
| 22 | +> sample versus 9.0% for a quarterly-retrained gradient-boosting model that uses the assessed value (10.7% without it): | |
| 23 | +> the cost of transparency is 2.2 points, and the chosen rule ranks 2nd of 1,370 eligible configurations out of sample. | |
| 24 | +> Accuracy is governed by submarket liquidity. | |
| 23 | 25 | |
| 24 | 26 | --- |
| 25 | 27 | |
@@ -27,47 +29,53 @@ Value Reconciliation — Evidence from 391,000 Single-Family Transactions in Que | ||
| 27 | 29 | |
| 28 | 30 | | Item | Value | |
| 29 | 31 | |---|---| |
| 30 | −| Version | 1.0 (October 2026) | | |
| 31 | −| SSRN | abstract **7626720** (PRELIMINARY_UPLOAD, 2026-10-17) — uploaded PDF carries the old cover "Working Paper No. 11"; **re-upload `paper/main.pdf` (No. 16)** | | |
| 32 | −| Journal | — | | |
| 32 | +| Version | **1.1 (October 2026)** — revised 2026-10-17 after a full editorial-board review (see `docs/REVIEW-REPORT.md`, `docs/CHANGELOG-REVISION.md`); v1.0 frozen as git tag `pre-revision-20261017` and `docs/main_pre-revision-20261017.pdf` | | |
| 33 | +| SSRN | abstract **7626720** (PRELIMINARY_UPLOAD, 2026-10-17) — uploaded PDF carries the old cover "Working Paper No. 11" and the v1.0 text; **re-upload `paper/main.pdf` (No. 16, v1.1)** | | |
| 34 | +| Journal | target *Journal of Real Estate Finance and Economics*; fallback *Journal of Real Estate Research*. Submission copies: `Recherche-UQO/_soumissions/revues/sales-comparison-validation/` (anonymous and with-authors PDFs) | | |
| 33 | 35 | | Code & data | pipeline in `pipeline/`, run on the MacLustr node M2U64; the Vrai Prix database snapshot is not redistributed | |
| 34 | 36 | |
| 35 | 37 | ## 🎯 What this paper does |
| 36 | 38 | |
| 37 | 39 | For every target sale the price is masked and the approach is applied with only the information available on the |
| 38 | −valuation date: candidate comparables are selected (nearest / most similar / compromise, k ∈ {1 … 50}, radius d, | |
| 39 | −window T), adjusted (none / time only / stylized conventional grid / hedonic grid estimated quarterly / assessment | |
| 40 | −ratio), weighted and reconciled (mean, median, weighted). Error surfaces Error(k, d, T) are mapped, the best | |
| 41 | −configuration is chosen on the 2024 validation year and evaluated out of sample on 2025–2026 against hedonic, | |
| 42 | −gradient-boosting, assessment-ratio and hybrid benchmarks. Accuracy is then decomposed by submarket liquidity, region, | |
| 43 | −price stratum and property type. | |
| 40 | +valuation date: candidate comparables are selected (nearest / most similar / compromise / least adjusted, k ∈ {1 … 50}, | |
| 41 | +radius d, window t), adjusted (none / time only / stylized conventional grid / hedonic grid estimated quarterly / | |
| 42 | +regional grid / local ridge grid / GBM adjustments / assessment ratio), weighted and reconciled (median, mean, weighted). | |
| 43 | +Error surfaces Error(k, d, t) are mapped, the best configuration is chosen on the 2024 validation year and evaluated out | |
| 44 | +of sample on 2025–2026 against hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks. Accuracy is then | |
| 45 | +decomposed by submarket liquidity, region, price stratum and property type, and the selection step itself is audited | |
| 46 | +(validation → test rank stability). | |
| 44 | 47 | |
| 45 | 48 | ## 🏆 Headline results |
| 46 | 49 | |
| 47 | 50 | | Object | Estimate (test 2025–2026, MdAPE) | |
| 48 | 51 | |---|---| |
| 49 | 52 | | 5 nearest sales, time adjustment only (M2) | 18.1% | |
| 50 | −| 5 nearest sales, hedonic adjustment grid (M7) | 13.0% | | |
| 51 | −| 5 most similar sales, no physical adjustment | ≈ same gain as the grid | | |
| 52 | −| Number of comparables with a grid, k = 1 → 3 → 5 → 10 → 20–30 | 16.5% → 13.8% → 13.0% → 12.4% → minimum | | |
| 53 | −| Best transparent rule chosen in 2024 (M0: compromise, k = 30, ≤ 25 km, 24 months, hedonic grid) | **11.1%** | | |
| 54 | −| Assessment-ratio comparable (M13) | 11.5% | | |
| 55 | −| Quarterly-retrained GBM using assessed value (M11b) / hybrid (M12) | ≈ 10–11% | | |
| 56 | −| Liquidity effect (M7): > 200 sales within 2 km / year vs < 20 | 10.2% vs 25.3% | | |
| 53 | +| 5 nearest sales, hedonic adjustment grid (M7) | 13.0% (M7 − M2: −5.1 pts, 95% CI [−5.26; −4.88]) | | |
| 54 | +| 5 most similar sales, no physical adjustment (M3) | 13.0% | | |
| 55 | +| Number of comparables with a grid, k = 1 → 3 → 5 → 10 → 20–50 | 16.5% → 13.8% → 13.0% → 12.4% → 12.1% (flat) | | |
| 56 | +| Best transparent rule chosen in 2024 (M0: compromise, k = 30, ≤ 25 km, 24 months, hedonic grid) | **11.1%** [10.8; 11.5]; rank 2 / 1,370 out of sample | | |
| 57 | +| Assessment-ratio comparable (M13, 90.3% coverage) | 11.5% | | |
| 58 | +| Quarterly-retrained GBM without / with assessed value (M11a / M11b) | 10.7% / 9.0% [8.8; 9.2] | | |
| 59 | +| Hybrid comparables + GBM (M12) | 8.9% (M12 − M11b: −0.06 pts, t = −8.3) | | |
| 60 | +| Cost of transparency (M11b − M0) | 2.2 pts [2.0; 2.3], t = −36; 1.4 pts in the most homogeneous tercile | | |
| 61 | +| Liquidity effect (M7): ≥ 200 sales within 2 km / year vs < 5 | 10.2% vs 25.3% | | |
| 62 | +| Calibrated 80% value interval (hedonic grid, k = 10) | median width 66.5% of value (≈ ±33%) | | |
| 57 | 63 | |
| 58 | 64 | ## 🗂 Repository layout |
| 59 | 65 | |
| 60 | 66 | ``` |
| 61 | 67 | sales-comparison-validation/ |
| 62 | 68 | ├── pipeline/ # 01_dataset → 02_index → 03_models → 04_backtest (+04b_gather) → 05_analyze → 06_report |
| 69 | +│ └── 07_revision_analyses.py # laptop-side complementary analyses (bootstrap B=999, selection stability, R8, seasonality) | |
| 63 | 70 | ├── run_all.sh # full chain on the compute node (CV_VERSION=YYYY-MM-DD) |
| 64 | 71 | ├── run_distributed.sh # 15-shard backtest across 5 MacLustr nodes, gather on M2U64 |
| 65 | 72 | ├── make_pdf.sh # pull out/<version>/ + docs/ from M2U64, compile docs/RAPPORT.pdf (French report) |
| 66 | −├── docs/ # PROTOCOLE.md (protocol), RAPPORT.md/.pdf (French results report), README-fr.md | |
| 73 | +├── docs/ # PROTOCOLE.md, RAPPORT.md/.pdf (French), README-fr.md, REVIEW-REPORT.md, CHANGELOG-REVISION.md | |
| 74 | +├── results/revision-2026-10-17/ # outputs of 07_revision_analyses.py (csv/json, committed) | |
| 67 | 75 | ├── ref/ # MUN.csv — municipality → MRC → region (MAMH) |
| 68 | 76 | ├── out/<version>/ # sales.parquet, index_asof.npy, models_*.json, est_E*.npy, tables/, fig/, results.json — not committed |
| 69 | 77 | ├── data/ # vraiprix-<date>.db snapshot (node only) — not committed |
| 70 | −└── paper/ # main.tex + sections/ + tables/ (generated) + figures/ → main.pdf | |
| 78 | +└── paper/ # main.tex (+ main_anon.tex) + sections/ + tables/ (generated) + figures/ → main.pdf | |
| 71 | 79 | ``` |
| 72 | 80 | |
| 73 | 81 | ## 🚀 Quick start |
@@ -76,10 +84,12 @@ sales-comparison-validation/ | ||
| 76 | 84 | rsync -a --exclude out --exclude data --exclude .git ./ M2U64:comparables-validation/ |
| 77 | 85 | ssh M2U64 'cd ~/comparables-validation && CV_VERSION=2026-10-10 ./run_all.sh' # ≈ 30 min (or ./run_distributed.sh) |
| 78 | 86 | ./make_pdf.sh # pull out/ + docs/, build RAPPORT.pdf |
| 79 | −cd paper && python3 make_paper_tables.py && make # tables + numbers.tex macros → main.pdf | |
| 87 | +python3 -I pipeline/07_revision_analyses.py # ≈ 80 s on the laptop: bootstrap, selection stability, R8, seasonality | |
| 88 | +cd paper && python3 make_paper_tables.py && make && latexmk -pdf main_anon.tex # tables + numbers*.tex macros → main.pdf, main_anon.pdf | |
| 80 | 89 | ``` |
| 81 | 90 | |
| 82 | −Variables: `CV_VERSION` (output folder), `CV_SAMPLE=3000` (target subsample for a dry run), `CV_PROCS` (processes, 20). | |
| 91 | +Variables: `CV_VERSION` (output folder), `CV_SAMPLE=3000` (target subsample for a dry run), `CV_PROCS` (processes, 20), | |
| 92 | +`CV_BOOT` (bootstrap replications in `07_revision_analyses.py`, 999). | |
| 83 | 93 | Environment on M2U64: `~/uqo-eval-terrains/env` (Python 3.14, pandas, numpy, scipy, lightgbm, matplotlib). |
| 84 | 94 | |
| 85 | 95 | ## 🗃 Data |
@@ -87,8 +97,9 @@ Environment on M2U64: `~/uqo-eval-terrains/env` (Python 3.14, pandas, numpy, sci | ||
| 87 | 97 | Snapshot of the Vrai Prix production database (2026-10-10): 766,596 residential transactions and 3.75 M assessment-roll |
| 88 | 98 | units. Single-family sales matched to the roll, cleaned by logged rules R0–R11 (`out/<v>/exclusions.csv`): |
| 89 | 99 | 456,950 → **390,996** sales (2021-01-05 → 2026-09-29), split train 201,642 / validation 64,795 (2024) / test 124,559 |
| 90 | −(2025–2026). Median price $400,000. All numbers in the paper are macros in `paper/tables/numbers.tex`, regenerated by | |
| 91 | −`paper/make_paper_tables.py` from `out/<version>/`. | |
| 100 | +(2025–2026). Median price $400,000. All numbers in the paper are macros in `paper/tables/numbers.tex` (regenerated by | |
| 101 | +`paper/make_paper_tables.py` from `out/<version>/`) and `paper/tables/numbers_revision.tex` (regenerated by | |
| 102 | +`pipeline/07_revision_analyses.py`). | |
| 92 | 103 | |
| 93 | 104 | ## 🔬 Methodology |
| 94 | 105 | |
@@ -96,6 +107,7 @@ units. Single-family sales matched to the roll, cleaned by logged rules R0–R11 | ||
| 96 | 107 | 2. **Backtest of 2,256 configurations** × benchmarks on 189,354 targets (`04_backtest.py`, sharded; `04b_gather.py`). |
| 97 | 108 | 3. **Selection on 2024, evaluation on 2025–2026**; error surfaces, marginal effects of each rule, conditional rules by stratum (`05_analyze.py`). |
| 98 | 109 | 4. **Decomposition** by liquidity, region, price, type; stability over time; prediction intervals (`06_report.py`, `paper/make_paper_tables.py`). |
| 110 | +5. **Inference and audit of the selection step** (`07_revision_analyses.py`): cluster bootstrap (999 replications, three alternative cluster definitions), cluster-robust Diebold–Mariano-type t statistics, per-model confidence intervals, validation → test rank stability of all configurations, sensitivity to the R8 screen, seasonal bias by calendar quarter. | |
| 99 | 111 | |
| 100 | 112 | ## 🖼 Figures & tables inventory |
| 101 | 113 | |
@@ -106,17 +118,22 @@ units. Single-family sales matched to the roll, cleaned by logged rules R0–R11 | ||
| 106 | 118 | | `f3_ajustements` | contribution of adjustment types | |
| 107 | 119 | | `f4_competition` | comparables vs hedonic / GBM / ratio / hybrid | |
| 108 | 120 | | `f5_densite`, `f6_k_par_densite` | accuracy and optimal k by submarket liquidity | |
| 109 | −| `f7_indice` | point-in-time price index | | |
| 121 | +| `f7_indice` | descriptive price index, province and five regions | | |
| 110 | 122 | |
| 111 | −Tables (`paper/tables/*.tex`, 28 generated): descriptives, exclusions, selection, adjustments, grid, weighting, | |
| 112 | −surfaces, competition, density, regions, strata, stability, intervals, conditional rules, hypotheses H2–H5. | |
| 123 | +Tables (`paper/tables/*.tex`, 33 generated): descriptives, exclusions, selection, adjustments, grid, weighting, | |
| 124 | +surfaces, competition (+ bootstrap CIs), paired differences (B = 999, t), cluster-definition sensitivity, density, | |
| 125 | +regions, strata, stability, seasonal bias, intervals, conditional rules, hypotheses H2–H5, selection stability | |
| 126 | +(top-10, near-optimal set), R8 sensitivity, regional index growth. | |
| 113 | 127 | |
| 114 | 128 | ## ⚠️ Limitations |
| 115 | 129 | |
| 116 | 130 | - Adjustments simulated on observable characteristics (area, lot, age, storeys, attachment, assessed value); unobserved |
| 117 | − condition and quality adjustments made by appraisers are not modelled. | |
| 131 | + condition and quality adjustments made by appraisers are not modelled (within-municipality R² of the grid ≈ 0.36). | |
| 118 | 132 | - Single-family houses in Quebec only; results for condos/plexes not covered. |
| 119 | −- Registry prices may include non-arm's-length transfers surviving the screens. | |
| 133 | +- Registry prices may include non-arm's-length transfers surviving the price/assessment screen; sales dated by | |
| 134 | + registration, publication lag ignored. | |
| 135 | +- The seasonal bias of the comparables estimate (−2.4 pts in Q1, +1.6 pts in Q3 for M7) is documented but not corrected; | |
| 136 | + the backtest was not re-run with alternative screening bounds for the comparable pool (target-side sensitivity only). | |
| 120 | 137 | |
| 121 | 138 | ## 📝 Citation |
| 122 | 139 | |
@@ -128,7 +145,8 @@ surfaces, competition, density, regions, strata, stability, intervals, condition | ||
| 128 | 145 | type = {Working Paper}, |
| 129 | 146 | number = {16}, |
| 130 | 147 | year = {2026}, |
| 131 | − month = {October} | |
| 148 | + month = {October}, | |
| 149 | + note = {Version 1.1} | |
| 132 | 150 | } |
| 133 | 151 | ``` |
| 134 | 152 | |
@@ -140,10 +158,16 @@ Université du Québec en Outaouais (UQO), Gatineau, QC. | ||
| 140 | 158 | |
| 141 | 159 | ## 🗒 Notes |
| 142 | 160 | |
| 161 | +- **Revision 2026-10-17 (v1.1)**: full scientific review by an editorial-board pass — see `docs/REVIEW-REPORT.md` | |
| 162 | + (diagnosis, verdicts, remaining limitations, analyses recommended but not executed) and `docs/CHANGELOG-REVISION.md` | |
| 163 | + (audit trail of every change, references added with DOIs, re-analyses executed). No headline estimate changed; the | |
| 164 | + regional index figure, one incorrect regional-growth statement and one unsupported mechanism (GBM seasonality) were corrected. | |
| 143 | 165 | - Numbering: issued on 2026-10-10 as "Working Paper No. 11" (number already used by *Half a Million Prices, Twenty |
| 144 | − Models*); renumbered **16** on 2026-10-17. The SSRN upload 7626720 predates the renumbering. | |
| 166 | + Models*); renumbered **16** on 2026-10-17. The SSRN upload 7626720 predates the renumbering and the revision. | |
| 145 | 167 | - Deviation from `Recherche-UQO/HARMONISATION.md` §2: the pipeline keeps `pipeline/` (not `scripts/`) and `out/` (not |
| 146 | − `results/`) because the same tree runs on M2U64 under `~/comparables-validation` with these paths. | |
| 147 | −- French protocol and report in `docs/` (`PROTOCOLE.md`, `RAPPORT.md/.pdf`); original French README in `docs/README-fr.md`. | |
| 168 | + `results/`) because the same tree runs on M2U64 under `~/comparables-validation` with these paths; the laptop-side | |
| 169 | + revision analyses write to `results/revision-2026-10-17/` (committed). | |
| 170 | +- French protocol and report in `docs/` (`PROTOCOLE.md`, `RAPPORT.md/.pdf`, generated by `06_report.py` from the | |
| 171 | + 2026-10-10 run and left unchanged); original French README in `docs/README-fr.md`. | |
| 148 | 172 | - Per-node copies of the working tree pulled on 2026-10-10 (`M*omparables-validation/`) were removed from the repository |
| 149 | 173 | and archived in `Recherche-UQO/_archives/sales-comparison-node-copies-20261010/`. |
added
docs/CHANGELOG-REVISION.md
+117 −0
@@ -0,0 +1,117 @@ | ||
| 1 | +# Journal d'audit — révision scientifique complète (editorial board, 2026-10-17) | |
| 2 | + | |
| 3 | +WP 16 — *Does the Sales Comparison Approach Actually Work?* — version 1.0 (2026-10-10, 38 p.) → **version 1.1 (October 2026, 52 p.)**. | |
| 4 | +État de départ figé : tag git `pre-revision-20261017`, `docs/main_pre-revision-20261017.pdf`. Aucun fichier d'origine n'a été | |
| 5 | +supprimé hors des deux gabarits parasites signalés ci-dessous ; aucun chiffre du pipeline distribué n'a été recalculé (données | |
| 6 | +absentes du poste de travail, interdiction de relancer le cluster). | |
| 7 | + | |
| 8 | +## 1. Corrections majeures (où / avant / après / pourquoi / preuve) | |
| 9 | + | |
| 10 | +| # | Où | Avant (v1.0) | Après (v1.1) | Pourquoi / preuve | | |
| 11 | +|---|---|---|---|---| | |
| 12 | +| 1 | `data.tex` §3.3 | « over 2025–2026 prices in the Montréal region rose by roughly a third of the increase recorded in the Quebec City region and half of that in the Montérégie » | « between December 2024 and September 2026 prices rose by about 25% in the Capitale-Nationale and 14% in the Montérégie, against 4% on the island of Montréal » (+ gains 2021-2026 : province 79 %, Montréal 33 %, Estrie 112 % ; repli 2022-23 : −13 %) | Recalcul depuis `out/2026-10-10/index_full.csv` (script `07_revision_analyses.py`, section f ; `results/revision-2026-10-17/index_growth.csv`). Montréal : +3,6 % vs 25,4 % (≈ 1/7, pas 1/3) et vs 13,9 % (≈ 1/4, pas 1/2). Nouvelle table A `tab:index_growth`. | | |
| 13 | +| 2 | `paper/figures/f7_indice.pdf` (Fig. 1) | Légende « Hedonic price index by region » mais une seule courbe (province) tracée | Province + Montréal, Capitale-Nationale, Montérégie, Outaouais, Estrie | Bogue de `make_paper_tables.py` : codes de région comparés à `"6"`, `"3"`… alors que le CSV contient `"06"`, `"03"` (zéro initial). Corrigé (`str.zfill(2)`), figure régénérée. | | |
| 14 | +| 3 | `robustness.tex` §6.1 | « The gradient-boosting benchmark, which includes the month of sale as a feature, absorbs most of this seasonality » | Le mécanisme est retiré : à la prédiction, la variable mois du GBM est fixée au dernier mois d'entraînement (`03_models.py`, `Gall["month"] = m_asof − 1`) et le niveau est reporté par le même indice que les comparables ; le biais saisonnier est documenté par trimestre civil (nouvelle table `tab:seasonal`) pour 8 modèles, l'absence de saisonnalité des GBM est constatée sans être expliquée, et la correction saisonnière est proposée comme extension non exécutée | Lecture du code ; recalcul des biais médians par trimestre civil depuis `models_est.parquet` : M7 −2,4 / +1,6 ; M0 −1,6 / +2,4 ; M10 −1,2 / +2,8 ; M11b +1,4 / −0,4 (Q1 / Q3). | | |
| 15 | +| 4 | `results.tex` §5.4 (H4) | « distance and similarity weights help slightly in dense markets … neutral or slightly harmful in sparse ones. H4 is thus supported in direction » | Relecture conforme à la table H4 : médiane égale meilleure ou à 0,1 pt du meilleur dans les 3 strates denses ; pondération par ajustement brut utile vs moyenne égale en marché dense (10,5 vs 10,9) mais pas en marché rare ; distance neutre partout ; produit des poids + médiane = pire combinaison dans les 2 strates les plus rares. « The density profile does not support H4 in its stated form » | Table `h4.tex` inchangée (chiffres de `results.json`) ; le texte v1.0 contredisait la table. | | |
| 16 | +| 5 | `results.tex` §5.2 (H3) | « The non-linear GBM adjustments are the most accurate scheme in dense markets and lose their advantage in sparse ones » ; verdict H3 implicite | GBM meilleur dans toutes les strates, avantage ≈ 1 pt en marché dense et minimal en marché rare ; H3 explicitement rejetée dans sa première partie (le local ne bat pas la grille fixe même en marché dense) et confirmée dans la seconde | Table `h3.tex`. | | |
| 17 | +| 6 | `results.tex` §5.3 (H1) | Test direct de H1 sans le point à rayon constant | Ajout de « five nearest within ten kilometres 17.6% » (macro `\hOneGeoDten`), qui isole l'effet du critère de sélection à rayon égal (17,6 → 13,0) | `results.json › hypotheses.H1.geo_d10` (déjà calculé, non rapporté dans le papier). | | |
| 18 | +| 7 | `results.tex` §5.5 (Table `tab:pairs`) | IC bootstrap : 400 réplications (MAPE), **150** (MdAPE) ; pas de statistique de test | Nouvelle table `pairs_b999.tex` : **999** réplications pour MdAPE et MAPE, statistique *t* robuste aux grappes (type Diebold–Mariano, éq. 8), 4 paires ajoutées (M0−M13, M0−M9, M0−M7, M13−M11b), nombre de grappes | `07_revision_analyses.py` (section b) depuis `models_est.parquet` + `targets.parquet` ; `results/revision-2026-10-17/pairs_B999.csv`. Les IC v1.0 sont contenus dans les nouveaux à ±0,02 pt. | | |
| 19 | +| 8 | Nouveau §6.2 « Selection stability and data snooping » | Absent (le risque de « winner's curse » sur 2 256 configurations n'était pas traité) | Corrélation de rang validation/test 0,996 (0,994 parmi les 1 370 éligibles) ; M0 2ᵉ/1 370 en test, regret 0,00 pt vs la configuration ex post optimale (compromis, k=10, 10 km, 24 mois, ajustements GBM : 11,13 %) ; 10 meilleures de 2024 toutes dans les 12 meilleures de 2025-26 ; 37 configurations à ≤ 0,25 pt du meilleur en validation → test entre 11,13 et 11,39 % | `surface_E1.csv`, `facteurs_E2.csv` ; tables `selection_top10.tex`, `selection_near.tex`. | | |
| 20 | +| 9 | Nouveau §6.4 (R8) | « A sensitivity analysis to the bounds of R8 and to the flip rule R9 is left for the next version » | Sensibilité côté cibles exécutée : bornes [0,40 ; 2,5], [0,50 ; 2], [0,67 ; 1,5] (96,1 % des cibles conservées pour [0,5 ; 2] ; M0 10,6 %, M11b 8,5 % ; ordre et écarts inchangés). Le canal « bassin de comparables » reste **non exécuté** (nécessite le cluster) et est dit tel | Table `r8_sensitivity.tex`. | | |
| 21 | +| 10 | Nouveau §6.5 | Une seule définition de grappes | Sensibilité des IC à 4 définitions de grappes (municipalité×trimestre, MRC×trimestre, municipalité, région×trimestre, 399 rép.) pour 4 paires clés ; conclusions inchangées | Table `pairs_cluster_sens.tex`. | | |
| 22 | +| 11 | Annexe, nouvelle table `tab:competition_ci` | Pas d'IC sur les niveaux | MdAPE et MAPE de chacun des 16 modèles avec IC bootstrap 95 % (999 rép.) | `competition_ci.tex` ; M0 11,13 [10,84 ; 11,45], M11b 8,97 [8,77 ; 9,20]. | | |
| 23 | +| 12 | `methodology.tex` | Définitions verbales de l'APE/MdAPE/biais/couverture, de la grille conventionnelle, des pondérations, de la grille locale, de l'inférence | Équations numérotées : dissimilarité (3), grille conventionnelle (4), grille locale ridge (5), réconciliation (6), APE et erreur signée (7), MdAPE/MAPE/biais (8), statistique *t* en grappes (9) ; scores de sélection explicités (`sim` = D + d/25, `mix` = D + d/2, `adj` = ajustement brut + d/25, lus dans `04_backtest.py`) ; précision que les ajustements GBM (schéma g) utilisent le modèle **sans** valeur au rôle ; indice : somme sur s ≤ m−2, base m−1, régions < 200 ventes → indice provincial | Code `02_index.py`, `04_backtest.py`. | | |
| 24 | +| 13 | `methodology.tex`, `results.tex` | « Thirteen named models » | « Sixteen named models » (M0, M1–M8a/b, M9, M10, M11a/b, M12, M13 = 16 lignes de la table) | Table `competition.tex`. | | |
| 25 | +| 14 | `introduction.tex` | « to our knowledge, the largest and most systematic empirical validation … to date » ; « measuring, for the first time, the price of interpretability » ; « In homogeneous submarkets the gap narrows to about a point » | Formulations sobres (« population-scale, out-of-sample validation ») ; « about a point and a half » (écart H5 homogène = 1,4 pt) | Protocole (pas de superlatifs) ; `results.json › H5`. | | |
| 26 | +| 15 | `paper/tables/strata_other.tex` | Lignes d'âge dans le désordre (« 16–40, 4–15, 41–70, >70, ≤3 ») ; lien physique étiqueté « 0.0, 1.0, 2.0, 3.0 » | Ordre croissant ; libellés « detached / semi-detached / row / integrated » | Bogue de remplacement textuel dans `make_paper_tables.py` (clés `"0"` vs valeurs `0.0`) ; fonction `strata()` dotée d'un paramètre `labels`. | | |
| 27 | +| 16 | `tables/weighting.tex`, `marginal.tex`, `h4.tex` | Pondération « most similar » | « 1/gross adjustment » (le poids est 1/(0,1 + ajustement hédonique brut), cf. `04_backtest.py`) ; lignes de `marginal.tex` dans l'ordre des facteurs | Lecture du code. | | |
| 28 | +| 17 | `tables/h2.tex` | Un seul panneau (grille hédonique) | Deux panneaux (grille hédonique / temps seul) : la première moitié de H2 (seuil sans ajustement, croissant avec la rareté) est désormais lisible dans le papier | `results.json › hypotheses.H2` (« none » déjà calculé). | | |
| 29 | +| 18 | `data.tex` | Délai de publication au registre non mentionné ; millésime du rôle non défini | Phrase sur la datation par enregistrement (délai ignoré, favorise toutes les méthodes de même) ; définition du « roll vintage » ; note explicative sous la table des descriptives ; ratio médian prix/rôle 1,39 → 1,20 relié à la restriction « même rôle » du ratio | `PROTOCOLE.md` §2 ; `01_dataset.py` (`role_vintage`). | | |
| 30 | +| 19 | `results.tex` §5.5 | Écarts dans le texte (2,2 pts) vs niveaux arrondis (11,1 − 9,0 = 2,1) sans explication | Phrase liminaire : les écarts sont des différences appariées calculées avant arrondi | — | | |
| 31 | +| 20 | `results.tex` §5.6 | — | Mise en garde sur l'effet dénominateur du quintile de prix le plus bas (APE mécaniquement gonflée) | Raisonnement ; table `strata_other.tex`. | | |
| 32 | +| 21 | `conclusion.tex` | « Three limitations » ; « as accurate as an opaque model in most of the market » | « Four limitations » (ajout : filtre R8, datation, délai) ; « nearly as accurate » ; deux extensions explicitement **non exécutées** (correction saisonnière, re-backtest du bassin) | — | | |
| 33 | +| 22 | `paper/tables/regression.tex`, `summary_stats.tex` | Gabarits finance (« Firm FE », « Book-to-Market ») non référencés | Supprimés | Fichiers parasites du gabarit. | | |
| 34 | +| 23 | `docs/PROTOCOLE.md` §2 | « modèles : estimés sur les ventes < 2024-01 pour la validation 2024 et < 2025-01 pour le test » | Ré-estimation trimestrielle 2024-01 → 2026-07 (ce que fait `03_models.py`) | Incohérence documentation / code. | | |
| 35 | +| 24 | `main.tex` | Version 1.0 ; résumé sans mention de l'inférence ni de la stabilité de la sélection (≈ 370 mots) | Version 1.1 ; résumé réécrit (≈ 300 mots) ; `\input{tables/numbers_revision}` ; `main_anon.tex` régénéré à l'identique (page titre anonyme, `pdfauthor={}`) | — | | |
| 36 | +| 25 | `f3_ajustements.pdf` | Étiquettes « GBM » / « assessment ratio » se chevauchant | « assess. ratio » | Inspection visuelle. | | |
| 37 | + | |
| 38 | +Aucun nombre issu du pipeline n'a changé : `tables/numbers.tex` régénéré est identique à la version v1.0 (diff vide, puis 16 macros | |
| 39 | +ajoutées : `\hOneGeoDten`, poids de M9/M12, `\densLowN`, `\ivCovLo/Hi`, `\ivHalf`, `\gridRtwo`, `\gridN`, `\hFourSimDense`, `\hFourEqualMeanDense`). | |
| 40 | + | |
| 41 | +## 2. Références | |
| 42 | + | |
| 43 | +**Avant** : 28 entrées, aucune avec DOI. **Après** : 64 entrées, toutes vérifiées le 2026-10-17 (Crossref `api.crossref.org/works?query.bibliographic=…` | |
| 44 | +via WebFetch, sauf mention), DOI inscrit quand il existe. | |
| 45 | + | |
| 46 | +Corrections d'entrées existantes : `bourassa2010` pages 139–159 → 139–160 (Crossref) ; `calem2021` ajouté avec le 4ᵉ auteur (Kenney) ; `cameron2008` | |
| 47 | +DOI 10.1162/rest.90.3.414 vérifié par `works/{DOI}` ; `case1989` (AER 1989) sans DOI (antérieur à l'attribution AER, vérifié via NBER w2506 et la | |
| 48 | +citation RePEc) ; standards IAAO 2017/2018 vérifiés sur iaao.org (WebSearch : `Standard_on_Automated_Valuation_Models.pdf`, approuvé juillet 2018 ; | |
| 49 | +`StandardOnMassAppraisal.pdf`, approuvé juillet 2017) ; `pace2020` conservé à l'année de parution imprimée (vol. 60, 2020 ; Crossref « issued 2019 ») ; | |
| 50 | +`hill2013` idem (vol. 27(5), 2013 ; Crossref 2012). | |
| 51 | + | |
| 52 | +Références ajoutées (36), toutes citées dans le texte : | |
| 53 | + | |
| 54 | +| Clé | Référence | DOI | Rôle dans l'argument | | |
| 55 | +|---|---|---|---| | |
| 56 | +| green1994 | Green, RE Econ 22(4) | 10.1111/1540-6229.00653 | débat Vandell–Gau et al. sur la pondération optimale | | |
| 57 | +| epley1997 | Epley, JRER 14(2) | 10.1080/10835547.1997.12090895 | praticabilité des règles optimales | | |
| 58 | +| pace1996 | Pace, JREFE 13(3) | 10.1007/BF00217391 (pages 203–218 via Springer/ACNP) | grille vs plus proche voisin vs MCO (Monte Carlo) — antécédent direct | | |
| 59 | +| pacegilley1998 | Pace & Gilley, RE Econ 26(2) | 10.1111/1540-6229.00748 | généralisation grille/MCO | | |
| 60 | +| lipscomb1990 | Lipscomb & Gray, JRER 5(1) | 10.1080/10835547.1990.12090602 | quatre méthodes d'ajustement dérivées du marché | | |
| 61 | +| cannaday1986 | Cannaday & Sunderman, RE Econ 14(2) | 10.1111/1540-6229.00386 | estimation de la dépréciation (≈ 1 %/an) | | |
| 62 | +| isakson1988 | Isakson, Growth and Change 19(2) | 10.1111/j.1468-2257.1988.tb00466.x | plus proches voisins | | |
| 63 | +| lai2008 | Lai, Vandell, Wang & Welke, JRER 30(4) | 10.1080/10835547.2008.12091226 | estimateur par réplication | | |
| 64 | +| quan1991 | Quan & Quigley, JREFE 4(2) | 10.1007/BF00173120 (pages 127–146 via RePEc) | évaluateur comme extracteur de signal bayésien | | |
| 65 | +| shiller1999 | Shiller & Weiss, JREFE 18(2) | 10.1023/A:1007756607862 | évaluation hors échantillon des systèmes d'évaluation | | |
| 66 | +| clapp1992 | Clapp & Giaccotto, JASA 87(418) | 10.1080/01621459.1992.10475209 | valeur au rôle comme information | | |
| 67 | +| thibodeau2003 | Thibodeau, RE Econ 31(1) | 10.1111/j.1080-8620.2003.00055.x | « marking assessed values to market » | | |
| 68 | +| clapham2006 | Clapham, Englund, Quigley & Redfearn, RE Econ 34(2) | 10.1111/j.1540-6229.2006.00167.x | révision des indices (point-in-time) | | |
| 69 | +| hillscholz2018 | Hill & Scholz, Rev. Income & Wealth 64(4) | 10.1111/roiw.12303 | imputation hédonique géospatiale | | |
| 70 | +| ngai2014 | Ngai & Tenreyro, AER 104(12) | 10.1257/aer.104.12.3991 | saisonnalité des marchés du logement | | |
| 71 | +| desrosiers2000 | Des Rosiers, Thériault & Villeneuve, JPIF 18(3) | 10.1108/14635780010338245 | hédonique québécois, accessibilité/voisinage | | |
| 72 | +| bourassa2003 | Bourassa, Hoesli & Peng, JHE 12(1) | 10.1016/S1051-1377(03)00003-2 | sous-marchés et prédiction | | |
| 73 | +| goodman2003 | Goodman & Thibodeau, JHE 12(3) | 10.1016/S1051-1377(03)00031-7 | segmentation et précision prédictive | | |
| 74 | +| bourassa2007 | Bourassa, Cantoni & Hoesli, JREFE 35(2) | 10.1007/s11146-007-9036-8 | dépendance spatiale, sous-marchés | | |
| 75 | +| dube2013 | Dubé & Legros, Papers in Reg. Sci. 92(1) | 10.1111/j.1435-5957.2011.00402.x | dépendance spatio-temporelle | | |
| 76 | +| dube2014 | Dubé & Legros, JPR 31(4) | 10.1080/09599916.2014.913655 | dimension temporelle de l'hédonique spatial | | |
| 77 | +| zurada2011 | Zurada, Levitan & Guan, JRER 33(3) | 10.1080/10835547.2011.12091311 | régression vs IA en évaluation de masse | | |
| 78 | +| kok2017 | Kok, Koponen & Martínez-Barbosa, JPM 43(6) | 10.3905/jpm.2017.43.6.202 | AVM et mégadonnées | | |
| 79 | +| bogin2020 | Bogin & Shui, JREFE 60(1–2) | 10.1007/s11146-019-09712-0 | précision des AVM et des évaluateurs en zone rurale | | |
| 80 | +| krause2020 | Krause, Martin & Fix, JPR 37(4) | 10.1080/09599916.2020.1807587 | incertitude des AVM (intervalles) | | |
| 81 | +| steurer2021 | Steurer, Hill & Pfeifer, JPR 38(2) | 10.1080/09599916.2020.1858937 | métriques d'évaluation des AVM | | |
| 82 | +| lorenz2023 | Lorenz, Willwersch, Cajias & Fuerst, RE Econ 51(5) | 10.1111/1540-6229.12397 | apprentissage automatique interprétable | | |
| 83 | +| eriksen2019 | Eriksen, Fout, Palim & Rosenblatt, JUE 111 | 10.1016/j.jue.2019.04.007 | biais d'évaluation vers le prix du contrat | | |
| 84 | +| calem2021 | Calem, Kenney, Lambie-Hanson & Nakamura, RE Econ 49(S1) | 10.1111/1540-6229.12326 | idem | | |
| 85 | +| diebold1995 | Diebold & Mariano, JBES 13(3) | 10.1080/07350015.1995.10524599 | comparaison appariée de pertes | | |
| 86 | +| giacomini2006 | Giacomini & White, Econometrica 74(6) | 10.1111/j.1468-0262.2006.00718.x | idem (conditionnel) | | |
| 87 | +| white2000 | White, Econometrica 68(5) | 10.1111/1468-0262.00152 | data snooping | | |
| 88 | +| hansen2005 | Hansen, JBES 23(4) | 10.1198/073500105000000063 | idem (SPA) | | |
| 89 | +| gneiting2011 | Gneiting, JASA 106(494) | 10.1198/jasa.2011.r10138 | médiane = prévision optimale sous perte absolue | | |
| 90 | +| hyndman2006 | Hyndman & Koehler, IJF 22(4) | 10.1016/j.ijforecast.2006.03.001 | mesures d'erreur en pourcentage | | |
| 91 | +| lei2018 | Lei, G'Sell, Rinaldo, Tibshirani & Wasserman, JASA 113(523) | 10.1080/01621459.2017.1307116 | calibration conformelle des intervalles | | |
| 92 | + | |
| 93 | +Références existantes vérifiées (DOI ajoutés) : colwell1983, vandell1991, gau1992, gau1994, isakson1986, isakson2002, lentz1998, kang1991, pace1998, | |
| 94 | +pace1997, rosen1974, bourassa2010, case2004, bailey1963, hill2013, pagourtzi2003, mccluskey2013, mayer2019, pace2020, diaz1990, diaz2001, | |
| 95 | +northcraft1987, cameron2008 ; sans DOI : case1989 (AER 1989), appraisal2020, iaao2017, iaao2018, gloudemans1999 (ouvrages/standards). | |
| 96 | +Aucune référence retirée. | |
| 97 | + | |
| 98 | +## 3. Réanalyses exécutées (poste de travail, sans accès à la base ni au cluster) | |
| 99 | + | |
| 100 | +| Script | Entrées | Durée | Sorties | | |
| 101 | +|---|---|---|---| | |
| 102 | +| `pipeline/07_revision_analyses.py` (nouveau, `python3 -I`) | `out/2026-10-10/{targets.parquet, models_est.parquet, index_full.csv, tables/surface_E1.csv, tables/facteurs_E2.csv}` | ≈ 75 s | `results/revision-2026-10-17/{revision_results.json, selection_top10.csv, pairs_B999.csv, model_ci.csv, r8_sensitivity.csv, seasonal_bias.csv, index_growth.csv}` ; `paper/tables/{selection_top10, selection_near, pairs_b999, pairs_cluster_sens, competition_ci, r8_sensitivity, seasonal_bias, index_growth}.tex` ; `paper/tables/numbers_revision.tex` (37 macros) | | |
| 103 | +| `paper/make_paper_tables.py` (corrigé) | `out/2026-10-10/` | ≈ 10 s | tables et figures régénérées ; `numbers.tex` inchangé pour les 97 macros d'origine | | |
| 104 | + | |
| 105 | +Résultats clés des réanalyses : Spearman val/test 0,996 ; M0 rang 2/1 370 en test ; regret 0,00 pt ; IC M7−M2 [−5,26 ; −4,88] (t = −58) ; | |
| 106 | +M11b−M0 [−2,30 ; −2,04] (t = −36) ; M12−M11b [−0,09 ; −0,03] (t = −8,3) ; IC robustes aux 4 définitions de grappes ; R8 serré à [0,5 ; 2] : | |
| 107 | +toutes les MdAPE baissent d'≈ 0,5 pt, ordre inchangé ; biais saisonnier Q1/Q3 : M7 −2,4/+1,6, M11b +1,4/−0,4. | |
| 108 | + | |
| 109 | +## 4. Éléments non modifiés faute de données / de calcul | |
| 110 | + | |
| 111 | +- Re-backtest avec d'autres bornes R8 ou R9 appliquées au **bassin** de comparables (nécessite `data/vraiprix-20261010.db` et le cluster). | |
| 112 | +- Correction saisonnière des estimations par comparables (nécessite les estimations par configuration `est_E1/E2.npy`, absentes du poste ; les seules | |
| 113 | + estimations disponibles localement sont celles des 16 modèles nommés). | |
| 114 | +- Terciles d'hétérogénéité (H5) : les dispersions `sd_E2` sont présentes en fragments mais l'analyse n'a pas été refaite (résultats v1.0 conservés tels quels). | |
| 115 | +- `docs/RAPPORT.md/.pdf` (rapport français généré par `06_report.py` le 2026-10-10) : non régénéré (le script tourne sur le nœud) ; ses chiffres restent | |
| 116 | + cohérents avec le papier, mais il ne contient pas les nouvelles analyses. | |
| 117 | +- Les résultats de 2 256 configurations ont été vérifiés pour cohérence (macros `numbers.tex` identiques), non recalculés. | |
modified
docs/PROTOCOLE.md
+1 −1
@@ -24,7 +24,7 @@ La méthode des comparables, telle qu'enseignée et pratiquée par les évaluate | ||
| 24 | 24 | Pour chaque transaction cible : |
| 25 | 25 | |
| 26 | 26 | 1. **Masquage** du prix de vente réel. |
| 27 | −2. **Reconstitution du marché à la date d'évaluation** : seules les ventes conclues *strictement avant* la date de la cible sont admissibles (le bien lui-même, s'il a déjà été vendu, est exclu du bassin : on évalue par comparables, pas par vente répétée). L'indice temporel, les coefficients hédoniques et les modèles d'apprentissage sont ré-estimés avec les seules ventes antérieures à la période d'évaluation (indice : ré-estimé chaque mois avec toutes les ventes des mois précédents ; modèles : estimés sur les ventes < 2024-01 pour la validation 2024 et < 2025-01 pour le test 2025-26). | |
| 27 | +2. **Reconstitution du marché à la date d'évaluation** : seules les ventes conclues *strictement avant* la date de la cible sont admissibles (le bien lui-même, s'il a déjà été vendu, est exclu du bassin : on évalue par comparables, pas par vente répétée). L'indice temporel, les coefficients hédoniques et les modèles d'apprentissage sont ré-estimés avec les seules ventes antérieures à la période d'évaluation (indice : ré-estimé chaque mois avec toutes les ventes des mois précédents ; grilles d'ajustement et modèles : ré-estimés au début de chaque trimestre, de 2024-01 à 2026-07, avec les seules ventes antérieures au trimestre — protocole effectivement implémenté dans `03_models.py` ; la version initiale de ce document indiquait deux estimations seulement, corrigé le 2026-10-17). | |
| 28 | 28 | 3. **Application** de chaque configuration (sélection × rayon × fenêtre × nombre × ajustement × pondération × réconciliation) sur exactement les mêmes cibles. |
| 29 | 29 | 4. **Comparaison** avec le prix réel : erreur absolue en pourcentage, biais, part des estimations à ±5/10/20 %, couverture (part des cibles pour lesquelles la configuration produit une valeur), dispersion des prix ajustés (intervalle de valeur). |
| 30 | 30 | |
added
docs/REVIEW-REPORT.md
+115 −0
@@ -0,0 +1,115 @@ | ||
| 1 | +# Rapport de l'« editorial board » — WP 16, *Does the Sales Comparison Approach Actually Work?* | |
| 2 | + | |
| 3 | +Révision scientifique complète du 2026-10-17 (master prompt `Recherche-UQO/REVISION-MASTER-PROMPT.md`, protocole | |
| 4 | +`REVISION-PROTOCOLE.md`). Version examinée : 1.0 (2026-10-10, 38 p., SSRN 7626720). Version produite : 1.1 (52 p.). | |
| 5 | +Journal d'audit détaillé : `CHANGELOG-REVISION.md`. Revue cible : *Journal of Real Estate Finance and Economics* (repli *JRER*). | |
| 6 | + | |
| 7 | +## 1. Diagnostic général | |
| 8 | + | |
| 9 | +Le papier pose une question nette et peu traitée — la précision prédictive des **règles** de la méthode des comparables — et y répond | |
| 10 | +avec un dispositif rare dans cette littérature : population exhaustive des ventes unifamiliales d'une province, rétro-évaluation | |
| 11 | +« point dans le temps » (indice mensuel et grilles trimestrielles ré-estimés sur les seules ventes antérieures), 2 256 configurations | |
| 12 | +et 16 modèles appliqués aux **mêmes** 189 354 cibles, séparation stricte entre année de sélection (2024) et années d'évaluation | |
| 13 | +(2025-26), inférence par bootstrap en grappes. La contribution originale est réelle et doit être préservée : (i) hiérarchie | |
| 14 | +ajustement > sélection > nombre/rayon > pondération, (ii) k optimal ≈ 20-30 avec grille, (iii) coût de la transparence 2,2 pts, | |
| 15 | +(iv) auto-adaptation de la règle « filet large » à la densité, (v) intervalles calibrés larges (±33 %). | |
| 16 | + | |
| 17 | +Les faiblesses de la v1.0 étaient d'ordre (a) éditorial et de cohérence (une affirmation chiffrée fausse sur les indices régionaux, | |
| 18 | +une figure ne montrant pas ce que dit sa légende, un paragraphe H4 contredit par sa table, un mécanisme attribué au GBM que le code | |
| 19 | +n'implémente pas, tables aux libellés bruts), (b) méthodologique sur l'inférence (IC de MdAPE à 150 réplications, pas de statistique | |
| 20 | +de test, pas de traitement du risque de sélection parmi 2 256 configurations, pas d'IC sur les niveaux), (c) formel (définitions | |
| 21 | +verbales là où des équations étaient attendues : erreurs, grille conventionnelle, scores de sélection, ridge local, bootstrap), | |
| 22 | +(d) bibliographique (28 références, aucun DOI, antécédents directs manquants : Pace 1996, Lai et al. 2008, Quan–Quigley 1991, | |
| 23 | +Shiller–Weiss 1999, Bogin–Shui 2020, Steurer et al. 2021, Krause et al. 2020, littérature d'évaluation des prévisions). Tous ces | |
| 24 | +points ont été traités ; les chiffres du pipeline n'ont pas changé. | |
| 25 | + | |
| 26 | +## 2. Diagnostic par section | |
| 27 | + | |
| 28 | +| Section | Forces | Faiblesses trouvées (v1.0) | Traitement (v1.1) | | |
| 29 | +|---|---|---|---| | |
| 30 | +| Résumé / Introduction | Question claire, cinq résultats bien hiérarchisés, périmètre honnête (« règles simulées ») | Superlatifs (« largest … to date », « for the first time ») ; « about a point » pour un écart de 1,4 ; aucune mention de l'inférence ni du risque de sélection ; « thirteen named models » (il y en a 16) | Réécrits ; résumé ramené à ≈ 300 mots avec inférence et stabilité de la sélection | | |
| 31 | +| Littérature | Bonne structure (conventions / ajustements / AVM / comportement) | Antécédents directs manquants ; aucune référence sur l'évaluation des prévisions, la révision des indices, la saisonnalité, l'incertitude des AVM ; Québec absent | 36 références ajoutées et intégrées à l'argument (voir §4) | | |
| 32 | +| Données | Nettoyage journalisé exemplaire ; densité définie à la date d'évaluation | Affirmation fausse sur Montréal vs Québec vs Montérégie ; Fig. 1 monocourbe ; délai d'enregistrement et millésime du rôle non définis ; chute du ratio prix/rôle non commentée | Corrigés ; table A1 des croissances régionales ; notes | | |
| 33 | +| Méthodologie | Protocole point-in-time rigoureux et conforme au code ; couverture rapportée | Pas d'équations pour l'APE/MdAPE/biais, la grille conventionnelle, les scores de sélection (poids d/25 et d/2 non dits), le ridge local, le bootstrap ; ambiguïté sur le GBM utilisé pour les ajustements ; somme de l'indice écrite s < m−1 | Sept équations ajoutées/numérotées ; précisions lues dans `04_backtest.py` ; sous-sections « Accuracy measures » et « Inference » | | |
| 34 | +| Résultats | Tables générées, macros, cohérence chiffres/texte très bonne | H4 contredit par la table ; H3 sans verdict ; test H1 sans le point à rayon constant ; IC à 150 rép. sans statistique de test ; écarts (2,2) vs niveaux arrondis (2,1) non expliqués ; effet dénominateur du Q1 non signalé ; libellé « most similar » pour un poids 1/ajustement | Tous corrigés ; table des paires à 999 rép. avec *t* ; poids M9/M12 rapportés | | |
| 35 | +| Robustesse | Stabilité trimestrielle, couverture, mesures alternatives, spécification des benchmarks | Mécanisme saisonnier du GBM non fondé ; sensibilité R8 « laissée pour la prochaine version » ; aucune analyse du risque de sélection ; une seule définition de grappes | Table saisonnière, §6.2 (stabilité de la sélection), §6.4 (R8 côté cibles), §6.5 (grappes) | | |
| 36 | +| Conclusion | Recommandations enseignables, limites honnêtes | Trois limites seulement ; « as accurate as » trop fort ; extensions non distinguées des résultats | Quatre limites, « nearly as accurate », deux extensions déclarées non exécutées | | |
| 37 | +| Annexes / LaTeX | Compilation propre | Deux gabarits finance parasites ; tables d'âge désordonnées ; codes « 0.0 » ; équation hors marge | Supprimés / corrigés ; `make` propre (0 erreur, 0 référence indéfinie, 1 overfull de 1,7 pt dans une légende) | | |
| 38 | + | |
| 39 | +## 3. Méthodologie — verdict motivé sur chaque point du §3 du master prompt | |
| 40 | + | |
| 41 | +| Point | Verdict | Motivation | | |
| 42 | +|---|---|---| | |
| 43 | +| Spécification justifiée | **Oui** | Le plan factoriel E2 et la surface E1 couvrent les règles débattues ; les ancres et grilles sont argumentées ; la grille conventionnelle est désormais écrite (éq. 4). | | |
| 44 | +| Hypothèses d'estimation explicites | **Oui, après révision** | Indice within-municipalité (éq. 2), grille hédonique within, ridge local (éq. 5), GBM L1 ; le choix L1 est relié à Gneiting (2011). | | |
| 45 | +| Variables correctement définies | **Oui, après révision** | APE/erreur signée/MdAPE/MAPE/biais/couverture (éq. 7-8), dissimilarité (éq. 3), millésime du rôle, densité. | | |
| 46 | +| Effets fixes appropriés | **Oui** | Effets de municipalité dans l'indice et la grille ; effets région/MRC/municipalité/unité de voisinage rétrécis dans M10. | | |
| 47 | +| Dépendance spatiale/temporelle défendable | **Oui** | Grappes municipalité × trimestre (6 346 grappes) ; sensibilité à MRC × trimestre, municipalité seule, région × trimestre : IC élargis mais conclusions inchangées (table 25). | | |
| 48 | +| Erreurs types correctement spécifiées | **Oui, après révision** | 999 réplications pour MdAPE et MAPE (150/400 en v1.0), statistique *t* robuste aux grappes (éq. 9), IC sur les niveaux (table A6). | | |
| 49 | +| Construction de l'échantillon transparente | **Oui** | 13 règles journalisées ; sensibilité aux bornes de R8 côté cibles exécutée (table 24) ; canal « bassin » déclaré non exécuté. | | |
| 50 | +| Robustesse face aux vraies menaces | **Oui, avec une réserve** | Menaces traitées : fuite d'information (point-in-time + sélection 2024/évaluation 2025-26), winner's curse (§6.2 : rang 2/1 370, Spearman 0,996), couverture, grappes, R8, saisonnalité. Réserve : le re-backtest du bassin et la correction saisonnière exigent le cluster. | | |
| 51 | +| Interprétation des coefficients | **Oui** | Élasticités et dépréciation lues correctement ; R² within = 0,36 commenté ; effet dénominateur du Q1 signalé. | | |
| 52 | +| Significativité statistique vs économique | **Oui, après révision** | M12 − M11b : −0,06 pt significatif (t = −8,3) mais « économiquement négligeable » ; M7 − M6 petit mais significatif. | | |
| 53 | +| Allégations causales | **Aucune** | Le papier reste strictement prédictif ; la phrase sur la saisonnalité du GBM a été remplacée par un constat et une hypothèse étiquetée comme telle. | | |
| 54 | + | |
| 55 | +Point examiné en priorité — **fuite d'information entre sélection (2024) et évaluation (2025-26)** : aucune fuite détectée. Les cibles | |
| 56 | +de 2024 ne servent qu'à choisir configurations, poids NNLS (M9, M12), recentrage médian de M10 et z des intervalles ; les modèles | |
| 57 | +trimestriels utilisés pour une cible de 2025 sont entraînés sur les ventes antérieures au trimestre (ce qui inclut légitimement | |
| 58 | +2024 comme données, non comme cibles). Le GBM ne voit pas le mois de la cible (variable mois plafonnée au dernier mois | |
| 59 | +d'entraînement) et son niveau est reporté par l'indice connu à la date d'évaluation, comme pour les comparables. | |
| 60 | + | |
| 61 | +## 4. Littérature | |
| 62 | + | |
| 63 | +- **Vérifiées** (28, toutes existantes ; DOI ajoutés à 23, 5 ouvrages/standards sans DOI) ; `bourassa2010` pages corrigées. | |
| 64 | +- **Ajoutées** (36, toutes vérifiées Crossref/éditeur, DOI dans le .bib, liste et rôles dans `CHANGELOG-REVISION.md` §2) : fondements | |
| 65 | + (Green 1994 ; Epley 1997 ; Pace 1996 ; Pace & Gilley 1998 ; Lipscomb & Gray 1990 ; Cannaday & Sunderman 1986 ; Isakson 1988 ; Lai et al. 2008 ; | |
| 66 | + Quan & Quigley 1991), indices et rôle (Clapp & Giaccotto 1992 ; Thibodeau 2003 ; Clapham et al. 2006 ; Hill & Scholz 2018 ; Ngai & Tenreyro 2014), | |
| 67 | + sous-marchés et dépendance (Des Rosiers et al. 2000 ; Bourassa et al. 2003, 2007 ; Goodman & Thibodeau 2003 ; Dubé & Legros 2013, 2014), | |
| 68 | + AVM (Shiller & Weiss 1999 ; Zurada et al. 2011 ; Kok et al. 2017 ; Bogin & Shui 2020 ; Krause et al. 2020 ; Steurer et al. 2021 ; Lorenz et al. 2023), | |
| 69 | + comportement (Eriksen et al. 2019 ; Calem et al. 2021), inférence (Diebold & Mariano 1995 ; Giacomini & White 2006 ; White 2000 ; Hansen 2005 ; | |
| 70 | + Gneiting 2011 ; Hyndman & Koehler 2006 ; Lei et al. 2018). | |
| 71 | +- **Retirées** : aucune. | |
| 72 | +- Références citées dans la demande et non ajoutées : « Colwell–Cannaday–Wu » et « Gau–Lai–Wang » étaient déjà présentes ; aucune autre | |
| 73 | + référence suggérée n'a été écartée. | |
| 74 | + | |
| 75 | +## 5. Résultats — incohérences trouvées | |
| 76 | + | |
| 77 | +1. Indices régionaux 2025-26 (texte v1.0 : Montréal ≈ ⅓ de Québec et ½ de Montérégie ; réel : +3,6 % vs +25,4 % et +13,9 %). **Corrigé.** | |
| 78 | +2. Figure 1 : une seule courbe au lieu de cinq régions (bogue de code). **Corrigé.** | |
| 79 | +3. H4 : texte contredit par la table (pondération par distance « utile en marché dense » alors qu'elle est neutre ; médiane égale meilleure). **Corrigé.** | |
| 80 | +4. Saisonnalité attribuée à la variable mois du GBM alors que celle-ci est fixée à la prédiction. **Corrigé.** | |
| 81 | +5. « Thirteen named models » pour 16 lignes. **Corrigé.** | |
| 82 | +6. Table des strates : âges désordonnés, codes de lien physique bruts ; pondération « most similar » = 1/ajustement brut. **Corrigé.** | |
| 83 | +7. Écart 2,2 (apparié, avant arrondi) vs 11,1 − 9,0 = 2,1 : expliqué en tête de la section Résultats. | |
| 84 | +8. Couverture des intervalles nominaux « 67–78 % » : valeurs réelles 66,5–78,5 → macro `\ivCovLo`–`\ivCovHi` (66–78). | |
| 85 | +9. `docs/PROTOCOLE.md` : « modèles estimés < 2024-01 et < 2025-01 » vs ré-estimation trimestrielle effective. **Corrigé.** | |
| 86 | +Aucune incohérence entre `results.json`, les tables et `docs/RAPPORT.md` n'a été trouvée (toutes les macros `numbers.tex` régénérées sont identiques). | |
| 87 | + | |
| 88 | +## 6. Limites restantes | |
| 89 | + | |
| 90 | +- Règles simulées, non dossiers d'évaluateurs ; caractéristiques du rôle seulement (R² within 0,36) ; une province, un type de bien. | |
| 91 | +- Filtre R8 appliqué aussi au **bassin** : sa sensibilité n'a été mesurée que côté cibles. | |
| 92 | +- Biais saisonnier des comparables (−2,4 / +1,6 pt en Q1/Q3) documenté, non corrigé ; l'absence de saisonnalité des GBM n'est pas expliquée. | |
| 93 | +- IC de MdAPE par bootstrap percentile sans correction (les grappes sont nombreuses : 6 346 ; l'ordre de grandeur est robuste aux définitions). | |
| 94 | +- Coût de la transparence mesuré contre un GBM d'une seule famille (LightGBM) ; d'autres apprenants (forêts, réseaux, modèles spatiaux) ne sont pas testés. | |
| 95 | +- Délai de publication au registre ignoré ; sélection des cibles sur la réalisation d'une vente (pas d'inférence sur les biens non vendus). | |
| 96 | +- 52 pages avec interligne 1,5 : la revue demandera sans doute de reléguer une partie des tables en annexe en ligne. | |
| 97 | + | |
| 98 | +## 7. Analyses recommandées non exécutées (toutes nécessitent le pipeline distribué ou les estimations par configuration) | |
| 99 | + | |
| 100 | +1. Re-backtest avec bornes R8 ∈ {[0,4 ; 2,5], [0,5 ; 2]} et sans R9 appliquées au bassin de comparables. | |
| 101 | +2. Correction saisonnière des estimations par comparables (facteur mensuel estimé sur les années antérieures), puis réévaluation des biais et MdAPE. | |
| 102 | +3. Test SPA de Hansen (2005) / « reality check » de White (2000) formel sur les 1 370 configurations éligibles (les statistiques de §6.2 en tiennent lieu de façon descriptive). | |
| 103 | +4. Benchmarks ML supplémentaires (forêt aléatoire, modèle spatial type Bourassa et al. 2010) avec le même jeu d'information. | |
| 104 | +5. Intervalles conformels par strate de densité (couverture conditionnelle), à partir de `sd_E2`. | |
| 105 | +6. Extension aux copropriétés et plex (données Vrai Prix disponibles sur le nœud). | |
| 106 | +7. Analyse du canal « composition du bassin » pour expliquer la saisonnalité (part des comparables printemps/été par trimestre de la cible). | |
| 107 | + | |
| 108 | +## 8. Maturité pour publication | |
| 109 | + | |
| 110 | +**Verdict : prêt pour soumission à JREFE après ajustement de la longueur** (relégation d'une partie des tables en annexe en ligne si demandé). | |
| 111 | +Points forts : question, échelle, protocole point-in-time, séparation sélection/évaluation désormais auditée, inférence complète, résultats | |
| 112 | +enseignables. Risques de rejet : (i) « un seul marché, un seul type de bien » (externe) ; (ii) absence des caractéristiques de listing | |
| 113 | +(garage, piscine, état) que la méthode professionnelle ajuste — un rapporteur pourra estimer que la version « simulée » sous-estime la | |
| 114 | +méthode réelle, ce que le papier reconnaît ; (iii) le benchmark ML unique ; (iv) longueur. Pour *JRER* (repli), le papier est au-dessus | |
| 115 | +des standards habituels de la revue et le risque principal est la longueur. | |
modified
paper/appendix/appendix.tex
+35 −4
@@ -4,11 +4,25 @@ | ||
| 4 | 4 | \label{app:tables} |
| 5 | 5 | % ============================================================================ |
| 6 | 6 | |
| 7 | +\begin{table}[H] | |
| 8 | +\centering | |
| 9 | +\caption{Hedonic price index (descriptive, full sample): cumulative growth by region (\%)} | |
| 10 | +\label{tab:index_growth} | |
| 11 | +\footnotesize | |
| 12 | +\resizebox{\textwidth}{!}{\input{tables/index_growth}} | |
| 13 | +\begin{tablenotes}\footnotesize | |
| 14 | +\item Note: monthly effects of a within-municipality hedonic regression estimated on the full sample (Figure~\ref{fig:index}); the peak is the highest month before 2023 and the trough the lowest month between the peak and December 2023. | |
| 15 | +\end{tablenotes} | |
| 16 | +\end{table} | |
| 17 | + | |
| 7 | 18 | \begin{table}[H] |
| 8 | 19 | \centering |
| 9 | 20 | \caption{Coverage of the error surface: share of test targets (\%) for which a value is returned, by radius and window} |
| 10 | 21 | \label{tab:surface_cov} |
| 11 | 22 | \input{tables/surface_cov} |
| 23 | +\begin{tablenotes}\footnotesize | |
| 24 | +\item Note: selection by distance, $k=5$, hedonic grid; coverage depends only on the radius and the window. | |
| 25 | +\end{tablenotes} | |
| 12 | 26 | \end{table} |
| 13 | 27 | |
| 14 | 28 | \begin{table}[H] |
@@ -33,6 +47,20 @@ | ||
| 33 | 47 | \label{tab:e1best} |
| 34 | 48 | \small |
| 35 | 49 | \input{tables/e1_best} |
| 50 | +\begin{tablenotes}\footnotesize | |
| 51 | +\item Note: best configuration among those with at least 97\% coverage on the validation targets. | |
| 52 | +\end{tablenotes} | |
| 53 | +\end{table} | |
| 54 | + | |
| 55 | +\begin{table}[H] | |
| 56 | +\centering | |
| 57 | +\caption{Horse race with cluster-bootstrap 95\% confidence intervals, 2025--2026} | |
| 58 | +\label{tab:competition_ci} | |
| 59 | +\footnotesize | |
| 60 | +\input{tables/competition_ci} | |
| 61 | +\begin{tablenotes}\footnotesize | |
| 62 | +\item Note: MdAPE and MAPE in per cent on the test targets covered by each model; intervals from 999 cluster-bootstrap replications over municipality--quarter cells. Intervals of different models overlap even where their paired difference is significant, because the paired comparison removes the target-specific component of the error. | |
| 63 | +\end{tablenotes} | |
| 36 | 64 | \end{table} |
| 37 | 65 | |
| 38 | 66 | \begin{table}[H] |
@@ -49,17 +77,20 @@ | ||
| 49 | 77 | \label{tab:price} |
| 50 | 78 | \footnotesize |
| 51 | 79 | \input{tables/strata_other} |
| 80 | +\begin{tablenotes}\footnotesize | |
| 81 | +\item Note: price quintiles are defined on the realised sale price of the test targets; see Section~\ref{sec:strata} on the denominator effect in the lowest quintile. | |
| 82 | +\end{tablenotes} | |
| 52 | 83 | \end{table} |
| 53 | 84 | |
| 54 | 85 | \section{Implementation details} |
| 55 | 86 | \label{app:impl} |
| 56 | 87 | |
| 57 | −\paragraph{Computation.} The backtest evaluates \nConfigs{} configurations on \nTargetsAll{} targets. For each target the candidate pool is retrieved with a $k$-d tree on projected coordinates, the time, conventional, hedonic, regional, local, GBM and ratio adjustments are computed once for every candidate, and the selection--window--count--weighting--reconciliation combinations are evaluated as operations on the resulting vectors. The computation is embarrassingly parallel over targets and was distributed over five Apple-silicon nodes of a local cluster; a full run takes under ten minutes. The quarterly re-estimation of the adjustment grids and gradient-boosting benchmarks (eleven as-of dates) takes a comparable time. All code is in Python (pandas, NumPy, SciPy, LightGBM) and is available from the author. | |
| 88 | +\paragraph{Computation.} The backtest evaluates \nConfigs{} configurations on \nTargetsAll{} targets. For each target the candidate pool is retrieved with a $k$-d tree on projected coordinates, the time, conventional, hedonic, regional, local, GBM and ratio adjustments are computed once for every candidate, and the selection--window--count--weighting--reconciliation combinations are evaluated as operations on the resulting vectors. The computation is embarrassingly parallel over targets and was distributed over five Apple-silicon nodes of a local cluster; the backtest itself takes under ten minutes once the index and the quarterly models are estimated, and the full chain (sample construction, index, eleven quarterly re-estimations of the grids and gradient-boosting models, backtest, analysis) about half an hour. All code is in Python (pandas, NumPy, SciPy, LightGBM) and is available from the author; the analyses of Sections~\ref{sec:selection_stability}, \ref{sec:exclusion} and \ref{sec:clusters}, the seasonal decomposition and the per-model confidence intervals are computed from the stored per-target estimates by a separate script that runs in about a minute. | |
| 58 | 89 | |
| 59 | −\paragraph{Local grid.} The local regression (adjustment scheme f) estimates, on the subject's candidates within 10~km and 24 months, the coefficients of log living area, log lot area, age, the two attached-dwelling indicators and the number of storeys, by ridge regression with penalty $\lambda=50$ towards the regional grid; with fewer than 40 candidates the regional grid is used. | |
| 90 | +\paragraph{Local grid.} The local regression (adjustment scheme f) estimates, on the subject's candidates within 10~km and 24 months, the coefficients of log living area, log lot area, age, the two attached-dwelling indicators and the number of storeys, by ridge regression with penalty $\lambda=50$ towards the regional grid (equation~\ref{eq:local}); with fewer than 40 candidates the regional grid is used. | |
| 60 | 91 | |
| 61 | −\paragraph{Gradient boosting.} LightGBM with 127 leaves, minimum 40 observations per leaf, learning rate 0.05, feature and bagging fractions 0.8, $L_2$ regularisation 1, L1 objective, number of trees by early stopping (patience 100) on the chronologically last 10\% of the training sample with a floor of 300 trees, then refit on the whole training sample. Features: log living area, log lot area, age, age$^2$, physical-link and construction-type indicators, storeys, new-build indicator, latitude, longitude, projected coordinates, region, building type, month of sale (capped at the last training month), and for M11b the log assessed value, its land share and its roll vintage. | |
| 92 | +\paragraph{Gradient boosting.} LightGBM with 127 leaves, minimum 40 observations per leaf, learning rate 0.05, feature and bagging fractions 0.8, $L_2$ regularisation 1, $L_1$ objective, number of trees by early stopping (patience 100) on the chronologically last 10\% of the training sample with a floor of 300 trees, then refit on the whole training sample. Features: log living area, log lot area, age, age$^2$, physical-link and construction-type indicators, storeys, new-build indicator, latitude, longitude, projected coordinates, region, building type, month of sale (set to the last training month when predicting), and for M11b the log assessed value, its land share and its roll vintage. The target is the log price brought to the level of the last training month by the point-in-time index; predictions are carried to the level of the month preceding the valuation by the index known at valuation time. The GBM adjustments of scheme (g) use the model without the assessed value. | |
| 62 | 93 | |
| 63 | −\paragraph{Cluster bootstrap.} Municipality--quarter cells of the test period are resampled with replacement 400 times; for each replication the mean of the paired APE differences is recomputed from cell sums, and for 150 replications the difference in medians is recomputed from the resampled observations. | |
| 94 | +\paragraph{Cluster bootstrap.} Municipality--quarter cells of the test period (several thousand cells) are resampled with replacement 999 times. For each replication, the resampled cells define integer multiplicities for every target; the mean of the paired APE differences and the two medians are recomputed with these multiplicities as weights, so that all models and all pairs are evaluated on the same replications. The cluster-robust $t$ statistic of equation~\eqref{eq:dm} uses the same cells. The sensitivity analysis of Table~\ref{tab:cluster_sens} repeats the procedure with MRC--quarter, municipality and region--quarter cells and 399 replications. | |
| 64 | 95 | |
| 65 | 96 | \end{appendices} |
modified
paper/figures/f1_mdape_vs_k.pdf
+0 −0
Binary file not shown.
modified
paper/figures/f2_surface_d_T.pdf
+0 −0
Binary file not shown.
modified
paper/figures/f3_ajustements.pdf
+0 −0
Binary file not shown.
modified
paper/figures/f4_competition.pdf
+0 −0
Binary file not shown.
modified
paper/figures/f5_densite.pdf
+0 −0
Binary file not shown.
modified
paper/figures/f6_k_par_densite.pdf
+0 −0
Binary file not shown.
modified
paper/figures/f7_indice.pdf
+0 −0
Binary file not shown.
modified
paper/main.pdf
+0 −0
Binary file not shown.
modified
paper/main.tex
+3 −2
@@ -95,9 +95,9 @@ | ||
| 95 | 95 | \newcommand{\WPtitle}{Does the Sales Comparison Approach Actually Work? A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation} |
| 96 | 96 | \newcommand{\WPsubtitle}{Evidence from 391,000 Single-Family Transactions in Quebec, 2021--2026} |
| 97 | 97 | \newcommand{\WPdate}{October 2026} |
| 98 | −\newcommand{\WPversion}{1.0} | |
| 98 | +\newcommand{\WPversion}{1.1} | |
| 99 | 99 | \newcommand{\WPabstract}{% |
| 100 | − The sales comparison approach is the dominant method of residential valuation, yet its operating rules---how many comparables to use, how far and how far back to search, which adjustments to apply, how to weight and reconcile---rest largely on professional convention. We subject these rules to a large-scale, point-in-time backtest on \nSalesClean{} arm's-length single-family transactions in Quebec (2021--2026) matched to the municipal assessment roll. For each of \nTargetsAll{} target sales in 2024--2026, the price is masked and \nConfigs{} configurations of the approach---together with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks---are applied using only information available on the valuation date; configurations are selected on 2024 and evaluated out of sample on 2025--2026. Adjustments dominate: the median absolute error of the five nearest sales falls from \mdapeMtwo{} with a time adjustment only to \mdapeMseven{} with a hedonic adjustment grid, and a stylized conventional grid captures most of that gain. Selecting physically similar rather than merely nearby comparables delivers a comparable gain without any adjustment. The optimal number of comparables is well above the conventional three---around twenty once a grid is applied, with no penalty up to fifty---and short time windows are dominated; the search radius matters far less than the number and similarity of the sales retained. Weighting and reconciliation rules matter little. The best transparent configuration chosen in 2024 reaches \mdapeMzero{} out of sample, against \mdapeMelevenB{} for a quarterly-retrained gradient-boosting model that uses the assessed value and \mdapeMtwelve{} for a hybrid; the cost of transparency is small and narrows in homogeneous submarkets. Accuracy is governed primarily by submarket liquidity; a wide-net rule that retains the most similar sales is self-adapting, and stratum-specific rules do not improve on it. The results turn a convention-based method into a set of empirically validated, reproducible rules. | |
| 100 | + The sales comparison approach is the dominant method of residential valuation, yet its operating rules---how many comparables to use, how far and how far back to search, which adjustments to apply, how to weight and reconcile---rest largely on professional convention. We subject these rules to a population-scale, point-in-time backtest on \nSalesClean{} arm's-length single-family transactions in Quebec (2021--2026) matched to the municipal assessment roll. For each of \nTargetsAll{} target sales in 2024--2026 the price is masked and \nConfigs{} configurations of the approach, together with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks, are applied using only information available on the valuation date; configurations are selected on 2024 and evaluated out of sample on 2025--2026, and paired differences are tested with a cluster bootstrap over municipality--quarter cells. Adjustments dominate: the median absolute error of the five nearest sales falls from \mdapeMtwo{} with a time adjustment only to \mdapeMseven{} with a hedonic grid, and a stylised conventional grid captures most of that gain. Selecting physically similar rather than merely nearby comparables delivers the same gain without any adjustment. The optimal number of comparables is about twenty once a grid is applied, with no penalty up to fifty; short windows are dominated and the search radius matters little. Weighting and reconciliation rules matter little. The best transparent configuration chosen in 2024 reaches \mdapeMzero{} out of sample, against \mdapeMelevenB{} for a quarterly-retrained gradient-boosting model using the assessed value; the cost of transparency, \transparencyCost{} points, is precisely estimated and narrows in homogeneous submarkets, and the chosen rule ranks second among \nConfigsCov{} configurations out of sample. Accuracy is governed primarily by submarket liquidity, and a wide-net rule retaining the most similar sales adapts to it on its own. | |
| 101 | 101 | } |
| 102 | 102 | \newcommand{\WPkeywords}{sales comparison approach, comparable selection, adjustment grid, automated valuation models, hedonic pricing, backtesting, residential real estate, Quebec} |
| 103 | 103 | \newcommand{\WPjel}{R31, R32, C52, C53, G12} |
@@ -122,6 +122,7 @@ | ||
| 122 | 122 | |
| 123 | 123 | % --- Title Page --- |
| 124 | 124 | \input{tables/numbers} |
| 125 | +\input{tables/numbers_revision} | |
| 125 | 126 | \input{sections/titlepage} |
| 126 | 127 | |
| 127 | 128 | % --- Main Body --- |
modified
paper/main_anon.pdf
+0 −0
Binary file not shown.
modified
paper/main_anon.tex
+3 −2
@@ -95,9 +95,9 @@ | ||
| 95 | 95 | \newcommand{\WPtitle}{Does the Sales Comparison Approach Actually Work? A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation} |
| 96 | 96 | \newcommand{\WPsubtitle}{Evidence from 391,000 Single-Family Transactions in Quebec, 2021--2026} |
| 97 | 97 | \newcommand{\WPdate}{October 2026} |
| 98 | −\newcommand{\WPversion}{1.0} | |
| 98 | +\newcommand{\WPversion}{1.1} | |
| 99 | 99 | \newcommand{\WPabstract}{% |
| 100 | − The sales comparison approach is the dominant method of residential valuation, yet its operating rules---how many comparables to use, how far and how far back to search, which adjustments to apply, how to weight and reconcile---rest largely on professional convention. We subject these rules to a large-scale, point-in-time backtest on \nSalesClean{} arm's-length single-family transactions in Quebec (2021--2026) matched to the municipal assessment roll. For each of \nTargetsAll{} target sales in 2024--2026, the price is masked and \nConfigs{} configurations of the approach---together with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks---are applied using only information available on the valuation date; configurations are selected on 2024 and evaluated out of sample on 2025--2026. Adjustments dominate: the median absolute error of the five nearest sales falls from \mdapeMtwo{} with a time adjustment only to \mdapeMseven{} with a hedonic adjustment grid, and a stylized conventional grid captures most of that gain. Selecting physically similar rather than merely nearby comparables delivers a comparable gain without any adjustment. The optimal number of comparables is well above the conventional three---around twenty once a grid is applied, with no penalty up to fifty---and short time windows are dominated; the search radius matters far less than the number and similarity of the sales retained. Weighting and reconciliation rules matter little. The best transparent configuration chosen in 2024 reaches \mdapeMzero{} out of sample, against \mdapeMelevenB{} for a quarterly-retrained gradient-boosting model that uses the assessed value and \mdapeMtwelve{} for a hybrid; the cost of transparency is small and narrows in homogeneous submarkets. Accuracy is governed primarily by submarket liquidity; a wide-net rule that retains the most similar sales is self-adapting, and stratum-specific rules do not improve on it. The results turn a convention-based method into a set of empirically validated, reproducible rules. | |
| 100 | + The sales comparison approach is the dominant method of residential valuation, yet its operating rules---how many comparables to use, how far and how far back to search, which adjustments to apply, how to weight and reconcile---rest largely on professional convention. We subject these rules to a population-scale, point-in-time backtest on \nSalesClean{} arm's-length single-family transactions in Quebec (2021--2026) matched to the municipal assessment roll. For each of \nTargetsAll{} target sales in 2024--2026 the price is masked and \nConfigs{} configurations of the approach, together with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks, are applied using only information available on the valuation date; configurations are selected on 2024 and evaluated out of sample on 2025--2026, and paired differences are tested with a cluster bootstrap over municipality--quarter cells. Adjustments dominate: the median absolute error of the five nearest sales falls from \mdapeMtwo{} with a time adjustment only to \mdapeMseven{} with a hedonic grid, and a stylised conventional grid captures most of that gain. Selecting physically similar rather than merely nearby comparables delivers the same gain without any adjustment. The optimal number of comparables is about twenty once a grid is applied, with no penalty up to fifty; short windows are dominated and the search radius matters little. Weighting and reconciliation rules matter little. The best transparent configuration chosen in 2024 reaches \mdapeMzero{} out of sample, against \mdapeMelevenB{} for a quarterly-retrained gradient-boosting model using the assessed value; the cost of transparency, \transparencyCost{} points, is precisely estimated and narrows in homogeneous submarkets, and the chosen rule ranks second among \nConfigsCov{} configurations out of sample. Accuracy is governed primarily by submarket liquidity, and a wide-net rule retaining the most similar sales adapts to it on its own. | |
| 101 | 101 | } |
| 102 | 102 | \newcommand{\WPkeywords}{sales comparison approach, comparable selection, adjustment grid, automated valuation models, hedonic pricing, backtesting, residential real estate, Quebec} |
| 103 | 103 | \newcommand{\WPjel}{R31, R32, C52, C53, G12} |
@@ -122,6 +122,7 @@ | ||
| 122 | 122 | |
| 123 | 123 | % --- Title Page --- |
| 124 | 124 | \input{tables/numbers} |
| 125 | +\input{tables/numbers_revision} | |
| 125 | 126 | \input{sections/titlepage_anon} |
| 126 | 127 | |
| 127 | 128 | % --- Main Body --- |
modified
paper/make_paper_tables.py
+34 −18
@@ -73,9 +73,15 @@ eb["Selection"] = eb["index"].map(lambda s: LAB[s.split("|")[0]]); eb["Adjustmen | ||
| 73 | 73 | eb = eb.rename(columns={"k": "$k$", "d": "$d$ (km)", "T": "$t$ (months)", "mdape_val": "MdAPE 2024", "mdape_test": "MdAPE 2025--26", "couverture_test": "Coverage"}) |
| 74 | 74 | write("e1_best", tabular(eb, ["Selection", "Adjustment", "$k$", "$d$ (km)", "$t$ (months)", "MdAPE 2024", "MdAPE 2025--26", "Coverage"], "llrrrrrr", fmt={"$k$": lambda v: str(int(v)), "$t$ (months)": lambda v: str(int(v)), "$d$ (km)": lambda v: f"{v:g}"})) |
| 75 | 75 | H = R["hypotheses"] |
| 76 | −h2 = pd.DataFrame({DENS_EN[k.split("|")[1]]: v for k, v in H["H2"].items() if k.startswith("hedo")}).T.loc[[DENS_EN[d] for d in DENS]].reset_index() | |
| 77 | −h2.columns = ["Density"] + [f"$k={c}$" for c in h2.columns[1:]] | |
| 78 | −write("h2", tabular(h2, list(h2.columns), "l" + "r" * (len(h2.columns) - 1), bold_min_cols=list(h2.columns[1:]))) | |
| 76 | +def h2tab(prefix): | |
| 77 | + h = pd.DataFrame({DENS_EN[k.split("|")[1]]: v for k, v in H["H2"].items() if k.startswith(prefix)}).T.loc[[DENS_EN[d] for d in DENS]].reset_index() | |
| 78 | + h.columns = ["Density"] + [f"$k={c}$" for c in h.columns[1:]] | |
| 79 | + return tabular(h, list(h.columns), "l" + "r" * (len(h.columns) - 1), bold_min_cols=list(h.columns[1:])) | |
| 80 | +t_h, t_n = h2tab("hedo"), h2tab("none") | |
| 81 | +t_h = t_h.replace("\\midrule", "\\midrule\n\\multicolumn{10}{l}{\\emph{Hedonic adjustment grid}} \\\\", 1) | |
| 82 | +body_n = t_n.split("\\midrule\n", 1)[1].replace("\\bottomrule\n\\end{tabular}\n", "") | |
| 83 | +t_h = t_h.replace("\\bottomrule", "\\midrule\n\\multicolumn{10}{l}{\\emph{Time adjustment only}} \\\\\n" + body_n + "\\bottomrule") | |
| 84 | +write("h2", t_h) | |
| 79 | 85 | fac = pd.read_csv(TAB / "facteurs_E2.csv"); anchor = fac[(fac.k == 5) & (fac.d == 5) & (fac["T"] == 12)] |
| 80 | 86 | adj = anchor[(anchor.sel == "geo") & (anchor.w == "equal") & (anchor.recon == "median")].set_index("adj").loc[["raw", "none", "conv", "hedo", "hedo_reg", "local", "gbm", "role"]].reset_index() |
| 81 | 87 | adj["Adjustment"] = adj["adj"].map(LAB) |
@@ -98,16 +104,18 @@ write("grid", tabular(gd, list(gd.columns), "lr") + f"\\par\\footnotesize Within | ||
| 98 | 104 | sel = anchor[(anchor.adj == "hedo") & (anchor.w == "equal") & (anchor.recon == "median")].copy(); sel["Selection"] = sel["sel"].map(LAB) |
| 99 | 105 | sel = sel.rename(columns={"mdape_val": "MdAPE 2024", "mdape_test": "MdAPE 2025--26", "biais_test": "Bias", "p10_test": "Within $\\pm10\\%$"}) |
| 100 | 106 | write("selection", tabular(sel, ["Selection", "MdAPE 2024", "MdAPE 2025--26", "Bias", "Within $\\pm10\\%$"], "lrrrr")) |
| 101 | −wr = anchor[(anchor.adj == "hedo") & (anchor.sel == "geo")].copy(); wr["Weighting"] = wr["w"].map({**LAB, "all": "product of the three"}); wr["Reconciliation"] = wr["recon"].map(LAB) | |
| 107 | +wr = anchor[(anchor.adj == "hedo") & (anchor.sel == "geo")].copy(); wr["Weighting"] = wr["w"].map({**LAB, "sim": "1/gross adjustment", "all": "product of the three"}); wr["Reconciliation"] = wr["recon"].map(LAB) | |
| 102 | 108 | wr = wr.rename(columns={"mdape_val": "MdAPE 2024", "mdape_test": "MdAPE 2025--26", "biais_test": "Bias", "p10_test": "Within $\\pm10\\%$"}) |
| 103 | 109 | write("weighting", tabular(wr, ["Weighting", "Reconciliation", "MdAPE 2024", "MdAPE 2025--26", "Bias", "Within $\\pm10\\%$"], "llrrrr")) |
| 104 | 110 | h4 = pd.DataFrame(H["H4"]).T.loc[DENS].reset_index(); h4["index"] = h4["index"].map(DENS_EN) |
| 105 | −SH = {"equal": "equal", "dist": "dist.", "rec": "rec.", "sim": "sim.", "all": "prod.", "median": "med.", "mean": "mean"} | |
| 111 | +SH = {"equal": "equal", "dist": "dist.", "rec": "rec.", "sim": "1/adj.", "all": "prod.", "median": "med.", "mean": "mean"} | |
| 106 | 112 | h4.columns = ["Density"] + [f"{SH[c.split('|')[0]]}/{SH[c.split('|')[1]]}" for c in h4.columns[1:]] |
| 107 | 113 | write("h4", tabular(h4, list(h4.columns), "l" + "r" * (len(h4.columns) - 1), bold_min_cols=list(h4.columns[1:]))) |
| 108 | 114 | marg = R["E2_marginal"]; rows = [] |
| 115 | +ORDER = {"sel": ["geo", "sim", "mix", "adj"], "adj": ["raw", "none", "conv", "hedo", "hedo_reg", "local", "gbm", "role"], "w": ["equal", "dist", "rec", "sim", "all"], "recon": ["median", "mean"]} | |
| 116 | +WLAB = {**LAB, "sim": "1/gross adjustment", "all": "product of the three"} | |
| 109 | 117 | for f, lab in [("sel", "Selection"), ("adj", "Adjustment"), ("w", "Weighting"), ("recon", "Reconciliation")]: |
| 110 | − for k, v in marg[f].items(): rows.append([lab, LAB[k], v]) | |
| 118 | + for k in ORDER[f]: rows.append([lab, (WLAB if f == "w" else LAB)[k], marg[f][k]]) | |
| 111 | 119 | mg = pd.DataFrame(rows, columns=["Factor", "Level", "Mean MdAPE"]); write("marginal", tabular(mg, list(mg.columns), "llr")) |
| 112 | 120 | cm = pd.DataFrame(comp).T; cm["val"] = [compv[m]["mdape"] for m in cm.index]; cm = cm.reset_index().rename(columns={"index": "Model"}) |
| 113 | 121 | cm["Model"] = cm["Model"].str.replace("M0 meilleure configuration empirique (choisie en 2024, couverture ≥ 97 %)", "M0 best configuration, chosen on 2024", regex=False) |
@@ -126,26 +134,26 @@ pr["$\\Delta$ MdAPE [95\\% CI]"] = pr.apply(lambda r: f"{r.diff_mdape:.2f} [{eva | ||
| 126 | 134 | pr["$\\Delta$ MAPE [95\\% CI]"] = pr.apply(lambda r: f"{r.diff_mape:.2f} [{eval(r.ci_mape)[0]:.2f}; {eval(r.ci_mape)[1]:.2f}]", axis=1) |
| 127 | 135 | pr["$A$ more accurate (\\%)"] = pr["part_A_meilleur"].map(f1); pr["$n$"] = pr["n"].map(n_) |
| 128 | 136 | write("pairs", tabular(pr, ["Comparison", "$\\Delta$ MdAPE [95\\% CI]", "$\\Delta$ MAPE [95\\% CI]", "$A$ more accurate (\\%)", "$n$"], "lrrrr")) |
| 129 | −def strata(by, label, order=None): | |
| 137 | +def strata(by, label, order=None, labels=None): | |
| 130 | 138 | st = pd.read_csv(TAB / f"strates_{by}.csv"); g = st.pivot(index=by, columns="modele", values="mdape").round(1) |
| 131 | 139 | g["n"] = st[st.modele == "M7"].set_index(by)["n"]; g = g[["n", "M2", "M7", "M8a", "M13", "M0", "M10", "M11b", "M12"]] |
| 132 | 140 | if order: g = g.loc[order] |
| 133 | − g = g.reset_index(); g[by] = g[by].map(lambda x: DENS_EN.get(x, x)); g = g.rename(columns={by: label, "n": "$n$"}) | |
| 141 | + labels = labels or {} | |
| 142 | + g = g.reset_index(); g[by] = g[by].map(lambda x: labels.get(x, DENS_EN.get(x, x))); g = g.rename(columns={by: label, "n": "$n$"}) | |
| 134 | 143 | return tabular(g, list(g.columns), "lr" + "r" * 8, fmt={"$n$": n_}) |
| 135 | 144 | write("density", strata("dens", "Density", DENS)) |
| 136 | 145 | st = pd.read_csv(TAB / "strates_region_nom.csv"); reg_order = st[st.modele == "M7"].sort_values("n", ascending=False)["region_nom"].tolist() |
| 137 | 146 | write("regions", strata("region_nom", "Region", reg_order)) |
| 138 | −parts = [] | |
| 139 | −for by, lab in [("prix_q", "Price quintile"), ("age_cl", "Age"), ("lien", "Physical link")]: | |
| 140 | − t = strata(by, lab); parts.append(t) | |
| 141 | −LIEN = {"0": "detached", "1": "semi-detached", "2": "row", "3": "integrated"}; AGE = {"≤ 3 ans": "$\\le 3$ years", "4-15": "4--15", "16-40": "16--40", "41-70": "41--70", "> 70": "$>70$"} | |
| 147 | +LIEN = {0.0: "detached", 1.0: "semi-detached", 2.0: "row", 3.0: "integrated"}; AGE = {"≤ 3 ans": "$\\le 3$ years", "4-15": "4--15", "16-40": "16--40", "41-70": "41--70", "> 70": "$>70$"} | |
| 142 | 148 | PQ = {"Q1 (bas)": "Q1 (low)", "Q5 (haut)": "Q5 (high)"} |
| 149 | +parts = [strata("prix_q", "Price quintile", ["Q1 (bas)", "Q2", "Q3", "Q4", "Q5 (haut)"], PQ), | |
| 150 | + strata("age_cl", "Age (years)", ["≤ 3 ans", "4-15", "16-40", "41-70", "> 70"], AGE), | |
| 151 | + strata("lien", "Physical link", [0.0, 1.0, 2.0, 3.0], LIEN)] | |
| 143 | 152 | txt = parts[0] |
| 144 | 153 | for p_ in parts[1:]: |
| 145 | 154 | body = p_.split("\\midrule\n", 1)[1].replace("\\bottomrule\n\\end{tabular}\n", "") |
| 146 | 155 | head = p_.split("\\toprule\n", 1)[1].split(" \\\\", 1)[0] |
| 147 | 156 | txt = txt.replace("\\bottomrule", "\\midrule\n" + head + " \\\\\n\\midrule\n" + body + "\\bottomrule") |
| 148 | −for k, v in {**LIEN, **AGE, **PQ}.items(): txt = txt.replace(f"\n{esc(k)} &", f"\n{v} &") | |
| 149 | 157 | write("strata_other", txt) |
| 150 | 158 | h5 = pd.DataFrame(H["H5"]).T.reset_index().rename(columns={"index": "Submarket", "sd_median": "Median $\\hat\\sigma$"}) |
| 151 | 159 | h5["Submarket"] = h5["Submarket"].map({"homogène": "homogeneous", "moyen": "intermediate", "hétérogène": "heterogeneous"}) |
@@ -194,7 +202,7 @@ savefig(fig, "f2_surface_d_T") | ||
| 194 | 202 | # f3 |
| 195 | 203 | fig, ax = plt.subplots(figsize=(7, 3.2)) |
| 196 | 204 | g = fac[(fac.k == 5) & (fac.d == 5) & (fac["T"] == 12) & (fac.sel == "geo") & (fac.w == "equal") & (fac.recon == "median")].set_index("adj").loc[["raw", "none", "conv", "hedo", "hedo_reg", "local", "gbm", "role"]] |
| 197 | −lab = {"raw": "none", "none": "time only", "conv": "conventional", "hedo": "hedonic", "hedo_reg": "regional", "local": "local", "gbm": "GBM", "role": "assessment ratio"} | |
| 205 | +lab = {"raw": "none", "none": "time only", "conv": "conventional", "hedo": "hedonic", "hedo_reg": "regional", "local": "local", "gbm": "GBM", "role": "assess. ratio"} | |
| 198 | 206 | x = np.arange(len(g)); ax.bar(x - 0.2, g.mdape_val, 0.38, color=PAL[0], label="validation 2024"); ax.bar(x + 0.2, g.mdape_test, 0.38, color=PAL[1], label="test 2025–26") |
| 199 | 207 | for i, (v1, v2) in enumerate(zip(g.mdape_val, g.mdape_test)): |
| 200 | 208 | ax.text(i - 0.2, v1 + 0.2, f"{v1:.1f}", ha="center", fontsize=7); ax.text(i + 0.2, v2 + 0.2, f"{v2:.1f}", ha="center", fontsize=7) |
@@ -221,9 +229,9 @@ for ax, a in zip(axes, ["none", "hedo"]): | ||
| 221 | 229 | ax.set_xscale("log"); ax.set_xticks([1, 3, 5, 10, 20, 50]); ax.set_xticklabels([1, 3, 5, 10, 20, 50]); ax.set_xlabel("k"); ax.set_title({"none": "time only", "hedo": "hedonic grid"}[a] + " — d ≤ 10 km, 12 months") |
| 222 | 230 | axes[0].set_ylabel("MdAPE, test (%)"); axes[1].legend(fontsize=8); savefig(fig, "f6_k_par_densite") |
| 223 | 231 | # f7 |
| 224 | −fig, ax = plt.subplots(figsize=(7, 3.2)); idxf = pd.read_csv(OUT / "index_full.csv") | |
| 225 | −for j, (r_, lab_) in enumerate([("QC", "Quebec (province)"), ("6", "Montréal"), ("3", "Capitale-Nationale"), ("7", "Outaouais"), ("5", "Estrie")]): | |
| 226 | − g = idxf[idxf.region.astype(str) == r_] | |
| 232 | +fig, ax = plt.subplots(figsize=(7, 3.2)); idxf = pd.read_csv(OUT / "index_full.csv", dtype={"region": str}) | |
| 233 | +for j, (r_, lab_) in enumerate([("QC", "Quebec (province)"), ("06", "Montréal"), ("03", "Capitale-Nationale"), ("16", "Montérégie"), ("07", "Outaouais"), ("05", "Estrie")]): | |
| 234 | + g = idxf[idxf.region.astype(str).str.zfill(2) == r_.zfill(2)] | |
| 227 | 235 | if len(g): ax.plot(pd.to_datetime(g.ym), np.exp(g.log_idx) * 100, color=PAL[j], lw=2 if r_ == "QC" else 1.4, label=lab_) |
| 228 | 236 | ax.set_ylabel("hedonic index (2021-01 = 100)"); ax.legend(fontsize=8); ax.set_title("Monthly price index (full sample, descriptive)") |
| 229 | 237 | savefig(fig, "f7_indice") |
@@ -263,7 +271,7 @@ m0 = R["M0"]; m["mZeroDesc"] = f"{LAB[m0['sel']]}, $k={int(m0['k'])}$, $\\le {fl | ||
| 263 | 271 | mu = R["M0_sans_contrainte"]; m["mZeroUncDesc"] = f"{LAB[mu['sel']]}, $k={int(mu['k'])}$, $\\le {float(mu['d']):g}$ km, {int(mu['T'])} months, {LAB[mu['adj']]}"; m["mZeroUncMdape"] = pc(mu["mdape_test"]); m["mZeroUncCov"] = pc(mu["couverture_test"]) |
| 264 | 272 | dn = R["strates_dens"]; m["densHighMseven"] = pc(dn["≥ 200"]["M7"]); m["densLowMseven"] = pc(dn["< 5"]["M7"]) |
| 265 | 273 | h1 = H["H1"]; m["hOneGeoDone"] = pc(h1["geo_d1"]["mdape_test"]); m["hOneGeoDoneCov"] = pc(h1["geo_d1"]["couverture_test"]); m["hOneSimDten"] = pc(h1["sim_d10"]["mdape_test"]) |
| 266 | −m["hOneGeoDtenHedo"] = pc(h1["geo_d10_hedo"]["mdape_test"]); m["hOneSimDtenHedo"] = pc(h1["sim_d10_hedo"]["mdape_test"]); m["hOneMixDtenHedo"] = pc(h1["mix_d10_hedo"]["mdape_test"]) | |
| 274 | +m["hOneGeoDten"] = pc(h1["geo_d10"]["mdape_test"]); m["hOneGeoDtenHedo"] = pc(h1["geo_d10_hedo"]["mdape_test"]); m["hOneSimDtenHedo"] = pc(h1["sim_d10_hedo"]["mdape_test"]); m["hOneMixDtenHedo"] = pc(h1["mix_d10_hedo"]["mdape_test"]) | |
| 267 | 275 | h5 = H["H5"]; m["hFiveGapHomog"] = f1(h5["homogène"]["M0"] - h5["homogène"]["M11b"]); m["hFiveGapHeter"] = f1(h5["hétérogène"]["M0"] - h5["hétérogène"]["M11b"]) |
| 268 | 276 | ivh = R["intervalles"]["hedo|10|10|24"]; m["ivWidth"] = pc(ivh["largeur_mediane_pct"]) |
| 269 | 277 | cg = R["regles_conditionnelles_global_test"]; m["condMdape"] = pc(cg["mdape"]); m["condPten"] = pc(cg["p10"]); m["condCov"] = pc(cg["couverture"]) |
@@ -274,6 +282,14 @@ kn = surf[(surf.sel == "geo") & (surf.adj == "none") & (surf.d == 10) & (surf["T | ||
| 274 | 282 | m["noneKten"] = pc(kn[10]); m["noneKfifty"] = pc(kn[50]) |
| 275 | 283 | pq = R["strates_prix_q"]; m["prixQoneMseven"] = pc(pq["Q1 (bas)"]["M7"]); m["prixQfourMseven"] = pc(pq["Q4"]["M7"]) |
| 276 | 284 | stb = {t["trimestre"]: t for t in R["stabilite"]}; m["biasQoneMseven"] = f"{stb['2025Q1']['biais_M7']:+.1f}\\%"; m["biasQthreeMseven"] = f"{stb['2025Q3']['biais_M7']:+.1f}\\%" |
| 285 | +w9 = R["M9_weights"]; w12 = R["M12_weights"] | |
| 286 | +m["wNineNone"] = f"{w9['none']:.2f}"; m["wNineGbm"] = f"{w9['gbm']:.2f}"; m["wNineRole"] = f"{w9['role']:.2f}" | |
| 287 | +m["wTwelveGbm"] = f"{w12['gbm_role']:.2f}"; m["wTwelveCompGbm"] = f"{w12['comps_gbm']:.2f}"; m["wTwelveCompHedo"] = f"{w12['comps_hedo']:.2f}" | |
| 288 | +dn_n = R["strates_dens_n"]; m["densLowN"] = n_(dn_n["< 5"]); m["densLowNall"] = n_(R["densite_distribution"]["< 5"]) | |
| 289 | +m["ivCovLo"] = f"{min(v['couverture_nominale_80'] for v in R['intervalles'].values()):.0f}"; m["ivCovHi"] = f"{max(v['couverture_nominale_80'] for v in R['intervalles'].values()):.0f}" | |
| 290 | +m["ivHalf"] = f"{ivh['largeur_mediane_pct'] / 2:.0f}\\%" | |
| 291 | +h4d = H["H4"]["≥ 200"]; m["hFourSimDense"] = pc(h4d["sim|mean"]); m["hFourEqualMeanDense"] = pc(h4d["equal|mean"]) | |
| 292 | +m["gridRtwo"] = f"{mod['r2_within']:.2f}"; m["gridN"] = n_(mod["n_train"]) | |
| 277 | 293 | lines = [f"\\newcommand{{\\{k}}}{{{v}}}" for k, v in m.items()] |
| 278 | 294 | (P / "tables" / "numbers.tex").write_text("% généré par make_paper_tables.py\n" + "\n".join(lines) + "\n") |
| 279 | 295 | print(f"{len(m)} macros, tables et figures écrites") |
modified
paper/references.bib
+513 −87
@@ -1,7 +1,9 @@ | ||
| 1 | 1 | % ============================================================================ |
| 2 | −% References — Working Paper 11 | |
| 2 | +% References — UQO Working Paper No. 16 (revised 2026-10-17; all entries verified | |
| 3 | +% against Crossref / publisher pages, DOIs recorded where they exist) | |
| 3 | 4 | % ============================================================================ |
| 4 | 5 | |
| 6 | +% ---------------------------------------------------------------- standards and books | |
| 5 | 7 | @book{appraisal2020, |
| 6 | 8 | author = {{Appraisal Institute}}, |
| 7 | 9 | title = {The Appraisal of Real Estate}, |
@@ -27,6 +29,15 @@ | ||
| 27 | 29 | year = {2018} |
| 28 | 30 | } |
| 29 | 31 | |
| 32 | +@book{gloudemans1999, | |
| 33 | + author = {Gloudemans, Robert J.}, | |
| 34 | + title = {Mass Appraisal of Real Property}, | |
| 35 | + publisher = {International Association of Assessing Officers}, | |
| 36 | + address = {Chicago}, | |
| 37 | + year = {1999} | |
| 38 | +} | |
| 39 | + | |
| 40 | +% ---------------------------------------------------------------- the sales comparison approach as an estimator | |
| 30 | 41 | @article{colwell1983, |
| 31 | 42 | author = {Colwell, Peter F. and Cannaday, Roger E. and Wu, Chunchi}, |
| 32 | 43 | title = {The Analytical Foundations of Adjustment Grid Methods}, |
@@ -34,7 +45,52 @@ | ||
| 34 | 45 | volume = {11}, |
| 35 | 46 | number = {1}, |
| 36 | 47 | pages = {11--29}, |
| 37 | − year = {1983} | |
| 48 | + year = {1983}, | |
| 49 | + doi = {10.1111/1540-6229.00277} | |
| 50 | +} | |
| 51 | + | |
| 52 | +@article{cannaday1986, | |
| 53 | + author = {Cannaday, Roger E. and Sunderman, Mark A.}, | |
| 54 | + title = {Estimation of Depreciation for Single-Family Appraisals}, | |
| 55 | + journal = {Real Estate Economics}, | |
| 56 | + volume = {14}, | |
| 57 | + number = {2}, | |
| 58 | + pages = {255--273}, | |
| 59 | + year = {1986}, | |
| 60 | + doi = {10.1111/1540-6229.00386} | |
| 61 | +} | |
| 62 | + | |
| 63 | +@article{isakson1986, | |
| 64 | + author = {Isakson, Hans R.}, | |
| 65 | + title = {The Nearest Neighbors Appraisal Technique: An Alternative to the Adjustment Grid Methods}, | |
| 66 | + journal = {Real Estate Economics}, | |
| 67 | + volume = {14}, | |
| 68 | + number = {2}, | |
| 69 | + pages = {274--286}, | |
| 70 | + year = {1986}, | |
| 71 | + doi = {10.1111/1540-6229.00387} | |
| 72 | +} | |
| 73 | + | |
| 74 | +@article{isakson1988, | |
| 75 | + author = {Isakson, Hans R.}, | |
| 76 | + title = {Valuation Analysis of Commercial Real Estate Using the Nearest Neighbors Appraisal Technique}, | |
| 77 | + journal = {Growth and Change}, | |
| 78 | + volume = {19}, | |
| 79 | + number = {2}, | |
| 80 | + pages = {11--24}, | |
| 81 | + year = {1988}, | |
| 82 | + doi = {10.1111/j.1468-2257.1988.tb00466.x} | |
| 83 | +} | |
| 84 | + | |
| 85 | +@article{lipscomb1990, | |
| 86 | + author = {Lipscomb, Joseph B. and Gray, J. Brian}, | |
| 87 | + title = {An Empirical Investigation of Four Market-Derived Adjustment Methods}, | |
| 88 | + journal = {Journal of Real Estate Research}, | |
| 89 | + volume = {5}, | |
| 90 | + number = {1}, | |
| 91 | + pages = {53--66}, | |
| 92 | + year = {1990}, | |
| 93 | + doi = {10.1080/10835547.1990.12090602} | |
| 38 | 94 | } |
| 39 | 95 | |
| 40 | 96 | @article{vandell1991, |
@@ -44,7 +100,30 @@ | ||
| 44 | 100 | volume = {19}, |
| 45 | 101 | number = {2}, |
| 46 | 102 | pages = {213--239}, |
| 47 | − year = {1991} | |
| 103 | + year = {1991}, | |
| 104 | + doi = {10.1111/1540-6229.00550} | |
| 105 | +} | |
| 106 | + | |
| 107 | +@article{kang1991, | |
| 108 | + author = {Kang, Han-Bin and Reichert, Alan K.}, | |
| 109 | + title = {An Empirical Analysis of Hedonic Regression and Grid-Adjustment Techniques in Real Estate Appraisal}, | |
| 110 | + journal = {Real Estate Economics}, | |
| 111 | + volume = {19}, | |
| 112 | + number = {1}, | |
| 113 | + pages = {70--91}, | |
| 114 | + year = {1991}, | |
| 115 | + doi = {10.1111/1540-6229.00541} | |
| 116 | +} | |
| 117 | + | |
| 118 | +@article{quan1991, | |
| 119 | + author = {Quan, Daniel C. and Quigley, John M.}, | |
| 120 | + title = {Price Formation and the Appraisal Function in Real Estate Markets}, | |
| 121 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 122 | + volume = {4}, | |
| 123 | + number = {2}, | |
| 124 | + pages = {127--146}, | |
| 125 | + year = {1991}, | |
| 126 | + doi = {10.1007/BF00173120} | |
| 48 | 127 | } |
| 49 | 128 | |
| 50 | 129 | @article{gau1992, |
@@ -54,7 +133,19 @@ | ||
| 54 | 133 | volume = {20}, |
| 55 | 134 | number = {1}, |
| 56 | 135 | pages = {107--123}, |
| 57 | − year = {1992} | |
| 136 | + year = {1992}, | |
| 137 | + doi = {10.1111/1540-6229.00574} | |
| 138 | +} | |
| 139 | + | |
| 140 | +@article{green1994, | |
| 141 | + author = {Green, Richard K.}, | |
| 142 | + title = {Optimal Comparable Weighting and Selection: A Comment}, | |
| 143 | + journal = {Real Estate Economics}, | |
| 144 | + volume = {22}, | |
| 145 | + number = {4}, | |
| 146 | + pages = {647--654}, | |
| 147 | + year = {1994}, | |
| 148 | + doi = {10.1111/1540-6229.00653} | |
| 58 | 149 | } |
| 59 | 150 | |
| 60 | 151 | @article{gau1994, |
@@ -64,27 +155,63 @@ | ||
| 64 | 155 | volume = {22}, |
| 65 | 156 | number = {4}, |
| 66 | 157 | pages = {655--663}, |
| 67 | − year = {1994} | |
| 158 | + year = {1994}, | |
| 159 | + doi = {10.1111/1540-6229.00654} | |
| 68 | 160 | } |
| 69 | 161 | |
| 70 | −@article{isakson1986, | |
| 71 | − author = {Isakson, Hans R.}, | |
| 72 | − title = {The Nearest Neighbors Appraisal Technique: An Alternative to the Adjustment Grid Methods}, | |
| 73 | − journal = {Real Estate Economics}, | |
| 162 | +@article{pace1996, | |
| 163 | + author = {Pace, R. Kelley}, | |
| 164 | + title = {Relative Performance of the Grid, Nearest Neighbor, and {OLS} Estimators}, | |
| 165 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 166 | + volume = {13}, | |
| 167 | + number = {3}, | |
| 168 | + pages = {203--218}, | |
| 169 | + year = {1996}, | |
| 170 | + doi = {10.1007/BF00217391} | |
| 171 | +} | |
| 172 | + | |
| 173 | +@article{epley1997, | |
| 174 | + author = {Epley, Donald R.}, | |
| 175 | + title = {A Note on the Optimal Selection and Weighting of Comparable Properties}, | |
| 176 | + journal = {Journal of Real Estate Research}, | |
| 74 | 177 | volume = {14}, |
| 75 | 178 | number = {2}, |
| 76 | − pages = {274--286}, | |
| 77 | − year = {1986} | |
| 179 | + pages = {175--182}, | |
| 180 | + year = {1997}, | |
| 181 | + doi = {10.1080/10835547.1997.12090895} | |
| 78 | 182 | } |
| 79 | 183 | |
| 80 | −@article{isakson2002, | |
| 81 | − author = {Isakson, Hans R.}, | |
| 82 | − title = {The Linear Algebra of the Sales Comparison Approach}, | |
| 184 | +@article{pace1997, | |
| 185 | + author = {Pace, R. Kelley and Gilley, Otis W.}, | |
| 186 | + title = {Using the Spatial Configuration of the Data to Improve Estimation}, | |
| 187 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 188 | + volume = {14}, | |
| 189 | + number = {3}, | |
| 190 | + pages = {333--340}, | |
| 191 | + year = {1997}, | |
| 192 | + doi = {10.1023/A:1007762613901} | |
| 193 | +} | |
| 194 | + | |
| 195 | +@article{pace1998, | |
| 196 | + author = {Pace, R. Kelley}, | |
| 197 | + title = {Appraisal Using Generalized Additive Models}, | |
| 83 | 198 | journal = {Journal of Real Estate Research}, |
| 84 | − volume = {24}, | |
| 199 | + volume = {15}, | |
| 200 | + number = {1}, | |
| 201 | + pages = {77--99}, | |
| 202 | + year = {1998}, | |
| 203 | + doi = {10.1080/10835547.1998.12090916} | |
| 204 | +} | |
| 205 | + | |
| 206 | +@article{pacegilley1998, | |
| 207 | + author = {Pace, R. Kelley and Gilley, Otis W.}, | |
| 208 | + title = {Generalizing the {OLS} and Grid Estimators}, | |
| 209 | + journal = {Real Estate Economics}, | |
| 210 | + volume = {26}, | |
| 85 | 211 | number = {2}, |
| 86 | − pages = {117--128}, | |
| 87 | − year = {2002} | |
| 212 | + pages = {331--347}, | |
| 213 | + year = {1998}, | |
| 214 | + doi = {10.1111/1540-6229.00748} | |
| 88 | 215 | } |
| 89 | 216 | |
| 90 | 217 | @article{lentz1998, |
@@ -94,39 +221,33 @@ | ||
| 94 | 221 | volume = {15}, |
| 95 | 222 | number = {1}, |
| 96 | 223 | pages = {11--39}, |
| 97 | − year = {1998} | |
| 98 | −} | |
| 99 | − | |
| 100 | −@article{kang1991, | |
| 101 | − author = {Kang, Han-Bin and Reichert, Alan K.}, | |
| 102 | − title = {An Empirical Analysis of Hedonic Regression and Grid-Adjustment Techniques in Real Estate Appraisal}, | |
| 103 | − journal = {Real Estate Economics}, | |
| 104 | − volume = {19}, | |
| 105 | − number = {1}, | |
| 106 | − pages = {70--91}, | |
| 107 | − year = {1991} | |
| 224 | + year = {1998}, | |
| 225 | + doi = {10.1080/10835547.1998.12090912} | |
| 108 | 226 | } |
| 109 | 227 | |
| 110 | −@article{pace1998, | |
| 111 | − author = {Pace, R. Kelley}, | |
| 112 | − title = {Appraisal Using Generalized Additive Models}, | |
| 228 | +@article{isakson2002, | |
| 229 | + author = {Isakson, Hans R.}, | |
| 230 | + title = {The Linear Algebra of the Sales Comparison Approach}, | |
| 113 | 231 | journal = {Journal of Real Estate Research}, |
| 114 | − volume = {15}, | |
| 115 | − number = {1}, | |
| 116 | − pages = {77--99}, | |
| 117 | − year = {1998} | |
| 232 | + volume = {24}, | |
| 233 | + number = {2}, | |
| 234 | + pages = {117--128}, | |
| 235 | + year = {2002}, | |
| 236 | + doi = {10.1080/10835547.2002.12091090} | |
| 118 | 237 | } |
| 119 | 238 | |
| 120 | −@article{pace1997, | |
| 121 | − author = {Pace, R. Kelley and Gilley, Otis W.}, | |
| 122 | − title = {Using the Spatial Configuration of the Data to Improve Estimation}, | |
| 123 | − journal = {Journal of Real Estate Finance and Economics}, | |
| 124 | − volume = {14}, | |
| 125 | − number = {3}, | |
| 126 | − pages = {333--340}, | |
| 127 | − year = {1997} | |
| 239 | +@article{lai2008, | |
| 240 | + author = {Lai, Tsong-Yue and Vandell, Kerry and Wang, Ko and Welke, Gerd}, | |
| 241 | + title = {Estimating Property Values by Replication: An Alternative to the Traditional Grid and Regression Methods}, | |
| 242 | + journal = {Journal of Real Estate Research}, | |
| 243 | + volume = {30}, | |
| 244 | + number = {4}, | |
| 245 | + pages = {441--460}, | |
| 246 | + year = {2008}, | |
| 247 | + doi = {10.1080/10835547.2008.12091226} | |
| 128 | 248 | } |
| 129 | 249 | |
| 250 | +% ---------------------------------------------------------------- hedonic theory, indices and the assessed value | |
| 130 | 251 | @article{rosen1974, |
| 131 | 252 | author = {Rosen, Sherwin}, |
| 132 | 253 | title = {Hedonic Prices and Implicit Markets: Product Differentiation in Pure Competition}, |
@@ -134,27 +255,8 @@ | ||
| 134 | 255 | volume = {82}, |
| 135 | 256 | number = {1}, |
| 136 | 257 | pages = {34--55}, |
| 137 | − year = {1974} | |
| 138 | −} | |
| 139 | − | |
| 140 | −@article{bourassa2010, | |
| 141 | − author = {Bourassa, Steven C. and Cantoni, Eva and Hoesli, Martin}, | |
| 142 | − title = {Predicting House Prices with Spatial Dependence: A Comparison of Alternative Methods}, | |
| 143 | − journal = {Journal of Real Estate Research}, | |
| 144 | − volume = {32}, | |
| 145 | − number = {2}, | |
| 146 | − pages = {139--159}, | |
| 147 | − year = {2010} | |
| 148 | −} | |
| 149 | − | |
| 150 | −@article{case2004, | |
| 151 | − author = {Case, Bradford and Clapp, John and Dubin, Robin and Rodriguez, Mauricio}, | |
| 152 | − title = {Modeling Spatial and Temporal House Price Patterns: A Comparison of Four Models}, | |
| 153 | − journal = {Journal of Real Estate Finance and Economics}, | |
| 154 | − volume = {29}, | |
| 155 | − number = {2}, | |
| 156 | − pages = {167--191}, | |
| 157 | − year = {2004} | |
| 258 | + year = {1974}, | |
| 259 | + doi = {10.1086/260169} | |
| 158 | 260 | } |
| 159 | 261 | |
| 160 | 262 | @article{bailey1963, |
@@ -164,7 +266,8 @@ | ||
| 164 | 266 | volume = {58}, |
| 165 | 267 | number = {304}, |
| 166 | 268 | pages = {933--942}, |
| 167 | − year = {1963} | |
| 269 | + year = {1963}, | |
| 270 | + doi = {10.1080/01621459.1963.10480679} | |
| 168 | 271 | } |
| 169 | 272 | |
| 170 | 273 | @article{case1989, |
@@ -177,6 +280,39 @@ | ||
| 177 | 280 | year = {1989} |
| 178 | 281 | } |
| 179 | 282 | |
| 283 | +@article{clapp1992, | |
| 284 | + author = {Clapp, John M. and Giaccotto, Carmelo}, | |
| 285 | + title = {Estimating Price Indices for Residential Property: A Comparison of Repeat Sales and Assessed Value Methods}, | |
| 286 | + journal = {Journal of the American Statistical Association}, | |
| 287 | + volume = {87}, | |
| 288 | + number = {418}, | |
| 289 | + pages = {300--306}, | |
| 290 | + year = {1992}, | |
| 291 | + doi = {10.1080/01621459.1992.10475209} | |
| 292 | +} | |
| 293 | + | |
| 294 | +@article{thibodeau2003, | |
| 295 | + author = {Thibodeau, Thomas G.}, | |
| 296 | + title = {Marking Single-Family Property Values to Market}, | |
| 297 | + journal = {Real Estate Economics}, | |
| 298 | + volume = {31}, | |
| 299 | + number = {1}, | |
| 300 | + pages = {1--22}, | |
| 301 | + year = {2003}, | |
| 302 | + doi = {10.1111/j.1080-8620.2003.00055.x} | |
| 303 | +} | |
| 304 | + | |
| 305 | +@article{clapham2006, | |
| 306 | + author = {Clapham, Eric and Englund, Peter and Quigley, John M. and Redfearn, Christian L.}, | |
| 307 | + title = {Revisiting the Past and Settling the Score: Index Revision for House Price Derivatives}, | |
| 308 | + journal = {Real Estate Economics}, | |
| 309 | + volume = {34}, | |
| 310 | + number = {2}, | |
| 311 | + pages = {275--302}, | |
| 312 | + year = {2006}, | |
| 313 | + doi = {10.1111/j.1540-6229.2006.00167.x} | |
| 314 | +} | |
| 315 | + | |
| 180 | 316 | @article{hill2013, |
| 181 | 317 | author = {Hill, Robert J.}, |
| 182 | 318 | title = {Hedonic Price Indexes for Residential Housing: A Survey, Evaluation and Taxonomy}, |
@@ -184,15 +320,131 @@ | ||
| 184 | 320 | volume = {27}, |
| 185 | 321 | number = {5}, |
| 186 | 322 | pages = {879--914}, |
| 187 | − year = {2013} | |
| 323 | + year = {2013}, | |
| 324 | + doi = {10.1111/j.1467-6419.2012.00731.x} | |
| 188 | 325 | } |
| 189 | 326 | |
| 190 | −@book{gloudemans1999, | |
| 191 | − author = {Gloudemans, Robert J.}, | |
| 192 | − title = {Mass Appraisal of Real Property}, | |
| 193 | − publisher = {International Association of Assessing Officers}, | |
| 194 | − address = {Chicago}, | |
| 195 | − year = {1999} | |
| 327 | +@article{hillscholz2018, | |
| 328 | + author = {Hill, Robert J. and Scholz, Michael}, | |
| 329 | + title = {Can Geospatial Data Improve House Price Indexes? {A} Hedonic Imputation Approach with Splines}, | |
| 330 | + journal = {Review of Income and Wealth}, | |
| 331 | + volume = {64}, | |
| 332 | + number = {4}, | |
| 333 | + pages = {737--756}, | |
| 334 | + year = {2018}, | |
| 335 | + doi = {10.1111/roiw.12303} | |
| 336 | +} | |
| 337 | + | |
| 338 | +@article{ngai2014, | |
| 339 | + author = {Ngai, L. Rachel and Tenreyro, Silvana}, | |
| 340 | + title = {Hot and Cold Seasons in the Housing Market}, | |
| 341 | + journal = {American Economic Review}, | |
| 342 | + volume = {104}, | |
| 343 | + number = {12}, | |
| 344 | + pages = {3991--4026}, | |
| 345 | + year = {2014}, | |
| 346 | + doi = {10.1257/aer.104.12.3991} | |
| 347 | +} | |
| 348 | + | |
| 349 | +% ---------------------------------------------------------------- spatial dependence, submarkets and local estimation | |
| 350 | +@article{desrosiers2000, | |
| 351 | + author = {{Des Rosiers}, Fran{\c{c}}ois and Th{\'e}riault, Marius and Villeneuve, Paul-Y.}, | |
| 352 | + title = {Sorting out Access and Neighbourhood Factors in Hedonic Price Modelling}, | |
| 353 | + journal = {Journal of Property Investment \& Finance}, | |
| 354 | + volume = {18}, | |
| 355 | + number = {3}, | |
| 356 | + pages = {291--315}, | |
| 357 | + year = {2000}, | |
| 358 | + doi = {10.1108/14635780010338245} | |
| 359 | +} | |
| 360 | + | |
| 361 | +@article{bourassa2003, | |
| 362 | + author = {Bourassa, Steven C. and Hoesli, Martin and Peng, Vincent S.}, | |
| 363 | + title = {Do Housing Submarkets Really Matter?}, | |
| 364 | + journal = {Journal of Housing Economics}, | |
| 365 | + volume = {12}, | |
| 366 | + number = {1}, | |
| 367 | + pages = {12--28}, | |
| 368 | + year = {2003}, | |
| 369 | + doi = {10.1016/S1051-1377(03)00003-2} | |
| 370 | +} | |
| 371 | + | |
| 372 | +@article{goodman2003, | |
| 373 | + author = {Goodman, Allen C. and Thibodeau, Thomas G.}, | |
| 374 | + title = {Housing Market Segmentation and Hedonic Prediction Accuracy}, | |
| 375 | + journal = {Journal of Housing Economics}, | |
| 376 | + volume = {12}, | |
| 377 | + number = {3}, | |
| 378 | + pages = {181--201}, | |
| 379 | + year = {2003}, | |
| 380 | + doi = {10.1016/S1051-1377(03)00031-7} | |
| 381 | +} | |
| 382 | + | |
| 383 | +@article{case2004, | |
| 384 | + author = {Case, Bradford and Clapp, John and Dubin, Robin and Rodriguez, Mauricio}, | |
| 385 | + title = {Modeling Spatial and Temporal House Price Patterns: A Comparison of Four Models}, | |
| 386 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 387 | + volume = {29}, | |
| 388 | + number = {2}, | |
| 389 | + pages = {167--191}, | |
| 390 | + year = {2004}, | |
| 391 | + doi = {10.1023/B:REAL.0000035309.60607.53} | |
| 392 | +} | |
| 393 | + | |
| 394 | +@article{bourassa2007, | |
| 395 | + author = {Bourassa, Steven C. and Cantoni, Eva and Hoesli, Martin}, | |
| 396 | + title = {Spatial Dependence, Housing Submarkets, and House Price Prediction}, | |
| 397 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 398 | + volume = {35}, | |
| 399 | + number = {2}, | |
| 400 | + pages = {143--160}, | |
| 401 | + year = {2007}, | |
| 402 | + doi = {10.1007/s11146-007-9036-8} | |
| 403 | +} | |
| 404 | + | |
| 405 | +@article{bourassa2010, | |
| 406 | + author = {Bourassa, Steven C. and Cantoni, Eva and Hoesli, Martin}, | |
| 407 | + title = {Predicting House Prices with Spatial Dependence: A Comparison of Alternative Methods}, | |
| 408 | + journal = {Journal of Real Estate Research}, | |
| 409 | + volume = {32}, | |
| 410 | + number = {2}, | |
| 411 | + pages = {139--160}, | |
| 412 | + year = {2010}, | |
| 413 | + doi = {10.1080/10835547.2010.12091276} | |
| 414 | +} | |
| 415 | + | |
| 416 | +@article{dube2013, | |
| 417 | + author = {Dub{\'e}, Jean and Legros, Diègo}, | |
| 418 | + title = {A Spatio-Temporal Measure of Spatial Dependence: An Example Using Real Estate Data}, | |
| 419 | + journal = {Papers in Regional Science}, | |
| 420 | + volume = {92}, | |
| 421 | + number = {1}, | |
| 422 | + pages = {19--31}, | |
| 423 | + year = {2013}, | |
| 424 | + doi = {10.1111/j.1435-5957.2011.00402.x} | |
| 425 | +} | |
| 426 | + | |
| 427 | +@article{dube2014, | |
| 428 | + author = {Dub{\'e}, Jean and Legros, Diègo}, | |
| 429 | + title = {Spatial Econometrics and the Hedonic Pricing Model: What About the Temporal Dimension?}, | |
| 430 | + journal = {Journal of Property Research}, | |
| 431 | + volume = {31}, | |
| 432 | + number = {4}, | |
| 433 | + pages = {333--359}, | |
| 434 | + year = {2014}, | |
| 435 | + doi = {10.1080/09599916.2014.913655} | |
| 436 | +} | |
| 437 | + | |
| 438 | +% ---------------------------------------------------------------- automated valuation and machine learning | |
| 439 | +@article{shiller1999, | |
| 440 | + author = {Shiller, Robert J. and Weiss, Allan N.}, | |
| 441 | + title = {Evaluating Real Estate Valuation Systems}, | |
| 442 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 443 | + volume = {18}, | |
| 444 | + number = {2}, | |
| 445 | + pages = {147--161}, | |
| 446 | + year = {1999}, | |
| 447 | + doi = {10.1023/A:1007756607862} | |
| 196 | 448 | } |
| 197 | 449 | |
| 198 | 450 | @article{pagourtzi2003, |
@@ -202,7 +454,19 @@ | ||
| 202 | 454 | volume = {21}, |
| 203 | 455 | number = {4}, |
| 204 | 456 | pages = {383--401}, |
| 205 | − year = {2003} | |
| 457 | + year = {2003}, | |
| 458 | + doi = {10.1108/14635780310483656} | |
| 459 | +} | |
| 460 | + | |
| 461 | +@article{zurada2011, | |
| 462 | + author = {Zurada, Jozef and Levitan, Alan S. and Guan, Jian}, | |
| 463 | + title = {A Comparison of Regression and Artificial Intelligence Methods in a Mass Appraisal Context}, | |
| 464 | + journal = {Journal of Real Estate Research}, | |
| 465 | + volume = {33}, | |
| 466 | + number = {3}, | |
| 467 | + pages = {349--388}, | |
| 468 | + year = {2011}, | |
| 469 | + doi = {10.1080/10835547.2011.12091311} | |
| 206 | 470 | } |
| 207 | 471 | |
| 208 | 472 | @article{mccluskey2013, |
@@ -212,7 +476,19 @@ | ||
| 212 | 476 | volume = {30}, |
| 213 | 477 | number = {4}, |
| 214 | 478 | pages = {239--265}, |
| 215 | − year = {2013} | |
| 479 | + year = {2013}, | |
| 480 | + doi = {10.1080/09599916.2013.781204} | |
| 481 | +} | |
| 482 | + | |
| 483 | +@article{kok2017, | |
| 484 | + author = {Kok, Nils and Koponen, Eija-Leena and Mart{\'\i}nez-Barbosa, Carmen Adriana}, | |
| 485 | + title = {Big Data in Real Estate? {From} Manual Appraisal to Automated Valuation}, | |
| 486 | + journal = {Journal of Portfolio Management}, | |
| 487 | + volume = {43}, | |
| 488 | + number = {6}, | |
| 489 | + pages = {202--211}, | |
| 490 | + year = {2017}, | |
| 491 | + doi = {10.3905/jpm.2017.43.6.202} | |
| 216 | 492 | } |
| 217 | 493 | |
| 218 | 494 | @article{mayer2019, |
@@ -222,7 +498,8 @@ | ||
| 222 | 498 | volume = {12}, |
| 223 | 499 | number = {1}, |
| 224 | 500 | pages = {134--150}, |
| 225 | − year = {2019} | |
| 501 | + year = {2019}, | |
| 502 | + doi = {10.1108/JERER-08-2018-0035} | |
| 226 | 503 | } |
| 227 | 504 | |
| 228 | 505 | @article{pace2020, |
@@ -232,7 +509,64 @@ | ||
| 232 | 509 | volume = {60}, |
| 233 | 510 | number = {1--2}, |
| 234 | 511 | pages = {170--180}, |
| 235 | − year = {2020} | |
| 512 | + year = {2020}, | |
| 513 | + doi = {10.1007/s11146-019-09724-w} | |
| 514 | +} | |
| 515 | + | |
| 516 | +@article{bogin2020, | |
| 517 | + author = {Bogin, Alexander N. and Shui, Jessica}, | |
| 518 | + title = {Appraisal Accuracy and Automated Valuation Models in Rural Areas}, | |
| 519 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 520 | + volume = {60}, | |
| 521 | + number = {1--2}, | |
| 522 | + pages = {40--52}, | |
| 523 | + year = {2020}, | |
| 524 | + doi = {10.1007/s11146-019-09712-0} | |
| 525 | +} | |
| 526 | + | |
| 527 | +@article{krause2020, | |
| 528 | + author = {Krause, Andy and Martin, Andrew and Fix, Miriam}, | |
| 529 | + title = {Uncertainty in Automated Valuation Models: Error- versus Model-Based Approaches}, | |
| 530 | + journal = {Journal of Property Research}, | |
| 531 | + volume = {37}, | |
| 532 | + number = {4}, | |
| 533 | + pages = {308--339}, | |
| 534 | + year = {2020}, | |
| 535 | + doi = {10.1080/09599916.2020.1807587} | |
| 536 | +} | |
| 537 | + | |
| 538 | +@article{steurer2021, | |
| 539 | + author = {Steurer, Miriam and Hill, Robert J. and Pfeifer, Norbert}, | |
| 540 | + title = {Metrics for Evaluating the Performance of Machine Learning Based Automated Valuation Models}, | |
| 541 | + journal = {Journal of Property Research}, | |
| 542 | + volume = {38}, | |
| 543 | + number = {2}, | |
| 544 | + pages = {99--129}, | |
| 545 | + year = {2021}, | |
| 546 | + doi = {10.1080/09599916.2020.1858937} | |
| 547 | +} | |
| 548 | + | |
| 549 | +@article{lorenz2023, | |
| 550 | + author = {Lorenz, Felix and Willwersch, Jonas and Cajias, Marcelo and Fuerst, Franz}, | |
| 551 | + title = {Interpretable Machine Learning for Real Estate Market Analysis}, | |
| 552 | + journal = {Real Estate Economics}, | |
| 553 | + volume = {51}, | |
| 554 | + number = {5}, | |
| 555 | + pages = {1178--1208}, | |
| 556 | + year = {2023}, | |
| 557 | + doi = {10.1111/1540-6229.12397} | |
| 558 | +} | |
| 559 | + | |
| 560 | +% ---------------------------------------------------------------- appraiser behaviour | |
| 561 | +@article{northcraft1987, | |
| 562 | + author = {Northcraft, Gregory B. and Neale, Margaret A.}, | |
| 563 | + title = {Experts, Amateurs, and Real Estate: An Anchoring-and-Adjustment Perspective on Property Pricing Decisions}, | |
| 564 | + journal = {Organizational Behavior and Human Decision Processes}, | |
| 565 | + volume = {39}, | |
| 566 | + number = {1}, | |
| 567 | + pages = {84--97}, | |
| 568 | + year = {1987}, | |
| 569 | + doi = {10.1016/0749-5978(87)90046-X} | |
| 236 | 570 | } |
| 237 | 571 | |
| 238 | 572 | @article{diaz1990, |
@@ -242,7 +576,8 @@ | ||
| 242 | 576 | volume = {5}, |
| 243 | 577 | number = {1}, |
| 244 | 578 | pages = {1--15}, |
| 245 | − year = {1990} | |
| 579 | + year = {1990}, | |
| 580 | + doi = {10.1080/10835547.1990.12090601} | |
| 246 | 581 | } |
| 247 | 582 | |
| 248 | 583 | @article{diaz2001, |
@@ -252,17 +587,85 @@ | ||
| 252 | 587 | volume = {18}, |
| 253 | 588 | number = {2}, |
| 254 | 589 | pages = {141--148}, |
| 255 | − year = {2001} | |
| 590 | + year = {2001}, | |
| 591 | + doi = {10.1080/09599910110039897} | |
| 256 | 592 | } |
| 257 | 593 | |
| 258 | −@article{northcraft1987, | |
| 259 | − author = {Northcraft, Gregory B. and Neale, Margaret A.}, | |
| 260 | − title = {Experts, Amateurs, and Real Estate: An Anchoring-and-Adjustment Perspective on Property Pricing Decisions}, | |
| 261 | − journal = {Organizational Behavior and Human Decision Processes}, | |
| 262 | − volume = {39}, | |
| 263 | − number = {1}, | |
| 264 | − pages = {84--97}, | |
| 265 | − year = {1987} | |
| 594 | +@article{eriksen2019, | |
| 595 | + author = {Eriksen, Michael D. and Fout, Hamilton B. and Palim, Mark and Rosenblatt, Eric}, | |
| 596 | + title = {The Influence of Contract Prices and Relationships on Appraisal Bias}, | |
| 597 | + journal = {Journal of Urban Economics}, | |
| 598 | + volume = {111}, | |
| 599 | + pages = {132--143}, | |
| 600 | + year = {2019}, | |
| 601 | + doi = {10.1016/j.jue.2019.04.007} | |
| 602 | +} | |
| 603 | + | |
| 604 | +@article{calem2021, | |
| 605 | + author = {Calem, Paul S. and Kenney, Jeanna and Lambie-Hanson, Lauren and Nakamura, Leonard I.}, | |
| 606 | + title = {Appraising Home Purchase Appraisals}, | |
| 607 | + journal = {Real Estate Economics}, | |
| 608 | + volume = {49}, | |
| 609 | + number = {S1}, | |
| 610 | + pages = {134--168}, | |
| 611 | + year = {2021}, | |
| 612 | + doi = {10.1111/1540-6229.12326} | |
| 613 | +} | |
| 614 | + | |
| 615 | +% ---------------------------------------------------------------- forecast evaluation and inference | |
| 616 | +@article{diebold1995, | |
| 617 | + author = {Diebold, Francis X. and Mariano, Roberto S.}, | |
| 618 | + title = {Comparing Predictive Accuracy}, | |
| 619 | + journal = {Journal of Business \& Economic Statistics}, | |
| 620 | + volume = {13}, | |
| 621 | + number = {3}, | |
| 622 | + pages = {253--263}, | |
| 623 | + year = {1995}, | |
| 624 | + doi = {10.1080/07350015.1995.10524599} | |
| 625 | +} | |
| 626 | + | |
| 627 | +@article{white2000, | |
| 628 | + author = {White, Halbert}, | |
| 629 | + title = {A Reality Check for Data Snooping}, | |
| 630 | + journal = {Econometrica}, | |
| 631 | + volume = {68}, | |
| 632 | + number = {5}, | |
| 633 | + pages = {1097--1126}, | |
| 634 | + year = {2000}, | |
| 635 | + doi = {10.1111/1468-0262.00152} | |
| 636 | +} | |
| 637 | + | |
| 638 | +@article{hansen2005, | |
| 639 | + author = {Hansen, Peter Reinhard}, | |
| 640 | + title = {A Test for Superior Predictive Ability}, | |
| 641 | + journal = {Journal of Business \& Economic Statistics}, | |
| 642 | + volume = {23}, | |
| 643 | + number = {4}, | |
| 644 | + pages = {365--380}, | |
| 645 | + year = {2005}, | |
| 646 | + doi = {10.1198/073500105000000063} | |
| 647 | +} | |
| 648 | + | |
| 649 | +@article{giacomini2006, | |
| 650 | + author = {Giacomini, Raffaella and White, Halbert}, | |
| 651 | + title = {Tests of Conditional Predictive Ability}, | |
| 652 | + journal = {Econometrica}, | |
| 653 | + volume = {74}, | |
| 654 | + number = {6}, | |
| 655 | + pages = {1545--1578}, | |
| 656 | + year = {2006}, | |
| 657 | + doi = {10.1111/j.1468-0262.2006.00718.x} | |
| 658 | +} | |
| 659 | + | |
| 660 | +@article{hyndman2006, | |
| 661 | + author = {Hyndman, Rob J. and Koehler, Anne B.}, | |
| 662 | + title = {Another Look at Measures of Forecast Accuracy}, | |
| 663 | + journal = {International Journal of Forecasting}, | |
| 664 | + volume = {22}, | |
| 665 | + number = {4}, | |
| 666 | + pages = {679--688}, | |
| 667 | + year = {2006}, | |
| 668 | + doi = {10.1016/j.ijforecast.2006.03.001} | |
| 266 | 669 | } |
| 267 | 670 | |
| 268 | 671 | @article{cameron2008, |
@@ -272,5 +675,28 @@ | ||
| 272 | 675 | volume = {90}, |
| 273 | 676 | number = {3}, |
| 274 | 677 | pages = {414--427}, |
| 275 | − year = {2008} | |
| 678 | + year = {2008}, | |
| 679 | + doi = {10.1162/rest.90.3.414} | |
| 680 | +} | |
| 681 | + | |
| 682 | +@article{gneiting2011, | |
| 683 | + author = {Gneiting, Tilmann}, | |
| 684 | + title = {Making and Evaluating Point Forecasts}, | |
| 685 | + journal = {Journal of the American Statistical Association}, | |
| 686 | + volume = {106}, | |
| 687 | + number = {494}, | |
| 688 | + pages = {746--762}, | |
| 689 | + year = {2011}, | |
| 690 | + doi = {10.1198/jasa.2011.r10138} | |
| 691 | +} | |
| 692 | + | |
| 693 | +@article{lei2018, | |
| 694 | + author = {Lei, Jing and G'Sell, Max and Rinaldo, Alessandro and Tibshirani, Ryan J. and Wasserman, Larry}, | |
| 695 | + title = {Distribution-Free Predictive Inference for Regression}, | |
| 696 | + journal = {Journal of the American Statistical Association}, | |
| 697 | + volume = {113}, | |
| 698 | + number = {523}, | |
| 699 | + pages = {1094--1111}, | |
| 700 | + year = {2018}, | |
| 701 | + doi = {10.1080/01621459.2017.1307116} | |
| 276 | 702 | } |
modified
paper/sections/conclusion.tex
+4 −4
@@ -3,10 +3,10 @@ | ||
| 3 | 3 | \label{sec:conclusion} |
| 4 | 4 | % ============================================================================ |
| 5 | 5 | |
| 6 | −The sales comparison approach is usually defended on the grounds of its logic and its transparency and criticised on the grounds of its subjectivity. This paper has asked a different question: taken as a set of explicit rules, how accurate is it, and which rules are empirically justified? Using the universe of single-family transactions in Quebec and a backtest that respects the information available on each valuation date, we have measured the accuracy of \nConfigs{} configurations of the method and of the natural alternatives on \nTargetsTest{} out-of-sample sales. | |
| 6 | +The sales comparison approach is usually defended on the grounds of its logic and its transparency and criticised on the grounds of its subjectivity. This paper has asked a different question: taken as a set of explicit rules, how accurate is it, and which rules are empirically justified? Using the universe of single-family transactions in Quebec and a backtest that respects the information available on each valuation date, we have measured the accuracy of \nConfigs{} configurations of the method and of the natural alternatives on \nTargetsTest{} out-of-sample sales, with configurations chosen on a separate year and differences tested against the spatial and temporal dependence of valuation errors. | |
| 7 | 7 | |
| 8 | −The answers are clear enough to be taught. Adjust: a grid---even a crude one---is worth several points of error. Choose similar comparables, not merely near ones, and search widely in space and time. Use more comparables than three: twenty or more when a grid is applied, in sparse and dense markets alike, and about ten when it is not. Do not agonise over weights or over the choice between median and mean. Use the assessed value: the ratio method is the most accurate single adjustment scheme where it applies. And report an interval, calibrated so that it means what it says---and be prepared for it to be wide. A transparent method configured on these principles and frozen in 2024 reaches a median absolute error of \mdapeMzero{} on 2025--2026 sales, within \transparencyCost{} points of the best quarterly-retrained machine-learning model given the same data, and within less than that in homogeneous submarkets. | |
| 8 | +The answers are clear enough to be taught. Adjust: a grid---even a crude one---is worth four to six points of error. Choose similar comparables, not merely near ones, and search widely in space and time. Use more comparables than three: twenty or more when a grid is applied, in sparse and dense markets alike, and about ten when it is not. Do not agonise over weights or over the choice between median and mean. Use the assessed value: the ratio method is the most accurate single adjustment scheme where it applies. And report an interval, calibrated so that it means what it says---and be prepared for it to be wide. A transparent method configured on these principles and frozen in 2024 reaches a median absolute error of \mdapeMzero{} on 2025--2026 sales, within \transparencyCost{} points of the best quarterly-retrained machine-learning model given the same data, and within about a point and a half in homogeneous submarkets. The choice of that configuration is robust: the ten best configurations on 2024 differ only in the number of comparables and the radius, and all rank among the twelve best out of sample. | |
| 9 | 9 | |
| 10 | −The results have implications for practice, for education and for regulation. For practice, they suggest that the appraiser's effort is best spent on selecting similar comparables and on estimating adjustments from the largest homogeneous pool of evidence, rather than on reconciling a handful of nearby sales by judgement. For education, they provide a set of validated rules that can be taught alongside the conventions, and a laboratory---the backtesting engine---in which students can test their own. For regulation and for the design of automated valuation models, they show that a comparable-based method can be made as accurate as an opaque model in most of the market while remaining fully explainable, and that a wide-net rule retaining the most similar sales adapts to market density on its own. | |
| 10 | +The results have implications for practice, for education and for regulation. For practice, they suggest that the appraiser's effort is best spent on selecting similar comparables and on estimating adjustments from the largest homogeneous pool of evidence, rather than on reconciling a handful of nearby sales by judgement. For education, they provide a set of validated rules that can be taught alongside the conventions, and a laboratory---the backtesting engine---in which students can test their own. For regulation and for the design of automated valuation models, they show that a comparable-based method can be made nearly as accurate as an opaque model in most of the market while remaining fully explainable, and that a wide-net rule retaining the most similar sales adapts to market density on its own. | |
| 11 | 11 | |
| 12 | −Three limitations bound the conclusions. We simulate rules; we do not observe appraisers, whose files may contain information (condition, renovations, listing history) that would improve on any rule, and whose judgement may also degrade it. We observe the characteristics of the assessment roll, not those of listings or inspections, so that adjustments for garages, pools, bedrooms and quality are untested and their effects remain in the residual error. And we study single-family dwellings in one province; condominiums, income properties and other markets may yield different optima, though we expect the qualitative hierarchy---adjustment and selection first, number and radius second, weighting last---to carry over. A companion paper develops the recommendation system of Section~\ref{sec:recommend} into a transparent, empirically optimised comparable-sales algorithm and integrates it into a teaching platform. | |
| 12 | +Four limitations bound the conclusions. We simulate rules; we do not observe appraisers, whose files may contain information (condition, renovations, listing history) that would improve on any rule, and whose judgement may also degrade it. We observe the characteristics of the assessment roll, not those of listings or inspections, so that adjustments for garages, pools, bedrooms and quality are untested and their effects remain in the residual error, which the within-municipality $R^2$ of about a third makes visible. We screen non-arm's-length transfers with a price-to-assessment ratio, date sales by their registration and ignore the publication lag of the register, three choices that favour every method alike but that a user of the rules in real time would have to confront. And we study single-family dwellings in one province; condominiums, income properties and other markets may yield different optima, though we expect the qualitative hierarchy---adjustment and selection first, number and radius second, weighting last---to carry over. Two extensions follow directly from the results and remain to be done: a seasonal correction of the comparables estimate, which Section~\ref{sec:stability} suggests would remove a bias of one to two points in the first and third quarters, and a re-execution of the backtest under alternative screening rules for the comparable pool. A companion project develops the recommendation system of Section~\ref{sec:recommend} into a transparent, empirically optimised comparable-sales algorithm and integrates it into a teaching platform. | |
modified
paper/sections/data.tex
+11 −7
@@ -5,15 +5,15 @@ | ||
| 5 | 5 | |
| 6 | 6 | \subsection{The Vrai Prix transaction database} |
| 7 | 7 | |
| 8 | −Our data come from Vrai Prix, a research database that assembles every residential transfer registered in the Quebec land register (\emph{Registre foncier du Québec}) since January 2021 and links each transfer to the corresponding unit of the municipal assessment roll (\emph{rôle d'évaluation foncière}). The snapshot used here was taken on 10 October 2026 and contains \nTransactionsDb{} transactions and \nUnits{} assessment units covering the whole province. For each transaction we observe the registration date, the price, the civic address and coordinates, the property type, and---through the link to the roll---the physical characteristics recorded by the municipal assessor: living area (sum of floor areas), lot area, year of construction, number of storeys, physical link (detached, semi-detached, row, integrated), construction type, the neighbourhood unit used by the assessor, and the assessed value in force at the date of sale together with its land and building components. Assessed values are revised on a three-year cycle that differs across municipalities; we record, for each sale, whether the assessed value in force belongs to the current roll, the previous roll, or an older one. | |
| 8 | +Our data come from Vrai Prix, a research database that assembles every residential transfer registered in the Quebec land register (\emph{Registre foncier du Québec}) since January 2021 and links each transfer to the corresponding unit of the municipal assessment roll (\emph{rôle d'évaluation foncière}). The snapshot used here was taken on 10 October 2026 and contains \nTransactionsDb{} transactions and \nUnits{} assessment units covering the whole province. For each transaction we observe the registration date, the price, the civic address and coordinates, the property type, and---through the link to the roll---the physical characteristics recorded by the municipal assessor: living area (sum of floor areas), lot area, year of construction, number of storeys, physical link (detached, semi-detached, row, integrated), construction type, the neighbourhood unit used by the assessor, and the assessed value in force at the date of sale together with its land and building components. Assessed values are revised on a three-year cycle that differs across municipalities; we record, for each sale, whether the assessed value in force belongs to the roll that was current at the snapshot date, to the previous roll, or to an older one, and we call this the roll \emph{vintage}. | |
| 9 | 9 | |
| 10 | −Two features of this database are essential for our purpose. First, it is a population, not a sample: every registered sale is present, so that the comparable pool available on any valuation date is the one an appraiser with full access to the register would have had. Second, the characteristics are those of the assessment roll, harmonised across the province by the \emph{Manuel d'évaluation foncière du Québec}, rather than listing descriptions whose coding varies by broker. The price of this harmonisation is that some characteristics commonly adjusted for in practice---garage, pool, number of bedrooms and bathrooms, interior quality, condition, renovations---are not recorded in the roll extract and are therefore unavailable. We return to this limitation in Section~\ref{sec:conclusion}. | |
| 10 | +Two features of this database are essential for our purpose. First, it is a population, not a sample: every registered sale is present, so that the comparable pool available on any valuation date is the one an appraiser with full access to the register would have had. We date each sale by its registration date and ignore the lag between the deed and its publication in the register, which slightly favours every method in the same way. Second, the characteristics are those of the assessment roll, harmonised across the province by the \emph{Manuel d'évaluation foncière du Québec}, rather than listing descriptions whose coding varies by broker. The price of this harmonisation is that some characteristics commonly adjusted for in practice---garage, pool, number of bedrooms and bathrooms, interior quality, condition, renovations---are not recorded in the roll extract and are therefore unavailable. We return to this limitation in Section~\ref{sec:conclusion}. | |
| 11 | 11 | |
| 12 | 12 | \subsection{Sample construction} |
| 13 | 13 | |
| 14 | 14 | We restrict attention to single-family dwellings, the segment for which the sales comparison approach is the reference method. Starting from \nSalesRaw{} transactions coded as single-family, we apply a sequence of explicit exclusion rules, each logged with the number of observations it removes (Table~\ref{tab:exclusions}). The rules remove transactions that cannot be matched to the roll; duplicates; units whose roll record is not a single dwelling; implausible or missing living area, lot area or year of construction; sales closed before the dwelling was built (which are land sales) or in the year of construction (whose prices are, on inspection, those of lots or unfinished buildings: a median of \$346{,}108 and a price-to-assessment ratio of 1.15, against \$450{,}000--480{,}000 and 1.30 for dwellings aged one to four years); prices outside \$50{,}000--\$3{,}000{,}000; and resales within 180 days with a price change above 40\%, which signal renovations or flips for which the recorded characteristics no longer describe the dwelling. |
| 15 | 15 | |
| 16 | −The most consequential rule is the exclusion of sales whose price-to-assessment ratio lies outside $[0.33, 3]$. The register does not flag non-arm's-length transfers (intra-family sales, transfers at nominal prices, distress sales), and an extreme ratio is the only observable indicator. An appraiser would screen such sales out of the comparable pool; we therefore exclude them both as targets and as comparables, and report the count. All other rules are standard. The final sample comprises \nSalesClean{} sales between \dateMin{} and \dateMax{}, with a median price of \$\medianPrice{}. | |
| 16 | +The most consequential rule is the exclusion of sales whose price-to-assessment ratio lies outside $[0.33, 3]$. The register does not flag non-arm's-length transfers (intra-family sales, transfers at nominal prices, distress sales), and an extreme ratio is the only observable indicator. An appraiser would screen such sales out of the comparable pool; we therefore exclude them both as targets and as comparables, and report the count. Because the rule uses the price, it also removes some genuine but unusual sales, and Section~\ref{sec:robustness} examines how the results depend on its bounds. All other rules are standard. The final sample comprises \nSalesClean{} sales between \dateMin{} and \dateMax{}, with a median price of \$\medianPrice{}. | |
| 17 | 17 | |
| 18 | 18 | \begin{table}[H] |
| 19 | 19 | \centering |
@@ -24,27 +24,31 @@ The most consequential rule is the exclusion of sales whose price-to-assessment | ||
| 24 | 24 | |
| 25 | 25 | \subsection{Chronological partition and descriptive statistics} |
| 26 | 26 | |
| 27 | −The sample is partitioned chronologically. Sales from January 2021 to December 2023 (\nTrain{} observations) serve to develop the adjustment grids and benchmark models. Sales in 2024 (\nVal{} observations) are the \emph{validation} targets, on which configurations and hyperparameters are chosen. Sales from January 2025 to September 2026 (\nTest{} observations) are the \emph{test} targets, on which all results reported as out of sample are computed. Every sale in the sample, whatever its period, is eligible as a comparable for any later target; what distinguishes the periods is the role of the sale as a target. | |
| 27 | +The sample is partitioned chronologically. Sales from January 2021 to December 2023 (\nTrain{} observations) serve to develop the adjustment grids and benchmark models. Sales in 2024 (\nVal{} observations) are the \emph{validation} targets, on which configurations and hyperparameters are chosen. Sales from January 2025 to September 2026 (\nTest{} observations) are the \emph{test} targets, on which all results reported as out of sample are computed. Every sale in the sample, whatever its period, is eligible as a comparable for any later target, and the grids and benchmark models are re-estimated each quarter on all prior sales, including those of the validation period once it has elapsed; what distinguishes the periods is the role of the sale as a target. | |
| 28 | 28 | |
| 29 | −Table~\ref{tab:descriptives} reports descriptive statistics by period and Figure~\ref{fig:index} shows the hedonic price index by region over the sample. The period covers the end of the pandemic boom, a correction in 2022--2023 and a renewed rise in 2024--2026 that is strongly heterogeneous across regions: over 2025--2026 prices in the Montréal region rose by roughly a third of the increase recorded in the Quebec City region and half of that in the Montérégie. This heterogeneity is a feature, not a nuisance: it is precisely the environment in which the quality of time adjustments and the locality of comparables matter. | |
| 29 | +Table~\ref{tab:descriptives} reports descriptive statistics by period and Figure~\ref{fig:index} shows the hedonic price index by region over the sample. Prices rose steeply until mid-2022, fell by about \idxDrawdown{} to a trough in early 2023, and rose again from 2024 onwards; over the whole period the provincial index gained \idxProvAll{}. The recovery was strongly heterogeneous across regions (Table~\ref{tab:index_growth} in the Appendix): between December 2024 and September 2026 prices rose by about \idxQc{} in the Capitale-Nationale and \idxMonteregie{} in the Montérégie, against \idxMtl{} on the island of Montréal, whose cumulative gain since 2021 (\idxMtlAll{}) is less than half the provincial one, while the Estrie more than doubled (\idxEstrieAll{}). This heterogeneity is a feature, not a nuisance: it is precisely the environment in which the quality of time adjustments and the locality of comparables matter. The median price-to-assessment ratio also falls from 1.39 in 2021--2023 to 1.20 in 2025--2026 as the 2025 and 2026 rolls incorporate the boom, which is why the assessment-ratio adjustment must be restricted to comparables on the same roll (Section~\ref{sec:config}). | |
| 30 | 30 | |
| 31 | 31 | \begin{table}[H] |
| 32 | 32 | \centering |
| 33 | 33 | \caption{Descriptive statistics by period} |
| 34 | 34 | \label{tab:descriptives} |
| 35 | 35 | \input{tables/descriptives} |
| 36 | +\begin{tablenotes}\footnotesize | |
| 37 | +\item Note: ``Sales within 2~km, 12 mo.'' is the number of sample sales within 2~km of the target in the twelve months preceding its date (the density measure of Section~\ref{sec:density}); ``candidate pool'' is the number of prior sales within 25~km and 24 months. Both are defined for targets only. | |
| 38 | +\end{tablenotes} | |
| 36 | 39 | \end{table} |
| 37 | 40 | |
| 38 | 41 | \begin{figure}[H] |
| 39 | 42 | \centering |
| 40 | 43 | \includegraphics[width=0.85\textwidth]{f7_indice.pdf} |
| 41 | −\caption{Hedonic price index by region, full sample (descriptive). Monthly effects from a within-municipality regression of log price on characteristics; January 2021 $=100$. The point-in-time indices used in the backtest are re-estimated every month from prior sales only.} | |
| 44 | +\caption{Hedonic price index, province and selected regions, full sample (descriptive). Monthly effects from a within-municipality regression of log price on characteristics, January 2021 $=100$. The point-in-time indices used in the backtest are re-estimated every month from prior sales only and are not shown.} | |
| 42 | 45 | \label{fig:index} |
| 43 | 46 | \end{figure} |
| 44 | 47 | |
| 45 | 48 | \subsection{Market density} |
| 49 | +\label{sec:density} | |
| 46 | 50 | |
| 47 | −Because the accuracy of any comparable-based method depends on the supply of comparables, we characterise each target by the number of sample sales that occurred within 2~km during the twelve months preceding its date. This \emph{density} measure is computed from the target's own comparable pool and is therefore available at valuation time. We group targets into five strata---fewer than 5, 5--19, 20--49, 50--199 and 200 or more sales---which range from rural municipalities and small towns to the dense suburbs of Montréal and Quebec City. Table~\ref{tab:density_dist} gives the distribution of test targets across strata. | |
| 51 | +Because the accuracy of any comparable-based method depends on the supply of comparables, we characterise each target by the number of sample sales that occurred within 2~km during the twelve months preceding its date, excluding prior sales of the subject itself. This \emph{density} measure is computed from the target's own comparable pool and is therefore available at valuation time. We group targets into five strata---fewer than 5, 5--19, 20--49, 50--199 and 200 or more sales---which range from rural municipalities and small towns to the dense suburbs of Montréal and Quebec City. Table~\ref{tab:density_dist} gives the distribution of test targets across strata; a fifth of the test targets are in markets with fewer than twenty sales within 2~km in a year. | |
| 48 | 52 | |
| 49 | 53 | \begin{table}[H] |
| 50 | 54 | \centering |
modified
paper/sections/introduction.tex
+6 −6
@@ -3,14 +3,14 @@ | ||
| 3 | 3 | \label{sec:introduction} |
| 4 | 4 | % ============================================================================ |
| 5 | 5 | |
| 6 | −The sales comparison approach is the workhorse of residential property valuation. Appraisers, assessors, lenders and courts rely on it daily: a small set of recent sales of similar properties is selected, each sale price is adjusted for the differences between the comparable and the subject, and the adjusted prices are reconciled into a single opinion of value \citep{appraisal2020,iaao2017}. The approach is taught in every valuation curriculum and codified in professional standards, and its logic---that the market itself reveals what buyers pay for a bundle of characteristics---is intuitively compelling. Yet the \emph{operating rules} that turn this logic into a number are, to a remarkable degree, matters of convention rather than evidence. How many comparables should be used? Three is the traditional answer; some standards say ``at least three.'' How far should one search, and how far back in time? Which adjustments are warranted, and of what magnitude? Should comparables be weighted equally, by proximity, by recency, or by similarity? Should adjusted prices be averaged, or should the appraiser ``reconcile'' them by judgement? Each of these questions has been debated in the professional and academic literature for four decades \citep{colwell1983,vandell1991,gau1992,isakson2002}, but the empirical evidence remains thin, dated, and almost always drawn from small samples in a single metropolitan area. | |
| 6 | +The sales comparison approach is the workhorse of residential property valuation. Appraisers, assessors, lenders and courts rely on it daily: a small set of recent sales of similar properties is selected, each sale price is adjusted for the differences between the comparable and the subject, and the adjusted prices are reconciled into a single opinion of value \citep{appraisal2020,iaao2017}. The approach is taught in every valuation curriculum and codified in professional standards, and its logic---that the market itself reveals what buyers pay for a bundle of characteristics---is compelling. Yet the \emph{operating rules} that turn this logic into a number are, to a remarkable degree, matters of convention rather than evidence. How many comparables should be used? Three is the traditional answer; some standards say ``at least three.'' How far should one search, and how far back in time? Which adjustments are warranted, and of what magnitude? Should comparables be weighted equally, by proximity, by recency, or by similarity? Should adjusted prices be averaged, or should the appraiser ``reconcile'' them by judgement? Each of these questions has been debated for four decades \citep{colwell1983,vandell1991,gau1992,pace1996,isakson2002,lai2008}, but the empirical evidence remains thin, dated, and almost always drawn from a few hundred sales in a single metropolitan area. | |
| 7 | 7 | |
| 8 | −This paper provides, to our knowledge, the largest and most systematic empirical validation of the sales comparison approach to date. We exploit a new research database---Vrai Prix---that links the universe of registered residential transactions in Quebec to the municipal assessment roll, giving us \nSalesClean{} arm's-length single-family sales from January 2021 to September 2026 with harmonised physical characteristics, coordinates, and assessed values. On this base we build a \emph{point-in-time backtesting} engine. For each of \nTargetsAll{} target sales occurring in 2024, 2025 and 2026, the sale price is masked and the property is valued as an appraiser would have had to value it on the day of the sale: only transactions closed strictly before that date are admissible as comparables; the price index used to time-adjust them is re-estimated every month from prior sales only; the adjustment grids and machine-learning benchmarks are re-estimated every quarter from prior sales only. The masked price is then compared with the estimate. Because every configuration of the method is applied to exactly the same targets, differences in accuracy are paired differences, and we quantify their uncertainty with a cluster bootstrap over municipality--quarter cells that respects spatial and temporal dependence. | |
| 8 | +This paper provides a population-scale, out-of-sample validation of the sales comparison approach. We exploit a research database---Vrai Prix---that links the universe of registered residential transactions in Quebec to the municipal assessment roll, giving us \nSalesClean{} arm's-length single-family sales from January 2021 to September 2026 with harmonised physical characteristics, coordinates and assessed values. On this base we build a \emph{point-in-time backtesting} engine. For each of \nTargetsAll{} target sales occurring in 2024, 2025 and 2026, the sale price is masked and the property is valued as an appraiser would have had to value it on the day of the sale: only transactions closed strictly before that date are admissible as comparables; the price index used to time-adjust them is re-estimated every month from prior sales only; the adjustment grids and machine-learning benchmarks are re-estimated every quarter from prior sales only. The masked price is then compared with the estimate. Because every configuration of the method is applied to exactly the same targets, differences in accuracy are paired differences of forecast losses in the sense of \citet{diebold1995}, and we quantify their uncertainty with a cluster bootstrap over municipality--quarter cells \citep{cameron2008} that respects the spatial and temporal dependence of valuation errors. Because configurations are chosen on 2024 sales and evaluated on 2025--2026 sales, the headline figures are protected against the data-snooping bias that a search over thousands of specifications would otherwise create \citep{white2000,hansen2005}; we verify directly that the configuration chosen on 2024 ranks \mZeroRankTest{}nd among the \nConfigsCov{} eligible configurations in 2025--2026. | |
| 9 | 9 | |
| 10 | −Within this engine we run \nConfigs{} configurations of the sales comparison approach. A first experiment maps the \emph{error surface} $\mathrm{Error}(k, d, t)$ over the number of comparables $k \in \{1,\dots,50\}$, the search radius $d \in \{0.5,\dots,25\}$~km and the look-back window $t \in \{3,\dots,24\}$ months, for three selection rules (nearest, most physically similar, and a compromise) and two adjustment regimes (time only, hedonic grid). A second experiment is a full factorial design over the \emph{rules} of the method at three anchor configurations: four selection criteria, eight adjustment schemes (none; time only; a stylised conventional grid; a global hedonic grid; a regional grid; a locally re-estimated grid; non-linear adjustments from a gradient-boosting model; and the assessment-ratio adjustment), five weighting schemes, and two reconciliation rules. Thirteen named models, from ``median of the three nearest sales'' to a quarterly-retrained gradient-boosting model and a hybrid, are then placed in a horse race, and five hypotheses about the trade-offs between proximity, similarity, sample size, local estimation, weighting and market liquidity are tested directly. | |
| 10 | +Within this engine we run \nConfigs{} configurations of the sales comparison approach. A first experiment maps the \emph{error surface} $\mathrm{Error}(k, d, t)$ over the number of comparables $k \in \{1,\dots,50\}$, the search radius $d \in \{0.5,\dots,25\}$~km and the look-back window $t \in \{3,\dots,24\}$ months, for three selection rules (nearest, most physically similar, and a compromise) and two adjustment regimes (time only, hedonic grid). A second experiment is a full factorial design over the \emph{rules} of the method at three anchor configurations: four selection criteria, eight adjustment schemes (none; time only; a stylised conventional grid; a global hedonic grid; a regional grid; a locally re-estimated grid; non-linear adjustments from a gradient-boosting model; and the assessment-ratio adjustment), five weighting schemes, and two reconciliation rules. Sixteen named models, from ``median of the three nearest sales'' to a quarterly-retrained gradient-boosting model and a hybrid, are then placed in a horse race, and five hypotheses about the trade-offs between proximity, similarity, sample size, local estimation, weighting and market liquidity are tested directly. | |
| 11 | 11 | |
| 12 | −Five results stand out. First, \emph{adjustments dominate}. Averaging the five nearest time-adjusted sales yields a median absolute percentage error (MdAPE) of \mdapeMtwo{}; applying a hedonic adjustment grid to the same five sales cuts it to \mdapeMseven{}, and a stylised conventional grid---half the comparable's price per square metre for living area, half the implied land value per square metre for lot size, half a percent per year of age---captures most of that gain (\mdapeMsix{}). What matters is \emph{that} one adjusts, far more than the sophistication of the grid: regional, local and non-linear refinements add little, and the locally re-estimated grid is unstable where sales are sparse. Second, \emph{similarity beats proximity}. With no physical adjustment at all, selecting the five most similar sales within a wide radius rather than the five nearest yields \mdapeMthree{}, essentially the same accuracy as adjusting the nearest ones. A nearby but dissimilar sale is a worse comparable than a more distant but similar one. Third, \emph{three comparables is too few}. Without physical adjustment, accuracy improves up to about ten comparables and then deteriorates as more distant sales are added; with a grid it improves up to roughly \kOpt{} and stays flat to fifty, in sparse and dense markets alike. Three-month windows are dominated everywhere, whereas the search radius, once enough similar sales are retained, matters little for accuracy. Fourth, \emph{weighting and reconciliation matter little}: the choice between median and mean, or between equal, distance, recency and similarity weights, moves the MdAPE by a few tenths of a point, an order of magnitude less than selection and adjustment. Fifth, \emph{the cost of transparency is small}. The best configuration of the method chosen on 2024 data reaches \mdapeMzero{} out of sample in 2025--2026, against \mdapeMelevenB{} for a gradient-boosting model retrained every quarter with access to the assessed value, and \mdapeMtwelve{} for a hybrid that combines the two. In homogeneous submarkets the gap narrows to about a point. Accuracy is, above all, a function of submarket liquidity: the same method is two and a half times more accurate where hundreds of sales occur within two kilometres in a year than where fewer than five do. A wide-net rule that retains the most similar sales within a large radius adapts to this on its own, and rules tuned stratum by stratum do not improve on it. | |
| 12 | +Five results stand out. First, \emph{adjustments dominate}. Averaging the five nearest time-adjusted sales yields a median absolute percentage error (MdAPE) of \mdapeMtwo{}; applying a hedonic adjustment grid to the same five sales cuts it to \mdapeMseven{}, and a stylised conventional grid---half the comparable's price per square metre for living area, half the implied land value per square metre for lot size, half a percent per year of age---captures most of that gain (\mdapeMsix{}). What matters is \emph{that} one adjusts, far more than the sophistication of the grid: regional, local and non-linear refinements add little, and the locally re-estimated grid is less accurate where sales are sparse. Second, \emph{similarity beats proximity}. With no physical adjustment at all, selecting the five most similar sales within a wide radius rather than the five nearest yields \mdapeMthree{}, essentially the same accuracy as adjusting the nearest ones. A nearby but dissimilar sale is a worse comparable than a more distant but similar one. Third, \emph{three comparables is too few}. Without physical adjustment, accuracy improves up to about ten comparables and then deteriorates as more distant sales are added; with a grid it improves up to roughly \kOpt{} and stays flat to fifty, in sparse and dense markets alike. Three-month windows are dominated everywhere, whereas the search radius, once enough similar sales are retained, matters little. Fourth, \emph{weighting and reconciliation matter little}: the choice between median and mean, or between equal, distance, recency and similarity weights, moves the MdAPE by a few tenths of a point, an order of magnitude less than selection and adjustment. Fifth, \emph{the cost of transparency is small}. The best configuration of the method chosen on 2024 data reaches \mdapeMzero{} out of sample in 2025--2026, against \mdapeMelevenB{} for a gradient-boosting model retrained every quarter with access to the assessed value, and \mdapeMtwelve{} for a hybrid that combines the two; the paired difference of \transparencyCost{} points is precisely estimated, and it falls to about a point and a half in homogeneous submarkets. Accuracy is, above all, a function of submarket liquidity: the same method is two and a half times more accurate where hundreds of sales occur within two kilometres in a year than where fewer than five do. A wide-net rule that retains the most similar sales within a large radius adapts to this on its own, and rules tuned stratum by stratum do not improve on it. | |
| 13 | 13 | |
| 14 | −The paper contributes to three literatures. It extends the small-sample studies of comparable selection and weighting \citep{colwell1983,vandell1991,gau1992,gau1994,isakson2002} to a population-scale, out-of-sample, point-in-time setting, and resolves several of their open questions with confidence intervals rather than illustrations. It speaks to the literature comparing hedonic and grid-adjustment techniques \citep{kang1991,lentz1998,pace1998} and the more recent comparisons of machine learning with traditional valuation \citep{mccluskey2013,mayer2019,bourassa2010,pace2020}, by holding the information set fixed across methods and measuring, for the first time, the price of interpretability in a transparent method tuned to the same data. And it offers the profession and its regulators a reproducible empirical basis for rules that are currently taught as conventions. We are careful about what the evidence can and cannot say: we validate the \emph{predictive accuracy of simulated professional rules} on observable characteristics, not the judgement of actual appraisers, whose files, comparables and adjustments we do not observe. | |
| 14 | +The paper contributes to three literatures. It extends the theoretical and small-sample studies of comparable selection and weighting \citep{colwell1983,vandell1991,gau1992,green1994,pace1996,epley1997,lai2008} to a population-scale, out-of-sample, point-in-time setting, and resolves several of their open questions with confidence intervals rather than illustrations. It speaks to the literature comparing hedonic and grid-adjustment techniques \citep{kang1991,lipscomb1990,pacegilley1998} and to the comparisons of machine learning with traditional valuation \citep{mccluskey2013,mayer2019,bogin2020,pace2020}, by holding the information set fixed across methods and measuring the price of interpretability in a transparent method tuned to the same data. And it offers the profession and its regulators a reproducible empirical basis for rules that are currently taught as conventions. We are careful about what the evidence can and cannot say: we validate the \emph{predictive accuracy of simulated professional rules} on observable characteristics, not the judgement of actual appraisers, whose files, comparables and adjustments we do not observe. | |
| 15 | 15 | |
| 16 | −The remainder of the paper is organised as follows. Section~\ref{sec:literature} reviews the literature on the sales comparison approach, adjustment estimation and automated valuation. Section~\ref{sec:data} describes the Vrai Prix database and the construction of the analysis sample. Section~\ref{sec:methodology} presents the point-in-time backtesting protocol, the configuration space, the benchmark models and the inference strategy. Section~\ref{sec:results} reports the results, Section~\ref{sec:robustness} examines their stability across submarkets, time and specifications, and Section~\ref{sec:conclusion} concludes. | |
| 16 | +The remainder of the paper is organised as follows. Section~\ref{sec:literature} reviews the literature on the sales comparison approach, adjustment estimation and automated valuation. Section~\ref{sec:data} describes the Vrai Prix database and the construction of the analysis sample. Section~\ref{sec:methodology} presents the point-in-time backtesting protocol, the configuration space, the benchmark models and the inference strategy. Section~\ref{sec:results} reports the results, Section~\ref{sec:robustness} examines their stability across submarkets, time, specifications and the selection step itself, and Section~\ref{sec:conclusion} concludes. | |
modified
paper/sections/literature.tex
+9 −7
@@ -5,22 +5,24 @@ | ||
| 5 | 5 | |
| 6 | 6 | \subsection{The sales comparison approach and its conventions} |
| 7 | 7 | |
| 8 | −The sales comparison approach rests on the principle of substitution: a buyer will pay no more for a property than the cost of acquiring an equally desirable substitute \citep{appraisal2020}. Its implementation follows a sequence that has barely changed since the method was formalised: identify comparable sales, verify that they are arm's-length, adjust each sale price for differences in transaction conditions, market conditions (time), location and physical characteristics, and reconcile the adjusted indications into a point estimate. Professional texts and standards give guidance on each step but rarely quantify it. The Appraisal Institute recommends ``a sufficient number'' of comparables and treats three as a practical minimum; the International Association of Assessing Officers' standard on automated valuation models recommends that comparable-sales algorithms use ``three to ten'' sales and weight them by similarity \citep{iaao2018}. Adjustments are to be ``market-derived,'' typically through paired-sales analysis, but in practice they are frequently taken from rules of thumb, published grids, or the appraiser's experience \citep{diaz1990,diaz2001}. | |
| 8 | +The sales comparison approach rests on the principle of substitution: a buyer will pay no more for a property than the cost of acquiring an equally desirable substitute \citep{appraisal2020}. Its implementation follows a sequence that has barely changed since the method was formalised: identify comparable sales, verify that they are arm's-length, adjust each sale price for differences in transaction conditions, market conditions (time), location and physical characteristics, and reconcile the adjusted indications into a point estimate. Professional texts and standards give guidance on each step but rarely quantify it. The Appraisal Institute recommends ``a sufficient number'' of comparables and treats three as a practical minimum; the International Association of Assessing Officers' standard on automated valuation models recommends that comparable-sales algorithms use ``three to ten'' sales and weight them by similarity \citep{iaao2018}. Adjustments are to be ``market-derived,'' typically through paired-sales analysis, but in practice they are frequently taken from rules of thumb, published grids, or the appraiser's experience \citep{diaz1990,diaz2001}. \citet{lipscomb1990} compared four market-derived adjustment methods on a single-market sample and found that the choice of method materially changes the adjustments, which is itself an argument for testing them. | |
| 9 | 9 | |
| 10 | −A long-standing academic literature has questioned whether these conventions are optimal. \citet{colwell1983} formalised the approach as a statistical estimator and showed that the familiar adjustment grid is a special case of a regression in which the coefficients are fixed a priori. \citet{vandell1991} derived the optimal weights to assign to comparables as a function of their adjustment variance, and showed that equal weighting is optimal only under restrictive conditions. \citet{gau1992,gau1994} extended this to the joint problem of selecting and weighting comparables, proposing a minimum-variance criterion that trades off the number of comparables against their heterogeneity. \citet{isakson1986,isakson2002} examined the nearest-neighbour logic of comparable selection and the role of distance metrics in characteristic space. \citet{lentz1998} survey the appraisal issues raised by the lending process, including the reliance on a small number of comparables and on judgemental adjustments. These contributions are theoretical or illustrated on small samples---typically a few hundred sales in one city---and none evaluates the rules out of sample on a large, heterogeneous market. Our design implements their estimators (nearest, most similar, minimum-adjustment selection; equal, distance, recency and similarity weights; median and mean reconciliation) as configurations of a single engine and measures their accuracy on hundreds of thousands of masked sales. | |
| 10 | +A long-standing academic literature has treated the approach as a statistical estimator and questioned whether its conventions are optimal. \citet{colwell1983} showed that the familiar adjustment grid is a special case of a regression in which the coefficients are fixed a priori, so that its accuracy depends on how close the pre-set coefficients are to the implicit prices. \citet{isakson1986,isakson1988} proposed a nearest-neighbour technique in characteristic space as an alternative to the grid, and \citet{isakson2002} set out the linear algebra that unifies the two. \citet{quan1991} modelled the appraiser as a Bayesian signal extractor who combines noisy comparable prices with a prior; in that framework the number of comparables and their weights follow from the relative variances of the signals. \citet{vandell1991} derived the optimal weights to assign to comparables as a function of their adjustment variance and showed that equal weighting is optimal only under restrictive conditions; \citet{gau1992} extended the problem to the joint selection and weighting of comparables under a minimum-variance criterion, which trades off the number of comparables against their heterogeneity, and the exchange with \citet{green1994} and \citet{gau1994} clarified the role of the covariance between adjustment errors. \citet{epley1997} noted that these optimal rules require variance estimates that practitioners rarely possess. \citet{pace1996} is the closest antecedent to our design: he interprets the grid estimator as a combination of a nearest-neighbour estimator and a parametric estimator and shows, analytically and by Monte Carlo simulation, that ordinary least squares dominates the grid, which dominates the nearest neighbour, under correct specification, and that the ranking is fragile under misspecification; \citet{pacegilley1998} generalise the two estimators into a single family. \citet{lai2008} propose a ``replication'' estimator that selects comparables so as to reproduce the subject's characteristics in combination rather than one at a time. \citet{lentz1998} survey the appraisal issues raised by the lending process, including the reliance on a small number of comparables and on judgemental adjustments. These contributions are theoretical, simulated, or illustrated on a few hundred sales in one city; none evaluates the rules out of sample on a large, heterogeneous market. Our design implements their estimators (nearest, most similar, minimum-adjustment selection; equal, distance, recency and similarity weights; median and mean reconciliation) as configurations of a single engine and measures their accuracy on more than a hundred thousand masked sales. | |
| 11 | 11 | |
| 12 | 12 | \subsection{Estimating adjustments: grids, hedonics and local models} |
| 13 | 13 | |
| 14 | −The estimation of adjustment magnitudes is where the sales comparison approach meets hedonic price theory. Since \citet{rosen1974}, the hedonic regression of price on characteristics has been the canonical tool for recovering implicit prices, and its coefficients are natural candidates for adjustment factors \citep{kang1991,pace1998}. \citet{kang1991} compared hedonic regression with grid-adjustment techniques on sales in a single county and found hedonic adjustments superior; \citet{pace1998} and \citet{pace1997} argued for non-parametric and spatial estimation of adjustments. More recent work has emphasised spatial heterogeneity in implicit prices \citep{bourassa2010,case2004} and the use of geographically weighted or locally estimated models. Whether locally estimated adjustments improve on a global grid in practice is an empirical question that depends on the bias--variance trade-off: local models track heterogeneity but are noisy where sales are few. We estimate adjustments at four levels of locality---a global within-municipality hedonic grid, a regional grid shrunk towards the global one, a per-valuation local regression on the candidate pool, and a non-linear gradient-boosting model---and test the trade-off directly across density strata. | |
| 14 | +The estimation of adjustment magnitudes is where the sales comparison approach meets hedonic price theory. Since \citet{rosen1974}, the hedonic regression of price on characteristics has been the canonical tool for recovering implicit prices, and its coefficients are natural candidates for adjustment factors \citep{kang1991,pace1998}. \citet{kang1991} compared hedonic regression with grid-adjustment techniques on sales in a single county and found hedonic adjustments superior; \citet{cannaday1986} showed how depreciation adjustments can be estimated from the same regressions; \citet{pace1998} and \citet{pace1997} argued for non-parametric and spatial estimation of adjustments. A large literature has since emphasised spatial heterogeneity in implicit prices and the value of submarket definitions for prediction \citep{desrosiers2000,bourassa2003,goodman2003,case2004,bourassa2007,bourassa2010}, and \citet{dube2013,dube2014} have stressed that spatial dependence in housing data is also temporal, since only prior sales can influence a current price. Whether locally estimated adjustments improve on a global grid in practice is an empirical question that depends on the bias--variance trade-off: local models track heterogeneity but are noisy where sales are few. We estimate adjustments at four levels of locality---a global within-municipality hedonic grid, a regional grid shrunk towards the global one, a per-valuation local regression on the candidate pool, and a non-linear gradient-boosting model---and test the trade-off directly across density strata. | |
| 15 | 15 | |
| 16 | −Time adjustments raise a separate issue. Price indices are routinely estimated with the full sample and applied retroactively, which leaks information from the future into every ``historical'' valuation. \citet{bailey1963} and \citet{case1989} introduced repeat-sales indices and \citet{hill2013} reviews hedonic index construction; we follow the hedonic route but re-estimate the regional monthly index every month using only prior sales, so that the time adjustment applied to a comparable on a given valuation date is one that could actually have been computed on that date. | |
| 16 | +Time adjustments raise a separate issue. Price indices are routinely estimated with the full sample and applied retroactively, which leaks information from the future into every ``historical'' valuation; \citet{clapham2006} document how much repeat-sales and hedonic indices are revised as later data arrive. \citet{bailey1963} and \citet{case1989} introduced repeat-sales indices, \citet{hill2013} reviews hedonic index construction and \citet{hillscholz2018} show the gains from geospatial information in hedonic imputation; we follow the hedonic route but re-estimate the regional monthly index every month using only prior sales, so that the time adjustment applied to a comparable on a given valuation date is one that could actually have been computed on that date. The seasonality of housing markets documented by \citet{ngai2014} matters here: because the price level of the valuation month is never observed, every point-in-time method inherits a seasonal error that we report rather than remove. | |
| 17 | + | |
| 18 | +The assessed value is a further source of information that the appraisal literature treats ambiguously. \citet{clapp1992} showed that assessed values can be used to construct price indices, and \citet{thibodeau2003} that marking assessed values to market with a small number of sales yields accurate estimates. Assessors' sales-ratio methods rest on the same idea. We treat the assessment ratio as one adjustment scheme among eight and quantify what the assessed value adds to both the traditional and the machine-learning methods. | |
| 17 | 19 | |
| 18 | 20 | \subsection{Automated valuation and machine learning} |
| 19 | 21 | |
| 20 | −Automated valuation models (AVMs) have moved from mass appraisal \citep{gloudemans1999} to lending and consumer applications, and the literature comparing them with traditional valuation has grown rapidly. \citet{pagourtzi2003} survey the methods; \citet{mccluskey2013} compare neural networks with hedonic models for mass appraisal; \citet{mayer2019} evaluate several machine-learning algorithms on a large Swiss dataset and find that gradient boosting and random forests dominate linear hedonics, with mixed-effects and spatial models close behind; \citet{bourassa2010} and \citet{case2004} compare spatial econometric specifications; \citet{pace2020} review the state of AVMs. A consistent finding is that tree ensembles with location features achieve median absolute errors of roughly 8--15\% on residential sales, depending on market and data richness. Comparable-sales algorithms are themselves a major class of AVM \citep{iaao2018}, and hybrid approaches that use a model to adjust comparables are common in industry but rarely evaluated in the academic literature. | |
| 22 | +Automated valuation models (AVMs) have moved from mass appraisal \citep{gloudemans1999} to lending and consumer applications, and the literature comparing them with traditional valuation has grown rapidly. \citet{shiller1999} set out the logic of evaluating valuation systems on their out-of-sample forecast errors rather than on in-sample fit. \citet{pagourtzi2003} survey the methods; \citet{zurada2011} and \citet{mccluskey2013} compare neural networks and other learners with hedonic models for mass appraisal; \citet{kok2017} document the move from manual appraisal to automated valuation in institutional practice; \citet{mayer2019} evaluate several machine-learning algorithms on a large Swiss dataset and find that gradient boosting and random forests dominate linear hedonics, with mixed-effects and spatial models close behind; \citet{pace2020} show that tree ensembles extract information from the residuals of hedonic and spatial models; \citet{bogin2020} find that both AVMs and appraisers are less accurate in rural areas, where comparables are scarce. \citet{steurer2021} review the metrics used to evaluate AVMs and argue for reporting medians, shares within tolerance bands and coverage together, which is the convention we follow; \citet{krause2020} compare error-based and model-based measures of AVM uncertainty, a question we take up when calibrating value intervals; \citet{lorenz2023} show how interpretable machine learning can recover implicit prices from tree ensembles, which narrows but does not close the gap between opaque and transparent methods. A consistent finding is that tree ensembles with location features achieve median absolute errors of roughly 8--15\% on residential sales, depending on market and data richness. Comparable-sales algorithms are themselves a major class of AVM \citep{iaao2018}, and hybrid approaches that use a model to adjust comparables are common in industry but rarely evaluated in the academic literature. | |
| 21 | 23 | |
| 22 | −Two features of this literature motivate our design. First, machine-learning benchmarks are usually trained on random splits of the data, which overstate accuracy relative to a temporal split \citep{mayer2019}; we use strict chronological separation and quarterly retraining, which is how such models are deployed. Second, the information set is rarely held constant: AVMs often exploit variables (assessed values, listing information) that the traditional method in the same study does not. We give every method the same characteristics and the same point-in-time index, and we separately quantify the value of the assessed value as an input to both the traditional and the machine-learning methods. | |
| 24 | +Two features of this literature motivate our design. First, machine-learning benchmarks are usually trained on random splits of the data, which overstate accuracy relative to a temporal split \citep{mayer2019}; we use strict chronological separation and quarterly retraining, which is how such models are deployed. Second, the information set is rarely held constant: AVMs often exploit variables (assessed values, listing information) that the traditional method in the same study does not. We give every method the same characteristics and the same point-in-time index, and we separately quantify the value of the assessed value as an input to both the traditional and the machine-learning methods. Our inference draws on the forecast-evaluation literature: paired comparisons of losses on common targets \citep{diebold1995,giacomini2006}, the median as the loss-minimising point forecast under absolute error \citep{gneiting2011,hyndman2006}, the protection against data snooping that a separate selection sample affords when many specifications are searched \citep{white2000,hansen2005}, and distribution-free calibration of prediction intervals on a hold-out sample \citep{lei2018}. | |
| 23 | 25 | |
| 24 | 26 | \subsection{Appraiser behaviour and the limits of simulation} |
| 25 | 27 | |
| 26 | −Finally, a behavioural literature studies how appraisers actually select and adjust comparables and documents anchoring on contract prices, prior appraisals and assessed values \citep{diaz1990,diaz2001,northcraft1987}. We do not observe appraisers' files, and our results concern the predictive accuracy of \emph{rules}, not of practitioners. The distinction matters in both directions: a practitioner may do better than any rule by using information we do not observe (condition, renovations, listing history), and worse by anchoring. Our contribution is to establish what the observable record supports, so that professional rules and training can be benchmarked against it. | |
| 28 | +Finally, a behavioural literature studies how appraisers actually select and adjust comparables and documents anchoring on contract prices, prior appraisals and assessed values \citep{northcraft1987,diaz1990,diaz2001}; on administrative data, \citet{eriksen2019} and \citet{calem2021} show that purchase appraisals cluster at or just above the contract price. We do not observe appraisers' files, and our results concern the predictive accuracy of \emph{rules}, not of practitioners. The distinction matters in both directions: a practitioner may do better than any rule by using information we do not observe (condition, renovations, listing history), and worse by anchoring. Our contribution is to establish what the observable record supports, so that professional rules and training can be benchmarked against it. | |
modified
paper/sections/methodology.tex
+64 −18
@@ -10,48 +10,71 @@ The core of the design is a backtest that reproduces, for each target sale $i$ w | ||
| 10 | 10 | \mathcal{C}_i = \{\, j \in \mathcal{S} : \tau_j < \tau_i,\; \tau_i - \tau_j \le 24 \text{ months},\; d_{ij} \le 25 \text{ km},\; u_j \neq u_i \,\}, |
| 11 | 11 | \label{eq:pool} |
| 12 | 12 | \end{equation} |
| 13 | −where $d_{ij}$ is the great-circle distance and $u_j$ the assessment unit; the last condition excludes prior sales of the subject itself, so that the method is evaluated as a comparable-sales method rather than a repeat-sales one. Every quantity entering a valuation on $\tau_i$ is computed from sales with $\tau_j < \tau_i$: the regional price index used to time-adjust comparables is re-estimated at the start of every month $m$ from all sales in months $< m$ (Section~\ref{sec:index}); adjustment coefficients and benchmark models are re-estimated at the start of every quarter $q$ from all sales in quarters $< q$ (Section~\ref{sec:config}). No configuration, hyperparameter or weight is chosen on test-period data: all are selected on the 2024 validation targets and frozen. | |
| 13 | +where $d_{ij}$ is the distance between the two properties and $u_j$ the assessment unit; the last condition excludes prior sales of the subject itself, so that the method is evaluated as a comparable-sales method rather than a repeat-sales one. Every quantity entering a valuation on $\tau_i$ is computed from sales with $\tau_j < \tau_i$: the regional price index used to time-adjust comparables is re-estimated at the start of every month $m$ from all sales in months before $m$ (Section~\ref{sec:index}); adjustment coefficients and benchmark models are re-estimated at the start of every quarter $q$ from all sales in quarters before $q$ (Sections~\ref{sec:config} and \ref{sec:benchmarks}). No configuration, hyperparameter or weight is chosen on test-period data: all are selected on the 2024 validation targets and frozen. | |
| 14 | 14 | |
| 15 | −A subtle but important consequence is that the price level of the target's own month is never observed. All methods, traditional and machine-learning alike, work at the level of the last fully observed month, exactly as a valuer must. This introduces a small, common and seasonal bias that we report rather than remove. | |
| 15 | +A subtle but important consequence is that the price level of the target's own month is never observed. All methods, traditional and machine-learning alike, work at the level of the last fully observed month, exactly as a valuer must. This introduces a small, common and seasonal bias that we report rather than remove (Section~\ref{sec:stability}). | |
| 16 | 16 | |
| 17 | 17 | \subsection{Point-in-time price index} |
| 18 | 18 | \label{sec:index} |
| 19 | 19 | |
| 20 | 20 | For each evaluation month $m$ and administrative region $r$ we estimate, on all sales of region $r$ in months $0,\dots,m-1$, the within-municipality regression |
| 21 | 21 | \begin{equation} |
| 22 | −\ln P_j = \alpha_{\mu(j)} + \mathbf{x}_j'\boldsymbol{\beta} + \sum_{s<m-1} \delta_s^{(r,m)} \mathbb{1}[\text{month}_j = s] + \varepsilon_j, | |
| 22 | +\ln P_j = \alpha_{\mu(j)} + \mathbf{x}_j'\boldsymbol{\beta} + \sum_{s \le m-2} \delta_s^{(r,m)}\, \mathbb{1}[\text{month}_j = s] + \varepsilon_j, | |
| 23 | 23 | \label{eq:index} |
| 24 | 24 | \end{equation} |
| 25 | −where $\alpha_{\mu}$ is a municipality fixed effect, $\mathbf{x}_j$ the characteristics (log living area, log lot area, age and its square, physical link, construction type, storeys, new-build indicator) and $\delta_s^{(r,m)}$ the effect of month $s$ relative to the last observed month $m-1$. The municipality fixed effects prevent the index from being contaminated by shifts in the composition of sales between expensive and inexpensive municipalities within a region. For regions with few sales the monthly effects are shrunk towards the provincial estimate with weight $n_s/(n_s+40)$. A comparable sold in month $s$ and valued in month $m$ is time-adjusted by $\exp(-\delta_s^{(r,m)})$; comparables sold in the same month as the target are not adjusted. The index is thus a three-dimensional object (evaluation month, region, sale month), and the estimate of any historical month is allowed to change as later data arrive---as it does in reality. | |
| 25 | +where $\alpha_{\mu}$ is a municipality fixed effect, $\mathbf{x}_j$ the characteristics (log living area, log lot area, age and its square, physical link, construction type, storeys, new-build indicator) and $\delta_s^{(r,m)}$ the effect of month $s$ relative to the last observed month $m-1$, whose effect is normalised to zero. The municipality fixed effects prevent the index from being contaminated by shifts in the composition of sales between expensive and inexpensive municipalities within a region. For regions with fewer than 200 sales the provincial index is used; otherwise the monthly effects are shrunk towards the provincial estimate with weight $n_s/(n_s+40)$, where $n_s$ is the number of regional sales in month $s$. A comparable sold in month $s < m$ and valued in month $m$ is time-adjusted by the factor $\exp(-\delta_s^{(r,m)})$, i.e.\ brought to the level of month $m-1$; comparables sold in the same month as the target, for which no effect can be estimated, are not adjusted. The index is thus a three-dimensional object (evaluation month, region, sale month), and the estimate of any historical month is allowed to change as later data arrive, as it does in reality \citep{clapham2006}. | |
| 26 | 26 | |
| 27 | 27 | \subsection{Configurations of the sales comparison approach} |
| 28 | 28 | \label{sec:config} |
| 29 | 29 | |
| 30 | −A configuration is a tuple $(\text{selection}, d, t, k, \text{adjustment}, \text{weighting}, \text{reconciliation})$. Given the pool $\mathcal{C}_i$, the method proceeds in four steps. | |
| 30 | +A configuration is a tuple $c = (\text{selection}, d, t, k, \text{adjustment}, \text{weighting}, \text{reconciliation})$. Given the pool $\mathcal{C}_i$, the method proceeds in four steps. | |
| 31 | 31 | |
| 32 | −\paragraph{Selection.} Candidates satisfying $d_{ij}\le d$ and $\tau_i-\tau_j \le t$ are ranked by a score and the $k$ best are retained. Four scores are implemented: \emph{nearest} (distance $d_{ij}$); \emph{most similar}, a physical dissimilarity $D_{ij} = \big[(\Delta\ln A/0.25)^2 + (\Delta\ln L/0.6)^2 + (\Delta\text{age}/15)^2 + \mathbb{1}[\text{link}\neq] + 0.5\,\mathbb{1}[\text{storeys}\neq]\big]^{1/2}$ in units of typical within-market standard deviations, with distance as a tie-breaker; \emph{compromise}, $D_{ij} + d_{ij}/2$; and \emph{least adjusted}, the gross hedonic adjustment $\sum_m |\hat\beta_m \Delta x_{ijm}|$, which operationalises the professional advice to prefer comparables requiring the fewest adjustments. | |
| 32 | +\paragraph{Selection.} Candidates satisfying $d_{ij}\le d$ and $\tau_i-\tau_j \le t$ are ranked by a score and the $k$ best are retained. Physical dissimilarity is measured by | |
| 33 | +\begin{equation} | |
| 34 | +D_{ij} = \Big[\Big(\tfrac{\Delta\ln A_{ij}}{0.25}\Big)^2 + \Big(\tfrac{\Delta\ln L_{ij}}{0.6}\Big)^2 + \Big(\tfrac{\Delta \text{age}_{ij}}{15}\Big)^2 + \mathbb{1}[\text{link}_i\neq\text{link}_j] + 0.5\,\mathbb{1}[\text{storeys}_i\neq\text{storeys}_j]\Big]^{1/2}, | |
| 35 | +\label{eq:dissim} | |
| 36 | +\end{equation} | |
| 37 | +where $A$ is living area, $L$ lot area and $\Delta$ denotes the difference between subject and comparable; the denominators are typical within-market standard deviations, so that $D_{ij}$ is in standard-deviation units. Four scores are implemented: \emph{nearest}, $s^{\mathrm{geo}}_{ij} = d_{ij}$; \emph{most similar}, $s^{\mathrm{sim}}_{ij} = D_{ij} + d_{ij}/25$, in which distance (in km) acts only as a tie-breaker; \emph{compromise}, $s^{\mathrm{mix}}_{ij} = D_{ij} + d_{ij}/2$, which trades one standard deviation of dissimilarity against two kilometres; and \emph{least adjusted}, $s^{\mathrm{adj}}_{ij} = \sum_m |\hat\beta_m \Delta x_{ijm}| + d_{ij}/25$, the gross hedonic adjustment, which operationalises the professional advice to prefer comparables requiring the fewest adjustments. | |
| 33 | 38 | |
| 34 | −\paragraph{Adjustment.} Each retained comparable's time-adjusted price $P_j^{T}$ is converted into an indication of the subject's value $\hat P_{i|j}$. Eight schemes are compared: | |
| 39 | +\paragraph{Adjustment.} Let $P_j^{T}$ denote the time-adjusted price of comparable $j$. Each retained comparable is converted into an indication of the subject's value $\hat P_{i|j}$ under one of eight schemes: | |
| 35 | 40 | \begin{enumerate}[label=(\alph*)] |
| 36 | 41 | \item \emph{none}: $\hat P_{i|j} = P_j$ (not even time-adjusted); |
| 37 | 42 | \item \emph{time only}: $\hat P_{i|j} = P_j^{T}$; |
| 38 | −\item \emph{conventional grid}: dollar adjustments in the spirit of professional practice---living area at one half of the comparable's price per square metre, lot area at one half of the comparable's implied land value per square metre (capped at $\pm 15\%$), age at $-0.5\%$ per year of difference (capped at $\pm 20\%$), and $\pm 5\%$ between detached and attached dwellings; | |
| 39 | −\item \emph{hedonic grid}: $\hat P_{i|j} = P_j^{T}\exp\!\big(\hat{\boldsymbol\beta}'(\mathbf{x}_i-\mathbf{x}_j)\big)$ with $\hat{\boldsymbol\beta}$ from a within-municipality regression on all prior sales (re-estimated quarterly); | |
| 43 | +\item \emph{conventional grid}: dollar adjustments in the spirit of professional practice, | |
| 44 | +\begin{equation} | |
| 45 | +\begin{aligned} | |
| 46 | +\hat P_{i|j} = P_j^{T} &+ \tfrac{1}{2}\tfrac{P_j^{T}}{A_j}(A_i - A_j) + \Big[\tfrac{1}{2}\tfrac{\lambda_j P_j^{T}}{L_j}(L_i - L_j)\Big]_{\pm 0.15 P_j^{T}} \\ | |
| 47 | +&+ \Big[-0.005\,P_j^{T}(\text{age}_i - \text{age}_j)\Big]_{\pm 0.20 P_j^{T}} \pm 0.05\,P_j^{T}\,\mathbb{1}[\text{link differs}], | |
| 48 | +\end{aligned} | |
| 49 | +\label{eq:conv} | |
| 50 | +\end{equation} | |
| 51 | +where $\lambda_j$ is the land share of the comparable's assessed value, $[\cdot]_{\pm b}$ denotes capping at $\pm b$, and the link adjustment is positive when the subject is detached and the comparable attached; the total adjustment is floored at $-60\%$ of $P_j^T$; | |
| 52 | +\item \emph{hedonic grid}: $\hat P_{i|j} = P_j^{T}\exp\!\big(\hat{\boldsymbol\beta}'(\mathbf{x}_i-\mathbf{x}_j)\big)$, with $\hat{\boldsymbol\beta}$ the within-municipality coefficients of a regression of the time-adjusted log price on $\mathbf{x}$, estimated on all prior sales and re-estimated quarterly; | |
| 40 | 53 | \item \emph{regional grid}: as (d) with region-specific coefficients shrunk towards the global ones with weight $n_r/(n_r+300)$; |
| 41 | −\item \emph{local grid}: as (d) with coefficients re-estimated on the subject's own pool within 10~km (at least 40 sales), by ridge regression penalised towards the regional grid; | |
| 42 | −\item \emph{non-linear (GBM)}: $\hat P_{i|j} = P_j^{T}\exp\!\big(f(\mathbf{z}_i)-f(\mathbf{z}_j)\big)$ where $f$ is a gradient-boosting model of the time-adjusted log price on characteristics and location (Section~\ref{sec:benchmarks}); | |
| 43 | −\item \emph{assessment ratio}: $\hat P_{i|j} = V_i \cdot P_j^{T}/V_j$ where $V$ is the assessed value, restricted to comparables in the same municipality and on the same assessment roll as the subject. | |
| 54 | +\item \emph{local grid}: as (d) with coefficients $\hat{\mathbf{b}}_i$ re-estimated on the subject's own candidates within 10~km (at least 40 sales) by ridge regression penalised towards the regional grid $\hat{\boldsymbol\beta}^{\mathrm{reg}}$, | |
| 55 | +\begin{equation} | |
| 56 | +\hat{\mathbf{b}}_i = \arg\min_{\mathbf{b}} \sum_{j \in \mathcal{C}_i,\, d_{ij}\le 10} \big(\tilde y_j - \tilde{\mathbf{x}}_j'\mathbf{b}\big)^2 + \lambda\, \|\mathbf{b} - \hat{\boldsymbol\beta}^{\mathrm{reg}}\|^2, \qquad \lambda = 50, | |
| 57 | +\label{eq:local} | |
| 58 | +\end{equation} | |
| 59 | +where tildes denote variables centred on the local pool; | |
| 60 | +\item \emph{non-linear (GBM)}: $\hat P_{i|j} = P_j^{T}\exp\!\big(f(\mathbf{z}_i)-f(\mathbf{z}_j)\big)$, where $f$ is the gradient-boosting model of the time-adjusted log price on characteristics and location described in Section~\ref{sec:benchmarks} (the version without the assessed value); | |
| 61 | +\item \emph{assessment ratio}: $\hat P_{i|j} = V_i \cdot P_j^{T}/V_j$, where $V$ is the assessed value in force at the sale, restricted to comparables in the same municipality and of the same roll vintage as the subject. | |
| 44 | 62 | \end{enumerate} |
| 45 | 63 | Scheme (h) is the ratio method used by assessors and, informally, by many appraisers; it uses information (the assessed value) that schemes (a)--(g) do not, and we treat it as a distinct model in the horse race. |
| 46 | 64 | |
| 47 | −\paragraph{Weighting and reconciliation.} Adjusted indications are combined by the median or the (dollar) mean, with equal weights or with weights proportional to $1/(0.25+d_{ij})$, to $\exp(-\text{months}_{ij}/6)$, to $1/(0.1+\text{gross adjustment}_{ij})$, or to their product; weighted medians are used when weights are not equal. The dispersion of the adjusted indications is recorded as a measure of uncertainty. | |
| 65 | +\paragraph{Weighting and reconciliation.} Let $v_{ij} = \ln \hat P_{i|j}$ and let $w_{ij}$ be the weight of comparable $j$. Five weighting schemes are compared: equal weights; distance, $w_{ij} = 1/(0.25 + d_{ij})$; recency, $w_{ij} = \exp(-\text{months}_{ij}/6)$; similarity, $w_{ij} = 1/(0.1 + g_{ij})$ where $g_{ij}$ is the gross hedonic adjustment in log points; and the product of the three. Two reconciliation rules are compared: the (weighted) median of the $v_{ij}$, and the dollar-weighted mean | |
| 66 | +\begin{equation} | |
| 67 | +\hat P_i = \frac{\sum_j w_{ij}\, \hat P_{i|j}}{\sum_j w_{ij}} . | |
| 68 | +\label{eq:recon} | |
| 69 | +\end{equation} | |
| 70 | +The standard deviation $\hat\sigma_i$ of the $v_{ij}$ across the retained comparables is recorded as a measure of uncertainty (Section~\ref{sec:intervals}). | |
| 48 | 71 | |
| 49 | 72 | \paragraph{Experiments.} Experiment E1 maps the error surface over $k \in \{1,3,5,7,10,15,20,30,50\}$, $d \in \{0.5,1,2,5,10,25\}$~km and $t \in \{3,6,12,24\}$ months, for the nearest, most-similar and compromise selections and the time-only and hedonic adjustments, with equal weights and the median (1{,}296 configurations). Experiment E2 is the full factorial of selection (4) $\times$ adjustment (8) $\times$ weighting (5) $\times$ reconciliation (2) at three anchors $(k,d,t) = (3, 2\text{ km}, 6\text{ mo})$, $(5, 5\text{ km}, 12\text{ mo})$ and $(10, 10\text{ km}, 24\text{ mo})$ (960 configurations). All \nConfigs{} configurations are evaluated on every target. |
| 50 | 73 | |
| 51 | 74 | \subsection{Benchmark models} |
| 52 | 75 | \label{sec:benchmarks} |
| 53 | 76 | |
| 54 | −Thirteen named models are compared (Table~\ref{tab:models}). M1--M8 and M13 are configurations of the sales comparison approach that correspond to recognisable professional practices. M10 is a direct global hedonic model: the within-municipality coefficients augmented with region, county (MRC), municipality and neighbourhood-unit effects estimated as shrunken residual means, plus a median correction. M11 is a gradient-boosting model (LightGBM, 127 leaves, learning rate 0.05, L1 objective so that it targets the conditional median, number of trees chosen by early stopping on the last 10\% of the training period) of the time-adjusted log price on characteristics, coordinates, region, construction type and the month of sale; M11a excludes and M11b includes the assessed value, its land share and its roll vintage. Both are retrained at the start of every quarter on all prior sales. M9 is an \emph{optimised reconciliation}: a convex combination, estimated by non-negative least squares on the validation targets, of the seven comparable-based indications at the $(5,5,12)$ anchor. M12 is a hybrid combining M11b with the GBM-adjusted and hedonic-adjusted comparable estimates, with weights estimated the same way. M0 is the single best of the \nConfigs{} configurations on the validation targets, among those that return a value for at least 97\% of targets. | |
| 77 | +Sixteen named models are compared (Table~\ref{tab:models}). M1--M8 and M13 are configurations of the sales comparison approach that correspond to recognisable professional practices. M10 is a direct global hedonic model: the within-municipality coefficients augmented with region, county (MRC), municipality and neighbourhood-unit effects estimated as shrunken residual means, plus a median correction estimated on the validation targets. M11 is a gradient-boosting model (LightGBM, 127 leaves, learning rate 0.05, $L_1$ objective, number of trees chosen by early stopping on the chronologically last 10\% of the training period) of the time-adjusted log price on characteristics, coordinates, region, construction type and the month of sale; M11a excludes and M11b includes the assessed value, its land share and its roll vintage. Both are retrained at the start of every quarter on all prior sales. Because the $L_1$ objective targets the conditional median of the log price, the models estimate the point forecast that minimises expected absolute error \citep{gneiting2011}, which is the loss the MdAPE rewards; a squared-error objective would target the conditional mean of a left-skewed distribution and produce a negative median bias (Section~\ref{sec:benchspec}). M9 is an \emph{optimised reconciliation}: a convex combination, estimated by non-negative least squares on the validation targets, of the seven comparable-based indications at the $(5,5,12)$ anchor. M12 is a hybrid combining M11b with the GBM-adjusted and hedonic-adjusted comparable estimates, with weights estimated the same way. M0 is the single best of the \nConfigs{} configurations on the validation targets, among those that return a value for at least 97\% of targets. | |
| 55 | 78 | |
| 56 | 79 | \begin{table}[H] |
| 57 | 80 | \centering |
@@ -75,17 +98,40 @@ M9 & Optimised reconciliation: NNLS combination of seven comparable-based indica | ||
| 75 | 98 | M10 & Direct global hedonic model with region, MRC, municipality and neighbourhood effects \\ |
| 76 | 99 | M11a / M11b & Gradient boosting without / with the assessed value, retrained quarterly \\ |
| 77 | 100 | M12 & Hybrid: NNLS combination of M11b and GBM- and hedonic-adjusted comparables (weights fitted on 2024) \\ |
| 78 | −M13 & Assessment ratio: $V_i \times$ median price-to-assessment ratio of the 5 nearest same-roll sales \\ | |
| 101 | +M13 & Assessment ratio: $V_i \times$ median price-to-assessment ratio of the 5 nearest same-roll sales ($\le 5$ km, $\le 12$ months) \\ | |
| 79 | 102 | M0 & Best of all \nConfigs{} configurations on 2024 (coverage $\ge 97\%$), frozen \\ |
| 80 | 103 | \bottomrule |
| 81 | 104 | \end{tabularx} |
| 82 | 105 | \end{table} |
| 83 | 106 | |
| 84 | −\subsection{Accuracy measures and inference} | |
| 107 | +\subsection{Accuracy measures} | |
| 108 | +\label{sec:measures} | |
| 85 | 109 | |
| 86 | −For each target and method we compute the absolute percentage error $\mathrm{APE}_i = |\hat P_i - P_i|/P_i$. We report the median APE (MdAPE), the mean APE (MAPE), the median signed error (bias), the shares of estimates within $\pm5$, $\pm10$ and $\pm20\%$ of the price, and the \emph{coverage}, the share of targets for which the method returns a value. Coverage is reported because a restrictive configuration does not fail on difficult targets---it abstains---and an MdAPE computed on the targets it covers would flatter it. Configuration selection on the validation set is restricted to configurations with coverage of at least 97\%. | |
| 110 | +For each target $i$ and method (or configuration) $c$ that returns a value, the absolute percentage error is | |
| 111 | +\begin{equation} | |
| 112 | +\mathrm{APE}_i(c) = \frac{|\hat P_i(c) - P_i|}{P_i}, \qquad e_i(c) = \frac{\hat P_i(c) - P_i}{P_i}. | |
| 113 | +\label{eq:ape} | |
| 114 | +\end{equation} | |
| 115 | +Over a set of targets $\mathcal{T}$, let $\mathcal{T}_c \subseteq \mathcal{T}$ be the targets for which $c$ returns a value. We report | |
| 116 | +\begin{equation} | |
| 117 | +\mathrm{MdAPE}(c) = \operatorname*{med}_{i \in \mathcal{T}_c} \mathrm{APE}_i(c), \qquad | |
| 118 | +\mathrm{MAPE}(c) = \frac{1}{|\mathcal{T}_c|}\sum_{i \in \mathcal{T}_c} \mathrm{APE}_i(c), \qquad | |
| 119 | +\mathrm{Bias}(c) = \operatorname*{med}_{i \in \mathcal{T}_c} e_i(c), | |
| 120 | +\label{eq:metrics} | |
| 121 | +\end{equation} | |
| 122 | +together with the shares of estimates within $\pm5$, $\pm10$ and $\pm20\%$ of the price and the \emph{coverage} $|\mathcal{T}_c|/|\mathcal{T}|$ \citep{steurer2021,hyndman2006}. The MdAPE is the headline measure because valuation errors are heavy-tailed and the median is the natural loss of an appraiser who is judged on the typical file; the MAPE and the tolerance shares are reported alongside because they weight the tails differently. Coverage is reported because a restrictive configuration does not fail on difficult targets---it abstains---and an MdAPE computed on the targets it covers would flatter it. Configuration selection on the validation set is restricted to configurations with coverage of at least 97\%. | |
| 123 | + | |
| 124 | +\subsection{Inference} | |
| 125 | +\label{sec:inference} | |
| 126 | + | |
| 127 | +Because all methods are evaluated on the same targets, comparisons are paired. For a pair of methods $(A,B)$ and the common set of covered targets, the difference in MAPE is the mean of the loss differential $\Delta_i = \mathrm{APE}_i(A) - \mathrm{APE}_i(B)$, the quantity tested by \citet{diebold1995} and \citet{giacomini2006}, and the difference in MdAPE is the difference of the two medians. Valuation errors are correlated within local markets and within periods, because neighbouring sales share unobserved amenities and the same index estimate; we therefore treat municipality--quarter cells as clusters. Confidence intervals are obtained from a cluster bootstrap \citep{cameron2008} that resamples cells with replacement $B = 999$ times and recomputes both statistics on each replication, and we also report a cluster-robust $t$ statistic for the mean differential, | |
| 128 | +\begin{equation} | |
| 129 | +t = \frac{\bar\Delta}{\Big[\sum_{g} \big(\sum_{i \in g} (\Delta_i - \bar\Delta)\big)^2\Big]^{1/2} / N}, | |
| 130 | +\label{eq:dm} | |
| 131 | +\end{equation} | |
| 132 | +where $g$ indexes the clusters and $N$ the number of common targets. We also report the share of targets on which $A$ is more accurate than $B$. Section~\ref{sec:clusters} shows that the intervals are insensitive to the cluster definition. | |
| 87 | 133 | |
| 88 | −Because all methods are evaluated on the same targets, comparisons are paired. For a pair of methods $(A,B)$ we report the difference in MdAPE and in MAPE together with 95\% confidence intervals from a cluster bootstrap \citep{cameron2008} that resamples municipality--quarter cells with replacement, thereby preserving the spatial and temporal dependence of valuation errors within local markets. We also report the share of targets on which $A$ is more accurate than $B$. | |
| 134 | +The second inferential issue is selection. Choosing the best of \nConfigs{} configurations on the same data that are used to report its accuracy would overstate it \citep{white2000,hansen2005}. We avoid this by selecting on the 2024 validation targets and reporting on the 2025--2026 test targets, and Section~\ref{sec:selection_stability} examines how the validation ranking carries over to the test period. | |
| 89 | 135 | |
| 90 | 136 | \subsection{Hypotheses} |
| 91 | 137 | |
modified
paper/sections/results.tex
+35 −23
@@ -3,7 +3,7 @@ | ||
| 3 | 3 | \label{sec:results} |
| 4 | 4 | % ============================================================================ |
| 5 | 5 | |
| 6 | −All results in this section are computed on the \nTargetsTest{} test targets of January 2025 to September 2026, with every configuration, weight and model frozen on the basis of the \nTargetsVal{} validation targets of 2024. Validation-period figures are reported alongside to document the stability of the rankings. | |
| 6 | +All results in this section are computed on the \nTargetsTest{} test targets of January 2025 to September 2026, with every configuration, weight and model frozen on the basis of the \nTargetsVal{} validation targets of 2024. Validation-period figures are reported alongside to document the stability of the rankings. Differences between methods quoted in the text are paired differences on the common targets, computed before rounding, so that they may differ by a tenth of a point from the difference of the rounded levels. | |
| 7 | 7 | |
| 8 | 8 | \subsection{How many comparables, how far, how far back?} |
| 9 | 9 | \label{sec:surface} |
@@ -31,13 +31,13 @@ Figure~\ref{fig:surface_k} plots the out-of-sample MdAPE against the number of c | ||
| 31 | 31 | \small |
| 32 | 32 | \input{tables/surface_geo_hedo} |
| 33 | 33 | \begin{tablenotes}\footnotesize |
| 34 | −\item Note: MdAPE on the 2025--2026 test targets for which the configuration returns a value. Coverage falls below 90\% for radii of 1~km and less in low-density markets; see Table~\ref{tab:surface_cov} in the Appendix. | |
| 34 | +\item Note: MdAPE on the 2025--2026 test targets for which the configuration returns a value; the minimum of each row is in bold. Coverage falls below 90\% for radii of 1~km and less in low-density markets; see Table~\ref{tab:surface_cov} in the Appendix. | |
| 35 | 35 | \end{tablenotes} |
| 36 | 36 | \end{table} |
| 37 | 37 | |
| 38 | −Three regularities emerge. First, without physical adjustment, accuracy improves with $k$ up to about ten comparables (\noneKten{} at $k=10$ for a 10~km radius and twelve-month window) and then deteriorates (\noneKfifty{} at $k=50$): beyond the nearest few, additional sales are increasingly dissimilar and the median of their time-adjusted prices drifts away from the subject's value. Second, with the hedonic grid the picture changes qualitatively. The error falls from \mdapeKone{} with a single comparable to \mdapeKthree{} with three, \mdapeKfive{} with five, \mdapeKten{} with ten, reaches \mdapeKopt{} at $k=\kOpt{}$ and stays there to $k=50$ (\mdapeKfifty{}). Adjustment converts dissimilar sales into usable evidence, so that the variance reduction from averaging more of them is no longer offset by their heterogeneity. Third, the window matters and the radius does not. The three-month window is dominated by the twelve- and twenty-four-month windows at every radius (Figure~\ref{fig:surface_dt}); but at the best $k$ the MdAPE with a 24-month window is 11.9--12.0\% for every radius from 0.5 to 25~km, because $k$ is a maximum and a wide radius only adds sales where the nearby ones run out. The conventional combination---three comparables, within a kilometre, within three months---yields an MdAPE of \narrowMdape{} with the hedonic grid and returns a value for only \narrowCov{} of targets; the best configuration of the surface chosen on the 2024 validation set (\bestSel{}, $k=\bestK$, $d\le\bestD$~km, $t=\bestT$ months, \bestAdj{}) yields \bestValMdape{} on validation and \bestTestMdape{} on the test set. | |
| 38 | +Three regularities emerge. First, without physical adjustment, accuracy improves with $k$ up to about ten comparables (\noneKten{} at $k=10$ for a 10~km radius and twelve-month window) and then deteriorates (\noneKfifty{} at $k=50$): beyond the nearest few, additional sales are increasingly dissimilar and the median of their time-adjusted prices drifts away from the subject's value. This is the pattern that \citet{pace1996} predicts for a nearest-neighbour estimator whose bias grows with the number of neighbours. Second, with the hedonic grid the picture changes qualitatively. The error falls from \mdapeKone{} with a single comparable to \mdapeKthree{} with three, \mdapeKfive{} with five, \mdapeKten{} with ten, reaches \mdapeKopt{} at $k=\kOpt{}$ and stays there to $k=50$ (\mdapeKfifty{}). Adjustment converts dissimilar sales into usable evidence, so that the variance reduction from averaging more of them is no longer offset by their heterogeneity. Third, the window matters and the radius does not. The three-month window is dominated by the twelve- and twenty-four-month windows at every radius (Figure~\ref{fig:surface_dt}); but at the best $k$ the MdAPE with a 24-month window lies between 11.9 and 12.0\% for every radius from 0.5 to 25~km, because $k$ is a maximum and a wide radius only adds sales where the nearby ones run out. The conventional combination---three comparables, within a kilometre, within three months---yields an MdAPE of \narrowMdape{} with the hedonic grid and returns a value for only \narrowCov{} of targets; the best configuration of the surface chosen on the 2024 validation set (\bestSel{}, $k=\bestK$, $d\le\bestD$~km, $t=\bestT$ months, \bestAdj{}) yields \bestValMdape{} on validation and \bestTestMdape{} on the test set. | |
| 39 | 39 | |
| 40 | −Table~\ref{tab:h2} tests H2 by stratifying the $k$-curve by market density. The threshold beyond which additional comparables hurt exists only without physical adjustment: with time adjustment alone the optimum is about ten in dense markets (where the eleventh-nearest sale is already dissimilar) and larger in sparse ones (where every sale within reach is needed). With the hedonic grid there is no threshold within the range studied: the error is minimised at twenty to fifty comparables in every density stratum. H2 is therefore confirmed for the unadjusted method and rejected for the adjusted one---adjustment, not density, determines whether more comparables help. | |
| 40 | +Table~\ref{tab:h2} tests H2 by stratifying the $k$-curve by market density. The threshold beyond which additional comparables hurt exists only without physical adjustment (lower panel): with time adjustment alone the optimum is seven to ten comparables in the three densest strata, where the eleventh-nearest sale is already dissimilar, fifteen in the 5--19 stratum, and fifty in the sparsest one, where every sale within reach is needed. With the hedonic grid (upper panel) there is no threshold within the range studied: the error is minimised at twenty to fifty comparables in every density stratum and the curve is flat beyond twenty. H2 is therefore confirmed for the unadjusted method and rejected for the adjusted one---adjustment, not density, determines whether more comparables help. | |
| 41 | 41 | |
| 42 | 42 | \begin{table}[H] |
| 43 | 43 | \centering |
@@ -46,14 +46,14 @@ Table~\ref{tab:h2} tests H2 by stratifying the $k$-curve by market density. The | ||
| 46 | 46 | \small |
| 47 | 47 | \input{tables/h2} |
| 48 | 48 | \begin{tablenotes}\footnotesize |
| 49 | −\item Note: selection by distance, $d\le 10$~km, twelve-month window, hedonic grid, median. Density is the number of sample sales within 2~km in the twelve months preceding the target's date. The minimum of each row is in bold. | |
| 49 | +\item Note: selection by distance, $d\le 10$~km, twelve-month window, median. Density is the number of sample sales within 2~km in the twelve months preceding the target's date. The minimum of each row is in bold. | |
| 50 | 50 | \end{tablenotes} |
| 51 | 51 | \end{table} |
| 52 | 52 | |
| 53 | 53 | \subsection{Which adjustments are warranted?} |
| 54 | 54 | \label{sec:adjustments} |
| 55 | 55 | |
| 56 | −Table~\ref{tab:adjustments} and Figure~\ref{fig:adjustments} compare the eight adjustment schemes at the $(5, 5\text{ km}, 12\text{ mo})$ anchor with selection by distance, equal weights and the median. The time adjustment alone is worth only \gainTime{} points of MdAPE relative to raw prices with a twelve-month window and \gainTimeLong{} points with a 24-month window: comparables are recent, and a monthly index estimated in real time is itself noisy. Any physical grid is worth a further \gainGridLo{} to \gainGridHi{} points. The stylised conventional grid---which uses no estimation whatsoever---recovers most of the gain of the hedonic grid (\convMdape{} against \mdapeMseven{}), consistent with \citet{colwell1983}'s observation that the grid is a regression with pre-set coefficients that need only be roughly right. The refinements add little: the regional grid is indistinguishable from the global one, the local grid is \localVsHedo{} (\localMdape{}), and the non-linear GBM adjustments improve on the hedonic grid by \gbmVsHedo{} points. The assessment-ratio adjustment stands apart: with \mdapeMthirteen{} it is the most accurate single scheme, but it returns a value for only \covMthirteen{} of targets because it requires comparables on the same assessment roll. | |
| 56 | +Table~\ref{tab:adjustments} and Figure~\ref{fig:adjustments} compare the eight adjustment schemes at the $(5, 5\text{ km}, 12\text{ mo})$ anchor with selection by distance, equal weights and the median. The time adjustment alone is worth only \gainTime{} points of MdAPE relative to raw prices with a twelve-month window and \gainTimeLong{} points with a 24-month window: comparables are recent, and a monthly index estimated in real time is itself noisy. Any physical grid is worth a further \gainGridLo{} to \gainGridHi{} points. The stylised conventional grid---which uses no estimation whatsoever---recovers most of the gain of the hedonic grid (\convMdape{} against \mdapeMseven{}), consistent with the observation of \citet{colwell1983} that the grid is a regression with pre-set coefficients that need only be roughly right. The refinements add little: the regional grid is indistinguishable from the global one, the local grid is \localVsHedo{} (\localMdape{}), and the non-linear GBM adjustments improve on the hedonic grid by \gbmVsHedo{} points. The assessment-ratio adjustment stands apart: with \mdapeMthirteen{} it is the most accurate single scheme, but it returns a value for only \covMthirteen{} of targets because it requires comparables on the same assessment roll. | |
| 57 | 57 | |
| 58 | 58 | \begin{table}[H] |
| 59 | 59 | \centering |
@@ -62,7 +62,7 @@ Table~\ref{tab:adjustments} and Figure~\ref{fig:adjustments} compare the eight a | ||
| 62 | 62 | \small |
| 63 | 63 | \input{tables/adjustments} |
| 64 | 64 | \begin{tablenotes}\footnotesize |
| 65 | −\item Note: five nearest sales, equal weights, median. MdAPE, MAPE, bias and the share within $\pm10\%$ in per cent; coverage is the share of targets for which the scheme returns a value. | |
| 65 | +\item Note: five nearest sales, equal weights, median. MdAPE, MAPE, bias (median signed error) and the share within $\pm10\%$ in per cent, on the 2025--2026 test targets unless indicated; coverage is the share of test targets for which the scheme returns a value. | |
| 66 | 66 | \end{tablenotes} |
| 67 | 67 | \end{table} |
| 68 | 68 | |
@@ -73,7 +73,7 @@ Table~\ref{tab:adjustments} and Figure~\ref{fig:adjustments} compare the eight a | ||
| 73 | 73 | \label{fig:adjustments} |
| 74 | 74 | \end{figure} |
| 75 | 75 | |
| 76 | −Table~\ref{tab:h3} examines H3 by crossing the adjustment schemes with market density at two anchors. The local grid tracks the hedonic grid closely where sales are abundant and falls behind where they are scarce, as the bias--variance argument predicts; at the tight $(3, 2\text{ km}, 6\text{ mo})$ anchor, where the local regression is estimated on the smallest pools, the gap widens. The non-linear GBM adjustments are the most accurate scheme in dense markets and lose their advantage in sparse ones. The practical implication is that adjustment grids should be estimated on the largest pool that is homogeneous in implicit prices---in these data, the province with municipality fixed effects---rather than on the handful of sales around the subject. | |
| 76 | +Table~\ref{tab:h3} examines H3 by crossing the adjustment schemes with market density at two anchors. The local grid tracks the hedonic grid closely where sales are abundant and falls behind where they are scarce, as the bias--variance argument predicts; at the tight $(3, 2\text{ km}, 6\text{ mo})$ anchor, where the local regression is estimated on the smallest pools, the gap widens. The non-linear GBM adjustments are the most accurate scheme in every stratum, but their advantage over the hedonic grid is largest in dense markets (about a point) and smallest in the sparsest one. H3 is therefore rejected in its first part---local re-estimation does not outperform the fixed grid even where sales are plentiful---and confirmed in its second. The practical implication is that adjustment grids should be estimated on the largest pool that is homogeneous in implicit prices---in these data, the province with municipality fixed effects---rather than on the handful of sales around the subject. | |
| 77 | 77 | |
| 78 | 78 | \begin{table}[H] |
| 79 | 79 | \centering |
@@ -82,11 +82,11 @@ Table~\ref{tab:h3} examines H3 by crossing the adjustment schemes with market de | ||
| 82 | 82 | \small |
| 83 | 83 | \input{tables/h3} |
| 84 | 84 | \begin{tablenotes}\footnotesize |
| 85 | −\item Note: selection by distance, equal weights, median. Upper panel: $(5, 5\text{ km}, 12\text{ mo})$; lower panel: $(3, 2\text{ km}, 6\text{ mo})$. | |
| 85 | +\item Note: selection by distance, equal weights, median. Upper panel: $(5, 5\text{ km}, 12\text{ mo})$; lower panel: $(3, 2\text{ km}, 6\text{ mo})$. The minimum of each row is in bold. | |
| 86 | 86 | \end{tablenotes} |
| 87 | 87 | \end{table} |
| 88 | 88 | |
| 89 | −The estimated hedonic grid itself is of independent interest (Table~\ref{tab:grid}). Within a municipality, the elasticity of price with respect to living area is \betaFloor{}, that with respect to lot area is \betaLand{}, and depreciation is \betaAge{} per year at the mean age with a convex profile; semi-detached and row dwellings sell at discounts of \betaSemi{} and \betaRow{} relative to detached dwellings of the same size. These coefficients are stable across the eleven quarterly re-estimations to the second decimal. | |
| 89 | +The estimated hedonic grid itself is of independent interest (Table~\ref{tab:grid}). Within a municipality, the elasticity of price with respect to living area is \betaFloor{}, that with respect to lot area is \betaLand{}, and depreciation is \betaAge{} per year at the mean age with a convex profile, a magnitude in line with the single-family depreciation estimates of \citet{cannaday1986}; semi-detached and row dwellings sell at discounts of \betaSemi{} and \betaRow{} relative to detached dwellings of the same size. The within-municipality $R^2$ is \gridRtwo{}: the roll characteristics explain about a third of the price variation among dwellings of the same municipality, and the rest---location within the municipality, condition, quality---is what the comparables themselves must supply. These coefficients are stable across the eleven quarterly re-estimations to the second decimal. | |
| 90 | 90 | |
| 91 | 91 | \begin{table}[H] |
| 92 | 92 | \centering |
@@ -94,7 +94,7 @@ The estimated hedonic grid itself is of independent interest (Table~\ref{tab:gri | ||
| 94 | 94 | \label{tab:grid} |
| 95 | 95 | \input{tables/grid} |
| 96 | 96 | \begin{tablenotes}\footnotesize |
| 97 | −\item Note: coefficients of the within-municipality regression of the time-adjusted log price on characteristics, estimated on sales before 2025 (test-period grid). Living and lot areas in logs; age in years; age$^2$/100. | |
| 97 | +\item Note: coefficients of the within-municipality regression of the time-adjusted log price on characteristics, estimated on all sales before 2025 (grid in force in the first test quarter). Living and lot areas in logs; age in years; age$^2$/100. Omitted categories: detached, single-storey construction type. | |
| 98 | 98 | \end{tablenotes} |
| 99 | 99 | \end{table} |
| 100 | 100 | |
@@ -110,12 +110,12 @@ Table~\ref{tab:selection} compares the four selection criteria at the $(5,5,12)$ | ||
| 110 | 110 | \input{tables/selection} |
| 111 | 111 | \end{table} |
| 112 | 112 | |
| 113 | −The direct test of H1 is starker. Without any physical adjustment, the five nearest sales within one kilometre yield an MdAPE of \hOneGeoDone{} (coverage \hOneGeoDoneCov{}); the five most similar sales within ten kilometres yield \hOneSimDten{}. The nearby-but-dissimilar comparable is worse than the distant-but-similar one, by a wide margin. The reason is that, in these data, the physical characteristics explain a larger share of the within-market variance of prices than micro-location does once the municipality is fixed. With the hedonic grid the two criteria converge (\hOneGeoDtenHedo{} against \hOneSimDtenHedo{}), because adjustment does for the nearest sales what selection does for the most similar; the compromise criterion, which uses both, is best (\hOneMixDtenHedo{}). | |
| 113 | +The direct test of H1 is starker. Without any physical adjustment, the five nearest sales within one kilometre yield an MdAPE of \hOneGeoDone{} (coverage \hOneGeoDoneCov{}), and the five nearest within ten kilometres \hOneGeoDten{}; the five most similar sales within ten kilometres yield \hOneSimDten{}. Holding the radius fixed, the selection criterion alone is worth more than four points, as much as the hedonic grid is worth to the nearest sales: the nearby-but-dissimilar comparable is worse than the distant-but-similar one by a wide margin. The reason is that, in these data, the physical characteristics explain a larger share of the within-market variance of prices than micro-location does once the municipality is fixed. With the hedonic grid the two criteria converge (\hOneGeoDtenHedo{} against \hOneSimDtenHedo{}), because adjustment does for the nearest sales what selection does for the most similar; the compromise criterion, which uses both, is best (\hOneMixDtenHedo{}). H1 is confirmed. | |
| 114 | 114 | |
| 115 | 115 | \subsection{Weighting and reconciliation (H4)} |
| 116 | 116 | \label{sec:weighting} |
| 117 | 117 | |
| 118 | −Table~\ref{tab:weighting} reports the ten weighting--reconciliation combinations at the $(5,5,12)$ anchor with the hedonic grid, and Table~\ref{tab:h4} stratifies them by density. The range of the MdAPE across all ten combinations is \wrRange{} points. The median is marginally more robust than the mean, as expected from the heavy tails of adjusted indications; distance and similarity weights help slightly in dense markets, where the nearest and most similar comparables are also the most informative, and are neutral or slightly harmful in sparse ones, where they concentrate weight on one or two sales. H4 is thus supported in direction but the effects are an order of magnitude smaller than those of selection and adjustment. The marginal effects over the whole E2 design (Table~\ref{tab:marginal}) make the hierarchy explicit: the adjustment factor spans \margAdjLo{}--\margAdjHi{} points of MdAPE, selection \margSelLo{}--\margSelHi{}, weighting \margWLo{}--\margWHi{}, and reconciliation \margReconMedian{} (median) against \margReconMean{} (mean). | |
| 118 | +Table~\ref{tab:weighting} reports the ten weighting--reconciliation combinations at the $(5,5,12)$ anchor with the hedonic grid, and Table~\ref{tab:h4} stratifies seven of them by density. The range of the MdAPE across all ten combinations is \wrRange{} points. The median is more robust than the mean, as expected from the heavy tails of adjusted indications, and the mean carries a negative bias of about $-1.5$ points because averaging in dollars overweights the high indications in logs. The density profile does not support H4 in its stated form. The equal-weight median is the best or within a tenth of a point of the best in the three densest strata; weighting by gross adjustment improves on the equal-weight mean in dense markets (\hFourSimDense{} against \hFourEqualMeanDense{} where 200 or more sales are available) but not in sparse ones; distance weights are neutral everywhere; and the product of the three weights with the median is the worst combination in the two sparsest strata, because it concentrates the weight on one or two sales. Weighting thus matters, if anywhere, in liquid markets, which is the opposite of what H4 predicts, and the effects are an order of magnitude smaller than those of selection and adjustment. The marginal effects over the whole E2 design (Table~\ref{tab:marginal}) make the hierarchy explicit: the adjustment factor spans \margAdjLo{}--\margAdjHi{} points of MdAPE, selection \margSelLo{}--\margSelHi{}, weighting \margWLo{}--\margWHi{}, and reconciliation \margReconMedian{} (median) against \margReconMean{} (mean). | |
| 119 | 119 | |
| 120 | 120 | \begin{table}[H] |
| 121 | 121 | \centering |
@@ -123,6 +123,9 @@ Table~\ref{tab:weighting} reports the ten weighting--reconciliation combinations | ||
| 123 | 123 | \label{tab:weighting} |
| 124 | 124 | \small |
| 125 | 125 | \input{tables/weighting} |
| 126 | +\begin{tablenotes}\footnotesize | |
| 127 | +\item Note: ``1/gross adjustment'' weights each comparable by the inverse of the absolute hedonic adjustment it requires; ``product of the three'' multiplies the distance, recency and gross-adjustment weights. | |
| 128 | +\end{tablenotes} | |
| 126 | 129 | \end{table} |
| 127 | 130 | |
| 128 | 131 | \begin{table}[H] |
@@ -131,6 +134,9 @@ Table~\ref{tab:weighting} reports the ten weighting--reconciliation combinations | ||
| 131 | 134 | \label{tab:h4} |
| 132 | 135 | \footnotesize |
| 133 | 136 | \input{tables/h4} |
| 137 | +\begin{tablenotes}\footnotesize | |
| 138 | +\item Note: $(5, 5\text{ km}, 12\text{ mo})$ anchor, selection by distance, hedonic grid. Columns: weighting/reconciliation; ``1/adj.'' is the inverse gross-adjustment weight, ``prod.'' the product of the distance, recency and gross-adjustment weights. The minimum of each row is in bold. | |
| 139 | +\end{tablenotes} | |
| 134 | 140 | \end{table} |
| 135 | 141 | |
| 136 | 142 | \begin{table}[H] |
@@ -146,7 +152,7 @@ Table~\ref{tab:weighting} reports the ten weighting--reconciliation combinations | ||
| 146 | 152 | \subsection{The horse race} |
| 147 | 153 | \label{sec:horserace} |
| 148 | 154 | |
| 149 | −Table~\ref{tab:competition} and Figure~\ref{fig:competition} report the thirteen named models and the empirically optimised configurations. The ordering is informative at every level. The traditional practices M1--M5---a few nearest sales, time-adjusted, weighted in one way or another---cluster at \mdapeMoneRange{} of MdAPE, with fewer than a third of estimates within $\pm10\%$ of the price. Adding a grid (M6, M7) or selecting by similarity (M3) brings the error into the \mdapeGridRange{} range. The assessment ratio (M13) reaches \mdapeMthirteen{} on the targets it covers. The best single transparent configuration frozen on 2024 (M0: \mZeroDesc) reaches \mdapeMzero{} with \pTenMzero{} of estimates within $\pm10\%$, and the optimised reconciliation M9 \mdapeMnine{}. Among the external benchmarks, the direct hedonic model M10 is the least accurate (\mdapeMten{}): a global model with municipality effects cannot match the local information of comparables. The gradient-boosting model without the assessed value (M11a, \mdapeMelevenA{}) is only \transparencyCostNoRole{} points better than the best frozen comparables configuration; with the assessed value (M11b) it reaches \mdapeMelevenB{} and \pTenMelevenB{} within $\pm10\%$, and the hybrid M12 \mdapeMtwelve{}. | |
| 155 | +Table~\ref{tab:competition} and Figure~\ref{fig:competition} report the sixteen named models; Table~\ref{tab:competition_ci} in the Appendix adds cluster-bootstrap confidence intervals for each model's MdAPE and MAPE. The ordering is informative at every level. The traditional practices M1--M5---a few nearest sales, time-adjusted, weighted in one way or another---cluster at \mdapeMoneRange{} of MdAPE, with fewer than a third of estimates within $\pm10\%$ of the price. Adding a grid (M6, M7) or selecting by similarity (M3) brings the error into the \mdapeGridRange{} range. The assessment ratio (M13) reaches \mdapeMthirteen{} on the targets it covers. The best single transparent configuration frozen on 2024 (M0: \mZeroDesc) reaches \mdapeMzero{} (95\% interval \ciMzeroLo{}--\ciMzeroHi{}) with \pTenMzero{} of estimates within $\pm10\%$, and the optimised reconciliation M9 \mdapeMnine{}; M9 places weights of \wNineRole{} on the assessment-ratio indication, \wNineGbm{} on the GBM-adjusted one and \wNineNone{} on the time-only one, and nothing on the other grids. Among the external benchmarks, the direct hedonic model M10 is the least accurate (\mdapeMten{}): a global model with municipality and neighbourhood effects cannot match the local information of comparables. The gradient-boosting model without the assessed value (M11a, \mdapeMelevenA{}) is only \transparencyCostNoRole{} points better than the best frozen comparables configuration; with the assessed value (M11b) it reaches \mdapeMelevenB{} (\ciMelevenBLo{}--\ciMelevenBHi{}) and \pTenMelevenB{} within $\pm10\%$, and the hybrid M12 \mdapeMtwelve{}, with \wTwelveGbm{} of its weight on M11b and the rest on the GBM- and hedonic-adjusted comparables. | |
| 150 | 156 | |
| 151 | 157 | \begin{table}[H] |
| 152 | 158 | \centering |
@@ -154,7 +160,7 @@ Table~\ref{tab:competition} and Figure~\ref{fig:competition} report the thirteen | ||
| 154 | 160 | \label{tab:competition} |
| 155 | 161 | \resizebox{\textwidth}{!}{\input{tables/competition}} |
| 156 | 162 | \begin{tablenotes}\footnotesize |
| 157 | −\item Note: MdAPE, MAPE, median signed error and the shares within $\pm5$, $\pm10$ and $\pm20\%$ in per cent, on the test targets for which the model returns a value; coverage is the share of such targets. The validation column gives the MdAPE on the 2024 targets. Weights of M9 and M12 and the configuration of M0 are chosen on 2024 only. | |
| 163 | +\item Note: MdAPE, MAPE, median signed error and the shares within $\pm5$, $\pm10$ and $\pm20\%$ in per cent, on the test targets for which the model returns a value; coverage is the share of such targets. The validation column gives the MdAPE on the 2024 targets. Weights of M9 and M12 and the configuration of M0 are chosen on 2024 only. Bootstrap confidence intervals for each entry are in Table~\ref{tab:competition_ci}. | |
| 158 | 164 | \end{tablenotes} |
| 159 | 165 | \end{table} |
| 160 | 166 | |
@@ -165,25 +171,25 @@ Table~\ref{tab:competition} and Figure~\ref{fig:competition} report the thirteen | ||
| 165 | 171 | \label{fig:competition} |
| 166 | 172 | \end{figure} |
| 167 | 173 | |
| 168 | −Table~\ref{tab:pairs} gives the paired differences with cluster-bootstrap confidence intervals for the comparisons that matter. The gain from the hedonic grid over time-only adjustment (M7 $-$ M2), the gain from similarity selection (M3 $-$ M2), and the gain of M11b over M0 are all precisely estimated and far from zero. The difference between the conventional and hedonic grids (M7 $-$ M6) is small; so is that between local and global grids. The hybrid M12 improves significantly on M11b, which shows that comparables carry local information that a quarterly-retrained tree model with coordinates does not fully extract. | |
| 174 | +Table~\ref{tab:pairs} gives the paired differences with cluster-bootstrap confidence intervals and cluster-robust $t$ statistics for the comparisons that matter. The gain from the hedonic grid over time-only adjustment (M7 $-$ M2, $t = \tMsevenMtwo{}$), the gain from similarity selection (M3 $-$ M2), and the gain of M11b over M0 ($t = \tMelevenBMzero{}$) are all precisely estimated and far from zero. The difference between the conventional and hedonic grids (M7 $-$ M6) is small but significant, as is the loss from the local grid (M8a $-$ M7). The hybrid M12 improves on M11b by only six hundredths of a point of MdAPE, a difference that is statistically distinguishable from zero ($t = \tMtwelveMelevenB{}$) but economically negligible: a quarterly-retrained tree model with coordinates and the assessed value already extracts almost all of the local information that comparables carry. On the common support of the ratio method, M0 has a lower MdAPE than M13 but a higher MAPE, so that the two are not ranked by all measures. | |
| 169 | 175 | |
| 170 | 176 | \begin{table}[H] |
| 171 | 177 | \centering |
| 172 | 178 | \caption{Paired differences in accuracy with cluster-bootstrap 95\% confidence intervals} |
| 173 | 179 | \label{tab:pairs} |
| 174 | 180 | \footnotesize |
| 175 | −\input{tables/pairs} | |
| 181 | +\resizebox{\textwidth}{!}{\input{tables/pairs_b999}} | |
| 176 | 182 | \begin{tablenotes}\footnotesize |
| 177 | −\item Note: $\Delta$ is the difference $A-B$ in MdAPE and in MAPE (percentage points) on the common test targets; a negative value means $A$ is more accurate. Confidence intervals from 400 (MAPE) and 150 (MdAPE) cluster-bootstrap replications over municipality--quarter cells. The last column is the share of targets on which $A$ has the smaller error. | |
| 183 | +\item Note: $\Delta$ is the difference $A-B$ in MdAPE and in MAPE (percentage points) on the common test targets; a negative value means $A$ is more accurate. Confidence intervals from 999 cluster-bootstrap replications over municipality--quarter cells; $t$ is the cluster-robust statistic of equation~\eqref{eq:dm} for the MAPE difference. ``$A$ better'' is the share of targets on which $A$ has the smaller error. | |
| 178 | 184 | \end{tablenotes} |
| 179 | 185 | \end{table} |
| 180 | 186 | |
| 181 | −Two findings deserve emphasis. The first is the \emph{cost of transparency}: \transparencyCost{} points of MdAPE separate the best frozen configuration of the sales comparison approach from the best opaque model with the same information plus the assessed value, and \transparencyCostNoRole{} points separate it from the same model without the assessed value. Every estimate of M0 is the median of \bestK{} identified sales adjusted by published coefficients; every estimate of M11b is the output of a thousand trees. The second is the \emph{value of the assessed value}. The ratio method M13 and the GBM with roll information M11b dominate their counterparts without it by \roleGainRatio{} and \roleGainGbm{} points respectively. The assessed value summarises characteristics that the roll extract does not expose to us---quality, condition, outbuildings---and a local price level that the assessor has already estimated; it is, in effect, a professional valuation available for every property, and the evidence says it should be used. | |
| 187 | +Two findings deserve emphasis. The first is the \emph{cost of transparency}: \transparencyCost{} points of MdAPE separate the best frozen configuration of the sales comparison approach from the best opaque model with the same information plus the assessed value, and \transparencyCostNoRole{} points separate it from the same model without the assessed value. Every estimate of M0 is the median of \bestK{} identified sales adjusted by published coefficients; every estimate of M11b is the output of several hundred trees. The second is the \emph{value of the assessed value}. The ratio method M13 and the GBM with roll information M11b dominate their counterparts without it by \roleGainRatio{} and \roleGainGbm{} points respectively. The assessed value summarises characteristics that the roll extract does not expose to us---quality, condition, outbuildings---and a local price level that the assessor has already estimated; it is, in effect, a professional valuation available for every property, and the evidence says it should be used, as \citet{thibodeau2003} argued for marking assessed values to market. | |
| 182 | 188 | |
| 183 | 189 | \subsection{Where does the method hold, and where does it fail?} |
| 184 | 190 | \label{sec:strata} |
| 185 | 191 | |
| 186 | −Table~\ref{tab:density} and Figure~\ref{fig:density} report accuracy by market density for the main models. The dependence is steep and common to all methods: the MdAPE of the hedonic-grid method M7 is \densHighMseven{} where two hundred or more sales occurred within 2~km in the past year and \densLowMseven{} where fewer than five did. The gap between the best transparent method and the best opaque one is widest in the sparsest markets, where the tree model's ability to borrow strength from a wider area is most valuable, and narrowest in the densest ones. Tables~\ref{tab:regions}--\ref{tab:price} in the Appendix report the same comparison by region, price quintile, age and physical link; accuracy is highest in the dense suburban regions (Laval, Outaouais, Montérégie) and lowest in the remote regions (Gaspésie, Nord-du-Québec, Côte-Nord), in the lowest price quintile (\prixQoneMseven{} for M7, against \prixQfourMseven{} in the fourth), for dwellings older than seventy years and for integrated dwellings---the segments where condition and unobserved characteristics carry the largest share of value. | |
| 192 | +Table~\ref{tab:density} and Figure~\ref{fig:density} report accuracy by market density for the main models. The dependence is steep and common to all methods: the MdAPE of the hedonic-grid method M7 is \densHighMseven{} where two hundred or more sales occurred within 2~km in the past year and \densLowMseven{} where fewer than five did. The gap between the best transparent method and the best opaque one is widest in the sparsest markets, where the tree model's ability to borrow strength from a wider area is most valuable, and narrowest in the densest ones; the pattern is the comparable-sales counterpart of the rural accuracy penalty that \citet{bogin2020} document for both appraisers and AVMs. Tables~\ref{tab:regions}--\ref{tab:price} in the Appendix report the same comparison by region, price quintile, age and physical link; accuracy is highest in the dense suburban regions (Laval, Outaouais, Montérégie) and lowest in the remote regions (Gaspésie, Nord-du-Québec, Côte-Nord), in the lowest price quintile (\prixQoneMseven{} for M7, against \prixQfourMseven{} in the fourth), for dwellings older than seventy years and for integrated dwellings---the segments where condition and unobserved characteristics carry the largest share of value. The lowest quintile deserves a word of caution: because quintiles are defined on the realised price, the APE of a cheap sale is mechanically inflated by any given dollar error, so part of the gradient across quintiles is a denominator effect rather than a failure of the method. | |
| 187 | 193 | |
| 188 | 194 | \begin{table}[H] |
| 189 | 195 | \centering |
@@ -191,6 +197,9 @@ Table~\ref{tab:density} and Figure~\ref{fig:density} report accuracy by market d | ||
| 191 | 197 | \label{tab:density} |
| 192 | 198 | \footnotesize |
| 193 | 199 | \input{tables/density} |
| 200 | +\begin{tablenotes}\footnotesize | |
| 201 | +\item Note: $n$ is the number of test targets of the stratum covered by M7; in the sparsest stratum it is \densLowN{} of \densLowNall{} targets because five sales within 5~km and 12 months are not always available. Each model's MdAPE is computed on the targets it covers. | |
| 202 | +\end{tablenotes} | |
| 194 | 203 | \end{table} |
| 195 | 204 | |
| 196 | 205 | \begin{figure}[H] |
@@ -200,7 +209,7 @@ Table~\ref{tab:density} and Figure~\ref{fig:density} report accuracy by market d | ||
| 200 | 209 | \label{fig:density} |
| 201 | 210 | \end{figure} |
| 202 | 211 | |
| 203 | −Table~\ref{tab:h5} addresses H5 by splitting the test targets into terciles of submarket heterogeneity, measured by the dispersion of the adjusted indications of the ten nearest comparables within 10~km and 24 months---a quantity available at valuation time. In the homogeneous tercile the transparent configuration M0 is within \hFiveGapHomog{} points of M11b; in the heterogeneous tercile the gap is \hFiveGapHeter{} points. Simple, explainable rules rival machine learning precisely where the market is informative enough for a few well-chosen sales to pin down value; machine learning earns its keep where the market is thin or heterogeneous. | |
| 212 | +Table~\ref{tab:h5} addresses H5 by splitting the test targets into terciles of submarket heterogeneity, measured by the dispersion of the adjusted indications of the ten nearest comparables within 10~km and 24 months---a quantity available at valuation time. In the homogeneous tercile the transparent configuration M0 is within \hFiveGapHomog{} points of M11b; in the heterogeneous tercile the gap is \hFiveGapHeter{} points. Simple, explainable rules rival machine learning precisely where the market is informative enough for a few well-chosen sales to pin down value; machine learning earns its keep where the market is thin or heterogeneous. H5 is confirmed in direction, although the gap never closes entirely. | |
| 204 | 213 | |
| 205 | 214 | \begin{table}[H] |
| 206 | 215 | \centering |
@@ -209,20 +218,23 @@ Table~\ref{tab:h5} addresses H5 by splitting the test targets into terciles of s | ||
| 209 | 218 | \footnotesize |
| 210 | 219 | \input{tables/h5} |
| 211 | 220 | \begin{tablenotes}\footnotesize |
| 212 | −\item Note: terciles of the standard deviation of the log adjusted indications of the ten nearest sales within 10~km and 24 months (hedonic grid), computed on the test targets. | |
| 221 | +\item Note: terciles of the standard deviation $\hat\sigma$ of the log adjusted indications of the ten nearest sales within 10~km and 24 months (hedonic grid), computed on the test targets. | |
| 213 | 222 | \end{tablenotes} |
| 214 | 223 | \end{table} |
| 215 | 224 | |
| 216 | 225 | \subsection{Value intervals} |
| 217 | 226 | \label{sec:intervals} |
| 218 | 227 | |
| 219 | −The dispersion of the adjusted indications provides an interval of value. Table~\ref{tab:intervals} reports, for the time-only, hedonic and GBM adjustments at two anchors, the coverage of the interval $\hat P \exp(\pm z\,\hat\sigma)$ with $z=1.28$ (nominal 80\% under normality) and with $z$ calibrated on the validation targets to achieve 80\% coverage, together with the median width of the calibrated interval. The nominal intervals under-cover (67--78\% for a nominal 80\%): the dispersion of five or ten adjusted sales understates the uncertainty of the estimate, because the comparables share unobserved characteristics with each other but not with the subject. Calibrated on 2024, the intervals achieve their target coverage in 2025--2026 to within half a point, but at a median width of \ivWidth{} of value: an honest 80\% interval around a comparables-based estimate spans roughly $\pm$30\%. The dispersion of the comparables is a valid but coarse measure of uncertainty, and the breadth of the calibrated interval is itself a finding about the precision the method can claim. | |
| 228 | +The dispersion of the adjusted indications provides an interval of value. Table~\ref{tab:intervals} reports, for the time-only, hedonic and GBM adjustments at two anchors, the coverage of the interval $\hat P_i \exp(\pm z\,\hat\sigma_i)$ with $z=1.28$ (nominal 80\% under normality) and with $z$ calibrated on the validation targets as the 80th percentile of the standardised absolute errors $|\ln P_i - \ln \hat P_i|/\hat\sigma_i$---a split-conformal calibration in the sense of \citet{lei2018}---together with the median width of the calibrated interval. The nominal intervals under-cover (\ivCovLo{}--\ivCovHi{}\% for a nominal 80\%): the dispersion of five or ten adjusted sales understates the uncertainty of the estimate, because the comparables share unobserved characteristics with each other but not with the subject, which is the error-based versus model-based distinction drawn by \citet{krause2020}. Calibrated on 2024, the intervals achieve their target coverage in 2025--2026 to within half a point, but at a median width of \ivWidth{} of value: an honest 80\% interval around a comparables-based estimate spans roughly $\pm$\ivHalf{}. The dispersion of the comparables is a valid but coarse measure of uncertainty, and the breadth of the calibrated interval is itself a finding about the precision the method can claim. | |
| 220 | 229 | |
| 221 | 230 | \begin{table}[H] |
| 222 | 231 | \centering |
| 223 | 232 | \caption{Coverage of value intervals derived from the dispersion of adjusted indications} |
| 224 | 233 | \label{tab:intervals} |
| 225 | 234 | \resizebox{\textwidth}{!}{\input{tables/intervals}} |
| 235 | +\begin{tablenotes}\footnotesize | |
| 236 | +\item Note: selection by distance, equal weights, median. Coverage is the share of test targets whose price lies within the interval; width is $2z\hat\sigma$ in per cent of value. | |
| 237 | +\end{tablenotes} | |
| 226 | 238 | \end{table} |
| 227 | 239 | |
| 228 | 240 | \subsection{Towards an empirical comparable-recommendation system} |
modified
paper/sections/robustness.tex
+65 −7
@@ -3,9 +3,10 @@ | ||
| 3 | 3 | \label{sec:robustness} |
| 4 | 4 | % ============================================================================ |
| 5 | 5 | |
| 6 | −\subsection{Stability over time} | |
| 6 | +\subsection{Stability over time and seasonality} | |
| 7 | +\label{sec:stability} | |
| 7 | 8 | |
| 8 | −Table~\ref{tab:stability} reports the MdAPE and bias of the main models by quarter of the test period. The rankings are stable quarter by quarter, and the validation-period (2024) and test-period (2025--2026) MdAPEs of every configuration in Tables~\ref{tab:adjustments}--\ref{tab:competition} differ by about a point in the same direction, which indicates that the rules selected on 2024 did not overfit it. The comparable-based methods display a seasonal bias: estimates are too low in the first quarter (\biasQoneMseven{} for M7) and too high in the third (\biasQthreeMseven{}). This is the signature of the point-in-time constraint---every method works at the level of the previous month, and Quebec prices rise into the spring and soften after the summer---and of the fact that the price level of the valuation month is not knowable on the valuation date. The gradient-boosting benchmark, which includes the month of sale as a feature, absorbs most of this seasonality; a seasonal adjustment estimated on prior years would do the same for the comparables method. | |
| 9 | +Table~\ref{tab:stability} reports the MdAPE and bias of the main models by quarter of the test period. The rankings are stable quarter by quarter, and the validation-period (2024) and test-period (2025--2026) MdAPEs of every configuration in Tables~\ref{tab:adjustments}--\ref{tab:competition} differ by about a point in the same direction, which indicates that the rules selected on 2024 did not overfit it; the test period was simply slightly easier for every method. The comparable-based methods display a seasonal bias: pooling the two years by calendar quarter (Table~\ref{tab:seasonal}), the hedonic-grid method M7 is too low in the first quarter (\seasQoneMseven{}) and too high in the third (\seasQthreeMseven{}), and the best configuration M0 and the direct hedonic model M10 show the same pattern (\seasQoneMzero{} and \seasQthreeMzero{}; \seasQoneMten{} and \seasQthreeMten{}). This is the signature of the point-in-time constraint: every method works at the level of the previous month, Quebec prices rise into the spring and soften after the summer \citep{ngai2014}, and the price level of the valuation month is not knowable on the valuation date. The gradient-boosting benchmarks do not share the pattern (M11b: \seasQoneMelevenB{} and \seasQthreeMelevenB{}). The reason is not that they observe the month of sale---at prediction time their month feature is fixed at the last training month, and their level is carried forward by the same index as the comparables---and we have not identified it; it may reflect the composition of the comparable pool, which over-represents the busy spring and summer months when the target sells in winter. A seasonal correction estimated on prior years is the obvious remedy for the comparables method, and we flag it as an extension that we have not implemented. | |
| 9 | 10 | |
| 10 | 11 | \begin{table}[H] |
| 11 | 12 | \centering |
@@ -15,22 +16,79 @@ Table~\ref{tab:stability} reports the MdAPE and bias of the main models by quart | ||
| 15 | 16 | \input{tables/stability} |
| 16 | 17 | \end{table} |
| 17 | 18 | |
| 19 | +\begin{table}[H] | |
| 20 | +\centering | |
| 21 | +\caption{Median signed error (\%) by calendar quarter, test targets of 2025 and 2026 pooled} | |
| 22 | +\label{tab:seasonal} | |
| 23 | +\footnotesize | |
| 24 | +\input{tables/seasonal_bias} | |
| 25 | +\end{table} | |
| 26 | + | |
| 27 | +\subsection{Selection stability and data snooping} | |
| 28 | +\label{sec:selection_stability} | |
| 29 | + | |
| 30 | +The configuration M0 is the best of \nConfigsCov{} configurations with at least 97\% coverage on the 2024 validation targets, and its test-period accuracy is reported on sales that played no role in that choice. Two further checks establish that the choice is not fragile. First, the validation and test rankings of the \nConfigs{} configurations are almost identical: the Spearman rank correlation between their 2024 and 2025--2026 MdAPEs is \spearmanAll{} over all configurations and \spearmanCov{} among those eligible for selection. Second, M0 ranks \mZeroRankTest{}nd of \nConfigsCov{} in the test period, within a hundredth of a point of the configuration that would have been chosen with hindsight (\oracleTest{}), which replaces the hedonic grid by GBM adjustments and is not transparent. Table~\ref{tab:selection_top10} lists the ten best configurations on 2024 and their test ranks: all ten combine the compromise selection, a 24-month window and the hedonic grid, differ only in $k$ (20 to 50) and $d$ (2 to 25~km), and all ten rank among the twelve best out of sample. Table~\ref{tab:selection_near} shows that the \nearQuarterN{} configurations within a quarter of a point of the best validation MdAPE have test MdAPEs between \nearQuarterMin{} and \nearQuarterMax{}. The accuracy attributed to M0 is therefore a property of a broad plateau of wide-net, similarity-based, adjusted configurations rather than of one lucky draw, which is what protects the headline figure from the data-snooping bias of \citet{white2000}. | |
| 31 | + | |
| 32 | +\begin{table}[H] | |
| 33 | +\centering | |
| 34 | +\caption{The ten best configurations on the 2024 validation targets and their out-of-sample rank} | |
| 35 | +\label{tab:selection_top10} | |
| 36 | +\footnotesize | |
| 37 | +\resizebox{\textwidth}{!}{\input{tables/selection_top10}} | |
| 38 | +\begin{tablenotes}\footnotesize | |
| 39 | +\item Note: configurations with at least 97\% coverage on 2024 (\nConfigsCov{} of \nConfigs{}). ``Rank 2025--26'' is the rank of the configuration's test MdAPE among the same \nConfigsCov{} configurations. | |
| 40 | +\end{tablenotes} | |
| 41 | +\end{table} | |
| 42 | + | |
| 43 | +\begin{table}[H] | |
| 44 | +\centering | |
| 45 | +\caption{Near-optimal configurations on 2024 and the range of their out-of-sample MdAPE (\%)} | |
| 46 | +\label{tab:selection_near} | |
| 47 | +\footnotesize | |
| 48 | +\input{tables/selection_near} | |
| 49 | +\end{table} | |
| 50 | + | |
| 18 | 51 | \subsection{Coverage and the common-support comparison} |
| 19 | 52 | |
| 20 | −Configurations with narrow radii or short windows abstain on a substantial share of targets, and their MdAPE on the targets they cover is not comparable to that of configurations that cover all targets. Table~\ref{tab:surface_cov} in the Appendix reports the coverage of every point of the error surface. Our configuration selection enforces 97\% coverage; relaxing the constraint selects a narrower configuration (\mZeroUncDesc) with a lower MdAPE on its own support (\mZeroUncMdape{}) but \mZeroUncCov{} coverage. The conclusions of Section~\ref{sec:surface} hold on the common support of all configurations with at least 97\% coverage. | |
| 53 | +Configurations with narrow radii or short windows abstain on a substantial share of targets, and their MdAPE on the targets they cover is not comparable to that of configurations that cover all targets. Table~\ref{tab:surface_cov} in the Appendix reports the coverage of every point of the error surface. Our configuration selection enforces 97\% coverage; relaxing the constraint selects a narrower configuration (\mZeroUncDesc) with a lower MdAPE on its own support (\mZeroUncMdape{}) but \mZeroUncCov{} coverage. The conclusions of Section~\ref{sec:surface} hold on the common support of all configurations with at least 97\% coverage, and the paired comparisons of Table~\ref{tab:pairs} are always computed on the targets covered by both methods. | |
| 21 | 54 | |
| 22 | 55 | \subsection{Exclusion rules} |
| 56 | +\label{sec:exclusion} | |
| 23 | 57 | |
| 24 | −The exclusion of extreme price-to-assessment ratios (rule R8) is the only rule that removes a non-trivial share of sales on a criterion related to price. The excluded sales are, by construction, those on which no method based on market evidence can succeed, and they are excluded from the comparable pool as an appraiser would exclude them; their inclusion as targets would raise the error of every method by a similar amount without affecting the comparisons, which are all paired on the same targets. A sensitivity analysis to the bounds of R8 and to the flip rule R9 is left for the next version of this paper. | |
| 58 | +The exclusion of extreme price-to-assessment ratios (rule R8) is the only rule that removes a non-trivial share of sales on a criterion related to price. Its bounds affect the results through two channels: the composition of the target set, on which every method is evaluated, and the composition of the comparable pool. Table~\ref{tab:r8} examines the first channel with the existing estimates by tightening the bounds on the targets' ratio while leaving the pool unchanged. Restricting the test targets to ratios in $[0.5, 2]$ keeps \rEightTightShare{} of them and lowers every MdAPE by about half a point (M0 \rEightTightMzero{}, M11b \rEightTightMelevenB{}); the ordering of the models and the gaps between them are unchanged, and even the drastic $[0.67, 1.5]$ window, which keeps four targets in five, preserves them. The sales removed by tighter bounds are those with the largest errors for every method, as one would expect of transactions whose price departs from any market reference. The second channel---re-running the backtest with a different pool---requires the distributed computation and was not re-executed; because the baseline bounds are wide and remove only 1.8\% of the raw sales, and because an appraiser would in any case discard such sales from the comparable set, we do not expect it to alter the comparisons. The flip rule R9 removes 794 sales and cannot matter at the scale of the sample. | |
| 25 | 59 | |
| 26 | −\subsection{Alternative accuracy measures} | |
| 60 | +\begin{table}[H] | |
| 61 | +\centering | |
| 62 | +\caption{Sensitivity to the price-to-assessment bounds of rule R8, test targets: MdAPE (\%)} | |
| 63 | +\label{tab:r8} | |
| 64 | +\footnotesize | |
| 65 | +\resizebox{\textwidth}{!}{\input{tables/r8_sensitivity}} | |
| 66 | +\begin{tablenotes}\footnotesize | |
| 67 | +\item Note: the test targets are restricted to the stated interval of the ratio of sale price to assessed value; comparable pools, grids and models are unchanged. ``Share'' is the share of the \nTargetsTest{} test targets retained. | |
| 68 | +\end{tablenotes} | |
| 69 | +\end{table} | |
| 27 | 70 | |
| 28 | −The MdAPE is our headline measure because the distribution of valuation errors is heavy-tailed and the median is the appraiser's natural loss. Table~\ref{tab:competition} shows that the MAPE and the shares within $\pm5$, $\pm10$ and $\pm20\%$ rank the models identically; the paired bootstrap on MAPE differences in Table~\ref{tab:pairs} yields the same conclusions as on MdAPE differences. The bias column shows that no comparable-based method is materially biased out of sample. | |
| 71 | +\subsection{Alternative accuracy measures and cluster definition} | |
| 72 | +\label{sec:clusters} | |
| 73 | + | |
| 74 | +The MdAPE is our headline measure because the distribution of valuation errors is heavy-tailed and the median is the appraiser's natural loss. Table~\ref{tab:competition} shows that the MAPE and the shares within $\pm5$, $\pm10$ and $\pm20\%$ rank the models almost identically---the only inversion is between M9 and M11a, which the MAPE separates more clearly than the MdAPE---and the paired bootstrap on MAPE differences in Table~\ref{tab:pairs} yields the same conclusions as on MdAPE differences. The bias column shows that no comparable-based method is materially biased out of sample once the whole test period is pooled. Table~\ref{tab:cluster_sens} re-estimates the confidence intervals of four key paired differences under alternative cluster definitions, from municipality--quarter cells (the baseline, with several thousand clusters) to region--quarter cells (about a hundred). The intervals widen as clusters become coarser, as they should when dependence at wider scales is allowed for, but none of the conclusions changes: the smallest difference that we call significant, M12 $-$ M11b, remains distinguishable from zero even with region--quarter clusters. | |
| 75 | + | |
| 76 | +\begin{table}[H] | |
| 77 | +\centering | |
| 78 | +\caption{Sensitivity of the bootstrap confidence intervals to the cluster definition: 95\% intervals of the MdAPE difference (points)} | |
| 79 | +\label{tab:cluster_sens} | |
| 80 | +\footnotesize | |
| 81 | +\resizebox{\textwidth}{!}{\input{tables/pairs_cluster_sens}} | |
| 82 | +\begin{tablenotes}\footnotesize | |
| 83 | +\item Note: 399 cluster-bootstrap replications for each alternative definition (999 for the baseline, Table~\ref{tab:pairs}). | |
| 84 | +\end{tablenotes} | |
| 85 | +\end{table} | |
| 29 | 86 | |
| 30 | 87 | \subsection{Index construction} |
| 31 | 88 | |
| 32 | 89 | The point-in-time index uses an expanding window of all prior sales and municipality fixed effects. Because comparables are at most 24 months old and the time adjustment contributes only \gainTime{}--\gainTimeLong{} points of accuracy (Section~\ref{sec:adjustments}), the comparable-based results are insensitive to the details of the index. The global benchmarks are more exposed: their level in 2025--2026 is carried entirely by the index from the last training quarter to the valuation month, and we verified that an index without municipality fixed effects, which is contaminated by the shift of sales towards less expensive municipalities, lowers their level. |
| 33 | 90 | |
| 34 | 91 | \subsection{Benchmark specification} |
| 92 | +\label{sec:benchspec} | |
| 35 | 93 | |
| 36 | −The gradient-boosting benchmarks are trained with an L1 objective so that they estimate the conditional median, which is the quantity the MdAPE rewards; the same models trained with a squared-error objective have a median bias of about $-5\%$ in-sample and out of sample, because the distribution of log prices conditional on characteristics is left-skewed, and a correspondingly higher MdAPE; the direct hedonic model M10, estimated by least squares, is for the same reason recentred on the median of its validation residuals. The number of trees chosen by early stopping on the last 10\% of the training window is floored at 300 because the final, partial months of the sample make that slice unreliable. The hybrid weights of M12 and the reconciliation weights of M9 are convex combinations fitted on 2024 and applied unchanged; M9 places most of its weight on the assessment-ratio and GBM-adjusted indications, and M12 over 90\% of its weight on M11b. | |
| 94 | +The gradient-boosting benchmarks are trained with an $L_1$ objective so that they estimate the conditional median, which is the quantity the MdAPE rewards; the same models trained with a squared-error objective have a median bias of about $-5\%$ in-sample and out of sample, because the distribution of log prices conditional on characteristics is left-skewed, and a correspondingly higher MdAPE; the direct hedonic model M10, estimated by least squares, is for the same reason recentred on the median of its validation residuals. The number of trees chosen by early stopping on the last 10\% of the training window is floored at 300 because the final, partial months of the sample make that slice unreliable. The hybrid weights of M12 and the reconciliation weights of M9 are convex combinations fitted on 2024 and applied unchanged; M9 places \wNineRole{} of its weight on the assessment-ratio indication and \wNineGbm{} on the GBM-adjusted one, and M12 places \wTwelveGbm{} on M11b. Neither combination was tuned beyond the choice of its components. | |
added
paper/tables/competition_ci.tex
+22 −0
@@ -0,0 +1,22 @@ | ||
| 1 | +\begin{tabular}{lrrrrr} | |
| 2 | +\toprule | |
| 3 | +Model & MdAPE & 95\% CI & MAPE & 95\% CI & $n$ \\ | |
| 4 | +\midrule | |
| 5 | +M12 & 8.91 & [8.73; 9.13] & 19.85 & [19.40; 20.37] & 124,559 \\ | |
| 6 | +M11b & 8.97 & [8.77; 9.20] & 19.93 & [19.46; 20.45] & 124,559 \\ | |
| 7 | +M9 & 10.64 & [10.38; 10.92] & 21.79 & [21.28; 22.33] & 124,310 \\ | |
| 8 | +M11a & 10.74 & [10.43; 11.04] & 23.12 & [22.53; 23.76] & 124,559 \\ | |
| 9 | +M0 & 11.13 & [10.84; 11.45] & 23.62 & [23.04; 24.25] & 124,555 \\ | |
| 10 | +M13 & 11.50 & [11.22; 11.80] & 22.48 & [21.98; 23.03] & 112,423 \\ | |
| 11 | +M8b & 11.93 & [11.63; 12.28] & 24.08 & [23.49; 24.70] & 124,310 \\ | |
| 12 | +M3 & 13.00 & [12.65; 13.40] & 26.19 & [25.52; 26.90] & 124,310 \\ | |
| 13 | +M7 & 13.01 & [12.71; 13.35] & 25.12 & [24.52; 25.75] & 124,310 \\ | |
| 14 | +M6 & 13.38 & [13.06; 13.74] & 26.44 & [25.78; 27.16] & 124,310 \\ | |
| 15 | +M8a & 13.47 & [13.12; 13.86] & 25.71 & [25.12; 26.33] & 124,310 \\ | |
| 16 | +M10 & 13.84 & [13.46; 14.31] & 25.90 & [25.28; 26.54] & 124,559 \\ | |
| 17 | +M4 & 17.85 & [17.44; 18.32] & 31.39 & [30.71; 32.18] & 124,310 \\ | |
| 18 | +M2 & 18.09 & [17.67; 18.55] & 31.77 & [31.07; 32.57] & 124,310 \\ | |
| 19 | +M5 & 18.50 & [18.06; 18.99] & 32.29 & [31.58; 33.09] & 124,310 \\ | |
| 20 | +M1 & 19.15 & [18.69; 19.68] & 32.80 & [32.12; 33.58] & 120,372 \\ | |
| 21 | +\bottomrule | |
| 22 | +\end{tabular} | |
modified
paper/tables/h2.tex
+8 −0
@@ -2,10 +2,18 @@ | ||
| 2 | 2 | \toprule |
| 3 | 3 | Density & $k=1$ & $k=3$ & $k=5$ & $k=7$ & $k=10$ & $k=15$ & $k=20$ & $k=30$ & $k=50$ \\ |
| 4 | 4 | \midrule |
| 5 | +\multicolumn{10}{l}{\emph{Hedonic adjustment grid}} \\ | |
| 5 | 6 | $<5$ & 31.3 & 26.9 & 25.3 & 24.4 & 24.1 & 23.7 & 23.8 & \textbf{23.6} & 23.7 \\ |
| 6 | 7 | 5--19 & 25.4 & 21.6 & 20.4 & 19.6 & 19.1 & 18.7 & 18.4 & 18.4 & \textbf{18.2} \\ |
| 7 | 8 | 20--49 & 20.1 & 16.6 & 15.4 & 14.9 & 14.5 & 14.3 & 14.1 & 14.1 & \textbf{13.9} \\ |
| 8 | 9 | 50--199 & 14.7 & 12.2 & 11.5 & 11.2 & 11.0 & 10.9 & \textbf{10.9} & \textbf{10.9} & 10.9 \\ |
| 9 | 10 | $\ge 200$ & 12.9 & 10.8 & 10.2 & 9.9 & 9.7 & 9.6 & 9.6 & \textbf{9.6} & 9.7 \\ |
| 11 | +\midrule | |
| 12 | +\multicolumn{10}{l}{\emph{Time adjustment only}} \\ | |
| 13 | +$<5$ & 39.2 & 34.7 & 33.2 & 32.2 & 32.1 & 32.1 & 31.7 & 31.7 & \textbf{31.5} \\ | |
| 14 | +5--19 & 33.8 & 29.4 & 28.3 & 27.7 & 27.2 & \textbf{27.2} & 27.4 & 27.6 & 27.8 \\ | |
| 15 | +20--49 & 26.6 & 23.0 & 22.3 & 21.9 & \textbf{21.4} & 21.7 & 21.7 & 22.2 & 22.4 \\ | |
| 16 | +50--199 & 18.5 & 15.9 & 15.4 & \textbf{15.3} & 15.3 & 15.6 & 15.7 & 16.1 & 16.6 \\ | |
| 17 | +$\ge 200$ & 16.0 & 13.7 & 13.1 & 13.1 & \textbf{12.9} & 13.0 & 13.2 & 13.7 & 14.3 \\ | |
| 10 | 18 | \bottomrule |
| 11 | 19 | \end{tabular} |
modified
paper/tables/h4.tex
+1 −1
@@ -1,6 +1,6 @@ | ||
| 1 | 1 | \begin{tabular}{lrrrrrrr} |
| 2 | 2 | \toprule |
| 3 | −Density & equal/med. & equal/mean & dist./mean & rec./mean & sim./mean & prod./mean & prod./med. \\ | |
| 3 | +Density & equal/med. & equal/mean & dist./mean & rec./mean & 1/adj./mean & prod./mean & prod./med. \\ | |
| 4 | 4 | \midrule |
| 5 | 5 | $<5$ & 25.3 & 25.1 & \textbf{25.0} & 25.9 & 25.2 & 25.8 & 28.1 \\ |
| 6 | 6 | 5--19 & 20.4 & 20.6 & 20.5 & 21.0 & \textbf{20.3} & 20.6 & 21.8 \\ |
added
paper/tables/index_growth.tex
+15 −0
@@ -0,0 +1,15 @@ | ||
| 1 | +\begin{tabular}{lrrrrr} | |
| 2 | +\toprule | |
| 3 | +Region & 2021-01 to 2026-09 (\%) & Peak & Trough & Peak to trough (\%) & 2024-12 to 2026-09 (\%) \\ | |
| 4 | +\midrule | |
| 5 | +Quebec (province) & 79.0 & 2022-06 & 2023-02 & -13.1 & 19.6 \\ | |
| 6 | +Montréal & 33.2 & 2022-06 & 2023-03 & -11.9 & 3.6 \\ | |
| 7 | +Capitale-Nationale & 81.0 & 2022-08 & 2023-02 & -14.1 & 25.4 \\ | |
| 8 | +Montérégie & 66.4 & 2022-06 & 2023-02 & -18.0 & 13.9 \\ | |
| 9 | +Laval & 48.3 & 2022-06 & 2023-02 & -16.2 & 12.6 \\ | |
| 10 | +Outaouais & 68.6 & 2022-06 & 2023-02 & -11.5 & 13.9 \\ | |
| 11 | +Laurentides & 73.6 & 2022-06 & 2023-02 & -14.3 & 16.1 \\ | |
| 12 | +Lanaudière & 88.0 & 2022-06 & 2023-02 & -15.4 & 20.1 \\ | |
| 13 | +Estrie & 112.0 & 2022-08 & 2023-02 & -15.4 & 26.1 \\ | |
| 14 | +\bottomrule | |
| 15 | +\end{tabular} | |
modified
paper/tables/marginal.tex
+9 −9
@@ -2,24 +2,24 @@ | ||
| 2 | 2 | \toprule |
| 3 | 3 | Factor & Level & Mean MdAPE \\ |
| 4 | 4 | \midrule |
| 5 | −Selection & least adjusted & 13.1 \\ | |
| 6 | 5 | Selection & nearest & 14.4 \\ |
| 7 | −Selection & compromise & 12.9 \\ | |
| 8 | 6 | Selection & most similar & 13.2 \\ |
| 7 | +Selection & compromise & 12.9 \\ | |
| 8 | +Selection & least adjusted & 13.1 \\ | |
| 9 | +Adjustment & none & 15.4 \\ | |
| 10 | +Adjustment & time only & 14.7 \\ | |
| 9 | 11 | Adjustment & conventional grid & 13.2 \\ |
| 10 | −Adjustment & GBM adjustments & 12.6 \\ | |
| 11 | 12 | Adjustment & hedonic grid & 13.1 \\ |
| 12 | 13 | Adjustment & regional grid & 13.1 \\ |
| 13 | 14 | Adjustment & local grid & 13.3 \\ |
| 14 | −Adjustment & time only & 14.7 \\ | |
| 15 | −Adjustment & none & 15.4 \\ | |
| 15 | +Adjustment & GBM adjustments & 12.6 \\ | |
| 16 | 16 | Adjustment & assessment ratio & 11.8 \\ |
| 17 | −Weighting & distance $\times$ recency $\times$ similarity & 13.6 \\ | |
| 18 | −Weighting & 1/distance & 13.4 \\ | |
| 19 | 17 | Weighting & equal & 13.2 \\ |
| 18 | +Weighting & 1/distance & 13.4 \\ | |
| 20 | 19 | Weighting & recency & 13.6 \\ |
| 21 | −Weighting & most similar & 13.1 \\ | |
| 22 | −Reconciliation & mean & 13.6 \\ | |
| 20 | +Weighting & 1/gross adjustment & 13.1 \\ | |
| 21 | +Weighting & product of the three & 13.6 \\ | |
| 23 | 22 | Reconciliation & median & 13.2 \\ |
| 23 | +Reconciliation & mean & 13.6 \\ | |
| 24 | 24 | \bottomrule |
| 25 | 25 | \end{tabular} |
modified
paper/tables/numbers.tex
+16 −0
@@ -75,6 +75,7 @@ | ||
| 75 | 75 | \newcommand{\hOneGeoDone}{17.2\%} |
| 76 | 76 | \newcommand{\hOneGeoDoneCov}{96.0\%} |
| 77 | 77 | \newcommand{\hOneSimDten}{13.0\%} |
| 78 | +\newcommand{\hOneGeoDten}{17.6\%} | |
| 78 | 79 | \newcommand{\hOneGeoDtenHedo}{13.0\%} |
| 79 | 80 | \newcommand{\hOneSimDtenHedo}{12.7\%} |
| 80 | 81 | \newcommand{\hOneMixDtenHedo}{12.1\%} |
@@ -96,3 +97,18 @@ | ||
| 96 | 97 | \newcommand{\prixQfourMseven}{10.0\%} |
| 97 | 98 | \newcommand{\biasQoneMseven}{-2.4\%} |
| 98 | 99 | \newcommand{\biasQthreeMseven}{+1.5\%} |
| 100 | +\newcommand{\wNineNone}{0.07} | |
| 101 | +\newcommand{\wNineGbm}{0.33} | |
| 102 | +\newcommand{\wNineRole}{0.59} | |
| 103 | +\newcommand{\wTwelveGbm}{0.91} | |
| 104 | +\newcommand{\wTwelveCompGbm}{0.03} | |
| 105 | +\newcommand{\wTwelveCompHedo}{0.07} | |
| 106 | +\newcommand{\densLowN}{9,177} | |
| 107 | +\newcommand{\densLowNall}{9,426} | |
| 108 | +\newcommand{\ivCovLo}{66} | |
| 109 | +\newcommand{\ivCovHi}{78} | |
| 110 | +\newcommand{\ivHalf}{33\%} | |
| 111 | +\newcommand{\hFourSimDense}{10.5\%} | |
| 112 | +\newcommand{\hFourEqualMeanDense}{10.9\%} | |
| 113 | +\newcommand{\gridRtwo}{0.36} | |
| 114 | +\newcommand{\gridN}{266,437} | |
added
paper/tables/numbers_revision.tex
+38 −0
@@ -0,0 +1,38 @@ | ||
| 1 | +% généré par pipeline/07_revision_analyses.py | |
| 2 | +\newcommand{\spearmanAll}{0.996} | |
| 3 | +\newcommand{\spearmanCov}{0.994} | |
| 4 | +\newcommand{\mZeroRankTest}{2} | |
| 5 | +\newcommand{\nConfigsCov}{1,370} | |
| 6 | +\newcommand{\oracleTest}{11.1\%} | |
| 7 | +\newcommand{\regretTest}{0.0} | |
| 8 | +\newcommand{\nearQuarterN}{37} | |
| 9 | +\newcommand{\nearQuarterMin}{11.1\%} | |
| 10 | +\newcommand{\nearQuarterMax}{11.4\%} | |
| 11 | +\newcommand{\ciMzeroLo}{10.8} | |
| 12 | +\newcommand{\ciMzeroHi}{11.5} | |
| 13 | +\newcommand{\ciMelevenBLo}{8.8} | |
| 14 | +\newcommand{\ciMelevenBHi}{9.2} | |
| 15 | +\newcommand{\tMsevenMtwo}{-58} | |
| 16 | +\newcommand{\tMelevenBMzero}{-36} | |
| 17 | +\newcommand{\tMtwelveMelevenB}{-8.3} | |
| 18 | +\newcommand{\tMsevenMsix}{-28} | |
| 19 | +\newcommand{\tMzeroMthirteen}{7.4} | |
| 20 | +\newcommand{\dMzeroMthirteen}{-0.48} | |
| 21 | +\newcommand{\rEightTightShare}{96.1\%} | |
| 22 | +\newcommand{\rEightTightMzero}{10.6\%} | |
| 23 | +\newcommand{\rEightTightMelevenB}{8.5\%} | |
| 24 | +\newcommand{\idxMtl}{4\%} | |
| 25 | +\newcommand{\idxQc}{25\%} | |
| 26 | +\newcommand{\idxMonteregie}{14\%} | |
| 27 | +\newcommand{\idxProvAll}{79\%} | |
| 28 | +\newcommand{\idxMtlAll}{33\%} | |
| 29 | +\newcommand{\idxEstrieAll}{112\%} | |
| 30 | +\newcommand{\idxDrawdown}{13\%} | |
| 31 | +\newcommand{\seasQoneMseven}{-2.4\%} | |
| 32 | +\newcommand{\seasQthreeMseven}{+1.6\%} | |
| 33 | +\newcommand{\seasQoneMelevenB}{+1.4\%} | |
| 34 | +\newcommand{\seasQthreeMelevenB}{-0.4\%} | |
| 35 | +\newcommand{\seasQoneMzero}{-1.6\%} | |
| 36 | +\newcommand{\seasQthreeMzero}{+2.4\%} | |
| 37 | +\newcommand{\seasQoneMten}{-1.2\%} | |
| 38 | +\newcommand{\seasQthreeMten}{+2.8\%} | |
added
paper/tables/pairs_b999.tex
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +\begin{tabular}{lrrrrrr} | |
| 2 | +\toprule | |
| 3 | +Comparison & $\Delta$ MdAPE [95\% CI] & $\Delta$ MAPE [95\% CI] & $t$ & $A$ better (\%) & $n$ & Clusters \\ | |
| 4 | +\midrule | |
| 5 | +M7 $-$ M2 & -5.08 [-5.26; -4.88] & -6.65 [-6.90; -6.43] & -57.6 & 61.1 & 124,310 & 6,346 \\ | |
| 6 | +M7 $-$ M6 & -0.37 [-0.45; -0.28] & -1.32 [-1.42; -1.24] & -28.4 & 52.3 & 124,310 & 6,346 \\ | |
| 7 | +M8a $-$ M7 & 0.46 [0.33; 0.59] & 0.59 [0.47; 0.72] & 8.7 & 48.0 & 124,310 & 6,346 \\ | |
| 8 | +M8b $-$ M7 & -1.08 [-1.17; -1.00] & -1.04 [-1.11; -0.98] & -31.3 & 54.3 & 124,310 & 6,346 \\ | |
| 9 | +M11b $-$ M0 & -2.17 [-2.30; -2.04] & -3.69 [-3.92; -3.49] & -36.1 & 59.1 & 124,555 & 6,412 \\ | |
| 10 | +M11a $-$ M0 & -0.39 [-0.49; -0.32] & -0.50 [-0.57; -0.43] & -14.1 & 51.8 & 124,555 & 6,412 \\ | |
| 11 | +M12 $-$ M11b & -0.06 [-0.09; -0.03] & -0.07 [-0.09; -0.06] & -8.3 & 52.6 & 124,559 & 6,415 \\ | |
| 12 | +M13 $-$ M7 & -1.40 [-1.58; -1.24] & -2.39 [-2.64; -2.16] & -20.2 & 54.5 & 112,423 & 5,986 \\ | |
| 13 | +M3 $-$ M2 & -5.10 [-5.26; -4.92] & -5.58 [-5.79; -5.38] & -52.1 & 59.8 & 124,310 & 6,346 \\ | |
| 14 | +M9 $-$ M7 & -2.38 [-2.51; -2.26] & -3.33 [-3.51; -3.16] & -38.3 & 59.7 & 124,310 & 6,346 \\ | |
| 15 | +M1 $-$ M2 & 1.41 [1.26; 1.61] & 1.82 [1.69; 1.97] & 23.2 & 45.8 & 120,372 & 5,760 \\ | |
| 16 | +M0 $-$ M13 & -0.48 [-0.64; -0.32] & 0.85 [0.63; 1.09] & 7.4 & 50.1 & 112,423 & 5,986 \\ | |
| 17 | +M0 $-$ M9 & 0.48 [0.36; 0.59] & 1.79 [1.62; 1.96] & 20.6 & 46.9 & 124,310 & 6,346 \\ | |
| 18 | +M0 $-$ M7 & -1.90 [-2.00; -1.80] & -1.54 [-1.64; -1.44] & -30.7 & 56.6 & 124,310 & 6,346 \\ | |
| 19 | +M13 $-$ M11b & 2.65 [2.51; 2.80] & 2.91 [2.76; 3.07] & 36.6 & 40.5 & 112,423 & 5,986 \\ | |
| 20 | +\bottomrule | |
| 21 | +\end{tabular} | |
added
paper/tables/pairs_cluster_sens.tex
+10 −0
@@ -0,0 +1,10 @@ | ||
| 1 | +\begin{tabular}{lrrrrr} | |
| 2 | +\toprule | |
| 3 | +Cluster definition & Clusters & M7 $-$ M2 & M11b $-$ M0 & M12 $-$ M11b & M7 $-$ M6 \\ | |
| 4 | +\midrule | |
| 5 | +municipality $\times$ quarter & 6,346 & [-5.28; -4.89] & [-2.29; -2.03] & [-0.09; -0.03] & [-0.45; -0.28] \\ | |
| 6 | +MRC $\times$ quarter & 605 & [-5.79; -4.73] & [-2.72; -1.91] & [-0.10; -0.03] & [-0.56; -0.26] \\ | |
| 7 | +municipality & 1,079 & [-5.40; -4.79] & [-2.43; -1.93] & [-0.09; -0.03] & [-0.47; -0.26] \\ | |
| 8 | +region $\times$ quarter & 119 & [-5.30; -4.88] & [-2.37; -1.97] & [-0.09; -0.02] & [-0.48; -0.27] \\ | |
| 9 | +\bottomrule | |
| 10 | +\end{tabular} | |
added
paper/tables/r8_sensitivity.tex
+10 −0
@@ -0,0 +1,10 @@ | ||
| 1 | +\begin{tabular}{lrrrrrrrrrr} | |
| 2 | +\toprule | |
| 3 | +Target price/assessment ratio & $n$ & Share (\%) & M2 & M7 & M13 & M0 & M10 & M11a & M11b & M12 \\ | |
| 4 | +\midrule | |
| 5 | +$[0.33, 3]$ (baseline) & 124,559 & 100.0 & 18.1 & 13.0 & 11.5 & 11.1 & 13.8 & 10.7 & 9.0 & 8.9 \\ | |
| 6 | +$[0.40, 2.5]$ & 123,112 & 98.8 & 17.8 & 12.8 & 11.3 & 11.0 & 13.7 & 10.6 & 8.8 & 8.8 \\ | |
| 7 | +$[0.50, 2.0]$ & 119,681 & 96.1 & 17.4 & 12.5 & 11.0 & 10.6 & 13.3 & 10.3 & 8.5 & 8.5 \\ | |
| 8 | +$[0.67, 1.5]$ & 102,333 & 82.2 & 16.3 & 11.5 & 9.8 & 9.7 & 12.4 & 9.4 & 7.5 & 7.4 \\ | |
| 9 | +\bottomrule | |
| 10 | +\end{tabular} | |
deleted
paper/tables/regression.tex
+0 −37
@@ -1,37 +0,0 @@ | ||
| 1 | −% ============================================================================ | |
| 2 | −% Table: Regression Results | |
| 3 | −% ============================================================================ | |
| 4 | −\begin{table}[htbp] | |
| 5 | − \centering | |
| 6 | − \begin{threeparttable} | |
| 7 | − \caption{Baseline Regression Results} | |
| 8 | − \label{tab:regression} | |
| 9 | − \begin{tabular}{@{}lcccc@{}} | |
| 10 | − \toprule | |
| 11 | − & (1) & (2) & (3) & (4) \\ | |
| 12 | − \midrule | |
| 13 | − $X_{i,t}$ & 0.245$^{***}$ & 0.198$^{***}$ & 0.183$^{***}$ & 0.172$^{***}$ \\ | |
| 14 | − & (0.042) & (0.038) & (0.035) & (0.033) \\[6pt] | |
| 15 | − Size & & & $-$0.032$^{**}$ & $-$0.028$^{**}$ \\ | |
| 16 | − & & & (0.014) & (0.013) \\[6pt] | |
| 17 | − Book-to-Market & & & 0.051$^{***}$ & 0.047$^{***}$ \\ | |
| 18 | − & & & (0.018) & (0.017) \\[6pt] | |
| 19 | − Leverage & & & & $-$0.089$^{**}$ \\ | |
| 20 | − & & & & (0.041) \\[6pt] | |
| 21 | − ROA & & & & 0.015$^{*}$ \\ | |
| 22 | − & & & & (0.008) \\[6pt] | |
| 23 | − \midrule | |
| 24 | − Firm FE & No & Yes & Yes & Yes \\ | |
| 25 | − Time FE & No & No & Yes & Yes \\ | |
| 26 | − Controls & No & No & Yes & Yes \\[3pt] | |
| 27 | − Observations & 125{,}400 & 125{,}400 & 125{,}400 & 125{,}400 \\ | |
| 28 | − $R^2$ & 0.032 & 0.148 & 0.215 & 0.228 \\ | |
| 29 | − Adjusted $R^2$ & 0.031 & 0.112 & 0.178 & 0.190 \\ | |
| 30 | − \bottomrule | |
| 31 | − \end{tabular} | |
| 32 | − \begin{tablenotes}[flushleft] | |
| 33 | − \footnotesize | |
| 34 | − \item \textit{Notes:} This table reports OLS estimates of Equation~\eqref{eq:baseline}. The dependent variable is $y_{i,t}$. $X_{i,t}$ is the main explanatory variable. Control variables include firm size, book-to-market ratio, leverage, and return on assets. Standard errors clustered at the firm level are reported in parentheses. $^{***}$, $^{**}$, and $^{*}$ denote statistical significance at the 1\%, 5\%, and 10\% levels, respectively. | |
| 35 | − \end{tablenotes} | |
| 36 | − \end{threeparttable} | |
| 37 | −\end{table} | |
added
paper/tables/seasonal_bias.tex
+10 −0
@@ -0,0 +1,10 @@ | ||
| 1 | +\begin{tabular}{lrrrrrrrrr} | |
| 2 | +\toprule | |
| 3 | +Calendar quarter & $n$ & M2 & M7 & M13 & M0 & M10 & M11a & M11b & M12 \\ | |
| 4 | +\midrule | |
| 5 | +Q1 & 21,956 & -1.1 & -2.4 & -2.2 & -1.6 & -1.2 & -1.4 & +1.4 & +1.0 \\ | |
| 6 | +Q2 & 47,648 & -1.2 & -1.2 & -0.4 & -0.5 & -0.9 & -0.1 & +0.0 & -0.1 \\ | |
| 7 | +Q3 & 38,263 & +1.7 & +1.6 & +1.6 & +2.4 & +2.8 & -0.4 & -0.4 & -0.4 \\ | |
| 8 | +Q4 & 16,692 & +0.2 & -0.7 & -1.2 & +0.1 & +0.7 & +0.2 & -0.1 & -0.3 \\ | |
| 9 | +\bottomrule | |
| 10 | +\end{tabular} | |
added
paper/tables/selection_near.tex
+10 −0
@@ -0,0 +1,10 @@ | ||
| 1 | +\begin{tabular}{lrrrr} | |
| 2 | +\toprule | |
| 3 | +Validation MdAPE within (pts of best) & Configurations & Test MdAPE min & median & max \\ | |
| 4 | +\midrule | |
| 5 | +$\le 0.10$ & 12 & 11.13 & 11.17 & 11.25 \\ | |
| 6 | +$\le 0.25$ & 37 & 11.13 & 11.26 & 11.39 \\ | |
| 7 | +$\le 0.50$ & 92 & 11.13 & 11.37 & 11.66 \\ | |
| 8 | +$\le 1.00$ & 252 & 11.13 & 11.72 & 12.18 \\ | |
| 9 | +\bottomrule | |
| 10 | +\end{tabular} | |
added
paper/tables/selection_top10.tex
+16 −0
@@ -0,0 +1,16 @@ | ||
| 1 | +\begin{tabular}{lrrrllrrr} | |
| 2 | +\toprule | |
| 3 | +Selection & $k$ & $d$ (km) & $t$ (mo) & Adjustment & Weights/recon. & MdAPE 2024 & MdAPE 2025--26 & Rank 2025--26 \\ | |
| 4 | +\midrule | |
| 5 | +compromise & 30 & 25 & 24 & hedonic grid & equal/median & 11.86 & 11.13 & 2 \\ | |
| 6 | +compromise & 30 & 10 & 24 & hedonic grid & equal/median & 11.87 & 11.14 & 3 \\ | |
| 7 | +compromise & 10 & 10 & 24 & GBM adjustments & equal/median & 11.89 & 11.13 & 1 \\ | |
| 8 | +compromise & 30 & 5 & 24 & hedonic grid & equal/median & 11.90 & 11.16 & 5 \\ | |
| 9 | +compromise & 50 & 10 & 24 & hedonic grid & equal/median & 11.91 & 11.23 & 12 \\ | |
| 10 | +compromise & 50 & 25 & 24 & hedonic grid & equal/median & 11.92 & 11.21 & 11 \\ | |
| 11 | +compromise & 20 & 25 & 24 & hedonic grid & equal/median & 11.93 & 11.17 & 7 \\ | |
| 12 | +compromise & 20 & 5 & 24 & hedonic grid & equal/median & 11.93 & 11.17 & 6 \\ | |
| 13 | +compromise & 20 & 10 & 24 & hedonic grid & equal/median & 11.93 & 11.17 & 8 \\ | |
| 14 | +compromise & 20 & 2 & 24 & hedonic grid & equal/median & 11.94 & 11.16 & 4 \\ | |
| 15 | +\bottomrule | |
| 16 | +\end{tabular} | |
modified
paper/tables/strata_other.tex
+7 −7
@@ -8,19 +8,19 @@ Q3 & 23,834 & 13.6 & 10.4 & 10.6 & 9.7 & 9.0 & 11.5 & 7.1 & 7.1 \\ | ||
| 8 | 8 | Q4 & 26,783 & 13.3 & 10.0 & 10.1 & 8.9 & 8.2 & 11.3 & 6.7 & 6.7 \\ |
| 9 | 9 | Q5 (high) & 27,354 & 19.3 & 11.6 & 12.4 & 9.7 & 10.1 & 13.2 & 8.2 & 8.2 \\ |
| 10 | 10 | \midrule |
| 11 | −Age & $n$ & M2 & M7 & M8a & M13 & M0 & M10 & M11b & M12 \\ | |
| 11 | +Age (years) & $n$ & M2 & M7 & M8a & M13 & M0 & M10 & M11b & M12 \\ | |
| 12 | 12 | \midrule |
| 13 | −16--40 & 39,475 & 16.6 & 11.1 & 11.3 & 9.8 & 9.2 & 11.9 & 7.5 & 7.4 \\ | |
| 13 | +$\le 3$ years & 2,373 & 20.1 & 11.9 & 12.1 & 9.8 & 9.8 & 12.9 & 8.0 & 7.8 \\ | |
| 14 | 14 | 4--15 & 17,344 & 17.0 & 9.9 & 10.1 & 8.8 & 8.4 & 12.1 & 6.7 & 6.6 \\ |
| 15 | +16--40 & 39,475 & 16.6 & 11.1 & 11.3 & 9.8 & 9.2 & 11.9 & 7.5 & 7.4 \\ | |
| 15 | 16 | 41--70 & 48,592 & 17.7 & 14.4 & 14.9 & 13.0 & 12.5 & 14.5 & 10.2 & 10.2 \\ |
| 16 | 17 | $>70$ & 16,526 & 25.1 & 19.7 & 21.6 & 16.9 & 18.0 & 20.9 & 14.2 & 14.2 \\ |
| 17 | −$\le 3$ years & 2,373 & 20.1 & 11.9 & 12.1 & 9.8 & 9.8 & 12.9 & 8.0 & 7.8 \\ | |
| 18 | 18 | \midrule |
| 19 | 19 | Physical link & $n$ & M2 & M7 & M8a & M13 & M0 & M10 & M11b & M12 \\ |
| 20 | 20 | \midrule |
| 21 | −0.0 & 100,999 & 19.6 & 14.3 & 14.7 & 12.4 & 12.2 & 14.2 & 9.8 & 9.7 \\ | |
| 22 | −1.0 & 15,936 & 12.9 & 9.1 & 9.4 & 8.6 & 7.7 & 11.8 & 6.6 & 6.4 \\ | |
| 23 | −2.0 & 7,297 & 11.9 & 8.2 & 8.6 & 8.0 & 7.8 & 14.5 & 6.2 & 6.1 \\ | |
| 24 | −3.0 & 78 & 34.3 & 22.3 & 18.0 & 10.3 & 15.3 & 31.0 & 10.9 & 9.9 \\ | |
| 21 | +detached & 100,999 & 19.6 & 14.3 & 14.7 & 12.4 & 12.2 & 14.2 & 9.8 & 9.7 \\ | |
| 22 | +semi-detached & 15,936 & 12.9 & 9.1 & 9.4 & 8.6 & 7.7 & 11.8 & 6.6 & 6.4 \\ | |
| 23 | +row & 7,297 & 11.9 & 8.2 & 8.6 & 8.0 & 7.8 & 14.5 & 6.2 & 6.1 \\ | |
| 24 | +integrated & 78 & 34.3 & 22.3 & 18.0 & 10.3 & 15.3 & 31.0 & 10.9 & 9.9 \\ | |
| 25 | 25 | \bottomrule |
| 26 | 26 | \end{tabular} |
deleted
paper/tables/summary_stats.tex
+0 −29
@@ -1,29 +0,0 @@ | ||
| 1 | −% ============================================================================ | |
| 2 | −% Table: Summary Statistics | |
| 3 | −% ============================================================================ | |
| 4 | −\begin{table}[htbp] | |
| 5 | − \centering | |
| 6 | − \begin{threeparttable} | |
| 7 | − \caption{Summary Statistics} | |
| 8 | − \label{tab:summary_stats} | |
| 9 | − \begin{tabular}{@{}lcccccc@{}} | |
| 10 | − \toprule | |
| 11 | − Variable & $N$ & Mean & Std.\ Dev. & $p_{25}$ & Median & $p_{75}$ \\ | |
| 12 | − \midrule | |
| 13 | − \multicolumn{7}{@{}l}{\textit{Panel A: Dependent Variables}} \\[3pt] | |
| 14 | − $r_{i,t}$ (\%) & 125{,}400 & 0.82 & 4.35 & $-$1.92 & 0.65 & 3.48 \\ | |
| 15 | − $\sigma_{i,t}$ (\%) & 125{,}400 & 2.14 & 1.87 & 0.98 & 1.65 & 2.78 \\[6pt] | |
| 16 | − \multicolumn{7}{@{}l}{\textit{Panel B: Explanatory Variables}} \\[3pt] | |
| 17 | − $X_{i,t}$ & 125{,}400 & 0.45 & 0.32 & 0.21 & 0.40 & 0.65 \\ | |
| 18 | − Size (log) & 125{,}400 & 7.23 & 1.85 & 5.92 & 7.15 & 8.47 \\ | |
| 19 | − Book-to-Market & 125{,}400 & 0.68 & 0.54 & 0.30 & 0.56 & 0.91 \\ | |
| 20 | − Leverage & 125{,}400 & 0.34 & 0.21 & 0.17 & 0.32 & 0.48 \\ | |
| 21 | − ROA (\%) & 125{,}400 & 3.15 & 8.42 & 0.52 & 3.28 & 6.74 \\ | |
| 22 | − \bottomrule | |
| 23 | − \end{tabular} | |
| 24 | − \begin{tablenotes}[flushleft] | |
| 25 | − \footnotesize | |
| 26 | − \item \textit{Notes:} This table reports summary statistics for the main variables used in the analysis. The sample covers the period January~1990 to December~2023. $r_{i,t}$ is the monthly return of firm~$i$. $\sigma_{i,t}$ is the realized volatility computed from daily returns within the month. $X_{i,t}$ is the main explanatory variable. Size is the natural logarithm of market capitalization. Book-to-Market is the ratio of book equity to market equity. Leverage is total debt over total assets. ROA is return on assets. All continuous variables are winsorized at the 1st and 99th percentiles. | |
| 27 | − \end{tablenotes} | |
| 28 | − \end{threeparttable} | |
| 29 | −\end{table} | |
modified
paper/tables/weighting.tex
+2 −2
@@ -8,8 +8,8 @@ equal & mean & 14.6 & 13.7 & -1.8 & 38.4 \\ | ||
| 8 | 8 | 1/distance & mean & 14.6 & 13.7 & -1.8 & 38.6 \\ |
| 9 | 9 | recency & median & 14.5 & 13.7 & -0.5 & 38.8 \\ |
| 10 | 10 | recency & mean & 14.8 & 13.9 & -1.6 & 38.0 \\ |
| 11 | −most similar & median & 13.8 & 12.9 & -0.3 & 40.8 \\ | |
| 12 | −most similar & mean & 14.3 & 13.3 & -1.5 & 39.7 \\ | |
| 11 | +1/gross adjustment & median & 13.8 & 12.9 & -0.3 & 40.8 \\ | |
| 12 | +1/gross adjustment & mean & 14.3 & 13.3 & -1.5 & 39.7 \\ | |
| 13 | 13 | product of the three & median & 14.6 & 13.6 & -0.3 & 39.2 \\ |
| 14 | 14 | product of the three & mean & 14.5 & 13.5 & -1.4 & 39.3 \\ |
| 15 | 15 | \bottomrule |
added
pipeline/07_revision_analyses.py
+209 −0
@@ -0,0 +1,209 @@ | ||
| 1 | +# -*- coding: utf-8 -*- | |
| 2 | +"""07 — Analyses complémentaires de la révision scientifique (editorial board, 2026-10-17). | |
| 3 | + | |
| 4 | +Tourne sur le poste de travail à partir des sorties déjà rapatriées dans out/<version>/ (aucun accès à la base | |
| 5 | +Vrai Prix ni au cluster ; < 5 min) : | |
| 6 | + targets.parquet, models_est.parquet (estimations par cible des modèles nommés M0–M13), | |
| 7 | + tables/surface_E1.csv, tables/facteurs_E2.csv, index_full.csv. | |
| 8 | + | |
| 9 | +Produit results/revision-<date>/*.csv|json et les fragments LaTeX correspondants dans paper/tables/ : | |
| 10 | + (a) stabilité validation → test du classement des 2 256 configurations (corrélation de rang, rang en test de la | |
| 11 | + configuration choisie en 2024, ensemble des configurations quasi optimales en validation et leur erreur en test) ; | |
| 12 | + (b) bootstrap par grappes à 999 réplications des différences appariées (MdAPE et MAPE), statistique t robuste aux | |
| 13 | + grappes de la différence de MAPE (type Diebold–Mariano), sensibilité au choix des grappes ; | |
| 14 | + (c) intervalles de confiance bootstrap de la MdAPE de chaque modèle nommé ; | |
| 15 | + (d) sensibilité aux bornes de la règle R8 (ratio prix/rôle) côté cibles ; | |
| 16 | + (e) biais médian par trimestre civil (saisonnalité) pour les principaux modèles ; | |
| 17 | + (f) croissance régionale de l'indice descriptif (vérification des chiffres du texte). | |
| 18 | +""" | |
| 19 | +import json, os, sys, time | |
| 20 | +from pathlib import Path | |
| 21 | +import numpy as np, pandas as pd | |
| 22 | +from scipy import stats | |
| 23 | + | |
| 24 | +ROOT = Path(__file__).resolve().parents[1] | |
| 25 | +V = os.environ.get("CV_VERSION", "2026-10-10") | |
| 26 | +OUT = ROOT / "out" / V; TAB = OUT / "tables" | |
| 27 | +RES = ROOT / "results" / "revision-2026-10-17"; RES.mkdir(parents=True, exist_ok=True) | |
| 28 | +PT = ROOT / "paper" / "tables" | |
| 29 | +B = int(os.environ.get("CV_BOOT", 999)); rng = np.random.default_rng(20261017) | |
| 30 | +t0 = time.time() | |
| 31 | +def log(m): print(f"[{time.time() - t0:6.1f}s] {m}", flush=True) | |
| 32 | +def n_(x): return f"{int(round(x)):,}" | |
| 33 | +def esc(s): return str(s).replace("%", "\\%").replace("&", "\\&") | |
| 34 | +def tabular(rows, cols, align): | |
| 35 | + L = ["\\begin{tabular}{" + align + "}", "\\toprule", " & ".join(cols) + " \\\\", "\\midrule"] | |
| 36 | + L += [" & ".join(str(c) for c in r) + " \\\\" for r in rows] | |
| 37 | + L += ["\\bottomrule", "\\end{tabular}"] | |
| 38 | + return "\n".join(L) + "\n" | |
| 39 | + | |
| 40 | +T = pd.read_parquet(OUT / "targets.parquet"); M = pd.read_parquet(OUT / "models_est.parquet") | |
| 41 | +assert (T["idx"].to_numpy() == M["idx"].to_numpy()).all() | |
| 42 | +TEST = (T["period"] == "test").to_numpy(); VAL = (T["period"] == "val").to_numpy() | |
| 43 | +lnP = T["ln_price"].to_numpy(float) | |
| 44 | +MODELS = ["M1", "M2", "M3", "M4", "M5", "M6", "M7", "M8a", "M8b", "M13", "M10", "M11a", "M11b", "M0", "M9", "M12"] | |
| 45 | +ERR = {m: np.exp(M[m].to_numpy(float) - lnP) - 1 for m in MODELS} | |
| 46 | +T["quarter"] = pd.to_datetime(T["date"]).dt.to_period("Q").astype(str) | |
| 47 | +T["cal_q"] = pd.to_datetime(T["date"]).dt.quarter | |
| 48 | +res = {} | |
| 49 | + | |
| 50 | +# ---------------------------------------------------------------- (a) stabilité validation → test des 2 256 configurations | |
| 51 | +S = pd.read_csv(TAB / "surface_E1.csv").assign(src="E1"); F = pd.read_csv(TAB / "facteurs_E2.csv").assign(src="E2") | |
| 52 | +A = pd.concat([S, F], ignore_index=True) | |
| 53 | +ok = A["couverture_val"] >= 97 | |
| 54 | +def spear(d): return float(stats.spearmanr(d["mdape_val"], d["mdape_test"]).statistic) | |
| 55 | +stab = {"n_configs": int(len(A)), "n_configs_cov97": int(ok.sum()), | |
| 56 | + "spearman_E1": spear(S), "spearman_E2": spear(F), "spearman_all": spear(A), "spearman_cov97": spear(A[ok])} | |
| 57 | +A97 = A[ok].copy(); A97["rank_val"] = A97["mdape_val"].rank(method="min"); A97["rank_test"] = A97["mdape_test"].rank(method="min") | |
| 58 | +chosen = A97.sort_values("mdape_val").iloc[0]; oracle = A97.sort_values("mdape_test").iloc[0] | |
| 59 | +stab.update(chosen_val=float(chosen.mdape_val), chosen_test=float(chosen.mdape_test), chosen_rank_test=int(chosen.rank_test), | |
| 60 | + oracle_test=float(oracle.mdape_test), oracle_desc=f"{oracle.sel}|k={int(oracle.k)}|d={oracle.d}|T={int(oracle['T'])}|{oracle.adj}|{oracle.w}|{oracle.recon}", | |
| 61 | + regret_test=float(chosen.mdape_test - oracle.mdape_test)) | |
| 62 | +near = [] | |
| 63 | +for tol in [0.1, 0.25, 0.5, 1.0]: | |
| 64 | + g = A97[A97["mdape_val"] <= chosen.mdape_val + tol] | |
| 65 | + near.append(dict(tol=tol, n=int(len(g)), test_min=float(g.mdape_test.min()), test_max=float(g.mdape_test.max()), test_median=float(g.mdape_test.median()))) | |
| 66 | +stab["near_optimal"] = near | |
| 67 | +top = A97.sort_values("mdape_val").head(10) | |
| 68 | +stab["top10_val"] = top[["src", "sel", "k", "d", "T", "adj", "w", "recon", "mdape_val", "mdape_test", "couverture_test", "rank_test"]].to_dict("records") | |
| 69 | +res["selection_stability"] = stab | |
| 70 | +LAB = {"geo": "nearest", "sim": "most similar", "mix": "compromise", "adj": "least adjusted", "none": "time only", "hedo": "hedonic grid", | |
| 71 | + "role": "assessment ratio", "gbm": "GBM adjustments", "conv": "conventional grid", "hedo_reg": "regional grid", "local": "local grid", "raw": "none", | |
| 72 | + "equal": "equal", "dist": "1/distance", "rec": "recency", "all": "product", "median": "median", "mean": "mean"} | |
| 73 | +rows = [[LAB[r.sel], int(r.k), f"{r.d:g}", int(r["T"]), LAB[r.adj], f"{LAB[r.w]}/{LAB[r.recon]}", f"{r.mdape_val:.2f}", f"{r.mdape_test:.2f}", int(r.rank_test)] for _, r in top.iterrows()] | |
| 74 | +(PT / "selection_top10.tex").write_text(tabular(rows, ["Selection", "$k$", "$d$ (km)", "$t$ (mo)", "Adjustment", "Weights/recon.", "MdAPE 2024", "MdAPE 2025--26", "Rank 2025--26"], "lrrrllrrr")) | |
| 75 | +rows = [[f"$\\le {d['tol']:.2f}$", d["n"], f"{d['test_min']:.2f}", f"{d['test_median']:.2f}", f"{d['test_max']:.2f}"] for d in near] | |
| 76 | +(PT / "selection_near.tex").write_text(tabular(rows, ["Validation MdAPE within (pts of best)", "Configurations", "Test MdAPE min", "median", "max"], "lrrrr")) | |
| 77 | +log(f"(a) stabilité : Spearman all {stab['spearman_all']:.4f}, rang test de M0 {stab['chosen_rank_test']}/{stab['n_configs_cov97']}, regret {stab['regret_test']:.2f}") | |
| 78 | + | |
| 79 | +# ---------------------------------------------------------------- bootstrap par grappes : outils | |
| 80 | +def cluster_ids(key): | |
| 81 | + return pd.factorize(key)[0] | |
| 82 | +CLUSTERS = {"municipality x quarter": cluster_ids(T["code_mun"].astype(str) + "|" + T["quarter"]), | |
| 83 | + "MRC x quarter": cluster_ids(T["mrc"].astype(str) + "|" + T["quarter"]), | |
| 84 | + "municipality": cluster_ids(T["code_mun"].astype(str)), | |
| 85 | + "region x quarter": cluster_ids(T["region"].astype(str) + "|" + T["quarter"])} | |
| 86 | +def boot_weights(cl, mask, B): | |
| 87 | + """Multiplicités bootstrap (grappes tirées avec remise) pour les observations de `mask` ; matrice B x n.""" | |
| 88 | + c = cl[mask]; ids, inv = np.unique(c, return_inverse=True); nc = len(ids) | |
| 89 | + W = np.empty((B, mask.sum()), np.int32) | |
| 90 | + for b in range(B): | |
| 91 | + pick = rng.integers(0, nc, nc); mult = np.bincount(pick, minlength=nc); W[b] = mult[inv] | |
| 92 | + return W | |
| 93 | +def wmedian_sorted(v_sorted, w_sorted_rows): | |
| 94 | + """Médiane pondérée pour chaque ligne de poids (observations déjà triées par v).""" | |
| 95 | + cw = np.cumsum(w_sorted_rows, axis=1); half = cw[:, -1] / 2.0 | |
| 96 | + pos = np.array([np.searchsorted(cw[b], half[b]) for b in range(cw.shape[0])]); return v_sorted[np.minimum(pos, len(v_sorted) - 1)] | |
| 97 | +def ci(x): return [float(np.percentile(x, 2.5)), float(np.percentile(x, 97.5))] | |
| 98 | + | |
| 99 | +# ---------------------------------------------------------------- (c) IC bootstrap de la MdAPE de chaque modèle (grappes municipalité x trimestre) | |
| 100 | +Wmain = boot_weights(CLUSTERS["municipality x quarter"], TEST, B) | |
| 101 | +log(f"poids bootstrap principaux : {Wmain.shape}") | |
| 102 | +mod_ci = {} | |
| 103 | +for m in MODELS: | |
| 104 | + a = np.abs(ERR[m][TEST]); okm = ~np.isnan(a) | |
| 105 | + v = a[okm]; o = np.argsort(v); vs = v[o]; W = Wmain[:, okm][:, o] | |
| 106 | + med = wmedian_sorted(vs, W) * 100; mean = (W @ vs) / W.sum(axis=1) * 100 | |
| 107 | + mod_ci[m] = dict(mdape=float(np.median(v) * 100), ci_mdape=ci(med), mape=float(v.mean() * 100), ci_mape=ci(mean), n=int(okm.sum())) | |
| 108 | +res["model_ci"] = mod_ci | |
| 109 | +order = sorted(MODELS, key=lambda m: mod_ci[m]["mdape"]) | |
| 110 | +rows = [[m, f"{mod_ci[m]['mdape']:.2f}", f"[{mod_ci[m]['ci_mdape'][0]:.2f}; {mod_ci[m]['ci_mdape'][1]:.2f}]", f"{mod_ci[m]['mape']:.2f}", f"[{mod_ci[m]['ci_mape'][0]:.2f}; {mod_ci[m]['ci_mape'][1]:.2f}]", n_(mod_ci[m]["n"])] for m in order] | |
| 111 | +(PT / "competition_ci.tex").write_text(tabular(rows, ["Model", "MdAPE", "95\\% CI", "MAPE", "95\\% CI", "$n$"], "lrrrrr")) | |
| 112 | +log("(c) IC par modèle écrits") | |
| 113 | + | |
| 114 | +# ---------------------------------------------------------------- (b) différences appariées : B = 999, t robuste aux grappes, autres grappes | |
| 115 | +PAIRS = [("M7", "M2"), ("M7", "M6"), ("M8a", "M7"), ("M8b", "M7"), ("M11b", "M0"), ("M11a", "M0"), ("M12", "M11b"), ("M13", "M7"), ("M3", "M2"), ("M9", "M7"), ("M1", "M2"), | |
| 116 | + ("M0", "M13"), ("M0", "M9"), ("M0", "M7"), ("M13", "M11b")] | |
| 117 | +def paired(a_, b_, mask, cl_name, B_): | |
| 118 | + cl = CLUSTERS[cl_name] | |
| 119 | + a = np.abs(a_[mask]); b = np.abs(b_[mask]); okp = ~np.isnan(a) & ~np.isnan(b) | |
| 120 | + a, b = a[okp], b[okp]; d = a - b | |
| 121 | + sub = np.zeros(mask.sum(), bool); sub[okp] = True | |
| 122 | + W = Wmain[:, okp] if (cl_name == "municipality x quarter" and B_ == B) else boot_weights(cl, _mask_sub(mask, okp), B_) | |
| 123 | + oa = np.argsort(a); ob = np.argsort(b) | |
| 124 | + med_d = (wmedian_sorted(a[oa], W[:, oa]) - wmedian_sorted(b[ob], W[:, ob])) * 100 | |
| 125 | + mean_d = (W @ d) / W.sum(axis=1) * 100 | |
| 126 | + # statistique t robuste aux grappes (Diebold–Mariano sur la perte absolue en pourcentage) | |
| 127 | + c = cl[_mask_sub(mask, okp)]; ids, inv = np.unique(c, return_inverse=True) | |
| 128 | + dbar = d.mean(); S_c = np.bincount(inv, d - dbar); se = np.sqrt((S_c ** 2).sum()) / len(d) | |
| 129 | + return dict(n=int(len(d)), n_clusters=int(len(ids)), diff_mdape=float((np.median(a) - np.median(b)) * 100), ci_mdape=ci(med_d), | |
| 130 | + diff_mape=float(dbar * 100), ci_mape=ci(mean_d), t_cluster=float(dbar / se), share_A_better=float((a < b).mean() * 100)) | |
| 131 | +def _mask_sub(mask, okp): | |
| 132 | + m = np.zeros(len(mask), bool); m[np.flatnonzero(mask)[okp]] = True; return m | |
| 133 | +pairs_main = {f"{a}-{b}": paired(ERR[a], ERR[b], TEST, "municipality x quarter", B) for a, b in PAIRS} | |
| 134 | +res["pairs_B999"] = pairs_main | |
| 135 | +rows = [[f"{a} $-$ {b}", f"{p['diff_mdape']:.2f} [{p['ci_mdape'][0]:.2f}; {p['ci_mdape'][1]:.2f}]", f"{p['diff_mape']:.2f} [{p['ci_mape'][0]:.2f}; {p['ci_mape'][1]:.2f}]", f"{p['t_cluster']:.1f}", f"{p['share_A_better']:.1f}", n_(p["n"]), n_(p["n_clusters"])] | |
| 136 | + for (a, b), p in zip(PAIRS, pairs_main.values())] | |
| 137 | +(PT / "pairs_b999.tex").write_text(tabular(rows, ["Comparison", "$\\Delta$ MdAPE [95\\% CI]", "$\\Delta$ MAPE [95\\% CI]", "$t$", "$A$ better (\\%)", "$n$", "Clusters"], "lrrrrrr")) | |
| 138 | +log("(b) paires B=999 écrites") | |
| 139 | +# sensibilité au choix des grappes (B = 399) pour trois comparaisons clés | |
| 140 | +SENS = [("M7", "M2"), ("M11b", "M0"), ("M12", "M11b"), ("M7", "M6")] | |
| 141 | +sens = {} | |
| 142 | +for cl_name in CLUSTERS: | |
| 143 | + for a, b in SENS: | |
| 144 | + sens[f"{cl_name}|{a}-{b}"] = paired(ERR[a], ERR[b], TEST, cl_name, 399) | |
| 145 | +res["pairs_cluster_sensitivity"] = sens | |
| 146 | +rows = [] | |
| 147 | +for cl_name in CLUSTERS: | |
| 148 | + r = [cl_name.replace(" x ", " $\\times$ "), n_(sens[f"{cl_name}|M7-M2"]["n_clusters"])] | |
| 149 | + for a, b in SENS: | |
| 150 | + p = sens[f"{cl_name}|{a}-{b}"]; r.append(f"[{p['ci_mdape'][0]:.2f}; {p['ci_mdape'][1]:.2f}]") | |
| 151 | + rows.append(r) | |
| 152 | +(PT / "pairs_cluster_sens.tex").write_text(tabular(rows, ["Cluster definition", "Clusters"] + [f"{a} $-$ {b}" for a, b in SENS], "lr" + "r" * len(SENS))) | |
| 153 | +log("(b') sensibilité aux grappes écrite") | |
| 154 | + | |
| 155 | +# ---------------------------------------------------------------- (d) sensibilité aux bornes de R8 (cibles) | |
| 156 | +ratio = (T["amount"] / T["role_vente"]).to_numpy() | |
| 157 | +KEY = ["M2", "M7", "M13", "M0", "M10", "M11a", "M11b", "M12"] | |
| 158 | +r8 = {} | |
| 159 | +for lab, lo, hi in [("$[0.33, 3]$ (baseline)", 0.33, 3.0), ("$[0.40, 2.5]$", 0.40, 2.5), ("$[0.50, 2.0]$", 0.50, 2.0), ("$[0.67, 1.5]$", 0.67, 1.5)]: | |
| 160 | + m = TEST & (ratio >= lo) & (ratio <= hi) | |
| 161 | + r8[lab] = dict(n=int(m.sum()), share=float(m.sum() / TEST.sum() * 100), **{k: float(np.nanmedian(np.abs(ERR[k][m])) * 100) for k in KEY}) | |
| 162 | +res["r8_sensitivity"] = r8 | |
| 163 | +rows = [[lab, n_(d["n"]), f"{d['share']:.1f}"] + [f"{d[k]:.1f}" for k in KEY] for lab, d in r8.items()] | |
| 164 | +(PT / "r8_sensitivity.tex").write_text(tabular(rows, ["Target price/assessment ratio", "$n$", "Share (\\%)"] + KEY, "lrr" + "r" * len(KEY))) | |
| 165 | +log("(d) sensibilité R8 écrite") | |
| 166 | + | |
| 167 | +# ---------------------------------------------------------------- (e) biais médian par trimestre civil | |
| 168 | +seas = {} | |
| 169 | +for q in [1, 2, 3, 4]: | |
| 170 | + m = TEST & (T["cal_q"].to_numpy() == q) | |
| 171 | + seas[f"Q{q}"] = dict(n=int(m.sum()), **{k: float(np.nanmedian(ERR[k][m]) * 100) for k in KEY}) | |
| 172 | +res["seasonal_bias"] = seas | |
| 173 | +rows = [[q, n_(d["n"])] + [f"{d[k]:+.1f}" for k in KEY] for q, d in seas.items()] | |
| 174 | +(PT / "seasonal_bias.tex").write_text(tabular(rows, ["Calendar quarter", "$n$"] + KEY, "lr" + "r" * len(KEY))) | |
| 175 | +log("(e) biais saisonnier écrit") | |
| 176 | + | |
| 177 | +# ---------------------------------------------------------------- (f) indice descriptif : croissance régionale | |
| 178 | +I = pd.read_csv(OUT / "index_full.csv", dtype={"region": str}) | |
| 179 | +NAMES = {"QC": "Quebec (province)", "06": "Montréal", "03": "Capitale-Nationale", "16": "Montérégie", "13": "Laval", "07": "Outaouais", "15": "Laurentides", "14": "Lanaudière", "05": "Estrie"} | |
| 180 | +idx = {} | |
| 181 | +for r, nm in NAMES.items(): | |
| 182 | + g = I[I.region == r].set_index("ym")["log_idx"] | |
| 183 | + if len(g): | |
| 184 | + peak = g.loc["2021-01":"2022-12"].idxmax(); trough = g.loc[peak:"2023-12"].idxmin() | |
| 185 | + idx[nm] = dict(g_2021_2026=float((np.exp(g.loc["2026-09"] - g.loc["2021-01"]) - 1) * 100), g_2025_2026=float((np.exp(g.loc["2026-09"] - g.loc["2024-12"]) - 1) * 100), | |
| 186 | + peak_2022=peak, trough_2023=trough, drawdown=float((np.exp(g.loc[trough] - g.loc[peak]) - 1) * 100)) | |
| 187 | +res["index_growth"] = idx | |
| 188 | +rows = [[nm, f"{d['g_2021_2026']:.1f}", f"{d['peak_2022']}", f"{d['trough_2023']}", f"{d['drawdown']:.1f}", f"{d['g_2025_2026']:.1f}"] for nm, d in idx.items()] | |
| 189 | +(PT / "index_growth.tex").write_text(tabular(rows, ["Region", "2021-01 to 2026-09 (\\%)", "Peak", "Trough", "Peak to trough (\\%)", "2024-12 to 2026-09 (\\%)"], "lrrrrr")) | |
| 190 | +log("(f) indice écrit") | |
| 191 | + | |
| 192 | +with open(RES / "revision_results.json", "w") as f: json.dump(res, f, indent=2, default=lambda o: float(o) if isinstance(o, (np.floating,)) else (int(o) if isinstance(o, np.integer) else str(o))) | |
| 193 | +pd.DataFrame(top).to_csv(RES / "selection_top10.csv", index=False) | |
| 194 | +pd.DataFrame([dict(pair=k, **v) for k, v in pairs_main.items()]).to_csv(RES / "pairs_B999.csv", index=False) | |
| 195 | +pd.DataFrame(mod_ci).T.to_csv(RES / "model_ci.csv") | |
| 196 | +pd.DataFrame(r8).T.to_csv(RES / "r8_sensitivity.csv"); pd.DataFrame(seas).T.to_csv(RES / "seasonal_bias.csv"); pd.DataFrame(idx).T.to_csv(RES / "index_growth.csv") | |
| 197 | +# macros pour le texte | |
| 198 | +mac = {"spearmanAll": f"{stab['spearman_all']:.3f}", "spearmanCov": f"{stab['spearman_cov97']:.3f}", "mZeroRankTest": str(stab["chosen_rank_test"]), "nConfigsCov": n_(stab["n_configs_cov97"]), | |
| 199 | + "oracleTest": f"{stab['oracle_test']:.1f}\\%", "regretTest": f"{stab['regret_test']:.1f}", "nearQuarterN": str(near[1]["n"]), "nearQuarterMin": f"{near[1]['test_min']:.1f}\\%", "nearQuarterMax": f"{near[1]['test_max']:.1f}\\%", | |
| 200 | + "ciMzeroLo": f"{mod_ci['M0']['ci_mdape'][0]:.1f}", "ciMzeroHi": f"{mod_ci['M0']['ci_mdape'][1]:.1f}", "ciMelevenBLo": f"{mod_ci['M11b']['ci_mdape'][0]:.1f}", "ciMelevenBHi": f"{mod_ci['M11b']['ci_mdape'][1]:.1f}", | |
| 201 | + "tMsevenMtwo": f"{pairs_main['M7-M2']['t_cluster']:.0f}", "tMelevenBMzero": f"{pairs_main['M11b-M0']['t_cluster']:.0f}", "tMtwelveMelevenB": f"{pairs_main['M12-M11b']['t_cluster']:.1f}", "tMsevenMsix": f"{pairs_main['M7-M6']['t_cluster']:.0f}", | |
| 202 | + "tMzeroMthirteen": f"{pairs_main['M0-M13']['t_cluster']:.1f}", "dMzeroMthirteen": f"{pairs_main['M0-M13']['diff_mdape']:.2f}", | |
| 203 | + "rEightTightShare": f"{r8['$[0.50, 2.0]$']['share']:.1f}\\%", "rEightTightMzero": f"{r8['$[0.50, 2.0]$']['M0']:.1f}\\%", "rEightTightMelevenB": f"{r8['$[0.50, 2.0]$']['M11b']:.1f}\\%", | |
| 204 | + "idxMtl": f"{idx['Montréal']['g_2025_2026']:.0f}\\%", "idxQc": f"{idx['Capitale-Nationale']['g_2025_2026']:.0f}\\%", "idxMonteregie": f"{idx['Montérégie']['g_2025_2026']:.0f}\\%", | |
| 205 | + "idxProvAll": f"{idx['Quebec (province)']['g_2021_2026']:.0f}\\%", "idxMtlAll": f"{idx['Montréal']['g_2021_2026']:.0f}\\%", "idxEstrieAll": f"{idx['Estrie']['g_2021_2026']:.0f}\\%", "idxDrawdown": f"{abs(idx['Quebec (province)']['drawdown']):.0f}\\%", | |
| 206 | + "seasQoneMseven": f"{seas['Q1']['M7']:+.1f}\\%", "seasQthreeMseven": f"{seas['Q3']['M7']:+.1f}\\%", "seasQoneMelevenB": f"{seas['Q1']['M11b']:+.1f}\\%", "seasQthreeMelevenB": f"{seas['Q3']['M11b']:+.1f}\\%", | |
| 207 | + "seasQoneMzero": f"{seas['Q1']['M0']:+.1f}\\%", "seasQthreeMzero": f"{seas['Q3']['M0']:+.1f}\\%", "seasQoneMten": f"{seas['Q1']['M10']:+.1f}\\%", "seasQthreeMten": f"{seas['Q3']['M10']:+.1f}\\%"} | |
| 208 | +(PT / "numbers_revision.tex").write_text("% généré par pipeline/07_revision_analyses.py\n" + "\n".join(f"\\newcommand{{\\{k}}}{{{v}}}" for k, v in mac.items()) + "\n") | |
| 209 | +log("terminé") | |
added
results/revision-2026-10-17/index_growth.csv
+10 −0
@@ -0,0 +1,10 @@ | ||
| 1 | +,g_2021_2026,g_2025_2026,peak_2022,trough_2023,drawdown | |
| 2 | +Quebec (province),79.04431543669928,19.61999619891963,2022-06,2023-02,-13.12164637096278 | |
| 3 | +Montréal,33.20268525439665,3.6170870275668676,2022-06,2023-03,-11.908718929212625 | |
| 4 | +Capitale-Nationale,80.98962076006319,25.43581179618868,2022-08,2023-02,-14.119876764484074 | |
| 5 | +Montérégie,66.37243322386337,13.858996204029284,2022-06,2023-02,-17.999345619512997 | |
| 6 | +Laval,48.315874338361155,12.644214753995154,2022-06,2023-02,-16.174889100779176 | |
| 7 | +Outaouais,68.56080325943911,13.850136879571838,2022-06,2023-02,-11.541935090435974 | |
| 8 | +Laurentides,73.56776142890568,16.0720157166234,2022-06,2023-02,-14.34485609503222 | |
| 9 | +Lanaudière,88.03729864981979,20.12011409183565,2022-06,2023-02,-15.446544713387533 | |
| 10 | +Estrie,111.96526044417152,26.129426233831676,2022-08,2023-02,-15.442819274665375 | |
added
results/revision-2026-10-17/model_ci.csv
+17 −0
@@ -0,0 +1,17 @@ | ||
| 1 | +,mdape,ci_mdape,mape,ci_mape,n | |
| 2 | +M1,19.146214848372246,"[18.685031183072766, 19.683348977052354]",32.800846529772485,"[32.122338749775516, 33.580296967360205]",120372 | |
| 3 | +M2,18.09403544314227,"[17.66825564715647, 18.550683626386977]",31.768819681471815,"[31.07159938200675, 32.57120626282716]",124310 | |
| 4 | +M3,12.996518301617666,"[12.646356484326454, 13.395039917871683]",26.1875684081453,"[25.52101411706073, 26.901367094523625]",124310 | |
| 5 | +M4,17.845039223505278,"[17.43549007960399, 18.31632606539743]",31.393630966552198,"[30.714842511391947, 32.17551221382804]",124310 | |
| 6 | +M5,18.500483669907887,"[18.06136224001957, 18.98670190413732]",32.28588625761257,"[31.579002770800916, 33.092144223949724]",124310 | |
| 7 | +M6,13.381342937804336,"[13.059504197488089, 13.744105357492264]",26.437070405143366,"[25.784908089385414, 27.158469799135688]",124310 | |
| 8 | +M7,13.013809369802908,"[12.714529475654354, 13.35244600835715]",25.11622709344079,"[24.520925576464077, 25.754323095646676]",124310 | |
| 9 | +M8a,13.47302998888682,"[13.120802670871669, 13.86299660279164]",25.705687211092737,"[25.123307421561712, 26.327239151692837]",124310 | |
| 10 | +M8b,11.931505499718048,"[11.633998772014268, 12.282466095441764]",24.075223589694446,"[23.49071882984309, 24.702058885939795]",124310 | |
| 11 | +M13,11.496705653902662,"[11.224051681471648, 11.798197002051694]",22.47922961792988,"[21.977887965134165, 23.029988342848682]",112423 | |
| 12 | +M10,13.835821507708479,"[13.464553572134326, 14.307419270864123]",25.898033630797336,"[25.280532617204084, 26.541255369690628]",124559 | |
| 13 | +M11a,10.739739317150843,"[10.432481977624406, 11.040690082305156]",23.119191766161666,"[22.53046826820057, 23.757778522653194]",124559 | |
| 14 | +M11b,8.967863128030018,"[8.773061391719711, 9.201311937764464]",19.926441862417057,"[19.462644173099918, 20.446830651945625]",124559 | |
| 15 | +M0,11.133125874033468,"[10.842794648724862, 11.453300396249132]",23.617600461764702,"[23.037622351260083, 24.250191986775633]",124555 | |
| 16 | +M9,10.637894275323067,"[10.378412412493105, 10.91923195917346]",21.78627195184587,"[21.280735693377956, 22.330297653813886]",124310 | |
| 17 | +M12,8.912072032345009,"[8.728129360436599, 9.134098588539496]",19.853462924041754,"[19.40133802849491, 20.37297004230134]",124559 | |
added
results/revision-2026-10-17/pairs_B999.csv
+16 −0
@@ -0,0 +1,16 @@ | ||
| 1 | +pair,n,n_clusters,diff_mdape,ci_mdape,diff_mape,ci_mape,t_cluster,share_A_better | |
| 2 | +M7-M2,124310,6346,-5.080226073339361,"[-5.256699131109255, -4.87872951697911]",-6.652592588031027,"[-6.900161453990993, -6.432232383650261]",-57.56448596802397,61.09082133376237 | |
| 3 | +M7-M6,124310,6346,-0.3675335680014269,"[-0.4469048266234854, -0.2785061226606056]",-1.3208433117025686,"[-1.4188744943733822, -1.2382776422006383]",-28.414737891275667,52.32000643552409 | |
| 4 | +M8a-M7,124310,6346,0.459220619083911,"[0.33003880838509747, 0.5874563420225992]",0.589460117651948,"[0.4656602932050286, 0.7228317369548676]",8.723315149160682,48.02670742498592 | |
| 5 | +M8b-M7,124310,6346,-1.0823038700848608,"[-1.1704477078902336, -0.9990520037054463]",-1.0410035037463432,"[-1.1102905562792775, -0.9786011296381123]",-31.27254030581421,54.33915211970075 | |
| 6 | +M11b-M0,124555,6412,-2.165733184796992,"[-2.3010453770472226, -2.0352543487211383]",-3.6921813066515026,"[-3.917617385677863, -3.4942324933129165]",-36.11141907283843,59.0710930914054 | |
| 7 | +M11a-M0,124555,6412,-0.3934626039773126,"[-0.4858772595755506, -0.32403804031274075]",-0.5009655321073974,"[-0.5700053325890613, -0.4324345602027148]",-14.08686715182151,51.83653807554896 | |
| 8 | +M12-M11b,124559,6415,-0.055791095685009395,"[-0.0882174361169985, -0.026170325516347022]",-0.07297893837530553,"[-0.09155086841765518, -0.05628757081132548]",-8.304695774432023,52.587930217808434 | |
| 9 | +M13-M7,112423,5986,-1.4031003594741742,"[-1.579824076330834, -1.2394040463044353]",-2.3935482710226417,"[-2.6407707041102433, -2.1621707075854677]",-20.162489430341008,54.50930859343729 | |
| 10 | +M3-M2,124310,6346,-5.097517141524604,"[-5.256787429842242, -4.920367461142634]",-5.581251273326516,"[-5.792671901092297, -5.376841660452177]",-52.13794398103222,59.820609765907804 | |
| 11 | +M9-M7,124310,6346,-2.3759150944798413,"[-2.5083684405611955, -2.2567855349842136]",-3.329955141594919,"[-3.5133610009942533, -3.162055456900493]",-38.339352975154185,59.68787708148982 | |
| 12 | +M1-M2,120372,5760,1.4143751719224162,"[1.2606079722601327, 1.6107793232686518]",1.824002308424994,"[1.6940734197889777, 1.9701925045304898]",23.22616210239385,45.79802611903101 | |
| 13 | +M0-M13,112423,5986,-0.4807801668167411,"[-0.6409032425930045, -0.3212825775757433]",0.8546683109389558,"[0.629246016724235, 1.085380013591885]",7.399522226037632,50.14276438095408 | |
| 14 | +M0-M9,124310,6346,0.47751432277509664,"[0.36355514904645725, 0.5931530092006615]",1.7856171175442008,"[1.6154733284918703, 1.9606704226547598]",20.597414130011938,46.87474861234012 | |
| 15 | +M0-M7,124310,6346,-1.8984007717047446,"[-2.0021522529118534, -1.7980729469798729]",-1.5443380240507172,"[-1.6444939863625099, -1.44020706858024]",-30.662005925447986,56.55297240769045 | |
| 16 | +M13-M11b,112423,5986,2.647301196336005,"[2.5056589156464484, 2.7967472829070883]",2.911390539266983,"[2.758833553976696, 3.074993097793895]",36.64004370805747,40.51662026453662 | |
added
results/revision-2026-10-17/r8_sensitivity.csv
+5 −0
@@ -0,0 +1,5 @@ | ||
| 1 | +,n,share,M2,M7,M13,M0,M10,M11a,M11b,M12 | |
| 2 | +"$[0.33, 3]$ (baseline)",124559.0,100.0,18.09403544314227,13.013809369802908,11.496705653902662,11.133125874033468,13.835821507708479,10.739739317150843,8.967863128030018,8.912072032345009 | |
| 3 | +"$[0.40, 2.5]$",123112.0,98.83830152779004,17.8369390192443,12.829950135079994,11.322220267951376,10.972716069842004,13.657399106054369,10.573991967167585,8.828711981499671,8.775052848550613 | |
| 4 | +"$[0.50, 2.0]$",119681.0,96.08378358850022,17.40591349398839,12.475219006753774,10.953531311273284,10.648230988421359,13.317440789968115,10.261795015888831,8.519286232482425,8.472341533475635 | |
| 5 | +"$[0.67, 1.5]$",102333.0,82.15624724026365,16.33201238418479,11.46748065922355,9.75126750607252,9.745530375503076,12.358315431772738,9.36069739024542,7.452909166736821,7.4068276566026325 | |
added
results/revision-2026-10-17/revision_results.json
+1059 −0
@@ -0,0 +1,1059 @@ | ||
| 1 | +{ | |
| 2 | + "selection_stability": { | |
| 3 | + "n_configs": 2256, | |
| 4 | + "n_configs_cov97": 1370, | |
| 5 | + "spearman_E1": 0.9982740706649024, | |
| 6 | + "spearman_E2": 0.9858890086369867, | |
| 7 | + "spearman_all": 0.9961530369665204, | |
| 8 | + "spearman_cov97": 0.9943808217392556, | |
| 9 | + "chosen_val": 11.862763, | |
| 10 | + "chosen_test": 11.13311, | |
| 11 | + "chosen_rank_test": 2, | |
| 12 | + "oracle_test": 11.129716, | |
| 13 | + "oracle_desc": "mix|k=10|d=10.0|T=24|gbm|equal|median", | |
| 14 | + "regret_test": 0.003394000000000119, | |
| 15 | + "near_optimal": [ | |
| 16 | + { | |
| 17 | + "tol": 0.1, | |
| 18 | + "n": 12, | |
| 19 | + "test_min": 11.129716, | |
| 20 | + "test_max": 11.25456, | |
| 21 | + "test_median": 11.170390000000001 | |
| 22 | + }, | |
| 23 | + { | |
| 24 | + "tol": 0.25, | |
| 25 | + "n": 37, | |
| 26 | + "test_min": 11.129716, | |
| 27 | + "test_max": 11.389613, | |
| 28 | + "test_median": 11.260313 | |
| 29 | + }, | |
| 30 | + { | |
| 31 | + "tol": 0.5, | |
| 32 | + "n": 92, | |
| 33 | + "test_min": 11.129716, | |
| 34 | + "test_max": 11.6600275, | |
| 35 | + "test_median": 11.365953000000001 | |
| 36 | + }, | |
| 37 | + { | |
| 38 | + "tol": 1.0, | |
| 39 | + "n": 252, | |
| 40 | + "test_min": 11.129716, | |
| 41 | + "test_max": 12.178909, | |
| 42 | + "test_median": 11.715694 | |
| 43 | + } | |
| 44 | + ], | |
| 45 | + "top10_val": [ | |
| 46 | + { | |
| 47 | + "src": "E1", | |
| 48 | + "sel": "mix", | |
| 49 | + "k": 30, | |
| 50 | + "d": 25.0, | |
| 51 | + "T": 24, | |
| 52 | + "adj": "hedo", | |
| 53 | + "w": "equal", | |
| 54 | + "recon": "median", | |
| 55 | + "mdape_val": 11.862763, | |
| 56 | + "mdape_test": 11.13311, | |
| 57 | + "couverture_test": 99.99678867042928, | |
| 58 | + "rank_test": 2.0 | |
| 59 | + }, | |
| 60 | + { | |
| 61 | + "src": "E1", | |
| 62 | + "sel": "mix", | |
| 63 | + "k": 30, | |
| 64 | + "d": 10.0, | |
| 65 | + "T": 24, | |
| 66 | + "adj": "hedo", | |
| 67 | + "w": "equal", | |
| 68 | + "recon": "median", | |
| 69 | + "mdape_val": 11.872429, | |
| 70 | + "mdape_test": 11.136251, | |
| 71 | + "couverture_test": 99.98635184932442, | |
| 72 | + "rank_test": 3.0 | |
| 73 | + }, | |
| 74 | + { | |
| 75 | + "src": "E2", | |
| 76 | + "sel": "mix", | |
| 77 | + "k": 10, | |
| 78 | + "d": 10.0, | |
| 79 | + "T": 24, | |
| 80 | + "adj": "gbm", | |
| 81 | + "w": "equal", | |
| 82 | + "recon": "median", | |
| 83 | + "mdape_val": 11.89453, | |
| 84 | + "mdape_test": 11.129716, | |
| 85 | + "couverture_test": 99.98635184932442, | |
| 86 | + "rank_test": 1.0 | |
| 87 | + }, | |
| 88 | + { | |
| 89 | + "src": "E1", | |
| 90 | + "sel": "mix", | |
| 91 | + "k": 30, | |
| 92 | + "d": 5.0, | |
| 93 | + "T": 24, | |
| 94 | + "adj": "hedo", | |
| 95 | + "w": "equal", | |
| 96 | + "recon": "median", | |
| 97 | + "mdape_val": 11.89692, | |
| 98 | + "mdape_test": 11.159075, | |
| 99 | + "couverture_test": 99.9181110959465, | |
| 100 | + "rank_test": 5.0 | |
| 101 | + }, | |
| 102 | + { | |
| 103 | + "src": "E1", | |
| 104 | + "sel": "mix", | |
| 105 | + "k": 50, | |
| 106 | + "d": 10.0, | |
| 107 | + "T": 24, | |
| 108 | + "adj": "hedo", | |
| 109 | + "w": "equal", | |
| 110 | + "recon": "median", | |
| 111 | + "mdape_val": 11.9100685, | |
| 112 | + "mdape_test": 11.225182, | |
| 113 | + "couverture_test": 99.98635184932442, | |
| 114 | + "rank_test": 12.0 | |
| 115 | + }, | |
| 116 | + { | |
| 117 | + "src": "E1", | |
| 118 | + "sel": "mix", | |
| 119 | + "k": 50, | |
| 120 | + "d": 25.0, | |
| 121 | + "T": 24, | |
| 122 | + "adj": "hedo", | |
| 123 | + "w": "equal", | |
| 124 | + "recon": "median", | |
| 125 | + "mdape_val": 11.920926, | |
| 126 | + "mdape_test": 11.213673, | |
| 127 | + "couverture_test": 99.99678867042928, | |
| 128 | + "rank_test": 11.0 | |
| 129 | + }, | |
| 130 | + { | |
| 131 | + "src": "E1", | |
| 132 | + "sel": "mix", | |
| 133 | + "k": 20, | |
| 134 | + "d": 25.0, | |
| 135 | + "T": 24, | |
| 136 | + "adj": "hedo", | |
| 137 | + "w": "equal", | |
| 138 | + "recon": "median", | |
| 139 | + "mdape_val": 11.927212, | |
| 140 | + "mdape_test": 11.17115, | |
| 141 | + "couverture_test": 99.99678867042928, | |
| 142 | + "rank_test": 7.0 | |
| 143 | + }, | |
| 144 | + { | |
| 145 | + "src": "E1", | |
| 146 | + "sel": "mix", | |
| 147 | + "k": 20, | |
| 148 | + "d": 5.0, | |
| 149 | + "T": 24, | |
| 150 | + "adj": "hedo", | |
| 151 | + "w": "equal", | |
| 152 | + "recon": "median", | |
| 153 | + "mdape_val": 11.927473, | |
| 154 | + "mdape_test": 11.16963, | |
| 155 | + "couverture_test": 99.9181110959465, | |
| 156 | + "rank_test": 6.0 | |
| 157 | + }, | |
| 158 | + { | |
| 159 | + "src": "E1", | |
| 160 | + "sel": "mix", | |
| 161 | + "k": 20, | |
| 162 | + "d": 10.0, | |
| 163 | + "T": 24, | |
| 164 | + "adj": "hedo", | |
| 165 | + "w": "equal", | |
| 166 | + "recon": "median", | |
| 167 | + "mdape_val": 11.93099, | |
| 168 | + "mdape_test": 11.174917, | |
| 169 | + "couverture_test": 99.98635184932442, | |
| 170 | + "rank_test": 8.0 | |
| 171 | + }, | |
| 172 | + { | |
| 173 | + "src": "E1", | |
| 174 | + "sel": "mix", | |
| 175 | + "k": 20, | |
| 176 | + "d": 2.0, | |
| 177 | + "T": 24, | |
| 178 | + "adj": "hedo", | |
| 179 | + "w": "equal", | |
| 180 | + "recon": "median", | |
| 181 | + "mdape_val": 11.93775, | |
| 182 | + "mdape_test": 11.156839, | |
| 183 | + "couverture_test": 99.11367303847976, | |
| 184 | + "rank_test": 4.0 | |
| 185 | + } | |
| 186 | + ] | |
| 187 | + }, | |
| 188 | + "model_ci": { | |
| 189 | + "M1": { | |
| 190 | + "mdape": 19.146214848372246, | |
| 191 | + "ci_mdape": [ | |
| 192 | + 18.685031183072766, | |
| 193 | + 19.683348977052354 | |
| 194 | + ], | |
| 195 | + "mape": 32.800846529772485, | |
| 196 | + "ci_mape": [ | |
| 197 | + 32.122338749775516, | |
| 198 | + 33.580296967360205 | |
| 199 | + ], | |
| 200 | + "n": 120372 | |
| 201 | + }, | |
| 202 | + "M2": { | |
| 203 | + "mdape": 18.09403544314227, | |
| 204 | + "ci_mdape": [ | |
| 205 | + 17.66825564715647, | |
| 206 | + 18.550683626386977 | |
| 207 | + ], | |
| 208 | + "mape": 31.768819681471815, | |
| 209 | + "ci_mape": [ | |
| 210 | + 31.07159938200675, | |
| 211 | + 32.57120626282716 | |
| 212 | + ], | |
| 213 | + "n": 124310 | |
| 214 | + }, | |
| 215 | + "M3": { | |
| 216 | + "mdape": 12.996518301617666, | |
| 217 | + "ci_mdape": [ | |
| 218 | + 12.646356484326454, | |
| 219 | + 13.395039917871683 | |
| 220 | + ], | |
| 221 | + "mape": 26.1875684081453, | |
| 222 | + "ci_mape": [ | |
| 223 | + 25.52101411706073, | |
| 224 | + 26.901367094523625 | |
| 225 | + ], | |
| 226 | + "n": 124310 | |
| 227 | + }, | |
| 228 | + "M4": { | |
| 229 | + "mdape": 17.845039223505278, | |
| 230 | + "ci_mdape": [ | |
| 231 | + 17.43549007960399, | |
| 232 | + 18.31632606539743 | |
| 233 | + ], | |
| 234 | + "mape": 31.393630966552198, | |
| 235 | + "ci_mape": [ | |
| 236 | + 30.714842511391947, | |
| 237 | + 32.17551221382804 | |
| 238 | + ], | |
| 239 | + "n": 124310 | |
| 240 | + }, | |
| 241 | + "M5": { | |
| 242 | + "mdape": 18.500483669907887, | |
| 243 | + "ci_mdape": [ | |
| 244 | + 18.06136224001957, | |
| 245 | + 18.98670190413732 | |
| 246 | + ], | |
| 247 | + "mape": 32.28588625761257, | |
| 248 | + "ci_mape": [ | |
| 249 | + 31.579002770800916, | |
| 250 | + 33.092144223949724 | |
| 251 | + ], | |
| 252 | + "n": 124310 | |
| 253 | + }, | |
| 254 | + "M6": { | |
| 255 | + "mdape": 13.381342937804336, | |
| 256 | + "ci_mdape": [ | |
| 257 | + 13.059504197488089, | |
| 258 | + 13.744105357492264 | |
| 259 | + ], | |
| 260 | + "mape": 26.437070405143366, | |
| 261 | + "ci_mape": [ | |
| 262 | + 25.784908089385414, | |
| 263 | + 27.158469799135688 | |
| 264 | + ], | |
| 265 | + "n": 124310 | |
| 266 | + }, | |
| 267 | + "M7": { | |
| 268 | + "mdape": 13.013809369802908, | |
| 269 | + "ci_mdape": [ | |
| 270 | + 12.714529475654354, | |
| 271 | + 13.35244600835715 | |
| 272 | + ], | |
| 273 | + "mape": 25.11622709344079, | |
| 274 | + "ci_mape": [ | |
| 275 | + 24.520925576464077, | |
| 276 | + 25.754323095646676 | |
| 277 | + ], | |
| 278 | + "n": 124310 | |
| 279 | + }, | |
| 280 | + "M8a": { | |
| 281 | + "mdape": 13.47302998888682, | |
| 282 | + "ci_mdape": [ | |
| 283 | + 13.120802670871669, | |
| 284 | + 13.86299660279164 | |
| 285 | + ], | |
| 286 | + "mape": 25.705687211092737, | |
| 287 | + "ci_mape": [ | |
| 288 | + 25.123307421561712, | |
| 289 | + 26.327239151692837 | |
| 290 | + ], | |
| 291 | + "n": 124310 | |
| 292 | + }, | |
| 293 | + "M8b": { | |
| 294 | + "mdape": 11.931505499718048, | |
| 295 | + "ci_mdape": [ | |
| 296 | + 11.633998772014268, | |
| 297 | + 12.282466095441764 | |
| 298 | + ], | |
| 299 | + "mape": 24.075223589694446, | |
| 300 | + "ci_mape": [ | |
| 301 | + 23.49071882984309, | |
| 302 | + 24.702058885939795 | |
| 303 | + ], | |
| 304 | + "n": 124310 | |
| 305 | + }, | |
| 306 | + "M13": { | |
| 307 | + "mdape": 11.496705653902662, | |
| 308 | + "ci_mdape": [ | |
| 309 | + 11.224051681471648, | |
| 310 | + 11.798197002051694 | |
| 311 | + ], | |
| 312 | + "mape": 22.47922961792988, | |
| 313 | + "ci_mape": [ | |
| 314 | + 21.977887965134165, | |
| 315 | + 23.029988342848682 | |
| 316 | + ], | |
| 317 | + "n": 112423 | |
| 318 | + }, | |
| 319 | + "M10": { | |
| 320 | + "mdape": 13.835821507708479, | |
| 321 | + "ci_mdape": [ | |
| 322 | + 13.464553572134326, | |
| 323 | + 14.307419270864123 | |
| 324 | + ], | |
| 325 | + "mape": 25.898033630797336, | |
| 326 | + "ci_mape": [ | |
| 327 | + 25.280532617204084, | |
| 328 | + 26.541255369690628 | |
| 329 | + ], | |
| 330 | + "n": 124559 | |
| 331 | + }, | |
| 332 | + "M11a": { | |
| 333 | + "mdape": 10.739739317150843, | |
| 334 | + "ci_mdape": [ | |
| 335 | + 10.432481977624406, | |
| 336 | + 11.040690082305156 | |
| 337 | + ], | |
| 338 | + "mape": 23.119191766161666, | |
| 339 | + "ci_mape": [ | |
| 340 | + 22.53046826820057, | |
| 341 | + 23.757778522653194 | |
| 342 | + ], | |
| 343 | + "n": 124559 | |
| 344 | + }, | |
| 345 | + "M11b": { | |
| 346 | + "mdape": 8.967863128030018, | |
| 347 | + "ci_mdape": [ | |
| 348 | + 8.773061391719711, | |
| 349 | + 9.201311937764464 | |
| 350 | + ], | |
| 351 | + "mape": 19.926441862417057, | |
| 352 | + "ci_mape": [ | |
| 353 | + 19.462644173099918, | |
| 354 | + 20.446830651945625 | |
| 355 | + ], | |
| 356 | + "n": 124559 | |
| 357 | + }, | |
| 358 | + "M0": { | |
| 359 | + "mdape": 11.133125874033468, | |
| 360 | + "ci_mdape": [ | |
| 361 | + 10.842794648724862, | |
| 362 | + 11.453300396249132 | |
| 363 | + ], | |
| 364 | + "mape": 23.617600461764702, | |
| 365 | + "ci_mape": [ | |
| 366 | + 23.037622351260083, | |
| 367 | + 24.250191986775633 | |
| 368 | + ], | |
| 369 | + "n": 124555 | |
| 370 | + }, | |
| 371 | + "M9": { | |
| 372 | + "mdape": 10.637894275323067, | |
| 373 | + "ci_mdape": [ | |
| 374 | + 10.378412412493105, | |
| 375 | + 10.91923195917346 | |
| 376 | + ], | |
| 377 | + "mape": 21.78627195184587, | |
| 378 | + "ci_mape": [ | |
| 379 | + 21.280735693377956, | |
| 380 | + 22.330297653813886 | |
| 381 | + ], | |
| 382 | + "n": 124310 | |
| 383 | + }, | |
| 384 | + "M12": { | |
| 385 | + "mdape": 8.912072032345009, | |
| 386 | + "ci_mdape": [ | |
| 387 | + 8.728129360436599, | |
| 388 | + 9.134098588539496 | |
| 389 | + ], | |
| 390 | + "mape": 19.853462924041754, | |
| 391 | + "ci_mape": [ | |
| 392 | + 19.40133802849491, | |
| 393 | + 20.37297004230134 | |
| 394 | + ], | |
| 395 | + "n": 124559 | |
| 396 | + } | |
| 397 | + }, | |
| 398 | + "pairs_B999": { | |
| 399 | + "M7-M2": { | |
| 400 | + "n": 124310, | |
| 401 | + "n_clusters": 6346, | |
| 402 | + "diff_mdape": -5.080226073339361, | |
| 403 | + "ci_mdape": [ | |
| 404 | + -5.256699131109255, | |
| 405 | + -4.87872951697911 | |
| 406 | + ], | |
| 407 | + "diff_mape": -6.652592588031027, | |
| 408 | + "ci_mape": [ | |
| 409 | + -6.900161453990993, | |
| 410 | + -6.432232383650261 | |
| 411 | + ], | |
| 412 | + "t_cluster": -57.56448596802397, | |
| 413 | + "share_A_better": 61.09082133376237 | |
| 414 | + }, | |
| 415 | + "M7-M6": { | |
| 416 | + "n": 124310, | |
| 417 | + "n_clusters": 6346, | |
| 418 | + "diff_mdape": -0.3675335680014269, | |
| 419 | + "ci_mdape": [ | |
| 420 | + -0.4469048266234854, | |
| 421 | + -0.2785061226606056 | |
| 422 | + ], | |
| 423 | + "diff_mape": -1.3208433117025686, | |
| 424 | + "ci_mape": [ | |
| 425 | + -1.4188744943733822, | |
| 426 | + -1.2382776422006383 | |
| 427 | + ], | |
| 428 | + "t_cluster": -28.414737891275667, | |
| 429 | + "share_A_better": 52.32000643552409 | |
| 430 | + }, | |
| 431 | + "M8a-M7": { | |
| 432 | + "n": 124310, | |
| 433 | + "n_clusters": 6346, | |
| 434 | + "diff_mdape": 0.459220619083911, | |
| 435 | + "ci_mdape": [ | |
| 436 | + 0.33003880838509747, | |
| 437 | + 0.5874563420225992 | |
| 438 | + ], | |
| 439 | + "diff_mape": 0.589460117651948, | |
| 440 | + "ci_mape": [ | |
| 441 | + 0.4656602932050286, | |
| 442 | + 0.7228317369548676 | |
| 443 | + ], | |
| 444 | + "t_cluster": 8.723315149160682, | |
| 445 | + "share_A_better": 48.02670742498592 | |
| 446 | + }, | |
| 447 | + "M8b-M7": { | |
| 448 | + "n": 124310, | |
| 449 | + "n_clusters": 6346, | |
| 450 | + "diff_mdape": -1.0823038700848608, | |
| 451 | + "ci_mdape": [ | |
| 452 | + -1.1704477078902336, | |
| 453 | + -0.9990520037054463 | |
| 454 | + ], | |
| 455 | + "diff_mape": -1.0410035037463432, | |
| 456 | + "ci_mape": [ | |
| 457 | + -1.1102905562792775, | |
| 458 | + -0.9786011296381123 | |
| 459 | + ], | |
| 460 | + "t_cluster": -31.27254030581421, | |
| 461 | + "share_A_better": 54.33915211970075 | |
| 462 | + }, | |
| 463 | + "M11b-M0": { | |
| 464 | + "n": 124555, | |
| 465 | + "n_clusters": 6412, | |
| 466 | + "diff_mdape": -2.165733184796992, | |
| 467 | + "ci_mdape": [ | |
| 468 | + -2.3010453770472226, | |
| 469 | + -2.0352543487211383 | |
| 470 | + ], | |
| 471 | + "diff_mape": -3.6921813066515026, | |
| 472 | + "ci_mape": [ | |
| 473 | + -3.917617385677863, | |
| 474 | + -3.4942324933129165 | |
| 475 | + ], | |
| 476 | + "t_cluster": -36.11141907283843, | |
| 477 | + "share_A_better": 59.0710930914054 | |
| 478 | + }, | |
| 479 | + "M11a-M0": { | |
| 480 | + "n": 124555, | |
| 481 | + "n_clusters": 6412, | |
| 482 | + "diff_mdape": -0.3934626039773126, | |
| 483 | + "ci_mdape": [ | |
| 484 | + -0.4858772595755506, | |
| 485 | + -0.32403804031274075 | |
| 486 | + ], | |
| 487 | + "diff_mape": -0.5009655321073974, | |
| 488 | + "ci_mape": [ | |
| 489 | + -0.5700053325890613, | |
| 490 | + -0.4324345602027148 | |
| 491 | + ], | |
| 492 | + "t_cluster": -14.08686715182151, | |
| 493 | + "share_A_better": 51.83653807554896 | |
| 494 | + }, | |
| 495 | + "M12-M11b": { | |
| 496 | + "n": 124559, | |
| 497 | + "n_clusters": 6415, | |
| 498 | + "diff_mdape": -0.055791095685009395, | |
| 499 | + "ci_mdape": [ | |
| 500 | + -0.0882174361169985, | |
| 501 | + -0.026170325516347022 | |
| 502 | + ], | |
| 503 | + "diff_mape": -0.07297893837530553, | |
| 504 | + "ci_mape": [ | |
| 505 | + -0.09155086841765518, | |
| 506 | + -0.05628757081132548 | |
| 507 | + ], | |
| 508 | + "t_cluster": -8.304695774432023, | |
| 509 | + "share_A_better": 52.587930217808434 | |
| 510 | + }, | |
| 511 | + "M13-M7": { | |
| 512 | + "n": 112423, | |
| 513 | + "n_clusters": 5986, | |
| 514 | + "diff_mdape": -1.4031003594741742, | |
| 515 | + "ci_mdape": [ | |
| 516 | + -1.579824076330834, | |
| 517 | + -1.2394040463044353 | |
| 518 | + ], | |
| 519 | + "diff_mape": -2.3935482710226417, | |
| 520 | + "ci_mape": [ | |
| 521 | + -2.6407707041102433, | |
| 522 | + -2.1621707075854677 | |
| 523 | + ], | |
| 524 | + "t_cluster": -20.162489430341008, | |
| 525 | + "share_A_better": 54.50930859343729 | |
| 526 | + }, | |
| 527 | + "M3-M2": { | |
| 528 | + "n": 124310, | |
| 529 | + "n_clusters": 6346, | |
| 530 | + "diff_mdape": -5.097517141524604, | |
| 531 | + "ci_mdape": [ | |
| 532 | + -5.256787429842242, | |
| 533 | + -4.920367461142634 | |
| 534 | + ], | |
| 535 | + "diff_mape": -5.581251273326516, | |
| 536 | + "ci_mape": [ | |
| 537 | + -5.792671901092297, | |
| 538 | + -5.376841660452177 | |
| 539 | + ], | |
| 540 | + "t_cluster": -52.13794398103222, | |
| 541 | + "share_A_better": 59.820609765907804 | |
| 542 | + }, | |
| 543 | + "M9-M7": { | |
| 544 | + "n": 124310, | |
| 545 | + "n_clusters": 6346, | |
| 546 | + "diff_mdape": -2.3759150944798413, | |
| 547 | + "ci_mdape": [ | |
| 548 | + -2.5083684405611955, | |
| 549 | + -2.2567855349842136 | |
| 550 | + ], | |
| 551 | + "diff_mape": -3.329955141594919, | |
| 552 | + "ci_mape": [ | |
| 553 | + -3.5133610009942533, | |
| 554 | + -3.162055456900493 | |
| 555 | + ], | |
| 556 | + "t_cluster": -38.339352975154185, | |
| 557 | + "share_A_better": 59.68787708148982 | |
| 558 | + }, | |
| 559 | + "M1-M2": { | |
| 560 | + "n": 120372, | |
| 561 | + "n_clusters": 5760, | |
| 562 | + "diff_mdape": 1.4143751719224162, | |
| 563 | + "ci_mdape": [ | |
| 564 | + 1.2606079722601327, | |
| 565 | + 1.6107793232686518 | |
| 566 | + ], | |
| 567 | + "diff_mape": 1.824002308424994, | |
| 568 | + "ci_mape": [ | |
| 569 | + 1.6940734197889777, | |
| 570 | + 1.9701925045304898 | |
| 571 | + ], | |
| 572 | + "t_cluster": 23.22616210239385, | |
| 573 | + "share_A_better": 45.79802611903101 | |
| 574 | + }, | |
| 575 | + "M0-M13": { | |
| 576 | + "n": 112423, | |
| 577 | + "n_clusters": 5986, | |
| 578 | + "diff_mdape": -0.4807801668167411, | |
| 579 | + "ci_mdape": [ | |
| 580 | + -0.6409032425930045, | |
| 581 | + -0.3212825775757433 | |
| 582 | + ], | |
| 583 | + "diff_mape": 0.8546683109389558, | |
| 584 | + "ci_mape": [ | |
| 585 | + 0.629246016724235, | |
| 586 | + 1.085380013591885 | |
| 587 | + ], | |
| 588 | + "t_cluster": 7.399522226037632, | |
| 589 | + "share_A_better": 50.14276438095408 | |
| 590 | + }, | |
| 591 | + "M0-M9": { | |
| 592 | + "n": 124310, | |
| 593 | + "n_clusters": 6346, | |
| 594 | + "diff_mdape": 0.47751432277509664, | |
| 595 | + "ci_mdape": [ | |
| 596 | + 0.36355514904645725, | |
| 597 | + 0.5931530092006615 | |
| 598 | + ], | |
| 599 | + "diff_mape": 1.7856171175442008, | |
| 600 | + "ci_mape": [ | |
| 601 | + 1.6154733284918703, | |
| 602 | + 1.9606704226547598 | |
| 603 | + ], | |
| 604 | + "t_cluster": 20.597414130011938, | |
| 605 | + "share_A_better": 46.87474861234012 | |
| 606 | + }, | |
| 607 | + "M0-M7": { | |
| 608 | + "n": 124310, | |
| 609 | + "n_clusters": 6346, | |
| 610 | + "diff_mdape": -1.8984007717047446, | |
| 611 | + "ci_mdape": [ | |
| 612 | + -2.0021522529118534, | |
| 613 | + -1.7980729469798729 | |
| 614 | + ], | |
| 615 | + "diff_mape": -1.5443380240507172, | |
| 616 | + "ci_mape": [ | |
| 617 | + -1.6444939863625099, | |
| 618 | + -1.44020706858024 | |
| 619 | + ], | |
| 620 | + "t_cluster": -30.662005925447986, | |
| 621 | + "share_A_better": 56.55297240769045 | |
| 622 | + }, | |
| 623 | + "M13-M11b": { | |
| 624 | + "n": 112423, | |
| 625 | + "n_clusters": 5986, | |
| 626 | + "diff_mdape": 2.647301196336005, | |
| 627 | + "ci_mdape": [ | |
| 628 | + 2.5056589156464484, | |
| 629 | + 2.7967472829070883 | |
| 630 | + ], | |
| 631 | + "diff_mape": 2.911390539266983, | |
| 632 | + "ci_mape": [ | |
| 633 | + 2.758833553976696, | |
| 634 | + 3.074993097793895 | |
| 635 | + ], | |
| 636 | + "t_cluster": 36.64004370805747, | |
| 637 | + "share_A_better": 40.51662026453662 | |
| 638 | + } | |
| 639 | + }, | |
| 640 | + "pairs_cluster_sensitivity": { | |
| 641 | + "municipality x quarter|M7-M2": { | |
| 642 | + "n": 124310, | |
| 643 | + "n_clusters": 6346, | |
| 644 | + "diff_mdape": -5.080226073339361, | |
| 645 | + "ci_mdape": [ | |
| 646 | + -5.275356600148439, | |
| 647 | + -4.888405744768826 | |
| 648 | + ], | |
| 649 | + "diff_mape": -6.652592588031027, | |
| 650 | + "ci_mape": [ | |
| 651 | + -6.895089962297712, | |
| 652 | + -6.439157807570946 | |
| 653 | + ], | |
| 654 | + "t_cluster": -57.56448596802397, | |
| 655 | + "share_A_better": 61.09082133376237 | |
| 656 | + }, | |
| 657 | + "municipality x quarter|M11b-M0": { | |
| 658 | + "n": 124555, | |
| 659 | + "n_clusters": 6412, | |
| 660 | + "diff_mdape": -2.165733184796992, | |
| 661 | + "ci_mdape": [ | |
| 662 | + -2.291781744930284, | |
| 663 | + -2.0347154832996317 | |
| 664 | + ], | |
| 665 | + "diff_mape": -3.6921813066515026, | |
| 666 | + "ci_mape": [ | |
| 667 | + -3.905927125647312, | |
| 668 | + -3.513150895839829 | |
| 669 | + ], | |
| 670 | + "t_cluster": -36.11141907283843, | |
| 671 | + "share_A_better": 59.0710930914054 | |
| 672 | + }, | |
| 673 | + "municipality x quarter|M12-M11b": { | |
| 674 | + "n": 124559, | |
| 675 | + "n_clusters": 6415, | |
| 676 | + "diff_mdape": -0.055791095685009395, | |
| 677 | + "ci_mdape": [ | |
| 678 | + -0.08598024915158875, | |
| 679 | + -0.02738958127513456 | |
| 680 | + ], | |
| 681 | + "diff_mape": -0.07297893837530553, | |
| 682 | + "ci_mape": [ | |
| 683 | + -0.08987387331172658, | |
| 684 | + -0.05698856424742702 | |
| 685 | + ], | |
| 686 | + "t_cluster": -8.304695774432023, | |
| 687 | + "share_A_better": 52.587930217808434 | |
| 688 | + }, | |
| 689 | + "municipality x quarter|M7-M6": { | |
| 690 | + "n": 124310, | |
| 691 | + "n_clusters": 6346, | |
| 692 | + "diff_mdape": -0.3675335680014269, | |
| 693 | + "ci_mdape": [ | |
| 694 | + -0.45108556641359376, | |
| 695 | + -0.27943734636590944 | |
| 696 | + ], | |
| 697 | + "diff_mape": -1.3208433117025686, | |
| 698 | + "ci_mape": [ | |
| 699 | + -1.421749966256078, | |
| 700 | + -1.2398886186059752 | |
| 701 | + ], | |
| 702 | + "t_cluster": -28.414737891275667, | |
| 703 | + "share_A_better": 52.32000643552409 | |
| 704 | + }, | |
| 705 | + "MRC x quarter|M7-M2": { | |
| 706 | + "n": 124310, | |
| 707 | + "n_clusters": 605, | |
| 708 | + "diff_mdape": -5.080226073339361, | |
| 709 | + "ci_mdape": [ | |
| 710 | + -5.794081488753455, | |
| 711 | + -4.730315225467635 | |
| 712 | + ], | |
| 713 | + "diff_mape": -6.652592588031027, | |
| 714 | + "ci_mape": [ | |
| 715 | + -7.7458357728552665, | |
| 716 | + -6.021956078354418 | |
| 717 | + ], | |
| 718 | + "t_cluster": -13.614417269500166, | |
| 719 | + "share_A_better": 61.09082133376237 | |
| 720 | + }, | |
| 721 | + "MRC x quarter|M11b-M0": { | |
| 722 | + "n": 124555, | |
| 723 | + "n_clusters": 606, | |
| 724 | + "diff_mdape": -2.165733184796992, | |
| 725 | + "ci_mdape": [ | |
| 726 | + -2.7169243164203034, | |
| 727 | + -1.907398286358361 | |
| 728 | + ], | |
| 729 | + "diff_mape": -3.6921813066515026, | |
| 730 | + "ci_mape": [ | |
| 731 | + -4.784447379102366, | |
| 732 | + -3.0745584805397757 | |
| 733 | + ], | |
| 734 | + "t_cluster": -7.164213796079076, | |
| 735 | + "share_A_better": 59.0710930914054 | |
| 736 | + }, | |
| 737 | + "MRC x quarter|M12-M11b": { | |
| 738 | + "n": 124559, | |
| 739 | + "n_clusters": 607, | |
| 740 | + "diff_mdape": -0.055791095685009395, | |
| 741 | + "ci_mdape": [ | |
| 742 | + -0.09775312739142172, | |
| 743 | + -0.02940621650670105 | |
| 744 | + ], | |
| 745 | + "diff_mape": -0.07297893837530553, | |
| 746 | + "ci_mape": [ | |
| 747 | + -0.09274881992025787, | |
| 748 | + -0.03520994860753721 | |
| 749 | + ], | |
| 750 | + "t_cluster": -4.615032828883292, | |
| 751 | + "share_A_better": 52.587930217808434 | |
| 752 | + }, | |
| 753 | + "MRC x quarter|M7-M6": { | |
| 754 | + "n": 124310, | |
| 755 | + "n_clusters": 605, | |
| 756 | + "diff_mdape": -0.3675335680014269, | |
| 757 | + "ci_mdape": [ | |
| 758 | + -0.559629226000643, | |
| 759 | + -0.2642948936716699 | |
| 760 | + ], | |
| 761 | + "diff_mape": -1.3208433117025686, | |
| 762 | + "ci_mape": [ | |
| 763 | + -1.7663514886791307, | |
| 764 | + -1.053462850671406 | |
| 765 | + ], | |
| 766 | + "t_cluster": -5.92518080338482, | |
| 767 | + "share_A_better": 52.32000643552409 | |
| 768 | + }, | |
| 769 | + "municipality|M7-M2": { | |
| 770 | + "n": 124310, | |
| 771 | + "n_clusters": 1079, | |
| 772 | + "diff_mdape": -5.080226073339361, | |
| 773 | + "ci_mdape": [ | |
| 774 | + -5.403486164505034, | |
| 775 | + -4.78735812575733 | |
| 776 | + ], | |
| 777 | + "diff_mape": -6.652592588031027, | |
| 778 | + "ci_mape": [ | |
| 779 | + -7.156470448862836, | |
| 780 | + -6.260029147523143 | |
| 781 | + ], | |
| 782 | + "t_cluster": -29.762323927149772, | |
| 783 | + "share_A_better": 61.09082133376237 | |
| 784 | + }, | |
| 785 | + "municipality|M11b-M0": { | |
| 786 | + "n": 124555, | |
| 787 | + "n_clusters": 1085, | |
| 788 | + "diff_mdape": -2.165733184796992, | |
| 789 | + "ci_mdape": [ | |
| 790 | + -2.4312015349580895, | |
| 791 | + -1.9344881366526112 | |
| 792 | + ], | |
| 793 | + "diff_mape": -3.6921813066515026, | |
| 794 | + "ci_mape": [ | |
| 795 | + -4.171948031877074, | |
| 796 | + -3.32728508548716 | |
| 797 | + ], | |
| 798 | + "t_cluster": -17.381442590817773, | |
| 799 | + "share_A_better": 59.0710930914054 | |
| 800 | + }, | |
| 801 | + "municipality|M12-M11b": { | |
| 802 | + "n": 124559, | |
| 803 | + "n_clusters": 1087, | |
| 804 | + "diff_mdape": -0.055791095685009395, | |
| 805 | + "ci_mdape": [ | |
| 806 | + -0.09271631387223866, | |
| 807 | + -0.025555403659782695 | |
| 808 | + ], | |
| 809 | + "diff_mape": -0.07297893837530553, | |
| 810 | + "ci_mape": [ | |
| 811 | + -0.08872883790595999, | |
| 812 | + -0.05321487513296427 | |
| 813 | + ], | |
| 814 | + "t_cluster": -8.172725284009474, | |
| 815 | + "share_A_better": 52.587930217808434 | |
| 816 | + }, | |
| 817 | + "municipality|M7-M6": { | |
| 818 | + "n": 124310, | |
| 819 | + "n_clusters": 1079, | |
| 820 | + "diff_mdape": -0.3675335680014269, | |
| 821 | + "ci_mdape": [ | |
| 822 | + -0.4688643170879958, | |
| 823 | + -0.26325876250399777 | |
| 824 | + ], | |
| 825 | + "diff_mape": -1.3208433117025686, | |
| 826 | + "ci_mape": [ | |
| 827 | + -1.5151019073472378, | |
| 828 | + -1.1734358107601057 | |
| 829 | + ], | |
| 830 | + "t_cluster": -14.468941665996562, | |
| 831 | + "share_A_better": 52.32000643552409 | |
| 832 | + }, | |
| 833 | + "region x quarter|M7-M2": { | |
| 834 | + "n": 124310, | |
| 835 | + "n_clusters": 119, | |
| 836 | + "diff_mdape": -5.080226073339361, | |
| 837 | + "ci_mdape": [ | |
| 838 | + -5.30211057304464, | |
| 839 | + -4.880557496456786 | |
| 840 | + ], | |
| 841 | + "diff_mape": -6.652592588031027, | |
| 842 | + "ci_mape": [ | |
| 843 | + -6.923181921946154, | |
| 844 | + -6.390046089091484 | |
| 845 | + ], | |
| 846 | + "t_cluster": -52.11871434289954, | |
| 847 | + "share_A_better": 61.09082133376237 | |
| 848 | + }, | |
| 849 | + "region x quarter|M11b-M0": { | |
| 850 | + "n": 124555, | |
| 851 | + "n_clusters": 119, | |
| 852 | + "diff_mdape": -2.165733184796992, | |
| 853 | + "ci_mdape": [ | |
| 854 | + -2.374730037455125, | |
| 855 | + -1.9668543846358748 | |
| 856 | + ], | |
| 857 | + "diff_mape": -3.6921813066515026, | |
| 858 | + "ci_mape": [ | |
| 859 | + -4.0365615650750115, | |
| 860 | + -3.3465098730221503 | |
| 861 | + ], | |
| 862 | + "t_cluster": -20.424646539383705, | |
| 863 | + "share_A_better": 59.0710930914054 | |
| 864 | + }, | |
| 865 | + "region x quarter|M12-M11b": { | |
| 866 | + "n": 124559, | |
| 867 | + "n_clusters": 119, | |
| 868 | + "diff_mdape": -0.055791095685009395, | |
| 869 | + "ci_mdape": [ | |
| 870 | + -0.09065726622101089, | |
| 871 | + -0.022888134340171007 | |
| 872 | + ], | |
| 873 | + "diff_mape": -0.07297893837530553, | |
| 874 | + "ci_mape": [ | |
| 875 | + -0.10689883881723176, | |
| 876 | + -0.044440407356515735 | |
| 877 | + ], | |
| 878 | + "t_cluster": -4.849476732411317, | |
| 879 | + "share_A_better": 52.587930217808434 | |
| 880 | + }, | |
| 881 | + "region x quarter|M7-M6": { | |
| 882 | + "n": 124310, | |
| 883 | + "n_clusters": 119, | |
| 884 | + "diff_mdape": -0.3675335680014269, | |
| 885 | + "ci_mdape": [ | |
| 886 | + -0.4769374263113396, | |
| 887 | + -0.26684833401399877 | |
| 888 | + ], | |
| 889 | + "diff_mape": -1.3208433117025686, | |
| 890 | + "ci_mape": [ | |
| 891 | + -1.45371068425048, | |
| 892 | + -1.2119661367078811 | |
| 893 | + ], | |
| 894 | + "t_cluster": -22.210817715407053, | |
| 895 | + "share_A_better": 52.32000643552409 | |
| 896 | + } | |
| 897 | + }, | |
| 898 | + "r8_sensitivity": { | |
| 899 | + "$[0.33, 3]$ (baseline)": { | |
| 900 | + "n": 124559, | |
| 901 | + "share": 100.0, | |
| 902 | + "M2": 18.09403544314227, | |
| 903 | + "M7": 13.013809369802908, | |
| 904 | + "M13": 11.496705653902662, | |
| 905 | + "M0": 11.133125874033468, | |
| 906 | + "M10": 13.835821507708479, | |
| 907 | + "M11a": 10.739739317150843, | |
| 908 | + "M11b": 8.967863128030018, | |
| 909 | + "M12": 8.912072032345009 | |
| 910 | + }, | |
| 911 | + "$[0.40, 2.5]$": { | |
| 912 | + "n": 123112, | |
| 913 | + "share": 98.83830152779004, | |
| 914 | + "M2": 17.8369390192443, | |
| 915 | + "M7": 12.829950135079994, | |
| 916 | + "M13": 11.322220267951376, | |
| 917 | + "M0": 10.972716069842004, | |
| 918 | + "M10": 13.657399106054369, | |
| 919 | + "M11a": 10.573991967167585, | |
| 920 | + "M11b": 8.828711981499671, | |
| 921 | + "M12": 8.775052848550613 | |
| 922 | + }, | |
| 923 | + "$[0.50, 2.0]$": { | |
| 924 | + "n": 119681, | |
| 925 | + "share": 96.08378358850022, | |
| 926 | + "M2": 17.40591349398839, | |
| 927 | + "M7": 12.475219006753774, | |
| 928 | + "M13": 10.953531311273284, | |
| 929 | + "M0": 10.648230988421359, | |
| 930 | + "M10": 13.317440789968115, | |
| 931 | + "M11a": 10.261795015888831, | |
| 932 | + "M11b": 8.519286232482425, | |
| 933 | + "M12": 8.472341533475635 | |
| 934 | + }, | |
| 935 | + "$[0.67, 1.5]$": { | |
| 936 | + "n": 102333, | |
| 937 | + "share": 82.15624724026365, | |
| 938 | + "M2": 16.33201238418479, | |
| 939 | + "M7": 11.46748065922355, | |
| 940 | + "M13": 9.75126750607252, | |
| 941 | + "M0": 9.745530375503076, | |
| 942 | + "M10": 12.358315431772738, | |
| 943 | + "M11a": 9.36069739024542, | |
| 944 | + "M11b": 7.452909166736821, | |
| 945 | + "M12": 7.4068276566026325 | |
| 946 | + } | |
| 947 | + }, | |
| 948 | + "seasonal_bias": { | |
| 949 | + "Q1": { | |
| 950 | + "n": 21956, | |
| 951 | + "M2": -1.068875400253888, | |
| 952 | + "M7": -2.3761061236467818, | |
| 953 | + "M13": -2.2339609368983213, | |
| 954 | + "M0": -1.5661924429749008, | |
| 955 | + "M10": -1.1737993799618207, | |
| 956 | + "M11a": -1.4248910590112684, | |
| 957 | + "M11b": 1.3958174620385622, | |
| 958 | + "M12": 0.9892310441998564 | |
| 959 | + }, | |
| 960 | + "Q2": { | |
| 961 | + "n": 47648, | |
| 962 | + "M2": -1.1646965873868598, | |
| 963 | + "M7": -1.2175155135622062, | |
| 964 | + "M13": -0.431849558975822, | |
| 965 | + "M0": -0.49261115847897274, | |
| 966 | + "M10": -0.870346596408661, | |
| 967 | + "M11a": -0.1456884651830248, | |
| 968 | + "M11b": 0.042384521698624056, | |
| 969 | + "M12": -0.1456256254450783 | |
| 970 | + }, | |
| 971 | + "Q3": { | |
| 972 | + "n": 38263, | |
| 973 | + "M2": 1.6562630899128905, | |
| 974 | + "M7": 1.5847252345029594, | |
| 975 | + "M13": 1.570493210118229, | |
| 976 | + "M0": 2.4369414849285342, | |
| 977 | + "M10": 2.8330368740560496, | |
| 978 | + "M11a": -0.3836734065058578, | |
| 979 | + "M11b": -0.41298228966486494, | |
| 980 | + "M12": -0.40229743502875337 | |
| 981 | + }, | |
| 982 | + "Q4": { | |
| 983 | + "n": 16692, | |
| 984 | + "M2": 0.17325287391389832, | |
| 985 | + "M7": -0.6645680938810239, | |
| 986 | + "M13": -1.20531268957772, | |
| 987 | + "M0": 0.08939227367225078, | |
| 988 | + "M10": 0.6773160464125016, | |
| 989 | + "M11a": 0.23456491919153777, | |
| 990 | + "M11b": -0.10665514003103671, | |
| 991 | + "M12": -0.30992142146609725 | |
| 992 | + } | |
| 993 | + }, | |
| 994 | + "index_growth": { | |
| 995 | + "Quebec (province)": { | |
| 996 | + "g_2021_2026": 79.04431543669928, | |
| 997 | + "g_2025_2026": 19.61999619891963, | |
| 998 | + "peak_2022": "2022-06", | |
| 999 | + "trough_2023": "2023-02", | |
| 1000 | + "drawdown": -13.12164637096278 | |
| 1001 | + }, | |
| 1002 | + "Montr\u00e9al": { | |
| 1003 | + "g_2021_2026": 33.20268525439665, | |
| 1004 | + "g_2025_2026": 3.6170870275668676, | |
| 1005 | + "peak_2022": "2022-06", | |
| 1006 | + "trough_2023": "2023-03", | |
| 1007 | + "drawdown": -11.908718929212625 | |
| 1008 | + }, | |
| 1009 | + "Capitale-Nationale": { | |
| 1010 | + "g_2021_2026": 80.98962076006319, | |
| 1011 | + "g_2025_2026": 25.43581179618868, | |
| 1012 | + "peak_2022": "2022-08", | |
| 1013 | + "trough_2023": "2023-02", | |
| 1014 | + "drawdown": -14.119876764484074 | |
| 1015 | + }, | |
| 1016 | + "Mont\u00e9r\u00e9gie": { | |
| 1017 | + "g_2021_2026": 66.37243322386337, | |
| 1018 | + "g_2025_2026": 13.858996204029284, | |
| 1019 | + "peak_2022": "2022-06", | |
| 1020 | + "trough_2023": "2023-02", | |
| 1021 | + "drawdown": -17.999345619512997 | |
| 1022 | + }, | |
| 1023 | + "Laval": { | |
| 1024 | + "g_2021_2026": 48.315874338361155, | |
| 1025 | + "g_2025_2026": 12.644214753995154, | |
| 1026 | + "peak_2022": "2022-06", | |
| 1027 | + "trough_2023": "2023-02", | |
| 1028 | + "drawdown": -16.174889100779176 | |
| 1029 | + }, | |
| 1030 | + "Outaouais": { | |
| 1031 | + "g_2021_2026": 68.56080325943911, | |
| 1032 | + "g_2025_2026": 13.850136879571838, | |
| 1033 | + "peak_2022": "2022-06", | |
| 1034 | + "trough_2023": "2023-02", | |
| 1035 | + "drawdown": -11.541935090435974 | |
| 1036 | + }, | |
| 1037 | + "Laurentides": { | |
| 1038 | + "g_2021_2026": 73.56776142890568, | |
| 1039 | + "g_2025_2026": 16.0720157166234, | |
| 1040 | + "peak_2022": "2022-06", | |
| 1041 | + "trough_2023": "2023-02", | |
| 1042 | + "drawdown": -14.34485609503222 | |
| 1043 | + }, | |
| 1044 | + "Lanaudi\u00e8re": { | |
| 1045 | + "g_2021_2026": 88.03729864981979, | |
| 1046 | + "g_2025_2026": 20.12011409183565, | |
| 1047 | + "peak_2022": "2022-06", | |
| 1048 | + "trough_2023": "2023-02", | |
| 1049 | + "drawdown": -15.446544713387533 | |
| 1050 | + }, | |
| 1051 | + "Estrie": { | |
| 1052 | + "g_2021_2026": 111.96526044417152, | |
| 1053 | + "g_2025_2026": 26.129426233831676, | |
| 1054 | + "peak_2022": "2022-08", | |
| 1055 | + "trough_2023": "2023-02", | |
| 1056 | + "drawdown": -15.442819274665375 | |
| 1057 | + } | |
| 1058 | + } | |
| 1059 | +} | |
| \ No newline at end of file | ||
added
results/revision-2026-10-17/seasonal_bias.csv
+5 −0
@@ -0,0 +1,5 @@ | ||
| 1 | +,n,M2,M7,M13,M0,M10,M11a,M11b,M12 | |
| 2 | +Q1,21956.0,-1.068875400253888,-2.3761061236467818,-2.2339609368983213,-1.5661924429749008,-1.1737993799618207,-1.4248910590112684,1.3958174620385622,0.9892310441998564 | |
| 3 | +Q2,47648.0,-1.1646965873868598,-1.2175155135622062,-0.431849558975822,-0.49261115847897274,-0.870346596408661,-0.1456884651830248,0.042384521698624056,-0.1456256254450783 | |
| 4 | +Q3,38263.0,1.6562630899128905,1.5847252345029594,1.570493210118229,2.4369414849285342,2.8330368740560496,-0.3836734065058578,-0.41298228966486494,-0.40229743502875337 | |
| 5 | +Q4,16692.0,0.17325287391389832,-0.6645680938810239,-1.20531268957772,0.08939227367225078,0.6773160464125016,0.23456491919153777,-0.10665514003103671,-0.30992142146609725 | |
added
results/revision-2026-10-17/selection_top10.csv
+11 −0
@@ -0,0 +1,11 @@ | ||
| 1 | +cfg,sel,d,T,k,adj,w,recon,mdape_val,mape_val,biais_val,biais_moy_val,p5_val,p10_val,p20_val,couverture_val,n_val,mdape_test,mape_test,biais_test,biais_moy_test,p5_test,p10_test,p20_test,couverture_test,n_test,src,rank_val,rank_test | |
| 2 | +1293,mix,25.0,24,30,hedo,equal,median,11.862763,26.167511,0.5587876,12.975147,23.808935874681687,43.84597576973532,68.97291457674203,99.99845667103943,64794,11.13311,23.617788,0.32793283,11.195292,25.05639897558587,46.064917027272216,71.5259435287695,99.99678867042928,124555,E1,1.0,2.0 | |
| 3 | +1221,mix,10.0,24,30,hedo,equal,median,11.872429,26.146523,0.5249381,12.9044895,23.79350258507601,43.819739177405665,68.94359132649124,99.98456671039432,64785,11.136251,23.615755,0.2931118,11.130122,25.078075450188265,46.04966321181127,71.4962387302403,99.98635184932442,124542,E1,2.0,3.0 | |
| 4 | +860,mix,10.0,24,10,gbm,equal,median,11.89453,25.846062,-0.0039100647,11.774436,24.12223165367698,43.88301566478895,68.70591866656378,99.98456671039432,64785,11.129716,23.309465,-0.18427074,10.0763,25.1800351640588,46.07374818359171,71.19999357734086,99.98635184932442,124542,E2,3.0,1.0 | |
| 5 | +1149,mix,5.0,24,30,hedo,equal,median,11.89692,26.126862,0.45363903,12.7392,23.7410294004167,43.720966123929315,68.8201250096458,99.90585693340536,64734,11.159075,23.591171,0.19932985,10.940329,25.048370651659052,45.98142245843335,71.37741953612344,99.9181110959465,124457,E1,4.0,5.0 | |
| 6 | +1223,mix,10.0,24,50,hedo,equal,median,11.9100685,26.281542,0.5472779,13.097379,23.608303109807853,43.65768963654603,68.94822131337295,99.98456671039432,64785,11.225182,23.775053,0.33664107,11.329909,24.931959954720252,45.65627533939739,71.3878563572283,99.98635184932442,124542,E1,5.0,12.0 | |
| 7 | +1295,mix,25.0,24,50,hedo,equal,median,11.920926,26.326576,0.59916973,13.2426815,23.646886333822053,43.63916968901921,68.98680453738714,99.99845667103943,64794,11.213673,23.793894,0.39550066,11.461602,24.944805273003155,45.710867942099725,71.40551866986729,99.99678867042928,124555,E1,6.0,11.0 | |
| 8 | +1291,mix,25.0,24,20,hedo,equal,median,11.927212,26.104502,0.46283603,12.775752,23.881472335828384,43.73948607145613,68.7676518249865,99.99845667103943,64794,11.17115,23.565552,0.27427673,10.997648,25.22981077240504,45.937266676835876,71.43361780361114,99.99678867042928,124555,E1,7.0,7.0 | |
| 9 | +1147,mix,5.0,24,20,hedo,equal,median,11.927473,26.074326,0.42672157,12.632256,23.926228875684853,43.69009954471796,68.69202870591866,99.90585693340536,64734,11.16963,23.560242,0.19148588,10.840268,25.22981077240504,45.903547716343255,71.28348814617972,99.9181110959465,124457,E1,8.0,6.0 | |
| 10 | +1219,mix,10.0,24,20,hedo,equal,median,11.93099,26.094849,0.45114756,12.736564,23.86758237518327,43.71170615016591,68.74141523265685,99.98456671039432,64785,11.174917,23.568298,0.260067,10.964221,25.217768286514826,45.93004118530175,71.42799797686237,99.98635184932442,124542,E1,9.0,8.0 | |
| 11 | +1075,mix,2.0,24,20,hedo,equal,median,11.93775,26.05132,0.33410788,12.348404,23.61293309668956,43.165367698124854,68.03457056871672,98.98294621498572,64136,11.156839,23.511223,0.11584759,10.594615,25.013046026381076,45.564752446631715,70.69902616430768,99.11367303847976,123455,E1,10.0,4.0 | |
| 12 | ||