SPB Git

spb/wp11_uqo

Public

UQO Working Paper No. 11 — Half a million prices, twenty models: a systematic assessment of hedonic specifications.

révision scientifique complète (editorial board, 2026-10-17)

- scoreboard uniforme (règle médiane pour tous les modèles ; Duan en robustesse) : le
  « renversement » forward-in-time de la v1.0 était un artefact de retransformation
- inférence : bootstrap par grappes (municipalités) + tests DM/GW groupés sur toutes
  les comparaisons ; décomposition niveau/compétence du generalization gap
- robustesse : splits groupés parcelle/bâtiment, FE spatiaux emboîtés, Box–Cox fin,
  sensibilité HGB ; nouvelle section 7, 9 tables et 1 figure
- méthodologie formalisée (13 équations) ; littérature 33 → 58 références vérifiées
- docs/REVIEW-REPORT.md, docs/CHANGELOG-REVISION.md ; version 1.1, October 2026

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Oct 18, 2026) parent 984118f Browse files

62 changed files +3,545 −758

62 changed files
Unified Split
modified README.md +76 −36
@@ -4,7 +4,7 @@
4 4 **UQO Working Paper No. 11** — *A Systematic Assessment of Hedonic Specifications
5 5 and Estimation Methods for the Quebec Housing Market, 2021–2026*
6 6
7 −[![Paper](https://img.shields.io/badge/paper-PDF%20(23%20p.)-b31b1b?logo=latex&logoColor=white)](paper/main.pdf)
7 +[![Paper](https://img.shields.io/badge/paper-PDF%20(40%20p.)-b31b1b?logo=latex&logoColor=white)](paper/main.pdf)
8 8 [![Python](https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&logoColor=white)](requirements.txt)
9 9 [![Data](https://img.shields.io/badge/data-514%2C212%20sales%20·%20roll--matched%20parquet-4051b5)](data/raw/README.md)
10 10 [![Models](https://img.shields.io/badge/models-20%20·%204%20design%20axes-4051b5)](src/wp11/models.py)
@@ -15,13 +15,19 @@ and Estimation Methods for the Quebec Housing Market, 2021–2026*
15 15 > **TL;DR** — A controlled horse race of **20 hedonic models** on 514,212 Quebec sales
16 16 > (2021–2026) with assessor-grade structural attributes: 6 functional forms × a time-FE
17 17 > ladder × a spatial-FE ladder × 6 estimation methods, all scored on the **same
18 −> price-level scoreboard** (Duan smearing / Box–Cox inversion) under **two holdouts**
19 −> (random 80/20 and forward-in-time). Functional form is second-order (±2 pp of MdAPE);
20 −> spatial controls are first-order (~10 pp) but overfit at ~1 km granularity; **gradient
21 −> boosting wins under random validation** (MdAPE 14.4% vs 16.5% for the best linear
22 −> model) — but under the forward-in-time split *every* model degrades and the ML
23 −> advantage **reverses** (splines 19.1% vs boosting 21.2%, identical R²_ln): random
24 −> cross-validation is the wrong experiment for valuation models.
18 +> price-level scoreboard** (one retransformation rule for every model) under **two
19 +> holdouts** (random 80/20 and forward-in-time), with municipality-clustered bootstrap
20 +> intervals and Diebold–Mariano-type tests on every comparison. Functional form is
21 +> second-order (1.5 pp of MdAPE); spatial controls are first-order (~10 pp), and a ~1 km
22 +> grid overfits when used alone but becomes the best linear specification once nested in
23 +> municipalities; **gradient boosting wins under random validation** (14.4% vs 17.6% for
24 +> the best linear model, −3.2 pp [−4.7, −1.9]), a gain that survives parcel- and
25 +> building-grouped holdouts. Under the forward-in-time split (train < 2025, test 2025–26)
26 +> every model degrades and the gap shrinks to 0.5 pp and to insignificance — but the
27 +> decomposition shows this is a *level* (staleness) effect: re-levelled, boosting still
28 +> leads by 2.9 pp. Random cross-validation hides the staleness cost; it does not reverse
29 +> the ranking (the reversal reported in v1.0 was an artefact of an asymmetric
30 +> retransformation, corrected in v1.1).
25 31
26 32 ---
27 33
@@ -29,8 +35,8 @@ and Estimation Methods for the Quebec Housing Market, 2021–2026*
29 35
30 36 | Item | Value |
31 37 |---|---|
32 −| Version | 1.0 (August 2026) |
33 −| SSRN | [7626698](https://ssrn.com/abstract=7626698) — submitted 2026-10-17 (preliminary upload) |
38 +| Version | **1.1 (October 2026)** — revised 2026-10-17 by the editorial-board review (see `docs/`); v1.0 = August 2026 |
39 +| SSRN | [7626698](https://ssrn.com/abstract=7626698) — v1.0 uploaded 2026-10-17; v1.1 PDF to be uploaded as a revision |
34 40 | Journal | not yet submitted |
35 41 | Code & data | reproducible from `scripts/` (see Quick start); raw parquet snapshot not redistributed |
36 42
@@ -77,23 +83,35 @@ and accuracy by market segment.
77 83
78 84 ## 🏆 Headline results
79 85
86 +All MdAPE in %, median retransformation rule for every model (v1.1 scoreboard).
87 +
80 88 | Object | Random 80/20 holdout | Forward-in-time holdout |
81 89 |---|---|---|
82 −| Functional-form range (A1–A6), MdAPE | 16.5 % (splines) – 18.6 % (Box–Cox) | 19.1 % – 21.0 % |
83 −| Data-driven Box–Cox exponent | λ̂ = 0.25, buys nothing out of sample | — |
84 −| Dropping spatial FE (C1 vs C2) | 27.9 % vs 18.1 % (+10 pp) | 26.5 % vs 20.5 % |
85 −| ~1.1 km grid FE (C4) | 19.4 % — **overfits** vs 17.5 % at ~5.5 km | 22.1 % |
86 −| Dropping time FE (B1 vs B3) | 21.1 % vs 18.1 % | 25.4 % vs 20.5 % |
87 −| Gradient boosting (D4) | **14.4 %** (−13 % vs best linear 16.5 %) | 21.2 % |
88 −| Random forest (D3) | 14.6 % | 21.3 % |
89 −| Boosting without coordinates (D5) | 23.2 % | 26.2 % |
90 −| Semi-log + splines (A6) | 16.5 % | **19.1 %** — beats boosting, same R²_ln (0.52) |
91 −| Implied monthly indices across forms | agree within a few index points | |
92 −| Boosted implicit age / area profiles | track quadratic OLS closely | |
93 −
94 −Practical synthesis: a semi-log with splines, quarter effects and spatial FE matched to
95 −market thickness is within two points of the frontier in deployment conditions —
96 −transparent, auditable, fast. Validate against the future, not a shuffle of the past.
90 +| Functional-form range (A1–A6) | 17.6 (splines) – 19.2 (quadratics) | 20.4 (linear) – 23.3 |
91 +| Splines vs. semi-log | −1.4 pp [−1.8, −1.1] | −1.4 pp [−2.3, −0.8] |
92 +| Data-driven Box–Cox exponent | λ̂ = 0.25 (0.35 on a fine grid), −0.5 pp vs. semi-log, worse on MAPE | — |
93 +| Dropping spatial FE (C1 vs C2) | 28.8 vs 19.2 (+9.6 pp) | 29.0 vs 23.3 |
94 +| ~5.5 km grid vs. municipality FE (C3 vs C2) | 18.7 vs 19.2 — **not significant** (−0.45 [−1.07, +0.37]) | 24.3 vs 23.3 |
95 +| ~1.1 km grid alone (C4) | 21.0 — overfits (+1.8 pp [+1.2, +2.3]) | 25.3 |
96 +| ~1.1 km grid **nested in municipalities** (C4′) | **17.3** — best linear specification | 21.6 |
97 +| Dropping time FE (B1 vs B3) | 22.0 vs 19.2 | 29.3 vs 23.3 |
98 +| Gradient boosting (D4) vs. best linear (A6) | **14.4** vs 17.6, −3.2 pp [−4.7, −1.9], DM t = −3.6 | 21.2 vs 21.7, −0.5 pp [−1.7, +0.6], n.s. |
99 +| Random forest (D3) | 14.6 | 21.3 |
100 +| Boosting without coordinates (D5) | 23.2 | 26.2 |
101 +| Parcel- / building-grouped holdout, D4 vs A6 | 14.5 vs 17.6 / 14.6 vs 17.8 (gap unchanged) | — |
102 +| Staleness cost (forward − same-period random) | — | D4 +7.5 pp, A6 +4.7, A1 +3.3 |
103 +| Median log level error, forward split | — | D4 +0.18, A6 +0.16, A1 +0.10 |
104 +| Re-levelled forward MdAPE | — | **D4 13.9 vs A6 16.8** (gap 2.9 ≈ random 3.5) |
105 +| Boosting lead over splines by test quarter | — | 1.5 pp (2025Q1) → 0.0 (2026Q3) |
106 +| Monthly indices, four linear forms | agree within 2.5 index points | |
107 +| Boosted imputation index, July 2026 | 160.5 vs 168.6–171.0 for the linear forms | |
108 +| Boosted age profile vs. quadratic OLS | same shape, 0.10–0.13 log points flatter from age 20 | |
109 +
110 +Practical synthesis: a semi-log with splines, quarter effects and spatial fixed effects
111 +nested from municipality down to a fine grid is within three points of the frontier under
112 +random validation and indistinguishable from it in deployment. Supply the price level from
113 +an index or a frequent refit, spend the flexibility budget on the spatial surface, and
114 +validate against the future, not a shuffle of the past.
97 115
98 116 ## 🗂 Repository layout
99 117
@@ -148,8 +166,9 @@ peak RAM ≈ 8 GB.
148 166 | 01 | `01_build_sample.py` | residential use codes, roll match ≤ 50 m / score ≥ 150, price ≥ $50k, floor area 20–2,000 m², known year built, 1/99 % trims within sale year; builds grid cells | `data/processed/`, `results/reproduced/sample_counts.csv`, `summary_stats.csv` |
149 167 | 02 | `02_horserace.py` | fits the 20 models on each split, retransforms predictions to price levels, scores MdAPE/MAPE/RMSE_ln/R²_ln; Box–Cox profile likelihood | `horserace_random.csv`, `horserace_temporal.csv`, `boxcox.csv` |
150 168 | 03 | `03_extensions.py` | constant-quality monthly indices by form, implicit ln-price profiles (quadratic OLS vs boosting), learning curves, accuracy by segment | `index.csv`, `profiles.csv`, `learning.csv`, `segments.csv` |
151 −| 04 | `04_make_figures.py` | the 9 figures | `figures/fig_*.png` |
152 −| 05 | `05_make_tables.py` | the 7 LaTeX tables | `results/tables/*.tex` |
169 +| 04 | `04_make_figures.py` | the 10 figures | `figures/fig_*.png` |
170 +| 05 | `05_make_tables.py` | the 16 LaTeX tables | `results/tables/*.tex` |
171 +| 06 | `06_robustness.py` | cluster-bootstrap CIs + clustered DM tests, same-period and by-quarter decomposition, level re-levelling, leakage decomposition, parcel- and building-grouped races, nested spatial FE, fine Box–Cox grid, boosting hyper-parameter sensitivity (run after 02; ≈ 20 min) | `inference_*.csv`, `pairs.csv`, `same_period.csv`, `by_quarter.csv`, `relevel.csv`, `leakage.csv`, `horserace_group_*.csv`, `nested_fe.csv`, `boxcox_fine.csv`, `hgb_sensitivity.csv` |
153 172
154 173 ## 🗃 Data
155 174
@@ -161,23 +180,28 @@ physical configuration, use code. Sample cascade (paper §3): residential use co
161 180 high-confidence match → price ≥ $50,000 → plausible floor area → known year built →
162 181 1/99 % trims ⇒ **514,212 sales** (348,253 single-family, 75,734 condominiums, 74,909
163 182 plexes, 10,349 cottages, 4,881 mobile homes) over 1,115 municipalities, 5,255 cells of
164 −~5.5 km and 30,548 cells of ~1.1 km. Mean price $462,000 (median $399,000). The raw
183 +~5.5 km and 30,548 cells of ~1.1 km. Mean price $440,207 (median $397,375); 86 further dwellings of undetermined class. One fifth of sales are repeat sales of the same parcel and 30% share exact coordinates with another sale (92% of condominiums). The raw
165 184 parquet (100 MB, transaction microdata) is **not redistributed**.
166 185
167 186 ## 🔬 Methodology
168 187
169 188 1. **One attribute set, one sample** for every model (`src/wp11/sample.py`).
170 189 2. **Axis A** — six functional forms on a municipality + quarter FE backbone; Box–Cox λ by
171 − profile likelihood; log models retransformed with Duan smearing, Box–Cox inverted
172 − (`src/wp11/models.py`).
190 + profile likelihood. Every transformed-scale prediction is inverted with the same
191 + *median rule* (plain inversion); Duan smearing is computed for every model and
192 + reported as a robustness scoreboard (`src/wp11/models.py`).
173 193 3. **Axis B / C** — time-FE ladder (none → month) and spatial-FE ladder (none → ~1.1 km
174 194 grid) on the quadratic semi-log backbone.
175 195 4. **Axis D** — OLS, cross-validated ridge, random forest, gradient boosting with and
176 196 without coordinates, spatial k-NN comparables, all on the identical attribute set.
177 197 5. **Two holdouts** — random 80/20 and forward-in-time (train < 2025, test 2025–26, time
178 − effects carried forward); metrics on price levels, design clustered by municipality.
198 + effects carried forward), plus parcel- and building-grouped holdouts; metrics on price
199 + levels; municipality-clustered bootstrap intervals and clustered Diebold–Mariano tests
200 + (`src/wp11/inference.py`); level/skill decomposition of the forward error.
179 201 6. **Extensions** — implied monthly indices, implicit-price profiles, learning curves,
180 202 segment accuracy (`scripts/03_extensions.py`).
203 +7. **Robustness** — retransformation rule, leakage-free holdouts, nested spatial FE, fine
204 + Box–Cox grid, boosting hyper-parameters (`scripts/06_robustness.py`).
181 205
182 206 ## 🖼 Figures & tables inventory
183 207
@@ -192,10 +216,13 @@ parquet (100 MB, transaction microdata) is **not redistributed**.
192 216 | `fig_profiles` | implicit ln-price profiles: quadratic OLS vs gradient boosting |
193 217 | `fig_learning` | learning curves: MdAPE against training-set size |
194 218 | `fig_segments` | MdAPE by market segment (random holdout) |
219 +| `fig_horizon` | forward-in-time MdAPE by test quarter (distance from the training window) |
195 220
196 221 Tables (`results/tables/*.tex`): summary statistics, scoreboard random holdout,
197 −scoreboard forward-in-time holdout, Box–Cox profile, generalization gap, learning
198 −curves, segments.
222 +scoreboard forward-in-time holdout, inference (bootstrap CIs + DM tests), targeted pairs,
223 +Box–Cox profile, generalization gap, same-period / level decomposition, by-quarter
224 +accuracy, learning curves, segments, median vs. Duan retransformation, grouped holdouts,
225 +seen/unseen leakage decomposition, nested spatial FE, boosting hyper-parameters.
199 226
200 227 ## ⚠️ Limitations
201 228
@@ -206,8 +233,13 @@ curves, segments.
206 233 - Attributes are those of the assessment roll (no interior quality, renovations or
207 234 listing text); the ML advantage measured here is the value of flexible *spatial*
208 235 learning on that attribute set, not of richer data.
209 −- Tree ensembles are tuned lightly (fixed hyper-parameters); the ranking is robust to
210 − the choices tried but not exhaustively optimised.
236 +- Tree ensembles use fixed, conventional hyper-parameters; four materially different
237 + configurations (including early stopping on a time-ordered validation set) leave the
238 + ranking unchanged (paper §7.5), but no linear model with a continuous spatial surface
239 + (kriging, bivariate splines) is estimated as a competitor.
240 +- The forward-in-time test covers one regime (the 2025–26 recovery); the level
241 + re-levelling uses the test-set median ratio (an oracle upper bound for what a published
242 + index would deliver).
211 243
212 244 ## 📝 Citation
213 245
@@ -222,7 +254,8 @@ curves, segments.
222 254 type = {Working Paper},
223 255 number = {11},
224 256 year = {2026},
225 − month = {August}
257 + month = {October},
258 + note = {Version 1.1}
226 259 }
227 260 ```
228 261
@@ -239,3 +272,10 @@ Université du Québec en Outaouais (UQO), Gatineau, QC.
239 272 Language-Model and Vision Features* (now **WP 14**) and *Does the Sales Comparison
240 273 Approach Actually Work?* (now **WP 16**).
241 274 - Cover page, metadata and layout follow `Recherche-UQO/HARMONISATION.md` (2026-10-17).
275 +- **Revision 2026-10-17 (v1.1)**: full editorial-board review. Scientific report in
276 + `docs/REVIEW-REPORT.md`, audit trail (every changed number, old → new) in
277 + `docs/CHANGELOG-REVISION.md`; pre-revision PDF kept as `docs/main_pre-revision-20261017.pdf`
278 + (git tag `pre-revision-20261017`). Main correction: v1.0 applied Duan smearing to the
279 + linear log models but not to the tree ensembles; under a uniform rule the forward-in-time
280 + "reversal" disappears (the gap shrinks to 0.5 pp, n.s.) and the paper now decomposes the
281 + forward error into a common level (staleness) component and intact relative skill.
added docs/CHANGELOG-REVISION.md +165 −0
@@ -0,0 +1,165 @@
1 +# Changelog — révision scientifique complète (editorial board, 2026-10-17)
2 +
3 +Papier : **WP 11 — Half a Million Prices, Twenty Models** (v1.0, August 2026 → **v1.1, October 2026**).
4 +État de référence : tag git `pre-revision-20261017`, PDF `docs/main_pre-revision-20261017.pdf` (23 pages).
5 +Rapport scientifique associé : `docs/REVIEW-REPORT.md`.
6 +
7 +## 1. Corrections majeures (où / avant / après / pourquoi / preuve)
8 +
9 +### 1.1 Règle de retransformation asymétrique → scoreboard uniforme (correction centrale)
10 +- **Où** : `src/wp11/models.py` (`fit_predict`, `run_horserace`), §4.3 Methodology, §5, §7.1, abstract, intro, conclusion.
11 +- **Avant** : les modèles linéaires en log (A2, A3, A5, A6, B*, C*, D1, D2) étaient multipliés par le facteur de
12 + smearing de Duan (ŝ ≈ 1.06–1.08) ; les ensembles d'arbres (D3, D4, D5) et le k-NN (D6) ne l'étaient pas
13 + (`np.exp(pred)` nu). Les métriques log (RMSE_ln, R²_ln) étaient calculées sur la prédiction smearée pour une
14 + famille et non pour l'autre.
15 +- **Après** : règle « médiane » (inversion directe de la prédiction transformée, éq. 10) appliquée **à tous** les
16 + modèles pour le tableau principal ; règle « moyenne » (Duan, éq. 11, généralisée au Box–Cox par moyenne sur
17 + 1 000 résidus) calculée pour tous et rapportée en robustesse (Table 12).
18 +- **Pourquoi** : la comparaison v1.0 opposait un A6 smearé (19.1 %) à un D4 non smearé (21.2 %) dans le split
19 + temporel — le « renversement » annoncé en titre de la v1.0 était un artefact : (i) résidus log asymétriques
20 + à gauche (skewness −1.4 à −1.8, médiane +0.06) donc le smearing abaisse par coïncidence la MdAPE de tout modèle
21 + log ; (ii) marché haussier en 2025-26 : tout facteur > 1 compense une partie de la staleness. Appliquée
22 + uniformément, aucune des deux règles ne renverse le classement (médiane : A6 21.7 vs D4 21.2 ; moyenne :
23 + 19.1 vs 18.4).
24 +- **Preuve** : `results/reproduced/horserace_{random,temporal}.csv` (colonnes `mdape` vs `mdape_mean`, `smear`),
25 + `results/tables/retransformation.tex`, médianes de résidus vérifiées sur `data/processed/predictions_*.parquet`.
26 +- **Chiffres modifiés (ancien → nouveau, MdAPE %)** — random : A1 18.2→18.2, A2 18.2→19.0, A3 17.0→18.0, A4 18.6→18.6,
27 + A5 18.1→19.2, A6 16.5→17.6, B1 21.1→22.0, B2 18.1→19.1, B4 18.3→19.3, C1 27.9→28.8, C3 17.5→18.7, C4 19.4→21.0,
28 + D2 16.7→17.7 ; D3/D4/D5/D6 inchangés (14.6/14.4/23.2/21.2). Forward : A1 20.4→20.4, A2 20.6→23.2, A3 19.2→21.8,
29 + A4 21.0→21.0, A5 20.5→23.3, A6 19.1→21.7, B1 25.4→29.3, B2 21.0→23.7, B4 20.3→22.9, C1 26.5→29.0, C3 21.3→24.3,
30 + C4 22.1→25.3, D2 19.5→22.3 ; D3/D4/D5/D6 inchangés (21.3/21.2/26.2/25.6). R²_ln des modèles log : +0.01 à +0.04
31 + (random), −0.03 à −0.04 (forward).
32 +- **Conséquence narrative** : « the ranking reverses » (v1.0) → « the gap shrinks from 3.2 to 0.5 points and loses
33 + significance ; relative skill transfers intact, the level does not » (v1.1). Abstract, §1, §5.5, §8, §9 réécrits.
34 +
35 +### 1.2 Inférence statistique ajoutée (absente en v1.0)
36 +- **Avant** : aucun intervalle, aucun test ; le README affirmait « design clustered by municipality » sans aucune
37 + implémentation.
38 +- **Après** : `src/wp11/inference.py` — bootstrap par grappes (municipalités, B = 500) pour la MdAPE de chaque
39 + modèle et pour les différences ; test de type Diebold–Mariano / Giacomini–White à erreurs-types groupées sur les
40 + pertes APE et log² (éq. 13). Tables 4 (tous les modèles vs A6) et 5 (13 paires ciblées).
41 +- **Résultats clés** : D4 vs A6 random −3.2 [−4.7, −1.9] (t = −3.6) ; forward −0.5 [−1.7, +0.6] (t = −1.6, p = 0.11) ;
42 + C3 vs C2 random −0.45 [−1.07, +0.37] (n.s.) ; C4 vs C2 +1.8 [+1.2, +2.3] ; B4 vs B3 +0.19 [+0.01, +0.37] ;
43 + D2 vs A6 +0.06 [−0.02, +0.16] (n.s.).
44 +- **Conséquence** : la phrase « the 5.5 km grid beats municipality effects » est ramenée à « nominally better,
45 + within bootstrap noise » ; « month effects indistinguishable from quarter » précisé (légèrement pires, significatif).
46 +
47 +### 1.3 Décomposition du « generalization gap » (nouveau)
48 +- Comparaison à période fixe (modèles random évalués sur les 31 465 ventes 2025-26 de leur test) : D4 13.7 → 21.2
49 + (coût de staleness 7.5), A6 17.0 → 21.7 (4.7), A1 17.1 → 20.4 (3.3). Biais de niveau médian forward : D4 +0.183,
50 + A6 +0.161, A1 +0.102. MdAPE re-nivelée forward : D4 13.9 vs A6 16.8 (écart 2.9 ≈ écart random 3.5).
51 + Par trimestre : avance D4 sur A6 de 1.5 pt (2025Q1) → 0 (2026Q3).
52 +- Fichiers : `same_period.csv`, `relevel.csv`, `by_quarter.csv` ; Tables 8–9, Figure 6 (nouvelle).
53 +
54 +### 1.4 Fuite d'information dans le split aléatoire (nouveau)
55 +- Faits : 20.6 % des ventes sont des reventes d'une même parcelle ; 30.3 % partagent des coordonnées exactes avec
56 + une autre vente (91.6 % des condos). Colonnes `parcel`, `coord` ajoutées à `sample.py` (reconstruction du
57 + parquet d'analyse, 514 212 ventes identiques).
58 +- Splits groupés (`split("group_parcel")`, `split("group_coord")`) : course complète. Parcelle : D4 14.5 / A6 17.6
59 + (≈ inchangé). Bâtiment : D4 14.6 / A6 17.8 (écart 3.2 identique) ; seul D6 se dégrade (21.2 → 22.3).
60 + Décomposition vu/non vu : D4 12.0 vs 15.3 (coord.), A6 17.8 vs 17.6 ; condos D4 8.9 vs 10.2.
61 +- Tables 13–14 ; §4.4, §7.2.
62 +
63 +### 1.5 Effets spatiaux fins : « seventeen K sales » et repli arbitraire
64 +- **Avant** (§5.3) : « its cells average seventeen K sales each » — faux : 514 212 / 30 548 = 16.8 ventes par cellule
65 + (médiane 3). Corrigé. Cellules non vues (2.5 % des ventes test) tarifées au niveau de la cellule de référence :
66 + désormais explicité.
67 +- **Nouveau** : C3′ (muni + grille 5.5 km) et C4′ (muni + grille 1.1 km) — solution de norme minimale (ridge α→0)
68 + ≈ pooling partiel vers la municipalité. Random : C4′ 17.3 % (meilleure spécification linéaire du papier),
69 + C3′ 17.7 % ; forward : 21.6 / 21.5 vs C2 23.3. Conclusion de l'axe C reformulée : « hierarchy, not coarseness ».
70 + Table 15 ; `nested_fe.csv`.
71 +
72 +### 1.6 Box–Cox : grille fine
73 +- Grille 0.05 sur [−0.25, 0.75] : λ̂ = 0.35 (profil plat 0.25–0.55 à la tolérance du solveur itératif) ;
74 + MdAPE random 18.4 % (vs 18.6 % à 0.25). `boxcox_fine.csv`. Texte §5.1 et §7.4.
75 +
76 +### 1.7 Hyper-paramètres du boosting
77 +- Quatre configurations alternatives (shallow, deep, régularisée, early stopping sur les 10 % derniers mois
78 + d'entraînement) sous les deux splits. `hgb_sensitivity.csv`, Table 16, §7.5. Random 14.3–14.7 % (baseline 14.4), forward 21.2–21.6 % (baseline 21.2) ; early stopping : 700 / 1 000 itérations, reproduit le baseline à 0.1 pt.
79 +
80 +### 1.8 Statistiques descriptives incohérentes
81 +- **Avant** (§3.2, README) : « Prices average $462,000 (median $399,000) ». **Table 1** : 440 207 / 397 375.
82 + **Après** : texte aligné sur la table ; ajout des 86 logements de classe indéterminée (la somme des classes v1.0
83 + faisait 514 126 ≠ 514 212) ; répartition par année ; nettoyage des lots > P99.5 explicité (était silencieux
84 + dans `sample.py`) ; bornes âge/étages/logements explicitées.
85 +
86 +### 1.9 Indices et profils : affirmations adoucies
87 +- **Avant** : « all five [indices] agree to within a few index points » ; « curves sit within a tenth of a log
88 + point ». **Après** : les quatre formes linéaires s'accordent à ≤ 2.5 points ; l'indice boosting finit 9 points
89 + plus bas (160.5 vs 168.6–171.0 ; écart max 10.9) ; profil d'âge du boosting uniformément plus plat de 0.10–0.13
90 + log-point à partir de 20 ans (−0.20 vs −0.31 à 40 ans), surface : ≤ 0.05. Chiffres vérifiés dans
91 + `index.csv`, `profiles.csv`.
92 +- Learning curves recalculées (règle médiane) : OLS 19.1 → 19.2 (v1.0 : 18.4 → 18.1) ; GB 17.6 → 14.5 ; écart
93 + 1.6 → 4.7 pts (v1.0 : < 1 → 3.6). Segments : condo 18.2 vs 9.1 (v1.0 OLS 20.4), grandes villes 17.7 vs 11.9,
94 + petites municipalités 24.8 vs 18.7 (v1.0 23.3), chalets 34.3 vs 32.1 (v1.0 33.9).
95 +
96 +### 1.10 Méthodologie formalisée
97 +- §4 réécrit : équations (1)–(13) — fonction hédonique générale, six formes A1–A6, profil de vraisemblance Box–Cox
98 + (avec jacobien), ridge, règle k-NN, règles médiane/Duan, métriques, test DM groupé. Documentation du traitement
99 + des catégories non vues, du carry-forward et de son analogue pour les arbres (correction de la phrase « clamped
100 + month counter » : rien n'est écrêté dans le code ; les arbres sont constants par morceaux).
101 +
102 +### 1.11 Autres corrections éditoriales
103 +- « 13 % improvement » → « 18 % » (14.4 vs 17.6) ; « 13–21 % » → « 18–25 % » ; intro « 27.9 % versus 17.5 % »
104 + vs README « vs 18.1 % » harmonisé (28.8 vs 19.2, référence = backbone C2).
105 +- Hiérarchie de la discussion : liste à puces → prose (règle de style du protocole).
106 +- Mots-clés harmonisés au vocabulaire commun (« hedonic price function », « forward-in-time holdout »,
107 + « automated valuation model »).
108 +- `\WPversion` 1.0 → 1.1 ; `\WPdate` August 2026 → October 2026 ; page titre canonique intacte ; macro
109 + `\companionpaper` conservée (main.tex / main_anon.tex).
110 +
111 +## 2. Références
112 +
113 +### Vérifiées (Crossref `api.crossref.org/works?query.bibliographic=…`, 2026-10-17) et corrigées
114 +Toutes les entrées conservées de la v1.0 (31) ont été retrouvées sur Crossref et dotées de leur DOI. Corrections :
115 +- `box1964analysis` : pages 211–252 → **211–243** (244–252 = discussion).
116 +- `bogin2020house` : number 1 → **1–2** ; DOI 10.1007/s11146-019-09712-0.
117 +- `silver2018house` : l'article EURONA 2018 n'est pas indexé ; remplacé par la version vérifiée **IMF Working Paper
118 + 16/213 (2016)**, DOI 10.5089/9781475552249.001, même titre (ordre des mots corrigé).
119 +- `ke2017lightgbm` : @article → **@inproceedings** (NIPS 2017, non indexé Crossref ; actes Curran Associates).
120 +- `malpezzi2003hedonic` : DOI 10.1002/9780470690680.ch5 ; `anselin1988spatial` : DOI 10.1007/978-94-015-7799-1.
121 +
122 +### Retirées (non citées dans le texte et bibliographiquement défectueuses)
123 +- `knight2003listing` (volume/pages vides ; en réalité chapitre Springer 2008, pp. 39–54).
124 +- `diewert2004hedonic` (« Mimeo » en champ journal ; non indexé).
125 +
126 +### Ajoutées (27, toutes vérifiées sur Crossref, DOI dans le .bib)
127 +hill2018geospatial (RIW 64(4) 737–756) · pace2020residuals (JREFE 60 170–180) · lorenz2023interpretable
128 +(REE 51(5) 1178–1208) · diewert2016tokyo (RSUE 60 300–315) · diewert2015decomposition (Econometric Reviews 34
129 +106–126) · diebold1995comparing (JBES 13(3) 253–263) · giacomini2006tests (Econometrica 74(6) 1545–1578) ·
130 +bergmeir2012crossvalidation (Information Sciences 191 192–213) · roberts2017crossvalidation (Ecography 40(8)
131 +913–929) · kuminoff2010which (JEEM 60(3) 145–160) · bishop2020best (REEP 14(2) 260–281) · bourassa2007spatial
132 +(JREFE 35(2) 143–160) · bourassa2003submarkets (JHE 12(1) 12–28) · goodman2003segmentation (JHE 12(3) 181–201) ·
133 +zurada2011comparison (JRER 33(3) 349–388) · antipov2012mass (ESWA 39(2) 1772–1778) · perezrave2019machine
134 +(J. Property Research 36(1) 59–96) · mccluskey2013prediction (JPR 30(4) 239–265) · schulz2014automated (JPR 31(2)
135 +131–153) · fik2003modeling (REE 31(4) 623–646) · clapp2003semiparametric (JREFE 27(3) 303–320) ·
136 +pace1998spatiotemporal (JREFE 17(1) 15–33) · dube2013pooled (LSRS 6(1) 1–18) · hill2009boom (JHE 18(3) 193–205) ·
137 +manning2001estimating (J. Health Econ. 20(4) 461–494) · thibodeau2003marking (REE 31(1) 1–22) ·
138 +khoshnoud2023evolution (JREL 31(1) 1–47).
139 +Total : 33 → **58** entrées (31 conservées + 27 ajoutées), toutes citées, toutes vérifiées.
140 +
141 +## 3. Réanalyses exécutées (laptop, Apple Silicon 18 cœurs, Python 3.14 / scikit-learn 1.6.1)
142 +
143 +| Script | Contenu | Durée | Sorties |
144 +|---|---|---|---|
145 +| `01_build_sample.py` | reconstruction de l'échantillon (+ clés `parcel`, `coord`) — 514 212 ventes identiques | 2 s | `data/processed/analysis.parquet` |
146 +| `02_horserace.py` | 20 modèles × 2 splits, règles médiane + Duan, prédictions sauvegardées | 3 min 28 s | `horserace_*.csv`, `boxcox.csv`, `predictions_*.parquet` (non commités) |
147 +| `03_extensions.py` | indices, profils, learning curves, segments (règle médiane) | 4 min 18 s | `index.csv`, `profiles.csv`, `learning.csv`, `segments.csv` |
148 +| `06_robustness.py` (nouveau) | bootstrap + DM, même période, par trimestre, fuite, re-nivellement, paires, splits groupés (2 courses complètes), FE emboîtés, Box–Cox fin, sensibilité HGB | 16 min 16 s | `inference_*.csv`, `same_period.csv`, `by_quarter.csv`, `leakage.csv`, `relevel.csv`, `pairs.csv`, `horserace_group_*.csv`, `nested_fe.csv`, `boxcox_fine.csv`, `hgb_sensitivity.csv` |
149 +| `04_make_figures.py` | 10 figures (+ `fig_horizon.png`) | < 1 min | `figures/` |
150 +| `05_make_tables.py` | 16 tables LaTeX (9 nouvelles) | < 1 s | `results/tables/` |
151 +
152 +Tous les nombres du texte proviennent de ces sorties ; aucune table n'est tapée à la main.
153 +
154 +## 4. Éléments non modifiés / analyses recommandées non exécutées
155 +- Modèle linéaire à surface spatiale (krigeage des résidus, splines bivariées en lat/lng, effets hiérarchiques
156 + bayésiens) : non estimé (hors cadre « 20 modèles ») — recommandé pour fermer l'écart linéaire/boosting par l'autre
157 + côté.
158 +- Holdout spatialement bloqué (cellules entières en test) : non exécuté (change la question en extrapolation
159 + spatiale ; les FE spatiaux y sont par construction inopérants).
160 +- Re-nivellement par un indice **publié** (hors échantillon, non oracle) : non exécuté (aucun indice externe
161 + chargé ; le re-nivellement présenté utilise le ratio médian du test, borne supérieure).
162 +- Variante « médiane calibrée » (exp(ẑ + médiane des résidus d'entraînement)) : discutée, non tabulée.
163 +- Fenêtres temporelles alternatives (p. ex. train < 2024, test 2024) : non exécutées — limite déclarée §8.4.
164 +- Comparaison avec LightGBM / XGBoost natifs ou réseaux de neurones : hors périmètre.
165 +- La règle de nettoyage des lots > P99.5 → 0 (héritée) est documentée mais conservée pour préserver l'échantillon.
added docs/REVIEW-REPORT.md +118 −0
@@ -0,0 +1,118 @@
1 +# Rapport de l'editorial board — WP 11 « Half a Million Prices, Twenty Models » (2026-10-17)
2 +
3 +Révision v1.0 (August 2026, 23 p.) → v1.1 (October 2026). Journal d'audit détaillé : `docs/CHANGELOG-REVISION.md`.
4 +Comité : économie immobilière, économétrie appliquée, économétrie spatiale, évaluation foncière, apprentissage
5 +automatique, édition scientifique. Revue cible : *Journal of Housing Economics* (repli JRER / IJF).
6 +
7 +## 1. Diagnostic général
8 +
9 +Le papier pose une question nette et utile — quels choix de spécification hédonique comptent réellement pour la
10 +précision hors échantillon — et y répond par un dispositif expérimental propre (un jeu d'attributs, quatre axes, un
11 +seul scoreboard, deux holdouts) sur un échantillon rare par sa taille et la qualité des descripteurs (rôle
12 +d'évaluation). La contribution est réelle et conservée. Trois faiblesses structurelles de la v1.0 ont été
13 +identifiées et corrigées :
14 +
15 +1. **Le résultat phare était un artefact.** Les modèles linéaires en log étaient retransformés par smearing de Duan
16 + et les ensembles d'arbres non. Dans le split temporel, le smearing (facteur ≈ 1.06) compense une partie de la
17 + staleness du niveau dans un marché haussier ; c'est ce qui produisait « A6 19.1 % vs D4 21.2 % » et le
18 + « renversement » affiché en abstract. Sous une règle uniforme, le boosting reste devant (21.2 vs 21.7, écart non
19 + significatif). Correction intégrale (code, tables, texte) ; la v1.1 le dit explicitement.
20 +2. **Aucune inférence.** Des différences de 0.2 à 0.6 point étaient interprétées comme des résultats. Ajout d'un
21 + bootstrap par grappes (municipalités) et de tests Diebold–Mariano/Giacomini–White groupés sur toutes les
22 + comparaisons. Plusieurs affirmations tombent (grille 5.5 km > municipalités ; mois ≈ trimestre) ou se précisent.
23 +3. **Mécanisme du « generalization gap » non identifié.** La v1.0 confondait difficulté de la période test et
24 + staleness du modèle. Décomposition ajoutée (même période, biais de niveau, re-nivellement, horizon par
25 + trimestre) : la compétence relative des ensembles transfère intacte (écart re-nivelé 2.9 pts ≈ 3.5 en random),
26 + c'est le niveau qui ne transfère pas, un peu plus pour les arbres (feuille terminale en t). Ce résultat est plus
27 + faible que le « renversement » mais plus utile et défendable.
28 +
29 +## 2. Diagnostic par section
30 +
31 +| Section | Forces | Faiblesses / erreurs trouvées | Action |
32 +|---|---|---|---|
33 +| Abstract | clair, chiffré | chiffres issus du scoreboard asymétrique ; « reverses », « identical R² » faux ; trop long pour la page 2 (débordait sur une page blanche) | réécrit, 4 leçons reformulées, mention de la correction |
34 +| 1 Introduction | bonne mise en situation Rosen → ML | « 13 % improvement » (vrai : 18 %) ; « 27.9 vs 17.5 » (comparait C1 à C3 au lieu du backbone) ; aucune mention de l'inférence ni des fuites | réécrit ; renvoi à la nouvelle §7 |
35 +| 2 Literature | couvre les trois courants | mince (33 réf., 29 citées) ; manquaient Hill–Scholz, Pace–Hayunga, Lorenz et al., Diewert–Shimizu, Kuminoff et al., DM/GW, Bergmeir–Benítez, Roberts et al., Dubé–Legros ; deux entrées non citées et défectueuses | +27 réf. vérifiées, −2 retirées ; Mayer et al. 2019 identifié comme antécédent direct (updating) |
36 +| 3 Data | cascade d'échantillon transparente | moyenne/médiane des prix fausses dans le texte (462k/399k vs 440k/397k) ; 86 ventes « other » omises ; nettoyage des lots > P99.5 non documenté ; bornes âge/étages non dites | corrigé ; répartition annuelle ; repeat sales et coordonnées partagées quantifiés |
37 +| 4 Methodology | design clair | aucune équation ; Duan non écrit ; inversion Box–Cox sans discussion du biais ; « clamped month counter » inexact (rien n'est écrêté ; arbres constants par morceaux) ; catégories non vues → référence arbitraire non dit ; hyper-paramètres « a priori » sans analyse | 13 équations ; règles médiane/moyenne ; carry-forward ; holdouts groupés ; inférence ; décomposition niveau/skill |
38 +| 5 Results | lecture par axe efficace | « seventeen K sales » (17) ; ordre A sous split temporel non discuté (A1 gagne par coïncidence de biais) ; différences non testées | réécrit avec IC/tests ; §5.5 refondue (décomposition) |
39 +| 6 Extensions | objets économiques pertinents | « all five agree within a few points » (indice GB −9 pts) ; « within a tenth of a log point » (0.10–0.13 systématique, profil plus plat) ; learning/segments chiffrés sous scoreboard asymétrique | affirmations ramenées aux chiffres ; mécanismes de l'écart d'indice discutés |
40 +| 7 Robustness | — (n'existait pas) | — | nouvelle section : retransformation, fuites, FE emboîtés, Box–Cox fin, hyper-paramètres HGB |
41 +| 8 Discussion | hiérarchie utile | liste à puces ; « the machines lose their edge » (faux) ; « 3–7 points » sans mécanisme | prose ; hiérarchie révisée ; mécanisme niveau vs skill ; implications réécrites |
42 +| 9 Conclusion | synthèse pratique | reprenait le renversement ; « within two points of the frontier in deployment » | réécrite |
43 +
44 +## 3. Méthodologie — verdict motivé (§3 du master prompt)
45 +
46 +- **Spécification justifiée** : oui, le design par axes est la force du papier ; la v1.1 le formalise (éq. 1–9).
47 +- **Hypothèses d'estimation explicites** : désormais oui (OLS via ridge α = 10⁻⁶, solution de norme minimale ;
48 + encodage one-hot avec référence ; splines : 6 nœuds uniformes, 8 fonctions de base ; RF/HGB : paramètres listés).
49 +- **Variables définies** : oui (notation unifiée §4.1 ; bornes et nettoyages §3.2).
50 +- **Effets fixes appropriés** : oui, avec la nuance majeure trouvée en révision : les FE fins doivent être emboîtés
51 + (C4′ 17.3 % = meilleure spécification linéaire ; C4 seule 21.0 %). Le repli « cellule de référence » pour les
52 + cellules non vues est désormais documenté et corrigé par l'emboîtement.
53 +- **Dépendance spatiale/temporelle** : traitée par l'inférence groupée (municipalités) et par les holdouts
54 + temporel, par parcelle et par bâtiment. Limite : pas de holdout bloqué spatialement (déclaré).
55 +- **Erreurs-types** : absentes en v1.0 → bootstrap par grappes + DM groupé (éq. 13). Le DM sur la perte APE teste la
56 + MAPE, pas la MdAPE ; le bootstrap couvre la médiane — distinction explicitée.
57 +- **Construction de l'échantillon transparente** : oui (cascade 745 119 → 514 212, reproduite à l'identique).
58 +- **Robustesse** : cinq analyses ajoutées (§7) ; toutes exécutées, aucune revendiquée sans sortie.
59 +- **Interprétation** : « 18 % improvement » = (17.6−14.4)/17.6 ; « ten points » = 28.8 − 19.2 ; profils en log-points.
60 +- **Significativité statistique vs économique** : distinguée systématiquement (ex. C3 vs C2, D2 vs A6, D4 vs A6
61 + forward : non significatifs ; D4 vs A6 random : −3.2 [−4.7, −1.9]).
62 +- **Langage causal** : aucun ; le papier est prédictif/descriptif.
63 +
64 +## 4. Résultats — incohérences trouvées et état des réanalyses
65 +
66 +- Toutes les incohérences texte/table de la v1.0 sont listées au CHANGELOG §1.1, 1.5, 1.8, 1.9, 1.11.
67 +- Nouveaux résultats (tous issus de `results/reproduced/*.csv`) :
68 + - Scoreboard uniforme (Tables 2–3) ; IC et tests (Tables 4–5) ; décomposition (Tables 8–9, Fig. 6) ;
69 + retransformation (Table 12) ; splits groupés (Table 13) ; vu/non vu (Table 14) ; FE emboîtés (Table 15) ;
70 + sensibilité HGB (Table 16) ; Box–Cox fin (texte §7.4).
71 + - Sensibilité HGB : quatre configurations alternatives (shallow, deep, régularisée, early stopping sur les 10 % derniers mois) donnent 14.3–14.7 % (random) et 21.2–21.6 % (forward) contre 14.4 / 21.2 pour le baseline ; le classement et la décomposition ne dépendent pas des hyper-paramètres fixés a priori.
72 +- Vérifications de cohérence finales : abstract ↔ §1 ↔ §5 ↔ §8–9 ↔ README ↔ tables (chiffres relus un à un après
73 + régénération des tables ; voir CHANGELOG §1.1 pour la liste ancien → nouveau).
74 +
75 +## 5. Littérature
76 +
77 +33 entrées en v1.0 → 58 en v1.1 ; 100 % vérifiées sur Crossref avec DOI (sauf LightGBM, actes NIPS non indexés,
78 +référence standard). 2 entrées retirées (non citées, défectueuses), 5 corrigées, 27 ajoutées et intégrées à
79 +l'argument (fonctionnelle : Kuminoff et al. 2010, Bishop et al. 2020, Khoshnoud et al. 2023 ; spatial : Pace et al.
80 +1998, Clapp 2003, Fik et al. 2003, Bourassa et al. 2003/2007, Goodman–Thibodeau 2003, Pace–Hayunga 2020,
81 +Dubé–Legros 2013 ; indices : Hill et al. 2009, Hill–Scholz 2018, Diewert et al. 2015, Diewert–Shimizu 2016 ;
82 +ML/AVM : Zurada et al. 2011, Antipov–Pokryshevskaya 2012, McCluskey et al. 2013, Schulz et al. 2014, Pérez-Rave et al.
83 +2019, Lorenz et al. 2023, Thibodeau 2003 ; validation : Diebold–Mariano 1995, Giacomini–White 2006,
84 +Bergmeir–Benítez 2012, Roberts et al. 2017 ; retransformation : Manning–Mullahy 2001).
85 +
86 +## 6. Limites restantes
87 +
88 +1. Un seul régime temporel testé (reprise 2025-26, +15 % en 18 mois) ; l'ordre des coûts de staleness entre
89 + familles pourrait différer dans un marché plat ou baissier.
90 +2. Le re-nivellement utilise le ratio médian du test (oracle) : borne supérieure de ce qu'un indice publié
91 + permettrait.
92 +3. Pas de modèle linéaire à surface spatiale continue (krigeage, splines bivariées) ni d'effets hiérarchiques
93 + proprement estimés ; l'emboîtement C4′ est un pooling partiel implicite (norme minimale), pas un modèle
94 + hiérarchique explicite.
95 +4. Les holdouts groupés neutralisent reventes et bâtiments partagés, pas le voisinage immédiat.
96 +5. Attributs du rôle seulement (pas de qualité intérieure, texte d'annonce, photos) ; l'avantage ML mesuré est celui
97 + de la flexibilité spatiale sur cet ensemble d'attributs.
98 +6. La règle « lots > P99.5 → 0 » héritée du pipeline est documentée mais discutable.
99 +7. Longueur : 40 pages avec 16 tables et 10 figures ; pour la soumission, les Tables 4–5, 12–16 iraient en annexe.
100 +
101 +## 7. Analyses recommandées non exécutées
102 +
103 +Voir CHANGELOG §4 : surface spatiale linéaire (krigeage / splines lat-lng / hiérarchique bayésien) ; holdout bloqué
104 +spatialement ; re-nivellement par indice externe ; fenêtres temporelles alternatives (rolling origin) ; variante
105 +« médiane calibrée » ; LightGBM/XGBoost natifs et réseaux de neurones ; coût de retraining mensuel (course
106 +glissante) — cette dernière est la plus importante pour la revue cible.
107 +
108 +## 8. Maturité pour publication
109 +
110 +- **Verdict** : manuscrit désormais défendable en revue (JHE) sous réserve d'une coupe (annexes) et, idéalement, d'une
111 + course glissante (retraining mensuel) qui transformerait la §5.5 en recommandation opérationnelle chiffrée.
112 +- **Risques de rejet** : (i) « un seul marché, une seule période » — mitigé par l'échelle provinciale et la
113 + décomposition ; (ii) absence d'un modèle linéaire spatialement sophistiqué comme concurrent (un rapporteur
114 + spatial le demandera) ; (iii) la correction d'un résultat phare publié en v1.0 sur SSRN doit être assumée
115 + explicitement (fait dans l'abstract et §7.1) — c'est un point d'intégrité, pas une faiblesse.
116 +- **Points forts pour la revue** : taille et qualité des données, design expérimental, inférence complète,
117 + décomposition niveau/compétence inédite dans la littérature AVM, résultat « hierarchy, not coarseness » sur les FE
118 + spatiaux.
modified figures/fig_forms.png +0 −0

Binary file not shown.

modified figures/fig_generalization.png +0 −0

Binary file not shown.

added figures/fig_horizon.png +0 −0

Binary file not shown.

modified figures/fig_learning.png +0 −0

Binary file not shown.

modified figures/fig_methods.png +0 −0

Binary file not shown.

modified figures/fig_segments.png +0 −0

Binary file not shown.

modified figures/fig_space.png +0 −0

Binary file not shown.

modified figures/fig_time.png +0 −0

Binary file not shown.

modified paper/main.pdf +0 −0

Binary file not shown.

modified paper/main.tex +41 −30
@@ -48,6 +48,11 @@
48 48 \usepackage{enumitem}
49 49 \setlist{nosep,leftmargin=*}
50 50
51 +% Float placement: let text and floats share pages
52 +\setcounter{topnumber}{3}\setcounter{bottomnumber}{2}\setcounter{totalnumber}{4}
53 +\renewcommand{\topfraction}{0.9}\renewcommand{\bottomfraction}{0.6}
54 +\renewcommand{\textfraction}{0.08}\renewcommand{\floatpagefraction}{0.5}
55 +
51 56 % ============================================================================
52 57 % METADATA
53 58 % ============================================================================
@@ -55,9 +60,9 @@
55 60 \newcommand{\WPtitle}{Half a Million Prices, Twenty Models}
56 61 \newcommand{\WPsubtitle}{A Systematic Assessment of Hedonic Specifications and
57 62 Estimation Methods for the Quebec Housing Market, 2021--2026}
58 −\newcommand{\WPdate}{August 2026}
59 −\newcommand{\WPversion}{1.0}
60 −\newcommand{\WPkeywords}{hedonic pricing, functional form, house price indexes, machine learning, automated valuation, out-of-sample prediction}
63 +\newcommand{\WPdate}{October 2026}
64 +\newcommand{\WPversion}{1.1}
65 +\newcommand{\WPkeywords}{hedonic price function, functional form, spatial fixed effects, machine learning, automated valuation model, forward-in-time holdout, house price index}
61 66 \newcommand{\WPjel}{R31, C21, C52, C53}
62 67
63 68 \newcommand{\WPauthor}{Simon-Pierre Boucher}
@@ -68,34 +73,39 @@
68 73 283, boulevard Alexandre-Tach\'e\\ Gatineau, Qu\'ebec, Canada J9A 1L8}
69 74
70 75 \newcommand{\WPabstract}{%
71 −Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still rests
72 −on a series of specification choices --- functional form, temporal controls,
73 −spatial controls, estimation method --- that are usually made by convention
74 −and rarely stress-tested jointly. Using 514{,}212 Quebec residential sales
76 +Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still
77 +rests on specification choices---functional form, temporal controls,
78 +spatial controls, estimation method---that are made by convention and
79 +rarely stress-tested jointly. Using 514{,}212 Quebec residential sales
75 80 (2021--2026) whose structural descriptors come from the provincial
76 81 assessment roll, we run a controlled horse race of twenty hedonic models
77 −organised along those four axes, scoring every model on identical holdouts
78 −and on the same price-level scoreboard (log models are Duan-retransformed,
79 −Box--Cox inverted). Four lessons emerge. (1)~Functional form is a
80 −second-order choice: from linear to splines, the median absolute error
81 −spans 16.5--18.6\%, and the data-driven Box--Cox exponent
82 −($\hat\lambda=0.25$) buys nothing out of sample. (2)~Controls are
83 −first-order: dropping spatial fixed effects costs ten error points ---
84 −location granularity matters far more than curvature, but only up to the
85 −point where cells thin out (a $\sim$1~km grid \emph{overfits}).
86 −(3)~Under random validation, estimation method matters most: gradient
87 −boosting reaches 14.4\% --- a 13\% improvement over the best linear model
88 −--- with random forests close behind, and coordinates alone reproduce
89 −what thousands of dummies buy. (4)~But the ranking is an artifact of the
90 −split: under a forward-in-time holdout (train $<$~2025, test 2025--26)
91 −every model degrades and the machine-learning advantage \emph{reverses}
92 −--- the spline hedonic posts a lower median error than gradient boosting
93 −and an identical log-scale $R^2$ --- a stark warning against evaluating
94 −valuation models with random cross-validation alone. Constant-quality
95 −monthly indices implied by the competing forms agree to within a few
96 −index points, and the boosted model's implicit age and floor-area
97 −profiles track the quadratic OLS closely --- interpretation and
98 −prediction disagree less than the random-validation gap suggests.}
82 +organised along those four axes, scoring every model on identical
83 +holdouts and on the same price-level scoreboard with a single
84 +retransformation rule, with a municipality-clustered bootstrap interval
85 +and a Diebold--Mariano-type test on every comparison. Four lessons
86 +emerge. (1)~Functional form is second-order: from linear to splines the
87 +median absolute error spans 17.6--19.2\%, and the data-driven Box--Cox
88 +exponent gains half a point. (2)~Controls are first-order: dropping
89 +spatial fixed effects costs ten error points; a $\sim$1~km grid used
90 +alone overfits, but nested within municipalities it becomes the most
91 +accurate linear specification---the bias--variance trade-off in
92 +absorbing location is resolved by hierarchy, not coarseness. (3)~Under
93 +random validation the estimation method dominates: gradient boosting
94 +reaches 14.4\%, an 18\% improvement over the best linear model
95 +(95\% interval 1.9--4.7 points) that survives holdouts grouped by parcel
96 +and by building and comes almost entirely from a flexible surface over
97 +space. (4)~Under a forward-in-time holdout (train $<$~2025, test
98 +2025--26) every model degrades and the gap shrinks to half a point and
99 +to insignificance. Decomposing the error shows that relative pricing
100 +skill transfers forward intact (re-levelled, boosting leads by 2.9
101 +points) while every frozen model under-prices the typical future sale
102 +by 10--20\%, the ensembles slightly more. Random cross-validation hides
103 +this staleness cost; it does not, as a previous version of this paper
104 +claimed on the basis of an asymmetric retransformation, reverse the
105 +ranking. Constant-quality indices agree across functional forms to
106 +within 2.5 points but a boosted imputation index rises nine points less,
107 +and the boosted model's implicit age and area profiles share the shape
108 +of the quadratic OLS gradients with a milder depreciation path.}
99 109
100 110 \hypersetup{pdftitle={Half a Million Prices, Twenty Models}, pdfauthor={Simon-Pierre Boucher}}
101 111
@@ -113,10 +123,11 @@ prediction disagree less than the random-validation gap suggests.}
113 123 \input{sections/methodology}
114 124 \input{sections/results}
115 125 \input{sections/extensions}
126 +\input{sections/robustness}
116 127 \input{sections/discussion}
117 128 \input{sections/conclusion}
118 129
119 −\newpage
130 +\clearpage
120 131 \bibliography{references}
121 132
122 133 \end{document}
modified paper/main_anon.pdf +0 −0

Binary file not shown.

modified paper/main_anon.tex +41 −30
@@ -48,6 +48,11 @@
48 48 \usepackage{enumitem}
49 49 \setlist{nosep,leftmargin=*}
50 50
51 +% Float placement: let text and floats share pages
52 +\setcounter{topnumber}{3}\setcounter{bottomnumber}{2}\setcounter{totalnumber}{4}
53 +\renewcommand{\topfraction}{0.9}\renewcommand{\bottomfraction}{0.6}
54 +\renewcommand{\textfraction}{0.08}\renewcommand{\floatpagefraction}{0.5}
55 +
51 56 % ============================================================================
52 57 % METADATA
53 58 % ============================================================================
@@ -55,9 +60,9 @@
55 60 \newcommand{\WPtitle}{Half a Million Prices, Twenty Models}
56 61 \newcommand{\WPsubtitle}{A Systematic Assessment of Hedonic Specifications and
57 62 Estimation Methods for the Quebec Housing Market, 2021--2026}
58 −\newcommand{\WPdate}{August 2026}
59 −\newcommand{\WPversion}{1.0}
60 −\newcommand{\WPkeywords}{hedonic pricing, functional form, house price indexes, machine learning, automated valuation, out-of-sample prediction}
63 +\newcommand{\WPdate}{October 2026}
64 +\newcommand{\WPversion}{1.1}
65 +\newcommand{\WPkeywords}{hedonic price function, functional form, spatial fixed effects, machine learning, automated valuation model, forward-in-time holdout, house price index}
61 66 \newcommand{\WPjel}{R31, C21, C52, C53}
62 67
63 68 \newcommand{\WPauthor}{Simon-Pierre Boucher}
@@ -68,34 +73,39 @@
68 73 283, boulevard Alexandre-Tach\'e\\ Gatineau, Qu\'ebec, Canada J9A 1L8}
69 74
70 75 \newcommand{\WPabstract}{%
71 −Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still rests
72 −on a series of specification choices --- functional form, temporal controls,
73 −spatial controls, estimation method --- that are usually made by convention
74 −and rarely stress-tested jointly. Using 514{,}212 Quebec residential sales
76 +Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still
77 +rests on specification choices---functional form, temporal controls,
78 +spatial controls, estimation method---that are made by convention and
79 +rarely stress-tested jointly. Using 514{,}212 Quebec residential sales
75 80 (2021--2026) whose structural descriptors come from the provincial
76 81 assessment roll, we run a controlled horse race of twenty hedonic models
77 −organised along those four axes, scoring every model on identical holdouts
78 −and on the same price-level scoreboard (log models are Duan-retransformed,
79 −Box--Cox inverted). Four lessons emerge. (1)~Functional form is a
80 −second-order choice: from linear to splines, the median absolute error
81 −spans 16.5--18.6\%, and the data-driven Box--Cox exponent
82 −($\hat\lambda=0.25$) buys nothing out of sample. (2)~Controls are
83 −first-order: dropping spatial fixed effects costs ten error points ---
84 −location granularity matters far more than curvature, but only up to the
85 −point where cells thin out (a $\sim$1~km grid \emph{overfits}).
86 −(3)~Under random validation, estimation method matters most: gradient
87 −boosting reaches 14.4\% --- a 13\% improvement over the best linear model
88 −--- with random forests close behind, and coordinates alone reproduce
89 −what thousands of dummies buy. (4)~But the ranking is an artifact of the
90 −split: under a forward-in-time holdout (train $<$~2025, test 2025--26)
91 −every model degrades and the machine-learning advantage \emph{reverses}
92 −--- the spline hedonic posts a lower median error than gradient boosting
93 −and an identical log-scale $R^2$ --- a stark warning against evaluating
94 −valuation models with random cross-validation alone. Constant-quality
95 −monthly indices implied by the competing forms agree to within a few
96 −index points, and the boosted model's implicit age and floor-area
97 −profiles track the quadratic OLS closely --- interpretation and
98 −prediction disagree less than the random-validation gap suggests.}
82 +organised along those four axes, scoring every model on identical
83 +holdouts and on the same price-level scoreboard with a single
84 +retransformation rule, with a municipality-clustered bootstrap interval
85 +and a Diebold--Mariano-type test on every comparison. Four lessons
86 +emerge. (1)~Functional form is second-order: from linear to splines the
87 +median absolute error spans 17.6--19.2\%, and the data-driven Box--Cox
88 +exponent gains half a point. (2)~Controls are first-order: dropping
89 +spatial fixed effects costs ten error points; a $\sim$1~km grid used
90 +alone overfits, but nested within municipalities it becomes the most
91 +accurate linear specification---the bias--variance trade-off in
92 +absorbing location is resolved by hierarchy, not coarseness. (3)~Under
93 +random validation the estimation method dominates: gradient boosting
94 +reaches 14.4\%, an 18\% improvement over the best linear model
95 +(95\% interval 1.9--4.7 points) that survives holdouts grouped by parcel
96 +and by building and comes almost entirely from a flexible surface over
97 +space. (4)~Under a forward-in-time holdout (train $<$~2025, test
98 +2025--26) every model degrades and the gap shrinks to half a point and
99 +to insignificance. Decomposing the error shows that relative pricing
100 +skill transfers forward intact (re-levelled, boosting leads by 2.9
101 +points) while every frozen model under-prices the typical future sale
102 +by 10--20\%, the ensembles slightly more. Random cross-validation hides
103 +this staleness cost; it does not, as a previous version of this paper
104 +claimed on the basis of an asymmetric retransformation, reverse the
105 +ranking. Constant-quality indices agree across functional forms to
106 +within 2.5 points but a boosted imputation index rises nine points less,
107 +and the boosted model's implicit age and area profiles share the shape
108 +of the quadratic OLS gradients with a milder depreciation path.}
99 109
100 110 \hypersetup{pdftitle={Half a Million Prices, Twenty Models}, pdfauthor={}}
101 111
@@ -113,10 +123,11 @@ prediction disagree less than the random-validation gap suggests.}
113 123 \input{sections/methodology}
114 124 \input{sections/results}
115 125 \input{sections/extensions}
126 +\input{sections/robustness}
116 127 \input{sections/discussion}
117 128 \input{sections/conclusion}
118 129
119 −\newpage
130 +\clearpage
120 131 \bibliography{references}
121 132
122 133 \end{document}
modified paper/references.bib +371 −60
@@ -7,7 +7,8 @@
7 7 year = {1974},
8 8 volume = {82},
9 9 number = {1},
10 − pages = {34--55}
10 + pages = {34--55},
11 + doi = {10.1086/260169}
11 12 }
12 13
13 14 @article{halvorsen1981choice,
@@ -17,7 +18,8 @@
17 18 year = {1981},
18 19 volume = {10},
19 20 number = {1},
20 − pages = {37--49}
21 + pages = {37--49},
22 + doi = {10.1016/0094-1190(81)90021-8}
21 23 }
22 24
23 25 @article{cassel1985cost,
@@ -27,7 +29,8 @@
27 29 year = {1985},
28 30 volume = {18},
29 31 number = {2},
30 − pages = {135--142}
32 + pages = {135--142},
33 + doi = {10.1016/0094-1190(85)90012-9}
31 34 }
32 35
33 36 @article{cropper1988choice,
@@ -37,7 +40,8 @@
37 40 year = {1988},
38 41 volume = {70},
39 42 number = {4},
40 − pages = {668--675}
43 + pages = {668--675},
44 + doi = {10.2307/1935831}
41 45 }
42 46
43 47 @article{box1964analysis,
@@ -47,7 +51,8 @@
47 51 year = {1964},
48 52 volume = {26},
49 53 number = {2},
50 − pages = {211--252}
54 + pages = {211--243},
55 + doi = {10.1111/j.2517-6161.1964.tb00553.x}
51 56 }
52 57
53 58 @incollection{malpezzi2003hedonic,
@@ -58,7 +63,8 @@
58 63 publisher = {Blackwell},
59 64 address = {Oxford},
60 65 year = {2003},
61 − pages = {67--89}
66 + pages = {67--89},
67 + doi = {10.1002/9780470690680.ch5}
62 68 }
63 69
64 70 @article{sirmans2005composition,
@@ -68,7 +74,8 @@
68 74 year = {2005},
69 75 volume = {13},
70 76 number = {1},
71 − pages = {1--44}
77 + pages = {1--44},
78 + doi = {10.1080/10835547.2005.12090154}
72 79 }
73 80
74 81 @article{duan1983smearing,
@@ -78,7 +85,8 @@
78 85 year = {1983},
79 86 volume = {78},
80 87 number = {383},
81 − pages = {605--610}
88 + pages = {605--610},
89 + doi = {10.1080/01621459.1983.10478017}
82 90 }
83 91
84 92 @article{goodman1998housing,
@@ -88,7 +96,8 @@
88 96 year = {1998},
89 97 volume = {7},
90 98 number = {2},
91 − pages = {121--143}
99 + pages = {121--143},
100 + doi = {10.1006/jhec.1998.0229}
92 101 }
93 102
94 103 @article{dubin1998spatial,
@@ -98,7 +107,8 @@
98 107 year = {1998},
99 108 volume = {7},
100 109 number = {4},
101 − pages = {304--327}
110 + pages = {304--327},
111 + doi = {10.1006/jhec.1998.0236}
102 112 }
103 113
104 114 @book{anselin1988spatial,
@@ -106,7 +116,8 @@
106 116 title = {Spatial Econometrics: Methods and Models},
107 117 publisher = {Kluwer Academic},
108 118 address = {Dordrecht},
109 − year = {1988}
119 + year = {1988},
120 + doi = {10.1007/978-94-015-7799-1}
110 121 }
111 122
112 123 @article{hill2013hedonic,
@@ -116,25 +127,19 @@
116 127 year = {2013},
117 128 volume = {27},
118 129 number = {5},
119 − pages = {879--914}
130 + pages = {879--914},
131 + doi = {10.1111/j.1467-6419.2012.00731.x}
120 132 }
121 133
122 −@article{silver2018house,
123 − author = {Silver, Mick},
124 − title = {How to Measure Hedonic Property Price Indexes Better},
125 − journal = {EURONA — Eurostat Review on National Accounts and Macroeconomic Indicators},
126 − year = {2018},
127 − volume = {1},
128 − pages = {35--66}
129 −}
130 −
131 −@article{diewert2004hedonic,
132 − author = {Diewert, W. Erwin},
133 − title = {Hedonic Regressions: A Review of Some Unresolved Issues},
134 − journal = {Mimeo, University of British Columbia},
135 − year = {2004},
136 − volume = {},
137 − pages = {}
134 +@techreport{silver2018house,
135 + author = {Silver, Mick},
136 + title = {How to Better Measure Hedonic Residential Property Price Indexes},
137 + institution = {International Monetary Fund},
138 + type = {IMF Working Paper},
139 + number = {16/213},
140 + address = {Washington, DC},
141 + year = {2016},
142 + doi = {10.5089/9781475552249.001}
138 143 }
139 144
140 145 @article{mullainathan2017machine,
@@ -144,7 +149,8 @@
144 149 year = {2017},
145 150 volume = {31},
146 151 number = {2},
147 − pages = {87--106}
152 + pages = {87--106},
153 + doi = {10.1257/jep.31.2.87}
148 154 }
149 155
150 156 @article{athey2019machine,
@@ -153,7 +159,8 @@
153 159 journal = {Annual Review of Economics},
154 160 year = {2019},
155 161 volume = {11},
156 − pages = {685--725}
162 + pages = {685--725},
163 + doi = {10.1146/annurev-economics-080217-053433}
157 164 }
158 165
159 166 @article{breiman2001random,
@@ -163,7 +170,8 @@
163 170 year = {2001},
164 171 volume = {45},
165 172 number = {1},
166 − pages = {5--32}
173 + pages = {5--32},
174 + doi = {10.1023/A:1010933404324}
167 175 }
168 176
169 177 @article{friedman2001greedy,
@@ -173,16 +181,18 @@
173 181 year = {2001},
174 182 volume = {29},
175 183 number = {5},
176 − pages = {1189--1232}
184 + pages = {1189--1232},
185 + doi = {10.1214/aos/1013203451}
177 186 }
178 187
179 −@article{ke2017lightgbm,
180 − author = {Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan},
181 − title = {{LightGBM}: A Highly Efficient Gradient Boosting Decision Tree},
182 − journal = {Advances in Neural Information Processing Systems},
183 − year = {2017},
184 − volume = {30},
185 − pages = {3146--3154}
188 +@inproceedings{ke2017lightgbm,
189 + author = {Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan},
190 + title = {{LightGBM}: A Highly Efficient Gradient Boosting Decision Tree},
191 + booktitle = {Advances in Neural Information Processing Systems 30 (NIPS 2017)},
192 + editor = {Guyon, I. and Von Luxburg, U. and Bengio, S. and Wallach, H. and Fergus, R. and Vishwanathan, S. and Garnett, R.},
193 + publisher = {Curran Associates},
194 + year = {2017},
195 + pages = {3146--3154}
186 196 }
187 197
188 198 @article{mayer2019estimation,
@@ -192,7 +202,8 @@
192 202 year = {2019},
193 203 volume = {12},
194 204 number = {1},
195 − pages = {134--150}
205 + pages = {134--150},
206 + doi = {10.1108/JERER-08-2018-0035}
196 207 }
197 208
198 209 @article{bogin2020house,
@@ -201,8 +212,9 @@
201 212 journal = {Journal of Real Estate Finance and Economics},
202 213 year = {2020},
203 214 volume = {60},
204 − number = {1},
205 − pages = {40--52}
215 + number = {1--2},
216 + pages = {40--52},
217 + doi = {10.1007/s11146-019-09712-0}
206 218 }
207 219
208 220 @article{steurer2021metrics,
@@ -212,7 +224,8 @@
212 224 year = {2021},
213 225 volume = {38},
214 226 number = {2},
215 − pages = {99--129}
227 + pages = {99--129},
228 + doi = {10.1080/09599916.2020.1858937}
216 229 }
217 230
218 231 @article{pace1997spatial,
@@ -222,7 +235,8 @@
222 235 year = {1997},
223 236 volume = {14},
224 237 number = {3},
225 − pages = {333--340}
238 + pages = {333--340},
239 + doi = {10.1023/A:1007762613901}
226 240 }
227 241
228 242 @article{case2004modeling,
@@ -232,7 +246,8 @@
232 246 year = {2004},
233 247 volume = {29},
234 248 number = {2},
235 − pages = {167--191}
249 + pages = {167--191},
250 + doi = {10.1023/B:REAL.0000035309.60607.53}
236 251 }
237 252
238 253 @article{bourassa2010predicting,
@@ -242,7 +257,8 @@
242 257 year = {2010},
243 258 volume = {32},
244 259 number = {2},
245 − pages = {139--159}
260 + pages = {139--159},
261 + doi = {10.1080/10835547.2010.12091276}
246 262 }
247 263
248 264 @article{desrosiers2000hedonic,
@@ -252,7 +268,8 @@
252 268 year = {2000},
253 269 volume = {18},
254 270 number = {3},
255 − pages = {291--315}
271 + pages = {291--315},
272 + doi = {10.1108/14635780010338245}
256 273 }
257 274
258 275 @article{gencay1996nonlinear,
@@ -262,7 +279,8 @@
262 279 year = {1996},
263 280 volume = {52},
264 281 number = {2},
265 − pages = {129--135}
282 + pages = {129--135},
283 + doi = {10.1016/S0165-1765(96)00851-8}
266 284 }
267 285
268 286 @article{kok2017big,
@@ -272,7 +290,8 @@
272 290 year = {2017},
273 291 volume = {43},
274 292 number = {6},
275 − pages = {202--211}
293 + pages = {202--211},
294 + doi = {10.3905/jpm.2017.43.6.202}
276 295 }
277 296
278 297 @article{hong2020machine,
@@ -282,16 +301,8 @@
282 301 year = {2020},
283 302 volume = {24},
284 303 number = {3},
285 − pages = {140--152}
286 −}
287 −
288 −@article{knight2003listing,
289 − author = {Knight, John R.},
290 − title = {Hedonic Modeling of the Home Selling Process},
291 − journal = {Journal of Real Estate Finance and Economics},
292 − year = {2003},
293 − volume = {},
294 − pages = {}
304 + pages = {140--152},
305 + doi = {10.3846/ijspm.2020.11544}
295 306 }
296 307
297 308 @book{iaao2013standard,
@@ -309,7 +320,8 @@
309 320 year = {2002},
310 321 volume = {24},
311 322 number = {1},
312 − pages = {1--26}
323 + pages = {1--26},
324 + doi = {10.1080/10835547.2002.12091087}
313 325 }
314 326
315 327 @techreport{oecd2013handbook,
@@ -319,3 +331,302 @@
319 331 address = {Luxembourg},
320 332 year = {2013}
321 333 }
334 +
335 +% ---------------------------------------------------------------------------
336 +% References added in the 2026-10-17 revision (all verified on Crossref)
337 +% ---------------------------------------------------------------------------
338 +
339 +@article{hill2018geospatial,
340 + author = {Hill, Robert J. and Scholz, Michael},
341 + title = {Can Geospatial Data Improve House Price Indexes? {A} Hedonic Imputation Approach with Splines},
342 + journal = {Review of Income and Wealth},
343 + year = {2018},
344 + volume = {64},
345 + number = {4},
346 + pages = {737--756},
347 + doi = {10.1111/roiw.12303}
348 +}
349 +
350 +@article{pace2020residuals,
351 + author = {Pace, R. Kelley and Hayunga, Darren},
352 + title = {Examining the Information Content of Residuals from Hedonic and Spatial Models Using Trees and Forests},
353 + journal = {Journal of Real Estate Finance and Economics},
354 + year = {2020},
355 + volume = {60},
356 + number = {1--2},
357 + pages = {170--180},
358 + doi = {10.1007/s11146-019-09724-w}
359 +}
360 +
361 +@article{lorenz2023interpretable,
362 + author = {Lorenz, Felix and Willwersch, Jonas and Cajias, Marcelo and Fuerst, Franz},
363 + title = {Interpretable Machine Learning for Real Estate Market Analysis},
364 + journal = {Real Estate Economics},
365 + year = {2023},
366 + volume = {51},
367 + number = {5},
368 + pages = {1178--1208},
369 + doi = {10.1111/1540-6229.12397}
370 +}
371 +
372 +@article{diewert2016tokyo,
373 + author = {Diewert, W. Erwin and Shimizu, Chihiro},
374 + title = {Hedonic Regression Models for {Tokyo} Condominium Sales},
375 + journal = {Regional Science and Urban Economics},
376 + year = {2016},
377 + volume = {60},
378 + pages = {300--315},
379 + doi = {10.1016/j.regsciurbeco.2016.08.002}
380 +}
381 +
382 +@article{diewert2015decomposition,
383 + author = {Diewert, W. Erwin and de Haan, Jan and Hendriks, Rens},
384 + title = {Hedonic Regressions and the Decomposition of a House Price Index into Land and Structure Components},
385 + journal = {Econometric Reviews},
386 + year = {2015},
387 + volume = {34},
388 + number = {1--2},
389 + pages = {106--126},
390 + doi = {10.1080/07474938.2014.944791}
391 +}
392 +
393 +@article{diebold1995comparing,
394 + author = {Diebold, Francis X. and Mariano, Roberto S.},
395 + title = {Comparing Predictive Accuracy},
396 + journal = {Journal of Business \& Economic Statistics},
397 + year = {1995},
398 + volume = {13},
399 + number = {3},
400 + pages = {253--263},
401 + doi = {10.1080/07350015.1995.10524599}
402 +}
403 +
404 +@article{giacomini2006tests,
405 + author = {Giacomini, Raffaella and White, Halbert},
406 + title = {Tests of Conditional Predictive Ability},
407 + journal = {Econometrica},
408 + year = {2006},
409 + volume = {74},
410 + number = {6},
411 + pages = {1545--1578},
412 + doi = {10.1111/j.1468-0262.2006.00718.x}
413 +}
414 +
415 +@article{bergmeir2012crossvalidation,
416 + author = {Bergmeir, Christoph and Ben{\'i}tez, Jos{\'e} M.},
417 + title = {On the Use of Cross-Validation for Time Series Predictor Evaluation},
418 + journal = {Information Sciences},
419 + year = {2012},
420 + volume = {191},
421 + pages = {192--213},
422 + doi = {10.1016/j.ins.2011.12.028}
423 +}
424 +
425 +@article{roberts2017crossvalidation,
426 + author = {Roberts, David R. and Bahn, Volker and Ciuti, Simone and Boyce, Mark S. and Elith, Jane and Guillera-Arroita, Gurutzeta and Hauenstein, Severin and Lahoz-Monfort, Jos{\'e} J. and Schr{\"o}der, Boris and Thuiller, Wilfried and Warton, David I. and Wintle, Brendan A. and Hartig, Florian and Dormann, Carsten F.},
427 + title = {Cross-Validation Strategies for Data with Temporal, Spatial, Hierarchical, or Phylogenetic Structure},
428 + journal = {Ecography},
429 + year = {2017},
430 + volume = {40},
431 + number = {8},
432 + pages = {913--929},
433 + doi = {10.1111/ecog.02881}
434 +}
435 +
436 +@article{kuminoff2010which,
437 + author = {Kuminoff, Nicolai V. and Parmeter, Christopher F. and Pope, Jaren C.},
438 + title = {Which Hedonic Models Can We Trust to Recover the Marginal Willingness to Pay for Environmental Amenities?},
439 + journal = {Journal of Environmental Economics and Management},
440 + year = {2010},
441 + volume = {60},
442 + number = {3},
443 + pages = {145--160},
444 + doi = {10.1016/j.jeem.2010.06.001}
445 +}
446 +
447 +@article{bishop2020best,
448 + author = {Bishop, Kelly C. and Kuminoff, Nicolai V. and Banzhaf, H. Spencer and Boyle, Kevin J. and von Gravenitz, Kathrine and Pope, Jaren C. and Smith, V. Kerry and Timmins, Christopher D.},
449 + title = {Best Practices for Using Hedonic Property Value Models to Measure Willingness to Pay for Environmental Quality},
450 + journal = {Review of Environmental Economics and Policy},
451 + year = {2020},
452 + volume = {14},
453 + number = {2},
454 + pages = {260--281},
455 + doi = {10.1093/reep/reaa001}
456 +}
457 +
458 +@article{bourassa2007spatial,
459 + author = {Bourassa, Steven C. and Cantoni, Eva and Hoesli, Martin},
460 + title = {Spatial Dependence, Housing Submarkets, and House Price Prediction},
461 + journal = {Journal of Real Estate Finance and Economics},
462 + year = {2007},
463 + volume = {35},
464 + number = {2},
465 + pages = {143--160},
466 + doi = {10.1007/s11146-007-9036-8}
467 +}
468 +
469 +@article{bourassa2003submarkets,
470 + author = {Bourassa, Steven C. and Hoesli, Martin and Peng, Vincent S.},
471 + title = {Do Housing Submarkets Really Matter?},
472 + journal = {Journal of Housing Economics},
473 + year = {2003},
474 + volume = {12},
475 + number = {1},
476 + pages = {12--28},
477 + doi = {10.1016/S1051-1377(03)00003-2}
478 +}
479 +
480 +@article{goodman2003segmentation,
481 + author = {Goodman, Allen C. and Thibodeau, Thomas G.},
482 + title = {Housing Market Segmentation and Hedonic Prediction Accuracy},
483 + journal = {Journal of Housing Economics},
484 + year = {2003},
485 + volume = {12},
486 + number = {3},
487 + pages = {181--201},
488 + doi = {10.1016/S1051-1377(03)00031-7}
489 +}
490 +
491 +@article{zurada2011comparison,
492 + author = {Zurada, Jozef and Levitan, Alan S. and Guan, Jian},
493 + title = {A Comparison of Regression and Artificial Intelligence Methods in a Mass Appraisal Context},
494 + journal = {Journal of Real Estate Research},
495 + year = {2011},
496 + volume = {33},
497 + number = {3},
498 + pages = {349--388},
499 + doi = {10.1080/10835547.2011.12091311}
500 +}
501 +
502 +@article{antipov2012mass,
503 + author = {Antipov, Evgeny A. and Pokryshevskaya, Elena B.},
504 + title = {Mass Appraisal of Residential Apartments: An Application of Random Forest for Valuation and a {CART}-Based Approach for Model Diagnostics},
505 + journal = {Expert Systems with Applications},
506 + year = {2012},
507 + volume = {39},
508 + number = {2},
509 + pages = {1772--1778},
510 + doi = {10.1016/j.eswa.2011.08.077}
511 +}
512 +
513 +@article{perezrave2019machine,
514 + author = {P{\'e}rez-Rave, Jorge Iv{\'a}n and Correa-Morales, Juan Carlos and Gonz{\'a}lez-Echavarr{\'i}a, Favi{\'a}n},
515 + title = {A Machine Learning Approach to Big Data Regression Analysis of Real Estate Prices for Inferential and Predictive Purposes},
516 + journal = {Journal of Property Research},
517 + year = {2019},
518 + volume = {36},
519 + number = {1},
520 + pages = {59--96},
521 + doi = {10.1080/09599916.2019.1587489}
522 +}
523 +
524 +@article{mccluskey2013prediction,
525 + author = {McCluskey, William J. and McCord, Michael and Davis, Peadar T. and Haran, Martin and McIlhatton, David},
526 + title = {Prediction Accuracy in Mass Appraisal: A Comparison of Modern Approaches},
527 + journal = {Journal of Property Research},
528 + year = {2013},
529 + volume = {30},
530 + number = {4},
531 + pages = {239--265},
532 + doi = {10.1080/09599916.2013.781204}
533 +}
534 +
535 +@article{schulz2014automated,
536 + author = {Schulz, Rainer and Wersing, Martin and Werwatz, Axel},
537 + title = {Automated Valuation Modelling: A Specification Exercise},
538 + journal = {Journal of Property Research},
539 + year = {2014},
540 + volume = {31},
541 + number = {2},
542 + pages = {131--153},
543 + doi = {10.1080/09599916.2013.846930}
544 +}
545 +
546 +@article{fik2003modeling,
547 + author = {Fik, Timothy J. and Ling, David C. and Mulligan, Gordon F.},
548 + title = {Modeling Spatial Variation in Housing Prices: A Variable Interaction Approach},
549 + journal = {Real Estate Economics},
550 + year = {2003},
551 + volume = {31},
552 + number = {4},
553 + pages = {623--646},
554 + doi = {10.1046/j.1080-8620.2003.00079.x}
555 +}
556 +
557 +@article{clapp2003semiparametric,
558 + author = {Clapp, John M.},
559 + title = {A Semiparametric Method for Valuing Residential Locations: Application to Automated Valuation},
560 + journal = {Journal of Real Estate Finance and Economics},
561 + year = {2003},
562 + volume = {27},
563 + number = {3},
564 + pages = {303--320},
565 + doi = {10.1023/A:1025838007297}
566 +}
567 +
568 +@article{pace1998spatiotemporal,
569 + author = {Pace, R. Kelley and Barry, Ronald and Clapp, John M. and Rodriquez, Mauricio},
570 + title = {Spatiotemporal Autoregressive Models of Neighborhood Effects},
571 + journal = {Journal of Real Estate Finance and Economics},
572 + year = {1998},
573 + volume = {17},
574 + number = {1},
575 + pages = {15--33},
576 + doi = {10.1023/A:1007799028599}
577 +}
578 +
579 +@article{dube2013pooled,
580 + author = {Dub{\'e}, Jean and Legros, Di{\`e}go},
581 + title = {Dealing with Spatial Data Pooled over Time in Statistical Models},
582 + journal = {Letters in Spatial and Resource Sciences},
583 + year = {2013},
584 + volume = {6},
585 + number = {1},
586 + pages = {1--18},
587 + doi = {10.1007/s12076-012-0082-3}
588 +}
589 +
590 +@article{hill2009boom,
591 + author = {Hill, Robert J. and Melser, Daniel and Syed, Iqbal},
592 + title = {Measuring a Boom and Bust: The {Sydney} Housing Market 2001--2006},
593 + journal = {Journal of Housing Economics},
594 + year = {2009},
595 + volume = {18},
596 + number = {3},
597 + pages = {193--205},
598 + doi = {10.1016/j.jhe.2009.07.010}
599 +}
600 +
601 +@article{manning2001estimating,
602 + author = {Manning, Willard G. and Mullahy, John},
603 + title = {Estimating Log Models: To Transform or Not to Transform?},
604 + journal = {Journal of Health Economics},
605 + year = {2001},
606 + volume = {20},
607 + number = {4},
608 + pages = {461--494},
609 + doi = {10.1016/S0167-6296(01)00086-8}
610 +}
611 +
612 +@article{thibodeau2003marking,
613 + author = {Thibodeau, Thomas G.},
614 + title = {Marking Single-Family Property Values to Market},
615 + journal = {Real Estate Economics},
616 + year = {2003},
617 + volume = {31},
618 + number = {1},
619 + pages = {1--22},
620 + doi = {10.1111/j.1080-8620.2003.00055.x}
621 +}
622 +
623 +@article{khoshnoud2023evolution,
624 + author = {Khoshnoud, Mahsa and Sirmans, G. Stacy and Zietz, Emily N.},
625 + title = {The Evolution of Hedonic Pricing Models},
626 + journal = {Journal of Real Estate Literature},
627 + year = {2023},
628 + volume = {31},
629 + number = {1},
630 + pages = {1--47},
631 + doi = {10.1080/09277544.2023.2201020}
632 +}
modified paper/sections/conclusion.tex +34 −24
@@ -5,28 +5,38 @@
5 5
6 6 We ran the hedonic specification debate as a controlled experiment:
7 7 twenty models, four design axes, one attribute set, one scoreboard, two
8 −holdouts, half a million sales. The verdict inverts the profession's
9 −implicit priorities. The choices that consume the most methodological
10 −attention --- functional form, transformation parameters --- move
11 −out-of-sample accuracy by at most two points of median error, while the
12 −choices often made by default --- whether and how finely to absorb space
13 −and time --- move it by three to ten. Machine-learning estimators
14 −dominate the standard scoreboard, but the standard scoreboard asks the
15 −wrong question: when the test set lies in the future, as it always does
16 −in deployment, the boosted ensemble's advantage reverses against a
17 −spline hedonic with good controls, because most of what it learned was a
18 −spatially fine price configuration that does not transfer across the
19 −market cycle.
8 +holdouts, half a million sales, and a bootstrap interval on every
9 +comparison. The verdict inverts the profession's implicit priorities.
10 +The choices that consume the most methodological attention---functional
11 +form, transformation parameters---move out-of-sample accuracy by at most
12 +two points of median error, while the choices often made by
13 +default---whether and how to absorb space and time---move it by three to
14 +ten, and the way fine spatial effects are implemented matters more than
15 +their resolution. Machine-learning estimators dominate the standard
16 +scoreboard by three points, a gain that is real, grows with the sample
17 +and comes almost entirely from a flexible surface over space. When the
18 +test set lies in the future, as it always does in deployment, that gain
19 +shrinks to half a point and loses significance, not because the learned
20 +surface fails to transfer---re-levelled, it prices future sales as well
21 +as it prices contemporaneous ones---but because every frozen model
22 +carries a stale price level, and a boosted model carries a slightly
23 +staler one than a linear model with carry-forward time effects.
20 24
21 −Meanwhile the objects economists actually harvest from hedonic models
22 −prove reassuringly stable: constant-quality indices agree across forms
23 −to within a few points over a full boom--bust--recovery cycle, and the
24 −machine's implicit age and area gradients replicate the quadratic OLS.
25 −The practical synthesis is almost embarrassingly simple: a semi-log with
26 −splines, quarter effects, and spatial fixed effects at a granularity
27 −matched to market thickness is within two points of the frontier in
28 −deployment conditions --- transparent, auditable, and fast. Where the
29 −extra accuracy of learning methods is worth its retraining cadence ---
30 −high-frequency AVMs, collateral monitoring --- our decomposition says to
31 −spend the flexibility budget on the spatial surface, and to validate,
32 −always, against the future rather than against a shuffle of the past.
25 +The objects economists harvest from hedonic models prove stable along
26 +the axis that generates the most debate and less so along the axis that
27 +generates the least: constant-quality indices agree across functional
28 +forms to within 2.5 points over a full boom--bust--recovery cycle but
29 +differ by nine points between a time-dummy index and a boosted
30 +imputation index, and the machine's implicit age and area gradients
31 +share the shape of the quadratic OLS with a milder depreciation path.
32 +The practical synthesis is simple. A semi-log with splines, quarter
33 +effects and spatial fixed effects nested from municipality down to a
34 +fine grid is within three points of the frontier under random
35 +validation and indistinguishable from it in deployment---transparent,
36 +auditable and fast. Where the extra within-period accuracy of learning
37 +methods is worth a retraining cadence---high-frequency automated
38 +valuation, collateral monitoring---our decomposition says to spend the
39 +flexibility budget on the spatial surface, to supply the price level
40 +from an index or a frequent refit rather than from the model, and to
41 +validate, always, against the future rather than against a shuffle of
42 +the past.
modified paper/sections/data.tex +43 −26
@@ -6,37 +6,53 @@
6 6 \subsection{Sources}
7 7
8 8 We use a registry of 745{,}119 Quebec residential-market transactions
9 −recorded between January 2021 and July 2026 --- sale price, date, address
10 −and coordinates --- each matched at the parcel level to the municipal
9 +recorded between January 2021 and July 2026---sale price, date, address
10 +and coordinates---each matched at the parcel level to the municipal
11 11 assessment roll in force at the sale date (median match distance
12 −0.6~metres, validated against the roll's recorded value). The roll
13 −contributes the structural descriptors that are notoriously missing from
14 −transaction registries: total floor area, lot area, year of construction,
15 −number of dwelling units, number of storeys, physical configuration
16 −(detached, semi-detached, row, apartment-linked) and the standardized use
17 −code. This combination --- market prices with assessor-grade structure
18 −for half a million sales across an entire province --- is what makes a
19 −controlled specification race feasible: every model sees exactly the same
20 −attributes.
12 +0.6~metres, validated against the roll's recorded value). The same
13 +snapshot underlies \companionpaper{}. The roll contributes the
14 +structural descriptors that are typically missing from transaction
15 +registries: total floor area, lot area, year of construction, number of
16 +dwelling units, number of storeys, physical configuration (detached,
17 +semi-detached, row, apartment-linked) and the standardised use code.
18 +This combination---market prices with assessor-grade structure for half a
19 +million sales across an entire province---is what makes a controlled
20 +specification race feasible: every model sees exactly the same
21 +attributes, and the attribute set is the one a mass-appraisal office
22 +actually holds.
21 23
22 24 \subsection{Sample}
23 25
24 26 We keep residential use codes (dwellings, cottages, mobile homes), require
25 −a high-confidence roll match (distance $\le 50$~m, score $\ge 150$), a
26 −price of at least \$50{,}000, a plausible floor area (20--2{,}000~m$^2$)
27 −and a known year of construction, and trim prices and floor areas at the
28 −1st and 99th percentiles within each sale year. The estimation sample
29 −contains \textbf{514{,}212 sales}: 348{,}253 single-family homes, 75{,}734
30 −condominiums, 74{,}909 plexes (2--5 units), 10{,}349 cottages and 4{,}881
31 −mobile homes, spread over 1{,}115 municipalities, 5{,}255 grid cells of
32 −$\sim$5.5~km and 30{,}548 cells of $\sim$1.1~km.
33 −Table~\ref{tab:sumstats} reports the descriptives. Prices average
34 −\$462{,}000 (median \$399{,}000); the window covers the tail of the
35 −post-pandemic boom, the 2022--2023 rate correction and the subsequent
36 −recovery --- a demanding environment for any static price model, which we
37 −exploit in the forward-in-time evaluation.
27 +a high-confidence roll match (distance $\le 50$~m, match score
28 +$\ge 150$), a price of at least \$50{,}000, a plausible floor area
29 +(20--2{,}000~m$^2$) and a known year of construction, and trim prices and
30 +floor areas at the 1st and 99th percentiles within each sale year.
31 +Building age is the sale year minus the year of construction, bounded
32 +between 0 and 150; storeys are bounded between 1 and 6 and dwelling units
33 +between 1 and 12; lot areas above the 99.5th percentile, which are
34 +implausible for residential parcels, are set to zero and treated as
35 +units without an exclusive lot, together with condominiums. The
36 +estimation sample contains \textbf{514{,}212 sales}: 348{,}253
37 +single-family homes, 75{,}734 condominiums, 74{,}909 plexes (two to five
38 +units), 10{,}349 cottages, 4{,}881 mobile homes and 86 dwellings of
39 +undetermined class, spread over 1{,}115 municipalities, 5{,}255 grid
40 +cells of $\sim$5.5~km and 30{,}548 cells of $\sim$1.1~km. Sales are
41 +spread across the window (97{,}868 in 2021, 90{,}733 in 2022, 76{,}411 in
42 +2023, 90{,}812 in 2024, 103{,}830 in 2025 and 54{,}558 in January--July
43 +2026). Table~\ref{tab:sumstats} reports the descriptive statistics.
44 +Prices average \$440{,}207 (median \$397{,}375), floor areas 132~m$^2$
45 +and building ages 44 years; the window covers the tail of the
46 +post-pandemic boom, the 2022--23 rate correction and the 2024--26
47 +recovery, during which the constant-quality price level rose by roughly
48 +70\% (Section~\ref{sec:ext}). This is a demanding environment for any
49 +static price model, which the forward-in-time evaluation exploits.
50 +One fifth of the sales (105{,}814) concern a parcel that sells more than
51 +once in the window and 30\% share their exact coordinates with another
52 +sale; Section~\ref{sec:holdouts} explains why these repeat and
53 +co-located sales matter for validation.
38 54
39 −\begin{table}[t]
55 +\begin{table}[tbp]
40 56 \centering
41 57 \begin{threeparttable}
42 58 \caption{Summary statistics, estimation sample}
@@ -47,7 +63,8 @@ exploit in the forward-in-time evaluation.
47 63 \item \textit{Notes:} 514{,}212 residential sales, January 2021 -- July
48 64 2026. Structural descriptors from the assessment roll in force at the
49 65 sale date. Lot area is zero for condominiums and other units without an
50 −exclusive lot.
66 +exclusive lot (and for the 0.5\% of lots above the 99.5th percentile,
67 +see text).
51 68 \end{tablenotes}
52 69 \end{threeparttable}
53 70 \end{table}
modified paper/sections/discussion.tex +94 −61
@@ -5,77 +5,110 @@
5 5
6 6 \subsection{A hierarchy of specification choices}
7 7
8 −Read jointly, the four axes imply a clear hierarchy of returns to
9 −specification effort, useful to anyone building a hedonic model:
8 +Read jointly, the four axes imply a hierarchy of returns to
9 +specification effort. The first and largest return is to location:
10 +any spatial absorption beats none by about ten points of median error, a
11 +margin that dwarfs every other choice, and the granularity question is
12 +one of implementation rather than of resolution. A fine grid used alone
13 +overfits because thin cells are estimated from a handful of sales and
14 +unseen cells are priced arbitrarily; the same grid nested within
15 +municipalities, which pools thin cells toward their municipality, is the
16 +most accurate linear specification we estimate. The second return is to
17 +time controls: in a moving market a model without time effects mistakes
18 +appreciation for attributes, at a cost of three points under random
19 +validation and six when the level must be carried forward, while
20 +granularity beyond quarters is cosmetic. The third return is to the
21 +estimator, and its size depends on the question asked. Under random
22 +validation gradient boosting buys three points over the best linear
23 +model, a gain that is statistically unambiguous and that grows with the
24 +training sample; in deployment the gain on the median error shrinks to
25 +half a point and loses significance, because the ensembles carry a
26 +slightly staler price level, while their relative pricing skill is
27 +intact. The fourth and smallest return is to functional form: splines
28 +are a cheap and reliable upgrade over the semi-log, and the Box--Cox
29 +machinery is not worth its complexity, exactly as
30 +\citet{cropper1988choice} concluded from simulations four decades ago.
10 31
11 −\begin{enumerate}
12 −\item \textbf{Get location right} ($\sim$10 points of median error):
13 −any spatial absorption beats none by a margin that dwarfs every other
14 −choice; the optimal granularity is interior --- around 5~km here ---
15 −because finer cells trade bias for variance.
16 −\item \textbf{Include some time control} ($\sim$3 points): in a moving
17 −market, a model without time effects mistakes appreciation for
18 −attributes; granularity beyond quarters is cosmetic.
19 −\item \textbf{Choose the estimator for the job} ($\sim$2 points
20 −deployed, $\sim$4 under random validation): boosting buys real accuracy
21 −in interpolation-heavy uses (filling gaps within a period, mass
22 −appraisal with contemporaneous comparables) and much less when the
23 −target is the future.
24 −\item \textbf{Stop worrying about functional form} ($\lesssim$2
25 −points): splines are a cheap upgrade; the Box--Cox machinery is not
26 −worth its complexity, exactly as \citet{cropper1988choice} concluded
27 −from simulations four decades ago.
28 −\end{enumerate}
32 +\subsection{Why the forward-in-time split compresses the ranking}
29 33
30 −\subsection{Why the machines lose their edge out of time}
31 −
32 −The decomposition rows explain the generalization gap. The boosting
33 −model's random-split advantage comes almost entirely from a flexible
34 −spatial surface (removing coordinates costs it 8.8 points). That surface
35 −is estimated \emph{jointly} with the time path, and trees clamp: beyond
36 −the last training month the model prices 2025--26 sales at a frozen
37 −level, exactly like the carry-forward linear models, while its
38 −fine-grained spatial fit --- calibrated on the 2021--24 price
39 −configuration --- partially decays as relative prices shift. The linear
40 −models, coarser but more rigid, carry a structure that transfers better.
41 −This is not an indictment of machine learning --- retrained monthly, the
42 −boosting model would presumably keep its interpolation advantage --- but
43 −it prices the \emph{retraining requirement} that random cross-validation
44 −hides, and it matches the practitioner evidence that AVM accuracy decays
45 −quickly out of sample period \citep{bogin2020house, kok2017big}.
34 +The decomposition of Section~\ref{sec:gap} clarifies what the
35 +forward-in-time holdout measures. The boosting model's random-split
36 +advantage comes almost entirely from a flexible spatial surface
37 +(removing the coordinates costs it 8.8 points), and that surface
38 +transfers to 2025--26 sales: re-levelled, the model prices them with a
39 +13.9\% median error, against 16.8\% for the spline hedonic. What does
40 +not transfer is the aggregate level. Trees are piecewise constant in the
41 +month counter and the terminal leaf in time averages the last training
42 +months rather than the last one, so a frozen boosted model enters the
43 +test period with a slightly staler level than a linear model whose last
44 +month effect is carried forward; by mid-2026 both are about 25\% below
45 +the market and indistinguishable. The lesson is not that machine
46 +learning fails out of sample. It is that the forward-in-time holdout
47 +conflates two things---the common level error that any static model
48 +incurs, and the relative accuracy the estimator contributes---and that
49 +the first dominates the second within a few quarters. Retrained monthly,
50 +or re-levelled with a published index as practitioners do
51 +\citep{thibodeau2003marking, mayer2019estimation}, the boosted model
52 +would presumably keep its interpolation advantage; the forward split
53 +prices the \emph{retraining or re-levelling requirement} that random
54 +cross-validation hides, and it matches the practitioner evidence that
55 +automated-valuation accuracy decays quickly outside the sample period
56 +\citep{bogin2020house, kok2017big}. It also explains why our first
57 +version, which smeared the linear models but not the ensembles, found a
58 +reversal: in a rising market an upward retransformation factor applied
59 +to one family only offsets part of that family's staleness.
46 60
47 61 \subsection{Implications}
48 62
49 −\paragraph{For research.} Hedonic coefficients and indices are robust
50 −objects: form and even estimator perturb them little
51 −(Section~\ref{sec:ext}). Researchers using hedonics to \emph{measure}
52 −--- indices, implicit prices, capitalization effects --- can keep simple
53 −forms with good controls and report validation under a temporal split
54 −when prediction claims are made.
63 +\paragraph{For research.} Hedonic indices are robust to functional form
64 +and implicit-price gradients are robust to the estimator in shape, if
65 +not in slope (Section~\ref{sec:ext}). Researchers using hedonic models to
66 +\emph{measure}---indices, implicit prices, capitalization effects---can
67 +keep simple forms with good, preferably hierarchical, spatial controls.
68 +When prediction claims are made, they should report a forward-in-time
69 +split alongside random cross-validation, and they should report
70 +intervals: several of the differences that organise the applied
71 +debate---a 5.5~km grid against municipality effects, month against
72 +quarter effects, boosting against splines in deployment---are not
73 +distinguishable from zero on half a million sales.
55 74
56 75 \paragraph{For mass appraisal.} Quebec's assessors face exactly this
57 −design problem every three years. Our results suggest the largest
58 −accuracy gains lie not in exotic estimators but in spatial resolution
59 −chosen by market thickness --- and that any model, linear or boosted,
60 −frozen at a reference date degrades by 3--7 points of median error
61 −within two years. This quantifies the staleness mechanism behind the
62 −assessment inequities documented in \companionpaper{}: the
63 −valuation technology's out-of-time decay is of the same magnitude as the
64 −inequities measured there.
76 +design problem every three years. Our results suggest that the largest
77 +accuracy gains lie in spatial resolution implemented hierarchically, that
78 +flexible learners add a further three points within the period they are
79 +estimated on, and that any model, linear or boosted, frozen at a
80 +reference date under-prices the typical sale by 10--20\% within eighteen
81 +months in a market like 2025--26. This quantifies the staleness
82 +mechanism behind the assessment inequities documented in
83 +\companionpaper{}: the valuation technology's out-of-time decay is of the
84 +same order as the inequities measured there, and it is a level problem
85 +that an index adjustment addresses before any change of estimator.
65 86
66 −\paragraph{For the AVM literature.} Every comparison should report a
67 −forward-in-time split alongside random cross-validation
68 −\citep{steurer2021metrics}. On our data, the choice of split changes not
69 −just magnitudes but the \emph{winner}.
87 +\paragraph{For the automated-valuation literature.} Every comparison
88 +should report a forward-in-time split, a leakage-free grouped split when
89 +properties sell repeatedly or share coordinates, and a single
90 +retransformation rule applied to all models
91 +\citep{steurer2021metrics}. On our data the first changes the size and
92 +significance of the headline gap, the second changes nothing, and the
93 +third, applied asymmetrically, is enough to manufacture a reversal.
70 94
71 95 \subsection{Limitations}
72 96
73 97 Our attribute set, though assessor-grade, omits listing-level quality
74 98 (renovations, finishes, views); richer features would likely raise the
75 −machines' interpolation advantage. Hyper-parameters were deliberately
76 −conventional --- a tuned boosted model would gain somewhat, as would a
77 −spatially smarter linear model (e.g.\ kriging residuals,
78 −\citealp{case2004modeling}). The forward split covers one specific
79 −regime change (the 2025--26 recovery); other windows would shift
80 −magnitudes. And Quebec's institutional homogeneity aids transferability
81 −of results across municipalities but may limit it across countries.
99 +learners' within-period advantage. Hyper-parameters were deliberately
100 +conventional, and Section~\ref{sec:robust} shows that materially
101 +different boosting configurations, including one tuned on a time-ordered
102 +validation set, leave the ranking unchanged; a spatially smarter linear
103 +model (kriged residuals, \citealp{case2004modeling}; semiparametric
104 +location surfaces, \citealp{clapp2003semiparametric}) would narrow the
105 +gap from the other side and is not estimated here. The forward split
106 +covers one specific regime (the 2025--26 recovery), in which prices rose
107 +by about 15\% in eighteen months; a flat or falling period would produce
108 +a smaller level error and possibly a different ordering of staleness
109 +across model families. The grouped holdouts remove leakage through
110 +repeat sales and shared coordinates but not through neighbouring parcels
111 +in the same street, which a spatially blocked holdout would address at
112 +the cost of changing the question to spatial extrapolation. Finally,
113 +Quebec's institutional homogeneity aids transferability of results
114 +across municipalities but may limit it across countries.
modified paper/sections/extensions.tex +90 −61
@@ -3,26 +3,41 @@
3 3 \section{Beyond the scoreboard: what the models imply}
4 4 \label{sec:ext}
5 5
6 −Accuracy is not the only output economists take from a hedonic model. This
7 −section asks whether the specification choices that move the scoreboard
8 −also move the two objects hedonic models are most often built to deliver:
9 −constant-quality price indices and implicit attribute prices.
6 +Accuracy is not the only output economists take from a hedonic model.
7 +This section asks whether the specification choices that move the
8 +scoreboard also move the two objects hedonic models are most often built
9 +to deliver, constant-quality price indices and implicit attribute prices,
10 +and then characterises where and when the learners' advantage
11 +materialises.
10 12
11 −\subsection{Price indices are form-robust}
13 +\subsection{Price indices: form-robust, estimator-sensitive}
12 14
13 −Figure~\ref{fig:index} plots the constant-quality monthly index implied by
14 −the month fixed effects of four linear forms --- semi-log, log-log,
15 −quadratic and spline --- together with a gradient-boosting index obtained
16 −by repricing a fixed 20{,}000-sale reference portfolio at each month
17 −(an imputation \emph{\`a la} \citealp{hill2013hedonic}). All five track
18 −the same cycle --- the 2021--22 boom, the 2022--23 correction, the
19 −2024--26 recovery --- and the four linear forms agree to within a few
20 −index points throughout. The choice that dominates the accuracy
21 −scoreboard barely perturbs the index: reassuring news for statistical
22 −agencies, and consistent with the RPPI handbook's pragmatism about form
23 −\citep{oecd2013handbook, silver2018house}.
15 +Figure~\ref{fig:index} plots the constant-quality monthly index implied
16 +by the month fixed effects of four linear forms---semi-log, log-log,
17 +quadratic and spline---together with a gradient-boosting index obtained
18 +by repricing a fixed 20{,}000-sale reference portfolio at each month, a
19 +hedonic imputation index in the sense of \citet{hill2013hedonic} and
20 +\citet{hill2018geospatial}. All five track the same cycle: the 2021--22
21 +boom, the 2022--23 correction and the 2024--26 recovery, over which the
22 +constant-quality level rises by about 70\%. The four linear forms agree
23 +to within 2.5 index points at every month, so the choice that generates
24 +the most methodological discussion barely perturbs the index, consistent
25 +with the RPPI handbook's pragmatism about form \citep{oecd2013handbook,
26 +silver2018house}. The estimator is a different matter: the boosting
27 +index rises less, ending in July 2026 at 160.5 against 168.6--171.0 for
28 +the linear forms, a gap that opens gradually from 2024 and reaches nine
29 +to eleven points. Two mechanisms contribute. The imputation index
30 +reprices a fixed portfolio, so its weights differ from the time-dummy
31 +index's implicit weights when appreciation is heterogeneous across
32 +segments; and the boosting model's month counter enters through tree
33 +splits that interact with location and attributes, so its ``time
34 +effect'' is an average of locally varying paths rather than a single
35 +common shift. Statistical agencies choosing between a time-dummy and an
36 +imputation index should expect this order of disagreement when the
37 +imputation model is a flexible learner; the agreement across linear
38 +forms is reassuring, the agreement across estimators is not automatic.
24 39
25 −\begin{figure}[t]
40 +\begin{figure}[tbp]
26 41 \centering
27 42 \includegraphics[width=\textwidth]{fig_index.png}
28 43 \caption{Constant-quality monthly price indices implied by competing
@@ -32,28 +47,36 @@ fixed effects. Gradient boosting: mean repriced value of a fixed
32 47 \label{fig:index}
33 48 \end{figure}
34 49
35 −\subsection{Implicit prices: the machine agrees with the quadratic}
50 +\subsection{Implicit prices: the machine broadly agrees with the quadratic}
36 51
37 −Figure~\ref{fig:profiles} compares the ln-price profiles in building age
38 −and floor area implied by the quadratic OLS coefficients with the partial-%
39 −dependence profiles of the boosting model \citep{friedman2001greedy}.
40 −The two agree over the bulk of the data: steep initial depreciation that
41 −flattens with age, and strongly concave returns to floor space. The
42 −visible divergences are where the quadratic \emph{cannot} bend --- the
43 −machine sees a milder gradient for very old stock (a vintage effect) and
44 −a small new-construction premium spike --- but through the ages and areas
45 −where 90\% of sales live, the curves sit within a tenth of a log point. The boosting model's advantage evidently comes from
46 −higher-order interactions and spatial flexibility, \emph{not} from a
47 −different view of the main attribute gradients --- so the interpretable
48 −quadratic form remains a faithful summary of the price surface even where
49 −it loses the prediction race \citep{mullainathan2017machine,
50 −athey2019machine}.
52 +Figure~\ref{fig:profiles} compares the log-price profiles in building age
53 +and floor area implied by the quadratic OLS coefficients with the
54 +partial-dependence profiles of the boosting model
55 +\citep{friedman2001greedy, lorenz2023interpretable}, each normalised to
56 +its leftmost point. The two agree on the shape: steep initial
57 +depreciation that flattens with age, and strongly concave returns to
58 +floor space. On floor area the curves sit within 0.05 log points of each
59 +other over the whole 50--400~m$^2$ range. On age the boosting profile is
60 +uniformly flatter: it shows a small new-construction premium (the
61 +profile peaks at five years, $+0.06$ log points above a new building,
62 +where the quadratic can only decline), and from twenty years onward its
63 +cumulative depreciation is 0.10--0.13 log points smaller than the
64 +quadratic's ($-0.20$ against $-0.31$ at forty years, $-0.28$ against
65 +$-0.41$ at sixty), with a milder gradient for very old stock, a vintage
66 +effect the quadratic cannot bend to accommodate. These are differences
67 +of degree in a shared shape, not of sign or ordering. The boosting
68 +model's accuracy advantage therefore does not come from a different view
69 +of the main attribute gradients; it comes from higher-order interactions
70 +and, above all, from the spatial surface (Section~\ref{sec:results}).
71 +The interpretable quadratic remains a faithful, if slightly steeper,
72 +summary of the age--price relation even where it loses the prediction
73 +race \citep{mullainathan2017machine, athey2019machine}.
51 74
52 −\begin{figure}[t]
75 +\begin{figure}[tbp]
53 76 \centering
54 77 \includegraphics[width=\textwidth]{fig_profiles.png}
55 −\caption{Implicit ln-price profiles: quadratic OLS coefficients versus
56 −gradient-boosting partial dependence, each normalized to its leftmost
78 +\caption{Implicit log-price profiles: quadratic OLS coefficients versus
79 +gradient-boosting partial dependence, each normalised to its leftmost
57 80 point. Panel A: building age. Panel B: floor area.}
58 81 \label{fig:profiles}
59 82 \end{figure}
@@ -61,18 +84,19 @@ point. Panel A: building age. Panel B: floor area.}
61 84 \subsection{Learning curves: data beat flexibility only at scale}
62 85
63 86 Figure~\ref{fig:learning} and Table~\ref{tab:learning} trace accuracy
64 −against training size from 10{,}000 to 400{,}000 sales. The OLS quadratic
65 −is essentially flat from the very first step (18.4\% at $n=10{,}000$,
66 −18.1\% at $n=400{,}000$) --- its bias floor binds almost immediately ---
67 −while the boosting model improves monotonically through the full sample
68 −(17.6\% $\to$ 14.5\%): the machine's advantage grows from under one
69 −error point at $n = 10{,}000$ to 3.6 points at
70 −$n = 400{,}000$. Flexible methods
71 −are not a free lunch in small markets: below a few tens of thousands of
72 −training sales, the machine's edge is modest, one reason rural and thin-%
73 −market AVMs underperform \citep{bogin2020house}.
87 +against training size from 10{,}000 to 400{,}000 sales on a fixed random
88 +holdout. The OLS quadratic is flat from the very first step (19.1\% at
89 +$n = 10{,}000$, 19.2\% at $n = 400{,}000$): its bias floor binds
90 +immediately, and more data cannot lower it. The boosting model improves
91 +monotonically through the full sample (17.6\% $\to$ 14.5\%), so its
92 +advantage grows from 1.6 points at $n = 10{,}000$ to 4.7 points at
93 +$n = 400{,}000$. Flexible methods are not a free lunch in small markets:
94 +below a few tens of thousands of training sales, the learner's edge is
95 +modest, one reason thin-market automated valuation underperforms
96 +\citep{bogin2020house}, and a reason the provincial scale of the present
97 +data matters for the size of the gap we measure.
74 98
75 −\begin{figure}[t]
99 +\begin{figure}[tbp]
76 100 \centering
77 101 \includegraphics[width=0.72\textwidth]{fig_learning.png}
78 102 \caption{Learning curves: median absolute error against training-set
@@ -80,7 +104,7 @@ size (random holdout fixed), OLS quadratic versus gradient boosting.}
80 104 \label{fig:learning}
81 105 \end{figure}
82 106
83 −\begin{table}[t]
107 +\begin{table}[tbp]
84 108 \centering
85 109 \begin{threeparttable}
86 110 \caption{Learning curves}
@@ -98,23 +122,28 @@ seeded subsamples of the indicated size.
98 122
99 123 Table~\ref{tab:segments} and Figure~\ref{fig:segments} decompose the
100 124 random-holdout MdAPE by property class and municipality size, and the
101 −pattern identifies the machine's edge with striking precision: it is
102 −spatial resolution. The gap is enormous for condominiums (9.1\% versus
103 −20.4\%) and in the largest cities (11.9\% versus 17.8\%) --- segments
104 −where structure is homogeneous but \emph{micro}-location varies hugely
105 −within a municipality, exactly the variation that municipality dummies
106 −cannot see and raw coordinates can. Once location is resolved, a condo
107 −is the most predictable object in the market. At the other extreme,
108 −cottages defeat both models (32.1\% versus 33.9\%): idiosyncratic,
109 −waterfront-driven stock is hard for everyone, and the machine's spatial
110 −surface cannot rescue attributes it does not observe. The complement of
111 −the assessment-inequity anatomy in \companionpaper{} is
125 +pattern locates the learner's edge: it is spatial resolution. The gap is
126 +largest for condominiums (9.1\% against 18.2\%) and in the largest
127 +cities (11.9\% against 17.7\%), segments where structure is homogeneous
128 +but \emph{micro}-location varies greatly within a municipality, exactly
129 +the variation that municipality dummies cannot see and raw coordinates
130 +can. Once location is resolved, a condominium is the most predictable
131 +object in the market. The gap is also large in the thinnest
132 +municipalities (18.7\% against 24.8\%), where a municipality effect is
133 +estimated from few sales and the learner can borrow strength from
134 +neighbouring municipalities through the coordinates. At the other
135 +extreme, cottages defeat both models (32.1\% against 34.3\%):
136 +idiosyncratic, waterfront-driven stock is hard for everyone, and a
137 +spatial surface cannot rescue attributes it does not observe. The
138 +complement with the assessment-inequity anatomy in \companionpaper{} is
112 139 instructive: the segments where assessors are most regressive
113 140 (heterogeneous, land-heavy stock) are also those where \emph{no}
114 141 standard technology, linear or boosted, prices accurately from roll
115 −attributes alone.
142 +attributes alone. Section~\ref{sec:robust} examines how much of the
143 +condominium advantage reflects genuine micro-location rather than recall
144 +of sales in the same building.
116 145
117 −\begin{figure}[t]
146 +\begin{figure}[tbp]
118 147 \centering
119 148 \includegraphics[width=0.8\textwidth]{fig_segments.png}
120 149 \caption{Median absolute error by market segment (random holdout):
@@ -122,7 +151,7 @@ OLS quadratic versus gradient boosting.}
122 151 \label{fig:segments}
123 152 \end{figure}
124 153
125 −\begin{table}[t]
154 +\begin{table}[tbp]
126 155 \centering
127 156 \begin{threeparttable}
128 157 \caption{Accuracy by market segment}
modified paper/sections/introduction.tex +96 −68
@@ -8,87 +8,115 @@ the price or not? Dummies for years or for months? Neighbourhood fixed
8 8 effects, and how fine? Least squares or, increasingly, a gradient-boosted
9 9 ensemble? Fifty years after \citet{rosen1974hedonic} established that
10 10 theory places no restriction on the shape of the price surface, these
11 −choices are still made largely by convention --- the semi-log survives as
12 −the default \citep{sirmans2005composition}, the classic form
13 −comparisons predate the out-of-sample era \citep{halvorsen1981choice,
14 −cropper1988choice}, and the machine-learning comparisons that have
15 −multiplied since \citet{mullainathan2017machine} typically change the
16 −sample, the features and the validation scheme all at once, making it
17 −impossible to say \emph{which} ingredient buys the reported gains.
11 +choices are still made largely by convention: the semi-log survives as
12 +the default \citep{sirmans2005composition, khoshnoud2023evolution}, the
13 +classic functional-form comparisons predate the out-of-sample era
14 +\citep{halvorsen1981choice, cropper1988choice}, and the machine-learning
15 +comparisons that have multiplied since \citet{mullainathan2017machine}
16 +typically change the sample, the features and the validation scheme at
17 +once, so that it is impossible to say \emph{which} ingredient buys the
18 +reported gains.
18 19
19 20 This paper runs the comparison the way one would design an experiment.
20 21 From 514{,}212 Quebec residential sales (2021--2026) matched at the
21 −parcel level to the provincial assessment roll --- so that every model
22 +parcel level to the provincial assessment roll---so that every model
22 23 sees the same assessor-grade attribute set: floor area, lot area, age,
23 −storeys, units, property class, physical configuration --- we estimate
24 −\textbf{twenty models organised along four axes}, varying one design
25 −choice at a time: six functional forms (linear to Box--Cox to splines),
26 −a ladder of time controls (none to month fixed effects), a ladder of
27 −spatial controls (none to a $\sim$1~km grid), and six estimation methods
28 −(OLS to random forests, gradient boosting, and a $k$-nearest-neighbour
24 +storeys, units, property class, physical configuration---we estimate
25 +twenty models organised along four axes, varying one design choice at a
26 +time: six functional forms (linear to Box--Cox to splines), a ladder of
27 +time controls (none to month fixed effects), a ladder of spatial
28 +controls (none to a $\sim$1~km grid), and six estimation methods (OLS to
29 +random forests, gradient boosting and a $k$-nearest-neighbour
29 30 comparables rule). Every model is scored on the same price-level
30 −scoreboard --- log models retransformed with Duan smearing, Box--Cox
31 −inverted --- and, crucially, under \textbf{two holdouts}: the standard
32 −random 80/20 split, and a forward-in-time split (train before 2025, test
33 −2025--26) that mimics how a valuation model is actually deployed.
31 +scoreboard, with a single retransformation rule for every model whose
32 +dependent variable is a transformed price, and under two holdouts: the
33 +standard random 80/20 split, and a forward-in-time split (train before
34 +2025, test 2025--26) that mimics how a valuation model is actually
35 +deployed. Because differences of a few tenths of a point between models
36 +evaluated on the same sales can be noise, every comparison comes with a
37 +municipality-clustered bootstrap interval and a Diebold--Mariano-type
38 +test of equal accuracy.
34 39
35 −Four results organise the paper.
36 −
37 −\textbf{First, functional form is a second-order choice.} Across linear,
38 −semi-log, log-log, profile-likelihood Box--Cox
39 −($\hat\lambda = 0.25$), quadratic and spline specifications --- holding
40 −controls fixed --- the median absolute error spans 16.5\% to 18.6\%.
41 −Splines beat the textbook semi-log by 1.6 points; the data-driven
42 −Box--Cox transformation, the great hope of the 1980s debate, buys
43 −nothing out of sample, vindicating \citet{cropper1988choice} and
40 +Four results organise the paper. \emph{First, functional form is a
41 +second-order choice.} Across linear, semi-log, log-log,
42 +profile-likelihood Box--Cox ($\hat\lambda = 0.25$), quadratic and spline
43 +specifications, holding controls fixed, the median absolute error spans
44 +17.6\% to 19.2\% under the random holdout. Splines beat the textbook
45 +semi-log by 1.4 points; the data-driven Box--Cox transformation, the
46 +great hope of the 1980s debate, gains half a point on the median error
47 +and nothing on the mean error, vindicating \citet{cropper1988choice} and
44 48 \citet{cassel1985cost} on modern data at scale.
45 49
46 −\textbf{Second, controls are first-order --- with a twist.} Removing all
47 −spatial controls costs ten error points (27.9\% versus 17.5\%), an order
48 −of magnitude more than any curvature decision; a $\sim$5.5~km grid beats
49 −both municipality effects and, notably, a $\sim$1.1~km grid, whose
50 −30{,}548 cells overfit --- the bias--variance trade-off in absorbing
51 −location is real and non-monotonic. Time effects show the same shape in
52 −miniature: anything beats nothing (three points), granularity beyond
53 −quarters buys nothing.
50 +\emph{Second, controls are first-order, with a twist.} Removing all
51 +spatial controls costs ten error points (28.8\% against 19.2\%), six
52 +times the entire functional-form range; a $\sim$5.5~km grid is nominally
53 +but not significantly better than municipality effects, and a
54 +$\sim$1.1~km grid used on its own overfits (21.0\%), its 30{,}548 cells
55 +holding three sales at the median. Nested within municipalities, so that
56 +thin and unseen cells inherit their municipality's level, the same fine
57 +grid becomes the most accurate linear specification in the paper
58 +(17.3\%): the bias--variance trade-off in absorbing location is real,
59 +and it is resolved by hierarchy rather than by coarseness. Time effects
60 +show the same shape in miniature: anything beats nothing (three points
61 +under the random split, six under the forward split), and granularity
62 +beyond quarters buys nothing.
54 63
55 −\textbf{Third, under random validation the estimation method dominates.}
64 +\emph{Third, under random validation the estimation method dominates.}
56 65 Gradient boosting reaches a median error of 14.4\% and random forests
57 −14.6\%, against 16.5\% for the best linear model --- a 13\% improvement
58 −--- and stripping the coordinates from the boosting model degrades it to
59 −23.2\%, showing that what the machine mainly learns is a flexible price
60 −surface over space \citep{bourassa2010predicting}. The na\"ive
61 −$k$-NN comparables rule (21.2\%) confirms that neither raw proximity nor
62 −raw flexibility suffices: the gains come from their combination.
66 +14.6\%, against 17.6\% for the best linear model, an 18\% improvement
67 +that is statistically unambiguous ($-3.2$ points, 95\% interval $-4.7$
68 +to $-1.9$). Stripping the coordinates from the boosting model degrades
69 +it to 23.2\%, so what the learner mainly buys is a flexible price
70 +surface over space \citep{bourassa2010predicting, pace2020residuals};
71 +the na\"ive $k$-NN comparables rule (21.2\%) shows that raw proximity
72 +without attribute adjustment does not suffice either. The advantage is
73 +not an artefact of repeat sales or of condominium units geocoded to the
74 +same building: it survives holdouts grouped by parcel and by coordinate
75 +pair essentially unchanged.
63 76
64 −\textbf{Fourth --- our headline --- the ranking is an artifact of the
65 −validation scheme.} Under the forward-in-time split every model degrades,
66 −but not equally: the machine-learning advantage \emph{reverses} on the
67 −median error (gradient boosting 21.2\% versus 19.1\% for the spline
68 −hedonic) and equalizes exactly on log-scale $R^2$ (0.52 for both).
69 −Random cross-validation lets flexible models interpolate the price level
70 −of their own test period; when the test period lies in the future --- the
71 −only case that matters for deployment --- that advantage is leakage, not
72 −skill. Comparisons of valuation models that report random cross-validation
73 −alone, i.e.\ most of the applied ML literature, systematically overstate
74 −the practical value of model flexibility \citep{clapp2002predicting,
75 −steurer2021metrics}.
77 +\emph{Fourth---our headline---the advantage is a within-period
78 +statement.} Under the forward-in-time split every model degrades, the
79 +ensembles most: gradient boosting loses 6.8 points (to 21.2\%) and the
80 +spline hedonic 4.1 (to 21.7\%), so the three-point gap shrinks to half a
81 +point and loses statistical significance, while the ensembles keep a
82 +modest lead on the mean error and on log-scale fit. Decomposing the
83 +forward error shows why. The 2025--26 sales are not harder to price:
84 +evaluated on the same sales, a boosting model that has seen their period
85 +posts 13.7\%. What a frozen model loses is the \emph{level}: every
86 +forward-split model under-prices the typical 2025--26 sale by 10--20\%,
87 +the ensembles slightly more than the linear models because their
88 +terminal leaf in time averages the last training months. Once each
89 +model's level is corrected, the boosting advantage over the spline
90 +hedonic is 2.9 points, essentially what the random split measures.
91 +Relative pricing skill transfers forward almost intact; the price level
92 +does not, and random cross-validation hides a staleness cost of three to
93 +seven points of median error within eighteen months that any deployed
94 +model, linear or boosted, must pay unless it is re-levelled by an index
95 +or retrained. A previous version of this paper reported that the
96 +ranking reversed in the forward split; the reversal was an artefact of
97 +an asymmetric retransformation and is corrected here.
76 98
77 99 The paper then asks what the specification choices do to the objects
78 −economists actually extract from hedonic models. Constant-quality monthly
79 −indices implied by the month effects of four linear forms --- and by
80 −repricing a fixed portfolio with the boosting model --- agree to within a
81 −few index points across the largest housing cycle in recent Canadian
82 −history. And the boosting model's partial-dependence profiles in age and
83 −floor area track the quadratic OLS closely: the machine agrees with the
84 −economist about the gradients and disagrees mainly about the residual
85 −surface. Interpretation and prediction, in other words, conflict far less
86 −than the accuracy scoreboard suggests.
100 +economists actually extract from hedonic models. Constant-quality
101 +monthly indices implied by the month effects of four linear forms agree
102 +to within 2.5 index points across the largest housing cycle in recent
103 +Canadian history, whereas the imputation index from the boosting model
104 +rises nine points less over five years: form does not move the index,
105 +the estimator can. The boosting model's partial-dependence profiles in
106 +age and floor area share the shape of the quadratic OLS gradients, with
107 +a uniformly milder depreciation path, so the machine disagrees with the
108 +economist about the residual surface rather than about the main
109 +gradients. Interpretation and prediction, in other words, conflict far
110 +less than the accuracy scoreboard suggests.
87 111
88 112 Section~\ref{sec:lit} situates the exercise in the functional-form,
89 −spatial-hedonic and ML-valuation literatures. Section~\ref{sec:data}
90 −describes the data, Section~\ref{sec:method} the experimental design.
91 −Section~\ref{sec:results} reports the horse race,
92 −Section~\ref{sec:ext} the index, implicit-price, learning-curve and
93 −segment extensions, Section~\ref{sec:disc} discusses implications, and
113 +spatial-hedonic and machine-learning valuation literatures.
114 +Section~\ref{sec:data} describes the data, Section~\ref{sec:method} the
115 +experimental design, the scoreboard and the inference procedures.
116 +Section~\ref{sec:results} reports the horse race and decomposes the
117 +generalization gap, Section~\ref{sec:ext} the index, implicit-price,
118 +learning-curve and segment extensions, Section~\ref{sec:robust} the
119 +robustness analyses (retransformation rule, leakage-free holdouts, nested
120 +spatial effects, Box--Cox grid, boosting hyper-parameters),
121 +Section~\ref{sec:disc} discusses implications, and
94 122 Section~\ref{sec:concl} concludes.
modified paper/sections/literature.tex +92 −50
@@ -6,62 +6,104 @@
6 6 \subsection{The functional-form debate}
7 7
8 8 Hedonic theory is silent about functional form. \citet{rosen1974hedonic}
9 −showed that the price surface is an equilibrium envelope of bids and
10 −offers, so nothing structural pins down its curvature ---
11 −form is an empirical choice. The classical exchange settled into a
12 −three-cornered debate: \citet{halvorsen1981choice} advocated data-driven
13 −Box--Cox transformations \citep{box1964analysis};
14 −\citet{cassel1985cost} cautioned that flexible transformations optimize
15 −in-sample fit while degrading the implicit prices researchers actually
16 −use; and \citet{cropper1988choice}, in an influential simulation study,
17 −found that simple forms --- linear and semi-log --- are the most robust to
18 −the misspecification and omitted variables that characterize real data.
19 −Fifty years on, the semi-log remains the profession's default
20 −\citep{malpezzi2003hedonic, sirmans2005composition}, but systematic
21 −head-to-head evidence on modern samples remains scarce, and the classic
22 −studies predate the out-of-sample validation culture. Two further
23 −technical strands feed our design: retransformation bias in log models
24 −\citep{duan1983smearing}, which we correct with the smearing factor so
25 −that every form competes in levels; and forecast-evaluation methodology
26 −for house prices \citep{clapp2002predicting, steurer2021metrics}.
9 +showed that the price surface is an equilibrium envelope of bid and
10 +offer functions, so nothing structural pins down its curvature: the form
11 +is an empirical choice. The classical exchange settled into a
12 +three-cornered debate. \citet{halvorsen1981choice} advocated letting the
13 +data choose the transformation through the Box--Cox family
14 +\citep{box1964analysis}; \citet{cassel1985cost} cautioned that flexible
15 +transformations optimise in-sample fit while degrading the implicit
16 +prices researchers actually use; and \citet{cropper1988choice}, in a
17 +simulation study that remains the standard reference, found that simple
18 +forms, linear and semi-log, are the most robust to the misspecification
19 +and omitted variables that characterise real data. The debate has been
20 +revisited for welfare measurement rather than prediction:
21 +\citet{kuminoff2010which} show in a Monte Carlo design calibrated to
22 +actual housing data that the choice of control variables (notably
23 +spatial fixed effects) matters more for recovering marginal willingness
24 +to pay than the choice of functional form, and \citet{bishop2020best}
25 +codify the resulting best practice. Fifty years on, the semi-log remains
26 +the profession's default \citep{malpezzi2003hedonic,
27 +sirmans2005composition, khoshnoud2023evolution}, but systematic
28 +head-to-head evidence on large modern samples remains scarce, and the
29 +classic studies predate the out-of-sample validation culture. Two
30 +technical strands feed our design. Retransformation bias in log models
31 +\citep{duan1983smearing, manning2001estimating} determines how
32 +predictions in logarithms are converted into price levels, a step that
33 +turns out to matter for the comparison between linear models and tree
34 +ensembles. Forecast-evaluation methodology, from the equal-accuracy tests
35 +of \citet{diebold1995comparing} and \citet{giacomini2006tests} to their
36 +application to house prices \citep{clapp2002predicting,
37 +steurer2021metrics}, supplies the inferential framework we use to decide
38 +whether differences on the scoreboard are real.
27 39
28 40 \subsection{Space and time in hedonic models}
29 41
30 42 A second literature concerns what the covariates cannot capture. House
31 43 prices are spatially autocorrelated at fine scale
32 −\citep{dubin1998spatial, anselin1988spatial}, and the practical remedies
33 −range from submarket segmentation \citep{goodman1998housing} to spatial
34 −econometrics and simple location fixed effects; comparative studies
35 −generally find that flexibly absorbing location buys more predictive
36 −accuracy than any parametric spatial process \citep{case2004modeling,
37 −bourassa2010predicting, pace1997spatial}. In the Quebec context,
44 +\citep{anselin1988spatial, dubin1998spatial, pace1998spatiotemporal}, and
45 +the practical remedies range from submarket segmentation
46 +\citep{goodman1998housing, goodman2003segmentation, bourassa2003submarkets}
47 +to spatial econometric processes, semiparametric location surfaces
48 +\citep{clapp2003semiparametric, fik2003modeling} and simple location
49 +fixed effects. Comparative studies generally find that flexibly
50 +absorbing location buys more predictive accuracy than any parametric
51 +spatial process, and that the gain depends on how submarkets are drawn
52 +\citep{pace1997spatial, case2004modeling, bourassa2007spatial,
53 +bourassa2010predicting}. \citet{pace2020residuals} show with tree
54 +ensembles that the residuals of standard hedonic and spatial models
55 +still contain exploitable spatial information, which is precisely the
56 +margin our coordinate-based learners target. In the Quebec context,
38 57 \citet{desrosiers2000hedonic} document the importance of access and
39 −neighbourhood attributes in Quebec City. On the time dimension, hedonic
40 −price-index theory --- the time-dummy versus imputation debate ---
41 −is surveyed by \citet{hill2013hedonic} and codified in the RPPI handbook
42 −\citep{oecd2013handbook, silver2018house}; a byproduct of our month-FE
43 −models is a direct test of how sensitive a constant-quality index is to
44 −the underlying form.
58 +neighbourhood attributes in Quebec City, and \citet{dube2013pooled}
59 +stress that sales pooled over time are not exchangeable: a sale can
60 +inform the price of a later neighbour but not of an earlier one, which
61 +is the asymmetry a forward-in-time holdout respects and a random split
62 +ignores. On the time dimension, the time-dummy versus imputation debate
63 +in hedonic price-index theory is surveyed by \citet{hill2013hedonic} and
64 +codified in the RPPI handbook \citep{oecd2013handbook, silver2018house};
65 +\citet{hill2009boom} and \citet{hill2018geospatial} show that imputation
66 +indices built from flexible, spline-based hedonic models track a boom
67 +and bust more faithfully, and \citet{diewert2015decomposition} and
68 +\citet{diewert2016tokyo} develop the hedonic decomposition of prices into
69 +land and structure components. A by-product of our month-effects models
70 +is a direct test of how sensitive a constant-quality index is to the
71 +underlying functional form and to the estimator.
45 72
46 73 \subsection{Machine learning and automated valuation}
47 74
48 −The third strand is the rapid colonization of mass valuation by machine
49 −learning. \citet{mullainathan2017machine} frame prediction as ML's
50 −comparative advantage, with hedonic pricing as their running example;
51 −\citet{athey2019machine} survey the toolkit. Applied comparisons ---
52 −random forests \citep{breiman2001random}, gradient boosting
53 −\citep{friedman2001greedy, ke2017lightgbm} --- consistently report 20--40\%
54 −error reductions over linear hedonics \citep{mayer2019estimation,
55 −hong2020machine, kok2017big}, with accuracy degrading in thin rural
56 −markets \citep{bogin2020house}; \citet{gencay1996nonlinear} anticipated
57 −the pattern with semiparametric estimators. Three gaps motivate our
58 −design. First, most comparisons change several ingredients at once
59 −(features, sample, validation scheme), so the \emph{sources} of the ML
60 −advantage --- curvature? interactions? spatial flexibility? --- are rarely
61 −isolated; our axis design holds features constant and varies one choice
62 −at a time. Second, almost all published comparisons use random
63 −cross-validation, which leaks future price levels into training and
64 −flatters any flexible model; we score every model under a
65 −forward-in-time split as well. Third, comparisons seldom examine what the
66 −flexible models \emph{imply} --- indices, implicit prices --- which is
67 −what economists need; our extensions section does exactly that.
75 +The third strand is the rapid adoption of machine learning in mass
76 +valuation. \citet{mullainathan2017machine} frame prediction as the
77 +comparative advantage of machine learning, with hedonic pricing as their
78 +running example; \citet{athey2019machine} survey the toolkit. The
79 +applied comparisons, from early neural networks and random forests
80 +\citep{zurada2011comparison, antipov2012mass, mccluskey2013prediction} to
81 +gradient boosting \citep{kok2017big, mayer2019estimation,
82 +perezrave2019machine, hong2020machine}, consistently report error
83 +reductions of 20--40\% over linear hedonics, with accuracy degrading in
84 +thin rural markets \citep{bogin2020house}; \citet{gencay1996nonlinear}
85 +anticipated the pattern with semiparametric estimators, and
86 +\citet{lorenz2023interpretable} show that the gains come with
87 +interpretable partial-dependence profiles that economists can read as
88 +implicit prices. \citet{mayer2019estimation} is the closest antecedent
89 +to our forward-in-time design: on Swiss data they compare estimation
90 +methods together with updating rules (moving windows, weighting) and
91 +find that the choice of updating matters as much as the estimator; we
92 +isolate the same question by scoring frozen models on a future period.
93 +Three gaps motivate our design. First, most comparisons change several
94 +ingredients at once (features, sample, validation scheme), so the sources
95 +of the machine-learning advantage---curvature, interactions, spatial
96 +flexibility---are rarely isolated; our axis design holds the features
97 +constant and varies one choice at a time
98 +\citep[compare the specification exercise of][]{schulz2014automated}.
99 +Second, almost all published comparisons use random cross-validation,
100 +which leaks the future price level into training, and, when properties
101 +sell repeatedly or share a building, lets a coordinate-based learner
102 +recall a property's own earlier price; the forecasting and ecology
103 +literatures have long documented how such temporal and spatial
104 +dependence inflates cross-validated accuracy
105 +\citep{bergmeir2012crossvalidation, roberts2017crossvalidation}. We score
106 +every model under a forward-in-time split and under parcel- and
107 +building-grouped splits as well. Third, comparisons seldom examine what
108 +the flexible models \emph{imply}---indices, implicit prices---which is
109 +what economists need; Section~\ref{sec:ext} does exactly that.
modified paper/sections/methodology.tex +264 −70
@@ -3,80 +3,274 @@
3 3 \section{Methodology: the design of the horse race}
4 4 \label{sec:method}
5 5
6 −The comparison is organised so that each design choice varies while
7 −everything else is held fixed. All models price the same sales from the
8 −same attribute set; only the functional mapping changes.
6 +The comparison is organised so that one design choice varies at a time
7 +while everything else is held fixed. All twenty models price the same
8 +sales from the same attribute vector; only the mapping from attributes to
9 +price changes. This section defines the models, the retransformation rule
10 +that puts them on a common scoreboard, the two holdouts and the inference
11 +procedures.
12 +
13 +\subsection{Notation and the baseline hedonic function}
14 +
15 +For sale $i$ let $P_i$ denote the price, $\mathbf{x}_i$ the vector of
16 +structural attributes supplied by the assessment roll (floor area $a_i$
17 +in m$^2$, lot area $l_i$ in m$^2$ with an indicator $h_i = \mathbb{1}[l_i
18 +> 0]$, building age $g_i$ in years, storeys, dwelling units, and dummies
19 +for property class and physical configuration), $m(i)$ the municipality,
20 +$c_5(i)$ and $c_1(i)$ the $\sim$5.5~km and $\sim$1.1~km grid cells
21 +containing the parcel, and $y(i)$, $q(i)$, $\tau(i)$ the year, quarter and
22 +month of sale. A linear hedonic model with a transformation $g(\cdot)$ of
23 +the price, a set $\mathcal{S}$ of spatial fixed effects and a set
24 +$\mathcal{T}$ of time fixed effects is
25 +\begin{equation}
26 + g(P_i) \;=\; \boldsymbol{\phi}(\mathbf{x}_i)'\boldsymbol{\beta}
27 + \;+\; \mu_{s(i)} \;+\; \delta_{t(i)} \;+\; \varepsilon_i,
28 + \qquad s(i)\in\mathcal{S},\; t(i)\in\mathcal{T},
29 + \label{eq:hedonic}
30 +\end{equation}
31 +where $\boldsymbol{\phi}(\cdot)$ is a basis expansion of the attributes
32 +(identity, logarithms, quadratics or splines), $\mu_s$ absorbs location
33 +and $\delta_t$ the market level. Following \citet{rosen1974hedonic},
34 +nothing in theory restricts $g$ or $\boldsymbol{\phi}$; the axes below
35 +make those choices explicit. The baseline, used as the backbone of
36 +axes~B and~C, takes $g = \ln$, quadratics in area and age,
37 +$\mathcal{S}$ = municipalities and $\mathcal{T}$ = quarters.
9 38
10 39 \subsection{The four axes}
11 40
12 41 \paragraph{A. Functional form.} With municipality and quarter fixed
13 −effects and the base attributes (floor area, lot area, building age,
14 −storeys, units, property class, physical configuration), we estimate:
15 −(A1) fully linear in levels; (A2) semi-log, $\ln P$ on levels
16 −\citep{malpezzi2003hedonic}; (A3) log-log, $\ln P$ on logged areas;
17 −(A4) Box--Cox on the price, with $\lambda$ chosen by profile likelihood
18 −on the training set over a grid \citep{box1964analysis,
19 −halvorsen1981choice}; (A5) semi-log with quadratics in area and age; and
20 −(A6) semi-log with cubic $B$-splines (six knots) in area, age and lot.
21 −
22 −\paragraph{B. Time effects.} On the A5 backbone: no time controls,
23 −year fixed effects, quarter fixed effects, month fixed effects ---
24 −the granularity ladder of the time-dummy index literature
25 −\citep{hill2013hedonic}.
26 −
27 −\paragraph{C. Spatial controls.} On the A5 backbone with quarter FE:
28 −no spatial controls, municipality FE (1{,}115), a $\sim$5.5~km grid
29 −(5{,}255 cells), and a $\sim$1.1~km grid (30{,}548 cells) --- a
30 −granularity ladder that brackets the bias--variance trade-off in
31 −absorbing location \citep{case2004modeling, bourassa2010predicting}.
32 −
33 −\paragraph{D. Estimation method.} Holding the attribute set fixed:
34 −(D1) OLS with municipality and month FE; (D2) cross-validated ridge on
35 −the same standardized design; (D3) a random forest
36 −\citep{breiman2001random} and (D4) gradient boosting
37 −\citep{friedman2001greedy} on the attributes plus raw coordinates and a
38 −continuous month counter; (D5) gradient boosting \emph{without}
39 −coordinates, to price what spatial flexibility is worth to the machine;
40 −and (D6) a spatial $k$-nearest-neighbour comparables rule --- the
41 −distance-weighted price per square metre of the ten nearest training
42 −sales --- as the na\"ive appraiser benchmark.
42 +effects held fixed, the six forms differ only in $g$ and
43 +$\boldsymbol{\phi}$:
44 +\begin{align}
45 + \text{A1 (linear):}\quad & P_i = \mathbf{x}_i'\boldsymbol{\beta}
46 + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a1}\\
47 + \text{A2 (semi-log):}\quad & \ln P_i = \mathbf{x}_i'\boldsymbol{\beta}
48 + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a2}\\
49 + \text{A3 (log-log):}\quad & \ln P_i = \beta_a \ln a_i
50 + + \beta_l\, h_i \ln l_i + \beta_h h_i + \tilde{\mathbf{x}}_i'\tilde{\boldsymbol{\beta}}
51 + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a3}\\
52 + \text{A4 (Box--Cox):}\quad & \frac{P_i^{\lambda}-1}{\lambda}
53 + = \mathbf{x}_i'\boldsymbol{\beta} + \mu_{m(i)} + \delta_{q(i)}
54 + + \varepsilon_i; \label{eq:a4}\\
55 + \text{A5 (quadratics):}\quad & \ln P_i = \mathbf{x}_i'\boldsymbol{\beta}
56 + + \beta_{aa}\Big(\tfrac{a_i}{100}\Big)^{2}
57 + + \beta_{gg}\Big(\tfrac{g_i}{10}\Big)^{2}
58 + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a5}\\
59 + \text{A6 (splines):}\quad & \ln P_i = \sum_{v\in\{a,l,g\}}
60 + \sum_{k=1}^{8} \beta_{vk}\, B_{vk}(v_i)
61 + + \tilde{\mathbf{x}}_i'\tilde{\boldsymbol{\beta}}
62 + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i, \label{eq:a6}
63 +\end{align}
64 +where $\tilde{\mathbf{x}}_i$ collects the remaining attributes and
65 +$B_{vk}$ are cubic $B$-spline basis functions with six uniformly spaced
66 +knots over the training range of each variable (eight basis functions
67 +per variable). In the Box--Cox model \citep{box1964analysis,
68 +halvorsen1981choice} the exponent is chosen on the training sample by
69 +maximising the profile log-likelihood
70 +\begin{equation}
71 + \ell(\lambda) \;=\; -\frac{n}{2}\ln\!\Big(\frac{\mathrm{SSE}(\lambda)}{n}\Big)
72 + \;+\; (\lambda - 1)\sum_{i=1}^{n}\ln P_i,
73 + \label{eq:profile}
74 +\end{equation}
75 +where $\mathrm{SSE}(\lambda)$ is the residual sum of squares of
76 +\eqref{eq:a4} at $\lambda$ and the second term is the Jacobian of the
77 +transformation. The main grid is $\lambda \in \{-0.5, -0.25, \dots, 1\}$;
78 +Section~\ref{sec:robust} refines it to steps of $0.05$.
79 +
80 +\paragraph{B. Time effects.} On the A5 backbone with municipality fixed
81 +effects, $\mathcal{T}$ runs through the granularity ladder of the
82 +time-dummy index literature \citep{hill2013hedonic}: no time controls
83 +(B1), year (B2), quarter (B3 $=$ A5) and month fixed effects (B4).
84 +
85 +\paragraph{C. Spatial controls.} On the A5 backbone with quarter fixed
86 +effects, $\mathcal{S}$ runs from no spatial controls (C1) to
87 +municipality fixed effects (C2 $=$ A5; 1{,}115 cells, median 79 sales
88 +per cell), a $0.05^{\circ}$ grid (C3; 5{,}255 cells of roughly
89 +5.5~km, median 10 sales) and a $0.01^{\circ}$ grid (C4; 30{,}548 cells of
90 +roughly 1.1~km, mean 17 and median 3 sales per cell). The ladder brackets
91 +the bias--variance trade-off in absorbing location
92 +\citep{case2004modeling, bourassa2010predicting}. Because the dummies
93 +are one-hot encoded with a dropped reference category, a test sale in a
94 +cell absent from the training set is priced at the reference cell's
95 +level; this affects 0.2\% of random-holdout sales under C3 and 2.5\%
96 +under C4. Section~\ref{sec:robust} reports nested variants (municipality
97 +plus grid effects) in which an unseen cell falls back on its
98 +municipality instead.
99 +
100 +\paragraph{D. Estimation method.} Holding the attribute set fixed, D1 is
101 +the semi-log quadratic OLS with municipality and month fixed effects
102 +(identical to B4); D2 replaces least squares by ridge regression on the
103 +same standardised design,
104 +\begin{equation}
105 + \hat{\boldsymbol{\beta}}_{\alpha} = \arg\min_{\boldsymbol{\beta}}
106 + \sum_i \big(\ln P_i - \mathbf{z}_i'\boldsymbol{\beta}\big)^2
107 + + \alpha \lVert \boldsymbol{\beta} \rVert_2^2,
108 + \qquad \alpha \in \{10^{-6}, 10^{-5.5}, \dots, 10^{2}\},
109 + \label{eq:ridge}
110 +\end{equation}
111 +with $\alpha$ selected by three-fold cross-validation within the training
112 +set. D3 is a random forest \citep{breiman2001random} and D4 a
113 +histogram-based gradient-boosting machine \citep{friedman2001greedy,
114 +ke2017lightgbm}, both fitted to $\ln P_i$ on the attribute vector
115 +augmented with the raw coordinates $(\mathrm{lat}_i, \mathrm{lng}_i)$
116 +and a continuous month counter $t_i$ (months since January 2021), with
117 +property class and physical configuration entered as integer codes. D5
118 +is the boosting machine without the coordinates, which prices what
119 +spatial flexibility is worth to the learner. D6 is a spatial
120 +$k$-nearest-neighbour comparables rule, the na\"ive appraiser benchmark:
121 +\begin{equation}
122 + \widehat{\ln P}_i \;=\; \ln a_i \;+\;
123 + \frac{\sum_{j\in N_{10}(i)} w_{ij}\,\big(\ln P_j - \ln a_j\big)}
124 + {\sum_{j\in N_{10}(i)} w_{ij}},
125 + \qquad w_{ij} = d_{ij}^{-1},
126 + \label{eq:knn}
127 +\end{equation}
128 +where $N_{10}(i)$ contains the ten training sales nearest to $i$ in
129 +kilometre-scaled coordinates and $d_{ij}$ is the planar distance.
130 +Hyper-parameters are fixed a priori at library-conventional values: the
131 +forest uses 120 trees, a minimum leaf size of 5 and half the features
132 +per split; the boosting machine uses 600 iterations, a learning rate of
133 +$0.08$, at most 63 leaves per tree, a minimum leaf size of 20 and an
134 +$\ell_2$ penalty of $0.01$. The ridge is the only method tuned on the
135 +data, and only within the training set. The exercise therefore compares
136 +model \emph{classes} as commonly used, not tuned frontier performance;
137 +Section~\ref{sec:robust} shows that the ranking is insensitive to
138 +materially different boosting configurations, including one tuned on a
139 +time-ordered validation set.
43 140
44 141 \subsection{One scoreboard: price levels}
142 +\label{sec:scoreboard}
143 +
144 +Comparing models whose dependent variables differ requires every model
145 +to be scored on the same object. We score \emph{predicted price levels}.
146 +Let $\hat{z}_i$ be the prediction of model~$M$ on the transformed scale
147 +($g(P)$ for the linear forms, $\ln P$ for the tree ensembles and the
148 +$k$-NN rule) on a test sale~$i$. Two inversion rules are available. The
149 +\emph{median rule} inverts the fitted value as is,
150 +\begin{equation}
151 + \hat{P}^{\mathrm{med}}_i = g^{-1}(\hat{z}_i)
152 + = \begin{cases} \exp(\hat{z}_i) & g = \ln,\\[2pt]
153 + (\lambda \hat{z}_i + 1)^{1/\lambda} & g = \text{Box--Cox},\end{cases}
154 + \label{eq:median}
155 +\end{equation}
156 +which under symmetric transformed-scale errors is the conditional median
157 +price and is the quantity the tree ensembles deliver natively. The
158 +\emph{mean rule} applies Duan's (\citeyear{duan1983smearing})
159 +nonparametric smearing estimator of the conditional mean,
160 +\begin{equation}
161 + \hat{P}^{\mathrm{mean}}_i = \frac{1}{n}\sum_{j=1}^{n}
162 + g^{-1}\big(\hat{z}_i + \hat{e}_j\big)
163 + \;\;\xrightarrow{\;g=\ln\;}\;\;
164 + \exp(\hat{z}_i)\cdot \hat{s}, \qquad
165 + \hat{s} = \frac{1}{n}\sum_{j=1}^{n} \exp(\hat{e}_j),
166 + \label{eq:duan}
167 +\end{equation}
168 +where $\hat{e}_j$ are the training residuals on the transformed scale.
169 +The smearing factor $\hat{s}$ exceeds one and corrects the downward bias
170 +of $\exp(\hat{z})$ as an estimator of $E[P\mid\mathbf{x}]$
171 +\citep{manning2001estimating}; for the Box--Cox model we evaluate
172 +\eqref{eq:duan} on a random subsample of 1{,}000 training residuals.
173 +The main scoreboard uses the median rule for every model, for two
174 +reasons. First, it is the only rule that treats all twenty models
175 +identically without a model-specific calibration constant: the training
176 +residuals of tree ensembles are shrunk by overfitting, so a smearing
177 +factor estimated from them is not comparable to one estimated from
178 +least-squares residuals. Second, our headline metric is a median error,
179 +whose natural target is the conditional median, and the log-scale
180 +metrics below are only meaningful on uncorrected log predictions.
181 +Section~\ref{sec:robust} reports the full scoreboard under the mean rule
182 +as well. Version~1.0 of this paper smeared the linear log models but not
183 +the tree ensembles; the asymmetry affected the comparison between the two
184 +families and is corrected here.
185 +
186 +Predictions are floored at \$20{,}000. With $\hat{P}_i$ the level
187 +prediction and $N$ the number of test sales, the four metrics are
188 +\begin{align}
189 + \mathrm{APE}_i &= \frac{\lvert \hat{P}_i - P_i\rvert}{P_i}, \qquad
190 + \mathrm{MdAPE} = \operatorname{median}_i \mathrm{APE}_i, \qquad
191 + \mathrm{MAPE} = \frac{1}{N}\sum_i \mathrm{APE}_i, \label{eq:ape}\\
192 + \mathrm{RMSE}_{\ln} &= \Big(\frac{1}{N}\sum_i
193 + \big(\ln P_i - \ln \hat{P}_i\big)^2\Big)^{1/2}, \qquad
194 + R^2_{\ln} = 1 - \frac{\sum_i (\ln P_i - \ln \hat{P}_i)^2}
195 + {\sum_i (\ln P_i - \overline{\ln P})^2}.
196 + \label{eq:r2}
197 +\end{align}
198 +MdAPE and MAPE are the accuracy standards of the automated-valuation
199 +industry and of ratio studies \citep{iaao2013standard,
200 +steurer2021metrics}; $R^2_{\ln}$ is an out-of-sample $R^2$ on the log
201 +scale, well defined for every model whatever its dependent variable.
202 +
203 +\subsection{Two holdouts, and leakage-free variants}
204 +\label{sec:holdouts}
205 +
206 +Each model is evaluated under two splits. The \emph{random} split assigns
207 +every sale independently to the test set with probability $0.2$ (seeded;
208 +411{,}283 training and 102{,}929 test sales). The \emph{forward-in-time}
209 +split trains on sales dated before 1~January 2025 (355{,}824) and tests
210 +on 2025--26 (158{,}388), the deployment situation of any valuation model
211 +that is estimated once and applied to subsequent sales. Static models
212 +carry no forecast of the price level, so we apply the standard
213 +carry-forward rule: a test sale whose time category (year, quarter or
214 +month) was never observed in training is priced at the last category
215 +observed in training, $\hat{\delta}_{t} = \hat{\delta}_{T}$ for $t > T$.
216 +The tree ensembles are piecewise constant in the month counter, so a
217 +test month beyond the training range is priced by the terminal leaf in
218 +$t$, the tree analogue of the same rule. Random cross-validation lets
219 +every model interpolate the price level of its own test period; the
220 +forward split reveals how much of measured accuracy depends on that
221 +interpolation \citep{bergmeir2012crossvalidation,
222 +roberts2017crossvalidation}.
223 +
224 +The random split has a second, less obvious leakage channel. One fifth of
225 +the sales in the sample are repeat sales of a parcel that sells again
226 +within the window, and 30\% share their exact geocoded coordinates with
227 +another sale, 92\% among condominiums, whose units are geocoded to the
228 +building. Under a random split, a model that reads the coordinates can
229 +learn a parcel's or a building's price from its other sales. We
230 +therefore also evaluate the full race under two grouped holdouts in
231 +which 20\% of \emph{parcels}, respectively 20\% of \emph{coordinate
232 +pairs}, are assigned to the test set, so that no property (respectively
233 +no building) appears on both sides
234 +\citep{roberts2017crossvalidation}.
235 +
236 +\subsection{Inference}
237 +\label{sec:inference}
238 +
239 +Differences of a few tenths of a point between models evaluated on the
240 +same 100{,}000 sales can be either meaningful or noise, and the sales are
241 +not independent: they cluster in municipalities whose local shocks move
242 +all errors together. We report two complementary tools. First, a
243 +\emph{cluster bootstrap} for MdAPE: municipalities are resampled with
244 +replacement ($B = 500$), each draw contributing all its test sales, and
245 +the median APE of every model and the difference between two models'
246 +MdAPEs are recomputed; we report percentile 95\% intervals. Second, a
247 +test of equal predictive accuracy in the spirit of
248 +\citet{diebold1995comparing} and \citet{giacomini2006tests}. For models
249 +$A$ and $B$ and a loss $L$, let $d_i = L(\hat{P}^A_i, P_i) -
250 +L(\hat{P}^B_i, P_i)$; the statistic
251 +\begin{equation}
252 + t_{\mathrm{DM}} = \frac{\bar{d}}{\widehat{\mathrm{se}}(\bar{d})},
253 + \qquad
254 + \widehat{\mathrm{se}}(\bar{d})^2 = \frac{G}{G-1}\,\frac{1}{N^2}
255 + \sum_{g=1}^{G}\Big(\sum_{i\in g}(d_i - \bar{d})\Big)^2,
256 + \label{eq:dm}
257 +\end{equation}
258 +uses standard errors clustered by municipality ($G = 1{,}115$), the
259 +cross-sectional analogue of the HAC variance in the time-series version.
260 +We use the absolute percentage error (the MAPE loss) and the squared log
261 +error (the RMSE$_{\ln}$ loss); a positive $\bar{d}$ means that $B$ is
262 +more accurate. The mean of the APE loss differential tests MAPE, not
263 +MdAPE; the bootstrap covers the median.
45 264
46 −Comparing forms whose dependent variables differ requires care. Every
47 −model is scored on \emph{predicted price levels}: log models are
48 −retransformed with Duan's smearing factor estimated on training
49 −residuals \citep{duan1983smearing}; Box--Cox predictions are analytically
50 −inverted; level predictions are floored at \$20{,}000. We report the
51 −median and mean absolute percentage errors (MdAPE, MAPE) --- the
52 −automated-valuation industry's standards \citep{steurer2021metrics,
53 −iaao2013standard} --- together with the RMSE and $R^2$ of log predicted
54 −versus log actual prices, which are well-defined for every model.
55 −
56 −\subsection{Two holdouts}
57 −
58 −Each model is evaluated under two splits. The \emph{random} split holds
59 −out 20\% of sales (seeded). The \emph{forward-in-time} split trains on
60 −sales before January 2025 (355{,}824) and tests on 2025--2026
61 −(158{,}388) --- the deployment situation of any valuation model. Static
62 −models carry no forecast of the price level, so we apply the standard
63 −carry-forward rule: unseen time categories in the test period are priced
64 −at the last level observed in training; the machine-learning models
65 −receive the same information through the clamped month counter. Random
66 −cross-validation lets every model interpolate the price level of its own
67 −test period --- the forward split reveals how much of measured accuracy
68 −is such leakage \citep{mullainathan2017machine, clapp2002predicting}.
69 −
70 −\subsection{Estimation details}
71 −
72 −All linear models are estimated as sparse ridge regressions with a
73 −vanishing penalty ($\alpha = 10^{-6}$) --- numerically OLS, but robust to
74 −collinear dummies, and unseen categories at prediction time are priced at
75 −the reference level. Dummies are one-hot with a dropped reference; the
76 −ridge (D2) standardizes columns and cross-validates
77 −$\alpha \in [10^{-6}, 10^{2}]$. The forest uses 120 trees
78 −(minimum leaf 5, half the features per split); the boosting machine uses
79 −600 trees of at most 63 leaves (learning rate 0.08, minimum leaf 20).
80 −Hyper-parameters were fixed a priori at library-conventional values ---
81 −the exercise compares model \emph{classes} as used in practice, not tuned
82 −frontier performance.
265 +Finally, because the forward-in-time holdout confounds the difficulty of
266 +the test period with the staleness of a frozen model, we decompose
267 +forward error into an aggregate level error and relative pricing skill.
268 +For each model, $\hat{b} = \operatorname{median}_i(\ln P_i - \ln
269 +\hat{P}_i)$ on the test set measures the common level error; the
270 +\emph{re-levelled} MdAPE, computed with $\hat{P}_i e^{\hat{b}}$, is the
271 +accuracy the model would attain if the aggregate price level of the test
272 +period were known, the time adjustment practitioners obtain from a
273 +published index \citep{thibodeau2003marking}. We also compare the
274 +random-split models on the 2025--26 sales of their own test set with the
275 +forward-split models on the full 2025--26 test set, which holds the test
276 +period fixed and varies only whether that period was seen in training.
modified paper/sections/results.tex +282 −86
@@ -4,9 +4,15 @@
4 4 \label{sec:results}
5 5
6 6 Table~\ref{tab:race_random} reports the full scoreboard under the random
7 −holdout (411{,}283 training sales, 102{,}929 test sales);
8 −Table~\ref{tab:race_temporal} repeats it under the forward-in-time split.
9 −Figures~\ref{fig:forms}--\ref{fig:methods} visualize each axis.
7 +holdout (411{,}283 training sales, 102{,}929 test sales) and
8 +Table~\ref{tab:race_temporal} under the forward-in-time split;
9 +Table~\ref{tab:inference} adds the cluster-bootstrap intervals and the
10 +equal-accuracy tests against the spline hedonic (A6), and
11 +Table~\ref{tab:pairs} the targeted comparisons along each axis.
12 +Figures~\ref{fig:forms}--\ref{fig:methods} visualise the four axes under
13 +the random holdout. All numbers use the median rule of
14 +Section~\ref{sec:scoreboard}; Section~\ref{sec:robust} shows how the
15 +Duan-smeared alternative moves them.
10 16
11 17 \begin{table}[p]
12 18 \centering
@@ -17,11 +23,12 @@ Figures~\ref{fig:forms}--\ref{fig:methods} visualize each axis.
17 23 \input{../results/tables/horserace_random}
18 24 \begin{tablenotes}[flushleft]\footnotesize
19 25 \item \textit{Notes:} All metrics on the held-out 20\% (102{,}929 sales).
20 −MdAPE/MAPE: median/mean absolute percentage error on price levels (log
21 −models retransformed with Duan smearing; Box--Cox inverted).
22 −RMSE$_{\ln}$ and $R^2_{\ln}$: computed on log predicted vs.\ log actual
23 −prices. Backbone rows repeat across axes by construction (A5 $=$ B3 $=$
24 −C2; B4 $=$ D1). Bold: best in axis.
26 +MdAPE/MAPE: median/mean absolute percentage error on price levels,
27 +eq.~\eqref{eq:ape}; transformed-scale predictions inverted with the
28 +median rule, eq.~\eqref{eq:median}. RMSE$_{\ln}$ and $R^2_{\ln}$:
29 +eq.~\eqref{eq:r2}. Fit: wall-clock seconds on an 18-core workstation.
30 +Backbone rows repeat across axes by construction (A5 $=$ B3 $=$ C2;
31 +B4 $=$ D1). Bold: lowest MdAPE within the axis.
25 32 \end{tablenotes}
26 33 \end{threeparttable}
27 34 \end{table}
@@ -35,31 +42,93 @@ C2; B4 $=$ D1). Bold: best in axis.
35 42 \input{../results/tables/horserace_temporal}
36 43 \begin{tablenotes}[flushleft]\footnotesize
37 44 \item \textit{Notes:} Training on sales before January 2025 (355{,}824);
38 −testing on 2025--2026 (158{,}388). Unseen time categories are priced at
39 −the last training period (carry-forward); the machine-learning models
40 −receive the same information through the clamped month counter.
45 +testing on 2025--26 (158{,}388). Unseen time categories are priced at
46 +the last training period (carry-forward); the tree ensembles are
47 +piecewise constant in the month counter and therefore apply the same
48 +rule through their terminal leaf. Other conventions as in
49 +Table~\ref{tab:race_random}.
50 +\end{tablenotes}
51 +\end{threeparttable}
52 +\end{table}
53 +
54 +\begin{table}[p]
55 +\centering
56 +\begin{threeparttable}
57 +\caption{Inference: bootstrap intervals and equal-accuracy tests against the spline hedonic}
58 +\label{tab:inference}
59 +\scriptsize
60 +\setlength{\tabcolsep}{3.5pt}
61 +\input{../results/tables/inference}
62 +\begin{tablenotes}[flushleft]\footnotesize
63 +\item \textit{Notes:} MdAPE in percent with a 95\% percentile interval
64 +from 500 cluster-bootstrap replicates (municipalities resampled with
65 +replacement). $\Delta$ vs.\ A6: MdAPE of the row model minus MdAPE of
66 +A6 on the same test sales, with its bootstrap interval; negative values
67 +favour the row model. $t_{\mathrm{DM}}$: clustered Diebold--Mariano
68 +statistic, eq.~\eqref{eq:dm}, for the absolute-percentage-error loss
69 +differential (row model minus A6); positive values mean A6 is more
70 +accurate on the mean loss. $^{*}$, $^{**}$, $^{***}$: $p < 0.10$,
71 +$0.05$, $0.01$ (two-sided). 1{,}115 municipality clusters.
72 +\end{tablenotes}
73 +\end{threeparttable}
74 +\end{table}
75 +
76 +\begin{table}[tbp]
77 +\centering
78 +\begin{threeparttable}
79 +\caption{Targeted comparisons along each axis}
80 +\label{tab:pairs}
81 +\scriptsize
82 +\setlength{\tabcolsep}{2.5pt}
83 +\input{../results/tables/pairs}
84 +\begin{tablenotes}[flushleft]\footnotesize
85 +\item \textit{Notes:} For each pair, the MdAPE difference (first model
86 +minus second) with its 95\% cluster-bootstrap interval and the clustered
87 +DM statistics for the APE and squared-log losses (positive: the second
88 +model is more accurate on the mean loss). Stars as in
89 +Table~\ref{tab:inference}.
41 90 \end{tablenotes}
42 91 \end{threeparttable}
43 92 \end{table}
44 93
45 94 \subsection{Axis A: functional form is second-order}
46 95
47 −Under the random split (Figure~\ref{fig:forms}), the six forms span
48 −16.5--18.6\% MdAPE. The spline specification wins (16.5\%), the log-log
49 −is second (17.0\%) --- floor area enters multiplicatively, as intuition
50 −suggests --- and the linear, semi-log and quadratic forms cluster within
51 −half a point of each other. The Box--Cox transformation selects
52 −$\hat\lambda = 0.25$ by profile likelihood (Table~\ref{tab:boxcox}), i.e.\
53 −the data reject both the linear ($\lambda=1$) and the log ($\lambda=0$)
54 −--- and then \emph{underperforms} the simple semi-log out of sample
55 −(18.6\%), because optimizing curvature in-sample does not survive
56 −prediction, precisely the warning of \citet{cassel1985cost} and
57 −\citet{cropper1988choice}. The ordering is preserved under the temporal
58 −split. Two free lessons: curvature in \emph{attributes} (splines) is
59 −worth more than curvature in the \emph{price} (Box--Cox); and no form
60 −choice moves the needle by more than two points.
61 −
62 −\begin{figure}[t]
96 +Under the random split (Figure~\ref{fig:forms}) the six forms span
97 +17.6--19.2\% MdAPE, a range of 1.5 points. The spline specification wins
98 +(17.6\%), the log-log is second (18.0\%)---floor area enters
99 +multiplicatively, as intuition suggests---and the linear (18.2\%),
100 +Box--Cox (18.6\%), semi-log (19.0\%) and quadratic (19.2\%) forms follow
101 +within a point of each other. The spline gain over the textbook
102 +semi-log, $-1.4$ points [$-1.8$, $-1.1$], is precisely estimated and
103 +holds on every metric (Table~\ref{tab:pairs}). The Box--Cox
104 +transformation selects $\hat\lambda = 0.25$ by profile likelihood
105 +(Table~\ref{tab:boxcox}), so the data reject both the linear
106 +($\lambda = 1$) and the logarithmic ($\lambda = 0$) specification, yet
107 +the model gains only half a point on the semi-log's median error, loses
108 +on the mean error (DM $t = 9.0$ in favour of the semi-log) and remains a
109 +full point behind the splines. Optimising the curvature of the
110 +\emph{price} in sample buys little out of sample, the warning of
111 +\citet{cassel1985cost} and \citet{cropper1988choice}; curvature in the
112 +\emph{attributes}, which splines supply cheaply, is worth more.
113 +
114 +Under the forward-in-time split (Table~\ref{tab:race_temporal}) the
115 +range widens to 20.4--23.3\% and the ordering changes: the linear model
116 +in levels posts the lowest median error (20.4\%), followed by the
117 +Box--Cox (21.0\%), spline (21.7\%) and log-log (21.8\%) forms, with the
118 +semi-log and quadratic forms last (23.2--23.3\%). The linear model's
119 +forward-split advantage is a bias coincidence rather than a property of
120 +the form: being a conditional-mean predictor in levels, its log
121 +prediction over-predicts the typical sale by about 5\% under the random
122 +split (median log residual $+0.004$ against $+0.06$ for the log models),
123 +and in a rising market that upward tilt offsets part of the staleness of
124 +a frozen price level. Once the common level error is removed
125 +(Section~\ref{sec:gap}), the spline form is again the most accurate
126 +linear specification (16.8\% against 18.0\% for the linear model).
127 +Two lessons survive both splits: no functional-form choice moves the
128 +median error by more than about two points, and splines dominate the
129 +semi-log everywhere.
130 +
131 +\begin{figure}[tbp]
63 132 \centering
64 133 \includegraphics[width=0.85\textwidth]{fig_forms.png}
65 134 \caption{Axis A --- functional form. MdAPE on the random holdout;
@@ -68,7 +137,7 @@ fixed throughout.}
68 137 \label{fig:forms}
69 138 \end{figure}
70 139
71 −\begin{table}[t]
140 +\begin{table}[tbp]
72 141 \centering
73 142 \begin{threeparttable}
74 143 \caption{Box--Cox profile likelihood}
@@ -76,26 +145,36 @@ fixed throughout.}
76 145 \small
77 146 \input{../results/tables/boxcox}
78 147 \begin{tablenotes}[flushleft]\footnotesize
79 −\item \textit{Notes:} $^{\dagger}$Thousands, relative to the maximum
80 −(zero = chosen $\lambda$). Estimated on the random-split training sample
81 −with the A-axis design.
148 +\item \textit{Notes:} $^{\dagger}$Profile log-likelihood,
149 +eq.~\eqref{eq:profile}, in thousands relative to the maximum (zero at
150 +the chosen $\lambda$). Estimated on the random-split training sample
151 +with the axis-A design. A finer grid (steps of $0.05$,
152 +Section~\ref{sec:robust}) selects $\hat\lambda = 0.35$ and lowers the
153 +random-holdout MdAPE from 18.6\% to 18.4\%.
82 154 \end{tablenotes}
83 155 \end{threeparttable}
84 156 \end{table}
85 157
86 158 \subsection{Axis B: time controls --- anything beats nothing}
87 159
88 −Omitting time controls entirely costs three points (21.1\% versus 18.1\%)
89 −in a window containing a boom, a correction and a recovery
90 −(Figure~\ref{fig:time}). But the granularity ladder flattens immediately:
91 −year, quarter and month effects are indistinguishable to within a quarter
92 −point under the random split. Under the temporal split the ordering
93 −inverts at the bottom --- \emph{no} time effects (25.4\%) is the worst
94 −choice again, but month effects with carry-forward (20.3\%) beat quarter
95 −effects (20.5\%) only marginally: once the level must be extrapolated,
96 −fine within-sample time resolution has little to offer.
97 −
98 −\begin{figure}[t]
160 +Omitting time controls entirely costs three points (22.0\% against
161 +19.2\%, difference $-2.9$ [$-3.8$, $-2.1$] for year effects against
162 +none) in a window containing a boom, a correction and a recovery
163 +(Figure~\ref{fig:time}). The granularity ladder then flattens at once:
164 +year, quarter and month effects lie within 0.2 points of each other
165 +under the random split, and month effects are, if anything, marginally
166 +worse than quarter effects ($+0.19$ [$+0.01$, $+0.37$]), the first sign
167 +that finer within-sample time resolution buys nothing for prediction.
168 +Under the forward-in-time split the cost of omitting time controls rises
169 +to six points (29.3\% against 23.3\%), because a model without time
170 +effects prices 2025--26 sales at the \emph{average} 2021--24 level rather
171 +than at the last observed level, and month effects with carry-forward
172 +(22.9\%) beat quarter effects (23.3\%) by a statistically fragile margin
173 +($-0.43$ [$-0.95$, $+0.05$]). Once the level must be extrapolated, fine
174 +time resolution has little to offer; what matters is that the carried
175 +level be the most recent one.
176 +
177 +\begin{figure}[tbp]
99 178 \centering
100 179 \includegraphics[width=0.85\textwidth]{fig_time.png}
101 180 \caption{Axis B --- time effects (quadratic semi-log backbone,
@@ -106,18 +185,31 @@ municipality FE). MdAPE on the random holdout.}
106 185 \subsection{Axis C: spatial controls are first-order --- up to a point}
107 186
108 187 Location dominates every other design margin
109 −(Figure~\ref{fig:space}). Stripping all spatial controls costs
110 −\emph{ten} points (27.9\% versus 17.5\%) --- five times the entire
188 +(Figure~\ref{fig:space}). Stripping all spatial controls costs ten
189 +points (28.8\% against 19.2\% with municipality effects, 11.2 points
190 +[10.1, 12.3] relative to the spline hedonic), six times the entire
111 191 functional-form range. The granularity ladder, however, is
112 −non-monotonic: the $\sim$5.5~km grid (5{,}255 cells, 17.5\%) beats
113 −municipality effects (18.1\%), but the $\sim$1.1~km grid (30{,}548
114 −cells, 19.4\%) \emph{overfits} --- its cells average seventeen K sales
115 −each, unseen cells in the holdout revert to the baseline, and estimated
116 −effects are noisy. The optimal spatial resolution is an interior
117 −solution, echoing \citet{goodman1998housing} on submarket definition:
118 −finer is better only while cells remain thick enough to estimate.
119 −
120 −\begin{figure}[t]
192 +non-monotonic. The $\sim$5.5~km grid (5{,}255 cells) is nominally better
193 +than municipality effects (18.7\% against 19.2\%), but the difference,
194 +$-0.45$ [$-1.07$, $+0.37$], is within bootstrap noise and the DM tests
195 +do not reject equal accuracy; the $\sim$1.1~km grid (30{,}548 cells)
196 +is clearly worse ($+1.8$ [$+1.2$, $+2.3$]). Its cells contain 17 sales on
197 +average and 3 at the median, so most cell effects are estimated from a
198 +handful of sales, and the 2.5\% of test sales that fall in a cell with
199 +no training sale are priced at an arbitrary reference cell. Under the
200 +forward-in-time split the ordering becomes monotonic in the wrong
201 +direction: finer cells are worse at every step (23.3\%, 24.3\%, 25.3\%),
202 +because a fine spatial configuration estimated on 2021--24 prices
203 +transfers less well than a coarse one. The optimal spatial resolution is
204 +an interior solution that depends on market thickness, echoing
205 +\citet{goodman1998housing} and \citet{bourassa2007spatial} on submarket
206 +definition: finer is better only while cells remain thick enough to
207 +estimate, and the relevant thickness is larger when the model must be
208 +carried forward in time. Section~\ref{sec:robust} shows that nesting the
209 +grid within municipalities, so that an unseen cell falls back on its
210 +municipality, repairs part but not all of the fine grid's deficit.
211 +
212 +\begin{figure}[tbp]
121 213 \centering
122 214 \includegraphics[width=0.85\textwidth]{fig_space.png}
123 215 \caption{Axis C --- spatial controls (quadratic semi-log backbone,
@@ -127,22 +219,33 @@ quarter FE). MdAPE on the random holdout.}
127 219
128 220 \subsection{Axis D: the machines --- and what they actually buy}
129 221
130 −Under the random split (Figure~\ref{fig:methods}), gradient boosting
131 −posts 14.4\% and the random forest 14.6\%, against 16.5\% for the best
132 −linear model and 18.3\% for the textbook OLS: a 13--21\% error
133 −reduction, squarely in the range reported by the AVM literature
134 −\citep{mayer2019estimation, hong2020machine}. The decomposition rows
135 −identify the source. Boosting \emph{without coordinates} collapses to
136 −23.2\% --- worse than plain OLS with municipality dummies --- so the
137 −machine's edge is overwhelmingly a flexible surface over space, not
138 −exotic structure--attribute interactions. Pure proximity without
139 −structure (the $k$-NN comparables rule, 21.2\%) fails too: it is the
140 −\emph{combination} of spatial flexibility with attribute adjustment that
141 −wins. The cross-validated ridge (16.7\%) confirms that regularizing the
142 −high-dimensional dummy design buys a little; it cannot manufacture the
143 −missing interactions.
144 −
145 −\begin{figure}[t]
222 +Under the random split (Figure~\ref{fig:methods}) gradient boosting
223 +posts 14.4\% [13.7, 15.7] and the random forest 14.6\%, against 17.6\%
224 +for the best linear model and 19.3\% for the textbook OLS with
225 +municipality and month effects: an 18--25\% reduction in the median
226 +error, squarely in the range reported by the automated-valuation
227 +literature \citep{mayer2019estimation, hong2020machine,
228 +perezrave2019machine}. The boosting advantage over the spline hedonic,
229 +$-3.2$ points [$-4.7$, $-1.9$], is statistically unambiguous on every
230 +metric (DM $t = -3.6$ on the APE loss, $-3.0$ on the squared-log loss),
231 +and boosting edges the forest by a quarter point ($-0.24$ [$-0.46$,
232 +$-0.05$]). The decomposition rows identify the source. Boosting
233 +\emph{without coordinates} collapses to 23.2\%, worse than plain OLS
234 +with municipality dummies, so the learner's edge is overwhelmingly a
235 +flexible surface over space rather than exotic structure--attribute
236 +interactions, consistent with the residual-information results of
237 +\citet{pace2020residuals}. Pure proximity without structure fails too:
238 +the $k$-NN comparables rule (21.2\%) is three and a half points worse
239 +than the spline hedonic. It is the \emph{combination} of spatial
240 +flexibility with attribute adjustment that wins. The cross-validated
241 +ridge (17.7\%) is a further lesson: on the identical design, shrinking
242 +the 1{,}115 municipality effects and the month effects toward zero
243 +improves on OLS by 1.7 points [1.1, 2.4] and ties the spline hedonic
244 +(difference $+0.06$ [$-0.02$, $+0.16$]). Regularising the thin
245 +fixed effects is as valuable as adding curvature; neither manufactures
246 +the spatial surface the trees learn.
247 +
248 +\begin{figure}[tbp]
146 249 \centering
147 250 \includegraphics[width=0.85\textwidth]{fig_methods.png}
148 251 \caption{Axis D --- estimation methods on the identical attribute set.
@@ -151,25 +254,32 @@ MdAPE on the random holdout.}
151 254 \end{figure}
152 255
153 256 \subsection{The generalization gap: when the test set is the future}
257 +\label{sec:gap}
154 258
155 259 Table~\ref{tab:generalization} and Figure~\ref{fig:generalization}
156 260 contain the paper's central result. Moving from the random to the
157 −forward-in-time split degrades every model --- the price level of
158 −2025--26 must be carried forward, not interpolated --- but the
159 −degradation is systematically larger for the flexible methods.
160 −Gradient boosting loses 6.8 points of MdAPE (14.4 $\to$ 21.2) and the
161 −random forest 6.7, while the spline hedonic loses 2.6 (16.5 $\to$ 19.1)
162 −and the log-log 2.2. The ranking \emph{reverses}: the best linear models
163 −beat the machines on the median error in the deployment scenario, and
164 −tie them exactly on $R^2_{\ln}$ ($\approx 0.52$). The machines remain
165 −better on MAPE (29.7 versus 30.5), i.e.\ in the tails, but the headline
166 −random-validation gap --- the number the ML-valuation literature
167 −reports --- overstates their deployable advantage entirely. The lesson
168 −is methodological and general: for models whose job is to price the
169 −future, random cross-validation is the wrong experiment
170 −\citep{mullainathan2017machine, clapp2002predicting}.
171 −
172 −\begin{table}[t]
261 +forward-in-time split degrades every model---the price level of 2025--26
262 +must be carried forward, not interpolated---but the degradation is
263 +systematically larger for the flexible methods. Gradient boosting loses
264 +6.8 points of MdAPE (14.4 $\to$ 21.2) and the random forest 6.7, while
265 +the spline hedonic loses 4.1 (17.6 $\to$ 21.7), the ridge 4.6 and the
266 +linear model in levels 2.1. The ranking compresses: the five best
267 +forward-split models, linear (20.4\%), Box--Cox (21.0\%), boosting
268 +(21.2\%), forest (21.3\%) and splines (21.7\%), lie within 1.3 points
269 +and are statistically indistinguishable on the median error---boosting
270 +against splines gives $-0.51$ [$-1.68$, $+0.55$], boosting against the
271 +linear model $+0.84$ [$-1.19$, $+2.59$]. The ensembles keep a clear lead
272 +on the mean error (MAPE 29.7\% against 30.9\% for splines and 32.9\% for
273 +the linear model; DM $t = -3.5$ against the linear model) and on the log
274 +scale ($R^2_{\ln}$ 0.52 against 0.49 and 0.46). Version~1.0 of this paper
275 +reported that the ranking \emph{reversed}; that reversal was an artefact
276 +of smearing the linear log models but not the tree ensembles
277 +(Section~\ref{sec:robust}). The corrected statement is weaker but still
278 +consequential: the three-point advantage that random validation
279 +attributes to gradient boosting shrinks to half a point, and to
280 +insignificance, when the test set lies in the future.
281 +
282 +\begin{table}[tbp]
173 283 \centering
174 284 \begin{threeparttable}
175 285 \caption{Random versus forward-in-time evaluation}
@@ -183,7 +293,7 @@ under both splits.
183 293 \end{threeparttable}
184 294 \end{table}
185 295
186 −\begin{figure}[t]
296 +\begin{figure}[tbp]
187 297 \centering
188 298 \includegraphics[width=0.85\textwidth]{fig_generalization.png}
189 299 \caption{The generalization gap. MdAPE under the random holdout (filled
@@ -191,3 +301,89 @@ circles) and the forward-in-time holdout (open squares); the grey
191 301 segment joins the two evaluations of the same model.}
192 302 \label{fig:generalization}
193 303 \end{figure}
304 +
305 +Where does the gap come from? Three further cuts of the data answer the
306 +question (Table~\ref{tab:same_period}). First, holding the test period
307 +fixed: evaluated on the 31{,}465 sales of 2025--26 that happen to fall in
308 +the random test set, the random-split boosting model posts 13.7\%, so
309 +the 2025--26 sales are not intrinsically harder to price; the entire
310 +degradation to 21.2\% comes from not having seen their period. The
311 +\emph{staleness cost}, defined as the forward-split minus the
312 +same-period random-split MdAPE, is 7.5 points for boosting and 7.5 for
313 +the forest, against 4.7 for the spline hedonic, 5.2 for the ridge and
314 +3.3 for the linear model. Second, the level decomposition of
315 +Section~\ref{sec:inference}: the median log residual of every
316 +forward-split model is positive and large---$+0.16$ for the spline
317 +hedonic, $+0.18$ for the two ensembles, $+0.10$ for the linear
318 +model---so every frozen model under-prices the typical 2025--26 sale by
319 +10--20\%, the ensembles slightly more because their terminal leaf in the
320 +month counter averages the last few training months rather than the last
321 +one. Once each model's predictions are re-levelled by its own median
322 +ratio, the forward-split MdAPE falls to 13.9\% for boosting and 16.8\%
323 +for the spline hedonic: a gap of 2.9 points, essentially the 3.5-point
324 +gap measured under the random split with the same correction
325 +(13.0\% against 16.5\%). Third, the horizon profile
326 +(Table~\ref{tab:by_quarter}, Figure~\ref{fig:horizon}): in the first
327 +quarter after the training window boosting still leads the spline
328 +hedonic by 1.5 points (17.1\% against 18.6\%); the lead shrinks every
329 +quarter and vanishes by mid-2026, when both models post about 25\%.
330 +
331 +The picture is therefore not that the learners have memorised something
332 +that fails to transfer. Their \emph{relative} pricing skill---which
333 +sales are dear and which are cheap within a period---transfers forward
334 +almost intact. What does not transfer is the \emph{level}, and a frozen
335 +boosted model carries a slightly staler level than a frozen linear model
336 +with carry-forward month effects. The forward-in-time holdout thus
337 +quantifies two things that random cross-validation hides: a level
338 +staleness of 3--7 points of median error within eighteen months, common
339 +to every static model and largest for the trees, and the fact that the
340 +headline gap between learners and linear hedonics is a within-period
341 +statement. For models whose job is to price the future, random
342 +cross-validation is the wrong experiment \citep{bergmeir2012crossvalidation,
343 +mullainathan2017machine}; the right one separates the level, which an
344 +index or a retraining schedule must supply, from the relative accuracy
345 +the estimator contributes.
346 +
347 +\begin{table}[tbp]
348 +\centering
349 +\begin{threeparttable}
350 +\caption{Decomposing the generalization gap: same-period comparison and level error}
351 +\label{tab:same_period}
352 +\footnotesize
353 +\setlength{\tabcolsep}{4pt}
354 +\input{../results/tables/same_period}
355 +\begin{tablenotes}[flushleft]\footnotesize
356 +\item \textit{Notes:} MdAPE in percent. Columns 2--3: models estimated on
357 +the random-split training sample, evaluated on all random test sales and
358 +on the 2025--26 subset of those sales. Column 4: models estimated on
359 +sales before 2025 and evaluated on all 2025--26 sales. Staleness cost:
360 +column 4 minus column 3. Column 5: median of $\ln P_i - \ln\hat{P}_i$ on
361 +the forward test set (common level error). Column 6: forward-split MdAPE
362 +after multiplying each model's predictions by $e^{\hat{b}}$
363 +(Section~\ref{sec:inference}).
364 +\end{tablenotes}
365 +\end{threeparttable}
366 +\end{table}
367 +
368 +\begin{table}[tbp]
369 +\centering
370 +\begin{threeparttable}
371 +\caption{Forward-in-time accuracy by test quarter}
372 +\label{tab:by_quarter}
373 +\small
374 +\input{../results/tables/by_quarter}
375 +\begin{tablenotes}[flushleft]\footnotesize
376 +\item \textit{Notes:} MdAPE (\%) of models trained on sales before
377 +January 2025, by quarter of the test sale. 2026Q3 covers July 2026
378 +only.
379 +\end{tablenotes}
380 +\end{threeparttable}
381 +\end{table}
382 +
383 +\begin{figure}[tbp]
384 +\centering
385 +\includegraphics[width=0.8\textwidth]{fig_horizon.png}
386 +\caption{Forward-in-time accuracy by horizon. MdAPE by quarter of the
387 +test sale for models trained on sales before January 2025.}
388 +\label{fig:horizon}
389 +\end{figure}
added paper/sections/robustness.tex +216 −0
@@ -0,0 +1,216 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% ============================================================================
3 +\section{Robustness}
4 +\label{sec:robust}
5 +
6 +This section examines the five choices in the design that a sceptical
7 +reader would most want varied: the rule that converts transformed-scale
8 +predictions into price levels, the leakage channels of the random
9 +holdout, the implementation of fine spatial effects, the resolution of
10 +the Box--Cox grid and the hyper-parameters of the boosting machine.
11 +
12 +\subsection{The retransformation rule}
13 +
14 +Table~\ref{tab:retransformation} reports the scoreboard for the
15 +transformed-scale models under the median rule used throughout
16 +(eq.~\ref{eq:median}) and under Duan's mean rule
17 +(eq.~\ref{eq:duan}). The smearing factors $\hat{s}$ lie between 1.06
18 +and 1.07 for the least-squares models and between 1.03 and 1.05 for the
19 +ensembles, whose in-sample residuals are shrunk by overfitting. Two
20 +regularities stand out. First, the mean rule lowers the median error of
21 +\emph{every} log-scale model under the random split, by about one point
22 +for the linear models and 1.3 points for the ensembles. This is not what
23 +a conditional-mean correction should do to a median-based metric; it
24 +happens because the log residuals are left-skewed (skewness $-1.4$ to
25 +$-1.8$), so that their median is $+0.06$ while their mean is zero, and
26 +$\exp(\hat{z})$ under-prices the typical sale by about 6\%---which is,
27 +by coincidence, the size of the smearing factor. Second, under the
28 +forward-in-time split the mean rule lowers median errors by 2.5--3
29 +points for all models, because the market rose after the training
30 +window and any upward factor offsets part of the staleness. Neither
31 +effect changes a ranking when the rule is applied uniformly: boosting
32 +leads the spline hedonic by 3.4 points under the random split and by
33 +0.7 points under the forward split with the mean rule, against 3.2 and
34 +0.5 with the median rule. What does change a ranking is applying the
35 +rule to one family only. Version~1.0 of this paper smeared the linear
36 +log models and not the ensembles; the result was a forward-split
37 +comparison of 19.1\% for the smeared spline hedonic against 21.2\% for
38 +the unsmeared boosting model, reported as a reversal. The comparable
39 +numbers are 21.7\% against 21.2\% (median rule) or 19.1\% against
40 +18.4\% (mean rule). The log-scale metrics are affected in the same
41 +direction: smearing adds $\ln\hat{s} \approx 0.06$ to every log
42 +prediction, which inflates RMSE$_{\ln}$ and depresses $R^2_{\ln}$ for
43 +the smeared family only, so that the ``identical $R^2_{\ln}$'' of the
44 +first version (0.52 for both) becomes 0.49 against 0.52 in favour of
45 +boosting under the uniform rule.
46 +
47 +\begin{table}[tbp]
48 +\centering
49 +\begin{threeparttable}
50 +\caption{Median rule versus Duan smearing}
51 +\label{tab:retransformation}
52 +\scriptsize
53 +\setlength{\tabcolsep}{4pt}
54 +\input{../results/tables/retransformation}
55 +\begin{tablenotes}[flushleft]\footnotesize
56 +\item \textit{Notes:} Median: transformed-scale prediction inverted as
57 +is, eq.~\eqref{eq:median} (main scoreboard). Mean: Duan smearing,
58 +eq.~\eqref{eq:duan}; $\hat{s}$ is the smearing factor estimated on
59 +training residuals (random split), undefined for the Box--Cox model,
60 +whose mean prediction averages the inverse transformation over 1{,}000
61 +training residuals. $R^2_{\ln}$ under the mean rule is computed on the
62 +smeared log prediction.
63 +\end{tablenotes}
64 +\end{threeparttable}
65 +\end{table}
66 +
67 +\subsection{Leakage-free holdouts}
68 +
69 +Table~\ref{tab:group_splits} re-runs the full race under the two grouped
70 +holdouts of Section~\ref{sec:holdouts}, and Table~\ref{tab:leakage}
71 +decomposes the random holdout by whether the test sale's parcel, or its
72 +exact coordinates, also appear in training. Repeat sales are irrelevant:
73 +the parcel-grouped race reproduces the random race to within 0.1--0.4
74 +points for every model, and within the random holdout the boosting
75 +model prices test sales whose parcel sold before in training (14.5\%)
76 +no better than the others (14.4\%). Shared coordinates matter more, but
77 +for a specific reason. Test sales that share coordinates with a training
78 +sale are priced at 12.0\% by boosting against 15.3\% for the rest, and
79 +at 16.7\% against 22.9\% by the $k$-NN rule, whose nearest neighbour is
80 +then literally the same building; the linear models are unaffected
81 +(17.8\% against 17.6\% for the splines). Condominiums account for most
82 +of these co-located sales, and the boosting model prices them at 8.9\%
83 +when their building is in training and 10.2\% when it is not, against
84 +18.0\% and 20.3\% for the quadratic OLS. The condominium advantage of
85 +Section~\ref{sec:ext} is therefore genuine micro-location, with a modest
86 +recall component. The building-grouped race, in which no coordinate pair
87 +straddles the split, confirms it: boosting posts 14.6\% against 17.8\%
88 +for the spline hedonic, a 3.2-point gap identical to the random race,
89 +and the only model that loses materially is the $k$-NN rule (22.3\%
90 +against 21.2\%).
91 +
92 +\begin{table}[p]
93 +\centering
94 +\begin{threeparttable}
95 +\caption{The race under random, parcel-grouped and building-grouped holdouts}
96 +\label{tab:group_splits}
97 +\small
98 +\input{../results/tables/group_splits}
99 +\begin{tablenotes}[flushleft]\footnotesize
100 +\item \textit{Notes:} MdAPE in percent. Parcel-grouped: 20\% of
101 +assessment-roll parcels assigned to the test set, so that repeat sales
102 +of a property never straddle the split. Building-grouped: 20\% of exact
103 +coordinate pairs assigned to the test set, so that units geocoded to the
104 +same building or complex never straddle the split.
105 +\end{tablenotes}
106 +\end{threeparttable}
107 +\end{table}
108 +
109 +\begin{table}[tbp]
110 +\centering
111 +\begin{threeparttable}
112 +\caption{Random holdout: accuracy on test sales seen and unseen through their parcel or coordinates}
113 +\label{tab:leakage}
114 +\small
115 +\input{../results/tables/leakage}
116 +\begin{tablenotes}[flushleft]\footnotesize
117 +\item \textit{Notes:} MdAPE (\%) on the random holdout. ``Same parcel in
118 +training'': the parcel of the test sale sold at least once in the
119 +training sample. ``Same coordinates in training'': at least one training
120 +sale shares the exact geocoded coordinates of the test sale.
121 +\end{tablenotes}
122 +\end{threeparttable}
123 +\end{table}
124 +
125 +\subsection{Nested spatial fixed effects}
126 +
127 +Section~\ref{sec:results} attributes the poor showing of the
128 +$\sim$1.1~km grid to thin cells and to the arbitrary pricing of unseen
129 +cells. Table~\ref{tab:nested_fe} adds two specifications in which the
130 +grid effects enter \emph{together with} municipality effects. Because
131 +the two sets of dummies are nearly collinear, the vanishing-penalty
132 +ridge returns the minimum-norm least-squares solution, which assigns the
133 +component common to a municipality to the municipality effect and only
134 +the deviation to the cell effect---a crude form of partial pooling---and
135 +a test sale in an unseen cell inherits its municipality's level. The
136 +effect is large. Nested within municipalities, the 5.5~km grid improves
137 +from 18.7\% to 17.7\% and the 1.1~km grid from 21.0\% to 17.3\%, the
138 +lowest median error of any linear specification in the paper, below the
139 +spline hedonic (17.6\%) with the same $R^2_{\ln}$ (0.59). Under the
140 +forward-in-time split the nested fine grid posts 21.6\% against 23.3\%
141 +for municipality effects alone and 25.3\% for the fine grid alone. The
142 +conclusion for axis~C is therefore sharper than ``finer overfits'':
143 +fine spatial effects are valuable when they are anchored to a coarser
144 +level that supplies the prior for thin and unseen cells, and harmful
145 +when they are not. A hierarchical or kriged location effect
146 +\citep{case2004modeling, hill2018geospatial} would push the linear
147 +frontier further; we leave its estimation to future work.
148 +
149 +\begin{table}[tbp]
150 +\centering
151 +\begin{threeparttable}
152 +\caption{Nested spatial fixed effects}
153 +\label{tab:nested_fe}
154 +\small
155 +\input{../results/tables/nested_fe}
156 +\begin{tablenotes}[flushleft]\footnotesize
157 +\item \textit{Notes:} Quadratic semi-log backbone with quarter fixed
158 +effects. C3$'$ and C4$'$ add the grid effects to the municipality
159 +effects; with a vanishing ridge penalty the estimator is the
160 +minimum-norm least-squares solution and unseen cells fall back on their
161 +municipality.
162 +\end{tablenotes}
163 +\end{threeparttable}
164 +\end{table}
165 +
166 +\subsection{A finer Box--Cox grid}
167 +
168 +The main grid steps $\lambda$ by $0.25$. Refining it to steps of $0.05$
169 +over $[-0.25, 0.75]$ moves the profile-likelihood maximum to
170 +$\hat\lambda = 0.35$, with a profile that is flat between $0.25$ and
171 +$0.55$ (differences of under one thousandth of the log-likelihood,
172 +within the tolerance of the sparse iterative solver) and falls steeply
173 +on either side. The Box--Cox model at $\hat\lambda = 0.35$ posts a
174 +random-holdout MdAPE of 18.4\% against 18.6\% at $0.25$: the exponent is
175 +imprecisely identified in a range that excludes both the linear and the
176 +logarithmic case, and its exact value is immaterial for prediction.
177 +
178 +\subsection{Boosting hyper-parameters}
179 +
180 +Table~\ref{tab:hgb} re-estimates the boosting model under four
181 +materially different configurations: shallow trees with more iterations,
182 +deep trees, heavy regularisation (minimum leaf 200, $\ell_2 = 1$), and a
183 +configuration whose number of iterations is chosen by early stopping on
184 +the last 10\% of training sales in date order, the time-honest tuning
185 +rule a careful practitioner would use for a forward-looking model.
186 +Under the random holdout the four alternatives span 14.3--14.7\% MdAPE
187 +against 14.4\% for the baseline, and under the forward-in-time holdout
188 +21.2--21.6\% against 21.2\%; $R^2_{\ln}$ moves by at most 0.006. The
189 +time-honest early-stopping rule selects 700 iterations on the random
190 +training sample and 1{,}000 on the pre-2025 sample and reproduces the
191 +baseline to within 0.1 point under both splits. The conclusions of Sections~\ref{sec:results}
192 +and~\ref{sec:gap} do not depend on the particular configuration we
193 +fixed a priori, and tuning on a time-ordered validation set does not
194 +remove the level staleness that dominates the forward-split error,
195 +because no choice of hyper-parameters lets a static tree extrapolate a
196 +price level it has not seen.
197 +
198 +\begin{table}[tbp]
199 +\centering
200 +\begin{threeparttable}
201 +\caption{Gradient-boosting hyper-parameter sensitivity}
202 +\label{tab:hgb}
203 +\footnotesize
204 +\setlength{\tabcolsep}{4pt}
205 +\input{../results/tables/hgb_sensitivity}
206 +\begin{tablenotes}[flushleft]\footnotesize
207 +\item \textit{Notes:} All configurations use the D4 feature set
208 +(attributes, coordinates, month counter). Baseline: 600 iterations,
209 +learning rate 0.08, 63 leaves, minimum leaf 20, $\ell_2 = 0.01$. Early
210 +stopping: iterations added in blocks of 100 (learning rate 0.05) until
211 +the squared log error on the last 10\% of training sales in date order
212 +stops improving; the selected number of iterations is shown in brackets
213 +for the random and forward splits.
214 +\end{tablenotes}
215 +\end{threeparttable}
216 +\end{table}
added results/reproduced/boxcox_fine.csv +22 −0
@@ -0,0 +1,22 @@
1 +lambda,loglik,chosen,mdape_at_chosen
2 +-0.25,-4938790.1592812985,False,
3 +-0.2,-4928326.613673929,False,
4 +-0.15,-4921149.036684653,False,
5 +-0.1,-4911809.9533146825,False,
6 +-0.05,-4905852.743838613,False,
7 +-0.0,-4896133.282457808,False,
8 +0.05,-4895821.141372284,False,
9 +0.1,-4891760.972508729,False,
10 +0.15,-4888345.89894094,False,
11 +0.2,-4885580.902667309,False,
12 +0.25,-4879097.21142742,False,
13 +0.3,-4877690.751161514,False,
14 +0.35,-4876918.904257714,True,18.38848861136389
15 +0.4,-4881082.917829343,False,
16 +0.45,-4877369.476074193,False,
17 +0.5,-4882782.367887354,False,
18 +0.55,-4880462.865948945,False,
19 +0.6,-4881961.43143256,False,
20 +0.65,-4886168.783351108,False,
21 +0.7,-4889986.164861776,False,
22 +0.75,-4893437.953075144,False,
added results/reproduced/by_quarter.csv +36 −0
@@ -0,0 +1,36 @@
1 +quarter,model,n,mdape
2 +2025Q1,A5 Semi-log + quadratics,17031,20.16069824377374
3 +2025Q1,A6 Semi-log + splines,17031,18.55293878398712
4 +2025Q1,D2 Ridge (cross-validated),17031,18.97125923373471
5 +2025Q1,D3 Random forest,17031,17.15973919906848
6 +2025Q1,D4 Gradient boosting,17031,17.069787319853706
7 +2025Q2,A5 Semi-log + quadratics,34546,21.09424889763385
8 +2025Q2,A6 Semi-log + splines,34546,19.546001101493225
9 +2025Q2,D2 Ridge (cross-validated),34546,20.128071582190238
10 +2025Q2,D3 Random forest,34546,19.17503183093982
11 +2025Q2,D4 Gradient boosting,34546,19.056435785828757
12 +2025Q3,A5 Semi-log + quadratics,28097,22.81785959565014
13 +2025Q3,A6 Semi-log + splines,28097,21.26342871883413
14 +2025Q3,D2 Ridge (cross-validated),28097,21.787565277767396
15 +2025Q3,D3 Random forest,28097,20.640199696692722
16 +2025Q3,D4 Gradient boosting,28097,20.570509508119454
17 +2025Q4,A5 Semi-log + quadratics,24156,23.84990757727795
18 +2025Q4,A6 Semi-log + splines,24156,22.156927074847843
19 +2025Q4,D2 Ridge (cross-validated),24156,22.704586080935236
20 +2025Q4,D3 Random forest,24156,21.686460771814904
21 +2025Q4,D4 Gradient boosting,24156,21.604075720079475
22 +2026Q1,A5 Semi-log + quadratics,15611,24.828798398064173
23 +2026Q1,A6 Semi-log + splines,15611,23.136954110839774
24 +2026Q1,D2 Ridge (cross-validated),15611,23.555344810884428
25 +2026Q1,D3 Random forest,15611,22.809749906308703
26 +2026Q1,D4 Gradient boosting,15611,22.683544277340335
27 +2026Q2,A5 Semi-log + quadratics,30950,25.82329388343181
28 +2026Q2,A6 Semi-log + splines,30950,24.490104774430893
29 +2026Q2,D2 Ridge (cross-validated),30950,25.114690369909486
30 +2026Q2,D3 Random forest,30950,24.366584102876867
31 +2026Q2,D4 Gradient boosting,30950,24.313275442733904
32 +2026Q3,A5 Semi-log + quadratics,7997,25.960227538726045
33 +2026Q3,A6 Semi-log + splines,7997,24.87544352852322
34 +2026Q3,D2 Ridge (cross-validated),7997,25.158903757045657
35 +2026Q3,D3 Random forest,7997,24.996277735314912
36 +2026Q3,D4 Gradient boosting,7997,24.87390841981474
added results/reproduced/hgb_sensitivity.csv +11 −0
@@ -0,0 +1,11 @@
1 +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test,scheme
2 +D4 Gradient boosting (baseline),S,14.388300681085338,27.766499817027036,0.3471568687492957,0.6422912683312305,13.136043786916385,28.339593011975516,0.3501358518462874,0.6361258693434015,1.0475010409695187,,,40.8,411283,102929,random
3 +"D4 shallow (15 leaves, lr 0.05, 1500 iter)",S,14.719243860105477,28.154919687681947,0.35039008957492424,0.63559725098862,13.401236660872359,28.83674263008007,0.35391692116024087,0.6282245817693768,1.051807579516185,,,28.7,411283,102929,random
4 +"D4 deep (255 leaves, lr 0.05, 600 iter)",S,14.289611237615919,27.61056923649104,0.34601169250653546,0.6446473438310576,13.117602210229256,28.098909164979524,0.34851529339791887,0.6394863630851599,1.043402259531175,,,121.5,411283,102929,random
5 +"D4 regularised (min leaf 200, L2 = 1)",S,14.26217307195608,27.69234020959297,0.3467049485484332,0.6432219748599041,13.01108639038474,28.277180185855872,0.3498336625404618,0.636753690914108,1.0485848722394113,,,40.9,411283,102929,random
6 +D4 early stopping on last 10% of training months,S,14.486038756539063,27.894520070852053,0.34801538571621565,0.6405198573634903,13.222127726623725,28.493460749856283,0.35117203837899774,0.6339689957921757,1.0488581267072552,,700.0,126.3,411283,102929,random
7 +D4 Gradient boosting (baseline),S,21.21243467837447,29.71192710420828,0.37236811207724885,0.5214213758996993,18.362851253802294,28.605014412660662,0.3587539272757954,0.5557763676216326,1.0485637089645152,,,40.7,355824,158388,temporal
8 +"D4 shallow (15 leaves, lr 0.05, 1500 iter)",S,21.41075901094633,29.814111901291206,0.37240688652603166,0.521321702550176,18.299953587404797,28.62964905180962,0.3577628776257503,0.5582272931995136,1.0532570669661585,,,28.7,355824,158388,temporal
9 +"D4 deep (255 leaves, lr 0.05, 600 iter)",S,21.599331939568586,29.87909605759482,0.3738085572244618,0.5177116081480675,18.958152089809033,28.790557674838784,0.3604324382094834,0.55160984301614,1.0448264651027446,,,107.7,355824,158388,temporal
10 +"D4 regularised (min leaf 200, L2 = 1)",S,21.53671243851688,29.763998225562823,0.37211824841062413,0.5220634249166158,18.523654776944497,28.560638550076707,0.3575714411737002,0.5586999457753024,1.0503781019780596,,,33.9,355824,158388,temporal
11 +D4 early stopping on last 10% of training months,S,21.211205965026554,29.680715279436637,0.37128678154177347,0.5241968567695916,18.364288650180356,28.557949065565285,0.3576920591594777,0.5584021719866923,1.04834813285858,,1000.0,158.8,355824,158388,temporal
added results/reproduced/horserace_group_coord.csv +21 −0
@@ -0,0 +1,21 @@
1 +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test
2 +A1 Linear (levels),A. Functional form,18.389455311057525,35.77737768430297,0.42150872762642705,0.4729684614765287,18.389455311057525,35.77737768430297,0.42150872762642705,0.4729684614765287,,,,0.4,411263,102949
3 +A2 Semi-log,A. Functional form,19.45406141657741,33.53341061653312,0.3982220564706349,0.5295926664345789,18.534228750178155,35.515597429659906,0.40458544391763035,0.5144388071443484,1.0716405125795019,,,0.6,411263,102949
4 +A3 Log-log,A. Functional form,18.09423906311878,31.5648056784115,0.3795231689636993,0.5727323131971047,17.13689016366873,33.083726302727165,0.3848295353676579,0.5607009592529427,1.0637378029740763,,,0.8,411263,102949
5 +A4 Box-Cox,A. Functional form,18.34720115572638,33.91816652516567,0.39588027012021865,0.5351089578471608,18.01369176345465,35.11846758212266,0.4005581443205167,0.5240573822295058,,0.5,,4.4,411263,102949
6 +A5 Semi-log + quadratics,A. Functional form,19.047250871870013,32.633713129691856,0.38879112913610764,0.5516097539820513,18.060274251722916,34.383320754266194,0.39457920958428305,0.5381596654068279,1.0678021020773296,,,0.7,411263,102949
7 +A6 Semi-log + splines,A. Functional form,17.84160255200089,31.17235135800901,0.375404013662821,0.5819566827043615,16.719953387602672,32.5440257224398,0.38032165138598156,0.5709325567849641,1.06173604546364,,,2.3,411263,102949
8 +B1 No time effects,B. Time effects,22.020713610007295,36.07330397660628,0.41771180424287685,0.4824206287010825,21.320623105141713,38.615749714380435,0.42497748545191755,0.4642584779943505,1.0797929429062483,,,0.5,411263,102949
9 +B2 Year FE,B. Time effects,19.216445989382176,33.21932498640056,0.3943360634939339,0.5387286770024654,18.306678366552863,35.06233519240754,0.4003352095717885,0.5245870163050035,1.069862710157588,,,0.7,411263,102949
10 +B3 Quarter FE,B. Time effects,19.047250871870013,32.633713129691856,0.38879112913610764,0.5516097539820513,18.060274251722916,34.383320754266194,0.39457920958428305,0.5381596654068279,1.0678021020773296,,,0.7,411263,102949
11 +B4 Month FE,B. Time effects,19.398625101840388,33.282018368201285,0.39485525067670135,0.537513247647079,18.49481494077501,35.151184915530756,0.40092705375610255,0.5231803031914969,1.070168533613627,,,0.6,411263,102949
12 +C1 No spatial controls,C. Spatial controls,28.689547544675424,43.840616324962724,0.48386013502120434,0.30551455745412537,27.854471463760017,48.59662417250643,0.4957431862109964,0.27098415190902636,1.1117746881974258,,,0.5,411263,102949
13 +C2 Municipality FE,C. Spatial controls,19.047250871870013,32.633713129691856,0.38879112913610764,0.5516097539820513,18.060274251722916,34.383320754266194,0.39457920958428305,0.5381596654068279,1.0678021020773296,,,0.7,411263,102949
14 +C3 Grid-cell FE (~5.5 km),C. Spatial controls,18.750020280736805,32.64356649270374,0.38885559208588105,0.5514610522654697,17.681345309644815,34.2696279297555,0.3942232975475682,0.538992453333233,1.0671696493930205,,,0.7,411263,102949
15 +C4 Grid-cell FE (~1.1 km),C. Spatial controls,20.225376123786642,34.842860847732275,0.40626236607303856,0.5104053661049823,18.962836326256582,36.70306684092424,0.41241718088755186,0.49545842259587813,1.0720020497888785,,,0.8,411263,102949
16 +D1 OLS (muni + month FE),D. Estimation method,19.398625101840388,33.282018368201285,0.39485525067670135,0.537513247647079,18.49481494077501,35.151184915530756,0.40092705375610255,0.5231803031914969,1.070168533613627,,,0.6,411263,102949
17 +D2 Ridge (cross-validated),D. Estimation method,17.890529516201568,31.251732783077244,0.3763885157696496,0.5797611590637215,16.8893677800818,32.68394222313682,0.3814143837676687,0.5684634391789163,1.0622804510287405,,,10.1,411263,102949
18 +D3 Random forest,D. Estimation method,14.883188133292904,28.737382658558698,0.3573031529738494,0.6212983831644259,14.053333531225153,29.108973566446117,0.35869427463082987,0.6183437740317796,1.0318463155697535,,,8.4,411263,102949
19 +D4 Gradient boosting,D. Estimation method,14.59794086326427,28.210269818504578,0.3511407425669361,0.6342486746266742,13.424406660101518,28.85748490533166,0.35438418409430017,0.6274606736319336,1.047165791723387,,,42.1,411263,102949
20 +"D5 Gradient boosting, no coordinates",D. Estimation method,23.288069281216877,38.626422603992054,0.44226646657865465,0.4197816156316563,22.324919746724586,41.540491694908305,0.45047464198749365,0.39804481488791,1.086467333318932,,,20.8,411263,102949
21 +D6 Spatial k-NN comparables,D. Estimation method,22.263894535107575,37.95804725523757,0.4404191782505073,0.4246184833703136,22.194183511129985,38.13596309275768,0.44042512037220527,0.42460295720693186,1.0066905499496188,,,0.3,411263,102949
added results/reproduced/horserace_group_parcel.csv +21 −0
@@ -0,0 +1,21 @@
1 +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test
2 +A1 Linear (levels),A. Functional form,18.394879964095797,35.96893018809802,0.42358366884898346,0.4688087852186674,18.394879964095797,35.96893018809802,0.42358366884898346,0.4688087852186674,,,,0.4,411369,102843
3 +A2 Semi-log,A. Functional form,19.40388065006023,33.60742551231478,0.3991437427562464,0.5283377491612558,18.459707672156842,35.527166516466494,0.40507307833118583,0.5142204489174887,1.0716280541158298,,,0.6,411369,102843
4 +A3 Log-log,A. Functional form,18.04901279537473,31.62346448383144,0.38036202721805884,0.5716815568957894,17.060785648509054,33.08835585008513,0.3853279508875882,0.5604244841914658,1.0636869814479837,,,0.9,411369,102843
5 +A4 Box-Cox,A. Functional form,18.24702711324504,34.006661029680444,0.3967848815219953,0.5338961387758547,17.86556740305991,35.46526823103805,0.40274905686877965,0.5197785744597814,,0.5,,4.4,411369,102843
6 +A5 Semi-log + quadratics,A. Functional form,19.19420945605029,33.23478458992403,0.3946076985401183,0.5389971863448217,18.178626389812674,35.017880574991494,0.40026491744886666,0.5256842765856499,1.0696108179421064,,,0.6,411369,102843
7 +A6 Semi-log + splines,A. Functional form,17.62071626632672,31.207237958155154,0.376000477877242,0.5814481534939215,16.60974843964176,32.54559297854598,0.38074837993673477,0.5708109860855053,1.0617470052055027,,,2.3,411369,102843
8 +B1 No time effects,B. Time effects,21.649567560303716,35.80598610682794,0.4155121629945933,0.4888599034728587,20.805949301729967,38.19532183877957,0.4222433482128658,0.47216510031035164,1.078199634331365,,,0.5,411369,102843
9 +B2 Year FE,B. Time effects,18.990446124535953,32.997820257572705,0.3921780438977923,0.5446566268137394,18.058282132701265,34.74906901015738,0.3977502821212147,0.5316252641639805,1.068487535996415,,,0.7,411369,102843
10 +B3 Quarter FE,B. Time effects,19.19420945605029,33.23478458992403,0.3946076985401183,0.5389971863448217,18.178626389812674,35.017880574991494,0.40026491744886666,0.5256842765856499,1.0696108179421064,,,0.6,411369,102843
11 +B4 Month FE,B. Time effects,19.332436660424253,33.33207533297025,0.3954934330533667,0.5369253343882607,18.402816630486033,35.160733297104976,0.40121192424899693,0.5234372068832924,1.0701374458919553,,,0.6,411369,102843
12 +C1 No spatial controls,C. Spatial controls,28.887143937365718,44.050532124221256,0.4859586541235692,0.30084879639246065,28.095541551675375,48.734151973096715,0.49707893152453125,0.2684850929794075,1.1115281859261688,,,0.4,411369,102843
13 +C2 Municipality FE,C. Spatial controls,19.19420945605029,33.23478458992403,0.3946076985401183,0.5389971863448217,18.178626389812674,35.017880574991494,0.40026491744886666,0.5256842765856499,1.0696108179421064,,,0.6,411369,102843
14 +C3 Grid-cell FE (~5.5 km),C. Spatial controls,18.73260474151463,32.799326462608065,0.38980765046740884,0.5501443438715339,17.53502943411876,34.40695892966617,0.3952679237455016,0.5374532717263956,1.0671611573206463,,,0.7,411369,102843
15 +C4 Grid-cell FE (~1.1 km),C. Spatial controls,21.097621825427847,35.95310914429783,0.4147863521944663,0.49064404850887544,19.553592455622823,37.9613459138871,0.4212809812757885,0.47456841574603015,1.0757010147272146,,,0.8,411369,102843
16 +D1 OLS (muni + month FE),D. Estimation method,19.332436660424253,33.33207533297025,0.3954934330533667,0.5369253343882607,18.402816630486033,35.160733297104976,0.40121192424899693,0.5234372068832924,1.0701374458919553,,,0.6,411369,102843
17 +D2 Ridge (cross-validated),D. Estimation method,17.70592620550421,31.251501981456574,0.3767730607223491,0.579726356784688,16.72857367165446,32.65172599817906,0.3815465570975194,0.569009649787606,1.062317122081801,,,10.2,411369,102843
18 +D3 Random forest,D. Estimation method,14.622820199263606,28.743565661551727,0.3582854740346365,0.6199586341211347,13.749624355032527,29.092018626347237,0.3596181757160165,0.6171261233153971,1.0317651338894,,,9.2,411369,102843
19 +D4 Gradient boosting,D. Estimation method,14.507935468925822,28.29190277596338,0.3525188079219081,0.6320938445206743,13.24585810355578,28.915575305176098,0.3556681265266408,0.6254909102009842,1.0478640275860414,,,43.0,411369,102843
20 +"D5 Gradient boosting, no coordinates",D. Estimation method,23.41157162533723,38.73698634397927,0.44364203470006636,0.4173096237076285,22.212467734820184,41.63446817144011,0.4514086798765861,0.39672923483220146,1.0877176588778272,,,17.5,411369,102843
21 +D6 Spatial k-NN comparables,D. Estimation method,21.308170377003776,37.246066218610544,0.43397656727509987,0.442422771348263,21.25721677508758,37.43458435469481,0.434126877127783,0.442036465398573,1.0066433972846687,,,0.3,411369,102843
modified results/reproduced/horserace_random.csv +21 −21
@@ -1,21 +1,21 @@
1 −name,group,mdape,mape,rmse_ln,r2_ln,seconds,n_train,n_test
2 −A1 Linear (levels),A. Functional form,18.247735441468322,35.554833553436424,0.4197326280373289,0.4770941777730925,0.5,411283,102929
3 −A2 Semi-log,A. Functional form,18.174885439840832,34.4795393377222,0.39663369462465925,0.5330641412991447,0.6,411283,102929
4 −A3 Log-log,A. Functional form,16.965877929132013,32.69009595755662,0.38125191583247026,0.5685782006102371,0.9,411283,102929
5 −A4 Box-Cox,A. Functional form,18.567718342220903,33.001281736849975,0.3903031252718359,0.5478504801090737,4.3,411283,102929
6 −A5 Semi-log + quadratics,A. Functional form,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,0.7,411283,102929
7 −A6 Semi-log + splines,A. Functional form,16.52578223473714,32.10851818544987,0.37644836156803374,0.5793810460819198,2.6,411283,102929
8 −B1 No time effects,B. Time effects,21.1450312090911,38.37918215557233,0.42321207728034005,0.4683888002641269,0.5,411283,102929
9 −B2 Year FE,B. Time effects,18.091245071945412,34.663597249791685,0.39704996735836223,0.5320835152583053,0.7,411283,102929
10 −B3 Quarter FE,B. Time effects,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,0.6,411283,102929
11 −B4 Month FE,B. Time effects,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,0.7,411283,102929
12 −C1 No spatial controls,C. Spatial controls,27.922352076812572,48.39219298797349,0.4944639622666287,0.2743164045351353,0.5,411283,102929
13 −C2 Municipality FE,C. Spatial controls,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,0.7,411283,102929
14 −C3 Grid-cell FE (~5.5 km),C. Spatial controls,17.53978076473437,34.03632099310664,0.3918420758470441,0.5442778333854188,0.7,411283,102929
15 −C4 Grid-cell FE (~1.1 km),C. Spatial controls,19.389847159546143,37.50800513353381,0.41828640746256024,0.48069139310554865,0.8,411283,102929
16 −D1 OLS (muni + month FE),D. Estimation method,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,0.7,411283,102929
17 −D2 Ridge (cross-validated),D. Estimation method,16.671722823569887,32.2521071548757,0.37758629277068595,0.5768343018663071,10.8,411283,102929
18 −D3 Random forest,D. Estimation method,14.62750218203232,28.181960170119,0.35257338948901984,0.6310418755396041,7.6,411283,102929
19 −D4 Gradient boosting,D. Estimation method,14.388300681085338,27.766499817027036,0.3471568687492957,0.6422912683312305,42.5,411283,102929
20 −"D5 Gradient boosting, no coordinates",D. Estimation method,23.225090677534933,38.30004358013209,0.4398837230475839,0.4256801846835939,17.7,411283,102929
21 −D6 Spatial k-NN comparables,D. Estimation method,21.18226600985226,36.73270267320528,0.43669681006238664,0.43397181715912814,0.1,411283,102929
1 +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test
2 +A1 Linear (levels),A. Functional form,18.247735441468322,35.554833553436424,0.4197326280373289,0.4770941777730925,18.247735441468322,35.554833553436424,0.4197326280373289,0.4770941777730925,,,,0.4,411283,102929
3 +A2 Semi-log,A. Functional form,19.034569938583708,32.66020106922096,0.39092568564912344,0.5464069104539773,18.174885439840832,34.4795393377222,0.39663369462465925,0.5330641412991447,1.0704523740369927,,,0.6,411283,102929
4 +A3 Log-log,A. Functional form,18.00422814105048,31.24431556205711,0.3762917738410556,0.5797308952207196,16.965877929132013,32.69009595755662,0.38125191583247026,0.5685782006102371,1.0640208207942232,,,0.9,411283,102929
5 +A4 Box-Cox,A. Functional form,18.567718342220903,33.001281736849975,0.3903031252718359,0.5478504801090737,17.931354368937967,34.39115498001405,0.3955659580558204,0.5355747370020278,,0.25,,5.1,411283,102929
6 +A5 Semi-log + quadratics,A. Functional form,19.15743025152905,32.8527013727642,0.3911358710025644,0.5459190210301912,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,1.0698913891782036,,,0.7,411283,102929
7 +A6 Semi-log + splines,A. Functional form,17.6216157169721,30.791561379112448,0.37171474821867573,0.5898926040038202,16.52578223473714,32.10851818544987,0.37644836156803374,0.5793810460819198,1.0620544125859206,,,2.6,411283,102929
8 +B1 No time effects,B. Time effects,22.038141454991422,35.894437273924325,0.4162524756898916,0.4857294257045265,21.1450312090911,38.37918215557233,0.42321207728034005,0.4683888002641269,1.0800144916543517,,,0.5,411283,102929
9 +B2 Year FE,B. Time effects,19.12497097548873,32.91021487912301,0.3914089242770463,0.5452848088000786,18.091245071945412,34.663597249791685,0.39704996735836223,0.5320835152583053,1.06983370962192,,,0.7,411283,102929
10 +B3 Quarter FE,B. Time effects,19.15743025152905,32.8527013727642,0.3911358710025644,0.5459190210301912,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,1.0698913891782036,,,0.7,411283,102929
11 +B4 Month FE,B. Time effects,19.342062726308214,33.01798132147254,0.39253988395462874,0.5426532514337992,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,1.0704345170610976,,,0.7,411283,102929
12 +C1 No spatial controls,C. Spatial controls,28.785665634746415,43.740789664459314,0.48315578380182944,0.30712900384495423,27.922352076812572,48.39219298797349,0.4944639622666287,0.2743164045351353,1.1117959257819008,,,0.4,411283,102929
13 +C2 Municipality FE,C. Spatial controls,19.15743025152905,32.8527013727642,0.3911358710025644,0.5459190210301912,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,1.0698913891782036,,,0.7,411283,102929
14 +C3 Grid-cell FE (~5.5 km),C. Spatial controls,18.703553400119855,32.45802918238758,0.38651653174174216,0.556581136808286,17.53978076473437,34.03632099310664,0.3918420758470441,0.5442778333854188,1.067449819592543,,,0.7,411283,102929
15 +C4 Grid-cell FE (~1.1 km),C. Spatial controls,20.962941599826827,35.57153971188526,0.41212679150885995,0.49587328501243466,19.389847159546143,37.50800513353381,0.41828640746256024,0.48069139310554865,1.0756884611181337,,,0.9,411283,102929
16 +D1 OLS (muni + month FE),D. Estimation method,19.342062726308214,33.01798132147254,0.39253988395462874,0.5426532514337992,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,1.0704345170610976,,,0.7,411283,102929
17 +D2 Ridge (cross-validated),D. Estimation method,17.684559394393904,30.874200209239945,0.372797514554587,0.5874999240199299,16.671722823569887,32.2521071548757,0.37758629277068595,0.5768343018663071,1.0626009803138499,,,10.9,411283,102929
18 +D3 Random forest,D. Estimation method,14.62750218203232,28.181960170119,0.35257338948901984,0.6310418755396041,13.68191573226618,28.50111285576457,0.3537359214449655,0.6286047502994752,1.0320010783517677,,,12.5,411283,102929
19 +D4 Gradient boosting,D. Estimation method,14.388300681085338,27.766499817027036,0.3471568687492957,0.6422912683312305,13.136043786916385,28.339593011975516,0.3501358518462874,0.6361258693434015,1.0475010409695187,,,46.7,411283,102929
20 +"D5 Gradient boosting, no coordinates",D. Estimation method,23.225090677534933,38.30004358013209,0.4398837230475839,0.4256801846835939,22.11744594443843,41.15658845539997,0.4476945241929057,0.4051032704375488,1.0876216107611343,,,17.3,411283,102929
21 +D6 Spatial k-NN comparables,D. Estimation method,21.18226600985226,36.73270267320528,0.43669681006238664,0.43397181715912814,21.128130305413727,36.898833705701435,0.436786904121283,0.43373824074782497,1.0056888675291784,,,0.3,411283,102929
modified results/reproduced/horserace_temporal.csv +21 −21
@@ -1,21 +1,21 @@
1 −name,group,mdape,mape,rmse_ln,r2_ln,seconds,n_train,n_test
2 −A1 Linear (levels),A. Functional form,20.369511213481317,32.86005001443525,0.39728743801570204,0.4552239614410175,0.4,355824,158388
3 −A2 Semi-log,A. Functional form,20.611640132255836,33.09824319242239,0.3964345928808567,0.45756036006756484,0.5,355824,158388
4 −A3 Log-log,A. Functional form,19.189006525695262,30.749155508273613,0.3757555688314752,0.5126744490905257,0.8,355824,158388
5 −A4 Box-Cox,A. Functional form,20.957181720703545,31.940298396980698,0.38962419749983584,0.47603753989498354,3.8,355824,158388
6 −A5 Semi-log + quadratics,A. Functional form,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,0.6,355824,158388
7 −A6 Semi-log + splines,A. Functional form,19.097413080353302,30.49261711294047,0.37228529384601217,0.5216342331571537,2.1,355824,158388
8 −B1 No time effects,B. Time effects,25.384810383977687,33.162107080537304,0.41811180324070163,0.39661683797944225,0.5,355824,158388
9 −B2 Year FE,B. Time effects,21.006054587504984,32.62538772051225,0.39253658926914514,0.46817515745089067,0.6,355824,158388
10 −B3 Quarter FE,B. Time effects,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,0.6,355824,158388
11 −B4 Month FE,B. Time effects,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,0.6,355824,158388
12 −C1 No spatial controls,C. Spatial controls,26.45586672540444,41.108374756471996,0.45691339109589413,0.2794301211279968,0.4,355824,158388
13 −C2 Municipality FE,C. Spatial controls,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,0.6,355824,158388
14 −C3 Grid-cell FE (~5.5 km),C. Spatial controls,21.279243946393596,32.712624877555356,0.39371599730320833,0.464974534611328,0.6,355824,158388
15 −C4 Grid-cell FE (~1.1 km),C. Spatial controls,22.091478382056824,36.02185035008858,0.41700099046919065,0.39981863895351477,0.7,355824,158388
16 −D1 OLS (muni + month FE),D. Estimation method,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,0.6,355824,158388
17 −D2 Ridge (cross-validated),D. Estimation method,19.520136279141678,30.594351068378263,0.3747175016019065,0.5153633136379877,8.6,355824,158388
18 −D3 Random forest,D. Estimation method,21.34011982105225,29.76443050482997,0.37332513269156825,0.5189582317282835,6.8,355824,158388
19 −D4 Gradient boosting,D. Estimation method,21.21243467837447,29.71192710420828,0.37236811207724885,0.5214213758996993,41.5,355824,158388
20 −"D5 Gradient boosting, no coordinates",D. Estimation method,26.207233073287274,35.736727341535484,0.44725743322834494,0.3095639406099918,21.1,355824,158388
21 −D6 Spatial k-NN comparables,D. Estimation method,25.63120139114109,34.234843881635754,0.4653714833711972,0.2525057333957209,0.1,355824,158388
1 +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test
2 +A1 Linear (levels),A. Functional form,20.369511213481317,32.86005001443525,0.39728743801570204,0.4552239614410175,20.369511213481317,32.86005001443525,0.39728743801570204,0.4552239614410175,,,,0.4,355824,158388
3 +A2 Semi-log,A. Functional form,23.154885105735325,32.88199068728801,0.4078149103607638,0.42597007752801586,20.611640132255836,33.09824319242239,0.3964345928808567,0.45756036006756484,1.076793566376698,,,0.5,355824,158388
4 +A3 Log-log,A. Functional form,21.769614210046917,31.036455799131453,0.38771693156325343,0.4811547262154816,19.189006525695262,30.749155508273613,0.3757555688314752,0.5126744490905257,1.0655530599282332,,,0.8,355824,158388
5 +A4 Box-Cox,A. Functional form,20.957181720703545,31.940298396980698,0.38962419749983584,0.47603753989498354,19.571597393733516,31.96261263566212,0.3844157306522595,0.48995248826084103,,0.5,,4.0,355824,158388
6 +A5 Semi-log + quadratics,A. Functional form,23.30291738895638,32.33980518990814,0.4022452672417709,0.4415423854252667,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,1.0711855946796898,,,0.6,355824,158388
7 +A6 Semi-log + splines,A. Functional form,21.722504213272607,30.87403330589372,0.384133203104169,0.4907019347955833,19.097413080353302,30.49261711294047,0.37228529384601217,0.5216342331571537,1.06346689326762,,,2.0,355824,158388
8 +B1 No time effects,B. Time effects,29.296772516609654,34.68364172805182,0.4503543461363537,0.29996936286225007,25.384810383977687,33.162107080537304,0.41811180324070163,0.39661683797944225,1.0779549442640406,,,0.5,355824,158388
9 +B2 Year FE,B. Time effects,23.651696383010147,32.69839717790235,0.40614244890268997,0.43066865187652315,21.006054587504984,32.62538772051225,0.39253658926914514,0.46817515745089067,1.0741977461673156,,,0.6,355824,158388
10 +B3 Quarter FE,B. Time effects,23.30291738895638,32.33980518990814,0.4022452672417709,0.4415423854252667,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,1.0711855946796898,,,0.6,355824,158388
11 +B4 Month FE,B. Time effects,22.8719130040602,32.42550461740692,0.400924502771694,0.4452037338739415,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,1.0724108768578227,,,0.6,355824,158388
12 +C1 No spatial controls,C. Spatial controls,29.015774167241894,38.87352353861925,0.4685413801986148,0.24228787650947192,26.45586672540444,41.108374756471996,0.45691339109589413,0.2794301211279968,1.1188483034696548,,,0.4,355824,158388
13 +C2 Municipality FE,C. Spatial controls,23.30291738895638,32.33980518990814,0.4022452672417709,0.4415423854252667,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,1.0711855946796898,,,0.6,355824,158388
14 +C3 Grid-cell FE (~5.5 km),C. Spatial controls,24.2625431905187,33.00942406765742,0.40874325687600593,0.42335366891858794,21.279243946393596,32.712624877555356,0.39371599730320833,0.464974534611328,1.0720367816988154,,,0.6,355824,158388
15 +C4 Grid-cell FE (~1.1 km),C. Spatial controls,25.324930433263678,35.50206863036151,0.42919028135360904,0.3642182032203949,22.091478382056824,36.02185035008858,0.41700099046919065,0.39981863895351477,1.0921887847980092,,,0.6,355824,158388
16 +D1 OLS (muni + month FE),D. Estimation method,22.8719130040602,32.42550461740692,0.400924502771694,0.4452037338739415,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,1.0724108768578227,,,0.6,355824,158388
17 +D2 Ridge (cross-validated),D. Estimation method,22.257318062238202,31.071606388474528,0.38804376521331146,0.480279615769711,19.520136279141678,30.594351068378263,0.3747175016019065,0.5153633136379877,1.0640215124102546,,,8.2,355824,158388
18 +D3 Random forest,D. Estimation method,21.340119821052188,29.76443050482997,0.37332513269156825,0.5189582317282835,19.394248052512626,28.951513572891475,0.36347962322867167,0.5439961919671212,1.0324597692592685,,,10.7,355824,158388
19 +D4 Gradient boosting,D. Estimation method,21.21243467837447,29.71192710420828,0.37236811207724885,0.5214213758996993,18.362851253802294,28.605014412660662,0.3587539272757954,0.5557763676216326,1.0485637089645152,,,41.3,355824,158388
20 +"D5 Gradient boosting, no coordinates",D. Estimation method,26.207233073287274,35.736727341535484,0.44725743322834494,0.3095639406099918,23.004852680221028,35.9605098787384,0.42968275070306156,0.3627583254241229,1.0912898778385383,,,21.7,355824,158388
21 +D6 Spatial k-NN comparables,D. Estimation method,25.63120139114109,34.234843881635754,0.4653714833711972,0.2525057333957209,25.38964897166118,34.13469069386835,0.46349083531773794,0.25853503771828157,1.0043584323305443,,,0.2,355824,158388
added results/reproduced/inference_random.csv +21 −0
@@ -0,0 +1,21 @@
1 +model,mdape,ci_lo,ci_hi,boot_se,diff_vs_ref,diff_lo,diff_hi,diff_vs_d4,diff_d4_lo,diff_d4_hi,dm_ape_vs_a6_t,dm_ape_vs_a6_p,dm_ape_vs_a6_diff,dm_sqlog_vs_a6_t,dm_sqlog_vs_a6_p,dm_sqlog_vs_a6_diff,dm_ape_vs_d4_t,dm_ape_vs_d4_p,dm_ape_vs_d4_diff,dm_sqlog_vs_d4_t,dm_sqlog_vs_d4_p,dm_sqlog_vs_d4_diff,n_clusters
2 +A1 Linear (levels),18.247735441468322,16.9113510560629,19.60250248921421,0.6798156719309989,0.6261197244962204,0.13177818927938997,1.1288670458109282,3.8594347603829835,2.3810890282818233,5.572402821114919,12.297632954401742,0.0,0.04763272174323981,7.51790454474203,5.573319583618286e-14,0.03800362499584921,10.403881445314044,0.0,0.077883337364094,11.88110067074241,0.0,0.05565758751930701,1091
3 +A2 Semi-log,19.034569938583708,17.948468947394765,20.177409718978527,0.5933974925595471,1.4129542216116064,1.0925363990812282,1.8335429697484,4.6462692574983695,3.410686792542463,6.182775708649027,10.174542669661014,0.0,0.01868639690108518,8.898715388706915,0.0,0.014651037656963798,5.814778194584789,6.071433578114238e-09,0.04893701252193937,4.763341767974322,1.9041293313204477e-06,0.03230500018042161,1091
4 +A3 Log-log,18.00422814105048,17.07752319183666,18.977226098783678,0.5097956635710524,0.3826124240783777,0.28563151196801645,0.48671649465618905,3.6159274599651408,2.268004875035067,5.04018984221415,4.640377326082088,3.477735786239222e-06,0.0045275418294466125,3.28780657498537,0.001009711859182305,0.0034236450171746756,3.7605346164843136,0.00016955060891321594,0.034778157450300794,3.1024469169219993,0.0019192796548725966,0.02107760754063249,1091
5 +A4 Box-Cox,18.567718342220903,17.46589758655213,19.707557217076772,0.5895493292913278,0.9461026252488018,0.6659357616138547,1.3168750827110465,4.179417661135565,2.9293738028577057,5.753351071060692,10.694909755887176,0.0,0.02209720357737525,10.463728364076685,0.0,0.014164675553688949,6.2558305116106006,3.954059302202495e-10,0.05234781919822943,5.180699999292202,2.2105477048839361e-07,0.03181863807714676,1091
6 +A5 Semi-log + quadratics,19.15743025152905,18.201941036517315,20.15445926287562,0.5026923733244107,1.5358145345569483,1.1398198349247033,2.1391093582834935,4.769129570443711,3.684746880089903,5.930644511229821,5.530637638741666,3.190688269150144e-08,0.020611399936517463,6.846564545749681,7.564393555981042e-12,0.014815415541661218,8.696091789215284,0.0,0.050862015557371645,7.103680897159478,1.2148060335448463e-12,0.03246937806511903,1091
7 +A6 Semi-log + splines,17.6216157169721,16.733927881204792,18.583767461456866,0.4920101568426174,,,,3.233315035886763,1.8746102788184382,4.657062294398633,,,,,,,3.5997911983137305,0.0003184728086869093,0.030250615620854186,3.021002919921985,0.002519389304698194,0.017653962523457807,1091
8 +B1 No time effects,22.038141454991422,21.08712269256635,23.267211253160106,0.5612291624566031,4.41652573801932,3.4200841630827816,5.582948377502827,7.649840773906083,6.748344719706512,8.338009838204949,8.206346390560523,2.220446049250313e-16,0.05102875894811877,7.860550382221196,3.774758283725532e-15,0.035094269474690284,19.848983759841108,0.0,0.08127937456897295,17.415570392509096,0.0,0.0527482319981481,1091
9 +B2 Year FE,19.12497097548873,18.135266091728134,20.16483252720224,0.5420996135180365,1.5033552585166277,1.0652509808425514,2.1819410747332237,4.736670294403391,3.6133353152983663,5.9326990304884655,6.124427672343136,9.101013276335834e-10,0.021186535000105613,7.1455155771433745,8.966161146872764e-13,0.015029091960441074,8.76902162424838,0.0,0.0514371506209598,7.29281506461723,3.035349749325178e-13,0.03268305448389889,1091
10 +B3 Quarter FE,19.15743025152905,18.201941036517315,20.15445926287562,0.5026923733244107,1.5358145345569483,1.1398198349247033,2.1391093582834935,4.769129570443711,3.684746880089903,5.930644511229821,5.530637638741666,3.190688269150144e-08,0.020611399936517463,6.846564545749681,7.564393555981042e-12,0.014815415541661218,8.696091789215284,0.0,0.050862015557371645,7.103680897159478,1.2148060335448463e-12,0.03246937806511903,1091
11 +B4 Month FE,19.342062726308214,18.41666765136561,20.381917170225584,0.5186977109248657,1.7204470093361124,1.2291247555472724,2.4534949420758574,4.953762045222875,3.8934157966761633,6.07316437462031,5.731933731822563,9.929206790104672e-09,0.02226419942360097,6.544042648781664,5.987765838710857e-11,0.0159157064518399,9.414342745938619,0.0,0.052514815044455154,7.726539400085093,1.1102230246251565e-14,0.033569668975297715,1091
12 +C1 No spatial controls,28.785665634746415,27.341346218160815,30.074976710188,0.7025099701791481,11.164049917774314,10.142872490107974,12.280345112224204,14.397364953661077,12.691537820133453,15.890935706600027,7.742609666008332,9.769962616701378e-15,0.12949228285346864,15.131594441078725,0.0,0.09526765737788669,14.938137987865796,0.0,0.15974289847432283,13.859739811417462,0.0,0.1129216199013445,1091
13 +C2 Municipality FE,19.15743025152905,18.201941036517315,20.15445926287562,0.5026923733244107,1.5358145345569483,1.1398198349247033,2.1391093582834935,4.769129570443711,3.684746880089903,5.930644511229821,5.530637638741666,3.190688269150144e-08,0.020611399936517463,6.846564545749681,7.564393555981042e-12,0.014815415541661218,8.696091789215284,0.0,0.050862015557371645,7.103680897159478,1.2148060335448463e-12,0.03246937806511903,1091
14 +C3 Grid-cell FE (~5.5 km),18.703553400119855,17.78729355112193,20.032142200057542,0.5678254955514357,1.0819376831477534,0.21901537947672517,2.308285560296583,4.3152527190345165,3.6452950369483923,5.089221243343746,2.5295453955100333,0.011421039859825566,0.01666467803275125,2.356369638453933,0.01845454380735978,0.01122317526639169,14.03126557996909,0.0,0.04691529365360543,13.706499864516653,0.0,0.0288771377898495,1091
15 +C4 Grid-cell FE (~1.1 km),20.962941599826827,20.085903076339296,21.854065484343828,0.4556213754459016,3.3413258828547256,2.8904906643605672,3.837012817774882,6.574640918741489,5.336615577939174,7.744405878759988,6.7530133957275,1.4480638910185917e-11,0.04779978332772806,7.5711644369648194,3.68594044175552e-14,0.03167663823611381,13.14987798988361,0.0,0.07805039894858226,12.025634909259105,0.0,0.049330600759571616,1091
16 +D1 OLS (muni + month FE),19.342062726308214,18.41666765136561,20.381917170225584,0.5186977109248657,1.7204470093361124,1.2291247555472724,2.4534949420758574,4.953762045222875,3.8934157966761633,6.07316437462031,5.731933731822563,9.929206790104672e-09,0.02226419942360097,6.544042648781664,5.987765838710857e-11,0.0159157064518399,9.414342745938619,0.0,0.052514815044455154,7.726539400085093,1.1102230246251565e-14,0.033569668975297715,1091
17 +D2 Ridge (cross-validated),17.684559394393904,16.768330917331383,18.6836218961783,0.5035043504152524,0.06294367742180285,-0.01676159248276264,0.1637807429146053,3.296258713308566,1.9811849496440535,4.724217428807863,2.6118019768463383,0.009006640622825302,0.0008263883012749337,2.7683584874087472,0.005633944669363622,0.0008061328148040056,3.6535115239679725,0.0002586781730240517,0.03107700392212912,3.060857865195947,0.002207038376347503,0.018460095338261817,1091
18 +D3 Random forest,14.62750218203232,13.737999221508268,16.19555708116927,0.6178221405912262,-2.994113534939782,-4.555739590156278,-1.4648756603277908,0.23920150094698123,0.0513885363794455,0.4588555165642077,-2.85524195730287,0.00430040529118747,-0.02609601208993448,-2.205886325821924,0.027391959893569684,-0.01386385906749741,4.672835254507822,2.970700491378153e-06,0.004154603530919706,6.693591526753016,2.177591440499782e-11,0.003790103455960403,1091
19 +D4 Gradient boosting,14.388300681085338,13.681350877219709,15.743536219678433,0.5263969787549507,-3.233315035886763,-4.657062294398633,-1.8746102788184382,,,,-3.5997911983137305,0.0003184728086869093,-0.030250615620854186,-3.021002919921985,0.002519389304698194,-0.017653962523457807,,,,,,,1091
20 +"D5 Gradient boosting, no coordinates",23.225090677534933,22.138529223958216,25.069137709119865,0.7566123985799357,5.603474960562831,3.9352537791374327,7.507040455957043,8.836789996449594,8.10748542572112,9.787971484731155,4.50536824487449,6.625793987247164e-06,0.07508482201019637,6.433439358856359,1.2474843380516631e-10,0.055325835758930006,10.594870291718431,0.0,0.10533543763105055,16.137980864260527,0.0,0.07297979828238782,1091
21 +D6 Spatial k-NN comparables,21.18226600985226,19.390028219965146,23.548071796694504,1.100889921651293,3.560650292880158,0.9519338843862455,5.8622156243977095,6.793965328766921,5.527326690571327,8.03181255741793,4.798111691143396,1.6016846118294126e-06,0.05941141294092833,5.960580367571443,2.513436569984151e-09,0.05253224987539075,19.297132345753973,0.0,0.08966202856178251,20.248240203175722,0.0,0.07018621239884855,1091
added results/reproduced/inference_temporal.csv +21 −0
@@ -0,0 +1,21 @@
1 +model,mdape,ci_lo,ci_hi,boot_se,diff_vs_ref,diff_lo,diff_hi,diff_vs_d4,diff_d4_lo,diff_d4_hi,dm_ape_vs_a6_t,dm_ape_vs_a6_p,dm_ape_vs_a6_diff,dm_sqlog_vs_a6_t,dm_sqlog_vs_a6_p,dm_sqlog_vs_a6_diff,dm_ape_vs_d4_t,dm_ape_vs_d4_p,dm_ape_vs_d4_diff,dm_sqlog_vs_d4_t,dm_sqlog_vs_d4_p,dm_sqlog_vs_d4_diff,n_clusters
2 +A1 Linear (levels),20.369511213481317,19.050772109018073,21.69348139159165,0.6849601233930911,-1.3529929997912902,-2.2770702594184407,-0.42539919706467916,-0.8429234648931541,-2.5898121968575603,1.1895847489303906,6.317663783346769,2.6554691778812867e-10,0.01986016708541528,6.164003649673723,7.092830767163605e-10,0.010278990678011518,3.520049279845853,0.0004314666251414856,0.03148122910226971,3.9079109150864686,9.309761836728825e-05,0.0191792975131057,1071
3 +A2 Semi-log,23.154885105735325,21.6887719627368,24.674870127680457,0.7875237764863192,1.4323808924627173,0.7670651228422551,2.2976528083716783,1.9424504273608534,0.5295622515203463,3.4375141515059062,7.796357304870725,6.439293542825908e-15,0.020079573813942913,7.223963685447007,5.049294315995212e-13,0.01875468338548904,3.965893940282591,7.312137370973915e-05,0.03170063583079734,5.055071810178792,4.3022885698462687e-07,0.02765499022058323,1071
4 +A3 Log-log,21.769614210046917,20.51940280719328,23.227311337108237,0.6739267745802049,0.04710999677431005,-0.04315938950036894,0.18763013229223433,0.5571795316724462,-0.5423402505386862,1.8391490439319191,1.804661231143382,0.07112771002474405,0.0016242249323772874,2.543121816787227,0.010986691828438477,0.0027661012937557784,1.646327603990528,0.09969632924843053,0.013245286949231714,2.098866610291465,0.03582866095871928,0.011666408128849963,1071
5 +A4 Box-Cox,20.957181720703545,19.742168516904147,22.156377541760673,0.6381565190731153,-0.7653224925690623,-1.4473333084469813,-0.09482024347911353,-0.25525295767092615,-1.7949341953230935,1.4690422686693982,4.264381230833118,2.0045693571724144e-05,0.010662650910869802,2.2871023500844068,0.022189849754430613,0.004248697550322327,2.5043936657330677,0.012266146996653404,0.022283712927724228,2.2903116877035994,0.02200325477123455,0.01314900438541651,1071
6 +A5 Semi-log + quadratics,23.30291738895638,21.869038097481326,24.922485533073313,0.8102019586928445,1.5804131756837734,1.0783669250409043,2.22065102463182,2.0904827105819095,0.9486003551587855,3.181577878834679,6.065811464275289,1.3128906850568e-09,0.014657718840144185,7.152656593716022,8.51096970677645e-13,0.014242937291334905,4.603816313944034,4.148183673891026e-06,0.026278780856998613,6.444665276370954,1.1585576942252374e-10,0.023143244126429093,1071
7 +A6 Semi-log + splines,21.722504213272607,20.45126930153131,23.100552763771457,0.6610274998462855,,,,0.5100695348981361,-0.5516760349102061,1.677839052305929,,,,,,,1.615823456025302,0.1061324886702475,0.011621062016854429,1.9765879301563614,0.04808822757633413,0.008900306835094185,1071
8 +B1 No time effects,29.296772516609654,27.25307447396948,31.453026176290106,1.0868368852006105,7.574268303337046,6.039750998516674,9.027043210519999,8.084337838235182,7.297235701980839,9.07468513889715,5.307042338766459,1.1141830524863394e-07,0.03809608422158093,9.288145872580824,0.0,0.05526071935683395,19.75478536885888,0.0,0.04971714623843536,16.92421738378873,0.0,0.06416102619192815,1071
9 +B2 Year FE,23.651696383010147,22.215432187910814,25.268875459898798,0.8096188915960055,1.92919216973754,1.3131800189892668,2.7530865495071235,2.439261704635676,1.3173521256992164,3.562209825093329,6.1579767061896264,7.368019527831393e-10,0.018243638720086247,6.017796983225028,1.7680674702091892e-09,0.017393371073605394,5.139555821884859,2.7538867297671743e-07,0.029864700736940673,7.23985001734488,4.4919623576333834e-13,0.026293677908699582,1071
10 +B3 Quarter FE,23.30291738895638,21.869038097481326,24.922485533073313,0.8102019586928445,1.5804131756837734,1.0783669250409043,2.22065102463182,2.0904827105819095,0.9486003551587855,3.181577878834679,6.065811464275289,1.3128906850568e-09,0.014657718840144185,7.152656593716022,8.51096970677645e-13,0.014242937291334905,4.603816313944034,4.148183673891026e-06,0.026278780856998613,6.444665276370954,1.1585576942252374e-10,0.023143244126429093,1071
11 +B4 Month FE,22.8719130040602,21.532938038433826,24.438984492545977,0.7619484470830038,1.149408790787593,0.6893140083694228,1.8098306190197673,1.659478325685729,0.5920810512262819,2.7576763092148138,5.991209061016688,2.082866767949554e-09,0.015514713115132013,5.990884007795864,2.0870345451839967e-09,0.013182139195661349,4.567473208327871,4.936387487219918e-06,0.027135775131986443,6.024141823255467,1.7000925112142795e-09,0.02208244603075553,1071
12 +C1 No spatial controls,29.015774167241894,27.422211588155974,30.325868927192477,0.7462222947018358,7.293269953969286,4.96861955562622,9.193766804289842,7.803339488867422,4.996711778534667,10.324147123803586,11.088323777905881,0.0,0.07999490232725531,7.508284601340977,5.995204332975845e-14,0.07197270723135413,17.010072661797665,0.0,0.09161596434410973,6.338419778689569,2.3213364563900996e-10,0.08087301406644831,1071
13 +C2 Municipality FE,23.30291738895638,21.869038097481326,24.922485533073313,0.8102019586928445,1.5804131756837734,1.0783669250409043,2.22065102463182,2.0904827105819095,0.9486003551587855,3.181577878834679,6.065811464275289,1.3128906850568e-09,0.014657718840144185,7.152656593716022,8.51096970677645e-13,0.014242937291334905,4.603816313944034,4.148183673891026e-06,0.026278780856998613,6.444665276370954,1.1585576942252374e-10,0.023143244126429093,1071
14 +C3 Grid-cell FE (~5.5 km),24.2625431905187,22.503220945710353,26.363285194699493,1.0301736045564578,2.540038977246091,1.5785786294135338,3.4839259747851203,3.0501085121442273,2.0572172007955043,3.9118766585912756,4.503146745467939,6.695461754135934e-06,0.02135390761763703,4.450105287824345,8.582820140379255e-06,0.019512732314535812,8.314219574865609,0.0,0.03297496963449146,12.3741576283692,0.0,0.028413039149629993,1071
15 +C4 Grid-cell FE (~1.1 km),25.324930433263678,24.251360863509277,26.29352775446495,0.49390977597366215,3.60242621999107,2.1189111166682935,4.91703661788902,4.112495754889206,2.25480020889692,5.928810388694186,12.43144220323286,0.0,0.046280353244677816,9.969202355421647,0.0,0.03664597988132135,7.149992224263942,8.677503160470224e-13,0.057901415261532245,6.861875584202508,6.796341267545358e-12,0.045546286716415536,1071
16 +D1 OLS (muni + month FE),22.8719130040602,21.532938038433826,24.438984492545977,0.7619484470830038,1.149408790787593,0.6893140083694228,1.8098306190197673,1.659478325685729,0.5920810512262819,2.7576763092148138,5.991209061016688,2.082866767949554e-09,0.015514713115132013,5.990884007795864,2.0870345451839967e-09,0.013182139195661349,4.567473208327871,4.936387487219918e-06,0.027135775131986443,6.024141823255467,1.7000925112142795e-09,0.02208244603075553,1071
17 +D2 Ridge (cross-validated),22.257318062238202,20.96191993033077,23.668021827486456,0.6692303373351438,0.5348138489655945,0.48101405213608955,0.6088791910137628,1.0448833838637306,0.0012400068282902282,2.185150952674026,6.580369386315953,4.692801702788074e-11,0.0019757308258080398,16.00748490968894,0.0,0.0030196459938548717,1.9584584383888557,0.05017624333623605,0.01359679284266247,2.647413747037119,0.008111004795629206,0.011919952828949056,1071
18 +D3 Random forest,21.340119821052188,19.907687448517617,22.980295616439495,0.8411843763724454,-0.38238439222041976,-1.5015448169575225,0.5571785755217394,0.12768514267771636,-0.14568791249818178,0.36811779443404413,-1.568790205531711,0.11669683137166498,-0.011096028010637537,-1.9109037795897155,0.05601694727646822,-0.008186663027891686,0.9790991544458619,0.3275309894032459,0.0005250340062168912,1.2895287248262932,0.19721433670042376,0.0007136438072025015,1071
19 +D4 Gradient boosting,21.21243467837447,19.70747318840484,22.945391194003907,0.8986037102049517,-0.5100695348981361,-1.6778390523059303,0.551676034910206,,,,-1.615823456025302,0.1061324886702475,-0.011621062016854429,-1.9765879301563614,0.04808822757633413,-0.008900306835094185,,,,,,,1071
20 +"D5 Gradient boosting, no coordinates",26.207233073287274,25.23007220277521,27.506849996204814,0.608040527286014,4.484728860014666,2.8779728042965105,6.210941230892228,4.994798394912802,3.1215058615470648,6.500510794984376,5.875060368764621,4.2268928357458435e-09,0.04862694035641765,9.378346817879406,0.0,0.05248089385093869,13.890923788390463,0.0,0.06024800237327208,11.078685604065148,0.0,0.06138120068603288,1071
21 +D6 Spatial k-NN comparables,25.63120139114109,23.14828800408425,28.333593401968223,1.407855973507387,3.9086971778684827,1.7817973469367552,5.637733529113384,4.418766712766619,3.332669523407553,5.470721729412763,4.036460402580431,5.4263679257982034e-05,0.0336081057574204,7.832489620597485,4.884981308350689e-15,0.0690122998080397,26.494126476851566,0.0,0.04522916777427483,15.848240562584424,0.0,0.0779126066431339,1071
added results/reproduced/leakage.csv +37 −0
@@ -0,0 +1,37 @@
1 +key,seen_in_training,segment,n,model,mdape
2 +in_train_parcel,True,all,17219,A5 Semi-log + quadratics,19.786608950655705
3 +in_train_parcel,True,all,17219,A6 Semi-log + splines,18.27278654110786
4 +in_train_parcel,True,all,17219,D2 Ridge (cross-validated),18.21506909664557
5 +in_train_parcel,True,all,17219,D3 Random forest,14.568143855697022
6 +in_train_parcel,True,all,17219,D4 Gradient boosting,14.494590514304903
7 +in_train_parcel,True,all,17219,"D5 Gradient boosting, no coordinates",22.887248058209508
8 +in_train_parcel,True,all,17219,D6 Spatial k-NN comparables,22.21141825253498
9 +in_train_parcel,True,condo,3374,A5 Semi-log + quadratics,17.414213423610217
10 +in_train_parcel,True,condo,3374,D4 Gradient boosting,9.035672175351072
11 +in_train_parcel,False,all,85710,A5 Semi-log + quadratics,19.05005201781079
12 +in_train_parcel,False,all,85710,A6 Semi-log + splines,17.48581012929675
13 +in_train_parcel,False,all,85710,D2 Ridge (cross-validated),17.576292121577303
14 +in_train_parcel,False,all,85710,D3 Random forest,14.639597756778661
15 +in_train_parcel,False,all,85710,D4 Gradient boosting,14.368856147710904
16 +in_train_parcel,False,all,85710,"D5 Gradient boosting, no coordinates",23.28075103236257
17 +in_train_parcel,False,all,85710,D6 Spatial k-NN comparables,20.990025690392308
18 +in_train_parcel,False,condo,11474,A5 Semi-log + quadratics,18.43144280197513
19 +in_train_parcel,False,condo,11474,D4 Gradient boosting,9.080011346798681
20 +in_train_coord,True,all,27475,A5 Semi-log + quadratics,19.16845806222931
21 +in_train_coord,True,all,27475,A6 Semi-log + splines,17.776290704974276
22 +in_train_coord,True,all,27475,D2 Ridge (cross-validated),17.859099147831706
23 +in_train_coord,True,all,27475,D3 Random forest,11.74898846453675
24 +in_train_coord,True,all,27475,D4 Gradient boosting,11.994995797797987
25 +in_train_coord,True,all,27475,"D5 Gradient boosting, no coordinates",20.04914260589526
26 +in_train_coord,True,all,27475,D6 Spatial k-NN comparables,16.652769091793402
27 +in_train_coord,True,condo,13261,A5 Semi-log + quadratics,17.97470023475135
28 +in_train_coord,True,condo,13261,D4 Gradient boosting,8.934702433522762
29 +in_train_coord,False,all,75454,A5 Semi-log + quadratics,19.15384674472759
30 +in_train_coord,False,all,75454,A6 Semi-log + splines,17.555794642798645
31 +in_train_coord,False,all,75454,D2 Ridge (cross-validated),17.625875167760007
32 +in_train_coord,False,all,75454,D3 Random forest,15.749377923105962
33 +in_train_coord,False,all,75454,D4 Gradient boosting,15.321150846402826
34 +in_train_coord,False,all,75454,"D5 Gradient boosting, no coordinates",24.512690151286638
35 +in_train_coord,False,all,75454,D6 Spatial k-NN comparables,22.907703956528476
36 +in_train_coord,False,condo,1587,A5 Semi-log + quadratics,20.34309569440084
37 +in_train_coord,False,condo,1587,D4 Gradient boosting,10.243202880340052
modified results/reproduced/learning.csv +6 −6
@@ -1,13 +1,13 @@
1 1 n_train,model,mdape,mape,rmse_ln,r2_ln
2 −10000,OLS quadratic (muni+quarter FE),18.38436183742385,34.439941317616764,0.39722768960842764,0.531664536345561
2 +10000,OLS quadratic (muni+quarter FE),19.136226571989205,32.82348826216586,0.39279102092321205,0.5420678667536525
3 3 10000,Gradient boosting,17.55817415103931,31.777076909845096,0.386462587914794,0.5567048988691092
4 −25000,OLS quadratic (muni+quarter FE),18.13123412223344,34.71752541710843,0.3987497135923394,0.5280686972389094
4 +25000,OLS quadratic (muni+quarter FE),19.056034269404236,32.98471009782084,0.39338974517982517,0.5406707674234809
5 5 25000,Gradient boosting,16.117235891494122,29.823687390173077,0.36598650222996054,0.6024349909662393
6 −50000,OLS quadratic (muni+quarter FE),17.85098430954251,34.291813175933555,0.394668048572139,0.5376807754235925
6 +50000,OLS quadratic (muni+quarter FE),18.801486657084954,32.60613267913101,0.38924551758611076,0.5502975469627178
7 7 50000,Gradient boosting,15.46029775645706,29.149938590306927,0.3602557920963508,0.6147878645125984
8 −100000,OLS quadratic (muni+quarter FE),17.858528109660003,33.97231033671203,0.39154828747222153,0.5449609437080014
8 +100000,OLS quadratic (muni+quarter FE),18.79513254400627,32.30474782180091,0.3861646547014708,0.5573881289110587
9 9 100000,Gradient boosting,15.201224181879457,28.789207604531935,0.35629888749041316,0.6232034228186022
10 −200000,OLS quadratic (muni+quarter FE),17.926781291106366,34.286171684612285,0.3940853634933891,0.5390448972702662
10 +200000,OLS quadratic (muni+quarter FE),18.95754804582277,32.59698579952596,0.3886557888504686,0.5516591634684497
11 11 200000,Gradient boosting,14.75286425569271,28.18065043214689,0.3510692811837181,0.6341831746695208
12 −400000,OLS quadratic (muni+quarter FE),18.06403273608106,34.590640697337626,0.3967353319490308,0.5328248061516689
12 +400000,OLS quadratic (muni+quarter FE),19.163049017655457,32.85043942607389,0.3911538943677605,0.5458771723710507
13 13 400000,Gradient boosting,14.485229661097984,27.850855998019064,0.3478423367115942,0.640877268082553
added results/reproduced/nested_fe.csv +5 −0
@@ -0,0 +1,5 @@
1 +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test,scheme
2 +C3' Municipality + grid-cell FE (~5.5 km),R. Nested spatial FE,17.665652290445884,31.12198554971308,0.37510903497229636,0.5823686750564678,16.57229702106549,32.4806122435959,0.37992439164733227,0.5715774026320801,1.0631574832813349,,,0.8,411283,102929,random
3 +C4' Municipality + grid-cell FE (~1.1 km),R. Nested spatial FE,17.29729629720419,30.733211555044505,0.3721271623880836,0.5889820782402844,16.375718827618094,32.05295192430275,0.3767837714918572,0.5786311816346446,1.0618679756677316,,,0.9,411283,102929,random
4 +C3' Municipality + grid-cell FE (~5.5 km),R. Nested spatial FE,21.540172002578217,31.319651378571518,0.387790697263982,0.4809572799709757,18.969417428034074,31.146859757968027,0.3774487316843718,0.5082727550291839,1.0655936591628847,,,0.7,355824,158388,temporal
5 +C4' Municipality + grid-cell FE (~1.1 km),R. Nested spatial FE,21.551985610380243,31.053622599219104,0.3860253906004687,0.48567211222806106,19.040236558535824,30.77780074936307,0.37492649963695734,0.5148225523371934,1.063943524835854,,,0.8,355824,158388,temporal
added results/reproduced/pairs.csv +27 −0
@@ -0,0 +1,27 @@
1 +scheme,model_a,model_b,mdape_a,mdape_b,diff,diff_lo,diff_hi,dm_ape_t,dm_ape_p,dm_sqlog_t,dm_sqlog_p
2 +random,A2 Semi-log,A1 Linear (levels),19.034569938583708,18.247735441468322,0.786834497115386,0.41777153663505734,1.2226677871864655,-10.686601368220524,0.0,-4.195670171129506,2.720659259725089e-05
3 +random,A6 Semi-log + splines,A2 Semi-log,17.6216157169721,19.034569938583708,-1.4129542216116064,-1.8335429697484,-1.0925363990812282,-10.174542669661014,0.0,-8.898715388706915,0.0
4 +random,A4 Box-Cox,A2 Semi-log,18.567718342220903,19.034569938583708,-0.4668515963628046,-0.5884632595730912,-0.3515787120379936,8.968814092671723,0.0,-0.6913546659882105,0.48934268588585117
5 +random,A3 Log-log,A2 Semi-log,18.00422814105048,19.034569938583708,-1.0303417975332287,-1.4512106300041965,-0.7229610971299634,-7.8961780030161535,2.886579864025407e-15,-10.077109850055015,0.0
6 +random,B2 Year FE,B1 No time effects,19.12497097548873,22.038141454991422,-2.9131704795026927,-3.840391706025526,-2.058685463008183,-8.108170173583066,4.440892098500626e-16,-7.414195626155693,1.2234657731369225e-13
7 +random,B4 Month FE,B3 Quarter FE,19.342062726308214,19.15743025152905,0.1846324747791641,0.009302948022620466,0.36545434556285866,2.8512317585347584,0.0043550214198446735,2.8914469178564333,0.003834723670044893
8 +random,C3 Grid-cell FE (~5.5 km),C2 Municipality FE,18.703553400119855,19.15743025152905,-0.45387685140919487,-1.0674805259265627,0.36564802296516835,-1.1325549920922209,0.25740117846477517,-1.1710051289695664,0.24159671553609452
9 +random,C4 Grid-cell FE (~1.1 km),C2 Municipality FE,20.962941599826827,19.15743025152905,1.8055113482977774,1.2410975614274538,2.3151293348045763,7.123073216501024,1.0556000518135988e-12,7.327255973885574,2.349231920106831e-13
10 +random,D2 Ridge (cross-validated),D1 OLS (muni + month FE),17.684559394393904,19.342062726308214,-1.6575033319143095,-2.3738182373546737,-1.1306026474848994,-5.474659566022061,4.383534979446324e-08,-5.9766089127788495,2.278302435598789e-09
11 +random,D4 Gradient boosting,D3 Random forest,14.388300681085338,14.62750218203232,-0.23920150094698123,-0.45885551656420775,-0.0513885363794455,-4.672835254507822,2.970700491378153e-06,-6.693591526753016,2.177591440499782e-11
12 +random,D4 Gradient boosting,"D5 Gradient boosting, no coordinates",14.388300681085338,23.225090677534933,-8.836789996449594,-9.787971484731155,-8.10748542572112,-10.594870291718431,0.0,-16.137980864260527,0.0
13 +random,D4 Gradient boosting,A1 Linear (levels),14.388300681085338,18.247735441468322,-3.8594347603829835,-5.572402821114921,-2.3810890282818233,-10.403881445314044,0.0,-11.88110067074241,0.0
14 +random,A6 Semi-log + splines,A1 Linear (levels),17.6216157169721,18.247735441468322,-0.6261197244962204,-1.1288670458109282,-0.13177818927939008,-12.297632954401742,0.0,-7.51790454474203,5.573319583618286e-14
15 +temporal,A2 Semi-log,A1 Linear (levels),23.154885105735325,20.369511213481317,2.7853738922540074,1.7788182735293514,3.8803663265918216,0.12007360831229617,0.9044248386245604,3.5326533859479348,0.00041141148678836004
16 +temporal,A6 Semi-log + splines,A2 Semi-log,21.722504213272607,23.154885105735325,-1.4323808924627173,-2.2976528083716783,-0.7670651228422554,-7.796357304870725,6.439293542825908e-15,-7.223963685447007,5.049294315995212e-13
17 +temporal,A4 Box-Cox,A2 Semi-log,20.957181720703545,23.154885105735325,-2.1977033850317795,-3.1466875936389673,-1.4637657862566353,-5.293840398286759,1.197739760705474e-07,-6.972682156866287,3.1095126473701384e-12
18 +temporal,A3 Log-log,A2 Semi-log,21.769614210046917,23.154885105735325,-1.3852708956884072,-2.2367115754092746,-0.7387385691927764,-7.105911135498072,1.1952661083114435e-12,-6.302689050518644,2.9252533728651997e-10
19 +temporal,B2 Year FE,B1 No time effects,23.651696383010147,29.296772516609654,-5.645076133599506,-6.510622685144795,-4.523812050946931,-3.8445189114832927,0.00012078917028457425,-10.351974415660491,0.0
20 +temporal,B4 Month FE,B3 Quarter FE,22.8719130040602,23.30291738895638,-0.4310043848961804,-0.9466994932147289,0.049163437435324885,0.7433560498426371,0.45726614558734036,-0.8851075462469108,0.37609862982402076
21 +temporal,C3 Grid-cell FE (~5.5 km),C2 Municipality FE,24.2625431905187,23.30291738895638,0.9596258015623178,0.16372589677996133,1.8087259847256496,2.296878919423751,0.02162567786746883,1.6301068446540665,0.1030789173649842
22 +temporal,C4 Grid-cell FE (~1.1 km),C2 Municipality FE,25.324930433263678,23.30291738895638,2.022013044307297,0.1532742784899062,3.589326531283127,8.122062031800938,4.440892098500626e-16,4.79314541245226,1.6418652810656909e-06
23 +temporal,D2 Ridge (cross-validated),D1 OLS (muni + month FE),22.257318062238202,22.8719130040602,-0.6145949418219985,-1.25927559041227,-0.16597374782475421,-5.560559072361206,2.6891181059696123e-08,-4.694048528809288,2.6785044802135616e-06
24 +temporal,D4 Gradient boosting,D3 Random forest,21.21243467837447,21.340119821052188,-0.12768514267771636,-0.36811779443404413,0.1456879124981816,-0.9790991544458619,0.3275309894032459,-1.2895287248262932,0.19721433670042376
25 +temporal,D4 Gradient boosting,"D5 Gradient boosting, no coordinates",21.21243467837447,26.207233073287274,-4.994798394912802,-6.500510794984377,-3.1215058615470657,-13.890923788390463,0.0,-11.078685604065148,0.0
26 +temporal,D4 Gradient boosting,A1 Linear (levels),21.21243467837447,20.369511213481317,0.8429234648931541,-1.1895847489303917,2.58981219685756,-3.520049279845853,0.0004314666251414856,-3.9079109150864686,9.309761836728825e-05
27 +temporal,A6 Semi-log + splines,A1 Linear (levels),21.722504213272607,20.369511213481317,1.3529929997912902,0.42539919706467916,2.27707025941844,-6.317663783346769,2.6554691778812867e-10,-6.164003649673723,7.092830767163605e-10
added results/reproduced/relevel.csv +41 −0
@@ -0,0 +1,41 @@
1 +scheme,model,median_log_resid,mdape,mdape_relevelled,mape_relevelled,rmse_ln_centered
2 +random,A1 Linear (levels),0.003941579311327104,18.247735441468322,18.23004891488436,35.69297195166703,0.4202401848567488
3 +random,A2 Semi-log,0.05718358954988467,19.034569938583708,18.231160761954637,34.0869684129567,0.3949373865541228
4 +random,A3 Log-log,0.058544005681735456,18.00422814105048,16.999353648099166,32.573565774441384,0.380703144661566
5 +random,A4 Box-Cox,0.04130325060083884,18.567718342220903,17.954579371141705,34.12392448255233,0.3943404790319506
6 +random,A5 Semi-log + quadratics,0.060482363271615824,19.15743025152905,18.11709775973182,34.346470692604235,0.39562549011238385
7 +random,A6 Semi-log + splines,0.05992694422787537,17.6216157169721,16.524937550487323,32.09956942745145,0.37640447767238644
8 +random,B1 No time effects,0.05783820151938279,22.038141454991422,21.21973153779817,37.60444848077603,0.42017760445683755
9 +random,B2 Year FE,0.060634056819226956,19.12497097548873,18.120790003428947,34.4190972555926,0.39595399652847946
10 +random,B3 Quarter FE,0.060482363271615824,19.15743025152905,18.11709775973182,34.346470692604235,0.39562549011238385
11 +random,B4 Month FE,0.05989058589285712,19.342062726308214,18.354164512138155,34.515768803825544,0.39693480132805525
12 +random,C1 No spatial controls,0.064660279848761,28.785665634746415,28.084183838836008,46.280530679601654,0.4873531649156765
13 +random,C2 Municipality FE,0.060482363271615824,19.15743025152905,18.11709775973182,34.346470692604235,0.39562549011238385
14 +random,C3 Grid-cell FE (~5.5 km),0.06211418728747908,18.703553400119855,17.560654074769303,33.92747210229044,0.3913354959102772
15 +random,C4 Grid-cell FE (~1.1 km),0.07346052233407185,20.962941599826827,19.37902139629324,37.526713965938505,0.41837214294707703
16 +random,D1 OLS (muni + month FE),0.05989058589285712,19.342062726308214,18.354164512138155,34.515768803825544,0.39693480132805525
17 +random,D2 Ridge (cross-validated),0.05859458753725022,17.684559394393904,16.68303730586329,32.182831210501654,0.377254725760277
18 +random,D3 Random forest,0.059849487924283906,14.62750218203232,13.315627506545855,29.149851711862564,0.3571621747065756
19 +random,D4 Gradient boosting,0.06070519978533717,14.388300681085338,12.968025826820522,28.70404274157882,0.3522826247099593
20 +random,"D5 Gradient boosting, no coordinates",0.06518367898153521,23.225090677534933,22.231894512368584,40.34607372325065,0.4445804591769491
21 +random,D6 Spatial k-NN comparables,0.031091961266028534,21.18226600985226,21.08883377849876,37.76094936816919,0.43809139600270497
22 +temporal,A1 Linear (levels),0.1023593997120793,20.369511213481317,17.983628345464886,34.78005470885447,0.4002842762670569
23 +temporal,A2 Semi-log,0.1537984152326084,23.154885105735325,19.250951132236665,34.93065853825993,0.39945237558539437
24 +temporal,A3 Log-log,0.15909109773775665,21.769614210046917,17.266368727247855,32.391262877862694,0.377692222226945
25 +temporal,A4 Box-Cox,0.1258160183976118,20.957181720703545,17.619997411558572,33.74049661940829,0.3889147051380237
26 +temporal,A5 Semi-log + quadratics,0.17129196839132454,23.30291738895638,18.276550062146548,33.989570593169596,0.39008182451924517
27 +temporal,A6 Semi-log + splines,0.1614247324548579,21.722504213272607,16.82576888128111,32.08286270087081,0.37439667304872754
28 +temporal,B1 No time effects,0.2826557411184121,29.296772516609654,19.01095994641687,34.65256914399475,0.3950407735178907
29 +temporal,B2 Year FE,0.17007926384480498,23.651696383010147,18.99713708925793,34.63025036534526,0.39482039546313313
30 +temporal,B3 Quarter FE,0.17129196839132454,23.30291738895638,18.276550062146548,33.989570593169596,0.39008182451924517
31 +temporal,B4 Month FE,0.15736851631187143,22.8719130040602,18.68918931880011,34.336746664507935,0.3927079521397681
32 +temporal,C1 No spatial controls,0.16321364030035745,29.015774167241894,25.986670604102326,42.99352712008042,0.4606526213784797
33 +temporal,C2 Municipality FE,0.17129196839132454,23.30291738895638,18.276550062146548,33.989570593169596,0.39008182451924517
34 +temporal,C3 Grid-cell FE (~5.5 km),0.18220234989278072,24.2625431905187,18.814878020376376,34.76783893858608,0.3949793249194573
35 +temporal,C4 Grid-cell FE (~1.1 km),0.1709061348026415,25.324930433263678,20.522125855535506,38.230322256162516,0.42231730885691315
36 +temporal,D1 OLS (muni + month FE),0.15736851631187143,22.8719130040602,18.68918931880011,34.336746664507935,0.3927079521397681
37 +temporal,D2 Ridge (cross-validated),0.17022474494589535,22.257318062238202,17.037619921036498,32.23095939217706,0.3756435542037618
38 +temporal,D3 Random forest,0.18360250057646788,21.340119821052188,13.905947637222784,29.094464580355663,0.3542908632942389
39 +temporal,D4 Gradient boosting,0.18339005785301854,21.21243467837447,13.865439528037044,29.06582352581495,0.35369501531983066
40 +temporal,"D5 Gradient boosting, no coordinates",0.18807982851052696,26.207233073287274,21.276298850287766,38.44281213948979,0.431047370174503
41 +temporal,D6 Spatial k-NN comparables,0.22093905599935848,25.63120139114109,19.23963333094165,34.78903287000047,0.41913080372044065
added results/reproduced/same_period.csv +21 −0
@@ -0,0 +1,21 @@
1 +model,mdape_random_all,mdape_random_2025_26,mdape_forward_2025_26,n_random_2025_26,n_forward
2 +A1 Linear (levels),18.247735441468322,17.09006288548948,20.369511213481317,31465,158388
3 +A2 Semi-log,19.034569938583708,18.049772389120093,23.154885105735325,31465,158388
4 +A3 Log-log,18.00422814105048,17.301575106616333,21.769614210046917,31465,158388
5 +A4 Box-Cox,18.567718342220903,17.63772856887288,20.957181720703545,31465,158388
6 +A5 Semi-log + quadratics,19.15743025152905,18.496565926231643,23.30291738895638,31465,158388
7 +A6 Semi-log + splines,17.6216157169721,16.986964776398153,21.722504213272607,31465,158388
8 +B1 No time effects,22.038141454991422,25.46195922621089,29.296772516609654,31465,158388
9 +B2 Year FE,19.12497097548873,18.587477688637772,23.651696383010147,31465,158388
10 +B3 Quarter FE,19.15743025152905,18.496565926231643,23.30291738895638,31465,158388
11 +B4 Month FE,19.342062726308214,18.91583380701899,22.8719130040602,31465,158388
12 +C1 No spatial controls,28.785665634746415,26.419945359140563,29.015774167241894,31465,158388
13 +C2 Municipality FE,19.15743025152905,18.496565926231643,23.30291738895638,31465,158388
14 +C3 Grid-cell FE (~5.5 km),18.703553400119855,18.261595423633086,24.2625431905187,31465,158388
15 +C4 Grid-cell FE (~1.1 km),20.962941599826827,19.84981063507152,25.324930433263678,31465,158388
16 +D1 OLS (muni + month FE),19.342062726308214,18.91583380701899,22.8719130040602,31465,158388
17 +D2 Ridge (cross-validated),17.684559394393904,17.098506259471893,22.257318062238202,31465,158388
18 +D3 Random forest,14.62750218203232,13.8581263765865,21.340119821052188,31465,158388
19 +D4 Gradient boosting,14.388300681085338,13.704684482746913,21.21243467837447,31465,158388
20 +"D5 Gradient boosting, no coordinates",23.225090677534933,20.99193671205706,26.207233073287274,31465,158388
21 +D6 Spatial k-NN comparables,21.18226600985226,22.55888464294253,25.63120139114109,31465,158388
modified results/reproduced/segments.csv +7 −7
@@ -1,15 +1,15 @@
1 1 segment,model,n,mdape,mape,rmse_ln,r2_ln
2 −Class: single_family,OLS quadratic,69835,17.15697828017771,35.09144196354416,0.4018837509330969,0.5269797760968626
2 +Class: single_family,OLS quadratic,69835,18.818747368969795,33.62957142912425,0.39648525503248905,0.5396025632666579
3 3 Class: single_family,Gradient boosting,69835,15.08887366879076,29.429305197147464,0.3608165339944521,0.6187132829338855
4 −Class: condo,OLS quadratic,14848,20.42743440870622,27.094418864984917,0.32651881058598764,0.4074914996647663
4 +Class: condo,OLS quadratic,14848,18.197646141652584,24.35300386684896,0.32154382366324963,0.4254093992708138
5 5 Class: condo,Gradient boosting,14848,9.071686847966884,14.485209967611992,0.20596083242967297,0.7642523075225905
6 −Class: plex,OLS quadratic,15115,18.50099130749293,36.98046001501068,0.41103973211699246,0.5454994242149013
6 +Class: plex,OLS quadratic,15115,20.245072332644707,35.37422177148882,0.4052926027997528,0.5581201635953541
7 7 Class: plex,Gradient boosting,15115,16.52761611544543,30.817583909526963,0.36806946728568257,0.635559683430239
8 −Class: cottage,OLS quadratic,2109,33.91947109274449,53.22661347754297,0.5482899168598493,0.3136130552394867
8 +Class: cottage,OLS quadratic,2109,34.25391889121872,49.5560691019703,0.5417910818656525,0.32978799887448207
9 9 Class: cottage,Gradient boosting,2109,32.09640332106328,45.661889354803385,0.5034612691648649,0.4212638262473417
10 −Muni < 1k sales,OLS quadratic,29253,23.257955712558935,42.00606759369818,0.4521889409919308,0.4895905571585859
10 +Muni < 1k sales,OLS quadratic,29253,24.83881625772472,39.847712541492584,0.44558852842475133,0.5043822749230567
11 11 Muni < 1k sales,Gradient boosting,29253,18.686729079471053,32.7580249582477,0.38939943295154483,0.621496886358641
12 −Muni 1k–10k sales,OLS quadratic,37419,15.193131785630404,32.29520768278317,0.3776824491741606,0.5133709854088806
12 +Muni 1k–10k sales,OLS quadratic,37419,16.825938909928517,31.11245095169157,0.3718321556965634,0.5283299710327505
13 13 Muni 1k–10k sales,Gradient boosting,37419,14.504639160801878,28.498332907599565,0.3521566764912874,0.576926089020657
14 −Muni > 10k sales,OLS quadratic,36257,17.82460628493704,30.99741904583036,0.3669850806325136,0.46899139390723055
14 +Muni > 10k sales,OLS quadratic,36257,17.685506151376927,29.004985587875904,0.3625286510651555,0.48180954127111475
15 15 Muni > 10k sales,Gradient boosting,36257,11.91348390081652,22.983932385296278,0.30299483947274597,0.6380278097783667
added results/tables/by_quarter.tex +13 −0
@@ -0,0 +1,13 @@
1 +\begin{tabular}{lrccccc}
2 +\toprule
3 +Test quarter & $n$ & A5 Quadratics & A6 Splines & D2 Ridge & D3 Random forest & D4 Boosting \\
4 +\midrule
5 +2025Q1 & 17,031 & 20.2 & 18.6 & 19.0 & 17.2 & 17.1 \\
6 +2025Q2 & 34,546 & 21.1 & 19.5 & 20.1 & 19.2 & 19.1 \\
7 +2025Q3 & 28,097 & 22.8 & 21.3 & 21.8 & 20.6 & 20.6 \\
8 +2025Q4 & 24,156 & 23.8 & 22.2 & 22.7 & 21.7 & 21.6 \\
9 +2026Q1 & 15,611 & 24.8 & 23.1 & 23.6 & 22.8 & 22.7 \\
10 +2026Q2 & 30,950 & 25.8 & 24.5 & 25.1 & 24.4 & 24.3 \\
11 +2026Q3 & 7,997 & 26.0 & 24.9 & 25.2 & 25.0 & 24.9 \\
12 +\bottomrule
13 +\end{tabular}
\ No newline at end of file
modified results/tables/generalization.tex +5 −5
@@ -4,11 +4,11 @@
4 4 \cmidrule{2-3}\cmidrule{5-6}
5 5 Model & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & $R^2_{\ln}$ \\
6 6 \midrule
7 −A5 Semi-log + quadratics & 18.1 & 0.533 & & 20.5 & 0.479 \\
8 −B4 Month FE & 18.3 & 0.529 & & 20.3 & 0.476 \\
9 −C4 Grid-cell FE ($\sim$1.1 km) & 19.4 & 0.481 & & 22.1 & 0.400 \\
10 −D1 OLS (muni + month FE) & 18.3 & 0.529 & & 20.3 & 0.476 \\
11 −D2 Ridge (cross-validated) & 16.7 & 0.577 & & 19.5 & 0.515 \\
7 +A5 Semi-log + quadratics & 19.2 & 0.546 & & 23.3 & 0.442 \\
8 +B4 Month FE & 19.3 & 0.543 & & 22.9 & 0.445 \\
9 +C4 Grid-cell FE ($\sim$1.1 km) & 21.0 & 0.496 & & 25.3 & 0.364 \\
10 +D1 OLS (muni + month FE) & 19.3 & 0.543 & & 22.9 & 0.445 \\
11 +D2 Ridge (cross-validated) & 17.7 & 0.587 & & 22.3 & 0.480 \\
12 12 D3 Random forest & 14.6 & 0.631 & & 21.3 & 0.519 \\
13 13 D4 Gradient boosting & 14.4 & 0.642 & & 21.2 & 0.521 \\
14 14 D5 Gradient boosting, no coordinates & 23.2 & 0.426 & & 26.2 & 0.310 \\
added results/tables/group_splits.tex +34 −0
@@ -0,0 +1,34 @@
1 +\begin{tabular}{l cc c cc c cc}
2 +\toprule
3 + & \multicolumn{2}{c}{Random 80/20} & & \multicolumn{2}{c}{Parcel-grouped} & & \multicolumn{2}{c}{Building-grouped} \\
4 +\cmidrule{2-3}\cmidrule{5-6}\cmidrule{8-9}
5 +Model & MdAPE & $R^2_{\ln}$ & & MdAPE & $R^2_{\ln}$ & & MdAPE & $R^2_{\ln}$ \\
6 +\midrule
7 +\multicolumn{9}{l}{\itshape A. Functional form}\\
8 +\quad A1 Linear & 18.2 & 0.477 & & 18.4 & 0.469 & & 18.4 & 0.473 \\
9 +\quad A2 Semi-log & 19.0 & 0.546 & & 19.4 & 0.528 & & 19.5 & 0.530 \\
10 +\quad A3 Log-log & 18.0 & 0.580 & & 18.0 & 0.572 & & 18.1 & 0.573 \\
11 +\quad A4 Box--Cox & 18.6 & 0.548 & & 18.2 & 0.534 & & 18.3 & 0.535 \\
12 +\quad A5 Quadratics & 19.2 & 0.546 & & 19.2 & 0.539 & & 19.0 & 0.552 \\
13 +\quad A6 Splines & 17.6 & 0.590 & & 17.6 & 0.581 & & 17.8 & 0.582 \\
14 +\multicolumn{9}{l}{\itshape B. Time effects}\\
15 +\quad B1 No time FE & 22.0 & 0.486 & & 21.6 & 0.489 & & 22.0 & 0.482 \\
16 +\quad B2 Year FE & 19.1 & 0.545 & & 19.0 & 0.545 & & 19.2 & 0.539 \\
17 +\quad B3 Quarter FE & 19.2 & 0.546 & & 19.2 & 0.539 & & 19.0 & 0.552 \\
18 +\quad B4 Month FE & 19.3 & 0.543 & & 19.3 & 0.537 & & 19.4 & 0.538 \\
19 +\multicolumn{9}{l}{\itshape C. Spatial controls}\\
20 +\quad C1 No spatial FE & 28.8 & 0.307 & & 28.9 & 0.301 & & 28.7 & 0.306 \\
21 +\quad C2 Municipality FE & 19.2 & 0.546 & & 19.2 & 0.539 & & 19.0 & 0.552 \\
22 +\quad C3 Grid 5.5 km & 18.7 & 0.557 & & 18.7 & 0.550 & & 18.8 & 0.551 \\
23 +\quad C4 Grid 1.1 km & 21.0 & 0.496 & & 21.1 & 0.491 & & 20.2 & 0.510 \\
24 +\multicolumn{9}{l}{\itshape D. Estimation method}\\
25 +\quad D1 OLS & 19.3 & 0.543 & & 19.3 & 0.537 & & 19.4 & 0.538 \\
26 +\quad D2 Ridge & 17.7 & 0.587 & & 17.7 & 0.580 & & 17.9 & 0.580 \\
27 +\quad D3 Random forest & 14.6 & 0.631 & & 14.6 & 0.620 & & 14.9 & 0.621 \\
28 +\quad D4 Boosting & 14.4 & 0.642 & & 14.5 & 0.632 & & 14.6 & 0.634 \\
29 +\quad D5 Boosting, no coord. & 23.2 & 0.426 & & 23.4 & 0.417 & & 23.3 & 0.420 \\
30 +\quad D6 $k$-NN comparables & 21.2 & 0.434 & & 21.3 & 0.442 & & 22.3 & 0.425 \\
31 +\midrule
32 +Test sales & 102,929 & & & 102,843 & & & 102,949 & \\
33 +\bottomrule
34 +\end{tabular}
\ No newline at end of file
added results/tables/hgb_sensitivity.tex +13 −0
@@ -0,0 +1,13 @@
1 +\begin{tabular}{l cc c cc}
2 +\toprule
3 + & \multicolumn{2}{c}{Random holdout} & & \multicolumn{2}{c}{Forward-in-time} \\
4 +\cmidrule{2-3}\cmidrule{5-6}
5 +Gradient-boosting configuration & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & $R^2_{\ln}$ \\
6 +\midrule
7 +Gradient boosting (baseline) & 14.4 & 0.642 & & 21.2 & 0.521 \\
8 +Shallow (15 leaves, lr 0.05, 1500 iter) & 14.7 & 0.636 & & 21.4 & 0.521 \\
9 +Deep (255 leaves, lr 0.05, 600 iter) & 14.3 & 0.645 & & 21.6 & 0.518 \\
10 +Regularised (min leaf 200, L2 = 1) & 14.3 & 0.643 & & 21.5 & 0.522 \\
11 +Early stopping on last 10\% of training months [700 / 1000 it.] & 14.5 & 0.641 & & 21.2 & 0.524 \\
12 +\bottomrule
13 +\end{tabular}
\ No newline at end of file
modified results/tables/horserace_random.tex +18 −18
@@ -4,30 +4,30 @@ Model & MdAPE (\%) & MAPE (\%) & RMSE$_{\ln}$ & $R^2_{\ln}$ & Fit (s) \\
4 4 \midrule
5 5 \multicolumn{6}{l}{\itshape A. Functional form}\\
6 6 \quad A1 Linear (levels) & 18.2 & 35.6 & 0.420 & 0.477 & 0 \\
7 −\quad A2 Semi-log & 18.2 & 34.5 & 0.397 & 0.533 & 1 \\
8 −\quad A3 Log-log & 17.0 & 32.7 & 0.381 & 0.569 & 1 \\
9 −\quad A4 Box-Cox & 18.6 & 33.0 & 0.390 & 0.548 & 4 \\
10 −\quad A5 Semi-log + quadratics & 18.1 & 34.6 & 0.397 & 0.533 & 1 \\
11 −\quad A6 Semi-log + splines & \textbf{16.5} & 32.1 & 0.376 & 0.579 & 3 \\
7 +\quad A2 Semi-log & 19.0 & 32.7 & 0.391 & 0.546 & 1 \\
8 +\quad A3 Log-log & 18.0 & 31.2 & 0.376 & 0.580 & 1 \\
9 +\quad A4 Box-Cox & 18.6 & 33.0 & 0.390 & 0.548 & 5 \\
10 +\quad A5 Semi-log + quadratics & 19.2 & 32.9 & 0.391 & 0.546 & 1 \\
11 +\quad A6 Semi-log + splines & \textbf{17.6} & 30.8 & 0.372 & 0.590 & 3 \\
12 12 \midrule
13 13 \multicolumn{6}{l}{\itshape B. Time effects}\\
14 −\quad B1 No time effects & 21.1 & 38.4 & 0.423 & 0.468 & 0 \\
15 −\quad B2 Year FE & 18.1 & 34.7 & 0.397 & 0.532 & 1 \\
16 −\quad B3 Quarter FE & \textbf{18.1} & 34.6 & 0.397 & 0.533 & 1 \\
17 −\quad B4 Month FE & 18.3 & 34.8 & 0.398 & 0.529 & 1 \\
14 +\quad B1 No time effects & 22.0 & 35.9 & 0.416 & 0.486 & 0 \\
15 +\quad B2 Year FE & \textbf{19.1} & 32.9 & 0.391 & 0.545 & 1 \\
16 +\quad B3 Quarter FE & 19.2 & 32.9 & 0.391 & 0.546 & 1 \\
17 +\quad B4 Month FE & 19.3 & 33.0 & 0.393 & 0.543 & 1 \\
18 18 \midrule
19 19 \multicolumn{6}{l}{\itshape C. Spatial controls}\\
20 −\quad C1 No spatial controls & 27.9 & 48.4 & 0.494 & 0.274 & 0 \\
21 −\quad C2 Municipality FE & 18.1 & 34.6 & 0.397 & 0.533 & 1 \\
22 −\quad C3 Grid-cell FE ($\sim$5.5 km) & \textbf{17.5} & 34.0 & 0.392 & 0.544 & 1 \\
23 −\quad C4 Grid-cell FE ($\sim$1.1 km) & 19.4 & 37.5 & 0.418 & 0.481 & 1 \\
20 +\quad C1 No spatial controls & 28.8 & 43.7 & 0.483 & 0.307 & 0 \\
21 +\quad C2 Municipality FE & 19.2 & 32.9 & 0.391 & 0.546 & 1 \\
22 +\quad C3 Grid-cell FE ($\sim$5.5 km) & \textbf{18.7} & 32.5 & 0.387 & 0.557 & 1 \\
23 +\quad C4 Grid-cell FE ($\sim$1.1 km) & 21.0 & 35.6 & 0.412 & 0.496 & 1 \\
24 24 \midrule
25 25 \multicolumn{6}{l}{\itshape D. Estimation method}\\
26 −\quad D1 OLS (muni + month FE) & 18.3 & 34.8 & 0.398 & 0.529 & 1 \\
27 −\quad D2 Ridge (cross-validated) & 16.7 & 32.3 & 0.378 & 0.577 & 11 \\
28 −\quad D3 Random forest & 14.6 & 28.2 & 0.353 & 0.631 & 8 \\
29 −\quad D4 Gradient boosting & \textbf{14.4} & 27.8 & 0.347 & 0.642 & 42 \\
30 −\quad D5 Gradient boosting, no coordinates & 23.2 & 38.3 & 0.440 & 0.426 & 18 \\
26 +\quad D1 OLS (muni + month FE) & 19.3 & 33.0 & 0.393 & 0.543 & 1 \\
27 +\quad D2 Ridge (cross-validated) & 17.7 & 30.9 & 0.373 & 0.587 & 11 \\
28 +\quad D3 Random forest & 14.6 & 28.2 & 0.353 & 0.631 & 12 \\
29 +\quad D4 Gradient boosting & \textbf{14.4} & 27.8 & 0.347 & 0.642 & 47 \\
30 +\quad D5 Gradient boosting, no coordinates & 23.2 & 38.3 & 0.440 & 0.426 & 17 \\
31 31 \quad D6 Spatial k-NN comparables & 21.2 & 36.7 & 0.437 & 0.434 & 0 \\
32 32 \bottomrule
33 33 \end{tabular}
\ No newline at end of file
modified results/tables/horserace_temporal.tex +18 −18
@@ -3,31 +3,31 @@
3 3 Model & MdAPE (\%) & MAPE (\%) & RMSE$_{\ln}$ & $R^2_{\ln}$ & Fit (s) \\
4 4 \midrule
5 5 \multicolumn{6}{l}{\itshape A. Functional form}\\
6 −\quad A1 Linear (levels) & 20.4 & 32.9 & 0.397 & 0.455 & 0 \\
7 −\quad A2 Semi-log & 20.6 & 33.1 & 0.396 & 0.458 & 0 \\
8 −\quad A3 Log-log & 19.2 & 30.7 & 0.376 & 0.513 & 1 \\
6 +\quad A1 Linear (levels) & \textbf{20.4} & 32.9 & 0.397 & 0.455 & 0 \\
7 +\quad A2 Semi-log & 23.2 & 32.9 & 0.408 & 0.426 & 0 \\
8 +\quad A3 Log-log & 21.8 & 31.0 & 0.388 & 0.481 & 1 \\
9 9 \quad A4 Box-Cox & 21.0 & 31.9 & 0.390 & 0.476 & 4 \\
10 −\quad A5 Semi-log + quadratics & 20.5 & 32.1 & 0.388 & 0.479 & 1 \\
11 −\quad A6 Semi-log + splines & \textbf{19.1} & 30.5 & 0.372 & 0.522 & 2 \\
10 +\quad A5 Semi-log + quadratics & 23.3 & 32.3 & 0.402 & 0.442 & 1 \\
11 +\quad A6 Semi-log + splines & 21.7 & 30.9 & 0.384 & 0.491 & 2 \\
12 12 \midrule
13 13 \multicolumn{6}{l}{\itshape B. Time effects}\\
14 −\quad B1 No time effects & 25.4 & 33.2 & 0.418 & 0.397 & 0 \\
15 −\quad B2 Year FE & 21.0 & 32.6 & 0.393 & 0.468 & 1 \\
16 −\quad B3 Quarter FE & 20.5 & 32.1 & 0.388 & 0.479 & 1 \\
17 −\quad B4 Month FE & \textbf{20.3} & 32.5 & 0.390 & 0.476 & 1 \\
14 +\quad B1 No time effects & 29.3 & 34.7 & 0.450 & 0.300 & 0 \\
15 +\quad B2 Year FE & 23.7 & 32.7 & 0.406 & 0.431 & 1 \\
16 +\quad B3 Quarter FE & 23.3 & 32.3 & 0.402 & 0.442 & 1 \\
17 +\quad B4 Month FE & \textbf{22.9} & 32.4 & 0.401 & 0.445 & 1 \\
18 18 \midrule
19 19 \multicolumn{6}{l}{\itshape C. Spatial controls}\\
20 −\quad C1 No spatial controls & 26.5 & 41.1 & 0.457 & 0.279 & 0 \\
21 −\quad C2 Municipality FE & \textbf{20.5} & 32.1 & 0.388 & 0.479 & 1 \\
22 −\quad C3 Grid-cell FE ($\sim$5.5 km) & 21.3 & 32.7 & 0.394 & 0.465 & 1 \\
23 −\quad C4 Grid-cell FE ($\sim$1.1 km) & 22.1 & 36.0 & 0.417 & 0.400 & 1 \\
20 +\quad C1 No spatial controls & 29.0 & 38.9 & 0.469 & 0.242 & 0 \\
21 +\quad C2 Municipality FE & \textbf{23.3} & 32.3 & 0.402 & 0.442 & 1 \\
22 +\quad C3 Grid-cell FE ($\sim$5.5 km) & 24.3 & 33.0 & 0.409 & 0.423 & 1 \\
23 +\quad C4 Grid-cell FE ($\sim$1.1 km) & 25.3 & 35.5 & 0.429 & 0.364 & 1 \\
24 24 \midrule
25 25 \multicolumn{6}{l}{\itshape D. Estimation method}\\
26 −\quad D1 OLS (muni + month FE) & 20.3 & 32.5 & 0.390 & 0.476 & 1 \\
27 −\quad D2 Ridge (cross-validated) & \textbf{19.5} & 30.6 & 0.375 & 0.515 & 9 \\
28 −\quad D3 Random forest & 21.3 & 29.8 & 0.373 & 0.519 & 7 \\
29 −\quad D4 Gradient boosting & 21.2 & 29.7 & 0.372 & 0.521 & 42 \\
30 −\quad D5 Gradient boosting, no coordinates & 26.2 & 35.7 & 0.447 & 0.310 & 21 \\
26 +\quad D1 OLS (muni + month FE) & 22.9 & 32.4 & 0.401 & 0.445 & 1 \\
27 +\quad D2 Ridge (cross-validated) & 22.3 & 31.1 & 0.388 & 0.480 & 8 \\
28 +\quad D3 Random forest & 21.3 & 29.8 & 0.373 & 0.519 & 11 \\
29 +\quad D4 Gradient boosting & \textbf{21.2} & 29.7 & 0.372 & 0.521 & 41 \\
30 +\quad D5 Gradient boosting, no coordinates & 26.2 & 35.7 & 0.447 & 0.310 & 22 \\
31 31 \quad D6 Spatial k-NN comparables & 25.6 & 34.2 & 0.465 & 0.253 & 0 \\
32 32 \bottomrule
33 33 \end{tabular}
\ No newline at end of file
added results/tables/inference.tex +32 −0
@@ -0,0 +1,32 @@
1 +\begin{tabular}{l ccc c ccc}
2 +\toprule
3 + & \multicolumn{3}{c}{Random holdout} & & \multicolumn{3}{c}{Forward-in-time holdout} \\
4 +\cmidrule{2-4}\cmidrule{6-8}
5 +Model & MdAPE [95\% CI] & $\Delta$ vs.\ A6 [95\% CI] & $t_{\mathrm{DM}}$ & & MdAPE [95\% CI] & $\Delta$ vs.\ A6 [95\% CI] & $t_{\mathrm{DM}}$ \\
6 +\midrule
7 +\multicolumn{8}{l}{\itshape A. Functional form}\\
8 +\quad A1 Linear & 18.2 [16.9, 19.6] & +0.6 [+0.1, +1.1] & 12.3$^{***}$ & & 20.4 [19.1, 21.7] & -1.4 [-2.3, -0.4] & 6.3$^{***}$ \\
9 +\quad A2 Semi-log & 19.0 [17.9, 20.2] & +1.4 [+1.1, +1.8] & 10.2$^{***}$ & & 23.2 [21.7, 24.7] & +1.4 [+0.8, +2.3] & 7.8$^{***}$ \\
10 +\quad A3 Log-log & 18.0 [17.1, 19.0] & +0.4 [+0.3, +0.5] & 4.6$^{***}$ & & 21.8 [20.5, 23.2] & +0.0 [-0.0, +0.2] & 1.8$^{*}$ \\
11 +\quad A4 Box--Cox & 18.6 [17.5, 19.7] & +0.9 [+0.7, +1.3] & 10.7$^{***}$ & & 21.0 [19.7, 22.2] & -0.8 [-1.4, -0.1] & 4.3$^{***}$ \\
12 +\quad A5 Quadratics & 19.2 [18.2, 20.2] & +1.5 [+1.1, +2.1] & 5.5$^{***}$ & & 23.3 [21.9, 24.9] & +1.6 [+1.1, +2.2] & 6.1$^{***}$ \\
13 +\quad A6 Splines & 17.6 [16.7, 18.6] & --- & --- & & 21.7 [20.5, 23.1] & --- & --- \\
14 +\multicolumn{8}{l}{\itshape B. Time effects}\\
15 +\quad B1 No time FE & 22.0 [21.1, 23.3] & +4.4 [+3.4, +5.6] & 8.2$^{***}$ & & 29.3 [27.3, 31.5] & +7.6 [+6.0, +9.0] & 5.3$^{***}$ \\
16 +\quad B2 Year FE & 19.1 [18.1, 20.2] & +1.5 [+1.1, +2.2] & 6.1$^{***}$ & & 23.7 [22.2, 25.3] & +1.9 [+1.3, +2.8] & 6.2$^{***}$ \\
17 +\quad B3 Quarter FE & 19.2 [18.2, 20.2] & +1.5 [+1.1, +2.1] & 5.5$^{***}$ & & 23.3 [21.9, 24.9] & +1.6 [+1.1, +2.2] & 6.1$^{***}$ \\
18 +\quad B4 Month FE & 19.3 [18.4, 20.4] & +1.7 [+1.2, +2.5] & 5.7$^{***}$ & & 22.9 [21.5, 24.4] & +1.1 [+0.7, +1.8] & 6.0$^{***}$ \\
19 +\multicolumn{8}{l}{\itshape C. Spatial controls}\\
20 +\quad C1 No spatial FE & 28.8 [27.3, 30.1] & +11.2 [+10.1, +12.3] & 7.7$^{***}$ & & 29.0 [27.4, 30.3] & +7.3 [+5.0, +9.2] & 11.1$^{***}$ \\
21 +\quad C2 Municipality FE & 19.2 [18.2, 20.2] & +1.5 [+1.1, +2.1] & 5.5$^{***}$ & & 23.3 [21.9, 24.9] & +1.6 [+1.1, +2.2] & 6.1$^{***}$ \\
22 +\quad C3 Grid 5.5 km & 18.7 [17.8, 20.0] & +1.1 [+0.2, +2.3] & 2.5$^{**}$ & & 24.3 [22.5, 26.4] & +2.5 [+1.6, +3.5] & 4.5$^{***}$ \\
23 +\quad C4 Grid 1.1 km & 21.0 [20.1, 21.9] & +3.3 [+2.9, +3.8] & 6.8$^{***}$ & & 25.3 [24.3, 26.3] & +3.6 [+2.1, +4.9] & 12.4$^{***}$ \\
24 +\multicolumn{8}{l}{\itshape D. Estimation method}\\
25 +\quad D1 OLS & 19.3 [18.4, 20.4] & +1.7 [+1.2, +2.5] & 5.7$^{***}$ & & 22.9 [21.5, 24.4] & +1.1 [+0.7, +1.8] & 6.0$^{***}$ \\
26 +\quad D2 Ridge & 17.7 [16.8, 18.7] & +0.1 [-0.0, +0.2] & 2.6$^{***}$ & & 22.3 [21.0, 23.7] & +0.5 [+0.5, +0.6] & 6.6$^{***}$ \\
27 +\quad D3 Random forest & 14.6 [13.7, 16.2] & -3.0 [-4.6, -1.5] & -2.9$^{***}$ & & 21.3 [19.9, 23.0] & -0.4 [-1.5, +0.6] & -1.6 \\
28 +\quad D4 Boosting & 14.4 [13.7, 15.7] & -3.2 [-4.7, -1.9] & -3.6$^{***}$ & & 21.2 [19.7, 22.9] & -0.5 [-1.7, +0.6] & -1.6 \\
29 +\quad D5 Boosting, no coord. & 23.2 [22.1, 25.1] & +5.6 [+3.9, +7.5] & 4.5$^{***}$ & & 26.2 [25.2, 27.5] & +4.5 [+2.9, +6.2] & 5.9$^{***}$ \\
30 +\quad D6 $k$-NN comparables & 21.2 [19.4, 23.5] & +3.6 [+1.0, +5.9] & 4.8$^{***}$ & & 25.6 [23.1, 28.3] & +3.9 [+1.8, +5.6] & 4.0$^{***}$ \\
31 +\bottomrule
32 +\end{tabular}
\ No newline at end of file
added results/tables/leakage.tex +21 −0
@@ -0,0 +1,21 @@
1 +\begin{tabular}{l cc c cc}
2 +\toprule
3 + & \multicolumn{2}{c}{Same parcel in training} & & \multicolumn{2}{c}{Same coordinates in training} \\
4 +\cmidrule{2-3}\cmidrule{5-6}
5 +Model & Yes & No & & Yes & No \\
6 +\midrule
7 +\multicolumn{6}{l}{\itshape All test sales}\\
8 +\quad A5 Quadratics & 19.8 & 19.1 & & 19.2 & 19.2 \\
9 +\quad A6 Splines & 18.3 & 17.5 & & 17.8 & 17.6 \\
10 +\quad D3 Random forest & 14.6 & 14.6 & & 11.7 & 15.7 \\
11 +\quad D4 Boosting & 14.5 & 14.4 & & 12.0 & 15.3 \\
12 +\quad D5 Boosting, no coord. & 22.9 & 23.3 & & 20.0 & 24.5 \\
13 +\quad D6 $k$-NN comparables & 22.2 & 21.0 & & 16.7 & 22.9 \\
14 +\quad Test sales & 17,219 & 85,710 & & 27,475 & 75,454 \\
15 +\midrule
16 +\multicolumn{6}{l}{\itshape Condominiums only}\\
17 +\quad A5 Quadratics & 17.4 & 18.4 & & 18.0 & 20.3 \\
18 +\quad D4 Boosting & 9.0 & 9.1 & & 8.9 & 10.2 \\
19 +\quad Test sales & 3,374 & 11,474 & & 13,261 & 1,587 \\
20 +\bottomrule
21 +\end{tabular}
\ No newline at end of file
modified results/tables/learning.tex +6 −6
@@ -2,11 +2,11 @@
2 2 \toprule
3 3 Training sales & Gradient boosting & OLS quadratic (muni+quarter FE) \\
4 4 \midrule
5 −10,000 & 17.6 & 18.4 \\
6 −25,000 & 16.1 & 18.1 \\
7 −50,000 & 15.5 & 17.9 \\
8 −100,000 & 15.2 & 17.9 \\
9 −200,000 & 14.8 & 17.9 \\
10 −400,000 & 14.5 & 18.1 \\
5 +10,000 & 17.6 & 19.1 \\
6 +25,000 & 16.1 & 19.1 \\
7 +50,000 & 15.5 & 18.8 \\
8 +100,000 & 15.2 & 18.8 \\
9 +200,000 & 14.8 & 19.0 \\
10 +400,000 & 14.5 & 19.2 \\
11 11 \bottomrule
12 12 \end{tabular}
\ No newline at end of file
added results/tables/nested_fe.tex +13 −0
@@ -0,0 +1,13 @@
1 +\begin{tabular}{l cc c cc}
2 +\toprule
3 + & \multicolumn{2}{c}{Random holdout} & & \multicolumn{2}{c}{Forward-in-time} \\
4 +\cmidrule{2-3}\cmidrule{5-6}
5 +Spatial controls & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & $R^2_{\ln}$ \\
6 +\midrule
7 +C2 Municipality FE (1,115 cells) & 19.2 & 0.546 & & 23.3 & 0.442 \\
8 +C3 Grid 5.5 km (5,255 cells) & 18.7 & 0.557 & & 24.3 & 0.423 \\
9 +C4 Grid 1.1 km (30,548 cells) & 21.0 & 0.496 & & 25.3 & 0.364 \\
10 +C3$'$ Municipality + grid 5.5 km & 17.7 & 0.582 & & 21.5 & 0.481 \\
11 +C4$'$ Municipality + grid 1.1 km & 17.3 & 0.589 & & 21.6 & 0.486 \\
12 +\bottomrule
13 +\end{tabular}
\ No newline at end of file
added results/tables/pairs.tex +21 −0
@@ -0,0 +1,21 @@
1 +\begin{tabular}{ll c cc c cc}
2 +\toprule
3 + & & & \multicolumn{2}{c}{Random holdout} & & \multicolumn{2}{c}{Forward-in-time} \\
4 +\cmidrule{4-5}\cmidrule{7-8}
5 +Comparison & & MdAPE & $\Delta$MdAPE [95\% CI] & $t_{\mathrm{DM}}$ APE / sq.\ log & & $\Delta$MdAPE [95\% CI] & $t_{\mathrm{DM}}$ APE / sq.\ log \\
6 +\midrule
7 +A2 Semi-log vs. & A1 Linear & 19.0 / 18.2 & +0.79 [+0.42, +1.22] & -10.7$^{***}$ / -4.2$^{***}$ & & +2.79 [+1.78, +3.88] & 0.1 / 3.5$^{***}$ \\
8 +A6 Splines vs. & A2 Semi-log & 17.6 / 19.0 & -1.41 [-1.83, -1.09] & -10.2$^{***}$ / -8.9$^{***}$ & & -1.43 [-2.30, -0.77] & -7.8$^{***}$ / -7.2$^{***}$ \\
9 +A4 Box--Cox vs. & A2 Semi-log & 18.6 / 19.0 & -0.47 [-0.59, -0.35] & 9.0$^{***}$ / -0.7 & & -2.20 [-3.15, -1.46] & -5.3$^{***}$ / -7.0$^{***}$ \\
10 +A3 Log-log vs. & A2 Semi-log & 18.0 / 19.0 & -1.03 [-1.45, -0.72] & -7.9$^{***}$ / -10.1$^{***}$ & & -1.39 [-2.24, -0.74] & -7.1$^{***}$ / -6.3$^{***}$ \\
11 +B2 Year FE vs. & B1 No time FE & 19.1 / 22.0 & -2.91 [-3.84, -2.06] & -8.1$^{***}$ / -7.4$^{***}$ & & -5.65 [-6.51, -4.52] & -3.8$^{***}$ / -10.4$^{***}$ \\
12 +B4 Month FE vs. & B3 Quarter FE & 19.3 / 19.2 & +0.18 [+0.01, +0.37] & 2.9$^{***}$ / 2.9$^{***}$ & & -0.43 [-0.95, +0.05] & 0.7 / -0.9 \\
13 +C3 Grid 5.5 km vs. & C2 Municipality FE & 18.7 / 19.2 & -0.45 [-1.07, +0.37] & -1.1 / -1.2 & & +0.96 [+0.16, +1.81] & 2.3$^{**}$ / 1.6 \\
14 +C4 Grid 1.1 km vs. & C2 Municipality FE & 21.0 / 19.2 & +1.81 [+1.24, +2.32] & 7.1$^{***}$ / 7.3$^{***}$ & & +2.02 [+0.15, +3.59] & 8.1$^{***}$ / 4.8$^{***}$ \\
15 +D2 Ridge vs. & D1 OLS & 17.7 / 19.3 & -1.66 [-2.37, -1.13] & -5.5$^{***}$ / -6.0$^{***}$ & & -0.61 [-1.26, -0.17] & -5.6$^{***}$ / -4.7$^{***}$ \\
16 +D4 Boosting vs. & D3 Random forest & 14.4 / 14.6 & -0.24 [-0.46, -0.05] & -4.7$^{***}$ / -6.7$^{***}$ & & -0.13 [-0.37, +0.15] & -1.0 / -1.3 \\
17 +D4 Boosting vs. & D5 Boosting, no coord. & 14.4 / 23.2 & -8.84 [-9.79, -8.11] & -10.6$^{***}$ / -16.1$^{***}$ & & -4.99 [-6.50, -3.12] & -13.9$^{***}$ / -11.1$^{***}$ \\
18 +D4 Boosting vs. & A1 Linear & 14.4 / 18.2 & -3.86 [-5.57, -2.38] & -10.4$^{***}$ / -11.9$^{***}$ & & +0.84 [-1.19, +2.59] & -3.5$^{***}$ / -3.9$^{***}$ \\
19 +A6 Splines vs. & A1 Linear & 17.6 / 18.2 & -0.63 [-1.13, -0.13] & -12.3$^{***}$ / -7.5$^{***}$ & & +1.35 [+0.43, +2.28] & -6.3$^{***}$ / -6.2$^{***}$ \\
20 +\bottomrule
21 +\end{tabular}
\ No newline at end of file
added results/tables/retransformation.tex +18 −0
@@ -0,0 +1,18 @@
1 +\begin{tabular}{l c cc cc c cc cc}
2 +\toprule
3 + & & \multicolumn{4}{c}{Random holdout} & & \multicolumn{4}{c}{Forward-in-time holdout} \\
4 +\cmidrule{3-6}\cmidrule{8-11}
5 + & & \multicolumn{2}{c}{MdAPE (\%)} & \multicolumn{2}{c}{$R^2_{\ln}$} & & \multicolumn{2}{c}{MdAPE (\%)} & \multicolumn{2}{c}{$R^2_{\ln}$} \\
6 +\cmidrule{3-4}\cmidrule{5-6}\cmidrule{8-9}\cmidrule{10-11}
7 +Model & $\hat{s}$ & Median & Mean & Median & Mean & & Median & Mean & Median & Mean \\
8 +\midrule
9 +A2 Semi-log & 1.070 & 19.0 & 18.2 & 0.546 & 0.533 & & 23.2 & 20.6 & 0.426 & 0.458 \\
10 +A3 Log-log & 1.064 & 18.0 & 17.0 & 0.580 & 0.569 & & 21.8 & 19.2 & 0.481 & 0.513 \\
11 +A4 Box--Cox & --- & 18.6 & 17.9 & 0.548 & 0.536 & & 21.0 & 19.6 & 0.476 & 0.490 \\
12 +A5 Quadratics & 1.070 & 19.2 & 18.1 & 0.546 & 0.533 & & 23.3 & 20.5 & 0.442 & 0.479 \\
13 +A6 Splines & 1.062 & 17.6 & 16.5 & 0.590 & 0.579 & & 21.7 & 19.1 & 0.491 & 0.522 \\
14 +D2 Ridge & 1.063 & 17.7 & 16.7 & 0.587 & 0.577 & & 22.3 & 19.5 & 0.480 & 0.515 \\
15 +D3 Random forest & 1.032 & 14.6 & 13.7 & 0.631 & 0.629 & & 21.3 & 19.4 & 0.519 & 0.544 \\
16 +D4 Boosting & 1.048 & 14.4 & 13.1 & 0.642 & 0.636 & & 21.2 & 18.4 & 0.521 & 0.556 \\
17 +\bottomrule
18 +\end{tabular}
\ No newline at end of file
added results/tables/same_period.tex +21 −0
@@ -0,0 +1,21 @@
1 +\begin{tabular}{l cc c c c c}
2 +\toprule
3 + & \multicolumn{2}{c}{Random split} & Forward split & Staleness & Level error & Re-levelled \\
4 +\cmidrule{2-3}\cmidrule{4-4}
5 +Model & All test sales & 2025--26 sales & 2025--26 sales & cost & $\hat{b}$ & MdAPE \\
6 +\midrule
7 +A1 Linear & 18.2 & 17.1 & 20.4 & +3.3 & +0.102 & 18.0 \\
8 +A5 Quadratics & 19.2 & 18.5 & 23.3 & +4.8 & +0.171 & 18.3 \\
9 +A6 Splines & 17.6 & 17.0 & 21.7 & +4.7 & +0.161 & 16.8 \\
10 +B4 Month FE & 19.3 & 18.9 & 22.9 & +4.0 & +0.157 & 18.7 \\
11 +C3 Grid 5.5 km & 18.7 & 18.3 & 24.3 & +6.0 & +0.182 & 18.8 \\
12 +C4 Grid 1.1 km & 21.0 & 19.8 & 25.3 & +5.5 & +0.171 & 20.5 \\
13 +D2 Ridge & 17.7 & 17.1 & 22.3 & +5.2 & +0.170 & 17.0 \\
14 +D3 Random forest & 14.6 & 13.9 & 21.3 & +7.5 & +0.184 & 13.9 \\
15 +D4 Boosting & 14.4 & 13.7 & 21.2 & +7.5 & +0.183 & 13.9 \\
16 +D5 Boosting, no coord. & 23.2 & 21.0 & 26.2 & +5.2 & +0.188 & 21.3 \\
17 +D6 $k$-NN comparables & 21.2 & 22.6 & 25.6 & +3.1 & +0.221 & 19.2 \\
18 +\midrule
19 +Test sales & 102,929 & 31,465 & 158,388 & & & \\
20 +\bottomrule
21 +\end{tabular}
\ No newline at end of file
modified results/tables/segments.tex +7 −7
@@ -2,12 +2,12 @@
2 2 \toprule
3 3 Segment & $n$ (test) & OLS quadratic & Gradient boosting \\
4 4 \midrule
5 −Condo & 14,848 & 20.4 & 9.1 \\
6 −Cottage & 2,109 & 33.9 & 32.1 \\
7 −Plex & 15,115 & 18.5 & 16.5 \\
8 −Single family & 69,835 & 17.2 & 15.1 \\
9 −Muni 1k--10k sales & 37,419 & 15.2 & 14.5 \\
10 −Muni $<$ 1k sales & 29,253 & 23.3 & 18.7 \\
11 −Muni $>$ 10k sales & 36,257 & 17.8 & 11.9 \\
5 +Condo & 14,848 & 18.2 & 9.1 \\
6 +Cottage & 2,109 & 34.3 & 32.1 \\
7 +Plex & 15,115 & 20.2 & 16.5 \\
8 +Single family & 69,835 & 18.8 & 15.1 \\
9 +Muni 1k--10k sales & 37,419 & 16.8 & 14.5 \\
10 +Muni $<$ 1k sales & 29,253 & 24.8 & 18.7 \\
11 +Muni $>$ 10k sales & 36,257 & 17.7 & 11.9 \\
12 12 \bottomrule
13 13 \end{tabular}
\ No newline at end of file
modified scripts/02_horserace.py +7 −2
@@ -7,7 +7,9 @@ Fits the twenty registry models under (i) a random 80/20 holdout and
7 7 Box–Cox profile likelihood and summary statistics.
8 8
9 9 Writes: horserace_random.csv, horserace_temporal.csv, boxcox.csv,
10 −summary_stats.csv, sample_counts.csv
10 +summary_stats.csv, sample_counts.csv, and the test-set predictions of
11 +every model (data/processed/predictions_<scheme>.parquet, not committed)
12 +used by scripts/06_robustness.py for the inference and leakage analyses.
11 13
12 14 Usage: python scripts/02_horserace.py
13 15 """
@@ -56,8 +58,11 @@ def main() -> None:
56 58
57 59 # ------------------------------------------------------------ the race
58 60 for scheme in ("random", "temporal"):
59 − res = models.run_horserace(df, scheme)
61 + res, preds = models.run_horserace(df, scheme,
62 + return_predictions=True)
60 63 res.to_csv(out / f"horserace_{scheme}.csv", index=False)
64 + preds.to_parquet(config.PROCESSED / f"predictions_{scheme}.parquet",
65 + index=False)
61 66
62 67
63 68 if __name__ == "__main__":
modified scripts/04_make_figures.py +33 −0
@@ -186,6 +186,38 @@ def fig_segments():
186 186 _save(fig, "fig_segments.png")
187 187
188 188
189 +# ---------------------------------------------------------------- fig 10
190 +def fig_horizon():
191 + """Forward-in-time MdAPE by test quarter (distance from the last
192 + training month), selected models."""
193 + f = RES / "by_quarter.csv"
194 + if not f.exists():
195 + return
196 + q = pd.read_csv(f)
197 + fig, ax = plt.subplots(figsize=(0.8 * TEXTWIDTH, 3.2))
198 + series = (("A6 Semi-log + splines", BLUE, "o", "-", "A6 Splines"),
199 + ("A5 Semi-log + quadratics", BLUES[2], "o", (0, (5, 2)),
200 + "A5 Quadratics"),
201 + ("D2 Ridge (cross-validated)", BLUES[5], "^", (0, (1, 1.2)),
202 + "D2 Ridge"),
203 + ("D4 Gradient boosting", RED, "s", "-", "D4 Boosting"),
204 + ("D3 Random forest", "#e08a8a", "s", (0, (5, 2)),
205 + "D3 Random forest"))
206 + for m, color, mk, ls, label in series:
207 + g = q[q["model"] == m].sort_values("quarter")
208 + ax.plot(range(len(g)), g["mdape"], marker=mk, ms=4, color=color,
209 + ls=ls, lw=1.1, mfc="white" if mk == "s" else color,
210 + label=label)
211 + g = q[q["model"] == series[0][0]].sort_values("quarter")
212 + ax.set_xticks(range(len(g)))
213 + ax.set_xticklabels(g["quarter"], fontsize=8, rotation=0)
214 + ax.set_xlabel("Test quarter (training ends December 2024)")
215 + ax.set_ylabel("Median absolute error (%)")
216 + ax.legend(fontsize=7.5, ncol=2, loc="upper left")
217 + ygrid(ax)
218 + _save(fig, "fig_horizon.png")
219 +
220 +
189 221 def main() -> None:
190 222 config.ensure_dirs()
191 223 apply_style()
@@ -195,6 +227,7 @@ def main() -> None:
195 227 fig_profiles()
196 228 fig_learning()
197 229 fig_segments()
230 + fig_horizon()
198 231 made = sorted(p.name for p in OUT.glob("fig_*.png"))
199 232 print(f"{len(made)} figures written:", ", ".join(made))
200 233
modified scripts/05_make_tables.py +306 −0
@@ -133,6 +133,305 @@ def t_segments():
133 133 (TAB / "segments.tex").write_text("\n".join(lines))
134 134
135 135
136 +# ---------------------------------------------------------------- revision 1.1
137 +SHORT = {"A1 Linear (levels)": "A1 Linear", "A2 Semi-log": "A2 Semi-log",
138 + "A3 Log-log": "A3 Log-log", "A4 Box-Cox": "A4 Box--Cox",
139 + "A5 Semi-log + quadratics": "A5 Quadratics",
140 + "A6 Semi-log + splines": "A6 Splines",
141 + "B1 No time effects": "B1 No time FE", "B2 Year FE": "B2 Year FE",
142 + "B3 Quarter FE": "B3 Quarter FE", "B4 Month FE": "B4 Month FE",
143 + "C1 No spatial controls": "C1 No spatial FE",
144 + "C2 Municipality FE": "C2 Municipality FE",
145 + "C3 Grid-cell FE (~5.5 km)": "C3 Grid 5.5 km",
146 + "C4 Grid-cell FE (~1.1 km)": "C4 Grid 1.1 km",
147 + "D1 OLS (muni + month FE)": "D1 OLS", "D2 Ridge (cross-validated)":
148 + "D2 Ridge", "D3 Random forest": "D3 Random forest",
149 + "D4 Gradient boosting": "D4 Boosting",
150 + "D5 Gradient boosting, no coordinates": "D5 Boosting, no coord.",
151 + "D6 Spatial k-NN comparables": "D6 $k$-NN comparables"}
152 +A6, D4 = "A6 Semi-log + splines", "D4 Gradient boosting"
153 +
154 +
155 +def _stars(p):
156 + return "$^{***}$" if p < 0.01 else "$^{**}$" if p < 0.05 else \
157 + "$^{*}$" if p < 0.10 else ""
158 +
159 +
160 +def t_inference():
161 + """MdAPE with cluster-bootstrap 95% CI, difference versus the spline
162 + hedonic (A6) with its CI, and the clustered DM t statistic on the APE
163 + loss differential, under both holdouts."""
164 + lines = [r"\begin{tabular}{l ccc c ccc}", r"\toprule",
165 + r" & \multicolumn{3}{c}{Random holdout} & &"
166 + r" \multicolumn{3}{c}{Forward-in-time holdout} \\",
167 + r"\cmidrule{2-4}\cmidrule{6-8}",
168 + r"Model & MdAPE [95\% CI] & $\Delta$ vs.\ A6 [95\% CI] & "
169 + r"$t_{\mathrm{DM}}$ & & MdAPE [95\% CI] & "
170 + r"$\Delta$ vs.\ A6 [95\% CI] & $t_{\mathrm{DM}}$ \\",
171 + r"\midrule"]
172 + R = {s: pd.read_csv(RES / f"inference_{s}.csv").set_index("model")
173 + for s in ("random", "temporal")}
174 + groups = {"A": "A. Functional form", "B": "B. Time effects",
175 + "C": "C. Spatial controls", "D": "D. Estimation method"}
176 + cur = None
177 + for m in SHORT:
178 + if m[0] != cur:
179 + cur = m[0]
180 + lines.append(r"\multicolumn{8}{l}{\itshape " + groups[cur]
181 + + r"}\\")
182 + cells = [r"\quad " + SHORT[m]]
183 + for s in ("random", "temporal"):
184 + x = R[s].loc[m]
185 + cells.append(f"{x['mdape']:.1f} [{x['ci_lo']:.1f}, "
186 + f"{x['ci_hi']:.1f}]")
187 + if m == A6:
188 + cells += ["---", "---"]
189 + else:
190 + cells.append(f"{x['diff_vs_ref']:+.1f} [{x['diff_lo']:+.1f}, "
191 + f"{x['diff_hi']:+.1f}]")
192 + cells.append(f"{x['dm_ape_vs_a6_t']:.1f}"
193 + + _stars(x['dm_ape_vs_a6_p']))
194 + if s == "random":
195 + cells.append("")
196 + lines.append(" & ".join(cells) + r" \\")
197 + lines += [r"\bottomrule", r"\end{tabular}"]
198 + (TAB / "inference.tex").write_text("\n".join(lines))
199 +
200 +
201 +def t_retransformation():
202 + """Median-consistent versus Duan-smeared (mean-consistent) predictions."""
203 + sel = ["A2 Semi-log", "A3 Log-log", "A4 Box-Cox",
204 + "A5 Semi-log + quadratics", "A6 Semi-log + splines",
205 + "D2 Ridge (cross-validated)", "D3 Random forest",
206 + "D4 Gradient boosting"]
207 + R = {s: pd.read_csv(RES / f"horserace_{s}.csv").set_index("name")
208 + for s in ("random", "temporal")}
209 + lines = [r"\begin{tabular}{l c cc cc c cc cc}", r"\toprule",
210 + r" & & \multicolumn{4}{c}{Random holdout} & &"
211 + r" \multicolumn{4}{c}{Forward-in-time holdout} \\",
212 + r"\cmidrule{3-6}\cmidrule{8-11}",
213 + r" & & \multicolumn{2}{c}{MdAPE (\%)} & "
214 + r"\multicolumn{2}{c}{$R^2_{\ln}$} & & "
215 + r"\multicolumn{2}{c}{MdAPE (\%)} & "
216 + r"\multicolumn{2}{c}{$R^2_{\ln}$} \\",
217 + r"\cmidrule{3-4}\cmidrule{5-6}\cmidrule{8-9}\cmidrule{10-11}",
218 + r"Model & $\hat{s}$ & Median & Mean & Median & Mean & & "
219 + r"Median & Mean & Median & Mean \\", r"\midrule"]
220 + for m in sel:
221 + a, b = R["random"].loc[m], R["temporal"].loc[m]
222 + sm = "---" if pd.isna(a["smear"]) else f"{a['smear']:.3f}"
223 + lines.append(
224 + f"{SHORT[m]} & {sm} & {a['mdape']:.1f} & {a['mdape_mean']:.1f} & "
225 + f"{a['r2_ln']:.3f} & {a['r2_ln_mean']:.3f} & & "
226 + f"{b['mdape']:.1f} & {b['mdape_mean']:.1f} & "
227 + f"{b['r2_ln']:.3f} & {b['r2_ln_mean']:.3f} \\\\")
228 + lines += [r"\bottomrule", r"\end{tabular}"]
229 + (TAB / "retransformation.tex").write_text("\n".join(lines))
230 +
231 +
232 +def t_same_period():
233 + sp = pd.read_csv(RES / "same_period.csv").set_index("model")
234 + rl = pd.read_csv(RES / "relevel.csv")
235 + rl = rl[rl["scheme"] == "temporal"].set_index("model")
236 + sel = ["A1 Linear (levels)", "A5 Semi-log + quadratics",
237 + "A6 Semi-log + splines", "B4 Month FE",
238 + "C3 Grid-cell FE (~5.5 km)", "C4 Grid-cell FE (~1.1 km)",
239 + "D2 Ridge (cross-validated)", "D3 Random forest",
240 + "D4 Gradient boosting", "D5 Gradient boosting, no coordinates",
241 + "D6 Spatial k-NN comparables"]
242 + lines = [r"\begin{tabular}{l cc c c c c}", r"\toprule",
243 + r" & \multicolumn{2}{c}{Random split} & Forward split & "
244 + r"Staleness & Level error & Re-levelled \\",
245 + r"\cmidrule{2-3}\cmidrule{4-4}",
246 + r"Model & All test sales & 2025--26 sales & 2025--26 sales & "
247 + r"cost & $\hat{b}$ & MdAPE \\", r"\midrule"]
248 + for m in sel:
249 + x = sp.loc[m]
250 + lines.append(f"{SHORT[m]} & {x['mdape_random_all']:.1f} & "
251 + f"{x['mdape_random_2025_26']:.1f} & "
252 + f"{x['mdape_forward_2025_26']:.1f} & "
253 + f"{x['mdape_forward_2025_26'] - x['mdape_random_2025_26']:+.1f} & "
254 + f"{rl.loc[m, 'median_log_resid']:+.3f} & "
255 + f"{rl.loc[m, 'mdape_relevelled']:.1f} \\\\")
256 + n1, n2 = int(sp["n_random_2025_26"].iloc[0]), int(sp["n_forward"].iloc[0])
257 + lines += [r"\midrule", f"Test sales & 102,929 & {n1:,} & {n2:,} & & & \\\\",
258 + r"\bottomrule", r"\end{tabular}"]
259 + (TAB / "same_period.tex").write_text("\n".join(lines))
260 +
261 +
262 +def t_pairs():
263 + pr = pd.read_csv(RES / "pairs.csv")
264 + lines = [r"\begin{tabular}{ll c cc c cc}", r"\toprule",
265 + r" & & & \multicolumn{2}{c}{Random holdout} & &"
266 + r" \multicolumn{2}{c}{Forward-in-time} \\",
267 + r"\cmidrule{4-5}\cmidrule{7-8}",
268 + r"Comparison & & MdAPE & $\Delta$MdAPE [95\% CI] & "
269 + r"$t_{\mathrm{DM}}$ APE / sq.\ log & & "
270 + r"$\Delta$MdAPE [95\% CI] & $t_{\mathrm{DM}}$ APE / sq.\ log \\",
271 + r"\midrule"]
272 + pairs = list(dict.fromkeys(zip(pr["model_a"], pr["model_b"])))
273 + for a, b in pairs:
274 + r = pr[(pr["model_a"] == a) & (pr["model_b"] == b)].set_index("scheme")
275 + x, y = r.loc["random"], r.loc["temporal"]
276 + cells = [SHORT[a] + " vs.", SHORT[b],
277 + f"{x['mdape_a']:.1f} / {x['mdape_b']:.1f}",
278 + f"{x['diff']:+.2f} [{x['diff_lo']:+.2f}, {x['diff_hi']:+.2f}]",
279 + f"{x['dm_ape_t']:.1f}{_stars(x['dm_ape_p'])} / "
280 + f"{x['dm_sqlog_t']:.1f}{_stars(x['dm_sqlog_p'])}", "",
281 + f"{y['diff']:+.2f} [{y['diff_lo']:+.2f}, {y['diff_hi']:+.2f}]",
282 + f"{y['dm_ape_t']:.1f}{_stars(y['dm_ape_p'])} / "
283 + f"{y['dm_sqlog_t']:.1f}{_stars(y['dm_sqlog_p'])}"]
284 + lines.append(" & ".join(cells) + r" \\")
285 + lines += [r"\bottomrule", r"\end{tabular}"]
286 + (TAB / "pairs.tex").write_text("\n".join(lines))
287 +
288 +
289 +def t_group_splits():
290 + R = {s: pd.read_csv(RES / f"horserace_{s}.csv").set_index("name")
291 + for s in ("random", "group_parcel", "group_coord")}
292 + lines = [r"\begin{tabular}{l cc c cc c cc}", r"\toprule",
293 + r" & \multicolumn{2}{c}{Random 80/20} & &"
294 + r" \multicolumn{2}{c}{Parcel-grouped} & &"
295 + r" \multicolumn{2}{c}{Building-grouped} \\",
296 + r"\cmidrule{2-3}\cmidrule{5-6}\cmidrule{8-9}",
297 + r"Model & MdAPE & $R^2_{\ln}$ & & MdAPE & $R^2_{\ln}$ & & "
298 + r"MdAPE & $R^2_{\ln}$ \\", r"\midrule"]
299 + groups = {"A": "A. Functional form", "B": "B. Time effects",
300 + "C": "C. Spatial controls", "D": "D. Estimation method"}
301 + cur = None
302 + for m in SHORT:
303 + if m[0] != cur:
304 + cur = m[0]
305 + lines.append(r"\multicolumn{9}{l}{\itshape " + groups[cur]
306 + + r"}\\")
307 + cells = [r"\quad " + SHORT[m]]
308 + for i, s in enumerate(("random", "group_parcel", "group_coord")):
309 + x = R[s].loc[m]
310 + cells += [f"{x['mdape']:.1f}", f"{x['r2_ln']:.3f}"]
311 + if i < 2:
312 + cells.append("")
313 + lines.append(" & ".join(cells) + r" \\")
314 + ns = [f"{int(R[s]['n_test'].iloc[0]):,}" for s in
315 + ("random", "group_parcel", "group_coord")]
316 + lines += [r"\midrule",
317 + f"Test sales & {ns[0]} & & & {ns[1]} & & & {ns[2]} & \\\\",
318 + r"\bottomrule", r"\end{tabular}"]
319 + (TAB / "group_splits.tex").write_text("\n".join(lines))
320 +
321 +
322 +def t_leakage_decomp():
323 + L = pd.read_csv(RES / "leakage.csv")
324 + sel = ["A5 Semi-log + quadratics", "A6 Semi-log + splines",
325 + "D3 Random forest", "D4 Gradient boosting",
326 + "D5 Gradient boosting, no coordinates",
327 + "D6 Spatial k-NN comparables"]
328 + lines = [r"\begin{tabular}{l cc c cc}", r"\toprule",
329 + r" & \multicolumn{2}{c}{Same parcel in training} & &"
330 + r" \multicolumn{2}{c}{Same coordinates in training} \\",
331 + r"\cmidrule{2-3}\cmidrule{5-6}",
332 + r"Model & Yes & No & & Yes & No \\", r"\midrule"]
333 + def cell(key, flag, m, seg="all"):
334 + x = L[(L["key"] == key) & (L["seen_in_training"] == flag)
335 + & (L["model"] == m) & (L["segment"] == seg)]
336 + return f"{x['mdape'].iloc[0]:.1f}" if len(x) else "---"
337 + def n(key, flag, seg="all"):
338 + x = L[(L["key"] == key) & (L["seen_in_training"] == flag)
339 + & (L["segment"] == seg)]
340 + return f"{int(x['n'].iloc[0]):,}"
341 + lines.append(r"\multicolumn{6}{l}{\itshape All test sales}\\")
342 + for m in sel:
343 + lines.append(f"\\quad {SHORT[m]} & {cell('in_train_parcel', True, m)} & "
344 + f"{cell('in_train_parcel', False, m)} & & "
345 + f"{cell('in_train_coord', True, m)} & "
346 + f"{cell('in_train_coord', False, m)} \\\\")
347 + lines.append(f"\\quad Test sales & {n('in_train_parcel', True)} & "
348 + f"{n('in_train_parcel', False)} & & {n('in_train_coord', True)} & "
349 + f"{n('in_train_coord', False)} \\\\")
350 + lines.append(r"\midrule")
351 + lines.append(r"\multicolumn{6}{l}{\itshape Condominiums only}\\")
352 + for m in ("A5 Semi-log + quadratics", "D4 Gradient boosting"):
353 + lines.append(f"\\quad {SHORT[m]} & {cell('in_train_parcel', True, m, 'condo')} & "
354 + f"{cell('in_train_parcel', False, m, 'condo')} & & "
355 + f"{cell('in_train_coord', True, m, 'condo')} & "
356 + f"{cell('in_train_coord', False, m, 'condo')} \\\\")
357 + lines.append(f"\\quad Test sales & {n('in_train_parcel', True, 'condo')} & "
358 + f"{n('in_train_parcel', False, 'condo')} & & "
359 + f"{n('in_train_coord', True, 'condo')} & "
360 + f"{n('in_train_coord', False, 'condo')} \\\\")
361 + lines += [r"\bottomrule", r"\end{tabular}"]
362 + (TAB / "leakage.tex").write_text("\n".join(lines))
363 +
364 +
365 +def t_nested_fe():
366 + N = pd.read_csv(RES / "nested_fe.csv")
367 + R = {s: pd.read_csv(RES / f"horserace_{s}.csv").set_index("name")
368 + for s in ("random", "temporal")}
369 + rows = [("C2 Municipality FE", "C2 Municipality FE (1,115 cells)"),
370 + ("C3 Grid-cell FE (~5.5 km)", "C3 Grid 5.5 km (5,255 cells)"),
371 + ("C4 Grid-cell FE (~1.1 km)", "C4 Grid 1.1 km (30,548 cells)"),
372 + ("C3' Municipality + grid-cell FE (~5.5 km)",
373 + "C3$'$ Municipality + grid 5.5 km"),
374 + ("C4' Municipality + grid-cell FE (~1.1 km)",
375 + "C4$'$ Municipality + grid 1.1 km")]
376 + lines = [r"\begin{tabular}{l cc c cc}", r"\toprule",
377 + r" & \multicolumn{2}{c}{Random holdout} & &"
378 + r" \multicolumn{2}{c}{Forward-in-time} \\",
379 + r"\cmidrule{2-3}\cmidrule{5-6}",
380 + r"Spatial controls & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & "
381 + r"$R^2_{\ln}$ \\", r"\midrule"]
382 + for key, label in rows:
383 + vals = []
384 + for s in ("random", "temporal"):
385 + if key in R[s].index:
386 + x = R[s].loc[key]
387 + else:
388 + x = N[(N["name"] == key) & (N["scheme"] == s)].iloc[0]
389 + vals += [f"{x['mdape']:.1f}", f"{x['r2_ln']:.3f}"]
390 + lines.append(f"{label} & {vals[0]} & {vals[1]} & & {vals[2]} & "
391 + f"{vals[3]} \\\\")
392 + lines += [r"\bottomrule", r"\end{tabular}"]
393 + (TAB / "nested_fe.tex").write_text("\n".join(lines))
394 +
395 +
396 +def t_hgb_sensitivity():
397 + H = pd.read_csv(RES / "hgb_sensitivity.csv")
398 + names = list(dict.fromkeys(H["name"]))
399 + lines = [r"\begin{tabular}{l cc c cc}", r"\toprule",
400 + r" & \multicolumn{2}{c}{Random holdout} & &"
401 + r" \multicolumn{2}{c}{Forward-in-time} \\",
402 + r"\cmidrule{2-3}\cmidrule{5-6}",
403 + r"Gradient-boosting configuration & MdAPE (\%) & $R^2_{\ln}$ & & "
404 + r"MdAPE (\%) & $R^2_{\ln}$ \\", r"\midrule"]
405 + for nm in names:
406 + a = H[(H["name"] == nm) & (H["scheme"] == "random")].iloc[0]
407 + b = H[(H["name"] == nm) & (H["scheme"] == "temporal")].iloc[0]
408 + label = _esc(nm.replace("D4 ", "").replace("%", r"\%"))
409 + label = label[0].upper() + label[1:]
410 + if not pd.isna(b.get("best_iter", float("nan"))):
411 + label += f" [{int(a['best_iter'])} / {int(b['best_iter'])} it.]"
412 + lines.append(f"{label} & {a['mdape']:.1f} & {a['r2_ln']:.3f} & & "
413 + f"{b['mdape']:.1f} & {b['r2_ln']:.3f} \\\\")
414 + lines += [r"\bottomrule", r"\end{tabular}"]
415 + (TAB / "hgb_sensitivity.tex").write_text("\n".join(lines))
416 +
417 +
418 +def t_by_quarter():
419 + Q = pd.read_csv(RES / "by_quarter.csv")
420 + piv = Q.pivot_table(index="quarter", columns="model", values="mdape")
421 + ns = Q.groupby("quarter")["n"].first()
422 + cols = ["A5 Semi-log + quadratics", "A6 Semi-log + splines",
423 + "D2 Ridge (cross-validated)", "D3 Random forest",
424 + "D4 Gradient boosting"]
425 + lines = [r"\begin{tabular}{lr" + "c" * len(cols) + "}", r"\toprule",
426 + r"Test quarter & $n$ & " + " & ".join(SHORT[c] for c in cols)
427 + + r" \\", r"\midrule"]
428 + for q, row in piv.iterrows():
429 + lines.append(f"{q} & {ns[q]:,.0f} & "
430 + + " & ".join(f"{row[c]:.1f}" for c in cols) + r" \\")
431 + lines += [r"\bottomrule", r"\end{tabular}"]
432 + (TAB / "by_quarter.tex").write_text("\n".join(lines))
433 +
434 +
136 435 def main() -> None:
137 436 config.ensure_dirs()
138 437 t_summary()
@@ -142,6 +441,13 @@ def main() -> None:
142 441 t_boxcox()
143 442 t_learning()
144 443 t_segments()
444 + for fn in (t_inference, t_retransformation, t_same_period, t_pairs,
445 + t_group_splits, t_leakage_decomp, t_nested_fe,
446 + t_hgb_sensitivity, t_by_quarter):
447 + try:
448 + fn()
449 + except FileNotFoundError as e: # step 06 not run yet
450 + print(f" skipped {fn.__name__}: {e}")
145 451 made = sorted(p.name for p in TAB.glob("*.tex"))
146 452 print(f"{len(made)} tables written:", ", ".join(made))
147 453
added scripts/06_robustness.py +274 −0
@@ -0,0 +1,274 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +"""Step 06 — Inference and robustness (added in the 2026-10-17 revision).
4 +
5 +Reads the test-set predictions stored by step 02 and runs the additional
6 +analyses requested by the editorial review:
7 +
8 +1. cluster-bootstrap confidence intervals (municipality clusters) for the
9 + MdAPE of every model, and Diebold–Mariano / Giacomini–White-type tests
10 + of equal accuracy against the spline hedonic (A6) and gradient
11 + boosting (D4), under both holdouts -> inference_<scheme>.csv
12 +2. same-period comparison: random-split models evaluated on the 2025–26
13 + sales of the random test set versus forward-in-time models on the
14 + full 2025–26 test set -> same_period.csv
15 +3. forward-in-time accuracy by test quarter -> by_quarter.csv
16 +4. leakage decomposition of the random holdout: accuracy on test sales
17 + whose parcel / coordinate pair also appears in training versus the
18 + others -> leakage.csv
19 +5. leakage-free holdouts: the full race under parcel-grouped and
20 + coordinate-grouped 80/20 splits -> horserace_group_parcel.csv,
21 + horserace_group_coord.csv
22 +6. nested spatial fixed effects (municipality + grid) -> nested_fe.csv
23 +7. Box–Cox on a finer lambda grid -> boxcox_fine.csv
24 +8. gradient-boosting hyper-parameter sensitivity -> hgb_sensitivity.csv
25 +10. targeted pairwise comparisons along each axis (bootstrap CI of the
26 + MdAPE difference and clustered DM tests) -> pairs.csv
27 +9. level-bias decomposition: median log residual (aggregate level error)
28 + and MdAPE after re-levelling predictions by the test-set median ratio
29 + (relative pricing skill net of the level error) -> relevel.csv
30 +
31 +Usage: python scripts/06_robustness.py (about 15 min)
32 +"""
33 +import sys
34 +import time
35 +from pathlib import Path
36 +
37 +import numpy as np
38 +import pandas as pd
39 +
40 +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
41 +
42 +from wp11 import config, inference, models, sample # noqa: E402
43 +
44 +OUT = config.REPRODUCED
45 +MODEL_NAMES = [m["name"] for m in models.model_registry()]
46 +A6, D4, A5 = ("A6 Semi-log + splines", "D4 Gradient boosting",
47 + "A5 Semi-log + quadratics")
48 +
49 +
50 +def _mdape(y, p):
51 + return float(np.median(np.abs(p - y) / y) * 100)
52 +
53 +
54 +def step_inference(P: dict):
55 + for scheme, Ps in P.items():
56 + t0 = time.time()
57 + boot = inference.cluster_bootstrap_mdape(Ps, MODEL_NAMES, ref=A6)
58 + boot_d4 = inference.cluster_bootstrap_mdape(Ps, MODEL_NAMES, ref=D4)
59 + boot = boot.merge(boot_d4[["model", "diff_vs_ref", "diff_lo",
60 + "diff_hi"]]
61 + .rename(columns={"diff_vs_ref": "diff_vs_d4",
62 + "diff_lo": "diff_d4_lo",
63 + "diff_hi": "diff_d4_hi"}),
64 + on="model", how="left")
65 + rows = []
66 + for m in MODEL_NAMES:
67 + r = {"model": m}
68 + for ref, tag in ((A6, "a6"), (D4, "d4")):
69 + if m == ref:
70 + continue
71 + for loss, ltag in ((inference.ape, "ape"),
72 + (inference.sq_log, "sqlog")):
73 + t = inference.dm_test(Ps, m, ref, loss=loss)
74 + r[f"dm_{ltag}_vs_{tag}_t"] = t["t"]
75 + r[f"dm_{ltag}_vs_{tag}_p"] = t["p"]
76 + r[f"dm_{ltag}_vs_{tag}_diff"] = t["mean_diff"]
77 + rows.append(r)
78 + res = boot.merge(pd.DataFrame(rows), on="model")
79 + res["n_clusters"] = Ps["muni"].nunique()
80 + res.to_csv(OUT / f"inference_{scheme}.csv", index=False)
81 + print(f" inference [{scheme}] done in {time.time() - t0:.0f}s")
82 +
83 +
84 +def step_same_period(P: dict):
85 + cut = pd.Timestamp(config.TEMPORAL_CUTOFF)
86 + Pr = P["random"][P["random"]["sale_date"] >= cut]
87 + Pt = P["temporal"]
88 + rows = []
89 + for m in MODEL_NAMES:
90 + rows.append({"model": m,
91 + "mdape_random_all": _mdape(P["random"]["price"].to_numpy(),
92 + P["random"][m].to_numpy()),
93 + "mdape_random_2025_26": _mdape(Pr["price"].to_numpy(),
94 + Pr[m].to_numpy()),
95 + "mdape_forward_2025_26": _mdape(Pt["price"].to_numpy(),
96 + Pt[m].to_numpy()),
97 + "n_random_2025_26": len(Pr), "n_forward": len(Pt)})
98 + pd.DataFrame(rows).to_csv(OUT / "same_period.csv", index=False)
99 + print(" same-period comparison done")
100 +
101 +
102 +def step_by_quarter(P: dict):
103 + Pt = P["temporal"].copy()
104 + Pt["q"] = Pt["sale_date"].dt.to_period("Q").astype(str)
105 + rows = []
106 + for q, g in Pt.groupby("q"):
107 + for m in (A5, A6, "D2 Ridge (cross-validated)", "D3 Random forest",
108 + D4):
109 + rows.append({"quarter": q, "model": m, "n": len(g),
110 + "mdape": _mdape(g["price"].to_numpy(),
111 + g[m].to_numpy())})
112 + pd.DataFrame(rows).to_csv(OUT / "by_quarter.csv", index=False)
113 + print(" by-quarter accuracy done")
114 +
115 +
116 +def step_leakage(P: dict):
117 + Pr = P["random"]
118 + rows = []
119 + for key in ("in_train_parcel", "in_train_coord"):
120 + for flag in (True, False):
121 + g = Pr[Pr[key] == flag]
122 + for m in (A5, A6, "D2 Ridge (cross-validated)",
123 + "D3 Random forest", D4,
124 + "D5 Gradient boosting, no coordinates",
125 + "D6 Spatial k-NN comparables"):
126 + rows.append({"key": key, "seen_in_training": flag,
127 + "segment": "all", "n": len(g), "model": m,
128 + "mdape": _mdape(g["price"].to_numpy(),
129 + g[m].to_numpy())})
130 + gc = g[g["prop_class"] == "condo"]
131 + for m in (A5, D4):
132 + rows.append({"key": key, "seen_in_training": flag,
133 + "segment": "condo", "n": len(gc), "model": m,
134 + "mdape": _mdape(gc["price"].to_numpy(),
135 + gc[m].to_numpy())})
136 + pd.DataFrame(rows).to_csv(OUT / "leakage.csv", index=False)
137 + print(" leakage decomposition done")
138 +
139 +
140 +def step_relevel(P: dict):
141 + """Decompose forward-in-time error into an aggregate level error and
142 + relative pricing skill. For each model the median log residual
143 + b = median(ln y - ln p) measures the common level error (staleness);
144 + the re-levelled MdAPE uses p * exp(b), i.e. an oracle time adjustment
145 + of the kind practitioners obtain from a published index."""
146 + rows = []
147 + for scheme, Ps in P.items():
148 + y = Ps["price"].to_numpy(float)
149 + for m in MODEL_NAMES:
150 + p = Ps[m].to_numpy(float)
151 + b = float(np.median(np.log(y) - np.log(p)))
152 + rows.append({"scheme": scheme, "model": m,
153 + "median_log_resid": b,
154 + "mdape": _mdape(y, p),
155 + "mdape_relevelled": _mdape(y, p * np.exp(b)),
156 + "mape_relevelled": float(np.mean(
157 + np.abs(p * np.exp(b) - y) / y) * 100),
158 + "rmse_ln_centered": float(np.sqrt(np.mean(
159 + (np.log(y) - np.log(p) - b) ** 2)))})
160 + pd.DataFrame(rows).to_csv(OUT / "relevel.csv", index=False)
161 + print(" level-bias decomposition done")
162 +
163 +
164 +PAIRS = [("A2 Semi-log", "A1 Linear (levels)"),
165 + ("A6 Semi-log + splines", "A2 Semi-log"),
166 + ("A4 Box-Cox", "A2 Semi-log"),
167 + ("A3 Log-log", "A2 Semi-log"),
168 + ("B2 Year FE", "B1 No time effects"),
169 + ("B4 Month FE", "B3 Quarter FE"),
170 + ("C3 Grid-cell FE (~5.5 km)", "C2 Municipality FE"),
171 + ("C4 Grid-cell FE (~1.1 km)", "C2 Municipality FE"),
172 + ("D2 Ridge (cross-validated)", "D1 OLS (muni + month FE)"),
173 + ("D4 Gradient boosting", "D3 Random forest"),
174 + ("D4 Gradient boosting", "D5 Gradient boosting, no coordinates"),
175 + ("D4 Gradient boosting", "A1 Linear (levels)"),
176 + ("A6 Semi-log + splines", "A1 Linear (levels)")]
177 +
178 +
179 +def step_pairs(P: dict):
180 + """Targeted comparisons: for each (A, B) the MdAPE difference A - B
181 + with its cluster-bootstrap interval and the clustered DM statistics
182 + on the APE and squared-log losses (positive t = B more accurate)."""
183 + rows = []
184 + for scheme, Ps in P.items():
185 + for a, b in PAIRS:
186 + boot = inference.cluster_bootstrap_mdape(Ps, [a, b], ref=b)
187 + x = boot.set_index("model").loc[a]
188 + t1 = inference.dm_test(Ps, a, b, loss=inference.ape)
189 + t2 = inference.dm_test(Ps, a, b, loss=inference.sq_log)
190 + rows.append({"scheme": scheme, "model_a": a, "model_b": b,
191 + "mdape_a": x["mdape"],
192 + "mdape_b": boot.set_index("model").loc[b, "mdape"],
193 + "diff": x["diff_vs_ref"], "diff_lo": x["diff_lo"],
194 + "diff_hi": x["diff_hi"],
195 + "dm_ape_t": t1["t"], "dm_ape_p": t1["p"],
196 + "dm_sqlog_t": t2["t"], "dm_sqlog_p": t2["p"]})
197 + pd.DataFrame(rows).to_csv(OUT / "pairs.csv", index=False)
198 + print(" targeted pairwise comparisons done")
199 +
200 +
201 +def step_group_splits(df):
202 + for scheme in ("group_parcel", "group_coord"):
203 + res = models.run_horserace(df, scheme)
204 + res.to_csv(OUT / f"horserace_{scheme}.csv", index=False)
205 +
206 +
207 +def step_nested_fe(df):
208 + rows = []
209 + for scheme in ("random", "temporal"):
210 + res = models.run_horserace(df, scheme,
211 + registry=models.robustness_registry())
212 + res["scheme"] = scheme
213 + rows.append(res)
214 + pd.concat(rows).to_csv(OUT / "nested_fe.csv", index=False)
215 +
216 +
217 +def step_boxcox_fine(df):
218 + d = models.add_derived(df)
219 + d["sale_year_c"] = d["sale_year"].astype(str)
220 + train, test = models.split(d, "random")
221 + grid = [round(x, 2) for x in np.arange(-0.25, 0.751, 0.05)]
222 + lam, prof = models.fit_boxcox(train, models.NUM_LEVELS,
223 + models.CATS + ["muni", "quarter"],
224 + grid=grid)
225 + prof["chosen"] = prof["lambda"] == lam
226 + # out-of-sample accuracy of the Box–Cox model at the refined lambda
227 + spec = dict(name="A4 fine", group="x", kind="boxcox",
228 + numeric=models.NUM_LEVELS, fe=["muni", "quarter"],
229 + boxcox_grid=[lam])
230 + pred, _ = models.fit_predict(spec, train, test)
231 + prof.attrs = {}
232 + prof["mdape_at_chosen"] = np.where(prof["chosen"],
233 + _mdape(test["price"].to_numpy(float),
234 + np.clip(pred, config.PRED_FLOOR,
235 + None)), np.nan)
236 + prof.to_csv(OUT / "boxcox_fine.csv", index=False)
237 + print(f" Box–Cox fine grid: lambda* = {lam}")
238 +
239 +
240 +def step_hgb_sensitivity(df):
241 + rows = []
242 + for scheme in ("random", "temporal"):
243 + reg = [dict(name="D4 Gradient boosting (baseline)", group="S",
244 + kind="hgb")]
245 + reg += [dict(name=f"D4 {label}", group="S", kind="hgb",
246 + hgb_params=params)
247 + for label, params in models.HGB_ALTERNATIVES.items()]
248 + res = models.run_horserace(df, scheme, registry=reg)
249 + res["scheme"] = scheme
250 + rows.append(res)
251 + pd.concat(rows).to_csv(OUT / "hgb_sensitivity.csv", index=False)
252 +
253 +
254 +def main() -> None:
255 + config.ensure_dirs()
256 + df = sample.load()
257 + P = {s: pd.read_parquet(config.PROCESSED / f"predictions_{s}.parquet")
258 + for s in ("random", "temporal")}
259 + t0 = time.time()
260 + step_inference(P)
261 + step_same_period(P)
262 + step_by_quarter(P)
263 + step_leakage(P)
264 + step_relevel(P)
265 + step_pairs(P)
266 + step_group_splits(df)
267 + step_nested_fe(df)
268 + step_boxcox_fine(df)
269 + step_hgb_sensitivity(df)
270 + print(f"Robustness step finished in {(time.time() - t0) / 60:.1f} min")
271 +
272 +
273 +if __name__ == "__main__":
274 + main()
added src/wp11/inference.py +95 −0
@@ -0,0 +1,95 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +"""Inference for out-of-sample accuracy comparisons.
3 +
4 +Two tools, both operating on the stored test-set predictions:
5 +
6 +* a *cluster bootstrap* (resampling municipalities with replacement) for
7 + the MdAPE of each model and for the MdAPE difference between two
8 + models evaluated on the same test sales;
9 +* a Diebold–Mariano / Giacomini–White-type test of equal predictive
10 + accuracy: the mean of the per-sale loss differential d_i = L(e_i^A) -
11 + L(e_i^B) is tested against zero with standard errors clustered by
12 + municipality (the cross-sectional analogue of the HAC-robust DM
13 + statistic). Losses: absolute percentage error (the MAPE loss) and the
14 + squared log error (the RMSE_ln loss).
15 +"""
16 +from __future__ import annotations
17 +
18 +import numpy as np
19 +import pandas as pd
20 +
21 +from . import config
22 +
23 +
24 +def ape(y: np.ndarray, p: np.ndarray) -> np.ndarray:
25 + return np.abs(p - y) / y
26 +
27 +
28 +def sq_log(y: np.ndarray, p: np.ndarray) -> np.ndarray:
29 + return (np.log(y) - np.log(p)) ** 2
30 +
31 +
32 +def cluster_bootstrap_mdape(P: pd.DataFrame, models: list[str],
33 + cluster: str = "muni", B: int = 500,
34 + ref: str | None = None,
35 + seed: int = config.SEED) -> pd.DataFrame:
36 + """Percentile 95% intervals for MdAPE (and MdAPE minus the reference
37 + model's MdAPE) from B cluster-bootstrap replicates.
38 +
39 + Clusters are municipalities; each replicate draws clusters with
40 + replacement (so a drawn municipality contributes all its test sales,
41 + possibly several times) and recomputes the median APE.
42 + """
43 + y = P["price"].to_numpy(float)
44 + A = {m: ape(y, P[m].to_numpy(float)) for m in models}
45 + codes, uniq = pd.factorize(P[cluster])
46 + G = len(uniq)
47 + rng = np.random.default_rng(seed)
48 + # pre-compute row indices per cluster
49 + order = np.argsort(codes, kind="stable")
50 + bounds = np.searchsorted(codes[order], np.arange(G + 1))
51 + rows_by_cluster = [order[bounds[g]:bounds[g + 1]] for g in range(G)]
52 + reps = {m: np.empty(B) for m in models}
53 + for b in range(B):
54 + draw = rng.integers(0, G, G)
55 + idx = np.concatenate([rows_by_cluster[g] for g in draw])
56 + for m in models:
57 + reps[m][b] = np.median(A[m][idx])
58 + out = []
59 + for m in models:
60 + point = float(np.median(A[m]) * 100)
61 + lo, hi = np.percentile(reps[m], [2.5, 97.5]) * 100
62 + row = {"model": m, "mdape": point, "ci_lo": lo, "ci_hi": hi,
63 + "boot_se": float(reps[m].std(ddof=1) * 100)}
64 + if ref is not None and m != ref:
65 + d = (reps[m] - reps[ref]) * 100
66 + row.update({"diff_vs_ref": point - float(np.median(A[ref]) * 100),
67 + "diff_lo": float(np.percentile(d, 2.5)),
68 + "diff_hi": float(np.percentile(d, 97.5))})
69 + out.append(row)
70 + return pd.DataFrame(out)
71 +
72 +
73 +def dm_test(P: pd.DataFrame, model_a: str, model_b: str,
74 + loss=ape, cluster: str = "muni") -> dict:
75 + """Clustered test of E[d_i] = 0 for d_i = L_A(i) - L_B(i).
76 +
77 + Returns the mean differential (positive = A has larger loss, i.e.
78 + B is more accurate), its cluster-robust standard error, the t
79 + statistic and the two-sided normal p-value.
80 + """
81 + y = P["price"].to_numpy(float)
82 + d = loss(y, P[model_a].to_numpy(float)) - loss(y, P[model_b].to_numpy(float))
83 + codes, _ = pd.factorize(P[cluster])
84 + n, G = len(d), codes.max() + 1
85 + dbar = d.mean()
86 + # cluster-robust variance of the mean: sum over clusters of squared
87 + # within-cluster sums of demeaned d, divided by n^2
88 + sums = np.bincount(codes, weights=d - dbar, minlength=G)
89 + var = (sums ** 2).sum() / n ** 2 * G / (G - 1)
90 + se = float(np.sqrt(var))
91 + t = float(dbar / se) if se > 0 else np.nan
92 + from math import erf, sqrt
93 + p = 2 * (1 - 0.5 * (1 + erf(abs(t) / sqrt(2)))) if np.isfinite(t) else np.nan
94 + return {"model_a": model_a, "model_b": model_b, "mean_diff": float(dbar),
95 + "se": se, "t": t, "p": float(p), "n_clusters": int(G)}
modified src/wp11/models.py +220 −43
@@ -6,8 +6,8 @@ Twenty hedonic models organised along four design axes:
6 6 A. functional form — linear, semi-log, log-log, Box–Cox, quadratic,
7 7 splines (linear estimator, muni + quarter FE)
8 8 B. time effects — none / year / quarter / month FE (quadratic backbone)
9 − C. spatial controls — none / municipality / ~1.1 km grid / neighbourhood
10 − unit FE (quadratic backbone, quarter FE)
9 + C. spatial controls — none / municipality / ~5.5 km grid / ~1.1 km
10 + grid FE (quadratic backbone, quarter FE)
11 11 D. estimation method — OLS, ridge, random forest, gradient boosting
12 12 (with and without coordinates), spatial k-NN
13 13 comparables
@@ -16,9 +16,25 @@ Every linear model is estimated as a sparse ridge with a vanishing penalty
16 16 (numerically OLS, but it (i) survives collinear dummies and (ii) predicts
17 17 unseen categories at the reference level via ``handle_unknown='ignore'``).
18 18 Every model is evaluated on identical train/test splits, and all metrics
19 −are computed on *price levels* — log models are retransformed with Duan's
20 −smearing factor, Box–Cox by analytic inversion — so that functional forms
21 −compete on the same scoreboard.
19 +are computed on *price levels* so that functional forms compete on the
20 +same scoreboard. Two retransformation rules are available for every model
21 +whose dependent variable is a transformation of the price:
22 +
23 + * ``"median"`` (main scoreboard) — the fitted transformed value is
24 + inverted as is: exp(.) for log models, the analytic inverse for
25 + Box–Cox. Under symmetric transformed-scale errors this is the
26 + conditional *median* price, the target that MdAPE/MAPE reward and the
27 + quantity the tree ensembles (fit on ln P) deliver natively. Log-scale
28 + metrics (RMSE_ln, R2_ln) are always computed on this prediction.
29 + * ``"mean"`` (robustness) — Duan's (1983) smearing estimator of the
30 + conditional *mean*: exp(z_hat) * mean(exp(e_i)) for log models and
31 + mean_i g^{-1}(z_hat + e_i) over training residuals e_i for Box–Cox;
32 + tree ensembles receive the same correction from their (in-sample)
33 + training residuals.
34 +
35 +Version 1.0 of the paper smeared the linear log models but not the tree
36 +ensembles, which penalised the linear models on every metric; the
37 +``"median"`` rule applied uniformly is the scoreboard of version 1.1.
22 38 """
23 39 from __future__ import annotations
24 40
@@ -62,6 +78,16 @@ def split(df: pd.DataFrame, scheme: str):
62 78 if scheme == "temporal":
63 79 cut = pd.Timestamp(config.TEMPORAL_CUTOFF)
64 80 return df[df["sale_date"] < cut], df[df["sale_date"] >= cut]
81 + if scheme in ("group_parcel", "group_coord"):
82 + # Leakage-free random holdouts: 20% of *groups* (parcels, or exact
83 + # coordinate pairs = buildings/complexes) go to the test set, so
84 + # that no property (resp. building) is seen on both sides.
85 + key = "parcel" if scheme == "group_parcel" else "coord"
86 + rng = np.random.default_rng(config.SEED)
87 + groups = df[key].unique()
88 + test_groups = set(groups[rng.random(len(groups)) < config.TEST_SHARE])
89 + mask = df[key].isin(test_groups).to_numpy()
90 + return df[~mask], df[mask]
65 91 raise ValueError(scheme)
66 92
67 93
@@ -118,13 +144,30 @@ def _inv_boxcox(z: np.ndarray, lam: float) -> np.ndarray:
118 144 return base ** (1.0 / lam)
119 145
120 146
121 −def fit_boxcox(train: pd.DataFrame, numeric: list[str], cats: list[str]):
147 +def _smear_boxcox(z_hat: np.ndarray, resid: np.ndarray, lam: float,
148 + n_resid: int = 1000) -> np.ndarray:
149 + """Duan's smearing estimator of E[P | X] for a Box–Cox model:
150 + the average of g^{-1}(z_hat + e_i) over (a subsample of) the training
151 + residuals e_i. Reduces to exp(z_hat) * mean(exp(e)) when lam = 0."""
152 + rng = np.random.default_rng(config.SEED)
153 + e = resid if len(resid) <= n_resid else rng.choice(resid, n_resid,
154 + replace=False)
155 + out = np.zeros_like(z_hat)
156 + for chunk in np.array_split(np.arange(len(z_hat)),
157 + max(1, len(z_hat) // 5000)):
158 + out[chunk] = _inv_boxcox(z_hat[chunk, None] + e[None, :],
159 + lam).mean(axis=1)
160 + return out
161 +
162 +
163 +def fit_boxcox(train: pd.DataFrame, numeric: list[str], cats: list[str],
164 + grid: list[float] | None = None):
122 165 """Profile-likelihood choice of λ on the training sample."""
123 166 y = train["price"].to_numpy(float)
124 167 n = len(y)
125 168 sum_ln_y = np.log(y).sum()
126 169 rows = []
127 − for lam in config.BOXCOX_GRID:
170 + for lam in (grid if grid is not None else config.BOXCOX_GRID):
128 171 pipe = _linear_pipeline(numeric, cats)
129 172 z = _boxcox(y, lam)
130 173 pipe.fit(train, z)
@@ -136,6 +179,43 @@ def fit_boxcox(train: pd.DataFrame, numeric: list[str], cats: list[str]):
136 179 return lam_star, prof
137 180
138 181
182 +HGB_PARAMS = dict(max_iter=600, learning_rate=0.08, max_leaf_nodes=63,
183 + min_samples_leaf=20, l2_regularization=1e-2)
184 +
185 +# Alternative gradient-boosting configurations for the hyper-parameter
186 +# sensitivity analysis (Section "Robustness"). ``early_stopping`` holds out
187 +# the last 10% of the training *rows in time order* as a validation set,
188 +# i.e. a time-honest tuning rule.
189 +HGB_ALTERNATIVES = {
190 + "shallow (15 leaves, lr 0.05, 1500 iter)":
191 + dict(max_iter=1500, learning_rate=0.05, max_leaf_nodes=15,
192 + min_samples_leaf=20, l2_regularization=1e-2),
193 + "deep (255 leaves, lr 0.05, 600 iter)":
194 + dict(max_iter=600, learning_rate=0.05, max_leaf_nodes=255,
195 + min_samples_leaf=20, l2_regularization=1e-2),
196 + "regularised (min leaf 200, L2 = 1)":
197 + dict(max_iter=600, learning_rate=0.08, max_leaf_nodes=63,
198 + min_samples_leaf=200, l2_regularization=1.0),
199 + "early stopping on last 10% of training months":
200 + dict(max_iter=2000, learning_rate=0.05, max_leaf_nodes=63,
201 + min_samples_leaf=20, l2_regularization=1e-2,
202 + early_stopping=True),
203 +}
204 +
205 +
206 +def robustness_registry() -> list[dict]:
207 + """Additional specifications: nested spatial FE (unseen grid cells fall
208 + back to their municipality instead of an arbitrary reference cell)."""
209 + return [
210 + dict(name="C3' Municipality + grid-cell FE (~5.5 km)",
211 + group="R. Nested spatial FE", kind="linear_log",
212 + numeric=NUM_QUAD, fe=["muni", "grid5", "quarter"]),
213 + dict(name="C4' Municipality + grid-cell FE (~1.1 km)",
214 + group="R. Nested spatial FE", kind="linear_log",
215 + numeric=NUM_QUAD, fe=["muni", "grid", "quarter"]),
216 + ]
217 +
218 +
139 219 # ---------------------------------------------------------------- registry
140 220 def model_registry() -> list[dict]:
141 221 """The twenty models. ``fe`` lists categorical FE columns appended to
@@ -193,10 +273,27 @@ def model_registry() -> list[dict]:
193 273
194 274
195 275 # ---------------------------------------------------------------- fit/eval
196 −def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame):
197 − """Fit one registry entry, return level predictions on the test set."""
276 +def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame,
277 + retransform: str = "median"):
278 + """Fit one registry entry, return level predictions on the test set.
279 +
280 + ``retransform`` is ``"median"`` (plain inversion of the transformed
281 + prediction) or ``"mean"`` (Duan smearing). ``spec`` is updated in
282 + place with ``smear`` (log models), ``lambda`` (Box–Cox) and, when
283 + ``retransform="mean"`` is requested, both predictions are returned
284 + through ``spec["pred_median"]`` / ``spec["pred_mean"]``.
285 + """
198 286 kind = spec["kind"]
199 287 y_train = train["price"].to_numpy(float)
288 + both = retransform == "mean"
289 +
290 + def _log_out(z_hat_test, z_hat_train, model):
291 + resid = np.log(y_train) - z_hat_train
292 + smear = float(np.mean(np.exp(resid))) # Duan (1983)
293 + spec["smear"] = smear
294 + p_med = np.exp(z_hat_test)
295 + spec["pred_median"], spec["pred_mean"] = p_med, p_med * smear
296 + return (p_med * smear if both else p_med), model
200 297
201 298 if kind in ("linear_level", "linear_log", "boxcox", "ridge"):
202 299 cats = CATS + spec.get("fe", [])
@@ -204,7 +301,9 @@ def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame):
204 301 spline_cols=spec.get("spline"))
205 302 if kind == "linear_level":
206 303 pipe.fit(train, y_train)
207 − return pipe.predict(test), pipe
304 + p = pipe.predict(test)
305 + spec["pred_median"] = spec["pred_mean"] = p
306 + return p, pipe
208 307 if kind in ("linear_log", "ridge"):
209 308 if kind == "ridge":
210 309 from sklearn.preprocessing import StandardScaler
@@ -212,32 +311,72 @@ def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame):
212 311 ("sc", StandardScaler(with_mean=False)),
213 312 ("reg", RidgeCV(alphas=np.logspace(-6, 2, 17),
214 313 cv=3))]
215 − z = np.log(y_train)
216 − pipe.fit(train, z)
217 − resid = z - pipe.predict(train)
218 − smear = float(np.mean(np.exp(resid))) # Duan (1983)
219 − return np.exp(pipe.predict(test)) * smear, pipe
220 − lam, _ = fit_boxcox(train, spec["numeric"], cats)
314 + pipe.fit(train, np.log(y_train))
315 + return _log_out(pipe.predict(test), pipe.predict(train), pipe)
316 + lam, _ = fit_boxcox(train, spec["numeric"], cats,
317 + grid=spec.get("boxcox_grid"))
221 318 pipe = _linear_pipeline(spec["numeric"], cats)
222 − pipe.fit(train, _boxcox(y_train, lam))
319 + z = _boxcox(y_train, lam)
320 + pipe.fit(train, z)
223 321 spec["lambda"] = lam
224 − return _inv_boxcox(pipe.predict(test), lam), pipe
322 + z_hat = pipe.predict(test)
323 + p_med = _inv_boxcox(z_hat, lam)
324 + spec["pred_median"] = p_med
325 + if both:
326 + resid = z - pipe.predict(train)
327 + spec["pred_mean"] = _smear_boxcox(z_hat, resid, lam)
328 + return spec["pred_mean"], pipe
329 + return p_med, pipe
225 330
226 331 if kind == "rf":
227 332 m = RandomForestRegressor(n_estimators=120, min_samples_leaf=5,
228 333 max_features=0.5, n_jobs=-1,
229 334 random_state=config.SEED)
230 335 m.fit(_ml_frame(train), np.log(y_train))
231 − return np.exp(m.predict(_ml_frame(test))), m
336 + return _log_out(m.predict(_ml_frame(test)),
337 + m.predict(_ml_frame(train)) if both
338 + else np.log(y_train), m)
232 339
233 340 if kind in ("hgb", "hgb_nogeo"):
234 341 geo = kind == "hgb"
235 − m = HistGradientBoostingRegressor(
236 − max_iter=600, learning_rate=0.08, max_leaf_nodes=63,
237 − min_samples_leaf=20, l2_regularization=1e-2,
238 − random_state=config.SEED)
239 − m.fit(_ml_frame(train, geo), np.log(y_train))
240 − return np.exp(m.predict(_ml_frame(test, geo))), m
342 + params = dict(spec.get("hgb_params", HGB_PARAMS))
343 + tr = train
344 + if params.pop("early_stopping", False):
345 + # time-honest validation: the last 10% of training rows in
346 + # sale-date order serve as the early-stopping set
347 + tr = train.sort_values("sale_date")
348 + n_val = int(0.1 * len(tr))
349 + Xv, yv = _ml_frame(tr.tail(n_val), geo), np.log(
350 + tr.tail(n_val)["price"].to_numpy(float))
351 + tr = tr.head(len(tr) - n_val)
352 + m = HistGradientBoostingRegressor(random_state=config.SEED,
353 + **params)
354 + # scikit-learn's early stopping draws a random validation
355 + # split; emulate a time-ordered one by staged fitting
356 + m.set_params(early_stopping=False)
357 + best, best_iter = np.inf, params["max_iter"]
358 + m.set_params(warm_start=True, max_iter=100)
359 + for it in range(100, params["max_iter"] + 1, 100):
360 + m.set_params(max_iter=it)
361 + m.fit(_ml_frame(tr, geo),
362 + np.log(tr["price"].to_numpy(float)))
363 + mse = float(np.mean((yv - m.predict(Xv)) ** 2))
364 + if mse < best - 1e-5:
365 + best, best_iter = mse, it
366 + elif it - best_iter >= 300:
367 + break
368 + spec["best_iter"] = best_iter
369 + m = HistGradientBoostingRegressor(random_state=config.SEED,
370 + **{**params,
371 + "max_iter": best_iter})
372 + tr = train
373 + else:
374 + m = HistGradientBoostingRegressor(random_state=config.SEED,
375 + **params)
376 + m.fit(_ml_frame(tr, geo), np.log(tr["price"].to_numpy(float)))
377 + return _log_out(m.predict(_ml_frame(test, geo)),
378 + m.predict(_ml_frame(train, geo)) if both
379 + else np.log(y_train), m)
241 380
242 381 if kind == "knn":
243 382 # price-per-m2 of the 10 nearest sold neighbours (lat/lng, km-scaled)
@@ -245,21 +384,17 @@ def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame):
245 384 Xte = test[["lat", "lng"]].to_numpy() * np.array([111.0, 78.0])
246 385 m = KNeighborsRegressor(n_neighbors=10, weights="distance", n_jobs=-1)
247 386 m.fit(Xtr, np.log(y_train) - train["ln_area"].to_numpy())
248 − return np.exp(m.predict(Xte) + test["ln_area"].to_numpy()), m
387 + z_te = m.predict(Xte) + test["ln_area"].to_numpy()
388 + z_tr = (m.predict(Xtr) + train["ln_area"].to_numpy() if both
389 + else np.log(y_train))
390 + return _log_out(z_te, z_tr, m)
249 391
250 392 raise ValueError(kind)
251 393
252 394
253 −def run_horserace(df: pd.DataFrame, scheme: str) -> pd.DataFrame:
254 − """Evaluate the full registry under one split scheme.
255 −
256 − Under the forward-in-time split, test-period time categories (months,
257 − quarters, years) were never seen in training. A one-hot encoder would
258 − silently price them at the *baseline* period; the honest forecasting
259 − rule for a static model is to freeze the price level at the last
260 − period observed in training, so unseen time labels are remapped to the
261 − latest training label before prediction.
262 − """
395 +def prepare(df: pd.DataFrame, scheme: str):
396 + """Derived columns, split, and the carry-forward rule for the
397 + forward-in-time scheme. Returns (train, test)."""
263 398 d = add_derived(df)
264 399 d["sale_year_c"] = d["sale_year"].astype(str)
265 400 train, test = split(d, scheme)
@@ -268,16 +403,58 @@ def run_horserace(df: pd.DataFrame, scheme: str) -> pd.DataFrame:
268 403 for col in ("quarter", "month", "sale_year_c"):
269 404 last = train[col].max()
270 405 test.loc[~test[col].isin(train[col].unique()), col] = last
406 + return train, test
407 +
408 +
409 +def run_horserace(df: pd.DataFrame, scheme: str,
410 + registry: list[dict] | None = None,
411 + return_predictions: bool = False):
412 + """Evaluate a registry (default: the twenty models) under one split.
413 +
414 + Under the forward-in-time split, test-period time categories (months,
415 + quarters, years) were never seen in training. A one-hot encoder would
416 + silently price them at the *baseline* period; the honest forecasting
417 + rule for a static model is to freeze the price level at the last
418 + period observed in training, so unseen time labels are remapped to the
419 + latest training label before prediction. Tree ensembles are piecewise
420 + constant in the month counter, so a test month beyond the training
421 + range is priced by the terminal leaf — the tree analogue of the same
422 + carry-forward rule.
423 +
424 + Metrics are reported for the median-consistent prediction (main
425 + scoreboard, suffix none) and for the Duan-smeared mean prediction
426 + (suffix ``_mean``). With ``return_predictions`` the test-set
427 + predictions of every model are returned as a second DataFrame.
428 + """
429 + train, test = prepare(df, scheme)
271 430 y_test = test["price"].to_numpy(float)
272 − rows = []
273 − for spec in model_registry():
431 + rows, preds = [], {}
432 + for spec in (registry if registry is not None else model_registry()):
433 + spec = dict(spec)
274 434 t0 = time.time()
275 − pred, _ = fit_predict(dict(spec), train, test)
276 − met = _metrics(y_test, pred)
435 + fit_predict(spec, train, test, retransform="mean")
436 + p_med, p_mean = spec["pred_median"], spec["pred_mean"]
437 + met = _metrics(y_test, p_med)
438 + met_mean = {f"{k}_mean": v for k, v in
439 + _metrics(y_test, p_mean).items()}
277 440 rows.append({"name": spec["name"], "group": spec["group"],
278 − **met, "seconds": round(time.time() - t0, 1),
441 + **met, **met_mean,
442 + "smear": spec.get("smear", np.nan),
443 + "lambda": spec.get("lambda", np.nan),
444 + "best_iter": spec.get("best_iter", np.nan),
445 + "seconds": round(time.time() - t0, 1),
279 446 "n_train": len(train), "n_test": len(test)})
280 − print(f" [{scheme}] {spec['name']:<38} "
447 + preds[spec["name"]] = np.clip(p_med, config.PRED_FLOOR, None)
448 + print(f" [{scheme}] {spec['name']:<44} "
281 449 f"MdAPE={met['mdape']:5.1f}% R2_ln={met['r2_ln']:.3f} "
282 450 f"({rows[-1]['seconds']}s)")
283 − return pd.DataFrame(rows)
451 + res = pd.DataFrame(rows)
452 + if not return_predictions:
453 + return res
454 + P = test[["id", "sale_date", "price", "muni", "prop_class", "grid",
455 + "parcel", "coord"]].reset_index(drop=True)
456 + P["in_train_parcel"] = P["parcel"].isin(set(train["parcel"]))
457 + P["in_train_coord"] = P["coord"].isin(set(train["coord"]))
458 + for k, v in preds.items():
459 + P[k] = v
460 + return res, P
modified src/wp11/sample.py +8 −2
@@ -73,6 +73,12 @@ def build(raw: pd.DataFrame | None = None) -> pd.DataFrame:
73 73 df["prop_class"] = _property_class(df)
74 74 df["link"] = df["role_lien_physique"].map(LINK_MAP).fillna("unknown")
75 75 df["muni"] = df["role_code_mun"]
76 + # Grouping keys for leakage-free holdouts: the assessment-roll parcel
77 + # identifier (repeat sales of the same property) and the exact
78 + # geocoded coordinate pair (units sharing a building or complex).
79 + df["parcel"] = df["role_id_provinc"].astype(str)
80 + df["coord"] = (df["lat"].round(6).astype(str) + "_"
81 + + df["lng"].round(6).astype(str))
76 82 df["grid"] = ((df["lat"] / config.GRID_DEG).round().astype(int).astype(str)
77 83 + "_" + (df["lng"] / config.GRID_DEG).round().astype(int).astype(str))
78 84 df["grid5"] = ((df["lat"] / config.GRID5_DEG).round().astype(int).astype(str)
@@ -91,8 +97,8 @@ def build(raw: pd.DataFrame | None = None) -> pd.DataFrame:
91 97 keep = ["id", "sale_date", "sale_year", "month", "quarter", "t",
92 98 "price", "ln_price", "area", "ln_area", "lot", "ln_lot", "has_lot",
93 99 "age", "floors", "units", "prop_class", "link",
94 − "muni", "grid", "grid5", "lat", "lng", "city",
95 − "role_municipalite"]
100 + "muni", "grid", "grid5", "parcel", "coord", "lat", "lng",
101 + "city", "role_municipalite"]
96 102 return df[keep].reset_index(drop=True)
97 103
98 104
99 105
Menu

Theme