révision scientifique complète (editorial board, 2026-10-17)
- scoreboard uniforme (règle médiane pour tous les modèles ; Duan en robustesse) : le « renversement » forward-in-time de la v1.0 était un artefact de retransformation - inférence : bootstrap par grappes (municipalités) + tests DM/GW groupés sur toutes les comparaisons ; décomposition niveau/compétence du generalization gap - robustesse : splits groupés parcelle/bâtiment, FE spatiaux emboîtés, Box–Cox fin, sensibilité HGB ; nouvelle section 7, 9 tables et 1 figure - méthodologie formalisée (13 équations) ; littérature 33 → 58 références vérifiées - docs/REVIEW-REPORT.md, docs/CHANGELOG-REVISION.md ; version 1.1, October 2026 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
62 changed files +3,545 −758
modified
README.md
+76 −36
@@ -4,7 +4,7 @@ | ||
| 4 | 4 | **UQO Working Paper No. 11** — *A Systematic Assessment of Hedonic Specifications |
| 5 | 5 | and Estimation Methods for the Quebec Housing Market, 2021–2026* |
| 6 | 6 | |
| 7 | −[-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) | |
| 7 | +[-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) | |
| 8 | 8 | [](requirements.txt) |
| 9 | 9 | [](data/raw/README.md) |
| 10 | 10 | [](src/wp11/models.py) |
@@ -15,13 +15,19 @@ and Estimation Methods for the Quebec Housing Market, 2021–2026* | ||
| 15 | 15 | > **TL;DR** — A controlled horse race of **20 hedonic models** on 514,212 Quebec sales |
| 16 | 16 | > (2021–2026) with assessor-grade structural attributes: 6 functional forms × a time-FE |
| 17 | 17 | > ladder × a spatial-FE ladder × 6 estimation methods, all scored on the **same |
| 18 | −> price-level scoreboard** (Duan smearing / Box–Cox inversion) under **two holdouts** | |
| 19 | −> (random 80/20 and forward-in-time). Functional form is second-order (±2 pp of MdAPE); | |
| 20 | −> spatial controls are first-order (~10 pp) but overfit at ~1 km granularity; **gradient | |
| 21 | −> boosting wins under random validation** (MdAPE 14.4% vs 16.5% for the best linear | |
| 22 | −> model) — but under the forward-in-time split *every* model degrades and the ML | |
| 23 | −> advantage **reverses** (splines 19.1% vs boosting 21.2%, identical R²_ln): random | |
| 24 | −> cross-validation is the wrong experiment for valuation models. | |
| 18 | +> price-level scoreboard** (one retransformation rule for every model) under **two | |
| 19 | +> holdouts** (random 80/20 and forward-in-time), with municipality-clustered bootstrap | |
| 20 | +> intervals and Diebold–Mariano-type tests on every comparison. Functional form is | |
| 21 | +> second-order (1.5 pp of MdAPE); spatial controls are first-order (~10 pp), and a ~1 km | |
| 22 | +> grid overfits when used alone but becomes the best linear specification once nested in | |
| 23 | +> municipalities; **gradient boosting wins under random validation** (14.4% vs 17.6% for | |
| 24 | +> the best linear model, −3.2 pp [−4.7, −1.9]), a gain that survives parcel- and | |
| 25 | +> building-grouped holdouts. Under the forward-in-time split (train < 2025, test 2025–26) | |
| 26 | +> every model degrades and the gap shrinks to 0.5 pp and to insignificance — but the | |
| 27 | +> decomposition shows this is a *level* (staleness) effect: re-levelled, boosting still | |
| 28 | +> leads by 2.9 pp. Random cross-validation hides the staleness cost; it does not reverse | |
| 29 | +> the ranking (the reversal reported in v1.0 was an artefact of an asymmetric | |
| 30 | +> retransformation, corrected in v1.1). | |
| 25 | 31 | |
| 26 | 32 | --- |
| 27 | 33 | |
@@ -29,8 +35,8 @@ and Estimation Methods for the Quebec Housing Market, 2021–2026* | ||
| 29 | 35 | |
| 30 | 36 | | Item | Value | |
| 31 | 37 | |---|---| |
| 32 | −| Version | 1.0 (August 2026) | | |
| 33 | −| SSRN | [7626698](https://ssrn.com/abstract=7626698) — submitted 2026-10-17 (preliminary upload) | | |
| 38 | +| Version | **1.1 (October 2026)** — revised 2026-10-17 by the editorial-board review (see `docs/`); v1.0 = August 2026 | | |
| 39 | +| SSRN | [7626698](https://ssrn.com/abstract=7626698) — v1.0 uploaded 2026-10-17; v1.1 PDF to be uploaded as a revision | | |
| 34 | 40 | | Journal | not yet submitted | |
| 35 | 41 | | Code & data | reproducible from `scripts/` (see Quick start); raw parquet snapshot not redistributed | |
| 36 | 42 | |
@@ -77,23 +83,35 @@ and accuracy by market segment. | ||
| 77 | 83 | |
| 78 | 84 | ## 🏆 Headline results |
| 79 | 85 | |
| 86 | +All MdAPE in %, median retransformation rule for every model (v1.1 scoreboard). | |
| 87 | + | |
| 80 | 88 | | Object | Random 80/20 holdout | Forward-in-time holdout | |
| 81 | 89 | |---|---|---| |
| 82 | −| Functional-form range (A1–A6), MdAPE | 16.5 % (splines) – 18.6 % (Box–Cox) | 19.1 % – 21.0 % | | |
| 83 | −| Data-driven Box–Cox exponent | λ̂ = 0.25, buys nothing out of sample | — | | |
| 84 | −| Dropping spatial FE (C1 vs C2) | 27.9 % vs 18.1 % (+10 pp) | 26.5 % vs 20.5 % | | |
| 85 | −| ~1.1 km grid FE (C4) | 19.4 % — **overfits** vs 17.5 % at ~5.5 km | 22.1 % | | |
| 86 | −| Dropping time FE (B1 vs B3) | 21.1 % vs 18.1 % | 25.4 % vs 20.5 % | | |
| 87 | −| Gradient boosting (D4) | **14.4 %** (−13 % vs best linear 16.5 %) | 21.2 % | | |
| 88 | −| Random forest (D3) | 14.6 % | 21.3 % | | |
| 89 | −| Boosting without coordinates (D5) | 23.2 % | 26.2 % | | |
| 90 | −| Semi-log + splines (A6) | 16.5 % | **19.1 %** — beats boosting, same R²_ln (0.52) | | |
| 91 | −| Implied monthly indices across forms | agree within a few index points | | | |
| 92 | −| Boosted implicit age / area profiles | track quadratic OLS closely | | | |
| 93 | − | |
| 94 | −Practical synthesis: a semi-log with splines, quarter effects and spatial FE matched to | |
| 95 | −market thickness is within two points of the frontier in deployment conditions — | |
| 96 | −transparent, auditable, fast. Validate against the future, not a shuffle of the past. | |
| 90 | +| Functional-form range (A1–A6) | 17.6 (splines) – 19.2 (quadratics) | 20.4 (linear) – 23.3 | | |
| 91 | +| Splines vs. semi-log | −1.4 pp [−1.8, −1.1] | −1.4 pp [−2.3, −0.8] | | |
| 92 | +| Data-driven Box–Cox exponent | λ̂ = 0.25 (0.35 on a fine grid), −0.5 pp vs. semi-log, worse on MAPE | — | | |
| 93 | +| Dropping spatial FE (C1 vs C2) | 28.8 vs 19.2 (+9.6 pp) | 29.0 vs 23.3 | | |
| 94 | +| ~5.5 km grid vs. municipality FE (C3 vs C2) | 18.7 vs 19.2 — **not significant** (−0.45 [−1.07, +0.37]) | 24.3 vs 23.3 | | |
| 95 | +| ~1.1 km grid alone (C4) | 21.0 — overfits (+1.8 pp [+1.2, +2.3]) | 25.3 | | |
| 96 | +| ~1.1 km grid **nested in municipalities** (C4′) | **17.3** — best linear specification | 21.6 | | |
| 97 | +| Dropping time FE (B1 vs B3) | 22.0 vs 19.2 | 29.3 vs 23.3 | | |
| 98 | +| Gradient boosting (D4) vs. best linear (A6) | **14.4** vs 17.6, −3.2 pp [−4.7, −1.9], DM t = −3.6 | 21.2 vs 21.7, −0.5 pp [−1.7, +0.6], n.s. | | |
| 99 | +| Random forest (D3) | 14.6 | 21.3 | | |
| 100 | +| Boosting without coordinates (D5) | 23.2 | 26.2 | | |
| 101 | +| Parcel- / building-grouped holdout, D4 vs A6 | 14.5 vs 17.6 / 14.6 vs 17.8 (gap unchanged) | — | | |
| 102 | +| Staleness cost (forward − same-period random) | — | D4 +7.5 pp, A6 +4.7, A1 +3.3 | | |
| 103 | +| Median log level error, forward split | — | D4 +0.18, A6 +0.16, A1 +0.10 | | |
| 104 | +| Re-levelled forward MdAPE | — | **D4 13.9 vs A6 16.8** (gap 2.9 ≈ random 3.5) | | |
| 105 | +| Boosting lead over splines by test quarter | — | 1.5 pp (2025Q1) → 0.0 (2026Q3) | | |
| 106 | +| Monthly indices, four linear forms | agree within 2.5 index points | | | |
| 107 | +| Boosted imputation index, July 2026 | 160.5 vs 168.6–171.0 for the linear forms | | | |
| 108 | +| Boosted age profile vs. quadratic OLS | same shape, 0.10–0.13 log points flatter from age 20 | | | |
| 109 | + | |
| 110 | +Practical synthesis: a semi-log with splines, quarter effects and spatial fixed effects | |
| 111 | +nested from municipality down to a fine grid is within three points of the frontier under | |
| 112 | +random validation and indistinguishable from it in deployment. Supply the price level from | |
| 113 | +an index or a frequent refit, spend the flexibility budget on the spatial surface, and | |
| 114 | +validate against the future, not a shuffle of the past. | |
| 97 | 115 | |
| 98 | 116 | ## 🗂 Repository layout |
| 99 | 117 | |
@@ -148,8 +166,9 @@ peak RAM ≈ 8 GB. | ||
| 148 | 166 | | 01 | `01_build_sample.py` | residential use codes, roll match ≤ 50 m / score ≥ 150, price ≥ $50k, floor area 20–2,000 m², known year built, 1/99 % trims within sale year; builds grid cells | `data/processed/`, `results/reproduced/sample_counts.csv`, `summary_stats.csv` | |
| 149 | 167 | | 02 | `02_horserace.py` | fits the 20 models on each split, retransforms predictions to price levels, scores MdAPE/MAPE/RMSE_ln/R²_ln; Box–Cox profile likelihood | `horserace_random.csv`, `horserace_temporal.csv`, `boxcox.csv` | |
| 150 | 168 | | 03 | `03_extensions.py` | constant-quality monthly indices by form, implicit ln-price profiles (quadratic OLS vs boosting), learning curves, accuracy by segment | `index.csv`, `profiles.csv`, `learning.csv`, `segments.csv` | |
| 151 | −| 04 | `04_make_figures.py` | the 9 figures | `figures/fig_*.png` | | |
| 152 | −| 05 | `05_make_tables.py` | the 7 LaTeX tables | `results/tables/*.tex` | | |
| 169 | +| 04 | `04_make_figures.py` | the 10 figures | `figures/fig_*.png` | | |
| 170 | +| 05 | `05_make_tables.py` | the 16 LaTeX tables | `results/tables/*.tex` | | |
| 171 | +| 06 | `06_robustness.py` | cluster-bootstrap CIs + clustered DM tests, same-period and by-quarter decomposition, level re-levelling, leakage decomposition, parcel- and building-grouped races, nested spatial FE, fine Box–Cox grid, boosting hyper-parameter sensitivity (run after 02; ≈ 20 min) | `inference_*.csv`, `pairs.csv`, `same_period.csv`, `by_quarter.csv`, `relevel.csv`, `leakage.csv`, `horserace_group_*.csv`, `nested_fe.csv`, `boxcox_fine.csv`, `hgb_sensitivity.csv` | | |
| 153 | 172 | |
| 154 | 173 | ## 🗃 Data |
| 155 | 174 | |
@@ -161,23 +180,28 @@ physical configuration, use code. Sample cascade (paper §3): residential use co | ||
| 161 | 180 | high-confidence match → price ≥ $50,000 → plausible floor area → known year built → |
| 162 | 181 | 1/99 % trims ⇒ **514,212 sales** (348,253 single-family, 75,734 condominiums, 74,909 |
| 163 | 182 | plexes, 10,349 cottages, 4,881 mobile homes) over 1,115 municipalities, 5,255 cells of |
| 164 | −~5.5 km and 30,548 cells of ~1.1 km. Mean price $462,000 (median $399,000). The raw | |
| 183 | +~5.5 km and 30,548 cells of ~1.1 km. Mean price $440,207 (median $397,375); 86 further dwellings of undetermined class. One fifth of sales are repeat sales of the same parcel and 30% share exact coordinates with another sale (92% of condominiums). The raw | |
| 165 | 184 | parquet (100 MB, transaction microdata) is **not redistributed**. |
| 166 | 185 | |
| 167 | 186 | ## 🔬 Methodology |
| 168 | 187 | |
| 169 | 188 | 1. **One attribute set, one sample** for every model (`src/wp11/sample.py`). |
| 170 | 189 | 2. **Axis A** — six functional forms on a municipality + quarter FE backbone; Box–Cox λ by |
| 171 | − profile likelihood; log models retransformed with Duan smearing, Box–Cox inverted | |
| 172 | − (`src/wp11/models.py`). | |
| 190 | + profile likelihood. Every transformed-scale prediction is inverted with the same | |
| 191 | + *median rule* (plain inversion); Duan smearing is computed for every model and | |
| 192 | + reported as a robustness scoreboard (`src/wp11/models.py`). | |
| 173 | 193 | 3. **Axis B / C** — time-FE ladder (none → month) and spatial-FE ladder (none → ~1.1 km |
| 174 | 194 | grid) on the quadratic semi-log backbone. |
| 175 | 195 | 4. **Axis D** — OLS, cross-validated ridge, random forest, gradient boosting with and |
| 176 | 196 | without coordinates, spatial k-NN comparables, all on the identical attribute set. |
| 177 | 197 | 5. **Two holdouts** — random 80/20 and forward-in-time (train < 2025, test 2025–26, time |
| 178 | − effects carried forward); metrics on price levels, design clustered by municipality. | |
| 198 | + effects carried forward), plus parcel- and building-grouped holdouts; metrics on price | |
| 199 | + levels; municipality-clustered bootstrap intervals and clustered Diebold–Mariano tests | |
| 200 | + (`src/wp11/inference.py`); level/skill decomposition of the forward error. | |
| 179 | 201 | 6. **Extensions** — implied monthly indices, implicit-price profiles, learning curves, |
| 180 | 202 | segment accuracy (`scripts/03_extensions.py`). |
| 203 | +7. **Robustness** — retransformation rule, leakage-free holdouts, nested spatial FE, fine | |
| 204 | + Box–Cox grid, boosting hyper-parameters (`scripts/06_robustness.py`). | |
| 181 | 205 | |
| 182 | 206 | ## 🖼 Figures & tables inventory |
| 183 | 207 | |
@@ -192,10 +216,13 @@ parquet (100 MB, transaction microdata) is **not redistributed**. | ||
| 192 | 216 | | `fig_profiles` | implicit ln-price profiles: quadratic OLS vs gradient boosting | |
| 193 | 217 | | `fig_learning` | learning curves: MdAPE against training-set size | |
| 194 | 218 | | `fig_segments` | MdAPE by market segment (random holdout) | |
| 219 | +| `fig_horizon` | forward-in-time MdAPE by test quarter (distance from the training window) | | |
| 195 | 220 | |
| 196 | 221 | Tables (`results/tables/*.tex`): summary statistics, scoreboard random holdout, |
| 197 | −scoreboard forward-in-time holdout, Box–Cox profile, generalization gap, learning | |
| 198 | −curves, segments. | |
| 222 | +scoreboard forward-in-time holdout, inference (bootstrap CIs + DM tests), targeted pairs, | |
| 223 | +Box–Cox profile, generalization gap, same-period / level decomposition, by-quarter | |
| 224 | +accuracy, learning curves, segments, median vs. Duan retransformation, grouped holdouts, | |
| 225 | +seen/unseen leakage decomposition, nested spatial FE, boosting hyper-parameters. | |
| 199 | 226 | |
| 200 | 227 | ## ⚠️ Limitations |
| 201 | 228 | |
@@ -206,8 +233,13 @@ curves, segments. | ||
| 206 | 233 | - Attributes are those of the assessment roll (no interior quality, renovations or |
| 207 | 234 | listing text); the ML advantage measured here is the value of flexible *spatial* |
| 208 | 235 | learning on that attribute set, not of richer data. |
| 209 | −- Tree ensembles are tuned lightly (fixed hyper-parameters); the ranking is robust to | |
| 210 | − the choices tried but not exhaustively optimised. | |
| 236 | +- Tree ensembles use fixed, conventional hyper-parameters; four materially different | |
| 237 | + configurations (including early stopping on a time-ordered validation set) leave the | |
| 238 | + ranking unchanged (paper §7.5), but no linear model with a continuous spatial surface | |
| 239 | + (kriging, bivariate splines) is estimated as a competitor. | |
| 240 | +- The forward-in-time test covers one regime (the 2025–26 recovery); the level | |
| 241 | + re-levelling uses the test-set median ratio (an oracle upper bound for what a published | |
| 242 | + index would deliver). | |
| 211 | 243 | |
| 212 | 244 | ## 📝 Citation |
| 213 | 245 | |
@@ -222,7 +254,8 @@ curves, segments. | ||
| 222 | 254 | type = {Working Paper}, |
| 223 | 255 | number = {11}, |
| 224 | 256 | year = {2026}, |
| 225 | − month = {August} | |
| 257 | + month = {October}, | |
| 258 | + note = {Version 1.1} | |
| 226 | 259 | } |
| 227 | 260 | ``` |
| 228 | 261 | |
@@ -239,3 +272,10 @@ Université du Québec en Outaouais (UQO), Gatineau, QC. | ||
| 239 | 272 | Language-Model and Vision Features* (now **WP 14**) and *Does the Sales Comparison |
| 240 | 273 | Approach Actually Work?* (now **WP 16**). |
| 241 | 274 | - Cover page, metadata and layout follow `Recherche-UQO/HARMONISATION.md` (2026-10-17). |
| 275 | +- **Revision 2026-10-17 (v1.1)**: full editorial-board review. Scientific report in | |
| 276 | + `docs/REVIEW-REPORT.md`, audit trail (every changed number, old → new) in | |
| 277 | + `docs/CHANGELOG-REVISION.md`; pre-revision PDF kept as `docs/main_pre-revision-20261017.pdf` | |
| 278 | + (git tag `pre-revision-20261017`). Main correction: v1.0 applied Duan smearing to the | |
| 279 | + linear log models but not to the tree ensembles; under a uniform rule the forward-in-time | |
| 280 | + "reversal" disappears (the gap shrinks to 0.5 pp, n.s.) and the paper now decomposes the | |
| 281 | + forward error into a common level (staleness) component and intact relative skill. | |
added
docs/CHANGELOG-REVISION.md
+165 −0
@@ -0,0 +1,165 @@ | ||
| 1 | +# Changelog — révision scientifique complète (editorial board, 2026-10-17) | |
| 2 | + | |
| 3 | +Papier : **WP 11 — Half a Million Prices, Twenty Models** (v1.0, August 2026 → **v1.1, October 2026**). | |
| 4 | +État de référence : tag git `pre-revision-20261017`, PDF `docs/main_pre-revision-20261017.pdf` (23 pages). | |
| 5 | +Rapport scientifique associé : `docs/REVIEW-REPORT.md`. | |
| 6 | + | |
| 7 | +## 1. Corrections majeures (où / avant / après / pourquoi / preuve) | |
| 8 | + | |
| 9 | +### 1.1 Règle de retransformation asymétrique → scoreboard uniforme (correction centrale) | |
| 10 | +- **Où** : `src/wp11/models.py` (`fit_predict`, `run_horserace`), §4.3 Methodology, §5, §7.1, abstract, intro, conclusion. | |
| 11 | +- **Avant** : les modèles linéaires en log (A2, A3, A5, A6, B*, C*, D1, D2) étaient multipliés par le facteur de | |
| 12 | + smearing de Duan (ŝ ≈ 1.06–1.08) ; les ensembles d'arbres (D3, D4, D5) et le k-NN (D6) ne l'étaient pas | |
| 13 | + (`np.exp(pred)` nu). Les métriques log (RMSE_ln, R²_ln) étaient calculées sur la prédiction smearée pour une | |
| 14 | + famille et non pour l'autre. | |
| 15 | +- **Après** : règle « médiane » (inversion directe de la prédiction transformée, éq. 10) appliquée **à tous** les | |
| 16 | + modèles pour le tableau principal ; règle « moyenne » (Duan, éq. 11, généralisée au Box–Cox par moyenne sur | |
| 17 | + 1 000 résidus) calculée pour tous et rapportée en robustesse (Table 12). | |
| 18 | +- **Pourquoi** : la comparaison v1.0 opposait un A6 smearé (19.1 %) à un D4 non smearé (21.2 %) dans le split | |
| 19 | + temporel — le « renversement » annoncé en titre de la v1.0 était un artefact : (i) résidus log asymétriques | |
| 20 | + à gauche (skewness −1.4 à −1.8, médiane +0.06) donc le smearing abaisse par coïncidence la MdAPE de tout modèle | |
| 21 | + log ; (ii) marché haussier en 2025-26 : tout facteur > 1 compense une partie de la staleness. Appliquée | |
| 22 | + uniformément, aucune des deux règles ne renverse le classement (médiane : A6 21.7 vs D4 21.2 ; moyenne : | |
| 23 | + 19.1 vs 18.4). | |
| 24 | +- **Preuve** : `results/reproduced/horserace_{random,temporal}.csv` (colonnes `mdape` vs `mdape_mean`, `smear`), | |
| 25 | + `results/tables/retransformation.tex`, médianes de résidus vérifiées sur `data/processed/predictions_*.parquet`. | |
| 26 | +- **Chiffres modifiés (ancien → nouveau, MdAPE %)** — random : A1 18.2→18.2, A2 18.2→19.0, A3 17.0→18.0, A4 18.6→18.6, | |
| 27 | + A5 18.1→19.2, A6 16.5→17.6, B1 21.1→22.0, B2 18.1→19.1, B4 18.3→19.3, C1 27.9→28.8, C3 17.5→18.7, C4 19.4→21.0, | |
| 28 | + D2 16.7→17.7 ; D3/D4/D5/D6 inchangés (14.6/14.4/23.2/21.2). Forward : A1 20.4→20.4, A2 20.6→23.2, A3 19.2→21.8, | |
| 29 | + A4 21.0→21.0, A5 20.5→23.3, A6 19.1→21.7, B1 25.4→29.3, B2 21.0→23.7, B4 20.3→22.9, C1 26.5→29.0, C3 21.3→24.3, | |
| 30 | + C4 22.1→25.3, D2 19.5→22.3 ; D3/D4/D5/D6 inchangés (21.3/21.2/26.2/25.6). R²_ln des modèles log : +0.01 à +0.04 | |
| 31 | + (random), −0.03 à −0.04 (forward). | |
| 32 | +- **Conséquence narrative** : « the ranking reverses » (v1.0) → « the gap shrinks from 3.2 to 0.5 points and loses | |
| 33 | + significance ; relative skill transfers intact, the level does not » (v1.1). Abstract, §1, §5.5, §8, §9 réécrits. | |
| 34 | + | |
| 35 | +### 1.2 Inférence statistique ajoutée (absente en v1.0) | |
| 36 | +- **Avant** : aucun intervalle, aucun test ; le README affirmait « design clustered by municipality » sans aucune | |
| 37 | + implémentation. | |
| 38 | +- **Après** : `src/wp11/inference.py` — bootstrap par grappes (municipalités, B = 500) pour la MdAPE de chaque | |
| 39 | + modèle et pour les différences ; test de type Diebold–Mariano / Giacomini–White à erreurs-types groupées sur les | |
| 40 | + pertes APE et log² (éq. 13). Tables 4 (tous les modèles vs A6) et 5 (13 paires ciblées). | |
| 41 | +- **Résultats clés** : D4 vs A6 random −3.2 [−4.7, −1.9] (t = −3.6) ; forward −0.5 [−1.7, +0.6] (t = −1.6, p = 0.11) ; | |
| 42 | + C3 vs C2 random −0.45 [−1.07, +0.37] (n.s.) ; C4 vs C2 +1.8 [+1.2, +2.3] ; B4 vs B3 +0.19 [+0.01, +0.37] ; | |
| 43 | + D2 vs A6 +0.06 [−0.02, +0.16] (n.s.). | |
| 44 | +- **Conséquence** : la phrase « the 5.5 km grid beats municipality effects » est ramenée à « nominally better, | |
| 45 | + within bootstrap noise » ; « month effects indistinguishable from quarter » précisé (légèrement pires, significatif). | |
| 46 | + | |
| 47 | +### 1.3 Décomposition du « generalization gap » (nouveau) | |
| 48 | +- Comparaison à période fixe (modèles random évalués sur les 31 465 ventes 2025-26 de leur test) : D4 13.7 → 21.2 | |
| 49 | + (coût de staleness 7.5), A6 17.0 → 21.7 (4.7), A1 17.1 → 20.4 (3.3). Biais de niveau médian forward : D4 +0.183, | |
| 50 | + A6 +0.161, A1 +0.102. MdAPE re-nivelée forward : D4 13.9 vs A6 16.8 (écart 2.9 ≈ écart random 3.5). | |
| 51 | + Par trimestre : avance D4 sur A6 de 1.5 pt (2025Q1) → 0 (2026Q3). | |
| 52 | +- Fichiers : `same_period.csv`, `relevel.csv`, `by_quarter.csv` ; Tables 8–9, Figure 6 (nouvelle). | |
| 53 | + | |
| 54 | +### 1.4 Fuite d'information dans le split aléatoire (nouveau) | |
| 55 | +- Faits : 20.6 % des ventes sont des reventes d'une même parcelle ; 30.3 % partagent des coordonnées exactes avec | |
| 56 | + une autre vente (91.6 % des condos). Colonnes `parcel`, `coord` ajoutées à `sample.py` (reconstruction du | |
| 57 | + parquet d'analyse, 514 212 ventes identiques). | |
| 58 | +- Splits groupés (`split("group_parcel")`, `split("group_coord")`) : course complète. Parcelle : D4 14.5 / A6 17.6 | |
| 59 | + (≈ inchangé). Bâtiment : D4 14.6 / A6 17.8 (écart 3.2 identique) ; seul D6 se dégrade (21.2 → 22.3). | |
| 60 | + Décomposition vu/non vu : D4 12.0 vs 15.3 (coord.), A6 17.8 vs 17.6 ; condos D4 8.9 vs 10.2. | |
| 61 | +- Tables 13–14 ; §4.4, §7.2. | |
| 62 | + | |
| 63 | +### 1.5 Effets spatiaux fins : « seventeen K sales » et repli arbitraire | |
| 64 | +- **Avant** (§5.3) : « its cells average seventeen K sales each » — faux : 514 212 / 30 548 = 16.8 ventes par cellule | |
| 65 | + (médiane 3). Corrigé. Cellules non vues (2.5 % des ventes test) tarifées au niveau de la cellule de référence : | |
| 66 | + désormais explicité. | |
| 67 | +- **Nouveau** : C3′ (muni + grille 5.5 km) et C4′ (muni + grille 1.1 km) — solution de norme minimale (ridge α→0) | |
| 68 | + ≈ pooling partiel vers la municipalité. Random : C4′ 17.3 % (meilleure spécification linéaire du papier), | |
| 69 | + C3′ 17.7 % ; forward : 21.6 / 21.5 vs C2 23.3. Conclusion de l'axe C reformulée : « hierarchy, not coarseness ». | |
| 70 | + Table 15 ; `nested_fe.csv`. | |
| 71 | + | |
| 72 | +### 1.6 Box–Cox : grille fine | |
| 73 | +- Grille 0.05 sur [−0.25, 0.75] : λ̂ = 0.35 (profil plat 0.25–0.55 à la tolérance du solveur itératif) ; | |
| 74 | + MdAPE random 18.4 % (vs 18.6 % à 0.25). `boxcox_fine.csv`. Texte §5.1 et §7.4. | |
| 75 | + | |
| 76 | +### 1.7 Hyper-paramètres du boosting | |
| 77 | +- Quatre configurations alternatives (shallow, deep, régularisée, early stopping sur les 10 % derniers mois | |
| 78 | + d'entraînement) sous les deux splits. `hgb_sensitivity.csv`, Table 16, §7.5. Random 14.3–14.7 % (baseline 14.4), forward 21.2–21.6 % (baseline 21.2) ; early stopping : 700 / 1 000 itérations, reproduit le baseline à 0.1 pt. | |
| 79 | + | |
| 80 | +### 1.8 Statistiques descriptives incohérentes | |
| 81 | +- **Avant** (§3.2, README) : « Prices average $462,000 (median $399,000) ». **Table 1** : 440 207 / 397 375. | |
| 82 | + **Après** : texte aligné sur la table ; ajout des 86 logements de classe indéterminée (la somme des classes v1.0 | |
| 83 | + faisait 514 126 ≠ 514 212) ; répartition par année ; nettoyage des lots > P99.5 explicité (était silencieux | |
| 84 | + dans `sample.py`) ; bornes âge/étages/logements explicitées. | |
| 85 | + | |
| 86 | +### 1.9 Indices et profils : affirmations adoucies | |
| 87 | +- **Avant** : « all five [indices] agree to within a few index points » ; « curves sit within a tenth of a log | |
| 88 | + point ». **Après** : les quatre formes linéaires s'accordent à ≤ 2.5 points ; l'indice boosting finit 9 points | |
| 89 | + plus bas (160.5 vs 168.6–171.0 ; écart max 10.9) ; profil d'âge du boosting uniformément plus plat de 0.10–0.13 | |
| 90 | + log-point à partir de 20 ans (−0.20 vs −0.31 à 40 ans), surface : ≤ 0.05. Chiffres vérifiés dans | |
| 91 | + `index.csv`, `profiles.csv`. | |
| 92 | +- Learning curves recalculées (règle médiane) : OLS 19.1 → 19.2 (v1.0 : 18.4 → 18.1) ; GB 17.6 → 14.5 ; écart | |
| 93 | + 1.6 → 4.7 pts (v1.0 : < 1 → 3.6). Segments : condo 18.2 vs 9.1 (v1.0 OLS 20.4), grandes villes 17.7 vs 11.9, | |
| 94 | + petites municipalités 24.8 vs 18.7 (v1.0 23.3), chalets 34.3 vs 32.1 (v1.0 33.9). | |
| 95 | + | |
| 96 | +### 1.10 Méthodologie formalisée | |
| 97 | +- §4 réécrit : équations (1)–(13) — fonction hédonique générale, six formes A1–A6, profil de vraisemblance Box–Cox | |
| 98 | + (avec jacobien), ridge, règle k-NN, règles médiane/Duan, métriques, test DM groupé. Documentation du traitement | |
| 99 | + des catégories non vues, du carry-forward et de son analogue pour les arbres (correction de la phrase « clamped | |
| 100 | + month counter » : rien n'est écrêté dans le code ; les arbres sont constants par morceaux). | |
| 101 | + | |
| 102 | +### 1.11 Autres corrections éditoriales | |
| 103 | +- « 13 % improvement » → « 18 % » (14.4 vs 17.6) ; « 13–21 % » → « 18–25 % » ; intro « 27.9 % versus 17.5 % » | |
| 104 | + vs README « vs 18.1 % » harmonisé (28.8 vs 19.2, référence = backbone C2). | |
| 105 | +- Hiérarchie de la discussion : liste à puces → prose (règle de style du protocole). | |
| 106 | +- Mots-clés harmonisés au vocabulaire commun (« hedonic price function », « forward-in-time holdout », | |
| 107 | + « automated valuation model »). | |
| 108 | +- `\WPversion` 1.0 → 1.1 ; `\WPdate` August 2026 → October 2026 ; page titre canonique intacte ; macro | |
| 109 | + `\companionpaper` conservée (main.tex / main_anon.tex). | |
| 110 | + | |
| 111 | +## 2. Références | |
| 112 | + | |
| 113 | +### Vérifiées (Crossref `api.crossref.org/works?query.bibliographic=…`, 2026-10-17) et corrigées | |
| 114 | +Toutes les entrées conservées de la v1.0 (31) ont été retrouvées sur Crossref et dotées de leur DOI. Corrections : | |
| 115 | +- `box1964analysis` : pages 211–252 → **211–243** (244–252 = discussion). | |
| 116 | +- `bogin2020house` : number 1 → **1–2** ; DOI 10.1007/s11146-019-09712-0. | |
| 117 | +- `silver2018house` : l'article EURONA 2018 n'est pas indexé ; remplacé par la version vérifiée **IMF Working Paper | |
| 118 | + 16/213 (2016)**, DOI 10.5089/9781475552249.001, même titre (ordre des mots corrigé). | |
| 119 | +- `ke2017lightgbm` : @article → **@inproceedings** (NIPS 2017, non indexé Crossref ; actes Curran Associates). | |
| 120 | +- `malpezzi2003hedonic` : DOI 10.1002/9780470690680.ch5 ; `anselin1988spatial` : DOI 10.1007/978-94-015-7799-1. | |
| 121 | + | |
| 122 | +### Retirées (non citées dans le texte et bibliographiquement défectueuses) | |
| 123 | +- `knight2003listing` (volume/pages vides ; en réalité chapitre Springer 2008, pp. 39–54). | |
| 124 | +- `diewert2004hedonic` (« Mimeo » en champ journal ; non indexé). | |
| 125 | + | |
| 126 | +### Ajoutées (27, toutes vérifiées sur Crossref, DOI dans le .bib) | |
| 127 | +hill2018geospatial (RIW 64(4) 737–756) · pace2020residuals (JREFE 60 170–180) · lorenz2023interpretable | |
| 128 | +(REE 51(5) 1178–1208) · diewert2016tokyo (RSUE 60 300–315) · diewert2015decomposition (Econometric Reviews 34 | |
| 129 | +106–126) · diebold1995comparing (JBES 13(3) 253–263) · giacomini2006tests (Econometrica 74(6) 1545–1578) · | |
| 130 | +bergmeir2012crossvalidation (Information Sciences 191 192–213) · roberts2017crossvalidation (Ecography 40(8) | |
| 131 | +913–929) · kuminoff2010which (JEEM 60(3) 145–160) · bishop2020best (REEP 14(2) 260–281) · bourassa2007spatial | |
| 132 | +(JREFE 35(2) 143–160) · bourassa2003submarkets (JHE 12(1) 12–28) · goodman2003segmentation (JHE 12(3) 181–201) · | |
| 133 | +zurada2011comparison (JRER 33(3) 349–388) · antipov2012mass (ESWA 39(2) 1772–1778) · perezrave2019machine | |
| 134 | +(J. Property Research 36(1) 59–96) · mccluskey2013prediction (JPR 30(4) 239–265) · schulz2014automated (JPR 31(2) | |
| 135 | +131–153) · fik2003modeling (REE 31(4) 623–646) · clapp2003semiparametric (JREFE 27(3) 303–320) · | |
| 136 | +pace1998spatiotemporal (JREFE 17(1) 15–33) · dube2013pooled (LSRS 6(1) 1–18) · hill2009boom (JHE 18(3) 193–205) · | |
| 137 | +manning2001estimating (J. Health Econ. 20(4) 461–494) · thibodeau2003marking (REE 31(1) 1–22) · | |
| 138 | +khoshnoud2023evolution (JREL 31(1) 1–47). | |
| 139 | +Total : 33 → **58** entrées (31 conservées + 27 ajoutées), toutes citées, toutes vérifiées. | |
| 140 | + | |
| 141 | +## 3. Réanalyses exécutées (laptop, Apple Silicon 18 cœurs, Python 3.14 / scikit-learn 1.6.1) | |
| 142 | + | |
| 143 | +| Script | Contenu | Durée | Sorties | | |
| 144 | +|---|---|---|---| | |
| 145 | +| `01_build_sample.py` | reconstruction de l'échantillon (+ clés `parcel`, `coord`) — 514 212 ventes identiques | 2 s | `data/processed/analysis.parquet` | | |
| 146 | +| `02_horserace.py` | 20 modèles × 2 splits, règles médiane + Duan, prédictions sauvegardées | 3 min 28 s | `horserace_*.csv`, `boxcox.csv`, `predictions_*.parquet` (non commités) | | |
| 147 | +| `03_extensions.py` | indices, profils, learning curves, segments (règle médiane) | 4 min 18 s | `index.csv`, `profiles.csv`, `learning.csv`, `segments.csv` | | |
| 148 | +| `06_robustness.py` (nouveau) | bootstrap + DM, même période, par trimestre, fuite, re-nivellement, paires, splits groupés (2 courses complètes), FE emboîtés, Box–Cox fin, sensibilité HGB | 16 min 16 s | `inference_*.csv`, `same_period.csv`, `by_quarter.csv`, `leakage.csv`, `relevel.csv`, `pairs.csv`, `horserace_group_*.csv`, `nested_fe.csv`, `boxcox_fine.csv`, `hgb_sensitivity.csv` | | |
| 149 | +| `04_make_figures.py` | 10 figures (+ `fig_horizon.png`) | < 1 min | `figures/` | | |
| 150 | +| `05_make_tables.py` | 16 tables LaTeX (9 nouvelles) | < 1 s | `results/tables/` | | |
| 151 | + | |
| 152 | +Tous les nombres du texte proviennent de ces sorties ; aucune table n'est tapée à la main. | |
| 153 | + | |
| 154 | +## 4. Éléments non modifiés / analyses recommandées non exécutées | |
| 155 | +- Modèle linéaire à surface spatiale (krigeage des résidus, splines bivariées en lat/lng, effets hiérarchiques | |
| 156 | + bayésiens) : non estimé (hors cadre « 20 modèles ») — recommandé pour fermer l'écart linéaire/boosting par l'autre | |
| 157 | + côté. | |
| 158 | +- Holdout spatialement bloqué (cellules entières en test) : non exécuté (change la question en extrapolation | |
| 159 | + spatiale ; les FE spatiaux y sont par construction inopérants). | |
| 160 | +- Re-nivellement par un indice **publié** (hors échantillon, non oracle) : non exécuté (aucun indice externe | |
| 161 | + chargé ; le re-nivellement présenté utilise le ratio médian du test, borne supérieure). | |
| 162 | +- Variante « médiane calibrée » (exp(ẑ + médiane des résidus d'entraînement)) : discutée, non tabulée. | |
| 163 | +- Fenêtres temporelles alternatives (p. ex. train < 2024, test 2024) : non exécutées — limite déclarée §8.4. | |
| 164 | +- Comparaison avec LightGBM / XGBoost natifs ou réseaux de neurones : hors périmètre. | |
| 165 | +- La règle de nettoyage des lots > P99.5 → 0 (héritée) est documentée mais conservée pour préserver l'échantillon. | |
added
docs/REVIEW-REPORT.md
+118 −0
@@ -0,0 +1,118 @@ | ||
| 1 | +# Rapport de l'editorial board — WP 11 « Half a Million Prices, Twenty Models » (2026-10-17) | |
| 2 | + | |
| 3 | +Révision v1.0 (August 2026, 23 p.) → v1.1 (October 2026). Journal d'audit détaillé : `docs/CHANGELOG-REVISION.md`. | |
| 4 | +Comité : économie immobilière, économétrie appliquée, économétrie spatiale, évaluation foncière, apprentissage | |
| 5 | +automatique, édition scientifique. Revue cible : *Journal of Housing Economics* (repli JRER / IJF). | |
| 6 | + | |
| 7 | +## 1. Diagnostic général | |
| 8 | + | |
| 9 | +Le papier pose une question nette et utile — quels choix de spécification hédonique comptent réellement pour la | |
| 10 | +précision hors échantillon — et y répond par un dispositif expérimental propre (un jeu d'attributs, quatre axes, un | |
| 11 | +seul scoreboard, deux holdouts) sur un échantillon rare par sa taille et la qualité des descripteurs (rôle | |
| 12 | +d'évaluation). La contribution est réelle et conservée. Trois faiblesses structurelles de la v1.0 ont été | |
| 13 | +identifiées et corrigées : | |
| 14 | + | |
| 15 | +1. **Le résultat phare était un artefact.** Les modèles linéaires en log étaient retransformés par smearing de Duan | |
| 16 | + et les ensembles d'arbres non. Dans le split temporel, le smearing (facteur ≈ 1.06) compense une partie de la | |
| 17 | + staleness du niveau dans un marché haussier ; c'est ce qui produisait « A6 19.1 % vs D4 21.2 % » et le | |
| 18 | + « renversement » affiché en abstract. Sous une règle uniforme, le boosting reste devant (21.2 vs 21.7, écart non | |
| 19 | + significatif). Correction intégrale (code, tables, texte) ; la v1.1 le dit explicitement. | |
| 20 | +2. **Aucune inférence.** Des différences de 0.2 à 0.6 point étaient interprétées comme des résultats. Ajout d'un | |
| 21 | + bootstrap par grappes (municipalités) et de tests Diebold–Mariano/Giacomini–White groupés sur toutes les | |
| 22 | + comparaisons. Plusieurs affirmations tombent (grille 5.5 km > municipalités ; mois ≈ trimestre) ou se précisent. | |
| 23 | +3. **Mécanisme du « generalization gap » non identifié.** La v1.0 confondait difficulté de la période test et | |
| 24 | + staleness du modèle. Décomposition ajoutée (même période, biais de niveau, re-nivellement, horizon par | |
| 25 | + trimestre) : la compétence relative des ensembles transfère intacte (écart re-nivelé 2.9 pts ≈ 3.5 en random), | |
| 26 | + c'est le niveau qui ne transfère pas, un peu plus pour les arbres (feuille terminale en t). Ce résultat est plus | |
| 27 | + faible que le « renversement » mais plus utile et défendable. | |
| 28 | + | |
| 29 | +## 2. Diagnostic par section | |
| 30 | + | |
| 31 | +| Section | Forces | Faiblesses / erreurs trouvées | Action | | |
| 32 | +|---|---|---|---| | |
| 33 | +| Abstract | clair, chiffré | chiffres issus du scoreboard asymétrique ; « reverses », « identical R² » faux ; trop long pour la page 2 (débordait sur une page blanche) | réécrit, 4 leçons reformulées, mention de la correction | | |
| 34 | +| 1 Introduction | bonne mise en situation Rosen → ML | « 13 % improvement » (vrai : 18 %) ; « 27.9 vs 17.5 » (comparait C1 à C3 au lieu du backbone) ; aucune mention de l'inférence ni des fuites | réécrit ; renvoi à la nouvelle §7 | | |
| 35 | +| 2 Literature | couvre les trois courants | mince (33 réf., 29 citées) ; manquaient Hill–Scholz, Pace–Hayunga, Lorenz et al., Diewert–Shimizu, Kuminoff et al., DM/GW, Bergmeir–Benítez, Roberts et al., Dubé–Legros ; deux entrées non citées et défectueuses | +27 réf. vérifiées, −2 retirées ; Mayer et al. 2019 identifié comme antécédent direct (updating) | | |
| 36 | +| 3 Data | cascade d'échantillon transparente | moyenne/médiane des prix fausses dans le texte (462k/399k vs 440k/397k) ; 86 ventes « other » omises ; nettoyage des lots > P99.5 non documenté ; bornes âge/étages non dites | corrigé ; répartition annuelle ; repeat sales et coordonnées partagées quantifiés | | |
| 37 | +| 4 Methodology | design clair | aucune équation ; Duan non écrit ; inversion Box–Cox sans discussion du biais ; « clamped month counter » inexact (rien n'est écrêté ; arbres constants par morceaux) ; catégories non vues → référence arbitraire non dit ; hyper-paramètres « a priori » sans analyse | 13 équations ; règles médiane/moyenne ; carry-forward ; holdouts groupés ; inférence ; décomposition niveau/skill | | |
| 38 | +| 5 Results | lecture par axe efficace | « seventeen K sales » (17) ; ordre A sous split temporel non discuté (A1 gagne par coïncidence de biais) ; différences non testées | réécrit avec IC/tests ; §5.5 refondue (décomposition) | | |
| 39 | +| 6 Extensions | objets économiques pertinents | « all five agree within a few points » (indice GB −9 pts) ; « within a tenth of a log point » (0.10–0.13 systématique, profil plus plat) ; learning/segments chiffrés sous scoreboard asymétrique | affirmations ramenées aux chiffres ; mécanismes de l'écart d'indice discutés | | |
| 40 | +| 7 Robustness | — (n'existait pas) | — | nouvelle section : retransformation, fuites, FE emboîtés, Box–Cox fin, hyper-paramètres HGB | | |
| 41 | +| 8 Discussion | hiérarchie utile | liste à puces ; « the machines lose their edge » (faux) ; « 3–7 points » sans mécanisme | prose ; hiérarchie révisée ; mécanisme niveau vs skill ; implications réécrites | | |
| 42 | +| 9 Conclusion | synthèse pratique | reprenait le renversement ; « within two points of the frontier in deployment » | réécrite | | |
| 43 | + | |
| 44 | +## 3. Méthodologie — verdict motivé (§3 du master prompt) | |
| 45 | + | |
| 46 | +- **Spécification justifiée** : oui, le design par axes est la force du papier ; la v1.1 le formalise (éq. 1–9). | |
| 47 | +- **Hypothèses d'estimation explicites** : désormais oui (OLS via ridge α = 10⁻⁶, solution de norme minimale ; | |
| 48 | + encodage one-hot avec référence ; splines : 6 nœuds uniformes, 8 fonctions de base ; RF/HGB : paramètres listés). | |
| 49 | +- **Variables définies** : oui (notation unifiée §4.1 ; bornes et nettoyages §3.2). | |
| 50 | +- **Effets fixes appropriés** : oui, avec la nuance majeure trouvée en révision : les FE fins doivent être emboîtés | |
| 51 | + (C4′ 17.3 % = meilleure spécification linéaire ; C4 seule 21.0 %). Le repli « cellule de référence » pour les | |
| 52 | + cellules non vues est désormais documenté et corrigé par l'emboîtement. | |
| 53 | +- **Dépendance spatiale/temporelle** : traitée par l'inférence groupée (municipalités) et par les holdouts | |
| 54 | + temporel, par parcelle et par bâtiment. Limite : pas de holdout bloqué spatialement (déclaré). | |
| 55 | +- **Erreurs-types** : absentes en v1.0 → bootstrap par grappes + DM groupé (éq. 13). Le DM sur la perte APE teste la | |
| 56 | + MAPE, pas la MdAPE ; le bootstrap couvre la médiane — distinction explicitée. | |
| 57 | +- **Construction de l'échantillon transparente** : oui (cascade 745 119 → 514 212, reproduite à l'identique). | |
| 58 | +- **Robustesse** : cinq analyses ajoutées (§7) ; toutes exécutées, aucune revendiquée sans sortie. | |
| 59 | +- **Interprétation** : « 18 % improvement » = (17.6−14.4)/17.6 ; « ten points » = 28.8 − 19.2 ; profils en log-points. | |
| 60 | +- **Significativité statistique vs économique** : distinguée systématiquement (ex. C3 vs C2, D2 vs A6, D4 vs A6 | |
| 61 | + forward : non significatifs ; D4 vs A6 random : −3.2 [−4.7, −1.9]). | |
| 62 | +- **Langage causal** : aucun ; le papier est prédictif/descriptif. | |
| 63 | + | |
| 64 | +## 4. Résultats — incohérences trouvées et état des réanalyses | |
| 65 | + | |
| 66 | +- Toutes les incohérences texte/table de la v1.0 sont listées au CHANGELOG §1.1, 1.5, 1.8, 1.9, 1.11. | |
| 67 | +- Nouveaux résultats (tous issus de `results/reproduced/*.csv`) : | |
| 68 | + - Scoreboard uniforme (Tables 2–3) ; IC et tests (Tables 4–5) ; décomposition (Tables 8–9, Fig. 6) ; | |
| 69 | + retransformation (Table 12) ; splits groupés (Table 13) ; vu/non vu (Table 14) ; FE emboîtés (Table 15) ; | |
| 70 | + sensibilité HGB (Table 16) ; Box–Cox fin (texte §7.4). | |
| 71 | + - Sensibilité HGB : quatre configurations alternatives (shallow, deep, régularisée, early stopping sur les 10 % derniers mois) donnent 14.3–14.7 % (random) et 21.2–21.6 % (forward) contre 14.4 / 21.2 pour le baseline ; le classement et la décomposition ne dépendent pas des hyper-paramètres fixés a priori. | |
| 72 | +- Vérifications de cohérence finales : abstract ↔ §1 ↔ §5 ↔ §8–9 ↔ README ↔ tables (chiffres relus un à un après | |
| 73 | + régénération des tables ; voir CHANGELOG §1.1 pour la liste ancien → nouveau). | |
| 74 | + | |
| 75 | +## 5. Littérature | |
| 76 | + | |
| 77 | +33 entrées en v1.0 → 58 en v1.1 ; 100 % vérifiées sur Crossref avec DOI (sauf LightGBM, actes NIPS non indexés, | |
| 78 | +référence standard). 2 entrées retirées (non citées, défectueuses), 5 corrigées, 27 ajoutées et intégrées à | |
| 79 | +l'argument (fonctionnelle : Kuminoff et al. 2010, Bishop et al. 2020, Khoshnoud et al. 2023 ; spatial : Pace et al. | |
| 80 | +1998, Clapp 2003, Fik et al. 2003, Bourassa et al. 2003/2007, Goodman–Thibodeau 2003, Pace–Hayunga 2020, | |
| 81 | +Dubé–Legros 2013 ; indices : Hill et al. 2009, Hill–Scholz 2018, Diewert et al. 2015, Diewert–Shimizu 2016 ; | |
| 82 | +ML/AVM : Zurada et al. 2011, Antipov–Pokryshevskaya 2012, McCluskey et al. 2013, Schulz et al. 2014, Pérez-Rave et al. | |
| 83 | +2019, Lorenz et al. 2023, Thibodeau 2003 ; validation : Diebold–Mariano 1995, Giacomini–White 2006, | |
| 84 | +Bergmeir–Benítez 2012, Roberts et al. 2017 ; retransformation : Manning–Mullahy 2001). | |
| 85 | + | |
| 86 | +## 6. Limites restantes | |
| 87 | + | |
| 88 | +1. Un seul régime temporel testé (reprise 2025-26, +15 % en 18 mois) ; l'ordre des coûts de staleness entre | |
| 89 | + familles pourrait différer dans un marché plat ou baissier. | |
| 90 | +2. Le re-nivellement utilise le ratio médian du test (oracle) : borne supérieure de ce qu'un indice publié | |
| 91 | + permettrait. | |
| 92 | +3. Pas de modèle linéaire à surface spatiale continue (krigeage, splines bivariées) ni d'effets hiérarchiques | |
| 93 | + proprement estimés ; l'emboîtement C4′ est un pooling partiel implicite (norme minimale), pas un modèle | |
| 94 | + hiérarchique explicite. | |
| 95 | +4. Les holdouts groupés neutralisent reventes et bâtiments partagés, pas le voisinage immédiat. | |
| 96 | +5. Attributs du rôle seulement (pas de qualité intérieure, texte d'annonce, photos) ; l'avantage ML mesuré est celui | |
| 97 | + de la flexibilité spatiale sur cet ensemble d'attributs. | |
| 98 | +6. La règle « lots > P99.5 → 0 » héritée du pipeline est documentée mais discutable. | |
| 99 | +7. Longueur : 40 pages avec 16 tables et 10 figures ; pour la soumission, les Tables 4–5, 12–16 iraient en annexe. | |
| 100 | + | |
| 101 | +## 7. Analyses recommandées non exécutées | |
| 102 | + | |
| 103 | +Voir CHANGELOG §4 : surface spatiale linéaire (krigeage / splines lat-lng / hiérarchique bayésien) ; holdout bloqué | |
| 104 | +spatialement ; re-nivellement par indice externe ; fenêtres temporelles alternatives (rolling origin) ; variante | |
| 105 | +« médiane calibrée » ; LightGBM/XGBoost natifs et réseaux de neurones ; coût de retraining mensuel (course | |
| 106 | +glissante) — cette dernière est la plus importante pour la revue cible. | |
| 107 | + | |
| 108 | +## 8. Maturité pour publication | |
| 109 | + | |
| 110 | +- **Verdict** : manuscrit désormais défendable en revue (JHE) sous réserve d'une coupe (annexes) et, idéalement, d'une | |
| 111 | + course glissante (retraining mensuel) qui transformerait la §5.5 en recommandation opérationnelle chiffrée. | |
| 112 | +- **Risques de rejet** : (i) « un seul marché, une seule période » — mitigé par l'échelle provinciale et la | |
| 113 | + décomposition ; (ii) absence d'un modèle linéaire spatialement sophistiqué comme concurrent (un rapporteur | |
| 114 | + spatial le demandera) ; (iii) la correction d'un résultat phare publié en v1.0 sur SSRN doit être assumée | |
| 115 | + explicitement (fait dans l'abstract et §7.1) — c'est un point d'intégrité, pas une faiblesse. | |
| 116 | +- **Points forts pour la revue** : taille et qualité des données, design expérimental, inférence complète, | |
| 117 | + décomposition niveau/compétence inédite dans la littérature AVM, résultat « hierarchy, not coarseness » sur les FE | |
| 118 | + spatiaux. | |
modified
figures/fig_forms.png
+0 −0
Binary file not shown.
modified
figures/fig_generalization.png
+0 −0
Binary file not shown.
added
figures/fig_horizon.png
+0 −0
Binary file not shown.
modified
figures/fig_learning.png
+0 −0
Binary file not shown.
modified
figures/fig_methods.png
+0 −0
Binary file not shown.
modified
figures/fig_segments.png
+0 −0
Binary file not shown.
modified
figures/fig_space.png
+0 −0
Binary file not shown.
modified
figures/fig_time.png
+0 −0
Binary file not shown.
modified
paper/main.pdf
+0 −0
Binary file not shown.
modified
paper/main.tex
+41 −30
@@ -48,6 +48,11 @@ | ||
| 48 | 48 | \usepackage{enumitem} |
| 49 | 49 | \setlist{nosep,leftmargin=*} |
| 50 | 50 | |
| 51 | +% Float placement: let text and floats share pages | |
| 52 | +\setcounter{topnumber}{3}\setcounter{bottomnumber}{2}\setcounter{totalnumber}{4} | |
| 53 | +\renewcommand{\topfraction}{0.9}\renewcommand{\bottomfraction}{0.6} | |
| 54 | +\renewcommand{\textfraction}{0.08}\renewcommand{\floatpagefraction}{0.5} | |
| 55 | + | |
| 51 | 56 | % ============================================================================ |
| 52 | 57 | % METADATA |
| 53 | 58 | % ============================================================================ |
@@ -55,9 +60,9 @@ | ||
| 55 | 60 | \newcommand{\WPtitle}{Half a Million Prices, Twenty Models} |
| 56 | 61 | \newcommand{\WPsubtitle}{A Systematic Assessment of Hedonic Specifications and |
| 57 | 62 | Estimation Methods for the Quebec Housing Market, 2021--2026} |
| 58 | −\newcommand{\WPdate}{August 2026} | |
| 59 | −\newcommand{\WPversion}{1.0} | |
| 60 | −\newcommand{\WPkeywords}{hedonic pricing, functional form, house price indexes, machine learning, automated valuation, out-of-sample prediction} | |
| 63 | +\newcommand{\WPdate}{October 2026} | |
| 64 | +\newcommand{\WPversion}{1.1} | |
| 65 | +\newcommand{\WPkeywords}{hedonic price function, functional form, spatial fixed effects, machine learning, automated valuation model, forward-in-time holdout, house price index} | |
| 61 | 66 | \newcommand{\WPjel}{R31, C21, C52, C53} |
| 62 | 67 | |
| 63 | 68 | \newcommand{\WPauthor}{Simon-Pierre Boucher} |
@@ -68,34 +73,39 @@ | ||
| 68 | 73 | 283, boulevard Alexandre-Tach\'e\\ Gatineau, Qu\'ebec, Canada J9A 1L8} |
| 69 | 74 | |
| 70 | 75 | \newcommand{\WPabstract}{% |
| 71 | −Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still rests | |
| 72 | −on a series of specification choices --- functional form, temporal controls, | |
| 73 | −spatial controls, estimation method --- that are usually made by convention | |
| 74 | −and rarely stress-tested jointly. Using 514{,}212 Quebec residential sales | |
| 76 | +Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still | |
| 77 | +rests on specification choices---functional form, temporal controls, | |
| 78 | +spatial controls, estimation method---that are made by convention and | |
| 79 | +rarely stress-tested jointly. Using 514{,}212 Quebec residential sales | |
| 75 | 80 | (2021--2026) whose structural descriptors come from the provincial |
| 76 | 81 | assessment roll, we run a controlled horse race of twenty hedonic models |
| 77 | −organised along those four axes, scoring every model on identical holdouts | |
| 78 | −and on the same price-level scoreboard (log models are Duan-retransformed, | |
| 79 | −Box--Cox inverted). Four lessons emerge. (1)~Functional form is a | |
| 80 | −second-order choice: from linear to splines, the median absolute error | |
| 81 | −spans 16.5--18.6\%, and the data-driven Box--Cox exponent | |
| 82 | −($\hat\lambda=0.25$) buys nothing out of sample. (2)~Controls are | |
| 83 | −first-order: dropping spatial fixed effects costs ten error points --- | |
| 84 | −location granularity matters far more than curvature, but only up to the | |
| 85 | −point where cells thin out (a $\sim$1~km grid \emph{overfits}). | |
| 86 | −(3)~Under random validation, estimation method matters most: gradient | |
| 87 | −boosting reaches 14.4\% --- a 13\% improvement over the best linear model | |
| 88 | −--- with random forests close behind, and coordinates alone reproduce | |
| 89 | −what thousands of dummies buy. (4)~But the ranking is an artifact of the | |
| 90 | −split: under a forward-in-time holdout (train $<$~2025, test 2025--26) | |
| 91 | −every model degrades and the machine-learning advantage \emph{reverses} | |
| 92 | −--- the spline hedonic posts a lower median error than gradient boosting | |
| 93 | −and an identical log-scale $R^2$ --- a stark warning against evaluating | |
| 94 | −valuation models with random cross-validation alone. Constant-quality | |
| 95 | −monthly indices implied by the competing forms agree to within a few | |
| 96 | −index points, and the boosted model's implicit age and floor-area | |
| 97 | −profiles track the quadratic OLS closely --- interpretation and | |
| 98 | −prediction disagree less than the random-validation gap suggests.} | |
| 82 | +organised along those four axes, scoring every model on identical | |
| 83 | +holdouts and on the same price-level scoreboard with a single | |
| 84 | +retransformation rule, with a municipality-clustered bootstrap interval | |
| 85 | +and a Diebold--Mariano-type test on every comparison. Four lessons | |
| 86 | +emerge. (1)~Functional form is second-order: from linear to splines the | |
| 87 | +median absolute error spans 17.6--19.2\%, and the data-driven Box--Cox | |
| 88 | +exponent gains half a point. (2)~Controls are first-order: dropping | |
| 89 | +spatial fixed effects costs ten error points; a $\sim$1~km grid used | |
| 90 | +alone overfits, but nested within municipalities it becomes the most | |
| 91 | +accurate linear specification---the bias--variance trade-off in | |
| 92 | +absorbing location is resolved by hierarchy, not coarseness. (3)~Under | |
| 93 | +random validation the estimation method dominates: gradient boosting | |
| 94 | +reaches 14.4\%, an 18\% improvement over the best linear model | |
| 95 | +(95\% interval 1.9--4.7 points) that survives holdouts grouped by parcel | |
| 96 | +and by building and comes almost entirely from a flexible surface over | |
| 97 | +space. (4)~Under a forward-in-time holdout (train $<$~2025, test | |
| 98 | +2025--26) every model degrades and the gap shrinks to half a point and | |
| 99 | +to insignificance. Decomposing the error shows that relative pricing | |
| 100 | +skill transfers forward intact (re-levelled, boosting leads by 2.9 | |
| 101 | +points) while every frozen model under-prices the typical future sale | |
| 102 | +by 10--20\%, the ensembles slightly more. Random cross-validation hides | |
| 103 | +this staleness cost; it does not, as a previous version of this paper | |
| 104 | +claimed on the basis of an asymmetric retransformation, reverse the | |
| 105 | +ranking. Constant-quality indices agree across functional forms to | |
| 106 | +within 2.5 points but a boosted imputation index rises nine points less, | |
| 107 | +and the boosted model's implicit age and area profiles share the shape | |
| 108 | +of the quadratic OLS gradients with a milder depreciation path.} | |
| 99 | 109 | |
| 100 | 110 | \hypersetup{pdftitle={Half a Million Prices, Twenty Models}, pdfauthor={Simon-Pierre Boucher}} |
| 101 | 111 | |
@@ -113,10 +123,11 @@ prediction disagree less than the random-validation gap suggests.} | ||
| 113 | 123 | \input{sections/methodology} |
| 114 | 124 | \input{sections/results} |
| 115 | 125 | \input{sections/extensions} |
| 126 | +\input{sections/robustness} | |
| 116 | 127 | \input{sections/discussion} |
| 117 | 128 | \input{sections/conclusion} |
| 118 | 129 | |
| 119 | −\newpage | |
| 130 | +\clearpage | |
| 120 | 131 | \bibliography{references} |
| 121 | 132 | |
| 122 | 133 | \end{document} |
modified
paper/main_anon.pdf
+0 −0
Binary file not shown.
modified
paper/main_anon.tex
+41 −30
@@ -48,6 +48,11 @@ | ||
| 48 | 48 | \usepackage{enumitem} |
| 49 | 49 | \setlist{nosep,leftmargin=*} |
| 50 | 50 | |
| 51 | +% Float placement: let text and floats share pages | |
| 52 | +\setcounter{topnumber}{3}\setcounter{bottomnumber}{2}\setcounter{totalnumber}{4} | |
| 53 | +\renewcommand{\topfraction}{0.9}\renewcommand{\bottomfraction}{0.6} | |
| 54 | +\renewcommand{\textfraction}{0.08}\renewcommand{\floatpagefraction}{0.5} | |
| 55 | + | |
| 51 | 56 | % ============================================================================ |
| 52 | 57 | % METADATA |
| 53 | 58 | % ============================================================================ |
@@ -55,9 +60,9 @@ | ||
| 55 | 60 | \newcommand{\WPtitle}{Half a Million Prices, Twenty Models} |
| 56 | 61 | \newcommand{\WPsubtitle}{A Systematic Assessment of Hedonic Specifications and |
| 57 | 62 | Estimation Methods for the Quebec Housing Market, 2021--2026} |
| 58 | −\newcommand{\WPdate}{August 2026} | |
| 59 | −\newcommand{\WPversion}{1.0} | |
| 60 | −\newcommand{\WPkeywords}{hedonic pricing, functional form, house price indexes, machine learning, automated valuation, out-of-sample prediction} | |
| 63 | +\newcommand{\WPdate}{October 2026} | |
| 64 | +\newcommand{\WPversion}{1.1} | |
| 65 | +\newcommand{\WPkeywords}{hedonic price function, functional form, spatial fixed effects, machine learning, automated valuation model, forward-in-time holdout, house price index} | |
| 61 | 66 | \newcommand{\WPjel}{R31, C21, C52, C53} |
| 62 | 67 | |
| 63 | 68 | \newcommand{\WPauthor}{Simon-Pierre Boucher} |
@@ -68,34 +73,39 @@ | ||
| 68 | 73 | 283, boulevard Alexandre-Tach\'e\\ Gatineau, Qu\'ebec, Canada J9A 1L8} |
| 69 | 74 | |
| 70 | 75 | \newcommand{\WPabstract}{% |
| 71 | −Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still rests | |
| 72 | −on a series of specification choices --- functional form, temporal controls, | |
| 73 | −spatial controls, estimation method --- that are usually made by convention | |
| 74 | −and rarely stress-tested jointly. Using 514{,}212 Quebec residential sales | |
| 76 | +Fifty years after \mbox{Rosen (1974)}, applied hedonic practice still | |
| 77 | +rests on specification choices---functional form, temporal controls, | |
| 78 | +spatial controls, estimation method---that are made by convention and | |
| 79 | +rarely stress-tested jointly. Using 514{,}212 Quebec residential sales | |
| 75 | 80 | (2021--2026) whose structural descriptors come from the provincial |
| 76 | 81 | assessment roll, we run a controlled horse race of twenty hedonic models |
| 77 | −organised along those four axes, scoring every model on identical holdouts | |
| 78 | −and on the same price-level scoreboard (log models are Duan-retransformed, | |
| 79 | −Box--Cox inverted). Four lessons emerge. (1)~Functional form is a | |
| 80 | −second-order choice: from linear to splines, the median absolute error | |
| 81 | −spans 16.5--18.6\%, and the data-driven Box--Cox exponent | |
| 82 | −($\hat\lambda=0.25$) buys nothing out of sample. (2)~Controls are | |
| 83 | −first-order: dropping spatial fixed effects costs ten error points --- | |
| 84 | −location granularity matters far more than curvature, but only up to the | |
| 85 | −point where cells thin out (a $\sim$1~km grid \emph{overfits}). | |
| 86 | −(3)~Under random validation, estimation method matters most: gradient | |
| 87 | −boosting reaches 14.4\% --- a 13\% improvement over the best linear model | |
| 88 | −--- with random forests close behind, and coordinates alone reproduce | |
| 89 | −what thousands of dummies buy. (4)~But the ranking is an artifact of the | |
| 90 | −split: under a forward-in-time holdout (train $<$~2025, test 2025--26) | |
| 91 | −every model degrades and the machine-learning advantage \emph{reverses} | |
| 92 | −--- the spline hedonic posts a lower median error than gradient boosting | |
| 93 | −and an identical log-scale $R^2$ --- a stark warning against evaluating | |
| 94 | −valuation models with random cross-validation alone. Constant-quality | |
| 95 | −monthly indices implied by the competing forms agree to within a few | |
| 96 | −index points, and the boosted model's implicit age and floor-area | |
| 97 | −profiles track the quadratic OLS closely --- interpretation and | |
| 98 | −prediction disagree less than the random-validation gap suggests.} | |
| 82 | +organised along those four axes, scoring every model on identical | |
| 83 | +holdouts and on the same price-level scoreboard with a single | |
| 84 | +retransformation rule, with a municipality-clustered bootstrap interval | |
| 85 | +and a Diebold--Mariano-type test on every comparison. Four lessons | |
| 86 | +emerge. (1)~Functional form is second-order: from linear to splines the | |
| 87 | +median absolute error spans 17.6--19.2\%, and the data-driven Box--Cox | |
| 88 | +exponent gains half a point. (2)~Controls are first-order: dropping | |
| 89 | +spatial fixed effects costs ten error points; a $\sim$1~km grid used | |
| 90 | +alone overfits, but nested within municipalities it becomes the most | |
| 91 | +accurate linear specification---the bias--variance trade-off in | |
| 92 | +absorbing location is resolved by hierarchy, not coarseness. (3)~Under | |
| 93 | +random validation the estimation method dominates: gradient boosting | |
| 94 | +reaches 14.4\%, an 18\% improvement over the best linear model | |
| 95 | +(95\% interval 1.9--4.7 points) that survives holdouts grouped by parcel | |
| 96 | +and by building and comes almost entirely from a flexible surface over | |
| 97 | +space. (4)~Under a forward-in-time holdout (train $<$~2025, test | |
| 98 | +2025--26) every model degrades and the gap shrinks to half a point and | |
| 99 | +to insignificance. Decomposing the error shows that relative pricing | |
| 100 | +skill transfers forward intact (re-levelled, boosting leads by 2.9 | |
| 101 | +points) while every frozen model under-prices the typical future sale | |
| 102 | +by 10--20\%, the ensembles slightly more. Random cross-validation hides | |
| 103 | +this staleness cost; it does not, as a previous version of this paper | |
| 104 | +claimed on the basis of an asymmetric retransformation, reverse the | |
| 105 | +ranking. Constant-quality indices agree across functional forms to | |
| 106 | +within 2.5 points but a boosted imputation index rises nine points less, | |
| 107 | +and the boosted model's implicit age and area profiles share the shape | |
| 108 | +of the quadratic OLS gradients with a milder depreciation path.} | |
| 99 | 109 | |
| 100 | 110 | \hypersetup{pdftitle={Half a Million Prices, Twenty Models}, pdfauthor={}} |
| 101 | 111 | |
@@ -113,10 +123,11 @@ prediction disagree less than the random-validation gap suggests.} | ||
| 113 | 123 | \input{sections/methodology} |
| 114 | 124 | \input{sections/results} |
| 115 | 125 | \input{sections/extensions} |
| 126 | +\input{sections/robustness} | |
| 116 | 127 | \input{sections/discussion} |
| 117 | 128 | \input{sections/conclusion} |
| 118 | 129 | |
| 119 | −\newpage | |
| 130 | +\clearpage | |
| 120 | 131 | \bibliography{references} |
| 121 | 132 | |
| 122 | 133 | \end{document} |
modified
paper/references.bib
+371 −60
@@ -7,7 +7,8 @@ | ||
| 7 | 7 | year = {1974}, |
| 8 | 8 | volume = {82}, |
| 9 | 9 | number = {1}, |
| 10 | − pages = {34--55} | |
| 10 | + pages = {34--55}, | |
| 11 | + doi = {10.1086/260169} | |
| 11 | 12 | } |
| 12 | 13 | |
| 13 | 14 | @article{halvorsen1981choice, |
@@ -17,7 +18,8 @@ | ||
| 17 | 18 | year = {1981}, |
| 18 | 19 | volume = {10}, |
| 19 | 20 | number = {1}, |
| 20 | − pages = {37--49} | |
| 21 | + pages = {37--49}, | |
| 22 | + doi = {10.1016/0094-1190(81)90021-8} | |
| 21 | 23 | } |
| 22 | 24 | |
| 23 | 25 | @article{cassel1985cost, |
@@ -27,7 +29,8 @@ | ||
| 27 | 29 | year = {1985}, |
| 28 | 30 | volume = {18}, |
| 29 | 31 | number = {2}, |
| 30 | − pages = {135--142} | |
| 32 | + pages = {135--142}, | |
| 33 | + doi = {10.1016/0094-1190(85)90012-9} | |
| 31 | 34 | } |
| 32 | 35 | |
| 33 | 36 | @article{cropper1988choice, |
@@ -37,7 +40,8 @@ | ||
| 37 | 40 | year = {1988}, |
| 38 | 41 | volume = {70}, |
| 39 | 42 | number = {4}, |
| 40 | − pages = {668--675} | |
| 43 | + pages = {668--675}, | |
| 44 | + doi = {10.2307/1935831} | |
| 41 | 45 | } |
| 42 | 46 | |
| 43 | 47 | @article{box1964analysis, |
@@ -47,7 +51,8 @@ | ||
| 47 | 51 | year = {1964}, |
| 48 | 52 | volume = {26}, |
| 49 | 53 | number = {2}, |
| 50 | − pages = {211--252} | |
| 54 | + pages = {211--243}, | |
| 55 | + doi = {10.1111/j.2517-6161.1964.tb00553.x} | |
| 51 | 56 | } |
| 52 | 57 | |
| 53 | 58 | @incollection{malpezzi2003hedonic, |
@@ -58,7 +63,8 @@ | ||
| 58 | 63 | publisher = {Blackwell}, |
| 59 | 64 | address = {Oxford}, |
| 60 | 65 | year = {2003}, |
| 61 | − pages = {67--89} | |
| 66 | + pages = {67--89}, | |
| 67 | + doi = {10.1002/9780470690680.ch5} | |
| 62 | 68 | } |
| 63 | 69 | |
| 64 | 70 | @article{sirmans2005composition, |
@@ -68,7 +74,8 @@ | ||
| 68 | 74 | year = {2005}, |
| 69 | 75 | volume = {13}, |
| 70 | 76 | number = {1}, |
| 71 | − pages = {1--44} | |
| 77 | + pages = {1--44}, | |
| 78 | + doi = {10.1080/10835547.2005.12090154} | |
| 72 | 79 | } |
| 73 | 80 | |
| 74 | 81 | @article{duan1983smearing, |
@@ -78,7 +85,8 @@ | ||
| 78 | 85 | year = {1983}, |
| 79 | 86 | volume = {78}, |
| 80 | 87 | number = {383}, |
| 81 | − pages = {605--610} | |
| 88 | + pages = {605--610}, | |
| 89 | + doi = {10.1080/01621459.1983.10478017} | |
| 82 | 90 | } |
| 83 | 91 | |
| 84 | 92 | @article{goodman1998housing, |
@@ -88,7 +96,8 @@ | ||
| 88 | 96 | year = {1998}, |
| 89 | 97 | volume = {7}, |
| 90 | 98 | number = {2}, |
| 91 | − pages = {121--143} | |
| 99 | + pages = {121--143}, | |
| 100 | + doi = {10.1006/jhec.1998.0229} | |
| 92 | 101 | } |
| 93 | 102 | |
| 94 | 103 | @article{dubin1998spatial, |
@@ -98,7 +107,8 @@ | ||
| 98 | 107 | year = {1998}, |
| 99 | 108 | volume = {7}, |
| 100 | 109 | number = {4}, |
| 101 | − pages = {304--327} | |
| 110 | + pages = {304--327}, | |
| 111 | + doi = {10.1006/jhec.1998.0236} | |
| 102 | 112 | } |
| 103 | 113 | |
| 104 | 114 | @book{anselin1988spatial, |
@@ -106,7 +116,8 @@ | ||
| 106 | 116 | title = {Spatial Econometrics: Methods and Models}, |
| 107 | 117 | publisher = {Kluwer Academic}, |
| 108 | 118 | address = {Dordrecht}, |
| 109 | − year = {1988} | |
| 119 | + year = {1988}, | |
| 120 | + doi = {10.1007/978-94-015-7799-1} | |
| 110 | 121 | } |
| 111 | 122 | |
| 112 | 123 | @article{hill2013hedonic, |
@@ -116,25 +127,19 @@ | ||
| 116 | 127 | year = {2013}, |
| 117 | 128 | volume = {27}, |
| 118 | 129 | number = {5}, |
| 119 | − pages = {879--914} | |
| 130 | + pages = {879--914}, | |
| 131 | + doi = {10.1111/j.1467-6419.2012.00731.x} | |
| 120 | 132 | } |
| 121 | 133 | |
| 122 | −@article{silver2018house, | |
| 123 | − author = {Silver, Mick}, | |
| 124 | − title = {How to Measure Hedonic Property Price Indexes Better}, | |
| 125 | − journal = {EURONA — Eurostat Review on National Accounts and Macroeconomic Indicators}, | |
| 126 | − year = {2018}, | |
| 127 | − volume = {1}, | |
| 128 | − pages = {35--66} | |
| 129 | −} | |
| 130 | − | |
| 131 | −@article{diewert2004hedonic, | |
| 132 | − author = {Diewert, W. Erwin}, | |
| 133 | − title = {Hedonic Regressions: A Review of Some Unresolved Issues}, | |
| 134 | − journal = {Mimeo, University of British Columbia}, | |
| 135 | − year = {2004}, | |
| 136 | − volume = {}, | |
| 137 | − pages = {} | |
| 134 | +@techreport{silver2018house, | |
| 135 | + author = {Silver, Mick}, | |
| 136 | + title = {How to Better Measure Hedonic Residential Property Price Indexes}, | |
| 137 | + institution = {International Monetary Fund}, | |
| 138 | + type = {IMF Working Paper}, | |
| 139 | + number = {16/213}, | |
| 140 | + address = {Washington, DC}, | |
| 141 | + year = {2016}, | |
| 142 | + doi = {10.5089/9781475552249.001} | |
| 138 | 143 | } |
| 139 | 144 | |
| 140 | 145 | @article{mullainathan2017machine, |
@@ -144,7 +149,8 @@ | ||
| 144 | 149 | year = {2017}, |
| 145 | 150 | volume = {31}, |
| 146 | 151 | number = {2}, |
| 147 | − pages = {87--106} | |
| 152 | + pages = {87--106}, | |
| 153 | + doi = {10.1257/jep.31.2.87} | |
| 148 | 154 | } |
| 149 | 155 | |
| 150 | 156 | @article{athey2019machine, |
@@ -153,7 +159,8 @@ | ||
| 153 | 159 | journal = {Annual Review of Economics}, |
| 154 | 160 | year = {2019}, |
| 155 | 161 | volume = {11}, |
| 156 | − pages = {685--725} | |
| 162 | + pages = {685--725}, | |
| 163 | + doi = {10.1146/annurev-economics-080217-053433} | |
| 157 | 164 | } |
| 158 | 165 | |
| 159 | 166 | @article{breiman2001random, |
@@ -163,7 +170,8 @@ | ||
| 163 | 170 | year = {2001}, |
| 164 | 171 | volume = {45}, |
| 165 | 172 | number = {1}, |
| 166 | − pages = {5--32} | |
| 173 | + pages = {5--32}, | |
| 174 | + doi = {10.1023/A:1010933404324} | |
| 167 | 175 | } |
| 168 | 176 | |
| 169 | 177 | @article{friedman2001greedy, |
@@ -173,16 +181,18 @@ | ||
| 173 | 181 | year = {2001}, |
| 174 | 182 | volume = {29}, |
| 175 | 183 | number = {5}, |
| 176 | − pages = {1189--1232} | |
| 184 | + pages = {1189--1232}, | |
| 185 | + doi = {10.1214/aos/1013203451} | |
| 177 | 186 | } |
| 178 | 187 | |
| 179 | −@article{ke2017lightgbm, | |
| 180 | − author = {Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan}, | |
| 181 | − title = {{LightGBM}: A Highly Efficient Gradient Boosting Decision Tree}, | |
| 182 | − journal = {Advances in Neural Information Processing Systems}, | |
| 183 | − year = {2017}, | |
| 184 | − volume = {30}, | |
| 185 | − pages = {3146--3154} | |
| 188 | +@inproceedings{ke2017lightgbm, | |
| 189 | + author = {Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan}, | |
| 190 | + title = {{LightGBM}: A Highly Efficient Gradient Boosting Decision Tree}, | |
| 191 | + booktitle = {Advances in Neural Information Processing Systems 30 (NIPS 2017)}, | |
| 192 | + editor = {Guyon, I. and Von Luxburg, U. and Bengio, S. and Wallach, H. and Fergus, R. and Vishwanathan, S. and Garnett, R.}, | |
| 193 | + publisher = {Curran Associates}, | |
| 194 | + year = {2017}, | |
| 195 | + pages = {3146--3154} | |
| 186 | 196 | } |
| 187 | 197 | |
| 188 | 198 | @article{mayer2019estimation, |
@@ -192,7 +202,8 @@ | ||
| 192 | 202 | year = {2019}, |
| 193 | 203 | volume = {12}, |
| 194 | 204 | number = {1}, |
| 195 | − pages = {134--150} | |
| 205 | + pages = {134--150}, | |
| 206 | + doi = {10.1108/JERER-08-2018-0035} | |
| 196 | 207 | } |
| 197 | 208 | |
| 198 | 209 | @article{bogin2020house, |
@@ -201,8 +212,9 @@ | ||
| 201 | 212 | journal = {Journal of Real Estate Finance and Economics}, |
| 202 | 213 | year = {2020}, |
| 203 | 214 | volume = {60}, |
| 204 | − number = {1}, | |
| 205 | − pages = {40--52} | |
| 215 | + number = {1--2}, | |
| 216 | + pages = {40--52}, | |
| 217 | + doi = {10.1007/s11146-019-09712-0} | |
| 206 | 218 | } |
| 207 | 219 | |
| 208 | 220 | @article{steurer2021metrics, |
@@ -212,7 +224,8 @@ | ||
| 212 | 224 | year = {2021}, |
| 213 | 225 | volume = {38}, |
| 214 | 226 | number = {2}, |
| 215 | − pages = {99--129} | |
| 227 | + pages = {99--129}, | |
| 228 | + doi = {10.1080/09599916.2020.1858937} | |
| 216 | 229 | } |
| 217 | 230 | |
| 218 | 231 | @article{pace1997spatial, |
@@ -222,7 +235,8 @@ | ||
| 222 | 235 | year = {1997}, |
| 223 | 236 | volume = {14}, |
| 224 | 237 | number = {3}, |
| 225 | − pages = {333--340} | |
| 238 | + pages = {333--340}, | |
| 239 | + doi = {10.1023/A:1007762613901} | |
| 226 | 240 | } |
| 227 | 241 | |
| 228 | 242 | @article{case2004modeling, |
@@ -232,7 +246,8 @@ | ||
| 232 | 246 | year = {2004}, |
| 233 | 247 | volume = {29}, |
| 234 | 248 | number = {2}, |
| 235 | − pages = {167--191} | |
| 249 | + pages = {167--191}, | |
| 250 | + doi = {10.1023/B:REAL.0000035309.60607.53} | |
| 236 | 251 | } |
| 237 | 252 | |
| 238 | 253 | @article{bourassa2010predicting, |
@@ -242,7 +257,8 @@ | ||
| 242 | 257 | year = {2010}, |
| 243 | 258 | volume = {32}, |
| 244 | 259 | number = {2}, |
| 245 | − pages = {139--159} | |
| 260 | + pages = {139--159}, | |
| 261 | + doi = {10.1080/10835547.2010.12091276} | |
| 246 | 262 | } |
| 247 | 263 | |
| 248 | 264 | @article{desrosiers2000hedonic, |
@@ -252,7 +268,8 @@ | ||
| 252 | 268 | year = {2000}, |
| 253 | 269 | volume = {18}, |
| 254 | 270 | number = {3}, |
| 255 | − pages = {291--315} | |
| 271 | + pages = {291--315}, | |
| 272 | + doi = {10.1108/14635780010338245} | |
| 256 | 273 | } |
| 257 | 274 | |
| 258 | 275 | @article{gencay1996nonlinear, |
@@ -262,7 +279,8 @@ | ||
| 262 | 279 | year = {1996}, |
| 263 | 280 | volume = {52}, |
| 264 | 281 | number = {2}, |
| 265 | − pages = {129--135} | |
| 282 | + pages = {129--135}, | |
| 283 | + doi = {10.1016/S0165-1765(96)00851-8} | |
| 266 | 284 | } |
| 267 | 285 | |
| 268 | 286 | @article{kok2017big, |
@@ -272,7 +290,8 @@ | ||
| 272 | 290 | year = {2017}, |
| 273 | 291 | volume = {43}, |
| 274 | 292 | number = {6}, |
| 275 | − pages = {202--211} | |
| 293 | + pages = {202--211}, | |
| 294 | + doi = {10.3905/jpm.2017.43.6.202} | |
| 276 | 295 | } |
| 277 | 296 | |
| 278 | 297 | @article{hong2020machine, |
@@ -282,16 +301,8 @@ | ||
| 282 | 301 | year = {2020}, |
| 283 | 302 | volume = {24}, |
| 284 | 303 | number = {3}, |
| 285 | − pages = {140--152} | |
| 286 | −} | |
| 287 | − | |
| 288 | −@article{knight2003listing, | |
| 289 | − author = {Knight, John R.}, | |
| 290 | − title = {Hedonic Modeling of the Home Selling Process}, | |
| 291 | − journal = {Journal of Real Estate Finance and Economics}, | |
| 292 | − year = {2003}, | |
| 293 | − volume = {}, | |
| 294 | − pages = {} | |
| 304 | + pages = {140--152}, | |
| 305 | + doi = {10.3846/ijspm.2020.11544} | |
| 295 | 306 | } |
| 296 | 307 | |
| 297 | 308 | @book{iaao2013standard, |
@@ -309,7 +320,8 @@ | ||
| 309 | 320 | year = {2002}, |
| 310 | 321 | volume = {24}, |
| 311 | 322 | number = {1}, |
| 312 | − pages = {1--26} | |
| 323 | + pages = {1--26}, | |
| 324 | + doi = {10.1080/10835547.2002.12091087} | |
| 313 | 325 | } |
| 314 | 326 | |
| 315 | 327 | @techreport{oecd2013handbook, |
@@ -319,3 +331,302 @@ | ||
| 319 | 331 | address = {Luxembourg}, |
| 320 | 332 | year = {2013} |
| 321 | 333 | } |
| 334 | + | |
| 335 | +% --------------------------------------------------------------------------- | |
| 336 | +% References added in the 2026-10-17 revision (all verified on Crossref) | |
| 337 | +% --------------------------------------------------------------------------- | |
| 338 | + | |
| 339 | +@article{hill2018geospatial, | |
| 340 | + author = {Hill, Robert J. and Scholz, Michael}, | |
| 341 | + title = {Can Geospatial Data Improve House Price Indexes? {A} Hedonic Imputation Approach with Splines}, | |
| 342 | + journal = {Review of Income and Wealth}, | |
| 343 | + year = {2018}, | |
| 344 | + volume = {64}, | |
| 345 | + number = {4}, | |
| 346 | + pages = {737--756}, | |
| 347 | + doi = {10.1111/roiw.12303} | |
| 348 | +} | |
| 349 | + | |
| 350 | +@article{pace2020residuals, | |
| 351 | + author = {Pace, R. Kelley and Hayunga, Darren}, | |
| 352 | + title = {Examining the Information Content of Residuals from Hedonic and Spatial Models Using Trees and Forests}, | |
| 353 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 354 | + year = {2020}, | |
| 355 | + volume = {60}, | |
| 356 | + number = {1--2}, | |
| 357 | + pages = {170--180}, | |
| 358 | + doi = {10.1007/s11146-019-09724-w} | |
| 359 | +} | |
| 360 | + | |
| 361 | +@article{lorenz2023interpretable, | |
| 362 | + author = {Lorenz, Felix and Willwersch, Jonas and Cajias, Marcelo and Fuerst, Franz}, | |
| 363 | + title = {Interpretable Machine Learning for Real Estate Market Analysis}, | |
| 364 | + journal = {Real Estate Economics}, | |
| 365 | + year = {2023}, | |
| 366 | + volume = {51}, | |
| 367 | + number = {5}, | |
| 368 | + pages = {1178--1208}, | |
| 369 | + doi = {10.1111/1540-6229.12397} | |
| 370 | +} | |
| 371 | + | |
| 372 | +@article{diewert2016tokyo, | |
| 373 | + author = {Diewert, W. Erwin and Shimizu, Chihiro}, | |
| 374 | + title = {Hedonic Regression Models for {Tokyo} Condominium Sales}, | |
| 375 | + journal = {Regional Science and Urban Economics}, | |
| 376 | + year = {2016}, | |
| 377 | + volume = {60}, | |
| 378 | + pages = {300--315}, | |
| 379 | + doi = {10.1016/j.regsciurbeco.2016.08.002} | |
| 380 | +} | |
| 381 | + | |
| 382 | +@article{diewert2015decomposition, | |
| 383 | + author = {Diewert, W. Erwin and de Haan, Jan and Hendriks, Rens}, | |
| 384 | + title = {Hedonic Regressions and the Decomposition of a House Price Index into Land and Structure Components}, | |
| 385 | + journal = {Econometric Reviews}, | |
| 386 | + year = {2015}, | |
| 387 | + volume = {34}, | |
| 388 | + number = {1--2}, | |
| 389 | + pages = {106--126}, | |
| 390 | + doi = {10.1080/07474938.2014.944791} | |
| 391 | +} | |
| 392 | + | |
| 393 | +@article{diebold1995comparing, | |
| 394 | + author = {Diebold, Francis X. and Mariano, Roberto S.}, | |
| 395 | + title = {Comparing Predictive Accuracy}, | |
| 396 | + journal = {Journal of Business \& Economic Statistics}, | |
| 397 | + year = {1995}, | |
| 398 | + volume = {13}, | |
| 399 | + number = {3}, | |
| 400 | + pages = {253--263}, | |
| 401 | + doi = {10.1080/07350015.1995.10524599} | |
| 402 | +} | |
| 403 | + | |
| 404 | +@article{giacomini2006tests, | |
| 405 | + author = {Giacomini, Raffaella and White, Halbert}, | |
| 406 | + title = {Tests of Conditional Predictive Ability}, | |
| 407 | + journal = {Econometrica}, | |
| 408 | + year = {2006}, | |
| 409 | + volume = {74}, | |
| 410 | + number = {6}, | |
| 411 | + pages = {1545--1578}, | |
| 412 | + doi = {10.1111/j.1468-0262.2006.00718.x} | |
| 413 | +} | |
| 414 | + | |
| 415 | +@article{bergmeir2012crossvalidation, | |
| 416 | + author = {Bergmeir, Christoph and Ben{\'i}tez, Jos{\'e} M.}, | |
| 417 | + title = {On the Use of Cross-Validation for Time Series Predictor Evaluation}, | |
| 418 | + journal = {Information Sciences}, | |
| 419 | + year = {2012}, | |
| 420 | + volume = {191}, | |
| 421 | + pages = {192--213}, | |
| 422 | + doi = {10.1016/j.ins.2011.12.028} | |
| 423 | +} | |
| 424 | + | |
| 425 | +@article{roberts2017crossvalidation, | |
| 426 | + author = {Roberts, David R. and Bahn, Volker and Ciuti, Simone and Boyce, Mark S. and Elith, Jane and Guillera-Arroita, Gurutzeta and Hauenstein, Severin and Lahoz-Monfort, Jos{\'e} J. and Schr{\"o}der, Boris and Thuiller, Wilfried and Warton, David I. and Wintle, Brendan A. and Hartig, Florian and Dormann, Carsten F.}, | |
| 427 | + title = {Cross-Validation Strategies for Data with Temporal, Spatial, Hierarchical, or Phylogenetic Structure}, | |
| 428 | + journal = {Ecography}, | |
| 429 | + year = {2017}, | |
| 430 | + volume = {40}, | |
| 431 | + number = {8}, | |
| 432 | + pages = {913--929}, | |
| 433 | + doi = {10.1111/ecog.02881} | |
| 434 | +} | |
| 435 | + | |
| 436 | +@article{kuminoff2010which, | |
| 437 | + author = {Kuminoff, Nicolai V. and Parmeter, Christopher F. and Pope, Jaren C.}, | |
| 438 | + title = {Which Hedonic Models Can We Trust to Recover the Marginal Willingness to Pay for Environmental Amenities?}, | |
| 439 | + journal = {Journal of Environmental Economics and Management}, | |
| 440 | + year = {2010}, | |
| 441 | + volume = {60}, | |
| 442 | + number = {3}, | |
| 443 | + pages = {145--160}, | |
| 444 | + doi = {10.1016/j.jeem.2010.06.001} | |
| 445 | +} | |
| 446 | + | |
| 447 | +@article{bishop2020best, | |
| 448 | + author = {Bishop, Kelly C. and Kuminoff, Nicolai V. and Banzhaf, H. Spencer and Boyle, Kevin J. and von Gravenitz, Kathrine and Pope, Jaren C. and Smith, V. Kerry and Timmins, Christopher D.}, | |
| 449 | + title = {Best Practices for Using Hedonic Property Value Models to Measure Willingness to Pay for Environmental Quality}, | |
| 450 | + journal = {Review of Environmental Economics and Policy}, | |
| 451 | + year = {2020}, | |
| 452 | + volume = {14}, | |
| 453 | + number = {2}, | |
| 454 | + pages = {260--281}, | |
| 455 | + doi = {10.1093/reep/reaa001} | |
| 456 | +} | |
| 457 | + | |
| 458 | +@article{bourassa2007spatial, | |
| 459 | + author = {Bourassa, Steven C. and Cantoni, Eva and Hoesli, Martin}, | |
| 460 | + title = {Spatial Dependence, Housing Submarkets, and House Price Prediction}, | |
| 461 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 462 | + year = {2007}, | |
| 463 | + volume = {35}, | |
| 464 | + number = {2}, | |
| 465 | + pages = {143--160}, | |
| 466 | + doi = {10.1007/s11146-007-9036-8} | |
| 467 | +} | |
| 468 | + | |
| 469 | +@article{bourassa2003submarkets, | |
| 470 | + author = {Bourassa, Steven C. and Hoesli, Martin and Peng, Vincent S.}, | |
| 471 | + title = {Do Housing Submarkets Really Matter?}, | |
| 472 | + journal = {Journal of Housing Economics}, | |
| 473 | + year = {2003}, | |
| 474 | + volume = {12}, | |
| 475 | + number = {1}, | |
| 476 | + pages = {12--28}, | |
| 477 | + doi = {10.1016/S1051-1377(03)00003-2} | |
| 478 | +} | |
| 479 | + | |
| 480 | +@article{goodman2003segmentation, | |
| 481 | + author = {Goodman, Allen C. and Thibodeau, Thomas G.}, | |
| 482 | + title = {Housing Market Segmentation and Hedonic Prediction Accuracy}, | |
| 483 | + journal = {Journal of Housing Economics}, | |
| 484 | + year = {2003}, | |
| 485 | + volume = {12}, | |
| 486 | + number = {3}, | |
| 487 | + pages = {181--201}, | |
| 488 | + doi = {10.1016/S1051-1377(03)00031-7} | |
| 489 | +} | |
| 490 | + | |
| 491 | +@article{zurada2011comparison, | |
| 492 | + author = {Zurada, Jozef and Levitan, Alan S. and Guan, Jian}, | |
| 493 | + title = {A Comparison of Regression and Artificial Intelligence Methods in a Mass Appraisal Context}, | |
| 494 | + journal = {Journal of Real Estate Research}, | |
| 495 | + year = {2011}, | |
| 496 | + volume = {33}, | |
| 497 | + number = {3}, | |
| 498 | + pages = {349--388}, | |
| 499 | + doi = {10.1080/10835547.2011.12091311} | |
| 500 | +} | |
| 501 | + | |
| 502 | +@article{antipov2012mass, | |
| 503 | + author = {Antipov, Evgeny A. and Pokryshevskaya, Elena B.}, | |
| 504 | + title = {Mass Appraisal of Residential Apartments: An Application of Random Forest for Valuation and a {CART}-Based Approach for Model Diagnostics}, | |
| 505 | + journal = {Expert Systems with Applications}, | |
| 506 | + year = {2012}, | |
| 507 | + volume = {39}, | |
| 508 | + number = {2}, | |
| 509 | + pages = {1772--1778}, | |
| 510 | + doi = {10.1016/j.eswa.2011.08.077} | |
| 511 | +} | |
| 512 | + | |
| 513 | +@article{perezrave2019machine, | |
| 514 | + author = {P{\'e}rez-Rave, Jorge Iv{\'a}n and Correa-Morales, Juan Carlos and Gonz{\'a}lez-Echavarr{\'i}a, Favi{\'a}n}, | |
| 515 | + title = {A Machine Learning Approach to Big Data Regression Analysis of Real Estate Prices for Inferential and Predictive Purposes}, | |
| 516 | + journal = {Journal of Property Research}, | |
| 517 | + year = {2019}, | |
| 518 | + volume = {36}, | |
| 519 | + number = {1}, | |
| 520 | + pages = {59--96}, | |
| 521 | + doi = {10.1080/09599916.2019.1587489} | |
| 522 | +} | |
| 523 | + | |
| 524 | +@article{mccluskey2013prediction, | |
| 525 | + author = {McCluskey, William J. and McCord, Michael and Davis, Peadar T. and Haran, Martin and McIlhatton, David}, | |
| 526 | + title = {Prediction Accuracy in Mass Appraisal: A Comparison of Modern Approaches}, | |
| 527 | + journal = {Journal of Property Research}, | |
| 528 | + year = {2013}, | |
| 529 | + volume = {30}, | |
| 530 | + number = {4}, | |
| 531 | + pages = {239--265}, | |
| 532 | + doi = {10.1080/09599916.2013.781204} | |
| 533 | +} | |
| 534 | + | |
| 535 | +@article{schulz2014automated, | |
| 536 | + author = {Schulz, Rainer and Wersing, Martin and Werwatz, Axel}, | |
| 537 | + title = {Automated Valuation Modelling: A Specification Exercise}, | |
| 538 | + journal = {Journal of Property Research}, | |
| 539 | + year = {2014}, | |
| 540 | + volume = {31}, | |
| 541 | + number = {2}, | |
| 542 | + pages = {131--153}, | |
| 543 | + doi = {10.1080/09599916.2013.846930} | |
| 544 | +} | |
| 545 | + | |
| 546 | +@article{fik2003modeling, | |
| 547 | + author = {Fik, Timothy J. and Ling, David C. and Mulligan, Gordon F.}, | |
| 548 | + title = {Modeling Spatial Variation in Housing Prices: A Variable Interaction Approach}, | |
| 549 | + journal = {Real Estate Economics}, | |
| 550 | + year = {2003}, | |
| 551 | + volume = {31}, | |
| 552 | + number = {4}, | |
| 553 | + pages = {623--646}, | |
| 554 | + doi = {10.1046/j.1080-8620.2003.00079.x} | |
| 555 | +} | |
| 556 | + | |
| 557 | +@article{clapp2003semiparametric, | |
| 558 | + author = {Clapp, John M.}, | |
| 559 | + title = {A Semiparametric Method for Valuing Residential Locations: Application to Automated Valuation}, | |
| 560 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 561 | + year = {2003}, | |
| 562 | + volume = {27}, | |
| 563 | + number = {3}, | |
| 564 | + pages = {303--320}, | |
| 565 | + doi = {10.1023/A:1025838007297} | |
| 566 | +} | |
| 567 | + | |
| 568 | +@article{pace1998spatiotemporal, | |
| 569 | + author = {Pace, R. Kelley and Barry, Ronald and Clapp, John M. and Rodriquez, Mauricio}, | |
| 570 | + title = {Spatiotemporal Autoregressive Models of Neighborhood Effects}, | |
| 571 | + journal = {Journal of Real Estate Finance and Economics}, | |
| 572 | + year = {1998}, | |
| 573 | + volume = {17}, | |
| 574 | + number = {1}, | |
| 575 | + pages = {15--33}, | |
| 576 | + doi = {10.1023/A:1007799028599} | |
| 577 | +} | |
| 578 | + | |
| 579 | +@article{dube2013pooled, | |
| 580 | + author = {Dub{\'e}, Jean and Legros, Di{\`e}go}, | |
| 581 | + title = {Dealing with Spatial Data Pooled over Time in Statistical Models}, | |
| 582 | + journal = {Letters in Spatial and Resource Sciences}, | |
| 583 | + year = {2013}, | |
| 584 | + volume = {6}, | |
| 585 | + number = {1}, | |
| 586 | + pages = {1--18}, | |
| 587 | + doi = {10.1007/s12076-012-0082-3} | |
| 588 | +} | |
| 589 | + | |
| 590 | +@article{hill2009boom, | |
| 591 | + author = {Hill, Robert J. and Melser, Daniel and Syed, Iqbal}, | |
| 592 | + title = {Measuring a Boom and Bust: The {Sydney} Housing Market 2001--2006}, | |
| 593 | + journal = {Journal of Housing Economics}, | |
| 594 | + year = {2009}, | |
| 595 | + volume = {18}, | |
| 596 | + number = {3}, | |
| 597 | + pages = {193--205}, | |
| 598 | + doi = {10.1016/j.jhe.2009.07.010} | |
| 599 | +} | |
| 600 | + | |
| 601 | +@article{manning2001estimating, | |
| 602 | + author = {Manning, Willard G. and Mullahy, John}, | |
| 603 | + title = {Estimating Log Models: To Transform or Not to Transform?}, | |
| 604 | + journal = {Journal of Health Economics}, | |
| 605 | + year = {2001}, | |
| 606 | + volume = {20}, | |
| 607 | + number = {4}, | |
| 608 | + pages = {461--494}, | |
| 609 | + doi = {10.1016/S0167-6296(01)00086-8} | |
| 610 | +} | |
| 611 | + | |
| 612 | +@article{thibodeau2003marking, | |
| 613 | + author = {Thibodeau, Thomas G.}, | |
| 614 | + title = {Marking Single-Family Property Values to Market}, | |
| 615 | + journal = {Real Estate Economics}, | |
| 616 | + year = {2003}, | |
| 617 | + volume = {31}, | |
| 618 | + number = {1}, | |
| 619 | + pages = {1--22}, | |
| 620 | + doi = {10.1111/j.1080-8620.2003.00055.x} | |
| 621 | +} | |
| 622 | + | |
| 623 | +@article{khoshnoud2023evolution, | |
| 624 | + author = {Khoshnoud, Mahsa and Sirmans, G. Stacy and Zietz, Emily N.}, | |
| 625 | + title = {The Evolution of Hedonic Pricing Models}, | |
| 626 | + journal = {Journal of Real Estate Literature}, | |
| 627 | + year = {2023}, | |
| 628 | + volume = {31}, | |
| 629 | + number = {1}, | |
| 630 | + pages = {1--47}, | |
| 631 | + doi = {10.1080/09277544.2023.2201020} | |
| 632 | +} | |
modified
paper/sections/conclusion.tex
+34 −24
@@ -5,28 +5,38 @@ | ||
| 5 | 5 | |
| 6 | 6 | We ran the hedonic specification debate as a controlled experiment: |
| 7 | 7 | twenty models, four design axes, one attribute set, one scoreboard, two |
| 8 | −holdouts, half a million sales. The verdict inverts the profession's | |
| 9 | −implicit priorities. The choices that consume the most methodological | |
| 10 | −attention --- functional form, transformation parameters --- move | |
| 11 | −out-of-sample accuracy by at most two points of median error, while the | |
| 12 | −choices often made by default --- whether and how finely to absorb space | |
| 13 | −and time --- move it by three to ten. Machine-learning estimators | |
| 14 | −dominate the standard scoreboard, but the standard scoreboard asks the | |
| 15 | −wrong question: when the test set lies in the future, as it always does | |
| 16 | −in deployment, the boosted ensemble's advantage reverses against a | |
| 17 | −spline hedonic with good controls, because most of what it learned was a | |
| 18 | −spatially fine price configuration that does not transfer across the | |
| 19 | −market cycle. | |
| 8 | +holdouts, half a million sales, and a bootstrap interval on every | |
| 9 | +comparison. The verdict inverts the profession's implicit priorities. | |
| 10 | +The choices that consume the most methodological attention---functional | |
| 11 | +form, transformation parameters---move out-of-sample accuracy by at most | |
| 12 | +two points of median error, while the choices often made by | |
| 13 | +default---whether and how to absorb space and time---move it by three to | |
| 14 | +ten, and the way fine spatial effects are implemented matters more than | |
| 15 | +their resolution. Machine-learning estimators dominate the standard | |
| 16 | +scoreboard by three points, a gain that is real, grows with the sample | |
| 17 | +and comes almost entirely from a flexible surface over space. When the | |
| 18 | +test set lies in the future, as it always does in deployment, that gain | |
| 19 | +shrinks to half a point and loses significance, not because the learned | |
| 20 | +surface fails to transfer---re-levelled, it prices future sales as well | |
| 21 | +as it prices contemporaneous ones---but because every frozen model | |
| 22 | +carries a stale price level, and a boosted model carries a slightly | |
| 23 | +staler one than a linear model with carry-forward time effects. | |
| 20 | 24 | |
| 21 | −Meanwhile the objects economists actually harvest from hedonic models | |
| 22 | −prove reassuringly stable: constant-quality indices agree across forms | |
| 23 | −to within a few points over a full boom--bust--recovery cycle, and the | |
| 24 | −machine's implicit age and area gradients replicate the quadratic OLS. | |
| 25 | −The practical synthesis is almost embarrassingly simple: a semi-log with | |
| 26 | −splines, quarter effects, and spatial fixed effects at a granularity | |
| 27 | −matched to market thickness is within two points of the frontier in | |
| 28 | −deployment conditions --- transparent, auditable, and fast. Where the | |
| 29 | −extra accuracy of learning methods is worth its retraining cadence --- | |
| 30 | −high-frequency AVMs, collateral monitoring --- our decomposition says to | |
| 31 | −spend the flexibility budget on the spatial surface, and to validate, | |
| 32 | −always, against the future rather than against a shuffle of the past. | |
| 25 | +The objects economists harvest from hedonic models prove stable along | |
| 26 | +the axis that generates the most debate and less so along the axis that | |
| 27 | +generates the least: constant-quality indices agree across functional | |
| 28 | +forms to within 2.5 points over a full boom--bust--recovery cycle but | |
| 29 | +differ by nine points between a time-dummy index and a boosted | |
| 30 | +imputation index, and the machine's implicit age and area gradients | |
| 31 | +share the shape of the quadratic OLS with a milder depreciation path. | |
| 32 | +The practical synthesis is simple. A semi-log with splines, quarter | |
| 33 | +effects and spatial fixed effects nested from municipality down to a | |
| 34 | +fine grid is within three points of the frontier under random | |
| 35 | +validation and indistinguishable from it in deployment---transparent, | |
| 36 | +auditable and fast. Where the extra within-period accuracy of learning | |
| 37 | +methods is worth a retraining cadence---high-frequency automated | |
| 38 | +valuation, collateral monitoring---our decomposition says to spend the | |
| 39 | +flexibility budget on the spatial surface, to supply the price level | |
| 40 | +from an index or a frequent refit rather than from the model, and to | |
| 41 | +validate, always, against the future rather than against a shuffle of | |
| 42 | +the past. | |
modified
paper/sections/data.tex
+43 −26
@@ -6,37 +6,53 @@ | ||
| 6 | 6 | \subsection{Sources} |
| 7 | 7 | |
| 8 | 8 | We use a registry of 745{,}119 Quebec residential-market transactions |
| 9 | −recorded between January 2021 and July 2026 --- sale price, date, address | |
| 10 | −and coordinates --- each matched at the parcel level to the municipal | |
| 9 | +recorded between January 2021 and July 2026---sale price, date, address | |
| 10 | +and coordinates---each matched at the parcel level to the municipal | |
| 11 | 11 | assessment roll in force at the sale date (median match distance |
| 12 | −0.6~metres, validated against the roll's recorded value). The roll | |
| 13 | −contributes the structural descriptors that are notoriously missing from | |
| 14 | −transaction registries: total floor area, lot area, year of construction, | |
| 15 | −number of dwelling units, number of storeys, physical configuration | |
| 16 | −(detached, semi-detached, row, apartment-linked) and the standardized use | |
| 17 | −code. This combination --- market prices with assessor-grade structure | |
| 18 | −for half a million sales across an entire province --- is what makes a | |
| 19 | −controlled specification race feasible: every model sees exactly the same | |
| 20 | −attributes. | |
| 12 | +0.6~metres, validated against the roll's recorded value). The same | |
| 13 | +snapshot underlies \companionpaper{}. The roll contributes the | |
| 14 | +structural descriptors that are typically missing from transaction | |
| 15 | +registries: total floor area, lot area, year of construction, number of | |
| 16 | +dwelling units, number of storeys, physical configuration (detached, | |
| 17 | +semi-detached, row, apartment-linked) and the standardised use code. | |
| 18 | +This combination---market prices with assessor-grade structure for half a | |
| 19 | +million sales across an entire province---is what makes a controlled | |
| 20 | +specification race feasible: every model sees exactly the same | |
| 21 | +attributes, and the attribute set is the one a mass-appraisal office | |
| 22 | +actually holds. | |
| 21 | 23 | |
| 22 | 24 | \subsection{Sample} |
| 23 | 25 | |
| 24 | 26 | We keep residential use codes (dwellings, cottages, mobile homes), require |
| 25 | −a high-confidence roll match (distance $\le 50$~m, score $\ge 150$), a | |
| 26 | −price of at least \$50{,}000, a plausible floor area (20--2{,}000~m$^2$) | |
| 27 | −and a known year of construction, and trim prices and floor areas at the | |
| 28 | −1st and 99th percentiles within each sale year. The estimation sample | |
| 29 | −contains \textbf{514{,}212 sales}: 348{,}253 single-family homes, 75{,}734 | |
| 30 | −condominiums, 74{,}909 plexes (2--5 units), 10{,}349 cottages and 4{,}881 | |
| 31 | −mobile homes, spread over 1{,}115 municipalities, 5{,}255 grid cells of | |
| 32 | −$\sim$5.5~km and 30{,}548 cells of $\sim$1.1~km. | |
| 33 | −Table~\ref{tab:sumstats} reports the descriptives. Prices average | |
| 34 | −\$462{,}000 (median \$399{,}000); the window covers the tail of the | |
| 35 | −post-pandemic boom, the 2022--2023 rate correction and the subsequent | |
| 36 | −recovery --- a demanding environment for any static price model, which we | |
| 37 | −exploit in the forward-in-time evaluation. | |
| 27 | +a high-confidence roll match (distance $\le 50$~m, match score | |
| 28 | +$\ge 150$), a price of at least \$50{,}000, a plausible floor area | |
| 29 | +(20--2{,}000~m$^2$) and a known year of construction, and trim prices and | |
| 30 | +floor areas at the 1st and 99th percentiles within each sale year. | |
| 31 | +Building age is the sale year minus the year of construction, bounded | |
| 32 | +between 0 and 150; storeys are bounded between 1 and 6 and dwelling units | |
| 33 | +between 1 and 12; lot areas above the 99.5th percentile, which are | |
| 34 | +implausible for residential parcels, are set to zero and treated as | |
| 35 | +units without an exclusive lot, together with condominiums. The | |
| 36 | +estimation sample contains \textbf{514{,}212 sales}: 348{,}253 | |
| 37 | +single-family homes, 75{,}734 condominiums, 74{,}909 plexes (two to five | |
| 38 | +units), 10{,}349 cottages, 4{,}881 mobile homes and 86 dwellings of | |
| 39 | +undetermined class, spread over 1{,}115 municipalities, 5{,}255 grid | |
| 40 | +cells of $\sim$5.5~km and 30{,}548 cells of $\sim$1.1~km. Sales are | |
| 41 | +spread across the window (97{,}868 in 2021, 90{,}733 in 2022, 76{,}411 in | |
| 42 | +2023, 90{,}812 in 2024, 103{,}830 in 2025 and 54{,}558 in January--July | |
| 43 | +2026). Table~\ref{tab:sumstats} reports the descriptive statistics. | |
| 44 | +Prices average \$440{,}207 (median \$397{,}375), floor areas 132~m$^2$ | |
| 45 | +and building ages 44 years; the window covers the tail of the | |
| 46 | +post-pandemic boom, the 2022--23 rate correction and the 2024--26 | |
| 47 | +recovery, during which the constant-quality price level rose by roughly | |
| 48 | +70\% (Section~\ref{sec:ext}). This is a demanding environment for any | |
| 49 | +static price model, which the forward-in-time evaluation exploits. | |
| 50 | +One fifth of the sales (105{,}814) concern a parcel that sells more than | |
| 51 | +once in the window and 30\% share their exact coordinates with another | |
| 52 | +sale; Section~\ref{sec:holdouts} explains why these repeat and | |
| 53 | +co-located sales matter for validation. | |
| 38 | 54 | |
| 39 | −\begin{table}[t] | |
| 55 | +\begin{table}[tbp] | |
| 40 | 56 | \centering |
| 41 | 57 | \begin{threeparttable} |
| 42 | 58 | \caption{Summary statistics, estimation sample} |
@@ -47,7 +63,8 @@ exploit in the forward-in-time evaluation. | ||
| 47 | 63 | \item \textit{Notes:} 514{,}212 residential sales, January 2021 -- July |
| 48 | 64 | 2026. Structural descriptors from the assessment roll in force at the |
| 49 | 65 | sale date. Lot area is zero for condominiums and other units without an |
| 50 | −exclusive lot. | |
| 66 | +exclusive lot (and for the 0.5\% of lots above the 99.5th percentile, | |
| 67 | +see text). | |
| 51 | 68 | \end{tablenotes} |
| 52 | 69 | \end{threeparttable} |
| 53 | 70 | \end{table} |
modified
paper/sections/discussion.tex
+94 −61
@@ -5,77 +5,110 @@ | ||
| 5 | 5 | |
| 6 | 6 | \subsection{A hierarchy of specification choices} |
| 7 | 7 | |
| 8 | −Read jointly, the four axes imply a clear hierarchy of returns to | |
| 9 | −specification effort, useful to anyone building a hedonic model: | |
| 8 | +Read jointly, the four axes imply a hierarchy of returns to | |
| 9 | +specification effort. The first and largest return is to location: | |
| 10 | +any spatial absorption beats none by about ten points of median error, a | |
| 11 | +margin that dwarfs every other choice, and the granularity question is | |
| 12 | +one of implementation rather than of resolution. A fine grid used alone | |
| 13 | +overfits because thin cells are estimated from a handful of sales and | |
| 14 | +unseen cells are priced arbitrarily; the same grid nested within | |
| 15 | +municipalities, which pools thin cells toward their municipality, is the | |
| 16 | +most accurate linear specification we estimate. The second return is to | |
| 17 | +time controls: in a moving market a model without time effects mistakes | |
| 18 | +appreciation for attributes, at a cost of three points under random | |
| 19 | +validation and six when the level must be carried forward, while | |
| 20 | +granularity beyond quarters is cosmetic. The third return is to the | |
| 21 | +estimator, and its size depends on the question asked. Under random | |
| 22 | +validation gradient boosting buys three points over the best linear | |
| 23 | +model, a gain that is statistically unambiguous and that grows with the | |
| 24 | +training sample; in deployment the gain on the median error shrinks to | |
| 25 | +half a point and loses significance, because the ensembles carry a | |
| 26 | +slightly staler price level, while their relative pricing skill is | |
| 27 | +intact. The fourth and smallest return is to functional form: splines | |
| 28 | +are a cheap and reliable upgrade over the semi-log, and the Box--Cox | |
| 29 | +machinery is not worth its complexity, exactly as | |
| 30 | +\citet{cropper1988choice} concluded from simulations four decades ago. | |
| 10 | 31 | |
| 11 | −\begin{enumerate} | |
| 12 | −\item \textbf{Get location right} ($\sim$10 points of median error): | |
| 13 | −any spatial absorption beats none by a margin that dwarfs every other | |
| 14 | −choice; the optimal granularity is interior --- around 5~km here --- | |
| 15 | −because finer cells trade bias for variance. | |
| 16 | −\item \textbf{Include some time control} ($\sim$3 points): in a moving | |
| 17 | −market, a model without time effects mistakes appreciation for | |
| 18 | −attributes; granularity beyond quarters is cosmetic. | |
| 19 | −\item \textbf{Choose the estimator for the job} ($\sim$2 points | |
| 20 | −deployed, $\sim$4 under random validation): boosting buys real accuracy | |
| 21 | −in interpolation-heavy uses (filling gaps within a period, mass | |
| 22 | −appraisal with contemporaneous comparables) and much less when the | |
| 23 | −target is the future. | |
| 24 | −\item \textbf{Stop worrying about functional form} ($\lesssim$2 | |
| 25 | −points): splines are a cheap upgrade; the Box--Cox machinery is not | |
| 26 | −worth its complexity, exactly as \citet{cropper1988choice} concluded | |
| 27 | −from simulations four decades ago. | |
| 28 | −\end{enumerate} | |
| 32 | +\subsection{Why the forward-in-time split compresses the ranking} | |
| 29 | 33 | |
| 30 | −\subsection{Why the machines lose their edge out of time} | |
| 31 | − | |
| 32 | −The decomposition rows explain the generalization gap. The boosting | |
| 33 | −model's random-split advantage comes almost entirely from a flexible | |
| 34 | −spatial surface (removing coordinates costs it 8.8 points). That surface | |
| 35 | −is estimated \emph{jointly} with the time path, and trees clamp: beyond | |
| 36 | −the last training month the model prices 2025--26 sales at a frozen | |
| 37 | −level, exactly like the carry-forward linear models, while its | |
| 38 | −fine-grained spatial fit --- calibrated on the 2021--24 price | |
| 39 | −configuration --- partially decays as relative prices shift. The linear | |
| 40 | −models, coarser but more rigid, carry a structure that transfers better. | |
| 41 | −This is not an indictment of machine learning --- retrained monthly, the | |
| 42 | −boosting model would presumably keep its interpolation advantage --- but | |
| 43 | −it prices the \emph{retraining requirement} that random cross-validation | |
| 44 | −hides, and it matches the practitioner evidence that AVM accuracy decays | |
| 45 | −quickly out of sample period \citep{bogin2020house, kok2017big}. | |
| 34 | +The decomposition of Section~\ref{sec:gap} clarifies what the | |
| 35 | +forward-in-time holdout measures. The boosting model's random-split | |
| 36 | +advantage comes almost entirely from a flexible spatial surface | |
| 37 | +(removing the coordinates costs it 8.8 points), and that surface | |
| 38 | +transfers to 2025--26 sales: re-levelled, the model prices them with a | |
| 39 | +13.9\% median error, against 16.8\% for the spline hedonic. What does | |
| 40 | +not transfer is the aggregate level. Trees are piecewise constant in the | |
| 41 | +month counter and the terminal leaf in time averages the last training | |
| 42 | +months rather than the last one, so a frozen boosted model enters the | |
| 43 | +test period with a slightly staler level than a linear model whose last | |
| 44 | +month effect is carried forward; by mid-2026 both are about 25\% below | |
| 45 | +the market and indistinguishable. The lesson is not that machine | |
| 46 | +learning fails out of sample. It is that the forward-in-time holdout | |
| 47 | +conflates two things---the common level error that any static model | |
| 48 | +incurs, and the relative accuracy the estimator contributes---and that | |
| 49 | +the first dominates the second within a few quarters. Retrained monthly, | |
| 50 | +or re-levelled with a published index as practitioners do | |
| 51 | +\citep{thibodeau2003marking, mayer2019estimation}, the boosted model | |
| 52 | +would presumably keep its interpolation advantage; the forward split | |
| 53 | +prices the \emph{retraining or re-levelling requirement} that random | |
| 54 | +cross-validation hides, and it matches the practitioner evidence that | |
| 55 | +automated-valuation accuracy decays quickly outside the sample period | |
| 56 | +\citep{bogin2020house, kok2017big}. It also explains why our first | |
| 57 | +version, which smeared the linear models but not the ensembles, found a | |
| 58 | +reversal: in a rising market an upward retransformation factor applied | |
| 59 | +to one family only offsets part of that family's staleness. | |
| 46 | 60 | |
| 47 | 61 | \subsection{Implications} |
| 48 | 62 | |
| 49 | −\paragraph{For research.} Hedonic coefficients and indices are robust | |
| 50 | −objects: form and even estimator perturb them little | |
| 51 | −(Section~\ref{sec:ext}). Researchers using hedonics to \emph{measure} | |
| 52 | −--- indices, implicit prices, capitalization effects --- can keep simple | |
| 53 | −forms with good controls and report validation under a temporal split | |
| 54 | −when prediction claims are made. | |
| 63 | +\paragraph{For research.} Hedonic indices are robust to functional form | |
| 64 | +and implicit-price gradients are robust to the estimator in shape, if | |
| 65 | +not in slope (Section~\ref{sec:ext}). Researchers using hedonic models to | |
| 66 | +\emph{measure}---indices, implicit prices, capitalization effects---can | |
| 67 | +keep simple forms with good, preferably hierarchical, spatial controls. | |
| 68 | +When prediction claims are made, they should report a forward-in-time | |
| 69 | +split alongside random cross-validation, and they should report | |
| 70 | +intervals: several of the differences that organise the applied | |
| 71 | +debate---a 5.5~km grid against municipality effects, month against | |
| 72 | +quarter effects, boosting against splines in deployment---are not | |
| 73 | +distinguishable from zero on half a million sales. | |
| 55 | 74 | |
| 56 | 75 | \paragraph{For mass appraisal.} Quebec's assessors face exactly this |
| 57 | −design problem every three years. Our results suggest the largest | |
| 58 | −accuracy gains lie not in exotic estimators but in spatial resolution | |
| 59 | −chosen by market thickness --- and that any model, linear or boosted, | |
| 60 | −frozen at a reference date degrades by 3--7 points of median error | |
| 61 | −within two years. This quantifies the staleness mechanism behind the | |
| 62 | −assessment inequities documented in \companionpaper{}: the | |
| 63 | −valuation technology's out-of-time decay is of the same magnitude as the | |
| 64 | −inequities measured there. | |
| 76 | +design problem every three years. Our results suggest that the largest | |
| 77 | +accuracy gains lie in spatial resolution implemented hierarchically, that | |
| 78 | +flexible learners add a further three points within the period they are | |
| 79 | +estimated on, and that any model, linear or boosted, frozen at a | |
| 80 | +reference date under-prices the typical sale by 10--20\% within eighteen | |
| 81 | +months in a market like 2025--26. This quantifies the staleness | |
| 82 | +mechanism behind the assessment inequities documented in | |
| 83 | +\companionpaper{}: the valuation technology's out-of-time decay is of the | |
| 84 | +same order as the inequities measured there, and it is a level problem | |
| 85 | +that an index adjustment addresses before any change of estimator. | |
| 65 | 86 | |
| 66 | −\paragraph{For the AVM literature.} Every comparison should report a | |
| 67 | −forward-in-time split alongside random cross-validation | |
| 68 | −\citep{steurer2021metrics}. On our data, the choice of split changes not | |
| 69 | −just magnitudes but the \emph{winner}. | |
| 87 | +\paragraph{For the automated-valuation literature.} Every comparison | |
| 88 | +should report a forward-in-time split, a leakage-free grouped split when | |
| 89 | +properties sell repeatedly or share coordinates, and a single | |
| 90 | +retransformation rule applied to all models | |
| 91 | +\citep{steurer2021metrics}. On our data the first changes the size and | |
| 92 | +significance of the headline gap, the second changes nothing, and the | |
| 93 | +third, applied asymmetrically, is enough to manufacture a reversal. | |
| 70 | 94 | |
| 71 | 95 | \subsection{Limitations} |
| 72 | 96 | |
| 73 | 97 | Our attribute set, though assessor-grade, omits listing-level quality |
| 74 | 98 | (renovations, finishes, views); richer features would likely raise the |
| 75 | −machines' interpolation advantage. Hyper-parameters were deliberately | |
| 76 | −conventional --- a tuned boosted model would gain somewhat, as would a | |
| 77 | −spatially smarter linear model (e.g.\ kriging residuals, | |
| 78 | −\citealp{case2004modeling}). The forward split covers one specific | |
| 79 | −regime change (the 2025--26 recovery); other windows would shift | |
| 80 | −magnitudes. And Quebec's institutional homogeneity aids transferability | |
| 81 | −of results across municipalities but may limit it across countries. | |
| 99 | +learners' within-period advantage. Hyper-parameters were deliberately | |
| 100 | +conventional, and Section~\ref{sec:robust} shows that materially | |
| 101 | +different boosting configurations, including one tuned on a time-ordered | |
| 102 | +validation set, leave the ranking unchanged; a spatially smarter linear | |
| 103 | +model (kriged residuals, \citealp{case2004modeling}; semiparametric | |
| 104 | +location surfaces, \citealp{clapp2003semiparametric}) would narrow the | |
| 105 | +gap from the other side and is not estimated here. The forward split | |
| 106 | +covers one specific regime (the 2025--26 recovery), in which prices rose | |
| 107 | +by about 15\% in eighteen months; a flat or falling period would produce | |
| 108 | +a smaller level error and possibly a different ordering of staleness | |
| 109 | +across model families. The grouped holdouts remove leakage through | |
| 110 | +repeat sales and shared coordinates but not through neighbouring parcels | |
| 111 | +in the same street, which a spatially blocked holdout would address at | |
| 112 | +the cost of changing the question to spatial extrapolation. Finally, | |
| 113 | +Quebec's institutional homogeneity aids transferability of results | |
| 114 | +across municipalities but may limit it across countries. | |
modified
paper/sections/extensions.tex
+90 −61
@@ -3,26 +3,41 @@ | ||
| 3 | 3 | \section{Beyond the scoreboard: what the models imply} |
| 4 | 4 | \label{sec:ext} |
| 5 | 5 | |
| 6 | −Accuracy is not the only output economists take from a hedonic model. This | |
| 7 | −section asks whether the specification choices that move the scoreboard | |
| 8 | −also move the two objects hedonic models are most often built to deliver: | |
| 9 | −constant-quality price indices and implicit attribute prices. | |
| 6 | +Accuracy is not the only output economists take from a hedonic model. | |
| 7 | +This section asks whether the specification choices that move the | |
| 8 | +scoreboard also move the two objects hedonic models are most often built | |
| 9 | +to deliver, constant-quality price indices and implicit attribute prices, | |
| 10 | +and then characterises where and when the learners' advantage | |
| 11 | +materialises. | |
| 10 | 12 | |
| 11 | −\subsection{Price indices are form-robust} | |
| 13 | +\subsection{Price indices: form-robust, estimator-sensitive} | |
| 12 | 14 | |
| 13 | −Figure~\ref{fig:index} plots the constant-quality monthly index implied by | |
| 14 | −the month fixed effects of four linear forms --- semi-log, log-log, | |
| 15 | −quadratic and spline --- together with a gradient-boosting index obtained | |
| 16 | −by repricing a fixed 20{,}000-sale reference portfolio at each month | |
| 17 | −(an imputation \emph{\`a la} \citealp{hill2013hedonic}). All five track | |
| 18 | −the same cycle --- the 2021--22 boom, the 2022--23 correction, the | |
| 19 | −2024--26 recovery --- and the four linear forms agree to within a few | |
| 20 | −index points throughout. The choice that dominates the accuracy | |
| 21 | −scoreboard barely perturbs the index: reassuring news for statistical | |
| 22 | −agencies, and consistent with the RPPI handbook's pragmatism about form | |
| 23 | −\citep{oecd2013handbook, silver2018house}. | |
| 15 | +Figure~\ref{fig:index} plots the constant-quality monthly index implied | |
| 16 | +by the month fixed effects of four linear forms---semi-log, log-log, | |
| 17 | +quadratic and spline---together with a gradient-boosting index obtained | |
| 18 | +by repricing a fixed 20{,}000-sale reference portfolio at each month, a | |
| 19 | +hedonic imputation index in the sense of \citet{hill2013hedonic} and | |
| 20 | +\citet{hill2018geospatial}. All five track the same cycle: the 2021--22 | |
| 21 | +boom, the 2022--23 correction and the 2024--26 recovery, over which the | |
| 22 | +constant-quality level rises by about 70\%. The four linear forms agree | |
| 23 | +to within 2.5 index points at every month, so the choice that generates | |
| 24 | +the most methodological discussion barely perturbs the index, consistent | |
| 25 | +with the RPPI handbook's pragmatism about form \citep{oecd2013handbook, | |
| 26 | +silver2018house}. The estimator is a different matter: the boosting | |
| 27 | +index rises less, ending in July 2026 at 160.5 against 168.6--171.0 for | |
| 28 | +the linear forms, a gap that opens gradually from 2024 and reaches nine | |
| 29 | +to eleven points. Two mechanisms contribute. The imputation index | |
| 30 | +reprices a fixed portfolio, so its weights differ from the time-dummy | |
| 31 | +index's implicit weights when appreciation is heterogeneous across | |
| 32 | +segments; and the boosting model's month counter enters through tree | |
| 33 | +splits that interact with location and attributes, so its ``time | |
| 34 | +effect'' is an average of locally varying paths rather than a single | |
| 35 | +common shift. Statistical agencies choosing between a time-dummy and an | |
| 36 | +imputation index should expect this order of disagreement when the | |
| 37 | +imputation model is a flexible learner; the agreement across linear | |
| 38 | +forms is reassuring, the agreement across estimators is not automatic. | |
| 24 | 39 | |
| 25 | −\begin{figure}[t] | |
| 40 | +\begin{figure}[tbp] | |
| 26 | 41 | \centering |
| 27 | 42 | \includegraphics[width=\textwidth]{fig_index.png} |
| 28 | 43 | \caption{Constant-quality monthly price indices implied by competing |
@@ -32,28 +47,36 @@ fixed effects. Gradient boosting: mean repriced value of a fixed | ||
| 32 | 47 | \label{fig:index} |
| 33 | 48 | \end{figure} |
| 34 | 49 | |
| 35 | −\subsection{Implicit prices: the machine agrees with the quadratic} | |
| 50 | +\subsection{Implicit prices: the machine broadly agrees with the quadratic} | |
| 36 | 51 | |
| 37 | −Figure~\ref{fig:profiles} compares the ln-price profiles in building age | |
| 38 | −and floor area implied by the quadratic OLS coefficients with the partial-% | |
| 39 | −dependence profiles of the boosting model \citep{friedman2001greedy}. | |
| 40 | −The two agree over the bulk of the data: steep initial depreciation that | |
| 41 | −flattens with age, and strongly concave returns to floor space. The | |
| 42 | −visible divergences are where the quadratic \emph{cannot} bend --- the | |
| 43 | −machine sees a milder gradient for very old stock (a vintage effect) and | |
| 44 | −a small new-construction premium spike --- but through the ages and areas | |
| 45 | −where 90\% of sales live, the curves sit within a tenth of a log point. The boosting model's advantage evidently comes from | |
| 46 | −higher-order interactions and spatial flexibility, \emph{not} from a | |
| 47 | −different view of the main attribute gradients --- so the interpretable | |
| 48 | −quadratic form remains a faithful summary of the price surface even where | |
| 49 | −it loses the prediction race \citep{mullainathan2017machine, | |
| 50 | −athey2019machine}. | |
| 52 | +Figure~\ref{fig:profiles} compares the log-price profiles in building age | |
| 53 | +and floor area implied by the quadratic OLS coefficients with the | |
| 54 | +partial-dependence profiles of the boosting model | |
| 55 | +\citep{friedman2001greedy, lorenz2023interpretable}, each normalised to | |
| 56 | +its leftmost point. The two agree on the shape: steep initial | |
| 57 | +depreciation that flattens with age, and strongly concave returns to | |
| 58 | +floor space. On floor area the curves sit within 0.05 log points of each | |
| 59 | +other over the whole 50--400~m$^2$ range. On age the boosting profile is | |
| 60 | +uniformly flatter: it shows a small new-construction premium (the | |
| 61 | +profile peaks at five years, $+0.06$ log points above a new building, | |
| 62 | +where the quadratic can only decline), and from twenty years onward its | |
| 63 | +cumulative depreciation is 0.10--0.13 log points smaller than the | |
| 64 | +quadratic's ($-0.20$ against $-0.31$ at forty years, $-0.28$ against | |
| 65 | +$-0.41$ at sixty), with a milder gradient for very old stock, a vintage | |
| 66 | +effect the quadratic cannot bend to accommodate. These are differences | |
| 67 | +of degree in a shared shape, not of sign or ordering. The boosting | |
| 68 | +model's accuracy advantage therefore does not come from a different view | |
| 69 | +of the main attribute gradients; it comes from higher-order interactions | |
| 70 | +and, above all, from the spatial surface (Section~\ref{sec:results}). | |
| 71 | +The interpretable quadratic remains a faithful, if slightly steeper, | |
| 72 | +summary of the age--price relation even where it loses the prediction | |
| 73 | +race \citep{mullainathan2017machine, athey2019machine}. | |
| 51 | 74 | |
| 52 | −\begin{figure}[t] | |
| 75 | +\begin{figure}[tbp] | |
| 53 | 76 | \centering |
| 54 | 77 | \includegraphics[width=\textwidth]{fig_profiles.png} |
| 55 | −\caption{Implicit ln-price profiles: quadratic OLS coefficients versus | |
| 56 | −gradient-boosting partial dependence, each normalized to its leftmost | |
| 78 | +\caption{Implicit log-price profiles: quadratic OLS coefficients versus | |
| 79 | +gradient-boosting partial dependence, each normalised to its leftmost | |
| 57 | 80 | point. Panel A: building age. Panel B: floor area.} |
| 58 | 81 | \label{fig:profiles} |
| 59 | 82 | \end{figure} |
@@ -61,18 +84,19 @@ point. Panel A: building age. Panel B: floor area.} | ||
| 61 | 84 | \subsection{Learning curves: data beat flexibility only at scale} |
| 62 | 85 | |
| 63 | 86 | Figure~\ref{fig:learning} and Table~\ref{tab:learning} trace accuracy |
| 64 | −against training size from 10{,}000 to 400{,}000 sales. The OLS quadratic | |
| 65 | −is essentially flat from the very first step (18.4\% at $n=10{,}000$, | |
| 66 | −18.1\% at $n=400{,}000$) --- its bias floor binds almost immediately --- | |
| 67 | −while the boosting model improves monotonically through the full sample | |
| 68 | −(17.6\% $\to$ 14.5\%): the machine's advantage grows from under one | |
| 69 | −error point at $n = 10{,}000$ to 3.6 points at | |
| 70 | −$n = 400{,}000$. Flexible methods | |
| 71 | −are not a free lunch in small markets: below a few tens of thousands of | |
| 72 | −training sales, the machine's edge is modest, one reason rural and thin-% | |
| 73 | −market AVMs underperform \citep{bogin2020house}. | |
| 87 | +against training size from 10{,}000 to 400{,}000 sales on a fixed random | |
| 88 | +holdout. The OLS quadratic is flat from the very first step (19.1\% at | |
| 89 | +$n = 10{,}000$, 19.2\% at $n = 400{,}000$): its bias floor binds | |
| 90 | +immediately, and more data cannot lower it. The boosting model improves | |
| 91 | +monotonically through the full sample (17.6\% $\to$ 14.5\%), so its | |
| 92 | +advantage grows from 1.6 points at $n = 10{,}000$ to 4.7 points at | |
| 93 | +$n = 400{,}000$. Flexible methods are not a free lunch in small markets: | |
| 94 | +below a few tens of thousands of training sales, the learner's edge is | |
| 95 | +modest, one reason thin-market automated valuation underperforms | |
| 96 | +\citep{bogin2020house}, and a reason the provincial scale of the present | |
| 97 | +data matters for the size of the gap we measure. | |
| 74 | 98 | |
| 75 | −\begin{figure}[t] | |
| 99 | +\begin{figure}[tbp] | |
| 76 | 100 | \centering |
| 77 | 101 | \includegraphics[width=0.72\textwidth]{fig_learning.png} |
| 78 | 102 | \caption{Learning curves: median absolute error against training-set |
@@ -80,7 +104,7 @@ size (random holdout fixed), OLS quadratic versus gradient boosting.} | ||
| 80 | 104 | \label{fig:learning} |
| 81 | 105 | \end{figure} |
| 82 | 106 | |
| 83 | −\begin{table}[t] | |
| 107 | +\begin{table}[tbp] | |
| 84 | 108 | \centering |
| 85 | 109 | \begin{threeparttable} |
| 86 | 110 | \caption{Learning curves} |
@@ -98,23 +122,28 @@ seeded subsamples of the indicated size. | ||
| 98 | 122 | |
| 99 | 123 | Table~\ref{tab:segments} and Figure~\ref{fig:segments} decompose the |
| 100 | 124 | random-holdout MdAPE by property class and municipality size, and the |
| 101 | −pattern identifies the machine's edge with striking precision: it is | |
| 102 | −spatial resolution. The gap is enormous for condominiums (9.1\% versus | |
| 103 | −20.4\%) and in the largest cities (11.9\% versus 17.8\%) --- segments | |
| 104 | −where structure is homogeneous but \emph{micro}-location varies hugely | |
| 105 | −within a municipality, exactly the variation that municipality dummies | |
| 106 | −cannot see and raw coordinates can. Once location is resolved, a condo | |
| 107 | −is the most predictable object in the market. At the other extreme, | |
| 108 | −cottages defeat both models (32.1\% versus 33.9\%): idiosyncratic, | |
| 109 | −waterfront-driven stock is hard for everyone, and the machine's spatial | |
| 110 | −surface cannot rescue attributes it does not observe. The complement of | |
| 111 | −the assessment-inequity anatomy in \companionpaper{} is | |
| 125 | +pattern locates the learner's edge: it is spatial resolution. The gap is | |
| 126 | +largest for condominiums (9.1\% against 18.2\%) and in the largest | |
| 127 | +cities (11.9\% against 17.7\%), segments where structure is homogeneous | |
| 128 | +but \emph{micro}-location varies greatly within a municipality, exactly | |
| 129 | +the variation that municipality dummies cannot see and raw coordinates | |
| 130 | +can. Once location is resolved, a condominium is the most predictable | |
| 131 | +object in the market. The gap is also large in the thinnest | |
| 132 | +municipalities (18.7\% against 24.8\%), where a municipality effect is | |
| 133 | +estimated from few sales and the learner can borrow strength from | |
| 134 | +neighbouring municipalities through the coordinates. At the other | |
| 135 | +extreme, cottages defeat both models (32.1\% against 34.3\%): | |
| 136 | +idiosyncratic, waterfront-driven stock is hard for everyone, and a | |
| 137 | +spatial surface cannot rescue attributes it does not observe. The | |
| 138 | +complement with the assessment-inequity anatomy in \companionpaper{} is | |
| 112 | 139 | instructive: the segments where assessors are most regressive |
| 113 | 140 | (heterogeneous, land-heavy stock) are also those where \emph{no} |
| 114 | 141 | standard technology, linear or boosted, prices accurately from roll |
| 115 | −attributes alone. | |
| 142 | +attributes alone. Section~\ref{sec:robust} examines how much of the | |
| 143 | +condominium advantage reflects genuine micro-location rather than recall | |
| 144 | +of sales in the same building. | |
| 116 | 145 | |
| 117 | −\begin{figure}[t] | |
| 146 | +\begin{figure}[tbp] | |
| 118 | 147 | \centering |
| 119 | 148 | \includegraphics[width=0.8\textwidth]{fig_segments.png} |
| 120 | 149 | \caption{Median absolute error by market segment (random holdout): |
@@ -122,7 +151,7 @@ OLS quadratic versus gradient boosting.} | ||
| 122 | 151 | \label{fig:segments} |
| 123 | 152 | \end{figure} |
| 124 | 153 | |
| 125 | −\begin{table}[t] | |
| 154 | +\begin{table}[tbp] | |
| 126 | 155 | \centering |
| 127 | 156 | \begin{threeparttable} |
| 128 | 157 | \caption{Accuracy by market segment} |
modified
paper/sections/introduction.tex
+96 −68
@@ -8,87 +8,115 @@ the price or not? Dummies for years or for months? Neighbourhood fixed | ||
| 8 | 8 | effects, and how fine? Least squares or, increasingly, a gradient-boosted |
| 9 | 9 | ensemble? Fifty years after \citet{rosen1974hedonic} established that |
| 10 | 10 | theory places no restriction on the shape of the price surface, these |
| 11 | −choices are still made largely by convention --- the semi-log survives as | |
| 12 | −the default \citep{sirmans2005composition}, the classic form | |
| 13 | −comparisons predate the out-of-sample era \citep{halvorsen1981choice, | |
| 14 | −cropper1988choice}, and the machine-learning comparisons that have | |
| 15 | −multiplied since \citet{mullainathan2017machine} typically change the | |
| 16 | −sample, the features and the validation scheme all at once, making it | |
| 17 | −impossible to say \emph{which} ingredient buys the reported gains. | |
| 11 | +choices are still made largely by convention: the semi-log survives as | |
| 12 | +the default \citep{sirmans2005composition, khoshnoud2023evolution}, the | |
| 13 | +classic functional-form comparisons predate the out-of-sample era | |
| 14 | +\citep{halvorsen1981choice, cropper1988choice}, and the machine-learning | |
| 15 | +comparisons that have multiplied since \citet{mullainathan2017machine} | |
| 16 | +typically change the sample, the features and the validation scheme at | |
| 17 | +once, so that it is impossible to say \emph{which} ingredient buys the | |
| 18 | +reported gains. | |
| 18 | 19 | |
| 19 | 20 | This paper runs the comparison the way one would design an experiment. |
| 20 | 21 | From 514{,}212 Quebec residential sales (2021--2026) matched at the |
| 21 | −parcel level to the provincial assessment roll --- so that every model | |
| 22 | +parcel level to the provincial assessment roll---so that every model | |
| 22 | 23 | sees the same assessor-grade attribute set: floor area, lot area, age, |
| 23 | −storeys, units, property class, physical configuration --- we estimate | |
| 24 | −\textbf{twenty models organised along four axes}, varying one design | |
| 25 | −choice at a time: six functional forms (linear to Box--Cox to splines), | |
| 26 | −a ladder of time controls (none to month fixed effects), a ladder of | |
| 27 | −spatial controls (none to a $\sim$1~km grid), and six estimation methods | |
| 28 | −(OLS to random forests, gradient boosting, and a $k$-nearest-neighbour | |
| 24 | +storeys, units, property class, physical configuration---we estimate | |
| 25 | +twenty models organised along four axes, varying one design choice at a | |
| 26 | +time: six functional forms (linear to Box--Cox to splines), a ladder of | |
| 27 | +time controls (none to month fixed effects), a ladder of spatial | |
| 28 | +controls (none to a $\sim$1~km grid), and six estimation methods (OLS to | |
| 29 | +random forests, gradient boosting and a $k$-nearest-neighbour | |
| 29 | 30 | comparables rule). Every model is scored on the same price-level |
| 30 | −scoreboard --- log models retransformed with Duan smearing, Box--Cox | |
| 31 | −inverted --- and, crucially, under \textbf{two holdouts}: the standard | |
| 32 | −random 80/20 split, and a forward-in-time split (train before 2025, test | |
| 33 | −2025--26) that mimics how a valuation model is actually deployed. | |
| 31 | +scoreboard, with a single retransformation rule for every model whose | |
| 32 | +dependent variable is a transformed price, and under two holdouts: the | |
| 33 | +standard random 80/20 split, and a forward-in-time split (train before | |
| 34 | +2025, test 2025--26) that mimics how a valuation model is actually | |
| 35 | +deployed. Because differences of a few tenths of a point between models | |
| 36 | +evaluated on the same sales can be noise, every comparison comes with a | |
| 37 | +municipality-clustered bootstrap interval and a Diebold--Mariano-type | |
| 38 | +test of equal accuracy. | |
| 34 | 39 | |
| 35 | −Four results organise the paper. | |
| 36 | − | |
| 37 | −\textbf{First, functional form is a second-order choice.} Across linear, | |
| 38 | −semi-log, log-log, profile-likelihood Box--Cox | |
| 39 | −($\hat\lambda = 0.25$), quadratic and spline specifications --- holding | |
| 40 | −controls fixed --- the median absolute error spans 16.5\% to 18.6\%. | |
| 41 | −Splines beat the textbook semi-log by 1.6 points; the data-driven | |
| 42 | −Box--Cox transformation, the great hope of the 1980s debate, buys | |
| 43 | −nothing out of sample, vindicating \citet{cropper1988choice} and | |
| 40 | +Four results organise the paper. \emph{First, functional form is a | |
| 41 | +second-order choice.} Across linear, semi-log, log-log, | |
| 42 | +profile-likelihood Box--Cox ($\hat\lambda = 0.25$), quadratic and spline | |
| 43 | +specifications, holding controls fixed, the median absolute error spans | |
| 44 | +17.6\% to 19.2\% under the random holdout. Splines beat the textbook | |
| 45 | +semi-log by 1.4 points; the data-driven Box--Cox transformation, the | |
| 46 | +great hope of the 1980s debate, gains half a point on the median error | |
| 47 | +and nothing on the mean error, vindicating \citet{cropper1988choice} and | |
| 44 | 48 | \citet{cassel1985cost} on modern data at scale. |
| 45 | 49 | |
| 46 | −\textbf{Second, controls are first-order --- with a twist.} Removing all | |
| 47 | −spatial controls costs ten error points (27.9\% versus 17.5\%), an order | |
| 48 | −of magnitude more than any curvature decision; a $\sim$5.5~km grid beats | |
| 49 | −both municipality effects and, notably, a $\sim$1.1~km grid, whose | |
| 50 | −30{,}548 cells overfit --- the bias--variance trade-off in absorbing | |
| 51 | −location is real and non-monotonic. Time effects show the same shape in | |
| 52 | −miniature: anything beats nothing (three points), granularity beyond | |
| 53 | −quarters buys nothing. | |
| 50 | +\emph{Second, controls are first-order, with a twist.} Removing all | |
| 51 | +spatial controls costs ten error points (28.8\% against 19.2\%), six | |
| 52 | +times the entire functional-form range; a $\sim$5.5~km grid is nominally | |
| 53 | +but not significantly better than municipality effects, and a | |
| 54 | +$\sim$1.1~km grid used on its own overfits (21.0\%), its 30{,}548 cells | |
| 55 | +holding three sales at the median. Nested within municipalities, so that | |
| 56 | +thin and unseen cells inherit their municipality's level, the same fine | |
| 57 | +grid becomes the most accurate linear specification in the paper | |
| 58 | +(17.3\%): the bias--variance trade-off in absorbing location is real, | |
| 59 | +and it is resolved by hierarchy rather than by coarseness. Time effects | |
| 60 | +show the same shape in miniature: anything beats nothing (three points | |
| 61 | +under the random split, six under the forward split), and granularity | |
| 62 | +beyond quarters buys nothing. | |
| 54 | 63 | |
| 55 | −\textbf{Third, under random validation the estimation method dominates.} | |
| 64 | +\emph{Third, under random validation the estimation method dominates.} | |
| 56 | 65 | Gradient boosting reaches a median error of 14.4\% and random forests |
| 57 | −14.6\%, against 16.5\% for the best linear model --- a 13\% improvement | |
| 58 | −--- and stripping the coordinates from the boosting model degrades it to | |
| 59 | −23.2\%, showing that what the machine mainly learns is a flexible price | |
| 60 | −surface over space \citep{bourassa2010predicting}. The na\"ive | |
| 61 | −$k$-NN comparables rule (21.2\%) confirms that neither raw proximity nor | |
| 62 | −raw flexibility suffices: the gains come from their combination. | |
| 66 | +14.6\%, against 17.6\% for the best linear model, an 18\% improvement | |
| 67 | +that is statistically unambiguous ($-3.2$ points, 95\% interval $-4.7$ | |
| 68 | +to $-1.9$). Stripping the coordinates from the boosting model degrades | |
| 69 | +it to 23.2\%, so what the learner mainly buys is a flexible price | |
| 70 | +surface over space \citep{bourassa2010predicting, pace2020residuals}; | |
| 71 | +the na\"ive $k$-NN comparables rule (21.2\%) shows that raw proximity | |
| 72 | +without attribute adjustment does not suffice either. The advantage is | |
| 73 | +not an artefact of repeat sales or of condominium units geocoded to the | |
| 74 | +same building: it survives holdouts grouped by parcel and by coordinate | |
| 75 | +pair essentially unchanged. | |
| 63 | 76 | |
| 64 | −\textbf{Fourth --- our headline --- the ranking is an artifact of the | |
| 65 | −validation scheme.} Under the forward-in-time split every model degrades, | |
| 66 | −but not equally: the machine-learning advantage \emph{reverses} on the | |
| 67 | −median error (gradient boosting 21.2\% versus 19.1\% for the spline | |
| 68 | −hedonic) and equalizes exactly on log-scale $R^2$ (0.52 for both). | |
| 69 | −Random cross-validation lets flexible models interpolate the price level | |
| 70 | −of their own test period; when the test period lies in the future --- the | |
| 71 | −only case that matters for deployment --- that advantage is leakage, not | |
| 72 | −skill. Comparisons of valuation models that report random cross-validation | |
| 73 | −alone, i.e.\ most of the applied ML literature, systematically overstate | |
| 74 | −the practical value of model flexibility \citep{clapp2002predicting, | |
| 75 | −steurer2021metrics}. | |
| 77 | +\emph{Fourth---our headline---the advantage is a within-period | |
| 78 | +statement.} Under the forward-in-time split every model degrades, the | |
| 79 | +ensembles most: gradient boosting loses 6.8 points (to 21.2\%) and the | |
| 80 | +spline hedonic 4.1 (to 21.7\%), so the three-point gap shrinks to half a | |
| 81 | +point and loses statistical significance, while the ensembles keep a | |
| 82 | +modest lead on the mean error and on log-scale fit. Decomposing the | |
| 83 | +forward error shows why. The 2025--26 sales are not harder to price: | |
| 84 | +evaluated on the same sales, a boosting model that has seen their period | |
| 85 | +posts 13.7\%. What a frozen model loses is the \emph{level}: every | |
| 86 | +forward-split model under-prices the typical 2025--26 sale by 10--20\%, | |
| 87 | +the ensembles slightly more than the linear models because their | |
| 88 | +terminal leaf in time averages the last training months. Once each | |
| 89 | +model's level is corrected, the boosting advantage over the spline | |
| 90 | +hedonic is 2.9 points, essentially what the random split measures. | |
| 91 | +Relative pricing skill transfers forward almost intact; the price level | |
| 92 | +does not, and random cross-validation hides a staleness cost of three to | |
| 93 | +seven points of median error within eighteen months that any deployed | |
| 94 | +model, linear or boosted, must pay unless it is re-levelled by an index | |
| 95 | +or retrained. A previous version of this paper reported that the | |
| 96 | +ranking reversed in the forward split; the reversal was an artefact of | |
| 97 | +an asymmetric retransformation and is corrected here. | |
| 76 | 98 | |
| 77 | 99 | The paper then asks what the specification choices do to the objects |
| 78 | −economists actually extract from hedonic models. Constant-quality monthly | |
| 79 | −indices implied by the month effects of four linear forms --- and by | |
| 80 | −repricing a fixed portfolio with the boosting model --- agree to within a | |
| 81 | −few index points across the largest housing cycle in recent Canadian | |
| 82 | −history. And the boosting model's partial-dependence profiles in age and | |
| 83 | −floor area track the quadratic OLS closely: the machine agrees with the | |
| 84 | −economist about the gradients and disagrees mainly about the residual | |
| 85 | −surface. Interpretation and prediction, in other words, conflict far less | |
| 86 | −than the accuracy scoreboard suggests. | |
| 100 | +economists actually extract from hedonic models. Constant-quality | |
| 101 | +monthly indices implied by the month effects of four linear forms agree | |
| 102 | +to within 2.5 index points across the largest housing cycle in recent | |
| 103 | +Canadian history, whereas the imputation index from the boosting model | |
| 104 | +rises nine points less over five years: form does not move the index, | |
| 105 | +the estimator can. The boosting model's partial-dependence profiles in | |
| 106 | +age and floor area share the shape of the quadratic OLS gradients, with | |
| 107 | +a uniformly milder depreciation path, so the machine disagrees with the | |
| 108 | +economist about the residual surface rather than about the main | |
| 109 | +gradients. Interpretation and prediction, in other words, conflict far | |
| 110 | +less than the accuracy scoreboard suggests. | |
| 87 | 111 | |
| 88 | 112 | Section~\ref{sec:lit} situates the exercise in the functional-form, |
| 89 | −spatial-hedonic and ML-valuation literatures. Section~\ref{sec:data} | |
| 90 | −describes the data, Section~\ref{sec:method} the experimental design. | |
| 91 | −Section~\ref{sec:results} reports the horse race, | |
| 92 | −Section~\ref{sec:ext} the index, implicit-price, learning-curve and | |
| 93 | −segment extensions, Section~\ref{sec:disc} discusses implications, and | |
| 113 | +spatial-hedonic and machine-learning valuation literatures. | |
| 114 | +Section~\ref{sec:data} describes the data, Section~\ref{sec:method} the | |
| 115 | +experimental design, the scoreboard and the inference procedures. | |
| 116 | +Section~\ref{sec:results} reports the horse race and decomposes the | |
| 117 | +generalization gap, Section~\ref{sec:ext} the index, implicit-price, | |
| 118 | +learning-curve and segment extensions, Section~\ref{sec:robust} the | |
| 119 | +robustness analyses (retransformation rule, leakage-free holdouts, nested | |
| 120 | +spatial effects, Box--Cox grid, boosting hyper-parameters), | |
| 121 | +Section~\ref{sec:disc} discusses implications, and | |
| 94 | 122 | Section~\ref{sec:concl} concludes. |
modified
paper/sections/literature.tex
+92 −50
@@ -6,62 +6,104 @@ | ||
| 6 | 6 | \subsection{The functional-form debate} |
| 7 | 7 | |
| 8 | 8 | Hedonic theory is silent about functional form. \citet{rosen1974hedonic} |
| 9 | −showed that the price surface is an equilibrium envelope of bids and | |
| 10 | −offers, so nothing structural pins down its curvature --- | |
| 11 | −form is an empirical choice. The classical exchange settled into a | |
| 12 | −three-cornered debate: \citet{halvorsen1981choice} advocated data-driven | |
| 13 | −Box--Cox transformations \citep{box1964analysis}; | |
| 14 | −\citet{cassel1985cost} cautioned that flexible transformations optimize | |
| 15 | −in-sample fit while degrading the implicit prices researchers actually | |
| 16 | −use; and \citet{cropper1988choice}, in an influential simulation study, | |
| 17 | −found that simple forms --- linear and semi-log --- are the most robust to | |
| 18 | −the misspecification and omitted variables that characterize real data. | |
| 19 | −Fifty years on, the semi-log remains the profession's default | |
| 20 | −\citep{malpezzi2003hedonic, sirmans2005composition}, but systematic | |
| 21 | −head-to-head evidence on modern samples remains scarce, and the classic | |
| 22 | −studies predate the out-of-sample validation culture. Two further | |
| 23 | −technical strands feed our design: retransformation bias in log models | |
| 24 | −\citep{duan1983smearing}, which we correct with the smearing factor so | |
| 25 | −that every form competes in levels; and forecast-evaluation methodology | |
| 26 | −for house prices \citep{clapp2002predicting, steurer2021metrics}. | |
| 9 | +showed that the price surface is an equilibrium envelope of bid and | |
| 10 | +offer functions, so nothing structural pins down its curvature: the form | |
| 11 | +is an empirical choice. The classical exchange settled into a | |
| 12 | +three-cornered debate. \citet{halvorsen1981choice} advocated letting the | |
| 13 | +data choose the transformation through the Box--Cox family | |
| 14 | +\citep{box1964analysis}; \citet{cassel1985cost} cautioned that flexible | |
| 15 | +transformations optimise in-sample fit while degrading the implicit | |
| 16 | +prices researchers actually use; and \citet{cropper1988choice}, in a | |
| 17 | +simulation study that remains the standard reference, found that simple | |
| 18 | +forms, linear and semi-log, are the most robust to the misspecification | |
| 19 | +and omitted variables that characterise real data. The debate has been | |
| 20 | +revisited for welfare measurement rather than prediction: | |
| 21 | +\citet{kuminoff2010which} show in a Monte Carlo design calibrated to | |
| 22 | +actual housing data that the choice of control variables (notably | |
| 23 | +spatial fixed effects) matters more for recovering marginal willingness | |
| 24 | +to pay than the choice of functional form, and \citet{bishop2020best} | |
| 25 | +codify the resulting best practice. Fifty years on, the semi-log remains | |
| 26 | +the profession's default \citep{malpezzi2003hedonic, | |
| 27 | +sirmans2005composition, khoshnoud2023evolution}, but systematic | |
| 28 | +head-to-head evidence on large modern samples remains scarce, and the | |
| 29 | +classic studies predate the out-of-sample validation culture. Two | |
| 30 | +technical strands feed our design. Retransformation bias in log models | |
| 31 | +\citep{duan1983smearing, manning2001estimating} determines how | |
| 32 | +predictions in logarithms are converted into price levels, a step that | |
| 33 | +turns out to matter for the comparison between linear models and tree | |
| 34 | +ensembles. Forecast-evaluation methodology, from the equal-accuracy tests | |
| 35 | +of \citet{diebold1995comparing} and \citet{giacomini2006tests} to their | |
| 36 | +application to house prices \citep{clapp2002predicting, | |
| 37 | +steurer2021metrics}, supplies the inferential framework we use to decide | |
| 38 | +whether differences on the scoreboard are real. | |
| 27 | 39 | |
| 28 | 40 | \subsection{Space and time in hedonic models} |
| 29 | 41 | |
| 30 | 42 | A second literature concerns what the covariates cannot capture. House |
| 31 | 43 | prices are spatially autocorrelated at fine scale |
| 32 | −\citep{dubin1998spatial, anselin1988spatial}, and the practical remedies | |
| 33 | −range from submarket segmentation \citep{goodman1998housing} to spatial | |
| 34 | −econometrics and simple location fixed effects; comparative studies | |
| 35 | −generally find that flexibly absorbing location buys more predictive | |
| 36 | −accuracy than any parametric spatial process \citep{case2004modeling, | |
| 37 | −bourassa2010predicting, pace1997spatial}. In the Quebec context, | |
| 44 | +\citep{anselin1988spatial, dubin1998spatial, pace1998spatiotemporal}, and | |
| 45 | +the practical remedies range from submarket segmentation | |
| 46 | +\citep{goodman1998housing, goodman2003segmentation, bourassa2003submarkets} | |
| 47 | +to spatial econometric processes, semiparametric location surfaces | |
| 48 | +\citep{clapp2003semiparametric, fik2003modeling} and simple location | |
| 49 | +fixed effects. Comparative studies generally find that flexibly | |
| 50 | +absorbing location buys more predictive accuracy than any parametric | |
| 51 | +spatial process, and that the gain depends on how submarkets are drawn | |
| 52 | +\citep{pace1997spatial, case2004modeling, bourassa2007spatial, | |
| 53 | +bourassa2010predicting}. \citet{pace2020residuals} show with tree | |
| 54 | +ensembles that the residuals of standard hedonic and spatial models | |
| 55 | +still contain exploitable spatial information, which is precisely the | |
| 56 | +margin our coordinate-based learners target. In the Quebec context, | |
| 38 | 57 | \citet{desrosiers2000hedonic} document the importance of access and |
| 39 | −neighbourhood attributes in Quebec City. On the time dimension, hedonic | |
| 40 | −price-index theory --- the time-dummy versus imputation debate --- | |
| 41 | −is surveyed by \citet{hill2013hedonic} and codified in the RPPI handbook | |
| 42 | −\citep{oecd2013handbook, silver2018house}; a byproduct of our month-FE | |
| 43 | −models is a direct test of how sensitive a constant-quality index is to | |
| 44 | −the underlying form. | |
| 58 | +neighbourhood attributes in Quebec City, and \citet{dube2013pooled} | |
| 59 | +stress that sales pooled over time are not exchangeable: a sale can | |
| 60 | +inform the price of a later neighbour but not of an earlier one, which | |
| 61 | +is the asymmetry a forward-in-time holdout respects and a random split | |
| 62 | +ignores. On the time dimension, the time-dummy versus imputation debate | |
| 63 | +in hedonic price-index theory is surveyed by \citet{hill2013hedonic} and | |
| 64 | +codified in the RPPI handbook \citep{oecd2013handbook, silver2018house}; | |
| 65 | +\citet{hill2009boom} and \citet{hill2018geospatial} show that imputation | |
| 66 | +indices built from flexible, spline-based hedonic models track a boom | |
| 67 | +and bust more faithfully, and \citet{diewert2015decomposition} and | |
| 68 | +\citet{diewert2016tokyo} develop the hedonic decomposition of prices into | |
| 69 | +land and structure components. A by-product of our month-effects models | |
| 70 | +is a direct test of how sensitive a constant-quality index is to the | |
| 71 | +underlying functional form and to the estimator. | |
| 45 | 72 | |
| 46 | 73 | \subsection{Machine learning and automated valuation} |
| 47 | 74 | |
| 48 | −The third strand is the rapid colonization of mass valuation by machine | |
| 49 | −learning. \citet{mullainathan2017machine} frame prediction as ML's | |
| 50 | −comparative advantage, with hedonic pricing as their running example; | |
| 51 | −\citet{athey2019machine} survey the toolkit. Applied comparisons --- | |
| 52 | −random forests \citep{breiman2001random}, gradient boosting | |
| 53 | −\citep{friedman2001greedy, ke2017lightgbm} --- consistently report 20--40\% | |
| 54 | −error reductions over linear hedonics \citep{mayer2019estimation, | |
| 55 | −hong2020machine, kok2017big}, with accuracy degrading in thin rural | |
| 56 | −markets \citep{bogin2020house}; \citet{gencay1996nonlinear} anticipated | |
| 57 | −the pattern with semiparametric estimators. Three gaps motivate our | |
| 58 | −design. First, most comparisons change several ingredients at once | |
| 59 | −(features, sample, validation scheme), so the \emph{sources} of the ML | |
| 60 | −advantage --- curvature? interactions? spatial flexibility? --- are rarely | |
| 61 | −isolated; our axis design holds features constant and varies one choice | |
| 62 | −at a time. Second, almost all published comparisons use random | |
| 63 | −cross-validation, which leaks future price levels into training and | |
| 64 | −flatters any flexible model; we score every model under a | |
| 65 | −forward-in-time split as well. Third, comparisons seldom examine what the | |
| 66 | −flexible models \emph{imply} --- indices, implicit prices --- which is | |
| 67 | −what economists need; our extensions section does exactly that. | |
| 75 | +The third strand is the rapid adoption of machine learning in mass | |
| 76 | +valuation. \citet{mullainathan2017machine} frame prediction as the | |
| 77 | +comparative advantage of machine learning, with hedonic pricing as their | |
| 78 | +running example; \citet{athey2019machine} survey the toolkit. The | |
| 79 | +applied comparisons, from early neural networks and random forests | |
| 80 | +\citep{zurada2011comparison, antipov2012mass, mccluskey2013prediction} to | |
| 81 | +gradient boosting \citep{kok2017big, mayer2019estimation, | |
| 82 | +perezrave2019machine, hong2020machine}, consistently report error | |
| 83 | +reductions of 20--40\% over linear hedonics, with accuracy degrading in | |
| 84 | +thin rural markets \citep{bogin2020house}; \citet{gencay1996nonlinear} | |
| 85 | +anticipated the pattern with semiparametric estimators, and | |
| 86 | +\citet{lorenz2023interpretable} show that the gains come with | |
| 87 | +interpretable partial-dependence profiles that economists can read as | |
| 88 | +implicit prices. \citet{mayer2019estimation} is the closest antecedent | |
| 89 | +to our forward-in-time design: on Swiss data they compare estimation | |
| 90 | +methods together with updating rules (moving windows, weighting) and | |
| 91 | +find that the choice of updating matters as much as the estimator; we | |
| 92 | +isolate the same question by scoring frozen models on a future period. | |
| 93 | +Three gaps motivate our design. First, most comparisons change several | |
| 94 | +ingredients at once (features, sample, validation scheme), so the sources | |
| 95 | +of the machine-learning advantage---curvature, interactions, spatial | |
| 96 | +flexibility---are rarely isolated; our axis design holds the features | |
| 97 | +constant and varies one choice at a time | |
| 98 | +\citep[compare the specification exercise of][]{schulz2014automated}. | |
| 99 | +Second, almost all published comparisons use random cross-validation, | |
| 100 | +which leaks the future price level into training, and, when properties | |
| 101 | +sell repeatedly or share a building, lets a coordinate-based learner | |
| 102 | +recall a property's own earlier price; the forecasting and ecology | |
| 103 | +literatures have long documented how such temporal and spatial | |
| 104 | +dependence inflates cross-validated accuracy | |
| 105 | +\citep{bergmeir2012crossvalidation, roberts2017crossvalidation}. We score | |
| 106 | +every model under a forward-in-time split and under parcel- and | |
| 107 | +building-grouped splits as well. Third, comparisons seldom examine what | |
| 108 | +the flexible models \emph{imply}---indices, implicit prices---which is | |
| 109 | +what economists need; Section~\ref{sec:ext} does exactly that. | |
modified
paper/sections/methodology.tex
+264 −70
@@ -3,80 +3,274 @@ | ||
| 3 | 3 | \section{Methodology: the design of the horse race} |
| 4 | 4 | \label{sec:method} |
| 5 | 5 | |
| 6 | −The comparison is organised so that each design choice varies while | |
| 7 | −everything else is held fixed. All models price the same sales from the | |
| 8 | −same attribute set; only the functional mapping changes. | |
| 6 | +The comparison is organised so that one design choice varies at a time | |
| 7 | +while everything else is held fixed. All twenty models price the same | |
| 8 | +sales from the same attribute vector; only the mapping from attributes to | |
| 9 | +price changes. This section defines the models, the retransformation rule | |
| 10 | +that puts them on a common scoreboard, the two holdouts and the inference | |
| 11 | +procedures. | |
| 12 | + | |
| 13 | +\subsection{Notation and the baseline hedonic function} | |
| 14 | + | |
| 15 | +For sale $i$ let $P_i$ denote the price, $\mathbf{x}_i$ the vector of | |
| 16 | +structural attributes supplied by the assessment roll (floor area $a_i$ | |
| 17 | +in m$^2$, lot area $l_i$ in m$^2$ with an indicator $h_i = \mathbb{1}[l_i | |
| 18 | +> 0]$, building age $g_i$ in years, storeys, dwelling units, and dummies | |
| 19 | +for property class and physical configuration), $m(i)$ the municipality, | |
| 20 | +$c_5(i)$ and $c_1(i)$ the $\sim$5.5~km and $\sim$1.1~km grid cells | |
| 21 | +containing the parcel, and $y(i)$, $q(i)$, $\tau(i)$ the year, quarter and | |
| 22 | +month of sale. A linear hedonic model with a transformation $g(\cdot)$ of | |
| 23 | +the price, a set $\mathcal{S}$ of spatial fixed effects and a set | |
| 24 | +$\mathcal{T}$ of time fixed effects is | |
| 25 | +\begin{equation} | |
| 26 | + g(P_i) \;=\; \boldsymbol{\phi}(\mathbf{x}_i)'\boldsymbol{\beta} | |
| 27 | + \;+\; \mu_{s(i)} \;+\; \delta_{t(i)} \;+\; \varepsilon_i, | |
| 28 | + \qquad s(i)\in\mathcal{S},\; t(i)\in\mathcal{T}, | |
| 29 | + \label{eq:hedonic} | |
| 30 | +\end{equation} | |
| 31 | +where $\boldsymbol{\phi}(\cdot)$ is a basis expansion of the attributes | |
| 32 | +(identity, logarithms, quadratics or splines), $\mu_s$ absorbs location | |
| 33 | +and $\delta_t$ the market level. Following \citet{rosen1974hedonic}, | |
| 34 | +nothing in theory restricts $g$ or $\boldsymbol{\phi}$; the axes below | |
| 35 | +make those choices explicit. The baseline, used as the backbone of | |
| 36 | +axes~B and~C, takes $g = \ln$, quadratics in area and age, | |
| 37 | +$\mathcal{S}$ = municipalities and $\mathcal{T}$ = quarters. | |
| 9 | 38 | |
| 10 | 39 | \subsection{The four axes} |
| 11 | 40 | |
| 12 | 41 | \paragraph{A. Functional form.} With municipality and quarter fixed |
| 13 | −effects and the base attributes (floor area, lot area, building age, | |
| 14 | −storeys, units, property class, physical configuration), we estimate: | |
| 15 | −(A1) fully linear in levels; (A2) semi-log, $\ln P$ on levels | |
| 16 | −\citep{malpezzi2003hedonic}; (A3) log-log, $\ln P$ on logged areas; | |
| 17 | −(A4) Box--Cox on the price, with $\lambda$ chosen by profile likelihood | |
| 18 | −on the training set over a grid \citep{box1964analysis, | |
| 19 | −halvorsen1981choice}; (A5) semi-log with quadratics in area and age; and | |
| 20 | −(A6) semi-log with cubic $B$-splines (six knots) in area, age and lot. | |
| 21 | − | |
| 22 | −\paragraph{B. Time effects.} On the A5 backbone: no time controls, | |
| 23 | −year fixed effects, quarter fixed effects, month fixed effects --- | |
| 24 | −the granularity ladder of the time-dummy index literature | |
| 25 | −\citep{hill2013hedonic}. | |
| 26 | − | |
| 27 | −\paragraph{C. Spatial controls.} On the A5 backbone with quarter FE: | |
| 28 | −no spatial controls, municipality FE (1{,}115), a $\sim$5.5~km grid | |
| 29 | −(5{,}255 cells), and a $\sim$1.1~km grid (30{,}548 cells) --- a | |
| 30 | −granularity ladder that brackets the bias--variance trade-off in | |
| 31 | −absorbing location \citep{case2004modeling, bourassa2010predicting}. | |
| 32 | − | |
| 33 | −\paragraph{D. Estimation method.} Holding the attribute set fixed: | |
| 34 | −(D1) OLS with municipality and month FE; (D2) cross-validated ridge on | |
| 35 | −the same standardized design; (D3) a random forest | |
| 36 | −\citep{breiman2001random} and (D4) gradient boosting | |
| 37 | −\citep{friedman2001greedy} on the attributes plus raw coordinates and a | |
| 38 | −continuous month counter; (D5) gradient boosting \emph{without} | |
| 39 | −coordinates, to price what spatial flexibility is worth to the machine; | |
| 40 | −and (D6) a spatial $k$-nearest-neighbour comparables rule --- the | |
| 41 | −distance-weighted price per square metre of the ten nearest training | |
| 42 | −sales --- as the na\"ive appraiser benchmark. | |
| 42 | +effects held fixed, the six forms differ only in $g$ and | |
| 43 | +$\boldsymbol{\phi}$: | |
| 44 | +\begin{align} | |
| 45 | + \text{A1 (linear):}\quad & P_i = \mathbf{x}_i'\boldsymbol{\beta} | |
| 46 | + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a1}\\ | |
| 47 | + \text{A2 (semi-log):}\quad & \ln P_i = \mathbf{x}_i'\boldsymbol{\beta} | |
| 48 | + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a2}\\ | |
| 49 | + \text{A3 (log-log):}\quad & \ln P_i = \beta_a \ln a_i | |
| 50 | + + \beta_l\, h_i \ln l_i + \beta_h h_i + \tilde{\mathbf{x}}_i'\tilde{\boldsymbol{\beta}} | |
| 51 | + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a3}\\ | |
| 52 | + \text{A4 (Box--Cox):}\quad & \frac{P_i^{\lambda}-1}{\lambda} | |
| 53 | + = \mathbf{x}_i'\boldsymbol{\beta} + \mu_{m(i)} + \delta_{q(i)} | |
| 54 | + + \varepsilon_i; \label{eq:a4}\\ | |
| 55 | + \text{A5 (quadratics):}\quad & \ln P_i = \mathbf{x}_i'\boldsymbol{\beta} | |
| 56 | + + \beta_{aa}\Big(\tfrac{a_i}{100}\Big)^{2} | |
| 57 | + + \beta_{gg}\Big(\tfrac{g_i}{10}\Big)^{2} | |
| 58 | + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i; \label{eq:a5}\\ | |
| 59 | + \text{A6 (splines):}\quad & \ln P_i = \sum_{v\in\{a,l,g\}} | |
| 60 | + \sum_{k=1}^{8} \beta_{vk}\, B_{vk}(v_i) | |
| 61 | + + \tilde{\mathbf{x}}_i'\tilde{\boldsymbol{\beta}} | |
| 62 | + + \mu_{m(i)} + \delta_{q(i)} + \varepsilon_i, \label{eq:a6} | |
| 63 | +\end{align} | |
| 64 | +where $\tilde{\mathbf{x}}_i$ collects the remaining attributes and | |
| 65 | +$B_{vk}$ are cubic $B$-spline basis functions with six uniformly spaced | |
| 66 | +knots over the training range of each variable (eight basis functions | |
| 67 | +per variable). In the Box--Cox model \citep{box1964analysis, | |
| 68 | +halvorsen1981choice} the exponent is chosen on the training sample by | |
| 69 | +maximising the profile log-likelihood | |
| 70 | +\begin{equation} | |
| 71 | + \ell(\lambda) \;=\; -\frac{n}{2}\ln\!\Big(\frac{\mathrm{SSE}(\lambda)}{n}\Big) | |
| 72 | + \;+\; (\lambda - 1)\sum_{i=1}^{n}\ln P_i, | |
| 73 | + \label{eq:profile} | |
| 74 | +\end{equation} | |
| 75 | +where $\mathrm{SSE}(\lambda)$ is the residual sum of squares of | |
| 76 | +\eqref{eq:a4} at $\lambda$ and the second term is the Jacobian of the | |
| 77 | +transformation. The main grid is $\lambda \in \{-0.5, -0.25, \dots, 1\}$; | |
| 78 | +Section~\ref{sec:robust} refines it to steps of $0.05$. | |
| 79 | + | |
| 80 | +\paragraph{B. Time effects.} On the A5 backbone with municipality fixed | |
| 81 | +effects, $\mathcal{T}$ runs through the granularity ladder of the | |
| 82 | +time-dummy index literature \citep{hill2013hedonic}: no time controls | |
| 83 | +(B1), year (B2), quarter (B3 $=$ A5) and month fixed effects (B4). | |
| 84 | + | |
| 85 | +\paragraph{C. Spatial controls.} On the A5 backbone with quarter fixed | |
| 86 | +effects, $\mathcal{S}$ runs from no spatial controls (C1) to | |
| 87 | +municipality fixed effects (C2 $=$ A5; 1{,}115 cells, median 79 sales | |
| 88 | +per cell), a $0.05^{\circ}$ grid (C3; 5{,}255 cells of roughly | |
| 89 | +5.5~km, median 10 sales) and a $0.01^{\circ}$ grid (C4; 30{,}548 cells of | |
| 90 | +roughly 1.1~km, mean 17 and median 3 sales per cell). The ladder brackets | |
| 91 | +the bias--variance trade-off in absorbing location | |
| 92 | +\citep{case2004modeling, bourassa2010predicting}. Because the dummies | |
| 93 | +are one-hot encoded with a dropped reference category, a test sale in a | |
| 94 | +cell absent from the training set is priced at the reference cell's | |
| 95 | +level; this affects 0.2\% of random-holdout sales under C3 and 2.5\% | |
| 96 | +under C4. Section~\ref{sec:robust} reports nested variants (municipality | |
| 97 | +plus grid effects) in which an unseen cell falls back on its | |
| 98 | +municipality instead. | |
| 99 | + | |
| 100 | +\paragraph{D. Estimation method.} Holding the attribute set fixed, D1 is | |
| 101 | +the semi-log quadratic OLS with municipality and month fixed effects | |
| 102 | +(identical to B4); D2 replaces least squares by ridge regression on the | |
| 103 | +same standardised design, | |
| 104 | +\begin{equation} | |
| 105 | + \hat{\boldsymbol{\beta}}_{\alpha} = \arg\min_{\boldsymbol{\beta}} | |
| 106 | + \sum_i \big(\ln P_i - \mathbf{z}_i'\boldsymbol{\beta}\big)^2 | |
| 107 | + + \alpha \lVert \boldsymbol{\beta} \rVert_2^2, | |
| 108 | + \qquad \alpha \in \{10^{-6}, 10^{-5.5}, \dots, 10^{2}\}, | |
| 109 | + \label{eq:ridge} | |
| 110 | +\end{equation} | |
| 111 | +with $\alpha$ selected by three-fold cross-validation within the training | |
| 112 | +set. D3 is a random forest \citep{breiman2001random} and D4 a | |
| 113 | +histogram-based gradient-boosting machine \citep{friedman2001greedy, | |
| 114 | +ke2017lightgbm}, both fitted to $\ln P_i$ on the attribute vector | |
| 115 | +augmented with the raw coordinates $(\mathrm{lat}_i, \mathrm{lng}_i)$ | |
| 116 | +and a continuous month counter $t_i$ (months since January 2021), with | |
| 117 | +property class and physical configuration entered as integer codes. D5 | |
| 118 | +is the boosting machine without the coordinates, which prices what | |
| 119 | +spatial flexibility is worth to the learner. D6 is a spatial | |
| 120 | +$k$-nearest-neighbour comparables rule, the na\"ive appraiser benchmark: | |
| 121 | +\begin{equation} | |
| 122 | + \widehat{\ln P}_i \;=\; \ln a_i \;+\; | |
| 123 | + \frac{\sum_{j\in N_{10}(i)} w_{ij}\,\big(\ln P_j - \ln a_j\big)} | |
| 124 | + {\sum_{j\in N_{10}(i)} w_{ij}}, | |
| 125 | + \qquad w_{ij} = d_{ij}^{-1}, | |
| 126 | + \label{eq:knn} | |
| 127 | +\end{equation} | |
| 128 | +where $N_{10}(i)$ contains the ten training sales nearest to $i$ in | |
| 129 | +kilometre-scaled coordinates and $d_{ij}$ is the planar distance. | |
| 130 | +Hyper-parameters are fixed a priori at library-conventional values: the | |
| 131 | +forest uses 120 trees, a minimum leaf size of 5 and half the features | |
| 132 | +per split; the boosting machine uses 600 iterations, a learning rate of | |
| 133 | +$0.08$, at most 63 leaves per tree, a minimum leaf size of 20 and an | |
| 134 | +$\ell_2$ penalty of $0.01$. The ridge is the only method tuned on the | |
| 135 | +data, and only within the training set. The exercise therefore compares | |
| 136 | +model \emph{classes} as commonly used, not tuned frontier performance; | |
| 137 | +Section~\ref{sec:robust} shows that the ranking is insensitive to | |
| 138 | +materially different boosting configurations, including one tuned on a | |
| 139 | +time-ordered validation set. | |
| 43 | 140 | |
| 44 | 141 | \subsection{One scoreboard: price levels} |
| 142 | +\label{sec:scoreboard} | |
| 143 | + | |
| 144 | +Comparing models whose dependent variables differ requires every model | |
| 145 | +to be scored on the same object. We score \emph{predicted price levels}. | |
| 146 | +Let $\hat{z}_i$ be the prediction of model~$M$ on the transformed scale | |
| 147 | +($g(P)$ for the linear forms, $\ln P$ for the tree ensembles and the | |
| 148 | +$k$-NN rule) on a test sale~$i$. Two inversion rules are available. The | |
| 149 | +\emph{median rule} inverts the fitted value as is, | |
| 150 | +\begin{equation} | |
| 151 | + \hat{P}^{\mathrm{med}}_i = g^{-1}(\hat{z}_i) | |
| 152 | + = \begin{cases} \exp(\hat{z}_i) & g = \ln,\\[2pt] | |
| 153 | + (\lambda \hat{z}_i + 1)^{1/\lambda} & g = \text{Box--Cox},\end{cases} | |
| 154 | + \label{eq:median} | |
| 155 | +\end{equation} | |
| 156 | +which under symmetric transformed-scale errors is the conditional median | |
| 157 | +price and is the quantity the tree ensembles deliver natively. The | |
| 158 | +\emph{mean rule} applies Duan's (\citeyear{duan1983smearing}) | |
| 159 | +nonparametric smearing estimator of the conditional mean, | |
| 160 | +\begin{equation} | |
| 161 | + \hat{P}^{\mathrm{mean}}_i = \frac{1}{n}\sum_{j=1}^{n} | |
| 162 | + g^{-1}\big(\hat{z}_i + \hat{e}_j\big) | |
| 163 | + \;\;\xrightarrow{\;g=\ln\;}\;\; | |
| 164 | + \exp(\hat{z}_i)\cdot \hat{s}, \qquad | |
| 165 | + \hat{s} = \frac{1}{n}\sum_{j=1}^{n} \exp(\hat{e}_j), | |
| 166 | + \label{eq:duan} | |
| 167 | +\end{equation} | |
| 168 | +where $\hat{e}_j$ are the training residuals on the transformed scale. | |
| 169 | +The smearing factor $\hat{s}$ exceeds one and corrects the downward bias | |
| 170 | +of $\exp(\hat{z})$ as an estimator of $E[P\mid\mathbf{x}]$ | |
| 171 | +\citep{manning2001estimating}; for the Box--Cox model we evaluate | |
| 172 | +\eqref{eq:duan} on a random subsample of 1{,}000 training residuals. | |
| 173 | +The main scoreboard uses the median rule for every model, for two | |
| 174 | +reasons. First, it is the only rule that treats all twenty models | |
| 175 | +identically without a model-specific calibration constant: the training | |
| 176 | +residuals of tree ensembles are shrunk by overfitting, so a smearing | |
| 177 | +factor estimated from them is not comparable to one estimated from | |
| 178 | +least-squares residuals. Second, our headline metric is a median error, | |
| 179 | +whose natural target is the conditional median, and the log-scale | |
| 180 | +metrics below are only meaningful on uncorrected log predictions. | |
| 181 | +Section~\ref{sec:robust} reports the full scoreboard under the mean rule | |
| 182 | +as well. Version~1.0 of this paper smeared the linear log models but not | |
| 183 | +the tree ensembles; the asymmetry affected the comparison between the two | |
| 184 | +families and is corrected here. | |
| 185 | + | |
| 186 | +Predictions are floored at \$20{,}000. With $\hat{P}_i$ the level | |
| 187 | +prediction and $N$ the number of test sales, the four metrics are | |
| 188 | +\begin{align} | |
| 189 | + \mathrm{APE}_i &= \frac{\lvert \hat{P}_i - P_i\rvert}{P_i}, \qquad | |
| 190 | + \mathrm{MdAPE} = \operatorname{median}_i \mathrm{APE}_i, \qquad | |
| 191 | + \mathrm{MAPE} = \frac{1}{N}\sum_i \mathrm{APE}_i, \label{eq:ape}\\ | |
| 192 | + \mathrm{RMSE}_{\ln} &= \Big(\frac{1}{N}\sum_i | |
| 193 | + \big(\ln P_i - \ln \hat{P}_i\big)^2\Big)^{1/2}, \qquad | |
| 194 | + R^2_{\ln} = 1 - \frac{\sum_i (\ln P_i - \ln \hat{P}_i)^2} | |
| 195 | + {\sum_i (\ln P_i - \overline{\ln P})^2}. | |
| 196 | + \label{eq:r2} | |
| 197 | +\end{align} | |
| 198 | +MdAPE and MAPE are the accuracy standards of the automated-valuation | |
| 199 | +industry and of ratio studies \citep{iaao2013standard, | |
| 200 | +steurer2021metrics}; $R^2_{\ln}$ is an out-of-sample $R^2$ on the log | |
| 201 | +scale, well defined for every model whatever its dependent variable. | |
| 202 | + | |
| 203 | +\subsection{Two holdouts, and leakage-free variants} | |
| 204 | +\label{sec:holdouts} | |
| 205 | + | |
| 206 | +Each model is evaluated under two splits. The \emph{random} split assigns | |
| 207 | +every sale independently to the test set with probability $0.2$ (seeded; | |
| 208 | +411{,}283 training and 102{,}929 test sales). The \emph{forward-in-time} | |
| 209 | +split trains on sales dated before 1~January 2025 (355{,}824) and tests | |
| 210 | +on 2025--26 (158{,}388), the deployment situation of any valuation model | |
| 211 | +that is estimated once and applied to subsequent sales. Static models | |
| 212 | +carry no forecast of the price level, so we apply the standard | |
| 213 | +carry-forward rule: a test sale whose time category (year, quarter or | |
| 214 | +month) was never observed in training is priced at the last category | |
| 215 | +observed in training, $\hat{\delta}_{t} = \hat{\delta}_{T}$ for $t > T$. | |
| 216 | +The tree ensembles are piecewise constant in the month counter, so a | |
| 217 | +test month beyond the training range is priced by the terminal leaf in | |
| 218 | +$t$, the tree analogue of the same rule. Random cross-validation lets | |
| 219 | +every model interpolate the price level of its own test period; the | |
| 220 | +forward split reveals how much of measured accuracy depends on that | |
| 221 | +interpolation \citep{bergmeir2012crossvalidation, | |
| 222 | +roberts2017crossvalidation}. | |
| 223 | + | |
| 224 | +The random split has a second, less obvious leakage channel. One fifth of | |
| 225 | +the sales in the sample are repeat sales of a parcel that sells again | |
| 226 | +within the window, and 30\% share their exact geocoded coordinates with | |
| 227 | +another sale, 92\% among condominiums, whose units are geocoded to the | |
| 228 | +building. Under a random split, a model that reads the coordinates can | |
| 229 | +learn a parcel's or a building's price from its other sales. We | |
| 230 | +therefore also evaluate the full race under two grouped holdouts in | |
| 231 | +which 20\% of \emph{parcels}, respectively 20\% of \emph{coordinate | |
| 232 | +pairs}, are assigned to the test set, so that no property (respectively | |
| 233 | +no building) appears on both sides | |
| 234 | +\citep{roberts2017crossvalidation}. | |
| 235 | + | |
| 236 | +\subsection{Inference} | |
| 237 | +\label{sec:inference} | |
| 238 | + | |
| 239 | +Differences of a few tenths of a point between models evaluated on the | |
| 240 | +same 100{,}000 sales can be either meaningful or noise, and the sales are | |
| 241 | +not independent: they cluster in municipalities whose local shocks move | |
| 242 | +all errors together. We report two complementary tools. First, a | |
| 243 | +\emph{cluster bootstrap} for MdAPE: municipalities are resampled with | |
| 244 | +replacement ($B = 500$), each draw contributing all its test sales, and | |
| 245 | +the median APE of every model and the difference between two models' | |
| 246 | +MdAPEs are recomputed; we report percentile 95\% intervals. Second, a | |
| 247 | +test of equal predictive accuracy in the spirit of | |
| 248 | +\citet{diebold1995comparing} and \citet{giacomini2006tests}. For models | |
| 249 | +$A$ and $B$ and a loss $L$, let $d_i = L(\hat{P}^A_i, P_i) - | |
| 250 | +L(\hat{P}^B_i, P_i)$; the statistic | |
| 251 | +\begin{equation} | |
| 252 | + t_{\mathrm{DM}} = \frac{\bar{d}}{\widehat{\mathrm{se}}(\bar{d})}, | |
| 253 | + \qquad | |
| 254 | + \widehat{\mathrm{se}}(\bar{d})^2 = \frac{G}{G-1}\,\frac{1}{N^2} | |
| 255 | + \sum_{g=1}^{G}\Big(\sum_{i\in g}(d_i - \bar{d})\Big)^2, | |
| 256 | + \label{eq:dm} | |
| 257 | +\end{equation} | |
| 258 | +uses standard errors clustered by municipality ($G = 1{,}115$), the | |
| 259 | +cross-sectional analogue of the HAC variance in the time-series version. | |
| 260 | +We use the absolute percentage error (the MAPE loss) and the squared log | |
| 261 | +error (the RMSE$_{\ln}$ loss); a positive $\bar{d}$ means that $B$ is | |
| 262 | +more accurate. The mean of the APE loss differential tests MAPE, not | |
| 263 | +MdAPE; the bootstrap covers the median. | |
| 45 | 264 | |
| 46 | −Comparing forms whose dependent variables differ requires care. Every | |
| 47 | −model is scored on \emph{predicted price levels}: log models are | |
| 48 | −retransformed with Duan's smearing factor estimated on training | |
| 49 | −residuals \citep{duan1983smearing}; Box--Cox predictions are analytically | |
| 50 | −inverted; level predictions are floored at \$20{,}000. We report the | |
| 51 | −median and mean absolute percentage errors (MdAPE, MAPE) --- the | |
| 52 | −automated-valuation industry's standards \citep{steurer2021metrics, | |
| 53 | −iaao2013standard} --- together with the RMSE and $R^2$ of log predicted | |
| 54 | −versus log actual prices, which are well-defined for every model. | |
| 55 | − | |
| 56 | −\subsection{Two holdouts} | |
| 57 | − | |
| 58 | −Each model is evaluated under two splits. The \emph{random} split holds | |
| 59 | −out 20\% of sales (seeded). The \emph{forward-in-time} split trains on | |
| 60 | −sales before January 2025 (355{,}824) and tests on 2025--2026 | |
| 61 | −(158{,}388) --- the deployment situation of any valuation model. Static | |
| 62 | −models carry no forecast of the price level, so we apply the standard | |
| 63 | −carry-forward rule: unseen time categories in the test period are priced | |
| 64 | −at the last level observed in training; the machine-learning models | |
| 65 | −receive the same information through the clamped month counter. Random | |
| 66 | −cross-validation lets every model interpolate the price level of its own | |
| 67 | −test period --- the forward split reveals how much of measured accuracy | |
| 68 | −is such leakage \citep{mullainathan2017machine, clapp2002predicting}. | |
| 69 | − | |
| 70 | −\subsection{Estimation details} | |
| 71 | − | |
| 72 | −All linear models are estimated as sparse ridge regressions with a | |
| 73 | −vanishing penalty ($\alpha = 10^{-6}$) --- numerically OLS, but robust to | |
| 74 | −collinear dummies, and unseen categories at prediction time are priced at | |
| 75 | −the reference level. Dummies are one-hot with a dropped reference; the | |
| 76 | −ridge (D2) standardizes columns and cross-validates | |
| 77 | −$\alpha \in [10^{-6}, 10^{2}]$. The forest uses 120 trees | |
| 78 | −(minimum leaf 5, half the features per split); the boosting machine uses | |
| 79 | −600 trees of at most 63 leaves (learning rate 0.08, minimum leaf 20). | |
| 80 | −Hyper-parameters were fixed a priori at library-conventional values --- | |
| 81 | −the exercise compares model \emph{classes} as used in practice, not tuned | |
| 82 | −frontier performance. | |
| 265 | +Finally, because the forward-in-time holdout confounds the difficulty of | |
| 266 | +the test period with the staleness of a frozen model, we decompose | |
| 267 | +forward error into an aggregate level error and relative pricing skill. | |
| 268 | +For each model, $\hat{b} = \operatorname{median}_i(\ln P_i - \ln | |
| 269 | +\hat{P}_i)$ on the test set measures the common level error; the | |
| 270 | +\emph{re-levelled} MdAPE, computed with $\hat{P}_i e^{\hat{b}}$, is the | |
| 271 | +accuracy the model would attain if the aggregate price level of the test | |
| 272 | +period were known, the time adjustment practitioners obtain from a | |
| 273 | +published index \citep{thibodeau2003marking}. We also compare the | |
| 274 | +random-split models on the 2025--26 sales of their own test set with the | |
| 275 | +forward-split models on the full 2025--26 test set, which holds the test | |
| 276 | +period fixed and varies only whether that period was seen in training. | |
modified
paper/sections/results.tex
+282 −86
@@ -4,9 +4,15 @@ | ||
| 4 | 4 | \label{sec:results} |
| 5 | 5 | |
| 6 | 6 | Table~\ref{tab:race_random} reports the full scoreboard under the random |
| 7 | −holdout (411{,}283 training sales, 102{,}929 test sales); | |
| 8 | −Table~\ref{tab:race_temporal} repeats it under the forward-in-time split. | |
| 9 | −Figures~\ref{fig:forms}--\ref{fig:methods} visualize each axis. | |
| 7 | +holdout (411{,}283 training sales, 102{,}929 test sales) and | |
| 8 | +Table~\ref{tab:race_temporal} under the forward-in-time split; | |
| 9 | +Table~\ref{tab:inference} adds the cluster-bootstrap intervals and the | |
| 10 | +equal-accuracy tests against the spline hedonic (A6), and | |
| 11 | +Table~\ref{tab:pairs} the targeted comparisons along each axis. | |
| 12 | +Figures~\ref{fig:forms}--\ref{fig:methods} visualise the four axes under | |
| 13 | +the random holdout. All numbers use the median rule of | |
| 14 | +Section~\ref{sec:scoreboard}; Section~\ref{sec:robust} shows how the | |
| 15 | +Duan-smeared alternative moves them. | |
| 10 | 16 | |
| 11 | 17 | \begin{table}[p] |
| 12 | 18 | \centering |
@@ -17,11 +23,12 @@ Figures~\ref{fig:forms}--\ref{fig:methods} visualize each axis. | ||
| 17 | 23 | \input{../results/tables/horserace_random} |
| 18 | 24 | \begin{tablenotes}[flushleft]\footnotesize |
| 19 | 25 | \item \textit{Notes:} All metrics on the held-out 20\% (102{,}929 sales). |
| 20 | −MdAPE/MAPE: median/mean absolute percentage error on price levels (log | |
| 21 | −models retransformed with Duan smearing; Box--Cox inverted). | |
| 22 | −RMSE$_{\ln}$ and $R^2_{\ln}$: computed on log predicted vs.\ log actual | |
| 23 | −prices. Backbone rows repeat across axes by construction (A5 $=$ B3 $=$ | |
| 24 | −C2; B4 $=$ D1). Bold: best in axis. | |
| 26 | +MdAPE/MAPE: median/mean absolute percentage error on price levels, | |
| 27 | +eq.~\eqref{eq:ape}; transformed-scale predictions inverted with the | |
| 28 | +median rule, eq.~\eqref{eq:median}. RMSE$_{\ln}$ and $R^2_{\ln}$: | |
| 29 | +eq.~\eqref{eq:r2}. Fit: wall-clock seconds on an 18-core workstation. | |
| 30 | +Backbone rows repeat across axes by construction (A5 $=$ B3 $=$ C2; | |
| 31 | +B4 $=$ D1). Bold: lowest MdAPE within the axis. | |
| 25 | 32 | \end{tablenotes} |
| 26 | 33 | \end{threeparttable} |
| 27 | 34 | \end{table} |
@@ -35,31 +42,93 @@ C2; B4 $=$ D1). Bold: best in axis. | ||
| 35 | 42 | \input{../results/tables/horserace_temporal} |
| 36 | 43 | \begin{tablenotes}[flushleft]\footnotesize |
| 37 | 44 | \item \textit{Notes:} Training on sales before January 2025 (355{,}824); |
| 38 | −testing on 2025--2026 (158{,}388). Unseen time categories are priced at | |
| 39 | −the last training period (carry-forward); the machine-learning models | |
| 40 | −receive the same information through the clamped month counter. | |
| 45 | +testing on 2025--26 (158{,}388). Unseen time categories are priced at | |
| 46 | +the last training period (carry-forward); the tree ensembles are | |
| 47 | +piecewise constant in the month counter and therefore apply the same | |
| 48 | +rule through their terminal leaf. Other conventions as in | |
| 49 | +Table~\ref{tab:race_random}. | |
| 50 | +\end{tablenotes} | |
| 51 | +\end{threeparttable} | |
| 52 | +\end{table} | |
| 53 | + | |
| 54 | +\begin{table}[p] | |
| 55 | +\centering | |
| 56 | +\begin{threeparttable} | |
| 57 | +\caption{Inference: bootstrap intervals and equal-accuracy tests against the spline hedonic} | |
| 58 | +\label{tab:inference} | |
| 59 | +\scriptsize | |
| 60 | +\setlength{\tabcolsep}{3.5pt} | |
| 61 | +\input{../results/tables/inference} | |
| 62 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 63 | +\item \textit{Notes:} MdAPE in percent with a 95\% percentile interval | |
| 64 | +from 500 cluster-bootstrap replicates (municipalities resampled with | |
| 65 | +replacement). $\Delta$ vs.\ A6: MdAPE of the row model minus MdAPE of | |
| 66 | +A6 on the same test sales, with its bootstrap interval; negative values | |
| 67 | +favour the row model. $t_{\mathrm{DM}}$: clustered Diebold--Mariano | |
| 68 | +statistic, eq.~\eqref{eq:dm}, for the absolute-percentage-error loss | |
| 69 | +differential (row model minus A6); positive values mean A6 is more | |
| 70 | +accurate on the mean loss. $^{*}$, $^{**}$, $^{***}$: $p < 0.10$, | |
| 71 | +$0.05$, $0.01$ (two-sided). 1{,}115 municipality clusters. | |
| 72 | +\end{tablenotes} | |
| 73 | +\end{threeparttable} | |
| 74 | +\end{table} | |
| 75 | + | |
| 76 | +\begin{table}[tbp] | |
| 77 | +\centering | |
| 78 | +\begin{threeparttable} | |
| 79 | +\caption{Targeted comparisons along each axis} | |
| 80 | +\label{tab:pairs} | |
| 81 | +\scriptsize | |
| 82 | +\setlength{\tabcolsep}{2.5pt} | |
| 83 | +\input{../results/tables/pairs} | |
| 84 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 85 | +\item \textit{Notes:} For each pair, the MdAPE difference (first model | |
| 86 | +minus second) with its 95\% cluster-bootstrap interval and the clustered | |
| 87 | +DM statistics for the APE and squared-log losses (positive: the second | |
| 88 | +model is more accurate on the mean loss). Stars as in | |
| 89 | +Table~\ref{tab:inference}. | |
| 41 | 90 | \end{tablenotes} |
| 42 | 91 | \end{threeparttable} |
| 43 | 92 | \end{table} |
| 44 | 93 | |
| 45 | 94 | \subsection{Axis A: functional form is second-order} |
| 46 | 95 | |
| 47 | −Under the random split (Figure~\ref{fig:forms}), the six forms span | |
| 48 | −16.5--18.6\% MdAPE. The spline specification wins (16.5\%), the log-log | |
| 49 | −is second (17.0\%) --- floor area enters multiplicatively, as intuition | |
| 50 | −suggests --- and the linear, semi-log and quadratic forms cluster within | |
| 51 | −half a point of each other. The Box--Cox transformation selects | |
| 52 | −$\hat\lambda = 0.25$ by profile likelihood (Table~\ref{tab:boxcox}), i.e.\ | |
| 53 | −the data reject both the linear ($\lambda=1$) and the log ($\lambda=0$) | |
| 54 | −--- and then \emph{underperforms} the simple semi-log out of sample | |
| 55 | −(18.6\%), because optimizing curvature in-sample does not survive | |
| 56 | −prediction, precisely the warning of \citet{cassel1985cost} and | |
| 57 | −\citet{cropper1988choice}. The ordering is preserved under the temporal | |
| 58 | −split. Two free lessons: curvature in \emph{attributes} (splines) is | |
| 59 | −worth more than curvature in the \emph{price} (Box--Cox); and no form | |
| 60 | −choice moves the needle by more than two points. | |
| 61 | − | |
| 62 | −\begin{figure}[t] | |
| 96 | +Under the random split (Figure~\ref{fig:forms}) the six forms span | |
| 97 | +17.6--19.2\% MdAPE, a range of 1.5 points. The spline specification wins | |
| 98 | +(17.6\%), the log-log is second (18.0\%)---floor area enters | |
| 99 | +multiplicatively, as intuition suggests---and the linear (18.2\%), | |
| 100 | +Box--Cox (18.6\%), semi-log (19.0\%) and quadratic (19.2\%) forms follow | |
| 101 | +within a point of each other. The spline gain over the textbook | |
| 102 | +semi-log, $-1.4$ points [$-1.8$, $-1.1$], is precisely estimated and | |
| 103 | +holds on every metric (Table~\ref{tab:pairs}). The Box--Cox | |
| 104 | +transformation selects $\hat\lambda = 0.25$ by profile likelihood | |
| 105 | +(Table~\ref{tab:boxcox}), so the data reject both the linear | |
| 106 | +($\lambda = 1$) and the logarithmic ($\lambda = 0$) specification, yet | |
| 107 | +the model gains only half a point on the semi-log's median error, loses | |
| 108 | +on the mean error (DM $t = 9.0$ in favour of the semi-log) and remains a | |
| 109 | +full point behind the splines. Optimising the curvature of the | |
| 110 | +\emph{price} in sample buys little out of sample, the warning of | |
| 111 | +\citet{cassel1985cost} and \citet{cropper1988choice}; curvature in the | |
| 112 | +\emph{attributes}, which splines supply cheaply, is worth more. | |
| 113 | + | |
| 114 | +Under the forward-in-time split (Table~\ref{tab:race_temporal}) the | |
| 115 | +range widens to 20.4--23.3\% and the ordering changes: the linear model | |
| 116 | +in levels posts the lowest median error (20.4\%), followed by the | |
| 117 | +Box--Cox (21.0\%), spline (21.7\%) and log-log (21.8\%) forms, with the | |
| 118 | +semi-log and quadratic forms last (23.2--23.3\%). The linear model's | |
| 119 | +forward-split advantage is a bias coincidence rather than a property of | |
| 120 | +the form: being a conditional-mean predictor in levels, its log | |
| 121 | +prediction over-predicts the typical sale by about 5\% under the random | |
| 122 | +split (median log residual $+0.004$ against $+0.06$ for the log models), | |
| 123 | +and in a rising market that upward tilt offsets part of the staleness of | |
| 124 | +a frozen price level. Once the common level error is removed | |
| 125 | +(Section~\ref{sec:gap}), the spline form is again the most accurate | |
| 126 | +linear specification (16.8\% against 18.0\% for the linear model). | |
| 127 | +Two lessons survive both splits: no functional-form choice moves the | |
| 128 | +median error by more than about two points, and splines dominate the | |
| 129 | +semi-log everywhere. | |
| 130 | + | |
| 131 | +\begin{figure}[tbp] | |
| 63 | 132 | \centering |
| 64 | 133 | \includegraphics[width=0.85\textwidth]{fig_forms.png} |
| 65 | 134 | \caption{Axis A --- functional form. MdAPE on the random holdout; |
@@ -68,7 +137,7 @@ fixed throughout.} | ||
| 68 | 137 | \label{fig:forms} |
| 69 | 138 | \end{figure} |
| 70 | 139 | |
| 71 | −\begin{table}[t] | |
| 140 | +\begin{table}[tbp] | |
| 72 | 141 | \centering |
| 73 | 142 | \begin{threeparttable} |
| 74 | 143 | \caption{Box--Cox profile likelihood} |
@@ -76,26 +145,36 @@ fixed throughout.} | ||
| 76 | 145 | \small |
| 77 | 146 | \input{../results/tables/boxcox} |
| 78 | 147 | \begin{tablenotes}[flushleft]\footnotesize |
| 79 | −\item \textit{Notes:} $^{\dagger}$Thousands, relative to the maximum | |
| 80 | −(zero = chosen $\lambda$). Estimated on the random-split training sample | |
| 81 | −with the A-axis design. | |
| 148 | +\item \textit{Notes:} $^{\dagger}$Profile log-likelihood, | |
| 149 | +eq.~\eqref{eq:profile}, in thousands relative to the maximum (zero at | |
| 150 | +the chosen $\lambda$). Estimated on the random-split training sample | |
| 151 | +with the axis-A design. A finer grid (steps of $0.05$, | |
| 152 | +Section~\ref{sec:robust}) selects $\hat\lambda = 0.35$ and lowers the | |
| 153 | +random-holdout MdAPE from 18.6\% to 18.4\%. | |
| 82 | 154 | \end{tablenotes} |
| 83 | 155 | \end{threeparttable} |
| 84 | 156 | \end{table} |
| 85 | 157 | |
| 86 | 158 | \subsection{Axis B: time controls --- anything beats nothing} |
| 87 | 159 | |
| 88 | −Omitting time controls entirely costs three points (21.1\% versus 18.1\%) | |
| 89 | −in a window containing a boom, a correction and a recovery | |
| 90 | −(Figure~\ref{fig:time}). But the granularity ladder flattens immediately: | |
| 91 | −year, quarter and month effects are indistinguishable to within a quarter | |
| 92 | −point under the random split. Under the temporal split the ordering | |
| 93 | −inverts at the bottom --- \emph{no} time effects (25.4\%) is the worst | |
| 94 | −choice again, but month effects with carry-forward (20.3\%) beat quarter | |
| 95 | −effects (20.5\%) only marginally: once the level must be extrapolated, | |
| 96 | −fine within-sample time resolution has little to offer. | |
| 97 | − | |
| 98 | −\begin{figure}[t] | |
| 160 | +Omitting time controls entirely costs three points (22.0\% against | |
| 161 | +19.2\%, difference $-2.9$ [$-3.8$, $-2.1$] for year effects against | |
| 162 | +none) in a window containing a boom, a correction and a recovery | |
| 163 | +(Figure~\ref{fig:time}). The granularity ladder then flattens at once: | |
| 164 | +year, quarter and month effects lie within 0.2 points of each other | |
| 165 | +under the random split, and month effects are, if anything, marginally | |
| 166 | +worse than quarter effects ($+0.19$ [$+0.01$, $+0.37$]), the first sign | |
| 167 | +that finer within-sample time resolution buys nothing for prediction. | |
| 168 | +Under the forward-in-time split the cost of omitting time controls rises | |
| 169 | +to six points (29.3\% against 23.3\%), because a model without time | |
| 170 | +effects prices 2025--26 sales at the \emph{average} 2021--24 level rather | |
| 171 | +than at the last observed level, and month effects with carry-forward | |
| 172 | +(22.9\%) beat quarter effects (23.3\%) by a statistically fragile margin | |
| 173 | +($-0.43$ [$-0.95$, $+0.05$]). Once the level must be extrapolated, fine | |
| 174 | +time resolution has little to offer; what matters is that the carried | |
| 175 | +level be the most recent one. | |
| 176 | + | |
| 177 | +\begin{figure}[tbp] | |
| 99 | 178 | \centering |
| 100 | 179 | \includegraphics[width=0.85\textwidth]{fig_time.png} |
| 101 | 180 | \caption{Axis B --- time effects (quadratic semi-log backbone, |
@@ -106,18 +185,31 @@ municipality FE). MdAPE on the random holdout.} | ||
| 106 | 185 | \subsection{Axis C: spatial controls are first-order --- up to a point} |
| 107 | 186 | |
| 108 | 187 | Location dominates every other design margin |
| 109 | −(Figure~\ref{fig:space}). Stripping all spatial controls costs | |
| 110 | −\emph{ten} points (27.9\% versus 17.5\%) --- five times the entire | |
| 188 | +(Figure~\ref{fig:space}). Stripping all spatial controls costs ten | |
| 189 | +points (28.8\% against 19.2\% with municipality effects, 11.2 points | |
| 190 | +[10.1, 12.3] relative to the spline hedonic), six times the entire | |
| 111 | 191 | functional-form range. The granularity ladder, however, is |
| 112 | −non-monotonic: the $\sim$5.5~km grid (5{,}255 cells, 17.5\%) beats | |
| 113 | −municipality effects (18.1\%), but the $\sim$1.1~km grid (30{,}548 | |
| 114 | −cells, 19.4\%) \emph{overfits} --- its cells average seventeen K sales | |
| 115 | −each, unseen cells in the holdout revert to the baseline, and estimated | |
| 116 | −effects are noisy. The optimal spatial resolution is an interior | |
| 117 | −solution, echoing \citet{goodman1998housing} on submarket definition: | |
| 118 | −finer is better only while cells remain thick enough to estimate. | |
| 119 | − | |
| 120 | −\begin{figure}[t] | |
| 192 | +non-monotonic. The $\sim$5.5~km grid (5{,}255 cells) is nominally better | |
| 193 | +than municipality effects (18.7\% against 19.2\%), but the difference, | |
| 194 | +$-0.45$ [$-1.07$, $+0.37$], is within bootstrap noise and the DM tests | |
| 195 | +do not reject equal accuracy; the $\sim$1.1~km grid (30{,}548 cells) | |
| 196 | +is clearly worse ($+1.8$ [$+1.2$, $+2.3$]). Its cells contain 17 sales on | |
| 197 | +average and 3 at the median, so most cell effects are estimated from a | |
| 198 | +handful of sales, and the 2.5\% of test sales that fall in a cell with | |
| 199 | +no training sale are priced at an arbitrary reference cell. Under the | |
| 200 | +forward-in-time split the ordering becomes monotonic in the wrong | |
| 201 | +direction: finer cells are worse at every step (23.3\%, 24.3\%, 25.3\%), | |
| 202 | +because a fine spatial configuration estimated on 2021--24 prices | |
| 203 | +transfers less well than a coarse one. The optimal spatial resolution is | |
| 204 | +an interior solution that depends on market thickness, echoing | |
| 205 | +\citet{goodman1998housing} and \citet{bourassa2007spatial} on submarket | |
| 206 | +definition: finer is better only while cells remain thick enough to | |
| 207 | +estimate, and the relevant thickness is larger when the model must be | |
| 208 | +carried forward in time. Section~\ref{sec:robust} shows that nesting the | |
| 209 | +grid within municipalities, so that an unseen cell falls back on its | |
| 210 | +municipality, repairs part but not all of the fine grid's deficit. | |
| 211 | + | |
| 212 | +\begin{figure}[tbp] | |
| 121 | 213 | \centering |
| 122 | 214 | \includegraphics[width=0.85\textwidth]{fig_space.png} |
| 123 | 215 | \caption{Axis C --- spatial controls (quadratic semi-log backbone, |
@@ -127,22 +219,33 @@ quarter FE). MdAPE on the random holdout.} | ||
| 127 | 219 | |
| 128 | 220 | \subsection{Axis D: the machines --- and what they actually buy} |
| 129 | 221 | |
| 130 | −Under the random split (Figure~\ref{fig:methods}), gradient boosting | |
| 131 | −posts 14.4\% and the random forest 14.6\%, against 16.5\% for the best | |
| 132 | −linear model and 18.3\% for the textbook OLS: a 13--21\% error | |
| 133 | −reduction, squarely in the range reported by the AVM literature | |
| 134 | −\citep{mayer2019estimation, hong2020machine}. The decomposition rows | |
| 135 | −identify the source. Boosting \emph{without coordinates} collapses to | |
| 136 | −23.2\% --- worse than plain OLS with municipality dummies --- so the | |
| 137 | −machine's edge is overwhelmingly a flexible surface over space, not | |
| 138 | −exotic structure--attribute interactions. Pure proximity without | |
| 139 | −structure (the $k$-NN comparables rule, 21.2\%) fails too: it is the | |
| 140 | −\emph{combination} of spatial flexibility with attribute adjustment that | |
| 141 | −wins. The cross-validated ridge (16.7\%) confirms that regularizing the | |
| 142 | −high-dimensional dummy design buys a little; it cannot manufacture the | |
| 143 | −missing interactions. | |
| 144 | − | |
| 145 | −\begin{figure}[t] | |
| 222 | +Under the random split (Figure~\ref{fig:methods}) gradient boosting | |
| 223 | +posts 14.4\% [13.7, 15.7] and the random forest 14.6\%, against 17.6\% | |
| 224 | +for the best linear model and 19.3\% for the textbook OLS with | |
| 225 | +municipality and month effects: an 18--25\% reduction in the median | |
| 226 | +error, squarely in the range reported by the automated-valuation | |
| 227 | +literature \citep{mayer2019estimation, hong2020machine, | |
| 228 | +perezrave2019machine}. The boosting advantage over the spline hedonic, | |
| 229 | +$-3.2$ points [$-4.7$, $-1.9$], is statistically unambiguous on every | |
| 230 | +metric (DM $t = -3.6$ on the APE loss, $-3.0$ on the squared-log loss), | |
| 231 | +and boosting edges the forest by a quarter point ($-0.24$ [$-0.46$, | |
| 232 | +$-0.05$]). The decomposition rows identify the source. Boosting | |
| 233 | +\emph{without coordinates} collapses to 23.2\%, worse than plain OLS | |
| 234 | +with municipality dummies, so the learner's edge is overwhelmingly a | |
| 235 | +flexible surface over space rather than exotic structure--attribute | |
| 236 | +interactions, consistent with the residual-information results of | |
| 237 | +\citet{pace2020residuals}. Pure proximity without structure fails too: | |
| 238 | +the $k$-NN comparables rule (21.2\%) is three and a half points worse | |
| 239 | +than the spline hedonic. It is the \emph{combination} of spatial | |
| 240 | +flexibility with attribute adjustment that wins. The cross-validated | |
| 241 | +ridge (17.7\%) is a further lesson: on the identical design, shrinking | |
| 242 | +the 1{,}115 municipality effects and the month effects toward zero | |
| 243 | +improves on OLS by 1.7 points [1.1, 2.4] and ties the spline hedonic | |
| 244 | +(difference $+0.06$ [$-0.02$, $+0.16$]). Regularising the thin | |
| 245 | +fixed effects is as valuable as adding curvature; neither manufactures | |
| 246 | +the spatial surface the trees learn. | |
| 247 | + | |
| 248 | +\begin{figure}[tbp] | |
| 146 | 249 | \centering |
| 147 | 250 | \includegraphics[width=0.85\textwidth]{fig_methods.png} |
| 148 | 251 | \caption{Axis D --- estimation methods on the identical attribute set. |
@@ -151,25 +254,32 @@ MdAPE on the random holdout.} | ||
| 151 | 254 | \end{figure} |
| 152 | 255 | |
| 153 | 256 | \subsection{The generalization gap: when the test set is the future} |
| 257 | +\label{sec:gap} | |
| 154 | 258 | |
| 155 | 259 | Table~\ref{tab:generalization} and Figure~\ref{fig:generalization} |
| 156 | 260 | contain the paper's central result. Moving from the random to the |
| 157 | −forward-in-time split degrades every model --- the price level of | |
| 158 | −2025--26 must be carried forward, not interpolated --- but the | |
| 159 | −degradation is systematically larger for the flexible methods. | |
| 160 | −Gradient boosting loses 6.8 points of MdAPE (14.4 $\to$ 21.2) and the | |
| 161 | −random forest 6.7, while the spline hedonic loses 2.6 (16.5 $\to$ 19.1) | |
| 162 | −and the log-log 2.2. The ranking \emph{reverses}: the best linear models | |
| 163 | −beat the machines on the median error in the deployment scenario, and | |
| 164 | −tie them exactly on $R^2_{\ln}$ ($\approx 0.52$). The machines remain | |
| 165 | −better on MAPE (29.7 versus 30.5), i.e.\ in the tails, but the headline | |
| 166 | −random-validation gap --- the number the ML-valuation literature | |
| 167 | −reports --- overstates their deployable advantage entirely. The lesson | |
| 168 | −is methodological and general: for models whose job is to price the | |
| 169 | −future, random cross-validation is the wrong experiment | |
| 170 | −\citep{mullainathan2017machine, clapp2002predicting}. | |
| 171 | − | |
| 172 | −\begin{table}[t] | |
| 261 | +forward-in-time split degrades every model---the price level of 2025--26 | |
| 262 | +must be carried forward, not interpolated---but the degradation is | |
| 263 | +systematically larger for the flexible methods. Gradient boosting loses | |
| 264 | +6.8 points of MdAPE (14.4 $\to$ 21.2) and the random forest 6.7, while | |
| 265 | +the spline hedonic loses 4.1 (17.6 $\to$ 21.7), the ridge 4.6 and the | |
| 266 | +linear model in levels 2.1. The ranking compresses: the five best | |
| 267 | +forward-split models, linear (20.4\%), Box--Cox (21.0\%), boosting | |
| 268 | +(21.2\%), forest (21.3\%) and splines (21.7\%), lie within 1.3 points | |
| 269 | +and are statistically indistinguishable on the median error---boosting | |
| 270 | +against splines gives $-0.51$ [$-1.68$, $+0.55$], boosting against the | |
| 271 | +linear model $+0.84$ [$-1.19$, $+2.59$]. The ensembles keep a clear lead | |
| 272 | +on the mean error (MAPE 29.7\% against 30.9\% for splines and 32.9\% for | |
| 273 | +the linear model; DM $t = -3.5$ against the linear model) and on the log | |
| 274 | +scale ($R^2_{\ln}$ 0.52 against 0.49 and 0.46). Version~1.0 of this paper | |
| 275 | +reported that the ranking \emph{reversed}; that reversal was an artefact | |
| 276 | +of smearing the linear log models but not the tree ensembles | |
| 277 | +(Section~\ref{sec:robust}). The corrected statement is weaker but still | |
| 278 | +consequential: the three-point advantage that random validation | |
| 279 | +attributes to gradient boosting shrinks to half a point, and to | |
| 280 | +insignificance, when the test set lies in the future. | |
| 281 | + | |
| 282 | +\begin{table}[tbp] | |
| 173 | 283 | \centering |
| 174 | 284 | \begin{threeparttable} |
| 175 | 285 | \caption{Random versus forward-in-time evaluation} |
@@ -183,7 +293,7 @@ under both splits. | ||
| 183 | 293 | \end{threeparttable} |
| 184 | 294 | \end{table} |
| 185 | 295 | |
| 186 | −\begin{figure}[t] | |
| 296 | +\begin{figure}[tbp] | |
| 187 | 297 | \centering |
| 188 | 298 | \includegraphics[width=0.85\textwidth]{fig_generalization.png} |
| 189 | 299 | \caption{The generalization gap. MdAPE under the random holdout (filled |
@@ -191,3 +301,89 @@ circles) and the forward-in-time holdout (open squares); the grey | ||
| 191 | 301 | segment joins the two evaluations of the same model.} |
| 192 | 302 | \label{fig:generalization} |
| 193 | 303 | \end{figure} |
| 304 | + | |
| 305 | +Where does the gap come from? Three further cuts of the data answer the | |
| 306 | +question (Table~\ref{tab:same_period}). First, holding the test period | |
| 307 | +fixed: evaluated on the 31{,}465 sales of 2025--26 that happen to fall in | |
| 308 | +the random test set, the random-split boosting model posts 13.7\%, so | |
| 309 | +the 2025--26 sales are not intrinsically harder to price; the entire | |
| 310 | +degradation to 21.2\% comes from not having seen their period. The | |
| 311 | +\emph{staleness cost}, defined as the forward-split minus the | |
| 312 | +same-period random-split MdAPE, is 7.5 points for boosting and 7.5 for | |
| 313 | +the forest, against 4.7 for the spline hedonic, 5.2 for the ridge and | |
| 314 | +3.3 for the linear model. Second, the level decomposition of | |
| 315 | +Section~\ref{sec:inference}: the median log residual of every | |
| 316 | +forward-split model is positive and large---$+0.16$ for the spline | |
| 317 | +hedonic, $+0.18$ for the two ensembles, $+0.10$ for the linear | |
| 318 | +model---so every frozen model under-prices the typical 2025--26 sale by | |
| 319 | +10--20\%, the ensembles slightly more because their terminal leaf in the | |
| 320 | +month counter averages the last few training months rather than the last | |
| 321 | +one. Once each model's predictions are re-levelled by its own median | |
| 322 | +ratio, the forward-split MdAPE falls to 13.9\% for boosting and 16.8\% | |
| 323 | +for the spline hedonic: a gap of 2.9 points, essentially the 3.5-point | |
| 324 | +gap measured under the random split with the same correction | |
| 325 | +(13.0\% against 16.5\%). Third, the horizon profile | |
| 326 | +(Table~\ref{tab:by_quarter}, Figure~\ref{fig:horizon}): in the first | |
| 327 | +quarter after the training window boosting still leads the spline | |
| 328 | +hedonic by 1.5 points (17.1\% against 18.6\%); the lead shrinks every | |
| 329 | +quarter and vanishes by mid-2026, when both models post about 25\%. | |
| 330 | + | |
| 331 | +The picture is therefore not that the learners have memorised something | |
| 332 | +that fails to transfer. Their \emph{relative} pricing skill---which | |
| 333 | +sales are dear and which are cheap within a period---transfers forward | |
| 334 | +almost intact. What does not transfer is the \emph{level}, and a frozen | |
| 335 | +boosted model carries a slightly staler level than a frozen linear model | |
| 336 | +with carry-forward month effects. The forward-in-time holdout thus | |
| 337 | +quantifies two things that random cross-validation hides: a level | |
| 338 | +staleness of 3--7 points of median error within eighteen months, common | |
| 339 | +to every static model and largest for the trees, and the fact that the | |
| 340 | +headline gap between learners and linear hedonics is a within-period | |
| 341 | +statement. For models whose job is to price the future, random | |
| 342 | +cross-validation is the wrong experiment \citep{bergmeir2012crossvalidation, | |
| 343 | +mullainathan2017machine}; the right one separates the level, which an | |
| 344 | +index or a retraining schedule must supply, from the relative accuracy | |
| 345 | +the estimator contributes. | |
| 346 | + | |
| 347 | +\begin{table}[tbp] | |
| 348 | +\centering | |
| 349 | +\begin{threeparttable} | |
| 350 | +\caption{Decomposing the generalization gap: same-period comparison and level error} | |
| 351 | +\label{tab:same_period} | |
| 352 | +\footnotesize | |
| 353 | +\setlength{\tabcolsep}{4pt} | |
| 354 | +\input{../results/tables/same_period} | |
| 355 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 356 | +\item \textit{Notes:} MdAPE in percent. Columns 2--3: models estimated on | |
| 357 | +the random-split training sample, evaluated on all random test sales and | |
| 358 | +on the 2025--26 subset of those sales. Column 4: models estimated on | |
| 359 | +sales before 2025 and evaluated on all 2025--26 sales. Staleness cost: | |
| 360 | +column 4 minus column 3. Column 5: median of $\ln P_i - \ln\hat{P}_i$ on | |
| 361 | +the forward test set (common level error). Column 6: forward-split MdAPE | |
| 362 | +after multiplying each model's predictions by $e^{\hat{b}}$ | |
| 363 | +(Section~\ref{sec:inference}). | |
| 364 | +\end{tablenotes} | |
| 365 | +\end{threeparttable} | |
| 366 | +\end{table} | |
| 367 | + | |
| 368 | +\begin{table}[tbp] | |
| 369 | +\centering | |
| 370 | +\begin{threeparttable} | |
| 371 | +\caption{Forward-in-time accuracy by test quarter} | |
| 372 | +\label{tab:by_quarter} | |
| 373 | +\small | |
| 374 | +\input{../results/tables/by_quarter} | |
| 375 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 376 | +\item \textit{Notes:} MdAPE (\%) of models trained on sales before | |
| 377 | +January 2025, by quarter of the test sale. 2026Q3 covers July 2026 | |
| 378 | +only. | |
| 379 | +\end{tablenotes} | |
| 380 | +\end{threeparttable} | |
| 381 | +\end{table} | |
| 382 | + | |
| 383 | +\begin{figure}[tbp] | |
| 384 | +\centering | |
| 385 | +\includegraphics[width=0.8\textwidth]{fig_horizon.png} | |
| 386 | +\caption{Forward-in-time accuracy by horizon. MdAPE by quarter of the | |
| 387 | +test sale for models trained on sales before January 2025.} | |
| 388 | +\label{fig:horizon} | |
| 389 | +\end{figure} | |
added
paper/sections/robustness.tex
+216 −0
@@ -0,0 +1,216 @@ | ||
| 1 | +% Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +% ============================================================================ | |
| 3 | +\section{Robustness} | |
| 4 | +\label{sec:robust} | |
| 5 | + | |
| 6 | +This section examines the five choices in the design that a sceptical | |
| 7 | +reader would most want varied: the rule that converts transformed-scale | |
| 8 | +predictions into price levels, the leakage channels of the random | |
| 9 | +holdout, the implementation of fine spatial effects, the resolution of | |
| 10 | +the Box--Cox grid and the hyper-parameters of the boosting machine. | |
| 11 | + | |
| 12 | +\subsection{The retransformation rule} | |
| 13 | + | |
| 14 | +Table~\ref{tab:retransformation} reports the scoreboard for the | |
| 15 | +transformed-scale models under the median rule used throughout | |
| 16 | +(eq.~\ref{eq:median}) and under Duan's mean rule | |
| 17 | +(eq.~\ref{eq:duan}). The smearing factors $\hat{s}$ lie between 1.06 | |
| 18 | +and 1.07 for the least-squares models and between 1.03 and 1.05 for the | |
| 19 | +ensembles, whose in-sample residuals are shrunk by overfitting. Two | |
| 20 | +regularities stand out. First, the mean rule lowers the median error of | |
| 21 | +\emph{every} log-scale model under the random split, by about one point | |
| 22 | +for the linear models and 1.3 points for the ensembles. This is not what | |
| 23 | +a conditional-mean correction should do to a median-based metric; it | |
| 24 | +happens because the log residuals are left-skewed (skewness $-1.4$ to | |
| 25 | +$-1.8$), so that their median is $+0.06$ while their mean is zero, and | |
| 26 | +$\exp(\hat{z})$ under-prices the typical sale by about 6\%---which is, | |
| 27 | +by coincidence, the size of the smearing factor. Second, under the | |
| 28 | +forward-in-time split the mean rule lowers median errors by 2.5--3 | |
| 29 | +points for all models, because the market rose after the training | |
| 30 | +window and any upward factor offsets part of the staleness. Neither | |
| 31 | +effect changes a ranking when the rule is applied uniformly: boosting | |
| 32 | +leads the spline hedonic by 3.4 points under the random split and by | |
| 33 | +0.7 points under the forward split with the mean rule, against 3.2 and | |
| 34 | +0.5 with the median rule. What does change a ranking is applying the | |
| 35 | +rule to one family only. Version~1.0 of this paper smeared the linear | |
| 36 | +log models and not the ensembles; the result was a forward-split | |
| 37 | +comparison of 19.1\% for the smeared spline hedonic against 21.2\% for | |
| 38 | +the unsmeared boosting model, reported as a reversal. The comparable | |
| 39 | +numbers are 21.7\% against 21.2\% (median rule) or 19.1\% against | |
| 40 | +18.4\% (mean rule). The log-scale metrics are affected in the same | |
| 41 | +direction: smearing adds $\ln\hat{s} \approx 0.06$ to every log | |
| 42 | +prediction, which inflates RMSE$_{\ln}$ and depresses $R^2_{\ln}$ for | |
| 43 | +the smeared family only, so that the ``identical $R^2_{\ln}$'' of the | |
| 44 | +first version (0.52 for both) becomes 0.49 against 0.52 in favour of | |
| 45 | +boosting under the uniform rule. | |
| 46 | + | |
| 47 | +\begin{table}[tbp] | |
| 48 | +\centering | |
| 49 | +\begin{threeparttable} | |
| 50 | +\caption{Median rule versus Duan smearing} | |
| 51 | +\label{tab:retransformation} | |
| 52 | +\scriptsize | |
| 53 | +\setlength{\tabcolsep}{4pt} | |
| 54 | +\input{../results/tables/retransformation} | |
| 55 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 56 | +\item \textit{Notes:} Median: transformed-scale prediction inverted as | |
| 57 | +is, eq.~\eqref{eq:median} (main scoreboard). Mean: Duan smearing, | |
| 58 | +eq.~\eqref{eq:duan}; $\hat{s}$ is the smearing factor estimated on | |
| 59 | +training residuals (random split), undefined for the Box--Cox model, | |
| 60 | +whose mean prediction averages the inverse transformation over 1{,}000 | |
| 61 | +training residuals. $R^2_{\ln}$ under the mean rule is computed on the | |
| 62 | +smeared log prediction. | |
| 63 | +\end{tablenotes} | |
| 64 | +\end{threeparttable} | |
| 65 | +\end{table} | |
| 66 | + | |
| 67 | +\subsection{Leakage-free holdouts} | |
| 68 | + | |
| 69 | +Table~\ref{tab:group_splits} re-runs the full race under the two grouped | |
| 70 | +holdouts of Section~\ref{sec:holdouts}, and Table~\ref{tab:leakage} | |
| 71 | +decomposes the random holdout by whether the test sale's parcel, or its | |
| 72 | +exact coordinates, also appear in training. Repeat sales are irrelevant: | |
| 73 | +the parcel-grouped race reproduces the random race to within 0.1--0.4 | |
| 74 | +points for every model, and within the random holdout the boosting | |
| 75 | +model prices test sales whose parcel sold before in training (14.5\%) | |
| 76 | +no better than the others (14.4\%). Shared coordinates matter more, but | |
| 77 | +for a specific reason. Test sales that share coordinates with a training | |
| 78 | +sale are priced at 12.0\% by boosting against 15.3\% for the rest, and | |
| 79 | +at 16.7\% against 22.9\% by the $k$-NN rule, whose nearest neighbour is | |
| 80 | +then literally the same building; the linear models are unaffected | |
| 81 | +(17.8\% against 17.6\% for the splines). Condominiums account for most | |
| 82 | +of these co-located sales, and the boosting model prices them at 8.9\% | |
| 83 | +when their building is in training and 10.2\% when it is not, against | |
| 84 | +18.0\% and 20.3\% for the quadratic OLS. The condominium advantage of | |
| 85 | +Section~\ref{sec:ext} is therefore genuine micro-location, with a modest | |
| 86 | +recall component. The building-grouped race, in which no coordinate pair | |
| 87 | +straddles the split, confirms it: boosting posts 14.6\% against 17.8\% | |
| 88 | +for the spline hedonic, a 3.2-point gap identical to the random race, | |
| 89 | +and the only model that loses materially is the $k$-NN rule (22.3\% | |
| 90 | +against 21.2\%). | |
| 91 | + | |
| 92 | +\begin{table}[p] | |
| 93 | +\centering | |
| 94 | +\begin{threeparttable} | |
| 95 | +\caption{The race under random, parcel-grouped and building-grouped holdouts} | |
| 96 | +\label{tab:group_splits} | |
| 97 | +\small | |
| 98 | +\input{../results/tables/group_splits} | |
| 99 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 100 | +\item \textit{Notes:} MdAPE in percent. Parcel-grouped: 20\% of | |
| 101 | +assessment-roll parcels assigned to the test set, so that repeat sales | |
| 102 | +of a property never straddle the split. Building-grouped: 20\% of exact | |
| 103 | +coordinate pairs assigned to the test set, so that units geocoded to the | |
| 104 | +same building or complex never straddle the split. | |
| 105 | +\end{tablenotes} | |
| 106 | +\end{threeparttable} | |
| 107 | +\end{table} | |
| 108 | + | |
| 109 | +\begin{table}[tbp] | |
| 110 | +\centering | |
| 111 | +\begin{threeparttable} | |
| 112 | +\caption{Random holdout: accuracy on test sales seen and unseen through their parcel or coordinates} | |
| 113 | +\label{tab:leakage} | |
| 114 | +\small | |
| 115 | +\input{../results/tables/leakage} | |
| 116 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 117 | +\item \textit{Notes:} MdAPE (\%) on the random holdout. ``Same parcel in | |
| 118 | +training'': the parcel of the test sale sold at least once in the | |
| 119 | +training sample. ``Same coordinates in training'': at least one training | |
| 120 | +sale shares the exact geocoded coordinates of the test sale. | |
| 121 | +\end{tablenotes} | |
| 122 | +\end{threeparttable} | |
| 123 | +\end{table} | |
| 124 | + | |
| 125 | +\subsection{Nested spatial fixed effects} | |
| 126 | + | |
| 127 | +Section~\ref{sec:results} attributes the poor showing of the | |
| 128 | +$\sim$1.1~km grid to thin cells and to the arbitrary pricing of unseen | |
| 129 | +cells. Table~\ref{tab:nested_fe} adds two specifications in which the | |
| 130 | +grid effects enter \emph{together with} municipality effects. Because | |
| 131 | +the two sets of dummies are nearly collinear, the vanishing-penalty | |
| 132 | +ridge returns the minimum-norm least-squares solution, which assigns the | |
| 133 | +component common to a municipality to the municipality effect and only | |
| 134 | +the deviation to the cell effect---a crude form of partial pooling---and | |
| 135 | +a test sale in an unseen cell inherits its municipality's level. The | |
| 136 | +effect is large. Nested within municipalities, the 5.5~km grid improves | |
| 137 | +from 18.7\% to 17.7\% and the 1.1~km grid from 21.0\% to 17.3\%, the | |
| 138 | +lowest median error of any linear specification in the paper, below the | |
| 139 | +spline hedonic (17.6\%) with the same $R^2_{\ln}$ (0.59). Under the | |
| 140 | +forward-in-time split the nested fine grid posts 21.6\% against 23.3\% | |
| 141 | +for municipality effects alone and 25.3\% for the fine grid alone. The | |
| 142 | +conclusion for axis~C is therefore sharper than ``finer overfits'': | |
| 143 | +fine spatial effects are valuable when they are anchored to a coarser | |
| 144 | +level that supplies the prior for thin and unseen cells, and harmful | |
| 145 | +when they are not. A hierarchical or kriged location effect | |
| 146 | +\citep{case2004modeling, hill2018geospatial} would push the linear | |
| 147 | +frontier further; we leave its estimation to future work. | |
| 148 | + | |
| 149 | +\begin{table}[tbp] | |
| 150 | +\centering | |
| 151 | +\begin{threeparttable} | |
| 152 | +\caption{Nested spatial fixed effects} | |
| 153 | +\label{tab:nested_fe} | |
| 154 | +\small | |
| 155 | +\input{../results/tables/nested_fe} | |
| 156 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 157 | +\item \textit{Notes:} Quadratic semi-log backbone with quarter fixed | |
| 158 | +effects. C3$'$ and C4$'$ add the grid effects to the municipality | |
| 159 | +effects; with a vanishing ridge penalty the estimator is the | |
| 160 | +minimum-norm least-squares solution and unseen cells fall back on their | |
| 161 | +municipality. | |
| 162 | +\end{tablenotes} | |
| 163 | +\end{threeparttable} | |
| 164 | +\end{table} | |
| 165 | + | |
| 166 | +\subsection{A finer Box--Cox grid} | |
| 167 | + | |
| 168 | +The main grid steps $\lambda$ by $0.25$. Refining it to steps of $0.05$ | |
| 169 | +over $[-0.25, 0.75]$ moves the profile-likelihood maximum to | |
| 170 | +$\hat\lambda = 0.35$, with a profile that is flat between $0.25$ and | |
| 171 | +$0.55$ (differences of under one thousandth of the log-likelihood, | |
| 172 | +within the tolerance of the sparse iterative solver) and falls steeply | |
| 173 | +on either side. The Box--Cox model at $\hat\lambda = 0.35$ posts a | |
| 174 | +random-holdout MdAPE of 18.4\% against 18.6\% at $0.25$: the exponent is | |
| 175 | +imprecisely identified in a range that excludes both the linear and the | |
| 176 | +logarithmic case, and its exact value is immaterial for prediction. | |
| 177 | + | |
| 178 | +\subsection{Boosting hyper-parameters} | |
| 179 | + | |
| 180 | +Table~\ref{tab:hgb} re-estimates the boosting model under four | |
| 181 | +materially different configurations: shallow trees with more iterations, | |
| 182 | +deep trees, heavy regularisation (minimum leaf 200, $\ell_2 = 1$), and a | |
| 183 | +configuration whose number of iterations is chosen by early stopping on | |
| 184 | +the last 10\% of training sales in date order, the time-honest tuning | |
| 185 | +rule a careful practitioner would use for a forward-looking model. | |
| 186 | +Under the random holdout the four alternatives span 14.3--14.7\% MdAPE | |
| 187 | +against 14.4\% for the baseline, and under the forward-in-time holdout | |
| 188 | +21.2--21.6\% against 21.2\%; $R^2_{\ln}$ moves by at most 0.006. The | |
| 189 | +time-honest early-stopping rule selects 700 iterations on the random | |
| 190 | +training sample and 1{,}000 on the pre-2025 sample and reproduces the | |
| 191 | +baseline to within 0.1 point under both splits. The conclusions of Sections~\ref{sec:results} | |
| 192 | +and~\ref{sec:gap} do not depend on the particular configuration we | |
| 193 | +fixed a priori, and tuning on a time-ordered validation set does not | |
| 194 | +remove the level staleness that dominates the forward-split error, | |
| 195 | +because no choice of hyper-parameters lets a static tree extrapolate a | |
| 196 | +price level it has not seen. | |
| 197 | + | |
| 198 | +\begin{table}[tbp] | |
| 199 | +\centering | |
| 200 | +\begin{threeparttable} | |
| 201 | +\caption{Gradient-boosting hyper-parameter sensitivity} | |
| 202 | +\label{tab:hgb} | |
| 203 | +\footnotesize | |
| 204 | +\setlength{\tabcolsep}{4pt} | |
| 205 | +\input{../results/tables/hgb_sensitivity} | |
| 206 | +\begin{tablenotes}[flushleft]\footnotesize | |
| 207 | +\item \textit{Notes:} All configurations use the D4 feature set | |
| 208 | +(attributes, coordinates, month counter). Baseline: 600 iterations, | |
| 209 | +learning rate 0.08, 63 leaves, minimum leaf 20, $\ell_2 = 0.01$. Early | |
| 210 | +stopping: iterations added in blocks of 100 (learning rate 0.05) until | |
| 211 | +the squared log error on the last 10\% of training sales in date order | |
| 212 | +stops improving; the selected number of iterations is shown in brackets | |
| 213 | +for the random and forward splits. | |
| 214 | +\end{tablenotes} | |
| 215 | +\end{threeparttable} | |
| 216 | +\end{table} | |
added
results/reproduced/boxcox_fine.csv
+22 −0
@@ -0,0 +1,22 @@ | ||
| 1 | +lambda,loglik,chosen,mdape_at_chosen | |
| 2 | +-0.25,-4938790.1592812985,False, | |
| 3 | +-0.2,-4928326.613673929,False, | |
| 4 | +-0.15,-4921149.036684653,False, | |
| 5 | +-0.1,-4911809.9533146825,False, | |
| 6 | +-0.05,-4905852.743838613,False, | |
| 7 | +-0.0,-4896133.282457808,False, | |
| 8 | +0.05,-4895821.141372284,False, | |
| 9 | +0.1,-4891760.972508729,False, | |
| 10 | +0.15,-4888345.89894094,False, | |
| 11 | +0.2,-4885580.902667309,False, | |
| 12 | +0.25,-4879097.21142742,False, | |
| 13 | +0.3,-4877690.751161514,False, | |
| 14 | +0.35,-4876918.904257714,True,18.38848861136389 | |
| 15 | +0.4,-4881082.917829343,False, | |
| 16 | +0.45,-4877369.476074193,False, | |
| 17 | +0.5,-4882782.367887354,False, | |
| 18 | +0.55,-4880462.865948945,False, | |
| 19 | +0.6,-4881961.43143256,False, | |
| 20 | +0.65,-4886168.783351108,False, | |
| 21 | +0.7,-4889986.164861776,False, | |
| 22 | +0.75,-4893437.953075144,False, | |
added
results/reproduced/by_quarter.csv
+36 −0
@@ -0,0 +1,36 @@ | ||
| 1 | +quarter,model,n,mdape | |
| 2 | +2025Q1,A5 Semi-log + quadratics,17031,20.16069824377374 | |
| 3 | +2025Q1,A6 Semi-log + splines,17031,18.55293878398712 | |
| 4 | +2025Q1,D2 Ridge (cross-validated),17031,18.97125923373471 | |
| 5 | +2025Q1,D3 Random forest,17031,17.15973919906848 | |
| 6 | +2025Q1,D4 Gradient boosting,17031,17.069787319853706 | |
| 7 | +2025Q2,A5 Semi-log + quadratics,34546,21.09424889763385 | |
| 8 | +2025Q2,A6 Semi-log + splines,34546,19.546001101493225 | |
| 9 | +2025Q2,D2 Ridge (cross-validated),34546,20.128071582190238 | |
| 10 | +2025Q2,D3 Random forest,34546,19.17503183093982 | |
| 11 | +2025Q2,D4 Gradient boosting,34546,19.056435785828757 | |
| 12 | +2025Q3,A5 Semi-log + quadratics,28097,22.81785959565014 | |
| 13 | +2025Q3,A6 Semi-log + splines,28097,21.26342871883413 | |
| 14 | +2025Q3,D2 Ridge (cross-validated),28097,21.787565277767396 | |
| 15 | +2025Q3,D3 Random forest,28097,20.640199696692722 | |
| 16 | +2025Q3,D4 Gradient boosting,28097,20.570509508119454 | |
| 17 | +2025Q4,A5 Semi-log + quadratics,24156,23.84990757727795 | |
| 18 | +2025Q4,A6 Semi-log + splines,24156,22.156927074847843 | |
| 19 | +2025Q4,D2 Ridge (cross-validated),24156,22.704586080935236 | |
| 20 | +2025Q4,D3 Random forest,24156,21.686460771814904 | |
| 21 | +2025Q4,D4 Gradient boosting,24156,21.604075720079475 | |
| 22 | +2026Q1,A5 Semi-log + quadratics,15611,24.828798398064173 | |
| 23 | +2026Q1,A6 Semi-log + splines,15611,23.136954110839774 | |
| 24 | +2026Q1,D2 Ridge (cross-validated),15611,23.555344810884428 | |
| 25 | +2026Q1,D3 Random forest,15611,22.809749906308703 | |
| 26 | +2026Q1,D4 Gradient boosting,15611,22.683544277340335 | |
| 27 | +2026Q2,A5 Semi-log + quadratics,30950,25.82329388343181 | |
| 28 | +2026Q2,A6 Semi-log + splines,30950,24.490104774430893 | |
| 29 | +2026Q2,D2 Ridge (cross-validated),30950,25.114690369909486 | |
| 30 | +2026Q2,D3 Random forest,30950,24.366584102876867 | |
| 31 | +2026Q2,D4 Gradient boosting,30950,24.313275442733904 | |
| 32 | +2026Q3,A5 Semi-log + quadratics,7997,25.960227538726045 | |
| 33 | +2026Q3,A6 Semi-log + splines,7997,24.87544352852322 | |
| 34 | +2026Q3,D2 Ridge (cross-validated),7997,25.158903757045657 | |
| 35 | +2026Q3,D3 Random forest,7997,24.996277735314912 | |
| 36 | +2026Q3,D4 Gradient boosting,7997,24.87390841981474 | |
added
results/reproduced/hgb_sensitivity.csv
+11 −0
@@ -0,0 +1,11 @@ | ||
| 1 | +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test,scheme | |
| 2 | +D4 Gradient boosting (baseline),S,14.388300681085338,27.766499817027036,0.3471568687492957,0.6422912683312305,13.136043786916385,28.339593011975516,0.3501358518462874,0.6361258693434015,1.0475010409695187,,,40.8,411283,102929,random | |
| 3 | +"D4 shallow (15 leaves, lr 0.05, 1500 iter)",S,14.719243860105477,28.154919687681947,0.35039008957492424,0.63559725098862,13.401236660872359,28.83674263008007,0.35391692116024087,0.6282245817693768,1.051807579516185,,,28.7,411283,102929,random | |
| 4 | +"D4 deep (255 leaves, lr 0.05, 600 iter)",S,14.289611237615919,27.61056923649104,0.34601169250653546,0.6446473438310576,13.117602210229256,28.098909164979524,0.34851529339791887,0.6394863630851599,1.043402259531175,,,121.5,411283,102929,random | |
| 5 | +"D4 regularised (min leaf 200, L2 = 1)",S,14.26217307195608,27.69234020959297,0.3467049485484332,0.6432219748599041,13.01108639038474,28.277180185855872,0.3498336625404618,0.636753690914108,1.0485848722394113,,,40.9,411283,102929,random | |
| 6 | +D4 early stopping on last 10% of training months,S,14.486038756539063,27.894520070852053,0.34801538571621565,0.6405198573634903,13.222127726623725,28.493460749856283,0.35117203837899774,0.6339689957921757,1.0488581267072552,,700.0,126.3,411283,102929,random | |
| 7 | +D4 Gradient boosting (baseline),S,21.21243467837447,29.71192710420828,0.37236811207724885,0.5214213758996993,18.362851253802294,28.605014412660662,0.3587539272757954,0.5557763676216326,1.0485637089645152,,,40.7,355824,158388,temporal | |
| 8 | +"D4 shallow (15 leaves, lr 0.05, 1500 iter)",S,21.41075901094633,29.814111901291206,0.37240688652603166,0.521321702550176,18.299953587404797,28.62964905180962,0.3577628776257503,0.5582272931995136,1.0532570669661585,,,28.7,355824,158388,temporal | |
| 9 | +"D4 deep (255 leaves, lr 0.05, 600 iter)",S,21.599331939568586,29.87909605759482,0.3738085572244618,0.5177116081480675,18.958152089809033,28.790557674838784,0.3604324382094834,0.55160984301614,1.0448264651027446,,,107.7,355824,158388,temporal | |
| 10 | +"D4 regularised (min leaf 200, L2 = 1)",S,21.53671243851688,29.763998225562823,0.37211824841062413,0.5220634249166158,18.523654776944497,28.560638550076707,0.3575714411737002,0.5586999457753024,1.0503781019780596,,,33.9,355824,158388,temporal | |
| 11 | +D4 early stopping on last 10% of training months,S,21.211205965026554,29.680715279436637,0.37128678154177347,0.5241968567695916,18.364288650180356,28.557949065565285,0.3576920591594777,0.5584021719866923,1.04834813285858,,1000.0,158.8,355824,158388,temporal | |
added
results/reproduced/horserace_group_coord.csv
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test | |
| 2 | +A1 Linear (levels),A. Functional form,18.389455311057525,35.77737768430297,0.42150872762642705,0.4729684614765287,18.389455311057525,35.77737768430297,0.42150872762642705,0.4729684614765287,,,,0.4,411263,102949 | |
| 3 | +A2 Semi-log,A. Functional form,19.45406141657741,33.53341061653312,0.3982220564706349,0.5295926664345789,18.534228750178155,35.515597429659906,0.40458544391763035,0.5144388071443484,1.0716405125795019,,,0.6,411263,102949 | |
| 4 | +A3 Log-log,A. Functional form,18.09423906311878,31.5648056784115,0.3795231689636993,0.5727323131971047,17.13689016366873,33.083726302727165,0.3848295353676579,0.5607009592529427,1.0637378029740763,,,0.8,411263,102949 | |
| 5 | +A4 Box-Cox,A. Functional form,18.34720115572638,33.91816652516567,0.39588027012021865,0.5351089578471608,18.01369176345465,35.11846758212266,0.4005581443205167,0.5240573822295058,,0.5,,4.4,411263,102949 | |
| 6 | +A5 Semi-log + quadratics,A. Functional form,19.047250871870013,32.633713129691856,0.38879112913610764,0.5516097539820513,18.060274251722916,34.383320754266194,0.39457920958428305,0.5381596654068279,1.0678021020773296,,,0.7,411263,102949 | |
| 7 | +A6 Semi-log + splines,A. Functional form,17.84160255200089,31.17235135800901,0.375404013662821,0.5819566827043615,16.719953387602672,32.5440257224398,0.38032165138598156,0.5709325567849641,1.06173604546364,,,2.3,411263,102949 | |
| 8 | +B1 No time effects,B. Time effects,22.020713610007295,36.07330397660628,0.41771180424287685,0.4824206287010825,21.320623105141713,38.615749714380435,0.42497748545191755,0.4642584779943505,1.0797929429062483,,,0.5,411263,102949 | |
| 9 | +B2 Year FE,B. Time effects,19.216445989382176,33.21932498640056,0.3943360634939339,0.5387286770024654,18.306678366552863,35.06233519240754,0.4003352095717885,0.5245870163050035,1.069862710157588,,,0.7,411263,102949 | |
| 10 | +B3 Quarter FE,B. Time effects,19.047250871870013,32.633713129691856,0.38879112913610764,0.5516097539820513,18.060274251722916,34.383320754266194,0.39457920958428305,0.5381596654068279,1.0678021020773296,,,0.7,411263,102949 | |
| 11 | +B4 Month FE,B. Time effects,19.398625101840388,33.282018368201285,0.39485525067670135,0.537513247647079,18.49481494077501,35.151184915530756,0.40092705375610255,0.5231803031914969,1.070168533613627,,,0.6,411263,102949 | |
| 12 | +C1 No spatial controls,C. Spatial controls,28.689547544675424,43.840616324962724,0.48386013502120434,0.30551455745412537,27.854471463760017,48.59662417250643,0.4957431862109964,0.27098415190902636,1.1117746881974258,,,0.5,411263,102949 | |
| 13 | +C2 Municipality FE,C. Spatial controls,19.047250871870013,32.633713129691856,0.38879112913610764,0.5516097539820513,18.060274251722916,34.383320754266194,0.39457920958428305,0.5381596654068279,1.0678021020773296,,,0.7,411263,102949 | |
| 14 | +C3 Grid-cell FE (~5.5 km),C. Spatial controls,18.750020280736805,32.64356649270374,0.38885559208588105,0.5514610522654697,17.681345309644815,34.2696279297555,0.3942232975475682,0.538992453333233,1.0671696493930205,,,0.7,411263,102949 | |
| 15 | +C4 Grid-cell FE (~1.1 km),C. Spatial controls,20.225376123786642,34.842860847732275,0.40626236607303856,0.5104053661049823,18.962836326256582,36.70306684092424,0.41241718088755186,0.49545842259587813,1.0720020497888785,,,0.8,411263,102949 | |
| 16 | +D1 OLS (muni + month FE),D. Estimation method,19.398625101840388,33.282018368201285,0.39485525067670135,0.537513247647079,18.49481494077501,35.151184915530756,0.40092705375610255,0.5231803031914969,1.070168533613627,,,0.6,411263,102949 | |
| 17 | +D2 Ridge (cross-validated),D. Estimation method,17.890529516201568,31.251732783077244,0.3763885157696496,0.5797611590637215,16.8893677800818,32.68394222313682,0.3814143837676687,0.5684634391789163,1.0622804510287405,,,10.1,411263,102949 | |
| 18 | +D3 Random forest,D. Estimation method,14.883188133292904,28.737382658558698,0.3573031529738494,0.6212983831644259,14.053333531225153,29.108973566446117,0.35869427463082987,0.6183437740317796,1.0318463155697535,,,8.4,411263,102949 | |
| 19 | +D4 Gradient boosting,D. Estimation method,14.59794086326427,28.210269818504578,0.3511407425669361,0.6342486746266742,13.424406660101518,28.85748490533166,0.35438418409430017,0.6274606736319336,1.047165791723387,,,42.1,411263,102949 | |
| 20 | +"D5 Gradient boosting, no coordinates",D. Estimation method,23.288069281216877,38.626422603992054,0.44226646657865465,0.4197816156316563,22.324919746724586,41.540491694908305,0.45047464198749365,0.39804481488791,1.086467333318932,,,20.8,411263,102949 | |
| 21 | +D6 Spatial k-NN comparables,D. Estimation method,22.263894535107575,37.95804725523757,0.4404191782505073,0.4246184833703136,22.194183511129985,38.13596309275768,0.44042512037220527,0.42460295720693186,1.0066905499496188,,,0.3,411263,102949 | |
added
results/reproduced/horserace_group_parcel.csv
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test | |
| 2 | +A1 Linear (levels),A. Functional form,18.394879964095797,35.96893018809802,0.42358366884898346,0.4688087852186674,18.394879964095797,35.96893018809802,0.42358366884898346,0.4688087852186674,,,,0.4,411369,102843 | |
| 3 | +A2 Semi-log,A. Functional form,19.40388065006023,33.60742551231478,0.3991437427562464,0.5283377491612558,18.459707672156842,35.527166516466494,0.40507307833118583,0.5142204489174887,1.0716280541158298,,,0.6,411369,102843 | |
| 4 | +A3 Log-log,A. Functional form,18.04901279537473,31.62346448383144,0.38036202721805884,0.5716815568957894,17.060785648509054,33.08835585008513,0.3853279508875882,0.5604244841914658,1.0636869814479837,,,0.9,411369,102843 | |
| 5 | +A4 Box-Cox,A. Functional form,18.24702711324504,34.006661029680444,0.3967848815219953,0.5338961387758547,17.86556740305991,35.46526823103805,0.40274905686877965,0.5197785744597814,,0.5,,4.4,411369,102843 | |
| 6 | +A5 Semi-log + quadratics,A. Functional form,19.19420945605029,33.23478458992403,0.3946076985401183,0.5389971863448217,18.178626389812674,35.017880574991494,0.40026491744886666,0.5256842765856499,1.0696108179421064,,,0.6,411369,102843 | |
| 7 | +A6 Semi-log + splines,A. Functional form,17.62071626632672,31.207237958155154,0.376000477877242,0.5814481534939215,16.60974843964176,32.54559297854598,0.38074837993673477,0.5708109860855053,1.0617470052055027,,,2.3,411369,102843 | |
| 8 | +B1 No time effects,B. Time effects,21.649567560303716,35.80598610682794,0.4155121629945933,0.4888599034728587,20.805949301729967,38.19532183877957,0.4222433482128658,0.47216510031035164,1.078199634331365,,,0.5,411369,102843 | |
| 9 | +B2 Year FE,B. Time effects,18.990446124535953,32.997820257572705,0.3921780438977923,0.5446566268137394,18.058282132701265,34.74906901015738,0.3977502821212147,0.5316252641639805,1.068487535996415,,,0.7,411369,102843 | |
| 10 | +B3 Quarter FE,B. Time effects,19.19420945605029,33.23478458992403,0.3946076985401183,0.5389971863448217,18.178626389812674,35.017880574991494,0.40026491744886666,0.5256842765856499,1.0696108179421064,,,0.6,411369,102843 | |
| 11 | +B4 Month FE,B. Time effects,19.332436660424253,33.33207533297025,0.3954934330533667,0.5369253343882607,18.402816630486033,35.160733297104976,0.40121192424899693,0.5234372068832924,1.0701374458919553,,,0.6,411369,102843 | |
| 12 | +C1 No spatial controls,C. Spatial controls,28.887143937365718,44.050532124221256,0.4859586541235692,0.30084879639246065,28.095541551675375,48.734151973096715,0.49707893152453125,0.2684850929794075,1.1115281859261688,,,0.4,411369,102843 | |
| 13 | +C2 Municipality FE,C. Spatial controls,19.19420945605029,33.23478458992403,0.3946076985401183,0.5389971863448217,18.178626389812674,35.017880574991494,0.40026491744886666,0.5256842765856499,1.0696108179421064,,,0.6,411369,102843 | |
| 14 | +C3 Grid-cell FE (~5.5 km),C. Spatial controls,18.73260474151463,32.799326462608065,0.38980765046740884,0.5501443438715339,17.53502943411876,34.40695892966617,0.3952679237455016,0.5374532717263956,1.0671611573206463,,,0.7,411369,102843 | |
| 15 | +C4 Grid-cell FE (~1.1 km),C. Spatial controls,21.097621825427847,35.95310914429783,0.4147863521944663,0.49064404850887544,19.553592455622823,37.9613459138871,0.4212809812757885,0.47456841574603015,1.0757010147272146,,,0.8,411369,102843 | |
| 16 | +D1 OLS (muni + month FE),D. Estimation method,19.332436660424253,33.33207533297025,0.3954934330533667,0.5369253343882607,18.402816630486033,35.160733297104976,0.40121192424899693,0.5234372068832924,1.0701374458919553,,,0.6,411369,102843 | |
| 17 | +D2 Ridge (cross-validated),D. Estimation method,17.70592620550421,31.251501981456574,0.3767730607223491,0.579726356784688,16.72857367165446,32.65172599817906,0.3815465570975194,0.569009649787606,1.062317122081801,,,10.2,411369,102843 | |
| 18 | +D3 Random forest,D. Estimation method,14.622820199263606,28.743565661551727,0.3582854740346365,0.6199586341211347,13.749624355032527,29.092018626347237,0.3596181757160165,0.6171261233153971,1.0317651338894,,,9.2,411369,102843 | |
| 19 | +D4 Gradient boosting,D. Estimation method,14.507935468925822,28.29190277596338,0.3525188079219081,0.6320938445206743,13.24585810355578,28.915575305176098,0.3556681265266408,0.6254909102009842,1.0478640275860414,,,43.0,411369,102843 | |
| 20 | +"D5 Gradient boosting, no coordinates",D. Estimation method,23.41157162533723,38.73698634397927,0.44364203470006636,0.4173096237076285,22.212467734820184,41.63446817144011,0.4514086798765861,0.39672923483220146,1.0877176588778272,,,17.5,411369,102843 | |
| 21 | +D6 Spatial k-NN comparables,D. Estimation method,21.308170377003776,37.246066218610544,0.43397656727509987,0.442422771348263,21.25721677508758,37.43458435469481,0.434126877127783,0.442036465398573,1.0066433972846687,,,0.3,411369,102843 | |
modified
results/reproduced/horserace_random.csv
+21 −21
@@ -1,21 +1,21 @@ | ||
| 1 | −name,group,mdape,mape,rmse_ln,r2_ln,seconds,n_train,n_test | |
| 2 | −A1 Linear (levels),A. Functional form,18.247735441468322,35.554833553436424,0.4197326280373289,0.4770941777730925,0.5,411283,102929 | |
| 3 | −A2 Semi-log,A. Functional form,18.174885439840832,34.4795393377222,0.39663369462465925,0.5330641412991447,0.6,411283,102929 | |
| 4 | −A3 Log-log,A. Functional form,16.965877929132013,32.69009595755662,0.38125191583247026,0.5685782006102371,0.9,411283,102929 | |
| 5 | −A4 Box-Cox,A. Functional form,18.567718342220903,33.001281736849975,0.3903031252718359,0.5478504801090737,4.3,411283,102929 | |
| 6 | −A5 Semi-log + quadratics,A. Functional form,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,0.7,411283,102929 | |
| 7 | −A6 Semi-log + splines,A. Functional form,16.52578223473714,32.10851818544987,0.37644836156803374,0.5793810460819198,2.6,411283,102929 | |
| 8 | −B1 No time effects,B. Time effects,21.1450312090911,38.37918215557233,0.42321207728034005,0.4683888002641269,0.5,411283,102929 | |
| 9 | −B2 Year FE,B. Time effects,18.091245071945412,34.663597249791685,0.39704996735836223,0.5320835152583053,0.7,411283,102929 | |
| 10 | −B3 Quarter FE,B. Time effects,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,0.6,411283,102929 | |
| 11 | −B4 Month FE,B. Time effects,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,0.7,411283,102929 | |
| 12 | −C1 No spatial controls,C. Spatial controls,27.922352076812572,48.39219298797349,0.4944639622666287,0.2743164045351353,0.5,411283,102929 | |
| 13 | −C2 Municipality FE,C. Spatial controls,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,0.7,411283,102929 | |
| 14 | −C3 Grid-cell FE (~5.5 km),C. Spatial controls,17.53978076473437,34.03632099310664,0.3918420758470441,0.5442778333854188,0.7,411283,102929 | |
| 15 | −C4 Grid-cell FE (~1.1 km),C. Spatial controls,19.389847159546143,37.50800513353381,0.41828640746256024,0.48069139310554865,0.8,411283,102929 | |
| 16 | −D1 OLS (muni + month FE),D. Estimation method,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,0.7,411283,102929 | |
| 17 | −D2 Ridge (cross-validated),D. Estimation method,16.671722823569887,32.2521071548757,0.37758629277068595,0.5768343018663071,10.8,411283,102929 | |
| 18 | −D3 Random forest,D. Estimation method,14.62750218203232,28.181960170119,0.35257338948901984,0.6310418755396041,7.6,411283,102929 | |
| 19 | −D4 Gradient boosting,D. Estimation method,14.388300681085338,27.766499817027036,0.3471568687492957,0.6422912683312305,42.5,411283,102929 | |
| 20 | −"D5 Gradient boosting, no coordinates",D. Estimation method,23.225090677534933,38.30004358013209,0.4398837230475839,0.4256801846835939,17.7,411283,102929 | |
| 21 | −D6 Spatial k-NN comparables,D. Estimation method,21.18226600985226,36.73270267320528,0.43669681006238664,0.43397181715912814,0.1,411283,102929 | |
| 1 | +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test | |
| 2 | +A1 Linear (levels),A. Functional form,18.247735441468322,35.554833553436424,0.4197326280373289,0.4770941777730925,18.247735441468322,35.554833553436424,0.4197326280373289,0.4770941777730925,,,,0.4,411283,102929 | |
| 3 | +A2 Semi-log,A. Functional form,19.034569938583708,32.66020106922096,0.39092568564912344,0.5464069104539773,18.174885439840832,34.4795393377222,0.39663369462465925,0.5330641412991447,1.0704523740369927,,,0.6,411283,102929 | |
| 4 | +A3 Log-log,A. Functional form,18.00422814105048,31.24431556205711,0.3762917738410556,0.5797308952207196,16.965877929132013,32.69009595755662,0.38125191583247026,0.5685782006102371,1.0640208207942232,,,0.9,411283,102929 | |
| 5 | +A4 Box-Cox,A. Functional form,18.567718342220903,33.001281736849975,0.3903031252718359,0.5478504801090737,17.931354368937967,34.39115498001405,0.3955659580558204,0.5355747370020278,,0.25,,5.1,411283,102929 | |
| 6 | +A5 Semi-log + quadratics,A. Functional form,19.15743025152905,32.8527013727642,0.3911358710025644,0.5459190210301912,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,1.0698913891782036,,,0.7,411283,102929 | |
| 7 | +A6 Semi-log + splines,A. Functional form,17.6216157169721,30.791561379112448,0.37171474821867573,0.5898926040038202,16.52578223473714,32.10851818544987,0.37644836156803374,0.5793810460819198,1.0620544125859206,,,2.6,411283,102929 | |
| 8 | +B1 No time effects,B. Time effects,22.038141454991422,35.894437273924325,0.4162524756898916,0.4857294257045265,21.1450312090911,38.37918215557233,0.42321207728034005,0.4683888002641269,1.0800144916543517,,,0.5,411283,102929 | |
| 9 | +B2 Year FE,B. Time effects,19.12497097548873,32.91021487912301,0.3914089242770463,0.5452848088000786,18.091245071945412,34.663597249791685,0.39704996735836223,0.5320835152583053,1.06983370962192,,,0.7,411283,102929 | |
| 10 | +B3 Quarter FE,B. Time effects,19.15743025152905,32.8527013727642,0.3911358710025644,0.5459190210301912,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,1.0698913891782036,,,0.7,411283,102929 | |
| 11 | +B4 Month FE,B. Time effects,19.342062726308214,33.01798132147254,0.39253988395462874,0.5426532514337992,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,1.0704345170610976,,,0.7,411283,102929 | |
| 12 | +C1 No spatial controls,C. Spatial controls,28.785665634746415,43.740789664459314,0.48315578380182944,0.30712900384495423,27.922352076812572,48.39219298797349,0.4944639622666287,0.2743164045351353,1.1117959257819008,,,0.4,411283,102929 | |
| 13 | +C2 Municipality FE,C. Spatial controls,19.15743025152905,32.8527013727642,0.3911358710025644,0.5459190210301912,18.057575256256637,34.597939297430145,0.3967501193167754,0.5327899798098671,1.0698913891782036,,,0.7,411283,102929 | |
| 14 | +C3 Grid-cell FE (~5.5 km),C. Spatial controls,18.703553400119855,32.45802918238758,0.38651653174174216,0.556581136808286,17.53978076473437,34.03632099310664,0.3918420758470441,0.5442778333854188,1.067449819592543,,,0.7,411283,102929 | |
| 15 | +C4 Grid-cell FE (~1.1 km),C. Spatial controls,20.962941599826827,35.57153971188526,0.41212679150885995,0.49587328501243466,19.389847159546143,37.50800513353381,0.41828640746256024,0.48069139310554865,1.0756884611181337,,,0.9,411283,102929 | |
| 16 | +D1 OLS (muni + month FE),D. Estimation method,19.342062726308214,33.01798132147254,0.39253988395462874,0.5426532514337992,18.322750827125915,34.80915561246471,0.3982300277671944,0.5292980206991789,1.0704345170610976,,,0.7,411283,102929 | |
| 17 | +D2 Ridge (cross-validated),D. Estimation method,17.684559394393904,30.874200209239945,0.372797514554587,0.5874999240199299,16.671722823569887,32.2521071548757,0.37758629277068595,0.5768343018663071,1.0626009803138499,,,10.9,411283,102929 | |
| 18 | +D3 Random forest,D. Estimation method,14.62750218203232,28.181960170119,0.35257338948901984,0.6310418755396041,13.68191573226618,28.50111285576457,0.3537359214449655,0.6286047502994752,1.0320010783517677,,,12.5,411283,102929 | |
| 19 | +D4 Gradient boosting,D. Estimation method,14.388300681085338,27.766499817027036,0.3471568687492957,0.6422912683312305,13.136043786916385,28.339593011975516,0.3501358518462874,0.6361258693434015,1.0475010409695187,,,46.7,411283,102929 | |
| 20 | +"D5 Gradient boosting, no coordinates",D. Estimation method,23.225090677534933,38.30004358013209,0.4398837230475839,0.4256801846835939,22.11744594443843,41.15658845539997,0.4476945241929057,0.4051032704375488,1.0876216107611343,,,17.3,411283,102929 | |
| 21 | +D6 Spatial k-NN comparables,D. Estimation method,21.18226600985226,36.73270267320528,0.43669681006238664,0.43397181715912814,21.128130305413727,36.898833705701435,0.436786904121283,0.43373824074782497,1.0056888675291784,,,0.3,411283,102929 | |
modified
results/reproduced/horserace_temporal.csv
+21 −21
@@ -1,21 +1,21 @@ | ||
| 1 | −name,group,mdape,mape,rmse_ln,r2_ln,seconds,n_train,n_test | |
| 2 | −A1 Linear (levels),A. Functional form,20.369511213481317,32.86005001443525,0.39728743801570204,0.4552239614410175,0.4,355824,158388 | |
| 3 | −A2 Semi-log,A. Functional form,20.611640132255836,33.09824319242239,0.3964345928808567,0.45756036006756484,0.5,355824,158388 | |
| 4 | −A3 Log-log,A. Functional form,19.189006525695262,30.749155508273613,0.3757555688314752,0.5126744490905257,0.8,355824,158388 | |
| 5 | −A4 Box-Cox,A. Functional form,20.957181720703545,31.940298396980698,0.38962419749983584,0.47603753989498354,3.8,355824,158388 | |
| 6 | −A5 Semi-log + quadratics,A. Functional form,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,0.6,355824,158388 | |
| 7 | −A6 Semi-log + splines,A. Functional form,19.097413080353302,30.49261711294047,0.37228529384601217,0.5216342331571537,2.1,355824,158388 | |
| 8 | −B1 No time effects,B. Time effects,25.384810383977687,33.162107080537304,0.41811180324070163,0.39661683797944225,0.5,355824,158388 | |
| 9 | −B2 Year FE,B. Time effects,21.006054587504984,32.62538772051225,0.39253658926914514,0.46817515745089067,0.6,355824,158388 | |
| 10 | −B3 Quarter FE,B. Time effects,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,0.6,355824,158388 | |
| 11 | −B4 Month FE,B. Time effects,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,0.6,355824,158388 | |
| 12 | −C1 No spatial controls,C. Spatial controls,26.45586672540444,41.108374756471996,0.45691339109589413,0.2794301211279968,0.4,355824,158388 | |
| 13 | −C2 Municipality FE,C. Spatial controls,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,0.6,355824,158388 | |
| 14 | −C3 Grid-cell FE (~5.5 km),C. Spatial controls,21.279243946393596,32.712624877555356,0.39371599730320833,0.464974534611328,0.6,355824,158388 | |
| 15 | −C4 Grid-cell FE (~1.1 km),C. Spatial controls,22.091478382056824,36.02185035008858,0.41700099046919065,0.39981863895351477,0.7,355824,158388 | |
| 16 | −D1 OLS (muni + month FE),D. Estimation method,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,0.6,355824,158388 | |
| 17 | −D2 Ridge (cross-validated),D. Estimation method,19.520136279141678,30.594351068378263,0.3747175016019065,0.5153633136379877,8.6,355824,158388 | |
| 18 | −D3 Random forest,D. Estimation method,21.34011982105225,29.76443050482997,0.37332513269156825,0.5189582317282835,6.8,355824,158388 | |
| 19 | −D4 Gradient boosting,D. Estimation method,21.21243467837447,29.71192710420828,0.37236811207724885,0.5214213758996993,41.5,355824,158388 | |
| 20 | −"D5 Gradient boosting, no coordinates",D. Estimation method,26.207233073287274,35.736727341535484,0.44725743322834494,0.3095639406099918,21.1,355824,158388 | |
| 21 | −D6 Spatial k-NN comparables,D. Estimation method,25.63120139114109,34.234843881635754,0.4653714833711972,0.2525057333957209,0.1,355824,158388 | |
| 1 | +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test | |
| 2 | +A1 Linear (levels),A. Functional form,20.369511213481317,32.86005001443525,0.39728743801570204,0.4552239614410175,20.369511213481317,32.86005001443525,0.39728743801570204,0.4552239614410175,,,,0.4,355824,158388 | |
| 3 | +A2 Semi-log,A. Functional form,23.154885105735325,32.88199068728801,0.4078149103607638,0.42597007752801586,20.611640132255836,33.09824319242239,0.3964345928808567,0.45756036006756484,1.076793566376698,,,0.5,355824,158388 | |
| 4 | +A3 Log-log,A. Functional form,21.769614210046917,31.036455799131453,0.38771693156325343,0.4811547262154816,19.189006525695262,30.749155508273613,0.3757555688314752,0.5126744490905257,1.0655530599282332,,,0.8,355824,158388 | |
| 5 | +A4 Box-Cox,A. Functional form,20.957181720703545,31.940298396980698,0.38962419749983584,0.47603753989498354,19.571597393733516,31.96261263566212,0.3844157306522595,0.48995248826084103,,0.5,,4.0,355824,158388 | |
| 6 | +A5 Semi-log + quadratics,A. Functional form,23.30291738895638,32.33980518990814,0.4022452672417709,0.4415423854252667,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,1.0711855946796898,,,0.6,355824,158388 | |
| 7 | +A6 Semi-log + splines,A. Functional form,21.722504213272607,30.87403330589372,0.384133203104169,0.4907019347955833,19.097413080353302,30.49261711294047,0.37228529384601217,0.5216342331571537,1.06346689326762,,,2.0,355824,158388 | |
| 8 | +B1 No time effects,B. Time effects,29.296772516609654,34.68364172805182,0.4503543461363537,0.29996936286225007,25.384810383977687,33.162107080537304,0.41811180324070163,0.39661683797944225,1.0779549442640406,,,0.5,355824,158388 | |
| 9 | +B2 Year FE,B. Time effects,23.651696383010147,32.69839717790235,0.40614244890268997,0.43066865187652315,21.006054587504984,32.62538772051225,0.39253658926914514,0.46817515745089067,1.0741977461673156,,,0.6,355824,158388 | |
| 10 | +B3 Quarter FE,B. Time effects,23.30291738895638,32.33980518990814,0.4022452672417709,0.4415423854252667,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,1.0711855946796898,,,0.6,355824,158388 | |
| 11 | +B4 Month FE,B. Time effects,22.8719130040602,32.42550461740692,0.400924502771694,0.4452037338739415,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,1.0724108768578227,,,0.6,355824,158388 | |
| 12 | +C1 No spatial controls,C. Spatial controls,29.015774167241894,38.87352353861925,0.4685413801986148,0.24228787650947192,26.45586672540444,41.108374756471996,0.45691339109589413,0.2794301211279968,1.1188483034696548,,,0.4,355824,158388 | |
| 13 | +C2 Municipality FE,C. Spatial controls,23.30291738895638,32.33980518990814,0.4022452672417709,0.4415423854252667,20.4747492500219,32.08431941207347,0.3884352669819105,0.47923038313059785,1.0711855946796898,,,0.6,355824,158388 | |
| 14 | +C3 Grid-cell FE (~5.5 km),C. Spatial controls,24.2625431905187,33.00942406765742,0.40874325687600593,0.42335366891858794,21.279243946393596,32.712624877555356,0.39371599730320833,0.464974534611328,1.0720367816988154,,,0.6,355824,158388 | |
| 15 | +C4 Grid-cell FE (~1.1 km),C. Spatial controls,25.324930433263678,35.50206863036151,0.42919028135360904,0.3642182032203949,22.091478382056824,36.02185035008858,0.41700099046919065,0.39981863895351477,1.0921887847980092,,,0.6,355824,158388 | |
| 16 | +D1 OLS (muni + month FE),D. Estimation method,22.8719130040602,32.42550461740692,0.400924502771694,0.4452037338739415,20.347788061301657,32.47788422275795,0.38952457659313394,0.4763054439041552,1.0724108768578227,,,0.6,355824,158388 | |
| 17 | +D2 Ridge (cross-validated),D. Estimation method,22.257318062238202,31.071606388474528,0.38804376521331146,0.480279615769711,19.520136279141678,30.594351068378263,0.3747175016019065,0.5153633136379877,1.0640215124102546,,,8.2,355824,158388 | |
| 18 | +D3 Random forest,D. Estimation method,21.340119821052188,29.76443050482997,0.37332513269156825,0.5189582317282835,19.394248052512626,28.951513572891475,0.36347962322867167,0.5439961919671212,1.0324597692592685,,,10.7,355824,158388 | |
| 19 | +D4 Gradient boosting,D. Estimation method,21.21243467837447,29.71192710420828,0.37236811207724885,0.5214213758996993,18.362851253802294,28.605014412660662,0.3587539272757954,0.5557763676216326,1.0485637089645152,,,41.3,355824,158388 | |
| 20 | +"D5 Gradient boosting, no coordinates",D. Estimation method,26.207233073287274,35.736727341535484,0.44725743322834494,0.3095639406099918,23.004852680221028,35.9605098787384,0.42968275070306156,0.3627583254241229,1.0912898778385383,,,21.7,355824,158388 | |
| 21 | +D6 Spatial k-NN comparables,D. Estimation method,25.63120139114109,34.234843881635754,0.4653714833711972,0.2525057333957209,25.38964897166118,34.13469069386835,0.46349083531773794,0.25853503771828157,1.0043584323305443,,,0.2,355824,158388 | |
added
results/reproduced/inference_random.csv
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +model,mdape,ci_lo,ci_hi,boot_se,diff_vs_ref,diff_lo,diff_hi,diff_vs_d4,diff_d4_lo,diff_d4_hi,dm_ape_vs_a6_t,dm_ape_vs_a6_p,dm_ape_vs_a6_diff,dm_sqlog_vs_a6_t,dm_sqlog_vs_a6_p,dm_sqlog_vs_a6_diff,dm_ape_vs_d4_t,dm_ape_vs_d4_p,dm_ape_vs_d4_diff,dm_sqlog_vs_d4_t,dm_sqlog_vs_d4_p,dm_sqlog_vs_d4_diff,n_clusters | |
| 2 | +A1 Linear (levels),18.247735441468322,16.9113510560629,19.60250248921421,0.6798156719309989,0.6261197244962204,0.13177818927938997,1.1288670458109282,3.8594347603829835,2.3810890282818233,5.572402821114919,12.297632954401742,0.0,0.04763272174323981,7.51790454474203,5.573319583618286e-14,0.03800362499584921,10.403881445314044,0.0,0.077883337364094,11.88110067074241,0.0,0.05565758751930701,1091 | |
| 3 | +A2 Semi-log,19.034569938583708,17.948468947394765,20.177409718978527,0.5933974925595471,1.4129542216116064,1.0925363990812282,1.8335429697484,4.6462692574983695,3.410686792542463,6.182775708649027,10.174542669661014,0.0,0.01868639690108518,8.898715388706915,0.0,0.014651037656963798,5.814778194584789,6.071433578114238e-09,0.04893701252193937,4.763341767974322,1.9041293313204477e-06,0.03230500018042161,1091 | |
| 4 | +A3 Log-log,18.00422814105048,17.07752319183666,18.977226098783678,0.5097956635710524,0.3826124240783777,0.28563151196801645,0.48671649465618905,3.6159274599651408,2.268004875035067,5.04018984221415,4.640377326082088,3.477735786239222e-06,0.0045275418294466125,3.28780657498537,0.001009711859182305,0.0034236450171746756,3.7605346164843136,0.00016955060891321594,0.034778157450300794,3.1024469169219993,0.0019192796548725966,0.02107760754063249,1091 | |
| 5 | +A4 Box-Cox,18.567718342220903,17.46589758655213,19.707557217076772,0.5895493292913278,0.9461026252488018,0.6659357616138547,1.3168750827110465,4.179417661135565,2.9293738028577057,5.753351071060692,10.694909755887176,0.0,0.02209720357737525,10.463728364076685,0.0,0.014164675553688949,6.2558305116106006,3.954059302202495e-10,0.05234781919822943,5.180699999292202,2.2105477048839361e-07,0.03181863807714676,1091 | |
| 6 | +A5 Semi-log + quadratics,19.15743025152905,18.201941036517315,20.15445926287562,0.5026923733244107,1.5358145345569483,1.1398198349247033,2.1391093582834935,4.769129570443711,3.684746880089903,5.930644511229821,5.530637638741666,3.190688269150144e-08,0.020611399936517463,6.846564545749681,7.564393555981042e-12,0.014815415541661218,8.696091789215284,0.0,0.050862015557371645,7.103680897159478,1.2148060335448463e-12,0.03246937806511903,1091 | |
| 7 | +A6 Semi-log + splines,17.6216157169721,16.733927881204792,18.583767461456866,0.4920101568426174,,,,3.233315035886763,1.8746102788184382,4.657062294398633,,,,,,,3.5997911983137305,0.0003184728086869093,0.030250615620854186,3.021002919921985,0.002519389304698194,0.017653962523457807,1091 | |
| 8 | +B1 No time effects,22.038141454991422,21.08712269256635,23.267211253160106,0.5612291624566031,4.41652573801932,3.4200841630827816,5.582948377502827,7.649840773906083,6.748344719706512,8.338009838204949,8.206346390560523,2.220446049250313e-16,0.05102875894811877,7.860550382221196,3.774758283725532e-15,0.035094269474690284,19.848983759841108,0.0,0.08127937456897295,17.415570392509096,0.0,0.0527482319981481,1091 | |
| 9 | +B2 Year FE,19.12497097548873,18.135266091728134,20.16483252720224,0.5420996135180365,1.5033552585166277,1.0652509808425514,2.1819410747332237,4.736670294403391,3.6133353152983663,5.9326990304884655,6.124427672343136,9.101013276335834e-10,0.021186535000105613,7.1455155771433745,8.966161146872764e-13,0.015029091960441074,8.76902162424838,0.0,0.0514371506209598,7.29281506461723,3.035349749325178e-13,0.03268305448389889,1091 | |
| 10 | +B3 Quarter FE,19.15743025152905,18.201941036517315,20.15445926287562,0.5026923733244107,1.5358145345569483,1.1398198349247033,2.1391093582834935,4.769129570443711,3.684746880089903,5.930644511229821,5.530637638741666,3.190688269150144e-08,0.020611399936517463,6.846564545749681,7.564393555981042e-12,0.014815415541661218,8.696091789215284,0.0,0.050862015557371645,7.103680897159478,1.2148060335448463e-12,0.03246937806511903,1091 | |
| 11 | +B4 Month FE,19.342062726308214,18.41666765136561,20.381917170225584,0.5186977109248657,1.7204470093361124,1.2291247555472724,2.4534949420758574,4.953762045222875,3.8934157966761633,6.07316437462031,5.731933731822563,9.929206790104672e-09,0.02226419942360097,6.544042648781664,5.987765838710857e-11,0.0159157064518399,9.414342745938619,0.0,0.052514815044455154,7.726539400085093,1.1102230246251565e-14,0.033569668975297715,1091 | |
| 12 | +C1 No spatial controls,28.785665634746415,27.341346218160815,30.074976710188,0.7025099701791481,11.164049917774314,10.142872490107974,12.280345112224204,14.397364953661077,12.691537820133453,15.890935706600027,7.742609666008332,9.769962616701378e-15,0.12949228285346864,15.131594441078725,0.0,0.09526765737788669,14.938137987865796,0.0,0.15974289847432283,13.859739811417462,0.0,0.1129216199013445,1091 | |
| 13 | +C2 Municipality FE,19.15743025152905,18.201941036517315,20.15445926287562,0.5026923733244107,1.5358145345569483,1.1398198349247033,2.1391093582834935,4.769129570443711,3.684746880089903,5.930644511229821,5.530637638741666,3.190688269150144e-08,0.020611399936517463,6.846564545749681,7.564393555981042e-12,0.014815415541661218,8.696091789215284,0.0,0.050862015557371645,7.103680897159478,1.2148060335448463e-12,0.03246937806511903,1091 | |
| 14 | +C3 Grid-cell FE (~5.5 km),18.703553400119855,17.78729355112193,20.032142200057542,0.5678254955514357,1.0819376831477534,0.21901537947672517,2.308285560296583,4.3152527190345165,3.6452950369483923,5.089221243343746,2.5295453955100333,0.011421039859825566,0.01666467803275125,2.356369638453933,0.01845454380735978,0.01122317526639169,14.03126557996909,0.0,0.04691529365360543,13.706499864516653,0.0,0.0288771377898495,1091 | |
| 15 | +C4 Grid-cell FE (~1.1 km),20.962941599826827,20.085903076339296,21.854065484343828,0.4556213754459016,3.3413258828547256,2.8904906643605672,3.837012817774882,6.574640918741489,5.336615577939174,7.744405878759988,6.7530133957275,1.4480638910185917e-11,0.04779978332772806,7.5711644369648194,3.68594044175552e-14,0.03167663823611381,13.14987798988361,0.0,0.07805039894858226,12.025634909259105,0.0,0.049330600759571616,1091 | |
| 16 | +D1 OLS (muni + month FE),19.342062726308214,18.41666765136561,20.381917170225584,0.5186977109248657,1.7204470093361124,1.2291247555472724,2.4534949420758574,4.953762045222875,3.8934157966761633,6.07316437462031,5.731933731822563,9.929206790104672e-09,0.02226419942360097,6.544042648781664,5.987765838710857e-11,0.0159157064518399,9.414342745938619,0.0,0.052514815044455154,7.726539400085093,1.1102230246251565e-14,0.033569668975297715,1091 | |
| 17 | +D2 Ridge (cross-validated),17.684559394393904,16.768330917331383,18.6836218961783,0.5035043504152524,0.06294367742180285,-0.01676159248276264,0.1637807429146053,3.296258713308566,1.9811849496440535,4.724217428807863,2.6118019768463383,0.009006640622825302,0.0008263883012749337,2.7683584874087472,0.005633944669363622,0.0008061328148040056,3.6535115239679725,0.0002586781730240517,0.03107700392212912,3.060857865195947,0.002207038376347503,0.018460095338261817,1091 | |
| 18 | +D3 Random forest,14.62750218203232,13.737999221508268,16.19555708116927,0.6178221405912262,-2.994113534939782,-4.555739590156278,-1.4648756603277908,0.23920150094698123,0.0513885363794455,0.4588555165642077,-2.85524195730287,0.00430040529118747,-0.02609601208993448,-2.205886325821924,0.027391959893569684,-0.01386385906749741,4.672835254507822,2.970700491378153e-06,0.004154603530919706,6.693591526753016,2.177591440499782e-11,0.003790103455960403,1091 | |
| 19 | +D4 Gradient boosting,14.388300681085338,13.681350877219709,15.743536219678433,0.5263969787549507,-3.233315035886763,-4.657062294398633,-1.8746102788184382,,,,-3.5997911983137305,0.0003184728086869093,-0.030250615620854186,-3.021002919921985,0.002519389304698194,-0.017653962523457807,,,,,,,1091 | |
| 20 | +"D5 Gradient boosting, no coordinates",23.225090677534933,22.138529223958216,25.069137709119865,0.7566123985799357,5.603474960562831,3.9352537791374327,7.507040455957043,8.836789996449594,8.10748542572112,9.787971484731155,4.50536824487449,6.625793987247164e-06,0.07508482201019637,6.433439358856359,1.2474843380516631e-10,0.055325835758930006,10.594870291718431,0.0,0.10533543763105055,16.137980864260527,0.0,0.07297979828238782,1091 | |
| 21 | +D6 Spatial k-NN comparables,21.18226600985226,19.390028219965146,23.548071796694504,1.100889921651293,3.560650292880158,0.9519338843862455,5.8622156243977095,6.793965328766921,5.527326690571327,8.03181255741793,4.798111691143396,1.6016846118294126e-06,0.05941141294092833,5.960580367571443,2.513436569984151e-09,0.05253224987539075,19.297132345753973,0.0,0.08966202856178251,20.248240203175722,0.0,0.07018621239884855,1091 | |
added
results/reproduced/inference_temporal.csv
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +model,mdape,ci_lo,ci_hi,boot_se,diff_vs_ref,diff_lo,diff_hi,diff_vs_d4,diff_d4_lo,diff_d4_hi,dm_ape_vs_a6_t,dm_ape_vs_a6_p,dm_ape_vs_a6_diff,dm_sqlog_vs_a6_t,dm_sqlog_vs_a6_p,dm_sqlog_vs_a6_diff,dm_ape_vs_d4_t,dm_ape_vs_d4_p,dm_ape_vs_d4_diff,dm_sqlog_vs_d4_t,dm_sqlog_vs_d4_p,dm_sqlog_vs_d4_diff,n_clusters | |
| 2 | +A1 Linear (levels),20.369511213481317,19.050772109018073,21.69348139159165,0.6849601233930911,-1.3529929997912902,-2.2770702594184407,-0.42539919706467916,-0.8429234648931541,-2.5898121968575603,1.1895847489303906,6.317663783346769,2.6554691778812867e-10,0.01986016708541528,6.164003649673723,7.092830767163605e-10,0.010278990678011518,3.520049279845853,0.0004314666251414856,0.03148122910226971,3.9079109150864686,9.309761836728825e-05,0.0191792975131057,1071 | |
| 3 | +A2 Semi-log,23.154885105735325,21.6887719627368,24.674870127680457,0.7875237764863192,1.4323808924627173,0.7670651228422551,2.2976528083716783,1.9424504273608534,0.5295622515203463,3.4375141515059062,7.796357304870725,6.439293542825908e-15,0.020079573813942913,7.223963685447007,5.049294315995212e-13,0.01875468338548904,3.965893940282591,7.312137370973915e-05,0.03170063583079734,5.055071810178792,4.3022885698462687e-07,0.02765499022058323,1071 | |
| 4 | +A3 Log-log,21.769614210046917,20.51940280719328,23.227311337108237,0.6739267745802049,0.04710999677431005,-0.04315938950036894,0.18763013229223433,0.5571795316724462,-0.5423402505386862,1.8391490439319191,1.804661231143382,0.07112771002474405,0.0016242249323772874,2.543121816787227,0.010986691828438477,0.0027661012937557784,1.646327603990528,0.09969632924843053,0.013245286949231714,2.098866610291465,0.03582866095871928,0.011666408128849963,1071 | |
| 5 | +A4 Box-Cox,20.957181720703545,19.742168516904147,22.156377541760673,0.6381565190731153,-0.7653224925690623,-1.4473333084469813,-0.09482024347911353,-0.25525295767092615,-1.7949341953230935,1.4690422686693982,4.264381230833118,2.0045693571724144e-05,0.010662650910869802,2.2871023500844068,0.022189849754430613,0.004248697550322327,2.5043936657330677,0.012266146996653404,0.022283712927724228,2.2903116877035994,0.02200325477123455,0.01314900438541651,1071 | |
| 6 | +A5 Semi-log + quadratics,23.30291738895638,21.869038097481326,24.922485533073313,0.8102019586928445,1.5804131756837734,1.0783669250409043,2.22065102463182,2.0904827105819095,0.9486003551587855,3.181577878834679,6.065811464275289,1.3128906850568e-09,0.014657718840144185,7.152656593716022,8.51096970677645e-13,0.014242937291334905,4.603816313944034,4.148183673891026e-06,0.026278780856998613,6.444665276370954,1.1585576942252374e-10,0.023143244126429093,1071 | |
| 7 | +A6 Semi-log + splines,21.722504213272607,20.45126930153131,23.100552763771457,0.6610274998462855,,,,0.5100695348981361,-0.5516760349102061,1.677839052305929,,,,,,,1.615823456025302,0.1061324886702475,0.011621062016854429,1.9765879301563614,0.04808822757633413,0.008900306835094185,1071 | |
| 8 | +B1 No time effects,29.296772516609654,27.25307447396948,31.453026176290106,1.0868368852006105,7.574268303337046,6.039750998516674,9.027043210519999,8.084337838235182,7.297235701980839,9.07468513889715,5.307042338766459,1.1141830524863394e-07,0.03809608422158093,9.288145872580824,0.0,0.05526071935683395,19.75478536885888,0.0,0.04971714623843536,16.92421738378873,0.0,0.06416102619192815,1071 | |
| 9 | +B2 Year FE,23.651696383010147,22.215432187910814,25.268875459898798,0.8096188915960055,1.92919216973754,1.3131800189892668,2.7530865495071235,2.439261704635676,1.3173521256992164,3.562209825093329,6.1579767061896264,7.368019527831393e-10,0.018243638720086247,6.017796983225028,1.7680674702091892e-09,0.017393371073605394,5.139555821884859,2.7538867297671743e-07,0.029864700736940673,7.23985001734488,4.4919623576333834e-13,0.026293677908699582,1071 | |
| 10 | +B3 Quarter FE,23.30291738895638,21.869038097481326,24.922485533073313,0.8102019586928445,1.5804131756837734,1.0783669250409043,2.22065102463182,2.0904827105819095,0.9486003551587855,3.181577878834679,6.065811464275289,1.3128906850568e-09,0.014657718840144185,7.152656593716022,8.51096970677645e-13,0.014242937291334905,4.603816313944034,4.148183673891026e-06,0.026278780856998613,6.444665276370954,1.1585576942252374e-10,0.023143244126429093,1071 | |
| 11 | +B4 Month FE,22.8719130040602,21.532938038433826,24.438984492545977,0.7619484470830038,1.149408790787593,0.6893140083694228,1.8098306190197673,1.659478325685729,0.5920810512262819,2.7576763092148138,5.991209061016688,2.082866767949554e-09,0.015514713115132013,5.990884007795864,2.0870345451839967e-09,0.013182139195661349,4.567473208327871,4.936387487219918e-06,0.027135775131986443,6.024141823255467,1.7000925112142795e-09,0.02208244603075553,1071 | |
| 12 | +C1 No spatial controls,29.015774167241894,27.422211588155974,30.325868927192477,0.7462222947018358,7.293269953969286,4.96861955562622,9.193766804289842,7.803339488867422,4.996711778534667,10.324147123803586,11.088323777905881,0.0,0.07999490232725531,7.508284601340977,5.995204332975845e-14,0.07197270723135413,17.010072661797665,0.0,0.09161596434410973,6.338419778689569,2.3213364563900996e-10,0.08087301406644831,1071 | |
| 13 | +C2 Municipality FE,23.30291738895638,21.869038097481326,24.922485533073313,0.8102019586928445,1.5804131756837734,1.0783669250409043,2.22065102463182,2.0904827105819095,0.9486003551587855,3.181577878834679,6.065811464275289,1.3128906850568e-09,0.014657718840144185,7.152656593716022,8.51096970677645e-13,0.014242937291334905,4.603816313944034,4.148183673891026e-06,0.026278780856998613,6.444665276370954,1.1585576942252374e-10,0.023143244126429093,1071 | |
| 14 | +C3 Grid-cell FE (~5.5 km),24.2625431905187,22.503220945710353,26.363285194699493,1.0301736045564578,2.540038977246091,1.5785786294135338,3.4839259747851203,3.0501085121442273,2.0572172007955043,3.9118766585912756,4.503146745467939,6.695461754135934e-06,0.02135390761763703,4.450105287824345,8.582820140379255e-06,0.019512732314535812,8.314219574865609,0.0,0.03297496963449146,12.3741576283692,0.0,0.028413039149629993,1071 | |
| 15 | +C4 Grid-cell FE (~1.1 km),25.324930433263678,24.251360863509277,26.29352775446495,0.49390977597366215,3.60242621999107,2.1189111166682935,4.91703661788902,4.112495754889206,2.25480020889692,5.928810388694186,12.43144220323286,0.0,0.046280353244677816,9.969202355421647,0.0,0.03664597988132135,7.149992224263942,8.677503160470224e-13,0.057901415261532245,6.861875584202508,6.796341267545358e-12,0.045546286716415536,1071 | |
| 16 | +D1 OLS (muni + month FE),22.8719130040602,21.532938038433826,24.438984492545977,0.7619484470830038,1.149408790787593,0.6893140083694228,1.8098306190197673,1.659478325685729,0.5920810512262819,2.7576763092148138,5.991209061016688,2.082866767949554e-09,0.015514713115132013,5.990884007795864,2.0870345451839967e-09,0.013182139195661349,4.567473208327871,4.936387487219918e-06,0.027135775131986443,6.024141823255467,1.7000925112142795e-09,0.02208244603075553,1071 | |
| 17 | +D2 Ridge (cross-validated),22.257318062238202,20.96191993033077,23.668021827486456,0.6692303373351438,0.5348138489655945,0.48101405213608955,0.6088791910137628,1.0448833838637306,0.0012400068282902282,2.185150952674026,6.580369386315953,4.692801702788074e-11,0.0019757308258080398,16.00748490968894,0.0,0.0030196459938548717,1.9584584383888557,0.05017624333623605,0.01359679284266247,2.647413747037119,0.008111004795629206,0.011919952828949056,1071 | |
| 18 | +D3 Random forest,21.340119821052188,19.907687448517617,22.980295616439495,0.8411843763724454,-0.38238439222041976,-1.5015448169575225,0.5571785755217394,0.12768514267771636,-0.14568791249818178,0.36811779443404413,-1.568790205531711,0.11669683137166498,-0.011096028010637537,-1.9109037795897155,0.05601694727646822,-0.008186663027891686,0.9790991544458619,0.3275309894032459,0.0005250340062168912,1.2895287248262932,0.19721433670042376,0.0007136438072025015,1071 | |
| 19 | +D4 Gradient boosting,21.21243467837447,19.70747318840484,22.945391194003907,0.8986037102049517,-0.5100695348981361,-1.6778390523059303,0.551676034910206,,,,-1.615823456025302,0.1061324886702475,-0.011621062016854429,-1.9765879301563614,0.04808822757633413,-0.008900306835094185,,,,,,,1071 | |
| 20 | +"D5 Gradient boosting, no coordinates",26.207233073287274,25.23007220277521,27.506849996204814,0.608040527286014,4.484728860014666,2.8779728042965105,6.210941230892228,4.994798394912802,3.1215058615470648,6.500510794984376,5.875060368764621,4.2268928357458435e-09,0.04862694035641765,9.378346817879406,0.0,0.05248089385093869,13.890923788390463,0.0,0.06024800237327208,11.078685604065148,0.0,0.06138120068603288,1071 | |
| 21 | +D6 Spatial k-NN comparables,25.63120139114109,23.14828800408425,28.333593401968223,1.407855973507387,3.9086971778684827,1.7817973469367552,5.637733529113384,4.418766712766619,3.332669523407553,5.470721729412763,4.036460402580431,5.4263679257982034e-05,0.0336081057574204,7.832489620597485,4.884981308350689e-15,0.0690122998080397,26.494126476851566,0.0,0.04522916777427483,15.848240562584424,0.0,0.0779126066431339,1071 | |
added
results/reproduced/leakage.csv
+37 −0
@@ -0,0 +1,37 @@ | ||
| 1 | +key,seen_in_training,segment,n,model,mdape | |
| 2 | +in_train_parcel,True,all,17219,A5 Semi-log + quadratics,19.786608950655705 | |
| 3 | +in_train_parcel,True,all,17219,A6 Semi-log + splines,18.27278654110786 | |
| 4 | +in_train_parcel,True,all,17219,D2 Ridge (cross-validated),18.21506909664557 | |
| 5 | +in_train_parcel,True,all,17219,D3 Random forest,14.568143855697022 | |
| 6 | +in_train_parcel,True,all,17219,D4 Gradient boosting,14.494590514304903 | |
| 7 | +in_train_parcel,True,all,17219,"D5 Gradient boosting, no coordinates",22.887248058209508 | |
| 8 | +in_train_parcel,True,all,17219,D6 Spatial k-NN comparables,22.21141825253498 | |
| 9 | +in_train_parcel,True,condo,3374,A5 Semi-log + quadratics,17.414213423610217 | |
| 10 | +in_train_parcel,True,condo,3374,D4 Gradient boosting,9.035672175351072 | |
| 11 | +in_train_parcel,False,all,85710,A5 Semi-log + quadratics,19.05005201781079 | |
| 12 | +in_train_parcel,False,all,85710,A6 Semi-log + splines,17.48581012929675 | |
| 13 | +in_train_parcel,False,all,85710,D2 Ridge (cross-validated),17.576292121577303 | |
| 14 | +in_train_parcel,False,all,85710,D3 Random forest,14.639597756778661 | |
| 15 | +in_train_parcel,False,all,85710,D4 Gradient boosting,14.368856147710904 | |
| 16 | +in_train_parcel,False,all,85710,"D5 Gradient boosting, no coordinates",23.28075103236257 | |
| 17 | +in_train_parcel,False,all,85710,D6 Spatial k-NN comparables,20.990025690392308 | |
| 18 | +in_train_parcel,False,condo,11474,A5 Semi-log + quadratics,18.43144280197513 | |
| 19 | +in_train_parcel,False,condo,11474,D4 Gradient boosting,9.080011346798681 | |
| 20 | +in_train_coord,True,all,27475,A5 Semi-log + quadratics,19.16845806222931 | |
| 21 | +in_train_coord,True,all,27475,A6 Semi-log + splines,17.776290704974276 | |
| 22 | +in_train_coord,True,all,27475,D2 Ridge (cross-validated),17.859099147831706 | |
| 23 | +in_train_coord,True,all,27475,D3 Random forest,11.74898846453675 | |
| 24 | +in_train_coord,True,all,27475,D4 Gradient boosting,11.994995797797987 | |
| 25 | +in_train_coord,True,all,27475,"D5 Gradient boosting, no coordinates",20.04914260589526 | |
| 26 | +in_train_coord,True,all,27475,D6 Spatial k-NN comparables,16.652769091793402 | |
| 27 | +in_train_coord,True,condo,13261,A5 Semi-log + quadratics,17.97470023475135 | |
| 28 | +in_train_coord,True,condo,13261,D4 Gradient boosting,8.934702433522762 | |
| 29 | +in_train_coord,False,all,75454,A5 Semi-log + quadratics,19.15384674472759 | |
| 30 | +in_train_coord,False,all,75454,A6 Semi-log + splines,17.555794642798645 | |
| 31 | +in_train_coord,False,all,75454,D2 Ridge (cross-validated),17.625875167760007 | |
| 32 | +in_train_coord,False,all,75454,D3 Random forest,15.749377923105962 | |
| 33 | +in_train_coord,False,all,75454,D4 Gradient boosting,15.321150846402826 | |
| 34 | +in_train_coord,False,all,75454,"D5 Gradient boosting, no coordinates",24.512690151286638 | |
| 35 | +in_train_coord,False,all,75454,D6 Spatial k-NN comparables,22.907703956528476 | |
| 36 | +in_train_coord,False,condo,1587,A5 Semi-log + quadratics,20.34309569440084 | |
| 37 | +in_train_coord,False,condo,1587,D4 Gradient boosting,10.243202880340052 | |
modified
results/reproduced/learning.csv
+6 −6
@@ -1,13 +1,13 @@ | ||
| 1 | 1 | n_train,model,mdape,mape,rmse_ln,r2_ln |
| 2 | −10000,OLS quadratic (muni+quarter FE),18.38436183742385,34.439941317616764,0.39722768960842764,0.531664536345561 | |
| 2 | +10000,OLS quadratic (muni+quarter FE),19.136226571989205,32.82348826216586,0.39279102092321205,0.5420678667536525 | |
| 3 | 3 | 10000,Gradient boosting,17.55817415103931,31.777076909845096,0.386462587914794,0.5567048988691092 |
| 4 | −25000,OLS quadratic (muni+quarter FE),18.13123412223344,34.71752541710843,0.3987497135923394,0.5280686972389094 | |
| 4 | +25000,OLS quadratic (muni+quarter FE),19.056034269404236,32.98471009782084,0.39338974517982517,0.5406707674234809 | |
| 5 | 5 | 25000,Gradient boosting,16.117235891494122,29.823687390173077,0.36598650222996054,0.6024349909662393 |
| 6 | −50000,OLS quadratic (muni+quarter FE),17.85098430954251,34.291813175933555,0.394668048572139,0.5376807754235925 | |
| 6 | +50000,OLS quadratic (muni+quarter FE),18.801486657084954,32.60613267913101,0.38924551758611076,0.5502975469627178 | |
| 7 | 7 | 50000,Gradient boosting,15.46029775645706,29.149938590306927,0.3602557920963508,0.6147878645125984 |
| 8 | −100000,OLS quadratic (muni+quarter FE),17.858528109660003,33.97231033671203,0.39154828747222153,0.5449609437080014 | |
| 8 | +100000,OLS quadratic (muni+quarter FE),18.79513254400627,32.30474782180091,0.3861646547014708,0.5573881289110587 | |
| 9 | 9 | 100000,Gradient boosting,15.201224181879457,28.789207604531935,0.35629888749041316,0.6232034228186022 |
| 10 | −200000,OLS quadratic (muni+quarter FE),17.926781291106366,34.286171684612285,0.3940853634933891,0.5390448972702662 | |
| 10 | +200000,OLS quadratic (muni+quarter FE),18.95754804582277,32.59698579952596,0.3886557888504686,0.5516591634684497 | |
| 11 | 11 | 200000,Gradient boosting,14.75286425569271,28.18065043214689,0.3510692811837181,0.6341831746695208 |
| 12 | −400000,OLS quadratic (muni+quarter FE),18.06403273608106,34.590640697337626,0.3967353319490308,0.5328248061516689 | |
| 12 | +400000,OLS quadratic (muni+quarter FE),19.163049017655457,32.85043942607389,0.3911538943677605,0.5458771723710507 | |
| 13 | 13 | 400000,Gradient boosting,14.485229661097984,27.850855998019064,0.3478423367115942,0.640877268082553 |
added
results/reproduced/nested_fe.csv
+5 −0
@@ -0,0 +1,5 @@ | ||
| 1 | +name,group,mdape,mape,rmse_ln,r2_ln,mdape_mean,mape_mean,rmse_ln_mean,r2_ln_mean,smear,lambda,best_iter,seconds,n_train,n_test,scheme | |
| 2 | +C3' Municipality + grid-cell FE (~5.5 km),R. Nested spatial FE,17.665652290445884,31.12198554971308,0.37510903497229636,0.5823686750564678,16.57229702106549,32.4806122435959,0.37992439164733227,0.5715774026320801,1.0631574832813349,,,0.8,411283,102929,random | |
| 3 | +C4' Municipality + grid-cell FE (~1.1 km),R. Nested spatial FE,17.29729629720419,30.733211555044505,0.3721271623880836,0.5889820782402844,16.375718827618094,32.05295192430275,0.3767837714918572,0.5786311816346446,1.0618679756677316,,,0.9,411283,102929,random | |
| 4 | +C3' Municipality + grid-cell FE (~5.5 km),R. Nested spatial FE,21.540172002578217,31.319651378571518,0.387790697263982,0.4809572799709757,18.969417428034074,31.146859757968027,0.3774487316843718,0.5082727550291839,1.0655936591628847,,,0.7,355824,158388,temporal | |
| 5 | +C4' Municipality + grid-cell FE (~1.1 km),R. Nested spatial FE,21.551985610380243,31.053622599219104,0.3860253906004687,0.48567211222806106,19.040236558535824,30.77780074936307,0.37492649963695734,0.5148225523371934,1.063943524835854,,,0.8,355824,158388,temporal | |
added
results/reproduced/pairs.csv
+27 −0
@@ -0,0 +1,27 @@ | ||
| 1 | +scheme,model_a,model_b,mdape_a,mdape_b,diff,diff_lo,diff_hi,dm_ape_t,dm_ape_p,dm_sqlog_t,dm_sqlog_p | |
| 2 | +random,A2 Semi-log,A1 Linear (levels),19.034569938583708,18.247735441468322,0.786834497115386,0.41777153663505734,1.2226677871864655,-10.686601368220524,0.0,-4.195670171129506,2.720659259725089e-05 | |
| 3 | +random,A6 Semi-log + splines,A2 Semi-log,17.6216157169721,19.034569938583708,-1.4129542216116064,-1.8335429697484,-1.0925363990812282,-10.174542669661014,0.0,-8.898715388706915,0.0 | |
| 4 | +random,A4 Box-Cox,A2 Semi-log,18.567718342220903,19.034569938583708,-0.4668515963628046,-0.5884632595730912,-0.3515787120379936,8.968814092671723,0.0,-0.6913546659882105,0.48934268588585117 | |
| 5 | +random,A3 Log-log,A2 Semi-log,18.00422814105048,19.034569938583708,-1.0303417975332287,-1.4512106300041965,-0.7229610971299634,-7.8961780030161535,2.886579864025407e-15,-10.077109850055015,0.0 | |
| 6 | +random,B2 Year FE,B1 No time effects,19.12497097548873,22.038141454991422,-2.9131704795026927,-3.840391706025526,-2.058685463008183,-8.108170173583066,4.440892098500626e-16,-7.414195626155693,1.2234657731369225e-13 | |
| 7 | +random,B4 Month FE,B3 Quarter FE,19.342062726308214,19.15743025152905,0.1846324747791641,0.009302948022620466,0.36545434556285866,2.8512317585347584,0.0043550214198446735,2.8914469178564333,0.003834723670044893 | |
| 8 | +random,C3 Grid-cell FE (~5.5 km),C2 Municipality FE,18.703553400119855,19.15743025152905,-0.45387685140919487,-1.0674805259265627,0.36564802296516835,-1.1325549920922209,0.25740117846477517,-1.1710051289695664,0.24159671553609452 | |
| 9 | +random,C4 Grid-cell FE (~1.1 km),C2 Municipality FE,20.962941599826827,19.15743025152905,1.8055113482977774,1.2410975614274538,2.3151293348045763,7.123073216501024,1.0556000518135988e-12,7.327255973885574,2.349231920106831e-13 | |
| 10 | +random,D2 Ridge (cross-validated),D1 OLS (muni + month FE),17.684559394393904,19.342062726308214,-1.6575033319143095,-2.3738182373546737,-1.1306026474848994,-5.474659566022061,4.383534979446324e-08,-5.9766089127788495,2.278302435598789e-09 | |
| 11 | +random,D4 Gradient boosting,D3 Random forest,14.388300681085338,14.62750218203232,-0.23920150094698123,-0.45885551656420775,-0.0513885363794455,-4.672835254507822,2.970700491378153e-06,-6.693591526753016,2.177591440499782e-11 | |
| 12 | +random,D4 Gradient boosting,"D5 Gradient boosting, no coordinates",14.388300681085338,23.225090677534933,-8.836789996449594,-9.787971484731155,-8.10748542572112,-10.594870291718431,0.0,-16.137980864260527,0.0 | |
| 13 | +random,D4 Gradient boosting,A1 Linear (levels),14.388300681085338,18.247735441468322,-3.8594347603829835,-5.572402821114921,-2.3810890282818233,-10.403881445314044,0.0,-11.88110067074241,0.0 | |
| 14 | +random,A6 Semi-log + splines,A1 Linear (levels),17.6216157169721,18.247735441468322,-0.6261197244962204,-1.1288670458109282,-0.13177818927939008,-12.297632954401742,0.0,-7.51790454474203,5.573319583618286e-14 | |
| 15 | +temporal,A2 Semi-log,A1 Linear (levels),23.154885105735325,20.369511213481317,2.7853738922540074,1.7788182735293514,3.8803663265918216,0.12007360831229617,0.9044248386245604,3.5326533859479348,0.00041141148678836004 | |
| 16 | +temporal,A6 Semi-log + splines,A2 Semi-log,21.722504213272607,23.154885105735325,-1.4323808924627173,-2.2976528083716783,-0.7670651228422554,-7.796357304870725,6.439293542825908e-15,-7.223963685447007,5.049294315995212e-13 | |
| 17 | +temporal,A4 Box-Cox,A2 Semi-log,20.957181720703545,23.154885105735325,-2.1977033850317795,-3.1466875936389673,-1.4637657862566353,-5.293840398286759,1.197739760705474e-07,-6.972682156866287,3.1095126473701384e-12 | |
| 18 | +temporal,A3 Log-log,A2 Semi-log,21.769614210046917,23.154885105735325,-1.3852708956884072,-2.2367115754092746,-0.7387385691927764,-7.105911135498072,1.1952661083114435e-12,-6.302689050518644,2.9252533728651997e-10 | |
| 19 | +temporal,B2 Year FE,B1 No time effects,23.651696383010147,29.296772516609654,-5.645076133599506,-6.510622685144795,-4.523812050946931,-3.8445189114832927,0.00012078917028457425,-10.351974415660491,0.0 | |
| 20 | +temporal,B4 Month FE,B3 Quarter FE,22.8719130040602,23.30291738895638,-0.4310043848961804,-0.9466994932147289,0.049163437435324885,0.7433560498426371,0.45726614558734036,-0.8851075462469108,0.37609862982402076 | |
| 21 | +temporal,C3 Grid-cell FE (~5.5 km),C2 Municipality FE,24.2625431905187,23.30291738895638,0.9596258015623178,0.16372589677996133,1.8087259847256496,2.296878919423751,0.02162567786746883,1.6301068446540665,0.1030789173649842 | |
| 22 | +temporal,C4 Grid-cell FE (~1.1 km),C2 Municipality FE,25.324930433263678,23.30291738895638,2.022013044307297,0.1532742784899062,3.589326531283127,8.122062031800938,4.440892098500626e-16,4.79314541245226,1.6418652810656909e-06 | |
| 23 | +temporal,D2 Ridge (cross-validated),D1 OLS (muni + month FE),22.257318062238202,22.8719130040602,-0.6145949418219985,-1.25927559041227,-0.16597374782475421,-5.560559072361206,2.6891181059696123e-08,-4.694048528809288,2.6785044802135616e-06 | |
| 24 | +temporal,D4 Gradient boosting,D3 Random forest,21.21243467837447,21.340119821052188,-0.12768514267771636,-0.36811779443404413,0.1456879124981816,-0.9790991544458619,0.3275309894032459,-1.2895287248262932,0.19721433670042376 | |
| 25 | +temporal,D4 Gradient boosting,"D5 Gradient boosting, no coordinates",21.21243467837447,26.207233073287274,-4.994798394912802,-6.500510794984377,-3.1215058615470657,-13.890923788390463,0.0,-11.078685604065148,0.0 | |
| 26 | +temporal,D4 Gradient boosting,A1 Linear (levels),21.21243467837447,20.369511213481317,0.8429234648931541,-1.1895847489303917,2.58981219685756,-3.520049279845853,0.0004314666251414856,-3.9079109150864686,9.309761836728825e-05 | |
| 27 | +temporal,A6 Semi-log + splines,A1 Linear (levels),21.722504213272607,20.369511213481317,1.3529929997912902,0.42539919706467916,2.27707025941844,-6.317663783346769,2.6554691778812867e-10,-6.164003649673723,7.092830767163605e-10 | |
added
results/reproduced/relevel.csv
+41 −0
@@ -0,0 +1,41 @@ | ||
| 1 | +scheme,model,median_log_resid,mdape,mdape_relevelled,mape_relevelled,rmse_ln_centered | |
| 2 | +random,A1 Linear (levels),0.003941579311327104,18.247735441468322,18.23004891488436,35.69297195166703,0.4202401848567488 | |
| 3 | +random,A2 Semi-log,0.05718358954988467,19.034569938583708,18.231160761954637,34.0869684129567,0.3949373865541228 | |
| 4 | +random,A3 Log-log,0.058544005681735456,18.00422814105048,16.999353648099166,32.573565774441384,0.380703144661566 | |
| 5 | +random,A4 Box-Cox,0.04130325060083884,18.567718342220903,17.954579371141705,34.12392448255233,0.3943404790319506 | |
| 6 | +random,A5 Semi-log + quadratics,0.060482363271615824,19.15743025152905,18.11709775973182,34.346470692604235,0.39562549011238385 | |
| 7 | +random,A6 Semi-log + splines,0.05992694422787537,17.6216157169721,16.524937550487323,32.09956942745145,0.37640447767238644 | |
| 8 | +random,B1 No time effects,0.05783820151938279,22.038141454991422,21.21973153779817,37.60444848077603,0.42017760445683755 | |
| 9 | +random,B2 Year FE,0.060634056819226956,19.12497097548873,18.120790003428947,34.4190972555926,0.39595399652847946 | |
| 10 | +random,B3 Quarter FE,0.060482363271615824,19.15743025152905,18.11709775973182,34.346470692604235,0.39562549011238385 | |
| 11 | +random,B4 Month FE,0.05989058589285712,19.342062726308214,18.354164512138155,34.515768803825544,0.39693480132805525 | |
| 12 | +random,C1 No spatial controls,0.064660279848761,28.785665634746415,28.084183838836008,46.280530679601654,0.4873531649156765 | |
| 13 | +random,C2 Municipality FE,0.060482363271615824,19.15743025152905,18.11709775973182,34.346470692604235,0.39562549011238385 | |
| 14 | +random,C3 Grid-cell FE (~5.5 km),0.06211418728747908,18.703553400119855,17.560654074769303,33.92747210229044,0.3913354959102772 | |
| 15 | +random,C4 Grid-cell FE (~1.1 km),0.07346052233407185,20.962941599826827,19.37902139629324,37.526713965938505,0.41837214294707703 | |
| 16 | +random,D1 OLS (muni + month FE),0.05989058589285712,19.342062726308214,18.354164512138155,34.515768803825544,0.39693480132805525 | |
| 17 | +random,D2 Ridge (cross-validated),0.05859458753725022,17.684559394393904,16.68303730586329,32.182831210501654,0.377254725760277 | |
| 18 | +random,D3 Random forest,0.059849487924283906,14.62750218203232,13.315627506545855,29.149851711862564,0.3571621747065756 | |
| 19 | +random,D4 Gradient boosting,0.06070519978533717,14.388300681085338,12.968025826820522,28.70404274157882,0.3522826247099593 | |
| 20 | +random,"D5 Gradient boosting, no coordinates",0.06518367898153521,23.225090677534933,22.231894512368584,40.34607372325065,0.4445804591769491 | |
| 21 | +random,D6 Spatial k-NN comparables,0.031091961266028534,21.18226600985226,21.08883377849876,37.76094936816919,0.43809139600270497 | |
| 22 | +temporal,A1 Linear (levels),0.1023593997120793,20.369511213481317,17.983628345464886,34.78005470885447,0.4002842762670569 | |
| 23 | +temporal,A2 Semi-log,0.1537984152326084,23.154885105735325,19.250951132236665,34.93065853825993,0.39945237558539437 | |
| 24 | +temporal,A3 Log-log,0.15909109773775665,21.769614210046917,17.266368727247855,32.391262877862694,0.377692222226945 | |
| 25 | +temporal,A4 Box-Cox,0.1258160183976118,20.957181720703545,17.619997411558572,33.74049661940829,0.3889147051380237 | |
| 26 | +temporal,A5 Semi-log + quadratics,0.17129196839132454,23.30291738895638,18.276550062146548,33.989570593169596,0.39008182451924517 | |
| 27 | +temporal,A6 Semi-log + splines,0.1614247324548579,21.722504213272607,16.82576888128111,32.08286270087081,0.37439667304872754 | |
| 28 | +temporal,B1 No time effects,0.2826557411184121,29.296772516609654,19.01095994641687,34.65256914399475,0.3950407735178907 | |
| 29 | +temporal,B2 Year FE,0.17007926384480498,23.651696383010147,18.99713708925793,34.63025036534526,0.39482039546313313 | |
| 30 | +temporal,B3 Quarter FE,0.17129196839132454,23.30291738895638,18.276550062146548,33.989570593169596,0.39008182451924517 | |
| 31 | +temporal,B4 Month FE,0.15736851631187143,22.8719130040602,18.68918931880011,34.336746664507935,0.3927079521397681 | |
| 32 | +temporal,C1 No spatial controls,0.16321364030035745,29.015774167241894,25.986670604102326,42.99352712008042,0.4606526213784797 | |
| 33 | +temporal,C2 Municipality FE,0.17129196839132454,23.30291738895638,18.276550062146548,33.989570593169596,0.39008182451924517 | |
| 34 | +temporal,C3 Grid-cell FE (~5.5 km),0.18220234989278072,24.2625431905187,18.814878020376376,34.76783893858608,0.3949793249194573 | |
| 35 | +temporal,C4 Grid-cell FE (~1.1 km),0.1709061348026415,25.324930433263678,20.522125855535506,38.230322256162516,0.42231730885691315 | |
| 36 | +temporal,D1 OLS (muni + month FE),0.15736851631187143,22.8719130040602,18.68918931880011,34.336746664507935,0.3927079521397681 | |
| 37 | +temporal,D2 Ridge (cross-validated),0.17022474494589535,22.257318062238202,17.037619921036498,32.23095939217706,0.3756435542037618 | |
| 38 | +temporal,D3 Random forest,0.18360250057646788,21.340119821052188,13.905947637222784,29.094464580355663,0.3542908632942389 | |
| 39 | +temporal,D4 Gradient boosting,0.18339005785301854,21.21243467837447,13.865439528037044,29.06582352581495,0.35369501531983066 | |
| 40 | +temporal,"D5 Gradient boosting, no coordinates",0.18807982851052696,26.207233073287274,21.276298850287766,38.44281213948979,0.431047370174503 | |
| 41 | +temporal,D6 Spatial k-NN comparables,0.22093905599935848,25.63120139114109,19.23963333094165,34.78903287000047,0.41913080372044065 | |
added
results/reproduced/same_period.csv
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +model,mdape_random_all,mdape_random_2025_26,mdape_forward_2025_26,n_random_2025_26,n_forward | |
| 2 | +A1 Linear (levels),18.247735441468322,17.09006288548948,20.369511213481317,31465,158388 | |
| 3 | +A2 Semi-log,19.034569938583708,18.049772389120093,23.154885105735325,31465,158388 | |
| 4 | +A3 Log-log,18.00422814105048,17.301575106616333,21.769614210046917,31465,158388 | |
| 5 | +A4 Box-Cox,18.567718342220903,17.63772856887288,20.957181720703545,31465,158388 | |
| 6 | +A5 Semi-log + quadratics,19.15743025152905,18.496565926231643,23.30291738895638,31465,158388 | |
| 7 | +A6 Semi-log + splines,17.6216157169721,16.986964776398153,21.722504213272607,31465,158388 | |
| 8 | +B1 No time effects,22.038141454991422,25.46195922621089,29.296772516609654,31465,158388 | |
| 9 | +B2 Year FE,19.12497097548873,18.587477688637772,23.651696383010147,31465,158388 | |
| 10 | +B3 Quarter FE,19.15743025152905,18.496565926231643,23.30291738895638,31465,158388 | |
| 11 | +B4 Month FE,19.342062726308214,18.91583380701899,22.8719130040602,31465,158388 | |
| 12 | +C1 No spatial controls,28.785665634746415,26.419945359140563,29.015774167241894,31465,158388 | |
| 13 | +C2 Municipality FE,19.15743025152905,18.496565926231643,23.30291738895638,31465,158388 | |
| 14 | +C3 Grid-cell FE (~5.5 km),18.703553400119855,18.261595423633086,24.2625431905187,31465,158388 | |
| 15 | +C4 Grid-cell FE (~1.1 km),20.962941599826827,19.84981063507152,25.324930433263678,31465,158388 | |
| 16 | +D1 OLS (muni + month FE),19.342062726308214,18.91583380701899,22.8719130040602,31465,158388 | |
| 17 | +D2 Ridge (cross-validated),17.684559394393904,17.098506259471893,22.257318062238202,31465,158388 | |
| 18 | +D3 Random forest,14.62750218203232,13.8581263765865,21.340119821052188,31465,158388 | |
| 19 | +D4 Gradient boosting,14.388300681085338,13.704684482746913,21.21243467837447,31465,158388 | |
| 20 | +"D5 Gradient boosting, no coordinates",23.225090677534933,20.99193671205706,26.207233073287274,31465,158388 | |
| 21 | +D6 Spatial k-NN comparables,21.18226600985226,22.55888464294253,25.63120139114109,31465,158388 | |
modified
results/reproduced/segments.csv
+7 −7
@@ -1,15 +1,15 @@ | ||
| 1 | 1 | segment,model,n,mdape,mape,rmse_ln,r2_ln |
| 2 | −Class: single_family,OLS quadratic,69835,17.15697828017771,35.09144196354416,0.4018837509330969,0.5269797760968626 | |
| 2 | +Class: single_family,OLS quadratic,69835,18.818747368969795,33.62957142912425,0.39648525503248905,0.5396025632666579 | |
| 3 | 3 | Class: single_family,Gradient boosting,69835,15.08887366879076,29.429305197147464,0.3608165339944521,0.6187132829338855 |
| 4 | −Class: condo,OLS quadratic,14848,20.42743440870622,27.094418864984917,0.32651881058598764,0.4074914996647663 | |
| 4 | +Class: condo,OLS quadratic,14848,18.197646141652584,24.35300386684896,0.32154382366324963,0.4254093992708138 | |
| 5 | 5 | Class: condo,Gradient boosting,14848,9.071686847966884,14.485209967611992,0.20596083242967297,0.7642523075225905 |
| 6 | −Class: plex,OLS quadratic,15115,18.50099130749293,36.98046001501068,0.41103973211699246,0.5454994242149013 | |
| 6 | +Class: plex,OLS quadratic,15115,20.245072332644707,35.37422177148882,0.4052926027997528,0.5581201635953541 | |
| 7 | 7 | Class: plex,Gradient boosting,15115,16.52761611544543,30.817583909526963,0.36806946728568257,0.635559683430239 |
| 8 | −Class: cottage,OLS quadratic,2109,33.91947109274449,53.22661347754297,0.5482899168598493,0.3136130552394867 | |
| 8 | +Class: cottage,OLS quadratic,2109,34.25391889121872,49.5560691019703,0.5417910818656525,0.32978799887448207 | |
| 9 | 9 | Class: cottage,Gradient boosting,2109,32.09640332106328,45.661889354803385,0.5034612691648649,0.4212638262473417 |
| 10 | −Muni < 1k sales,OLS quadratic,29253,23.257955712558935,42.00606759369818,0.4521889409919308,0.4895905571585859 | |
| 10 | +Muni < 1k sales,OLS quadratic,29253,24.83881625772472,39.847712541492584,0.44558852842475133,0.5043822749230567 | |
| 11 | 11 | Muni < 1k sales,Gradient boosting,29253,18.686729079471053,32.7580249582477,0.38939943295154483,0.621496886358641 |
| 12 | −Muni 1k–10k sales,OLS quadratic,37419,15.193131785630404,32.29520768278317,0.3776824491741606,0.5133709854088806 | |
| 12 | +Muni 1k–10k sales,OLS quadratic,37419,16.825938909928517,31.11245095169157,0.3718321556965634,0.5283299710327505 | |
| 13 | 13 | Muni 1k–10k sales,Gradient boosting,37419,14.504639160801878,28.498332907599565,0.3521566764912874,0.576926089020657 |
| 14 | −Muni > 10k sales,OLS quadratic,36257,17.82460628493704,30.99741904583036,0.3669850806325136,0.46899139390723055 | |
| 14 | +Muni > 10k sales,OLS quadratic,36257,17.685506151376927,29.004985587875904,0.3625286510651555,0.48180954127111475 | |
| 15 | 15 | Muni > 10k sales,Gradient boosting,36257,11.91348390081652,22.983932385296278,0.30299483947274597,0.6380278097783667 |
added
results/tables/by_quarter.tex
+13 −0
@@ -0,0 +1,13 @@ | ||
| 1 | +\begin{tabular}{lrccccc} | |
| 2 | +\toprule | |
| 3 | +Test quarter & $n$ & A5 Quadratics & A6 Splines & D2 Ridge & D3 Random forest & D4 Boosting \\ | |
| 4 | +\midrule | |
| 5 | +2025Q1 & 17,031 & 20.2 & 18.6 & 19.0 & 17.2 & 17.1 \\ | |
| 6 | +2025Q2 & 34,546 & 21.1 & 19.5 & 20.1 & 19.2 & 19.1 \\ | |
| 7 | +2025Q3 & 28,097 & 22.8 & 21.3 & 21.8 & 20.6 & 20.6 \\ | |
| 8 | +2025Q4 & 24,156 & 23.8 & 22.2 & 22.7 & 21.7 & 21.6 \\ | |
| 9 | +2026Q1 & 15,611 & 24.8 & 23.1 & 23.6 & 22.8 & 22.7 \\ | |
| 10 | +2026Q2 & 30,950 & 25.8 & 24.5 & 25.1 & 24.4 & 24.3 \\ | |
| 11 | +2026Q3 & 7,997 & 26.0 & 24.9 & 25.2 & 25.0 & 24.9 \\ | |
| 12 | +\bottomrule | |
| 13 | +\end{tabular} | |
| \ No newline at end of file | ||
modified
results/tables/generalization.tex
+5 −5
@@ -4,11 +4,11 @@ | ||
| 4 | 4 | \cmidrule{2-3}\cmidrule{5-6} |
| 5 | 5 | Model & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & $R^2_{\ln}$ \\ |
| 6 | 6 | \midrule |
| 7 | −A5 Semi-log + quadratics & 18.1 & 0.533 & & 20.5 & 0.479 \\ | |
| 8 | −B4 Month FE & 18.3 & 0.529 & & 20.3 & 0.476 \\ | |
| 9 | −C4 Grid-cell FE ($\sim$1.1 km) & 19.4 & 0.481 & & 22.1 & 0.400 \\ | |
| 10 | −D1 OLS (muni + month FE) & 18.3 & 0.529 & & 20.3 & 0.476 \\ | |
| 11 | −D2 Ridge (cross-validated) & 16.7 & 0.577 & & 19.5 & 0.515 \\ | |
| 7 | +A5 Semi-log + quadratics & 19.2 & 0.546 & & 23.3 & 0.442 \\ | |
| 8 | +B4 Month FE & 19.3 & 0.543 & & 22.9 & 0.445 \\ | |
| 9 | +C4 Grid-cell FE ($\sim$1.1 km) & 21.0 & 0.496 & & 25.3 & 0.364 \\ | |
| 10 | +D1 OLS (muni + month FE) & 19.3 & 0.543 & & 22.9 & 0.445 \\ | |
| 11 | +D2 Ridge (cross-validated) & 17.7 & 0.587 & & 22.3 & 0.480 \\ | |
| 12 | 12 | D3 Random forest & 14.6 & 0.631 & & 21.3 & 0.519 \\ |
| 13 | 13 | D4 Gradient boosting & 14.4 & 0.642 & & 21.2 & 0.521 \\ |
| 14 | 14 | D5 Gradient boosting, no coordinates & 23.2 & 0.426 & & 26.2 & 0.310 \\ |
added
results/tables/group_splits.tex
+34 −0
@@ -0,0 +1,34 @@ | ||
| 1 | +\begin{tabular}{l cc c cc c cc} | |
| 2 | +\toprule | |
| 3 | + & \multicolumn{2}{c}{Random 80/20} & & \multicolumn{2}{c}{Parcel-grouped} & & \multicolumn{2}{c}{Building-grouped} \\ | |
| 4 | +\cmidrule{2-3}\cmidrule{5-6}\cmidrule{8-9} | |
| 5 | +Model & MdAPE & $R^2_{\ln}$ & & MdAPE & $R^2_{\ln}$ & & MdAPE & $R^2_{\ln}$ \\ | |
| 6 | +\midrule | |
| 7 | +\multicolumn{9}{l}{\itshape A. Functional form}\\ | |
| 8 | +\quad A1 Linear & 18.2 & 0.477 & & 18.4 & 0.469 & & 18.4 & 0.473 \\ | |
| 9 | +\quad A2 Semi-log & 19.0 & 0.546 & & 19.4 & 0.528 & & 19.5 & 0.530 \\ | |
| 10 | +\quad A3 Log-log & 18.0 & 0.580 & & 18.0 & 0.572 & & 18.1 & 0.573 \\ | |
| 11 | +\quad A4 Box--Cox & 18.6 & 0.548 & & 18.2 & 0.534 & & 18.3 & 0.535 \\ | |
| 12 | +\quad A5 Quadratics & 19.2 & 0.546 & & 19.2 & 0.539 & & 19.0 & 0.552 \\ | |
| 13 | +\quad A6 Splines & 17.6 & 0.590 & & 17.6 & 0.581 & & 17.8 & 0.582 \\ | |
| 14 | +\multicolumn{9}{l}{\itshape B. Time effects}\\ | |
| 15 | +\quad B1 No time FE & 22.0 & 0.486 & & 21.6 & 0.489 & & 22.0 & 0.482 \\ | |
| 16 | +\quad B2 Year FE & 19.1 & 0.545 & & 19.0 & 0.545 & & 19.2 & 0.539 \\ | |
| 17 | +\quad B3 Quarter FE & 19.2 & 0.546 & & 19.2 & 0.539 & & 19.0 & 0.552 \\ | |
| 18 | +\quad B4 Month FE & 19.3 & 0.543 & & 19.3 & 0.537 & & 19.4 & 0.538 \\ | |
| 19 | +\multicolumn{9}{l}{\itshape C. Spatial controls}\\ | |
| 20 | +\quad C1 No spatial FE & 28.8 & 0.307 & & 28.9 & 0.301 & & 28.7 & 0.306 \\ | |
| 21 | +\quad C2 Municipality FE & 19.2 & 0.546 & & 19.2 & 0.539 & & 19.0 & 0.552 \\ | |
| 22 | +\quad C3 Grid 5.5 km & 18.7 & 0.557 & & 18.7 & 0.550 & & 18.8 & 0.551 \\ | |
| 23 | +\quad C4 Grid 1.1 km & 21.0 & 0.496 & & 21.1 & 0.491 & & 20.2 & 0.510 \\ | |
| 24 | +\multicolumn{9}{l}{\itshape D. Estimation method}\\ | |
| 25 | +\quad D1 OLS & 19.3 & 0.543 & & 19.3 & 0.537 & & 19.4 & 0.538 \\ | |
| 26 | +\quad D2 Ridge & 17.7 & 0.587 & & 17.7 & 0.580 & & 17.9 & 0.580 \\ | |
| 27 | +\quad D3 Random forest & 14.6 & 0.631 & & 14.6 & 0.620 & & 14.9 & 0.621 \\ | |
| 28 | +\quad D4 Boosting & 14.4 & 0.642 & & 14.5 & 0.632 & & 14.6 & 0.634 \\ | |
| 29 | +\quad D5 Boosting, no coord. & 23.2 & 0.426 & & 23.4 & 0.417 & & 23.3 & 0.420 \\ | |
| 30 | +\quad D6 $k$-NN comparables & 21.2 & 0.434 & & 21.3 & 0.442 & & 22.3 & 0.425 \\ | |
| 31 | +\midrule | |
| 32 | +Test sales & 102,929 & & & 102,843 & & & 102,949 & \\ | |
| 33 | +\bottomrule | |
| 34 | +\end{tabular} | |
| \ No newline at end of file | ||
added
results/tables/hgb_sensitivity.tex
+13 −0
@@ -0,0 +1,13 @@ | ||
| 1 | +\begin{tabular}{l cc c cc} | |
| 2 | +\toprule | |
| 3 | + & \multicolumn{2}{c}{Random holdout} & & \multicolumn{2}{c}{Forward-in-time} \\ | |
| 4 | +\cmidrule{2-3}\cmidrule{5-6} | |
| 5 | +Gradient-boosting configuration & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & $R^2_{\ln}$ \\ | |
| 6 | +\midrule | |
| 7 | +Gradient boosting (baseline) & 14.4 & 0.642 & & 21.2 & 0.521 \\ | |
| 8 | +Shallow (15 leaves, lr 0.05, 1500 iter) & 14.7 & 0.636 & & 21.4 & 0.521 \\ | |
| 9 | +Deep (255 leaves, lr 0.05, 600 iter) & 14.3 & 0.645 & & 21.6 & 0.518 \\ | |
| 10 | +Regularised (min leaf 200, L2 = 1) & 14.3 & 0.643 & & 21.5 & 0.522 \\ | |
| 11 | +Early stopping on last 10\% of training months [700 / 1000 it.] & 14.5 & 0.641 & & 21.2 & 0.524 \\ | |
| 12 | +\bottomrule | |
| 13 | +\end{tabular} | |
| \ No newline at end of file | ||
modified
results/tables/horserace_random.tex
+18 −18
@@ -4,30 +4,30 @@ Model & MdAPE (\%) & MAPE (\%) & RMSE$_{\ln}$ & $R^2_{\ln}$ & Fit (s) \\ | ||
| 4 | 4 | \midrule |
| 5 | 5 | \multicolumn{6}{l}{\itshape A. Functional form}\\ |
| 6 | 6 | \quad A1 Linear (levels) & 18.2 & 35.6 & 0.420 & 0.477 & 0 \\ |
| 7 | −\quad A2 Semi-log & 18.2 & 34.5 & 0.397 & 0.533 & 1 \\ | |
| 8 | −\quad A3 Log-log & 17.0 & 32.7 & 0.381 & 0.569 & 1 \\ | |
| 9 | −\quad A4 Box-Cox & 18.6 & 33.0 & 0.390 & 0.548 & 4 \\ | |
| 10 | −\quad A5 Semi-log + quadratics & 18.1 & 34.6 & 0.397 & 0.533 & 1 \\ | |
| 11 | −\quad A6 Semi-log + splines & \textbf{16.5} & 32.1 & 0.376 & 0.579 & 3 \\ | |
| 7 | +\quad A2 Semi-log & 19.0 & 32.7 & 0.391 & 0.546 & 1 \\ | |
| 8 | +\quad A3 Log-log & 18.0 & 31.2 & 0.376 & 0.580 & 1 \\ | |
| 9 | +\quad A4 Box-Cox & 18.6 & 33.0 & 0.390 & 0.548 & 5 \\ | |
| 10 | +\quad A5 Semi-log + quadratics & 19.2 & 32.9 & 0.391 & 0.546 & 1 \\ | |
| 11 | +\quad A6 Semi-log + splines & \textbf{17.6} & 30.8 & 0.372 & 0.590 & 3 \\ | |
| 12 | 12 | \midrule |
| 13 | 13 | \multicolumn{6}{l}{\itshape B. Time effects}\\ |
| 14 | −\quad B1 No time effects & 21.1 & 38.4 & 0.423 & 0.468 & 0 \\ | |
| 15 | −\quad B2 Year FE & 18.1 & 34.7 & 0.397 & 0.532 & 1 \\ | |
| 16 | −\quad B3 Quarter FE & \textbf{18.1} & 34.6 & 0.397 & 0.533 & 1 \\ | |
| 17 | −\quad B4 Month FE & 18.3 & 34.8 & 0.398 & 0.529 & 1 \\ | |
| 14 | +\quad B1 No time effects & 22.0 & 35.9 & 0.416 & 0.486 & 0 \\ | |
| 15 | +\quad B2 Year FE & \textbf{19.1} & 32.9 & 0.391 & 0.545 & 1 \\ | |
| 16 | +\quad B3 Quarter FE & 19.2 & 32.9 & 0.391 & 0.546 & 1 \\ | |
| 17 | +\quad B4 Month FE & 19.3 & 33.0 & 0.393 & 0.543 & 1 \\ | |
| 18 | 18 | \midrule |
| 19 | 19 | \multicolumn{6}{l}{\itshape C. Spatial controls}\\ |
| 20 | −\quad C1 No spatial controls & 27.9 & 48.4 & 0.494 & 0.274 & 0 \\ | |
| 21 | −\quad C2 Municipality FE & 18.1 & 34.6 & 0.397 & 0.533 & 1 \\ | |
| 22 | −\quad C3 Grid-cell FE ($\sim$5.5 km) & \textbf{17.5} & 34.0 & 0.392 & 0.544 & 1 \\ | |
| 23 | −\quad C4 Grid-cell FE ($\sim$1.1 km) & 19.4 & 37.5 & 0.418 & 0.481 & 1 \\ | |
| 20 | +\quad C1 No spatial controls & 28.8 & 43.7 & 0.483 & 0.307 & 0 \\ | |
| 21 | +\quad C2 Municipality FE & 19.2 & 32.9 & 0.391 & 0.546 & 1 \\ | |
| 22 | +\quad C3 Grid-cell FE ($\sim$5.5 km) & \textbf{18.7} & 32.5 & 0.387 & 0.557 & 1 \\ | |
| 23 | +\quad C4 Grid-cell FE ($\sim$1.1 km) & 21.0 & 35.6 & 0.412 & 0.496 & 1 \\ | |
| 24 | 24 | \midrule |
| 25 | 25 | \multicolumn{6}{l}{\itshape D. Estimation method}\\ |
| 26 | −\quad D1 OLS (muni + month FE) & 18.3 & 34.8 & 0.398 & 0.529 & 1 \\ | |
| 27 | −\quad D2 Ridge (cross-validated) & 16.7 & 32.3 & 0.378 & 0.577 & 11 \\ | |
| 28 | −\quad D3 Random forest & 14.6 & 28.2 & 0.353 & 0.631 & 8 \\ | |
| 29 | −\quad D4 Gradient boosting & \textbf{14.4} & 27.8 & 0.347 & 0.642 & 42 \\ | |
| 30 | −\quad D5 Gradient boosting, no coordinates & 23.2 & 38.3 & 0.440 & 0.426 & 18 \\ | |
| 26 | +\quad D1 OLS (muni + month FE) & 19.3 & 33.0 & 0.393 & 0.543 & 1 \\ | |
| 27 | +\quad D2 Ridge (cross-validated) & 17.7 & 30.9 & 0.373 & 0.587 & 11 \\ | |
| 28 | +\quad D3 Random forest & 14.6 & 28.2 & 0.353 & 0.631 & 12 \\ | |
| 29 | +\quad D4 Gradient boosting & \textbf{14.4} & 27.8 & 0.347 & 0.642 & 47 \\ | |
| 30 | +\quad D5 Gradient boosting, no coordinates & 23.2 & 38.3 & 0.440 & 0.426 & 17 \\ | |
| 31 | 31 | \quad D6 Spatial k-NN comparables & 21.2 & 36.7 & 0.437 & 0.434 & 0 \\ |
| 32 | 32 | \bottomrule |
| 33 | 33 | \end{tabular} |
| \ No newline at end of file | ||
modified
results/tables/horserace_temporal.tex
+18 −18
@@ -3,31 +3,31 @@ | ||
| 3 | 3 | Model & MdAPE (\%) & MAPE (\%) & RMSE$_{\ln}$ & $R^2_{\ln}$ & Fit (s) \\ |
| 4 | 4 | \midrule |
| 5 | 5 | \multicolumn{6}{l}{\itshape A. Functional form}\\ |
| 6 | −\quad A1 Linear (levels) & 20.4 & 32.9 & 0.397 & 0.455 & 0 \\ | |
| 7 | −\quad A2 Semi-log & 20.6 & 33.1 & 0.396 & 0.458 & 0 \\ | |
| 8 | −\quad A3 Log-log & 19.2 & 30.7 & 0.376 & 0.513 & 1 \\ | |
| 6 | +\quad A1 Linear (levels) & \textbf{20.4} & 32.9 & 0.397 & 0.455 & 0 \\ | |
| 7 | +\quad A2 Semi-log & 23.2 & 32.9 & 0.408 & 0.426 & 0 \\ | |
| 8 | +\quad A3 Log-log & 21.8 & 31.0 & 0.388 & 0.481 & 1 \\ | |
| 9 | 9 | \quad A4 Box-Cox & 21.0 & 31.9 & 0.390 & 0.476 & 4 \\ |
| 10 | −\quad A5 Semi-log + quadratics & 20.5 & 32.1 & 0.388 & 0.479 & 1 \\ | |
| 11 | −\quad A6 Semi-log + splines & \textbf{19.1} & 30.5 & 0.372 & 0.522 & 2 \\ | |
| 10 | +\quad A5 Semi-log + quadratics & 23.3 & 32.3 & 0.402 & 0.442 & 1 \\ | |
| 11 | +\quad A6 Semi-log + splines & 21.7 & 30.9 & 0.384 & 0.491 & 2 \\ | |
| 12 | 12 | \midrule |
| 13 | 13 | \multicolumn{6}{l}{\itshape B. Time effects}\\ |
| 14 | −\quad B1 No time effects & 25.4 & 33.2 & 0.418 & 0.397 & 0 \\ | |
| 15 | −\quad B2 Year FE & 21.0 & 32.6 & 0.393 & 0.468 & 1 \\ | |
| 16 | −\quad B3 Quarter FE & 20.5 & 32.1 & 0.388 & 0.479 & 1 \\ | |
| 17 | −\quad B4 Month FE & \textbf{20.3} & 32.5 & 0.390 & 0.476 & 1 \\ | |
| 14 | +\quad B1 No time effects & 29.3 & 34.7 & 0.450 & 0.300 & 0 \\ | |
| 15 | +\quad B2 Year FE & 23.7 & 32.7 & 0.406 & 0.431 & 1 \\ | |
| 16 | +\quad B3 Quarter FE & 23.3 & 32.3 & 0.402 & 0.442 & 1 \\ | |
| 17 | +\quad B4 Month FE & \textbf{22.9} & 32.4 & 0.401 & 0.445 & 1 \\ | |
| 18 | 18 | \midrule |
| 19 | 19 | \multicolumn{6}{l}{\itshape C. Spatial controls}\\ |
| 20 | −\quad C1 No spatial controls & 26.5 & 41.1 & 0.457 & 0.279 & 0 \\ | |
| 21 | −\quad C2 Municipality FE & \textbf{20.5} & 32.1 & 0.388 & 0.479 & 1 \\ | |
| 22 | −\quad C3 Grid-cell FE ($\sim$5.5 km) & 21.3 & 32.7 & 0.394 & 0.465 & 1 \\ | |
| 23 | −\quad C4 Grid-cell FE ($\sim$1.1 km) & 22.1 & 36.0 & 0.417 & 0.400 & 1 \\ | |
| 20 | +\quad C1 No spatial controls & 29.0 & 38.9 & 0.469 & 0.242 & 0 \\ | |
| 21 | +\quad C2 Municipality FE & \textbf{23.3} & 32.3 & 0.402 & 0.442 & 1 \\ | |
| 22 | +\quad C3 Grid-cell FE ($\sim$5.5 km) & 24.3 & 33.0 & 0.409 & 0.423 & 1 \\ | |
| 23 | +\quad C4 Grid-cell FE ($\sim$1.1 km) & 25.3 & 35.5 & 0.429 & 0.364 & 1 \\ | |
| 24 | 24 | \midrule |
| 25 | 25 | \multicolumn{6}{l}{\itshape D. Estimation method}\\ |
| 26 | −\quad D1 OLS (muni + month FE) & 20.3 & 32.5 & 0.390 & 0.476 & 1 \\ | |
| 27 | −\quad D2 Ridge (cross-validated) & \textbf{19.5} & 30.6 & 0.375 & 0.515 & 9 \\ | |
| 28 | −\quad D3 Random forest & 21.3 & 29.8 & 0.373 & 0.519 & 7 \\ | |
| 29 | −\quad D4 Gradient boosting & 21.2 & 29.7 & 0.372 & 0.521 & 42 \\ | |
| 30 | −\quad D5 Gradient boosting, no coordinates & 26.2 & 35.7 & 0.447 & 0.310 & 21 \\ | |
| 26 | +\quad D1 OLS (muni + month FE) & 22.9 & 32.4 & 0.401 & 0.445 & 1 \\ | |
| 27 | +\quad D2 Ridge (cross-validated) & 22.3 & 31.1 & 0.388 & 0.480 & 8 \\ | |
| 28 | +\quad D3 Random forest & 21.3 & 29.8 & 0.373 & 0.519 & 11 \\ | |
| 29 | +\quad D4 Gradient boosting & \textbf{21.2} & 29.7 & 0.372 & 0.521 & 41 \\ | |
| 30 | +\quad D5 Gradient boosting, no coordinates & 26.2 & 35.7 & 0.447 & 0.310 & 22 \\ | |
| 31 | 31 | \quad D6 Spatial k-NN comparables & 25.6 & 34.2 & 0.465 & 0.253 & 0 \\ |
| 32 | 32 | \bottomrule |
| 33 | 33 | \end{tabular} |
| \ No newline at end of file | ||
added
results/tables/inference.tex
+32 −0
@@ -0,0 +1,32 @@ | ||
| 1 | +\begin{tabular}{l ccc c ccc} | |
| 2 | +\toprule | |
| 3 | + & \multicolumn{3}{c}{Random holdout} & & \multicolumn{3}{c}{Forward-in-time holdout} \\ | |
| 4 | +\cmidrule{2-4}\cmidrule{6-8} | |
| 5 | +Model & MdAPE [95\% CI] & $\Delta$ vs.\ A6 [95\% CI] & $t_{\mathrm{DM}}$ & & MdAPE [95\% CI] & $\Delta$ vs.\ A6 [95\% CI] & $t_{\mathrm{DM}}$ \\ | |
| 6 | +\midrule | |
| 7 | +\multicolumn{8}{l}{\itshape A. Functional form}\\ | |
| 8 | +\quad A1 Linear & 18.2 [16.9, 19.6] & +0.6 [+0.1, +1.1] & 12.3$^{***}$ & & 20.4 [19.1, 21.7] & -1.4 [-2.3, -0.4] & 6.3$^{***}$ \\ | |
| 9 | +\quad A2 Semi-log & 19.0 [17.9, 20.2] & +1.4 [+1.1, +1.8] & 10.2$^{***}$ & & 23.2 [21.7, 24.7] & +1.4 [+0.8, +2.3] & 7.8$^{***}$ \\ | |
| 10 | +\quad A3 Log-log & 18.0 [17.1, 19.0] & +0.4 [+0.3, +0.5] & 4.6$^{***}$ & & 21.8 [20.5, 23.2] & +0.0 [-0.0, +0.2] & 1.8$^{*}$ \\ | |
| 11 | +\quad A4 Box--Cox & 18.6 [17.5, 19.7] & +0.9 [+0.7, +1.3] & 10.7$^{***}$ & & 21.0 [19.7, 22.2] & -0.8 [-1.4, -0.1] & 4.3$^{***}$ \\ | |
| 12 | +\quad A5 Quadratics & 19.2 [18.2, 20.2] & +1.5 [+1.1, +2.1] & 5.5$^{***}$ & & 23.3 [21.9, 24.9] & +1.6 [+1.1, +2.2] & 6.1$^{***}$ \\ | |
| 13 | +\quad A6 Splines & 17.6 [16.7, 18.6] & --- & --- & & 21.7 [20.5, 23.1] & --- & --- \\ | |
| 14 | +\multicolumn{8}{l}{\itshape B. Time effects}\\ | |
| 15 | +\quad B1 No time FE & 22.0 [21.1, 23.3] & +4.4 [+3.4, +5.6] & 8.2$^{***}$ & & 29.3 [27.3, 31.5] & +7.6 [+6.0, +9.0] & 5.3$^{***}$ \\ | |
| 16 | +\quad B2 Year FE & 19.1 [18.1, 20.2] & +1.5 [+1.1, +2.2] & 6.1$^{***}$ & & 23.7 [22.2, 25.3] & +1.9 [+1.3, +2.8] & 6.2$^{***}$ \\ | |
| 17 | +\quad B3 Quarter FE & 19.2 [18.2, 20.2] & +1.5 [+1.1, +2.1] & 5.5$^{***}$ & & 23.3 [21.9, 24.9] & +1.6 [+1.1, +2.2] & 6.1$^{***}$ \\ | |
| 18 | +\quad B4 Month FE & 19.3 [18.4, 20.4] & +1.7 [+1.2, +2.5] & 5.7$^{***}$ & & 22.9 [21.5, 24.4] & +1.1 [+0.7, +1.8] & 6.0$^{***}$ \\ | |
| 19 | +\multicolumn{8}{l}{\itshape C. Spatial controls}\\ | |
| 20 | +\quad C1 No spatial FE & 28.8 [27.3, 30.1] & +11.2 [+10.1, +12.3] & 7.7$^{***}$ & & 29.0 [27.4, 30.3] & +7.3 [+5.0, +9.2] & 11.1$^{***}$ \\ | |
| 21 | +\quad C2 Municipality FE & 19.2 [18.2, 20.2] & +1.5 [+1.1, +2.1] & 5.5$^{***}$ & & 23.3 [21.9, 24.9] & +1.6 [+1.1, +2.2] & 6.1$^{***}$ \\ | |
| 22 | +\quad C3 Grid 5.5 km & 18.7 [17.8, 20.0] & +1.1 [+0.2, +2.3] & 2.5$^{**}$ & & 24.3 [22.5, 26.4] & +2.5 [+1.6, +3.5] & 4.5$^{***}$ \\ | |
| 23 | +\quad C4 Grid 1.1 km & 21.0 [20.1, 21.9] & +3.3 [+2.9, +3.8] & 6.8$^{***}$ & & 25.3 [24.3, 26.3] & +3.6 [+2.1, +4.9] & 12.4$^{***}$ \\ | |
| 24 | +\multicolumn{8}{l}{\itshape D. Estimation method}\\ | |
| 25 | +\quad D1 OLS & 19.3 [18.4, 20.4] & +1.7 [+1.2, +2.5] & 5.7$^{***}$ & & 22.9 [21.5, 24.4] & +1.1 [+0.7, +1.8] & 6.0$^{***}$ \\ | |
| 26 | +\quad D2 Ridge & 17.7 [16.8, 18.7] & +0.1 [-0.0, +0.2] & 2.6$^{***}$ & & 22.3 [21.0, 23.7] & +0.5 [+0.5, +0.6] & 6.6$^{***}$ \\ | |
| 27 | +\quad D3 Random forest & 14.6 [13.7, 16.2] & -3.0 [-4.6, -1.5] & -2.9$^{***}$ & & 21.3 [19.9, 23.0] & -0.4 [-1.5, +0.6] & -1.6 \\ | |
| 28 | +\quad D4 Boosting & 14.4 [13.7, 15.7] & -3.2 [-4.7, -1.9] & -3.6$^{***}$ & & 21.2 [19.7, 22.9] & -0.5 [-1.7, +0.6] & -1.6 \\ | |
| 29 | +\quad D5 Boosting, no coord. & 23.2 [22.1, 25.1] & +5.6 [+3.9, +7.5] & 4.5$^{***}$ & & 26.2 [25.2, 27.5] & +4.5 [+2.9, +6.2] & 5.9$^{***}$ \\ | |
| 30 | +\quad D6 $k$-NN comparables & 21.2 [19.4, 23.5] & +3.6 [+1.0, +5.9] & 4.8$^{***}$ & & 25.6 [23.1, 28.3] & +3.9 [+1.8, +5.6] & 4.0$^{***}$ \\ | |
| 31 | +\bottomrule | |
| 32 | +\end{tabular} | |
| \ No newline at end of file | ||
added
results/tables/leakage.tex
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +\begin{tabular}{l cc c cc} | |
| 2 | +\toprule | |
| 3 | + & \multicolumn{2}{c}{Same parcel in training} & & \multicolumn{2}{c}{Same coordinates in training} \\ | |
| 4 | +\cmidrule{2-3}\cmidrule{5-6} | |
| 5 | +Model & Yes & No & & Yes & No \\ | |
| 6 | +\midrule | |
| 7 | +\multicolumn{6}{l}{\itshape All test sales}\\ | |
| 8 | +\quad A5 Quadratics & 19.8 & 19.1 & & 19.2 & 19.2 \\ | |
| 9 | +\quad A6 Splines & 18.3 & 17.5 & & 17.8 & 17.6 \\ | |
| 10 | +\quad D3 Random forest & 14.6 & 14.6 & & 11.7 & 15.7 \\ | |
| 11 | +\quad D4 Boosting & 14.5 & 14.4 & & 12.0 & 15.3 \\ | |
| 12 | +\quad D5 Boosting, no coord. & 22.9 & 23.3 & & 20.0 & 24.5 \\ | |
| 13 | +\quad D6 $k$-NN comparables & 22.2 & 21.0 & & 16.7 & 22.9 \\ | |
| 14 | +\quad Test sales & 17,219 & 85,710 & & 27,475 & 75,454 \\ | |
| 15 | +\midrule | |
| 16 | +\multicolumn{6}{l}{\itshape Condominiums only}\\ | |
| 17 | +\quad A5 Quadratics & 17.4 & 18.4 & & 18.0 & 20.3 \\ | |
| 18 | +\quad D4 Boosting & 9.0 & 9.1 & & 8.9 & 10.2 \\ | |
| 19 | +\quad Test sales & 3,374 & 11,474 & & 13,261 & 1,587 \\ | |
| 20 | +\bottomrule | |
| 21 | +\end{tabular} | |
| \ No newline at end of file | ||
modified
results/tables/learning.tex
+6 −6
@@ -2,11 +2,11 @@ | ||
| 2 | 2 | \toprule |
| 3 | 3 | Training sales & Gradient boosting & OLS quadratic (muni+quarter FE) \\ |
| 4 | 4 | \midrule |
| 5 | −10,000 & 17.6 & 18.4 \\ | |
| 6 | −25,000 & 16.1 & 18.1 \\ | |
| 7 | −50,000 & 15.5 & 17.9 \\ | |
| 8 | −100,000 & 15.2 & 17.9 \\ | |
| 9 | −200,000 & 14.8 & 17.9 \\ | |
| 10 | −400,000 & 14.5 & 18.1 \\ | |
| 5 | +10,000 & 17.6 & 19.1 \\ | |
| 6 | +25,000 & 16.1 & 19.1 \\ | |
| 7 | +50,000 & 15.5 & 18.8 \\ | |
| 8 | +100,000 & 15.2 & 18.8 \\ | |
| 9 | +200,000 & 14.8 & 19.0 \\ | |
| 10 | +400,000 & 14.5 & 19.2 \\ | |
| 11 | 11 | \bottomrule |
| 12 | 12 | \end{tabular} |
| \ No newline at end of file | ||
added
results/tables/nested_fe.tex
+13 −0
@@ -0,0 +1,13 @@ | ||
| 1 | +\begin{tabular}{l cc c cc} | |
| 2 | +\toprule | |
| 3 | + & \multicolumn{2}{c}{Random holdout} & & \multicolumn{2}{c}{Forward-in-time} \\ | |
| 4 | +\cmidrule{2-3}\cmidrule{5-6} | |
| 5 | +Spatial controls & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & $R^2_{\ln}$ \\ | |
| 6 | +\midrule | |
| 7 | +C2 Municipality FE (1,115 cells) & 19.2 & 0.546 & & 23.3 & 0.442 \\ | |
| 8 | +C3 Grid 5.5 km (5,255 cells) & 18.7 & 0.557 & & 24.3 & 0.423 \\ | |
| 9 | +C4 Grid 1.1 km (30,548 cells) & 21.0 & 0.496 & & 25.3 & 0.364 \\ | |
| 10 | +C3$'$ Municipality + grid 5.5 km & 17.7 & 0.582 & & 21.5 & 0.481 \\ | |
| 11 | +C4$'$ Municipality + grid 1.1 km & 17.3 & 0.589 & & 21.6 & 0.486 \\ | |
| 12 | +\bottomrule | |
| 13 | +\end{tabular} | |
| \ No newline at end of file | ||
added
results/tables/pairs.tex
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +\begin{tabular}{ll c cc c cc} | |
| 2 | +\toprule | |
| 3 | + & & & \multicolumn{2}{c}{Random holdout} & & \multicolumn{2}{c}{Forward-in-time} \\ | |
| 4 | +\cmidrule{4-5}\cmidrule{7-8} | |
| 5 | +Comparison & & MdAPE & $\Delta$MdAPE [95\% CI] & $t_{\mathrm{DM}}$ APE / sq.\ log & & $\Delta$MdAPE [95\% CI] & $t_{\mathrm{DM}}$ APE / sq.\ log \\ | |
| 6 | +\midrule | |
| 7 | +A2 Semi-log vs. & A1 Linear & 19.0 / 18.2 & +0.79 [+0.42, +1.22] & -10.7$^{***}$ / -4.2$^{***}$ & & +2.79 [+1.78, +3.88] & 0.1 / 3.5$^{***}$ \\ | |
| 8 | +A6 Splines vs. & A2 Semi-log & 17.6 / 19.0 & -1.41 [-1.83, -1.09] & -10.2$^{***}$ / -8.9$^{***}$ & & -1.43 [-2.30, -0.77] & -7.8$^{***}$ / -7.2$^{***}$ \\ | |
| 9 | +A4 Box--Cox vs. & A2 Semi-log & 18.6 / 19.0 & -0.47 [-0.59, -0.35] & 9.0$^{***}$ / -0.7 & & -2.20 [-3.15, -1.46] & -5.3$^{***}$ / -7.0$^{***}$ \\ | |
| 10 | +A3 Log-log vs. & A2 Semi-log & 18.0 / 19.0 & -1.03 [-1.45, -0.72] & -7.9$^{***}$ / -10.1$^{***}$ & & -1.39 [-2.24, -0.74] & -7.1$^{***}$ / -6.3$^{***}$ \\ | |
| 11 | +B2 Year FE vs. & B1 No time FE & 19.1 / 22.0 & -2.91 [-3.84, -2.06] & -8.1$^{***}$ / -7.4$^{***}$ & & -5.65 [-6.51, -4.52] & -3.8$^{***}$ / -10.4$^{***}$ \\ | |
| 12 | +B4 Month FE vs. & B3 Quarter FE & 19.3 / 19.2 & +0.18 [+0.01, +0.37] & 2.9$^{***}$ / 2.9$^{***}$ & & -0.43 [-0.95, +0.05] & 0.7 / -0.9 \\ | |
| 13 | +C3 Grid 5.5 km vs. & C2 Municipality FE & 18.7 / 19.2 & -0.45 [-1.07, +0.37] & -1.1 / -1.2 & & +0.96 [+0.16, +1.81] & 2.3$^{**}$ / 1.6 \\ | |
| 14 | +C4 Grid 1.1 km vs. & C2 Municipality FE & 21.0 / 19.2 & +1.81 [+1.24, +2.32] & 7.1$^{***}$ / 7.3$^{***}$ & & +2.02 [+0.15, +3.59] & 8.1$^{***}$ / 4.8$^{***}$ \\ | |
| 15 | +D2 Ridge vs. & D1 OLS & 17.7 / 19.3 & -1.66 [-2.37, -1.13] & -5.5$^{***}$ / -6.0$^{***}$ & & -0.61 [-1.26, -0.17] & -5.6$^{***}$ / -4.7$^{***}$ \\ | |
| 16 | +D4 Boosting vs. & D3 Random forest & 14.4 / 14.6 & -0.24 [-0.46, -0.05] & -4.7$^{***}$ / -6.7$^{***}$ & & -0.13 [-0.37, +0.15] & -1.0 / -1.3 \\ | |
| 17 | +D4 Boosting vs. & D5 Boosting, no coord. & 14.4 / 23.2 & -8.84 [-9.79, -8.11] & -10.6$^{***}$ / -16.1$^{***}$ & & -4.99 [-6.50, -3.12] & -13.9$^{***}$ / -11.1$^{***}$ \\ | |
| 18 | +D4 Boosting vs. & A1 Linear & 14.4 / 18.2 & -3.86 [-5.57, -2.38] & -10.4$^{***}$ / -11.9$^{***}$ & & +0.84 [-1.19, +2.59] & -3.5$^{***}$ / -3.9$^{***}$ \\ | |
| 19 | +A6 Splines vs. & A1 Linear & 17.6 / 18.2 & -0.63 [-1.13, -0.13] & -12.3$^{***}$ / -7.5$^{***}$ & & +1.35 [+0.43, +2.28] & -6.3$^{***}$ / -6.2$^{***}$ \\ | |
| 20 | +\bottomrule | |
| 21 | +\end{tabular} | |
| \ No newline at end of file | ||
added
results/tables/retransformation.tex
+18 −0
@@ -0,0 +1,18 @@ | ||
| 1 | +\begin{tabular}{l c cc cc c cc cc} | |
| 2 | +\toprule | |
| 3 | + & & \multicolumn{4}{c}{Random holdout} & & \multicolumn{4}{c}{Forward-in-time holdout} \\ | |
| 4 | +\cmidrule{3-6}\cmidrule{8-11} | |
| 5 | + & & \multicolumn{2}{c}{MdAPE (\%)} & \multicolumn{2}{c}{$R^2_{\ln}$} & & \multicolumn{2}{c}{MdAPE (\%)} & \multicolumn{2}{c}{$R^2_{\ln}$} \\ | |
| 6 | +\cmidrule{3-4}\cmidrule{5-6}\cmidrule{8-9}\cmidrule{10-11} | |
| 7 | +Model & $\hat{s}$ & Median & Mean & Median & Mean & & Median & Mean & Median & Mean \\ | |
| 8 | +\midrule | |
| 9 | +A2 Semi-log & 1.070 & 19.0 & 18.2 & 0.546 & 0.533 & & 23.2 & 20.6 & 0.426 & 0.458 \\ | |
| 10 | +A3 Log-log & 1.064 & 18.0 & 17.0 & 0.580 & 0.569 & & 21.8 & 19.2 & 0.481 & 0.513 \\ | |
| 11 | +A4 Box--Cox & --- & 18.6 & 17.9 & 0.548 & 0.536 & & 21.0 & 19.6 & 0.476 & 0.490 \\ | |
| 12 | +A5 Quadratics & 1.070 & 19.2 & 18.1 & 0.546 & 0.533 & & 23.3 & 20.5 & 0.442 & 0.479 \\ | |
| 13 | +A6 Splines & 1.062 & 17.6 & 16.5 & 0.590 & 0.579 & & 21.7 & 19.1 & 0.491 & 0.522 \\ | |
| 14 | +D2 Ridge & 1.063 & 17.7 & 16.7 & 0.587 & 0.577 & & 22.3 & 19.5 & 0.480 & 0.515 \\ | |
| 15 | +D3 Random forest & 1.032 & 14.6 & 13.7 & 0.631 & 0.629 & & 21.3 & 19.4 & 0.519 & 0.544 \\ | |
| 16 | +D4 Boosting & 1.048 & 14.4 & 13.1 & 0.642 & 0.636 & & 21.2 & 18.4 & 0.521 & 0.556 \\ | |
| 17 | +\bottomrule | |
| 18 | +\end{tabular} | |
| \ No newline at end of file | ||
added
results/tables/same_period.tex
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +\begin{tabular}{l cc c c c c} | |
| 2 | +\toprule | |
| 3 | + & \multicolumn{2}{c}{Random split} & Forward split & Staleness & Level error & Re-levelled \\ | |
| 4 | +\cmidrule{2-3}\cmidrule{4-4} | |
| 5 | +Model & All test sales & 2025--26 sales & 2025--26 sales & cost & $\hat{b}$ & MdAPE \\ | |
| 6 | +\midrule | |
| 7 | +A1 Linear & 18.2 & 17.1 & 20.4 & +3.3 & +0.102 & 18.0 \\ | |
| 8 | +A5 Quadratics & 19.2 & 18.5 & 23.3 & +4.8 & +0.171 & 18.3 \\ | |
| 9 | +A6 Splines & 17.6 & 17.0 & 21.7 & +4.7 & +0.161 & 16.8 \\ | |
| 10 | +B4 Month FE & 19.3 & 18.9 & 22.9 & +4.0 & +0.157 & 18.7 \\ | |
| 11 | +C3 Grid 5.5 km & 18.7 & 18.3 & 24.3 & +6.0 & +0.182 & 18.8 \\ | |
| 12 | +C4 Grid 1.1 km & 21.0 & 19.8 & 25.3 & +5.5 & +0.171 & 20.5 \\ | |
| 13 | +D2 Ridge & 17.7 & 17.1 & 22.3 & +5.2 & +0.170 & 17.0 \\ | |
| 14 | +D3 Random forest & 14.6 & 13.9 & 21.3 & +7.5 & +0.184 & 13.9 \\ | |
| 15 | +D4 Boosting & 14.4 & 13.7 & 21.2 & +7.5 & +0.183 & 13.9 \\ | |
| 16 | +D5 Boosting, no coord. & 23.2 & 21.0 & 26.2 & +5.2 & +0.188 & 21.3 \\ | |
| 17 | +D6 $k$-NN comparables & 21.2 & 22.6 & 25.6 & +3.1 & +0.221 & 19.2 \\ | |
| 18 | +\midrule | |
| 19 | +Test sales & 102,929 & 31,465 & 158,388 & & & \\ | |
| 20 | +\bottomrule | |
| 21 | +\end{tabular} | |
| \ No newline at end of file | ||
modified
results/tables/segments.tex
+7 −7
@@ -2,12 +2,12 @@ | ||
| 2 | 2 | \toprule |
| 3 | 3 | Segment & $n$ (test) & OLS quadratic & Gradient boosting \\ |
| 4 | 4 | \midrule |
| 5 | −Condo & 14,848 & 20.4 & 9.1 \\ | |
| 6 | −Cottage & 2,109 & 33.9 & 32.1 \\ | |
| 7 | −Plex & 15,115 & 18.5 & 16.5 \\ | |
| 8 | −Single family & 69,835 & 17.2 & 15.1 \\ | |
| 9 | −Muni 1k--10k sales & 37,419 & 15.2 & 14.5 \\ | |
| 10 | −Muni $<$ 1k sales & 29,253 & 23.3 & 18.7 \\ | |
| 11 | −Muni $>$ 10k sales & 36,257 & 17.8 & 11.9 \\ | |
| 5 | +Condo & 14,848 & 18.2 & 9.1 \\ | |
| 6 | +Cottage & 2,109 & 34.3 & 32.1 \\ | |
| 7 | +Plex & 15,115 & 20.2 & 16.5 \\ | |
| 8 | +Single family & 69,835 & 18.8 & 15.1 \\ | |
| 9 | +Muni 1k--10k sales & 37,419 & 16.8 & 14.5 \\ | |
| 10 | +Muni $<$ 1k sales & 29,253 & 24.8 & 18.7 \\ | |
| 11 | +Muni $>$ 10k sales & 36,257 & 17.7 & 11.9 \\ | |
| 12 | 12 | \bottomrule |
| 13 | 13 | \end{tabular} |
| \ No newline at end of file | ||
modified
scripts/02_horserace.py
+7 −2
@@ -7,7 +7,9 @@ Fits the twenty registry models under (i) a random 80/20 holdout and | ||
| 7 | 7 | Box–Cox profile likelihood and summary statistics. |
| 8 | 8 | |
| 9 | 9 | Writes: horserace_random.csv, horserace_temporal.csv, boxcox.csv, |
| 10 | −summary_stats.csv, sample_counts.csv | |
| 10 | +summary_stats.csv, sample_counts.csv, and the test-set predictions of | |
| 11 | +every model (data/processed/predictions_<scheme>.parquet, not committed) | |
| 12 | +used by scripts/06_robustness.py for the inference and leakage analyses. | |
| 11 | 13 | |
| 12 | 14 | Usage: python scripts/02_horserace.py |
| 13 | 15 | """ |
@@ -56,8 +58,11 @@ def main() -> None: | ||
| 56 | 58 | |
| 57 | 59 | # ------------------------------------------------------------ the race |
| 58 | 60 | for scheme in ("random", "temporal"): |
| 59 | − res = models.run_horserace(df, scheme) | |
| 61 | + res, preds = models.run_horserace(df, scheme, | |
| 62 | + return_predictions=True) | |
| 60 | 63 | res.to_csv(out / f"horserace_{scheme}.csv", index=False) |
| 64 | + preds.to_parquet(config.PROCESSED / f"predictions_{scheme}.parquet", | |
| 65 | + index=False) | |
| 61 | 66 | |
| 62 | 67 | |
| 63 | 68 | if __name__ == "__main__": |
modified
scripts/04_make_figures.py
+33 −0
@@ -186,6 +186,38 @@ def fig_segments(): | ||
| 186 | 186 | _save(fig, "fig_segments.png") |
| 187 | 187 | |
| 188 | 188 | |
| 189 | +# ---------------------------------------------------------------- fig 10 | |
| 190 | +def fig_horizon(): | |
| 191 | + """Forward-in-time MdAPE by test quarter (distance from the last | |
| 192 | + training month), selected models.""" | |
| 193 | + f = RES / "by_quarter.csv" | |
| 194 | + if not f.exists(): | |
| 195 | + return | |
| 196 | + q = pd.read_csv(f) | |
| 197 | + fig, ax = plt.subplots(figsize=(0.8 * TEXTWIDTH, 3.2)) | |
| 198 | + series = (("A6 Semi-log + splines", BLUE, "o", "-", "A6 Splines"), | |
| 199 | + ("A5 Semi-log + quadratics", BLUES[2], "o", (0, (5, 2)), | |
| 200 | + "A5 Quadratics"), | |
| 201 | + ("D2 Ridge (cross-validated)", BLUES[5], "^", (0, (1, 1.2)), | |
| 202 | + "D2 Ridge"), | |
| 203 | + ("D4 Gradient boosting", RED, "s", "-", "D4 Boosting"), | |
| 204 | + ("D3 Random forest", "#e08a8a", "s", (0, (5, 2)), | |
| 205 | + "D3 Random forest")) | |
| 206 | + for m, color, mk, ls, label in series: | |
| 207 | + g = q[q["model"] == m].sort_values("quarter") | |
| 208 | + ax.plot(range(len(g)), g["mdape"], marker=mk, ms=4, color=color, | |
| 209 | + ls=ls, lw=1.1, mfc="white" if mk == "s" else color, | |
| 210 | + label=label) | |
| 211 | + g = q[q["model"] == series[0][0]].sort_values("quarter") | |
| 212 | + ax.set_xticks(range(len(g))) | |
| 213 | + ax.set_xticklabels(g["quarter"], fontsize=8, rotation=0) | |
| 214 | + ax.set_xlabel("Test quarter (training ends December 2024)") | |
| 215 | + ax.set_ylabel("Median absolute error (%)") | |
| 216 | + ax.legend(fontsize=7.5, ncol=2, loc="upper left") | |
| 217 | + ygrid(ax) | |
| 218 | + _save(fig, "fig_horizon.png") | |
| 219 | + | |
| 220 | + | |
| 189 | 221 | def main() -> None: |
| 190 | 222 | config.ensure_dirs() |
| 191 | 223 | apply_style() |
@@ -195,6 +227,7 @@ def main() -> None: | ||
| 195 | 227 | fig_profiles() |
| 196 | 228 | fig_learning() |
| 197 | 229 | fig_segments() |
| 230 | + fig_horizon() | |
| 198 | 231 | made = sorted(p.name for p in OUT.glob("fig_*.png")) |
| 199 | 232 | print(f"{len(made)} figures written:", ", ".join(made)) |
| 200 | 233 | |
modified
scripts/05_make_tables.py
+306 −0
@@ -133,6 +133,305 @@ def t_segments(): | ||
| 133 | 133 | (TAB / "segments.tex").write_text("\n".join(lines)) |
| 134 | 134 | |
| 135 | 135 | |
| 136 | +# ---------------------------------------------------------------- revision 1.1 | |
| 137 | +SHORT = {"A1 Linear (levels)": "A1 Linear", "A2 Semi-log": "A2 Semi-log", | |
| 138 | + "A3 Log-log": "A3 Log-log", "A4 Box-Cox": "A4 Box--Cox", | |
| 139 | + "A5 Semi-log + quadratics": "A5 Quadratics", | |
| 140 | + "A6 Semi-log + splines": "A6 Splines", | |
| 141 | + "B1 No time effects": "B1 No time FE", "B2 Year FE": "B2 Year FE", | |
| 142 | + "B3 Quarter FE": "B3 Quarter FE", "B4 Month FE": "B4 Month FE", | |
| 143 | + "C1 No spatial controls": "C1 No spatial FE", | |
| 144 | + "C2 Municipality FE": "C2 Municipality FE", | |
| 145 | + "C3 Grid-cell FE (~5.5 km)": "C3 Grid 5.5 km", | |
| 146 | + "C4 Grid-cell FE (~1.1 km)": "C4 Grid 1.1 km", | |
| 147 | + "D1 OLS (muni + month FE)": "D1 OLS", "D2 Ridge (cross-validated)": | |
| 148 | + "D2 Ridge", "D3 Random forest": "D3 Random forest", | |
| 149 | + "D4 Gradient boosting": "D4 Boosting", | |
| 150 | + "D5 Gradient boosting, no coordinates": "D5 Boosting, no coord.", | |
| 151 | + "D6 Spatial k-NN comparables": "D6 $k$-NN comparables"} | |
| 152 | +A6, D4 = "A6 Semi-log + splines", "D4 Gradient boosting" | |
| 153 | + | |
| 154 | + | |
| 155 | +def _stars(p): | |
| 156 | + return "$^{***}$" if p < 0.01 else "$^{**}$" if p < 0.05 else \ | |
| 157 | + "$^{*}$" if p < 0.10 else "" | |
| 158 | + | |
| 159 | + | |
| 160 | +def t_inference(): | |
| 161 | + """MdAPE with cluster-bootstrap 95% CI, difference versus the spline | |
| 162 | + hedonic (A6) with its CI, and the clustered DM t statistic on the APE | |
| 163 | + loss differential, under both holdouts.""" | |
| 164 | + lines = [r"\begin{tabular}{l ccc c ccc}", r"\toprule", | |
| 165 | + r" & \multicolumn{3}{c}{Random holdout} & &" | |
| 166 | + r" \multicolumn{3}{c}{Forward-in-time holdout} \\", | |
| 167 | + r"\cmidrule{2-4}\cmidrule{6-8}", | |
| 168 | + r"Model & MdAPE [95\% CI] & $\Delta$ vs.\ A6 [95\% CI] & " | |
| 169 | + r"$t_{\mathrm{DM}}$ & & MdAPE [95\% CI] & " | |
| 170 | + r"$\Delta$ vs.\ A6 [95\% CI] & $t_{\mathrm{DM}}$ \\", | |
| 171 | + r"\midrule"] | |
| 172 | + R = {s: pd.read_csv(RES / f"inference_{s}.csv").set_index("model") | |
| 173 | + for s in ("random", "temporal")} | |
| 174 | + groups = {"A": "A. Functional form", "B": "B. Time effects", | |
| 175 | + "C": "C. Spatial controls", "D": "D. Estimation method"} | |
| 176 | + cur = None | |
| 177 | + for m in SHORT: | |
| 178 | + if m[0] != cur: | |
| 179 | + cur = m[0] | |
| 180 | + lines.append(r"\multicolumn{8}{l}{\itshape " + groups[cur] | |
| 181 | + + r"}\\") | |
| 182 | + cells = [r"\quad " + SHORT[m]] | |
| 183 | + for s in ("random", "temporal"): | |
| 184 | + x = R[s].loc[m] | |
| 185 | + cells.append(f"{x['mdape']:.1f} [{x['ci_lo']:.1f}, " | |
| 186 | + f"{x['ci_hi']:.1f}]") | |
| 187 | + if m == A6: | |
| 188 | + cells += ["---", "---"] | |
| 189 | + else: | |
| 190 | + cells.append(f"{x['diff_vs_ref']:+.1f} [{x['diff_lo']:+.1f}, " | |
| 191 | + f"{x['diff_hi']:+.1f}]") | |
| 192 | + cells.append(f"{x['dm_ape_vs_a6_t']:.1f}" | |
| 193 | + + _stars(x['dm_ape_vs_a6_p'])) | |
| 194 | + if s == "random": | |
| 195 | + cells.append("") | |
| 196 | + lines.append(" & ".join(cells) + r" \\") | |
| 197 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 198 | + (TAB / "inference.tex").write_text("\n".join(lines)) | |
| 199 | + | |
| 200 | + | |
| 201 | +def t_retransformation(): | |
| 202 | + """Median-consistent versus Duan-smeared (mean-consistent) predictions.""" | |
| 203 | + sel = ["A2 Semi-log", "A3 Log-log", "A4 Box-Cox", | |
| 204 | + "A5 Semi-log + quadratics", "A6 Semi-log + splines", | |
| 205 | + "D2 Ridge (cross-validated)", "D3 Random forest", | |
| 206 | + "D4 Gradient boosting"] | |
| 207 | + R = {s: pd.read_csv(RES / f"horserace_{s}.csv").set_index("name") | |
| 208 | + for s in ("random", "temporal")} | |
| 209 | + lines = [r"\begin{tabular}{l c cc cc c cc cc}", r"\toprule", | |
| 210 | + r" & & \multicolumn{4}{c}{Random holdout} & &" | |
| 211 | + r" \multicolumn{4}{c}{Forward-in-time holdout} \\", | |
| 212 | + r"\cmidrule{3-6}\cmidrule{8-11}", | |
| 213 | + r" & & \multicolumn{2}{c}{MdAPE (\%)} & " | |
| 214 | + r"\multicolumn{2}{c}{$R^2_{\ln}$} & & " | |
| 215 | + r"\multicolumn{2}{c}{MdAPE (\%)} & " | |
| 216 | + r"\multicolumn{2}{c}{$R^2_{\ln}$} \\", | |
| 217 | + r"\cmidrule{3-4}\cmidrule{5-6}\cmidrule{8-9}\cmidrule{10-11}", | |
| 218 | + r"Model & $\hat{s}$ & Median & Mean & Median & Mean & & " | |
| 219 | + r"Median & Mean & Median & Mean \\", r"\midrule"] | |
| 220 | + for m in sel: | |
| 221 | + a, b = R["random"].loc[m], R["temporal"].loc[m] | |
| 222 | + sm = "---" if pd.isna(a["smear"]) else f"{a['smear']:.3f}" | |
| 223 | + lines.append( | |
| 224 | + f"{SHORT[m]} & {sm} & {a['mdape']:.1f} & {a['mdape_mean']:.1f} & " | |
| 225 | + f"{a['r2_ln']:.3f} & {a['r2_ln_mean']:.3f} & & " | |
| 226 | + f"{b['mdape']:.1f} & {b['mdape_mean']:.1f} & " | |
| 227 | + f"{b['r2_ln']:.3f} & {b['r2_ln_mean']:.3f} \\\\") | |
| 228 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 229 | + (TAB / "retransformation.tex").write_text("\n".join(lines)) | |
| 230 | + | |
| 231 | + | |
| 232 | +def t_same_period(): | |
| 233 | + sp = pd.read_csv(RES / "same_period.csv").set_index("model") | |
| 234 | + rl = pd.read_csv(RES / "relevel.csv") | |
| 235 | + rl = rl[rl["scheme"] == "temporal"].set_index("model") | |
| 236 | + sel = ["A1 Linear (levels)", "A5 Semi-log + quadratics", | |
| 237 | + "A6 Semi-log + splines", "B4 Month FE", | |
| 238 | + "C3 Grid-cell FE (~5.5 km)", "C4 Grid-cell FE (~1.1 km)", | |
| 239 | + "D2 Ridge (cross-validated)", "D3 Random forest", | |
| 240 | + "D4 Gradient boosting", "D5 Gradient boosting, no coordinates", | |
| 241 | + "D6 Spatial k-NN comparables"] | |
| 242 | + lines = [r"\begin{tabular}{l cc c c c c}", r"\toprule", | |
| 243 | + r" & \multicolumn{2}{c}{Random split} & Forward split & " | |
| 244 | + r"Staleness & Level error & Re-levelled \\", | |
| 245 | + r"\cmidrule{2-3}\cmidrule{4-4}", | |
| 246 | + r"Model & All test sales & 2025--26 sales & 2025--26 sales & " | |
| 247 | + r"cost & $\hat{b}$ & MdAPE \\", r"\midrule"] | |
| 248 | + for m in sel: | |
| 249 | + x = sp.loc[m] | |
| 250 | + lines.append(f"{SHORT[m]} & {x['mdape_random_all']:.1f} & " | |
| 251 | + f"{x['mdape_random_2025_26']:.1f} & " | |
| 252 | + f"{x['mdape_forward_2025_26']:.1f} & " | |
| 253 | + f"{x['mdape_forward_2025_26'] - x['mdape_random_2025_26']:+.1f} & " | |
| 254 | + f"{rl.loc[m, 'median_log_resid']:+.3f} & " | |
| 255 | + f"{rl.loc[m, 'mdape_relevelled']:.1f} \\\\") | |
| 256 | + n1, n2 = int(sp["n_random_2025_26"].iloc[0]), int(sp["n_forward"].iloc[0]) | |
| 257 | + lines += [r"\midrule", f"Test sales & 102,929 & {n1:,} & {n2:,} & & & \\\\", | |
| 258 | + r"\bottomrule", r"\end{tabular}"] | |
| 259 | + (TAB / "same_period.tex").write_text("\n".join(lines)) | |
| 260 | + | |
| 261 | + | |
| 262 | +def t_pairs(): | |
| 263 | + pr = pd.read_csv(RES / "pairs.csv") | |
| 264 | + lines = [r"\begin{tabular}{ll c cc c cc}", r"\toprule", | |
| 265 | + r" & & & \multicolumn{2}{c}{Random holdout} & &" | |
| 266 | + r" \multicolumn{2}{c}{Forward-in-time} \\", | |
| 267 | + r"\cmidrule{4-5}\cmidrule{7-8}", | |
| 268 | + r"Comparison & & MdAPE & $\Delta$MdAPE [95\% CI] & " | |
| 269 | + r"$t_{\mathrm{DM}}$ APE / sq.\ log & & " | |
| 270 | + r"$\Delta$MdAPE [95\% CI] & $t_{\mathrm{DM}}$ APE / sq.\ log \\", | |
| 271 | + r"\midrule"] | |
| 272 | + pairs = list(dict.fromkeys(zip(pr["model_a"], pr["model_b"]))) | |
| 273 | + for a, b in pairs: | |
| 274 | + r = pr[(pr["model_a"] == a) & (pr["model_b"] == b)].set_index("scheme") | |
| 275 | + x, y = r.loc["random"], r.loc["temporal"] | |
| 276 | + cells = [SHORT[a] + " vs.", SHORT[b], | |
| 277 | + f"{x['mdape_a']:.1f} / {x['mdape_b']:.1f}", | |
| 278 | + f"{x['diff']:+.2f} [{x['diff_lo']:+.2f}, {x['diff_hi']:+.2f}]", | |
| 279 | + f"{x['dm_ape_t']:.1f}{_stars(x['dm_ape_p'])} / " | |
| 280 | + f"{x['dm_sqlog_t']:.1f}{_stars(x['dm_sqlog_p'])}", "", | |
| 281 | + f"{y['diff']:+.2f} [{y['diff_lo']:+.2f}, {y['diff_hi']:+.2f}]", | |
| 282 | + f"{y['dm_ape_t']:.1f}{_stars(y['dm_ape_p'])} / " | |
| 283 | + f"{y['dm_sqlog_t']:.1f}{_stars(y['dm_sqlog_p'])}"] | |
| 284 | + lines.append(" & ".join(cells) + r" \\") | |
| 285 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 286 | + (TAB / "pairs.tex").write_text("\n".join(lines)) | |
| 287 | + | |
| 288 | + | |
| 289 | +def t_group_splits(): | |
| 290 | + R = {s: pd.read_csv(RES / f"horserace_{s}.csv").set_index("name") | |
| 291 | + for s in ("random", "group_parcel", "group_coord")} | |
| 292 | + lines = [r"\begin{tabular}{l cc c cc c cc}", r"\toprule", | |
| 293 | + r" & \multicolumn{2}{c}{Random 80/20} & &" | |
| 294 | + r" \multicolumn{2}{c}{Parcel-grouped} & &" | |
| 295 | + r" \multicolumn{2}{c}{Building-grouped} \\", | |
| 296 | + r"\cmidrule{2-3}\cmidrule{5-6}\cmidrule{8-9}", | |
| 297 | + r"Model & MdAPE & $R^2_{\ln}$ & & MdAPE & $R^2_{\ln}$ & & " | |
| 298 | + r"MdAPE & $R^2_{\ln}$ \\", r"\midrule"] | |
| 299 | + groups = {"A": "A. Functional form", "B": "B. Time effects", | |
| 300 | + "C": "C. Spatial controls", "D": "D. Estimation method"} | |
| 301 | + cur = None | |
| 302 | + for m in SHORT: | |
| 303 | + if m[0] != cur: | |
| 304 | + cur = m[0] | |
| 305 | + lines.append(r"\multicolumn{9}{l}{\itshape " + groups[cur] | |
| 306 | + + r"}\\") | |
| 307 | + cells = [r"\quad " + SHORT[m]] | |
| 308 | + for i, s in enumerate(("random", "group_parcel", "group_coord")): | |
| 309 | + x = R[s].loc[m] | |
| 310 | + cells += [f"{x['mdape']:.1f}", f"{x['r2_ln']:.3f}"] | |
| 311 | + if i < 2: | |
| 312 | + cells.append("") | |
| 313 | + lines.append(" & ".join(cells) + r" \\") | |
| 314 | + ns = [f"{int(R[s]['n_test'].iloc[0]):,}" for s in | |
| 315 | + ("random", "group_parcel", "group_coord")] | |
| 316 | + lines += [r"\midrule", | |
| 317 | + f"Test sales & {ns[0]} & & & {ns[1]} & & & {ns[2]} & \\\\", | |
| 318 | + r"\bottomrule", r"\end{tabular}"] | |
| 319 | + (TAB / "group_splits.tex").write_text("\n".join(lines)) | |
| 320 | + | |
| 321 | + | |
| 322 | +def t_leakage_decomp(): | |
| 323 | + L = pd.read_csv(RES / "leakage.csv") | |
| 324 | + sel = ["A5 Semi-log + quadratics", "A6 Semi-log + splines", | |
| 325 | + "D3 Random forest", "D4 Gradient boosting", | |
| 326 | + "D5 Gradient boosting, no coordinates", | |
| 327 | + "D6 Spatial k-NN comparables"] | |
| 328 | + lines = [r"\begin{tabular}{l cc c cc}", r"\toprule", | |
| 329 | + r" & \multicolumn{2}{c}{Same parcel in training} & &" | |
| 330 | + r" \multicolumn{2}{c}{Same coordinates in training} \\", | |
| 331 | + r"\cmidrule{2-3}\cmidrule{5-6}", | |
| 332 | + r"Model & Yes & No & & Yes & No \\", r"\midrule"] | |
| 333 | + def cell(key, flag, m, seg="all"): | |
| 334 | + x = L[(L["key"] == key) & (L["seen_in_training"] == flag) | |
| 335 | + & (L["model"] == m) & (L["segment"] == seg)] | |
| 336 | + return f"{x['mdape'].iloc[0]:.1f}" if len(x) else "---" | |
| 337 | + def n(key, flag, seg="all"): | |
| 338 | + x = L[(L["key"] == key) & (L["seen_in_training"] == flag) | |
| 339 | + & (L["segment"] == seg)] | |
| 340 | + return f"{int(x['n'].iloc[0]):,}" | |
| 341 | + lines.append(r"\multicolumn{6}{l}{\itshape All test sales}\\") | |
| 342 | + for m in sel: | |
| 343 | + lines.append(f"\\quad {SHORT[m]} & {cell('in_train_parcel', True, m)} & " | |
| 344 | + f"{cell('in_train_parcel', False, m)} & & " | |
| 345 | + f"{cell('in_train_coord', True, m)} & " | |
| 346 | + f"{cell('in_train_coord', False, m)} \\\\") | |
| 347 | + lines.append(f"\\quad Test sales & {n('in_train_parcel', True)} & " | |
| 348 | + f"{n('in_train_parcel', False)} & & {n('in_train_coord', True)} & " | |
| 349 | + f"{n('in_train_coord', False)} \\\\") | |
| 350 | + lines.append(r"\midrule") | |
| 351 | + lines.append(r"\multicolumn{6}{l}{\itshape Condominiums only}\\") | |
| 352 | + for m in ("A5 Semi-log + quadratics", "D4 Gradient boosting"): | |
| 353 | + lines.append(f"\\quad {SHORT[m]} & {cell('in_train_parcel', True, m, 'condo')} & " | |
| 354 | + f"{cell('in_train_parcel', False, m, 'condo')} & & " | |
| 355 | + f"{cell('in_train_coord', True, m, 'condo')} & " | |
| 356 | + f"{cell('in_train_coord', False, m, 'condo')} \\\\") | |
| 357 | + lines.append(f"\\quad Test sales & {n('in_train_parcel', True, 'condo')} & " | |
| 358 | + f"{n('in_train_parcel', False, 'condo')} & & " | |
| 359 | + f"{n('in_train_coord', True, 'condo')} & " | |
| 360 | + f"{n('in_train_coord', False, 'condo')} \\\\") | |
| 361 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 362 | + (TAB / "leakage.tex").write_text("\n".join(lines)) | |
| 363 | + | |
| 364 | + | |
| 365 | +def t_nested_fe(): | |
| 366 | + N = pd.read_csv(RES / "nested_fe.csv") | |
| 367 | + R = {s: pd.read_csv(RES / f"horserace_{s}.csv").set_index("name") | |
| 368 | + for s in ("random", "temporal")} | |
| 369 | + rows = [("C2 Municipality FE", "C2 Municipality FE (1,115 cells)"), | |
| 370 | + ("C3 Grid-cell FE (~5.5 km)", "C3 Grid 5.5 km (5,255 cells)"), | |
| 371 | + ("C4 Grid-cell FE (~1.1 km)", "C4 Grid 1.1 km (30,548 cells)"), | |
| 372 | + ("C3' Municipality + grid-cell FE (~5.5 km)", | |
| 373 | + "C3$'$ Municipality + grid 5.5 km"), | |
| 374 | + ("C4' Municipality + grid-cell FE (~1.1 km)", | |
| 375 | + "C4$'$ Municipality + grid 1.1 km")] | |
| 376 | + lines = [r"\begin{tabular}{l cc c cc}", r"\toprule", | |
| 377 | + r" & \multicolumn{2}{c}{Random holdout} & &" | |
| 378 | + r" \multicolumn{2}{c}{Forward-in-time} \\", | |
| 379 | + r"\cmidrule{2-3}\cmidrule{5-6}", | |
| 380 | + r"Spatial controls & MdAPE (\%) & $R^2_{\ln}$ & & MdAPE (\%) & " | |
| 381 | + r"$R^2_{\ln}$ \\", r"\midrule"] | |
| 382 | + for key, label in rows: | |
| 383 | + vals = [] | |
| 384 | + for s in ("random", "temporal"): | |
| 385 | + if key in R[s].index: | |
| 386 | + x = R[s].loc[key] | |
| 387 | + else: | |
| 388 | + x = N[(N["name"] == key) & (N["scheme"] == s)].iloc[0] | |
| 389 | + vals += [f"{x['mdape']:.1f}", f"{x['r2_ln']:.3f}"] | |
| 390 | + lines.append(f"{label} & {vals[0]} & {vals[1]} & & {vals[2]} & " | |
| 391 | + f"{vals[3]} \\\\") | |
| 392 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 393 | + (TAB / "nested_fe.tex").write_text("\n".join(lines)) | |
| 394 | + | |
| 395 | + | |
| 396 | +def t_hgb_sensitivity(): | |
| 397 | + H = pd.read_csv(RES / "hgb_sensitivity.csv") | |
| 398 | + names = list(dict.fromkeys(H["name"])) | |
| 399 | + lines = [r"\begin{tabular}{l cc c cc}", r"\toprule", | |
| 400 | + r" & \multicolumn{2}{c}{Random holdout} & &" | |
| 401 | + r" \multicolumn{2}{c}{Forward-in-time} \\", | |
| 402 | + r"\cmidrule{2-3}\cmidrule{5-6}", | |
| 403 | + r"Gradient-boosting configuration & MdAPE (\%) & $R^2_{\ln}$ & & " | |
| 404 | + r"MdAPE (\%) & $R^2_{\ln}$ \\", r"\midrule"] | |
| 405 | + for nm in names: | |
| 406 | + a = H[(H["name"] == nm) & (H["scheme"] == "random")].iloc[0] | |
| 407 | + b = H[(H["name"] == nm) & (H["scheme"] == "temporal")].iloc[0] | |
| 408 | + label = _esc(nm.replace("D4 ", "").replace("%", r"\%")) | |
| 409 | + label = label[0].upper() + label[1:] | |
| 410 | + if not pd.isna(b.get("best_iter", float("nan"))): | |
| 411 | + label += f" [{int(a['best_iter'])} / {int(b['best_iter'])} it.]" | |
| 412 | + lines.append(f"{label} & {a['mdape']:.1f} & {a['r2_ln']:.3f} & & " | |
| 413 | + f"{b['mdape']:.1f} & {b['r2_ln']:.3f} \\\\") | |
| 414 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 415 | + (TAB / "hgb_sensitivity.tex").write_text("\n".join(lines)) | |
| 416 | + | |
| 417 | + | |
| 418 | +def t_by_quarter(): | |
| 419 | + Q = pd.read_csv(RES / "by_quarter.csv") | |
| 420 | + piv = Q.pivot_table(index="quarter", columns="model", values="mdape") | |
| 421 | + ns = Q.groupby("quarter")["n"].first() | |
| 422 | + cols = ["A5 Semi-log + quadratics", "A6 Semi-log + splines", | |
| 423 | + "D2 Ridge (cross-validated)", "D3 Random forest", | |
| 424 | + "D4 Gradient boosting"] | |
| 425 | + lines = [r"\begin{tabular}{lr" + "c" * len(cols) + "}", r"\toprule", | |
| 426 | + r"Test quarter & $n$ & " + " & ".join(SHORT[c] for c in cols) | |
| 427 | + + r" \\", r"\midrule"] | |
| 428 | + for q, row in piv.iterrows(): | |
| 429 | + lines.append(f"{q} & {ns[q]:,.0f} & " | |
| 430 | + + " & ".join(f"{row[c]:.1f}" for c in cols) + r" \\") | |
| 431 | + lines += [r"\bottomrule", r"\end{tabular}"] | |
| 432 | + (TAB / "by_quarter.tex").write_text("\n".join(lines)) | |
| 433 | + | |
| 434 | + | |
| 136 | 435 | def main() -> None: |
| 137 | 436 | config.ensure_dirs() |
| 138 | 437 | t_summary() |
@@ -142,6 +441,13 @@ def main() -> None: | ||
| 142 | 441 | t_boxcox() |
| 143 | 442 | t_learning() |
| 144 | 443 | t_segments() |
| 444 | + for fn in (t_inference, t_retransformation, t_same_period, t_pairs, | |
| 445 | + t_group_splits, t_leakage_decomp, t_nested_fe, | |
| 446 | + t_hgb_sensitivity, t_by_quarter): | |
| 447 | + try: | |
| 448 | + fn() | |
| 449 | + except FileNotFoundError as e: # step 06 not run yet | |
| 450 | + print(f" skipped {fn.__name__}: {e}") | |
| 145 | 451 | made = sorted(p.name for p in TAB.glob("*.tex")) |
| 146 | 452 | print(f"{len(made)} tables written:", ", ".join(made)) |
| 147 | 453 | |
added
scripts/06_robustness.py
+274 −0
@@ -0,0 +1,274 @@ | ||
| 1 | +#!/usr/bin/env python3 | |
| 2 | +# Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 3 | +"""Step 06 — Inference and robustness (added in the 2026-10-17 revision). | |
| 4 | + | |
| 5 | +Reads the test-set predictions stored by step 02 and runs the additional | |
| 6 | +analyses requested by the editorial review: | |
| 7 | + | |
| 8 | +1. cluster-bootstrap confidence intervals (municipality clusters) for the | |
| 9 | + MdAPE of every model, and Diebold–Mariano / Giacomini–White-type tests | |
| 10 | + of equal accuracy against the spline hedonic (A6) and gradient | |
| 11 | + boosting (D4), under both holdouts -> inference_<scheme>.csv | |
| 12 | +2. same-period comparison: random-split models evaluated on the 2025–26 | |
| 13 | + sales of the random test set versus forward-in-time models on the | |
| 14 | + full 2025–26 test set -> same_period.csv | |
| 15 | +3. forward-in-time accuracy by test quarter -> by_quarter.csv | |
| 16 | +4. leakage decomposition of the random holdout: accuracy on test sales | |
| 17 | + whose parcel / coordinate pair also appears in training versus the | |
| 18 | + others -> leakage.csv | |
| 19 | +5. leakage-free holdouts: the full race under parcel-grouped and | |
| 20 | + coordinate-grouped 80/20 splits -> horserace_group_parcel.csv, | |
| 21 | + horserace_group_coord.csv | |
| 22 | +6. nested spatial fixed effects (municipality + grid) -> nested_fe.csv | |
| 23 | +7. Box–Cox on a finer lambda grid -> boxcox_fine.csv | |
| 24 | +8. gradient-boosting hyper-parameter sensitivity -> hgb_sensitivity.csv | |
| 25 | +10. targeted pairwise comparisons along each axis (bootstrap CI of the | |
| 26 | + MdAPE difference and clustered DM tests) -> pairs.csv | |
| 27 | +9. level-bias decomposition: median log residual (aggregate level error) | |
| 28 | + and MdAPE after re-levelling predictions by the test-set median ratio | |
| 29 | + (relative pricing skill net of the level error) -> relevel.csv | |
| 30 | + | |
| 31 | +Usage: python scripts/06_robustness.py (about 15 min) | |
| 32 | +""" | |
| 33 | +import sys | |
| 34 | +import time | |
| 35 | +from pathlib import Path | |
| 36 | + | |
| 37 | +import numpy as np | |
| 38 | +import pandas as pd | |
| 39 | + | |
| 40 | +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) | |
| 41 | + | |
| 42 | +from wp11 import config, inference, models, sample # noqa: E402 | |
| 43 | + | |
| 44 | +OUT = config.REPRODUCED | |
| 45 | +MODEL_NAMES = [m["name"] for m in models.model_registry()] | |
| 46 | +A6, D4, A5 = ("A6 Semi-log + splines", "D4 Gradient boosting", | |
| 47 | + "A5 Semi-log + quadratics") | |
| 48 | + | |
| 49 | + | |
| 50 | +def _mdape(y, p): | |
| 51 | + return float(np.median(np.abs(p - y) / y) * 100) | |
| 52 | + | |
| 53 | + | |
| 54 | +def step_inference(P: dict): | |
| 55 | + for scheme, Ps in P.items(): | |
| 56 | + t0 = time.time() | |
| 57 | + boot = inference.cluster_bootstrap_mdape(Ps, MODEL_NAMES, ref=A6) | |
| 58 | + boot_d4 = inference.cluster_bootstrap_mdape(Ps, MODEL_NAMES, ref=D4) | |
| 59 | + boot = boot.merge(boot_d4[["model", "diff_vs_ref", "diff_lo", | |
| 60 | + "diff_hi"]] | |
| 61 | + .rename(columns={"diff_vs_ref": "diff_vs_d4", | |
| 62 | + "diff_lo": "diff_d4_lo", | |
| 63 | + "diff_hi": "diff_d4_hi"}), | |
| 64 | + on="model", how="left") | |
| 65 | + rows = [] | |
| 66 | + for m in MODEL_NAMES: | |
| 67 | + r = {"model": m} | |
| 68 | + for ref, tag in ((A6, "a6"), (D4, "d4")): | |
| 69 | + if m == ref: | |
| 70 | + continue | |
| 71 | + for loss, ltag in ((inference.ape, "ape"), | |
| 72 | + (inference.sq_log, "sqlog")): | |
| 73 | + t = inference.dm_test(Ps, m, ref, loss=loss) | |
| 74 | + r[f"dm_{ltag}_vs_{tag}_t"] = t["t"] | |
| 75 | + r[f"dm_{ltag}_vs_{tag}_p"] = t["p"] | |
| 76 | + r[f"dm_{ltag}_vs_{tag}_diff"] = t["mean_diff"] | |
| 77 | + rows.append(r) | |
| 78 | + res = boot.merge(pd.DataFrame(rows), on="model") | |
| 79 | + res["n_clusters"] = Ps["muni"].nunique() | |
| 80 | + res.to_csv(OUT / f"inference_{scheme}.csv", index=False) | |
| 81 | + print(f" inference [{scheme}] done in {time.time() - t0:.0f}s") | |
| 82 | + | |
| 83 | + | |
| 84 | +def step_same_period(P: dict): | |
| 85 | + cut = pd.Timestamp(config.TEMPORAL_CUTOFF) | |
| 86 | + Pr = P["random"][P["random"]["sale_date"] >= cut] | |
| 87 | + Pt = P["temporal"] | |
| 88 | + rows = [] | |
| 89 | + for m in MODEL_NAMES: | |
| 90 | + rows.append({"model": m, | |
| 91 | + "mdape_random_all": _mdape(P["random"]["price"].to_numpy(), | |
| 92 | + P["random"][m].to_numpy()), | |
| 93 | + "mdape_random_2025_26": _mdape(Pr["price"].to_numpy(), | |
| 94 | + Pr[m].to_numpy()), | |
| 95 | + "mdape_forward_2025_26": _mdape(Pt["price"].to_numpy(), | |
| 96 | + Pt[m].to_numpy()), | |
| 97 | + "n_random_2025_26": len(Pr), "n_forward": len(Pt)}) | |
| 98 | + pd.DataFrame(rows).to_csv(OUT / "same_period.csv", index=False) | |
| 99 | + print(" same-period comparison done") | |
| 100 | + | |
| 101 | + | |
| 102 | +def step_by_quarter(P: dict): | |
| 103 | + Pt = P["temporal"].copy() | |
| 104 | + Pt["q"] = Pt["sale_date"].dt.to_period("Q").astype(str) | |
| 105 | + rows = [] | |
| 106 | + for q, g in Pt.groupby("q"): | |
| 107 | + for m in (A5, A6, "D2 Ridge (cross-validated)", "D3 Random forest", | |
| 108 | + D4): | |
| 109 | + rows.append({"quarter": q, "model": m, "n": len(g), | |
| 110 | + "mdape": _mdape(g["price"].to_numpy(), | |
| 111 | + g[m].to_numpy())}) | |
| 112 | + pd.DataFrame(rows).to_csv(OUT / "by_quarter.csv", index=False) | |
| 113 | + print(" by-quarter accuracy done") | |
| 114 | + | |
| 115 | + | |
| 116 | +def step_leakage(P: dict): | |
| 117 | + Pr = P["random"] | |
| 118 | + rows = [] | |
| 119 | + for key in ("in_train_parcel", "in_train_coord"): | |
| 120 | + for flag in (True, False): | |
| 121 | + g = Pr[Pr[key] == flag] | |
| 122 | + for m in (A5, A6, "D2 Ridge (cross-validated)", | |
| 123 | + "D3 Random forest", D4, | |
| 124 | + "D5 Gradient boosting, no coordinates", | |
| 125 | + "D6 Spatial k-NN comparables"): | |
| 126 | + rows.append({"key": key, "seen_in_training": flag, | |
| 127 | + "segment": "all", "n": len(g), "model": m, | |
| 128 | + "mdape": _mdape(g["price"].to_numpy(), | |
| 129 | + g[m].to_numpy())}) | |
| 130 | + gc = g[g["prop_class"] == "condo"] | |
| 131 | + for m in (A5, D4): | |
| 132 | + rows.append({"key": key, "seen_in_training": flag, | |
| 133 | + "segment": "condo", "n": len(gc), "model": m, | |
| 134 | + "mdape": _mdape(gc["price"].to_numpy(), | |
| 135 | + gc[m].to_numpy())}) | |
| 136 | + pd.DataFrame(rows).to_csv(OUT / "leakage.csv", index=False) | |
| 137 | + print(" leakage decomposition done") | |
| 138 | + | |
| 139 | + | |
| 140 | +def step_relevel(P: dict): | |
| 141 | + """Decompose forward-in-time error into an aggregate level error and | |
| 142 | + relative pricing skill. For each model the median log residual | |
| 143 | + b = median(ln y - ln p) measures the common level error (staleness); | |
| 144 | + the re-levelled MdAPE uses p * exp(b), i.e. an oracle time adjustment | |
| 145 | + of the kind practitioners obtain from a published index.""" | |
| 146 | + rows = [] | |
| 147 | + for scheme, Ps in P.items(): | |
| 148 | + y = Ps["price"].to_numpy(float) | |
| 149 | + for m in MODEL_NAMES: | |
| 150 | + p = Ps[m].to_numpy(float) | |
| 151 | + b = float(np.median(np.log(y) - np.log(p))) | |
| 152 | + rows.append({"scheme": scheme, "model": m, | |
| 153 | + "median_log_resid": b, | |
| 154 | + "mdape": _mdape(y, p), | |
| 155 | + "mdape_relevelled": _mdape(y, p * np.exp(b)), | |
| 156 | + "mape_relevelled": float(np.mean( | |
| 157 | + np.abs(p * np.exp(b) - y) / y) * 100), | |
| 158 | + "rmse_ln_centered": float(np.sqrt(np.mean( | |
| 159 | + (np.log(y) - np.log(p) - b) ** 2)))}) | |
| 160 | + pd.DataFrame(rows).to_csv(OUT / "relevel.csv", index=False) | |
| 161 | + print(" level-bias decomposition done") | |
| 162 | + | |
| 163 | + | |
| 164 | +PAIRS = [("A2 Semi-log", "A1 Linear (levels)"), | |
| 165 | + ("A6 Semi-log + splines", "A2 Semi-log"), | |
| 166 | + ("A4 Box-Cox", "A2 Semi-log"), | |
| 167 | + ("A3 Log-log", "A2 Semi-log"), | |
| 168 | + ("B2 Year FE", "B1 No time effects"), | |
| 169 | + ("B4 Month FE", "B3 Quarter FE"), | |
| 170 | + ("C3 Grid-cell FE (~5.5 km)", "C2 Municipality FE"), | |
| 171 | + ("C4 Grid-cell FE (~1.1 km)", "C2 Municipality FE"), | |
| 172 | + ("D2 Ridge (cross-validated)", "D1 OLS (muni + month FE)"), | |
| 173 | + ("D4 Gradient boosting", "D3 Random forest"), | |
| 174 | + ("D4 Gradient boosting", "D5 Gradient boosting, no coordinates"), | |
| 175 | + ("D4 Gradient boosting", "A1 Linear (levels)"), | |
| 176 | + ("A6 Semi-log + splines", "A1 Linear (levels)")] | |
| 177 | + | |
| 178 | + | |
| 179 | +def step_pairs(P: dict): | |
| 180 | + """Targeted comparisons: for each (A, B) the MdAPE difference A - B | |
| 181 | + with its cluster-bootstrap interval and the clustered DM statistics | |
| 182 | + on the APE and squared-log losses (positive t = B more accurate).""" | |
| 183 | + rows = [] | |
| 184 | + for scheme, Ps in P.items(): | |
| 185 | + for a, b in PAIRS: | |
| 186 | + boot = inference.cluster_bootstrap_mdape(Ps, [a, b], ref=b) | |
| 187 | + x = boot.set_index("model").loc[a] | |
| 188 | + t1 = inference.dm_test(Ps, a, b, loss=inference.ape) | |
| 189 | + t2 = inference.dm_test(Ps, a, b, loss=inference.sq_log) | |
| 190 | + rows.append({"scheme": scheme, "model_a": a, "model_b": b, | |
| 191 | + "mdape_a": x["mdape"], | |
| 192 | + "mdape_b": boot.set_index("model").loc[b, "mdape"], | |
| 193 | + "diff": x["diff_vs_ref"], "diff_lo": x["diff_lo"], | |
| 194 | + "diff_hi": x["diff_hi"], | |
| 195 | + "dm_ape_t": t1["t"], "dm_ape_p": t1["p"], | |
| 196 | + "dm_sqlog_t": t2["t"], "dm_sqlog_p": t2["p"]}) | |
| 197 | + pd.DataFrame(rows).to_csv(OUT / "pairs.csv", index=False) | |
| 198 | + print(" targeted pairwise comparisons done") | |
| 199 | + | |
| 200 | + | |
| 201 | +def step_group_splits(df): | |
| 202 | + for scheme in ("group_parcel", "group_coord"): | |
| 203 | + res = models.run_horserace(df, scheme) | |
| 204 | + res.to_csv(OUT / f"horserace_{scheme}.csv", index=False) | |
| 205 | + | |
| 206 | + | |
| 207 | +def step_nested_fe(df): | |
| 208 | + rows = [] | |
| 209 | + for scheme in ("random", "temporal"): | |
| 210 | + res = models.run_horserace(df, scheme, | |
| 211 | + registry=models.robustness_registry()) | |
| 212 | + res["scheme"] = scheme | |
| 213 | + rows.append(res) | |
| 214 | + pd.concat(rows).to_csv(OUT / "nested_fe.csv", index=False) | |
| 215 | + | |
| 216 | + | |
| 217 | +def step_boxcox_fine(df): | |
| 218 | + d = models.add_derived(df) | |
| 219 | + d["sale_year_c"] = d["sale_year"].astype(str) | |
| 220 | + train, test = models.split(d, "random") | |
| 221 | + grid = [round(x, 2) for x in np.arange(-0.25, 0.751, 0.05)] | |
| 222 | + lam, prof = models.fit_boxcox(train, models.NUM_LEVELS, | |
| 223 | + models.CATS + ["muni", "quarter"], | |
| 224 | + grid=grid) | |
| 225 | + prof["chosen"] = prof["lambda"] == lam | |
| 226 | + # out-of-sample accuracy of the Box–Cox model at the refined lambda | |
| 227 | + spec = dict(name="A4 fine", group="x", kind="boxcox", | |
| 228 | + numeric=models.NUM_LEVELS, fe=["muni", "quarter"], | |
| 229 | + boxcox_grid=[lam]) | |
| 230 | + pred, _ = models.fit_predict(spec, train, test) | |
| 231 | + prof.attrs = {} | |
| 232 | + prof["mdape_at_chosen"] = np.where(prof["chosen"], | |
| 233 | + _mdape(test["price"].to_numpy(float), | |
| 234 | + np.clip(pred, config.PRED_FLOOR, | |
| 235 | + None)), np.nan) | |
| 236 | + prof.to_csv(OUT / "boxcox_fine.csv", index=False) | |
| 237 | + print(f" Box–Cox fine grid: lambda* = {lam}") | |
| 238 | + | |
| 239 | + | |
| 240 | +def step_hgb_sensitivity(df): | |
| 241 | + rows = [] | |
| 242 | + for scheme in ("random", "temporal"): | |
| 243 | + reg = [dict(name="D4 Gradient boosting (baseline)", group="S", | |
| 244 | + kind="hgb")] | |
| 245 | + reg += [dict(name=f"D4 {label}", group="S", kind="hgb", | |
| 246 | + hgb_params=params) | |
| 247 | + for label, params in models.HGB_ALTERNATIVES.items()] | |
| 248 | + res = models.run_horserace(df, scheme, registry=reg) | |
| 249 | + res["scheme"] = scheme | |
| 250 | + rows.append(res) | |
| 251 | + pd.concat(rows).to_csv(OUT / "hgb_sensitivity.csv", index=False) | |
| 252 | + | |
| 253 | + | |
| 254 | +def main() -> None: | |
| 255 | + config.ensure_dirs() | |
| 256 | + df = sample.load() | |
| 257 | + P = {s: pd.read_parquet(config.PROCESSED / f"predictions_{s}.parquet") | |
| 258 | + for s in ("random", "temporal")} | |
| 259 | + t0 = time.time() | |
| 260 | + step_inference(P) | |
| 261 | + step_same_period(P) | |
| 262 | + step_by_quarter(P) | |
| 263 | + step_leakage(P) | |
| 264 | + step_relevel(P) | |
| 265 | + step_pairs(P) | |
| 266 | + step_group_splits(df) | |
| 267 | + step_nested_fe(df) | |
| 268 | + step_boxcox_fine(df) | |
| 269 | + step_hgb_sensitivity(df) | |
| 270 | + print(f"Robustness step finished in {(time.time() - t0) / 60:.1f} min") | |
| 271 | + | |
| 272 | + | |
| 273 | +if __name__ == "__main__": | |
| 274 | + main() | |
added
src/wp11/inference.py
+95 −0
@@ -0,0 +1,95 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher — contact@spboucher.ai | |
| 2 | +"""Inference for out-of-sample accuracy comparisons. | |
| 3 | + | |
| 4 | +Two tools, both operating on the stored test-set predictions: | |
| 5 | + | |
| 6 | +* a *cluster bootstrap* (resampling municipalities with replacement) for | |
| 7 | + the MdAPE of each model and for the MdAPE difference between two | |
| 8 | + models evaluated on the same test sales; | |
| 9 | +* a Diebold–Mariano / Giacomini–White-type test of equal predictive | |
| 10 | + accuracy: the mean of the per-sale loss differential d_i = L(e_i^A) - | |
| 11 | + L(e_i^B) is tested against zero with standard errors clustered by | |
| 12 | + municipality (the cross-sectional analogue of the HAC-robust DM | |
| 13 | + statistic). Losses: absolute percentage error (the MAPE loss) and the | |
| 14 | + squared log error (the RMSE_ln loss). | |
| 15 | +""" | |
| 16 | +from __future__ import annotations | |
| 17 | + | |
| 18 | +import numpy as np | |
| 19 | +import pandas as pd | |
| 20 | + | |
| 21 | +from . import config | |
| 22 | + | |
| 23 | + | |
| 24 | +def ape(y: np.ndarray, p: np.ndarray) -> np.ndarray: | |
| 25 | + return np.abs(p - y) / y | |
| 26 | + | |
| 27 | + | |
| 28 | +def sq_log(y: np.ndarray, p: np.ndarray) -> np.ndarray: | |
| 29 | + return (np.log(y) - np.log(p)) ** 2 | |
| 30 | + | |
| 31 | + | |
| 32 | +def cluster_bootstrap_mdape(P: pd.DataFrame, models: list[str], | |
| 33 | + cluster: str = "muni", B: int = 500, | |
| 34 | + ref: str | None = None, | |
| 35 | + seed: int = config.SEED) -> pd.DataFrame: | |
| 36 | + """Percentile 95% intervals for MdAPE (and MdAPE minus the reference | |
| 37 | + model's MdAPE) from B cluster-bootstrap replicates. | |
| 38 | + | |
| 39 | + Clusters are municipalities; each replicate draws clusters with | |
| 40 | + replacement (so a drawn municipality contributes all its test sales, | |
| 41 | + possibly several times) and recomputes the median APE. | |
| 42 | + """ | |
| 43 | + y = P["price"].to_numpy(float) | |
| 44 | + A = {m: ape(y, P[m].to_numpy(float)) for m in models} | |
| 45 | + codes, uniq = pd.factorize(P[cluster]) | |
| 46 | + G = len(uniq) | |
| 47 | + rng = np.random.default_rng(seed) | |
| 48 | + # pre-compute row indices per cluster | |
| 49 | + order = np.argsort(codes, kind="stable") | |
| 50 | + bounds = np.searchsorted(codes[order], np.arange(G + 1)) | |
| 51 | + rows_by_cluster = [order[bounds[g]:bounds[g + 1]] for g in range(G)] | |
| 52 | + reps = {m: np.empty(B) for m in models} | |
| 53 | + for b in range(B): | |
| 54 | + draw = rng.integers(0, G, G) | |
| 55 | + idx = np.concatenate([rows_by_cluster[g] for g in draw]) | |
| 56 | + for m in models: | |
| 57 | + reps[m][b] = np.median(A[m][idx]) | |
| 58 | + out = [] | |
| 59 | + for m in models: | |
| 60 | + point = float(np.median(A[m]) * 100) | |
| 61 | + lo, hi = np.percentile(reps[m], [2.5, 97.5]) * 100 | |
| 62 | + row = {"model": m, "mdape": point, "ci_lo": lo, "ci_hi": hi, | |
| 63 | + "boot_se": float(reps[m].std(ddof=1) * 100)} | |
| 64 | + if ref is not None and m != ref: | |
| 65 | + d = (reps[m] - reps[ref]) * 100 | |
| 66 | + row.update({"diff_vs_ref": point - float(np.median(A[ref]) * 100), | |
| 67 | + "diff_lo": float(np.percentile(d, 2.5)), | |
| 68 | + "diff_hi": float(np.percentile(d, 97.5))}) | |
| 69 | + out.append(row) | |
| 70 | + return pd.DataFrame(out) | |
| 71 | + | |
| 72 | + | |
| 73 | +def dm_test(P: pd.DataFrame, model_a: str, model_b: str, | |
| 74 | + loss=ape, cluster: str = "muni") -> dict: | |
| 75 | + """Clustered test of E[d_i] = 0 for d_i = L_A(i) - L_B(i). | |
| 76 | + | |
| 77 | + Returns the mean differential (positive = A has larger loss, i.e. | |
| 78 | + B is more accurate), its cluster-robust standard error, the t | |
| 79 | + statistic and the two-sided normal p-value. | |
| 80 | + """ | |
| 81 | + y = P["price"].to_numpy(float) | |
| 82 | + d = loss(y, P[model_a].to_numpy(float)) - loss(y, P[model_b].to_numpy(float)) | |
| 83 | + codes, _ = pd.factorize(P[cluster]) | |
| 84 | + n, G = len(d), codes.max() + 1 | |
| 85 | + dbar = d.mean() | |
| 86 | + # cluster-robust variance of the mean: sum over clusters of squared | |
| 87 | + # within-cluster sums of demeaned d, divided by n^2 | |
| 88 | + sums = np.bincount(codes, weights=d - dbar, minlength=G) | |
| 89 | + var = (sums ** 2).sum() / n ** 2 * G / (G - 1) | |
| 90 | + se = float(np.sqrt(var)) | |
| 91 | + t = float(dbar / se) if se > 0 else np.nan | |
| 92 | + from math import erf, sqrt | |
| 93 | + p = 2 * (1 - 0.5 * (1 + erf(abs(t) / sqrt(2)))) if np.isfinite(t) else np.nan | |
| 94 | + return {"model_a": model_a, "model_b": model_b, "mean_diff": float(dbar), | |
| 95 | + "se": se, "t": t, "p": float(p), "n_clusters": int(G)} | |
modified
src/wp11/models.py
+220 −43
@@ -6,8 +6,8 @@ Twenty hedonic models organised along four design axes: | ||
| 6 | 6 | A. functional form — linear, semi-log, log-log, Box–Cox, quadratic, |
| 7 | 7 | splines (linear estimator, muni + quarter FE) |
| 8 | 8 | B. time effects — none / year / quarter / month FE (quadratic backbone) |
| 9 | − C. spatial controls — none / municipality / ~1.1 km grid / neighbourhood | |
| 10 | − unit FE (quadratic backbone, quarter FE) | |
| 9 | + C. spatial controls — none / municipality / ~5.5 km grid / ~1.1 km | |
| 10 | + grid FE (quadratic backbone, quarter FE) | |
| 11 | 11 | D. estimation method — OLS, ridge, random forest, gradient boosting |
| 12 | 12 | (with and without coordinates), spatial k-NN |
| 13 | 13 | comparables |
@@ -16,9 +16,25 @@ Every linear model is estimated as a sparse ridge with a vanishing penalty | ||
| 16 | 16 | (numerically OLS, but it (i) survives collinear dummies and (ii) predicts |
| 17 | 17 | unseen categories at the reference level via ``handle_unknown='ignore'``). |
| 18 | 18 | Every model is evaluated on identical train/test splits, and all metrics |
| 19 | −are computed on *price levels* — log models are retransformed with Duan's | |
| 20 | −smearing factor, Box–Cox by analytic inversion — so that functional forms | |
| 21 | −compete on the same scoreboard. | |
| 19 | +are computed on *price levels* so that functional forms compete on the | |
| 20 | +same scoreboard. Two retransformation rules are available for every model | |
| 21 | +whose dependent variable is a transformation of the price: | |
| 22 | + | |
| 23 | + * ``"median"`` (main scoreboard) — the fitted transformed value is | |
| 24 | + inverted as is: exp(.) for log models, the analytic inverse for | |
| 25 | + Box–Cox. Under symmetric transformed-scale errors this is the | |
| 26 | + conditional *median* price, the target that MdAPE/MAPE reward and the | |
| 27 | + quantity the tree ensembles (fit on ln P) deliver natively. Log-scale | |
| 28 | + metrics (RMSE_ln, R2_ln) are always computed on this prediction. | |
| 29 | + * ``"mean"`` (robustness) — Duan's (1983) smearing estimator of the | |
| 30 | + conditional *mean*: exp(z_hat) * mean(exp(e_i)) for log models and | |
| 31 | + mean_i g^{-1}(z_hat + e_i) over training residuals e_i for Box–Cox; | |
| 32 | + tree ensembles receive the same correction from their (in-sample) | |
| 33 | + training residuals. | |
| 34 | + | |
| 35 | +Version 1.0 of the paper smeared the linear log models but not the tree | |
| 36 | +ensembles, which penalised the linear models on every metric; the | |
| 37 | +``"median"`` rule applied uniformly is the scoreboard of version 1.1. | |
| 22 | 38 | """ |
| 23 | 39 | from __future__ import annotations |
| 24 | 40 | |
@@ -62,6 +78,16 @@ def split(df: pd.DataFrame, scheme: str): | ||
| 62 | 78 | if scheme == "temporal": |
| 63 | 79 | cut = pd.Timestamp(config.TEMPORAL_CUTOFF) |
| 64 | 80 | return df[df["sale_date"] < cut], df[df["sale_date"] >= cut] |
| 81 | + if scheme in ("group_parcel", "group_coord"): | |
| 82 | + # Leakage-free random holdouts: 20% of *groups* (parcels, or exact | |
| 83 | + # coordinate pairs = buildings/complexes) go to the test set, so | |
| 84 | + # that no property (resp. building) is seen on both sides. | |
| 85 | + key = "parcel" if scheme == "group_parcel" else "coord" | |
| 86 | + rng = np.random.default_rng(config.SEED) | |
| 87 | + groups = df[key].unique() | |
| 88 | + test_groups = set(groups[rng.random(len(groups)) < config.TEST_SHARE]) | |
| 89 | + mask = df[key].isin(test_groups).to_numpy() | |
| 90 | + return df[~mask], df[mask] | |
| 65 | 91 | raise ValueError(scheme) |
| 66 | 92 | |
| 67 | 93 | |
@@ -118,13 +144,30 @@ def _inv_boxcox(z: np.ndarray, lam: float) -> np.ndarray: | ||
| 118 | 144 | return base ** (1.0 / lam) |
| 119 | 145 | |
| 120 | 146 | |
| 121 | −def fit_boxcox(train: pd.DataFrame, numeric: list[str], cats: list[str]): | |
| 147 | +def _smear_boxcox(z_hat: np.ndarray, resid: np.ndarray, lam: float, | |
| 148 | + n_resid: int = 1000) -> np.ndarray: | |
| 149 | + """Duan's smearing estimator of E[P | X] for a Box–Cox model: | |
| 150 | + the average of g^{-1}(z_hat + e_i) over (a subsample of) the training | |
| 151 | + residuals e_i. Reduces to exp(z_hat) * mean(exp(e)) when lam = 0.""" | |
| 152 | + rng = np.random.default_rng(config.SEED) | |
| 153 | + e = resid if len(resid) <= n_resid else rng.choice(resid, n_resid, | |
| 154 | + replace=False) | |
| 155 | + out = np.zeros_like(z_hat) | |
| 156 | + for chunk in np.array_split(np.arange(len(z_hat)), | |
| 157 | + max(1, len(z_hat) // 5000)): | |
| 158 | + out[chunk] = _inv_boxcox(z_hat[chunk, None] + e[None, :], | |
| 159 | + lam).mean(axis=1) | |
| 160 | + return out | |
| 161 | + | |
| 162 | + | |
| 163 | +def fit_boxcox(train: pd.DataFrame, numeric: list[str], cats: list[str], | |
| 164 | + grid: list[float] | None = None): | |
| 122 | 165 | """Profile-likelihood choice of λ on the training sample.""" |
| 123 | 166 | y = train["price"].to_numpy(float) |
| 124 | 167 | n = len(y) |
| 125 | 168 | sum_ln_y = np.log(y).sum() |
| 126 | 169 | rows = [] |
| 127 | − for lam in config.BOXCOX_GRID: | |
| 170 | + for lam in (grid if grid is not None else config.BOXCOX_GRID): | |
| 128 | 171 | pipe = _linear_pipeline(numeric, cats) |
| 129 | 172 | z = _boxcox(y, lam) |
| 130 | 173 | pipe.fit(train, z) |
@@ -136,6 +179,43 @@ def fit_boxcox(train: pd.DataFrame, numeric: list[str], cats: list[str]): | ||
| 136 | 179 | return lam_star, prof |
| 137 | 180 | |
| 138 | 181 | |
| 182 | +HGB_PARAMS = dict(max_iter=600, learning_rate=0.08, max_leaf_nodes=63, | |
| 183 | + min_samples_leaf=20, l2_regularization=1e-2) | |
| 184 | + | |
| 185 | +# Alternative gradient-boosting configurations for the hyper-parameter | |
| 186 | +# sensitivity analysis (Section "Robustness"). ``early_stopping`` holds out | |
| 187 | +# the last 10% of the training *rows in time order* as a validation set, | |
| 188 | +# i.e. a time-honest tuning rule. | |
| 189 | +HGB_ALTERNATIVES = { | |
| 190 | + "shallow (15 leaves, lr 0.05, 1500 iter)": | |
| 191 | + dict(max_iter=1500, learning_rate=0.05, max_leaf_nodes=15, | |
| 192 | + min_samples_leaf=20, l2_regularization=1e-2), | |
| 193 | + "deep (255 leaves, lr 0.05, 600 iter)": | |
| 194 | + dict(max_iter=600, learning_rate=0.05, max_leaf_nodes=255, | |
| 195 | + min_samples_leaf=20, l2_regularization=1e-2), | |
| 196 | + "regularised (min leaf 200, L2 = 1)": | |
| 197 | + dict(max_iter=600, learning_rate=0.08, max_leaf_nodes=63, | |
| 198 | + min_samples_leaf=200, l2_regularization=1.0), | |
| 199 | + "early stopping on last 10% of training months": | |
| 200 | + dict(max_iter=2000, learning_rate=0.05, max_leaf_nodes=63, | |
| 201 | + min_samples_leaf=20, l2_regularization=1e-2, | |
| 202 | + early_stopping=True), | |
| 203 | +} | |
| 204 | + | |
| 205 | + | |
| 206 | +def robustness_registry() -> list[dict]: | |
| 207 | + """Additional specifications: nested spatial FE (unseen grid cells fall | |
| 208 | + back to their municipality instead of an arbitrary reference cell).""" | |
| 209 | + return [ | |
| 210 | + dict(name="C3' Municipality + grid-cell FE (~5.5 km)", | |
| 211 | + group="R. Nested spatial FE", kind="linear_log", | |
| 212 | + numeric=NUM_QUAD, fe=["muni", "grid5", "quarter"]), | |
| 213 | + dict(name="C4' Municipality + grid-cell FE (~1.1 km)", | |
| 214 | + group="R. Nested spatial FE", kind="linear_log", | |
| 215 | + numeric=NUM_QUAD, fe=["muni", "grid", "quarter"]), | |
| 216 | + ] | |
| 217 | + | |
| 218 | + | |
| 139 | 219 | # ---------------------------------------------------------------- registry |
| 140 | 220 | def model_registry() -> list[dict]: |
| 141 | 221 | """The twenty models. ``fe`` lists categorical FE columns appended to |
@@ -193,10 +273,27 @@ def model_registry() -> list[dict]: | ||
| 193 | 273 | |
| 194 | 274 | |
| 195 | 275 | # ---------------------------------------------------------------- fit/eval |
| 196 | −def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame): | |
| 197 | − """Fit one registry entry, return level predictions on the test set.""" | |
| 276 | +def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame, | |
| 277 | + retransform: str = "median"): | |
| 278 | + """Fit one registry entry, return level predictions on the test set. | |
| 279 | + | |
| 280 | + ``retransform`` is ``"median"`` (plain inversion of the transformed | |
| 281 | + prediction) or ``"mean"`` (Duan smearing). ``spec`` is updated in | |
| 282 | + place with ``smear`` (log models), ``lambda`` (Box–Cox) and, when | |
| 283 | + ``retransform="mean"`` is requested, both predictions are returned | |
| 284 | + through ``spec["pred_median"]`` / ``spec["pred_mean"]``. | |
| 285 | + """ | |
| 198 | 286 | kind = spec["kind"] |
| 199 | 287 | y_train = train["price"].to_numpy(float) |
| 288 | + both = retransform == "mean" | |
| 289 | + | |
| 290 | + def _log_out(z_hat_test, z_hat_train, model): | |
| 291 | + resid = np.log(y_train) - z_hat_train | |
| 292 | + smear = float(np.mean(np.exp(resid))) # Duan (1983) | |
| 293 | + spec["smear"] = smear | |
| 294 | + p_med = np.exp(z_hat_test) | |
| 295 | + spec["pred_median"], spec["pred_mean"] = p_med, p_med * smear | |
| 296 | + return (p_med * smear if both else p_med), model | |
| 200 | 297 | |
| 201 | 298 | if kind in ("linear_level", "linear_log", "boxcox", "ridge"): |
| 202 | 299 | cats = CATS + spec.get("fe", []) |
@@ -204,7 +301,9 @@ def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame): | ||
| 204 | 301 | spline_cols=spec.get("spline")) |
| 205 | 302 | if kind == "linear_level": |
| 206 | 303 | pipe.fit(train, y_train) |
| 207 | − return pipe.predict(test), pipe | |
| 304 | + p = pipe.predict(test) | |
| 305 | + spec["pred_median"] = spec["pred_mean"] = p | |
| 306 | + return p, pipe | |
| 208 | 307 | if kind in ("linear_log", "ridge"): |
| 209 | 308 | if kind == "ridge": |
| 210 | 309 | from sklearn.preprocessing import StandardScaler |
@@ -212,32 +311,72 @@ def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame): | ||
| 212 | 311 | ("sc", StandardScaler(with_mean=False)), |
| 213 | 312 | ("reg", RidgeCV(alphas=np.logspace(-6, 2, 17), |
| 214 | 313 | cv=3))] |
| 215 | − z = np.log(y_train) | |
| 216 | − pipe.fit(train, z) | |
| 217 | − resid = z - pipe.predict(train) | |
| 218 | − smear = float(np.mean(np.exp(resid))) # Duan (1983) | |
| 219 | − return np.exp(pipe.predict(test)) * smear, pipe | |
| 220 | − lam, _ = fit_boxcox(train, spec["numeric"], cats) | |
| 314 | + pipe.fit(train, np.log(y_train)) | |
| 315 | + return _log_out(pipe.predict(test), pipe.predict(train), pipe) | |
| 316 | + lam, _ = fit_boxcox(train, spec["numeric"], cats, | |
| 317 | + grid=spec.get("boxcox_grid")) | |
| 221 | 318 | pipe = _linear_pipeline(spec["numeric"], cats) |
| 222 | − pipe.fit(train, _boxcox(y_train, lam)) | |
| 319 | + z = _boxcox(y_train, lam) | |
| 320 | + pipe.fit(train, z) | |
| 223 | 321 | spec["lambda"] = lam |
| 224 | − return _inv_boxcox(pipe.predict(test), lam), pipe | |
| 322 | + z_hat = pipe.predict(test) | |
| 323 | + p_med = _inv_boxcox(z_hat, lam) | |
| 324 | + spec["pred_median"] = p_med | |
| 325 | + if both: | |
| 326 | + resid = z - pipe.predict(train) | |
| 327 | + spec["pred_mean"] = _smear_boxcox(z_hat, resid, lam) | |
| 328 | + return spec["pred_mean"], pipe | |
| 329 | + return p_med, pipe | |
| 225 | 330 | |
| 226 | 331 | if kind == "rf": |
| 227 | 332 | m = RandomForestRegressor(n_estimators=120, min_samples_leaf=5, |
| 228 | 333 | max_features=0.5, n_jobs=-1, |
| 229 | 334 | random_state=config.SEED) |
| 230 | 335 | m.fit(_ml_frame(train), np.log(y_train)) |
| 231 | − return np.exp(m.predict(_ml_frame(test))), m | |
| 336 | + return _log_out(m.predict(_ml_frame(test)), | |
| 337 | + m.predict(_ml_frame(train)) if both | |
| 338 | + else np.log(y_train), m) | |
| 232 | 339 | |
| 233 | 340 | if kind in ("hgb", "hgb_nogeo"): |
| 234 | 341 | geo = kind == "hgb" |
| 235 | − m = HistGradientBoostingRegressor( | |
| 236 | − max_iter=600, learning_rate=0.08, max_leaf_nodes=63, | |
| 237 | − min_samples_leaf=20, l2_regularization=1e-2, | |
| 238 | − random_state=config.SEED) | |
| 239 | − m.fit(_ml_frame(train, geo), np.log(y_train)) | |
| 240 | − return np.exp(m.predict(_ml_frame(test, geo))), m | |
| 342 | + params = dict(spec.get("hgb_params", HGB_PARAMS)) | |
| 343 | + tr = train | |
| 344 | + if params.pop("early_stopping", False): | |
| 345 | + # time-honest validation: the last 10% of training rows in | |
| 346 | + # sale-date order serve as the early-stopping set | |
| 347 | + tr = train.sort_values("sale_date") | |
| 348 | + n_val = int(0.1 * len(tr)) | |
| 349 | + Xv, yv = _ml_frame(tr.tail(n_val), geo), np.log( | |
| 350 | + tr.tail(n_val)["price"].to_numpy(float)) | |
| 351 | + tr = tr.head(len(tr) - n_val) | |
| 352 | + m = HistGradientBoostingRegressor(random_state=config.SEED, | |
| 353 | + **params) | |
| 354 | + # scikit-learn's early stopping draws a random validation | |
| 355 | + # split; emulate a time-ordered one by staged fitting | |
| 356 | + m.set_params(early_stopping=False) | |
| 357 | + best, best_iter = np.inf, params["max_iter"] | |
| 358 | + m.set_params(warm_start=True, max_iter=100) | |
| 359 | + for it in range(100, params["max_iter"] + 1, 100): | |
| 360 | + m.set_params(max_iter=it) | |
| 361 | + m.fit(_ml_frame(tr, geo), | |
| 362 | + np.log(tr["price"].to_numpy(float))) | |
| 363 | + mse = float(np.mean((yv - m.predict(Xv)) ** 2)) | |
| 364 | + if mse < best - 1e-5: | |
| 365 | + best, best_iter = mse, it | |
| 366 | + elif it - best_iter >= 300: | |
| 367 | + break | |
| 368 | + spec["best_iter"] = best_iter | |
| 369 | + m = HistGradientBoostingRegressor(random_state=config.SEED, | |
| 370 | + **{**params, | |
| 371 | + "max_iter": best_iter}) | |
| 372 | + tr = train | |
| 373 | + else: | |
| 374 | + m = HistGradientBoostingRegressor(random_state=config.SEED, | |
| 375 | + **params) | |
| 376 | + m.fit(_ml_frame(tr, geo), np.log(tr["price"].to_numpy(float))) | |
| 377 | + return _log_out(m.predict(_ml_frame(test, geo)), | |
| 378 | + m.predict(_ml_frame(train, geo)) if both | |
| 379 | + else np.log(y_train), m) | |
| 241 | 380 | |
| 242 | 381 | if kind == "knn": |
| 243 | 382 | # price-per-m2 of the 10 nearest sold neighbours (lat/lng, km-scaled) |
@@ -245,21 +384,17 @@ def fit_predict(spec: dict, train: pd.DataFrame, test: pd.DataFrame): | ||
| 245 | 384 | Xte = test[["lat", "lng"]].to_numpy() * np.array([111.0, 78.0]) |
| 246 | 385 | m = KNeighborsRegressor(n_neighbors=10, weights="distance", n_jobs=-1) |
| 247 | 386 | m.fit(Xtr, np.log(y_train) - train["ln_area"].to_numpy()) |
| 248 | − return np.exp(m.predict(Xte) + test["ln_area"].to_numpy()), m | |
| 387 | + z_te = m.predict(Xte) + test["ln_area"].to_numpy() | |
| 388 | + z_tr = (m.predict(Xtr) + train["ln_area"].to_numpy() if both | |
| 389 | + else np.log(y_train)) | |
| 390 | + return _log_out(z_te, z_tr, m) | |
| 249 | 391 | |
| 250 | 392 | raise ValueError(kind) |
| 251 | 393 | |
| 252 | 394 | |
| 253 | −def run_horserace(df: pd.DataFrame, scheme: str) -> pd.DataFrame: | |
| 254 | − """Evaluate the full registry under one split scheme. | |
| 255 | − | |
| 256 | − Under the forward-in-time split, test-period time categories (months, | |
| 257 | − quarters, years) were never seen in training. A one-hot encoder would | |
| 258 | − silently price them at the *baseline* period; the honest forecasting | |
| 259 | − rule for a static model is to freeze the price level at the last | |
| 260 | − period observed in training, so unseen time labels are remapped to the | |
| 261 | − latest training label before prediction. | |
| 262 | − """ | |
| 395 | +def prepare(df: pd.DataFrame, scheme: str): | |
| 396 | + """Derived columns, split, and the carry-forward rule for the | |
| 397 | + forward-in-time scheme. Returns (train, test).""" | |
| 263 | 398 | d = add_derived(df) |
| 264 | 399 | d["sale_year_c"] = d["sale_year"].astype(str) |
| 265 | 400 | train, test = split(d, scheme) |
@@ -268,16 +403,58 @@ def run_horserace(df: pd.DataFrame, scheme: str) -> pd.DataFrame: | ||
| 268 | 403 | for col in ("quarter", "month", "sale_year_c"): |
| 269 | 404 | last = train[col].max() |
| 270 | 405 | test.loc[~test[col].isin(train[col].unique()), col] = last |
| 406 | + return train, test | |
| 407 | + | |
| 408 | + | |
| 409 | +def run_horserace(df: pd.DataFrame, scheme: str, | |
| 410 | + registry: list[dict] | None = None, | |
| 411 | + return_predictions: bool = False): | |
| 412 | + """Evaluate a registry (default: the twenty models) under one split. | |
| 413 | + | |
| 414 | + Under the forward-in-time split, test-period time categories (months, | |
| 415 | + quarters, years) were never seen in training. A one-hot encoder would | |
| 416 | + silently price them at the *baseline* period; the honest forecasting | |
| 417 | + rule for a static model is to freeze the price level at the last | |
| 418 | + period observed in training, so unseen time labels are remapped to the | |
| 419 | + latest training label before prediction. Tree ensembles are piecewise | |
| 420 | + constant in the month counter, so a test month beyond the training | |
| 421 | + range is priced by the terminal leaf — the tree analogue of the same | |
| 422 | + carry-forward rule. | |
| 423 | + | |
| 424 | + Metrics are reported for the median-consistent prediction (main | |
| 425 | + scoreboard, suffix none) and for the Duan-smeared mean prediction | |
| 426 | + (suffix ``_mean``). With ``return_predictions`` the test-set | |
| 427 | + predictions of every model are returned as a second DataFrame. | |
| 428 | + """ | |
| 429 | + train, test = prepare(df, scheme) | |
| 271 | 430 | y_test = test["price"].to_numpy(float) |
| 272 | − rows = [] | |
| 273 | − for spec in model_registry(): | |
| 431 | + rows, preds = [], {} | |
| 432 | + for spec in (registry if registry is not None else model_registry()): | |
| 433 | + spec = dict(spec) | |
| 274 | 434 | t0 = time.time() |
| 275 | − pred, _ = fit_predict(dict(spec), train, test) | |
| 276 | − met = _metrics(y_test, pred) | |
| 435 | + fit_predict(spec, train, test, retransform="mean") | |
| 436 | + p_med, p_mean = spec["pred_median"], spec["pred_mean"] | |
| 437 | + met = _metrics(y_test, p_med) | |
| 438 | + met_mean = {f"{k}_mean": v for k, v in | |
| 439 | + _metrics(y_test, p_mean).items()} | |
| 277 | 440 | rows.append({"name": spec["name"], "group": spec["group"], |
| 278 | − **met, "seconds": round(time.time() - t0, 1), | |
| 441 | + **met, **met_mean, | |
| 442 | + "smear": spec.get("smear", np.nan), | |
| 443 | + "lambda": spec.get("lambda", np.nan), | |
| 444 | + "best_iter": spec.get("best_iter", np.nan), | |
| 445 | + "seconds": round(time.time() - t0, 1), | |
| 279 | 446 | "n_train": len(train), "n_test": len(test)}) |
| 280 | − print(f" [{scheme}] {spec['name']:<38} " | |
| 447 | + preds[spec["name"]] = np.clip(p_med, config.PRED_FLOOR, None) | |
| 448 | + print(f" [{scheme}] {spec['name']:<44} " | |
| 281 | 449 | f"MdAPE={met['mdape']:5.1f}% R2_ln={met['r2_ln']:.3f} " |
| 282 | 450 | f"({rows[-1]['seconds']}s)") |
| 283 | − return pd.DataFrame(rows) | |
| 451 | + res = pd.DataFrame(rows) | |
| 452 | + if not return_predictions: | |
| 453 | + return res | |
| 454 | + P = test[["id", "sale_date", "price", "muni", "prop_class", "grid", | |
| 455 | + "parcel", "coord"]].reset_index(drop=True) | |
| 456 | + P["in_train_parcel"] = P["parcel"].isin(set(train["parcel"])) | |
| 457 | + P["in_train_coord"] = P["coord"].isin(set(train["coord"])) | |
| 458 | + for k, v in preds.items(): | |
| 459 | + P[k] = v | |
| 460 | + return res, P | |
modified
src/wp11/sample.py
+8 −2
@@ -73,6 +73,12 @@ def build(raw: pd.DataFrame | None = None) -> pd.DataFrame: | ||
| 73 | 73 | df["prop_class"] = _property_class(df) |
| 74 | 74 | df["link"] = df["role_lien_physique"].map(LINK_MAP).fillna("unknown") |
| 75 | 75 | df["muni"] = df["role_code_mun"] |
| 76 | + # Grouping keys for leakage-free holdouts: the assessment-roll parcel | |
| 77 | + # identifier (repeat sales of the same property) and the exact | |
| 78 | + # geocoded coordinate pair (units sharing a building or complex). | |
| 79 | + df["parcel"] = df["role_id_provinc"].astype(str) | |
| 80 | + df["coord"] = (df["lat"].round(6).astype(str) + "_" | |
| 81 | + + df["lng"].round(6).astype(str)) | |
| 76 | 82 | df["grid"] = ((df["lat"] / config.GRID_DEG).round().astype(int).astype(str) |
| 77 | 83 | + "_" + (df["lng"] / config.GRID_DEG).round().astype(int).astype(str)) |
| 78 | 84 | df["grid5"] = ((df["lat"] / config.GRID5_DEG).round().astype(int).astype(str) |
@@ -91,8 +97,8 @@ def build(raw: pd.DataFrame | None = None) -> pd.DataFrame: | ||
| 91 | 97 | keep = ["id", "sale_date", "sale_year", "month", "quarter", "t", |
| 92 | 98 | "price", "ln_price", "area", "ln_area", "lot", "ln_lot", "has_lot", |
| 93 | 99 | "age", "floors", "units", "prop_class", "link", |
| 94 | − "muni", "grid", "grid5", "lat", "lng", "city", | |
| 95 | − "role_municipalite"] | |
| 100 | + "muni", "grid", "grid5", "parcel", "coord", "lat", "lng", | |
| 101 | + "city", "role_municipalite"] | |
| 96 | 102 | return df[keep].reset_index(drop=True) |
| 97 | 103 | |
| 98 | 104 | |
| 99 | 105 | |