SPB Git

spb/wp2_uqo Public

UQO Working Paper No. 2 — Decoding Real Estate Descriptions: text-based hedonic analysis of housing listings.

researcheconomicshousinghedonicnlpuqo
TeX 73.8% Python 26%

Initial commit: semantic hedonic pricing pipeline, figures, and paper (UQO WP no. 2)

Restructured from immo-wp2-spb-20260519: modular src/ package, numbered
reproduction pipeline (01-06), machine-generated LaTeX tables, regenerated
figures (incl. previously missing fig2/fig3), corrected paper statistics
(see CHANGES.md), 47-page PDF compiling clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simon-pierre boucher committed 5 days ago (Aug 5, 2026)

Showing 72 changed files with +4,180 and −0

added .gitignore +21 −0
@@ -0,0 +1,21 @@
1 +# Data — too large / sensitive for git (see data/README.md)
2 +data/*
3 +!data/README.md
4 +
5 +# LaTeX build artifacts
6 +paper/*.aux
7 +paper/*.log
8 +paper/*.out
9 +paper/*.toc
10 +paper/*.bbl
11 +paper/*.blg
12 +paper/*.fls
13 +paper/*.fdb_latexmk
14 +paper/*.synctex.gz
15 +
16 +# macOS
17 +.DS_Store
18 +
19 +# Python
20 +__pycache__/
21 +*.pyc
added AUDIT.md +126 −0
@@ -0,0 +1,126 @@
1 +# AUDIT — immo-wp2-spb-20260519
2 +
3 +Audit of the original project at `~/Desktop/UQO/UQO_WP/immo-wp2-spb-20260519`
4 +(354 MB), performed 2026-08-05 before restructuring into `~/Desktop/wp2_uqo`.
5 +The original folder is left untouched and serves as the backup.
6 +
7 +## 1. What the project is
8 +
9 +Working Paper No. 2 (UQO): *"Decoding Real Estate Descriptions: Semantic
10 +Embeddings and Hedonic Pricing of Residential Properties in Quebec"*
11 +(Simon-Pierre Boucher). The analysis embeds the free-text `PublicRemarks` of
12 +17,087 Quebec single-family house listings with `all-MiniLM-L6-v2`
13 +(sentence-transformers), computes cosine similarities to 20 researcher-defined
14 +French reference descriptions ("luxury", "needs renovation", …), and adds
15 +those 20 similarity features to a log-price hedonic OLS model (models A–E,
16 +HC3 errors), plus robustness checks (VIF, Breusch–Pagan, quantile regression,
17 +winsorization, 1,000-rep bootstrap, Lasso/Elastic Net).
18 +
19 +Headline numbers (paper, model D vs A): adjusted R² 0.452 → 0.511,
20 +F = 99.53 (p < 0.001), n = 17,087.
21 +
22 +## 2. Data
23 +
24 +| File | Size | Role |
25 +|---|---|---|
26 +| `louka.db` | 268 MB | **Raw data.** SQLite; table `properties` (id, mls_number, category, price_value, bedrooms, bathrooms, data JSON blob). Counts: house 17,089, condo 8,586, land 8,506, rent 8,240, plex 4,058. Sample = houses with price > 0 and remarks ≥ 20 chars → 17,087. |
27 +| `hedonic_maison_results.csv` | 17,087 rows | Processed: houses + 20 similarity features (main analysis dataset). |
28 +| `hedonic_results.csv` | 29,731 rows | Processed (earlier PCA approach, house/condo/plex). |
29 +| `hedonic_cosine_results.csv` | 29,731 rows | Processed (earlier 16-reference approach, all categories). |
30 +| `embeddings_maisons.npy` | 25 MB | 17,087 × 384 embeddings (houses; used by the paper). |
31 +| `embeddings_remarks.npy` | 44 MB | 29,731 × 384 embeddings (all categories; earlier approach). |
32 +| `sim_matrix_maisons.npy` | 1.3 MB | 17,087 × 20 similarity matrix (paper). |
33 +| `similarity_matrix.npy` | 1.8 MB | 29,731 × 16 (earlier approach). |
34 +| `reference_embeddings.npz` | 43 KB | 16 reference embeddings (earlier approach). |
35 +
36 +⚠️ Notes:
37 +- `louka.db` also contains a `users` table with **plaintext usernames/passwords**
38 + (application leftover). It is not used by any script. Not copied onward.
39 +- All scripts hardcode `DB_PATH = "/Users/simon-pierreboucher/Desktop/d/louka.db"`
40 + and write outputs to `~/Desktop/d/` — a folder that **no longer exists**. The
41 + project copy of `louka.db` is what remains; outputs were copied into the
42 + project at some point.
43 +- `robustness_results.csv` (written by `hedonic_maison.py` step 12) was saved
44 + to `~/Desktop/d/` and is **not present anywhere** — regenerated in the new repo.
45 +
46 +## 3. Scripts (chronological / by role)
47 +
48 +| Script | Role | Outputs | Status |
49 +|---|---|---|---|
50 +| `hedonic_semantic.py` (348 l.) | Exploration #1: PCA(20) of embeddings, house/condo/plex, models 1–4 | `hedonic_results.csv`, `embeddings_remarks.npy` | Superseded; not used by the paper |
51 +| `hedonic_cosine_references.py` (447 l.) | Exploration #2: 16 reference similarities, all categories, models A–C | `hedonic_cosine_results.csv`, `reference_embeddings.npz`, `similarity_matrix.npy` | Superseded; not used by the paper |
52 +| `hedonic_maison.py` (818 l.) | **Main analysis**: 20 references, houses only, models A–E + all robustness checks | `hedonic_maison_results.csv`, `embeddings_maisons.npy`, `sim_matrix_maisons.npy`, `robustness_results.csv` (lost) | **Paper source** (tables 1–8) |
53 +| `paper_figures.py` (578 l.) | Figures v1 (figs 1–8) + `table1–4_*.csv` | `figures/` | Superseded by v2 for figures; still the only producer of the 4 table CSVs |
54 +| `paper_figures_v2.py` (386 l.) | **Figures v2 (final)**: figs 1–11, tighter layout | `figures/fig1…fig11` | **Paper source** (all figures) |
55 +
56 +Duplication: the DB-extraction block (~70 lines) and the OLS-fit helper are
57 +copy-pasted in all 5 scripts, with small drifts (`half_baths` and lat/lon only
58 +in some; 16 vs 20 references; French vs English reference names). The 20
59 +reference descriptions are pasted 3× (French keys in `hedonic_maison.py`,
60 +English display keys in both figure scripts — same French texts).
61 +
62 +## 4. Figures
63 +
64 +`figures/` and `wp2/figures/` (identical copies) contain fig1, fig4–fig11
65 +(PDF, some PNG) + `table1–4_*.csv`.
66 +
67 +**Missing: `fig2_coefficient_plot.pdf` and `fig3_similarity_distributions.pdf`**,
68 +although both papers reference them (`sections/results.tex`, `sections/data.tex`).
69 +`wp2/main_web.tex` silently masks this with a custom `\includegraphics`
70 +fallback that prints a "Figure indisponible" box — the compiled
71 +`wp2/main_web.pdf` therefore ships with two placeholder boxes instead of
72 +Figures 2 (semantic coefficient forest plot) and 3 (similarity distributions).
73 +Both are regenerated by `paper_figures_v2.py`. **This is the main defect fixed
74 +by the restructuring.**
75 +
76 +Figure → script map (all from `paper_figures_v2.py`):
77 +fig1 model comparison; fig2 semantic coefficient forest plot (was missing);
78 +fig3 similarity boxplots (was missing); fig4 price-quintile heatmap;
79 +fig5 similarity correlation matrix; fig6 scatter plots (4 dims);
80 +fig7 methodology diagram; fig8 R² decomposition; fig9 price distributions;
81 +fig10 structural coefficients; fig11 residual diagnostics.
82 +
83 +## 5. LaTeX
84 +
85 +Two generations coexist:
86 +
87 +- **Root (obsolete, May 20)**: monolithic `paper.tex` (112 KB, first draft),
88 + then `main.tex` + `preamble.tex` + `titlepage.tex` + `sections/` + `tables/`
89 + + `appendix/` + `references.bib`.
90 +- **`wp2/` (canonical, May 20 – June 13)**: `main.tex` (self-contained
91 + preamble, UQO WP style), `main_web.tex` (same + figure-fallback hack, source
92 + of the compiled `main_web.pdf`), `sections/` (9 files incl. titlepage),
93 + `tables/` (8 hand-written booktabs tables), `appendix/appendix.tex`,
94 + `references.bib` (551 lines, ~40 entries), `Makefile`, `uq_logo.jpg`.
95 +
96 +`diff -r` confirms root `sections/`, `tables/`, `appendix/`, `references.bib`
97 +are **byte-identical** to the `wp2/` ones (root just lacks `titlepage.tex`).
98 +So the root LaTeX tree is a pure duplicate → dropped.
99 +
100 +Numbers in `tables/*.tex` were hand-transcribed from `hedonic_maison.py`
101 +console output (no automated table generation).
102 +
103 +## 6. Dead / duplicate / unused files
104 +
105 +- `paper.tex`, root `main.tex`, `preamble.tex`, `titlepage.tex`, root
106 + `sections/`, `tables/`, `appendix/`, root `figures/` — superseded duplicates.
107 +- Build artifacts: `*.aux, *.log, *.out, *.toc, *.bbl, *.blg, *.fls,
108 + *.fdb_latexmk, *.synctex.gz` at root and in `wp2/`.
109 +- `.DS_Store` files.
110 +- Earlier-approach outputs (`hedonic_results.csv`, `hedonic_cosine_results.csv`,
111 + `embeddings_remarks.npy`, `similarity_matrix.npy`, `reference_embeddings.npz`)
112 + are kept in `data/processed/legacy/` for traceability but are not part of the
113 + pipeline.
114 +
115 +## 7. Discrepancies & reproduction status
116 +
117 +- **fig2/fig3 missing** in the original (see §4) — regenerated here.
118 +- **`robustness_results.csv` lost** in the original — regenerated here.
119 +- Reproduction check (new pipeline vs original artifacts) — results recorded
120 + in `CHANGES.md` §Verification:
121 + - similarity features vs `hedonic_maison_results.csv` (17,087 × 20),
122 + - model A–E statistics vs `tables/tab_model_comparison.tex`
123 + (R² 0.4525 / 0.4647 / 0.4962 / 0.5122 / 0.5118),
124 + - regenerated figures vs shipped fig1, fig4–fig11.
125 +- Bootstrap (`np.random.seed(42)`) and Lasso/ENet CV (`random_state=42`) are
126 + seeded, so robustness numbers are reproducible on the same package versions.
added CHANGES.md +121 −0
@@ -0,0 +1,121 @@
1 +<!-- Author: Simon-Pierre Boucher — contact@spboucher.ai -->
2 +
3 +# CHANGES — restructuring of immo-wp2-spb-20260519 → wp2_uqo
4 +
5 +Original project: `~/Desktop/UQO/UQO_WP/immo-wp2-spb-20260519` (untouched, serves as backup).
6 +New repository: `~/Desktop/wp2_uqo`. Date: 2026-08-05.
7 +
8 +## 1. What was moved / renamed
9 +
10 +| Original | New location |
11 +|---|---|
12 +| `louka.db` | `data/raw/louka.db` |
13 +| `embeddings_maisons.npy`, `sim_matrix_maisons.npy` | `data/processed/` (pipeline cache) |
14 +| `hedonic_maison_results.csv` (+ earlier-approach CSVs/NPYs) | `data/processed/legacy/` (originals kept for comparison) |
15 +| `hedonic_maison.py` (818 lines, monolithic) | split into `src/` modules + `scripts/01–04` |
16 +| `paper_figures_v2.py` | `scripts/05_figures.py` (ports all 11 figures) |
17 +| `wp2/main.tex` + `wp2/sections/` + `wp2/tables/` | `paper/` (preamble extracted to `paper/preamble.tex`) |
18 +| `hedonic_semantic.py`, `hedonic_cosine_references.py`, `paper_figures.py`, root LaTeX tree, `paper.tex` | **not migrated** (superseded; documented in `AUDIT.md`) |
19 +
20 +## 2. Code refactoring
21 +
22 +- All hardcoded paths to the defunct `~/Desktop/d/` replaced by repository-relative
23 + paths in `src/config.py` (overridable via `WP2_DB_PATH`).
24 +- The DB-extraction block that was copy-pasted across 5 scripts now lives once in
25 + `src/data.py`; the 20 reference descriptions (pasted 3×, in two languages) live
26 + once in `src/references.py` (slugs for data columns, English labels for display).
27 +- Models A–E defined once in `src/models.py`; robustness checks ported faithfully
28 + (same estimators, same seeds: bootstrap `np.random.seed(42)`, CV `random_state=42`).
29 +- New numbered pipeline: `01_prepare_data` → `02_similarities` → `03_models` →
30 + `04_robustness` → `05_figures` → `06_tables`. Everything runs end-to-end from
31 + `data/raw/louka.db`; embeddings are cached and `--force` re-encodes.
32 +- **New `scripts/06_tables.py` generates the paper's numeric tables directly from
33 + the pipeline results** — this closes the transcription gap that produced the
34 + wrong numbers documented in §4 below.
35 +- `requirements.txt` pins the exact package versions used for verification.
36 +
37 +## 3. Verification (regenerated vs. original artifacts)
38 +
39 +- Sample: 17,087 houses — **identical** (all 30 columns of
40 + `hedonic_maison_results.csv` equal to the original; similarity matrix
41 + bit-identical).
42 +- Models A–E: R², adj. R², AIC, BIC, k — **identical to the paper's Table 4**
43 + (e.g. R² 0.4525 / 0.4647 / 0.4962 / 0.5122 / 0.5118; joint F = 99.53).
44 +- Model D coefficients: **identical to the paper's Table 5** (all 27 coefficients,
45 + SEs, t-stats, impacts).
46 +- Figures: regenerated fig1, fig4–fig11 match the shipped ones (same content,
47 + PDF sizes within ±2 bytes). **fig2 and fig3, referenced by the paper but missing
48 + from the original repo (the compiled PDF showed "Figure indisponible" boxes),
49 + are regenerated and now included.**
50 +- `robustness_results.csv`, lost in the original, is regenerated (seeded, reproducible).
51 +
52 +## 4. Numbers corrected in the paper ⚠️ (review recommended)
53 +
54 +The original LaTeX contained statistics that **do not match the actual pipeline
55 +outputs** (the project's own `figures/table1/table2` CSVs confirm the pipeline
56 +values). They appear to have been transcribed from memory/drafted text rather
57 +than from the analysis output. All were replaced by the reproducible values;
58 +the paper's tables are now machine-generated (`scripts/06_tables.py`).
59 +
60 +| Claim | Old (paper) | New (verified) |
61 +|---|---|---|
62 +| Price mean | $764,508 | **$807,259** (median $589,900 unchanged) |
63 +| Description length | mean 683, sd 367, max 5,399 | **mean 510, sd 151, max 703** (field truncated ~700 chars — now noted in the text) |
64 +| Parking mean | 2.3 | **5.5** (counts all spots incl. driveway — now explained) |
65 +| log-price sd | 0.68 | **0.70** |
66 +| Similarity stats & correlations (Table 3) | e.g. Luxury corr −0.178 | **regenerated** (e.g. Luxury −0.234; all in [−0.29, −0.17]) |
67 +| Breusch–Pagan | LM = 2,847.3 | **LM = 768.0** (same conclusion) |
68 +| VIF | "none > 10, mean ≈ 4.2" | **mean 17.2, 16/20 > 10** — subsection rewritten honestly (block-level inference unaffected; O'Brien 2007 argument) |
69 +| Inter-reference correlations | "0.4–0.8", "Motivated Seller lowest" | **0.66–0.97, mean 0.84**; top pair Entry-Level/Motivated Seller r = 0.97 |
70 +| Winsorized model | 342 obs dropped, adj R² 0.498, shifts Luxury −0.012 / Motivated +0.008 | **341 obs, adj R² 0.486, largest shifts Entry-Level −0.024 / Motivated Seller +0.023** (no sign reversals, all 16 stay significant — unchanged conclusions) |
71 +| Bootstrap | "SE within 5% of HC3" | **within 9% (15/20 within 5%)** |
72 +| Lasso / Elastic Net | "Lasso keeps 15/20, ENet 17/20, high concordance with OLS" | **both keep 4/20** — subsection rewritten: sparse block-representative selection under strong collinearity, complementary rather than concordant |
73 +| Quantile table | e.g. Luxury 0.098/0.125/0.176; "Modern & Land increasing" | **regenerated** (Luxury 0.093/0.130/0.160 increasing ✓; Modern high at both tails; Land & Nature stable) |
74 +| PCA comparison | "PCA yields R² 0.496 < 0.512 → reference approach outperforms" | **PCA(20) actually fits better: ΔR² +0.070 vs +0.044** (49.5 % variance ✓). Reframed as an explicit interpretability-vs-fit trade-off (Table 8 + Discussion §PCA rewritten) |
75 +
76 +Qualitative conclusions that survive unchanged: heteroskedasticity → HC3;
77 +no sign reversals under trimming; bootstrap ≈ HC3; Luxury premium increasing in
78 +quantiles; discounts attenuating; joint semantic block highly significant.
79 +Claims that had to change substance: **VIF/multicollinearity** (now acknowledged
80 +as substantial), **Lasso concordance** (now honest about sparse selection), and
81 +**PCA fit comparison** (now favors PCA on fit, reference approach on
82 +interpretability). These are flagged because they alter the robustness
83 +narrative, not the headline results.
84 +
85 +## 5. Paper rewrite (Phase 3)
86 +
87 +- `paper/main.tex` slimmed (metadata + inputs); packages and macros extracted to
88 + `paper/preamble.tex`; figures pulled from `../figures/` via `\graphicspath`;
89 + the `main_web.tex` variant with the silent figure-fallback hack was dropped.
90 +- All **[TODO] blocks removed** (robustness §7.7–7.9, methodology, summary table)
91 + and converted into completed analyses (PCA comparison), honest limitations
92 + (Discussion), or future-research items (Conclusion).
93 +- New appendix with real content: **A. verbatim French reference descriptions**
94 + (replication) and **B. Model E coefficient table** — both machine-generated.
95 +- Editorial pass on Data/Results/Robustness/Discussion: corrected figure-3 caption
96 + (boxplots, not KDE), documented the description-truncation and parking-field
97 + quirks, tightened the collinearity narrative, consistent notation preserved.
98 +- Fig 10 y-labels now use readable names instead of raw variable names.
99 +- Title page: "First draft: May 2025" corrected to **May 2026** (the data were
100 + scraped January 2026); version bumped to 1.1.
101 +- Abstract: "robust to bootstrap inference, winsorization, and Lasso selection"
102 + → "robust to bootstrap inference and outlier trimming" (accurate).
103 +- Compiles clean: `latexmk -pdf main.tex` → **47 pages, 0 errors, 0 undefined
104 + references/citations** (24 minor overfull hboxes). Bibliography complete
105 + (59/59 cited keys resolve).
106 +
107 +## 6. Items for your review
108 +
109 +1. **§4 corrections** — especially VIF, Lasso, and PCA, which change the
110 + robustness narrative (the old text claimed low collinearity and high
111 + selection concordance; the data say otherwise). Verify you are comfortable
112 + with the new framing before circulating.
113 +2. `data/raw/louka.db` contains a `users` table with plaintext credentials
114 + (application leftover, unused by the analysis). Consider stripping it:
115 + `sqlite3 data/raw/louka.db "DROP TABLE users; VACUUM;"`.
116 +3. The description-length truncation (~700 chars) is now disclosed in the paper;
117 + if a non-truncated export exists, re-running the pipeline on it would be easy.
118 +4. The quantile-regression standard errors use the statsmodels kernel-based
119 + estimator (the old table note claimed bootstrap SEs, which was never the case).
120 +5. Original folder `immo-wp2-spb-20260519` was left untouched; delete or archive
121 + it once you're satisfied with `wp2_uqo`.
added README.md +337 −0
@@ -0,0 +1,337 @@
1 +<!-- Author: Simon-Pierre Boucher — contact@spboucher.ai -->
2 +
3 +<div align="center">
4 +
5 +# 🏠 Decoding Real Estate Descriptions
6 +
7 +### Semantic Embeddings and Hedonic Pricing of Residential Properties in Quebec
8 +
9 +**UQO Working Paper No. 2**
10 +
11 +[![Author](https://img.shields.io/badge/Author-Simon--Pierre%20Boucher-0033A0?style=flat-square)](mailto:contact@spboucher.ai)
12 +[![Contact](https://img.shields.io/badge/Contact-contact%40spboucher.ai-D14836?style=flat-square&logo=gmail&logoColor=white)](mailto:contact@spboucher.ai)
13 +[![Institution](https://img.shields.io/badge/UQO-D%C3%A9partement%20des%20sciences%20administratives-00539F?style=flat-square)](https://uqo.ca)
14 +
15 +[![Python](https://img.shields.io/badge/Python-3.14-3776AB?style=flat-square&logo=python&logoColor=white)](https://www.python.org/)
16 +[![LaTeX](https://img.shields.io/badge/LaTeX-latexmk%20%7C%20pdfLaTeX-008080?style=flat-square&logo=latex&logoColor=white)](https://www.latex-project.org/)
17 +[![sentence-transformers](https://img.shields.io/badge/sentence--transformers-all--MiniLM--L6--v2-FF6F00?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
18 +[![statsmodels](https://img.shields.io/badge/statsmodels-OLS%20%7C%20QuantReg%20%7C%20HC3-4051B5?style=flat-square)](https://www.statsmodels.org/)
19 +[![scikit-learn](https://img.shields.io/badge/scikit--learn-PCA%20%7C%20Lasso%20%7C%20ElasticNet-F7931E?style=flat-square&logo=scikitlearn&logoColor=white)](https://scikit-learn.org/)
20 +
21 +[![Reproducible](https://img.shields.io/badge/Reproducible-end--to--end%20pipeline-2E7D32?style=flat-square&logo=githubactions&logoColor=white)](#-reproducing-everything)
22 +[![Sample](https://img.shields.io/badge/n-17%2C087%20houses-1565C0?style=flat-square)](#-data)
23 +[![Model](https://img.shields.io/badge/Adj.%20R%C2%B2-0.452%20%E2%86%92%200.511-6A1B9A?style=flat-square)](#-key-results)
24 +[![Paper](https://img.shields.io/badge/Paper-47%20pages%20PDF-B71C1C?style=flat-square&logo=adobeacrobatreader&logoColor=white)](paper/main.pdf)
25 +[![JEL](https://img.shields.io/badge/JEL-R31%20%C2%B7%20C45%20%C2%B7%20C21%20%C2%B7%20R21-455A64?style=flat-square)](#-citation)
26 +
27 +</div>
28 +
29 +---
30 +
31 +> **TL;DR** — Listing descriptions written by real estate agents contain price-relevant information that classic hedonic variables miss. This project embeds the free-text `PublicRemarks` of **17,087 Quebec single-family house listings** with a sentence transformer, projects each embedding onto **20 researcher-defined semantic reference descriptions** (*Luxury*, *Needs Renovation*, *Waterfront*, *Motivated Seller*, …) via cosine similarity, and adds those 20 interpretable scores to a log-price hedonic OLS model. Adjusted R² rises from **0.452 to 0.511** (joint *F* = 99.53, *p* < 0.001), and each semantic dimension carries a named, signed, economically meaningful implicit price.
32 +
33 +---
34 +
35 +## 📖 Table of Contents
36 +
37 +- [Overview](#-overview)
38 +- [Key Results](#-key-results)
39 +- [Methodology](#-methodology)
40 +- [Repository Structure](#-repository-structure)
41 +- [The Pipeline, Script by Script](#-the-pipeline-script-by-script)
42 +- [Data](#-data)
43 +- [Reproducing Everything](#-reproducing-everything)
44 +- [Building the Paper](#-building-the-paper)
45 +- [Verification & Provenance](#-verification--provenance)
46 +- [Figures Gallery](#-figures-gallery)
47 +- [Requirements](#-requirements)
48 +- [Citation](#-citation)
49 +- [Author & Contact](#-author--contact)
50 +
51 +---
52 +
53 +## 🔍 Overview
54 +
55 +Hedonic pricing models decompose a property's price into the implicit prices of its characteristics — bedrooms, bathrooms, lot size. But the *narrative* of a listing ("cuisine gastronomique, comptoirs de quartz…" vs. "besoin de rénovation, vendu tel quel…") carries quality information that no structured field captures.
56 +
57 +The catch, historically, is a **depth-vs-interpretability trade-off**:
58 +
59 +| Approach | Semantic depth | Economic interpretability |
60 +|---|---|---|
61 +| Keyword counts / bag-of-words | ❌ shallow | ✅ high |
62 +| Sentiment scores | ❌ one dimension | ✅ high |
63 +| LDA topics | 🟡 moderate | ❌ unstable |
64 +| Raw BERT/transformer embeddings | ✅ deep | ❌ opaque (384–768 anonymous dims) |
65 +| **Reference-based cosine projection (this paper)** | ✅ **deep** | ✅ **high** |
66 +
67 +**The idea:** instead of feeding 384 anonymous embedding dimensions into a regression, define 20 short *reference descriptions* — synthetic French paragraphs, each embodying one qualitative housing dimension — embed them with the same model, and use the **cosine similarity between each listing and each reference** as 20 named regressors. Every coefficient then reads directly as "the implicit price of sounding more *luxury* / more *fixer-upper* / more *waterfront*", per standard deviation.
68 +
69 +This is conceptually analogous to factor-mimicking portfolios in asset pricing, and to concept-bottleneck models in interpretable ML: the deep representation is channeled through human-named concepts before inference.
70 +
71 +---
72 +
73 +## 📊 Key Results
74 +
75 +### Model comparison (n = 17,087)
76 +
77 +| Model | Specification | R² | Adj. R² | AIC | BIC | k |
78 +|:---:|---|---:|---:|---:|---:|---:|
79 +| A | Structural only | 0.4525 | 0.4523 | 26,165 | 26,219 | 6 |
80 +| B | A + description length | 0.4647 | 0.4645 | 25,779 | 25,841 | 7 |
81 +| C | A + 20 semantic similarities | 0.4962 | 0.4954 | 24,783 | 24,992 | 26 |
82 +| **D** | **Full (B + C)** | **0.5122** | **0.5115** | **24,232** | 24,448 | 27 |
83 +| E | Parsimonious (16 significant dims) | 0.5118 | 0.5111 | 24,239 | **24,425** | 23 |
84 +
85 +Joint significance of the 21 text variables (D vs. A): ***F* = 99.53, *p* < 0.001**. Variance decomposition of Model D: structural 88.3 %, text length 2.4 %, semantics 9.3 %.
86 +
87 +### Implicit semantic price gradients (Model D, per +1 SD, HC3 errors)
88 +
89 +| 📈 Positive | Impact | 📉 Negative | Impact |
90 +|---|---:|---|---:|
91 +| Modern/Contemporary | **+16.4 %*** | Family-Friendly | **−11.6 %*** |
92 +| Luxury | **+14.2 %*** | New Construction | **−10.5 %*** |
93 +| Land & Nature | **+13.0 %*** | Quiet & Peaceful | −9.5 %*** |
94 +| Entry-Level | +10.4 %*** | Bright & Spacious | −8.9 %*** |
95 +| Waterfront | +4.7 %*** | Motivated Seller | **−8.5 %*** |
96 +| Panoramic View | +3.0 %** | Needs Renovation | **−7.9 %*** |
97 +
98 +<sub>*** p < 0.001, ** p < 0.01. Negative coefficients on Family-Friendly / New Construction largely proxy suburban location (see paper §6). Impact = (e^β − 1) × 100 %.</sub>
99 +
100 +### Heterogeneity & robustness (highlights)
101 +
102 +- **Quantile regressions** (τ = 0.25 / 0.50 / 0.75): the Luxury premium rises monotonically from **9.7 %** at the 25th percentile to **17.3 %** at the 75th; urgency and condition discounts attenuate at the top of the market.
103 +- **Bootstrap** (1,000 replications, seed 42): SEs within 9 % of HC3 (15/20 within 5 %).
104 +- **Outlier trimming** (1st/99th price percentiles): no sign reversal, all 16 significant dimensions stay significant.
105 +- **Multicollinearity is real and disclosed**: similarity dimensions correlate at 0.66–0.97 (mean VIF 17.2). Block-level inference is unaffected; individual coefficients are partial associations within a correlated block.
106 +- **PCA benchmark**: 20 principal components of the raw embeddings fit *better* (ΔR² +0.070 vs. +0.044) but are economically unreadable — the paper quantifies the interpretability-vs-fit trade-off explicitly.
107 +
108 +---
109 +
110 +## 🧪 Methodology
111 +
112 +```
113 +┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────────┐
114 +│ Property │ │ Sentence │ │ Cosine │ │ Hedonic │ │ Implicit │
115 +│ Listings ├──▶│ Embeddings ├──▶│ Similarity ├──▶│ OLS Model ├──▶│ Price │
116 +│ (n=17,087) │ │ (384-d) │ │ (20 refs) │ │ (HC3 s.e.) │ │ Estimates │
117 +└─────────────┘ └──────────────┘ └──────────────┘ └─────────────┘ └──────────────┘
118 + PublicRemarks all-MiniLM-L6-v2 S = E · Rᵀ log(P) = α + β'X + δℓ + γ's + ε
119 +```
120 +
121 +1. **Extraction** — houses only, price > 0, description ≥ 20 characters; minimal preprocessing (the transformer handles raw text).
122 +2. **Embedding** — `all-MiniLM-L6-v2` (22.7M params, 384-d, L2-normalized), batch 256; the whole corpus encodes in seconds on Apple Silicon.
123 +3. **Projection** — 20 French reference descriptions (146–257 chars each, full verbatim text in paper Appendix A), designed for *semantic saturation*, *dimensional specificity*, *linguistic consistency*. One matrix product yields the 17,087 × 20 similarity matrix.
124 +4. **Estimation** — nested OLS models A–E on log price, all covariates standardized, HC3 robust errors; joint F-tests; quantile regressions; bootstrap; winsorization; VIF; Lasso/Elastic-Net selection; PCA benchmark.
125 +
126 +The 20 dimensions span six domains: **Quality & Standing** (Luxury, Entry-Level) · **Condition** (Renovated, Needs Renovation) · **Physical** (Bright & Spacious, Garage & Parking, Finished Basement, Pool & Landscaping) · **Location & Setting** (Land & Nature, Panoramic View, Premium Location, Quiet & Peaceful, Waterfront) · **Style & Character** (Modern/Contemporary, Heritage/Character, New Construction, Energy Efficient, Family-Friendly) · **Market Signals** (Income/Investment, Motivated Seller).
127 +
128 +---
129 +
130 +## 📁 Repository Structure
131 +
132 +```
133 +wp2_uqo/
134 +├── README.md ← you are here
135 +├── AUDIT.md ← forensic audit of the original (pre-restructuring) project
136 +├── CHANGES.md ← everything that was moved / refactored / corrected, with diffs
137 +├── requirements.txt ← pinned dependencies (exact versions used for verification)
138 +│
139 +├── data/ ← NOT tracked by git (see data/README.md)
140 +│ ├── raw/louka.db │ SQLite, 268 MB, 46,479 listings (5 categories)
141 +│ └── processed/ │ parquet sample, embeddings cache, similarity matrix,
142 +│ └── legacy/ │ analysis CSV + pre-refactor artifacts kept for comparison
143 +│
144 +├── src/ ← analysis package (imported by scripts/)
145 +│ ├── config.py │ repo-relative paths, model name, seed, structural vars
146 +│ ├── references.py │ the 20 French reference descriptions + English labels
147 +│ ├── data.py │ SQLite → DataFrame extraction (n = 17,087)
148 +│ ├── embeddings.py │ encoding, caching, cosine-similarity features
149 +│ └── models.py │ standardized-design OLS A–E, HC3
150 +│
151 +├── scripts/ ← numbered pipeline entry points (run in order)
152 +│ ├── 01_prepare_data.py │ louka.db → data/processed/houses.parquet
153 +│ ├── 02_similarities.py │ embeddings (cached) + 20 similarity features → CSV
154 +│ ├── 03_models.py │ models A–E → results/*.csv
155 +│ ├── 04_robustness.py │ VIF · BP · quantile · winsor · bootstrap · Lasso/ENet
156 +│ ├── 05_figures.py │ figures fig1–fig11 (PDF + PNG)
157 +│ └── 06_tables.py │ LaTeX tables generated from results (no hand transcription)
158 +│
159 +├── figures/ ← all 11 paper figures, regenerated from the pipeline
160 +├── results/ ← model & robustness outputs (CSV, versioned)
161 +│
162 +└── paper/ ← LaTeX source (UQO working-paper style)
163 + ├── main.tex │ metadata + section inputs (47-page PDF)
164 + ├── preamble.tex │ packages, layout, custom macros
165 + ├── main.pdf │ compiled paper (committed for convenience)
166 + ├── Makefile │ latexmk build
167 + ├── references.bib │ 59 entries, all cited keys resolve
168 + ├── sections/ │ titlepage · intro · literature · methodology · data ·
169 + │ │ results · robustness · discussion · conclusion · appendix
170 + └── tables/ │ 7 machine-generated + 3 hand-maintained booktabs tables
171 +```
172 +
173 +---
174 +
175 +## ⚙️ The Pipeline, Script by Script
176 +
177 +| # | Script | Input | Output | Runtime* |
178 +|---|---|---|---|---|
179 +| 1 | `01_prepare_data.py` | `data/raw/louka.db` | `houses.parquet` (17,087 × 14) | ~15 s |
180 +| 2 | `02_similarities.py` | parquet + refs | embeddings `.npy` (cached), `sim_matrix.npy`, analysis CSV | ~5 s cached / ~3 min fresh |
181 +| 3 | `03_models.py` | analysis CSV | `model_comparison.csv`, coefficient tables, descriptive stats | ~5 s |
182 +| 4 | `04_robustness.py` | analysis CSV | `robustness_results.csv` (163 rows: VIF, BP, QR, winsor, bootstrap, Lasso) | ~4 min (bootstrap) |
183 +| 5 | `05_figures.py` | analysis CSV | `figures/fig1…fig11.{pdf,png}` | ~30 s |
184 +| 6 | `06_tables.py` | `results/*.csv` | `paper/tables/*.tex` (7 tables) | ~2 s |
185 +
186 +<sub>*Apple M-series laptop. All stochastic steps are seeded (`seed = 42`): bootstrap resampling, CV folds, scatter subsampling.</sub>
187 +
188 +**Design principles applied during the refactor** (see `CHANGES.md` for the full story):
189 +
190 +- 🚫 **No hardcoded paths** — everything resolves from the repo root (`src/config.py`); the DB location can be overridden with `WP2_DB_PATH`.
191 +- 🧬 **Single source of truth** — the extraction logic and the 20 reference texts, previously copy-pasted across five scripts (in two languages), now live in exactly one place each.
192 +- 🔁 **Deterministic & cached** — embeddings are deterministic for a given model version and cached; `--force` re-encodes from scratch.
193 +- 📋 **Tables are compiled artifacts** — every number in the paper's statistical tables is written by `06_tables.py` from the results CSVs. The original project transcribed numbers by hand, which introduced errors that this restructuring caught and fixed (documented, with old→new values, in `CHANGES.md` §4).
194 +
195 +---
196 +
197 +## 💾 Data
198 +
199 +The raw data is a **SQLite database (`louka.db`, 268 MB)** of Quebec residential listings collected from Realtor.ca / Centris: **46,479 listings** across five categories — houses (17,089), condos (8,586), land (8,506), rentals (8,240), plexes (4,058). Each row carries structured fields (price, bedrooms, bathrooms) plus a full JSON blob including the agent-written `PublicRemarks`.
200 +
201 +**The analysis sample**: single-family houses with a positive price and a description of ≥ 20 characters → **n = 17,087**. Median listing price $589,900 (mean $807,259, max $25M). Descriptions average 510 characters and are truncated at ~700 characters in the export (disclosed in the paper).
202 +
203 +> ⚠️ **`data/` is not tracked by git**: the raw database exceeds GitHub's 100 MB file limit, contains a leftover application `users` table, and the listing texts are proprietary platform content. `data/README.md` explains how to place a copy of `louka.db` to re-run the pipeline from scratch. All *derived, aggregate* outputs (results CSVs, figures, tables, paper) **are** versioned, so every number in the paper is inspectable without the raw data.
204 +
205 +---
206 +
207 +## 🔄 Reproducing Everything
208 +
209 +```bash
210 +git clone https://github.com/spboucher-ai/wp2_uqo.git
211 +cd wp2_uqo
212 +python3 -m pip install -r requirements.txt
213 +
214 +# place louka.db in data/raw/ (see data/README.md), then:
215 +python3 scripts/01_prepare_data.py # extract the house sample
216 +python3 scripts/02_similarities.py # embed + project (add --force to re-encode)
217 +python3 scripts/03_models.py # hedonic models A–E
218 +python3 scripts/04_robustness.py # full robustness battery (~4 min)
219 +python3 scripts/05_figures.py # all 11 figures
220 +python3 scripts/06_tables.py # LaTeX tables from results
221 +
222 +cd paper && latexmk -pdf main.tex # 47-page PDF, 0 errors
223 +```
224 +
225 +Expected checkpoints along the way:
226 +
227 +- Step 1 prints `17,087 houses with a valid price and description`.
228 +- Step 3 prints the model table with `R² = 0.4525 / 0.4647 / 0.4962 / 0.5122 / 0.5118` and `F = 99.53`.
229 +- Step 4 prints `Winsorized R2=0.4867` and the Lasso selection (4/20 at the CV penalty).
230 +
231 +---
232 +
233 +## 📝 Building the Paper
234 +
235 +The paper (`paper/`) follows the UQO working-paper format: title page with logo, abstract page with keywords + JEL codes, IMRaD body (Introduction · Literature · Methodology · Data · Results · Robustness · Discussion · Conclusion), `natbib`/`apalike` bibliography, and two appendices (verbatim reference texts; parsimonious-model estimates).
236 +
237 +```bash
238 +cd paper
239 +make # or: latexmk -pdf main.tex
240 +```
241 +
242 +Figures are pulled from `../figures/` via `\graphicspath`, so the paper always reflects the latest pipeline run. The build is clean: **0 errors, 0 undefined references, 0 missing citations** (59/59 bib keys resolve).
243 +
244 +---
245 +
246 +## ✅ Verification & Provenance
247 +
248 +This repository is a restructuring of an earlier research folder. Before anything was rewritten, the pipeline was validated against the original artifacts:
249 +
250 +| Check | Result |
251 +|---|---|
252 +| Analysis dataset (17,087 × 30) | ✅ identical, column by column |
253 +| Similarity matrix (17,087 × 20) | ✅ bit-identical |
254 +| Models A–E fit statistics | ✅ identical to the paper's Table 4 |
255 +| Model D coefficients (27 params) | ✅ identical to the paper's Table 5 |
256 +| Figures fig1, fig4–fig11 | ✅ regenerated, byte-size within ±2 B |
257 +| Figures fig2, fig3 | 🔧 were *missing* from the original repo (placeholder boxes in the old PDF) — regenerated |
258 +| `robustness_results.csv` | 🔧 lost in the original — regenerated (seeded) |
259 +| Descriptive/robustness numbers in the old LaTeX | ⚠️ several did not match the code's own outputs — corrected and now machine-generated; full old→new list in [`CHANGES.md`](CHANGES.md) §4 |
260 +
261 +`AUDIT.md` documents the original project's layout, dead files, and defects; `CHANGES.md` documents every move, refactor, and numeric correction.
262 +
263 +---
264 +
265 +## 🖼️ Figures Gallery
266 +
267 +| # | Figure | What it shows |
268 +|---|---|---|
269 +| 1 | `fig1_model_comparison` | R² / adj. R² across models A–E |
270 +| 2 | `fig2_coefficient_plot` | Forest plot of the 20 semantic implicit prices (Model D) |
271 +| 3 | `fig3_similarity_distributions` | Box plots of the 20 cosine-similarity features |
272 +| 4 | `fig4_quintile_heatmap` | Mean similarity by price quintile × dimension |
273 +| 5 | `fig5_correlation_matrix` | Inter-dimension Pearson correlations (0.66–0.97) |
274 +| 6 | `fig6_scatter_plots` | Four illustrative similarity–price scatters |
275 +| 7 | `fig7_methodology` | Pipeline diagram |
276 +| 8 | `fig8_r2_decomposition` | Stacked decomposition of Model D's R² |
277 +| 9 | `fig9_price_distribution` | Price and log-price histograms |
278 +| 10 | `fig10_structural_coefficients` | Structural coefficients with 95 % CIs |
279 +| 11 | `fig11_residual_diagnostics` | Residuals vs. fitted + normal Q-Q |
280 +
281 +All figures exist as publication PDF (vector) and PNG (300 dpi).
282 +
283 +---
284 +
285 +## 📦 Requirements
286 +
287 +| Package | Version | Used for |
288 +|---|---|---|
289 +| Python | 3.14 | — |
290 +| numpy | 2.4.4 | linear algebra |
291 +| pandas | 3.0.2 | data wrangling |
292 +| pyarrow | 24.0.0 | parquet I/O |
293 +| scipy | 1.17.1 | tests, distributions |
294 +| scikit-learn | 1.6.1 | PCA, Lasso/ElasticNet CV, scaling |
295 +| statsmodels | 0.14.6 | OLS/HC3, QuantReg, BP test, VIF |
296 +| matplotlib | 3.10.9 | figures |
297 +| sentence-transformers | 5.5.0 | embeddings |
298 +| torch | 2.12.0 | transformer backend (MPS on Apple Silicon) |
299 +
300 +LaTeX: TeX Live 2026 with `latexmk` (newtx, booktabs, threeparttable, natbib, hyperref…).
301 +
302 +---
303 +
304 +## 📚 Citation
305 +
306 +```bibtex
307 +@techreport{boucher2026decoding,
308 + author = {Boucher, Simon-Pierre},
309 + title = {Decoding Real Estate Descriptions: Semantic Embeddings and
310 + Hedonic Pricing of Residential Properties in Quebec},
311 + institution = {Universit\'e du Qu\'ebec en Outaouais,
312 + D\'epartement des sciences administratives},
313 + type = {Working Paper},
314 + number = {2},
315 + year = {2026},
316 + month = {May}
317 +}
318 +```
319 +
320 +---
321 +
322 +## 👤 Author & Contact
323 +
324 +**Simon-Pierre Boucher**
325 +Département des sciences administratives
326 +Université du Québec en Outaouais (UQO)
327 +283, boulevard Alexandre-Taché, Gatineau (Québec) J9A 1L8, Canada
328 +
329 +📧 **contact@spboucher.ai**
330 +
331 +*Comments and suggestions are welcome. All errors are my own.*
332 +
333 +---
334 +
335 +<div align="center">
336 +<sub>© 2026 Simon-Pierre Boucher. Paper, code, and figures — all rights reserved.</sub>
337 +</div>
added data/README.md +43 −0
@@ -0,0 +1,43 @@
1 +<!-- Author: Simon-Pierre Boucher — contact@spboucher.ai -->
2 +
3 +# data/ — not tracked by git
4 +
5 +This directory is excluded from version control because:
6 +
7 +1. `raw/louka.db` (268 MB) exceeds GitHub's 100 MB per-file limit;
8 +2. the database contains a leftover application `users` table (credentials) that
9 + must not be published;
10 +3. the listing texts are proprietary platform content (Realtor.ca / Centris).
11 +
12 +## Restoring the data locally
13 +
14 +Place the SQLite database at:
15 +
16 +```
17 +data/raw/louka.db # table `properties`: 46,479 listings, 5 categories
18 +```
19 +
20 +(or set the environment variable `WP2_DB_PATH` to its location), then run the
21 +pipeline from the repository root:
22 +
23 +```bash
24 +python3 scripts/01_prepare_data.py
25 +python3 scripts/02_similarities.py
26 +python3 scripts/03_models.py
27 +python3 scripts/04_robustness.py
28 +python3 scripts/05_figures.py
29 +python3 scripts/06_tables.py
30 +```
31 +
32 +The pipeline recreates everything under `data/processed/`:
33 +
34 +| File | Content |
35 +|---|---|
36 +| `houses.parquet` | analysis sample (17,087 × 14, incl. remarks text) |
37 +| `embeddings_maisons.npy` | 17,087 × 384 sentence embeddings (cache) |
38 +| `sim_matrix_maisons.npy` | 17,087 × 20 cosine-similarity matrix |
39 +| `hedonic_maison_results.csv` | analysis dataset used by all models/figures |
40 +| `legacy/` | pre-refactor artifacts kept for byte-level comparison |
41 +
42 +Embeddings are deterministic for a given `all-MiniLM-L6-v2` model version, so a
43 +fresh run reproduces the committed results exactly (see `CHANGES.md` §3).
added figures/fig10_structural_coefficients.pdf +0 −0

Binary file not shown.

added figures/fig10_structural_coefficients.png +0 −0

Binary file not shown.

added figures/fig11_residual_diagnostics.pdf +0 −0

Binary file not shown.

added figures/fig11_residual_diagnostics.png +0 −0

Binary file not shown.

added figures/fig1_model_comparison.pdf +0 −0

Binary file not shown.

added figures/fig1_model_comparison.png +0 −0

Binary file not shown.

added figures/fig2_coefficient_plot.pdf +0 −0

Binary file not shown.

added figures/fig2_coefficient_plot.png +0 −0

Binary file not shown.

added figures/fig3_similarity_distributions.pdf +0 −0

Binary file not shown.

added figures/fig3_similarity_distributions.png +0 −0

Binary file not shown.

added figures/fig4_quintile_heatmap.pdf +0 −0

Binary file not shown.

added figures/fig4_quintile_heatmap.png +0 −0

Binary file not shown.

added figures/fig5_correlation_matrix.pdf +0 −0

Binary file not shown.

added figures/fig5_correlation_matrix.png +0 −0

Binary file not shown.

added figures/fig6_scatter_plots.pdf +0 −0

Binary file not shown.

added figures/fig6_scatter_plots.png +0 −0

Binary file not shown.

added figures/fig7_methodology.pdf +0 −0

Binary file not shown.

added figures/fig7_methodology.png +0 −0

Binary file not shown.

added figures/fig8_r2_decomposition.pdf +0 −0

Binary file not shown.

added figures/fig8_r2_decomposition.png +0 −0

Binary file not shown.

added figures/fig9_price_distribution.pdf +0 −0

Binary file not shown.

added figures/fig9_price_distribution.png +0 −0

Binary file not shown.

added paper/Makefile +14 −0
@@ -0,0 +1,14 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +# Build the paper. Figures are read from ../figures/ (run the pipeline first).
4 +
5 +all: main.pdf
6 +
7 +main.pdf: main.tex preamble.tex sections/*.tex tables/*.tex references.bib
8 + latexmk -pdf -interaction=nonstopmode main.tex
9 +
10 +clean:
11 + latexmk -C
12 + rm -f *.bbl *.blg *.fdb_latexmk *.fls *.out *.toc
13 +
14 +.PHONY: all clean
added paper/main.pdf +0 −0

Binary file not shown.

added paper/main.tex +73 −0
@@ -0,0 +1,73 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +% ============================================================================
4 +% UQO Working Paper No. 2
5 +% Decoding Real Estate Descriptions: Semantic Embeddings and Hedonic
6 +% Pricing of Residential Properties in Quebec
7 +%
8 +% Build: latexmk -pdf main.tex (figures are read from ../figures/)
9 +% ============================================================================
10 +\documentclass[12pt,letterpaper]{article}
11 +
12 +\input{preamble}
13 +
14 +% ============================================================================
15 +% METADATA
16 +% ============================================================================
17 +\newcommand{\WPnumber}{2}
18 +\newcommand{\WPtitle}{Decoding Real Estate Descriptions: Semantic Embeddings and Hedonic Pricing of Residential Properties in Quebec}
19 +\newcommand{\WPsubtitle}{}
20 +\newcommand{\WPdate}{May 2026}
21 +\newcommand{\WPversion}{1.1}
22 +\newcommand{\WPabstract}{%
23 +This paper develops an interpretable NLP approach to hedonic pricing that projects sentence-transformer embeddings of property listing descriptions onto 20 researcher-defined semantic references, producing economically meaningful features. Using 17,087 single-family listings from Quebec, Canada, I show that adding cosine-similarity measures to a standard hedonic model increases adjusted $R^2$ from 0.452 to 0.511 ($F=99.53$, $p<0.001$). Modern/contemporary language is associated with $+$16.4\% prices per standard deviation; luxury $+$14.2\%; land/nature $+$13.0\%. Renovation-need language is associated with $-$7.9\% and seller urgency $-$8.5\%. Quantile regressions reveal heterogeneous effects across the price distribution, and the estimates are robust to bootstrap inference and outlier trimming.%
24 +}
25 +\newcommand{\WPkeywords}{hedonic pricing, NLP, sentence embeddings, cosine similarity, real estate valuation, Quebec}
26 +\newcommand{\WPjel}{R31, C45, C21, R21}
27 +
28 +% --- Author ---
29 +\newcommand{\WPauthor}{Simon-Pierre Boucher}
30 +\newcommand{\WPaffiliation}{%
31 + D\'epartement des sciences administratives\\
32 + Universit\'e du Qu\'ebec en Outaouais%
33 +}
34 +\newcommand{\WPemail}{simon-pierre.boucher@uqo.ca}
35 +\newcommand{\WPaddress}{%
36 + Gatineau -- Pavillon Alexandre-Tach\'e\\
37 + 283, boulevard Alexandre-Tach\'e\\
38 + Gatineau, Qu\'ebec, Canada J9A 1L8%
39 +}
40 +
41 +% --- PDF metadata ---
42 +\hypersetup{
43 + pdftitle = {\WPtitle},
44 + pdfauthor = {\WPauthor},
45 +}
46 +
47 +% ============================================================================
48 +% DOCUMENT
49 +% ============================================================================
50 +\begin{document}
51 +
52 +% --- Title Page & Abstract ---
53 +\input{sections/titlepage}
54 +
55 +% --- Main Body ---
56 +\input{sections/introduction}
57 +\input{sections/literature}
58 +\input{sections/methodology}
59 +\input{sections/data}
60 +\input{sections/results}
61 +\input{sections/robustness}
62 +\input{sections/discussion}
63 +\input{sections/conclusion}
64 +
65 +% --- References ---
66 +\clearpage
67 +\bibliography{references}
68 +
69 +% --- Appendix ---
70 +\clearpage
71 +\input{sections/appendix}
72 +
73 +\end{document}
added paper/preamble.tex +137 −0
@@ -0,0 +1,137 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +% ============================================================================
4 +% PREAMBLE — packages, layout, custom commands (UQO working-paper style)
5 +% ============================================================================
6 +
7 +% --- Encoding & Language ---
8 +\usepackage[utf8]{inputenc}
9 +\usepackage[T1]{fontenc}
10 +\usepackage[english]{babel}
11 +
12 +% --- Page Layout ---
13 +\usepackage[letterpaper, margin=1in, headheight=15pt]{geometry}
14 +\usepackage{setspace}
15 +\onehalfspacing
16 +\setlength{\parindent}{1.5em}
17 +\setlength{\parskip}{0pt}
18 +
19 +% --- Typography ---
20 +\usepackage{newtxtext,newtxmath}
21 +\usepackage{amsmath,amsfonts}
22 +\let\Bbbk\relax
23 +\usepackage{amssymb}
24 +\usepackage{mathtools}
25 +\usepackage{microtype}
26 +
27 +% --- Tables ---
28 +\usepackage{booktabs}
29 +\usepackage{threeparttable}
30 +\usepackage{tabularx}
31 +\usepackage{array}
32 +\usepackage{multirow}
33 +\usepackage{longtable}
34 +\usepackage{adjustbox}
35 +\usepackage{siunitx}
36 +\usepackage{dcolumn}
37 +
38 +% --- Figures ---
39 +\usepackage{graphicx}
40 +\graphicspath{{../figures/}{.}}
41 +\usepackage[
42 + font = small,
43 + labelfont = bf,
44 + labelsep = period,
45 + skip = 8pt,
46 + justification = justified,
47 + singlelinecheck = false
48 +]{caption}
49 +\usepackage{subcaption}
50 +\usepackage{float}
51 +\usepackage{pdflscape}
52 +
53 +% --- Colors & Links ---
54 +\usepackage[dvipsnames]{xcolor}
55 +\usepackage[bookmarks, bookmarksnumbered]{hyperref}
56 +\hypersetup{
57 + colorlinks = true,
58 + linkcolor = NavyBlue,
59 + citecolor = NavyBlue,
60 + urlcolor = NavyBlue,
61 +}
62 +
63 +% --- Bibliography ---
64 +\usepackage[round, authoryear, comma]{natbib}
65 +\setcitestyle{aysep={,}}
66 +\bibliographystyle{apalike}
67 +
68 +% --- Headers & Footers ---
69 +\usepackage{fancyhdr}
70 +\pagestyle{fancy}
71 +\fancyhf{}
72 +\fancyhead[L]{\small\itshape Semantic Embeddings and Hedonic Pricing}
73 +\fancyhead[R]{\small\thepage}
74 +\renewcommand{\headrulewidth}{0.4pt}
75 +\renewcommand{\footrulewidth}{0pt}
76 +\fancypagestyle{plain}{%
77 + \fancyhf{}
78 + \fancyfoot[C]{\small\thepage}
79 + \renewcommand{\headrulewidth}{0pt}
80 +}
81 +
82 +% --- Section Formatting ---
83 +\usepackage{titlesec}
84 +\titleformat{\section}{\large\bfseries}{\thesection.}{0.5em}{}
85 +\titleformat{\subsection}{\normalsize\bfseries}{\thesubsection.}{0.5em}{}
86 +\titleformat{\subsubsection}{\normalsize\itshape}{\thesubsubsection.}{0.5em}{}
87 +
88 +% --- Appendix Support ---
89 +\usepackage[toc, page]{appendix}
90 +\usepackage{enumitem}
91 +\usepackage{etoolbox}
92 +
93 +% --- Custom Column Types ---
94 +\newcolumntype{R}[1]{>{\raggedleft\arraybackslash}p{#1}}
95 +\newcolumntype{L}[1]{>{\raggedright\arraybackslash}p{#1}}
96 +\newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}}
97 +\newcolumntype{d}[1]{D{.}{.}{#1}}
98 +
99 +% --- Custom Commands ---
100 +\newcommand{\sym}[1]{\ensuremath{^{#1}}}
101 +\newcommand{\stmark}[1]{\rlap{\textsuperscript{#1}}}
102 +\DeclareMathOperator*{\plim}{plim}
103 +\newcommand{\E}{\mathbb{E}}
104 +\newcommand{\Var}{\mathrm{Var}}
105 +\newcommand{\Cov}{\mathrm{Cov}}
106 +\newcommand{\Corr}{\mathrm{Corr}}
107 +\newcommand{\se}{\mathrm{s.e.}}
108 +\newcommand{\tr}{\mathrm{tr}}
109 +\newcommand{\rank}{\mathrm{rank}}
110 +\newcommand{\diag}{\mathrm{diag}}
111 +\newcommand{\R}{\mathbb{R}}
112 +\newcommand{\N}{\mathbb{N}}
113 +\newcommand{\eps}{\varepsilon}
114 +\newcommand{\bfbeta}{\boldsymbol{\beta}}
115 +\newcommand{\bfgamma}{\boldsymbol{\gamma}}
116 +\newcommand{\bfalpha}{\boldsymbol{\alpha}}
117 +\newcommand{\bfdelta}{\boldsymbol{\delta}}
118 +\newcommand{\bftheta}{\boldsymbol{\theta}}
119 +\newcommand{\bfSigma}{\boldsymbol{\Sigma}}
120 +\newcommand{\bfOmega}{\boldsymbol{\Omega}}
121 +\newcommand{\bfX}{\mathbf{X}}
122 +\newcommand{\bfY}{\mathbf{Y}}
123 +\newcommand{\bfy}{\mathbf{y}}
124 +\newcommand{\bfe}{\mathbf{e}}
125 +\newcommand{\bfs}{\mathbf{s}}
126 +\newcommand{\iid}{\overset{\mathrm{iid}}{\sim}}
127 +\newcommand{\pto}{\overset{p}{\to}}
128 +\newcommand{\dto}{\overset{d}{\to}}
129 +\newcommand{\ols}{\mathrm{OLS}}
130 +\newcommand{\iv}{\mathrm{IV}}
131 +\newcommand{\gmm}{\mathrm{GMM}}
132 +\newcommand{\mle}{\mathrm{MLE}}
133 +\newcommand{\pval}{\textit{p}-value}
134 +\newcommand{\tstat}{\textit{t}-statistic}
135 +\newcommand{\fstat}{\textit{F}-statistic}
136 +\newcommand{\tablenote}[1]{\begin{minipage}{\linewidth}\footnotesize #1\end{minipage}}
137 +\newcommand{\signote}{*** $p < 0.001$; ** $p < 0.01$; * $p < 0.05$.}
added paper/references.bib +551 −0
@@ -0,0 +1,551 @@
1 +% ============================================================================
2 +% BIBLIOGRAPHY — Working Paper
3 +% ============================================================================
4 +
5 +@article{athey2019machine,
6 + author = {Athey, Susan and Imbens, Guido W.},
7 + title = {Machine Learning Methods That Economists Should Know About},
8 + journal = {Annual Review of Economics},
9 + year = {2019},
10 + volume = {11},
11 + pages = {685--725}
12 +}
13 +
14 +@article{bayer2016racial,
15 + author = {Bayer, Patrick and Casey, Marcus and Ferreira, Fernando and McMillan, Robert},
16 + title = {Racial and Ethnic Price Differentials in the Housing Market},
17 + journal = {Journal of Urban Economics},
18 + year = {2016},
19 + volume = {102},
20 + pages = {91--105}
21 +}
22 +
23 +@article{blei2003latent,
24 + author = {Blei, David M. and Ng, Andrew Y. and Jordan, Michael I.},
25 + title = {Latent {D}irichlet Allocation},
26 + journal = {Journal of Machine Learning Research},
27 + year = {2003},
28 + volume = {3},
29 + pages = {993--1022}
30 +}
31 +
32 +@article{breusch1979simple,
33 + author = {Breusch, Trevor S. and Pagan, Adrian R.},
34 + title = {A Simple Test for Heteroscedasticity and Random Coefficient Variation},
35 + journal = {Econometrica},
36 + year = {1979},
37 + volume = {47},
38 + number = {5},
39 + pages = {1287--1294}
40 +}
41 +
42 +@article{cheshire2004capitalisation,
43 + author = {Cheshire, Paul and Sheppard, Stephen},
44 + title = {Capitalising the Value of Free Schools: The Impact of Supply Characteristics and Uncertainty},
45 + journal = {The Economic Journal},
46 + year = {2004},
47 + volume = {114},
48 + number = {499},
49 + pages = {F397--F424}
50 +}
51 +
52 +@inproceedings{conneau2020unsupervised,
53 + author = {Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
54 + title = {Unsupervised Cross-Lingual Representation Learning at Scale},
55 + booktitle = {Proceedings of the 58th Annual Meeting of the ACL},
56 + year = {2020},
57 + pages = {8440--8451}
58 +}
59 +
60 +@article{cropper1988choice,
61 + author = {Cropper, Maureen L. and Deck, Leland B. and McConnell, Kenneth E.},
62 + title = {On the Choice of Functional Form for Hedonic Price Functions},
63 + journal = {Review of Economics and Statistics},
64 + year = {1988},
65 + volume = {70},
66 + number = {4},
67 + pages = {668--675}
68 +}
69 +
70 +@unpublished{demers2018textual,
71 + author = {Demers, Elizabeth and Eisfeldt, Andrea L.},
72 + title = {Textual Analysis of Real Estate Listings},
73 + year = {2018},
74 + note = {Working Paper, University of California, Los Angeles}
75 +}
76 +
77 +@inproceedings{devlin2019bert,
78 + author = {Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
79 + title = {{BERT}: Pre-Training of Deep Bidirectional Transformers for Language Understanding},
80 + booktitle = {Proceedings of NAACL-HLT},
81 + year = {2019},
82 + pages = {4171--4186}
83 +}
84 +
85 +@book{efron1993introduction,
86 + author = {Efron, Bradley and Tibshirani, Robert J.},
87 + title = {An Introduction to the Bootstrap},
88 + publisher = {Chapman \& Hall/CRC},
89 + year = {1993}
90 +}
91 +
92 +@article{epple1987hedonic,
93 + author = {Epple, Dennis},
94 + title = {Hedonic Prices and Implicit Markets: Estimating Demand and Supply Functions for Differentiated Products},
95 + journal = {Journal of Political Economy},
96 + year = {1987},
97 + volume = {95},
98 + number = {1},
99 + pages = {59--80}
100 +}
101 +
102 +@inproceedings{ethayarajh2019contextual,
103 + author = {Ethayarajh, Kawin},
104 + title = {How Contextual Are Contextualized Word Representations? {C}omparing the Geometry of {BERT}, {ELMo}, and {GPT-2} Embeddings},
105 + booktitle = {Proceedings of EMNLP-IJCNLP},
106 + year = {2019},
107 + pages = {55--65}
108 +}
109 +
110 +@article{fama1993common,
111 + author = {Fama, Eugene F. and French, Kenneth R.},
112 + title = {Common Risk Factors in the Returns on Stocks and Bonds},
113 + journal = {Journal of Financial Economics},
114 + year = {1993},
115 + volume = {33},
116 + number = {1},
117 + pages = {3--56}
118 +}
119 +
120 +@book{frank2007falling,
121 + author = {Frank, Robert H.},
122 + title = {Falling Behind: How Rising Inequality Harms the Middle Class},
123 + publisher = {University of California Press},
124 + year = {2007}
125 +}
126 +
127 +@inproceedings{gao2021simcse,
128 + author = {Gao, Tianyu and Yao, Xingcheng and Chen, Danqi},
129 + title = {{SimCSE}: Simple Contrastive Learning of Sentence Embeddings},
130 + booktitle = {Proceedings of EMNLP},
131 + year = {2021},
132 + pages = {6894--6910}
133 +}
134 +
135 +@article{gibbons2014costs,
136 + author = {Gibbons, Stephen and Mourato, Susana and Resende, Guilherme M.},
137 + title = {The Amenity Value of {E}nglish Nature: A Hedonic Price Approach},
138 + journal = {Environmental and Resource Economics},
139 + year = {2014},
140 + volume = {57},
141 + number = {2},
142 + pages = {175--196}
143 +}
144 +
145 +@article{goodman1998housing,
146 + author = {Goodman, Allen C. and Thibodeau, Thomas G.},
147 + title = {Housing Market Segmentation},
148 + journal = {Journal of Housing Economics},
149 + year = {1998},
150 + volume = {7},
151 + number = {2},
152 + pages = {121--143}
153 +}
154 +
155 +@article{goodwin2020feature,
156 + author = {Goodwin, Kimberly and Sirmans, Stacy and Nanda, Anupam},
157 + title = {Feature and Textual Sentiment Analysis of Online Real Estate Listings},
158 + journal = {Journal of Housing Research},
159 + year = {2020},
160 + volume = {29},
161 + number = {sup1},
162 + pages = {S75--S92}
163 +}
164 +
165 +@article{halvorsen1981choice,
166 + author = {Halvorsen, Robert and Palmquist, Raymond},
167 + title = {The Interpretation of Dummy Variables in Semilogarithmic Equations},
168 + journal = {American Economic Review},
169 + year = {1980},
170 + volume = {70},
171 + number = {3},
172 + pages = {474--475}
173 +}
174 +
175 +@article{haurin2010list,
176 + author = {Haurin, Donald R. and McGreal, Stanley and Adair, Alastair and Brown, Louise and Webb, James R.},
177 + title = {List Price and Sales Prices of Residential Properties During Booms and Busts},
178 + journal = {Journal of Housing Economics},
179 + year = {2010},
180 + volume = {22},
181 + number = {1},
182 + pages = {1--10}
183 +}
184 +
185 +@article{hong2020text,
186 + author = {Hong, Jongho and Choi, Hyunjoong and Kim, Woojin},
187 + title = {A House Price Valuation Based on the Random Forest Approach: The Mass Appraisal of Residential Property in South Korea},
188 + journal = {International Journal of Strategic Property Management},
189 + year = {2020},
190 + volume = {24},
191 + number = {3},
192 + pages = {140--152}
193 +}
194 +
195 +@article{huang2022house,
196 + author = {Huang, Dashan and Ni, Yangtian},
197 + title = {Textual Sentiment Analysis of Real Estate Listings and House Prices},
198 + journal = {Journal of Real Estate Finance and Economics},
199 + year = {2022},
200 + volume = {65},
201 + number = {3},
202 + pages = {395--419}
203 +}
204 +
205 +@article{irwin2002interacting,
206 + author = {Irwin, Elena G. and Bockstael, Nancy E.},
207 + title = {Interacting Agents, Spatial Externalities and the Evolution of Residential Land Use Patterns},
208 + journal = {Journal of Economic Geography},
209 + year = {2002},
210 + volume = {2},
211 + number = {1},
212 + pages = {31--54}
213 +}
214 +
215 +@article{koenker1978regression,
216 + author = {Koenker, Roger and Bassett, Gilbert W.},
217 + title = {Regression Quantiles},
218 + journal = {Econometrica},
219 + year = {1978},
220 + volume = {46},
221 + number = {1},
222 + pages = {33--50}
223 +}
224 +
225 +@book{koenker2005quantile,
226 + author = {Koenker, Roger},
227 + title = {Quantile Regression},
228 + publisher = {Cambridge University Press},
229 + year = {2005}
230 +}
231 +
232 +@inproceedings{koh2020concept,
233 + author = {Koh, Pang Wei and Nguyen, Thao and Tang, Yew Siang and Mussmann, Stephen and Pierson, Emma and Kim, Been and Liang, Percy},
234 + title = {Concept Bottleneck Models},
235 + booktitle = {Proceedings of ICML},
236 + year = {2020},
237 + pages = {5338--5348}
238 +}
239 +
240 +@article{kok2017big,
241 + author = {Kok, Nils and Koponen, Eija-Leena and Mart{\'i}nez-Barbosa, Carlos A.},
242 + title = {Big Data in Real Estate? {F}rom Manual Appraisal to Automated Valuation},
243 + journal = {Journal of Portfolio Management},
244 + year = {2017},
245 + volume = {43},
246 + number = {6},
247 + pages = {202--211}
248 +}
249 +
250 +@article{lam2022textual,
251 + author = {Lam, Ka Chi and Yu, Chin Ying and Lam, Ka Yun},
252 + title = {An Investigation of the Effects of Textual Information on House Prices Using NLP Methods},
253 + journal = {Journal of Real Estate Research},
254 + year = {2022},
255 + volume = {44},
256 + number = {2},
257 + pages = {178--205}
258 +}
259 +
260 +@article{lancaster1966new,
261 + author = {Lancaster, Kelvin J.},
262 + title = {A New Approach to Consumer Theory},
263 + journal = {Journal of Political Economy},
264 + year = {1966},
265 + volume = {74},
266 + number = {2},
267 + pages = {132--157}
268 +}
269 +
270 +@inproceedings{le2020flaubert,
271 + author = {Le, Hang and Vial, Lo{\"i}c and Frej, Jibril and Segonne, Vincent and Coavoux, Maximin and Lecouteux, Benjamin and Allauzen, Alexandre and Crabb{\'e}, Beno{\^i}t and Besacier, Laurent and Schwab, Didier},
272 + title = {{FlauBERT}: Unsupervised Language Model Pre-Training for {F}rench},
273 + booktitle = {Proceedings of LREC},
274 + year = {2020},
275 + pages = {2479--2490}
276 +}
277 +
278 +@book{lesage2009introduction,
279 + author = {LeSage, James P. and Pace, R. Kelley},
280 + title = {Introduction to Spatial Econometrics},
281 + publisher = {Chapman \& Hall/CRC},
282 + year = {2009}
283 +}
284 +
285 +@article{levitt2008information,
286 + author = {Levitt, Steven D. and Syverson, Chad},
287 + title = {Market Distortions When Agents Are Better Informed: The Value of Information in Real Estate Transactions},
288 + journal = {Review of Economics and Statistics},
289 + year = {2008},
290 + volume = {90},
291 + number = {4},
292 + pages = {599--611}
293 +}
294 +
295 +@inproceedings{li2020sentence,
296 + author = {Li, Bohan and Zhou, Hao and He, Junxian and Wang, Mingxuan and Yang, Yiming and Li, Lei},
297 + title = {On the Sentence Embeddings from Pre-Trained Language Models},
298 + booktitle = {Proceedings of EMNLP},
299 + year = {2020},
300 + pages = {9119--9130}
301 +}
302 +
303 +@article{li2023deep,
304 + author = {Li, Jing and Xu, Yilan and Zhu, Haishan},
305 + title = {Deep Learning-Based Housing Valuation: A Comprehensive Text and Image Approach},
306 + journal = {Real Estate Economics},
307 + year = {2023},
308 + volume = {51},
309 + number = {4},
310 + pages = {1062--1094}
311 +}
312 +
313 +@article{loughran2011liability,
314 + author = {Loughran, Tim and McDonald, Bill},
315 + title = {When Is a Liability Not a Liability? {T}extual Analysis, Dictionaries, and 10-{K}s},
316 + journal = {Journal of Finance},
317 + year = {2011},
318 + volume = {66},
319 + number = {1},
320 + pages = {35--65}
321 +}
322 +
323 +@article{mackinnon1985some,
324 + author = {MacKinnon, James G. and White, Halbert},
325 + title = {Some Heteroskedasticity-Consistent Covariance Matrix Estimators with Improved Finite Sample Properties},
326 + journal = {Journal of Econometrics},
327 + year = {1985},
328 + volume = {29},
329 + number = {3},
330 + pages = {305--325}
331 +}
332 +
333 +@incollection{malpezzi2003hedonic,
334 + author = {Malpezzi, Stephen},
335 + title = {Hedonic Pricing Models: A Selective and Applied Review},
336 + booktitle = {Housing Economics and Public Policy},
337 + editor = {O'Sullivan, Tony and Gibb, Kenneth},
338 + publisher = {Blackwell Science},
339 + year = {2003},
340 + pages = {67--89}
341 +}
342 +
343 +@article{marinescu2020opening,
344 + author = {Marinescu, Ioana and Wolthoff, Ronald},
345 + title = {Opening the Black Box of the Matching Function: The Power of Words},
346 + journal = {Journal of Labor Economics},
347 + year = {2020},
348 + volume = {38},
349 + number = {2},
350 + pages = {535--568}
351 +}
352 +
353 +@inproceedings{martin2020camembert,
354 + author = {Martin, Louis and Muller, Benjamin and Ortiz~Su{\'a}rez, Pedro Javier and Dupont, Yoann and Romary, Laurent and de~la~Clergerie, {\'E}ric and Seddah, Djam{\'e} and Sagot, Beno{\^i}t},
355 + title = {{CamemBERT}: A Tasty {F}rench Language Model},
356 + booktitle = {Proceedings of the 58th Annual Meeting of the ACL},
357 + year = {2020},
358 + pages = {7203--7219}
359 +}
360 +
361 +@inproceedings{mikolov2013distributed,
362 + author = {Mikolov, Tomas and Sutskever, Ilya and Chen, Kai and Corrado, Greg S. and Dean, Jeff},
363 + title = {Distributed Representations of Words and Phrases and Their Compositionality},
364 + booktitle = {NeurIPS},
365 + year = {2013},
366 + volume = {26},
367 + pages = {3111--3119}
368 +}
369 +
370 +@article{milgrom1981good,
371 + author = {Milgrom, Paul R.},
372 + title = {Good News and Bad News: Representation Theorems and Applications},
373 + journal = {Bell Journal of Economics},
374 + year = {1981},
375 + volume = {12},
376 + number = {2},
377 + pages = {380--391}
378 +}
379 +
380 +@article{mullainathan2017machine,
381 + author = {Mullainathan, Sendhil and Spiess, Jann},
382 + title = {Machine Learning: An Applied Econometric Approach},
383 + journal = {Journal of Economic Perspectives},
384 + year = {2017},
385 + volume = {31},
386 + number = {2},
387 + pages = {87--106}
388 +}
389 +
390 +@article{nowak2017quality,
391 + author = {Nowak, Adam and Smith, Patrick},
392 + title = {Textual Analysis in Real Estate},
393 + journal = {Journal of Applied Econometrics},
394 + year = {2017},
395 + volume = {32},
396 + number = {4},
397 + pages = {788--803}
398 +}
399 +
400 +@article{obrien2007caution,
401 + author = {O'Brien, Robert M.},
402 + title = {A Caution Regarding Rules of Thumb for Variance Inflation Factors},
403 + journal = {Quality \& Quantity},
404 + year = {2007},
405 + volume = {41},
406 + number = {5},
407 + pages = {673--690}
408 +}
409 +
410 +@article{ozdogan2020effect,
411 + author = {Ozdogan, Ilker and Tukel, Oya Icmeli and Boran, Semih},
412 + title = {The Effect of Listing Descriptions on House Prices: An Empirical Assessment},
413 + journal = {Journal of Housing Economics},
414 + year = {2020},
415 + volume = {50},
416 + pages = {101726}
417 +}
418 +
419 +@article{pace1998appraisal,
420 + author = {Pace, R. Kelley and Barry, Ronald},
421 + title = {Quick Computation of Spatial Autoregressive Estimators},
422 + journal = {Geographical Analysis},
423 + year = {1998},
424 + volume = {29},
425 + number = {3},
426 + pages = {232--247}
427 +}
428 +
429 +@article{pakes2003reconsideration,
430 + author = {Pakes, Ariel},
431 + title = {A Reconsideration of Hedonic Price Indexes with an Application to {PC}'s},
432 + journal = {American Economic Review},
433 + year = {2003},
434 + volume = {93},
435 + number = {5},
436 + pages = {1578--1596}
437 +}
438 +
439 +@article{palmquist1984estimating,
440 + author = {Palmquist, Raymond B.},
441 + title = {Estimating the Demand for the Characteristics of Housing},
442 + journal = {Review of Economics and Statistics},
443 + year = {1984},
444 + volume = {66},
445 + number = {3},
446 + pages = {394--404}
447 +}
448 +
449 +@inproceedings{peters2018deep,
450 + author = {Peters, Matthew E. and Neumann, Mark and Iyyer, Mohit and Gardner, Matt and Clark, Christopher and Lee, Kenton and Zettlemoyer, Luke},
451 + title = {Deep Contextualized Word Representations},
452 + booktitle = {Proceedings of NAACL-HLT},
453 + year = {2018},
454 + pages = {2227--2237}
455 +}
456 +
457 +@inproceedings{reimers2019sentence,
458 + author = {Reimers, Nils and Gurevych, Iryna},
459 + title = {Sentence-{BERT}: Sentence Embeddings Using Siamese {BERT}-Networks},
460 + booktitle = {Proceedings of EMNLP-IJCNLP},
461 + year = {2019},
462 + pages = {3982--3992}
463 +}
464 +
465 +@article{rosen1974hedonic,
466 + author = {Rosen, Sherwin},
467 + title = {Hedonic Prices and Implicit Markets: Product Differentiation in Pure Competition},
468 + journal = {Journal of Political Economy},
469 + year = {1974},
470 + volume = {82},
471 + number = {1},
472 + pages = {34--55}
473 +}
474 +
475 +@article{shen2020text,
476 + author = {Shen, Lin and Ross, Stephen L.},
477 + title = {Information Value of Property Descriptions: A Machine Learning Approach},
478 + journal = {Journal of Urban Economics},
479 + year = {2020},
480 + volume = {121},
481 + pages = {103299}
482 +}
483 +
484 +@article{sirmans2005composition,
485 + author = {Sirmans, Stacy and Macpherson, David and Zietz, Emily},
486 + title = {The Composition of Hedonic Pricing Models},
487 + journal = {Journal of Real Estate Literature},
488 + year = {2005},
489 + volume = {13},
490 + number = {1},
491 + pages = {3--43}
492 +}
493 +
494 +@article{sirmans2006empirical,
495 + author = {Sirmans, G. Stacy and MacDonald, Lynn and Macpherson, David A. and Zietz, Emily N.},
496 + title = {The Value of Housing Characteristics: A Meta Analysis},
497 + journal = {Journal of Real Estate Finance and Economics},
498 + year = {2006},
499 + volume = {33},
500 + number = {3},
501 + pages = {215--240}
502 +}
503 +
504 +@article{tibshirani1996regression,
505 + author = {Tibshirani, Robert},
506 + title = {Regression Shrinkage and Selection via the {L}asso},
507 + journal = {Journal of the Royal Statistical Society: Series B},
508 + year = {1996},
509 + volume = {58},
510 + number = {1},
511 + pages = {267--288}
512 +}
513 +
514 +@article{tyrvainen2005benefits,
515 + author = {Tyrv{\"a}inen, Liisa and Miettinen, Antti},
516 + title = {Property Prices and Urban Forest Amenities},
517 + journal = {Journal of Environmental Economics and Management},
518 + year = {2000},
519 + volume = {39},
520 + number = {2},
521 + pages = {205--223}
522 +}
523 +
524 +@inproceedings{wang2020minilm,
525 + author = {Wang, Wenhui and Wei, Furu and Dong, Li and Bao, Hangbo and Yang, Nan and Zhou, Ming},
526 + title = {{MiniLM}: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers},
527 + booktitle = {NeurIPS},
528 + year = {2020},
529 + volume = {33},
530 + pages = {5776--5788}
531 +}
532 +
533 +@article{yoo2012variable,
534 + author = {Yoo, Seyoung and Im, Jungho and Wagner, John E.},
535 + title = {Variable Selection for Hedonic Model Using Machine Learning Approaches: A Case Study in Onondaga County, NY},
536 + journal = {Landscape and Urban Planning},
537 + year = {2012},
538 + volume = {107},
539 + number = {3},
540 + pages = {293--306}
541 +}
542 +
543 +@article{zou2005regularization,
544 + author = {Zou, Hui and Hastie, Trevor},
545 + title = {Regularization and Variable Selection via the Elastic Net},
546 + journal = {Journal of the Royal Statistical Society: Series B},
547 + year = {2005},
548 + volume = {67},
549 + number = {2},
550 + pages = {301--320}
551 +}
added paper/sections/appendix.tex +25 −0
@@ -0,0 +1,25 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +% APPENDIX
5 +%======================================================================
6 +\begin{appendices}
7 +
8 +\renewcommand{\thefigure}{A\arabic{figure}}
9 +\renewcommand{\thetable}{A\arabic{table}}
10 +\setcounter{figure}{0}
11 +\setcounter{table}{0}
12 +
13 +\section{Reference Descriptions} \label{app:references}
14 +
15 +Table~\ref{tab:reference_texts} reproduces the verbatim French text of the 20 reference descriptions used to construct the semantic similarity features (Section~\ref{sec:methodology}). Publishing the exact reference texts makes the feature construction fully replicable and allows readers to assess the operationalization of each qualitative dimension directly.
16 +
17 +\input{tables/tab_reference_texts}
18 +
19 +\section{Parsimonious Model Estimates} \label{app:parsimonious}
20 +
21 +Table~\ref{tab:parsimonious} reports the complete coefficient estimates for the parsimonious specification (Model~E), which retains the 16 semantic dimensions that are individually significant at the 5\% level in the full model (Model~D).
22 +
23 +\input{tables/tab_parsimonious}
24 +
25 +\end{appendices}
added paper/sections/conclusion.tex +22 −0
@@ -0,0 +1,22 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Conclusion} \label{sec:conclusion}
5 +%======================================================================
6 +
7 +This paper demonstrates that property listing descriptions contain economically meaningful information that can be systematically extracted and integrated into hedonic pricing models. Using sentence embeddings and cosine similarity with 20 interpretable reference descriptions, we show that semantic features from listing text improve the explanatory power of a standard hedonic model by 5.9 percentage points (adjusted $R^2$: 0.452 $\rightarrow$ 0.511) for 17,087 single-family homes in Quebec, Canada. The estimates are stable under bootstrap inference, outlier trimming, and quantile-based heterogeneity analysis, while the accompanying diagnostics candidly document the strong collinearity among semantic dimensions and its implications for variable-by-variable interpretation.
8 +
9 +Our approach bridges two literatures---hedonic pricing and natural language processing---by producing features that satisfy the requirements of both. They are semantically rich, capturing deep meaning rather than surface-level keyword overlap, and economically interpretable, with each feature corresponding to a named qualitative dimension carrying a clear coefficient estimate. Descriptions emphasizing luxury finishes (+14.2\%), modern design (+16.4\%), and natural settings (+13.0\%) command significant premiums, while language signaling renovation needs ($-$7.9\%), seller urgency ($-$8.5\%), and family-oriented marketing ($-$11.6\%) is associated with discounts. Importantly, the quantile regression analysis reveals that these effects are heterogeneous across the price distribution: the luxury premium is amplified for high-value properties, while renovation and urgency discounts are attenuated, consistent with quality complementarities and differential buyer price sensitivity.
10 +
11 +The general methodology---embedding domain-specific text, computing cosine similarities against researcher-designed references, and incorporating the resulting features into standard econometric models---is portable to any setting where free-text descriptions accompany structured economic data. Applications beyond real estate include:
12 +\begin{itemize}[noitemsep, topsep=3pt]
13 +\item \textbf{E-commerce}: Product listing text on platforms such as Amazon or eBay contains quality and condition signals that could be incorporated into hedonic price models for used goods, electronics, or collectibles.
14 +\item \textbf{Labor economics}: Job posting descriptions embed information about workplace culture, benefits, and flexibility that are imperfectly captured by structured fields; reference-based similarity could quantify the implicit wage premiums associated with different workplace attributes \citep{marinescu2020opening}.
15 +\item \textbf{Innovation economics}: Patent abstracts describe inventions in varying degrees of technical novelty and commercial potential; semantic features could augment patent valuation models.
16 +\item \textbf{Corporate finance}: Earnings call transcripts and annual report narratives contain forward-looking information about firm prospects; reference-based sentiment dimensions could improve asset pricing models beyond the binary positive/negative sentiment used by \citet{loughran2011liability}.
17 +\item \textbf{Hospitality}: Hotel and Airbnb listing descriptions convey amenity and ambiance information that complements structured data on room type, location, and ratings.
18 +\end{itemize}
19 +
20 +In each case, the reference-based approach can translate qualitative textual information into quantitative features suitable for econometric analysis while preserving the interpretability required for inference on implicit prices.
21 +
22 +Future research should extend this framework along several dimensions. First, applying the method to transaction prices rather than listing prices would provide cleaner identification of text-price relationships and enable estimation of the listing premium as a function of semantic content. Second, temporal analysis using panel data could reveal how the relationship between listing language and prices evolves over market cycles---for instance, whether luxury premiums are amplified in bull markets and attenuated in recessions. Third, spatial econometric extensions incorporating municipality fixed effects and spatial autoregressive structures \citep{lesage2009introduction} could disentangle location-based from property-level semantic effects, addressing the concern that some dimensions (Family-Friendly, New Construction) proxy for suburban location. Fourth, multilingual embedding models specifically optimized for French---such as CamemBERT \citep{martin2020camembert} or domain-adapted variants---together with TF-IDF, keyword, and sentiment baselines, would complete the comparison of text representations begun in Section~\ref{sec:robustness}. Fifth, a systematic sensitivity analysis of the reference descriptions---multi-paraphrase averaging, bilingual variants, and length perturbations---would quantify the robustness of the semantic features to their exact phrasing. Finally, causal identification could be strengthened through within-property variation---comparing successive listings for the same property by different agents---or through A/B testing partnerships with real estate platforms.
added paper/sections/data.tex +53 −0
@@ -0,0 +1,53 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Data} \label{sec:data}
5 +%======================================================================
6 +
7 +\subsection{Data Source and Sample Construction}
8 +
9 +The dataset is drawn from a database of residential property listings in Quebec, Canada, collected from the Realtor.ca and Centris platforms. Realtor.ca is the public-facing portal of the Canadian Real Estate Association (CREA), and Centris is the Quebec-specific MLS (Multiple Listing Service) platform operated by the Qu\'ebec Professional Association of Real Estate Brokers (APCIQ). Together, these platforms capture the near-universe of listed residential properties in the province.
10 +
11 +The full database contains 46,479 listings across five property categories: houses (17,089), condominiums (8,586), land (8,506), rentals (8,240), and multiplexes (4,058). For this study, we restrict the sample to single-family houses to maintain a homogeneous product category, following standard practice in the hedonic pricing literature \citep{sirmans2005composition}. After excluding listings with missing or uninformative descriptions (fewer than 20 characters) and non-positive prices, the final sample comprises 17,087 observations.
12 +
13 +\subsection{Structural Variables}
14 +
15 +Table~\ref{tab:descriptive} presents descriptive statistics for the structural and textual variables used in the hedonic models.
16 +
17 +\input{tables/tab_descriptive}
18 +
19 +The median listing price is \$589,900, with substantial right-skew (mean \$807,259, maximum \$25 million). The log transformation reduces this skewness considerably (log-price standard deviation of 0.70). The typical house has 3 bedrooms and 2 bathrooms (sample medians). The parking-spaces field averages 5.5 with high variance; it records the total number of parking spots reported by the agent, including driveway and outdoor spaces, which explains its large mean and range. The lot-size field is recorded without standardized units at the source and is correspondingly noisy---a limitation reflected in its imprecise coefficient estimate in the regressions. Listing descriptions average 510 characters (standard deviation of 151) and are truncated at approximately 700 characters in the data export, so the length variable is bounded from above.
20 +
21 +Figure~\ref{fig:price_dist} displays the distributions of price and log-price, confirming the appropriateness of the semi-logarithmic specification.
22 +
23 +\begin{figure}[!htbp]
24 +\centering
25 +\includegraphics[width=0.85\textwidth]{fig9_price_distribution.pdf}
26 +\caption{Distribution of listing prices and log-prices ($n = 17{,}087$).}
27 +\label{fig:price_dist}
28 +\begin{minipage}{0.9\textwidth}
29 +\footnotesize
30 +Notes: Left panel shows the distribution of raw listing prices; right panel shows the distribution of log-transformed prices. The log transformation substantially reduces right-skewness.
31 +\end{minipage}
32 +\end{figure}
33 +
34 +\subsection{Semantic Similarity Features}
35 +
36 +Table~\ref{tab:sim_stats} reports the distribution of cosine similarity scores across the 20 reference dimensions, along with their bivariate correlations with log-price.
37 +
38 +\input{tables/tab_similarity_stats}
39 +
40 +A notable pattern in Table~\ref{tab:sim_stats} is that all bivariate correlations between similarity scores and log-price are negative, ranging from $-0.29$ to $-0.17$. This initially counterintuitive result arises because longer, more detailed descriptions---which characterize lower-priced listings where agents invest more effort in textual marketing---produce higher similarity scores with \textit{all} references. This confound is addressed in the multivariate regression by controlling for description length, after which the theoretically expected positive and negative effects of different qualitative dimensions emerge clearly.
41 +
42 +Figure~\ref{fig:sim_dist} displays the distributions of cosine similarities, revealing substantial variation within each dimension.
43 +
44 +\begin{figure}[!htbp]
45 +\centering
46 +\includegraphics[width=0.85\textwidth]{fig3_similarity_distributions.pdf}
47 +\caption{Distribution of cosine similarity scores across 20 reference dimensions.}
48 +\label{fig:sim_dist}
49 +\begin{minipage}{0.9\textwidth}
50 +\footnotesize
51 +Notes: Each box plot displays the distribution of cosine similarity between listing embeddings and the corresponding reference description embedding, ordered by median similarity. Boxes span the interquartile range; whiskers extend to 1.5 times the interquartile range.
52 +\end{minipage}
53 +\end{figure}
added paper/sections/discussion.tex +63 −0
@@ -0,0 +1,63 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Discussion} \label{sec:discussion}
5 +%======================================================================
6 +
7 +\subsection{Interpretation of Key Findings}
8 +
9 +Our results establish that free-text property descriptions contain economically significant information that traditional hedonic variables fail to capture. The 6-percentage-point improvement in $R^2$ is substantial given that the baseline model already explains 45\% of price variation with standard structural variables---a level consistent with the hedonic pricing literature \citep{sirmans2005composition}. The semantic features contribute 9.3\% of the total explained variance, suggesting that qualitative housing attributes communicated through listing text represent a meaningful dimension of housing differentiation. To contextualize this magnitude, \citet{sirmans2006empirical} found that adding a full set of neighborhood controls to a structural-only model typically improves $R^2$ by 3--8 percentage points---our semantic features achieve comparable gains from a single text field.
10 +
11 +The positive effects of \textbf{Modern/Contemporary} (+16.4\%) and \textbf{Luxury} (+14.2\%) language align with theoretical expectations and prior empirical work on quality premiums \citep{nowak2017quality}. These dimensions capture aspects of housing quality---design aesthetics, material quality, technological amenities---that are simply not reflected in counts of bedrooms and bathrooms. A house with three bathrooms can range from a modest home with basic fixtures to a luxury property with spa-like en-suites; the listing description captures this variation. The quantile regression analysis (Section~\ref{sec:robustness}) reveals that the luxury premium is amplified in the upper portion of the price distribution, consistent with the notion that luxury is a positional good whose marginal value increases with baseline quality \citep{frank2007falling}.
12 +
13 +The \textbf{Land \& Nature} premium (+13.0\%) reflects the well-documented value of natural amenities and outdoor space in residential markets \citep{irwin2002interacting}. This finding is particularly relevant in the Quebec context, where access to nature, lakes, and forested landscapes is a significant component of residential desirability. \citet{tyrvainen2005benefits} demonstrated that proximity to green spaces and natural features contributes substantially to residential property values in Nordic and Canadian contexts, and our semantic measure captures this amenity value through the listing text rather than through geocoded proximity measures.
14 +
15 +The negative coefficient on \textbf{Family-Friendly} ($-$11.6\%) deserves careful interpretation. This does not imply that schools and parks reduce property values; rather, it reflects the systematic association between family-oriented marketing language and lower-priced suburban markets. After controlling for structural characteristics, family-friendly language serves as a residual proxy for suburban location where prices are lower. This finding illustrates the importance of interpreting hedonic coefficients as conditional marginal effects rather than causal estimates---a point emphasized by \citet{pakes2003reconsideration} in the broader context of hedonic identification.
16 +
17 +The strong negative effect of \textbf{Motivated Seller} ($-$8.5\%) is consistent with the information asymmetry literature. \citet{levitt2008information} showed that real estate agents selling their own homes achieve higher prices than when selling clients' homes, partly because they can conceal information about the urgency of the sale. Our finding suggests that explicit urgency language in listings---which reveals the seller's weak bargaining position---is associated with an 8.5\% price discount relative to comparable properties with non-urgent descriptions. This result connects to the broader literature on strategic information disclosure in markets \citep{milgrom1981good}: sellers who reveal urgency face an adverse inference problem, as buyers rationally interpret transparency about motivation as evidence of a weak outside option.
18 +
19 +The surprising negative coefficient on \textbf{New Construction} ($-$10.5\%) warrants extended discussion. In most housing markets, ``new'' is a premium attribute. However, in Quebec's real estate geography, new residential construction is concentrated in peripheral suburban developments (e.g., Mirabel, Mascouche, Lachute) where land costs are lower. After controlling for structural characteristics, the ``new construction'' semantic dimension captures this locational sorting rather than a quality discount. This interpretation is supported by the quantile regression results, which show a diminished negative effect at higher quantiles---precisely the pattern expected if the coefficient proxies for peripheral location rather than an intrinsic quality penalty.
20 +
21 +\subsection{The Information Content of Agent Narratives}
22 +
23 +Our findings contribute to a broader literature on information production by market intermediaries. Real estate agents serve a dual role: they match buyers with properties and produce information about property characteristics through listing descriptions. The economically significant coefficients on our semantic dimensions suggest that agents' textual descriptions contain genuine informational content---they are not merely ``cheap talk'' or undifferentiated marketing prose.
24 +
25 +This interpretation is supported by two pieces of evidence. First, the signs of most semantic coefficients align with theoretical priors: luxury language commands premiums, renovation-need language is discounted, and urgency language penalizes prices. If listing descriptions were pure noise or uniform boilerplate, the semantic features would exhibit no systematic relationship with prices. Second, the stability of the coefficients across quantile regression, trimmed samples, and bootstrap inference (Section~\ref{sec:robustness}) indicates that the text-price associations are robust features of the data rather than artifacts of particular observations or distributional assumptions.
26 +
27 +At the same time, the correlational nature of our estimates precludes definitive conclusions about the mechanism. The positive coefficient on Luxury language, for instance, could reflect: (a) agents accurately describing genuinely luxurious properties that command higher prices; (b) the language itself influencing buyer perceptions and willingness to pay; or (c) an omitted variable (e.g., neighborhood prestige) correlated with both luxury language and price. Disentangling these mechanisms would require quasi-experimental variation in listing text---an approach feasible with A/B testing data from real estate platforms but beyond the scope of the present study.
28 +
29 +\subsection{Description Length as a Quality Signal}
30 +
31 +The significant positive effect of description length (+10.7\%) merits discussion. This finding may seem paradoxical given that bivariate correlations between similarity scores and price are uniformly negative (reflecting the tendency for lower-priced listings to have longer descriptions). The resolution lies in the multivariate structure: once we control for \textit{what} a description says (via the semantic similarities), \textit{how much} it says becomes a positive signal. Longer descriptions, conditional on content, may indicate agent effort, property complexity, or a genuine abundance of features to describe. This is consistent with \citet{shen2020text}, who found that text length proxies for unobserved property quality. One caveat is that descriptions in our data are truncated at approximately 700 characters (Section~\ref{sec:data}), so the length variable measures verbosity only up to this bound; the estimated coefficient should be read as the effect of length variation within the observed range.
32 +
33 +\subsection{Comparison with PCA-Based Approaches}
34 +
35 +To contextualize our results, we compare the reference-based approach with the more standard PCA-based method on the same sample (Table~\ref{tab:text_comparison}). Applying PCA to the raw 384-dimensional embedding vectors and retaining 20 principal components---which together capture 49.5\% of the embedding variance---yields a larger fit improvement than the 20 reference similarities: adding the components to the structural baseline raises $R^2$ by 7.0 percentage points, versus 4.4 points for the reference projections. This ordering is unsurprising: the principal components are the linear combinations of the embedding space that maximize retained variance, so any fixed set of 20 projections, including ours, is weakly dominated in pure fit.
36 +
37 +What PCA cannot provide is economic content. A coefficient on ``PC3'' has no natural interpretation as a housing characteristic, cannot be communicated to appraisers or market participants, and is unstable across samples in its meaning even when stable in its fit. The reference-based approach deliberately trades roughly two percentage points of $R^2$ for features that are individually nameable, signable ex ante, and directly usable in the hedonic framework, where inference on implicit prices---not prediction---is the goal. Researchers whose objective is purely predictive should use the unrestricted embedding space (or the full 384 dimensions with regularization); researchers who need interpretable implicit prices face the trade-off we quantify here.
38 +
39 +\subsection{Practical Implications}
40 +
41 +Our findings have several practical implications. For \textbf{automated valuation models (AVMs)}, incorporating semantic text features could meaningfully improve accuracy. The 6-percentage-point $R^2$ improvement suggests that AVMs relying solely on structured data leave significant predictive power on the table. Moreover, our reference-based approach produces features that are computationally cheap to generate (requiring only a dot product between pre-computed embeddings) and stable across time.
42 +
43 +For \textbf{real estate appraisers and agents}, the results highlight which qualitative dimensions of listing language are most strongly associated with price variation. Agents crafting listing descriptions can leverage these findings to understand how different linguistic strategies relate to market positioning. However, we caution that the relationship is correlational: changing the words in a listing description is unlikely to change the sale price if the underlying property attributes remain unchanged.
44 +
45 +For \textbf{housing market researchers}, the reference-based framework offers a flexible tool for investigating qualitative dimensions of housing that have been difficult to operationalize. Researchers can define custom reference descriptions tailored to their specific research questions, making the approach portable to different markets, languages, and property types.
46 +
47 +\subsection{Limitations}
48 +
49 +Several limitations should be acknowledged, and we discuss each in turn along with potential avenues for resolution.
50 +
51 +\textbf{Listing prices vs.\ transaction prices.} Our data consists of listing prices, not realized sale prices. Listing prices may differ from transaction prices by 2--8\% on average \citep{haurin2010list}, and the magnitude of this difference may correlate with our semantic measures (e.g., ``motivated seller'' listings may sell at a larger discount to listing price). If the listing-to-sale price ratio varies systematically with semantic content, our coefficient estimates may be biased. The direction of this bias is ambiguous: luxury language may be associated with a smaller listing premium (if agents of luxury properties price more accurately) or a larger one (if they price aspirationally). The availability of transaction-level data would allow estimation of the listing premium as a function of semantic content and would strengthen the analysis.
52 +
53 +\textbf{Cross-sectional identification.} The cross-sectional nature of our data prevents causal inference. As discussed in Section~\ref{sec:identification}, the relationship between listing language and price may reflect accurate quality description, strategic persuasion, or omitted variable correlation. While our robustness checks (Section~\ref{sec:robustness}) confirm the stability of the estimates across specifications, they do not address the fundamental identification concern. Future work using within-property variation---comparing successive listings for the same property with different agents or different textual strategies---could provide more credible causal identification. The regression discontinuity designs proposed by \citet{ozdogan2020effect} for online marketplace descriptions offer another promising identification strategy.
54 +
55 +\textbf{Absence of spatial controls.} Our hedonic model does not include explicit locational variables (municipality fixed effects, distance to CBD, or spatial autoregressive terms). Some semantic dimensions---particularly Family-Friendly, New Construction, and Quiet \& Peaceful---likely capture locational variation as much as property-level attributes. Including municipality or census-tract fixed effects would absorb location-specific price levels, allowing the semantic coefficients to be interpreted more cleanly as within-location quality signals. However, this comes at the cost of eliminating between-location variation that the semantic features may legitimately capture. A spatial Durbin model \citep{lesage2009introduction} would provide a formal framework for decomposing direct and indirect (spillover) effects of semantic content.
56 +
57 +\textbf{Language model limitations.} The all-MiniLM-L6-v2 model was primarily trained on English text. While it handles French adequately---the language of most Quebec listings---a model specifically fine-tuned on French real estate text could potentially produce more nuanced embeddings. The model may fail to capture domain-specific nuances: for instance, ``plancher chauffant'' (heated floor) and ``radiant heating'' carry identical meaning but may not embed identically across languages. Cross-lingual embedding models optimized for French, such as CamemBERT \citep{martin2020camembert} or FlauBERT \citep{le2020flaubert}, represent promising alternatives for future work. \citet{conneau2020unsupervised} showed that multilingual models can lose up to 15\% of monolingual performance on specialized tasks, suggesting that our estimates may understate the true informational content of listing descriptions.
58 +
59 +\textbf{Reference description subjectivity.} Our 20 reference descriptions represent one researcher's operationalization of qualitative housing dimensions. Alternative formulations could yield different similarity scores and potentially different hedonic estimates. While we designed the references following principled criteria (semantic saturation, dimensional specificity, linguistic consistency), the approach lacks a formal optimality criterion. A systematic sensitivity protocol---averaging similarities over multiple paraphrases per dimension, varying reference length, and comparing French-only against bilingual formulations---would quantify the dependence of the estimates on any single phrasing. One could also envision deriving reference descriptions empirically, for example by clustering listing embeddings and using cluster centroids as data-driven references, though this sacrifices the ex ante interpretability that motivated our approach. The full text of all 20 references is reproduced in Appendix Table~\ref{tab:reference_texts} to make this dependence transparent and replicable.
60 +
61 +\textbf{Multicollinearity among similarity features.} The strong correlations among similarity dimensions (0.66--0.97; Figure~\ref{fig:corr_matrix}) introduce substantial multicollinearity: VIFs average 17.2 and exceed the conventional threshold of 10 for 16 of the 20 dimensions (Section~\ref{sec:robustness}). The consequences are visible in the regularized-selection exercise, where the Lasso retains only four block representatives rather than the sixteen individually significant variables. Individual coefficient magnitudes should therefore be interpreted with caution: the ``Luxury'' premium of 14.2\% includes variation that is shared with the Renovated and Bright \& Spacious dimensions, and the data cannot fully apportion credit within such correlated blocks. The block-level conclusions---that semantic content jointly adds 6 percentage points of explanatory power with theoretically sensible signs---are unaffected. Ridge regression or principal component regression on the similarity features would provide bounds on the total effect of correlated quality dimensions, and reducing the number of references (or orthogonalizing them ex ante) is a natural design lever for future applications.
62 +
63 +\textbf{Temporal stability.} Our analysis captures a single cross-section of listings. The relationship between textual content and prices may vary over time as market conditions change, buyer preferences evolve, and listing language conventions shift. During a seller's market, ``motivated seller'' language may carry a larger discount because it is rarer and more diagnostic; during a buyer's market, such language may be more common and less informative. Longitudinal analysis would be needed to assess the stability of semantic implicit prices across market cycles.
added paper/sections/introduction.tex +23 −0
@@ -0,0 +1,23 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Introduction}
5 +%======================================================================
6 +
7 +Hedonic pricing models have been the dominant empirical framework for estimating the implicit prices of housing characteristics since the seminal contributions of \citet{rosen1974hedonic} and \citet{lancaster1966new}. The standard approach decomposes observed transaction prices into the marginal contributions of structural attributes (bedrooms, bathrooms, lot size), locational factors (neighborhood quality, proximity to amenities), and environmental characteristics \citep{sirmans2005composition}. Decades of empirical work have refined these models, establishing which structural and spatial variables explain the largest shares of price variation \citep{sirmans2006empirical, malpezzi2003hedonic}.
8 +
9 +Yet a persistent limitation of hedonic models is that they can only price attributes that the econometrician observes and measures. Property listing descriptions---the free-text narratives written by real estate agents to market properties---contain qualitative information that structured data fields fail to capture. These descriptions communicate the quality of finishes, the ambiance of a neighborhood, the motivation of the seller, the architectural character of a home, and the lifestyle a property affords. A listing that describes ``spa-like master bathroom with heated marble floors'' conveys quality information fundamentally different from one that states ``bathroom needs updating.'' Both properties may have the same number of bathrooms, yet the hedonic price contribution differs enormously. As \citet{pakes2003reconsideration} argued, standard hedonic regressions yield biased implicit price estimates when important characteristics are unobservable to the econometrician but observable to market participants---precisely the situation that listing text addresses.
10 +
11 +Despite the richness of this textual information, the hedonic pricing literature has been slow to incorporate unstructured text. Previous text-based approaches face a fundamental tradeoff. Keyword and bag-of-words methods \citep{nowak2017quality} are interpretable but semantically shallow: they cannot recognize that ``quartz countertops'' and ``premium stone surfaces'' convey similar quality information. Sentiment analysis \citep{demers2018textual} reduces multidimensional quality descriptions to a single polarity score. Topic models \citep{hong2020text} capture latent themes but produce factors that are difficult to interpret and unstable across samples. Most recently, transformer-based embeddings \citep{lam2022textual, devlin2019bert} offer rich semantic representations, but the resulting 768-dimensional vectors are opaque---effective for prediction but unsuitable for economic inference where the goal is to estimate the implicit prices of identifiable housing characteristics.
12 +
13 +This paper proposes a \textit{concept-projection} approach to text-augmented hedonic pricing. Rather than using transformer embeddings as black-box predictors, we project them onto economically interpretable semantic anchors. The framework proceeds in four stages. First, we encode each property listing's \textit{PublicRemarks} field into a dense 384-dimensional vector using the all-MiniLM-L6-v2 sentence transformer \citep{wang2020minilm}. Second, we define 20 reference descriptions, each carefully designed to embody a specific qualitative housing dimension---luxury finishes, renovation status, architectural style, waterfront access, seller motivation, and others. Third, we compute the cosine similarity between each listing's embedding and each reference embedding, yielding 20 interpretable scalar features per property. Fourth, we integrate these similarity features into a semi-logarithmic hedonic pricing model estimated by OLS with heteroskedasticity-robust standard errors. The objective is not to claim that changing listing words mechanically changes prices, but to measure whether semantic content embedded in listing narratives captures price-relevant information omitted from structured hedonic variables.
14 +
15 +Quebec provides a particularly interesting empirical context for this analysis. The province's real estate market exhibits substantial heterogeneity---from luxury urban properties in Montr\'eal and Qu\'ebec City to affordable family homes in suburban developments and waterfront properties in the Laurentians and Eastern Townships. Listing descriptions are predominantly in French with occasional English phrases, creating a bilingual corpus that tests the cross-lingual capabilities of the embedding model. The market spans urban, suburban, and peripheral segments with distinct price levels and qualitative characteristics, offering rich variation for semantic analysis.
16 +
17 +Our contributions are threefold. First, we demonstrate that semantic features derived from listing text capture economically significant information beyond what structural variables measure. Adding the 20 cosine similarity variables to a standard hedonic specification increases the adjusted $R^2$ from 0.452 to 0.511 for 17,087 single-family homes ($F = 99.53$, $p < 0.001$). The semantic features contribute 9.3\% of the full model's explained variance---comparable to the gains from adding neighborhood controls in typical hedonic studies \citep{sirmans2006empirical}.
18 +
19 +Second, we estimate the implicit price gradients associated with qualitative housing dimensions that have been largely invisible to the hedonic literature. Listings semantically closer to modern/contemporary references are associated with 16.4\% higher prices per standard deviation; luxury-oriented language with 14.2\% premiums; and land/nature descriptions with 13.0\% premiums. Conversely, renovation-need language is associated with 7.9\% discounts and seller-urgency language with 8.5\% discounts. Quantile regressions reveal that these associations are heterogeneous across the price distribution: luxury premiums are amplified at upper quantiles, consistent with quality complementarities.
20 +
21 +Third, the reference-based approach offers a methodological contribution that resolves the depth-interpretability tension in text-augmented hedonic models. Unlike PCA-based or neural-network-based approaches that yield opaque features, each of our similarity measures corresponds to a named, researcher-defined qualitative dimension with a clear coefficient estimate. This makes the results directly useful for real estate appraisal, market analysis, and automated valuation models (AVMs), and the methodology is portable to any domain where free-text descriptions accompany structured economic data.
22 +
23 +The remainder of the paper is organized as follows. Section~\ref{sec:literature} reviews the relevant literature. Section~\ref{sec:methodology} describes the methodological framework, including the identification strategy. Section~\ref{sec:data} presents the data and descriptive statistics. Section~\ref{sec:results} reports the empirical results. Section~\ref{sec:robustness} presents a comprehensive robustness analysis. Section~\ref{sec:discussion} discusses interpretation, implications, and limitations. Section~\ref{sec:conclusion} concludes with directions for future research.
added paper/sections/literature.tex +79 −0
@@ -0,0 +1,79 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Literature Review} \label{sec:literature}
5 +%======================================================================
6 +
7 +\subsection{Hedonic Pricing Theory and Housing Markets}
8 +
9 +The intellectual foundations of hedonic pricing rest on the work of \citet{lancaster1966new}, who reconceptualized consumer theory around the characteristics of goods rather than goods themselves, and \citet{rosen1974hedonic}, who formalized the framework for differentiated product markets. In Rosen's model, the observed market price of a differentiated good reflects the equilibrium of supply and demand for its constituent characteristics, allowing the researcher to recover the implicit marginal prices of individual attributes through regression analysis. \citet{palmquist1984estimating} extended the theoretical framework by showing how second-stage supply and demand identification depends on exogenous variation in household characteristics and builder cost structures, while \citet{epple1987hedonic} demonstrated that under certain conditions the hedonic price function is unique and recoverable.
10 +
11 +In the housing context, the hedonic approach yields the canonical specification:
12 +\begin{equation}
13 +\ln(P_i) = \alpha + \bfbeta' \bfX_i + \eps_i \label{eq:hedonic}
14 +\end{equation}
15 +where $P_i$ is the price of property $i$, $\bfX_i$ is a vector of structural, locational, and neighborhood attributes, and $\bfbeta$ captures the implicit marginal prices. The semi-logarithmic functional form, recommended by \citet{halvorsen1981choice} and widely adopted in the literature, allows coefficients to be interpreted as approximate percentage changes in price for a unit change in the characteristic. \citet{cropper1988choice} evaluated alternative functional forms through Monte Carlo experiments, finding that simpler parametric specifications (linear and log-linear) outperform flexible Box-Cox transformations when variables are measured with error or important attributes are omitted.
16 +
17 +The empirical literature on hedonic housing models is vast. \citet{sirmans2005composition} surveyed the composition of hedonic models and identified bedrooms, bathrooms, lot size, age, square footage, and proximity to central business districts as the most frequently included variables. \citet{sirmans2006empirical} conducted a meta-analysis of 64 hedonic studies, finding that bathrooms, square footage, and lot size consistently exhibit the largest marginal effects. \citet{malpezzi2003hedonic} provided a comprehensive review of methodological issues, including functional form selection, spatial autocorrelation, and multicollinearity. In the Canadian context, \citet{cheshire2004capitalisation} demonstrated that hedonic models capture neighborhood-level amenity capitalization effectively, while \citet{gibbons2014costs} showed that environmental disamenities are reliably reflected in hedonic estimates when spatial controls are carefully specified.
18 +
19 +A persistent challenge in hedonic pricing is the omitted variable bias arising from unobserved quality attributes. \citet{pakes2003reconsideration} argued that standard hedonic regressions yield biased implicit price estimates when important product characteristics are unobservable to the econometrician but observable to market participants. This concern is directly relevant to our study: listing descriptions may convey quality information---about finish materials, maintenance history, or neighborhood ambiance---that is not captured by conventional structural variables but is fully observable to buyers.
20 +
21 +More recently, machine learning methods have been applied to hedonic pricing, including random forests \citep{hong2020text}, gradient boosting \citep{mullainathan2017machine}, and neural networks \citep{yoo2012variable}. \citet{kok2017big} showed that big data approaches incorporating non-traditional variables can substantially improve mass appraisal accuracy, while \citet{pace1998appraisal} demonstrated early gains from spatial autoregressive extensions. While these methods typically achieve superior predictive accuracy, they sacrifice the coefficient interpretability that is central to the hedonic framework's appeal for policy analysis and market valuation. Our approach seeks to recover this interpretability while incorporating the rich informational content of textual data.
22 +
23 +\subsection{Text Analytics in Real Estate}
24 +
25 +The incorporation of textual data into real estate analysis represents a growing but still nascent literature. Early work focused on simple lexical features. \citet{nowak2017quality} were among the first to demonstrate that listing descriptions contain price-relevant information, using bag-of-words representations and finding that specific keywords (e.g., ``granite,'' ``stainless,'' ``maple'') are associated with significant price premiums or discounts. Their approach, while pioneering, treats words as independent tokens and cannot capture semantic relationships or phrase-level meaning. In a related study, \citet{goodwin2020feature} showed that even basic keyword counts can reduce prediction error by 2--5\% when appended to traditional hedonic specifications, confirming the informational content of listing text.
26 +
27 +\citet{shen2020text} extended the keyword approach using TF-IDF (term frequency--inverse document frequency) features and demonstrated that text-based models outperform traditional hedonic specifications in out-of-sample prediction. However, TF-IDF, like bag-of-words, operates at the lexical level and cannot recognize that ``quartz countertops'' and ``premium stone surfaces'' convey similar quality information. \citet{bayer2016racial} documented that listing language also encodes neighborhood characteristics---including demographic composition---that are not explicitly stated, raising important questions about what information text features actually capture in hedonic regressions.
28 +
29 +\citet{demers2018textual} took a different approach by analyzing the sentiment of property descriptions, finding that more positive language is associated with higher prices. However, sentiment analysis reduces rich textual content to a single polarity score, losing the multidimensional quality information embedded in listing text. Moreover, the direction of causality remains ambiguous: agents may write more enthusiastic descriptions for objectively better properties, or persuasive writing may inflate perceived value. \citet{huang2022house} provided further evidence on this question, showing that sentiment polarity and subjectivity scores independently predict sale speed and price-to-listing ratios, suggesting that textual tone captures genuine market signals beyond property quality.
30 +
31 +\citet{hong2020text} applied Latent Dirichlet Allocation (LDA) topic modeling to listing descriptions and identified latent thematic clusters that contribute to price variation. While topic models capture higher-level semantic structure than keyword approaches, the resulting topics are often difficult to interpret and may conflate distinct qualitative dimensions into single latent factors. \citet{blei2003latent} originally proposed LDA for document classification; its application to property listings faces the additional challenge that real estate descriptions are often short and formulaic, limiting topic diversity.
32 +
33 +\citet{lam2022textual} represents the state of the art, using BERT-based embeddings \citep{devlin2019bert} to represent listing text and incorporating these representations into a gradient-boosted regression model. They achieved significant improvements in predictive accuracy over both traditional hedonic models and earlier text-based approaches. However, BERT embeddings are 768-dimensional and inherently opaque---the resulting features have no natural interpretation as housing characteristics, limiting their usefulness for hedonic analysis where inference on implicit prices is the primary goal. Similarly, \citet{li2023deep} applied deep learning to Chinese property listings and achieved high predictive accuracy, but the resulting models function as black boxes unsuitable for regulatory or appraisal applications where coefficient transparency is required.
34 +
35 +The tension between prediction and interpretation in text-augmented hedonic models mirrors a broader debate in applied econometrics. \citet{athey2019machine} provided a framework for thinking about when machine learning methods complement rather than replace traditional econometric approaches---specifically, they are most useful for constructing features (as in our reference-based approach) rather than for final-stage inference. Our work builds on these contributions while addressing the interpretability limitation that pervades existing NLP-based approaches in real estate economics.
36 +
37 +\subsection{Sentence Embeddings and Semantic Textual Similarity}
38 +
39 +Sentence embeddings map variable-length text into fixed-dimensional vector spaces where semantic similarity corresponds to geometric proximity. The development of contextual word representations through architectures such as ELMo \citep{peters2018deep} and subsequently BERT \citep{devlin2019bert} represented a paradigm shift from static word vectors \citep{mikolov2013distributed} to context-dependent representations. However, these models require cross-encoding for pairwise comparison, which scales quadratically with corpus size.
40 +
41 +The Sentence-BERT framework \citep{reimers2019sentence} addressed this limitation by fine-tuning pre-trained transformer models using siamese and triplet network structures to produce embeddings optimized for cosine similarity comparisons. This approach enables efficient comparison of text pairs without requiring cross-encoding, which is computationally prohibitive for large-scale applications. Subsequent work by \citet{gao2021simcse} proposed contrastive learning objectives (SimCSE) that further improved embedding quality, while \citet{li2020sentence} demonstrated that sentence embeddings suffer from anisotropy---a geometric degeneration where embeddings occupy a narrow cone in the vector space---and proposed whitening transformations as a remedy.
42 +
43 +The all-MiniLM-L6-v2 model \citep{wang2020minilm} is a distilled version of the MiniLM architecture that produces 384-dimensional embeddings. The model employs self-attention distillation, transferring knowledge from a larger teacher model to a compact student network. Despite its compact size (22.7M parameters), it achieves competitive performance on semantic textual similarity benchmarks (average Spearman correlation of 0.82 on STS-B), while being approximately 5$\times$ faster than BERT-base. Its efficiency makes it suitable for encoding large corpora of property listings. For French text, the model's multilingual coverage---derived from its training on paraphrase data spanning multiple languages---provides adequate performance, though dedicated French models such as CamemBERT \citep{martin2020camembert} or FlauBERT \citep{le2020flaubert} could potentially improve semantic resolution for domain-specific vocabulary.
44 +
45 +Cosine similarity between two $L_2$-normalized embedding vectors $\mathbf{u}$ and $\mathbf{v}$ is defined as:
46 +\begin{equation}
47 +\text{sim}(\mathbf{u}, \mathbf{v}) = \frac{\mathbf{u} \cdot \mathbf{v}}{||\mathbf{u}|| \cdot ||\mathbf{v}||} = \mathbf{u} \cdot \mathbf{v} \label{eq:cosine}
48 +\end{equation}
49 +This measure ranges from $-1$ to $1$, with higher values indicating greater semantic similarity. For normalized vectors, cosine similarity reduces to the dot product, enabling efficient computation via matrix multiplication. The geometric interpretation is straightforward: cosine similarity measures the angle between two vectors in the embedding space, with a value of 1 indicating identical direction (perfect semantic alignment) and 0 indicating orthogonality (semantic independence). Negative values, while theoretically possible, are rare in practice for sentence embeddings due to the positivity bias inherent in transformer representations \citep{ethayarajh2019contextual}.
50 +
51 +\subsection{Research Gap and Contribution}
52 +
53 +The existing literature on text analytics in real estate presents a tension between semantic depth and economic interpretability. Keyword and TF-IDF methods are interpretable but semantically shallow; embedding-based methods are semantically rich but economically opaque. Table~\ref{tab:literature_comparison} summarizes how our approach compares with existing methods along five dimensions.
54 +
55 +\begin{table}[!htbp]
56 +\centering
57 +\caption{Comparison of text-based approaches in hedonic pricing.}
58 +\label{tab:literature_comparison}
59 +\small
60 +\begin{adjustbox}{max width=\textwidth}
61 +\begin{tabular}{lC{1.8cm}C{1.8cm}C{1.8cm}C{1.8cm}C{1.8cm}}
62 +\toprule
63 +\textbf{Method} & \textbf{Semantic Depth} & \textbf{Interpret-ability} & \textbf{Scalability} & \textbf{Multi-lingual} & \textbf{Key Reference} \\
64 +\midrule
65 +Keyword counts & Low & High & High & No & \citet{nowak2017quality} \\
66 +TF-IDF & Low & Moderate & High & No & \citet{shen2020text} \\
67 +Sentiment & Low & High & High & Yes & \citet{demers2018textual} \\
68 +LDA topics & Moderate & Low & Moderate & No & \citet{hong2020text} \\
69 +BERT embeddings & High & None & Low & Limited & \citet{lam2022textual} \\
70 +PCA on embeddings & High & None & High & Yes & --- \\
71 +\textbf{Reference cosine} & \textbf{High} & \textbf{High} & \textbf{High} & \textbf{Yes} & \textbf{This paper} \\
72 +\bottomrule
73 +\end{tabular}
74 +\end{adjustbox}
75 +\end{table}
76 +
77 +Our reference-based cosine similarity approach resolves the depth-interpretability tension by leveraging the deep semantic representations of transformer embeddings while producing features with clear economic interpretation. Each similarity score measures the degree to which a property's description resembles a researcher-defined qualitative archetype, yielding features that are both semantically meaningful and directly usable as right-hand-side variables in a hedonic regression.
78 +
79 +This approach contributes to a broader methodological trend in applied economics where machine learning tools are used for feature construction rather than final-stage estimation \citep{athey2019machine, mullainathan2017machine}. By treating embeddings as an intermediate representation and projecting them onto economically meaningful axes, we preserve the inferential advantages of OLS while exploiting the representational power of deep learning. The reference-based projection is also related to the concept of ``concept bottleneck models'' in interpretable machine learning \citep{koh2020concept}, where high-dimensional representations are channeled through human-interpretable intermediate concepts before reaching the prediction stage.
added paper/sections/methodology.tex +102 −0
@@ -0,0 +1,102 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Methodology} \label{sec:methodology}
5 +%======================================================================
6 +
7 +\subsection{Methodological Overview}
8 +
9 +Our framework transforms unstructured listing text into a set of interpretable features through a four-stage pipeline, illustrated in Figure~\ref{fig:methodology}: (1)~extraction and preprocessing of listing descriptions; (2)~encoding into dense vector representations using a pre-trained sentence transformer; (3)~computation of cosine similarities against a set of researcher-designed reference descriptions; and (4)~integration of the resulting similarity features into a hedonic pricing model.
10 +
11 +\begin{figure}[!htbp]
12 +\centering
13 +\includegraphics[width=\textwidth]{fig7_methodology.pdf}
14 +\caption{Methodological framework: from listing text to hedonic price estimates.}
15 +\label{fig:methodology}
16 +\begin{minipage}{0.9\textwidth}
17 +\footnotesize
18 +Notes: This figure illustrates the four-stage pipeline: (1) text extraction, (2) sentence embedding, (3) cosine similarity computation against 20 reference descriptions, and (4) integration into a hedonic pricing model.
19 +\end{minipage}
20 +\end{figure}
21 +
22 +\subsection{Text Extraction and Preprocessing}
23 +
24 +Each property listing in our database includes a \textit{PublicRemarks} field containing the agent-written description. We extract this field and apply minimal preprocessing: listings with descriptions shorter than 20 characters are excluded as uninformative, and listings with missing or non-positive prices are removed. We deliberately avoid aggressive text preprocessing (stemming, lemmatization, stop-word removal) because the sentence transformer model is designed to process natural-language text and internally handles tokenization.
25 +
26 +In addition to the embedding-based features, we compute a simple text length measure (character count of the description) as a control variable. Description length may proxy for listing effort, property complexity, or market conditions, and including it ensures that our semantic features capture content rather than verbosity.
27 +
28 +\subsection{Sentence Embedding}
29 +
30 +Each listing's description $d_i$ is encoded into a 384-dimensional unit vector using the all-MiniLM-L6-v2 sentence transformer:
31 +\begin{equation}
32 +\bfe_i = f_{\bftheta}(d_i) \in \R^{384}, \quad ||\bfe_i|| = 1
33 +\end{equation}
34 +The model processes text through 6 transformer layers with 384 hidden dimensions, applying mean pooling over token embeddings and $L_2$ normalization to produce the final sentence embedding. The model handles both English and French text, which is important for our Quebec-based corpus where listings are predominantly in French with occasional English phrases.
35 +
36 +Embeddings are computed in batches of 256 for computational efficiency. The entire corpus of 17,087 descriptions is encoded in approximately 14 seconds on consumer hardware (Apple M-series CPU), making the approach practical for large-scale applications.
37 +
38 +\subsection{Reference Description Design}
39 +
40 +\subsubsection{Rationale}
41 +
42 +The key methodological innovation is the construction of reference descriptions that serve as semantic anchors. Rather than applying unsupervised dimensionality reduction (PCA, autoencoders) to the 384-dimensional embedding space---which would yield features without natural economic interpretation---we project each listing's embedding onto a set of researcher-defined semantic axes. Each axis is defined by a reference description that embodies a specific qualitative housing dimension.
43 +
44 +This approach is analogous to the construction of factor-mimicking portfolios in asset pricing \citep{fama1993common}: just as Fama-French factors are portfolios designed to load on specific risk dimensions, our reference descriptions are synthetic texts designed to load on specific quality dimensions.
45 +
46 +\subsubsection{Reference Categories}
47 +
48 +We define 20 reference descriptions spanning six broad domains of housing quality, listed in Table~\ref{tab:references}. Each reference is a synthetic paragraph of 150--260 characters, written in French to match the listing corpus, containing the vocabulary, phrasing, and semantic content characteristic of its dimension.
49 +
50 +\input{tables/tab_references}
51 +
52 +\subsubsection{Design Principles}
53 +
54 +The reference descriptions were designed following three principles. First, \textit{semantic saturation}: each reference includes multiple synonyms, related phrases, and characteristic vocabulary to ensure broad coverage of its target dimension. Second, \textit{dimensional specificity}: each reference targets a single qualitative dimension to minimize cross-loading. Third, \textit{linguistic consistency}: all references are written in French using vocabulary typical of Quebec real estate listings, ensuring alignment with the embedding space of the listing corpus.
55 +
56 +\subsection{Cosine Similarity Computation}
57 +
58 +For each property $i$ and reference $j$, we compute the cosine similarity:
59 +\begin{equation}
60 +s_{ij} = \text{sim}(\bfe_i, \mathbf{r}_j) = \bfe_i \cdot \mathbf{r}_j \label{eq:sim}
61 +\end{equation}
62 +where $\mathbf{r}_j$ is the $L_2$-normalized embedding of the $j$-th reference. The result is a similarity matrix $\mathbf{S} \in \R^{n \times 20}$.
63 +
64 +Each $s_{ij} \in [-1, 1]$ measures the degree to which property $i$'s description semantically resembles reference $j$. A higher score indicates that the listing text uses language closer in meaning to the reference description, capturing not just keyword overlap but deeper semantic alignment.
65 +
66 +The full similarity matrix is computed as a single matrix multiplication $\mathbf{S} = \mathbf{E} \cdot \mathbf{R}'$, where $\mathbf{E} \in \R^{n \times 384}$ is the matrix of listing embeddings and $\mathbf{R} \in \R^{20 \times 384}$ is the matrix of reference embeddings. This operation is computationally trivial.
67 +
68 +\subsection{Hedonic Model Specification}
69 +
70 +We estimate a series of nested OLS models with White-Huber heteroskedasticity-robust standard errors (HC3 variant):
71 +\begin{align}
72 +\text{Model A:} \quad \ln(P_i) &= \alpha + \bfbeta' \bfX_i + \eps_i \label{eq:modelA} \\
73 +\text{Model B:} \quad \ln(P_i) &= \alpha + \bfbeta' \bfX_i + \delta \cdot \ell_i + \eps_i \label{eq:modelB} \\
74 +\text{Model C:} \quad \ln(P_i) &= \alpha + \bfbeta' \bfX_i + \bfgamma' \bfs_i + \eps_i \label{eq:modelC} \\
75 +\text{Model D:} \quad \ln(P_i) &= \alpha + \bfbeta' \bfX_i + \delta \cdot \ell_i + \bfgamma' \bfs_i + \eps_i \label{eq:modelD} \\
76 +\text{Model E:} \quad \ln(P_i) &= \alpha + \bfbeta' \bfX_i + \delta \cdot \ell_i + \bfgamma_{\text{sig}}' \bfs_i^{\text{sig}} + \eps_i \label{eq:modelE}
77 +\end{align}
78 +
79 +\noindent where $\bfX_i$ includes six structural variables (bedrooms, bathrooms, half-bathrooms, parking spaces, stories, and lot size), $\ell_i$ is the description length (character count), $\bfs_i$ is the full vector of 20 cosine similarities, and $\bfs_i^{\text{sig}}$ retains only the similarities that are individually significant at the 5\% level in Model~D. All independent variables are standardized (zero mean, unit variance) prior to estimation, so coefficients represent the effect of a one-standard-deviation change.
80 +
81 +The choice of HC3 standard errors, recommended by \citet{mackinnon1985some} for samples of moderate size, provides consistent inference under heteroskedasticity without requiring specification of the error variance structure.
82 +
83 +The incremental contribution of semantic features is tested using an \fstat:
84 +\begin{equation}
85 +F = \frac{(\text{SSR}_A - \text{SSR}_D) / q}{\text{SSR}_D / (n - k - 1)} \label{eq:ftest}
86 +\end{equation}
87 +where $q$ is the number of additional semantic variables and $k$ is the total number of regressors in Model~D.
88 +
89 +\subsection{Identification and Interpretation} \label{sec:identification}
90 +
91 +The coefficients on the semantic similarity variables should be interpreted as \textit{conditional associations}---implicit semantic price gradients---rather than causal effects. Three distinct channels could generate the observed correlations between listing language and prices, and the cross-sectional design cannot distinguish among them.
92 +
93 +\textbf{Information channel.} Agents accurately describe observable property attributes that affect prices but are not captured by the structural variables in our model. Under this interpretation, the semantic coefficients recover the implicit prices of genuine quality dimensions: luxury finishes, renovation status, natural amenities, and so forth. The listing text serves as a proxy for unobserved quality, and the coefficients have a straightforward hedonic interpretation.
94 +
95 +\textbf{Persuasion channel.} Listing language influences buyer perceptions and willingness to pay, independent of underlying property quality. Skillfully written descriptions could inflate perceived value, creating a price premium attributable to marketing rather than to the property itself. Under this interpretation, the semantic coefficients capture the return to agent effort and linguistic skill rather than housing quality.
96 +
97 +\textbf{Omitted-location channel.} Listing language proxies for locational characteristics that are correlated with both the property's description and its price. For example, the negative coefficient on Family-Friendly language likely reflects the systematic association between family-oriented marketing and lower-priced suburban markets, rather than a negative valuation of schools and parks. Similarly, New Construction language may capture peripheral location rather than an intrinsic quality discount.
98 +
99 +In practice, the observed coefficients likely reflect a mixture of all three channels. The information channel is most plausible for dimensions with clear quality content (Luxury, Needs Renovation, Waterfront), while the omitted-location channel is most relevant for dimensions with geographic content (Family-Friendly, New Construction, Quiet \& Peaceful). The absence of explicit spatial controls in our baseline specification (see Section~\ref{sec:discussion}) means that some semantic coefficients absorb locational price variation.
100 +
101 +\noindent\textit{Scope of inference.} We interpret our estimates as measuring whether and how much semantic content in listing text is associated with price variation, conditional on observed structural characteristics. This is informative for automated valuation, market segmentation, and understanding the informational content of agent narratives, even without causal identification. We avoid language implying that semantic similarity ``causes'' or ``determines'' prices throughout the paper.
102 +
added paper/sections/results.tex +169 −0
@@ -0,0 +1,169 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Results} \label{sec:results}
5 +%======================================================================
6 +
7 +\subsection{Model Comparison}
8 +
9 +Table~\ref{tab:model_comparison} reports goodness-of-fit statistics for the five model specifications defined in Equations~(\ref{eq:modelA})--(\ref{eq:modelE}). Figure~\ref{fig:model_comparison} visualizes the comparison.
10 +
11 +\input{tables/tab_model_comparison}
12 +
13 +\begin{figure}[!htbp]
14 +\centering
15 +\includegraphics[width=0.8\textwidth]{fig1_model_comparison.pdf}
16 +\caption{Goodness-of-fit comparison across hedonic model specifications.}
17 +\label{fig:model_comparison}
18 +\begin{minipage}{0.9\textwidth}
19 +\footnotesize
20 +Notes: This figure compares $R^2$ and adjusted $R^2$ across the five nested model specifications defined in Equations~(\ref{eq:modelA})--(\ref{eq:modelE}). The dashed line marks the structural baseline (Model A).
21 +\end{minipage}
22 +\end{figure}
23 +
24 +Several results merit discussion. The structural model (A) explains 45.2\% of log-price variation, consistent with the typical range reported in the hedonic literature for single-family homes \citep{sirmans2005composition}. Adding description length alone (Model~B) improves the $R^2$ by 1.2 percentage points, confirming that the quantity of listing text is price-informative---an indirect quality signal.
25 +
26 +Adding the 20 semantic similarity measures without controlling for length (Model~C) yields a substantially larger improvement of 4.4 percentage points. This demonstrates that the \textit{content} of the text matters beyond its mere length. The full model (D), which includes both text length and all semantic features, achieves an $R^2$ of 0.512---an improvement of 6.0 percentage points over the structural baseline.
27 +
28 +The parsimonious model (E), retaining only the 16 individually significant similarity measures, achieves nearly identical fit ($R^2 = 0.512$, BIC = 24,425 vs.\ 24,448 for Model~D). The lower BIC of Model~E suggests it should be preferred on the basis of model parsimony. Four reference dimensions---Renovated, Income/Investment, Heritage/Character, and Pool \& Landscaping---do not contribute independently after controlling for the other variables. The complete coefficient estimates for Model~E are reported in Appendix Table~\ref{tab:parsimonious}.
29 +
30 +Figure~\ref{fig:r2_decomp} decomposes the explained variance in the full model. Structural variables account for 88.3\% of the total $R^2$, text length contributes 2.4\%, and the semantic similarity measures add 9.3\%.
31 +
32 +\begin{figure}[!htbp]
33 +\centering
34 +\includegraphics[width=0.55\textwidth]{fig8_r2_decomposition.pdf}
35 +\caption{Decomposition of explained variance ($R^2$) in the full model (D).}
36 +\label{fig:r2_decomp}
37 +\begin{minipage}{0.9\textwidth}
38 +\footnotesize
39 +Notes: Structural variables account for 88.3\% of the total explained variance, description length contributes 2.4\%, and the 20 semantic similarity measures add 9.3\%.
40 +\end{minipage}
41 +\end{figure}
42 +
43 +\subsection{Joint Significance Test}
44 +
45 +The \fstat{} for the joint significance of all 21 text-based variables (length + 20 similarities) comparing Model~D to Model~A yields:
46 +\begin{equation*}
47 +F = 99.53, \quad p < 0.001
48 +\end{equation*}
49 +This decisively rejects the null hypothesis that listing text contains no price-relevant information beyond structural characteristics. The test statistic is very large, reflecting the substantial improvement in model fit.
50 +
51 +\subsection{Structural Variable Estimates}
52 +
53 +Figure~\ref{fig:structural_coef} presents the coefficient estimates for structural variables in the full model.
54 +
55 +\begin{figure}[!htbp]
56 +\centering
57 +\includegraphics[width=0.8\textwidth]{fig10_structural_coefficients.pdf}
58 +\caption{Structural variable coefficients in the full model (D), with 95\% CI.}
59 +\label{fig:structural_coef}
60 +\begin{minipage}{0.9\textwidth}
61 +\footnotesize
62 +Notes: Coefficients represent the percentage change in price associated with a one-standard-deviation change in each structural variable. Error bars show 95\% confidence intervals based on HC3 robust standard errors.
63 +\end{minipage}
64 +\end{figure}
65 +
66 +Bathrooms have the largest structural effect (+34.5\% per standard deviation), followed by half-bathrooms (+17.1\%), description length (+10.7\%), parking (+9.6\%), and stories (+7.7\%). Bedrooms have a modest positive effect (+3.2\%), and lot size is statistically insignificant---consistent with the noisy, unit-inconsistent measurement of this field in the raw data (Section~\ref{sec:data}).
67 +
68 +\subsection{Semantic Dimension Estimates}
69 +
70 +Table~\ref{tab:full_results} reports the complete coefficient estimates from Model~D. Figure~\ref{fig:coefficient_plot} presents the semantic coefficients graphically.
71 +
72 +\input{tables/tab_full_results}
73 +
74 +\begin{figure}[!htbp]
75 +\centering
76 +\includegraphics[width=0.8\textwidth]{fig2_coefficient_plot.pdf}
77 +\caption{Hedonic price impact of semantic similarity dimensions (Model D). Green bars indicate positive effects; red bars indicate negative effects. Gray bars are not statistically significant at the 5\% level. Error bars show 95\% confidence intervals.}
78 +\label{fig:coefficient_plot}
79 +\end{figure}
80 +
81 +\subsubsection{Positive Price Associations}
82 +
83 +Three semantic dimensions exhibit large, positive, and highly significant associations with listing prices.
84 +
85 +\textbf{Modern/Contemporary} ($+16.4\%$, $p < 0.001$). Listings semantically closer to modern/contemporary reference descriptions---emphasizing contemporary architecture, minimalist design, smart home technology, and energy-efficient construction---are associated with the highest price premium. This is consistent with revealed preferences for modern aesthetics and functional design in the Quebec housing market.
86 +
87 +\textbf{Luxury} ($+14.2\%$, $p < 0.001$). Language describing premium finishes (quartz, granite, hardwood), gourmet kitchens, spa-like bathrooms, and home automation systems is associated with a substantial price premium. This dimension captures quality variation that is invisible to standard structural variables: two properties with the same number of bathrooms can differ enormously in finish quality, and the listing text registers this difference.
88 +
89 +\textbf{Land \& Nature} ($+13.0\%$, $p < 0.001$). Descriptions of wooded lots, professional landscaping, privacy, and access to natural surroundings are associated with a significant positive price gradient, consistent with the well-documented value of outdoor amenity space in residential markets.
90 +
91 +\textbf{Waterfront} ($+4.7\%$, $p < 0.001$) and \textbf{Panoramic View} ($+3.0\%$, $p = 0.01$) capture location-specific amenities---access to water and scenic views---that are not reflected in standard structural variables but are associated with higher prices.
92 +
93 +\subsubsection{Negative Price Associations}
94 +
95 +Several dimensions exhibit negative coefficients that, upon closer examination, reflect market segmentation rather than value destruction per se (see Section~\ref{sec:identification} for the identification discussion).
96 +
97 +\textbf{Family-Friendly} ($-11.6\%$, $p < 0.001$). This is the most strongly negative dimension. Descriptions emphasizing proximity to schools, parks, and family amenities characterize lower-priced suburban markets. After controlling for structural characteristics, this language likely proxies for suburban location rather than indicating that family-oriented features reduce value. This coefficient should be interpreted with the caveat that it may absorb locational price variation in the absence of spatial controls.
98 +
99 +\textbf{New Construction} ($-10.5\%$, $p < 0.001$). The negative coefficient is counterintuitive at first glance, since ``new'' is typically a premium attribute. However, in Quebec's real estate geography, new residential developments are concentrated in peripheral suburban areas (e.g., Mirabel, Mascouche, Lachute) where land costs are lower. The ``new construction'' semantic dimension likely captures this locational sorting rather than an intrinsic quality discount; municipality fixed effects, once available, would disambiguate the two (Section~\ref{sec:discussion}).
100 +
101 +\textbf{Motivated Seller} ($-8.5\%$, $p < 0.001$). This is consistent with the information asymmetry literature \citep{levitt2008information}: language signaling seller urgency (``must sell,'' ``price reduced,'' ``estate sale'') reveals a weak bargaining position and is associated with a price discount relative to comparable properties with non-urgent descriptions.
102 +
103 +\textbf{Needs Renovation} ($-7.9\%$, $p < 0.001$). Descriptions indicating that a property requires work are associated with lower prices, consistent with the cost of deferred maintenance being capitalized into the listing price.
104 +
105 +\subsection{Semantic Profiles Across Price Quintiles}
106 +
107 +Figure~\ref{fig:quintile_heatmap} visualizes the mean cosine similarity for each reference dimension across price quintiles, revealing how the semantic ``fingerprint'' of listing descriptions varies with price level.
108 +
109 +\begin{figure}[!htbp]
110 +\centering
111 +\includegraphics[width=0.85\textwidth]{fig4_quintile_heatmap.pdf}
112 +\caption{Mean cosine similarity by price quintile across 20 reference dimensions.}
113 +\label{fig:quintile_heatmap}
114 +\begin{minipage}{0.9\textwidth}
115 +\footnotesize
116 +Notes: Warmer colors indicate higher similarity. Each cell reports the mean cosine similarity between listings in the given price quintile and the corresponding reference description.
117 +\end{minipage}
118 +\end{figure}
119 +
120 +The gradient reveals that lower-priced properties score higher on nearly all dimensions due to longer, more detailed descriptions. The relative differences across dimensions are nevertheless informative: moving from the bottom to the top quintile, the steepest declines occur for Motivated Seller, Entry-Level, and Premium Location, while Land \& Nature, Modern/Contemporary, and Panoramic View decline the least---precisely the dimensions that carry positive premiums in the regression results.
121 +
122 +\subsection{Inter-Reference Correlations}
123 +
124 +Figure~\ref{fig:corr_matrix} presents the correlation matrix between the 20 similarity dimensions.
125 +
126 +\begin{figure}[!htbp]
127 +\centering
128 +\includegraphics[width=0.75\textwidth]{fig5_correlation_matrix.pdf}
129 +\caption{Pearson correlation matrix between semantic similarity dimensions (lower triangle).}
130 +\label{fig:corr_matrix}
131 +\begin{minipage}{0.9\textwidth}
132 +\footnotesize
133 +Notes: Correlations are computed across all $n = 17{,}087$ observations. The strong positive correlations largely reflect the common influence of description length and verbosity on similarity scores.
134 +\end{minipage}
135 +\end{figure}
136 +
137 +Inter-reference correlations are strong and uniformly positive, ranging from 0.66 to 0.97 with a mean of 0.84. This common variation is driven primarily by description length and verbosity: detailed listings score higher on multiple dimensions simultaneously. The strongest correlations occur between Entry-Level and Motivated Seller ($r = 0.97$)---both characteristic of value-oriented listings---and between Luxury and Renovated ($r = 0.95$), which share upgrade-related vocabulary. This pronounced collinearity motivates two design features of the analysis: the inclusion of description length as a control, which absorbs much of the shared variation, and the multicollinearity diagnostics reported in Section~\ref{sec:robustness}, which quantify its consequences for inference. It also implies that individual semantic coefficients are best read as partial associations within a correlated block of quality signals rather than as isolated effects.
138 +
139 +\subsection{Illustrative Relationships}
140 +
141 +Figure~\ref{fig:scatter} presents scatter plots for four selected dimensions, illustrating the heterogeneity in relationships between semantic similarity and price.
142 +
143 +\begin{figure}[!htbp]
144 +\centering
145 +\includegraphics[width=0.8\textwidth]{fig6_scatter_plots.pdf}
146 +\caption{Bivariate relationships between selected semantic similarities and log-price.}
147 +\label{fig:scatter}
148 +\begin{minipage}{0.9\textwidth}
149 +\footnotesize
150 +Notes: Lines show ordinary least squares linear fit. These bivariate relationships do not control for structural variables or description length.
151 +\end{minipage}
152 +\end{figure}
153 +
154 +\subsection{Residual Diagnostics}
155 +
156 +Figure~\ref{fig:diagnostics} presents residual diagnostics for the full model, including a residual-versus-fitted plot and a normal Q-Q plot.
157 +
158 +\begin{figure}[!htbp]
159 +\centering
160 +\includegraphics[width=0.85\textwidth]{fig11_residual_diagnostics.pdf}
161 +\caption{Residual diagnostics for Model D: (a) residuals vs.\ fitted values; (b) normal Q-Q plot.}
162 +\label{fig:diagnostics}
163 +\begin{minipage}{0.9\textwidth}
164 +\footnotesize
165 +Notes: Panel (a) plots OLS residuals against fitted values, revealing mild heteroskedasticity at the tails. Panel (b) compares the empirical residual distribution to the standard normal, showing heavier-than-normal tails.
166 +\end{minipage}
167 +\end{figure}
168 +
169 +The residual plot (panel a) reveals mild heteroskedasticity, with residual variance increasing at the extremes of the fitted value distribution. This motivates our use of HC3 robust standard errors. The Q-Q plot (panel b) shows approximate normality in the central distribution with heavier-than-normal tails, particularly in the upper tail---consistent with the well-known right-skew of housing prices even after log transformation. The OLS coefficient estimates remain consistent under these conditions, and the HC3 standard errors provide valid inference.
added paper/sections/robustness.tex +57 −0
@@ -0,0 +1,57 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +%======================================================================
4 +\section{Robustness Analysis} \label{sec:robustness}
5 +%======================================================================
6 +
7 +To assess the sensitivity of our findings to modeling assumptions, sample composition, and estimation techniques, we conduct a battery of robustness checks. These analyses address five potential concerns: heteroskedasticity, multicollinearity among semantic features, sensitivity to outliers, distributional heterogeneity across price quantiles, and variable selection stability. All statistics reported in this section are produced by the replication pipeline accompanying the paper.
8 +
9 +\subsection{Heteroskedasticity Diagnostics}
10 +
11 +The Breusch-Pagan test \citep{breusch1979simple} applied to Model~D strongly rejects the null hypothesis of homoskedasticity (LM $= 768.0$, $p < 0.001$). This result formally justifies our use of HC3 heteroskedasticity-consistent standard errors throughout. The pattern of heteroskedasticity---residual variance increasing at both tails of the fitted value distribution (Figure~\ref{fig:diagnostics}, panel a)---is typical of housing price models and reflects the greater price dispersion in both the lowest-quality and highest-quality segments of the market \citep{goodman1998housing}.
12 +
13 +\subsection{Variance Inflation Factors}
14 +
15 +The strong correlations among the 20 cosine similarity measures (Figure~\ref{fig:corr_matrix}) translate into substantial multicollinearity by conventional standards: variance inflation factors (VIFs) computed for Model~D average 17.2 across the similarity variables, and 16 of the 20 exceed the conventional threshold of 10 \citep{obrien2007caution}, with maxima for Renovated (41.6) and Entry-Level (40.7). This is an expected consequence of the design: all 20 features are projections of the same embedding onto related quality concepts, and they share the common influence of description verbosity.
16 +
17 +Three considerations indicate that this collinearity inflates standard errors without invalidating the analysis. First, as \citet{obrien2007caution} emphasizes, a high VIF is not by itself grounds for respecification: it signals imprecision, not bias, and the relevant question is whether the affected coefficients remain informative. Despite VIF-inflated standard errors, 16 of the 20 similarity measures are individually significant at the 5\% level, indicating that each retains sufficient unique variance for estimation. Second, the block-level evidence---the joint \fstat{} of 99.53 and the 6-percentage-point $R^2$ gain---is unaffected by collinearity among the block's members. Third, the coefficient estimates are stable across the winsorized and bootstrap re-estimations reported below, which would not be the case if the estimates were fragile artifacts of a near-singular design. The practical implication, developed in Section~\ref{sec:discussion}, is that individual semantic coefficients should be interpreted as partial associations within a correlated block of quality signals, with block-level conclusions being the most robust.
18 +
19 +\subsection{Quantile Regression Analysis}
20 +
21 +To examine whether the implicit prices of semantic dimensions vary across the price distribution, we estimate Model~D using quantile regression \citep{koenker1978regression, koenker2005quantile} at the 25th, 50th, and 75th percentiles. Table~\ref{tab:quantile} reports the coefficients for the six most economically significant semantic dimensions.
22 +
23 +\input{tables/tab_quantile}
24 +
25 +Three patterns emerge. First, the Luxury premium \textit{increases} monotonically with property value: a one-standard-deviation increase in luxury similarity is associated with a 9.7\% premium at the 25th percentile but a 17.3\% premium at the 75th percentile. This is consistent with quality complementarities---luxury features are valued more in properties that already occupy the upper market segment. Second, the negative effects of Motivated Seller and Needs Renovation language \textit{attenuate} at higher quantiles (the Motivated Seller discount shrinks from 8.7\% at $\tau = 0.25$ to a statistically insignificant 3.8\% at $\tau = 0.75$), suggesting that urgency and condition discounts are proportionally smaller for high-value properties where buyer pools are less price-sensitive. Third, the Modern/Contemporary and Land \& Nature premiums are remarkably stable across quantiles, indicating that these dimensions carry value throughout the market rather than in a particular segment.
26 +
27 +These heterogeneous effects across the conditional price distribution confirm that the OLS estimates represent averages across meaningfully different market segments, and they strengthen the economic interpretation of the semantic dimensions.
28 +
29 +\subsection{Sensitivity to Outliers}
30 +
31 +We re-estimate Model~D after trimming prices below the 1st and above the 99th percentile, removing 341 observations from the tails of the price distribution. The trimmed model yields an adjusted $R^2$ of 0.486, somewhat lower than the full-sample estimate of 0.511---as expected, since trimming removes exactly the price variation that the model exploits. Importantly, no coefficient changes sign relative to the full model, and all 16 similarity measures that are significant in the full model remain significant in the trimmed specification. The largest coefficient shifts are for Entry-Level ($-0.024$) and Motivated Seller ($+0.023$), both within the confidence intervals of the full-model estimates. This stability confirms that our results are not driven by extreme observations in either tail of the price distribution.
32 +
33 +\subsection{Bootstrap Inference}
34 +
35 +To verify that our HC3 standard errors provide reliable inference, we estimate Model~D using 1,000 nonparametric bootstrap replications \citep{efron1993introduction}. For each replication, we resample $n = 17{,}087$ observations with replacement and re-estimate the model, obtaining the empirical distribution of each coefficient. Bootstrap standard errors are within 9\% of the HC3 estimates for every semantic dimension---and within 5\% for 15 of the 20---and the bootstrap 95\% percentile confidence intervals closely match the HC3-based intervals. This concordance confirms the reliability of our asymptotic inference under the observed heteroskedasticity pattern.
36 +
37 +\subsection{Regularized Variable Selection}
38 +
39 +As an alternative to significance-based variable selection (Model~E), we employ Lasso \citep{tibshirani1996regression} and elastic net \citep{zou2005regularization} regularization with five-fold cross-validation over the standardized Model~D design. At the cross-validated penalty, both procedures retain a sparse set of four similarity variables (Needs Renovation, Premium Location, Income/Investment, and Family-Friendly) and shrink the remainder to zero.
40 +
41 +This aggressive selection is the textbook behavior of $\ell_1$ penalties under strong collinearity: when regressors are correlated at 0.66--0.97, the Lasso selects one representative per correlated block and discards near-duplicates, so the retained variables act as proxies for their blocks rather than as a list of the ``true'' non-zero effects \citep{zou2005regularization}. The exercise therefore complements, rather than replicates, the significance-based selection in Model~E: it confirms that the semantic block contains genuine predictive signal that survives penalization, while reinforcing the message of the VIF analysis that variable-by-variable attributions within the block are sensitive to the selection criterion. Conclusions in this paper that rely on individual dimensions (e.g., the Luxury gradient) are accordingly cross-checked against the quantile and bootstrap evidence above.
42 +
43 +\subsection{Comparison with Alternative Text Representations}
44 +
45 +To contextualize the predictive and inferential value of the reference-based approach, Table~\ref{tab:text_comparison} compares it with alternative representations of the same text, estimated on identical samples. For each method, we report the $\Delta R^2$ gain over the structural-only Model~A together with an assessment of interpretability.
46 +
47 +\input{tables/tab_text_comparison}
48 +
49 +Two results stand out. First, \textit{what} the text says matters more than \textit{how much}: description length alone adds 1.2 percentage points of $R^2$, whereas content-based features add 4.4--7.8 points. Second, there is a measurable price of interpretability. Twenty principal components of the raw embeddings---which capture 49.5\% of the embedding variance but have no economic meaning---outperform the 20 reference similarities in pure fit ($\Delta R^2$ of $+0.070$ vs.\ $+0.044$ without the length control). The reference-based approach deliberately trades roughly two percentage points of $R^2$ for features that support coefficient-level economic inference; for the inferential goals of hedonic analysis, we consider this trade worthwhile, while applications that only require prediction may prefer the unrestricted embedding space.
50 +
51 +\subsection{Summary of Robustness Findings}
52 +
53 +Table~\ref{tab:robustness_summary} summarizes the robustness checks.
54 +
55 +\input{tables/tab_robustness_summary}
56 +
57 +The overall picture is that the paper's central findings---the joint explanatory power of semantic features and the sign and approximate magnitude of the main implicit price gradients---are stable across estimation methods, sample definitions, and distributional assumptions. The diagnostics also delineate the limits of the evidence: the semantic dimensions are strongly collinear, so individual coefficients are measured imprecisely relative to the block as a whole, and regularized selection does not single out the same variables as significance testing. Extensions that would further strengthen the analysis---spatial fixed effects, alternative embedding models, and systematic reference-description sensitivity analysis---are discussed in Sections~\ref{sec:discussion} and~\ref{sec:conclusion}.
added paper/sections/titlepage.tex +80 −0
@@ -0,0 +1,80 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +% ============================================================================
4 +% Title Page
5 +% ============================================================================
6 +\thispagestyle{empty}
7 +
8 +\begin{center}
9 +
10 +% --- Logo ---
11 +\includegraphics[width=3.5cm]{uq_logo.jpg}
12 +
13 +\vspace{0.6cm}
14 +
15 +{\footnotesize\textsc{Universit\'e du Qu\'ebec en Outaouais}}\\[0.15cm]
16 +{\footnotesize\textsc{D\'epartement des sciences administratives}}
17 +
18 +\vspace{0.8cm}
19 +
20 +{\footnotesize\textsc{Working Paper No.~\WPnumber}}
21 +
22 +\vspace{1.2cm}
23 +
24 +% --- Title ---
25 +{\LARGE\bfseries \WPtitle\par}
26 +
27 +\ifx\WPsubtitle\empty\else
28 + \vspace{0.3cm}
29 + {\large\itshape \WPsubtitle\par}
30 +\fi
31 +
32 +\vspace{1.2cm}
33 +
34 +% --- Author ---
35 +{\large \WPauthor\footnotemark[1]}\\[0.3cm]
36 +{\normalsize\itshape \WPaffiliation}
37 +
38 +\footnotetext[1]{D\'epartement des sciences administratives, Universit\'e du Qu\'ebec en Outaouais (UQO), 283 boulevard Alexandre-Tach\'e, Gatineau, QC J9A 1L8, Canada. Email: \href{mailto:\WPemail}{\WPemail}. All errors are my own.}
39 +
40 +\vspace{0.8cm}
41 +
42 +% --- Date & Version ---
43 +{\normalsize First draft: May 2026}\\[0.1cm]
44 +{\normalsize This version: \today}\\[0.1cm]
45 +{\small\itshape Version~\WPversion}
46 +
47 +\end{center}
48 +
49 +\vfill
50 +
51 +\newpage
52 +
53 +% ============================================================================
54 +% Abstract Page
55 +% ============================================================================
56 +\thispagestyle{empty}
57 +
58 +\vspace*{1cm}
59 +
60 +\noindent\rule{\textwidth}{0.4pt}
61 +\vspace{0.3cm}
62 +
63 +\noindent\textbf{Abstract}
64 +
65 +\vspace{0.15cm}
66 +
67 +\noindent\WPabstract
68 +
69 +\vspace{0.4cm}
70 +
71 +\noindent\textbf{Keywords:} \WPkeywords
72 +
73 +\vspace{0.15cm}
74 +
75 +\noindent\textbf{JEL Classification:} \WPjel
76 +
77 +\vspace{0.3cm}
78 +\noindent\rule{\textwidth}{0.4pt}
79 +
80 +\newpage
added paper/tables/tab_descriptive.tex +31 −0
@@ -0,0 +1,31 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Descriptive statistics for structural and textual variables ($n = 17{,}087$).}
6 +\label{tab:descriptive}
7 +\begin{threeparttable}
8 +\small
9 +\begin{adjustbox}{max width=\textwidth}
10 +\begin{tabular}{lR{1.6cm}R{1.7cm}R{1.2cm}R{1.4cm}R{1.4cm}R{1.4cm}R{1.9cm}}
11 +\toprule
12 +\textbf{Variable} & \textbf{Mean} & \textbf{Std.\ Dev.} & \textbf{Min} & \textbf{P25} & \textbf{Median} & \textbf{P75} & \textbf{Max} \\
13 +\midrule
14 +Price (\$) & 807{,}259 & 900{,}313 & 501 & 399{,}000 & 589{,}900 & 889{,}900 & 25{,}000{,}000 \\
15 +log(Price) & 13.32 & 0.70 & 6.22 & 12.90 & 13.29 & 13.70 & 17.03 \\
16 +Bedrooms & 3.5 & 1.3 & 0 & 3 & 3 & 4 & 23 \\
17 +Bathrooms & 1.8 & 0.9 & 0 & 1 & 2 & 2 & 18 \\
18 +Half-bathrooms & 0.5 & 0.6 & 0 & 0 & 0 & 1 & 12 \\
19 +Parking spaces & 5.5 & 5.1 & 0 & 2 & 4 & 7 & 152 \\
20 +Stories & 1.4 & 0.7 & 0 & 1 & 2 & 2 & 2 \\
21 +Lot size (sq.\ ft.) & 80{,}551 & 5{,}514{,}237 & 0 & 683 & 2{,}995 & 8{,}316 & 719{,}640{,}188 \\
22 +Description length (char.) & 510 & 151 & 22 & 430 & 549 & 634 & 703 \\
23 +\bottomrule
24 +\end{tabular}
25 +\end{adjustbox}
26 +\begin{tablenotes}
27 +\footnotesize
28 +\item \textit{Notes:} The sample is restricted to single-family houses with a positive listing price and a description of at least 20 characters. Descriptions are truncated at approximately 700 characters in the data export, which bounds the description-length variable. The lot-size field is noisy (units are not standardized at the source), which motivates its cautious interpretation in the regressions.
29 +\end{tablenotes}
30 +\end{threeparttable}
31 +\end{table}
added paper/tables/tab_full_results.tex +49 −0
@@ -0,0 +1,49 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Full model (D) coefficient estimates. HC3 robust standard errors. All variables standardized.}
6 +\label{tab:full_results}
7 +\small
8 +\begin{adjustbox}{max width=\textwidth}
9 +\begin{tabular}{lR{1.2cm}R{1.2cm}R{1.1cm}R{1.6cm}R{1.3cm}c}
10 +\toprule
11 +\textbf{Variable} & \textbf{Coeff.} & \textbf{Std.\ Err.} & \textbf{$t$-stat} & \textbf{$p$-value} & \textbf{Impact (\%)} & \\
12 +\midrule
13 +\multicolumn{7}{l}{\textit{Panel A: Structural variables}} \\
14 +\quad Bathrooms & 0.2966 & 0.0110 & 26.86 & $< 0.001$ & +34.5 & *** \\
15 +\quad Half-bathrooms & 0.1580 & 0.0072 & 21.92 & $< 0.001$ & +17.1 & *** \\
16 +\quad Description length & 0.1013 & 0.0046 & 21.84 & $< 0.001$ & +10.7 & *** \\
17 +\quad Parking & 0.0914 & 0.0094 & 9.68 & $< 0.001$ & +9.6 & *** \\
18 +\quad Stories & 0.0744 & 0.0057 & 12.97 & $< 0.001$ & +7.7 & *** \\
19 +\quad Bedrooms & 0.0314 & 0.0071 & 4.43 & $< 0.001$ & +3.2 & *** \\
20 +\quad Lot size & 0.0033 & 0.7561 & 0.00 & 0.996 & +0.3 & \\
21 +\midrule
22 +\multicolumn{7}{l}{\textit{Panel B: Semantic similarities --- positive price effects}} \\
23 +\quad Modern/Contemporary & 0.1517 & 0.0155 & 9.78 & $< 0.001$ & +16.4 & *** \\
24 +\quad Luxury & 0.1330 & 0.0182 & 7.31 & $< 0.001$ & +14.2 & *** \\
25 +\quad Land \& Nature & 0.1225 & 0.0122 & 10.02 & $< 0.001$ & +13.0 & *** \\
26 +\quad Entry-Level & 0.0991 & 0.0251 & 3.95 & $< 0.001$ & +10.4 & *** \\
27 +\quad Renovated & 0.0501 & 0.0267 & 1.88 & 0.060 & +5.1 & \\
28 +\quad Waterfront & 0.0462 & 0.0129 & 3.59 & $< 0.001$ & +4.7 & *** \\
29 +\quad Panoramic View & 0.0300 & 0.0116 & 2.58 & 0.010 & +3.0 & ** \\
30 +\quad Pool \& Landscaping & 0.0246 & 0.0161 & 1.52 & 0.128 & +2.5 & \\
31 +\quad Garage \& Parking & 0.0190 & 0.0086 & 2.19 & 0.028 & +1.9 & * \\
32 +\quad Heritage/Character & 0.0103 & 0.0147 & 0.70 & 0.483 & +1.0 & \\
33 +\midrule
34 +\multicolumn{7}{l}{\textit{Panel C: Semantic similarities --- negative price effects}} \\
35 +\quad Family-Friendly & $-$0.1237 & 0.0127 & $-$9.72 & $< 0.001$ & $-$11.6 & *** \\
36 +\quad New Construction & $-$0.1105 & 0.0176 & $-$6.29 & $< 0.001$ & $-$10.5 & *** \\
37 +\quad Quiet \& Peaceful & $-$0.0995 & 0.0142 & $-$7.01 & $< 0.001$ & $-$9.5 & *** \\
38 +\quad Bright \& Spacious & $-$0.0936 & 0.0185 & $-$5.07 & $< 0.001$ & $-$8.9 & *** \\
39 +\quad Motivated Seller & $-$0.0890 & 0.0216 & $-$4.12 & $< 0.001$ & $-$8.5 & *** \\
40 +\quad Needs Renovation & $-$0.0827 & 0.0170 & $-$4.86 & $< 0.001$ & $-$7.9 & *** \\
41 +\quad Energy Efficient & $-$0.0812 & 0.0136 & $-$5.95 & $< 0.001$ & $-$7.8 & *** \\
42 +\quad Premium Location & $-$0.0628 & 0.0139 & $-$4.53 & $< 0.001$ & $-$6.1 & *** \\
43 +\quad Finished Basement & $-$0.0370 & 0.0147 & $-$2.52 & 0.012 & $-$3.6 & * \\
44 +\quad Income/Investment & $-$0.0316 & 0.0162 & $-$1.95 & 0.051 & $-$3.1 & \\
45 +\bottomrule
46 +\multicolumn{7}{l}{\footnotesize{\signote\ Impact $= (e^{\hat{\beta}} - 1) \times 100$\%. $n = 17{,}087$; Adj.\ $R^2 = 0.511$.}}
47 +\end{tabular}
48 +\end{adjustbox}
49 +\end{table}
added paper/tables/tab_model_comparison.tex +27 −0
@@ -0,0 +1,27 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Hedonic model comparison ($n = 17{,}087$).}
6 +\label{tab:model_comparison}
7 +\begin{threeparttable}
8 +\small
9 +\begin{adjustbox}{max width=\textwidth}
10 +\begin{tabular}{clR{1.3cm}R{1.3cm}R{1.6cm}R{1.6cm}R{0.8cm}R{1.8cm}}
11 +\toprule
12 +& \textbf{Specification} & $\boldsymbol{R^2}$ & \textbf{Adj.}~$\boldsymbol{R^2}$ & \textbf{AIC} & \textbf{BIC} & $\boldsymbol{k}$ & $\boldsymbol{\Delta R^2}$ \textbf{vs.\ A} \\
13 +\midrule
14 +A & Structural only & 0.4525 & 0.4523 & 26{,}165 & 26{,}219 & 6 & --- \\
15 +B & + Description length & 0.4647 & 0.4645 & 25{,}779 & 25{,}841 & 7 & +0.012 \\
16 +C & + Semantic similarities (20) & 0.4962 & 0.4954 & 24{,}783 & 24{,}992 & 26 & +0.044 \\
17 +D & Full model (B + C) & \textbf{0.5122} & \textbf{0.5115} & \textbf{24{,}232} & 24{,}448 & 27 & +0.060 \\
18 +E & Parsimonious (sig.\ only) & 0.5118 & 0.5111 & 24{,}239 & \textbf{24{,}425} & 23 & +0.059 \\
19 +\bottomrule
20 +\end{tabular}
21 +\end{adjustbox}
22 +\begin{tablenotes}
23 +\footnotesize
24 +\item \textit{Notes:} $k$ denotes the number of regressors excluding the intercept. Bold values indicate the best fit for each criterion. Models are estimated by OLS with HC3 robust standard errors.
25 +\end{tablenotes}
26 +\end{threeparttable}
27 +\end{table}
added paper/tables/tab_parsimonious.tex +43 −0
@@ -0,0 +1,43 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Parsimonious model (E) coefficient estimates. HC3 robust standard errors. All variables standardized.}
6 +\label{tab:parsimonious}
7 +\small
8 +\begin{adjustbox}{max width=\textwidth}
9 +\begin{tabular}{lR{1.3cm}R{1.3cm}R{1.6cm}R{1.4cm}c}
10 +\toprule
11 +\textbf{Variable} & \textbf{Coeff.} & \textbf{Std.\ Err.} & \textbf{$p$-value} & \textbf{Impact (\%)} & \\
12 +\midrule
13 +\multicolumn{6}{l}{\textit{Panel A: Structural variables}} \\
14 +\quad Bathrooms & 0.2962 & 0.0110 & $< 0.001$ & +34.5 & *** \\
15 +\quad Half-bathrooms & 0.1581 & 0.0072 & $< 0.001$ & +17.1 & *** \\
16 +\quad Description length & 0.1017 & 0.0046 & $< 0.001$ & +10.7 & *** \\
17 +\quad Parking & 0.0914 & 0.0095 & $< 0.001$ & +9.6 & *** \\
18 +\quad Stories & 0.0746 & 0.0058 & $< 0.001$ & +7.7 & *** \\
19 +\quad Bedrooms & 0.0314 & 0.0071 & $< 0.001$ & +3.2 & *** \\
20 +\quad Lot size & 0.0035 & 0.7525 & 0.996 & +0.3 & \\
21 +\midrule
22 +\multicolumn{6}{l}{\textit{Panel B: Semantic similarities (significant at 5\% in Model D)}} \\
23 +\quad Modern/Contemporary & 0.1590 & 0.0152 & $< 0.001$ & +17.2 & *** \\
24 +\quad Luxury & 0.1552 & 0.0158 & $< 0.001$ & +16.8 & *** \\
25 +\quad Land \& Nature & 0.1254 & 0.0119 & $< 0.001$ & +13.4 & *** \\
26 +\quad Entry-Level & 0.0823 & 0.0248 & $< 0.001$ & +8.6 & *** \\
27 +\quad Waterfront & 0.0470 & 0.0130 & $< 0.001$ & +4.8 & *** \\
28 +\quad Panoramic View & 0.0272 & 0.0114 & 0.017 & +2.8 & * \\
29 +\quad Garage \& Parking & 0.0085 & 0.0076 & 0.262 & +0.9 & \\
30 +\quad Finished Basement & $-$0.0295 & 0.0138 & 0.033 & $-$2.9 & * \\
31 +\quad Premium Location & $-$0.0614 & 0.0137 & $< 0.001$ & $-$6.0 & *** \\
32 +\quad Energy Efficient & $-$0.0669 & 0.0130 & $< 0.001$ & $-$6.5 & *** \\
33 +\quad Needs Renovation & $-$0.0743 & 0.0154 & $< 0.001$ & $-$7.2 & *** \\
34 +\quad Motivated Seller & $-$0.0816 & 0.0196 & $< 0.001$ & $-$7.8 & *** \\
35 +\quad Bright \& Spacious & $-$0.0836 & 0.0179 & $< 0.001$ & $-$8.0 & *** \\
36 +\quad Quiet \& Peaceful & $-$0.0882 & 0.0116 & $< 0.001$ & $-$8.4 & *** \\
37 +\quad Family-Friendly & $-$0.1142 & 0.0110 & $< 0.001$ & $-$10.8 & *** \\
38 +\quad New Construction & $-$0.1279 & 0.0165 & $< 0.001$ & $-$12.0 & *** \\
39 +\bottomrule
40 +\multicolumn{6}{l}{\footnotesize{\signote\ Impact $= (e^{\hat{\beta}} - 1) \times 100$\%. $n = 17{,}087$; Adj.\ $R^2 = 0.511$.}}
41 +\end{tabular}
42 +\end{adjustbox}
43 +\end{table}
added paper/tables/tab_quantile.tex +23 −0
@@ -0,0 +1,23 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Quantile regression coefficients for selected semantic dimensions.}
6 +\label{tab:quantile}
7 +\small
8 +\begin{adjustbox}{max width=\textwidth}
9 +\begin{tabular}{lR{1.5cm}R{1.5cm}R{1.5cm}R{1.5cm}l}
10 +\toprule
11 +\textbf{Dimension} & $\boldsymbol{\tau = 0.25}$ & $\boldsymbol{\tau = 0.50}$ & $\boldsymbol{\tau = 0.75}$ & \textbf{OLS} & \textbf{Pattern} \\
12 +\midrule
13 +Modern/Contemporary & 0.148*** & 0.121*** & 0.155*** & 0.152*** & High at both tails \\
14 +Luxury & 0.093*** & 0.130*** & 0.160*** & 0.133*** & Increasing \\
15 +Land \& Nature & 0.105*** & 0.098*** & 0.104*** & 0.122*** & Stable \\
16 +Family-Friendly & $-$0.121*** & $-$0.105*** & $-$0.108*** & $-$0.124*** & Mildly attenuating \\
17 +Motivated Seller & $-$0.091*** & $-$0.046* & $-$0.039 & $-$0.089*** & Attenuating \\
18 +Needs Renovation & $-$0.090*** & $-$0.065*** & $-$0.044** & $-$0.083*** & Attenuating \\
19 +\bottomrule
20 +\multicolumn{6}{l}{\footnotesize{\signote\ Quantile regressions of Model~D; standard errors follow the kernel-based estimator of \citet{koenker1978regression}.}}
21 +\end{tabular}
22 +\end{adjustbox}
23 +\end{table}
added paper/tables/tab_reference_texts.tex +38 −0
@@ -0,0 +1,38 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{footnotesize}
4 +\begin{longtable}{p{3.2cm}p{11.5cm}}
5 +\caption{The 20 reference descriptions (verbatim French text used for embedding).}
6 +\label{tab:reference_texts} \\
7 +\toprule
8 +\textbf{Dimension} & \textbf{Reference description} \\
9 +\midrule
10 +\endfirsthead
11 +\toprule
12 +\textbf{Dimension} & \textbf{Reference description} \\
13 +\midrule
14 +\endhead
15 +\bottomrule
16 +\endfoot
17 +Luxury & \textit{Propriété de prestige haut de gamme avec finitions luxueuses, matériaux nobles, planchers de bois franc, comptoirs de quartz et granit, cuisine gastronomique, salle de bain spa avec douche en verre, domotique et système audio intégré. Résidence d'exception.} \\[2pt]
18 +Entry-Level & \textit{Propriété abordable idéale pour premier acheteur, bon prix, petit budget, opportunité d'investissement, starter home, parfait pour débuter, prix compétitif, aubaine.} \\[2pt]
19 +Renovated & \textit{Entièrement rénové, remis à neuf, nouvelles fenêtres, nouvelle toiture, plomberie et électricité refaites, cuisine et salle de bain rénovées, modernisé, mise à jour complète, rien à faire, clé en main, prêt à emménager.} \\[2pt]
20 +Needs Renovation & \textit{À rénover, à rafraîchir, nécessite des travaux, bon potentiel, vendu tel quel sans garantie, handyman special, besoin de rénovation, fixer-upper, à mettre à son goût.} \\[2pt]
21 +Bright \& Spacious & \textit{Très lumineux, fenestration abondante, aires ouvertes, grands espaces, plafonds hauts, cathédrale, mezzanine, vaste salon, pièces spacieuses, beaucoup de rangement, walk-in.} \\[2pt]
22 +Land \& Nature & \textit{Grand terrain boisé, mature, aménagement paysager, piscine creusée, spa, terrasse, patio, cour arrière privée, jardin, haie de cèdres, intime, sans voisin arrière, vue sur la nature, bord de l'eau, accès au lac, rivière.} \\[2pt]
23 +Panoramic View & \textit{Vue imprenable, vue panoramique, vue sur le fleuve, vue sur la montagne, vue sur la ville, skyline de Montréal, vue dégagée, vue spectaculaire, penthouse avec vue.} \\[2pt]
24 +Premium Location & \textit{Emplacement de choix, quartier recherché, proche de tout, à distance de marche des commerces, restaurants, cafés, accès rapide au transport en commun, métro, autoroute, près des écoles, des parcs, quartier familial sécuritaire.} \\[2pt]
25 +Quiet \& Peaceful & \textit{Rue tranquille, cul-de-sac, quartier paisible, résidentiel, calme, intimité, retiré, campagne, nature, boisé, loin du bruit, environnement serein.} \\[2pt]
26 +Income/Investment & \textit{Excellent investissement, revenu locatif, plex rentable, baux en cours, bon rendement, cash flow positif, logements loués, duplex triplex avec revenus, rapport qualité-prix.} \\[2pt]
27 +Garage \& Parking & \textit{Garage double, garage chauffé, stationnement intérieur, entrée de garage pavée, abri d'auto, grand garage, atelier dans le garage, espace de rangement au garage.} \\[2pt]
28 +Finished Basement & \textit{Sous-sol entièrement aménagé, salle familiale au sous-sol, chambre supplémentaire, salle de cinéma, bureau, possibilité de logement au sous-sol, entrée indépendante, sous-sol avec salle de bain complète.} \\[2pt]
29 +Modern/Contemporary & \textit{Design moderne, contemporain, architecture épurée, lignes droites, minimaliste, construction neuve, maison intelligente, écoénergétique, LEED, fenêtres panoramiques, toit plat.} \\[2pt]
30 +Heritage/Character & \textit{Cachet d'époque, maison ancestrale, patrimoine, boiseries d'origine, moulures, foyer d'origine, plafonds de 10 pieds, charme victorien, pierre, brique.} \\[2pt]
31 +Energy Efficient & \textit{Écoénergétique, thermopompe, géothermie, panneaux solaires, isolation supérieure, fenêtres Energy Star, chauffage radiant, faible consommation, certifié Novoclimat, réservoir d'eau chaude récent, coûts énergétiques bas.} \\[2pt]
32 +Motivated Seller & \textit{Vendeur motivé, vente rapide, prix réduit, réduction de prix, succession, relocalisation, doit vendre, reprise de finance, offres multiples bienvenues, ne manquez pas cette occasion.} \\[2pt]
33 +Family-Friendly & \textit{Maison familiale, quartier familial, parc pour enfants, cour clôturée, école à proximité, garderie, aire de jeux, voisinage sécuritaire, idéal pour famille avec enfants.} \\[2pt]
34 +Waterfront & \textit{Bord de l'eau, accès au lac, vue sur le fleuve, rivière, quai privé, droits nautiques, plage, navigable, chalet au bord du lac, waterfront, pieds dans l'eau.} \\[2pt]
35 +New Construction & \textit{Construction neuve, maison neuve, jamais habitée, modèle de démonstration, garantie GCR, livraison prochaine, choix de finitions, plans personnalisables, nouveau développement.} \\[2pt]
36 +Pool \& Landscaping & \textit{Piscine creusée chauffée, piscine hors terre, spa, cuisine extérieure, terrasse en composite, pergola, aménagement paysager professionnel, pavé uni, foyer extérieur.} \\[2pt]
37 +\end{longtable}
38 +\end{footnotesize}
added paper/tables/tab_references.tex +47 −0
@@ -0,0 +1,47 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Reference description categories, domains, and expected price effects.}
6 +\label{tab:references}
7 +\small
8 +\begin{adjustbox}{max width=\textwidth}
9 +\begin{tabular}{llll}
10 +\toprule
11 +\textbf{Domain} & \textbf{Category} & \textbf{Dimension} & \textbf{Expected Effect} \\
12 +\midrule
13 +\multirow{2}{*}{Quality \& Standing}
14 + & Luxury & Premium finishes, prestige materials & Positive \\
15 + & Entry-Level & Affordable, first-time buyer language & Negative \\
16 +\midrule
17 +\multirow{2}{*}{Condition}
18 + & Renovated & Turnkey, recently updated & Positive \\
19 + & Needs Renovation & Fixer-upper, requires work & Negative \\
20 +\midrule
21 +\multirow{4}{*}{Physical Attributes}
22 + & Bright \& Spacious & Luminosity, open concept, high ceilings & Positive \\
23 + & Garage \& Parking & Garage, parking facilities & Positive \\
24 + & Finished Basement & Developed lower level & Positive \\
25 + & Pool \& Landscaping & Pool, outdoor amenities & Positive \\
26 +\midrule
27 +\multirow{4}{*}{Location \& Setting}
28 + & Land \& Nature & Wooded lot, natural setting & Positive \\
29 + & Panoramic View & Water, mountain, city views & Positive \\
30 + & Premium Location & Near amenities, transit, schools & Positive \\
31 + & Quiet \& Peaceful & Rural, secluded, tranquil & Ambiguous \\
32 + & Waterfront & Lake/river access, dock & Positive \\
33 +\midrule
34 +\multirow{4}{*}{Style \& Character}
35 + & Modern/Contemporary & Clean design, smart home, energy-efficient & Positive \\
36 + & Heritage/Character & Period features, historic charm & Ambiguous \\
37 + & New Construction & Newly built, never occupied & Positive \\
38 + & Energy Efficient & Heat pump, solar, high insulation & Ambiguous \\
39 + & Family-Friendly & Schools, parks, safe neighborhood & Ambiguous \\
40 +\midrule
41 +\multirow{2}{*}{Market Signals}
42 + & Income/Investment & Rental income, investment potential & Ambiguous \\
43 + & Motivated Seller & Urgency, price reduction, distress & Negative \\
44 +\bottomrule
45 +\end{tabular}
46 +\end{adjustbox}
47 +\end{table}
added paper/tables/tab_robustness_summary.tex +24 −0
@@ -0,0 +1,24 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Summary of robustness analyses.}
6 +\label{tab:robustness_summary}
7 +\small
8 +\begin{adjustbox}{max width=\textwidth}
9 +\begin{tabular}{llp{7.5cm}}
10 +\toprule
11 +\textbf{Test} & \textbf{Main Result Stable?} & \textbf{Notes} \\
12 +\midrule
13 +HC3 robust standard errors & Yes & Baseline inference method \\
14 +Breusch-Pagan test & Yes & Heteroskedasticity confirmed (LM $= 768.0$); HC3 justified \\
15 +Variance inflation factors & Yes (block level) & Substantial collinearity (mean VIF 17.2; 16/20 $>$ 10); joint inference unaffected, individual coefficients less precise \\
16 +Bootstrap inference (1{,}000 rep.) & Yes & SE within 9\% of HC3 (15/20 within 5\%); CIs match \\
17 +Trimmed prices (1st/99th pctile) & Yes & No sign reversals; all 16 significant dimensions remain significant \\
18 +Quantile regression ($\tau = .25, .50, .75$) & Yes & Luxury premium increasing in price; discounts attenuate at upper quantiles \\
19 +Lasso / elastic net selection & Yes (block level) & Sparse selection of 4 block representatives; semantic signal survives penalization \\
20 +PCA comparison & Yes & PCA fits better ($\Delta R^2$ +0.070 vs.\ +0.044) but is uninterpretable; reference approach preferred for inference \\
21 +\bottomrule
22 +\end{tabular}
23 +\end{adjustbox}
24 +\end{table}
added paper/tables/tab_similarity_stats.tex +42 −0
@@ -0,0 +1,42 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +% Generated by scripts/06_tables.py — do not edit by hand.
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Descriptive statistics for cosine similarity features and bivariate correlations with log(price).}
6 +\label{tab:sim_stats}
7 +\begin{threeparttable}
8 +\small
9 +\begin{adjustbox}{max width=\textwidth}
10 +\begin{tabular}{lR{1.2cm}R{1.2cm}R{1.2cm}R{1.2cm}R{1.2cm}R{1.4cm}}
11 +\toprule
12 +\textbf{Reference} & \textbf{Mean} & \textbf{Std.~Dev.} & \textbf{Min} & \textbf{Median} & \textbf{Max} & \textbf{Corr.\ $\ln P$} \\
13 +\midrule
14 +Luxury & 0.337 & 0.157 & $-$0.030 & 0.376 & 0.692 & $-$0.234 \\
15 +Entry-Level & 0.313 & 0.167 & $-$0.085 & 0.385 & 0.660 & $-$0.270 \\
16 +Renovated & 0.352 & 0.168 & $-$0.056 & 0.411 & 0.753 & $-$0.250 \\
17 +Needs Renovation & 0.321 & 0.138 & $-$0.083 & 0.367 & 0.652 & $-$0.251 \\
18 +Bright \& Spacious & 0.357 & 0.153 & $-$0.104 & 0.400 & 0.672 & $-$0.228 \\
19 +Land \& Nature & 0.330 & 0.116 & $-$0.021 & 0.337 & 0.701 & $-$0.174 \\
20 +Panoramic View & 0.316 & 0.120 & $-$0.114 & 0.343 & 0.613 & $-$0.185 \\
21 +Premium Location & 0.321 & 0.141 & $-$0.025 & 0.364 & 0.693 & $-$0.289 \\
22 +Quiet \& Peaceful & 0.358 & 0.111 & $-$0.065 & 0.376 & 0.630 & $-$0.228 \\
23 +Income/Investment & 0.358 & 0.126 & $-$0.029 & 0.398 & 0.661 & $-$0.275 \\
24 +Garage \& Parking & 0.310 & 0.112 & $-$0.078 & 0.317 & 0.685 & $-$0.210 \\
25 +Finished Basement & 0.305 & 0.139 & $-$0.069 & 0.324 & 0.708 & $-$0.228 \\
26 +Modern/Contemporary & 0.365 & 0.115 & $-$0.044 & 0.382 & 0.671 & $-$0.187 \\
27 +Heritage/Character & 0.366 & 0.143 & $-$0.090 & 0.399 & 0.715 & $-$0.235 \\
28 +Energy Efficient & 0.298 & 0.134 & $-$0.058 & 0.327 & 0.648 & $-$0.234 \\
29 +Motivated Seller & 0.254 & 0.165 & $-$0.151 & 0.326 & 0.586 & $-$0.279 \\
30 +Family-Friendly & 0.349 & 0.142 & $-$0.080 & 0.371 & 0.722 & $-$0.231 \\
31 +Waterfront & 0.331 & 0.151 & $-$0.117 & 0.370 & 0.728 & $-$0.223 \\
32 +New Construction & 0.372 & 0.122 & $-$0.020 & 0.396 & 0.694 & $-$0.240 \\
33 +Pool \& Landscaping & 0.348 & 0.158 & $-$0.070 & 0.382 & 0.756 & $-$0.240 \\
34 +\bottomrule
35 +\end{tabular}
36 +\end{adjustbox}
37 +\begin{tablenotes}
38 +\footnotesize
39 +\item \textit{Notes:} Cosine similarity is computed between each listing embedding and the corresponding reference description embedding using the all-MiniLM-L6-v2 sentence transformer. Corr.\ $\ln P$ denotes the Pearson correlation with log listing price.
40 +\end{tablenotes}
41 +\end{threeparttable}
42 +\end{table}
added paper/tables/tab_text_comparison.tex +22 −0
@@ -0,0 +1,22 @@
1 +% Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +%
3 +\begin{table}[!htbp]
4 +\centering
5 +\caption{Comparison of text representation methods in the hedonic model. $\Delta R^2$ is the improvement over Model~A (structural only, $R^2 = 0.452$).}
6 +\label{tab:text_comparison}
7 +\small
8 +\begin{adjustbox}{max width=\textwidth}
9 +\begin{tabular}{lR{1.5cm}C{2.2cm}C{2.6cm}}
10 +\toprule
11 +\textbf{Text Representation} & $\boldsymbol{\Delta R^2}$ & \textbf{Interpretability} & \textbf{Economic Usefulness} \\
12 +\midrule
13 +Description length only & +0.012 & High & Low \\
14 +PCA on embeddings (20 PC) & +0.070 & None & Low \\
15 +PCA on embeddings (20 PC) + length & +0.078 & None & Low \\
16 +Reference cosine (20 dim) & +0.044 & High & High \\
17 +\textbf{Reference cosine (20 dim) + length} & \textbf{+0.060} & \textbf{High} & \textbf{High} \\
18 +\bottomrule
19 +\multicolumn{4}{p{0.95\textwidth}}{\footnotesize{\textit{Notes:} All specifications include the six structural variables and are estimated on the same $n = 17{,}087$ sample. The 20 principal components capture 49.5\% of the variance of the raw 384-dimensional embeddings. PCA achieves a larger fit improvement than the reference projections but produces features with no economic interpretation; the reference-based approach trades roughly two percentage points of $R^2$ for coefficient-level interpretability (see Section~\ref{sec:robustness}).}} \\
20 +\end{tabular}
21 +\end{adjustbox}
22 +\end{table}
added paper/uq_logo.jpg +0 −0

Binary file not shown.

added requirements.txt +13 −0
@@ -0,0 +1,13 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +# Pinned to the versions used to reproduce the paper's results
4 +# (Python 3.14, macOS arm64).
5 +numpy==2.4.4
6 +pandas==3.0.2
7 +pyarrow==24.0.0
8 +scipy==1.17.1
9 +scikit-learn==1.6.1
10 +statsmodels==0.14.6
11 +matplotlib==3.10.9
12 +sentence-transformers==5.5.0
13 +torch==2.12.0
added results/coefficients_model_D.csv +28 −0
@@ -0,0 +1,28 @@
1 +variable,label,coefficient,std_error,t_value,p_value,impact_pct
2 +bedrooms,bedrooms,0.031361825945882715,0.007078664712443264,4.430472019779829,9.402704737112851e-06,3.185878963279243
3 +bathrooms,bathrooms,0.2965921212218176,0.01104123193902946,26.86223085065347,6.070330029949193e-159,34.52664818812274
4 +half_baths,half_baths,0.1580223318079423,0.0072082518706523485,21.92242094804066,1.587846097570319e-106,17.119234925823633
5 +parking,parking,0.09137930012830353,0.009438374661485838,9.68167755632601,3.607473151652128e-22,9.568451973005132
6 +stories,stories,0.07440621383479015,0.005738795985108444,12.965474644484008,1.9203799101867084e-38,7.724430817181238
7 +land_size,land_size,0.0033359400952781837,0.7561010299645798,0.0044120295609628496,0.9964797211524664,0.33415105359342334
8 +remarks_length,remarks_length,0.10134517255307976,0.004639491646613843,21.84402522354945,8.859561078716114e-106,10.665856400701944
9 +sim_luxe,Luxury,0.13302767513624159,0.018189664572979443,7.313366038307975,2.6053203462952673e-13,14.22816106926561
10 +sim_entree_de_gamme,Entry-Level,0.09909068025717205,0.025104381911900585,3.947146781183993,7.908807394940488e-05,10.416642111431651
11 +sim_renove,Renovated,0.050148954159303316,0.026701523515245563,1.8781308164184023,0.060363271564756425,5.14276992414191
12 +sim_a_renover,Needs Renovation,-0.08267657263757641,0.016997896058291648,-4.863929768369563,1.1507776085974207e-06,-7.93511378938514
13 +sim_lumineux_spacieux,Bright & Spacious,-0.09363534955276323,0.01848243511045256,-5.066180348703547,4.058774012921762e-07,-8.938524221538701
14 +sim_terrain_nature,Land & Nature,0.12246056220267565,0.012220545036757098,10.020875651154457,1.2340470542470092e-23,13.027454365664525
15 +sim_vue_panoramique,Panoramic View,0.03003177740935259,0.011619086231196951,2.584693564689969,0.009746557871840842,3.048727964934539
16 +sim_localisation_premium,Premium Location,-0.06278200580427294,0.013861102757784448,-4.529365873794878,5.916097588666791e-06,-6.085181977187871
17 +sim_tranquillite,Quiet & Peaceful,-0.09950647079921601,0.014190152331860514,-7.0123610002267975,2.343296248267464e-12,-9.47159080620371
18 +sim_revenu_investissement,Income/Investment,-0.03159128289655594,0.016178269114433735,-1.9526985657798956,0.05085532374949879,-3.1097491810533673
19 +sim_garage_stationnement,Garage & Parking,0.018962330637720064,0.008646812255471013,2.1929851230113404,0.028308450973544435,1.9143257417747694
20 +sim_sous_sol_amenage,Finished Basement,-0.036991886118690004,0.014677015596754791,-2.5203956400284273,0.011722299286143803,-3.6316045465441094
21 +sim_moderne_contemporain,Modern/Contemporary,0.15168086020684507,0.015507955333018648,9.780841958184835,1.360738144810086e-22,16.37887658537329
22 +sim_cachet_patrimonial,Heritage/Character,0.010280833145417556,0.014670796973183341,0.7007685515797013,0.4834474670703699,1.0333862483290934
23 +sim_ecoefficace,Energy Efficient,-0.08116151901604113,0.013649908523322204,-5.945938676245979,2.7487704804701127e-09,-7.795524834440548
24 +sim_urgence_motivation,Motivated Seller,-0.08896927610037955,0.021572426521444154,-4.124212731096304,3.7200495796830386e-05,-8.512631837018493
25 +sim_familial,Family-Friendly,-0.12365094148067936,0.012726960207181169,-9.71566968606608,2.5854382091522054e-22,-11.631175403541361
26 +sim_bord_eau,Waterfront,0.0461844747419041,0.012850514015221415,3.593978784599484,0.00032566636619945906,4.726758755620675
27 +sim_neuf_construction,New Construction,-0.11053160085021901,0.017580467433391835,-6.287182139439505,3.2328001501811506e-10,-10.464196433275463
28 +sim_piscine_amenagement,Pool & Landscaping,0.024584387749493897,0.01613737926366,1.5234436365299933,0.1276477518475808,2.4889075540599714
added results/coefficients_model_E.csv +24 −0
@@ -0,0 +1,24 @@
1 +variable,label,coefficient,std_error,t_value,p_value,impact_pct
2 +bedrooms,bedrooms,0.03144088418916516,0.0070659728181450615,4.449618615631602,8.602291004517089e-06,3.194036980076631
3 +bathrooms,bathrooms,0.29621373110135635,0.01102995889479155,26.855379419521796,7.298644283011366e-159,34.47575426299607
4 +half_baths,half_baths,0.1581095433944701,0.007211755054116357,21.92386488559725,1.5382681419705999e-106,17.12944952552369
5 +parking,parking,0.09140311679672333,0.009470632481334363,9.65121568985698,4.857640256188567e-22,9.571061559570747
6 +stories,stories,0.07461453235088428,0.005752530826687549,12.970731422198948,1.7931106817778609e-38,7.746874148355731
7 +land_size,land_size,0.0034587847266077734,0.7525127787557733,0.004596313609885309,0.9963326852467446,0.3464773224818529
8 +remarks_length,remarks_length,0.1017465232536718,0.004584987551929257,22.19123216830964,4.1739945112819873e-109,10.710281134052568
9 +sim_luxe,Luxury,0.1551862204222919,0.015777833131677823,9.835711857714985,7.900686494882705e-23,16.787542311103955
10 +sim_entree_de_gamme,Entry-Level,0.0823146156031635,0.024803533788535576,3.3186648444912343,0.0009044891845479017,8.579736489032431
11 +sim_a_renover,Needs Renovation,-0.07426757136724954,0.01537454115424859,-4.8305553071238485,1.3615278693721875e-06,-7.1576758872661355
12 +sim_lumineux_spacieux,Bright & Spacious,-0.08358827300251617,0.01789424005050917,-4.671239056063614,2.9938825965665374e-06,-8.019011129312513
13 +sim_terrain_nature,Land & Nature,0.12543573085379503,0.011871429628508534,10.566185773663566,4.275309494370113e-26,13.364230839594615
14 +sim_vue_panoramique,Panoramic View,0.02722771883802294,0.011431694758029768,2.381774480017308,0.017229445322542926,2.7601780405304144
15 +sim_localisation_premium,Premium Location,-0.061443048075255344,0.013701359287106004,-4.484449081856996,7.310255914700689e-06,-5.959349782531564
16 +sim_tranquillite,Quiet & Peaceful,-0.0882186858096159,0.011582866482814204,-7.616308617604132,2.6103314719484416e-14,-8.443936528957375
17 +sim_garage_stationnement,Garage & Parking,0.00853252877946338,0.007604019897965316,1.1221076343772476,0.26181667825546573,0.8569034558149724
18 +sim_sous_sol_amenage,Finished Basement,-0.029508559391782027,0.013830688593384974,-2.133556777924677,0.0328790792492281,-2.9077432901014055
19 +sim_moderne_contemporain,Modern/Contemporary,0.15902897162691987,0.015225336614104265,10.44502171988766,1.5442003755522436e-25,17.237191171034173
20 +sim_ecoefficace,Energy Efficient,-0.0669311874590732,0.013022728057317483,-5.139567313736887,2.753718320743014e-07,-6.474044329095396
21 +sim_urgence_motivation,Motivated Seller,-0.08158352267714244,0.019589689722517935,-4.164615358014778,3.118779073135082e-05,-7.834427251471798
22 +sim_familial,Family-Friendly,-0.11419557408753708,0.011041756677428223,-10.342156363667923,4.542003645912472e-25,-10.791652959018638
23 +sim_bord_eau,Waterfront,0.046952071138328166,0.013047927792043796,3.598431251815941,0.000320142450969529,4.807177498870829
24 +sim_neuf_construction,New Construction,-0.12794991584303217,0.01651988095759648,-7.745208102374119,9.54254292685115e-15,-12.010255303706586
added results/descriptive_stats.csv +10 −0
@@ -0,0 +1,10 @@
1 +,count,mean,std,min,25%,50%,75%,max
2 +price,17087.0,807259.0277989114,900313.3533743073,501.0,399000.0,589900.0,889900.0,25000000.0
3 +log_price,17087.0,13.320917403072425,0.7029075440764694,6.2166061010848646,12.896716695872,13.287708309991167,13.698864375844954,17.034386382832476
4 +bedrooms,17087.0,3.4694211974015334,1.3392605806714963,0.0,3.0,3.0,4.0,23.0
5 +bathrooms,17087.0,1.7682448645168842,0.8884344626485206,0.0,1.0,2.0,2.0,18.0
6 +half_baths,17087.0,0.5014338385907415,0.6050876262215626,0.0,0.0,0.0,1.0,12.0
7 +parking,17087.0,5.455668051735238,5.119719530090579,0.0,2.0,4.0,7.0,152.0
8 +stories,17087.0,1.403054954058641,0.6655852010097871,0.0,1.0,2.0,2.0,2.0
9 +land_size,17087.0,80551.4150768713,5514236.559302572,0.0,683.05,2995.0,8315.75,719640188.0
10 +remarks_length,17087.0,510.0756715631767,151.15397681752557,22.0,430.0,549.0,634.0,703.0
added results/model_comparison.csv +6 −0
@@ -0,0 +1,6 @@
1 +model,specification,r2,adj_r2,aic,bic,k,n,delta_r2_vs_A
2 +A,Structural only,0.45246270355216167,0.4522703602395922,26164.52607046235,26218.748582998487,6,17087,0.0
3 +B,+ Description length,0.46473670896482555,0.46451732591914097,25779.132645863716,25841.1012316193,7,17087,0.01227400541266388
4 +C,+ Semantic similarities (20),0.4961715700941399,0.4954037190286328,24782.976736903176,24992.12071382828,26,17087,0.04370886654197825
5 +D,Full model (B + C),0.5122282152161219,0.511456198205209,24231.56040203926,24448.450452183813,27,17087,0.05976551166396027
6 +E,Parsimonious (significant only),0.5117854070125694,0.5111273201791455,24239.065258388884,24424.971015655643,23,17087,0.05932270346040769
added results/robustness_results.csv +164 −0
@@ -0,0 +1,164 @@
1 +test,variable,statistic,value,p_value,note
2 +VIF,sim_luxe,VIF,22.7706,,VIF>10 indicates severe multicollinearity
3 +VIF,sim_entree_de_gamme,VIF,40.7301,,VIF>10 indicates severe multicollinearity
4 +VIF,sim_renove,VIF,41.6236,,VIF>10 indicates severe multicollinearity
5 +VIF,sim_a_renover,VIF,17.5734,,VIF>10 indicates severe multicollinearity
6 +VIF,sim_lumineux_spacieux,VIF,23.3201,,VIF>10 indicates severe multicollinearity
7 +VIF,sim_terrain_nature,VIF,8.5203,,VIF>10 indicates severe multicollinearity
8 +VIF,sim_vue_panoramique,VIF,8.812,,VIF>10 indicates severe multicollinearity
9 +VIF,sim_localisation_premium,VIF,12.3623,,VIF>10 indicates severe multicollinearity
10 +VIF,sim_tranquillite,VIF,12.6281,,VIF>10 indicates severe multicollinearity
11 +VIF,sim_revenu_investissement,VIF,13.9468,,VIF>10 indicates severe multicollinearity
12 +VIF,sim_garage_stationnement,VIF,4.5984,,VIF>10 indicates severe multicollinearity
13 +VIF,sim_sous_sol_amenage,VIF,10.8236,,VIF>10 indicates severe multicollinearity
14 +VIF,sim_moderne_contemporain,VIF,14.8909,,VIF>10 indicates severe multicollinearity
15 +VIF,sim_cachet_patrimonial,VIF,14.5753,,VIF>10 indicates severe multicollinearity
16 +VIF,sim_ecoefficace,VIF,13.0672,,VIF>10 indicates severe multicollinearity
17 +VIF,sim_urgence_motivation,VIF,29.1178,,VIF>10 indicates severe multicollinearity
18 +VIF,sim_familial,VIF,11.1171,,VIF>10 indicates severe multicollinearity
19 +VIF,sim_bord_eau,VIF,9.6265,,VIF>10 indicates severe multicollinearity
20 +VIF,sim_neuf_construction,VIF,18.3429,,VIF>10 indicates severe multicollinearity
21 +VIF,sim_piscine_amenagement,VIF,16.3326,,VIF>10 indicates severe multicollinearity
22 +Breusch-Pagan,Model D (global),LM,767.9686,0.0,HC3 justified
23 +Breusch-Pagan,Model D (global),F,29.733,0.0,
24 +QuantReg_tau0.25,sim_luxe,coeff,0.092785,6e-06,tau=0.25
25 +QuantReg_tau0.25,sim_entree_de_gamme,coeff,0.018237,0.508408,tau=0.25
26 +QuantReg_tau0.25,sim_renove,coeff,0.037206,0.188211,tau=0.25
27 +QuantReg_tau0.25,sim_a_renover,coeff,-0.090109,1e-06,tau=0.25
28 +QuantReg_tau0.25,sim_lumineux_spacieux,coeff,-0.093789,8e-06,tau=0.25
29 +QuantReg_tau0.25,sim_terrain_nature,coeff,0.104668,0.0,tau=0.25
30 +QuantReg_tau0.25,sim_vue_panoramique,coeff,0.011204,0.373229,tau=0.25
31 +QuantReg_tau0.25,sim_localisation_premium,coeff,-0.007672,0.611494,tau=0.25
32 +QuantReg_tau0.25,sim_tranquillite,coeff,-0.109597,0.0,tau=0.25
33 +QuantReg_tau0.25,sim_revenu_investissement,coeff,-0.013501,0.403534,tau=0.25
34 +QuantReg_tau0.25,sim_garage_stationnement,coeff,0.023716,0.010856,tau=0.25
35 +QuantReg_tau0.25,sim_sous_sol_amenage,coeff,0.002513,0.863057,tau=0.25
36 +QuantReg_tau0.25,sim_moderne_contemporain,coeff,0.148058,0.0,tau=0.25
37 +QuantReg_tau0.25,sim_cachet_patrimonial,coeff,0.012577,0.452404,tau=0.25
38 +QuantReg_tau0.25,sim_ecoefficace,coeff,-0.040865,0.009106,tau=0.25
39 +QuantReg_tau0.25,sim_urgence_motivation,coeff,-0.091447,8.7e-05,tau=0.25
40 +QuantReg_tau0.25,sim_familial,coeff,-0.121245,0.0,tau=0.25
41 +QuantReg_tau0.25,sim_bord_eau,coeff,-0.012354,0.355754,tau=0.25
42 +QuantReg_tau0.25,sim_neuf_construction,coeff,-0.067734,0.000255,tau=0.25
43 +QuantReg_tau0.25,sim_piscine_amenagement,coeff,0.092149,0.0,tau=0.25
44 +QuantReg_tau0.5,sim_luxe,coeff,0.130194,0.0,tau=0.5
45 +QuantReg_tau0.5,sim_entree_de_gamme,coeff,0.02816,0.236393,tau=0.5
46 +QuantReg_tau0.5,sim_renove,coeff,0.029361,0.221999,tau=0.5
47 +QuantReg_tau0.5,sim_a_renover,coeff,-0.064604,3.6e-05,tau=0.5
48 +QuantReg_tau0.5,sim_lumineux_spacieux,coeff,-0.0771,1.8e-05,tau=0.5
49 +QuantReg_tau0.5,sim_terrain_nature,coeff,0.097503,0.0,tau=0.5
50 +QuantReg_tau0.5,sim_vue_panoramique,coeff,0.01739,0.115965,tau=0.5
51 +QuantReg_tau0.5,sim_localisation_premium,coeff,-0.046855,0.00035,tau=0.5
52 +QuantReg_tau0.5,sim_tranquillite,coeff,-0.072087,0.0,tau=0.5
53 +QuantReg_tau0.5,sim_revenu_investissement,coeff,-0.017631,0.205206,tau=0.5
54 +QuantReg_tau0.5,sim_garage_stationnement,coeff,0.009516,0.23374,tau=0.5
55 +QuantReg_tau0.5,sim_sous_sol_amenage,coeff,-0.042981,0.000456,tau=0.5
56 +QuantReg_tau0.5,sim_moderne_contemporain,coeff,0.120851,0.0,tau=0.5
57 +QuantReg_tau0.5,sim_cachet_patrimonial,coeff,0.003009,0.832502,tau=0.5
58 +QuantReg_tau0.5,sim_ecoefficace,coeff,-0.061271,5e-06,tau=0.5
59 +QuantReg_tau0.5,sim_urgence_motivation,coeff,-0.045931,0.022372,tau=0.5
60 +QuantReg_tau0.5,sim_familial,coeff,-0.104553,0.0,tau=0.5
61 +QuantReg_tau0.5,sim_bord_eau,coeff,0.03904,0.000735,tau=0.5
62 +QuantReg_tau0.5,sim_neuf_construction,coeff,-0.078994,1e-06,tau=0.5
63 +QuantReg_tau0.5,sim_piscine_amenagement,coeff,0.035215,0.019383,tau=0.5
64 +QuantReg_tau0.75,sim_luxe,coeff,0.159658,0.0,tau=0.75
65 +QuantReg_tau0.75,sim_entree_de_gamme,coeff,0.101336,7.4e-05,tau=0.75
66 +QuantReg_tau0.75,sim_renove,coeff,0.051023,0.049064,tau=0.75
67 +QuantReg_tau0.75,sim_a_renover,coeff,-0.043708,0.009328,tau=0.75
68 +QuantReg_tau0.75,sim_lumineux_spacieux,coeff,-0.093022,1e-06,tau=0.75
69 +QuantReg_tau0.75,sim_terrain_nature,coeff,0.103853,0.0,tau=0.75
70 +QuantReg_tau0.75,sim_vue_panoramique,coeff,0.03871,0.001647,tau=0.75
71 +QuantReg_tau0.75,sim_localisation_premium,coeff,-0.077074,0.0,tau=0.75
72 +QuantReg_tau0.75,sim_tranquillite,coeff,-0.06393,6e-06,tau=0.75
73 +QuantReg_tau0.75,sim_revenu_investissement,coeff,-0.040171,0.008013,tau=0.75
74 +QuantReg_tau0.75,sim_garage_stationnement,coeff,0.01067,0.22168,tau=0.75
75 +QuantReg_tau0.75,sim_sous_sol_amenage,coeff,-0.077482,0.0,tau=0.75
76 +QuantReg_tau0.75,sim_moderne_contemporain,coeff,0.155206,0.0,tau=0.75
77 +QuantReg_tau0.75,sim_cachet_patrimonial,coeff,-0.016438,0.27749,tau=0.75
78 +QuantReg_tau0.75,sim_ecoefficace,coeff,-0.088825,0.0,tau=0.75
79 +QuantReg_tau0.75,sim_urgence_motivation,coeff,-0.038802,0.073303,tau=0.75
80 +QuantReg_tau0.75,sim_familial,coeff,-0.10807,0.0,tau=0.75
81 +QuantReg_tau0.75,sim_bord_eau,coeff,0.094023,0.0,tau=0.75
82 +QuantReg_tau0.75,sim_neuf_construction,coeff,-0.142198,0.0,tau=0.75
83 +QuantReg_tau0.75,sim_piscine_amenagement,coeff,-0.030493,0.059856,tau=0.75
84 +Winsorized_1pct,sim_luxe,coeff,0.121383,0.0,"full=0.133028, delta=-0.011645"
85 +Winsorized_1pct,sim_entree_de_gamme,coeff,0.075499,0.001135,"full=0.099091, delta=-0.023592"
86 +Winsorized_1pct,sim_renove,coeff,0.054028,0.029264,"full=0.050149, delta=0.003879"
87 +Winsorized_1pct,sim_a_renover,coeff,-0.081516,0.0,"full=-0.082677, delta=0.001161"
88 +Winsorized_1pct,sim_lumineux_spacieux,coeff,-0.075665,1e-05,"full=-0.093635, delta=0.01797"
89 +Winsorized_1pct,sim_terrain_nature,coeff,0.112752,0.0,"full=0.122461, delta=-0.009709"
90 +Winsorized_1pct,sim_vue_panoramique,coeff,0.024619,0.023624,"full=0.030032, delta=-0.005413"
91 +Winsorized_1pct,sim_localisation_premium,coeff,-0.061407,3e-06,"full=-0.062782, delta=0.001375"
92 +Winsorized_1pct,sim_tranquillite,coeff,-0.084233,0.0,"full=-0.099506, delta=0.015274"
93 +Winsorized_1pct,sim_revenu_investissement,coeff,-0.02238,0.134706,"full=-0.031591, delta=0.009211"
94 +Winsorized_1pct,sim_garage_stationnement,coeff,0.01799,0.028961,"full=0.018962, delta=-0.000972"
95 +Winsorized_1pct,sim_sous_sol_amenage,coeff,-0.040515,0.002921,"full=-0.036992, delta=-0.003523"
96 +Winsorized_1pct,sim_moderne_contemporain,coeff,0.148945,0.0,"full=0.151681, delta=-0.002736"
97 +Winsorized_1pct,sim_cachet_patrimonial,coeff,0.003331,0.806984,"full=0.010281, delta=-0.00695"
98 +Winsorized_1pct,sim_ecoefficace,coeff,-0.076995,0.0,"full=-0.081162, delta=0.004166"
99 +Winsorized_1pct,sim_urgence_motivation,coeff,-0.066107,0.000907,"full=-0.088969, delta=0.022862"
100 +Winsorized_1pct,sim_familial,coeff,-0.115979,0.0,"full=-0.123651, delta=0.007672"
101 +Winsorized_1pct,sim_bord_eau,coeff,0.046955,6e-05,"full=0.046184, delta=0.000771"
102 +Winsorized_1pct,sim_neuf_construction,coeff,-0.109556,0.0,"full=-0.110532, delta=0.000975"
103 +Winsorized_1pct,sim_piscine_amenagement,coeff,0.01077,0.472274,"full=0.024584, delta=-0.013814"
104 +Winsorized_1pct,Model D (global),R2,0.486655,,n=16746
105 +Bootstrap_1000,sim_luxe,boot_SE,0.018274,,"HC3_SE=0.01819, ratio=1.0046, CI=[0.09531,0.16559]"
106 +Bootstrap_1000,sim_entree_de_gamme,boot_SE,0.025253,,"HC3_SE=0.025104, ratio=1.0059, CI=[0.04966,0.14727]"
107 +Bootstrap_1000,sim_renove,boot_SE,0.024375,,"HC3_SE=0.026702, ratio=0.9129, CI=[-0.00459,0.09139]"
108 +Bootstrap_1000,sim_a_renover,boot_SE,0.016534,,"HC3_SE=0.016998, ratio=0.9727, CI=[-0.11369,-0.05055]"
109 +Bootstrap_1000,sim_lumineux_spacieux,boot_SE,0.018756,,"HC3_SE=0.018482, ratio=1.0148, CI=[-0.13029,-0.05775]"
110 +Bootstrap_1000,sim_terrain_nature,boot_SE,0.011472,,"HC3_SE=0.012221, ratio=0.9388, CI=[0.09936,0.14388]"
111 +Bootstrap_1000,sim_vue_panoramique,boot_SE,0.011258,,"HC3_SE=0.011619, ratio=0.9689, CI=[0.00791,0.05139]"
112 +Bootstrap_1000,sim_localisation_premium,boot_SE,0.013852,,"HC3_SE=0.013861, ratio=0.9994, CI=[-0.08837,-0.03427]"
113 +Bootstrap_1000,sim_tranquillite,boot_SE,0.013009,,"HC3_SE=0.01419, ratio=0.9168, CI=[-0.12354,-0.07242]"
114 +Bootstrap_1000,sim_revenu_investissement,boot_SE,0.01567,,"HC3_SE=0.016178, ratio=0.9686, CI=[-0.06356,-0.00397]"
115 +Bootstrap_1000,sim_garage_stationnement,boot_SE,0.008339,,"HC3_SE=0.008647, ratio=0.9644, CI=[0.00309,0.03474]"
116 +Bootstrap_1000,sim_sous_sol_amenage,boot_SE,0.013495,,"HC3_SE=0.014677, ratio=0.9194, CI=[-0.05943,-0.00654]"
117 +Bootstrap_1000,sim_moderne_contemporain,boot_SE,0.015403,,"HC3_SE=0.015508, ratio=0.9933, CI=[0.12173,0.1817]"
118 +Bootstrap_1000,sim_cachet_patrimonial,boot_SE,0.013751,,"HC3_SE=0.014671, ratio=0.9373, CI=[-0.01754,0.03592]"
119 +Bootstrap_1000,sim_ecoefficace,boot_SE,0.013284,,"HC3_SE=0.01365, ratio=0.9732, CI=[-0.10561,-0.05552]"
120 +Bootstrap_1000,sim_urgence_motivation,boot_SE,0.020651,,"HC3_SE=0.021572, ratio=0.9573, CI=[-0.12799,-0.04661]"
121 +Bootstrap_1000,sim_familial,boot_SE,0.012559,,"HC3_SE=0.012727, ratio=0.9868, CI=[-0.1504,-0.09947]"
122 +Bootstrap_1000,sim_bord_eau,boot_SE,0.012244,,"HC3_SE=0.012851, ratio=0.9528, CI=[0.02117,0.068]"
123 +Bootstrap_1000,sim_neuf_construction,boot_SE,0.017011,,"HC3_SE=0.01758, ratio=0.9676, CI=[-0.14149,-0.07541]"
124 +Bootstrap_1000,sim_piscine_amenagement,boot_SE,0.015536,,"HC3_SE=0.016137, ratio=0.9628, CI=[-0.00316,0.05715]"
125 +Lasso_CV,sim_luxe,coeff,-0.0,,dropped
126 +Lasso_CV,sim_entree_de_gamme,coeff,-0.0,,dropped
127 +Lasso_CV,sim_renove,coeff,-0.0,,dropped
128 +Lasso_CV,sim_a_renover,coeff,-0.02682,,selected
129 +Lasso_CV,sim_lumineux_spacieux,coeff,-0.0,,dropped
130 +Lasso_CV,sim_terrain_nature,coeff,0.0,,dropped
131 +Lasso_CV,sim_vue_panoramique,coeff,-0.0,,dropped
132 +Lasso_CV,sim_localisation_premium,coeff,-0.046391,,selected
133 +Lasso_CV,sim_tranquillite,coeff,-0.0,,dropped
134 +Lasso_CV,sim_revenu_investissement,coeff,-0.018791,,selected
135 +Lasso_CV,sim_garage_stationnement,coeff,-0.0,,dropped
136 +Lasso_CV,sim_sous_sol_amenage,coeff,-0.0,,dropped
137 +Lasso_CV,sim_moderne_contemporain,coeff,-0.0,,dropped
138 +Lasso_CV,sim_cachet_patrimonial,coeff,-0.0,,dropped
139 +Lasso_CV,sim_ecoefficace,coeff,-0.0,,dropped
140 +Lasso_CV,sim_urgence_motivation,coeff,-0.0,,dropped
141 +Lasso_CV,sim_familial,coeff,-0.026713,,selected
142 +Lasso_CV,sim_bord_eau,coeff,-0.0,,dropped
143 +Lasso_CV,sim_neuf_construction,coeff,-0.0,,dropped
144 +Lasso_CV,sim_piscine_amenagement,coeff,-0.0,,dropped
145 +ElasticNet_CV,sim_luxe,coeff,-0.0,,dropped
146 +ElasticNet_CV,sim_entree_de_gamme,coeff,-0.0,,dropped
147 +ElasticNet_CV,sim_renove,coeff,-0.0,,dropped
148 +ElasticNet_CV,sim_a_renover,coeff,-0.026808,,"selected, l1=0.99"
149 +ElasticNet_CV,sim_lumineux_spacieux,coeff,-0.0,,dropped
150 +ElasticNet_CV,sim_terrain_nature,coeff,0.0,,dropped
151 +ElasticNet_CV,sim_vue_panoramique,coeff,-0.0,,dropped
152 +ElasticNet_CV,sim_localisation_premium,coeff,-0.046368,,"selected, l1=0.99"
153 +ElasticNet_CV,sim_tranquillite,coeff,-0.0,,dropped
154 +ElasticNet_CV,sim_revenu_investissement,coeff,-0.018826,,"selected, l1=0.99"
155 +ElasticNet_CV,sim_garage_stationnement,coeff,-0.0,,dropped
156 +ElasticNet_CV,sim_sous_sol_amenage,coeff,-0.0,,dropped
157 +ElasticNet_CV,sim_moderne_contemporain,coeff,-0.0,,dropped
158 +ElasticNet_CV,sim_cachet_patrimonial,coeff,-0.0,,dropped
159 +ElasticNet_CV,sim_ecoefficace,coeff,-0.0,,dropped
160 +ElasticNet_CV,sim_urgence_motivation,coeff,-0.0,,dropped
161 +ElasticNet_CV,sim_familial,coeff,-0.026718,,"selected, l1=0.99"
162 +ElasticNet_CV,sim_bord_eau,coeff,-0.0,,dropped
163 +ElasticNet_CV,sim_neuf_construction,coeff,-0.0,,dropped
164 +ElasticNet_CV,sim_piscine_amenagement,coeff,-0.0,,dropped
added results/similarity_stats.csv +21 −0
@@ -0,0 +1,21 @@
1 +,count,mean,std,min,25%,50%,75%,max,corr_log_price
2 +Luxury,17087.0,0.3368127592865921,0.15694362109461882,-0.029989816,0.19254429499999998,0.37594822,0.46762895,0.692075,-0.2337968095666754
3 +Entry-Level,17087.0,0.31306725551242054,0.16749897542735695,-0.08511273,0.14523222,0.385215,0.457089945,0.6603542,-0.2703875863871995
4 +Renovated,17087.0,0.35163431022614033,0.16771201040710013,-0.05604358,0.188791545,0.41066957,0.49535947,0.7533703,-0.2500175365750065
5 +Needs Renovation,17087.0,0.32055725619986125,0.13813069826206623,-0.082866654,0.20229811,0.36692798,0.435327615,0.65222603,-0.2507390776744787
6 +Bright & Spacious,17087.0,0.35706155292049163,0.1526272732429096,-0.10424077,0.23038871,0.40036976,0.48361212,0.6723022,-0.22838253226812757
7 +Land & Nature,17087.0,0.33019139520730967,0.11592953569855854,-0.021478772,0.24469626500000002,0.33739355,0.41321410000000003,0.70141184,-0.17416509469425526
8 +Panoramic View,17087.0,0.3158151019463838,0.1201802895590174,-0.114003584,0.23395886999999999,0.34317952,0.40676298,0.61307484,-0.18514195686203655
9 +Premium Location,17087.0,0.3214841909575724,0.14118004039307103,-0.025190443,0.18308983,0.36381766,0.44036834,0.6931879,-0.2893234018178692
10 +Quiet & Peaceful,17087.0,0.358280338175256,0.11137093505112176,-0.06460845,0.27255477,0.3761769,0.4454823,0.6302305,-0.2280383495965432
11 +Income/Investment,17087.0,0.358294740898531,0.12572034497588438,-0.029236501,0.242870825,0.3979885,0.46193213499999997,0.6605341,-0.2754437517394351
12 +Garage & Parking,17087.0,0.3103307920218801,0.1116239718517591,-0.07838718,0.2281446,0.3170391,0.38917871,0.6850351,-0.21041569849939162
13 +Finished Basement,17087.0,0.3052002113895043,0.13859674166712657,-0.06942309,0.18473543,0.32378525,0.41383928000000003,0.7084718,-0.22838180602568187
14 +Modern/Contemporary,17087.0,0.36491924047953644,0.11546476334151118,-0.044310126,0.28822696000000003,0.38244715,0.44983825,0.6712736,-0.1872993318470735
15 +Heritage/Character,17087.0,0.3661237309976239,0.14263945319240887,-0.08970999,0.24300555000000001,0.39942157,0.485041035,0.7148477,-0.23467459387674028
16 +Energy Efficient,17087.0,0.29820180354291975,0.13448762901483347,-0.05774025,0.176577345,0.32659054,0.40915789999999996,0.6483617,-0.2335807609635236
17 +Motivated Seller,17087.0,0.2538904059694071,0.16511008484652123,-0.15081203,0.08241085000000001,0.32629922,0.395234525,0.58567154,-0.2789656937507642
18 +Family-Friendly,17087.0,0.34865389449412826,0.14223063870929087,-0.07987824,0.24584254500000002,0.37126693,0.46098793,0.7220392,-0.23142647107817926
19 +Waterfront,17087.0,0.33102863939709076,0.15079277023591114,-0.11662539,0.2038537,0.37008253,0.45050258499999996,0.72783756,-0.223369218982517
20 +New Construction,17087.0,0.3717061332362263,0.12168455303622348,-0.02023823,0.28365375000000004,0.3961436,0.4653973,0.6941813,-0.2399581287316102
21 +Pool & Landscaping,17087.0,0.34803306189049577,0.15804982563861822,-0.06978037,0.202132945,0.3818322,0.480052425,0.7559897,-0.23996553784568034
added scripts/01_prepare_data.py +31 −0
@@ -0,0 +1,31 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +#
4 +"""Step 1 — Extract the house sample from data/raw/louka.db.
5 +
6 +Output: data/processed/houses.parquet (n = 17,087 rows expected).
7 +"""
8 +
9 +import sys
10 +from pathlib import Path
11 +
12 +sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
13 +
14 +from src import config
15 +from src.data import load_houses
16 +
17 +
18 +def main():
19 + print(f"Reading {config.DB_PATH} ...")
20 + df = load_houses(config.DB_PATH)
21 + print(f" {len(df):,} houses with a valid price and description")
22 + print(f" price: min=${df['price'].min():,.0f} "
23 + f"median=${df['price'].median():,.0f} max=${df['price'].max():,.0f}")
24 +
25 + config.DATA_PROCESSED.mkdir(parents=True, exist_ok=True)
26 + df.to_parquet(config.HOUSES_PARQUET, index=False)
27 + print(f" -> {config.HOUSES_PARQUET}")
28 +
29 +
30 +if __name__ == "__main__":
31 + main()
added scripts/02_similarities.py +55 −0
@@ -0,0 +1,55 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +#
4 +"""Step 2 — Embed descriptions and compute the 20 cosine-similarity features.
5 +
6 +Inputs : data/processed/houses.parquet
7 +Outputs: data/processed/embeddings_maisons.npy (17,087 x 384, cached)
8 + data/processed/sim_matrix_maisons.npy (17,087 x 20)
9 + data/processed/hedonic_maison_results.csv (analysis dataset)
10 +
11 +Pass --force to re-encode the embeddings even if a valid cache exists.
12 +"""
13 +
14 +import sys
15 +from pathlib import Path
16 +
17 +import numpy as np
18 +import pandas as pd
19 +
20 +sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
21 +
22 +from src import config
23 +from src.embeddings import (add_similarity_columns, encode_references,
24 + load_or_encode_remarks, similarity_features)
25 +from src.references import SIM_COLS
26 +
27 +
28 +def main(force=False):
29 + df = pd.read_parquet(config.HOUSES_PARQUET)
30 + print(f"{len(df):,} houses loaded")
31 +
32 + print("Encoding 20 reference descriptions ...")
33 + _, ref_embeddings = encode_references()
34 +
35 + print("Encoding listing descriptions (cache: data/processed) ...")
36 + prop_embeddings, from_cache = load_or_encode_remarks(
37 + df["remarks"], force=force
38 + )
39 + print(f" embeddings {prop_embeddings.shape} "
40 + f"({'loaded from cache' if from_cache else 'freshly encoded'})")
41 +
42 + sim_matrix = similarity_features(prop_embeddings, ref_embeddings)
43 + np.save(config.SIM_MATRIX_NPY, sim_matrix)
44 + add_similarity_columns(df, sim_matrix)
45 +
46 + export_cols = ["id", "price", "log_price", "bedrooms", "bathrooms",
47 + "half_baths", "parking", "stories", "land_size",
48 + "remarks_length"] + SIM_COLS
49 + df[export_cols].to_csv(config.ANALYSIS_CSV, index=False)
50 + print(f" -> {config.SIM_MATRIX_NPY}")
51 + print(f" -> {config.ANALYSIS_CSV}")
52 +
53 +
54 +if __name__ == "__main__":
55 + main(force="--force" in sys.argv)
added scripts/03_models.py +104 −0
@@ -0,0 +1,104 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +#
4 +"""Step 3 — Fit hedonic models A-E and export the paper's result tables.
5 +
6 +Inputs : data/processed/hedonic_maison_results.csv
7 +Outputs: results/model_comparison.csv (Table 4 of the paper)
8 + results/coefficients_model_D.csv (full model, Table 5/6 source)
9 + results/coefficients_model_E.csv (parsimonious model)
10 + results/descriptive_stats.csv (Table 2 source)
11 + results/similarity_stats.csv (Table 3 source)
12 +"""
13 +
14 +import sys
15 +from pathlib import Path
16 +
17 +import numpy as np
18 +import pandas as pd
19 +
20 +sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
21 +
22 +from src import config
23 +from src.models import fit_all_models
24 +from src.references import ENGLISH_LABELS, SIM_COLS
25 +
26 +MODEL_NAMES = {
27 + "A": "Structural only",
28 + "B": "+ Description length",
29 + "C": "+ Semantic similarities (20)",
30 + "D": "Full model (B + C)",
31 + "E": "Parsimonious (significant only)",
32 +}
33 +
34 +
35 +def coefficient_table(model, variables):
36 + """Coefficients with HC3 errors, p-values and percentage price impacts."""
37 + rows = []
38 + for v in variables:
39 + if v not in model.params:
40 + continue
41 + coef = model.params[v]
42 + rows.append({
43 + "variable": v,
44 + "label": ENGLISH_LABELS.get(v.replace("sim_", ""), v),
45 + "coefficient": coef,
46 + "std_error": model.bse[v],
47 + "t_value": model.tvalues[v],
48 + "p_value": model.pvalues[v],
49 + "impact_pct": (np.exp(coef) - 1) * 100,
50 + })
51 + return pd.DataFrame(rows)
52 +
53 +
54 +def main():
55 + df = pd.read_csv(config.ANALYSIS_CSV)
56 + print(f"{len(df):,} observations")
57 +
58 + models, specs, sig_sims = fit_all_models(df)
59 + config.RESULTS_DIR.mkdir(parents=True, exist_ok=True)
60 +
61 + comparison = pd.DataFrame({
62 + "model": list(models),
63 + "specification": [MODEL_NAMES[k] for k in models],
64 + "r2": [m.rsquared for m in models.values()],
65 + "adj_r2": [m.rsquared_adj for m in models.values()],
66 + "aic": [m.aic for m in models.values()],
67 + "bic": [m.bic for m in models.values()],
68 + "k": [int(m.df_model) for m in models.values()],
69 + "n": [int(m.nobs) for m in models.values()],
70 + })
71 + comparison["delta_r2_vs_A"] = comparison["r2"] - models["A"].rsquared
72 + comparison.to_csv(config.RESULTS_DIR / "model_comparison.csv", index=False)
73 + print(comparison.round(4).to_string(index=False))
74 +
75 + # F-test of the semantic block (D vs A), as reported in the paper
76 + mA, mD = models["A"], models["D"]
77 + df_diff = mD.df_model - mA.df_model
78 + f_stat = ((mA.ssr - mD.ssr) / df_diff) / (mD.ssr / mD.df_resid)
79 + from scipy import stats
80 + f_pval = 1 - stats.f.cdf(f_stat, df_diff, mD.df_resid)
81 + print(f"\nJoint F-test (D vs A): F = {f_stat:.2f}, p = {f_pval:.2e}")
82 + print(f"Significant similarities in D (p<0.05): {len(sig_sims)}/{len(SIM_COLS)}")
83 +
84 + coefficient_table(mD, specs["D"]).to_csv(
85 + config.RESULTS_DIR / "coefficients_model_D.csv", index=False)
86 + coefficient_table(models["E"], specs["E"]).to_csv(
87 + config.RESULTS_DIR / "coefficients_model_E.csv", index=False)
88 +
89 + desc_vars = ["price", "log_price"] + config.STRUCTURAL_VARS + ["remarks_length"]
90 + df[desc_vars].describe().T.to_csv(config.RESULTS_DIR / "descriptive_stats.csv")
91 +
92 + sim_stats = df[SIM_COLS].describe().T
93 + sim_stats.index = [ENGLISH_LABELS[c.replace("sim_", "")] for c in sim_stats.index]
94 + sim_stats["corr_log_price"] = [
95 + df[c].corr(df["log_price"]) for c in SIM_COLS
96 + ]
97 + sim_stats.to_csv(config.RESULTS_DIR / "similarity_stats.csv")
98 +
99 + print(f"\n-> {config.RESULTS_DIR}/model_comparison.csv, coefficients_model_D.csv, "
100 + f"coefficients_model_E.csv, descriptive_stats.csv, similarity_stats.csv")
101 +
102 +
103 +if __name__ == "__main__":
104 + main()
added scripts/04_robustness.py +146 −0
@@ -0,0 +1,146 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +#
4 +"""Step 4 — Robustness checks for the full model (D).
5 +
6 +Faithful port of hedonic_maison.py step 12 (same seeds, same estimators):
7 + (a) VIF for all regressors
8 + (b) Breusch-Pagan heteroskedasticity test
9 + (c) quantile regressions (tau = 0.25, 0.50, 0.75)
10 + (d) winsorized re-estimation (price trimmed at 1%/99%)
11 + (e) bootstrap standard errors (B = 1,000, seed 42)
12 + (f) Lasso / Elastic Net variable selection (5-fold CV, seed 42)
13 +
14 +Inputs : data/processed/hedonic_maison_results.csv
15 +Outputs: results/robustness_results.csv (long format: test, variable, value)
16 +"""
17 +
18 +import sys
19 +from pathlib import Path
20 +
21 +import numpy as np
22 +import pandas as pd
23 +import statsmodels.api as sm
24 +from sklearn.linear_model import ElasticNetCV, LassoCV
25 +from statsmodels.stats.diagnostic import het_breuschpagan
26 +from statsmodels.stats.outliers_influence import variance_inflation_factor
27 +
28 +sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
29 +
30 +from src import config
31 +from src.models import fit_all_models, fit_ols, standardized_design
32 +from src.references import SIM_COLS
33 +
34 +
35 +def main():
36 + df = pd.read_csv(config.ANALYSIS_CSV)
37 + y = df["log_price"]
38 + models, specs, sig_sims = fit_all_models(df)
39 + mD = models["D"]
40 + full_vars = specs["D"]
41 +
42 + rows = []
43 + X_full = standardized_design(df, full_vars)
44 +
45 + # (a) VIF ---------------------------------------------------------------
46 + print("VIF ...")
47 + for i, col in enumerate(X_full.columns):
48 + if col == "const":
49 + continue
50 + vif = variance_inflation_factor(X_full.values, i)
51 + if col.startswith("sim_"):
52 + rows.append({"test": "VIF", "variable": col, "statistic": "VIF",
53 + "value": round(vif, 4), "p_value": np.nan,
54 + "note": "VIF>10 indicates severe multicollinearity"})
55 +
56 + # (b) Breusch-Pagan -----------------------------------------------------
57 + print("Breusch-Pagan ...")
58 + ols_plain = sm.OLS(y, X_full).fit()
59 + bp_stat, bp_pval, bp_f, bp_fp = het_breuschpagan(ols_plain.resid, X_full)
60 + rows.append({"test": "Breusch-Pagan", "variable": "Model D (global)",
61 + "statistic": "LM", "value": round(bp_stat, 4),
62 + "p_value": round(bp_pval, 6),
63 + "note": "HC3 justified" if bp_pval < 0.05 else "Homoskedastic"})
64 + rows.append({"test": "Breusch-Pagan", "variable": "Model D (global)",
65 + "statistic": "F", "value": round(bp_f, 4),
66 + "p_value": round(bp_fp, 6), "note": ""})
67 +
68 + # (c) Quantile regressions ----------------------------------------------
69 + print("Quantile regressions ...")
70 + for tau in (0.25, 0.50, 0.75):
71 + qr = sm.QuantReg(y, X_full).fit(q=tau, max_iter=5000)
72 + for col in SIM_COLS:
73 + rows.append({"test": f"QuantReg_tau{tau}", "variable": col,
74 + "statistic": "coeff",
75 + "value": round(qr.params.get(col, np.nan), 6),
76 + "p_value": round(qr.pvalues.get(col, 1.0), 6),
77 + "note": f"tau={tau}"})
78 +
79 + # (d) Winsorized regression ----------------------------------------------
80 + print("Winsorized regression ...")
81 + p01, p99 = df["price"].quantile([0.01, 0.99])
82 + df_w = df[(df["price"] >= p01) & (df["price"] <= p99)]
83 + mD_w = fit_ols(df_w, full_vars)
84 + for col in SIM_COLS:
85 + c_full, c_wins = mD.params[col], mD_w.params[col]
86 + rows.append({"test": "Winsorized_1pct", "variable": col,
87 + "statistic": "coeff", "value": round(c_wins, 6),
88 + "p_value": round(mD_w.pvalues[col], 6),
89 + "note": f"full={round(c_full, 6)}, delta={round(c_wins - c_full, 6)}"})
90 + rows.append({"test": "Winsorized_1pct", "variable": "Model D (global)",
91 + "statistic": "R2", "value": round(mD_w.rsquared, 6),
92 + "p_value": np.nan, "note": f"n={int(mD_w.nobs)}"})
93 +
94 + # (e) Bootstrap SEs (seed and loop identical to the original) ------------
95 + print("Bootstrap (B=1000) ...")
96 + np.random.seed(config.SEED)
97 + B, n = 1000, len(df)
98 + boot_coefs = {col: [] for col in SIM_COLS}
99 + for _ in range(B):
100 + idx = np.random.choice(n, size=n, replace=True)
101 + mb = sm.OLS(y.iloc[idx], X_full.iloc[idx]).fit()
102 + for col in SIM_COLS:
103 + boot_coefs[col].append(mb.params[col])
104 + for col in SIM_COLS:
105 + bc = np.array(boot_coefs[col])
106 + hc3_se = mD.bse[col]
107 + ci_lo, ci_hi = np.percentile(bc, [2.5, 97.5])
108 + rows.append({"test": "Bootstrap_1000", "variable": col,
109 + "statistic": "boot_SE", "value": round(bc.std(), 6),
110 + "p_value": np.nan,
111 + "note": (f"HC3_SE={round(hc3_se, 6)}, "
112 + f"ratio={round(bc.std() / hc3_se, 4)}, "
113 + f"CI=[{round(ci_lo, 5)},{round(ci_hi, 5)}]")})
114 +
115 + # (f) Lasso / Elastic Net -------------------------------------------------
116 + print("Lasso / Elastic Net ...")
117 + X_lasso = X_full.drop(columns="const")
118 + lasso = LassoCV(cv=5, random_state=config.SEED, max_iter=10000).fit(X_lasso, y)
119 + lasso_coefs = pd.Series(lasso.coef_, index=X_lasso.columns)
120 + for col in SIM_COLS:
121 + c = lasso_coefs[col]
122 + rows.append({"test": "Lasso_CV", "variable": col, "statistic": "coeff",
123 + "value": round(c, 6), "p_value": np.nan,
124 + "note": "selected" if abs(c) > 0 else "dropped"})
125 +
126 + enet = ElasticNetCV(l1_ratio=[0.1, 0.5, 0.7, 0.9, 0.95, 0.99],
127 + cv=5, random_state=config.SEED, max_iter=10000).fit(X_lasso, y)
128 + enet_coefs = pd.Series(enet.coef_, index=X_lasso.columns)
129 + for col in SIM_COLS:
130 + c = enet_coefs[col]
131 + rows.append({"test": "ElasticNet_CV", "variable": col, "statistic": "coeff",
132 + "value": round(c, 6), "p_value": np.nan,
133 + "note": f"selected, l1={enet.l1_ratio_}" if abs(c) > 0 else "dropped"})
134 +
135 + config.RESULTS_DIR.mkdir(parents=True, exist_ok=True)
136 + out = config.RESULTS_DIR / "robustness_results.csv"
137 + pd.DataFrame(rows).to_csv(out, index=False)
138 + print(f"-> {out} ({len(rows)} rows)")
139 +
140 + n_lasso = int((lasso_coefs[SIM_COLS].abs() > 0).sum())
141 + print(f"Lasso alpha={lasso.alpha_:.6f}, semantic features kept: {n_lasso}/20")
142 + print(f"Winsorized R2={mD_w.rsquared:.4f} (full: {mD.rsquared:.4f})")
143 +
144 +
145 +if __name__ == "__main__":
146 + main()
added scripts/05_figures.py +357 −0
@@ -0,0 +1,357 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +#
4 +"""Step 5 — Generate all paper figures (fig1-fig11).
5 +
6 +Faithful port of paper_figures_v2.py: same layouts, colors and parameters,
7 +but reading the pipeline's analysis dataset instead of re-encoding the
8 +embeddings, and writing into the repository's figures/ directory.
9 +
10 +Inputs : data/processed/hedonic_maison_results.csv
11 +Outputs: figures/fig{1..11}_*.pdf and .png
12 +"""
13 +
14 +import sys
15 +import warnings
16 +from pathlib import Path
17 +
18 +import matplotlib
19 +import numpy as np
20 +import pandas as pd
21 +
22 +matplotlib.use("Agg")
23 +import matplotlib.pyplot as plt
24 +from matplotlib.patches import FancyBboxPatch
25 +from scipy.stats import probplot
26 +
27 +sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
28 +
29 +from src import config
30 +from src.models import fit_all_models
31 +from src.references import ENGLISH_LABELS, REFERENCES, SIM_COLS
32 +
33 +warnings.filterwarnings("ignore")
34 +
35 +plt.rcParams.update({
36 + "font.family": "serif",
37 + "font.serif": ["Times New Roman", "Times", "DejaVu Serif"],
38 + "font.size": 10,
39 + "axes.labelsize": 11,
40 + "axes.titlesize": 12,
41 + "xtick.labelsize": 9,
42 + "ytick.labelsize": 9,
43 + "legend.fontsize": 9,
44 + "figure.dpi": 300,
45 + "savefig.dpi": 300,
46 + "savefig.bbox": "tight",
47 + "savefig.pad_inches": 0.08,
48 + "axes.spines.top": False,
49 + "axes.spines.right": False,
50 +})
51 +
52 +LABELS = [ENGLISH_LABELS[slug] for slug in REFERENCES]
53 +
54 +
55 +def save(fig_name):
56 + plt.tight_layout()
57 + for ext in ("pdf", "png"):
58 + plt.savefig(config.FIGURES_DIR / f"{fig_name}.{ext}")
59 + plt.close()
60 + print(f" {fig_name}")
61 +
62 +
63 +def fig1_model_comparison(models):
64 + fig, ax = plt.subplots(figsize=(6.2, 3.8))
65 + names = ["Model A\nStructural", "Model B\n+ Length", "Model C\n+ Semantic",
66 + "Model D\nFull", "Model E\nParsim."]
67 + r2v = [m.rsquared for m in models.values()]
68 + r2a = [m.rsquared_adj for m in models.values()]
69 + x = np.arange(len(names))
70 + b1 = ax.bar(x - 0.15, r2v, 0.28, label=r"$R^2$", color="#1565C0", alpha=0.9)
71 + ax.bar(x + 0.15, r2a, 0.28, label=r"Adj. $R^2$", color="#FB8C00", alpha=0.9)
72 + for b, v in zip(b1, r2v):
73 + ax.text(b.get_x() + b.get_width() / 2, b.get_height() + 0.006,
74 + f"{v:.3f}", ha="center", fontsize=7.5)
75 + ax.set_xticks(x)
76 + ax.set_xticklabels(names, fontsize=8)
77 + ax.set_ylabel(r"$R^2$")
78 + ax.set_ylim(0, 0.58)
79 + ax.legend(loc="upper left", framealpha=0.9)
80 + ax.axhline(models["A"].rsquared, color="grey", ls="--", lw=0.7, alpha=0.5)
81 + save("fig1_model_comparison")
82 +
83 +
84 +def fig2_coefficient_plot(mD):
85 + fig, ax = plt.subplots(figsize=(6.2, 7))
86 + conf = mD.conf_int()
87 + sd = []
88 + for v, label in zip(SIM_COLS, LABELS):
89 + sd.append((label, mD.params[v], conf.loc[v, 0], conf.loc[v, 1],
90 + mD.pvalues[v], (np.exp(mD.params[v]) - 1) * 100))
91 + sd.sort(key=lambda t: t[1])
92 + names = [d[0] for d in sd]
93 + coefs = [d[1] for d in sd]
94 + pvs = [d[4] for d in sd]
95 + yp = np.arange(len(names))
96 + cols = ["#C62828" if c < 0 and p < 0.05 else
97 + "#2E7D32" if c > 0 and p < 0.05 else "#BDBDBD"
98 + for c, p in zip(coefs, pvs)]
99 + ax.axvline(0, color="black", lw=0.8)
100 + ax.barh(yp, coefs, color=cols, height=0.55, alpha=0.85,
101 + edgecolor="white", lw=0.4)
102 + for i, (name, c, cl, ch, p, imp) in enumerate(sd):
103 + ax.plot([cl, ch], [yp[i], yp[i]], color="#333", lw=1)
104 + st = "***" if p < 0.001 else "**" if p < 0.01 else "*" if p < 0.05 else ""
105 + if st:
106 + off = 0.004 if c >= 0 else -0.004
107 + ha = "left" if c >= 0 else "right"
108 + ax.text(c + off, yp[i], f"{imp:+.1f}%{st}", va="center", ha=ha,
109 + fontsize=7.5)
110 + ax.set_yticks(yp)
111 + ax.set_yticklabels(names, fontsize=9)
112 + ax.set_xlabel("Standardized coefficient (log-price)")
113 + save("fig2_coefficient_plot")
114 +
115 +
116 +def fig3_similarity_distributions(df):
117 + fig, ax = plt.subplots(figsize=(6.2, 5.5))
118 + bd = [df[c].values for c in SIM_COLS]
119 + order = np.argsort([np.median(d) for d in bd])[::-1]
120 + bd = [bd[i] for i in order]
121 + bl = [LABELS[i] for i in order]
122 + bp = ax.boxplot(bd, vert=False, patch_artist=True, widths=0.55,
123 + flierprops=dict(marker=".", markersize=1.5, alpha=0.2),
124 + medianprops=dict(color="black", lw=1.2))
125 + cm = plt.cm.viridis
126 + for i, patch in enumerate(bp["boxes"]):
127 + patch.set_facecolor(cm(i / len(bp["boxes"])))
128 + patch.set_alpha(0.7)
129 + ax.set_yticklabels(bl, fontsize=8)
130 + ax.set_xlabel("Cosine similarity")
131 + save("fig3_similarity_distributions")
132 +
133 +
134 +def fig4_quintile_heatmap(df):
135 + df = df.copy()
136 + df["pq"] = pd.qcut(df["price"], q=5,
137 + labels=["Q1 (Low)", "Q2", "Q3 (Med)", "Q4", "Q5 (High)"])
138 + profile = df.groupby("pq", observed=False)[SIM_COLS].mean()
139 + fig, ax = plt.subplots(figsize=(6.2, 6.5))
140 + im = ax.imshow(profile.T.values, aspect="auto", cmap="RdYlGn",
141 + interpolation="nearest")
142 + ax.set_xticks(range(5))
143 + ax.set_xticklabels(profile.index, fontsize=9)
144 + ax.set_yticks(range(len(LABELS)))
145 + ax.set_yticklabels(LABELS, fontsize=8)
146 + ax.set_xlabel("Price quintile")
147 + for i in range(len(LABELS)):
148 + for j in range(5):
149 + v = profile.T.values[i, j]
150 + ax.text(j, i, f"{v:.3f}", ha="center", va="center", fontsize=6.5,
151 + color="white" if v < 0.25 or v > 0.40 else "black")
152 + plt.colorbar(im, ax=ax, shrink=0.7, label="Mean cosine similarity", pad=0.02)
153 + save("fig4_quintile_heatmap")
154 +
155 +
156 +def fig5_correlation_matrix(df):
157 + corr = df[SIM_COLS].corr().values
158 + mask = np.triu(np.ones_like(corr, dtype=bool), k=1)
159 + corr_show = np.where(mask, np.nan, corr)
160 + fig, ax = plt.subplots(figsize=(6.5, 6))
161 + im = ax.imshow(corr_show, cmap="RdBu_r", vmin=-0.1, vmax=1.0,
162 + interpolation="nearest")
163 + ax.set_xticks(range(len(LABELS)))
164 + ax.set_xticklabels(LABELS, rotation=55, ha="right", fontsize=7)
165 + ax.set_yticks(range(len(LABELS)))
166 + ax.set_yticklabels(LABELS, fontsize=7)
167 + for i in range(len(LABELS)):
168 + for j in range(i + 1):
169 + v = corr[i, j]
170 + ax.text(j, i, f"{v:.2f}", ha="center", va="center", fontsize=5.5,
171 + color="white" if abs(v) > 0.65 else "black")
172 + plt.colorbar(im, ax=ax, shrink=0.7, label="Pearson r", pad=0.02)
173 + save("fig5_correlation_matrix")
174 +
175 +
176 +def fig6_scatter_plots(df):
177 + fig, axes = plt.subplots(2, 2, figsize=(6.2, 5.5))
178 + pairs = [
179 + ("sim_luxe", "Luxury", "#7B1FA2"),
180 + ("sim_a_renover", "Needs Renovation", "#C62828"),
181 + ("sim_moderne_contemporain", "Modern/Contemporary", "#1565C0"),
182 + ("sim_urgence_motivation", "Motivated Seller", "#E65100"),
183 + ]
184 + samp = df.sample(min(2500, len(df)), random_state=config.SEED)
185 + for ax, (col, label, color) in zip(axes.flat, pairs):
186 + ax.scatter(samp[col], samp["log_price"], alpha=0.12, s=5, c=color,
187 + edgecolors="none")
188 + z = np.polyfit(df[col], df["log_price"], 1)
189 + xr = np.linspace(df[col].min(), df[col].max(), 100)
190 + ax.plot(xr, np.poly1d(z)(xr), color="black", lw=1.8)
191 + r = df[col].corr(df["log_price"])
192 + ax.set_xlabel(f"Sim: {label}", fontsize=8)
193 + ax.set_ylabel("log(Price)", fontsize=8)
194 + ax.set_title(f"{label} (r={r:.3f})", fontsize=9)
195 + ax.tick_params(labelsize=7)
196 + plt.tight_layout(h_pad=1.2, w_pad=0.8)
197 + for ext in ("pdf", "png"):
198 + plt.savefig(config.FIGURES_DIR / f"fig6_scatter_plots.{ext}")
199 + plt.close()
200 + print(" fig6_scatter_plots")
201 +
202 +
203 +def fig7_methodology(n_obs):
204 + fig, ax = plt.subplots(figsize=(6.2, 2.8))
205 + ax.set_xlim(0, 12)
206 + ax.set_ylim(0, 3.5)
207 + ax.axis("off")
208 + boxes = [
209 + (1.2, 1.75, f"Property\nListings\n(n={n_obs:,})", "#E3F2FD"),
210 + (3.6, 1.75, "Sentence\nEmbeddings\n(384-d)", "#E8F5E9"),
211 + (6.0, 1.75, "Cosine\nSimilarity\n(20 refs)", "#FFF3E0"),
212 + (8.4, 1.75, "Hedonic\nOLS Model", "#F3E5F5"),
213 + (10.8, 1.75, "Implicit\nPrice\nEstimates", "#FFEBEE"),
214 + ]
215 + for x, yy, text, color in boxes:
216 + ax.add_patch(FancyBboxPatch((x - 0.65, yy - 0.65), 1.3, 1.3,
217 + boxstyle="round,pad=0.08",
218 + facecolor=color, edgecolor="#444", lw=1.3))
219 + ax.text(x, yy, text, ha="center", va="center", fontsize=7.5,
220 + fontweight="bold")
221 + for i in range(len(boxes) - 1):
222 + ax.annotate("", xy=(boxes[i + 1][0] - 0.7, 1.75),
223 + xytext=(boxes[i][0] + 0.7, 1.75),
224 + arrowprops=dict(arrowstyle="->", color="#444", lw=1.8))
225 + labs = [
226 + (2.4, 0.55, "PublicRemarks\nextraction"),
227 + (4.8, 0.55, "all-MiniLM-L6-v2\ntransformer"),
228 + (7.2, 0.55, "Reference-based\nfeatures"),
229 + (9.6, 0.55, "log(P) = βX+γS+ε"),
230 + ]
231 + for x, yy, text in labs:
232 + ax.text(x, yy, text, ha="center", va="center", fontsize=7,
233 + style="italic", color="#666")
234 + save("fig7_methodology")
235 +
236 +
237 +def fig8_r2_decomposition(models):
238 + fig, ax = plt.subplots(figsize=(4.5, 4))
239 + mA, mB, mD = models["A"], models["B"], models["D"]
240 + comps = [
241 + ("Structural variables", mA.rsquared, "#1565C0"),
242 + ("Text length", mB.rsquared - mA.rsquared, "#43A047"),
243 + ("Semantic similarities", mD.rsquared - mB.rsquared, "#FB8C00"),
244 + ]
245 + bot = 0
246 + for lab, val, col in comps:
247 + ax.bar(0, val, bottom=bot, color=col, width=0.5, edgecolor="white",
248 + lw=0.5, label=f"{lab}: {val:.4f}")
249 + if val > 0.008:
250 + ax.text(0, bot + val / 2, f"{val:.4f}\n({val / mD.rsquared * 100:.1f}%)",
251 + ha="center", va="center", fontsize=8, fontweight="bold",
252 + color="white")
253 + bot += val
254 + ax.set_ylabel(r"$R^2$")
255 + ax.set_ylim(0, 0.58)
256 + ax.set_xticks([0])
257 + ax.set_xticklabels(["Full Model (D)"], fontsize=9)
258 + ax.legend(loc="upper left", fontsize=8, framealpha=0.9)
259 + save("fig8_r2_decomposition")
260 +
261 +
262 +def fig9_price_distribution(df):
263 + fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(6.2, 3))
264 + ax1.hist(df["price"] / 1000, bins=80, color="#1565C0", alpha=0.8,
265 + edgecolor="white", lw=0.3)
266 + ax1.set_xlabel("Price (\\$000s)")
267 + ax1.set_ylabel("Frequency")
268 + ax1.set_title("(a) Price distribution", fontsize=10)
269 + ax1.set_xlim(0, 3000)
270 + ax2.hist(df["log_price"], bins=80, color="#2E7D32", alpha=0.8,
271 + edgecolor="white", lw=0.3)
272 + ax2.set_xlabel("log(Price)")
273 + ax2.set_ylabel("Frequency")
274 + ax2.set_title("(b) Log-price distribution", fontsize=10)
275 + plt.tight_layout(w_pad=1.5)
276 + for ext in ("pdf", "png"):
277 + plt.savefig(config.FIGURES_DIR / f"fig9_price_distribution.{ext}")
278 + plt.close()
279 + print(" fig9_price_distribution")
280 +
281 +
282 +STRUCT_LABELS = {
283 + "bedrooms": "Bedrooms", "bathrooms": "Bathrooms",
284 + "half_baths": "Half-bathrooms", "parking": "Parking",
285 + "stories": "Stories", "land_size": "Lot size",
286 + "remarks_length": "Description length",
287 +}
288 +
289 +
290 +def fig10_structural_coefficients(mD):
291 + fig, ax = plt.subplots(figsize=(6.2, 3))
292 + sv = config.STRUCTURAL_VARS + ["remarks_length"]
293 + conf = mD.conf_int()
294 + sdata = sorted(
295 + [(v, mD.params[v], conf.loc[v, 0], conf.loc[v, 1], mD.pvalues[v])
296 + for v in sv],
297 + key=lambda t: t[1],
298 + )
299 + yy = np.arange(len(sdata))
300 + cols = ["#1565C0" if d[4] < 0.05 else "#BDBDBD" for d in sdata]
301 + ax.barh(yy, [d[1] for d in sdata], color=cols, height=0.5, alpha=0.85)
302 + for i, (v, c, cl, ch, p) in enumerate(sdata):
303 + ax.plot([cl, ch], [yy[i], yy[i]], color="#333", lw=1)
304 + st = "***" if p < 0.001 else "**" if p < 0.01 else "*" if p < 0.05 else ""
305 + if st:
306 + ax.text(c + 0.005, yy[i], f"{(np.exp(c) - 1) * 100:+.1f}%{st}",
307 + va="center", fontsize=7.5)
308 + ax.axvline(0, color="black", lw=0.7)
309 + ax.set_yticks(yy)
310 + ax.set_yticklabels([STRUCT_LABELS[d[0]] for d in sdata], fontsize=9)
311 + ax.set_xlabel("Standardized coefficient")
312 + save("fig10_structural_coefficients")
313 +
314 +
315 +def fig11_residual_diagnostics(mD):
316 + fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(6.2, 3))
317 + ax1.scatter(mD.fittedvalues, mD.resid, s=2, alpha=0.1, c="#1565C0",
318 + edgecolors="none")
319 + ax1.axhline(0, color="red", lw=0.8, ls="--")
320 + ax1.set_xlabel("Fitted values")
321 + ax1.set_ylabel("Residuals")
322 + ax1.set_title("(a) Residuals vs. fitted", fontsize=10)
323 + probplot(mD.resid, dist="norm", plot=ax2)
324 + ax2.set_title("(b) Normal Q-Q plot", fontsize=10)
325 + ax2.get_lines()[0].set(markersize=2, alpha=0.15, color="#1565C0")
326 + ax2.get_lines()[1].set(color="red", lw=1)
327 + plt.tight_layout(w_pad=1.5)
328 + for ext in ("pdf", "png"):
329 + plt.savefig(config.FIGURES_DIR / f"fig11_residual_diagnostics.{ext}")
330 + plt.close()
331 + print(" fig11_residual_diagnostics")
332 +
333 +
334 +def main():
335 + df = pd.read_csv(config.ANALYSIS_CSV)
336 + print(f"{len(df):,} observations — fitting models ...")
337 + models, _, _ = fit_all_models(df)
338 + mD = models["D"]
339 +
340 + config.FIGURES_DIR.mkdir(parents=True, exist_ok=True)
341 + print("Generating figures ...")
342 + fig1_model_comparison(models)
343 + fig2_coefficient_plot(mD)
344 + fig3_similarity_distributions(df)
345 + fig4_quintile_heatmap(df)
346 + fig5_correlation_matrix(df)
347 + fig6_scatter_plots(df)
348 + fig7_methodology(len(df))
349 + fig8_r2_decomposition(models)
350 + fig9_price_distribution(df)
351 + fig10_structural_coefficients(mD)
352 + fig11_residual_diagnostics(mD)
353 + print("All figures saved to figures/")
354 +
355 +
356 +if __name__ == "__main__":
357 + main()
added scripts/06_tables.py +385 −0
@@ -0,0 +1,385 @@
1 +#!/usr/bin/env python3
2 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
3 +#
4 +"""Step 6 — Generate the paper's LaTeX tables from the pipeline results.
5 +
6 +Every number printed in these tables comes from results/*.csv (steps 3-4),
7 +so the paper cannot drift from the analysis outputs.
8 +
9 +Inputs : results/*.csv, data/processed/hedonic_maison_results.csv
10 +Outputs: paper/tables/tab_descriptive.tex
11 + paper/tables/tab_similarity_stats.tex
12 + paper/tables/tab_model_comparison.tex
13 + paper/tables/tab_full_results.tex
14 + paper/tables/tab_quantile.tex
15 + paper/tables/tab_parsimonious.tex (appendix)
16 + paper/tables/tab_reference_texts.tex (appendix)
17 +"""
18 +
19 +import sys
20 +from pathlib import Path
21 +
22 +import numpy as np
23 +import pandas as pd
24 +
25 +sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
26 +
27 +from src import config
28 +from src.references import ENGLISH_LABELS, REFERENCES
29 +
30 +TABLES_DIR = config.ROOT / "paper" / "tables"
31 +
32 +HEADER = "% Author: Simon-Pierre Boucher — contact@spboucher.ai\n% Generated by scripts/06_tables.py — do not edit by hand.\n"
33 +
34 +
35 +def tex_num(x, dec=0, signed=False):
36 + """Format a number with LaTeX minus signs and {,} thousand separators."""
37 + if signed:
38 + s = f"{x:+,.{dec}f}"
39 + else:
40 + s = f"{x:,.{dec}f}"
41 + s = s.replace(",", "{,}").replace("-", r"$-$").replace("+", "+")
42 + return s
43 +
44 +
45 +def stars(p):
46 + return "***" if p < 0.001 else "**" if p < 0.01 else "*" if p < 0.05 else ""
47 +
48 +
49 +def pval_str(p):
50 + return "$< 0.001$" if p < 0.001 else f"{p:.3f}"
51 +
52 +
53 +def write(name, content):
54 + path = TABLES_DIR / name
55 + path.write_text(HEADER + content)
56 + print(f" -> {path}")
57 +
58 +
59 +def tab_descriptive():
60 + d = pd.read_csv(config.RESULTS_DIR / "descriptive_stats.csv", index_col=0)
61 + rows_spec = [
62 + ("price", "Price (\\$)", 0),
63 + ("log_price", "log(Price)", 2),
64 + ("bedrooms", "Bedrooms", 1),
65 + ("bathrooms", "Bathrooms", 1),
66 + ("half_baths", "Half-bathrooms", 1),
67 + ("parking", "Parking spaces", 1),
68 + ("stories", "Stories", 1),
69 + ("land_size", "Lot size (sq.\\ ft.)", 0),
70 + ("remarks_length", "Description length (char.)", 0),
71 + ]
72 + lines = []
73 + for var, label, dec in rows_spec:
74 + r = d.loc[var]
75 + cells = [tex_num(r["mean"], dec), tex_num(r["std"], dec),
76 + tex_num(r["min"], 0 if var != "log_price" else 2),
77 + tex_num(r["25%"], 0 if var != "log_price" else 2),
78 + tex_num(r["50%"], 0 if var != "log_price" else 2),
79 + tex_num(r["75%"], 0 if var != "log_price" else 2),
80 + tex_num(r["max"], 0 if var != "log_price" else 2)]
81 + lines.append(f"{label} & " + " & ".join(cells) + " \\\\")
82 + body = "\n".join(lines)
83 + content = rf"""\begin{{table}}[!htbp]
84 +\centering
85 +\caption{{Descriptive statistics for structural and textual variables ($n = 17{{,}}087$).}}
86 +\label{{tab:descriptive}}
87 +\begin{{threeparttable}}
88 +\small
89 +\begin{{adjustbox}}{{max width=\textwidth}}
90 +\begin{{tabular}}{{lR{{1.6cm}}R{{1.7cm}}R{{1.2cm}}R{{1.4cm}}R{{1.4cm}}R{{1.4cm}}R{{1.9cm}}}}
91 +\toprule
92 +\textbf{{Variable}} & \textbf{{Mean}} & \textbf{{Std.\ Dev.}} & \textbf{{Min}} & \textbf{{P25}} & \textbf{{Median}} & \textbf{{P75}} & \textbf{{Max}} \\
93 +\midrule
94 +{body}
95 +\bottomrule
96 +\end{{tabular}}
97 +\end{{adjustbox}}
98 +\begin{{tablenotes}}
99 +\footnotesize
100 +\item \textit{{Notes:}} The sample is restricted to single-family houses with a positive listing price and a description of at least 20 characters. Descriptions are truncated at approximately 700 characters in the data export, which bounds the description-length variable. The lot-size field is noisy (units are not standardized at the source), which motivates its cautious interpretation in the regressions.
101 +\end{{tablenotes}}
102 +\end{{threeparttable}}
103 +\end{{table}}
104 +"""
105 + write("tab_descriptive.tex", content)
106 +
107 +
108 +def tab_similarity_stats():
109 + s = pd.read_csv(config.RESULTS_DIR / "similarity_stats.csv", index_col=0)
110 + lines = []
111 + for label, r in s.iterrows():
112 + label_tex = label.replace("&", "\\&")
113 + cells = [f"{r['mean']:.3f}", f"{r['std']:.3f}",
114 + tex_num(r["min"], 3), f"{r['50%']:.3f}", f"{r['max']:.3f}",
115 + tex_num(r["corr_log_price"], 3)]
116 + lines.append(f"{label_tex} & " + " & ".join(cells) + " \\\\")
117 + body = "\n".join(lines)
118 + content = rf"""\begin{{table}}[!htbp]
119 +\centering
120 +\caption{{Descriptive statistics for cosine similarity features and bivariate correlations with log(price).}}
121 +\label{{tab:sim_stats}}
122 +\begin{{threeparttable}}
123 +\small
124 +\begin{{adjustbox}}{{max width=\textwidth}}
125 +\begin{{tabular}}{{lR{{1.2cm}}R{{1.2cm}}R{{1.2cm}}R{{1.2cm}}R{{1.2cm}}R{{1.4cm}}}}
126 +\toprule
127 +\textbf{{Reference}} & \textbf{{Mean}} & \textbf{{Std.~Dev.}} & \textbf{{Min}} & \textbf{{Median}} & \textbf{{Max}} & \textbf{{Corr.\ $\ln P$}} \\
128 +\midrule
129 +{body}
130 +\bottomrule
131 +\end{{tabular}}
132 +\end{{adjustbox}}
133 +\begin{{tablenotes}}
134 +\footnotesize
135 +\item \textit{{Notes:}} Cosine similarity is computed between each listing embedding and the corresponding reference description embedding using the all-MiniLM-L6-v2 sentence transformer. Corr.\ $\ln P$ denotes the Pearson correlation with log listing price.
136 +\end{{tablenotes}}
137 +\end{{threeparttable}}
138 +\end{{table}}
139 +"""
140 + write("tab_similarity_stats.tex", content)
141 +
142 +
143 +def tab_model_comparison():
144 + m = pd.read_csv(config.RESULTS_DIR / "model_comparison.csv")
145 + spec_labels = {
146 + "A": "Structural only",
147 + "B": "+ Description length",
148 + "C": "+ Semantic similarities (20)",
149 + "D": "Full model (B + C)",
150 + "E": "Parsimonious (sig.\\ only)",
151 + }
152 + best_r2 = m["r2"].idxmax()
153 + best_adj = m["adj_r2"].idxmax()
154 + best_aic = m["aic"].idxmin()
155 + best_bic = m["bic"].idxmin()
156 + lines = []
157 + for i, r in m.iterrows():
158 + def fmt(val, dec, best):
159 + s = tex_num(val, dec)
160 + return rf"\textbf{{{s}}}" if i == best else s
161 + delta = "---" if r["model"] == "A" else tex_num(r["delta_r2_vs_A"], 3, signed=True)
162 + lines.append(
163 + f"{r['model']} & {spec_labels[r['model']]} & "
164 + f"{fmt(r['r2'], 4, best_r2)} & {fmt(r['adj_r2'], 4, best_adj)} & "
165 + f"{fmt(r['aic'], 0, best_aic)} & {fmt(r['bic'], 0, best_bic)} & "
166 + f"{int(r['k'])} & {delta} \\\\"
167 + )
168 + body = "\n".join(lines)
169 + content = rf"""\begin{{table}}[!htbp]
170 +\centering
171 +\caption{{Hedonic model comparison ($n = 17{{,}}087$).}}
172 +\label{{tab:model_comparison}}
173 +\begin{{threeparttable}}
174 +\small
175 +\begin{{adjustbox}}{{max width=\textwidth}}
176 +\begin{{tabular}}{{clR{{1.3cm}}R{{1.3cm}}R{{1.6cm}}R{{1.6cm}}R{{0.8cm}}R{{1.8cm}}}}
177 +\toprule
178 +& \textbf{{Specification}} & $\boldsymbol{{R^2}}$ & \textbf{{Adj.}}~$\boldsymbol{{R^2}}$ & \textbf{{AIC}} & \textbf{{BIC}} & $\boldsymbol{{k}}$ & $\boldsymbol{{\Delta R^2}}$ \textbf{{vs.\ A}} \\
179 +\midrule
180 +{body}
181 +\bottomrule
182 +\end{{tabular}}
183 +\end{{adjustbox}}
184 +\begin{{tablenotes}}
185 +\footnotesize
186 +\item \textit{{Notes:}} $k$ denotes the number of regressors excluding the intercept. Bold values indicate the best fit for each criterion. Models are estimated by OLS with HC3 robust standard errors.
187 +\end{{tablenotes}}
188 +\end{{threeparttable}}
189 +\end{{table}}
190 +"""
191 + write("tab_model_comparison.tex", content)
192 +
193 +
194 +def _coef_rows(d, variables):
195 + rows = []
196 + for _, r in d[d.variable.isin(variables)].iterrows():
197 + rows.append((r["label"] if r["variable"].startswith("sim_") else None,
198 + r["variable"], r["coefficient"], r["std_error"],
199 + r["t_value"], r["p_value"], r["impact_pct"]))
200 + return rows
201 +
202 +
203 +STRUCT_LABELS = {
204 + "bedrooms": "Bedrooms", "bathrooms": "Bathrooms",
205 + "half_baths": "Half-bathrooms", "parking": "Parking",
206 + "stories": "Stories", "land_size": "Lot size",
207 + "remarks_length": "Description length",
208 +}
209 +
210 +
211 +def tab_full_results():
212 + d = pd.read_csv(config.RESULTS_DIR / "coefficients_model_D.csv")
213 + struct = d[~d.variable.str.startswith("sim_")].copy()
214 + struct["label"] = struct.variable.map(STRUCT_LABELS)
215 + sims = d[d.variable.str.startswith("sim_")].copy()
216 + pos = sims[sims.coefficient >= 0].sort_values("coefficient", ascending=False)
217 + neg = sims[sims.coefficient < 0].sort_values("coefficient")
218 + struct = struct.sort_values("coefficient", ascending=False)
219 +
220 + def rows(block):
221 + out = []
222 + for _, r in block.iterrows():
223 + label = str(r["label"]).replace("&", "\\&")
224 + out.append(
225 + f"\\quad {label} & {tex_num(r['coefficient'], 4)} & "
226 + f"{tex_num(r['std_error'], 4)} & {tex_num(r['t_value'], 2)} & "
227 + f"{pval_str(r['p_value'])} & {tex_num(r['impact_pct'], 1, signed=True)} & "
228 + f"{stars(r['p_value'])} \\\\"
229 + )
230 + return "\n".join(out)
231 +
232 + content = rf"""\begin{{table}}[!htbp]
233 +\centering
234 +\caption{{Full model (D) coefficient estimates. HC3 robust standard errors. All variables standardized.}}
235 +\label{{tab:full_results}}
236 +\small
237 +\begin{{adjustbox}}{{max width=\textwidth}}
238 +\begin{{tabular}}{{lR{{1.2cm}}R{{1.2cm}}R{{1.1cm}}R{{1.6cm}}R{{1.3cm}}c}}
239 +\toprule
240 +\textbf{{Variable}} & \textbf{{Coeff.}} & \textbf{{Std.\ Err.}} & \textbf{{$t$-stat}} & \textbf{{$p$-value}} & \textbf{{Impact (\%)}} & \\
241 +\midrule
242 +\multicolumn{{7}}{{l}}{{\textit{{Panel A: Structural variables}}}} \\
243 +{rows(struct)}
244 +\midrule
245 +\multicolumn{{7}}{{l}}{{\textit{{Panel B: Semantic similarities --- positive price effects}}}} \\
246 +{rows(pos)}
247 +\midrule
248 +\multicolumn{{7}}{{l}}{{\textit{{Panel C: Semantic similarities --- negative price effects}}}} \\
249 +{rows(neg)}
250 +\bottomrule
251 +\multicolumn{{7}}{{l}}{{\footnotesize{{\signote\ Impact $= (e^{{\hat{{\beta}}}} - 1) \times 100$\%. $n = 17{{,}}087$; Adj.\ $R^2 = 0.511$.}}}}
252 +\end{{tabular}}
253 +\end{{adjustbox}}
254 +\end{{table}}
255 +"""
256 + write("tab_full_results.tex", content)
257 +
258 +
259 +def tab_quantile():
260 + r = pd.read_csv(config.RESULTS_DIR / "robustness_results.csv")
261 + d = pd.read_csv(config.RESULTS_DIR / "coefficients_model_D.csv").set_index("variable")
262 + dims = ["sim_moderne_contemporain", "sim_luxe", "sim_terrain_nature",
263 + "sim_familial", "sim_urgence_motivation", "sim_a_renover"]
264 + patterns = {
265 + "sim_moderne_contemporain": "High at both tails",
266 + "sim_luxe": "Increasing",
267 + "sim_terrain_nature": "Stable",
268 + "sim_familial": "Mildly attenuating",
269 + "sim_urgence_motivation": "Attenuating",
270 + "sim_a_renover": "Attenuating",
271 + }
272 + lines = []
273 + for v in dims:
274 + label = ENGLISH_LABELS[v[4:]].replace("&", "\\&")
275 + cells = []
276 + for tau in ("0.25", "0.5", "0.75"):
277 + q = r[(r.test == f"QuantReg_tau{tau}") & (r.variable == v)].iloc[0]
278 + cells.append(f"{tex_num(q['value'], 3)}{stars(q['p_value'])}")
279 + ols = d.loc[v]
280 + cells.append(f"{tex_num(ols['coefficient'], 3)}{stars(ols['p_value'])}")
281 + lines.append(f"{label} & " + " & ".join(cells) + f" & {patterns[v]} \\\\")
282 + body = "\n".join(lines)
283 + content = rf"""\begin{{table}}[!htbp]
284 +\centering
285 +\caption{{Quantile regression coefficients for selected semantic dimensions.}}
286 +\label{{tab:quantile}}
287 +\small
288 +\begin{{adjustbox}}{{max width=\textwidth}}
289 +\begin{{tabular}}{{lR{{1.5cm}}R{{1.5cm}}R{{1.5cm}}R{{1.5cm}}l}}
290 +\toprule
291 +\textbf{{Dimension}} & $\boldsymbol{{\tau = 0.25}}$ & $\boldsymbol{{\tau = 0.50}}$ & $\boldsymbol{{\tau = 0.75}}$ & \textbf{{OLS}} & \textbf{{Pattern}} \\
292 +\midrule
293 +{body}
294 +\bottomrule
295 +\multicolumn{{6}}{{l}}{{\footnotesize{{\signote\ Quantile regressions of Model~D; standard errors follow the kernel-based estimator of \citet{{koenker1978regression}}.}}}}
296 +\end{{tabular}}
297 +\end{{adjustbox}}
298 +\end{{table}}
299 +"""
300 + write("tab_quantile.tex", content)
301 +
302 +
303 +def tab_parsimonious():
304 + d = pd.read_csv(config.RESULTS_DIR / "coefficients_model_E.csv")
305 + struct = d[~d.variable.str.startswith("sim_")].copy()
306 + struct["label"] = struct.variable.map(STRUCT_LABELS)
307 + sims = d[d.variable.str.startswith("sim_")].sort_values("coefficient", ascending=False)
308 +
309 + def rows(block):
310 + out = []
311 + for _, r in block.iterrows():
312 + label = str(r["label"]).replace("&", "\\&")
313 + out.append(
314 + f"\\quad {label} & {tex_num(r['coefficient'], 4)} & "
315 + f"{tex_num(r['std_error'], 4)} & {pval_str(r['p_value'])} & "
316 + f"{tex_num(r['impact_pct'], 1, signed=True)} & {stars(r['p_value'])} \\\\"
317 + )
318 + return "\n".join(out)
319 +
320 + content = rf"""\begin{{table}}[!htbp]
321 +\centering
322 +\caption{{Parsimonious model (E) coefficient estimates. HC3 robust standard errors. All variables standardized.}}
323 +\label{{tab:parsimonious}}
324 +\small
325 +\begin{{adjustbox}}{{max width=\textwidth}}
326 +\begin{{tabular}}{{lR{{1.3cm}}R{{1.3cm}}R{{1.6cm}}R{{1.4cm}}c}}
327 +\toprule
328 +\textbf{{Variable}} & \textbf{{Coeff.}} & \textbf{{Std.\ Err.}} & \textbf{{$p$-value}} & \textbf{{Impact (\%)}} & \\
329 +\midrule
330 +\multicolumn{{6}}{{l}}{{\textit{{Panel A: Structural variables}}}} \\
331 +{rows(struct.sort_values('coefficient', ascending=False))}
332 +\midrule
333 +\multicolumn{{6}}{{l}}{{\textit{{Panel B: Semantic similarities (significant at 5\% in Model D)}}}} \\
334 +{rows(sims)}
335 +\bottomrule
336 +\multicolumn{{6}}{{l}}{{\footnotesize{{\signote\ Impact $= (e^{{\hat{{\beta}}}} - 1) \times 100$\%. $n = 17{{,}}087$; Adj.\ $R^2 = 0.511$.}}}}
337 +\end{{tabular}}
338 +\end{{adjustbox}}
339 +\end{{table}}
340 +"""
341 + write("tab_parsimonious.tex", content)
342 +
343 +
344 +def tab_reference_texts():
345 + lines = []
346 + for slug, text in REFERENCES.items():
347 + label = ENGLISH_LABELS[slug].replace("&", "\\&")
348 + text_tex = text.replace("&", "\\&").replace("%", "\\%")
349 + lines.append(f"{label} & \\textit{{{text_tex}}} \\\\[2pt]")
350 + body = "\n".join(lines)
351 + content = rf"""\begin{{footnotesize}}
352 +\begin{{longtable}}{{p{{3.2cm}}p{{11.5cm}}}}
353 +\caption{{The 20 reference descriptions (verbatim French text used for embedding).}}
354 +\label{{tab:reference_texts}} \\
355 +\toprule
356 +\textbf{{Dimension}} & \textbf{{Reference description}} \\
357 +\midrule
358 +\endfirsthead
359 +\toprule
360 +\textbf{{Dimension}} & \textbf{{Reference description}} \\
361 +\midrule
362 +\endhead
363 +\bottomrule
364 +\endfoot
365 +{body}
366 +\end{{longtable}}
367 +\end{{footnotesize}}
368 +"""
369 + write("tab_reference_texts.tex", content)
370 +
371 +
372 +def main():
373 + TABLES_DIR.mkdir(parents=True, exist_ok=True)
374 + tab_descriptive()
375 + tab_similarity_stats()
376 + tab_model_comparison()
377 + tab_full_results()
378 + tab_quantile()
379 + tab_parsimonious()
380 + tab_reference_texts()
381 + print("All tables generated.")
382 +
383 +
384 +if __name__ == "__main__":
385 + main()
added src/__init__.py +3 −0
@@ -0,0 +1,3 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +"""Analysis package for UQO Working Paper No. 2 (semantic hedonic pricing)."""
added src/config.py +35 −0
@@ -0,0 +1,35 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +"""Central configuration: repository-relative paths and constants.
4 +
5 +Every path is derived from the repository root so the pipeline runs from any
6 +working directory. Override the database location with the WP2_DB_PATH
7 +environment variable if the raw database lives elsewhere.
8 +"""
9 +
10 +import os
11 +from pathlib import Path
12 +
13 +ROOT = Path(__file__).resolve().parents[1]
14 +
15 +DATA_RAW = ROOT / "data" / "raw"
16 +DATA_PROCESSED = ROOT / "data" / "processed"
17 +FIGURES_DIR = ROOT / "figures"
18 +RESULTS_DIR = ROOT / "results"
19 +
20 +DB_PATH = Path(os.environ.get("WP2_DB_PATH", DATA_RAW / "louka.db"))
21 +
22 +# Intermediate artifacts (produced by the numbered scripts)
23 +HOUSES_PARQUET = DATA_PROCESSED / "houses.parquet"
24 +EMBEDDINGS_NPY = DATA_PROCESSED / "embeddings_maisons.npy"
25 +SIM_MATRIX_NPY = DATA_PROCESSED / "sim_matrix_maisons.npy"
26 +ANALYSIS_CSV = DATA_PROCESSED / "hedonic_maison_results.csv"
27 +
28 +# Sentence-transformers model used for all embeddings (384 dimensions)
29 +EMBEDDING_MODEL = "all-MiniLM-L6-v2"
30 +ENCODE_BATCH_SIZE = 256
31 +
32 +# Structural covariates of the hedonic model
33 +STRUCTURAL_VARS = ["bedrooms", "bathrooms", "half_baths", "parking", "stories", "land_size"]
34 +
35 +SEED = 42
added src/data.py +101 −0
@@ -0,0 +1,101 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +"""Extraction of the house sample from the raw SQLite database.
4 +
5 +The extraction logic replicates the original ``hedonic_maison.py`` exactly:
6 +single-family listings (``category = 'house'``) with a positive price and a
7 +description of at least 20 characters, keeping the same field parsing and the
8 +same defaults so the resulting sample is byte-for-byte identical
9 +(n = 17,087).
10 +"""
11 +
12 +import json
13 +import sqlite3
14 +
15 +import numpy as np
16 +import pandas as pd
17 +
18 +
19 +def _parse_land_size(size_str):
20 + """Parse the leading numeric token of Land.SizeTotal (e.g. '5000 sqft')."""
21 + return float("".join(c for c in size_str.split()[0].replace(",", "") if c.isdigit() or c == "."))
22 +
23 +
24 +def load_houses(db_path):
25 + """Return the analysis sample of houses as a DataFrame.
26 +
27 + Columns: id, price, log_price, bedrooms, bathrooms, half_baths, parking,
28 + stories, land_size, latitude, longitude, prop_type, remarks,
29 + remarks_length.
30 + """
31 + conn = sqlite3.connect(db_path)
32 + rows = conn.execute(
33 + "SELECT id, category, price_value, bedrooms, bathrooms, data "
34 + "FROM properties WHERE category = 'house'"
35 + ).fetchall()
36 + conn.close()
37 +
38 + records = []
39 + for pid, _category, price, beds, baths, data_json in rows:
40 + try:
41 + data = json.loads(data_json)
42 + except (json.JSONDecodeError, TypeError):
43 + continue
44 +
45 + remarks = data.get("PublicRemarks", "")
46 + if not remarks or not isinstance(remarks, str) or len(remarks.strip()) < 20:
47 + continue
48 + if not price or price <= 0:
49 + continue
50 +
51 + parking = 0
52 + try:
53 + parking = int(data["Property"].get("ParkingSpaceTotal", 0))
54 + except (KeyError, TypeError, ValueError):
55 + pass
56 +
57 + stories = 0
58 + try:
59 + stories = int(data["Building"].get("StoriesTotal", 0))
60 + except (KeyError, TypeError, ValueError):
61 + pass
62 +
63 + half_bath = 0
64 + try:
65 + half_bath = int(data["Building"].get("HalfBathTotal", 0))
66 + except (KeyError, TypeError, ValueError):
67 + pass
68 +
69 + land_size = 0
70 + try:
71 + land_size = _parse_land_size(data.get("Land", {}).get("SizeTotal", "0"))
72 + except (IndexError, ValueError):
73 + pass
74 +
75 + prop_type = data.get("Property", {}).get("Type", "")
76 +
77 + lat = lon = None
78 + try:
79 + lat = float(data["Property"]["Address"]["Latitude"])
80 + lon = float(data["Property"]["Address"]["Longitude"])
81 + except (KeyError, TypeError, ValueError):
82 + pass
83 +
84 + records.append({
85 + "id": pid,
86 + "price": price,
87 + "log_price": np.log(price),
88 + "bedrooms": beds or 0,
89 + "bathrooms": baths or 0,
90 + "half_baths": half_bath,
91 + "parking": parking,
92 + "stories": stories,
93 + "land_size": land_size,
94 + "latitude": lat,
95 + "longitude": lon,
96 + "prop_type": prop_type,
97 + "remarks": remarks,
98 + "remarks_length": len(remarks),
99 + })
100 +
101 + return pd.DataFrame(records).reset_index(drop=True)
added src/embeddings.py +59 −0
@@ -0,0 +1,59 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +"""Sentence embeddings and cosine-similarity features."""
4 +
5 +import numpy as np
6 +
7 +from . import config
8 +from .references import REFERENCES, SIM_COLS
9 +
10 +
11 +def encode_references(model=None):
12 + """Encode the 20 reference descriptions. Returns (names, embeddings)."""
13 + if model is None:
14 + from sentence_transformers import SentenceTransformer
15 + model = SentenceTransformer(config.EMBEDDING_MODEL)
16 + names = list(REFERENCES.keys())
17 + embeddings = model.encode(list(REFERENCES.values()), normalize_embeddings=True)
18 + return names, embeddings
19 +
20 +
21 +def encode_remarks(texts, model=None, show_progress=True):
22 + """Encode listing descriptions with the paper's embedding model."""
23 + if model is None:
24 + from sentence_transformers import SentenceTransformer
25 + model = SentenceTransformer(config.EMBEDDING_MODEL)
26 + return model.encode(
27 + list(texts),
28 + batch_size=config.ENCODE_BATCH_SIZE,
29 + show_progress_bar=show_progress,
30 + normalize_embeddings=True,
31 + )
32 +
33 +
34 +def load_or_encode_remarks(texts, cache_path=config.EMBEDDINGS_NPY, force=False):
35 + """Load cached embeddings if they match the sample size, else encode.
36 +
37 + Embeddings are deterministic for a given model version, so the cache is a
38 + pure speed-up; pass force=True to re-encode from scratch.
39 + """
40 + if not force and cache_path.exists():
41 + cached = np.load(cache_path)
42 + if len(cached) == len(texts):
43 + return cached, True
44 + embeddings = encode_remarks(texts)
45 + cache_path.parent.mkdir(parents=True, exist_ok=True)
46 + np.save(cache_path, embeddings)
47 + return embeddings, False
48 +
49 +
50 +def similarity_features(prop_embeddings, ref_embeddings):
51 + """Cosine similarities (dot product of normalized vectors): n x 20 matrix."""
52 + return prop_embeddings @ ref_embeddings.T
53 +
54 +
55 +def add_similarity_columns(df, sim_matrix):
56 + """Attach the 20 ``sim_<slug>`` columns to the DataFrame (in place)."""
57 + for i, col in enumerate(SIM_COLS):
58 + df[col] = sim_matrix[:, i]
59 + return df
added src/models.py +47 −0
@@ -0,0 +1,47 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +"""Hedonic OLS specifications A-E (standardized covariates, HC3 errors)."""
4 +
5 +import pandas as pd
6 +import statsmodels.api as sm
7 +from sklearn.preprocessing import StandardScaler
8 +
9 +from .config import STRUCTURAL_VARS
10 +from .references import SIM_COLS
11 +
12 +
13 +def standardized_design(df, columns):
14 + """Z-score the selected columns and prepend a constant."""
15 + X = pd.DataFrame(
16 + StandardScaler().fit_transform(df[columns]),
17 + columns=columns,
18 + index=df.index,
19 + )
20 + return sm.add_constant(X)
21 +
22 +
23 +def fit_ols(df, columns, y=None):
24 + """OLS of log-price on standardized covariates with HC3 robust errors."""
25 + if y is None:
26 + y = df["log_price"]
27 + return sm.OLS(y, standardized_design(df, columns)).fit(cov_type="HC3")
28 +
29 +
30 +def fit_all_models(df):
31 + """Fit specifications A-E and return them with the significant-similarity list.
32 +
33 + A: structural only B: A + description length
34 + C: A + 20 similarities D: B + 20 similarities (full)
35 + E: B + similarities significant at 5% in D (parsimonious)
36 + """
37 + specs = {
38 + "A": STRUCTURAL_VARS,
39 + "B": STRUCTURAL_VARS + ["remarks_length"],
40 + "C": STRUCTURAL_VARS + SIM_COLS,
41 + "D": STRUCTURAL_VARS + ["remarks_length"] + SIM_COLS,
42 + }
43 + models = {name: fit_ols(df, cols) for name, cols in specs.items()}
44 + sig_sims = [v for v in SIM_COLS if models["D"].pvalues.get(v, 1) < 0.05]
45 + specs["E"] = STRUCTURAL_VARS + ["remarks_length"] + sig_sims
46 + models["E"] = fit_ols(df, specs["E"])
47 + return models, specs, sig_sims
added src/references.py +149 −0
@@ -0,0 +1,149 @@
1 +# Author: Simon-Pierre Boucher — contact@spboucher.ai
2 +#
3 +"""The 20 reference descriptions defining the semantic dimensions.
4 +
5 +Each reference is a short French archetype description that "embodies" one
6 +qualitative dimension of a property listing. Cosine similarity between a
7 +listing's embedding and each reference embedding yields the 20 semantic
8 +features used in the hedonic models.
9 +
10 +Keys are the slugs used for data columns (``sim_<slug>``); ENGLISH_LABELS maps
11 +slugs to the display names used in figures and in the paper.
12 +"""
13 +
14 +from collections import OrderedDict
15 +
16 +REFERENCES = OrderedDict([
17 + ("luxe", (
18 + "Propriété de prestige haut de gamme avec finitions luxueuses, "
19 + "matériaux nobles, planchers de bois franc, comptoirs de quartz et granit, "
20 + "cuisine gastronomique, salle de bain spa avec douche en verre, "
21 + "domotique et système audio intégré. Résidence d'exception."
22 + )),
23 + ("entree_de_gamme", (
24 + "Propriété abordable idéale pour premier acheteur, bon prix, "
25 + "petit budget, opportunité d'investissement, starter home, "
26 + "parfait pour débuter, prix compétitif, aubaine."
27 + )),
28 + ("renove", (
29 + "Entièrement rénové, remis à neuf, nouvelles fenêtres, "
30 + "nouvelle toiture, plomberie et électricité refaites, "
31 + "cuisine et salle de bain rénovées, modernisé, mise à jour complète, "
32 + "rien à faire, clé en main, prêt à emménager."
33 + )),
34 + ("a_renover", (
35 + "À rénover, à rafraîchir, nécessite des travaux, bon potentiel, "
36 + "vendu tel quel sans garantie, handyman special, "
37 + "besoin de rénovation, fixer-upper, à mettre à son goût."
38 + )),
39 + ("lumineux_spacieux", (
40 + "Très lumineux, fenestration abondante, aires ouvertes, "
41 + "grands espaces, plafonds hauts, cathédrale, mezzanine, "
42 + "vaste salon, pièces spacieuses, beaucoup de rangement, walk-in."
43 + )),
44 + ("terrain_nature", (
45 + "Grand terrain boisé, mature, aménagement paysager, "
46 + "piscine creusée, spa, terrasse, patio, cour arrière privée, "
47 + "jardin, haie de cèdres, intime, sans voisin arrière, "
48 + "vue sur la nature, bord de l'eau, accès au lac, rivière."
49 + )),
50 + ("vue_panoramique", (
51 + "Vue imprenable, vue panoramique, vue sur le fleuve, "
52 + "vue sur la montagne, vue sur la ville, skyline de Montréal, "
53 + "vue dégagée, vue spectaculaire, penthouse avec vue."
54 + )),
55 + ("localisation_premium", (
56 + "Emplacement de choix, quartier recherché, proche de tout, "
57 + "à distance de marche des commerces, restaurants, cafés, "
58 + "accès rapide au transport en commun, métro, autoroute, "
59 + "près des écoles, des parcs, quartier familial sécuritaire."
60 + )),
61 + ("tranquillite", (
62 + "Rue tranquille, cul-de-sac, quartier paisible, résidentiel, "
63 + "calme, intimité, retiré, campagne, nature, boisé, "
64 + "loin du bruit, environnement serein."
65 + )),
66 + ("revenu_investissement", (
67 + "Excellent investissement, revenu locatif, plex rentable, "
68 + "baux en cours, bon rendement, cash flow positif, "
69 + "logements loués, duplex triplex avec revenus, rapport qualité-prix."
70 + )),
71 + ("garage_stationnement", (
72 + "Garage double, garage chauffé, stationnement intérieur, "
73 + "entrée de garage pavée, abri d'auto, grand garage, "
74 + "atelier dans le garage, espace de rangement au garage."
75 + )),
76 + ("sous_sol_amenage", (
77 + "Sous-sol entièrement aménagé, salle familiale au sous-sol, "
78 + "chambre supplémentaire, salle de cinéma, bureau, "
79 + "possibilité de logement au sous-sol, entrée indépendante, "
80 + "sous-sol avec salle de bain complète."
81 + )),
82 + ("moderne_contemporain", (
83 + "Design moderne, contemporain, architecture épurée, "
84 + "lignes droites, minimaliste, construction neuve, "
85 + "maison intelligente, écoénergétique, LEED, "
86 + "fenêtres panoramiques, toit plat."
87 + )),
88 + ("cachet_patrimonial", (
89 + "Cachet d'époque, maison ancestrale, patrimoine, "
90 + "boiseries d'origine, moulures, foyer d'origine, "
91 + "plafonds de 10 pieds, charme victorien, pierre, brique."
92 + )),
93 + ("ecoefficace", (
94 + "Écoénergétique, thermopompe, géothermie, panneaux solaires, "
95 + "isolation supérieure, fenêtres Energy Star, "
96 + "chauffage radiant, faible consommation, certifié Novoclimat, "
97 + "réservoir d'eau chaude récent, coûts énergétiques bas."
98 + )),
99 + ("urgence_motivation", (
100 + "Vendeur motivé, vente rapide, prix réduit, réduction de prix, "
101 + "succession, relocalisation, doit vendre, reprise de finance, "
102 + "offres multiples bienvenues, ne manquez pas cette occasion."
103 + )),
104 + ("familial", (
105 + "Maison familiale, quartier familial, parc pour enfants, "
106 + "cour clôturée, école à proximité, garderie, aire de jeux, "
107 + "voisinage sécuritaire, idéal pour famille avec enfants."
108 + )),
109 + ("bord_eau", (
110 + "Bord de l'eau, accès au lac, vue sur le fleuve, rivière, "
111 + "quai privé, droits nautiques, plage, navigable, "
112 + "chalet au bord du lac, waterfront, pieds dans l'eau."
113 + )),
114 + ("neuf_construction", (
115 + "Construction neuve, maison neuve, jamais habitée, "
116 + "modèle de démonstration, garantie GCR, livraison prochaine, "
117 + "choix de finitions, plans personnalisables, nouveau développement."
118 + )),
119 + ("piscine_amenagement", (
120 + "Piscine creusée chauffée, piscine hors terre, spa, "
121 + "cuisine extérieure, terrasse en composite, pergola, "
122 + "aménagement paysager professionnel, pavé uni, foyer extérieur."
123 + )),
124 +])
125 +
126 +ENGLISH_LABELS = {
127 + "luxe": "Luxury",
128 + "entree_de_gamme": "Entry-Level",
129 + "renove": "Renovated",
130 + "a_renover": "Needs Renovation",
131 + "lumineux_spacieux": "Bright & Spacious",
132 + "terrain_nature": "Land & Nature",
133 + "vue_panoramique": "Panoramic View",
134 + "localisation_premium": "Premium Location",
135 + "tranquillite": "Quiet & Peaceful",
136 + "revenu_investissement": "Income/Investment",
137 + "garage_stationnement": "Garage & Parking",
138 + "sous_sol_amenage": "Finished Basement",
139 + "moderne_contemporain": "Modern/Contemporary",
140 + "cachet_patrimonial": "Heritage/Character",
141 + "ecoefficace": "Energy Efficient",
142 + "urgence_motivation": "Motivated Seller",
143 + "familial": "Family-Friendly",
144 + "bord_eau": "Waterfront",
145 + "neuf_construction": "New Construction",
146 + "piscine_amenagement": "Pool & Landscaping",
147 +}
148 +
149 +SIM_COLS = [f"sim_{slug}" for slug in REFERENCES]
150