# 🏠 Decoding Real Estate Descriptions

UQO Working Paper No. 2 — Semantic Embeddings and Hedonic Pricing of Residential Properties in Quebec

TL;DR — The free-text PublicRemarks of 17,087 Quebec single-family listings (snapshot of 7 January 2026) are embedded with a sentence transformer, projected onto 20 researcher-defined reference descriptions (Luxury, Needs Renovation, Waterfront, Motivated Seller, …) by cosine similarity, and the 20 named scores enter a log-price hedonic model. Adjusted R² rises from 0.452 to 0.511, a gain confirmed out of sample (random and FSA-grouped folds). The 2026-10-17 revision shows that 40 % of the listings are in English and score lower against the French references: language alone explains 86 % of the variance of the mean similarity and more than half of the apparent semantic gain. Net of language and 415 FSA fixed effects, the block adds 2.0 pp of R² (baseline encoder) or 3.9 pp (multilingual encoder) and stays jointly significant; only five dimensions (Luxury, Land & Nature, Family-Friendly, Quiet & Peaceful, Needs Renovation) keep sign and significance across specifications and encoders.

# 📌 Status

# 🎯 What this paper does

Hedonic pricing models decompose a property's price into the implicit prices of its characteristics, but they can only price what the econometrician observes. Broker-written descriptions («cuisine gastronomique, comptoirs de quartz…» vs. «besoin de rénovation, vendu tel quel…») carry quality information that no structured field records. Existing text approaches trade depth for interpretability:

The paper defines 20 French reference descriptions, embeds them with the same model as the listings, and uses the cosine similarity between each listing and each reference as 20 named regressors:

log(P_i) = α + β'X_i + δ·len_i + γ'S_i + ε_i        S_i = E_i · Rᵀ   (17,087 × 20)
log(P_i) = α_FSA(i) + β'X_i + δ·len_i + λ'L_i + γ'S_i + ε_i   (extended: language L_i, 415 FSA fixed effects, clustered SE)

Each γ_j reads as the change in log price for a one-standard-deviation increase in similarity to reference j (impact = (e^γ − 1) × 100 %). The revision asks three questions the first version did not: how much of the gain is the language of the listing rather than its content; whether the gain survives location fixed effects and holds out of sample; and whether the 20 individual coefficients are stable to the phrasing of the references and to the encoder.

# 🏆 Headline results

Baseline (v1.1, reproduced exactly)

Joint test of the 21 text variables (D vs. A): F(21, 17,059) = 99.53, p < 0.001.

Revision 2026-10-17 (Section 7 of the paper)

• Language: 57.6 % French, 39.7 % English, 2.7 % mixed; mean similarity 0.434 (FR) vs 0.204 (EN) with MiniLM-L6, 0.395 vs 0.352 with the multilingual encoder; English listings are priced +19.5 % conditional on structure (+9.1 % within FSA).
• Reference phrasing: perturbing the reference texts leaves the similarities almost unchanged (corr. 0.93–0.99) and the block ΔR² at 0.042–0.054, but the vector of 20 coefficients correlates only 0.02–0.81 with the baseline.
• Earlier robustness (v1.1): quantile regressions (Luxury 9.7 % → 17.3 % from τ = .25 to .75), 1,000-rep bootstrap (SE within 9 % of HC3), 1 %/99 % trimming (no sign reversal), VIF mean 17.2, Lasso keeps 4/20, PCA-20 ΔR² +0.070 vs +0.044.

# 🗂 Repository layout

wp2_uqo/
├── README.md
├── requirements.txt         # pinned dependencies (Python 3.14, macOS arm64)
├── docs/                    # REVIEW-REPORT.md · CHANGELOG-REVISION.md (2026-10-17 revision)
│                            # AUDIT.md · CHANGES.md · PAPER_REVIEW.md · UPGRADE_REPORT.md (2026-08-05 restructuring)
│                            # main_pre-revision-20261017.pdf (v1.1 reference PDF)
├── data/                    # NOT tracked (see data/README.md)
│   ├── raw/louka.db         #   SQLite, 268 MB, 46,479 listings
│   └── processed/           #   parquet sample, embedding caches (2 models), similarity matrix, revision fields
├── src/                     # analysis package
│   ├── config.py            #   repo-relative paths, model name, seed, structural vars
│   ├── references.py        #   the 20 French reference descriptions + English labels
│   ├── data.py              #   SQLite → DataFrame extraction (n = 17,087)
│   ├── embeddings.py        #   encoding, caching, cosine-similarity features
│   ├── models.py            #   standardised-design OLS A–E, HC3
│   └── extensions.py        #   language detection, FSA/municipality, lot & floor area (m²), FE/cluster OLS, OOS metrics
├── scripts/                 # the pipeline, in order
│   ├── 01_prepare_data.py   #   louka.db → data/processed/houses.parquet
│   ├── 02_similarities.py   #   embeddings (cached) + 20 similarity features
│   ├── 03_models.py         #   hedonic models A–E → results/*.csv
│   ├── 04_robustness.py     #   VIF · BP · quantile · trimming · bootstrap · Lasso/ENet
│   ├── 05_figures.py        #   fig1–fig11 (PDF + PNG)
│   ├── 06_tables.py         #   LaTeX tables generated from results
│   ├── 07_revision_extensions.py  # language · FE · clustering · lot/floor area · block tests · OOS · reference sensitivity · multilingual
│   └── 08_revision_tables.py      # tables 10–15 + fig12 from results/revision_*.csv
├── results/                 # model, robustness and revision outputs (CSV)
├── figures/                 # 12 paper figures
└── paper/                   # main.tex + preamble.tex + sections/ + tables/ + references.bib → main.pdf

# 🚀 Quick start

python3 -m pip install -r requirements.txt

# place louka.db in data/raw/ (see data/README.md), then:
python3 scripts/01_prepare_data.py     # 17,087 houses with a valid price and description
python3 scripts/02_similarities.py     # embed + project (add --force to re-encode)
python3 scripts/03_models.py           # hedonic models A–E
python3 scripts/04_robustness.py       # robustness battery (~4 min, bootstrap)
python3 scripts/05_figures.py          # 11 figures
python3 scripts/06_tables.py           # LaTeX tables from results
python3 scripts/07_revision_extensions.py   # ~25 s (+ model downloads on first run)
python3 scripts/08_revision_tables.py       # revision tables + fig12

cd paper && make                       # latexmk → main.pdf (61 pages)

Total runtime ≈ 8 minutes on an Apple-silicon laptop (embeddings cached after the first run). All stochastic steps are seeded (seed = 42). Expected checkpoints: step 3 prints R² = 0.4525 / 0.4647 / 0.4962 / 0.5122 / 0.5118 and F = 99.53; step 7 prints dR2(sims)=0.0475 (D), 0.0174 (D-L), 0.0197 (D-L + FSA FE), 0.0392 (ML-D-L + FSA FE).

# 🗃 Data

A SQLite database (louka.db, 268 MB) of Quebec residential listings collected from Realtor.ca / Centris in a single extraction completed on 7 January 2026: 46,479 listings across five categories — houses (17,089), condos (8,586), land (8,506), rentals (8,240), plexes (4,058). Each row carries structured fields, coordinates, postal code and the broker-written PublicRemarks.

Analysis sample: single-family houses with a positive asking price and a description of ≥ 20 characters → n = 17,087, 415 forward sortation areas (FSA), 1,061 municipalities. Median asking price $589,900 (mean $807,259). Descriptions average 510 characters, truncated at ~700 characters in the export. The lot-size field mixes units at the source (61 % m², 32 % sq. ft., 2 % acres/ha, 4 % missing) — used raw in the baseline for comparability with v1.1, converted to m² in the extended models. Interior floor area is reported for 6,228 listings (36 %).

data/ is not tracked: the database exceeds 100 MB and listing texts are proprietary platform content. All derived, aggregate outputs (results CSVs, figures, tables, paper) are versioned, so every number in the paper is inspectable without the raw data. data/README.md explains how to place a copy of louka.db; the location can be overridden with WP2_DB_PATH.

# 🔬 Methodology

1. Extraction — houses only, price > 0, description ≥ 20 characters; minimal preprocessing (src/data.py); language classified from French/English stop-word counts (src/extensions.py).
2. Embedding — all-MiniLM-L6-v2 (22.7M params, 384-d, L2-normalised) for the baseline; paraphrase-multilingual-MiniLM-L12-v2 for the multilingual specifications (src/embeddings.py, scripts/07_…).
3. Projection — 20 French reference descriptions (146–257 chars, verbatim in paper Appendix A); one matrix product yields the 17,087 × 20 similarity matrix (src/references.py).
4. Estimation — nested OLS models A–E on log price, standardised covariates, HC3 errors; joint F-tests; quantile regressions; bootstrap; trimming; VIF; Lasso/Elastic-Net; PCA benchmark (src/models.py, scripts/04_…).
5. Extended specifications (v1.2) — language indicators; FSA and municipality fixed effects (within transformation) with standard errors clustered by area; unit-corrected lot area; floor-area subsample; language subsamples; domain-level Wald tests; 5-fold cross-validation with random and FSA-grouped folds (R²_oos, RMSE, MdAPE); reference-phrasing perturbations; multilingual encoder (scripts/07_…).

# 🖼 Figures & tables inventory

Tables (paper/tables/*.tex): descriptive statistics, language composition (new), similarity statistics, model comparison, full Model D, quantile regressions, text-representation comparison, robustness summary, spec summary (new), extended specifications (new), domain block tests (new), out-of-sample (new), reference sensitivity (new), reference categories, reference texts (appendix), parsimonious model (appendix). 13 are machine-generated by 06_tables.py / 08_revision_tables.py, 3 are hand-maintained (tab_references, tab_robustness_summary, tab_text_comparison).

# ⚠️ Limitations

• Asking prices from a single snapshot, not transaction prices; no dates → no forward-in-time holdout; coefficients are conditional associations (text is written after the price is set, by the same agent).
• 40 % of the corpus is English while the references are French and the baseline encoder is English-trained; the paper controls for language, estimates within language and uses a multilingual encoder, but a French-specific encoder is untested.
• Floor area available for 36 % of listings only; fixed effects absorb between-area location only.
• The 20 references have overlapping vocabularies; individual coefficients are sensitive to phrasing and encoder (quantified in Section 7). Block- and domain-level conclusions are the robust object; VIF mean 17.2.
• Descriptions truncated at ~700 characters in the export.

# 📝 Citation

@techreport{boucher2026decoding,
  author      = {Boucher, Simon-Pierre},
  title       = {Decoding Real Estate Descriptions: Semantic Embeddings and
                 Hedonic Pricing of Residential Properties in Quebec},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {2},
  year        = {2026},
  month       = {October},
  note        = {Version 1.2}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

• Format follows Recherche-UQO/HARMONISATION.md (2026-10-17): template title page, natbib + aer bibliography style, docs/ for the audit documents.
• Revision 2026-10-17 (v1.1 → v1.2): editorial-board review report in docs/REVIEW-REPORT.md (diagnosis by section, methodological verdicts, literature, remaining limitations, recommended analyses not executed, publication maturity) and audit trail in docs/CHANGELOG-REVISION.md (every correction with before/after/evidence, 8 added references with DOIs, re-analyses executed). The v1.1 numbers are reproduced and kept as the baseline; the new Section 7 qualifies their interpretation. Pre-revision state: git tag pre-revision-20261017, docs/main_pre-revision-20261017.pdf.
• The original flat May 2026 project (immo-wp2-spb-20260519) is archived in Recherche-UQO/_archives/versions-mai-2026/decoding-real-estate-descriptions--immo-wp2-20260519/; docs/AUDIT.md and docs/CHANGES.md document the 2026-08-05 restructuring and the numeric corrections it caught.
• Build verified 2026-10-17: 0 errors, 0 undefined references, 94 references rendered, 61 pages.