# 🏠 Decoding Real Estate Descriptions

UQO Working Paper No. 2 — Semantic Embeddings and Hedonic Pricing of Residential Properties in Quebec

TL;DR — Listing descriptions written by real-estate agents contain price-relevant information that classic hedonic variables miss. The free-text PublicRemarks of 17,087 Quebec single-family house listings are embedded with a sentence transformer, projected onto 20 researcher-defined semantic reference descriptions (Luxury, Needs Renovation, Waterfront, Motivated Seller, …) via cosine similarity, and the 20 interpretable scores are added to a log-price hedonic OLS model. Adjusted R² rises from 0.452 to 0.511 (joint F = 99.53, p < 0.001); each semantic dimension carries a named, signed, economically meaningful implicit price (Modern/Contemporary +16.4 %, Luxury +14.2 %, Needs Renovation −7.9 %, Motivated Seller −8.5 % per SD).

# 📌 Status

# 🎯 What this paper does

Hedonic pricing models decompose a property's price into the implicit prices of its characteristics — bedrooms, bathrooms, lot size. But the narrative of a listing («cuisine gastronomique, comptoirs de quartz…» vs. «besoin de rénovation, vendu tel quel…») carries quality information that no structured field captures. Historically there has been a depth-vs-interpretability trade-off:

Instead of feeding 384 anonymous embedding dimensions into a regression, the paper defines 20 short reference descriptions — synthetic French paragraphs, each embodying one qualitative housing dimension — embeds them with the same model, and uses the cosine similarity between each listing and each reference as 20 named regressors:

log(P_i) = α + β'X_i + δ·len_i + γ'S_i + ε_i        S_i = E_i · Rᵀ   (17,087 × 20)

Every coefficient then reads directly as «the implicit price of sounding more luxury / more fixer-upper / more waterfront», per standard deviation — analogous to factor-mimicking portfolios in asset pricing and to concept-bottleneck models in interpretable ML.

# 🏆 Headline results

Joint significance of the 21 text variables (D vs. A): F = 99.53, p < 0.001. Variance decomposition of Model D: structural 88.3 %, text length 2.4 %, semantics 9.3 %.

_{*** p < 0.001, ** p < 0.01. Negative coefficients on Family-Friendly / New Construction largely proxy suburban location (paper §6). Impact = (e^β − 1) × 100 %.}

• Quantile regressions (τ = 0.25 / 0.50 / 0.75): the Luxury premium rises monotonically from 9.7 % to 17.3 %; urgency and condition discounts attenuate at the top of the market.
• Bootstrap (1,000 replications): SEs within 9 % of HC3. Trimming (1st/99th percentiles): no sign reversal.
• Multicollinearity disclosed: similarity dimensions correlate at 0.66–0.97 (mean VIF 17.2); block inference unaffected.
• PCA benchmark: 20 principal components fit better (ΔR² +0.070 vs. +0.044) but are economically unreadable.

# 🗂 Repository layout

wp2_uqo/
├── README.md
├── requirements.txt         # pinned dependencies (Python 3.14, macOS arm64)
├── docs/                    # AUDIT.md · CHANGES.md · PAPER_REVIEW.md · UPGRADE_REPORT.md (restructuring history)
├── data/                    # NOT tracked (see data/README.md)
│   ├── raw/louka.db         #   SQLite, 268 MB, 46,479 listings
│   └── processed/           #   parquet sample, embeddings cache, similarity matrix
├── src/                     # analysis package
│   ├── config.py            #   repo-relative paths, model name, seed, structural vars
│   ├── references.py        #   the 20 French reference descriptions + English labels
│   ├── data.py              #   SQLite → DataFrame extraction (n = 17,087)
│   ├── embeddings.py        #   encoding, caching, cosine-similarity features
│   └── models.py            #   standardized-design OLS A–E, HC3
├── scripts/                 # the pipeline, in order
│   ├── 01_prepare_data.py   #   louka.db → data/processed/houses.parquet
│   ├── 02_similarities.py   #   embeddings (cached) + 20 similarity features
│   ├── 03_models.py         #   models A–E → results/*.csv
│   ├── 04_robustness.py     #   VIF · BP · quantile · winsor · bootstrap · Lasso/ENet
│   ├── 05_figures.py        #   fig1–fig11 (PDF + PNG)
│   └── 06_tables.py         #   LaTeX tables generated from results
├── results/                 # model & robustness outputs (CSV)
├── figures/                 # 11 paper figures
└── paper/                   # main.tex + preamble.tex + sections/ + tables/ + references.bib → main.pdf

# 🚀 Quick start

python3 -m pip install -r requirements.txt

# place louka.db in data/raw/ (see data/README.md), then:
python3 scripts/01_prepare_data.py     # 17,087 houses with a valid price and description
python3 scripts/02_similarities.py     # embed + project (add --force to re-encode)
python3 scripts/03_models.py           # hedonic models A–E
python3 scripts/04_robustness.py       # robustness battery (~4 min, bootstrap)
python3 scripts/05_figures.py          # 11 figures
python3 scripts/06_tables.py           # LaTeX tables from results

cd paper && make                       # latexmk → main.pdf (53 pages)

Total runtime ≈ 8 minutes on an Apple-silicon laptop (embeddings cached after the first run). All stochastic steps are seeded (seed = 42). Expected checkpoints: step 3 prints R² = 0.4525 / 0.4647 / 0.4962 / 0.5122 / 0.5118 and F = 99.53.

# 🗃 Data

A SQLite database (louka.db, 268 MB) of Quebec residential listings collected from Realtor.ca / Centris: 46,479 listings across five categories — houses (17,089), condos (8,586), land (8,506), rentals (8,240), plexes (4,058). Each row carries structured fields plus the agent-written PublicRemarks.

Analysis sample: single-family houses with a positive price and a description of ≥ 20 characters → n = 17,087. Median listing price $589,900 (mean $807,259). Descriptions average 510 characters, truncated at ~700 characters in the export (disclosed in the paper).

data/ is not tracked: the database exceeds 100 MB and listing texts are proprietary platform content. All derived, aggregate outputs (results CSVs, figures, tables, paper) are versioned, so every number in the paper is inspectable without the raw data. data/README.md explains how to place a copy of louka.db; the location can be overridden with WP2_DB_PATH.

# 🔬 Methodology

1. Extraction — houses only, price > 0, description ≥ 20 characters; minimal preprocessing (src/data.py).
2. Embedding — all-MiniLM-L6-v2 (22.7M params, 384-d, L2-normalised), batch 256 (src/embeddings.py).
3. Projection — 20 French reference descriptions (146–257 chars, verbatim in paper Appendix A), designed for semantic saturation, dimensional specificity and linguistic consistency; one matrix product yields the 17,087 × 20 similarity matrix (src/references.py).
4. Estimation — nested OLS models A–E on log price, standardised covariates, HC3 errors; joint F-tests; quantile regressions; bootstrap; winsorisation; VIF; Lasso/Elastic-Net selection; PCA benchmark (src/models.py, scripts/04_robustness.py).

The 20 dimensions span six domains: Quality & Standing · Condition · Physical · Location & Setting · Style & Character · Market Signals.

# 🖼 Figures & tables inventory

Tables (paper/tables/*.tex): descriptive statistics, similarity statistics, model comparison, full Model D results, parsimonious model, quantile regressions, robustness summary, reference texts, text comparison (7 machine-generated by 06_tables.py, 3 hand-maintained).

# ⚠️ Limitations

• Cross-sectional listing (asking) prices, not transaction prices; coefficients are conditional associations.
• Descriptions truncated at ~700 characters in the export.
• Similarity dimensions are highly collinear (mean VIF 17.2): individual coefficients are partial associations within a correlated block; block-level inference is the robust object.
• Reference texts embed the researcher's definitional choices; the PCA benchmark quantifies the fit cost of interpretability.

# 📝 Citation

@techreport{boucher2026decoding,
  author      = {Boucher, Simon-Pierre},
  title       = {Decoding Real Estate Descriptions: Semantic Embeddings and
                 Hedonic Pricing of Residential Properties in Quebec},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {2},
  year        = {2026},
  month       = {May}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

• Format follows Recherche-UQO/HARMONISATION.md (2026-10-17): template title page, natbib + aer bibliography style, docs/ for the restructuring reports.
• The original flat May 2026 project (immo-wp2-spb-20260519) is archived in Recherche-UQO/_archives/versions-mai-2026/decoding-real-estate-descriptions--immo-wp2-20260519/; docs/AUDIT.md and docs/CHANGES.md document the 2026-08-05 restructuring and the numeric corrections it caught (old → new values).
• Build verified 2026-10-17: 0 errors, 0 undefined references, 86 references rendered.