# π QHPI β Quebec Housing Price Index
### Quality-adjusted, hierarchically pooled housing price indexes for Quebec
*Province Β· 17 administrative regions Β· major municipalities Γ {all, unifamilial, condo, plex}*
[](https://www.indexqc.house)
[](#-methodology)
[%20%2B%20weekly%20(research)-8a2be2?style=flat-square)](#-methodology)
[](#-index-families--coverage)
[-orange?style=flat-square)](#-methodology)
[](#-data-policy--no-microdata-in-this-repo)
[](engine/pyproject.toml)
[](api/)
[](web/)
[](db/)
[](docker-compose.yml)
[](paper/)
[](mailto:contact@spboucher.ai)
[](mailto:contact@spboucher.ai)
**Author : Simon-Pierre Boucher β [contact@spboucher.ai](mailto:contact@spboucher.ai)**
---
## Table of contents
- [What this is](#-what-this-is)
- [Key numbers](#-key-numbers)
- [Data policy β no microdata in this repo](#-data-policy--no-microdata-in-this-repo)
- [Platform architecture](#-platform-architecture)
- [Methodology](#-methodology)
- [Validation](#-validation)
- [Index families & coverage](#-index-families--coverage)
- [Canonical dataset](#-canonical-dataset)
- [API](#-api)
- [Dashboard](#-dashboard)
- [Repository map](#-repository-map)
- [Quickstart](#-quickstart)
- [Engine pipeline](#-engine-pipeline)
- [Engineering standards](#-engineering-standards)
- [Research outputs & paper](#-research-outputs--paper)
- [Reproducibility](#-reproducibility)
- [Author & license](#-author--license)
---
## π― What this is
QHPI is a **production-grade economic-measurement platform**, not a median-price
tracker. It separates **price movement** from **composition movement**: a week
where only mansions sell must not register as a price increase.
The platform has four layers:
| Layer | What it does | Where |
|---|---|---|
| **Index Engine** | Reproducible econometric pipeline: cleaning β hedonic estimation β hierarchical pooling β validation β canonical Parquet | [`engine/`](engine/) |
| **Data Store** | Versioned Parquet lake (aggregated series only) + PostgreSQL serving layer | [`data/processed/`](data/processed/), [`db/`](db/) |
| **API** | FastAPI service exposing every published series with uncertainty, reliability and vintages | [`api/`](api/) |
| **Dashboard** | Interactive Next.js frontend β explore, compare, map, PDF reports, live at [www.indexqc.house](https://www.indexqc.house) | [`web/`](web/) |
Every published observation looks like this:
```
Geography: Quebec City | Type: Condo | Period: 2026-06
Index: 163.5 (2021 = 100) | 1m: +0.4% | YoY: +8.1%
Representative value: $389,200 | Transactions: 47 | Reliability: A
95% CI: [160.9, 166.1] | Vintage: 2026-08-08 | Model: 2.1.0
```
Uncertainty is **never hidden**: every row carries a 95% confidence interval,
an AβE reliability grade, effective sample size and shrinkage weight.
---
## π Key numbers




| Metric | Value |
|---|---|
| Raw transactions ingested | ~745,000 (2021-01 β present) |
| Estimation sample after documented exclusions | ~720,000 |
| Geographic units | Province + 17 administrative regions + major municipalities |
| Property types | `unifamilial`, `condo`, `plex` (+ `all` composite) |
| Published cells | 81 β 50 **direct** liquid cells + 31 **hierarchical** thin cells |
| Headline frequency | **Monthly** (v2.1) β weekly v1 retained as a research module |
| Base | 2021 average = 100 |
| Reliability grades | **A** β₯150 tx/mo Β· **B** β₯75 Β· **C** β₯40 Β· **D** β₯15 Β· **E** <15 (model-implied) |
| Growth horizons | 1m, 3m, 6m, YoY (weekly module: 1w, 4w, 13w, 26w, YoY) |
| Vintages | `first_release` preserved; every revision queryable |
---
## π Data policy β no microdata in this repo
> **This repository contains NO individual transaction data.**
| Category | In repo? | Detail |
|---|---|---|
| Raw transaction CSV (`data/raw/`) | β **excluded** | Individual sale records β never versioned |
| Transaction-level intermediates (`data/interim/`, `transactions_clean.parquet`, `geo_join.parquet`) | β **excluded** | Anything with one row per transaction |
| Cadastral boundary files (`SDA.gpkg`) | β excluded (size) | Source, URL, version and CRS documented in [`data/external/SDA_SOURCE.md`](data/external/SDA_SOURCE.md) |
| **Aggregated published series** | β
included | `qhpi_monthly.parquet`, `qwhpi_weekly.parquet`, liquidity, coverage matrices, assessment gap, first releases, vintages |
| **Aggregate tables & figures** (`outputs/`) | β
included | Audit counts, model comparisons, validation results, publication figures |
The [`.gitignore`](.gitignore) enforces this at the repo boundary. Aggregates
are cell-level (`geography Γ type Γ period`) with transaction **counts**, never
transaction rows. To rerun the full pipeline you must supply your own
`data/raw/province_transactions.csv` (see [Reproducibility](#-reproducibility)).
---
## π Platform architecture
```
βββββββββββββββββββββββββββββββββββββββββββββββ
β province_transactions.csv β
β (local only β NOT in this repo) β
ββββββββββββββββββββββββ¬βββββββββββββββββββββββ
βΌ
ββββββββββββββββββββββββββββββ ENGINE (Python) ββββββββββββββββββββββββββββββ
β 01 profile β 02 geography (spatial join, SDA boundaries) β 03 clean β
β β 04β08 weekly research module (baseline, hierarchical, validation) β
β β 09 monthly estimation (rolling time-dummy, Huber-IRLS) β
β β 10 monthly validation β 11 canonical parquet β
ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββ
βΌ
data/processed/qhpi_monthly.parquet β source of truth
(+ liquidity, coverage, vintages, assessment gap)
βΌ
βββββββββββββββ PostgreSQL (docker-compose, Alembic) βββββββββββββββ
β series Β· observations Β· geographies Β· liquidity Β· vintages β
ββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ
βΌ
FastAPI /v1/* (uncertainty always returned)
βΌ
Next.js dashboard β www.indexqc.house
(explore Β· compare Β· map Β· PDF reports Β· methodology)
```
Weekly refresh: `ops/scheduler/` ingests new rows β append-only clean β
re-estimation β new `data_vintage` β DB load β API cache invalidation.
---
## π§ͺ Methodology
The index is a **robust hedonic time-dummy index with hierarchical pooling** β
three estimation regimes arbitrated by empirical validation (repeat sales +
stratified matched-cell medians), not by convenience:
### 1. Province Γ type backbone β rolling time-dummy (RTD)
- Pooled robust regressions (`log(price)` on structural characteristics +
fine location fixed effects + period dummies) over **13-month rolling
windows** with **Huber-IRLS** weighting.
- Windows are combined by **mean splice**, making published history
**revision-free** by construction.
- Coefficient drift is monitored and published
(`outputs/tables/rtd_coefficient_drift.csv`) β e.g. the floor-area
elasticity moves 0.556 β 0.586 across windows, which a fixed pooled model
would have hidden.
### 2. Liquid cells (median β₯ 40 tx/month) β direct local estimation
Each of the **50 liquid `geography Γ type` cells** gets its **own robust
time-dummy regression** (own hedonic coefficients, own FSA fixed effects),
lightly smoothed by a state-space filter. This was arbitrated against
deviation-from-pooled-surface modelling, which **compressed genuine local
divergence** β Quebec City condos moved ~+55% (confirmed independently by
repeat sales *and* stratified matched-cell medians) while pooled deviations
reported only +42%.
### 3. Thin cells β hierarchical Kalman shrinkage
The remaining **31 thin cells** follow a latent random-walk **deviation from
their parent's published path**, estimated by Kalman filter/smoother with
observation noise ΟΒ²/nβ. Thin months shrink toward the parent trend; the
shrinkage weight is published on every row.
### Hedonic specification (headline = structural only)
- `log(amount)` on `log(floorArea)` (+ time-stable missingness indicator),
age from `yearBuilt` via **conditional-median imputation** (its missingness
drifts 15% β 2% over the sample, so a "missing" bin would leak time),
and fine location fixed effects (FSA).
- **`buildingType` is banned from all hedonic models.** The provider
backfilled it in region-staggered waves (most regions 2022-01, Montreal
2023-01); the time-correlated missingness fabricated a β38 log-point cliff
in local Montreal indexes. The ban is documented in
[`engine/src/qwhpi/features.py`](engine/src/qwhpi/features.py).
- Assessment values (`totalArValue`, `previousValue`) never enter the
headline (valuation leakage); they power the separate
**Assessment Gap Index** (`amount / totalArValue`).
### Cleaning
Raw data is immutable; every exclusion is economically justified and counted
in [`outputs/tables/exclusions.csv`](outputs/tables/exclusions.csv)
(assessment-ratio band, exact duplicates, non-arm's-length transfers, β¦).
Duplicates are **flagged, never silently deleted**. The `indΓ©terminΓ©`
property type never contaminates type-specific indexes.
### Geography
Transactions are spatially joined (`lat/lng`) against the authoritative
Quebec **SDA cadastral boundaries** (source, version and CRS documented in
[`data/external/SDA_SOURCE.md`](data/external/SDA_SOURCE.md)) β the free-text
`city` field alone is not trusted.
---
## β
Validation
Every claim below has a table or figure in [`outputs/`](outputs/):
| Check | Result |
|---|---|
| **Repeat-sales benchmark** (BMN, ~85k repeated addresses) | June 2026, 2021=100: quebec/condo **132.0** vs RS 132 (exact); quebec-city/condo **163.5** vs RS 162; montreal/condo **121.6** vs RS 122; montreal/unifamilial **126.2** between stratified 120 and RS 135 (RS renovation bias on older stock) |
| **Composition-shock simulation** | Mix shock moves the raw median **+11.7%**; the hedonic index moves **β0.7%** β the index measures prices, not composition |
| **Downsampling experiment** | Montreal condos thinned to 100/50/25/15/10/5 tx/period; RMSE, bias, volatility, turning points and CI coverage justify the AβE liquidity tiers empirically |
| **Weekly vs monthly** | Same methodology at both frequencies; signal-to-noise, turning-point detection and revision magnitude quantified in `outputs/tables/weekly_vs_monthly.csv` |
| **Seasonality** | NSA published; no stable seasonality demonstrated (`seasonality_tests.csv`) |
| **Alternative specs** | Hierarchical variants, mixed effects, ML residualization benchmarked in `estimation_methods.csv` β chosen model optimizes interpretability + stability, not transaction-level RMSE |
Full write-ups: [`outputs/reports/`](outputs/reports/) (audit, geography,
cleaning, hierarchical, validation v1 & v2, methodology research).
---
## πΊ Index families & coverage
| Family | Scope |
|---|---|
| **QHPI-QC** | Province of Quebec |
| **QHPI-REG** | 17 administrative regions |
| **QHPI-CITY** | Montréal, Québec, Laval, Gatineau, Longueuil, Sherbrooke, Trois-Rivières, Saguenay, Lévis, Drummondville, ⦠as liquidity permits |
| **QHPI-TYPE** | All / Unifamilial / Condo / Plex per supported geography |
| **Assessment Gap** | `amount / totalArValue` β separate concept, never mixed with the price index |
The **coverage matrix**
([`data/processed/coverage_matrix_monthly.parquet`](data/processed/),
[`outputs/tables/coverage_monthly.csv`](outputs/tables/coverage_monthly.csv))
declares per `municipality Γ type`: published / conditional / not published.
---
## π Canonical dataset
[`data/processed/qhpi_monthly.parquet`](data/processed/qhpi_monthly.parquet)
(7,504 rows) powers the DB, API and dashboard. Weekly research module:
[`data/processed/qwhpi_weekly.parquet`](data/processed/qwhpi_weekly.parquet)
(32,592 rows).
```
period, geography_level, geography_id, geography_name, property_type,
index, index_smoothed, representative_value,
transactions, effective_sample_size,
monthly_pct, three_month_pct, six_month_pct, yoy_pct,
lower_95, upper_95, reliability_grade, shrinkage_weight,
is_partial_month, model_version, data_vintage
```
- `index` = raw estimate, `index_smoothed` = **one-sided** (real-time safe)
smoother β no look-ahead in the published real-time series.
- `representative_value` = dollar value of a documented, fixed property
basket per segment.
- Vintage framework: first releases preserved under
`data/processed/vintages*/`, revisions queryable via the API.
---
## π API
FastAPI service (OpenAPI docs at `/docs`), CSV/JSON export toggle, ETag
caching, pagination, rate limiting, CORS. **Reliability and CIs are always
returned.**
| Endpoint | Purpose |
|---|---|
| `GET /v1/index?geography=quebec-city&type=condo&from=2021-01&to=latest` | Full series |
| `GET /v1/index/latest?geography=β¦&type=β¦` | Latest observation |
| `GET /v1/geographies` Β· `GET /v1/geographies/coverage` | Hierarchy + coverage matrix |
| `GET /v1/liquidity?geography=β¦&type=β¦` | Transaction counts & effective N |
| `GET /v1/compare?series=montreal:condo,quebec-city:condo` | Multi-series comparison |
| `GET /v1/map?metric=yoy&level=region` | Choropleth payloads (+ `period` for time-lapse) |
| `GET /v1/stats` Β· `GET /v1/stats/overview` | Peak/drawdown/CAGR/volatility/momentum/rank, heatmap rows |
| `GET /v1/report?series=β¦` Β· `GET /v1/report/market` | Publication-grade PDF reports (matplotlib) |
| `GET /v1/vintages?geography=β¦&type=β¦` | Revision history |
| `GET /v1/meta` | `model_version`, `data_vintage`, methodology link, author credit |
| `GET /v1/health` | Liveness |
Sample response:
```json
{
"geography": "Quebec City",
"property_type": "condo",
"frequency": "monthly",
"latest_index": 163.5,
"representative_value": 389200,
"monthly_change": 0.4,
"yoy_change": 8.1,
"transactions": 47,
"reliability": "A",
"lower_95": 160.9,
"upper_95": 166.1,
"is_partial_month": false,
"author": "Simon-Pierre Boucher",
"contact": "contact@spboucher.ai"
}
```
---
## π» Dashboard
Live at **[www.indexqc.house](https://www.indexqc.house)** β Next.js +
TypeScript, dark/light themes, fully responsive (audited at 390/360 px),
`prefers-reduced-motion` respected, zero-dependency custom hero chart,
βK command palette over all 112 series.
| Page | Contents |
|---|---|
| `/` | Market pulse: heat-shaded region table with sparklines, 12 metric tiles, ticker tape, top movers, market-report PDF |
| `/explore` | Series picker, CI band, raw-median & smoothed toggles, 1Y/3Y/5Y/YTD/MAX ranges, Bank-of-Canada event annotations, brush zoom, PNG/CSV export, sortable data table, shareable URLs |
| `/compare` | Multi-series with rebasing tool, toggleable legend chips, comparison PDF |
| `/map` | Region choropleth (YoY, 3m, level, assessment gap) with monthly slider + βΆ time-lapse |
| `/methodology` | Rendered methodology + link to the paper |
| `/api-docs` | Live playground (run requests in-page) + copyable curl examples |
Low-reliability series are **visually flagged, never hidden**. Footer credit:
*Simon-Pierre Boucher β contact@spboucher.ai*.
---
## π Repository map
```
βββ engine/ # Index engine (Python)
β βββ src/qwhpi/ # ingest Β· clean Β· geography Β· features Β· hedonic
β β # rtd Β· hierarchy Β· state_space Β· repeat_sales
β β # index Β· uncertainty Β· seasonal Β· nowcast
β β # vintages Β· export Β· plotting
β βββ scripts/ # 01_profile β¦ 11_monthly_canonical (ordered, idempotent)
β βββ tests/ # smoke pipeline, features/clean, state-space
βββ data/
β βββ external/ # boundary-source documentation (files excluded by size)
β βββ processed/ # β
aggregated series only (see Data policy)
βββ db/ # PostgreSQL schema + Alembic migrations
βββ api/ # FastAPI app: routers/ services/ tests/
βββ web/ # Next.js dashboard: app/ components/ lib/ styles/
βββ ops/ # weekly-refresh scheduler + CI pipeline
βββ outputs/
β βββ figures/ # publication figures (shared plotting config)
β βββ tables/ # 35+ audit/validation/model tables (aggregates)
β βββ reports/ # markdown reports for every pipeline stage
βββ paper/ # LaTeX methodology paper (+ compiled PDF)
βββ scripts/check_headers.py # CI gate: every file carries the author header
βββ Makefile # one-command targets
βββ docker-compose.yml # postgres + api + web + scheduler
```
---
## π Quickstart
> Requires the raw transaction CSV (not distributed β see
> [Data policy](#-data-policy--no-microdata-in-this-repo)) for `make pipeline`.
> The API and dashboard run off the **included aggregated Parquet** without it.
```bash
# Engine (full run ~ audit β geography β clean β estimate β validate β canonical)
make pipeline
# Incremental monthly refresh (clean β estimate β canonical, ~2 min)
make refresh
# Services
make api # FastAPI on :8080 β docs at /docs
make web # Next.js dashboard on :3000
make up # docker compose: postgres + api + web + scheduler
make down
# Quality
make test # engine + api test suites
make lint # ruff + mypy + tsc
make headers # author-header check (also a pre-commit hook + CI gate)
make figures # publication figures β outputs/figures/
make paper # compile the LaTeX paper
```
---
## βοΈ Engine pipeline
| Script | Stage |
|---|---|
| `01_profile.py` | Data audit: schema, descriptives, missingness, sentinel values, liquidity matrices |
| `02_geography.py` | SDA boundary spatial join, validation, persistence |
| `03_clean.py` | Research sample with fully documented exclusions |
| `04_baseline.py` | Weekly pooled hedonic time-dummy baseline *(research module)* |
| `05_hierarchical.py` | Weekly hierarchical state-space shrinkage *(research module)* |
| `06_validation.py` | Repeat sales, downsampling, composition shock *(weekly)* |
| `07_canonical.py` | Weekly canonical Parquet *(research module)* |
| `08_figures.py` | Publication figures |
| `09_monthly.py` | **Headline monthly estimation** (RTD + direct cells + Kalman) |
| `10_monthly_validation.py` | Monthly validation suite |
| `11_monthly_canonical.py` | **Canonical `qhpi_monthly.parquet`** + vintage write |
Every run emits a structured **run manifest** (input hash, row counts, model
version, timings).
---
## π Engineering standards
- **Every file** starts with the author header β enforced by
[`scripts/check_headers.py`](scripts/check_headers.py) in pre-commit **and** CI.
- Vectorized pandas/polars + numpy; no per-transaction Python loops; spatial
joins, features and model matrices cached as Parquet.
- Diagnostics for every production model: residual distribution,
heteroskedasticity, temporal/spatial residual patterns, coefficient
stability, effective N.
- Model selection optimizes **interpretability + stability + calibrated
uncertainty + reproducibility** β never transaction-level RMSE alone.
- CI (`ops/ci/ci.yml`): lint, mypy + tsc, tests, header check, small-sample
smoke run of the pipeline.
---
## π Research outputs & paper
- **Figures** ([`outputs/figures/`](outputs/figures/)): province aggregate,
region comparison, big-4 cities, condo by city, raw median vs hedonic,
volumes, reliability matrix, downsampling stability, assessment gap,
YoY appreciation map.
- **Tables** ([`outputs/tables/`](outputs/tables/)): audit suite, exclusions,
liquidity, coverage, model comparison, repeat-sales comparison,
downsampling, seasonality, coefficient drift, weekly-vs-monthly.
- **Paper** ([`paper/qwhpi.pdf`](paper/qwhpi.pdf)): *A High-Frequency Hedonic
Housing Price Index for Quebec* β real, researched citations only (hedonic
and repeat-sales index theory, Case-Shiller, hierarchical and state-space
indexes, spatial hedonics, index-number theory).
---
## π Reproducibility
1. Place your transaction extract at `data/raw/province_transactions.csv`
(columns: `id, date, amount, street, zipCode, city, lat, lng,
propertyType, yearBuilt, floorArea, buildingType, previousValue,
totalArValue, ownerType`).
2. Download the SDA boundary file per
[`data/external/SDA_SOURCE.md`](data/external/SDA_SOURCE.md).
3. `make pipeline` β deterministic given the same inputs; every run is
stamped with `model_version` + `data_vintage` and manifested.
Without microdata, everything downstream of the canonical Parquet (DB load,
API, dashboard, figures) is fully runnable from the aggregates included here.
---
## π€ Author & license
**Simon-Pierre Boucher**
[](mailto:contact@spboucher.ai)
[](https://www.indexqc.house)
Code, methodology, figures and documentation Β© Simon-Pierre Boucher.
All rights reserved β contact the author for reuse, data questions or
collaboration. The repository intentionally distributes **no individual
transaction records**; published aggregates carry full uncertainty metadata
so they are never mistaken for more precision than the data supports.