# CHANGES — Restructuring report (2026-08-05)

Original project: ~/Desktop/UQO/UQO_WP/immo-wp9-spb-20260529 (left untouched). New clean repository: ~/Desktop/wp9_uqo (this folder). No published number, result or scientific claim was changed; the paper's tables and number-bearing figures are built from the original outputs preserved in results/reference/.

# 1. What was moved / renamed

Values that existed only inside the published tables (regression coefficients, quantile estimates, summary statistics, model R²) were transcribed into results/reference/{coef_M*.csv, quantile.csv, summary_stats.csv, fit.json} so that every table/figure is now generated from a data file rather than hard-coded.

# 2. Code refactoring

The three live scripts (estimate_ext.py, estimate_ext2.py, make_assets.py, ~25 KB of duplicated single-letter-variable code with absolute machine-specific paths) were refactored into a package + numbered entry points:

• src/wp9/config.py — all paths (relative to the repo), constants, seeds; the metro coordinates and the sqft/acre conversions now live in exactly one place.
• src/wp9/parsing.py — documented parsers for every semi-structured raw field.
• src/wp9/sample.py — raw DuckDB → estimation sample (was upstream in the lost RE_DB_QC pipeline; reconstructed, see AUDIT.md §5).
• src/wp9/models.py — design matrix, M1–M5 ladder, absorbing least squares, FSA fixed-effect recovery, Duan smearing. The design() function, previously copy-pasted four times across scripts, exists once.
• src/wp9/plotstyle.py — shared figure style/palette.
• scripts/01_build_sample.py → data/processed/analysis.parquet
• scripts/02_estimate_core.py → ladder fits, coefficient tables, grand-model residuals, FSA premia
• scripts/03_estimate_extended.py → OOS, robustness, heterogeneity, quantile, nonlinearity, gradient, Moran's I, LOPO
• scripts/04_make_figures.py / 05_make_tables.py → all 17 figures / 6 tables, with --results {reference,reproduced} selecting the numbers tier (default: reference = published values).

Every code file carries the header Author: Simon-Pierre Boucher — contact@spboucher.ai. analysis/ml_benchmark.py was dropped (dead code: outputs absent, never cited by the paper). main_web.tex was dropped (web variant whose figure-fallback produced a PDF of placeholder boxes).

Pipeline verified end-to-end: raw DuckDB → sample → estimates → figures/tables → PDF. Regenerated tables are numerically identical to the originals; regenerated reference-driven figures are visually identical to the surviving originals; the reproduced tier matches the published estimates closely but not exactly (full side-by-side table in AUDIT.md §5).

# 3. Paper rewrite (paper/)

• Structure kept as main.tex + one file per section; preamble reorganised into labelled blocks; duplicate/unused packages removed (tabularx, multirow, float, appendix, amsthm retained only where used); metadata unchanged except Version 1.0 → 1.1.
• Tables are now \input from ../results/tables/ and figures from ../figures/, so recompiling after a pipeline run always uses current artifacts.
• Prose: editorial polish only — tightened sentences, fixed agreement/punctuation, consistent notation and hyphenation; all numbers, claims and 32 citations preserved.
• Fixed an internal inconsistency: the note under the quantile table said the size elasticity falls with price while the table and body text show it rising (0.557 → 0.598); the note now agrees with the table.
• The maps figure caption no longer promises Natural Earth provincial boundaries (the regenerated maps plot listing coordinates with province labels; the boundary shapefile dependency was removed).
• Compiles cleanly: paper/main.pdf, 26 pages, zero errors/undefined references, all 17 figures embedded (the old main_web.pdf contained none).

# 4. Items requiring your review

1. Condominium size-gradient sentence (results §5.2). The original fig_size_gradient.png is lost. Raw medians from the data show condominiums listing above houses of equal size (composition: condos sit in expensive metros), while the original text claimed they "lie below houses at every size". I rewrote that one sentence to describe the regenerated figure and the (data-supported) conditional version of the claim. Please confirm the new wording — or tell me how the original figure was constructed and I will match it.
2. Reproduced tier differences (AUDIT.md §5): sample +0.86%; M5 living-area elasticity 0.530 vs 0.547; lot coefficients differ because the original lot parser could not be fully recovered; LOPO mean 0.315 vs 0.362. If you still have any copy of the RE_DB_QC/hedonic pipeline (backup/other machine), I can close these gaps.
3. Moran scatter / fit / residual / nonlinearity-band figures are computed from the reconstructed sample (no stored micro-outputs existed); their annotated Moran's I values are taken from the original ext2.json while the scatter clouds are reproduced data.
4. ml_benchmark.py dropped — recover from the original folder if you want the ML comparison back; its outputs would need to be regenerated (requires xgboost, lightgbm, torch).
5. The paper still lists simon-pierre.boucher@uqo.ca as contact (unchanged); code headers use contact@spboucher.ai per your instruction.