# Research log — WP20 (From Listing Images to Property Valuation)

Chronological, append-only. Each entry: date, decision or finding, evidence/path. API keys are never written here.

# 2026-10-20 07:20 — Kick-off and environment

• Request: build WP20 in ~/Desktop/Recherche-UQO following the author's CLAUDE.md specification.
• Folder created: immobilier/multimodal-cost-approach-valuation (HARMONISATION.md §1: next free number is 20; WP18 was created earlier today in immobilier/housing-leverage-intergenerational-inequality; no local or remote WP19 was found — the number 20 is the one requested by the author and is used as-is; note kept for the README). Target spbgit repo wp20_uqo.git (to be created when the gateway is reachable again).
• Blocker: laptop off the cluster LAN (IP 192.0.0.2, Tailscale logged out) → M1M32/spbgit and M3U96a (Vrai Prix production code) unreachable. Work proceeds on local copies (see docs/implementation_audit.md §1). git push deferred.
• Tooling: TeX Live (latexmk, pdflatex), Python 3.14 with numpy 2.4 / pandas 3.0 / scipy / statsmodels / scikit-learn / lightgbm / matplotlib / pyarrow / geopandas; project venv .venv (system-site-packages) with anthropic 1.15.0 added.

# 2026-10-20 07:30 — Gate 1: repository and data audit (done)

• Verified local copies of Vrai Prix data (vraiprix.db, immoka.db copied 2026-10-17 for the Toveli project) and of the Vrai Prix estimation engine (toveli/src/lib/engine.ts). Details and counts in docs/implementation_audit.md.
• Finding: Vrai Prix has no inspectable cost-approach code locally; vp_eval.cost_est exists as an output (54,364 listings) of a script not available offline. WP20 will build its own transparent cost model and treat cost_est as an external production benchmark of unknown formula.
• Finding: 20,123 house listings join to a roll unit within 15 m with complete roll fields and live images; roll vs listing year built agree within ±1 year in 95.3 % of cases.
• Finding: listing image URLs still resolve (HTTP 200) for all major sources, 3 months after the WP14 crawl and 3 days after the Immo-Ka copy.
• Decision: primary sample = Immo-Ka house listings matched to the roll (verified administrative fields available), stratified by region, price tier, building age, and source; WP14's Realtor/DuProprio sample kept as a secondary, text-leakage-aware sample only if time permits (it has no roll link).
• Decision on API use: Anthropic Message Batches API (50 % price) for all extractions; three extractor tiers (Haiku 5.5 on the full sample, Sonnet 5.5 on the full sample, Opus 5.5 on a subsample) so that extractor capability is itself a treatment. Spend is logged per batch in results/raw/api_usage.jsonl (tokens only).
• Decision on rights: images are fetched transiently for analysis only, never stored in the repository, never reproduced in the manuscript; data/images/ is git-ignored and deleted at the end. See docs/data_rights.md.

# 2026-10-20 07:40 — Sampling frame and sample (Gate 3/4 preparation)

• src/ingestion/build_frame.py: 39,278 house listings evaluated by Vrai Prix → 7,745 after rules R1–R10 (cascade in results/tables/frame_exclusions.csv). Rule R10 (new, rights-driven): image hosts whose robots.txt disallows all automated retrieval (media.remax-quebec.com: User-agent: * / Disallow: /, checked 2026-10-20, results/raw/robots_check.txt) are excluded; the Anthropic URL fetcher independently refused those URLs ("disallowed by the website's robots.txt"). We do not route around this by downloading the files ourselves. 5,287 listings lost (RE/MAX Québec ≈ 40 % of the frame); balance check in results/tables/robots_exclusion_balance.csv (excluded listings: median price $589,900 vs $563,000 retained, land value $163,900 vs $132,600, floor area 125 vs 121 m²) → the retained frame under-represents RE/MAX's more urban, higher-land-value listings; reported as a limitation.
• src/ingestion/sample.py: stratified sample of 1,200 listings (cma × age band × price quartile, floor 20 per stratum, seed 20261020), design weights saved (weight), 14,245 image URLs (first ≤ 12 in seller order), Opus subsample of 200.
• Smoke test (scripts/03_smoke_test.py): structured-output schemas must not use min/max/maxItems/maxLength (API 400) → scales encoded as integer enums. Mean tokens per image call ≈ 4,800 input / 330 output; per text call ≈ 3,400 / 280.
• Text masking bug found and fixed before submission: DuProprio features carried "Évaluation municipale : 429 000 $" → all feature strings are now masked and assessment/tax/price keys dropped; 0 / 1,200 listings retain a dollar amount.

# 2026-10-20 07:55 — Extraction batches submitted (Anthropic Message Batches API, 50 % price)

Batch ids in results/raw/batches/*.batch_ids.txt; schema wp20-schema-1.0, prompts wp20-prompts-1.0 (run_meta.json). Expected spend ≈ US$110 at batch prices (dominant: Sonnet images); actual usage logged in results/raw/api_usage.jsonl.

# 2026-10-20 09:10 — Gate 2 (literature) status and an honesty note

• 104 candidate references → 104 verified against primary records (Crossref works API by DOI or bibliographic query with title similarity ≥ 0.80 and year ±1; arXiv API for 9 preprints/PMLR-NeurIPS papers), raw JSON/Atom kept in docs/verification/; paper/references.bib is generated from the verified metadata only. Two initial failures were resolved manually: Lindenthal & Johnson (2021) is published as « Machine Learning, Architectural Styles and Property Values » (JREFE, DOI 10.1007/s11146-021-09845-1), not under the working title first typed; Gal & Ghahramani (2016) verified via arXiv 1506.02142 after an arXiv 500/429. Dropped: Goodman & Thibodeau (1995, JHR, no record found → replaced by their 1998 REE article) and Cheng (1974, Public Finance, no record).
• docs/literature_matrix.csv content notes are written from the team's prior knowledge of each work at abstract level; the works were not re-read in this session. Consequently the manuscript cites them for general contributions only and never attributes specific figures to them.
• Opus 5.5 batches (1,994 image calls, 200 text calls) still processing at 09:10; all other tiers collected. API spend to date ≈ US$98 (Sonnet images ≈ US$79).

# 2026-10-20 10:30 — Gates 4–7: experiments, manuscript, validation

• Collection gap found and fixed: one Haiku image batch (4,000 requests, msgbatch_01QQiU…) had ended after the last collect and was missing from the first full run; collected, pipeline re-run end to end (05 → 06 → 07 → figures → tables → paper). Final counts: Haiku 14,245 image calls (97.1 % ok), Sonnet 11,916 (96.4 % ok), Opus text 200; Opus image batch (1,994) still in_progress at 10:30 (3 h) — manuscript v1.0 states this and defers Opus image results to v1.1 (make collect aggregate experiments figures tables paper).
• Key design decisions taken during analysis (all pre-dating the reading of results where they affect inference): anchor B (Québec 2026 guides) as headline unit cost with anchor A (Altus Montréal) as alternative; assessor-based calibration of a single parameter with Québec City hold-out as a diagnostic, never as the method; garage "none" reference from parking fields abandoned (shown unreliable against both channels); scenario effects reported among affected listings.
• Manuscript: 66 pages (main text ≈ 46 + appendices), 15 figures, 23 generated table fragments, 1,183 macros; 104 references; latexmk exit 0; 0 undefined references/citations; 3 overfull boxes left (table widths) to tidy.
• Validation (scripts/12_validate_results.py): 18/18 independent recomputations match the macros (docs/results_validation.md).
• Spend: ≈ US$98 before Opus images (Opus images ≈ US$18 more when billed).
• Not done / blocked: spbgit remote (gateway unreachable), production code re-inspection, MEFQ 3E text, human labels (see docs/CHECKLIST.md).

# 2026-10-20 11:05 — Remote repository

• Gateway still unreachable by LAN/Tailscale, but M1M32 answers on the MacLustr Tunnel hub (BHS64 tunnelctl peer ls → 10.67.0.2, handshake < 60 s). Bare repo srv/git/wp20_uqo.git created and main pushed with GIT_SSH_COMMAND="ssh -o ProxyJump=BHS64 -o HostName=10.67.0.2"; verified on the remote (7ccbf65, 2.78 MiB pack).