# Research log — WP20 (From Listing Images to Property Valuation)

Chronological, append-only. Each entry: date, decision or finding, evidence/path. API keys are never written here.

## 2026-10-20 07:20 — Kick-off and environment

- Request: build WP20 in `~/Desktop/Recherche-UQO` following the author's CLAUDE.md specification.
- Folder created: `immobilier/multimodal-cost-approach-valuation` (HARMONISATION.md §1: next free number is 20; WP18 was created
  earlier today in `immobilier/housing-leverage-intergenerational-inequality`; no local or remote WP19 was found — the number 20 is
  the one requested by the author and is used as-is; note kept for the README). Target spbgit repo `wp20_uqo.git` (to be created
  when the gateway is reachable again).
- Blocker: laptop off the cluster LAN (IP 192.0.0.2, Tailscale logged out) → M1M32/spbgit and M3U96a (Vrai Prix production code)
  unreachable. Work proceeds on local copies (see `docs/implementation_audit.md` §1). `git push` deferred.
- Tooling: TeX Live (latexmk, pdflatex), Python 3.14 with numpy 2.4 / pandas 3.0 / scipy / statsmodels / scikit-learn / lightgbm /
  matplotlib / pyarrow / geopandas; project venv `.venv` (system-site-packages) with `anthropic` 1.15.0 added.

## 2026-10-20 07:30 — Gate 1: repository and data audit (done)

- Verified local copies of Vrai Prix data (`vraiprix.db`, `immoka.db` copied 2026-10-17 for the Toveli project) and of the Vrai Prix
  estimation engine (`toveli/src/lib/engine.ts`). Details and counts in `docs/implementation_audit.md`.
- Finding: Vrai Prix has no inspectable cost-approach code locally; `vp_eval.cost_est` exists as an output (54,364 listings) of a
  script not available offline. WP20 will build its own transparent cost model and treat `cost_est` as an external production
  benchmark of unknown formula.
- Finding: 20,123 house listings join to a roll unit within 15 m with complete roll fields and live images; roll vs listing year
  built agree within ±1 year in 95.3 % of cases.
- Finding: listing image URLs still resolve (HTTP 200) for all major sources, 3 months after the WP14 crawl and 3 days after the
  Immo-Ka copy.
- Decision: primary sample = Immo-Ka house listings matched to the roll (verified administrative fields available), stratified by
  region, price tier, building age, and source; WP14's Realtor/DuProprio sample kept as a secondary, text-leakage-aware sample only
  if time permits (it has no roll link).
- Decision on API use: Anthropic Message Batches API (50 % price) for all extractions; three extractor tiers (Haiku 5.5 on the full
  sample, Sonnet 5.5 on the full sample, Opus 5.5 on a subsample) so that extractor capability is itself a treatment. Spend is
  logged per batch in `results/raw/api_usage.jsonl` (tokens only).
- Decision on rights: images are fetched transiently for analysis only, never stored in the repository, never reproduced in the
  manuscript; `data/images/` is git-ignored and deleted at the end. See `docs/data_rights.md`.

## 2026-10-20 07:40 — Sampling frame and sample (Gate 3/4 preparation)

- `src/ingestion/build_frame.py`: 39,278 house listings evaluated by Vrai Prix → 7,745 after rules R1–R10 (cascade in
  `results/tables/frame_exclusions.csv`). Rule R10 (new, rights-driven): image hosts whose robots.txt disallows all automated
  retrieval (`media.remax-quebec.com`: `User-agent: * / Disallow: /`, checked 2026-10-20, `results/raw/robots_check.txt`) are
  excluded; the Anthropic URL fetcher independently refused those URLs ("disallowed by the website's robots.txt"). We do not
  route around this by downloading the files ourselves. 5,287 listings lost (RE/MAX Québec ≈ 40 % of the frame); balance check
  in `results/tables/robots_exclusion_balance.csv` (excluded listings: median price $589,900 vs $563,000 retained, land value
  $163,900 vs $132,600, floor area 125 vs 121 m²) → the retained frame under-represents RE/MAX's more urban, higher-land-value
  listings; reported as a limitation.
- `src/ingestion/sample.py`: stratified sample of 1,200 listings (cma × age band × price quartile, floor 20 per stratum, seed
  20261020), design weights saved (`weight`), 14,245 image URLs (first ≤ 12 in seller order), Opus subsample of 200.
- Smoke test (`scripts/03_smoke_test.py`): structured-output schemas must not use min/max/maxItems/maxLength (API 400) → scales
  encoded as integer enums. Mean tokens per image call ≈ 4,800 input / 330 output; per text call ≈ 3,400 / 280.
- Text masking bug found and fixed before submission: DuProprio `features` carried "Évaluation municipale : 429 000 $" → all
  feature strings are now masked and assessment/tax/price keys dropped; 0 / 1,200 listings retain a dollar amount.

## 2026-10-20 07:55 — Extraction batches submitted (Anthropic Message Batches API, 50 % price)

| batch | requests | model | images per listing |
|---|---|---|---|
| img_haiku | 14,245 | claude-haiku-5-5 | ≤ 12 |
| img_sonnet | 11,916 | claude-sonnet-5-5 | ≤ 10 |
| img_opus | 1,994 | claude-opus-5-5 | ≤ 10 (200 listings) |
| txt_haiku / txt_sonnet / txt_opus | 1,200 / 1,200 / 200 | — | masked text |

Batch ids in `results/raw/batches/*.batch_ids.txt`; schema `wp20-schema-1.0`, prompts `wp20-prompts-1.0` (`run_meta.json`).
Expected spend ≈ US$110 at batch prices (dominant: Sonnet images); actual usage logged in `results/raw/api_usage.jsonl`.

## 2026-10-20 09:10 — Gate 2 (literature) status and an honesty note

- 104 candidate references → 104 verified against primary records (Crossref works API by DOI or bibliographic query with title
  similarity ≥ 0.80 and year ±1; arXiv API for 9 preprints/PMLR-NeurIPS papers), raw JSON/Atom kept in `docs/verification/`;
  `paper/references.bib` is generated from the verified metadata only. Two initial failures were resolved manually: Lindenthal &
  Johnson (2021) is published as « Machine Learning, Architectural Styles and Property Values » (JREFE, DOI 10.1007/s11146-021-09845-1),
  not under the working title first typed; Gal & Ghahramani (2016) verified via arXiv 1506.02142 after an arXiv 500/429.
  Dropped: Goodman & Thibodeau (1995, JHR, no record found → replaced by their 1998 REE article) and Cheng (1974, Public Finance, no record).
- `docs/literature_matrix.csv` content notes are written from the team's prior knowledge of each work at abstract level; the
  works were **not re-read in this session**. Consequently the manuscript cites them for general contributions only and never
  attributes specific figures to them.
- Opus 5.5 batches (1,994 image calls, 200 text calls) still processing at 09:10; all other tiers collected. API spend to date
  ≈ US$98 (Sonnet images ≈ US$79).

## 2026-10-20 10:30 — Gates 4–7: experiments, manuscript, validation

- Collection gap found and fixed: one Haiku image batch (4,000 requests, msgbatch_01QQiU…) had ended after the last collect
  and was missing from the first full run; collected, pipeline re-run end to end (05 → 06 → 07 → figures → tables → paper).
  Final counts: Haiku 14,245 image calls (97.1 % ok), Sonnet 11,916 (96.4 % ok), Opus text 200; Opus image batch (1,994)
  still `in_progress` at 10:30 (3 h) — manuscript v1.0 states this and defers Opus image results to v1.1
  (`make collect aggregate experiments figures tables paper`).
- Key design decisions taken during analysis (all pre-dating the reading of results where they affect inference): anchor B
  (Québec 2026 guides) as headline unit cost with anchor A (Altus Montréal) as alternative; assessor-based calibration of a
  single parameter with Québec City hold-out as a diagnostic, never as the method; garage "none" reference from parking
  fields abandoned (shown unreliable against both channels); scenario effects reported among affected listings.
- Manuscript: 66 pages (main text ≈ 46 + appendices), 15 figures, 23 generated table fragments, 1,183 macros; 104 references;
  latexmk exit 0; 0 undefined references/citations; 3 overfull boxes left (table widths) to tidy.
- Validation (`scripts/12_validate_results.py`): 18/18 independent recomputations match the macros (`docs/results_validation.md`).
- Spend: ≈ US$98 before Opus images (Opus images ≈ US$18 more when billed).
- Not done / blocked: spbgit remote (gateway unreachable), production code re-inspection, MEFQ 3E text, human labels (see
  `docs/CHECKLIST.md`).

## 2026-10-20 11:05 — Remote repository

- Gateway still unreachable by LAN/Tailscale, but M1M32 answers on the MacLustr Tunnel hub (BHS64 `tunnelctl peer ls` → 10.67.0.2, handshake < 60 s). Bare repo `srv/git/wp20_uqo.git` created and `main` pushed with `GIT_SSH_COMMAND="ssh -o ProxyJump=BHS64 -o HostName=10.67.0.2"`; verified on the remote (7ccbf65, 2.78 MiB pack).