SPB Git forge

spb/wp21_uqo

Public
0commits 0branches 0releases
0 Bsize
maindefault branch
—last push
2.3 KB

# WP21 — Limitations (final audit, 2026-10-20)

  1. Retrieval bound. Only 18.8 % of the 2,667 stage-one hedonic/unclear studies could be obtained as legal open-access full texts; the extracted sample (297 assessed, 242 eligible) over-represents recent, open-access and non-US work. The paywalled 1980s–2000s North American journal literature is under-represented.
  2. Discovery bound. OpenAlex was budget-capped (no API key); SSRN, Google Scholar, IDEAS/EconPapers, CORE, DOAJ and repositories were not harvested; multilingual queries returned mostly English records.
  3. Agent-based screening and extraction. Independent LLM agents applied written protocols; validation on 20 papers shows 95 % eligibility agreement and 100 % numerical agreement on 647 matched coefficients, but 78–79 % recall/precision on which auxiliary rows were recorded; the two-human-reviewer standard was not met.
  4. Harmonisation choices. Construction-year variables, standardised betas, interactions, squared terms and un-harmonisable units are excluded from pooling (about a quarter of rows); Kennedy correction assumes normality; semi-elasticities per m² pool houses and apartments.
  5. Heterogeneity. I² ≈ 100 % in almost every family; prediction intervals span zero for most characteristics; moderators explain little; ML meta-regressions have negative out-of-study R².
  6. Publication selection. Egger asymmetry in 46/67 families is not interpretable as selection alone under this heterogeneity and dependence; PET-PEESE and selection models reported as sensitivity only; some selection models produce implausible values.
  7. Temporal/crisis questions (RQ4). Pre-2000 evidence thin; no powered pre/post contrast for 2008 or COVID-19.
  8. AI synthesis (RQ6). 40 retrievable studies; metrics not comparable across studies; SHAP effects not pooled with coefficients.
  9. Sirmans comparison. The 2005/2006 full texts were not accessible; the comparison uses abstract-level findings only, and the "125 studies" figure could not be verified.
  10. Reproducibility. Raw PDFs not redistributed; agent outputs are stored artefacts, but re-running the agents is not deterministic in the strict sense; all downstream computations are deterministic (seed 20261020).