SPB Git

spb/wp37_uqo

Public
4.2 KB · 41 lines markdown
1# Referee report 2 — WP37 (internal, skeptical: data provenance, integrity, reproducibility, citations)23Role: data and integrity referee. Checks whether every statement in the manuscript is traceable to an authorised, logged artefact and whether the integrity rules of the mandate were respected.45## Provenance and authorisation6- Raw data: owner export (`.backup` through a read-only connection) of a database owned by the PI; SHA-256 recorded and re-verified at every ingestion (`src/common.verify_snapshot`). Production untouched (no write path exists in the pipeline). **Pass.**7- `robots.txt` disallows `/api/`; the public API was hit exactly once (homepage cross-check), logged in `reports/phase0_feasibility.md`. **Pass.**8- No credentials in logs: `logs/phase0_api_credentials.txt` shows only key length/prefix. `.env` is git-ignored. **Pass.**9- External data: Statistics Canada WDS, public, raw JSON kept with timestamps. **Pass.**1011## Integrity rules of the mandate12| Rule | Check |13|---|---|14| No fabricated observations | all tables generated by code from the snapshot; `tests/test_outputs.py` ties counts to source |15| Listed ≠ paid prices; disappearance ≠ sale; tag ≠ discount | stated in §1, §3, §9; promotions "inferred" throughout |16| Do not assume history/stores/IDs/API | Phase 0 report documents each; stores: unavailable; IDs: UPC for 2 feeds only |17| 54 banners ≠ 54 chains | retailers table: 61 feeds → 52 banners → 37 companies; channel split; same-company flag |18| Matched counted separately | attrition table: 13 088 matched of 124 579 |19| Exploratory vs causal | Model C explicitly descriptive; lead placebo reported as failed |20| Preregistration before results | `research/preregistration.md` hashed; three amendments logged with rationale |21| Null results reported | H5 market-level null; H3 economically null; Model F naive beats ML on MAE |22| No production change | prospective design described, not implemented |2324## Reproducibility25- Full pipeline re-run from the raw snapshot after the manuscript was written (`logs/rerun/full_rerun.log`); JSON summaries compared by hash before/after — see `reports/reproducibility.md` for the outcome.26- Independent recomputation of the weekly modal price, H1 median gap, AR(1), banner effects (pairwise within-estimator), promotion share, volatility ratio and chained Jevons with a different code path (DuckDB SQL + statsmodels) — `reports/audit_recompute.json`.27- Seeds fixed; LLM annotation stored so the pipeline is API-independent.2829## Citations30- 75 references; 73 resolved through Crossref DOIs, 2 (Varian 1980, Feenstra 1994) documented as DOI-less with JSTOR stable URLs, 1 report (Competition Bureau 2023) with URL. Three references first resolved to SSRN preprints were re-resolved to the journal versions (Chevalier–Kashyap–Rossi 2003 AER; Hottman–Redding–Weinstein 2016 QJE) or marked as a working paper (Gourville–Koehler 2004). `references.bib` is generated from the verified CSV; no hand-typed entries. **Pass.**31- Every claim in the literature section is linked to a verified key in `literature/evidence_matrix.csv`.3233## Concerns341. The LLM annotator and the reviewer share the same prompt context in effect (the operator read the pairs after seeing the LLM labels); independence is not established. The manuscript now says "PI adjudication pending".352. Company attributions for affiliated banners (Richelieu, Marché Ami, Axep, Inter-Marché, Club Entrepôt, Rachelle-Béry) come from the Food-Ka registry text; they affect only the same-/cross-company split and are marked `ownership_basis = registry`. Should be verified against corporate sources before publication.363. Three post-registration amendments were needed. Each is defensible and logged, but A-001 changes the basket sample materially (7 → 15–18 items); the preregistered version's result (7 items) is not reported in the paper — it should be mentioned in §8. (The decision log has it.)374. The `Autres` category (19 % of active listings) is a residual bucket; category-level results exclude it implicitly where n < 50 but the index results include it.3839## Recommendation40Provenance, integrity and citation gates pass. Reproducibility gate conditional on the re-run comparison in `reports/reproducibility.md`.41
Files · 6b98ba74da2dd3460762f3a83a17630795fba248
Menu

Theme