Referee report 1 — WP37 "The Hidden Architecture of Grocery Prices" (internal, skeptical)
Role: methodological referee (econometrics and measurement). Written against the compiled manuscript of 2026-11-01 and the replication package; every number quoted was checked against experiments/*.json and data/processed/*.csv.
Summary of claims checked
| Claim | Where | Check | Verdict |
|---|---|---|---|
| Snapshot counts (124 579 listings, 50 760 active, 300 613 events, 15 630 passes) | §4.1 | SQLite counts on the checksummed snapshot; public homepage cross-check | confirmed |
| Store-mixture oscillation (48–68 % A→B→A reversions, median 4 values, Jaccard 0.5) | §4.4 | logs/phase0_db_quality.txt, logs/phase0_store_mixture.txt |
confirmed; mechanism (proxy pool, no store selection) confirmed in connector code |
| Matching precision/recall (B: 0.913/0.984 UPC; 60/60 text) | §4.6 | match_threshold_calibration.csv, match_validation_summary.csv |
confirmed; see concern 2 |
| H1 median gap 0.154; AR(1) 0.663; placebo 0.004 | §7.1 | modelA_summary.json |
confirmed |
| H2 ratio 1.92 [1.84, 2.02]; promo share 4.6 % | §7.2 | modelB_summary.json |
confirmed |
| H3 return-episode Δlog ref −0.0003 (se 0.0001) | §7.3 | idem | confirmed; see concern 4 |
| H4 cross-company lead −0.108 (se 0.048) | §7.4 | modelC_summary.json |
confirmed |
| H5 gap 0.0006 [−0.0078, 0.0084] | §7.5 | modelD_summary.json |
confirmed |
| H6 15.4 % savings, 85 % at K=2 | §7.6 | modelE_summary.json |
confirmed |
Major concerns
- Within-company dominance of the matched sample. 93 % of matched group-weeks compare two banners of the same company. The paper is explicit about this, but the title and abstract speak of "retail competition"; the evidence is mostly about format segmentation within chains. Recommendation: keep the framing but add the share to the abstract (done in revision: "mostly inside the two chains"), and present the cross-company subsample (2 103 group-weeks) as the primary object for any competition statement.
- Validation labels. The final labels of the 240-pair sample were set by the research operator after reading the pairs and agreeing with the LLM annotator; no independent human coder was involved. Agreement of 100 % after review is therefore not an inter-rater statistic. The UPC-based precision (0.913) is the only fully independent number and should be the headline for tier B — it is, in Table 3, but the text should not describe the sample as "manually validated" without the qualifier. Recommendation: PI adjudication of at least the 60 tier-C pairs and a random 30 of tier B before any journal submission.
- Promotion inference. The 10 %-below-trailing-mode rule with an 8-week window on an 11-week panel is fragile at the panel edges; the first weeks' reference is the mode of one to three observations. The agreement of 0.90 with the current on-sale flag is reassuring but the flag itself is Food-Ka's derived field. The classification of 42 % of episodes as "reset down" shows how much the rule picks up regular-price cuts. Recommendation (addressed): report H3 on return episodes only, as done; add in the limitations that the first two weeks are mechanically promotion-poor (the QGPAI promotion component is already set to NaN in week 1).
- H3 significance vs economic size. −0.0003 with s.e. 0.0001 is "significant" only because n = 6 049 and because the return definition constrains the post price within ±2 % of the pre reference; the test is close to circular. The paper says so ("economically zero", "by construction |Δ price| ≤ 2 %"). Recommendation: state explicitly that H3 as registered cannot be powered by an 11-week panel and is reported for completeness — the manuscript does this in §6 and §9.
- Model C placebo failure. The cross-company lead coefficient (−0.108, s.e. 0.048) is reported as a failed placebo and no causal claim is made. Good. But the same-company lag-1 coefficient (0.042) is described as "coordination within a chain's pricing calendar": this is an interpretation, not a finding. Recommendation: label as interpretation (the text says "rather than competitive response", which is fine but should be flagged as hypothesis).
- Clustering. With 14 banners, two-way clustering on banner is unreliable; the paper reports group-clustered SEs as primary and two-way as sensitivity, and flags the small banner dimension. Acceptable. A wild-cluster bootstrap was not run (package not installed); mention.
- Representativeness. 52 % of active listings come from four exhaustively-read Shopify catalogues; the large chains are read on selected aisles. Market-level indices are therefore not representative of expenditure. The paper states this in §9. Fine, but Figure 1B could mislead a hurried reader — caption now says "active listings", not "market share".
Minor
- Table 5 header "Share ≥ 5 %" was truncated in the first compile (escaping bug) — fixed.
- The abstract spilled onto a third page — trimmed.
fixed Carli +2.7 %is upward-biased by construction; the text says so.- The event-study y-axis reads "Log price vs week −1"; add that week 0 is the first promoted week (caption does).
Recommendation
Acceptable as a working paper with the qualifications already in the text. Not publication-ready for a journal without (i) independent human adjudication of the validation sample, (ii) a longer panel for H3/H5, (iii) store-identified collection for H4/H6.