Reproducibility report — WP37 (2026-11-01)
1. Full re-run from the raw snapshot
After the manuscript was compiled, the entire pipeline was re-executed from the checksummed raw snapshot with
make ingest panel match validate phase4 phase5 robust tables figures (log: logs/rerun/full_rerun.log). The JSON summaries of
every model (experiments/*.json) were hashed before and after (logs/rerun/before_sha256.txt, after_sha256.txt):
| Summary | Changed keys (excluding NaN≠NaN artefacts) |
|---|---|
| modelA, modelB, modelC, modelD, modelE, robustness | 0 — byte-identical |
| modelF, qgpai | 0 numerical differences (only NaN fields compare unequal by definition) |
Determinism holds: fixed seed 20261101 for the bootstrap, permutation placebo, basket sampling, LightGBM; DuckDB and pandas operations are order-stable.
One incident was found by the re-run and fixed: the match step regenerated the validation sample file and overwrote the
annotated labels (the file is in the git-ignored data/processed/). The sample itself is deterministic (identical 240 pairs); the
LLM annotation was re-run (same model, labels compared with the archived round-1 file), the operator labels were re-applied, the
matching step now refuses to overwrite an annotated sample, and the annotated CSVs are force-added to git.
2. Independent recomputation (src/audit/independent_recompute.py → reports/audit_recompute.json)
A second implementation, using DuckDB SQL from the raw event log and statsmodels instead of the pipeline's pandas/pyfixest code, reproduces the central statistics:
| Quantity | Pipeline | Independent | Note |
|---|---|---|---|
| Weekly modal price (880 072 listing-weeks) | — | 99.98 % identical | residual: tie-breaking at equal durations |
| H1 median cross-banner log gap | 0.1542 | 0.1542 | 30 571 vs 30 042 group-weeks (SQL version does not drop unrealistic-price flags) |
| H1 cross-company median gap | 0.1825 | 0.1825 | |
| AR(1) of banner-pair gaps | 0.663 (0.014) | 0.669 (0.014) | pyfixest vs statsmodels cluster-robust OLS |
| Super C effect vs Metro | −0.100 (0.003) | −0.102 (0.004) | pairwise within-estimator on 35 430 obs |
| Maxi / Club Entrepôt / Provigo vs Metro | −0.184 / −0.136 / −0.043 | −0.248 / −0.177 / −0.053 | pairwise direct comparisons (n = 948 / 790 / 1 044) vs full two-way FE using all groups; same sign and ordering, magnitudes differ on these small direct subsamples |
| Promotion share (storefront, ≥ 8 weeks) | 0.0457 | 0.0458 | |
| Volatility ratio raw / reference | 1.922 | 1.915 | |
| Chained Jevons, storefront | +0.28 % (all listings) | +0.09 % (≥ 8-week listings) | different samples by construction |
3. Source-count validation
tests/test_outputs.py ties the processed tables to the snapshot (124 579 listings, 50 760 active, 300 613 events), checks index
bases, basket monotonicity in K, forecast origins and QGPAI shape. 15 tests pass (make test).
4. Environment
Python 3.12.x (uv), requirements-lock.txt (97 pinned packages; pulp<3 required), TeX Live 2026 / latexmk 4.88, macOS 27.
Wall-clock for the full pipeline on an 18-core laptop: ≈ 10 minutes (Model A ≈ 3 min, Model F ≈ 2 min).
5. Gates
| Gate | Status |
|---|---|
| Re-run identical | pass |
| Independent recomputation of central coefficients | pass (sign, order and magnitude for primary statistics; direct-pair banner effects differ in magnitude as expected) |
| Source counts | pass |
| Citation integrity | pass (75/75 resolved; references.bib generated) |
| Human adjudication of validation labels | pending (operator review only) |
| Publication-ready | no — working paper status; see referee_1.md, referee_2.md, limitations.md |