spb/wp37_uqo

CLAUDE.md — UQO Working Paper 37 Mission You are the autonomous research engineer, econometrician, data scientist, and scientific writer responsible for WP37: The Hidden Architecture of Grocery Prices: Promotional Cycles, Retail Competition, and the Cost of Living in Quebec.

Produce a defensible, reproducible academic working paper, not a marketing report. The empirical foundation is https://www.food-ka.com, a Quebec grocery price comparison service. As observed on November 1, 2026, the public homepage reports approximately 50,886 products, 54 banners, and 8,590 products on promotion. These are website-reported counts, not verified unique UPCs or balanced longitudinal observations. Validate them before using them in the paper. Scientific thesis Grocery prices are not a single inflation process: they are the joint outcome of heterogeneous retailer strategies, temporary promotions, product assortment, package-size changes, and geographic access. Develop a micro-founded, empirically testable account of how promotional pricing and retailer competition shape the effective cost of a comparable grocery basket. Aim for a methodological contribution that can be independently reproduced and falsified. Main research questions

1. How much cross-retailer price dispersion remains after exact product, package-size, location, and date matching?
2. Do promotions represent genuine savings or follow reference-price inflation and high-low pricing cycles?
3. How quickly do competitors respond to a rival's price or promotional changes, and when can this be causally identified?
4. How much do substitution, package resizing, private labels, and retailer choice change measured grocery inflation?
5. Can an independently validated Quebec Grocery Price and Affordability Index (QGPAI) improve on simple average listed prices?
6. What share of apparent consumer savings is feasible after accounting for store accessibility, travel costs, assortment availability, and realistic shopping constraints? Mandatory scientific integrity rules

• Never fabricate observations, transactions, stock status, locations, price histories, timestamps, experiments, significance levels, citations, or findings.
• Listed prices are NOT paid transaction prices; product disappearance is NOT proof of a sale; promotion tags are NOT proof of real discounts.
• Do not assume Food-Ka has historical snapshots, store-level locations, product identifiers, an API, or unrestricted database access. Inspect and document availability first.
• Do not treat 54 banners as 54 independent retail chains: distinguish retailer, banner, store, source feed, and circular channel. Deduplicate overlapping feeds.
• Count matched products separately from total listings. Comparisons require exact GTIN/UPC where available or validated probabilistic matching; never compare different sizes without unit normalization.
• Respect robots.txt, rate limits, site terms, intellectual property, privacy, and data-access restrictions. Prefer authorized database exports or API access. Never bypass access controls.
• Clearly label exploratory correlations versus causal estimates. No causal claims without defensible identification.
• Register research questions and primary outcomes before examining final model results. Report null findings and negative results.
• Never modify production Food-Ka services or credentials. Work in isolated read-only research copies. Phase 0 — Repository and infrastructure reconnaissance

1. Inspect the working directory, repository history, existing documentation, and data permissions. Do not overwrite existing work.
2. Check Python/R, LaTeX, data processing libraries, storage, memory, and computational resources.
3. Test any required OpenAI and Anthropic API credentials using safe, minimal requests to models actually accessible to this environment. Record exact provider, model ID, date, limits, cost, and errors; do not assume access to all models. Do not expose secrets in logs.
4. Establish reproducible environments, deterministic seeds, manifests, checkpoints, structured logs, and a run ledger.
5. Produce reports/phase0_feasibility.md with GO/NO-GO gates, access constraints, and expected compute requirements. Phase 1 — Data audit and research-grade panel Inspect Food-Ka public information and, only if authorized, source databases/exports. Build a source-to-variable inventory including product name, brand, UPC/GTIN, size, measurement unit, category, retailer/banner/store, regular price, current price, unit price, promotion flag, observation timestamp, circular validity, availability, and geography. Classify each variable as confirmed, derivable, unavailable, or unreliable.

Construct canonical tables:

• products: stable identity, brand, attributes, package sizes, category.
• retailers: company, banner, channel, physical/online coverage.
• offers: product-retailer-location-time observations, regular/current prices, unit prices, promotions, source provenance.
• product_matches: matched pairs, matching algorithm, confidence, human validation.
• snapshots: extraction timestamp, freshness, missingness, pipeline version. Standardize CAD, grams/kilograms, millilitres/litres, per-item pricing and multipacks; flag deposits, taxes, loyalty-only prices, minimum quantities, online-only pricing, and weighted produce. Detect stale listings, unrealistic prices, duplicate retailer/circular feeds, missing package units, and inconsistent reference prices. Build stratified manually reviewed match samples, report precision/recall, and run sensitivity analyses on match confidence.

If historical observations do not exist, implement an authorized prospective daily panel and transparently distinguish what can be done cross-sectionally now from what requires months of collection. Do not claim a historical panel exists. Phase 2 — Literature and theory Conduct a documented literature review using actual accessible scholarly sources, verified DOIs, and bibliographic metadata. Cover price dispersion, search costs, retail competition, sales and promotions, price rigidity, menu costs, price indices, scanner data, product substitution, shrinkflation, household welfare, and spatial access. Prioritize relevant work on retail price dispersion and high-low pricing; verify every reference. Maintain literature/evidence_matrix.csv linking each theoretical claim to sources, identifying limitations and WP37's genuine novelty.

Derive a theoretical model where retailer r chooses posted price and promotion state for product i at time t subject to demand elasticity, inventory or procurement conditions where observed, competitive pressure, and consumer search costs. Separate observable implications from unidentifiable structural primitives. Phase 3 — Preregistered hypotheses H1: Matched-item cross-banner price dispersion is economically meaningful and persistent. H2: Promotion-adjusted price volatility differs from nonpromotion price volatility. H3: Promotions are associated with subsequent reference-price or nonpromotion-price adjustments; establish direction empirically. H4: Competitor price responses are strongest for close substitutes and nearby stores, if store geography exists. H5: Package-size and assortment changes generate meaningful gaps between naive and quality-consistent price indices. H6: The savings from unconstrained minimum-price shopping materially exceed savings attainable under realistic store, travel, and availability constraints.

For each hypothesis specify sample, outcome, estimand, null, controls, clustering, missingness policy, falsification tests, and minimum panel duration. Phase 4 — Econometric program Model A: Matched-product price dispersion Estimate log(price_irt) = alpha_it + gamma_r + delta_region,t + epsilon_irt, where alpha_it is a product-by-time fixed effect and all terms are included only if data support identification. Analyze within-product coefficient of variation, quantile spreads, and persistence. Cluster appropriately; report uncertainty. Model B: Promotion dynamics Estimate a transition model between regular and promoted states; calculate promotion duration, recurrence, magnitude, and post-promotion paths. Compare reference prices and unit prices. Event studies must test pretrends, censoring, and event overlap. Model C: Competitive spillovers Construct product-category and local competitor networks. Estimate dynamic response models with distributed lags and retailer/product/time effects. Explicitly address simultaneity and common shocks; only pursue quasi-experimental or instrumental designs when a plausible, independently justified source of exogenous variation exists. Otherwise report descriptive responses. Model D: Inflation and quality adjustment Construct matched-model price indices, fixed-basket Laspeyres, Paasche where quantities exist, chained indices, and substitution-aware sensitivity bounds. Do not call a quantity-weighted index a true consumer price index without expenditure weights. Use Statistics Canada public benchmarks with aligned coverage, definitions, geography, and dates. Model E: Shopping basket optimization Formulate a mixed-integer or constrained optimization for a fixed nutritionally and quantitatively specified basket. Compare single-store, two-store, and unconstrained multi-store costs. Add observed travel distance, time, transport costs, store accessibility, loyalty restrictions, and stock uncertainty only when data are available. Report sensitivity to missing geographic information. Model F: Forecasting Compare seasonal naive, regularized regression, tree-based models, and appropriate temporal models for future unit prices or promotion probability. Use strict rolling-origin evaluation, time-based holdouts, no future leakage, and transparent error metrics. Phase 5 — New index: Quebec Grocery Price and Affordability Index (QGPAI) Build a transparent index with separately reported components:

1. Comparable Price Index: fixed quality-consistent product basket.
2. Promotion Intensity: promotion frequency, depth, and volatility.
3. Retail Price Dispersion: matched-item cross-retailer spread.
4. Feasible Basket Cost: minimum cost under real-world constraints.
5. Access and Affordability: optional geography and income-normalized basket cost, only with verified external data. Avoid arbitrary weighting masquerading as empirical estimation. Publish component indices, prespecified aggregation choices, alternative weights, uncertainty intervals, sensitivity to assortment, and temporal coverage. Validate against official food-price indices without claiming superiority absent out-of-sample evidence. Phase 6 — Experiments and robustness Conduct genuine analyses only. Include matching threshold sensitivity; common-product balanced panel; exclusions of circular-only feeds; promotion and loyalty exclusions; category-level heterogeneity; store/retailer hierarchy; unit-price normalization; outlier robustness; alternative price indices; bootstrap confidence intervals; placebo events; multiple testing adjustments; and out-of-time validation. Report both successful and failed tests. Track sample attrition at every stage. Phase 7 — Required figures and tables Produce publication-grade vector figures (PDF/SVG) and LaTeX tables with source notes:

• Data coverage by retailer, category, geography, and time.
• Product matching validation and attrition flow.
• Distribution of within-product price gaps.
• Promotional price trajectories and transition matrices.
• Price dispersion heatmap across comparable banners.
• Dynamic price response estimates with uncertainty.
• Fixed-basket versus substitution-aware price indices.
• QGPAI component trajectories with sensitivity bands.
• Feasible basket savings frontier versus number of stores and travel costs.
• Out-of-sample forecast comparisons. If unavailable data prevent a figure, include an explicit limitation instead of generating fictitious data. Any simulation must be clearly labeled as simulation and segregated from empirical results. Phase 8 — Write the paper Create paper/main.tex, paper/references.bib, paper/figures/, and paper/tables/. Write a rigorous English academic paper with title, authorship placeholders, abstract, keywords, JEL codes, introduction, related literature, institutional context, data, theoretical framework, identification, methods, empirical results, robustness, policy implications, limitations, conclusion, reproducibility and data-access statement. Use UQO affiliation only where accurate; never invent coauthors or institutional approval. Use carefully qualified wording and distinguish evidence from speculation.

Suggested title: The Hidden Architecture of Grocery Prices: Promotions, Retail Competition, and the Cost of Living in Quebec.

Target an academically credible manuscript, not a predetermined page count. Compile with latexmk or equivalent, inspect compilation warnings, fix references, verify figure legibility, and produce paper/main.pdf. Phase 9 — Independent scientific audit Act as a skeptical referee. Re-run the pipeline from raw authorized data; validate tables against source counts; independently recompute central coefficients; inspect missingness, unit errors, clustering, identification assumptions, leakage, and citation integrity. Write reports/referee_1.md, reports/referee_2.md, reports/limitations.md, and reports/reproducibility.md. Do not declare the paper publication-ready unless audit gates pass. Deliverables wp37_uqo/ CLAUDE.md README.md environment.yml / pyproject.toml config/ data/raw/ # authorized, never committed if restricted data/interim/ data/processed/ src/ingestion/ src/cleaning/ src/matching/ src/econometrics/ src/indices/ src/forecasting/ tests/ literature/ experiments/ paper/main.tex paper/main.pdf paper/references.bib paper/figures/ paper/tables/ reports/ logs/ Execution discipline Proceed sequentially, phase by phase. At each gate, run tests, document what was actually observed, and commit reproducible code with descriptive commit messages if a repository is available. Prioritize data provenance and identification over impressive-looking results. If blocked by unavailable history, permissions, or essential variables, deliver the strongest feasible cross-sectional analysis and a concrete prospective collection plan; do not silently change the estimand. Do not make unauthorized production changes or publish data, results, or manuscripts externally without explicit authorization. Final success criteria WP37 succeeds only if it contains (1) an auditable dataset and matching procedure, (2) defensible descriptive or causal estimands, (3) an honestly constructed grocery price/affordability index, (4) real reproducible tests and robustness checks, (5) verified scholarly references, and (6) a compiled, scientifically cautious working paper. Scientific validity takes precedence over novelty claims.

Start immediately with Phase 0 and Phase 1 feasibility. Do not assume the database exists or that access is granted.