Phase 0 — Environment, model and source audit (WP38)
Written 2026-11-01. All figures below were measured on the authorised snapshot or on the production node with read-only queries; nothing was modified in production.
0.1 Environment
| Item | Value |
|---|---|
| Working directory | ~/Desktop/UQO/Recherche/Recherche-UQO/economie/economic-geography-digital-production (git, local main; remote to be created on spbgit as wp38_uqo.git) |
| Laptop | Apple Silicon, 18 cores, 48 GB RAM, 1.0 TB free; macOS 27 |
| Cluster | MacLustr (29 Macs, 401 cores) reachable over SSH; only M4M64a (Fabri-Ka host) was used in Phase 0 |
| Python | uv venv .venv, Python 3.12, pinned via pyproject.toml (pandas, pyfixest, statsmodels, networkx, geopandas, duckdb, rapidfuzz, …) |
| LaTeX | TeX Live (latexmk, pdflatex), pandoc |
| Network | Outbound HTTPS allowed. Crossref, Statistics Canada WDS, ISQ and Données Québec are reachable |
| API keys | .env (git-ignored) holds one OpenAI key and one Anthropic key inherited from WP37; never printed |
0.1.1 Model inventory (official provider endpoints, tiny test calls)
See reports/model_inventory.md / logs/model_inventory.json (checked 2026-11-01T17:42Z). OpenAI GET /v1/models lists 140 IDs; gpt-5.5, gpt-5 and gpt-5-mini answered 200 (8-token budget fully consumed by reasoning, so empty text — expected for reasoning models with a tiny budget); gpt-5.1 returned 400 ("could not finish the message" under the 8-token budget). Anthropic lists 14 IDs; claude-fable-5-1, claude-opus-5-5, claude-sonnet-5-5, claude-haiku-5-5 all answered "OK" in about 1 s. Total cost of the inventory: below 0.002 USD. LLMs will be used only for (a) second annotation of a product-matching validation sample and (b) optional category-reclassification audit; every call will be logged in logs/run_ledger.jsonl with its cost. Update after Phase 2: the category audit showed 46 % precision of the rule labels, so the primary category labelling is a classifier trained on cached model annotations (A-002); every category-based result is also reported under the rule labels. No result depends on a live model call.
0.2 Source identification: Fabri-Ka
Fabri-Ka (www.fabri-ka.com) is an aggregator built and operated by the author (Groupe Ka). It does not sell: it discovers Quebec online stores from certification registries, sector associations, regional agri-food brands and marketplaces (deliverables/research_report.md), verifies each domain, classifies the store by origin class A–E with retained textual evidence, and then syncs each store's public catalogue endpoint (Shopify /products.json, WooCommerce Store API, Wix GraphQL, Squarespace ?format=json, Square, Votresite, generic JSON-LD) roughly every 6 hours (PM2 process fabri-ka-sync). Compliance per source is documented in docs/CONFORMITE.md (reviewed 2026-08-19/21): public endpoints only, nothing behind authentication, polite rates, removal on request.
0.2.1 Access and authorisation
- Authorised access: the author owns the platform and the database. The research copy is a first-party export; no scraping of fabri-ka.com and no use of its
/api/(whichrobots.txtdisallows for crawlers) beyond one control request to/api/statson 2026-11-01T17:37Z to date-stamp the public counters (products 390 108, stores_live 1 569, stores_registry 4 082, regions 17). The mandate's "approximately 390 456 products, 1 572 stores" was therefore an interface figure from a slightly earlier moment; the counts drift by a few hundred per 6-hour cycle. - Snapshot:
sqlite3.Connection.backup()ofM4M64a:~/fabri-ka/data/fabrika.dbon 2026-11-01T17:38:30Z, application commit689c7bb1…; tablesusers(2 rows) andinscriptions(1 row) — the only personal data — and the derived FTS index were dropped on the node before transfer. SHA-2568d963ba9…98fb, 1.24 GB, stored indata/raw/(git-ignored). The discovery registrystores.jsonandcollections_taxonomy.jsonwere copied with checksums (data/raw/README.md). - Production untouched: all queries used
mode=ro; the WAL file was not modified.
0.2.2 Schema (three tables kept)
| table | rows | key fields |
|---|---|---|
stores |
4 082 (1 569 enabled with ≥ 1 published product; 132 enabled with 0; 2 381 disabled = directory-only or dead) | id (canonical domain), name, platform, city, region (17 canonical names or blank), origin_class A/C/E, origin_confidence, origin_evidence (text), categories (JSON), discovery_sources (JSON), language, postal_prefix (RTA/FSA), lat,lng (FSA centroid via zippopotam/GeoNames, precision "fsa"), store_kind (revendeur/collectif/blank), certifications (JSON: Aliments du Québec, Les Produits du Québec, CARTV bio…), product_count, fail_streak, presumed_closed, shipping_info |
products |
465 388 (390 108 active=1 ∧ listing_status='published') |
uid (20-hex hash of store+external id), store_id, external_id, url, title, description, price (min variant), price_max, compare_at_price, currency, images (JSON), category (20 canonical keys, inferred by keyword rules fabrika/schema.py::infer_category from product type, tags, title and option names), product_type (merchant's own), tags (JSON), vendor (merchant's brand field), available, content_hash, first_seen, last_seen, active, miss_count, details (JSON: variants with price/sku/grams/available, options, attributes, categories), listing_status (published / excluded:{carte_cadeau, atelier_cours, abonnement, billet, test} / quarantine:{prix_hors_bornes, titre_vide}), price_on_request |
sync_log |
577 773 (ok 572 749, error 4 972, presumed_closed 52) | one row per store per sync: ts, store_id, found, added, updated, removed, status, message |
There is no price-history table. The database upserts by content_hash and keeps only the current price per listing; sync_log records counts, not prices. Consequently every price analysis in WP38 is cross-sectional at the snapshot date; temporal analysis is limited to listing entry/exit (first_seen/last_seen) and store-level sync counts.
0.2.3 Temporal coverage
first_seenruns 2026-08-12 → 2026-11-01 (81 days). 291 125 listings carryfirst_seen= 2026-08-12 (initial load) and a further 48 820 + 30 173 on 2026-08-19/23 (expansion waves); 1 266 stores were onboarded in ISO week 32 and 328 in week 33; only 15 stores were first observed later. Hence store onboarding ended by 2026-08-24, and product-level entry after that date reflects merchant catalogue changes (new listings, connector fixes) rather than platform expansion — this is the only usable "dynamic" margin and it must be audited connector by connector before any H4 statement.- Sync cadence: median 6.3 h between successive syncs of a store (p10 1.2 h, p90 8.8 h; 200-store sample, last 30 days). A listing becomes
active=0after 2 consecutive misses (MISS_GRACE = 2, ≈ 12 h). 73 618 published listings are inactive at the snapshot (last seen Aug 6 970, Sep 31 642, Oct 35 222). A disappearance is not a sale (sold-out products usually stay listed withavailable=0: 48 997 active listings).
0.2.4 Field-level quality and missingness (active published listings, N = 390 108)
| issue | measurement |
|---|---|
| Price missing | 14 786 (3.8 %) price IS NULL (all flagged price_on_request); 31 quarantined prix_hors_bornes (> 100 000 CAD) excluded upstream |
| Currency | CAD 389 378; USD 578 (8 stores); EUR 112; GBP 40 — non-CAD stores to be excluded from price analyses (12 stores) |
| Price distribution | p1 1.32, p25 12.95, median 25.00, p75 69.00, p95 766, p99 3 745 CAD; max 95 999.99 |
| Variants | price_max > price for 57 728 listings (14.8 %) — the listing price is the minimum variant price; details.variants carries per-variant prices for Shopify/WooCommerce |
| Compare-at (strike-through) price | 33 311 listings (8.5 %) have compare_at_price > price (on-sale indicator, merchant-declared) |
| Availability | available = 1: 336 038; = 0: 48 997; NULL: 5 073 |
| Category | 20 keys; autre = 46 094 (11.8 %); rule-based inference, no manual validation yet → WP38 will audit a stratified sample and report category precision |
| Brand / vendor | vendor non-empty for 290 827 (74.5 %); it is the merchant's own brand field (often the store name) |
| Product type / tags / description / images | non-empty for 305 860 / 261 421 / 339 265 / 360 508 |
| SKU / GTIN | SKU present inside details for Shopify/Woo variants; no GTIN/UPC field → cross-merchant matching must rely on vendor + normalised title (+ SKU where present) |
| Within-store duplicate titles | 7 411 (store, title) pairs with > 1 listing (variants listed as separate products, colour/format) |
| Cross-store identical titles (len > 15) | 2 595 distinct titles covering 5 961 listings — first candidate pool for H5 |
external_id collisions across stores |
15 033 — platform-local integer IDs, not product identity (ignored) |
0.2.5 Merchant-level quality
| issue | measurement |
|---|---|
| Region | 346 of 1 569 live stores (22 %) have a blank region, carrying 159 096 listings (41 %), including the four largest catalogues (qublivre.ca 24 946 books; ici-la.co 21 412 souvenirs; remorquetrailer.com 19 482; beteaunaturel.com 9 910). Of these 346, 117 have an FSA centroid (lat/lng) → region recoverable by point-in-polygon against the official administrative-region layer; 229 have no geographic field at all → to be resolved from stores.json (qc_postal, city mentions) or left as "unknown region" and excluded from regional analyses (sensitivity: included as an 18th pseudo-region) |
| Region provenance | assigned by (1) regional directory source (certain), (2) regions mentioned on the site, (3) FSA → region map; coordinates = FSA centroid (neighbourhood precision), never the civic address |
| Origin class | A 1 526 stores (322 007 listings), C 12 (63 885), E 31 (4 216); B and D unused. Class is a store-level prior from the discovery source (e.g. PPAQ maple producer = A 0.95; Les Produits du Québec retailer = C 0.75) adjusted by site signals; origin_evidence text retained. Many A entries read "aucun signal de localisation québécoise détecté sur le site (à vérifier)". It is not product-level manufacturing provenance |
| Certifications | 718 live stores carry at least one structured certification (Aliments du Québec, Les Produits du Québec, CARTV organic, …) — third-party store/enterprise certification, usable as a store-level provenance indicator with its own caveats |
| Platform | shopify 696 (273 504 listings), woocommerce 498 (91 100), wix 244 (8 798), votresite 42, square 36, squarespace 23, generic 20, lightspeed 5, others ≤ 2 |
| Catalogue size | median 36 listings per store, p90 324, p99 3 997; top-10 stores hold 139 735 listings (36 %) |
| Store kind | revendeur 23, collectif 3 (marketplaces such as epipresto.ca), hors_mission 1, blank 1 542 |
| Sync health | 26 stores with an error status in the last 7 days; stores.product_count equals the live published count for all 1 569 stores (consistency check passed) |
0.2.6 Known selection mechanisms (from the platform's own documentation)
Discovery favours certified, association-member and search-indexed businesses; under-represents CMAQ artisans, industrial manufacturers selling direct, Instagram/Facebook-only micro-shops, Nord-du-Québec, Etsy sellers (excluded by Etsy ToS), ASC farms; Wix stores were only connected in a later wave; Ecwid storefronts cannot be connected; Montréal was deliberately topped-up via a dedicated source (acheterquebecois_mtl, modemtl). Connector caps (max_pages 200) truncate the very largest catalogues. These mechanisms are region- and category-correlated and are central to the representativeness audit (Phase 5).
0.3 Feasibility verdict by hypothesis
| hypothesis | feasible with this source? | conditions |
|---|---|---|
| H1 regional specialization (RAA) | Yes | after region recovery; merchant-weighted and largest-merchant-excluded variants mandatory |
| H2 diversity vs regional scale | Yes (n = 17 regions → low power; report as descriptive association with bootstrap CIs) | external population/business counts from ISQ/StatCan |
| H3 concentration / long tail | Yes | rank-size of merchant catalogues, listing shares |
| H4 related diversification (dynamic) | Partially / exploratory only | no store onboarding after week 33; only within-store listing entry over ~10 weeks; no cross-snapshot category entry at region level beyond a handful of events → mark as exploratory, likely inconclusive |
| H5 price dispersion of matched products | Yes, cross-sectional only | no price history; matched products via vendor + title + variant; merchant/region collinearity: one merchant = one region → region effect identified only across merchants |
| H6 local-origin premium | Not testable at product level (no product provenance) | store-level certification contrast reported as a descriptive, clearly-labelled proxy, not as H6 |
| H7 digital-assortment complexity | Yes (17 × 20 matrix; small) | null models and merchant-preserving permutations required; eigenvector interpretation guarded |
0.4 Blockers and decisions carried forward
- No price history → temporal price analysis blocked (recorded in
reports/limitations.md). A prospective collection plan is written but not executed. - Region unknown for 22 % of live stores → Phase 2 region-recovery procedure with quality flags; all regional results reported with and without recovered stores.
- Origin class is store-level → wording throughout: "online assortment of Quebec-based merchants", never "Quebec production".
- Category labels are rule-based → stratified audit (manual + LLM second annotation, logged cost) to estimate confusion rates and propagate uncertainty.
- Public counters vs cleaned sample: 390 108 published active listings at the snapshot; the research sample will be smaller after exclusions (non-CAD, missing price, unknown region, duplicates) and the full attrition is reported in
reports/sample_flow.md.