# Data dictionary — WP38

Generated by `src/reports/data_dictionary.py` from the parquet schemas. Raw snapshot fields are described in `reports/00_source_audit.md`.

## `data/interim/listings_clean.parquet` — 390 108 rows × 56 columns

| column | dtype | non-null | description |
|---|---|---:|---|
| `uid` | object | 1.000 | Fabri-Ka listing id (hash of store + platform product id); stable across syncs |
| `store_id` | object | 1.000 | canonical store domain (merchant identifier) |
| `external_id` | object | 1.000 | platform-local product id (not unique across stores) |
| `url` | object | 1.000 | canonical product URL on the merchant site |
| `title` | object | 1.000 | listing title as published |
| `description_head` | object | 1.000 | first 600 characters of the merchant description |
| `description_len` | int64 | 1.000 | length of the full description (chars) |
| `price` | float64 | 0.962 | listed price = MINIMUM variant price at the snapshot (merchant currency) |
| `price_max` | float64 | 0.962 | maximum variant price |
| `compare_at_price` | float64 | 0.146 | merchant strike-through price (on-sale indicator) |
| `currency` | object | 1.000 | merchant currency code |
| `category` | object | 1.000 | Fabri-Ka canonical category (20 keys, rule-based inference; audited in reports/phase2_data_audit.md) |
| `product_type` | object | 1.000 | merchant's own product type |
| `vendor` | object | 1.000 | merchant's brand/vendor field (often the store's own name) |
| `available` | Int64 | 0.987 | merchant availability flag (1 in stock, 0 sold out, NULL unknown) |
| `content_hash` | object | 1.000 | hash of listing content used by the platform's upsert |
| `first_seen` | float64 | 1.000 | epoch seconds of first observation |
| `last_seen` | float64 | 1.000 | epoch seconds of last observation |
| `active` | int64 | 1.000 | 1 if seen in the last two syncs |
| `miss_count` | int64 | 1.000 | consecutive misses (0–2) |
| `listing_status` | object | 1.000 | published | excluded:<reason> | quarantine:<reason> (platform rules) |
| `price_on_request` | int64 | 1.000 | 1 if the merchant shows no price |
| `n_variants` | float64 | 0.756 | number of variants (Shopify/Woo/Wix details) |
| `has_sku` | bool | 1.000 | a merchant SKU is present |
| `sku` | object | 0.769 | first merchant SKU (64 chars max) |
| `grams` | float64 | 0.427 | variant weight in grams (Shopify grams or Woo kg×1000) — first variant |
| `n_options` | float64 | 0.472 | number of option dimensions |
| `variant_prices_min` | float64 | 0.697 | min of variant prices in details |
| `variant_prices_max` | float64 | 0.697 | max of variant prices in details |
| `n_variant_prices` | float64 | 0.697 | number of variant prices |
| `on_sale_flag` | object | 0.085 | Shopify on_sale flag |
| `merchant_categories` | object | 0.184 | merchant's own category path (Woo/Wix) |
| `published_at` | object | 0.701 | Shopify published_at |
| `created_at` | object | 0.701 | Shopify created_at |
| `n_images` | int64 | 1.000 | number of images |
| `tags_list` | object | 1.000 | merchant tags (JSON list) |
| `n_tags` | int64 | 1.000 | number of tags |
| `first_seen_utc` | datetime64[ns, UTC] | 1.000 | first observation (UTC) |
| `last_seen_utc` | datetime64[ns, UTC] | 1.000 | last observation (UTC) |
| `f_published` | bool | 1.000 | active = 1 and listing_status = 'published' |
| `f_live_store` | bool | 1.000 | store enabled with >= 1 published listing |
| `store_kind` | object | 1.000 | standard | revendeur | collectif | hors_mission |
| `f_hors_mission` | bool | 1.000 | store flagged out of mission by the platform |
| `title_norm` | object | 1.000 | ASCII-folded, lower-cased, punctuation-stripped title |
| `dup_group_size` | int64 | 1.000 | number of listings with the same normalised title in the store |
| `f_dup_in_store` | bool | 1.000 | duplicated title within store |
| `f_dup_rep` | bool | 1.000 | representative (first-seen) listing of its (store, title) group |
| `f_sibling_dup` | bool | 1.000 | store is a non-representative mirror domain of another store |
| `f_analysis_assort` | bool | 1.000 | ASSORTMENT SAMPLE membership |
| `store_cad_share` | float64 | 1.000 | share of the store's listings priced in CAD |
| `f_cad` | bool | 1.000 | CAD listing in a CAD store (>= 95 %) |
| `f_price_ok` | bool | 1.000 | 0 < price <= 50 000, not on request |
| `f_analysis_price` | bool | 1.000 | PRICE SAMPLE membership |
| `log_price` | float64 | 0.962 | natural log of price (price sample) |
| `on_sale` | bool | 1.000 | compare_at_price > price |
| `has_variants` | bool | 1.000 | price_max > price or n_variants > 1 |

## `data/interim/stores_master.parquet` — 1 569 rows × 41 columns

| column | dtype | non-null | description |
|---|---|---:|---|
| `id` | object | 1.000 | store domain |
| `name` | object | 1.000 | store name |
| `platform` | object | 1.000 | e-commerce platform (connector) |
| `city` | object | 1.000 | city/sector label from the FSA centroid or directory |
| `region_db` | object | 0.779 | Fabri-Ka region (directory / site / FSA map) |
| `region_pip` | object | 0.797 | region containing the FSA centroid (official MERN layer, point-in-polygon) |
| `region_city` | object | 0.250 |  |
| `region_dbfirst` | object | 1.000 |  |
| `region_final` | object | 1.000 | region_db, else region_pip, else 'Inconnue' |
| `region_source` | object | 1.000 | db | pip | unknown |
| `region_conflict` | bool | 1.000 | region_db ≠ region_pip (both present) |
| `has_geo` | bool | 1.000 | FSA centroid available |
| `lat` | float64 | 0.797 | FSA centroid latitude |
| `lng` | float64 | 0.797 | FSA centroid longitude |
| `postal_prefix` | object | 0.824 | forward sortation area (first 3 characters) |
| `phone_shared` | bool | 1.000 | phone shared by >= 3 stores (directory contamination; never used) |
| `origin_class` | object | 1.000 | store-level origin class A/C/E assigned by the platform from discovery-source priors + site signals (NOT product provenance) |
| `origin_confidence` | float64 | 1.000 | platform confidence in origin_class |
| `origin_evidence` | object | 1.000 | retained textual evidence for origin_class |
| `certified` | bool | 1.000 | store carries >= 1 structured third-party certification |
| `cert_adq` | bool | 1.000 | Aliments du Québec member |
| `cert_lpq` | bool | 1.000 | Les Produits du Québec certified |
| `cert_bio` | bool | 1.000 | CARTV organic |
| `cert_labels` | object | 1.000 | certification labels |
| `store_kind` | object | 1.000 | standard | revendeur | collectif | hors_mission |
| `language` | object | 1.000 | site language |
| `n_sources` | int64 | 1.000 | number of discovery sources |
| `first_source` | object | 1.000 | first discovery source |
| `reg_status` | object | 1.000 | registry verification status (verified/probable) |
| `product_count` | int64 | 1.000 | platform's published count |
| `n_listings` | int64 | 1.000 | assortment-sample listings |
| `n_categories` | float64 | 0.992 | distinct categories in the assortment sample |
| `share_autre` | float64 | 0.992 | share of 'autre' |
| `median_price` | float64 | 0.953 | median listed price (CAD) |
| `share_available` | float64 | 0.985 | share available = 1 |
| `share_on_sale` | float64 | 0.992 | share on sale |
| `main_category` | object | 0.992 | modal category |
| `first_listing_utc` | datetime64[ns, UTC] | 0.992 | earliest first_seen |
| `last_listing_utc` | datetime64[ns, UTC] | 0.992 | latest first_seen |
| `last_sync_utc` | datetime64[ns, UTC] | 1.000 | last sync |
| `fail_streak` | int64 | 1.000 | consecutive failed syncs |

## `data/interim/sibling_stores.parquet` — 22 rows × 5 columns

| column | dtype | non-null | description |
|---|---|---:|---|
| `sibling_group` | int64 | 1.000 | mirror-domain group id |
| `id` | object | 1.000 | store domain |
| `sibling_rep` | object | 1.000 | representative store of the group |
| `is_rep` | bool | 1.000 | is the representative |
| `n_titles` | int64 | 1.000 | distinct titles |

## `data/interim/match_pairs.parquet` — 5 453 rows × 16 columns

| column | dtype | non-null | description |
|---|---|---:|---|
| `uid_a` | object | 1.000 | listing a |
| `uid_b` | object | 1.000 | listing b |
| `tier` | object | 1.000 | matching tier A/B/C/D |
| `score` | float64 | 1.000 | rapidfuzz score |
| `store_a` | object | 1.000 |  |
| `price_a` | float64 | 1.000 |  |
| `nv_a` | int64 | 1.000 |  |
| `grams_a` | float64 | 0.469 |  |
| `cat_a` | object | 1.000 |  |
| `store_b` | object | 1.000 |  |
| `price_b` | float64 | 1.000 |  |
| `nv_b` | int64 | 1.000 |  |
| `grams_b` | float64 | 0.484 |  |
| `cat_b` | object | 1.000 |  |
| `comparable` | bool | 1.000 | variant-structure comparability flag |
| `log_gap` | float64 | 1.000 | |log price a − log price b| |

## `data/interim/match_groups.parquet` — 4 832 rows × 11 columns

| column | dtype | non-null | description |
|---|---|---:|---|
| `group_id` | int64 | 1.000 | match group (connected component over tiers A–C) |
| `uid` | object | 1.000 | Fabri-Ka listing id (hash of store + platform product id); stable across syncs |
| `store_id` | object | 1.000 | canonical store domain (merchant identifier) |
| `price` | float64 | 1.000 | listed price = MINIMUM variant price at the snapshot (merchant currency) |
| `category` | object | 1.000 | Fabri-Ka canonical category (20 keys, rule-based inference; audited in reports/phase2_data_audit.md) |
| `nv` | int64 | 1.000 | variant count |
| `grams` | float64 | 0.542 | variant weight in grams (Shopify grams or Woo kg×1000) — first variant |
| `vendor_norm` | object | 1.000 | normalised brand |
| `n` | int64 | 1.000 | group size |
| `n_stores` | int64 | 1.000 | stores in group |
| `n_cat` | int64 | 1.000 | categories in group |

## `data/interim/sync_log.parquet` — 577 773 rows × 8 columns

| column | dtype | non-null | description |
|---|---|---:|---|
| `ts` | float64 | 1.000 | epoch |
| `store_id` | object | 1.000 | canonical store domain (merchant identifier) |
| `found` | int64 | 1.000 | listings found in the sync |
| `added` | int64 | 1.000 | added |
| `updated` | int64 | 1.000 | updated |
| `removed` | int64 | 1.000 | removed |
| `status` | object | 1.000 | ok | error | presumed_closed |
| `ts_utc` | datetime64[ns, UTC] | 1.000 | UTC time |