spb/rareindex

1# Source connectors23Every source lives in `connectors/<engine>/<source-id>/index.ts` and is declared in `registry.json` (§106).4Adding a marketplace requires only (§197):561. **metadata** — an entry in `registry.json` (`module` points to the folder, e.g. `"api/scryfall"`);72. **extraction schema** — the payload shape the connector captures in `crawl()` (typed with Zod);83. **normalization adapter** — a pure `normalize(raw)` returning canonical `NormalizedRecord[]`;94. **fixtures** — at least one real captured payload in `data/fixtures/<source-id>/*.json` and a10   `*.test.ts` next to the connector calling `runFixtureSuite`.1112The generic pipeline (raw store → normalizer → entity resolution → dedupe → validation → canonical13tables → valuation → indices) then works without changes.1415Rules: never invent data, keep source quirks inside the connector, store the source's own dates,16keep the native currency, respect robots/terms/rate limits, and never bypass access controls (§179).1718Module contract: `export default (meta: ConnectorMeta) => RareIndexConnector`.1920Full guide: `docs/connectors/ADDING_A_CONNECTOR.md` (scaffold with `pnpm connector:new`, adapters for Shopify/21WooCommerce/sitemaps/RSS/PDF/JSON-LD, per-host policies in `domains.json` + `domains.d/`, source catalogue in22`data/sources/entries/`, resumable backfills, health and drift monitoring).23