Source connectors
Every source lives in connectors/<engine>/<source-id>/index.ts and is declared in registry.json (§106).
Adding a marketplace requires only (§197):
- metadata — an entry in
registry.json(modulepoints to the folder, e.g."api/scryfall"); - extraction schema — the payload shape the connector captures in
crawl()(typed with Zod); - normalization adapter — a pure
normalize(raw)returning canonicalNormalizedRecord[]; - fixtures — at least one real captured payload in
data/fixtures/<source-id>/*.jsonand a*.test.tsnext to the connector callingrunFixtureSuite.
The generic pipeline (raw store → normalizer → entity resolution → dedupe → validation → canonical tables → valuation → indices) then works without changes.
Rules: never invent data, keep source quirks inside the connector, store the source's own dates, keep the native currency, respect robots/terms/rate limits, and never bypass access controls (§179).
Module contract: export default (meta: ConnectorMeta) => RareIndexConnector.