1# RareIndex23**The global market for collectibles.** RareIndex.io is a market-intelligence, search, pricing and4index platform for collectible assets: trading cards, sports cards, watches, sneakers, LEGO, comics,5video games, toys, coins, art, wine and, over time, every category the collectibles economy contains.6It continuously collects public market data from marketplaces, auction houses, dealers, grading7companies, manufacturer catalogs and archives, resolves it into one canonical asset graph, and turns8it into research-grade data: valuations with confidence levels, comparable sales, live listings,9rarity and liquidity scores, price histories, and the RareIndex family of indices (`RARE` and its10category subindices).1112RareIndex is an intelligence platform, not a marketplace aggregator. Every number shown is traceable13to its source, every estimate carries a sample size and a confidence level, and when the evidence is14insufficient the platform says "Data unavailable" rather than inventing a figure. The full product15mandate is in [`docs/SPEC-original.md`](docs/SPEC-original.md); the condensed working version that16guides implementation is [`CLAUDE.md`](CLAUDE.md).1718## Architecture1920TypeScript end-to-end in a pnpm monorepo: Next.js 16 web app, Fastify public API, PostgreSQL 1721(pg_trgm, unaccent, pgvector) as system of record, pg-boss workers for the ingestion pipeline, and a22connector framework routing Direct API → Firecrawl → Scrapfly with quality scoring and health23metrics. See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the layout and data flow, and24[`docs/adr/`](docs/adr/) for the decisions behind it.2526```27apps/web · apps/api · packages/{shared,database,taxonomy,connectors,valuation,indices,search,ai}28workers/ · connectors/ (+ registry.json) · data/{taxonomy,fixtures} · docs/ · scripts/29```3031Data flow: `raw_records` (immutable) → `normalize()` → `normalized_records` → entity resolution →32`assets` / `asset_variants` → dedupe and price validation (flags + audit log, nothing deleted) →33`sales` / `listings` / `auction_lots` / `price_observations` → valuations and stats → indices.3435## Quick start3637Requirements: Node ≥ 22, pnpm 11, PostgreSQL 17 with the `pgvector` extension available.3839```bash40pnpm install41createdb rareindex42cp .env.example .env # fill DATABASE_URL, FIRECRAWL_API_KEY, SCRAPFLY_API_KEY, AI and Resend keys43pnpm db:migrate && pnpm db:seed # creates extensions + tables, seeds taxonomy, graders, indices, registry4445pnpm dev:web # Next.js on http://localhost:300046pnpm dev:api # public REST API (/v1)47pnpm worker # pipeline workers (crawl, normalize, resolve, value, index)48pnpm ri --help # operational CLI (run a connector, recompute valuations, health…)49```5051Useful scripts: `pnpm typecheck`, `pnpm test`, `pnpm build`, `pnpm db:generate` (after editing52`packages/database/src/schema/*`), `pnpm tsx scripts/build-registry.ts` (rebuilds53`connectors/registry.json` from per-connector `meta.json` files).5455## Adding a connector5657A new source needs only four things: connector metadata (`meta.json`), an extraction schema, a pure58`normalize()` adapter and real captured fixtures with tests. The rest of the pipeline works59unchanged. Conventions and the module contract are in [`connectors/README.md`](connectors/README.md).6061## Data honesty rules6263- No fabricated data, ever: no invented sales, prices, populations, market caps, auctions or listings.64- Price-guide values are stored as observations, never as transactions.65- Valuations are published only with sample size, confidence, low/high range and timestamp.66- Outliers are flagged and audited, never silently deleted.67- Sale dates come from the source; crawl time is never substituted.68- Historical prices are converted with the exchange rate of the sale date.6970Details: [`docs/adr/0002-data-honesty.md`](docs/adr/0002-data-honesty.md).7172## Deployment7374Production runs on a MacLustr node (PostgreSQL 17 + pgvector) deployed through the `mld` gateway:75PM2 processes `rareindex-web`, `rareindex-api`, `rareindex-worker` and an ngrok tunnel serving76**https://www.rareindex.io**. Secrets live only in the node's `.env` / the `mld` manifest.7778## Disclaimers7980RareIndex is an information platform. Valuations are estimates derived from observed public data.81Listing prices are not confirmed transactions. Past performance does not guarantee future results.82RareIndex does not authenticate items. Third-party names, marks and content belong to their owners83and are attributed to their sources; RareIndex respects source terms, robots policies, rate limits,84copyright, privacy and database rights, and never bypasses authentication, paywalls or CAPTCHAs.8586## License8788Proprietary — © 2026 RareIndex. All rights reserved.89