1# RareIndex methodology23RareIndex turns fragmented public collectibles data into structured market data. This page explains4how every number on the platform is produced, what evidence it rests on, and where its limits are.5Nothing on RareIndex is invented: every sale, listing, guide price, population count and index value6traces back to a source record with its URL, capture time, parser version and connector version.78## 1. Data sources and pipeline910Data enters through connectors (marketplaces, auction houses, price guides, grading companies,11catalogs). Each capture is stored immutably as a raw record, then normalised into a canonical shape,12matched to a canonical asset (entity resolution), de-duplicated, validated and only then written to13the sales, listings and observations tables that feed valuations and indices.1415Three kinds of price evidence are kept strictly apart:1617| Evidence | Meaning | Used for |18| --- | --- | --- |19| **Sale** | An observed transaction with a source date and price | RIV, indices, records, liquidity |20| **Listing** | An asking price; not a transaction | Value opportunity, spread, supply signals |21| **Price observation** | A published guide value (e.g. a marketplace's market price) | Fallback estimate only, flagged as guide-based |2223Currency: every price is stored in its native currency together with the USD conversion at the24**historical** ECB reference rate of the transaction date. Display currencies are converted at view25time; historical sales are never converted at today's rate.2627## 2. Entity resolution2829Descriptions of the same object differ across sources ("1999 Pokemon Base Charizard 1st PSA 10" vs30"Pokémon Base Set Charizard Holo 1st Edition #4/102 PSA GEM MT 10"). Records are resolved in order:31deterministic identifiers (catalog ids, style codes, set numbers, references, UPC/ISBN) → a32deterministic canonical key built from structured attributes per category → trigram similarity of33titles constrained by hard discriminators (card number, set, year) → creation of a new asset. The34match method and confidence are stored on every record, and grade/condition slices are kept as35variants of the asset (PSA 10, CIB, deadstock size 10 …) so that different grades are never mixed.3637## 3. RIV — RareIndex Valuation3839RIV is an ensemble estimate of the current market value of a variant, computed from verified40transactions in the last 365 days (extended to three years when fewer than five recent sales exist,41in which case confidence is capped).4243The estimators are: latest sale, medians of the last 5/10/20 sales, a trimmed mean (10 % each tail),44an exponentially weighted mean (90-day half-life) and a weighted mean using source trust,45identification confidence and recency. RIV is the median of the core estimators. The low/high band46comes from the inter-quartile range of the sales used (widened to at least ±5 %).4748When a variant has fewer than three sales, RIV may fall back to **grade-adjusted comparables**49(sales of other grades of the same asset, adjusted by empirical grade premiums) and, as a last50resort, to **guide observations**. Fallback estimates are labelled and their confidence is capped at510.5 (comps) or 0.45 (guide-based). With no evidence at all, RareIndex shows *Data unavailable*.5253### Confidence5455Confidence (0–1, shown as High ≥ 0.75, Medium ≥ 0.5, Low > 0.2, Insufficient otherwise) combines56sample size (log-scaled, 40 sales ≈ maximum), dispersion of log prices (robust MAD), recency of the57latest sale and the trust/identification quality of the sales used. Every RIV is displayed with its58sample size, confidence and computation time.5960### Outliers6162Within a variant, sales whose modified z-score on log price exceeds 3.5 are **flagged**, not deleted;63bundles and multi-quantity lots are excluded from pricing. Every flag is written to the audit log with64its reason.6566### Grade premiums6768For each category, grader and grade, RareIndex measures the median ratio between graded and raw69sales of the same assets (at least eight paired assets). PSA 10, BGS 10 and CGC 10 are never assumed70equivalent; premiums are only ever derived from paired evidence.7172## 4. Scores7374- **Liquidity (0–100)**: sales per month, active listings, number of sources, median days between75 sales and the spread between asks and RIV.76- **Rarity (0–100)**: graded population, production quantity when documented, and observed77 sale/listing frequency. Rarity is *null* when no supply signal exists — it is never guessed.78- **Momentum (−100…100)** over 7d/30d/90d/1y: price change against the RIV history blended with79 transaction-volume acceleration.80- **Value opportunity**: `(ask − RIV) / RIV` of an active ask against the RIV of its **own variant**81 (negative = below the valuation), shown only when the gate below passes. Analytical information,82 not investment advice.83- **Data quality (0–100)**: attribute completeness, source trust, identification confidence, images84 and transaction evidence.85- **Trending**: geometric blend of price, volume and listing momenta.8687### Ask vs RIV and the anomaly gate8889An asking price is compared with a valuation only when the comparison is legitimate (the live90constants are exported from `packages/valuation/src/scores.ts` and rendered on `/methodology`):9192- same variant — a slab whose grade could not be read is kept in a "grader · grade unknown" variant93 and is never compared against the raw valuation;94- the RIV rests on transactions (comps-only and guide-only estimates never qualify an ask);95- RIV confidence ≥ 0.5 and ≥ 5 transactions used; listing identification confidence ≥ 0.7;96- plausibility band: an ask below 0.1× or above 10× RIV is flagged `riv_anomaly` and shown as97 **Data/identity anomaly** — a wrong variant, lot, currency or identity until reviewed — never as a98 deal.99100Verdicts: **deal** ≤ −10 %, **fair** between −10 % and +10 %, **premium** ≥ +10 %. The **Deal Score**101(0–100) multiplies discount depth, RIV confidence, sample size, liquidity and match confidence, so a102large discount on an illiquid or poorly identified asset cannot score high. Discounts are recomputed103from scratch on every valuation pass; a stale discount never survives a changed valuation. Period104changes beyond ±500 % are treated as data artefacts and are not published as movers.105106## 5. Indices107108Each subindex (RARE-TCG, RARE-WATCH, …) is a **chain-linked** index of daily RIV changes across its109constituents: assets with a valuation confidence of at least 0.4 and at least three sales in the last110twelve months (one variant per asset, the most traded). Each day the index moves by the trimmed,111equal-weight average of constituents' log returns; composition can change daily without breaking112the level. Base = 1000 on 2024-01-01. An index publishes nothing until it has its minimum number of113constituents (10 for subindices, 25 for RARE), and every point stores its constituent count and114coverage. Indices that have not met their minimum are listed as *in development* outside the primary115ticker.116117**RARE**, the flagship index, is the transaction-weighted average of subindex daily returns and118publishes only once at least 25 constituents exist across the published subindices.119120A **repeat-sales** estimator (pairs of sales of the same variant, monthly periods, least squares)121is provided for research views; it is less exposed to composition bias but requires many pairs.122123Historical index and asset series are reconstructed from transactions only: for each week since the124first sale, the valuation is recomputed with the sales known at that date.125126## 6. Market capitalisation127128Category market capitalisation is estimated as Σ (graded population × RIV) **only** for assets with129a published population report; the estimate carries a coverage-based confidence label and is never130shown as a precise figure.131132## 7. Benchmarks and correlations133134Benchmark series (S&P 500, NASDAQ, Bitcoin, CPI, Case-Shiller) come from public datasets (FRED,135stooq). Correlations between subindices are Pearson coefficients of daily log returns over 30/90/365136days; collectibles trade far less frequently than securities, so daily comparisons should be read137with that in mind.138139## 8. Rare Radar and records140141Record sales are the highest verified transactions per category. Radar findings (price142discrepancies, first listings in years, ultra-low populations) are derived mechanically from stored143transactions, listings and population reports with the underlying evidence attached.144145## 9. Freshness and limits146147Connectors run on schedules tuned to each source; every statistic shows its last update. Coverage148is uneven by design: categories with few public transactions show fewer valuations and no index149rather than a fabricated one.150