# RareIndex methodology

RareIndex turns fragmented public collectibles data into structured market data. This page explains how every number on the platform is produced, what evidence it rests on, and where its limits are. Nothing on RareIndex is invented: every sale, listing, guide price, population count and index value traces back to a source record with its URL, capture time, parser version and connector version.

# 1. Data sources and pipeline

Data enters through connectors (marketplaces, auction houses, price guides, grading companies, catalogs). Each capture is stored immutably as a raw record, then normalised into a canonical shape, matched to a canonical asset (entity resolution), de-duplicated, validated and only then written to the sales, listings and observations tables that feed valuations and indices.

Three kinds of price evidence are kept strictly apart:

Currency: every price is stored in its native currency together with the USD conversion at the historical ECB reference rate of the transaction date. Display currencies are converted at view time; historical sales are never converted at today's rate.

# 2. Entity resolution

Descriptions of the same object differ across sources ("1999 Pokemon Base Charizard 1st PSA 10" vs "Pokémon Base Set Charizard Holo 1st Edition #4/102 PSA GEM MT 10"). Records are resolved in order: deterministic identifiers (catalog ids, style codes, set numbers, references, UPC/ISBN) → a deterministic canonical key built from structured attributes per category → trigram similarity of titles constrained by hard discriminators (card number, set, year) → creation of a new asset. The match method and confidence are stored on every record, and grade/condition slices are kept as variants of the asset (PSA 10, CIB, deadstock size 10 …) so that different grades are never mixed.

# 3. RIV — RareIndex Valuation

RIV is an ensemble estimate of the current market value of a variant, computed from verified transactions in the last 365 days (extended to three years when fewer than five recent sales exist, in which case confidence is capped).

The estimators are: latest sale, medians of the last 5/10/20 sales, a trimmed mean (10 % each tail), an exponentially weighted mean (90-day half-life) and a weighted mean using source trust, identification confidence and recency. RIV is the median of the core estimators. The low/high band comes from the inter-quartile range of the sales used (widened to at least ±5 %).

When a variant has fewer than three sales, RIV may fall back to grade-adjusted comparables (sales of other grades of the same asset, adjusted by empirical grade premiums) and, as a last resort, to guide observations. Fallback estimates are labelled and their confidence is capped at 0.5 (comps) or 0.45 (guide-based). With no evidence at all, RareIndex shows Data unavailable.

# Confidence

Confidence (0–1, shown as High ≥ 0.75, Medium ≥ 0.5, Low > 0.2, Insufficient otherwise) combines sample size (log-scaled, 40 sales ≈ maximum), dispersion of log prices (robust MAD), recency of the latest sale and the trust/identification quality of the sales used. Every RIV is displayed with its sample size, confidence and computation time.

# Outliers

Within a variant, sales whose modified z-score on log price exceeds 3.5 are flagged, not deleted; bundles and multi-quantity lots are excluded from pricing. Every flag is written to the audit log with its reason.

# Grade premiums

For each category, grader and grade, RareIndex measures the median ratio between graded and raw sales of the same assets (at least eight paired assets). PSA 10, BGS 10 and CGC 10 are never assumed equivalent; premiums are only ever derived from paired evidence.

# Buyer premiums and the buyer-pays price

A hammer price and a marketplace price are not the same number: the buyer of an auction lot also pays the house's buyer premium. Every sale therefore carries all_in_usd = price_usd plus the premium when the record is hammer-only (buyer_premium_included = false, or unknown for a listed auction house), computed from the schedule in data/fees/auction-houses.json with marginal tiers, minimums, caps and fixed fees, tier thresholds converted into the schedule currency at the sale-date rate. fee_basis records how the figure was obtained — included, added_published, added_approximate, added_default (house unknown, 22 %), none (no premium: marketplaces, ComicLink, Yahoo! Auctions Japan…) or unknown. RIV, ATH/ATL, grade premiums, index volumes and the auction assessments all use coalesce(all_in_usd, price_usd); the native price and price_usd are never overwritten. When a valuation's inputs include estimated premiums the valuation says so ("n % of inputs include an estimated buyer premium"). Live lots are assessed the same way: current bid (only when at least one bid exists) or low estimate + premium → all-in cost, compared with the RIV of the lot's own variant through the ask gates (bid_vs_riv, estimate_vs_riv, verdict). VAT or sales tax on the premium, import duties and shipping are not modelled and are stated as such.

# 4. Scores

• Liquidity (0–100): sales per month, active listings, number of sources, median days between sales and the spread between asks and RIV.
• Rarity (0–100): graded population, production quantity when documented, and observed sale/listing frequency. Rarity is null when no supply signal exists — it is never guessed.
• Momentum (−100…100) over 7d/30d/90d/1y: price change against the RIV history blended with transaction-volume acceleration.
• Value opportunity: (ask − RIV) / RIV of an active ask against the RIV of its own variant (negative = below the valuation), shown only when the gate below passes. Analytical information, not investment advice.
• Data quality (0–100): attribute completeness, source trust, identification confidence, images and transaction evidence.
• Trending: geometric blend of price, volume and listing momenta.

# Ask vs RIV and the anomaly gate

An asking price is compared with a valuation only when the comparison is legitimate (the live constants are exported from packages/valuation/src/scores.ts and rendered on /methodology):

• same variant — a slab whose grade could not be read is kept in a "grader · grade unknown" variant and is never compared against the raw valuation;
• the RIV rests on transactions (comps-only and guide-only estimates never qualify an ask);
• RIV confidence ≥ 0.5 and ≥ 5 transactions used; listing identification confidence ≥ 0.7;
• plausibility band: an ask below 0.1× or above 10× RIV is flagged riv_anomaly and shown as Data/identity anomaly — a wrong variant, lot, currency or identity until reviewed — never as a deal.

Verdicts: deal ≤ −10 %, fair between −10 % and +10 %, premium ≥ +10 %. The Deal Score (0–100) multiplies discount depth, RIV confidence, sample size, liquidity and match confidence, so a large discount on an illiquid or poorly identified asset cannot score high. Discounts are recomputed from scratch on every valuation pass; a stale discount never survives a changed valuation. Period changes beyond ±500 % are treated as data artefacts and are not published as movers.

# Auction fees and all-in cost

A hammer price is not what a buyer pays. Before any comparison with marketplace prices or with the RIV, RareIndex adds the house's buyer's premium (data/fees/auction-houses.json: marginal tiers, minimum/maximum, fixed fees, per-house confidence published / approximate / none; houses not on file fall back to a labelled default of 22 %). The basis is stored with every figure (sales.fee_basis, auction_lots.fee_basis): premium included, premium added (published / ≈ estimated / default), no buyer premium (fixed-price marketplaces, dealers, houses without a premium) or fees unknown (connector silent and house not on file — the price is kept as recorded and flagged). VAT/sales tax on the premium, import duties and shipping are excluded and stated as such.

Live lots are assessed hourly on the same basis: current bid (or the low estimate, labelled "est.", when no bid has been placed) → USD at the current ECB rate → + premium → compared with the RIV of the lot's variant under the listing gates (transaction-based RIV, ≥ 5 sales, confidence ≥ 0.5, plausibility band, −50 % review threshold). Opening prices without a bid are never called deals.

# 5. Indices

Each subindex (RARE-TCG, RARE-WATCH, …) is a chain-linked index of daily RIV changes across its constituents: assets with a valuation confidence of at least 0.4 and at least three sales in the last twelve months (one variant per asset, the most traded). Each day the index moves by the trimmed, equal-weight average of constituents' log returns; composition can change daily without breaking the level. Base = 1000 on 2024-01-01. An index publishes nothing until it has its minimum number of constituents (10 for subindices, 25 for RARE), and every point stores its constituent count and coverage. Indices that have not met their minimum are listed as in development outside the primary ticker.

RARE, the flagship index, is the transaction-weighted average of subindex daily returns and publishes only once at least 25 constituents exist across the published subindices.

A repeat-sales estimator (pairs of sales of the same variant, monthly periods, least squares) is provided for research views; it is less exposed to composition bias but requires many pairs.

Historical index and asset series are reconstructed from transactions only: for each week since the first sale, the valuation is recomputed with the sales known at that date.

# 6. Market capitalisation

Category market capitalisation is estimated as Σ (graded population × RIV) only for assets with a published population report; the estimate carries a coverage-based confidence label and is never shown as a precise figure.

# 7. Benchmarks and correlations

Benchmark series (S&P 500, NASDAQ, Bitcoin, CPI, Case-Shiller) come from public datasets (FRED, stooq). Correlations between subindices are Pearson coefficients of daily log returns over 30/90/365 days; collectibles trade far less frequently than securities, so daily comparisons should be read with that in mind.

# 8. Rare Radar and records

Record sales are the highest verified transactions per category. Radar findings (price discrepancies, first listings in years, ultra-low populations) are derived mechanically from stored transactions, listings and population reports with the underlying evidence attached.

# 9. Freshness and limits

Connectors run on schedules tuned to each source; every statistic shows its last update. Coverage is uneven by design: categories with few public transactions show fewer valuations and no index rather than a fabricated one.