FinData OS — Data dictionary and conventions
Bars (market_bars.*) — Arrow schema v1 (GET /api/v1/catalog/schema/bars)
| Column | Type | Meaning |
|---|---|---|
| instrument_id | string | firstrate:<asset_class>:<SYMBOL> (vendor-keyed). Delisted: …:<SYMBOL>-DELISTED |
| symbol_as_reported | string | vendor file ticker (uppercase) |
| venue | string? | unknown from vendor (US consolidated); NULL |
| asset_class | string | stock, etf, futures, index, fx, crypto |
| interval | string | 1m, 5m, 30m, 1h, 1d |
| timestamp_utc | timestamp[us, UTC] | bar OPEN instant. The 09:30 bar covers 09:30:00–09:30:59 ET |
| session_date | date | trading session date in the exchange timezone (America/New_York; crypto: UTC date) |
| open/high/low/close | float64 | prices in currency |
| volume | float64 | shares; integral for raw data (fractional only in vendor-adjusted series) |
| currency | string | USD |
| trade_count | int32? | number of 1-minute bars aggregated (derived datasets only) |
| vwap | float64? | volume-weighted close of the aggregated bars (derived datasets only; NULL when volume = 0) |
| adjustment_type | string | raw (stored) · split · split_div (query-time) · continuous (futures) |
| source_id | string | firstrate |
| ingested_at | timestamp[us, UTC] | when FinData normalized the row |
| quality_flags | list | see below; never alter prices |
| revision_id | int32 | increments when a session is re-published by the vendor |
Sessions, DST, calendars
- Vendor timestamps are US Eastern wall-clock (EST/EDT); conversion to UTC uses
zoneinfo America/New_York. Non-existent local times (spring-forward hour) are dropped and counted (rows_dropped_unparseablein the job result); extended hours never overlap the ambiguous fall-back hour. - Extended session 04:00–20:00 ET (last 1-minute bar opens 19:59). Regular session 09:30–16:00; the 16:00 bar is the closing print. Expected regular bars per session = minutes(open→close) + 1 (391 on a normal day, 211 on a 13:00 early close), from
exchange_calendarsXNYS. - Zero-volume minutes are absent at the source — FinData never forward-fills or manufactures bars. Coverage = bars present / expected regular bars.
- Overnight futures sessions: daily futures bars follow the contract trading day (≈18:00 ET previous day → 17:00);
session_dateis the vendor's date (dataset planned). - Crypto: UTC timestamps, 7 days a week (dataset planned).
Quality flags (row level) and anomaly codes
ohlc_inconsistent, nonpositive_price, negative_volume, zero_volume, fractional_volume, off_interval, duplicate, outside_session, non_trading_day, price_jump (> 25 % between consecutive bars within a session), gap (coverage level). Severity: error (ohlc/nonpositive/negative/duplicate/off_interval/non_trading_day) · warning (price_jump, outside_session) · info (zero_volume, fractional_volume, gap).
Adjustments
- Stored data is raw (vendor
UNADJUSTED).adjustment=split: prices × Π(1/ratio) and volume × Π(ratio) for splits with ex_date after the bar's session.adjustment=split_div: additionally × Π(1 − dividend / last close before ex_date) using the closes present in the requested range (if no prior close is in range the dividend factor is skipped and reported inmeta.adjustment_notes). Splits and cash dividends only (vendor meta files); no mergers/spin-offs. - Vendor-adjusted history is rewritten after each corporate action; FinData avoids that drift by storing raw bars and applying actions explicitly.
Symbols, delistings, survivorship
- Vendor merges ticker changes under the latest symbol (fully since 2022); earlier changes mapped by the vendor. Delisted instruments keep a
-DELISTEDsuffix;instrument_aliasesrecords the base ticker. The instrument master includes 8 069 delisted tickers (listing of 2026-11-01), so universes can be built without survivorship bias when delisted archives are ingested (planned). - Futures roll policy is the vendor's (fixed dates ≈ 7–10 days before expiry for ES/NQ); three continuous series (unadjusted, ratio, absolute); absolute-adjusted series may be negative (dataset planned).
Fundamentals (fundamentals_facts)
| Column | Meaning |
|---|---|
| entity_id | frd:<TICKER> (vendor) or sec:CIK########## (EDGAR) |
| security_id | firstrate:stock:<TICKER> when known |
| metric | vendor field name (rev, net_inc, eps_d, assets_tot, cash_op, shares, roe, …) — 76 fields, units in unit |
| value / unit / currency | numeric; USD, USD/share, shares, ratio |
| fiscal_period_start/end, fiscal_period_type (q/a), fiscal_year, fiscal_period | vendor period; start is approximated (vendor gives end only) |
| filing_date | SEC filing date (vendor) |
| published_at | unknown from vendor → NULL |
| available_at | PIT policy: filing_date + 1 day 00:00 America/New_York (fallback period_end + 91 d when filing_date missing, flagged pit_fallback) |
| observed_at | when FinData ingested this value |
| revision_id / restatement_flag / superseded_at | a later ingestion seeing a different value creates revision n+1 with restatement_flag=true and supersedes revision n |
| source_document / source_id | SEC 10-Q via FirstRate (filing_date=…) / firstrate_fundamentals |
As-of semantics (GET /api/v1/fundamentals/{entity}?as_of=): visible iff available_at <= as_of and (revision_id = 1 or observed_at <= as_of). Revision 1 is the proxy for "as originally filed"; restatements observed after as_of are never shown. Limitation: restatements that happened before FinData's first ingestion are not reconstructible from this vendor (SEC EDGAR connector, once configured, provides filing-level history).
Provenance recorded per row / file
vendor + product (sources), endpoint + sanitized request (raw_archives.request), retrieval timestamp (raw_archives.fetched_at, ingested_at), vendor update date (raw_archives.vendor_last_update), archive sha256, Parquet sha256 (lake_files), dataset version + snapshot id (dataset_versions, X-Dataset-Version header), transformation code version (ingest_runs.code_version = git short SHA), completeness (coverage), license class (datasets.license_class).