1# FinData OS — Data dictionary and conventions23## Bars (`market_bars.*`) — Arrow schema v1 (`GET /api/v1/catalog/schema/bars`)4| Column | Type | Meaning |5|---|---|---|6| instrument_id | string | `firstrate:<asset_class>:<SYMBOL>` (vendor-keyed). Delisted: `…:<SYMBOL>-DELISTED` |7| symbol_as_reported | string | vendor file ticker (uppercase) |8| venue | string? | unknown from vendor (US consolidated); NULL |9| asset_class | string | stock, etf, futures, index, fx, crypto |10| interval | string | 1m, 5m, 30m, 1h, 1d |11| **timestamp_utc** | timestamp[us, UTC] | **bar OPEN** instant. The 09:30 bar covers 09:30:00–09:30:59 ET |12| session_date | date | trading session date in the exchange timezone (America/New_York; crypto: UTC date) |13| open/high/low/close | float64 | prices in `currency` |14| volume | float64 | shares; integral for raw data (fractional only in vendor-adjusted series) |15| currency | string | USD |16| trade_count | int32? | number of 1-minute bars aggregated (derived datasets only) |17| vwap | float64? | volume-weighted close of the aggregated bars (derived datasets only; NULL when volume = 0) |18| adjustment_type | string | raw (stored) · split · split_div (query-time) · continuous (futures) |19| source_id | string | firstrate |20| ingested_at | timestamp[us, UTC] | when FinData normalized the row |21| quality_flags | list<string> | see below; never alter prices |22| revision_id | int32 | increments when a session is re-published by the vendor |2324### Sessions, DST, calendars25- Vendor timestamps are US Eastern wall-clock (EST/EDT); conversion to UTC uses `zoneinfo America/New_York`. Non-existent local times (spring-forward hour) are dropped and counted (`rows_dropped_unparseable` in the job result); extended hours never overlap the ambiguous fall-back hour.26- Extended session 04:00–20:00 ET (last 1-minute bar opens 19:59). Regular session 09:30–16:00; the 16:00 bar is the closing print. Expected regular bars per session = minutes(open→close) + 1 (391 on a normal day, 211 on a 13:00 early close), from `exchange_calendars` XNYS.27- **Zero-volume minutes are absent at the source** — FinData never forward-fills or manufactures bars. Coverage = bars present / expected regular bars.28- Overnight futures sessions: daily futures bars follow the contract trading day (≈18:00 ET previous day → 17:00); `session_date` is the vendor's date (dataset planned).29- Crypto: UTC timestamps, 7 days a week (dataset planned).3031### Quality flags (row level) and anomaly codes32`ohlc_inconsistent`, `nonpositive_price`, `negative_volume`, `zero_volume`, `fractional_volume`, `off_interval`, `duplicate`, `outside_session`, `non_trading_day`, `price_jump` (> 25 % between consecutive bars within a session), `gap` (coverage level). Severity: error (ohlc/nonpositive/negative/duplicate/off_interval/non_trading_day) · warning (price_jump, outside_session) · info (zero_volume, fractional_volume, gap).3334### Adjustments35- Stored data is **raw** (vendor `UNADJUSTED`). `adjustment=split`: prices × Π(1/ratio) and volume × Π(ratio) for splits with ex_date after the bar's session. `adjustment=split_div`: additionally × Π(1 − dividend / last close before ex_date) using the closes present in the requested range (if no prior close is in range the dividend factor is skipped and reported in `meta.adjustment_notes`). Splits and cash dividends only (vendor meta files); no mergers/spin-offs.36- Vendor-adjusted history is rewritten after each corporate action; FinData avoids that drift by storing raw bars and applying actions explicitly.3738### Symbols, delistings, survivorship39- Vendor merges ticker changes under the latest symbol (fully since 2022); earlier changes mapped by the vendor. Delisted instruments keep a `-DELISTED` suffix; `instrument_aliases` records the base ticker. The instrument master includes 8 069 delisted tickers (listing of 2026-11-01), so universes can be built without survivorship bias when delisted archives are ingested (planned).40- Futures roll policy is the vendor's (fixed dates ≈ 7–10 days before expiry for ES/NQ); three continuous series (unadjusted, ratio, absolute); absolute-adjusted series may be negative (dataset planned).4142## Fundamentals (`fundamentals_facts`)43| Column | Meaning |44|---|---|45| entity_id | `frd:<TICKER>` (vendor) or `sec:CIK##########` (EDGAR) |46| security_id | `firstrate:stock:<TICKER>` when known |47| metric | vendor field name (rev, net_inc, eps_d, assets_tot, cash_op, shares, roe, …) — 76 fields, units in `unit` |48| value / unit / currency | numeric; USD, USD/share, shares, ratio |49| fiscal_period_start/end, fiscal_period_type (q/a), fiscal_year, fiscal_period | vendor period; start is approximated (vendor gives end only) |50| filing_date | SEC filing date (vendor) |51| published_at | unknown from vendor → NULL |52| **available_at** | PIT policy: `filing_date + 1 day 00:00 America/New_York` (fallback period_end + 91 d when filing_date missing, flagged `pit_fallback`) |53| observed_at | when FinData ingested this value |54| revision_id / restatement_flag / superseded_at | a later ingestion seeing a different value creates revision n+1 with `restatement_flag=true` and supersedes revision n |55| source_document / source_id | `SEC 10-Q via FirstRate (filing_date=…)` / `firstrate_fundamentals` |5657**As-of semantics** (`GET /api/v1/fundamentals/{entity}?as_of=`): visible iff `available_at <= as_of` and (`revision_id = 1` or `observed_at <= as_of`). Revision 1 is the proxy for "as originally filed"; restatements observed after `as_of` are never shown. Limitation: restatements that happened before FinData's first ingestion are not reconstructible from this vendor (SEC EDGAR connector, once configured, provides filing-level history).5859## Provenance recorded per row / file60vendor + product (`sources`), endpoint + sanitized request (`raw_archives.request`), retrieval timestamp (`raw_archives.fetched_at`, `ingested_at`), vendor update date (`raw_archives.vendor_last_update`), archive sha256, Parquet sha256 (`lake_files`), dataset version + snapshot id (`dataset_versions`, `X-Dataset-Version` header), transformation code version (`ingest_runs.code_version` = git short SHA), completeness (`coverage`), license class (`datasets.license_class`).61