DATA_DICTIONARY.md
Every constructed variable: name, formula, timing, units, availability. Raw fields first; derived features appended as they are implemented (Stage D+).
Raw fields (per bar, source: mito0o852/OHLCV-1m @ 7763284)
| Field | Type | Units | Timing | Notes |
|---|---|---|---|---|
timestamp |
Datetime(ns, UTC) | — | start of the bar minute | verified UTC; convert to America/New_York for session logic |
open |
f64 | USD | first trade in minute | raw, NOT split-adjusted |
high |
f64 | USD | max trade price in minute | raw |
low |
f64 | f64 | min trade price in minute | raw |
close |
f64 | USD | last trade in minute | raw |
volume |
f64 | shares | sum within minute | integer-valued; venue scope unknown |
ticker |
str | — | contemporaneous symbol | NOT a permanent identifier; reuse possible |
Key property: a missing (ticker, minute) row means no reported trade, not zero return. There are no zero-volume rows in the source.
Derived variables
(to be added with formulas, lag discipline, and config references as Stage D/E code lands;
see config/shock_definitions.yaml for planned shock and horizon definitions)
Clean minute panel (data/processed/minute/year=YYYY/month=MM/data.parquet)
| Field | Type | Definition |
|---|---|---|
| ticker | str | contemporaneous symbol (identity unit = spell, see stage C) |
| timestamp | Datetime(ns, UTC) | bar-start minute, deduplicated key with ticker |
| open/high/low/close | f64 | USD, unadjusted; duplicates merged first/max/min/last |
| volume | f64 | shares, duplicates summed |
| date | Date | ET trading date |
| mod | i32 | minute of day ET (570 = 09:30) |
| session | str | pre / rth / post; rth truncated at true XNYS close (early closes) |
| flag_ohlc | bool | Low>min(O,C) or High<max(O,C) or Low>High — flagged, kept |
| flag_split_window | bool | within ±1 trading day of a high-confidence split event |
Universe table (data/metadata/universe_monthly.parquet)
Membership for month m computed from month m−1 only (no look-ahead): days, med_close, med_bars, dollar_vol (m−1 aggregates); eligible (screens); top500/top1000/top3000 (lagged dollar-volume ranks); flag_noncommon (ticker heuristic).
Split events (outputs/diagnostics/stage_d/split_events.parquet)
ticker, date, overnight_ratio, event ("split:R"/"reverse_split:R"), prev_close, rth_first_open. Windows in split_windows.parquet (ticker, date).