spb/wp46_uqo

1# FinData OS audit for WP46 (Phase 0)23Audited 2026-11-09T17:38:35.835958+00:00 against `https://findata.spboucher.ai/api/v1` by `scripts/audit_findata.py`. Machine-readable evidence: `results/manifests/findata_audit.json`; call log: `results/logs/findata_audit_provenance.jsonl`. The hostname `findata.spboucher.ai` (not `indata.`) is the one configured in the owner's FinData repository and its `docs/CLAUDE-API-USAGE.md`; the API contract is the live OpenAPI document (`/api/v1/openapi.json`) — no endpoint below was guessed.45## 1. Service and contract67- Health status: `degraded`; alerts: ['dead_jobs']; components: `{'postgres': 'ok', 'backup': '…', 'ingest_worker': 'running', 'lake': '…', 'ssd_smart': 'Verified'}`.8- OpenAPI 3.1.0, 41 paths. Used by WP46: `/catalog/datasets`, `/catalog/datasets/{id}`, `/catalog/schema/bars`, `/instruments/{symbol}`, `/market/bars`, `/query` (DuckDB SQL over the lake views), `/exports` (+ `/exports/{id}`, `/download`), `/health`.9- Bars schema: `{"schema_version": 1, "timestamp_convention": "bar_open", "timezone_storage": "UTC", "fields": [{"name": "instrument_id", "type": "string", "nullable": false}, {"name": "symbol_as_reported", "type": "string", "nullable": false}, {"name": "venue", "type": "string", "nullable": true}, {"name": "asset_class", "type": "string", "nullable": false}, {"name": "interval", "type": "string", "nullable": false}, {"name": "timestamp_utc", "type": "timestamp[us, tz=UTC]", "nullable": false}, {"name": "session_date", "type": "date32[day]", "nullable": false}, {"name": "open", "type": "double", "nullable": f`10- Response headers observed: `{'x-content-type-options': 'nosniff', 'x-frame-options': 'DENY', 'x-request-id': 'a269afe07a8a4356'}`. Rate limit (documented): 600 requests/min per key; the client uses a 5 req/s token bucket with Retry-After/backoff.11- Authentication: bearer API key read from the owner's secret file; key scopes used: read:market, query, export. Never logged.1213## 2. Datasets (catalog)1415| dataset | status | rows | first session | last session | version |16|---|---|---:|---|---|---:|17| `corporate_actions.stock` | active | 0 |  |  | 0 |18| `market_bars.stock.5m.raw` | active | 2,062,205,350 | 2000-01-03 | 2026-11-06 | 43 |19| `market_bars.crypto.1d.raw` | active | 181,957 |  |  | 75 |20| `market_bars.crypto.1h.raw` | active | 4,098,204 |  |  | 5 |21| `market_bars.crypto.1m.raw` | active | 177,377,622 |  |  | 83 |22| `market_bars.crypto.30m.raw` | active | 7,970,148 |  |  | 5 |23| `market_bars.crypto.5m.raw` | active | 42,849,352 |  |  | 5 |24| `market_bars.etf.1d.raw` | active | 9,073,895 | 2000-01-03 | 2026-11-06 | 5167 |25| `market_bars.etf.1d.split` | active | 9,051,347 |  |  | 270 |26| `market_bars.etf.1d.split_div` | active | 9,051,347 |  |  | 270 |27| `market_bars.etf.1h.split` | active | 55,703,678 |  |  | 270 |28| `market_bars.etf.1h.split_div` | active | 55,703,678 |  |  | 270 |29| `market_bars.etf.1m.raw` | active | 910,257,088 | 2000-01-03 | 2026-11-06 | 5167 |30| `market_bars.etf.1m.split` | active | 907,197,376 |  |  | 5161 |31| `market_bars.etf.1m.split_div` | active | 907,197,376 |  |  | 5161 |32| `market_bars.etf.30m.split` | active | 91,337,816 |  |  | 270 |33| `market_bars.etf.30m.split_div` | active | 91,337,816 |  |  | 270 |34| `market_bars.etf.5m.split` | active | 330,637,732 |  |  | 270 |35| `market_bars.etf.5m.split_div` | active | 330,637,732 |  |  | 270 |36| `market_bars.futures_contracts.1d.raw` | active | 8,430,644 |  |  | 20967 |37| `market_bars.futures_contracts.1h.raw` | active | 30,077,653 |  |  | 997 |38| `market_bars.futures_contracts.1m.raw` | active | 615,553,277 |  |  | 19699 |39| `market_bars.futures_contracts.30m.raw` | active | 51,126,242 |  |  | 997 |40| `market_bars.futures_contracts.5m.raw` | active | 199,339,369 |  |  | 997 |41| `market_bars.futures.1d.continuous_absolute` | active | 526,324 |  |  | 133 |42| `market_bars.futures.1d.continuous_ratio` | active | 526,324 |  |  | 133 |43| `market_bars.futures.1d.continuous_raw` | active | 526,324 |  |  | 133 |44| `market_bars.futures.1h.continuous_absolute` | active | 8,317,926 |  |  | 8 |45| `market_bars.futures.1h.continuous_ratio` | active | 8,317,926 |  |  | 8 |46| `market_bars.futures.1h.continuous_raw` | active | 8,317,926 |  |  | 8 |47| `market_bars.futures.1m.continuous_absolute` | active | 302,853,229 |  |  | 132 |48| `market_bars.futures.1m.continuous_ratio` | active | 302,853,229 |  |  | 132 |49| `market_bars.futures.1m.continuous_raw` | active | 302,853,229 |  |  | 132 |50| `market_bars.futures.30m.continuous_absolute` | active | 15,617,848 |  |  | 8 |51| `market_bars.futures.30m.continuous_ratio` | active | 15,617,848 |  |  | 8 |52| `market_bars.futures.30m.continuous_raw` | active | 15,617,848 |  |  | 8 |53| `market_bars.futures.5m.continuous_absolute` | active | 78,211,693 |  |  | 8 |54| `market_bars.futures.5m.continuous_ratio` | active | 78,211,693 |  |  | 8 |55| `market_bars.futures.5m.continuous_raw` | active | 78,211,693 |  |  | 8 |56| `market_bars.fx.1d.raw` | planned | 0 |  |  | 0 |57| `market_bars.fx.1h.raw` | active | 7,635,724 |  |  | 5 |58| `market_bars.fx.1m.raw` | active | 407,864,391 |  |  | 85 |59| `market_bars.fx.30m.raw` | active | 15,141,666 |  |  | 5 |60| `market_bars.fx.5m.raw` | active | 87,304,277 |  |  | 5 |61| `market_bars.index.1d.raw` | active | 556,868 |  |  | 124 |62| `market_bars.index.1h.raw` | active | 3,963,294 |  |  | 8 |63| `market_bars.index.1m.raw` | active | 203,832,449 | 2008-01-02 | 2026-11-06 | 128 |64| `market_bars.index.30m.raw` | active | 7,206,595 |  |  | 8 |65| `market_bars.index.5m.raw` | active | 41,215,377 |  |  | 8 |66| `market_bars.stock.1d.raw` | active | 21,088,134 | 2000-01-03 | 2026-11-06 | 7757 |67| `market_bars.stock.1d.split` | active | 38,316,176 |  |  | 827 |68| `market_bars.stock.1d.split_div` | active | 38,103,477 |  |  | 828 |69| `market_bars.stock.1h.split` | active | 286,709,713 |  |  | 840 |70| `market_bars.stock.1h.split_div` | active | 285,019,420 |  |  | 831 |71| `market_bars.stock.1m.raw` | active | 6,702,645,689 | 2000-01-03 | 2026-11-06 | 17382 |72| `market_bars.stock.1m.split` | active | 6,657,098,101 |  |  | 14008 |73| `market_bars.stock.1m.split_div` | active | 6,578,579,087 |  |  | 13811 |74| `market_bars.stock.30m.split` | active | 485,858,160 |  |  | 840 |75| `market_bars.stock.30m.split_div` | active | 482,672,255 |  |  | 831 |76| `market_bars.stock.5m.split` | active | 2,069,201,312 |  |  | 840 |77| `market_bars.stock.5m.split_div` | active | 2,053,780,780 |  |  | 831 |78| `options.eod` | active | 4,232,091,733 |  |  | 361 |7980### Dataset conventions (descriptor text)8182- `market_bars.etf.1m.raw`: {'name': 'US ETFs — 1-minute bars (unadjusted)', 'description': 'Same contract as stocks.', 'timestamp_convention': 'bar_open', 'timezone': 'America/New_York', 'notes': None, 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 5167, 'status': 'active'}83- `market_bars.etf.1d.split_div`: {'name': 'US ETFs — daily bars (split- and dividend-adjusted (vendor adj_splitdiv))', 'description': 'Vendor daily OHLCV bars, split- and dividend-adjusted (vendor adj_splitdiv). Includes delisted tickers (suffix -DELISTED) when their archives are ingested. Vendor-adjusted history is rewritten after corporate actions; refreshed weekly via full_update.', 'timestamp_convention': 'session', 'timezone': 'America/New_York', 'notes': None, 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 270, 'status': 'active'}84- `market_bars.stock.1m.raw`: {'name': 'US stocks — 1-minute bars (unadjusted)', 'description': 'Raw (unadjusted) 1-minute OHLCV bars for US-listed stocks, extended hours 04:00–20:00 ET, zero-volume bars absent at source. Includes delisted tickers (suffix -DELISTED).', 'timestamp_convention': 'bar_open', 'timezone': 'America/New_York', 'notes': 'Canonical dataset; adjustments are applied at query time from corporate_actions.', 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 17382, 'status': 'active'}85- `market_bars.stock.1d.split_div`: {'name': 'US stocks — daily bars (split- and dividend-adjusted (vendor adj_splitdiv))', 'description': 'Vendor daily OHLCV bars, split- and dividend-adjusted (vendor adj_splitdiv). Includes delisted tickers (suffix -DELISTED) when their archives are ingested. Vendor-adjusted history is rewritten after corporate actions; refreshed weekly via full_update.', 'timestamp_convention': 'session', 'timezone': 'America/New_York', 'notes': None, 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 828, 'status': 'active'}86- `market_bars.futures.1m.continuous_ratio`: {'name': 'Futures — 1-minute continuous, ratio back-adjusted', 'description': 'Vendor continuous series per root (continuous, ratio back-adjusted); roll dates fixed by vendor.', 'timestamp_convention': 'bar_open', 'timezone': 'America/New_York', 'notes': 'Daily bars follow the contract trading day; absolute-adjusted series may be negative.', 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 132, 'status': 'active'}87- `market_bars.futures.1d.continuous_ratio`: {'name': 'Futures — daily continuous, ratio back-adjusted', 'description': 'Vendor continuous series per root (continuous, ratio back-adjusted); roll dates fixed by vendor.', 'timestamp_convention': 'session', 'timezone': 'America/New_York', 'notes': 'Daily bars follow the contract trading day; absolute-adjusted series may be negative.', 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 133, 'status': 'active'}88- `market_bars.futures.1d.continuous_absolute`: {'name': 'Futures — daily continuous, absolute back-adjusted', 'description': 'Vendor continuous series per root (continuous, absolute back-adjusted); roll dates fixed by vendor.', 'timestamp_convention': 'session', 'timezone': 'America/New_York', 'notes': 'Daily bars follow the contract trading day; absolute-adjusted series may be negative.', 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 133, 'status': 'active'}89- `market_bars.fx.1m.raw`: {'name': 'FX majors — 1-minute bars', 'description': 'Aggregated from four money-center banks.', 'timestamp_convention': 'bar_open', 'timezone': 'America/New_York', 'notes': 'Two-field datetime in source files.', 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 85, 'status': 'active'}90- `market_bars.crypto.1m.raw`: {'name': 'Crypto — 1-minute bars', 'description': 'UTC timestamps, 7 days a week.', 'timestamp_convention': 'bar_open', 'timezone': 'UTC', 'notes': None, 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 83, 'status': 'active'}91- `market_bars.index.1d.raw`: {'name': 'Indices — daily bars', 'description': 'Vendor daily bars for indices. No volume (stored as 0).', 'timestamp_convention': 'session', 'timezone': 'America/New_York', 'notes': None, 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 124, 'status': 'active'}92- `corporate_actions.stock`: {'name': 'Splits and cash dividends', 'description': 'From vendor meta files; mergers/spin-offs not covered.', 'timestamp_convention': None, 'timezone': None, 'notes': None, 'license_class': 'frd-private', 'license_text': 'FirstRate Data license: private use by the subscriber only. Raw bars/tables may not be redistributed to third parties (no public API, CSV or tables). Derived results and charts may be published with attribution to FirstRate Data.', 'current_version': 0, 'status': 'active'}9394## 3. Candidate universe coverage9596| class | symbol | name | daily sessions | daily first | daily last | bad close | 1-min sessions | 1-min first | 1-min last | 1-min bars |97|---|---|---|---:|---|---|---:|---:|---|---|---:|98| etf | SPY | SPDR S&P 500 | 6748 | 2000-01-03 | 2026-10-30 | 0 | 6753 | 2000-01-03 | 2026-11-06 | 4675764 |99| etf | QQQ | PowerShares Qqq | 6711 | 2000-02-28 | 2026-10-30 | 0 | 6753 | 2000-01-03 | 2026-11-06 | 4293565 |100| etf | IWM | iShares Russell 2000 Index | 6646 | 2000-05-30 | 2026-10-30 | 0 | 6652 | 2000-05-26 | 2026-11-06 | 3475724 |101| etf | DIA | SPDR Dow Jones Industrial Average | 6748 | 2000-01-03 | 2026-10-30 | 0 | 6753 | 2000-01-03 | 2026-11-06 | 3043772 |102| etf | EEM | iShares MSCI Emerging Markets Index | 5926 | 2003-04-14 | 2026-10-30 | 0 | 5931 | 2003-04-14 | 2026-11-06 | 2514085 |103| etf | EFA | iShares MSCI Eafe Index | 6332 | 2001-08-27 | 2026-10-30 | 0 | 6337 | 2001-08-27 | 2026-11-06 | 2377628 |104| etf | XLF | Financial Select Sector SPDR | 6748 | 2000-01-03 | 2026-10-30 | 0 | 6753 | 2000-01-03 | 2026-11-06 | 2748952 |105| etf | XLE | Energy Select Sector SPDR | 6748 | 2000-01-03 | 2026-10-30 | 0 | 6753 | 2000-01-03 | 2026-11-06 | 2665653 |106| etf | TLT | iShares Barclays 20+ Year Treas Bond | 6104 | 2002-07-30 | 2026-10-30 | 0 | 6109 | 2002-07-30 | 2026-11-06 | 2660987 |107| etf | IEF | iShares Lehman 7-10 Year Treasu | 6104 | 2002-07-30 | 2026-10-30 | 0 | 6109 | 2002-07-30 | 2026-11-06 | 1883934 |108| etf | HYG | iShares Iboxx $ High Yield Corporate Bd | 4922 | 2007-04-11 | 2026-10-30 | 0 | 4926 | 2007-04-11 | 2026-11-06 | 1812239 |109| etf | LQD | iShares Iboxx $ Invest Grade Bond | 6104 | 2002-07-30 | 2026-10-30 | 0 | 6109 | 2002-07-30 | 2026-11-06 | 1947361 |110| etf | GLD | SPDR Gold Shares | 5522 | 2004-11-18 | 2026-10-30 | 0 | 5527 | 2004-11-18 | 2026-11-06 | 2878048 |111| etf | SLV | iShares Silver Trust | 5160 | 2006-04-28 | 2026-10-30 | 0 | 5165 | 2006-04-28 | 2026-11-06 | 2641783 |112| etf | USO | United States Oil | 5173 | 2006-04-10 | 2026-10-30 | 0 | 5178 | 2006-04-10 | 2026-11-06 | 2542011 |113| stock | AAPL | Apple | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 4184674 |114| stock | MSFT | Microsoft | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 3568310 |115| stock | AMZN | Amazon.com | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 3435049 |116| stock | NVDA | Nvidia Corporation | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 3609152 |117| stock | JPM | JP Morgan Chase & Co. | 6637 | 2000-06-12 | 2026-10-30 | 0 | 6642 | 2000-06-12 | 2026-11-06 | 2983348 |118| stock | XOM | Exxon Mobil | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 3031298 |119| stock | JNJ | Johnson & Johnson | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 2764519 |120| stock | PG | Procter & Gamble | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 2744358 |121| stock | KO | Coca-Cola Company | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 2839222 |122| stock | WMT | Walmart | 6747 | 2000-01-04 | 2026-10-30 | 0 | 6752 | 2000-01-04 | 2026-11-06 | 2885451 |123| futures | ES | Eversource Energy | 4754 | 2008-01-02 | 2026-11-06 | 0 | 5861 | 2008-01-02 | 2026-11-06 | 6596585 |124| futures | NQ |  | 4754 | 2008-01-02 | 2026-11-06 | 0 | 5861 | 2008-01-02 | 2026-11-06 | 6262856 |125| futures | CL | Colgate-Palmolive | 4749 | 2008-01-02 | 2026-11-06 | 0 | 5863 | 2008-01-02 | 2026-11-06 | 6478865 |126| futures | NG | Novagold Resources | 4709 | 2008-02-25 | 2026-11-06 | 0 | 5818 | 2008-02-24 | 2026-11-06 | 5186395 |127| futures | GC |  | 4732 | 2008-01-28 | 2026-11-06 | 0 | 5840 | 2008-01-27 | 2026-11-06 | 6571988 |128| futures | SI | Shoulder Innovations | 4749 | 2008-01-02 | 2026-11-06 | 0 | 5863 | 2008-01-02 | 2026-11-06 | 5981619 |129| futures | HG | Hamilton Insurance | 4749 | 2008-01-02 | 2026-11-06 | 0 | 5863 | 2008-01-02 | 2026-11-06 | 5795513 |130| futures | ZN |  | 4754 | 2008-01-02 | 2026-11-06 | 0 | 5860 | 2008-01-02 | 2026-11-06 | 5688309 |131| futures | ZC |  | 4724 | 2008-02-19 | 2026-11-06 | 0 | 5684 | 2008-02-19 | 2026-11-06 | 3300107 |132| fx | EURUSD |  |  |  |  |  | 5257 | 2010-01-03 | 2026-11-06 | 6251085 |133| fx | USDJPY |  |  |  |  |  | 5255 | 2010-01-03 | 2026-11-06 | 6244770 |134| fx | GBPUSD |  |  |  |  |  | 5255 | 2010-01-03 | 2026-11-06 | 6243077 |135| fx | AUDUSD |  |  |  |  |  | 5255 | 2010-01-03 | 2026-11-06 | 6239239 |136| fx | USDCAD |  |  |  |  |  | 5254 | 2010-01-03 | 2026-11-06 | 6211934 |137| fx | USDCHF |  |  |  |  |  | 5257 | 2010-01-03 | 2026-11-06 | 6212109 |138| crypto | BTC | Grayscale Bitcoin Mini Trust | 4967 | 2013-04-01 | 2026-11-06 | 0 | 4970 | 2013-04-01 | 2026-11-08 | 6402725 |139| crypto | ETH | Ethan Allen Interiors | 3885 | 2016-03-11 | 2026-11-06 | 0 | 3888 | 2016-03-11 | 2026-11-08 | 5270106 |140141FX has no daily dataset in the lake (`market_bars.fx.1d.raw` is `planned`, 0 rows): daily FX returns are built from the 1-minute bars (last bar of each vendor session). Futures daily rows come from the vendor continuous series.142143## 4. Session structure (2024, one representative symbol per class)144145- **etf / SPY**: {'n_sessions': 252, 'min_bars': 537, 'med_bars': 782, 'max_bars': 948, 'first_bar_utc_min': '2024-01-02T09:00:00+00:00', 'last_bar_utc_max': '2025-01-01T00:59:00+00:00'}; sample days: [{'session_date': '2024-03-08', 'n': 794, 't0': '2024-03-08T09:00:00+00:00', 't1': '2024-03-09T00:59:00+00:00'}, {'session_date': '2024-03-11', 'n': 805, 't0': '2024-03-11T08:00:00+00:00', 't1': '2024-03-11T23:59:00+00:00'}, {'session_date': '2024-07-03', 'n': 575, 't0': '2024-07-03T08:00:00+00:00', 't1': '2024-07-03T20:55:00+00:00'}, {'session_date': '2024-11-29', 'n': 579, 't0': '2024-11-29T09:00:00+00:00', 't1': '2024-11-29T21:59:00+00:00'}]146- **stock / AAPL**: {'n_sessions': 252, 'min_bars': 448, 'med_bars': 743, 'max_bars': 926, 'first_bar_utc_min': '2024-01-02T09:00:00+00:00', 'last_bar_utc_max': '2025-01-01T00:58:00+00:00'}; sample days: [{'session_date': '2024-03-08', 'n': 835, 't0': '2024-03-08T09:00:00+00:00', 't1': '2024-03-09T00:59:00+00:00'}, {'session_date': '2024-03-11', 'n': 821, 't0': '2024-03-11T08:00:00+00:00', 't1': '2024-03-11T23:59:00+00:00'}, {'session_date': '2024-07-03', 'n': 591, 't0': '2024-07-03T08:00:00+00:00', 't1': '2024-07-03T20:59:00+00:00'}, {'session_date': '2024-11-29', 'n': 467, 't0': '2024-11-29T09:00:00+00:00', 't1': '2024-11-29T21:59:00+00:00'}]147- **futures / ES**: {'n_sessions': 313, 'min_bars': 354, 'med_bars': 1379, 'max_bars': 1380, 'first_bar_utc_min': '2024-01-01T23:00:00+00:00', 'last_bar_utc_max': '2024-12-31T21:59:00+00:00'}; sample days: [{'session_date': '2024-03-08', 'n': 1019, 't0': '2024-03-08T05:00:00+00:00', 't1': '2024-03-08T21:59:00+00:00'}, {'session_date': '2024-03-11', 'n': 1374, 't0': '2024-03-11T04:00:00+00:00', 't1': '2024-03-12T03:59:00+00:00'}, {'session_date': '2024-07-03', 'n': 1151, 't0': '2024-07-03T04:00:00+00:00', 't1': '2024-07-04T03:59:00+00:00'}, {'session_date': '2024-07-07', 'n': 360, 't0': '2024-07-07T22:00:00+00:00', 't1': '2024-07-08T03:59:00+00:00'}, {'session_date': '2024-11-29', 'n': 795, 't0': '2024-11-29T05:00:00+00:00', 't1': '2024-11-29T18:14:00+00:00'}]148- **fx / EURUSD**: {'n_sessions': 312, 'min_bars': 393, 'med_bars': 1433, 'max_bars': 1440, 'first_bar_utc_min': '2024-01-02T05:01:00+00:00', 'last_bar_utc_max': '2024-12-31T20:00:00+00:00'}; sample days: [{'session_date': '2024-03-08', 'n': 1020, 't0': '2024-03-08T05:00:00+00:00', 't1': '2024-03-08T21:59:00+00:00'}, {'session_date': '2024-03-11', 'n': 1437, 't0': '2024-03-11T04:00:00+00:00', 't1': '2024-03-12T03:59:00+00:00'}, {'session_date': '2024-07-03', 'n': 1434, 't0': '2024-07-03T04:00:00+00:00', 't1': '2024-07-04T03:59:00+00:00'}, {'session_date': '2024-07-07', 'n': 419, 't0': '2024-07-07T21:00:00+00:00', 't1': '2024-07-08T03:59:00+00:00'}, {'session_date': '2024-11-29', 'n': 1019, 't0': '2024-11-29T05:00:00+00:00', 't1': '2024-11-29T21:58:00+00:00'}]149- **crypto / BTC**: {'n_sessions': 366, 'min_bars': 1010, 'med_bars': 1440, 'max_bars': 1440, 'first_bar_utc_min': '2024-01-01T00:00:00+00:00', 'last_bar_utc_max': '2024-12-31T23:58:00+00:00'}; sample days: [{'session_date': '2024-03-08', 'n': 1440, 't0': '2024-03-08T00:00:00+00:00', 't1': '2024-03-08T23:59:00+00:00'}, {'session_date': '2024-03-11', 'n': 1440, 't0': '2024-03-11T00:00:00+00:00', 't1': '2024-03-11T23:59:00+00:00'}, {'session_date': '2024-07-03', 'n': 1440, 't0': '2024-07-03T00:00:00+00:00', 't1': '2024-07-03T23:59:00+00:00'}, {'session_date': '2024-07-07', 'n': 1440, 't0': '2024-07-07T00:00:00+00:00', 't1': '2024-07-07T23:59:00+00:00'}, {'session_date': '2024-11-29', 'n': 1440, 't0': '2024-11-29T00:00:00+00:00', 't1': '2024-11-29T23:59:00+00:00'}]150- SPY regular hours 09:30–16:00 ET (incl. the 16:00 closing print) 2024: {'n_sessions': 252, 'min_bars': 319, 'med_bars': 391, 'max_bars': 391}151152## 5. Calendar reconciliation153154- SPY daily sessions vs `exchange_calendars` XNYS: {'vendor_sessions': 6748, 'xnys_sessions': 6748, 'missing_in_vendor': [], 'n_missing': 0, 'extra_in_vendor': [], 'n_extra': 0, 'exchange_calendars_version': '4.13.2'}155156## 6. Adjustment conventions (ratios, no price levels disclosed)157158- aapl_split_2020_08_31: [{'session_date': '2020-08-28', 'ratio_raw_over_split': 4.0}, {'session_date': '2020-08-31', 'ratio_raw_over_split': 1.0}]159- spy_exdiv_2024_03_15: [{'session_date': '2024-03-14', 'ratio_split_over_splitdiv': 1.0327383781585309}, {'session_date': '2024-03-15', 'ratio_split_over_splitdiv': 1.0295496480088433}, {'session_date': '2024-03-18', 'ratio_split_over_splitdiv': 1.0295497190449032}]160- es_roll_march_2024: [{'session_date': '2024-03-11', 'ratio_ratioadj_over_raw': 1.1122799141128246, 'diff_absadj_minus_raw': 660.5}, {'session_date': '2024-03-12', 'ratio_ratioadj_over_raw': 1.112279685161041, 'diff_absadj_minus_raw': 660.5}, {'session_date': '2024-03-13', 'ratio_ratioadj_over_raw': 1.112279411764706, 'diff_absadj_minus_raw': 660.5}, {'session_date': '2024-03-14', 'ratio_ratioadj_over_raw': 1.0985684170180146, 'diff_absadj_minus_raw': 596.0}, {'session_date': '2024-03-15', 'ratio_ratioadj_over_raw': 1.0985692923640924, 'diff_absadj_minus_raw': 596.0}, {'session_date': '2024-03-18', 'ratio_ratioadj_over_raw': 1.0985684836281702, 'diff_absadj_minus_raw': 596.0}, {'session_date': '2024-03-19', 'ratio_ratioadj_over_raw': 1.0985701340201268, 'diff_absadj_minus_raw': 596.0}]161- es_latest_ratio_vs_raw_equal: [{'d': '2026-11-06', 'ratio_close_over_raw_close': 1.0}, {'d': '2026-11-05', 'ratio_close_over_raw_close': 1.0}, {'d': '2026-11-04', 'ratio_close_over_raw_close': 1.0}]162163## 7. Quality flags and revisions (daily ETF/stock candidates)164165- flags: [{'flag': 'off_interval', 'n': 91696}]; revision ids: [{'revision_id': 1, 'n': 91696}]166167## 8. Export smoke test168169- {'export_id': '63a3f115-e934-4660-b688-a370e20e77ea', 'dataset_id': 'market_bars.etf.1d.split_div', 'dataset_version': 270, 'row_count': 252, 'bytes': 18126, 'sha256': 'cbb37f1924e4a6bebc80c84e2559ac62f13c591b300c6ae488a0be217158af5e'}170171## 9. Errors during the audit172173- none174175## 10. Facts, consequences and unresolved questions176177- **Timestamps** are bar OPEN in UTC; stocks/ETFs/futures/FX carry an ET `session_date`, crypto a UTC date. Stocks/ETFs: extended hours 04:00–20:00 ET are present in the 1-minute data; regular session 09:30–16:00 ET (391 bars on a full day, 211 on a 13:00 early close). Zero-volume minutes are absent (missingness is a mask, never filled).178- **Futures** continuous series: vendor-fixed roll dates (not documented per contract by the vendor); three variants (raw = unadjusted splice, ratio = ratio back-adjusted, absolute = difference back-adjusted). WP46 uses the **ratio-adjusted** series, whose log returns are free of roll jumps, and the vendor `session_date` (a calendar day in ET, 00:00–23:59 ET), not the CME trading day (18:00–17:00 ET). This is disclosed in the data section; daily futures returns are therefore 'ET-day close-to-close'.179- **FX**: 1-minute mid bars aggregated from four money-center banks (vendor description), ET session_date, 24h except the weekend gap (Friday ~17:00 ET → Sunday 17:00 ET); Sunday partial sessions exist and are dropped by the preprocessing rule (session must have ≥ 60 % of the modal bar count).180- **Crypto**: UTC days, 7 days a week, 1,440 bars per day.181- **Adjustments**: stocks/ETFs daily returns use the vendor split- and dividend-adjusted closes (`split_div`, price-return convention with dividends reinvested at the ex-date = total-return approximation); intraday returns within a session are unaffected by adjustments except on split days (handled by using the `split` 1-minute view for RV).182- **Licence**: FirstRate Data private use; raw bars are never redistributed; aggregate forecasts/losses and derived statistics are publishable with attribution.183- **Survivorship**: the universe is fixed ex ante to instruments alive on 2026-11-06; all candidates have been continuously listed over their sample, so there is no within-sample delisting, but the selection of liquid, surviving instruments is a documented selection bias (results describe liquid benchmarks, not the cross-section).184- **Unresolved**: (i) exact vendor roll rule for continuous futures (vendor does not expose it in the API; the ratio series preserves returns, so forecasts are unaffected, but RV on roll days mixes two contracts' intraday paths); (ii) corporate-actions table is empty in the lake (adjustments come from vendor-adjusted series only); (iii) 1-minute FX/crypto bars are not consolidated trade data — RV from them is a mid-quote/aggregate measure and is documented as such.185