README: final dataset metrics — 26.5B rows, 558,814 files, 350 GB, 66/66 options quarters
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 changed file +6 −6
modified
README.md
+6 −6
@@ -18,18 +18,18 @@ | ||
| 18 | 18 | |
| 19 | 19 | ## Dataset metrics |
| 20 | 20 | |
| 21 | −Live snapshot of the lake while ingestion is still running (2026-08-10): | |
| 21 | +Full ingestion completed 2026-08-10 — **745/745 jobs, zero data loss**: | |
| 22 | 22 | |
| 23 | 23 | | Metric | Value | |
| 24 | 24 | |---|---:| |
| 25 | −| Rows converted to Parquet | **1,785,837,666+** | | |
| 26 | −| Parquet files (one per instrument/segment) | **211,140+** | | |
| 27 | −| Lake size (zstd Parquet) | **34 GB+** (raw archives deleted on the fly) | | |
| 28 | −| Options quarters ingested | **30+ / 66** (2010_q1 → today) | | |
| 25 | +| Rows in the lake | **26,464,775,098** | | |
| 26 | +| Parquet files (one per instrument/segment) | **558,814** | | |
| 27 | +| Lake size (zstd Parquet) | **350 GB** (raw archives deleted on the fly) | | |
| 28 | +| Options quarters | **66 / 66** (2010_q1 → today) | | |
| 29 | 29 | | Options underlyings | **5,800+** US equities & indices | |
| 30 | +| Stocks · ETFs at 1-minute | 7,670 · 5,161 instruments | | |
| 30 | 31 | | Bar timeframes | 1min · 5min · 30min · 1hour · 1day | |
| 31 | 32 | | Asset classes | stocks · ETFs · futures (continuous + contracts) · crypto · indices · FX · options | |
| 32 | −| Download plan | 745 jobs, fully resumable, 0 conversion errors | | |
| 33 | 33 | |
| 34 | 34 | Query the live inventory anytime: [`GET /v1/status`](https://www.hfmarketdata.io/v1/status). |
| 35 | 35 | |
| 36 | 36 | |