SPB Git

Repository Public

BENCHMARKS.md 4.5 KB 60 lines markdown

1# SheetData benchmarks23Measured with `orix-data-bench` (this package). **These are measurements of one run on one machine, not guarantees**; every number moves with hardware, disk cache state, thread count and data shape.45## Setup67| | |8|---|---|9| Date | 2026-11-21 |10| Machine | Mac17,6, Apple M5 Max, 18 cores, 48 GiB RAM, macOS 27.0 (26A5425a) |11| Engine | DuckDB v2.0.1 (vendored universal dylib), 18 threads, `memory_limit` 24 GiB (default = half of RAM), release build |12| Dataset | 10,000,000 synthetic rows × 8 columns: `id BIGINT`, `category VARCHAR` (20 values), `region VARCHAR` (8), `order_date DATE` (2020–2024), `amount DOUBLE`, `quantity BIGINT` (1–50), `customer_id BIGINT` (100,000 values), `status VARCHAR` (5) |13| Files | CSV 559.9 MB (written by the tool, streamed through a 4 MiB buffer), Parquet 108.1 MB (zstd, written by DuckDB from the CSV), plus a 100,000-row `customers` CSV for the join |14| Cache state | the files were written seconds before being read, so they were in the OS page cache: import and scan times are CPU-bound, not disk-bound |1516Command:1718```sh19swift build -c release20.build/release/orix-data-bench --dir /tmp/orix-bench --rows 10000000 > report.json21```2223Queries are run three times; the table gives the median and the best run. One-off steps (generation, import, profile, writes to the grid) are run once.2425## Results2627| Step | Median (s) | Best (s) | Notes |28|---|---:|---:|---|29| Generate CSV (Swift, streamed) | 3.931 | | 10,000,000 rows, 559.9 MB, process RSS 15 MiB |30| Register CSV as a lazy view | 0.230 | | sniffing only, no rows loaded |31| Aggregate straight from the CSV view (`GROUP BY region`) | 0.138 | 0.129 | full parse of the file each time |32| Convert CSV → Parquet (`exportQuery`) | 0.671 | | 108.1 MB |33| **Import CSV into a table** (`importFile`) | **0.300** | | 10,000,000 rows, ≈ 33 M rows/s |34| Row count | 0.0004 | 0.0003 | |35| **Filtered aggregation** (2 `IN` values, date ≥, amount >, `GROUP BY category`) | **0.014** | 0.012 | 20 groups |36| **Pivot** region × category by year, sum + % of grand total (`PivotEngine.compute`) | **0.038** | 0.037 | 169 row items × 6 column items; DuckDB returned 1,014 grouped rows |37| **Join** with `customers` on `customer_id` + `GROUP BY segment, status` (transform pipeline) | **0.088** | 0.065 | 20 groups |38| **Preview window** offset 5,000,000, limit 200 (insertion order) | **0.005** | 0.003 | |39| Preview window sorted by `amount DESC`, offset 1,000, limit 200 | 0.033 | 0.032 | top-N with a deterministic tie-break |40| Preview window with a filter, offset 10,000, limit 200 | 0.007 | 0.006 | |41| Profile of the 8 columns (+ top 3 values each) | 1.127 | | exact quantiles on 10 M rows dominate |42| Pivot computed, rendered and written to a sheet | 0.146 | | 171 × 14 cells, one undo step |43| Materialize a 100,000-row × 8-column extract into a sheet | 0.925 | | 800,008 cells through `CalcEngine.set`, one undo step |44| Materialize the whole table into a sheet | refused | | `exceedsSheetLimits`: 10,000,001 rows × 8 columns do not fit 1,048,576 × 16,384 |45| **Parquet, pushdown filter** (`id BETWEEN 9,000,000 AND 9,010,000`) | **0.004** | 0.004 | row groups skipped from statistics; 10,001 rows matched |46| Parquet, projection + aggregate (`GROUP BY category`, 2 of 8 columns read) | 0.011 | 0.010 | |47| Parquet, the filtered aggregation above | 0.014 | 0.014 | no import needed |4849Peak resident memory of the whole process (`getrusage`, `ru_maxrss`): **2,941 MiB**, including the imported table (in-memory database), the 800,008-cell worksheet and the 1,048,576 rows fetched before the full-table materialization was refused. After the import alone it was 1,628 MiB.5051Consistency check at the end of the run: the table, the Parquet view and the CSV view all hold 10,000,000 rows and the same total amount (compared as integer cents).5253Two earlier runs of the same binary on the same machine gave 0.297 s and 0.364 s for the import, 0.011 s for the filtered aggregation, 0.029–0.031 s for the pivot, 0.028–0.029 s for the join and 0.003–0.004 s for the middle preview window: expect that kind of spread.5455## Reading the numbers5657- The grid never sees the 10 M rows: the UI reads 200-row windows (`preview`), and only bounded results (a pivot, an extract) cross into a worksheet.58- Writing to the grid is the slow door: about 0.86 M cells/s through SheetCore's per-cell `set`, against tens of millions of rows per second inside the engine.59- `swift test` does not run this benchmark. A 1,000,000-row end-to-end test exists behind `ORIX_DATA_LARGE_TESTS=1`.60
Files main
Menu

Theme