SPB Git

Repository Public

performance.md 25.3 KB

# Performance

Every measured number that exists for ORIX Sheet, with the machine, the build configuration, the date and the file it comes from. Nothing here is a guarantee: each table is one or a few runs on one machine.

Related documents: setup.md, testing.md, ../architecture/calculation-graph.md, ../architecture/grid-engine.md, ../architecture/data-engine.md, ../architecture/chart-engine.md, ../compatibility/formats.md.

# 1. How to read this document

  • One machine. All numbers were taken on the build machine: Mac17,6, Apple M5 Max, 18 cores, 48 GB RAM, macOS 27.0 (26A5425a), Swift 6.4.2 (docs/research/technology-evaluation.md, Packages/SheetData/BENCHMARKS.md; confirmed with sysctl and sw_vers on 2026-11-21). No measurement exists on another Mac, on an Intel Mac, or on macOS 14–26.
  • Two origins. Sections 2 to 6 copy numbers recorded in repository files by the engineers. Section 7 contains numbers measured for this document on 2026-11-21 by running benchmark harnesses that already exist in the repository but whose output had not been recorded in any file. They are labelled as such.
  • The machine was shared. Several of the source documents, and the runs of section 7, note that other builds were running at the same time. Expect a spread of tens of percent between runs (Packages/SheetData/BENCHMARKS.md gives 0.297 s to 0.364 s for the same import over three runs).
  • Release builds unless stated. A debug build is 2–4 times slower on I/O (Packages/SheetIO/COMPATIBILITY.md), and orix-bench prints a warning when it is not built in release.
  • No comparison with other products was measured. Nothing in the repository times Excel, Numbers or LibreOffice on the same workloads.

# 2. Calculation engine

Tool: orix-bench (Packages/SheetEngine/Sources/orix-bench/main.swift), run as swift run -c release orix-bench --json docs/benchmarks/engine.json. Run recorded: 2026-11-21T07:08:59Z, macOS 27.0 (26A5425a), 18 cores, 48 GB, 473 functions available (465 registered functions + 8 special forms). The text log contains no debug-build warning, which the tool prints in a debug build.

Model. A sheet of rows rows with 4 numeric input columns and 8 formula columns (12 cells per row), plus a 100-row rate table on a second sheet. The eight formula templates mix arithmetic, IF, ROUND, a per-row SUM, an exact-match VLOOKUP into the rate table, MAX/MIN, IFERROR and ABS/LN.

# Build, dependency graph and recalculation by size

Cells (formulas) Build the workbook Dependency graph Full recalculation Formulas per second Process RSS after recalculation
12 000 (8 000) 0.0097 s 0.0078 s 0.0092 s 867 020 19 MB
120 000 (80 000) 0.0926 s 0.0855 s 0.0954 s 838 965 93 MB
1 200 000 (800 000) 1.0733 s 1.2041 s 1.2120 s 660 051 974 MB
6 000 000 (4 000 000) 5.0490 s 6.3115 s 5.0114 s 798 184 4 501 MB

Source: docs/benchmarks/engine.txt and docs/benchmarks/engine.json (release build, build machine, 2026-11-21).

"Build the workbook" includes parsing every formula text. The RSS column is the resident size of the benchmark process at that point of the run; the sizes run one after the other in the same process, so the larger figures may include memory not yet returned by the smaller models.

# Incremental recalculation

Cells After one input edit Formulas evaluated After editing a whole input column Formulas evaluated
12 000 0.0001 s (75 µs) 8 0.0090 s (1 000 edits) 8 000
120 000 0.0001 s (101 µs) 8 0.1262 s (10 000 edits) 80 000
1 200 000 0.0002 s (152 µs) 8 0.9732 s (100 000 edits) 800 000
6 000 000 0.0001 s (121 µs) 8 4.7177 s (500 000 edits) 4 000 000

Source: docs/benchmarks/engine.txt (rounded seconds) and docs/benchmarks/engine.json (microsecond values: 7.5e-05, 0.000101, 0.000152, 0.000121 s).

The single-edit time does not grow with the size of the workbook: only the eight formulas of the edited row are evaluated. Editing a column that every row depends on is a full recalculation in disguise.

# Other engine workloads

Workload Time Rate
Parse 500 000 formulas (IF(SUM(…)>100,VLOOKUP(…)*1.05,"n/a")) 0.5306 s 942 330 formulas/s
Chain of 1 000 000 dependent formulas (A(n) = A(n−1) + 1), first calculation 0.3920 s 2 551 030 formulas/s
Chain: change the first cell (1 000 000 dependents) 0.3250 s
7 whole-column aggregates over 1 000 000 rows (SUM, AVERAGE, COUNTIF, SUMIFS, MAX−MIN, STDEV.S, MEDIAN on A:A / B:B) 0.5351 s
Format 2 000 000 numbers as #,##0.00 0.2902 s 6 892 954 renders/s
Insert 5 rows + recalculation (1 200 000 cells) 3.0839 s
Undo the insertion + recalculation (1 200 000 cells) 1.9569 s
Sort 100 000 rows × 4 columns + recalculation 2.5159 s

Source: docs/benchmarks/engine.txt and engine.json (release build, build machine, 2026-11-21); workload definitions read in Packages/SheetEngine/Sources/orix-bench/main.swift.

# Lookups

Packages/FunctionsLookup/COMPATIBILITY.md states, for a debug build: 2 000 lifted lookups over 200 000 rows in under 1.5 s (one hash-index build and the probes) and a single whole-column lookup in under 0.4 s. These are upper bounds, not timings; machine and date are not stated in that file.

# 3. Number formatting

Format Renders per second
#,##0.00 about 6 million
yyyy-mm-dd hh:mm:ss 2.7 million
width-limited General 1.5 million

Source: Packages/SheetFormat/COMPATIBILITY.md (micro-benchmark BenchmarkTests in Tests/SheetFormatTests/EdgeTests.swift, "release build, Apple Silicon"; the document gives no date and no machine model). The engine benchmark of section 2 measured 6 892 954 renders/s for #,##0.00 on the build machine.

# 4. File formats

Workbook: one sheet of N rows × 10 columns (6 numeric columns, 1 unique text, 1 repeated text, 1 date, 1 formula with cached value). Measured on an Apple M5 Max (48 GB), macOS 27, Swift 6.4, 2026-11-21, release build. Reproduce with ORIX_BENCH=1 swift test -c release -Xswiftc -enable-testing --filter PerformanceTests in Packages/SheetIO.

1,000,000 cells Write Read File size
.orixsheet 0.58 s — 1.7 M cells/s 0.10 s — 10.4 M cells/s 3.5 MB
.xlsx 1.19 s — 0.84 M cells/s 1.90 s — 0.53 M cells/s 8.5 MB
.csv 0.46 s — 2.2 M cells/s 0.19 s — 5.4 M cells/s 10.2 MB
.jsonl 0.46 s — 2.2 M cells/s 0.18 s — 5.7 M cells/s 10.8 MB
.ods 1.45 s — 0.69 M cells/s 4.58 s — 0.22 M cells/s 8.9 MB
5,000,000 cells Write Read
.orixsheet 3.1 s 0.5 s
.xlsx 6.5 s 10.2 s

Source: Packages/SheetIO/COMPATIBILITY.md ("Measured performance").

Notes from the same file:

  • The native read includes parsing 100,000 (respectively 500,000) formula texts and is checked cell by cell against the source workbook; the XLSX and CSV reads are checked the same way.
  • In a debug build, 240,000 string cells with 120,000 distinct strings: write 0.9 s, read 1.2 s.

# 5. Data engine (DuckDB)

Tool: orix-data-bench (Packages/SheetData), swift build -c release then .build/release/orix-data-bench --dir /tmp/orix-bench --rows 10000000 > report.json.

Date 2026-11-21
Machine Mac17,6, Apple M5 Max, 18 cores, 48 GiB RAM, macOS 27.0 (26A5425a)
Engine DuckDB v2.0.1 (vendored universal dylib), 18 threads, memory_limit 24 GiB (default = half of RAM), release build
Dataset 10,000,000 synthetic rows × 8 columns (id, category with 20 values, region with 8, order_date 2020–2024, amount, quantity 1–50, customer_id with 100,000 values, status with 5)
Files CSV 559.9 MB, Parquet 108.1 MB (zstd), plus a 100,000-row customers CSV for the join
Cache state files written seconds before being read, so in the OS page cache: import and scan times are CPU-bound, not disk-bound

Source: Packages/SheetData/BENCHMARKS.md ("Setup").

Queries are run three times (median and best); one-off steps are run once.

Step Median (s) Best (s) Notes
Generate CSV (Swift, streamed) 3.931 10,000,000 rows, 559.9 MB, process RSS 15 MiB
Register CSV as a lazy view 0.230 sniffing only, no rows loaded
Aggregate straight from the CSV view (GROUP BY region) 0.138 0.129 full parse of the file each time
Convert CSV → Parquet (exportQuery) 0.671 108.1 MB
Import CSV into a table (importFile) 0.300 10,000,000 rows, ≈ 33 M rows/s
Row count 0.0004 0.0003
Filtered aggregation (2 IN values, date ≥, amount >, GROUP BY category) 0.014 0.012 20 groups
Pivot region × category by year, sum + % of grand total (PivotEngine.compute) 0.038 0.037 169 row items × 6 column items; DuckDB returned 1,014 grouped rows
Join with customers on customer_id + GROUP BY segment, status (transform pipeline) 0.088 0.065 20 groups
Preview window offset 5,000,000, limit 200 (insertion order) 0.005 0.003
Preview window sorted by amount DESC, offset 1,000, limit 200 0.033 0.032 top-N with a deterministic tie-break
Preview window with a filter, offset 10,000, limit 200 0.007 0.006
Profile of the 8 columns (+ top 3 values each) 1.127 exact quantiles on 10 M rows dominate
Pivot computed, rendered and written to a sheet 0.146 171 × 14 cells, one undo step
Materialize a 100,000-row × 8-column extract into a sheet 0.925 800,008 cells through CalcEngine.set, one undo step
Materialize the whole table into a sheet refused exceedsSheetLimits: 10,000,001 rows × 8 columns do not fit 1,048,576 × 16,384
Parquet, pushdown filter (id BETWEEN 9,000,000 AND 9,010,000) 0.004 0.004 row groups skipped from statistics; 10,001 rows matched
Parquet, projection + aggregate (GROUP BY category, 2 of 8 columns read) 0.011 0.010
Parquet, the filtered aggregation above 0.014 0.014 no import needed

Source: Packages/SheetData/BENCHMARKS.md ("Results"; release build, build machine, 2026-11-21).

The same file reports the run-to-run spread: two earlier runs of the same binary gave 0.297 s and 0.364 s for the import, 0.011 s for the filtered aggregation, 0.029–0.031 s for the pivot, 0.028–0.029 s for the join and 0.003–0.004 s for the middle preview window. swift test does not run this benchmark; a 1,000,000-row end-to-end test exists behind ORIX_DATA_LARGE_TESTS=1.

data-preview --bench (Packages/FeatureData) measures the paging latency of the dataset grid (200-row pages at random offsets, sequential scroll, sorted, filtered, lazy Parquet view). No output of that harness is recorded in the repository and it was not run for this document.

# 6. Charts and intelligence

# Chart rendering at 100 000 points

Release build, 640 × 400 pt, best of three; resolve + layout + draw + encode.

100 000 points PNG 2x PDF SVG
line (decimated) 40–45 ms 8–9 ms 5–6 ms
line, decimation off 1.4–1.7 s — —
scatter 0.26–0.57 s 0.29–0.85 s 0.10–0.18 s

Source: Packages/SheetCharts/README.md ("Decimation"). The file says "measured on this machine", "two runs; the machine was shared with other builds, hence the ranges"; it gives no date.

Lines longer than 2 000 points are reduced to at most four points per 0.5 pt horizontal bucket (first, lowest, highest, last); scatter series above 20 000 opaque markers skip markers that fall on an occupied 0.5 pt cell. See ../architecture/chart-engine.md.

# On-device assistance

Measure Result
Mean time per formula request (20 fixed requests, greedy decoding) 28.2 s

Source: Packages/SheetIntelligence/EVALUATION.md (generated by RealModelTests.fullEvaluation on 2026-11-21T07:36:55Z, macOS 27.0, model "Apple on-device model (AFM 3 Core Advanced)").

The quality results of that evaluation are in ../architecture/intelligence.md.

# 7. Measured for this document (2026-11-21)

The repository contains benchmark harnesses whose output was not recorded in any file. They were run once or twice for this document, on the build machine (Mac17,6, M5 Max, macOS 27.0), from scratch build directories, while other builds and test runs were using the machine. Treat them as orders of magnitude. Each row gives the exact command.

# Grid repaint

technology-evaluation.md justifies CoreGraphics/CoreText over Metal with "a full repaint of a typical window measures a few milliseconds"; no file recorded the measurement. GridView stores the duration of its last draw(_:) in lastDrawDuration, and the preview harness prints it after rendering its window to a bitmap.

Window Appearance grid draw (last draw)
1280 × 800 pt light 4.12 ms, 3.14 ms, 1.29 ms (three launches)
1920 × 1200 pt dark 1.58 ms, 1.38 ms (two launches)

Source: measured for this document on 2026-11-21 with swift build -c release --product sheet-preview in Packages/SheetUI, then sheet-preview --shot out.png --size 1280x800 and sheet-preview --shot out.png --size 1920x1200 --dark; the value is the grid draw: … ms (2 draws) line printed by Packages/SheetUI/Sources/sheet-preview/main.swift.

What this does and does not show: the demo workbook of the harness is small (a budget table in rows 1–9 and two cells in row 12; about 33 rows × 9 columns of mostly empty cells are visible at 1280 × 800, as seen on the screenshot the harness wrote), and the timed draw is the one made into the offscreen bitmap of the screenshot. It is not a measurement of a window full of text, of scrolling, or of a sheet with many conditional formats.

The other repaint figures recorded in the repository, which the summary statements must agree with:

Case Figure Recorded in
Full repaint of a 1440 × 900 window 2–8 ms RELEASE-REPORT.md and README.md (no per-frame table at that size is kept in docs/; the dense-sheet table at 1280 × 800 in ../architecture/grid-engine.md has medians of 1.1–3.8 ms and, apart from one unexplained 25.76 ms frame, worst frames up to 10.1 ms)
Dense content at 2560 × 1440 median 9–16 ms (9.31 to 16.01 ms scrolling, 8.97 to 15.69 ms static) ../architecture/grid-engine.md, measurement 2
App-hosted large-sheet test worst case under 2 ms (1.8 ms) RELEASE-REPORT.md; printed by testLargeSheetStaysResponsive

testLargeSheetStaysResponsive (App/Tests/AppIntegrationTests.swift) fills 50 000 rows × 4 columns (200 000 cells, 100 000 of them formulas), lets the session recalculate in the background, scrolls to the last row, repaints the grid twenty times and prints large sheet repaint worst case: … ms. Its assertion is deliberately loose (under 50 ms). The test passes (docs/reports/app-tests.json); that file records the count and the failures, not the printed duration.

Reading: a repaint is a few milliseconds for an ordinary window and stays inside a 120 Hz frame there. On a 2560 × 1440 window in which every visible cell is filled, the median exceeds the 8.3 ms of a 120 Hz frame and reaches the 16.7 ms of a 60 Hz frame for the styled case, because every frame repaints all visible cells: there is no dirty-rectangle or scroll-blit optimisation.

# Grid with ten embedded charts

Case Per chart Whole grid redraw
Plain redraw, chart cache hit (500 chart draws) 0.0010 ms 0.62 ms, 0.61 ms
Redraw after a data change, cache miss (200 chart renders) 0.588 ms, 0.594 ms 6.73 ms, 6.79 ms

Source: measured for this document on 2026-11-21 with swift build -c release --product charts-preview in Packages/FeatureCharts, then charts-preview --state perf (two runs; the two values in a cell are the two runs). The harness is measure() in Packages/FeatureCharts/Sources/charts-preview/main.swift: 50 plain redraws, then 20 redraws each preceded by an edit of 12 source cells.

# Conditional-format effects

The grid asks an adapter for the visual effects (fills, data bars, icons) of the visible window. The harness uses a 100 000-row sheet with five rules (colour scale, data bars, icon set, top 10, and a cellIs > 900 rule over six columns) and a 50 × 30 window containing 164 effects.

Case Run 1 Run 2
First call (statistics of 4 rules over 100 000 rows) 33.77 ms 35.13 ms
Scrolling 3 rows per frame, 200 frames median 0.052 ms, p95 0.091 ms, max 0.235 ms median 0.052 ms, p95 0.094 ms, max 0.117 ms
Jumping to a new 50-row window, 50 jumps median 0.216 ms, p95 0.281 ms, max 0.373 ms median 0.225 ms, p95 0.253 ms, max 0.350 ms
Redraw of a window already seen, 200 draws median 0.046 ms, p95 0.048 ms, max 0.220 ms median 0.046 ms, p95 0.047 ms, max 0.224 ms

Source: measured for this document on 2026-11-21 with swift build -c release --product audit-preview in Packages/FeatureAudit, then audit-preview --bench (two runs). The harness is benchmark() in Packages/FeatureAudit/Sources/audit-preview/main.swift.

# Python data exchange

100 000 rows × 6 columns (numbers, text, a date column, blanks) read from the host and written back, in restricted mode, with /opt/homebrew/bin/python3 3.14.4.

Step Time
range.values (read 100 001 × 6 cells into Python lists) 0.440 s
range.to_dataframe() 0.503 s
groupby("category")["revenue"].sum() (pandas, for scale) 0.0060 s
range.write(df) (send the DataFrame back) 0.477 s
Whole run, interpreter start and import pandas included 1.542 s

Source: measured for this document on 2026-11-21 with swift test --filter hundredThousandRowsRoundTrip in Packages/SheetPython (debug build of the Swift host; one run). The line is printed by LargeDataTests.hundredThousandRowsRoundTrip in Tests/SheetPythonTests/DiscoveryAndRecordsTests.swift. The test uses a fake in-memory host, so the time to apply the write to a real workbook is not included.

FeaturePythonTests.largeWriteIsApplied prints the time to apply 50 000 × 4 cells to a real session as one undo step; that figure is not recorded in a file and was not measured here.

# 8. Memory

Situation Figure Source
Engine benchmark process after a full recalculation of 12 000 / 120 000 / 1 200 000 / 6 000 000 cells 19 / 93 / 974 / 4 501 MB resident docs/benchmarks/engine.json (rssMB)
SheetCore store "roughly 200 MB per million cells" Packages/SheetIO/COMPATIBILITY.md
I/O benchmark process after the 1 M-cell native round trip / after the XLSX one (holding the source workbook and the loaded copy) 0.47 GB / 0.79 GB peak Packages/SheetIO/COMPATIBILITY.md
I/O benchmark process at the end of the 5 M-cell run 3.7 GB Packages/SheetIO/COMPATIBILITY.md
XLSX reader one decompressed worksheet part in memory, about 50 MB per million cells, no DOM Packages/SheetIO/COMPATIBILITY.md
Data benchmark, whole process peak (ru_maxrss): imported 10 M-row table in memory + an 800,008-cell worksheet + 1,048,576 fetched rows 2,941 MiB Packages/SheetData/BENCHMARKS.md
Data benchmark, after the import alone 1,628 MiB Packages/SheetData/BENCHMARKS.md
CSV generation, streamed through a 4 MiB buffer 15 MiB process RSS Packages/SheetData/BENCHMARKS.md
DuckDB memory_limit default half of physical RAM (24 GiB on the build machine) Packages/SheetData/BENCHMARKS.md
Calculation thread stack 512 MiB reserved (t.stackSize = 1 << 29); this is address space, not resident memory Packages/SheetCore/Sources/SheetCore/Engine/CalcEngine.swift (runDeep)
Python script memory cap physical footprint of the process tree sampled every 0.2 s, killed above maxMemoryBytes (default 8 GB); a script can overshoot between two samples Packages/SheetPython/SECURITY.md

The engine figures put a worksheet at roughly 0.75–0.8 GB resident per million cells in this model (two thirds of the cells are formulas with their parsed expressions and dependency entries). A sheet of plain values is closer to the 200 MB per million cells quoted by the I/O document. Both are single observations.

No memory measurement of the running application (as opposed to the benchmark processes) is recorded.

# 9. Known slow paths

Each item is backed by a number above or by the code named.

  1. Structural edits rebuild the whole dependency graph and snapshot the workbook. Inserting or deleting rows and columns, and undoing them, go through CalcEngine.structural (Packages/SheetCore/Sources/SheetCore/Engine/Editing.swift): a whole-workbook snapshot for undo, the edit, then rebuildGraph(). Inserting 5 rows in a 1 200 000-cell model took 3.08 s with the recalculation; the undo 1.96 s.
  2. Sorting a large range — 2.52 s for 100 000 rows × 4 columns with the recalculation.
  3. Building the dependency graph costs about as much as a full recalculation (1.20 s against 1.21 s at 800 000 formulas; 6.31 s against 5.01 s at 4 000 000). Opening a large workbook pays both unless cached values are trusted on load.
  4. Formula evaluation is single-threaded. A recalculation runs on one thread with a large stack (CalcEngine.runDeep); there is no parallel evaluation in Packages/SheetCore/Sources. The 18 cores of the build machine are not used by recalculation. The app moves a recalculation off the main thread above a threshold of pending formulas (backgroundCalcThreshold, default 20 000, adjustable from 1 000 to 1 000 000 in Settings — App/Sources/AppSettings.swift).
  5. Per-cell materialization into the grid. Results of the data engine and of Python are written through CalcEngine.set, cell by cell: about 0.86 M cells/s (800,008 cells in 0.925 s), against tens of millions of rows per second inside DuckDB. SheetCore has no bulk write (Packages/SheetData/README.md, "Limitations").
  6. Python exchange is JSON over pipes: about 0.44–0.50 s per direction for 600 000 cells (section 7), plus the interpreter start and import pandas.
  7. XLSX and ODS reading. XLSX reads at 0.53 M cells/s and ODS at 0.22 M cells/s, against 10.4 M cells/s for the native format.
  8. Whole-column references are evaluated over the used range each time — 0.535 s for seven aggregates over 1 000 000 rows. There is no cache shared between cells for lookups: "it cannot be invalidated safely without a recalculation generation in SheetCore" (Packages/FunctionsLookup/COMPATIBILITY.md).
  9. First paint of statistical conditional formats on a long range: about 34 ms for four rules over 100 000 rows (section 7), then well under a millisecond per frame.
  10. Charts without decimation. A 100 000-point line with decimation off takes 1.4–1.7 s per PNG; smoothed and area series are never decimated; a 100 000-point scatter takes 0.26–0.57 s per PNG.
  11. Data profile. Exact quantiles dominate the column profile: 1.127 s for 8 columns over 10 M rows.
  12. On-device model latency: 28.2 s mean per formula request.
  13. Debug builds are 2–4 times slower on I/O; engine numbers from a debug build are not representative.

# 10. What has not been measured

  • Application launch time, time to open a document window, and time to first paint.
  • Scrolling frame rate or frame-time distribution in the real app (only the single-draw durations of section 7 exist).
  • Typing latency in the cell editor and the formula bar.
  • Memory of the running app with a large document, and behaviour under memory pressure.
  • Autosave and version-browsing cost on large documents.
  • Printing and PDF export time.
  • Energy use.
  • Any Intel Mac, any machine with less memory, any macOS version other than 27.0.
  • Paging latency of the dataset grid (data-preview --bench exists, no recorded output).
  • Automation workflows (run time of a workflow, of orix-sheet commands) and template construction time.

# 11. Reproducing

sh
# engine
cd Packages/SheetEngine && swift run -c release orix-bench --json ../../docs/benchmarks/engine.json
swift run -c release orix-bench --quick                      # smaller sizes, a few seconds

# file formats
cd Packages/SheetIO && ORIX_BENCH=1 swift test -c release -Xswiftc -enable-testing --filter PerformanceTests

# data engine (writes about 670 MB of files under --dir)
cd Packages/SheetData && swift build -c release
.build/release/orix-data-bench --dir /tmp/orix-bench --rows 10000000 > report.json

# harnesses of section 7
cd Packages/SheetUI && swift run -c release sheet-preview --shot /tmp/grid.png --size 1280x800
cd Packages/FeatureCharts && swift run -c release charts-preview --state perf
cd Packages/FeatureAudit && swift run -c release audit-preview --bench
cd Packages/FeatureData && swift run -c release data-preview --bench --rows 1000000
cd Packages/SheetPython && swift test --filter hundredThousandRowsRoundTrip

When a number in this document is refreshed, update the date, the machine and the source line of its table, and keep the earlier value in the commit history rather than in the text.

Files main
Menu

Theme