SPB Git

spb/metrika Public

Stata-class statistics, GPU-accelerated by Apple Silicon. Native Swift — no Electron, no Python runtime, no compromises.

Swift 92.4% HTML 3.3% R 3% Shell 1.3%

History of MetrikaKit/Sources/ZQData/DuckDBFastReader.swift · clear filter

  1. perf(data): 880x faster loads via DuckDB C-API bulk extraction
    - root cause: duckdb-swift's element(forColumn:at:) linearly rescans
      every chunk on every element access — O(rows x chunks); a 10M-row CSV
      load took 276 s
    - DuckDBFastReader: pointer-based duckdb C calls via @_silgen_name (the
      C library is statically linked; only pointer-argument functions are
      declared, so no struct-ABI exposure), private in-memory instance,
      contiguous column_data/nullmask_data extraction
    - ZQDataStore.dataFrame(fromQuery:): LIMIT-0 schema probe through the
      supported API, SQL-side casts to DOUBLE/VARCHAR, memcpy per numeric
      column; dead per-element materializer removed
    - measured (M4-class): 10M-row parquet 0.20 s, csv 0.31 s, summarize
      ~0.27 s, robust reg ~0.24 s — CLAUDE.md §8 budgets met; table updated
      with measured numbers
    - 116 tests green (identical numerics through the new path)
    - version 1.0.1
    
    Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
    simon-pierre boucher committed 5 days ago (Aug 5, 2026) · 1 file changed +187