SPB Git
spboucher.ai

spb/metrika

Public

Stata-class statistics, GPU-accelerated by Apple Silicon. Native Swift — no Electron, no Python runtime, no compromises.

macosswiftstatisticsgpu

feat: bootstrap Metrika v0.1 skeleton with working ZQL vertical slice

- MetrikaKit SPM package: ZQParser, ZQPlanner, ZQEngine, ZQData, ZQStats,
  ZQGPU, ZQGraphics, ZQPlugins (Swift 6, strict concurrency)
- ZQL parser: lexer, Pratt expressions, factor variables, prefix commands,
  column-cited errors with Levenshtein verb suggestions
- ZQData: DuckDB-backed load/save (parquet, csv, json, arrow)
- ZQStats: OLS via LAPACK QR, HC0-HC3 and cluster-robust SE, summarize,
  t/F distributions accurate in the far tails
- ZQGPU: Philox4x32-10 reference RNG, counter-addressable bootstrap
- ZQEngine: session actor with use/save/gen/replace/drop/keep/summarize/
  regress/count/list/graph/bootstrap/set seed/xtset/log
- SwiftUI app (xcodegen): console with history, variables sidebar,
  Swift Charts plots; sandboxed + hardened runtime entitlements
- Tests: 44 green (parser golden, R fixtures at 1e-10, Philox KAT,
  end-to-end engine); Tests/Fixtures/generate.R; Tests/Bench harness
- scripts: check_headers.sh + pre-commit hook, make_icns.sh, release.sh

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simon-pierre boucher committed (Aug 5, 2026) Browse files

69 changed files +6,758 −0

69 changed files
Load all 69 files Unified Split
added .gitignore +22 −0
@@ -0,0 +1,22 @@
1 +# macOS
2 +.DS_Store
3 +
4 +# Xcode
5 +build/
6 +DerivedData/
7 +*.xcuserstate
8 +xcuserdata/
9 +*.xcscmblueprint
10 +*.xccheckout
11 +
12 +# SPM
13 +.build/
14 +.swiftpm/
15 +
16 +# Generated artifacts
17 +*.icns
18 +*.dmg
19 +
20 +# Local credentials (never commit)
21 +Metrika-local/
22 +Zyquo-local/
added Assets/AppIcon/metrika-icon.svg +131 −0
@@ -0,0 +1,131 @@
1 +<?xml version="1.0" encoding="UTF-8"?>
2 +<!--
3 + metrika-icon.svg — Metrika
4 + Author: Simon-Pierre Boucher
5 + Contact: contact@spboucher.ai
6 + Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
7 +-->
8 +<svg width="1024" height="1024" viewBox="0 0 1024 1024" xmlns="http://www.w3.org/2000/svg">
9 + <defs>
10 + <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
11 + <stop offset="0%" stop-color="#0B1026"/>
12 + <stop offset="45%" stop-color="#141B3D"/>
13 + <stop offset="100%" stop-color="#1E0B38"/>
14 + </linearGradient>
15 + <radialGradient id="glow" cx="50%" cy="52%" r="55%">
16 + <stop offset="0%" stop-color="#7C5CFF" stop-opacity="0.55"/>
17 + <stop offset="55%" stop-color="#4A3AC8" stop-opacity="0.18"/>
18 + <stop offset="100%" stop-color="#000000" stop-opacity="0"/>
19 + </radialGradient>
20 + <linearGradient id="mstroke" x1="0%" y1="0%" x2="100%" y2="0%">
21 + <stop offset="0%" stop-color="#3EF2D4"/>
22 + <stop offset="50%" stop-color="#7C5CFF"/>
23 + <stop offset="100%" stop-color="#FF4FA3"/>
24 + </linearGradient>
25 + <linearGradient id="mcore" x1="0%" y1="0%" x2="100%" y2="0%">
26 + <stop offset="0%" stop-color="#EAFFFB"/>
27 + <stop offset="50%" stop-color="#E4DBFF"/>
28 + <stop offset="100%" stop-color="#FFE1F0"/>
29 + </linearGradient>
30 + <linearGradient id="area" x1="0%" y1="0%" x2="0%" y2="100%">
31 + <stop offset="0%" stop-color="#7C5CFF" stop-opacity="0.30"/>
32 + <stop offset="100%" stop-color="#7C5CFF" stop-opacity="0"/>
33 + </linearGradient>
34 + <filter id="blurBig" x="-40%" y="-40%" width="180%" height="180%">
35 + <feGaussianBlur stdDeviation="26"/>
36 + </filter>
37 + <filter id="blurSoft" x="-40%" y="-40%" width="180%" height="180%">
38 + <feGaussianBlur stdDeviation="7"/>
39 + </filter>
40 + <linearGradient id="sheen" x1="0%" y1="0%" x2="0%" y2="100%">
41 + <stop offset="0%" stop-color="#FFFFFF" stop-opacity="0.16"/>
42 + <stop offset="35%" stop-color="#FFFFFF" stop-opacity="0.04"/>
43 + <stop offset="60%" stop-color="#FFFFFF" stop-opacity="0"/>
44 + </linearGradient>
45 + <clipPath id="squircle">
46 + <rect x="64" y="64" width="896" height="896" rx="200"/>
47 + </clipPath>
48 + </defs>
49 +
50 + <rect x="64" y="64" width="896" height="896" rx="200" fill="url(#bg)"/>
51 +
52 + <g clip-path="url(#squircle)">
53 + <!-- Plot grid -->
54 + <g stroke="#8FA3FF" stroke-opacity="0.07" stroke-width="2">
55 + <line x1="64" y1="288" x2="960" y2="288"/>
56 + <line x1="64" y1="464" x2="960" y2="464"/>
57 + <line x1="64" y1="640" x2="960" y2="640"/>
58 + <line x1="64" y1="816" x2="960" y2="816"/>
59 + <line x1="256" y1="64" x2="256" y2="960"/>
60 + <line x1="432" y1="64" x2="432" y2="960"/>
61 + <line x1="608" y1="64" x2="608" y2="960"/>
62 + <line x1="784" y1="64" x2="784" y2="960"/>
63 + </g>
64 +
65 + <rect x="64" y="64" width="896" height="896" fill="url(#glow)"/>
66 +
67 + <!-- Area fill under the M trace (chart vibe) -->
68 + <path d="M 220 800 L 220 300 L 420 640 L 512 380 L 604 640 L 804 300 L 804 800 Z"
69 + fill="url(#area)"/>
70 +
71 + <!-- Scatter cloud around the trace -->
72 + <g fill="#9DB4FF" fill-opacity="0.55">
73 + <circle cx="286" cy="418" r="9"/>
74 + <circle cx="330" cy="512" r="7"/>
75 + <circle cx="378" cy="566" r="10"/>
76 + <circle cx="452" cy="536" r="8"/>
77 + <circle cx="486" cy="446" r="7"/>
78 + <circle cx="542" cy="452" r="9"/>
79 + <circle cx="580" cy="560" r="8"/>
80 + <circle cx="646" cy="576" r="10"/>
81 + <circle cx="700" cy="470" r="7"/>
82 + <circle cx="748" cy="392" r="9"/>
83 + <circle cx="264" cy="640" r="7"/>
84 + <circle cx="760" cy="640" r="8"/>
85 + </g>
86 + <g fill="#3EF2D4" fill-opacity="0.85">
87 + <circle cx="252" cy="356" r="6"/>
88 + <circle cx="470" cy="410" r="6"/>
89 + <circle cx="676" cy="520" r="6"/>
90 + </g>
91 + <g fill="#FF4FA3" fill-opacity="0.8">
92 + <circle cx="356" cy="600" r="6"/>
93 + <circle cx="556" cy="500" r="7"/>
94 + <circle cx="772" cy="340" r="6"/>
95 + </g>
96 +
97 + <!-- The M : time-series trace -->
98 + <path d="M 220 800 L 220 300 L 420 640 L 512 380 L 604 640 L 804 300 L 804 800"
99 + fill="none" stroke="url(#mstroke)" stroke-width="96"
100 + stroke-linecap="round" stroke-linejoin="round"
101 + filter="url(#blurBig)" opacity="0.75"/>
102 + <path d="M 220 800 L 220 300 L 420 640 L 512 380 L 604 640 L 804 300 L 804 800"
103 + fill="none" stroke="url(#mstroke)" stroke-width="74"
104 + stroke-linecap="round" stroke-linejoin="round"/>
105 + <path d="M 220 800 L 220 300 L 420 640 L 512 380 L 604 640 L 804 300 L 804 800"
106 + fill="none" stroke="url(#mcore)" stroke-width="20"
107 + stroke-linecap="round" stroke-linejoin="round"
108 + filter="url(#blurSoft)" opacity="0.9"/>
109 +
110 + <!-- Data-point nodes at the vertices of the trace -->
111 + <g fill="#FFFFFF">
112 + <circle cx="220" cy="300" r="16"/>
113 + <circle cx="420" cy="640" r="16"/>
114 + <circle cx="512" cy="380" r="16"/>
115 + <circle cx="604" cy="640" r="16"/>
116 + <circle cx="804" cy="300" r="16"/>
117 + </g>
118 + <g fill="none" stroke="#FFFFFF" stroke-opacity="0.35" stroke-width="6">
119 + <circle cx="220" cy="300" r="30"/>
120 + <circle cx="420" cy="640" r="30"/>
121 + <circle cx="512" cy="380" r="30"/>
122 + <circle cx="604" cy="640" r="30"/>
123 + <circle cx="804" cy="300" r="30"/>
124 + </g>
125 +
126 + <rect x="64" y="64" width="896" height="520" fill="url(#sheen)"/>
127 + </g>
128 +
129 + <rect x="66" y="66" width="892" height="892" rx="198" fill="none"
130 + stroke="#FFFFFF" stroke-opacity="0.10" stroke-width="3"/>
131 +</svg>
added CLAUDE.md +374 −0
@@ -0,0 +1,374 @@
1 +# CLAUDE.md — Metrika
2 +
3 +> **Metrika** — A next-generation statistical & data science application for macOS.
4 +> Stata-class command syntax, GPU-accelerated by Apple Silicon (MLX/Metal), powered by DuckDB.
5 +> Native Swift + SwiftUI. No Electron. No Python runtime dependency. No compromises.
6 +
7 +---
8 +
9 +## 0. Non-Negotiable Project Rules
10 +
11 +1. **Every source file MUST begin with this header** (adapt comment style to the language):
12 +
13 +```swift
14 +//
15 +// <FileName>.swift
16 +// Metrika
17 +//
18 +// Author: Simon-Pierre Boucher
19 +// Contact: contact@spboucher.ai
20 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
21 +//
22 +```
23 +
24 +For Metal shaders (`.metal`), shell scripts, and Python tooling scripts, use the equivalent comment syntax (`//` or `#`). **No file is committed without this header.** A pre-commit hook (see §12) enforces it.
25 +
26 +2. Language: **Swift 6**, strict concurrency enabled (`-strict-concurrency=complete`).
27 +3. UI: **SwiftUI first**, AppKit bridges (`NSViewRepresentable`) only where SwiftUI is insufficient (text editor, data grid virtualization).
28 +4. Minimum target: **macOS 14 (Sonoma)**, Apple Silicon only (`arm64`). No Intel builds.
29 +5. All numerical results must match reference implementations (R / Stata) to at least 1e-10 relative tolerance for CPU paths; GPU paths documented tolerance 1e-6 (float32) or 1e-10 (float64 where MLX supports it).
30 +6. English for all code, comments, commit messages, and docs.
31 +
32 +---
33 +
34 +## 1. Product Vision
35 +
36 +Metrika is a complete data science environment:
37 +
38 +- **Command-driven**, like Stata: `reg y x1 x2, robust cluster(id)` — one line, publication-ready output.
39 +- **Scriptable**: `.zyq` do-files with full reproducibility (logs, seeds, versioned datasets).
40 +- **GPU-native**: massively parallel workloads (bootstrap, Monte Carlo, MCMC, ML training, permutation tests) run on Apple GPU via MLX with unified memory.
41 +- **Extensible**: user-defined commands ("home-made" commands) via a plugin protocol — drop a Swift package or a `.zyq` ado-style script into `~/Library/Application Support/Metrika/Commands/`.
42 +- **Big-data capable**: DuckDB engine → out-of-core, columnar, multi-threaded. 100M+ rows on a laptop.
43 +
44 +### Feature pillars (v1 scope)
45 +
46 +| Pillar | Contents |
47 +|---|---|
48 +| Data management | import/export CSV, Parquet, Arrow, Excel, Stata `.dta`, JSON; merge, append, reshape, collapse, egen-style generators |
49 +| Estimation | summarize, tabulate, correlate, `regress` (OLS), `logit`, `probit`, `poisson`, `ivregress` (2SLS), `xtreg` (FE/RE), `areg`, quantile regression |
50 +| Inference | robust (HC0–HC3), cluster-robust SE, **GPU bootstrap** (pairs, wild, cluster), permutation tests, jackknife |
51 +| Simulation | Monte Carlo engine, random number generation (Philox counter-based, reproducible on GPU) |
52 +| ML | gradient boosting, elastic net, k-means, PCA — MLX-backed |
53 +| Graphics | scatter, line, histogram, kdensity, coefplot, binscatter — Swift Charts + custom Metal renderer for >1M points |
54 +| Reproducibility | do-file editor, logging (`log using`), `set seed` honored across CPU & GPU |
55 +
56 +---
57 +
58 +## 2. App Name, Bundle & Identity
59 +
60 +- **App name:** Metrika
61 +- **Bundle ID:** `ai.spboucher.metrika`
62 +- **Version scheme:** SemVer (`MARKETING_VERSION`), build number = CI run number.
63 +- **Icon:** `Assets/AppIcon/metrika-icon.svg` is the master (see §11). Rasterize to all required sizes via `scripts/make_icns.sh`.
64 +
65 +---
66 +
67 +## 3. Architecture Overview
68 +
69 +```
70 +┌─────────────────────────────────────────────────────────┐
71 +│ MetrikaApp (SwiftUI) │
72 +│ CommandConsole · DoFileEditor · DataBrowser · Plots │
73 +└──────────────────────────┬──────────────────────────────┘
74 + │
75 +┌──────────────────────────▼──────────────────────────────┐
76 +│ MetrikaKit (SPM package) │
77 +│ │
78 +│ ZQParser command grammar → AST │
79 +│ ZQPlanner AST → ExecutionPlan (CPU/GPU/hybrid) │
80 +│ ZQEngine orchestration, sessions, logging │
81 +│ ZQData DataFrame façade over DuckDB/Arrow │
82 +│ ZQStats CPU estimators (Accelerate/LAPACK) │
83 +│ ZQGPU MLX batched estimators, RNG, kernels │
84 +│ ZQGraphics plot spec → Swift Charts / Metal │
85 +│ ZQPlugins user command discovery & sandboxing │
86 +└─────────────────────────────────────────────────────────┘
87 +```
88 +
89 +### Module rules
90 +- `MetrikaKit` is a **Swift Package** with zero UI dependencies → fully testable via `swift test`.
91 +- The app target only imports `MetrikaKit` and renders.
92 +- `ZQGPU` is the **only** module allowed to import MLX/Metal. `ZQStats` is the only one importing Accelerate. This keeps backends swappable.
93 +
94 +### Dependencies (SPM)
95 +```
96 +duckdb/duckdb-swift — data engine
97 +ml-explore/mlx-swift — GPU tensor compute
98 +apple/swift-collections — deques, ordered dicts
99 +pointfreeco/swift-parsing — command grammar
100 +apple/swift-argument-parser — metrika-cli companion tool
101 +```
102 +
103 +---
104 +
105 +## 4. The Command Language (ZQL)
106 +
107 +Stata-compatible mental model, cleaner grammar:
108 +
109 +```
110 +command [varlist] [if expr] [in range] [weight] [, options]
111 +```
112 +
113 +Examples that MUST work in v1:
114 +
115 +```
116 +use sales.parquet
117 +summarize revenue price, detail
118 +gen log_rev = ln(revenue)
119 +reg log_rev price i.region, robust
120 +bootstrap, reps(100000) seed(42): reg log_rev price
121 +xtset firm_id year
122 +xtreg log_rev price, fe cluster(firm_id)
123 +graph scatter log_rev price, by(region)
124 +```
125 +
126 +### Parser (`ZQParser`)
127 +- Built with `swift-parsing`; grammar defined in `Sources/ZQParser/Grammar/`.
128 +- Produces a typed AST: `ZQCommand { verb, varlist, condition, range, weight, options }`.
129 +- Factor variables (`i.region`, `c.age#c.age`) expand at plan time, not parse time.
130 +- Error messages must cite column position and suggest fixes (Levenshtein on known verbs).
131 +
132 +### Home-made commands
133 +Two tiers:
134 +1. **Script commands** (`.zyq` files): sequences of ZQL with `args`/`syntax` declarations — the Stata ado-file analog.
135 +2. **Native plugins**: Swift packages conforming to:
136 +
137 +```swift
138 +public protocol ZQCommandPlugin: Sendable {
139 + static var verb: String { get }
140 + static var syntax: ZQSyntaxSpec { get }
141 + func execute(_ ctx: ZQContext) async throws -> ZQResult
142 +}
143 +```
144 +
145 +Discovery path: `~/Library/Application Support/Metrika/Commands/`. Plugins run in-process but receive a **read-only DataFrame view** unless they declare `mutates: true`.
146 +
147 +---
148 +
149 +## 5. Execution Planner — CPU vs GPU Dispatch
150 +
151 +The planner decides the backend **automatically**. The user never chooses.
152 +
153 +```swift
154 +enum Backend { case cpu, gpu, hybrid }
155 +
156 +func plan(_ cmd: ZQCommand, data: ZQDataFrame) -> ExecutionPlan {
157 + // Heuristics, tuned by benchmarks in Tests/Bench:
158 + // 1. Single estimation, n < 5_000_000 → .cpu (LAPACK QR, dqrls-style)
159 + // 2. reps ≥ 500 independent replicates → .gpu (batched solve)
160 + // 3. MCMC / iterative + large n → .hybrid (GPU likelihood, CPU control)
161 +}
162 +```
163 +
164 +### CPU path (`ZQStats`)
165 +- OLS: QR via LAPACK `dgeqrf`/`dormqr` (never form X'X for conditioning reasons).
166 +- Robust/cluster SE: sandwich estimators, computed with BLAS level-3.
167 +- MLE (logit/probit/poisson): Newton–Raphson with analytic Hessians; IRLS fallback.
168 +- All CPU compute in **Float64**.
169 +
170 +### GPU path (`ZQGPU`, MLX)
171 +- **Batched OLS**: resample indices as `(reps, n)` int tensor → gather → batched Cholesky/QR solve on `(reps, k, k)` — thousands of regressions per GPU pass.
172 +- **RNG**: Philox4x32 counter-based generator → identical streams CPU/GPU given same seed; `set seed` maps to Philox key. Reproducibility is a release blocker.
173 +- **Cluster bootstrap** (ragged n per replicate): pad-to-max with weight masks; custom Metal kernel only if profiling shows >2× overhead from padding.
174 +- Precision: MLX float32 by default; accumulate cross-products in float64 emulation (Kahan/two-sum) where it matters; document tolerances per command.
175 +- **Chunk reps** to respect memory: `chunk = min(reps, budget / (n*k*4))`, stream results, report progress to UI via `AsyncStream`.
176 +
177 +### Hybrid
178 +- Example: Bayesian regression — GPU computes log-likelihood over chains in parallel, CPU runs the sampler control loop.
179 +
180 +---
181 +
182 +## 6. Data Layer (`ZQData`)
183 +
184 +- Canonical store: **DuckDB** in-process database; DataFrames are Arrow views over query results.
185 +- Zero-copy handoff Arrow → MLX tensors where dtype allows; otherwise single cast pass.
186 +- `use file.parquet` → registers a DuckDB view, lazy. `gen`/`replace` compile to SQL expressions when possible; fall back to vectorized Swift over Arrow buffers.
187 +- Missing values: Arrow validity bitmaps end-to-end; every estimator does listwise deletion by default with an explicit report line ("(12 observations dropped due to missing values)").
188 +- `.dta` reader/writer implemented natively in `ZQData/Stata/` (spec v117–v121).
189 +
190 +---
191 +
192 +## 7. UI (SwiftUI)
193 +
194 +Windows/panes:
195 +1. **Console** — command input with history (↑/↓), autocomplete on verbs & varnames, ANSI-style rich results (monospaced, aligned tables via `AttributedString`).
196 +2. **Do-file editor** — `NSTextView` bridge, syntax highlighting for ZQL, ⌘R runs selection.
197 +3. **Data browser** — virtualized grid (custom `NSTableView` bridge; SwiftUI `Table` is too slow past ~100k rows), sortable, filter bar compiles to `if` expressions.
198 +4. **Plots** — Swift Charts for standard plots; custom `MTKView` renderer for scatter >1M points (instanced point sprites).
199 +5. **Sidebar** — variables list with types, labels, missing counts.
200 +
201 +State: `@Observable` session model; every command mutation goes through `ZQEngine` so console, do-files, and plugins share one execution path. Undo = dataset snapshots via DuckDB `CHECKPOINT` + copy-on-write.
202 +
203 +---
204 +
205 +## 8. Performance Budgets (release blockers)
206 +
207 +| Operation | Data | Budget (M3 Pro) |
208 +|---|---|---|
209 +| `use` parquet | 10M rows × 20 cols | < 1.5 s |
210 +| `summarize` all vars | 10M × 20 | < 300 ms |
211 +| `reg` 5 covariates | 10M rows | < 900 ms |
212 +| `bootstrap, reps(10000): reg` | 100k rows | < 3 s |
213 +| `bootstrap, reps(100000): reg` | 100k rows | < 25 s |
214 +| Scatter render | 2M points | 60 fps pan/zoom |
215 +
216 +Benchmarks live in `Tests/Bench/` and run in CI on self-hosted Apple Silicon runner; regressions >10% fail the build.
217 +
218 +---
219 +
220 +## 9. Testing
221 +
222 +- `swift test` on MetrikaKit: parser golden tests, estimator numerical tests vs fixtures generated by R (`Tests/Fixtures/generate.R`).
223 +- Property tests: OLS invariance (scaling, permutation), bootstrap CI coverage simulation (n small, known DGP).
224 +- GPU vs CPU cross-checks: same seed ⇒ identical resample indices; estimates within documented tolerance.
225 +- UI: XCUITest smoke — launch, load sample dataset, run `reg`, assert output table.
226 +
227 +---
228 +
229 +## 10. Build, Signing, Notarization & DMG
230 +
231 +> ⚠️ **PLACEHOLDERS** — the `Zyquo-local/` folder containing the real notarization credentials was not available when this file was generated. Fill every `<...>` below from `Zyquo-local/notarization.md` (or equivalent) before the first release build. Do **not** commit real credentials; keep them in the local keychain / CI secrets.
232 +
233 +### Identifiers
234 +```
235 +DEVELOPMENT_TEAM = <TEAM_ID> # from Metrika-local
236 +CODE_SIGN_IDENTITY = "Developer ID Application: Simon-Pierre Boucher (<TEAM_ID>)"
237 +NOTARY_PROFILE = "<KEYCHAIN_PROFILE_NAME>" # created via: xcrun notarytool store-credentials
238 +BUNDLE_ID = ai.spboucher.metrika
239 +```
240 +
241 +### One-time setup
242 +```bash
243 +xcrun notarytool store-credentials "<KEYCHAIN_PROFILE_NAME>" \
244 + --apple-id "<APPLE_ID_EMAIL>" \
245 + --team-id "<TEAM_ID>" \
246 + --password "<APP_SPECIFIC_PASSWORD>"
247 +```
248 +
249 +### Release pipeline — `scripts/release.sh`
250 +```bash
251 +#!/bin/bash
252 +#
253 +# release.sh — Metrika
254 +# Author: Simon-Pierre Boucher
255 +# Contact: contact@spboucher.ai
256 +#
257 +set -euo pipefail
258 +
259 +APP="Metrika"
260 +SCHEME="Metrika"
261 +BUILD_DIR="build"
262 +DMG="${APP}.dmg"
263 +
264 +# 1. Archive
265 +xcodebuild -scheme "$SCHEME" -configuration Release -arch arm64 \
266 + -archivePath "$BUILD_DIR/$APP.xcarchive" archive
267 +
268 +# 2. Export with Developer ID
269 +xcodebuild -exportArchive \
270 + -archivePath "$BUILD_DIR/$APP.xcarchive" \
271 + -exportOptionsPlist scripts/ExportOptions.plist \
272 + -exportPath "$BUILD_DIR/export"
273 +
274 +# 3. Verify hardened runtime + entitlements
275 +codesign -dv --verbose=4 "$BUILD_DIR/export/$APP.app"
276 +codesign --verify --deep --strict "$BUILD_DIR/export/$APP.app"
277 +
278 +# 4. Build DMG (create-dmg, background art in Assets/DMG/)
279 +create-dmg \
280 + --volname "$APP" \
281 + --window-size 540 380 \
282 + --icon-size 128 \
283 + --icon "$APP.app" 130 190 \
284 + --app-drop-link 400 190 \
285 + --background "Assets/DMG/background.png" \
286 + "$DMG" "$BUILD_DIR/export/"
287 +
288 +# 5. Sign the DMG itself
289 +codesign --sign "Developer ID Application: Simon-Pierre Boucher (<TEAM_ID>)" "$DMG"
290 +
291 +# 6. Notarize & wait
292 +xcrun notarytool submit "$DMG" \
293 + --keychain-profile "<KEYCHAIN_PROFILE_NAME>" \
294 + --wait
295 +
296 +# 7. Staple
297 +xcrun stapler staple "$DMG"
298 +xcrun stapler validate "$DMG"
299 +
300 +echo "✅ $DMG notarized and stapled."
301 +```
302 +
303 +### `scripts/ExportOptions.plist`
304 +```xml
305 +<?xml version="1.0" encoding="UTF-8"?>
306 +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
307 + "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
308 +<plist version="1.0"><dict>
309 + <key>method</key><string>developer-id</string>
310 + <key>teamID</key><string><TEAM_ID></string>
311 + <key>signingStyle</key><string>automatic</string>
312 + <key>destination</key><string>export</string>
313 +</dict></plist>
314 +```
315 +
316 +### Entitlements (`Metrika.entitlements`)
317 +- Hardened Runtime: ON.
318 +- App Sandbox: ON, with `com.apple.security.files.user-selected.read-write` (open/save datasets) and bookmark entitlements for recent files.
319 +- No JIT, no unsigned executable memory (MLX does not require them).
320 +- If plugin loading of external dylibs is added later, revisit `com.apple.security.cs.disable-library-validation` — avoid if possible; prefer SPM-compiled-in plugins.
321 +
322 +---
323 +
324 +## 11. App Icon
325 +
326 +- Master: `Assets/AppIcon/metrika-icon.svg` (1024×1024 design).
327 +- Concept: a glowing **M-shaped time-series line** — a zigzag chart trace cutting through a scatter of data points on a deep space-gradient squircle — econometrics meets GPU power.
328 +- Generate `.icns`: `scripts/make_icns.sh` rasterizes the SVG (via `rsvg-convert` or `qlmanage`) to 16→1024 px and runs `iconutil -c icns`.
329 +- macOS Sonoma+ icon grid: content within the 824×824 safe zone of the 1024 canvas; system applies the squircle mask — but we bake our own rounded-rect background per Big Sur+ convention.
330 +
331 +---
332 +
333 +## 12. Repo Conventions
334 +
335 +```
336 +Metrika/
337 +├── CLAUDE.md
338 +├── Metrika.xcodeproj
339 +├── Metrika/ # app target (UI)
340 +├── MetrikaKit/ # SPM package (all logic)
341 +│ └── Sources/{ZQParser,ZQPlanner,ZQEngine,ZQData,ZQStats,ZQGPU,ZQGraphics,ZQPlugins}
342 +├── Assets/{AppIcon,DMG}
343 +├── scripts/{release.sh,make_icns.sh,check_headers.sh}
344 +└── Tests/{Unit,Bench,Fixtures}
345 +```
346 +
347 +- **Pre-commit hook** runs `scripts/check_headers.sh`: rejects any staged source file (`.swift`, `.metal`, `.sh`, `.py`, `.zyq`) missing the `Author: Simon-Pierre Boucher` / `Contact: contact@spboucher.ai` header.
348 +- Commits: Conventional Commits (`feat(parser): factor variable expansion`).
349 +- Branches: `main` (protected), `dev`, feature branches `feat/*`.
350 +- CI: GitHub Actions on self-hosted arm64 macOS runner — build, test, bench, header check.
351 +
352 +---
353 +
354 +## 13. Roadmap
355 +
356 +**v0.1 (MVP, ~8 weeks of focused work)**
357 +use/save (parquet, csv, dta) · summarize · tabulate · gen/replace · reg (robust, cluster) · logit · basic scatter/histogram · console + data browser · do-file execution.
358 +
359 +**v0.2** — GPU bootstrap + permutation tests · xtreg FE · ivregress · Swift Charts full suite · plugin protocol.
360 +
361 +**v0.3** — MCMC/Bayesian module · gradient boosting & elastic net (MLX) · Metal 2M-point renderer · margins/predict.
362 +
363 +**v1.0** — notarized DMG release, docs site, sample datasets, command reference (`help regress` in-app).
364 +
365 +---
366 +
367 +## 14. What Claude Should Do When Working in This Repo
368 +
369 +1. Read this file first; respect §0 rules absolutely (headers, Swift 6 strict concurrency).
370 +2. Before touching numerics, read the matching fixture in `Tests/Fixtures/` and keep tests green.
371 +3. Any new command: grammar entry → AST case → planner rule → CPU impl → (optional) GPU impl → golden test → in-app help entry. All six or the PR is incomplete.
372 +4. Never introduce a Python or Node runtime dependency into the app bundle.
373 +5. When unsure between CPU and GPU, implement CPU first — correctness before speed — then add the GPU path behind the planner.
374 +6. Update the performance table (§8) whenever a benchmark materially changes.
added Metrika.xcodeproj/project.pbxproj +383 −0
@@ -0,0 +1,383 @@
1 +// !$*UTF8*$!
2 +{
3 + archiveVersion = 1;
4 + classes = {
5 + };
6 + objectVersion = 77;
7 + objects = {
8 +
9 +/* Begin PBXBuildFile section */
10 + 0F7000A49C9879593F09A0D3 /* MetrikaKit in Frameworks */ = {isa = PBXBuildFile; productRef = 498C4115F45BADDF16191A57 /* MetrikaKit */; };
11 + 3084AE2A328ED6746D01F46E /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 0404FDEAF8A57414EFB3C649 /* Assets.xcassets */; };
12 + 7E936A23CCC90823D6B08BCC /* SessionModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0453FB0F256E853D2C2F481C /* SessionModel.swift */; };
13 + 8B0CDB213D64E3EE87593808 /* ConsoleView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 159695CF89D6BB25DF883F07 /* ConsoleView.swift */; };
14 + 9BE65709781719FAE0CA740A /* MetrikaApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9C5393DC050C555E4FB69FC0 /* MetrikaApp.swift */; };
15 + DDCC2DC487AF9FAC7D6B8E3E /* PlotView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 4AD551AA707CD1853C3D6D21 /* PlotView.swift */; };
16 +/* End PBXBuildFile section */
17 +
18 +/* Begin PBXFileReference section */
19 + 0404FDEAF8A57414EFB3C649 /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = "<group>"; };
20 + 0453FB0F256E853D2C2F481C /* SessionModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SessionModel.swift; sourceTree = "<group>"; };
21 + 159695CF89D6BB25DF883F07 /* ConsoleView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ConsoleView.swift; sourceTree = "<group>"; };
22 + 24D84DAB354D702D8575D587 /* MetrikaKit */ = {isa = PBXFileReference; lastKnownFileType = folder; name = MetrikaKit; path = MetrikaKit; sourceTree = SOURCE_ROOT; };
23 + 4AD551AA707CD1853C3D6D21 /* PlotView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PlotView.swift; sourceTree = "<group>"; };
24 + 5A0213CC9FE05A339DEEFA28 /* Metrika.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Metrika.app; sourceTree = BUILT_PRODUCTS_DIR; };
25 + 6E94C5817E96C83B888FB579 /* Metrika.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; path = Metrika.entitlements; sourceTree = "<group>"; };
26 + 9C5393DC050C555E4FB69FC0 /* MetrikaApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MetrikaApp.swift; sourceTree = "<group>"; };
27 +/* End PBXFileReference section */
28 +
29 +/* Begin PBXFrameworksBuildPhase section */
30 + 168283EE9449D9A4324BFE40 /* Frameworks */ = {
31 + isa = PBXFrameworksBuildPhase;
32 + buildActionMask = 2147483647;
33 + files = (
34 + 0F7000A49C9879593F09A0D3 /* MetrikaKit in Frameworks */,
35 + );
36 + runOnlyForDeploymentPostprocessing = 0;
37 + };
38 +/* End PBXFrameworksBuildPhase section */
39 +
40 +/* Begin PBXGroup section */
41 + 133558B8D49331BB5120F8F8 /* Products */ = {
42 + isa = PBXGroup;
43 + children = (
44 + 5A0213CC9FE05A339DEEFA28 /* Metrika.app */,
45 + );
46 + name = Products;
47 + sourceTree = "<group>";
48 + };
49 + 58DB3AB5F820BD1E6C82E1CB /* Sources */ = {
50 + isa = PBXGroup;
51 + children = (
52 + 159695CF89D6BB25DF883F07 /* ConsoleView.swift */,
53 + 9C5393DC050C555E4FB69FC0 /* MetrikaApp.swift */,
54 + 4AD551AA707CD1853C3D6D21 /* PlotView.swift */,
55 + 0453FB0F256E853D2C2F481C /* SessionModel.swift */,
56 + );
57 + path = Sources;
58 + sourceTree = "<group>";
59 + };
60 + 63F77F28B4EF42E0407BEC4F /* Packages */ = {
61 + isa = PBXGroup;
62 + children = (
63 + 24D84DAB354D702D8575D587 /* MetrikaKit */,
64 + );
65 + name = Packages;
66 + sourceTree = "<group>";
67 + };
68 + A905F8EA05CC983967935417 /* Metrika */ = {
69 + isa = PBXGroup;
70 + children = (
71 + 0404FDEAF8A57414EFB3C649 /* Assets.xcassets */,
72 + 6E94C5817E96C83B888FB579 /* Metrika.entitlements */,
73 + 58DB3AB5F820BD1E6C82E1CB /* Sources */,
74 + );
75 + path = Metrika;
76 + sourceTree = "<group>";
77 + };
78 + DC657A037F59093C5529DD71 = {
79 + isa = PBXGroup;
80 + children = (
81 + A905F8EA05CC983967935417 /* Metrika */,
82 + 63F77F28B4EF42E0407BEC4F /* Packages */,
83 + 133558B8D49331BB5120F8F8 /* Products */,
84 + );
85 + sourceTree = "<group>";
86 + };
87 +/* End PBXGroup section */
88 +
89 +/* Begin PBXNativeTarget section */
90 + 5E03B33ABE9DE3162C326F87 /* Metrika */ = {
91 + isa = PBXNativeTarget;
92 + buildConfigurationList = A087F0DEACCF786E9B7CC9A5 /* Build configuration list for PBXNativeTarget "Metrika" */;
93 + buildPhases = (
94 + C8918D5868AE76549F9247E4 /* Sources */,
95 + D09C5D18E3FF44946887A6C9 /* Resources */,
96 + 168283EE9449D9A4324BFE40 /* Frameworks */,
97 + );
98 + buildRules = (
99 + );
100 + dependencies = (
101 + );
102 + name = Metrika;
103 + packageProductDependencies = (
104 + 498C4115F45BADDF16191A57 /* MetrikaKit */,
105 + );
106 + productName = Metrika;
107 + productReference = 5A0213CC9FE05A339DEEFA28 /* Metrika.app */;
108 + productType = "com.apple.product-type.application";
109 + };
110 +/* End PBXNativeTarget section */
111 +
112 +/* Begin PBXProject section */
113 + 8571D0059EB0A97DE47F96F7 /* Project object */ = {
114 + isa = PBXProject;
115 + attributes = {
116 + BuildIndependentTargetsInParallel = YES;
117 + LastUpgradeCheck = 1430;
118 + TargetAttributes = {
119 + 5E03B33ABE9DE3162C326F87 = {
120 + ProvisioningStyle = Automatic;
121 + };
122 + };
123 + };
124 + buildConfigurationList = 02D661894CE6081B5C88E20C /* Build configuration list for PBXProject "Metrika" */;
125 + developmentRegion = en;
126 + hasScannedForEncodings = 0;
127 + knownRegions = (
128 + Base,
129 + en,
130 + );
131 + mainGroup = DC657A037F59093C5529DD71;
132 + minimizedProjectReferenceProxies = 1;
133 + packageReferences = (
134 + D626AF14268CCEC6F4756ABC /* XCLocalSwiftPackageReference "MetrikaKit" */,
135 + );
136 + preferredProjectObjectVersion = 77;
137 + productRefGroup = 133558B8D49331BB5120F8F8 /* Products */;
138 + projectDirPath = "";
139 + projectRoot = "";
140 + targets = (
141 + 5E03B33ABE9DE3162C326F87 /* Metrika */,
142 + );
143 + };
144 +/* End PBXProject section */
145 +
146 +/* Begin PBXResourcesBuildPhase section */
147 + D09C5D18E3FF44946887A6C9 /* Resources */ = {
148 + isa = PBXResourcesBuildPhase;
149 + buildActionMask = 2147483647;
150 + files = (
151 + 3084AE2A328ED6746D01F46E /* Assets.xcassets in Resources */,
152 + );
153 + runOnlyForDeploymentPostprocessing = 0;
154 + };
155 +/* End PBXResourcesBuildPhase section */
156 +
157 +/* Begin PBXSourcesBuildPhase section */
158 + C8918D5868AE76549F9247E4 /* Sources */ = {
159 + isa = PBXSourcesBuildPhase;
160 + buildActionMask = 2147483647;
161 + files = (
162 + 8B0CDB213D64E3EE87593808 /* ConsoleView.swift in Sources */,
163 + 9BE65709781719FAE0CA740A /* MetrikaApp.swift in Sources */,
164 + DDCC2DC487AF9FAC7D6B8E3E /* PlotView.swift in Sources */,
165 + 7E936A23CCC90823D6B08BCC /* SessionModel.swift in Sources */,
166 + );
167 + runOnlyForDeploymentPostprocessing = 0;
168 + };
169 +/* End PBXSourcesBuildPhase section */
170 +
171 +/* Begin XCBuildConfiguration section */
172 + 3CB58B068179C9DB2CD1CAC1 /* Release */ = {
173 + isa = XCBuildConfiguration;
174 + buildSettings = {
175 + ALWAYS_SEARCH_USER_PATHS = NO;
176 + CLANG_ANALYZER_NONNULL = YES;
177 + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
178 + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14";
179 + CLANG_CXX_LIBRARY = "libc++";
180 + CLANG_ENABLE_MODULES = YES;
181 + CLANG_ENABLE_OBJC_ARC = YES;
182 + CLANG_ENABLE_OBJC_WEAK = YES;
183 + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES;
184 + CLANG_WARN_BOOL_CONVERSION = YES;
185 + CLANG_WARN_COMMA = YES;
186 + CLANG_WARN_CONSTANT_CONVERSION = YES;
187 + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES;
188 + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
189 + CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
190 + CLANG_WARN_EMPTY_BODY = YES;
191 + CLANG_WARN_ENUM_CONVERSION = YES;
192 + CLANG_WARN_INFINITE_RECURSION = YES;
193 + CLANG_WARN_INT_CONVERSION = YES;
194 + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES;
195 + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES;
196 + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES;
197 + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
198 + CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES;
199 + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
200 + CLANG_WARN_STRICT_PROTOTYPES = YES;
201 + CLANG_WARN_SUSPICIOUS_MOVE = YES;
202 + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
203 + CLANG_WARN_UNREACHABLE_CODE = YES;
204 + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES;
205 + COPY_PHASE_STRIP = NO;
206 + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
207 + ENABLE_NS_ASSERTIONS = NO;
208 + ENABLE_STRICT_OBJC_MSGSEND = YES;
209 + GCC_C_LANGUAGE_STANDARD = gnu11;
210 + GCC_NO_COMMON_BLOCKS = YES;
211 + GCC_WARN_64_TO_32_BIT_CONVERSION = YES;
212 + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
213 + GCC_WARN_UNDECLARED_SELECTOR = YES;
214 + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
215 + GCC_WARN_UNUSED_FUNCTION = YES;
216 + GCC_WARN_UNUSED_VARIABLE = YES;
217 + MACOSX_DEPLOYMENT_TARGET = 14.0;
218 + MTL_ENABLE_DEBUG_INFO = NO;
219 + MTL_FAST_MATH = YES;
220 + PRODUCT_NAME = "$(TARGET_NAME)";
221 + SDKROOT = macosx;
222 + SWIFT_COMPILATION_MODE = wholemodule;
223 + SWIFT_OPTIMIZATION_LEVEL = "-O";
224 + SWIFT_VERSION = 5.0;
225 + };
226 + name = Release;
227 + };
228 + 4D5E4B6EED8197B03EAD7E51 /* Debug */ = {
229 + isa = XCBuildConfiguration;
230 + buildSettings = {
231 + ARCHS = arm64;
232 + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
233 + CODE_SIGN_ENTITLEMENTS = Metrika/Metrika.entitlements;
234 + CODE_SIGN_STYLE = Automatic;
235 + COMBINE_HIDPI_IMAGES = YES;
236 + CURRENT_PROJECT_VERSION = 1;
237 + ENABLE_APP_SANDBOX = YES;
238 + ENABLE_HARDENED_RUNTIME = YES;
239 + GENERATE_INFOPLIST_FILE = YES;
240 + INFOPLIST_KEY_NSHumanReadableCopyright = "© 2026 Simon-Pierre Boucher. All rights reserved.";
241 + INFOPLIST_KEY_NSPrincipalClass = NSApplication;
242 + LD_RUNPATH_SEARCH_PATHS = (
243 + "$(inherited)",
244 + "@executable_path/../Frameworks",
245 + );
246 + MARKETING_VERSION = 0.1.0;
247 + ONLY_ACTIVE_ARCH = YES;
248 + PRODUCT_BUNDLE_IDENTIFIER = ai.spboucher.metrika;
249 + SDKROOT = macosx;
250 + SWIFT_STRICT_CONCURRENCY = complete;
251 + SWIFT_VERSION = 6.0;
252 + };
253 + name = Debug;
254 + };
255 + C241F8724816C2E6FEFA98F2 /* Release */ = {
256 + isa = XCBuildConfiguration;
257 + buildSettings = {
258 + ARCHS = arm64;
259 + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
260 + CODE_SIGN_ENTITLEMENTS = Metrika/Metrika.entitlements;
261 + CODE_SIGN_STYLE = Automatic;
262 + COMBINE_HIDPI_IMAGES = YES;
263 + CURRENT_PROJECT_VERSION = 1;
264 + ENABLE_APP_SANDBOX = YES;
265 + ENABLE_HARDENED_RUNTIME = YES;
266 + GENERATE_INFOPLIST_FILE = YES;
267 + INFOPLIST_KEY_NSHumanReadableCopyright = "© 2026 Simon-Pierre Boucher. All rights reserved.";
268 + INFOPLIST_KEY_NSPrincipalClass = NSApplication;
269 + LD_RUNPATH_SEARCH_PATHS = (
270 + "$(inherited)",
271 + "@executable_path/../Frameworks",
272 + );
273 + MARKETING_VERSION = 0.1.0;
274 + ONLY_ACTIVE_ARCH = YES;
275 + PRODUCT_BUNDLE_IDENTIFIER = ai.spboucher.metrika;
276 + SDKROOT = macosx;
277 + SWIFT_STRICT_CONCURRENCY = complete;
278 + SWIFT_VERSION = 6.0;
279 + };
280 + name = Release;
281 + };
282 + DC41815671596F5C0AC0A0A6 /* Debug */ = {
283 + isa = XCBuildConfiguration;
284 + buildSettings = {
285 + ALWAYS_SEARCH_USER_PATHS = NO;
286 + CLANG_ANALYZER_NONNULL = YES;
287 + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
288 + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14";
289 + CLANG_CXX_LIBRARY = "libc++";
290 + CLANG_ENABLE_MODULES = YES;
291 + CLANG_ENABLE_OBJC_ARC = YES;
292 + CLANG_ENABLE_OBJC_WEAK = YES;
293 + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES;
294 + CLANG_WARN_BOOL_CONVERSION = YES;
295 + CLANG_WARN_COMMA = YES;
296 + CLANG_WARN_CONSTANT_CONVERSION = YES;
297 + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES;
298 + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
299 + CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
300 + CLANG_WARN_EMPTY_BODY = YES;
301 + CLANG_WARN_ENUM_CONVERSION = YES;
302 + CLANG_WARN_INFINITE_RECURSION = YES;
303 + CLANG_WARN_INT_CONVERSION = YES;
304 + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES;
305 + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES;
306 + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES;
307 + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
308 + CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES;
309 + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
310 + CLANG_WARN_STRICT_PROTOTYPES = YES;
311 + CLANG_WARN_SUSPICIOUS_MOVE = YES;
312 + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
313 + CLANG_WARN_UNREACHABLE_CODE = YES;
314 + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES;
315 + COPY_PHASE_STRIP = NO;
316 + DEBUG_INFORMATION_FORMAT = dwarf;
317 + ENABLE_STRICT_OBJC_MSGSEND = YES;
318 + ENABLE_TESTABILITY = YES;
319 + GCC_C_LANGUAGE_STANDARD = gnu11;
320 + GCC_DYNAMIC_NO_PIC = NO;
321 + GCC_NO_COMMON_BLOCKS = YES;
322 + GCC_OPTIMIZATION_LEVEL = 0;
323 + GCC_PREPROCESSOR_DEFINITIONS = (
324 + "$(inherited)",
325 + "DEBUG=1",
326 + );
327 + GCC_WARN_64_TO_32_BIT_CONVERSION = YES;
328 + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
329 + GCC_WARN_UNDECLARED_SELECTOR = YES;
330 + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
331 + GCC_WARN_UNUSED_FUNCTION = YES;
332 + GCC_WARN_UNUSED_VARIABLE = YES;
333 + MACOSX_DEPLOYMENT_TARGET = 14.0;
334 + MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE;
335 + MTL_FAST_MATH = YES;
336 + ONLY_ACTIVE_ARCH = YES;
337 + PRODUCT_NAME = "$(TARGET_NAME)";
338 + SDKROOT = macosx;
339 + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG;
340 + SWIFT_OPTIMIZATION_LEVEL = "-Onone";
341 + SWIFT_VERSION = 5.0;
342 + };
343 + name = Debug;
344 + };
345 +/* End XCBuildConfiguration section */
346 +
347 +/* Begin XCConfigurationList section */
348 + 02D661894CE6081B5C88E20C /* Build configuration list for PBXProject "Metrika" */ = {
349 + isa = XCConfigurationList;
350 + buildConfigurations = (
351 + DC41815671596F5C0AC0A0A6 /* Debug */,
352 + 3CB58B068179C9DB2CD1CAC1 /* Release */,
353 + );
354 + defaultConfigurationIsVisible = 0;
355 + defaultConfigurationName = Debug;
356 + };
357 + A087F0DEACCF786E9B7CC9A5 /* Build configuration list for PBXNativeTarget "Metrika" */ = {
358 + isa = XCConfigurationList;
359 + buildConfigurations = (
360 + 4D5E4B6EED8197B03EAD7E51 /* Debug */,
361 + C241F8724816C2E6FEFA98F2 /* Release */,
362 + );
363 + defaultConfigurationIsVisible = 0;
364 + defaultConfigurationName = Debug;
365 + };
366 +/* End XCConfigurationList section */
367 +
368 +/* Begin XCLocalSwiftPackageReference section */
369 + D626AF14268CCEC6F4756ABC /* XCLocalSwiftPackageReference "MetrikaKit" */ = {
370 + isa = XCLocalSwiftPackageReference;
371 + relativePath = MetrikaKit;
372 + };
373 +/* End XCLocalSwiftPackageReference section */
374 +
375 +/* Begin XCSwiftPackageProductDependency section */
376 + 498C4115F45BADDF16191A57 /* MetrikaKit */ = {
377 + isa = XCSwiftPackageProductDependency;
378 + productName = MetrikaKit;
379 + };
380 +/* End XCSwiftPackageProductDependency section */
381 + };
382 + rootObject = 8571D0059EB0A97DE47F96F7 /* Project object */;
383 +}
added Metrika.xcodeproj/project.xcworkspace/contents.xcworkspacedata +7 −0
@@ -0,0 +1,7 @@
1 +<?xml version="1.0" encoding="UTF-8"?>
2 +<Workspace
3 + version = "1.0">
4 + <FileRef
5 + location = "self:">
6 + </FileRef>
7 +</Workspace>
added Metrika/Assets.xcassets/AppIcon.appiconset/Contents.json +15 −0
@@ -0,0 +1,15 @@
1 +{
2 + "images": [
3 + { "filename": "icon_16.png", "idiom": "mac", "scale": "1x", "size": "16x16" },
4 + { "filename": "icon_16@2x.png", "idiom": "mac", "scale": "2x", "size": "16x16" },
5 + { "filename": "icon_32.png", "idiom": "mac", "scale": "1x", "size": "32x32" },
6 + { "filename": "icon_32@2x.png", "idiom": "mac", "scale": "2x", "size": "32x32" },
7 + { "filename": "icon_128.png", "idiom": "mac", "scale": "1x", "size": "128x128" },
8 + { "filename": "icon_128@2x.png", "idiom": "mac", "scale": "2x", "size": "128x128" },
9 + { "filename": "icon_256.png", "idiom": "mac", "scale": "1x", "size": "256x256" },
10 + { "filename": "icon_256@2x.png", "idiom": "mac", "scale": "2x", "size": "256x256" },
11 + { "filename": "icon_512.png", "idiom": "mac", "scale": "1x", "size": "512x512" },
12 + { "filename": "icon_512@2x.png", "idiom": "mac", "scale": "2x", "size": "512x512" }
13 + ],
14 + "info": { "author": "xcode", "version": 1 }
15 +}
added Metrika/Assets.xcassets/AppIcon.appiconset/icon_128.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_128@2x.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_16.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_16@2x.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_256.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_256@2x.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_32.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_32@2x.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_512.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/AppIcon.appiconset/icon_512@2x.png +0 −0

Binary file not shown.

added Metrika/Assets.xcassets/Contents.json +1 −0
@@ -0,0 +1 @@
1 +{ "info": { "author": "xcode", "version": 1 } }
added Metrika/Metrika.entitlements +13 −0
@@ -0,0 +1,13 @@
1 +<?xml version="1.0" encoding="UTF-8"?>
2 +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
3 + "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
4 +<plist version="1.0">
5 +<dict>
6 + <key>com.apple.security.app-sandbox</key>
7 + <true/>
8 + <key>com.apple.security.files.user-selected.read-write</key>
9 + <true/>
10 + <key>com.apple.security.files.bookmarks.app-scope</key>
11 + <true/>
12 +</dict>
13 +</plist>
added Metrika/Sources/ConsoleView.swift +120 −0
@@ -0,0 +1,120 @@
1 +//
2 +// ConsoleView.swift
3 +// Metrika
4 +//
5 +// Author: Simon-Pierre Boucher
6 +// Contact: contact@spboucher.ai
7 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
8 +//
9 +
10 +import SwiftUI
11 +
12 +/// Command console: scrollback of executed commands with their rendered
13 +/// output, plus an input line with ↑/↓ history recall (CLAUDE.md §7).
14 +struct ConsoleView: View {
15 + @Environment(SessionModel.self) private var model
16 + @State private var input = ""
17 + @State private var historyCursor: Int?
18 + @FocusState private var inputFocused: Bool
19 +
20 + var body: some View {
21 + VStack(spacing: 0) {
22 + ScrollViewReader { proxy in
23 + ScrollView {
24 + LazyVStack(alignment: .leading, spacing: 10) {
25 + ForEach(model.entries) { entry in
26 + VStack(alignment: .leading, spacing: 2) {
27 + if !entry.command.isEmpty {
28 + Text(". \(entry.command)")
29 + .foregroundStyle(.secondary)
30 + }
31 + if !entry.output.isEmpty {
32 + Text(entry.output)
33 + .foregroundStyle(
34 + entry.isError ? Color.red : Color.primary
35 + )
36 + .textSelection(.enabled)
37 + }
38 + }
39 + .id(entry.id)
40 + }
41 + }
42 + .padding(12)
43 + .frame(maxWidth: .infinity, alignment: .leading)
44 + }
45 + .onChange(of: model.entries) {
46 + if let last = model.entries.last {
47 + proxy.scrollTo(last.id, anchor: .bottom)
48 + }
49 + }
50 + }
51 +
52 + Divider()
53 +
54 + HStack(spacing: 8) {
55 + Text(".")
56 + .foregroundStyle(.secondary)
57 + TextField("Type a command — e.g. use sales.parquet", text: $input)
58 + .textFieldStyle(.plain)
59 + .focused($inputFocused)
60 + .onSubmit(submit)
61 + .onKeyPress(.upArrow) { recallHistory(step: -1) }
62 + .onKeyPress(.downArrow) { recallHistory(step: 1) }
63 + if model.isRunning {
64 + ProgressView()
65 + .controlSize(.small)
66 + }
67 + }
68 + .padding(10)
69 + }
70 + .fontDesign(.monospaced)
71 + .font(.system(size: 12))
72 + .onAppear { inputFocused = true }
73 + }
74 +
75 + private func submit() {
76 + let command = input
77 + input = ""
78 + historyCursor = nil
79 + Task { await model.run(command) }
80 + }
81 +
82 + private func recallHistory(step: Int) -> KeyPress.Result {
83 + let history = model.history
84 + guard !history.isEmpty else { return .ignored }
85 + var cursor = historyCursor ?? history.count
86 + cursor = min(max(cursor + step, 0), history.count)
87 + historyCursor = cursor
88 + input = cursor < history.count ? history[cursor] : ""
89 + return .handled
90 + }
91 +}
92 +
93 +/// Sidebar: variables of the working dataset with type and missing count
94 +/// (CLAUDE.md §7, pane 5).
95 +struct VariablesSidebar: View {
96 + @Environment(SessionModel.self) private var model
97 +
98 + var body: some View {
99 + List {
100 + Section("Variables (\(model.observationCount) obs)") {
101 + ForEach(model.variables) { variable in
102 + HStack {
103 + Text(variable.name)
104 + .fontDesign(.monospaced)
105 + Spacer()
106 + if variable.missingCount > 0 {
107 + Text("\(variable.missingCount) mi")
108 + .foregroundStyle(.orange)
109 + .font(.caption)
110 + }
111 + Text(variable.type)
112 + .foregroundStyle(.secondary)
113 + .font(.caption)
114 + }
115 + }
116 + }
117 + }
118 + .listStyle(.sidebar)
119 + }
120 +}
added Metrika/Sources/MetrikaApp.swift +44 −0
@@ -0,0 +1,44 @@
1 +//
2 +// MetrikaApp.swift
3 +// Metrika
4 +//
5 +// Author: Simon-Pierre Boucher
6 +// Contact: contact@spboucher.ai
7 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
8 +//
9 +
10 +import SwiftUI
11 +
12 +@main
13 +struct MetrikaApp: App {
14 + @State private var model = SessionModel()
15 +
16 + var body: some Scene {
17 + WindowGroup {
18 + ContentView()
19 + .environment(model)
20 + .frame(minWidth: 900, minHeight: 560)
21 + }
22 + .windowStyle(.automatic)
23 + }
24 +}
25 +
26 +struct ContentView: View {
27 + @Environment(SessionModel.self) private var model
28 +
29 + var body: some View {
30 + NavigationSplitView {
31 + VariablesSidebar()
32 + } detail: {
33 + HSplitView {
34 + ConsoleView()
35 + .frame(minWidth: 480)
36 + if model.lastPlot != nil {
37 + PlotView()
38 + .frame(minWidth: 300)
39 + }
40 + }
41 + }
42 + .navigationTitle("Metrika")
43 + }
44 +}
added Metrika/Sources/PlotView.swift +58 −0
@@ -0,0 +1,58 @@
1 +//
2 +// PlotView.swift
3 +// Metrika
4 +//
5 +// Author: Simon-Pierre Boucher
6 +// Contact: contact@spboucher.ai
7 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
8 +//
9 +
10 +import Charts
11 +import SwiftUI
12 +import ZQGraphics
13 +
14 +/// Swift Charts rendering of the session's latest plot spec. The custom
15 +/// Metal renderer takes over past ~1M points (v0.3, CLAUDE.md §7).
16 +struct PlotView: View {
17 + @Environment(SessionModel.self) private var model
18 +
19 + var body: some View {
20 + if let plot = model.lastPlot {
21 + Chart {
22 + ForEach(Array(plot.series.enumerated()), id: \.offset) { _, series in
23 + ForEach(series.x.indices, id: \.self) { index in
24 + switch plot.kind {
25 + case .scatter:
26 + PointMark(
27 + x: .value(plot.xLabel, series.x[index]),
28 + y: .value(plot.yLabel, series.y[index])
29 + )
30 + .symbolSize(20)
31 + .foregroundStyle(by: .value("Series", series.label))
32 + case .line, .kdensity:
33 + LineMark(
34 + x: .value(plot.xLabel, series.x[index]),
35 + y: .value(plot.yLabel, series.y[index])
36 + )
37 + .foregroundStyle(by: .value("Series", series.label))
38 + case .histogram:
39 + BarMark(
40 + x: .value(plot.xLabel, series.x[index]),
41 + y: .value(plot.yLabel, series.y[index])
42 + )
43 + .foregroundStyle(by: .value("Series", series.label))
44 + }
45 + }
46 + }
47 + }
48 + .chartXAxisLabel(plot.xLabel)
49 + .chartYAxisLabel(plot.yLabel)
50 + .padding(12)
51 + } else {
52 + ContentUnavailableView(
53 + "No plot", systemImage: "chart.xyaxis.line",
54 + description: Text("Run a graph command to see it here.")
55 + )
56 + }
57 + }
58 +}
added Metrika/Sources/SessionModel.swift +84 −0
@@ -0,0 +1,84 @@
1 +//
2 +// SessionModel.swift
3 +// Metrika
4 +//
5 +// Author: Simon-Pierre Boucher
6 +// Contact: contact@spboucher.ai
7 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
8 +//
9 +
10 +import Observation
11 +import SwiftUI
12 +import ZQData
13 +import ZQEngine
14 +import ZQGraphics
15 +
16 +/// Observable session state for the UI. Every command — typed in the
17 +/// console or replayed from a do-file — goes through the one `ZQSession`
18 +/// execution path (CLAUDE.md §7).
19 +@Observable @MainActor
20 +final class SessionModel {
21 + struct ConsoleEntry: Identifiable, Equatable {
22 + let id = UUID()
23 + let command: String
24 + let output: String
25 + let isError: Bool
26 + }
27 +
28 + struct VariableInfo: Identifiable, Equatable {
29 + var id: String { name }
30 + let name: String
31 + let type: String
32 + let missingCount: Int
33 + }
34 +
35 + private(set) var entries: [ConsoleEntry] = []
36 + private(set) var variables: [VariableInfo] = []
37 + private(set) var observationCount = 0
38 + private(set) var lastPlot: ZQPlotSpec?
39 + private(set) var isRunning = false
40 + private(set) var history: [String] = []
41 +
42 + private var session: ZQSession?
43 +
44 + init() {
45 + do {
46 + session = try ZQSession()
47 + } catch {
48 + entries.append(ConsoleEntry(
49 + command: "",
50 + output: "failed to start engine: \(error)",
51 + isError: true
52 + ))
53 + }
54 + }
55 +
56 + func run(_ command: String) async {
57 + let line = command.trimmingCharacters(in: .whitespaces)
58 + guard !line.isEmpty, let session else { return }
59 + history.append(line)
60 + isRunning = true
61 + defer { isRunning = false }
62 + do {
63 + let result = try await session.execute(line)
64 + entries.append(ConsoleEntry(command: line, output: result.text, isError: false))
65 + } catch {
66 + entries.append(ConsoleEntry(command: line, output: "\(error)", isError: true))
67 + }
68 + await refreshDatasetState()
69 + }
70 +
71 + private func refreshDatasetState() async {
72 + guard let session else { return }
73 + let frame = await session.frame
74 + observationCount = frame.rowCount
75 + variables = frame.columns.map { column in
76 + VariableInfo(
77 + name: column.name,
78 + type: column.data.isNumeric ? "float64" : "string",
79 + missingCount: column.data.missingCount
80 + )
81 + }
82 + lastPlot = await session.lastPlot
83 + }
84 +}
added MetrikaKit/Package.resolved +33 −0
@@ -0,0 +1,33 @@
1 +{
2 + "originHash" : "e4ae512a67bec7ae66f2abd23f1e87240c39d28811734cdfafc7a91e4323f6db",
3 + "pins" : [
4 + {
5 + "identity" : "duckdb-swift",
6 + "kind" : "remoteSourceControl",
7 + "location" : "https://github.com/duckdb/duckdb-swift",
8 + "state" : {
9 + "revision" : "9583f44fedc20a8e0b6151d2e43f6d850016712f",
10 + "version" : "1.1.3"
11 + }
12 + },
13 + {
14 + "identity" : "swift-argument-parser",
15 + "kind" : "remoteSourceControl",
16 + "location" : "https://github.com/apple/swift-argument-parser",
17 + "state" : {
18 + "revision" : "6a52f3251125d74daf04fcbd5e6f08a75d074382",
19 + "version" : "1.8.2"
20 + }
21 + },
22 + {
23 + "identity" : "swift-collections",
24 + "kind" : "remoteSourceControl",
25 + "location" : "https://github.com/apple/swift-collections",
26 + "state" : {
27 + "revision" : "a0cb0954ecb21e4e31b0070e6ed5674e8556685a",
28 + "version" : "1.6.0"
29 + }
30 + }
31 + ],
32 + "version" : 3
33 +}
added MetrikaKit/Package.swift +96 −0
@@ -0,0 +1,96 @@
1 +// swift-tools-version: 6.0
2 +//
3 +// Package.swift
4 +// Metrika
5 +//
6 +// Author: Simon-Pierre Boucher
7 +// Contact: contact@spboucher.ai
8 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
9 +//
10 +
11 +import PackageDescription
12 +
13 +let strictConcurrency: [SwiftSetting] = [
14 + .swiftLanguageMode(.v6)
15 +]
16 +
17 +let package = Package(
18 + name: "MetrikaKit",
19 + platforms: [.macOS(.v14)],
20 + products: [
21 + .library(
22 + name: "MetrikaKit",
23 + targets: [
24 + "ZQParser", "ZQPlanner", "ZQEngine", "ZQData",
25 + "ZQStats", "ZQGPU", "ZQGraphics", "ZQPlugins",
26 + ]
27 + ),
28 + .executable(name: "metrika-cli", targets: ["MetrikaCLI"]),
29 + ],
30 + dependencies: [
31 + .package(url: "https://github.com/duckdb/duckdb-swift", from: "1.0.0"),
32 + .package(url: "https://github.com/apple/swift-collections", from: "1.1.0"),
33 + .package(url: "https://github.com/apple/swift-argument-parser", from: "1.3.0"),
34 + ],
35 + targets: [
36 + .target(
37 + name: "ZQParser",
38 + swiftSettings: strictConcurrency
39 + ),
40 + .target(
41 + name: "ZQData",
42 + dependencies: [
43 + .product(name: "DuckDB", package: "duckdb-swift"),
44 + .product(name: "Collections", package: "swift-collections"),
45 + ],
46 + swiftSettings: strictConcurrency
47 + ),
48 + .target(
49 + name: "ZQStats",
50 + dependencies: ["ZQData"],
51 + swiftSettings: strictConcurrency
52 + ),
53 + .target(
54 + name: "ZQGPU",
55 + dependencies: ["ZQData"],
56 + swiftSettings: strictConcurrency
57 + ),
58 + .target(
59 + name: "ZQPlanner",
60 + dependencies: ["ZQParser", "ZQData"],
61 + swiftSettings: strictConcurrency
62 + ),
63 + .target(
64 + name: "ZQGraphics",
65 + dependencies: ["ZQData"],
66 + swiftSettings: strictConcurrency
67 + ),
68 + .target(
69 + name: "ZQPlugins",
70 + dependencies: ["ZQParser", "ZQData"],
71 + swiftSettings: strictConcurrency
72 + ),
73 + .target(
74 + name: "ZQEngine",
75 + dependencies: [
76 + "ZQParser", "ZQPlanner", "ZQData", "ZQStats",
77 + "ZQGPU", "ZQGraphics", "ZQPlugins",
78 + ],
79 + swiftSettings: strictConcurrency
80 + ),
81 + .executableTarget(
82 + name: "MetrikaCLI",
83 + dependencies: [
84 + "ZQEngine",
85 + .product(name: "ArgumentParser", package: "swift-argument-parser"),
86 + ],
87 + swiftSettings: strictConcurrency
88 + ),
89 + .testTarget(
90 + name: "MetrikaKitTests",
91 + dependencies: ["ZQParser", "ZQPlanner", "ZQEngine", "ZQData", "ZQStats", "ZQGPU"],
92 + resources: [.copy("Fixtures")],
93 + swiftSettings: strictConcurrency
94 + ),
95 + ]
96 +)
added MetrikaKit/Sources/MetrikaCLI/MetrikaCLI.swift +65 −0
@@ -0,0 +1,65 @@
1 +//
2 +// MetrikaCLI.swift
3 +// Metrika
4 +//
5 +// Author: Simon-Pierre Boucher
6 +// Contact: contact@spboucher.ai
7 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
8 +//
9 +
10 +import ArgumentParser
11 +import Foundation
12 +import ZQEngine
13 +
14 +/// Companion command-line tool: runs ZQL commands or `.zyq` do-files
15 +/// against a headless session. Interactive REPL when no input is given.
16 +@main
17 +struct MetrikaCLI: AsyncParsableCommand {
18 + static let configuration = CommandConfiguration(
19 + commandName: "metrika-cli",
20 + abstract: "Metrika statistical engine — command-line interface."
21 + )
22 +
23 + @Option(
24 + name: [.short, .customLong("execute")],
25 + help: "Execute a ZQL command (repeatable, runs in order)."
26 + )
27 + var execute: [String] = []
28 +
29 + @Argument(help: "A .zyq do-file to run.")
30 + var doFile: String?
31 +
32 + mutating func run() async throws {
33 + let session = try ZQSession()
34 +
35 + if let doFile {
36 + let url = URL(fileURLWithPath: (doFile as NSString).expandingTildeInPath)
37 + let text = try String(contentsOf: url, encoding: .utf8)
38 + let result = try await session.executeScript(text)
39 + print(result.text)
40 + return
41 + }
42 +
43 + if !execute.isEmpty {
44 + for command in execute {
45 + let result = try await session.execute(command)
46 + if !result.text.isEmpty { print(result.text) }
47 + }
48 + return
49 + }
50 +
51 + // REPL
52 + print("Metrika — type ZQL commands, 'exit' to quit.")
53 + while true {
54 + print(". ", terminator: "")
55 + guard let line = readLine() else { break }
56 + if line == "exit" || line == "quit" { break }
57 + do {
58 + let result = try await session.execute(line)
59 + if !result.text.isEmpty { print(result.text) }
60 + } catch {
61 + print("error: \(error)")
62 + }
63 + }
64 + }
65 +}
added MetrikaKit/Sources/ZQData/DataFrame.swift +163 −0
@@ -0,0 +1,163 @@
1 +//
2 +// DataFrame.swift
3 +// Metrika
4 +//
5 +// Author: Simon-Pierre Boucher
6 +// Contact: contact@spboucher.ai
7 +// Copyright © 2026 Simon-Pierre Boucher. All rights reserved.
8 +//
9 +
10 +/// Column payload. Numeric data is Float64 end-to-end (CLAUDE.md §5: all
11 +/// CPU compute in Float64) with an explicit validity mask mirroring Arrow
12 +/// semantics: `missing[i] == true` means observation `i` is missing, and
13 +/// the corresponding value slot is undefined.
14 +public enum ZQColumnData: Equatable, Sendable {
15 + case float64(values: [Double], missing: [Bool])
16 + case string([String?])
17 +
18 + public var count: Int {
19 + switch self {
20 + case .float64(let values, _): return values.count
21 + case .string(let values): return values.count
22 + }
23 + }
24 +
25 + public var missingCount: Int {
26 + switch self {
27 + case .float64(_, let missing): return missing.count { $0 }
28 + case .string(let values): return values.count { $0 == nil }
29 + }
30 + }
31 +
32 + public var isNumeric: Bool {
33 + if case .float64 = self { return true }
34 + return false
35 + }
36 +}
37 +
38 +/// A named column of a data frame.
39 +public struct ZQColumn: Equatable, Sendable {
40 + public var name: String
41 + public var data: ZQColumnData
42 +
43 + public init(name: String, data: ZQColumnData) {
44 + self.name = name
45 + self.data = data
46 + }
47 +
48 + public var count: Int { data.count }
49 +
50 + /// Numeric view: values with their validity mask, or nil for string
51 + /// columns.
52 + public var numeric: (values: [Double], missing: [Bool])? {
53 + if case .float64(let values, let missing) = data {
54 + return (values, missing)
55 + }
56 + return nil
57 + }
58 +}
59 +
60 +public struct ZQDataError: Error, Equatable, Sendable, CustomStringConvertible {
61 + public var message: String
62 + public init(_ message: String) { self.message = message }
63 + public var description: String { message }
64 +}
65 +
66 +/// In-memory data frame: the working dataset of a session. Materialized
67 +/// from DuckDB query results; columnar, immutable value semantics.
68 +public struct ZQDataFrame: Equatable, Sendable {
69 + public private(set) var columns: [ZQColumn]
70 + public let rowCount: Int
71 +
72 + public init(columns: [ZQColumn] = []) throws(ZQDataError) {
73 + let counts = Set(columns.map(\.count))
74 + guard counts.count <= 1 else {
75 + throw ZQDataError("columns have inconsistent lengths: \(counts.sorted())")
76 + }
77 + self.columns = columns
78 + self.rowCount = counts.first ?? 0
79 + }
80 +
81 + public var isEmpty: Bool { columns.isEmpty || rowCount == 0 }
82 + public var columnNames: [String] { columns.map(\.name) }
83 +
84 + public func column(named name: String) -> ZQColumn? {
85 + columns.first { $0.name == name }
86 + }
87 +
88 + public func requireColumn(_ name: String) throws(ZQDataError) -> ZQColumn {
89 + guard let column = column(named: name) else {
90 + throw ZQDataError("variable '\(name)' not found")
91 + }
92 + return column
93 + }
94 +
95 + public func requireNumeric(
96 + _ name: String
97 + ) throws(ZQDataError) -> (values: [Double], missing: [Bool]) {
98 + guard let numeric = try requireColumn(name).numeric else {
99 + throw ZQDataError("variable '\(name)' is a string variable")
100 + }
101 + return numeric
102 + }
103 +
104 + /// Adds a new column; fails if the name already exists (Stata `generate`
105 + /// semantics — use `replaceColumn` for `replace`).
106 + public mutating func addColumn(_ column: ZQColumn) throws(ZQDataError) {
107 + guard self.column(named: column.name) == nil else {
108 + throw ZQDataError("variable '\(column.name)' already defined")
109 + }
110 + guard columns.isEmpty || column.count == rowCount else {
111 + throw ZQDataError(
112 + "column '\(column.name)' has \(column.count) rows, expected \(rowCount)"
113 + )
114 + }
115 + columns.append(column)
116 + }
117 +
118 + /// Replaces an existing column in place (Stata `replace`).
119 + public mutating func replaceColumn(_ column: ZQColumn) throws(ZQDataError) {
120 + guard let index = columns.firstIndex(where: { $0.name == column.name }) else {
121 + throw ZQDataError("variable '\(column.name)' not found")
122 + }
123 + guard column.count == rowCount else {
124 + throw ZQDataError(
125 + "column '\(column.name)' has \(column.count) rows, expected \(rowCount)"
126 + )
127 + }
128 + columns[index] = column
129 + }
130 +
131 + public mutating func dropColumns(_ names: [String]) throws(ZQDataError) {
132 + for name in names {
133 + guard columns.contains(where: { $0.name == name }) else {
134 + throw ZQDataError("variable '\(name)' not found")
135 + }
136 + }
137 + columns.removeAll { names.contains($0.name) }
138 + }
139 +
140 + /// Row subset by keep-mask (`mask[i] == true` keeps observation `i`).
141 + public func filtered(by mask: [Bool]) throws(ZQDataError) -> ZQDataFrame {
142 + guard mask.count == rowCount else {
143 + throw ZQDataError("filter mask length \(mask.count) != \(rowCount) rows")
144 + }
145 + let kept = (0..<rowCount).filter { mask[$0] }
146 + let newColumns = columns.map { column -> ZQColumn in
147 + switch column.data {
148 + case .float64(let values, let missing):
149 + return ZQColumn(
150 + name: column.name,
151 + data: .float64(
152 + values: kept.map { values[$0] },
153 + missing: kept.map { missing[$0] }
154 + )
155 + )
156 + case .string(let values):
157 + return ZQColumn(name: column.name, data: .string(kept.map { values[$0] }))
158 + }
159 + }
160 + // Lengths are uniform by construction.
161 + return try! ZQDataFrame(columns: newColumns)
162 + }
163 +}
added MetrikaKit/Sources/ZQData/DuckDBStore.swift +153 −0
Large diff (153 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQData/ZQDataModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQEngine/ExpressionEvaluator.swift +332 −0
Large diff (332 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQEngine/Session.swift +956 −0
Large diff (956 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQEngine/TableFormatter.swift +48 −0
Large diff (48 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQEngine/ZQEngineModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQGPU/Philox.swift +101 −0
Large diff (101 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQGPU/ZQGPUModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQGraphics/PlotSpec.swift +43 −0
Large diff (43 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQGraphics/ZQGraphicsModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/AST.swift +167 −0
Large diff (167 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/Grammar/CommandParser.swift +480 −0
Large diff (480 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/Grammar/ExpressionParser.swift +128 −0
Large diff (128 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/KnownVerbs.swift +109 −0
Large diff (109 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/Lexer.swift +169 −0
Large diff (169 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/ParseError.swift +52 −0
Large diff (52 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/Token.swift +55 −0
Large diff (55 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQParser/ZQParserModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQPlanner/Planner.swift +106 −0
Large diff (106 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQPlanner/ZQPlannerModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQPlugins/PluginProtocol.swift +128 −0
Large diff (128 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQPlugins/ZQPluginsModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQStats/Distributions.swift +149 −0
Large diff (149 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQStats/LinearAlgebra.swift +199 −0
Large diff (199 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQStats/OLS.swift +316 −0
Large diff (316 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQStats/Summarize.swift +121 −0
Large diff (121 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Sources/ZQStats/ZQStatsModule.swift +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/EngineTests.swift +147 −0
Large diff (147 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/FixtureSupport.swift +61 −0
Large diff (61 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/Fixtures/expected.tsv +47 −0
Large diff (47 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/Fixtures/regression.csv +61 −0
Large diff (61 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/ParserTests.swift +174 −0
Large diff (174 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/PhiloxTests.swift +92 −0
Large diff (92 lines) is not loaded by default to keep the page fast.Load diff
added MetrikaKit/Tests/MetrikaKitTests/StatsTests.swift +218 −0
Large diff (218 lines) is not loaded by default to keep the page fast.Load diff
added Tests/Bench/bench.sh +54 −0
Large diff (54 lines) is not loaded by default to keep the page fast.Load diff
added Tests/Fixtures/generate.R +161 −0
Large diff (161 lines) is not loaded by default to keep the page fast.Load diff
added project.yml +37 −0
Large diff (37 lines) is not loaded by default to keep the page fast.Load diff
added scripts/ExportOptions.plist +9 −0
Large diff (9 lines) is not loaded by default to keep the page fast.Load diff
added scripts/check_headers.sh +45 −0
Large diff (45 lines) is not loaded by default to keep the page fast.Load diff
added scripts/install_hooks.sh +14 −0
Large diff (14 lines) is not loaded by default to keep the page fast.Load diff
added scripts/make_icns.sh +41 −0
Large diff (41 lines) is not loaded by default to keep the page fast.Load diff
added scripts/pre-commit +13 −0
Large diff (13 lines) is not loaded by default to keep the page fast.Load diff
added scripts/release.sh +59 −0
Large diff (60 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere