Stata-class statistics, GPU-accelerated by Apple Silicon. Native Swift — no Electron, no Python runtime, no compromises.
-
Wed, Aug 5, 2026 13
-
feat(app): user manual — in-app reference and console help
…
- ZQCommandReference: one registry of every command (syntax, options, examples, notes) shared by the console and the app so they cannot drift; 35 commands across 10 categories - console: 'help' lists the reference by category, 'help <command>' (abbreviations resolve too) renders the full entry - app: Manual pane — searchable sidebar grouped by category, per-command pages (syntax/options/examples), and a 'How Metrika works' overview covering the command grammar, data handling, estimation conventions, the CPU/GPU planner, reproducibility, and user commands - release plumbing: real team ID + notary keychain-profile NAME in release.sh (both configuration, not credentials, per the security conventions), manual Developer ID signing with hardened runtime, -skipPackagePluginValidation, DMG background art - tests: help coverage test pins a manual entry for every implemented verb; UI smoke drives the Manual pane; 106 green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(stats): elastic net and lasso via coordinate descent
…
- ZQElasticNet: cyclic coordinate descent with residual updates, glmnet-convention objective ((1/2n)RSS + lambda(alpha*l1 + (1-alpha)/2*l2)), internal predictor standardization (1/n variance), unpenalized intercept, coefficients reported on the original scale; lambdaMax helper - matched glmnet's gaussian y-standardization quirk deliberately: the L1 penalty is invariant to it but the effective ridge penalty scales by 1/sd(y) — without this, alpha<1 fits diverge from glmnet by ~10% - engine: 'elasticnet y x…, lambda(#) [alpha(#)]' and 'lasso' (alpha fixed at 1); missing lambda() errors with the data's lambda_max as a hint; predict works afterwards, margins refuses (no VCE) - fixtures: glmnet 5.0 at thresh 1e-15 over deliberately correlated regressors; coefficients match at 1e-6 (documented tolerance for penalized iterative solvers) and the selection pattern (which coefficients are exactly zero) matches exactly - ZQCoefficient gains a public initializer - 104 tests green (swift test and xcodebuild with GPU suites) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(engine): margins dydx() with delta-method standard errors
…
- ZQOLSResult/ZQGLMResult expose the full covariance matrix (column-major k x k aligned with coefficients); IV passes it through - EstimationState carries vce, inference df, and the estimation-sample design (GLMs only — AMEs need it) - margins, dydx(varlist): OLS/IV effects are the coefficients with their SEs; GLM average marginal effects with analytic delta gradients (logit p(1-p)(1-2p), probit -xb*phi, poisson exp) over the estimation sample; t or z inference per model kind - factor/interaction dydx rejected with a clear message (discrete-change margins later); continuous terms of factor models work - R fixtures mirror the exact formulas at the converged coefficients; logit/poisson AME and delta SE match at 1e-10 - 98 tests green (swift test and xcodebuild with GPU suites) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(engine): predict with estimation state (e() analog)
…
- PredictorDefinition (column / indicator / product / constant) records how each fitted regressor is recomputed from the dataset; threaded through buildRegressionSample so factor expansions and interactions carry their recipes - EstimationState stored after regress, logit/probit/poisson, and ivregress; cleared after xtreg (predict there needs the estimated u_i) - predict newvar [, xb | residuals | pr | n]: evaluates over ALL current observations with missing propagation; kind-aware defaults (xb for ols/iv, pr for logit/probit, n for poisson) and statistic validation - ZQFactorExpansion.expand now returns the numeric level alongside each indicator column - tests: xb + residuals reconstruct the response exactly (plain and factor models), mean fitted probability/count equals ybar (score equations), option validation; 93 green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(plugins): harden the user-command protocol
…
- fix: script-command verbs were routed AFTER parsing, which rejects unknown verbs — .zyq commands could never run; they now route before the parser with raw macro arguments - .zyq scripts: ado-style 'args name…' declaration with backtick macro substitution (`0', `k', `name'); scratch-variable scripts verified end-to-end against the built-in path - ZQPluginRegistry: verb-keyed native plugin registry; verbs shadowing built-ins rejected at session start; parsed commands validated against the declared ZQSyntaxSpec (varlist/if/in/weights/options) - mutation gating: plugins return a replacement dataset via ZQResult.replacementFrame, installed only under 'mutates: true' — a non-mutating plugin returning one is an error and the dataset is untouched (ZQContext is now a plain value, no writable back-channel) - ZScorePlugin ships as the reference native plugin, registered by the app ('zscore varname' -> standardized z_varname) - CLAUDE.md: document the mlx-swift build quirks (xcodebuild for GPU tests, -skipPackagePluginValidation everywhere) - 87 tests green (swift test and xcodebuild with GPU suites); app builds Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> -
feat(stats): permutation tests and single-variable graphics
…
- ZQResampling.permutationIndices: permutation as argsort of 64-bit Philox keys — order-free and counter-addressable so a future GPU argsort path reproduces it exactly; dedicated stream offset (bit 63, applied after word expansion) keeps permutations disjoint from bootstrap draws under the same seed - engine: 'permute, reps(#) [seed(#)]: reg …' — permutes the response within the estimation sample, parallel chunked refits, empirical two-sided p per coefficient (c, reps, p, SE(p)); degenerate _cons row omitted - planner: GPU heuristic restricted to bootstrap until the permute argsort path lands - graphics: standalone histogram/scatter/kdensity verbs; single-variable histogram (Sturges default, bins() option) and Epanechnikov kdensity with Silverman bandwidth (density integrates to 1 in tests) - 81 tests green (swift test and xcodebuild with GPU suites) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(stats): xtreg fixed effects and ivregress 2sls
…
- parser: (endog = instruments) varlist groups -> ZQIVSpec; digit-led sub-commands (2sls) reassembled from number+identifier tokens via column adjacency - ZQFixedEffects: within estimator with Stata conventions (add-back means, reported _cons, df = N-K-G), within R-squared, panel-clustered VCE with G/(G-1) and t on G-1 df; v0.2 restriction: cluster variable must equal the panel variable - ZQIV: 2SLS via thin-Q projection of the instrument matrix (Z'Z never formed), residuals from original regressors, Stata 'small' inference, classical/HC1/cluster VCE built on projected regressors - engine: xtreg (requires xtset + fe), ivregress 2sls with dedicated listwise deletion across depvar/exog/endog/instruments; shared coefficient-table renderer extracted - fixtures: z1/z2 instrument columns (drawn after existing draws, earlier golden values bit-identical), manual within/2SLS algebra in R - 75 tests green (swift test and xcodebuild with GPU suites) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(gpu): MLX-batched pairs bootstrap with bit-identical Philox streams
…
- mlx-swift dependency in ZQGPU (the only module allowed to import MLX) - Philox4x32-10 as vectorized MLX uint64 ops with the reference key schedule: GPU resample indices are bit-identical to the CPU generator (asserted per replicate) — the §5 reproducibility contract - batched gather + float32 cross-products on GPU, float64 Cholesky solves on CPU (§5 hybrid); chunked to the memory budget - conditioning: predictors standardized by fixed full-sample mean/scale (exact reparametrization) before forming X'X — raw data with an intercept lost 2-3 digits in float32 - MLX workaround: batched GEMM mis-accumulates small k x k outputs at batch >= 2 (~6e-4 rel, stride/fusion-independent); X'X is built one column at a time through the exact (count,k,n)@(count,n,1) path, with regression tests pinning the behavior - planner: gpuAvailable detected from actual metallib presence (SwiftPM CLI cannot compile Metal shaders - swift test skips GPU suites, the xcodebuild-built app and tests get the GPU); reps >= 500 dispatches bootstrap to .gpu - engine: backend-aware bootstrap, drops singular resamples, output titled 'GPU batched' with gpu scalar - 68 tests green under xcodebuild (66 + GPU suites under swift test skip) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(ui): data browser and do-file editor panes
…
- DataBrowserPane (§7 pane 3): NSTableView bridge (natively virtualized), sortable columns with missing-last ordering, observation-number gutter, filter bar compiled to an if-expression through the real parser (ZQCommandParser.parseExpression + ZQSession.conditionMask are the new public entry points) - DoFileEditorPane (§7 pane 2): NSTextView bridge with ZQL highlighting (verbs resolved via ZQVerbTable, comments, strings), run selection or file with Cmd-R through the shared session path, open/save .zyq - ContentView: segmented Console / Data / Do-file switcher in the toolbar - UI smoke extended: pane switching, filter narrowing (2 of 5 obs), do-file run surfacing in the console and sidebar - 61 kit tests + 2 UI tests green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(data): native Stata .dta reader (117-119) and writer (118)
…
- DTAReader: formats 117/118/119, both byte orders, all numeric storage types widened to Float64 with Stata missing codes (., .a-.z) mapped to the validity mask; str# and strL (GSO) load as strings; value labels read past but not yet applied; pre-117 files rejected with a clear message - DTAWriter: format 118 (UTF-8, LSF), doubles + str# up to 2045 bytes, real map offsets so Stata/haven/pandas can seek - ZQDataStore routes .dta through the native path for use and save - ZQColumnData equality now ignores value slots at missing positions (undefined by contract, often NaN — synthesized == failed on NaN != NaN) - Fixtures: haven-written 117 + 118 files; tests assert bit-identical numeric loads vs the CSV, lossless write/read roundtrip; haven cross-reads Metrika-written files (verified) - 61 tests green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat(stats): logit/probit/poisson MLE, tabulate, correlate
…
- ZQGLM: Fisher-scoring IRLS through the LAPACK QR path (X'WX never formed), converged to |Δll| < 1e-13; classical, robust (HC0 score sandwich, Stata ML convention), and cluster (G/(G−1)) VCE; LR/Wald chi2, McFadden pseudo-R²; information matrix re-evaluated at the converged beta (R's vcov carries last-iteration weights and is only ~1e-6-accurate by its own stopping rule — fixtures compute expected information at the optimum explicitly) - Distributions: regularized incomplete gamma, chi-square CDF/p-value, normal quantile - ZQCorrelate: Pearson matrix on listwise-complete data - Engine: logit/probit/poisson tables (z, P>|z|, LR chi2, pseudo R²), one-way and two-way tabulate with totals, Stata-style lower-triangle correlate - Fixtures: binomial/poisson outcomes drawn after existing draws (earlier golden values bit-identical); glm at epsilon 1e-12 - 56 tests green (12 new) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
test(ui): add XCUITest console smoke per CLAUDE.md §9
…
- MetrikaUITests target (xcodegen scheme test action) - launch-environment hook (DEBUG only): sandboxed app materializes the test CSV in its own container, since the runner cannot stage files across the sandbox boundary reliably - queries predicate on AX 'value' (SwiftUI Text exposes value, not label) - smoke: launch, auto-load dataset, type gen + reg robust, assert regression table and sidebar variable Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-
feat: bootstrap Metrika v0.1 skeleton with working ZQL vertical slice
…
- MetrikaKit SPM package: ZQParser, ZQPlanner, ZQEngine, ZQData, ZQStats, ZQGPU, ZQGraphics, ZQPlugins (Swift 6, strict concurrency) - ZQL parser: lexer, Pratt expressions, factor variables, prefix commands, column-cited errors with Levenshtein verb suggestions - ZQData: DuckDB-backed load/save (parquet, csv, json, arrow) - ZQStats: OLS via LAPACK QR, HC0-HC3 and cluster-robust SE, summarize, t/F distributions accurate in the far tails - ZQGPU: Philox4x32-10 reference RNG, counter-addressable bootstrap - ZQEngine: session actor with use/save/gen/replace/drop/keep/summarize/ regress/count/list/graph/bootstrap/set seed/xtset/log - SwiftUI app (xcodegen): console with history, variables sidebar, Swift Charts plots; sandboxed + hardened runtime entitlements - Tests: 44 green (parser golden, R fixtures at 1e-10, Philox KAT, end-to-end engine); Tests/Fixtures/generate.R; Tests/Bench harness - scripts: check_headers.sh + pre-commit hook, make_icns.sh, release.sh Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
-