SPB Git

spb/zyquo-local Public MIT

Native macOS AI chat that runs LLMs 100% locally on Apple Silicon with MLX — no cloud, no API keys.

Swift 97.2% Shell 1.8% Makefile 1%
12.1 KB

# Zyquo Local — Build Plan

Machine: Apple M5 Max, 18 cores, 48 GB RAM, macOS 27.0 (beta), Swift 6.4 (Command Line Tools only, no Xcode IDE installed).

# Phase 0 — Intensive research — ✅ DONE (2026-07-30)

  • Research current mlx-swift packages, versions, SPM coordinates, min macOS
  • Research current MLXLLM/MLXLMCommon API surface (load, tokenize, chat template, streaming, params, KV cache, unload)
  • Document model directory format (required files)
  • Document supported architectures list
  • Resolve the no-Xcode build question EMPIRICALLY (scratch SPM package + swift build with CLT only)
  • Decide swift-transformers vs direct HTTP HubService
  • Document memory model + GPU cache limits + RAM estimation
  • Write docs/MLX-RESEARCH.md
  • Research HF Hub HTTP API (search, model info, resolve URLs, LFS, tokens)
  • Curate 20–30 Featured models, live-verified on the Hub with sizes
  • RAM recommendation table
  • Write docs/MODELS.md

Checkpoint: Stack pinned: mlx-swift-lm 3.31.4 (MLXLLM/MLXLMCommon/MLXHuggingFace)

  • swift-huggingface 0.9.0 + swift-transformers 1.3.x on mlx-swift 0.31.6. No-Xcode question resolved empirically: Swift 6.4 swift build compiles Metal kernels when a standalone Metal.xctoolchain is on PATH (copied from cluster node M4M36 → ~/Developer/Metal.xctoolchain); GPU execution verified. SwiftUI needs SDKROOT pinned to MacOSX26.5.sdk (macro-plugin issue on 27.x SDKs). Downloads: custom URLSession manager (Range-resume verified 206); tokenization via swift-transformers. 30-model catalog live-verified with real sizes.

# Phase 1 — Project setup — ✅ DONE (2026-07-30)

  • Package.swift (tools 6.1, mlx-swift-lm 3.31.4 + swift-huggingface + swift-transformers + swift-markdown)
  • Makefile (SDK pin + Metal PATH, bundle assembly incl. *.bundle resources, dev/release targets)
  • Support/Info.plist per spec (com.zyquo.local, LSMinimumSystemVersion 14.0, arm64)
  • @main entry with CLI dispatch (--poc) + SwiftUI app + Apple Silicon gate
  • scripts/check-headers.sh sweep (green)
  • make build green, zero warnings from our sources (68 s full graph)
  • make bundle-debug produces an ad-hoc-signed arm64 bundle; CLI dispatch smoke-tested

Checkpoint: Full dependency graph (2 040 tasks incl. Cmlx Metal kernels) compiles under the pinned recipe. Bundle carries mlx-swift_Cmlx.bundle (metallib), Hub + Crypto resource bundles. codesign -dv: com.zyquo.local, arm64, adhoc. Executable dispatches --poc and exits 64 as designed.

# Phase 2 — Architecture + inference PoC — ✅ DONE (2026-07-30)

  • Models/: Conversation, Message (+MessageStats), LocalModel, DownloadTask, Persona
  • Engine/GenerationParams.swift (Codable app params → GenerateParameters at engine boundary)
  • Engine/MemoryAdvisor.swift (verdicts 60 %/75 % of hw.memsize, live MLX.Memory readouts)
  • Engine/ChatSession.swift (history → chat template via MLX session, oldest-turn truncation keeping system prompt, KV reuse, per-call params without KV loss)
  • Engine/InferenceEngine.swift (actor; unloaded→loading→ready⇄generating; AsyncThrowingStream; cancellation wired via task cancel + onTermination; unload → clearCache)
  • PoCRunner: ZyquoLocal --poc <dir> "<prompt>"
  • PHASE GATE: Qwen3-0.6B-4bit loaded from local dir, streamed to stdout — 130.4 tok/s, TTFT 1.59 s, 329 tokens, peak 388 MB, stop reason stop.

Checkpoint: Zero warnings. Engine layer fully owns inference; stats are first-class; <think> content streams through (reasoning display feeds on it in Phase 6). Multi-turn KV reuse + cancellation get their formal end-to-end tests in the Phase 7 harness.

# Phase 3 — Hub browse & download — ✅ DONE (2026-07-30)

  • HubService: live search (scopes featured/mlx-community/all-MLX, config=true arch check), tree file listing with Link-header pagination, Codable structs throughout, HF-token support, human-readable error surfaces (gated/429/…)
  • SupportedArchitectures shipping the verified LLMTypeRegistry list
  • FileTransfer: delegate-backed chunked transport, Range resume (206), transparent restart (200), 416 handling, cross-host Authorization strip, cancellation
  • DownloadManager: queued, 2 concurrent files, pause/resume/cancel per model, transient-drop retry with backoff, atomic .partial → final, size verification, disk pre-check (5 % headroom), manifest persisted across relaunch, speed EMA + ETA
  • ModelStore: scan/validate (required-file check), size on disk, delete + reclaim, reveal in Finder, last-used + per-model default params sidecar
  • Services: PersistenceService (JSON per conversation + generic docs), ModelCatalog (30 featured entries mirroring docs/MODELS.md, starter picks per RAM)
  • PHASE GATE (--hub-poc, isolated temp store): search ✅ listing ✅ download 41 MB/s ✅ pause at 27 % ✅ Range-resume from 100 MB ✅ verify ✅ scan (arch qwen3, 4bit, ctx 40960, Fits) ✅ delete ✅

Checkpoint: Zero warnings. Coherence sweep: headers green, LocalModel is the single term, folder structure matches the Phase 2 tree (DesignSystem/, ViewModels/, Views/ arrive with Phases 4/6).

# Phase 4 — Design system & UI spec — ✅ DONE (2026-07-30)

  • ZyquoTheme tokens: exact light palette from spec, derived graphite-green dark, semantic status colors
  • Typography scale (title 20 sb / body 13.5 / emphasis / caption 11 / code 12.5 mono), 1.45 line-height helper, user-adjustable chat size 12–18
  • Spacing 4–32, radii 6/10/14, 760 pt message column, 0.5 pt hairlines, ultra-soft shadow tokens
  • AccentChoice (emerald default + graphite/sky/amber/rose) + ThemeStore (mode/accent/font persisted)

Checkpoint: Zero warnings. UI layout contract for Phase 6 is §4.2 of the project CLAUDE.md (exact spec); the design quality gate (§4.4) runs before Definition of Done.

# Phase 5 — App icon — ✅ DONE (2026-07-30)

  • Both creative directions rendered and compared at 512/64 px: Z-core chosen (bold circuit-trace Z, far stronger small-size legibility); Z-chip kept as exploration
  • Final zyquo-local.svg: graphite gradient squircle, radial top light (no banding), emerald-gradient Z trace with restrained glow, routed-net stubs, node micro-highlights
  • Simplified 16/32 px variant (no traces/glow, thicker Z) — verified legible at 16 px
  • Monochrome menu bar template (18/36 px PNGs)
  • scripts/make-icon.sh: SVG → iconset (simple for ≤32, full for ≥64) → AppIcon.icns via iconutil
  • Makefile bundles AppIcon.icns + menubar PNGs into Resources

Checkpoint: AppIcon.icns (652 KB) generated; LOOKed at 512/64/32/16 renders and iterated (fixed vignette band edge). rsvg-convert installed via Homebrew per the spec's pipeline.

# Phase 6 — Features / full UI — ✅ DONE (2026-07-30)

  • AppModel + ChatController (streaming, live parser robust to split tags, ≤4 Hz tok/s ticker, auto-titling with model + fallback)
  • Sidebar: wordmark, search (verified live), New Chat, Library entry with download badge, Pinned/Today/… groups, pin/rename/delete/export context menu, footer settings + loaded-model chip with RAM dot + live memory
  • Chat: 52 pt header (editable title, centered model chip → switcher popover with verdicts, perf toggle, export, info popover with system prompt + params + context bar), transcript (user accentSubtle right / assistant surface left, hover timestamps, jump-to-bottom pill, streaming caret), thinking disclosure, per-message stats line, message actions (copy/edit&resend/regenerate/quote/delete)
  • Markdown renderer (swift-markdown blocks + AttributedString inline) with syntax-highlighted code blocks + copy
  • Input bar: floating card r14, attach text files (picker + drag&drop, fenced injection), prompt-library picker, params popover, ⌘↩ send / stop
  • Onboarding hero with starter cards (RAM-fit for THIS Mac, inline download progress)
  • Library: Installed (badges, verdicts, load/unload, per-model params, reveal, delete with reclaimed GB) + Discover (Featured curated sections, live search w/ scopes+filters+sort, cards flip to progress with MB/s + ETA + pause/cancel) + downloads drawer
  • Settings 6 tabs (General/Models&Storage w/ masked HF token/Inference w/ GPU cache limit/Appearance w/ accents + live font preview/Shortcuts/Advanced w/ export-import)
  • Quick Chat panel (⌥Space Carbon global hotkey, Spotlight-style, one-click load hint)
  • Compare mode (two engines, RAM-gated, broadcast prompt, independent streams + stats)
  • Menu bar extra (state + downloads glance, template icon), shortcuts (⌘N/⌘K/⌘L/⌘F/⌘↩/⇧⌘E/⌥Space), export MD + PDF
  • Prompt library: 56 built-in templates ({{input}} variables) + user templates; Personas store with 4 built-ins

Checkpoint (verified live on screen): default model auto-loads on launch; full chat round-trip through the real UI — user bubble, thinking disclosure, “⚡ 107.0 tok/s · 467 tokens · 2.0s to first token” stats line, auto-title; sidebar search filters live; Installed + Discover screens render per spec. Zero warnings. Dark-mode + full design gate re-run before Definition of Done.

# Phase 7 — Local model verification — ✅ DONE (2026-07-30)

  • --verify harness: download via the app's own DownloadManager, validate, load, deterministic gen, multi-turn carry-over, streaming cancellation, unload + memory-release check, tok/s + TTFT recorded
  • 6 models across architectures (qwen3/llama/smollm3/gemma3/qwen2-coder/R1-distill): 36/36 checks green
  • Entire Featured catalog (30 repos) dry-verified live: exists + files + sizes within 10 % — 30/30, zero corrections needed (MODELS.md ↔ ModelCatalog in sync)
  • Test models deleted, smallest kept for dev; results table in docs/VERIFICATION.md

Checkpoint: VERIFY: ALL GREEN, exit 0. Best perf: Llama-3.2-1B at 222.8 tok/s; unloads verified down to KBs of MLX active memory.

# Phase 8 — Signing & notarization — ✅ DONE (2026-07-30)

  • zyquo-term setup inspected and reused: identity Developer ID Application: Simon-Pierre Boucher (3YM54G49SN), notarytool keychain profile MacLustr-Notarize (verified live before use)
  • Support/entitlements.plist: hardened runtime, NO JIT/unsigned-memory entitlements — verified unnecessary by running MLX inference in the signed build
  • scripts/release.sh (make release): release build → bundle (incl. Cmlx metallib bundle) → sign nested bundles then app → ditto zip → notarytool submit --wait → staple → spctl → signed+notarized DMG
  • First submission Accepted (app AND dmg); xcrun stapler validate
  • spctl -a -vv: accepted, source=Notarized Developer ID
  • Notarized app runs inference: 554.3 tok/s, TTFT 0.03 s (release build), GUI launches
  • Final coherence sweep: syntax palette moved into ZyquoTheme, naming fix, headers green

Checkpoint: dist/Zyquo Local.app (signed, notarized, stapled) + dist/ZyquoLocal.dmg (13 MB, notarized, stapled). make dev keeps ad-hoc signing for iteration.


# Definition of Done — final review (2026-07-30)

  • make release → Developer ID–signed, notarized, stapled app (spctl: Notarized Developer ID)
  • In-app HF browsing + one-click resumable downloads (pause/Range-resume verified live); Featured catalog live-verified 30/30
  • Chat with downloaded MLX models: streaming, multi-turn with context management, stop, stats (tok/s, TTFT), reasoning display — verified on screen and in the harness
  • MemoryAdvisor verdicts accurate for this Mac; load/unload verifiably frees memory (GB → KB)
  • Silicon-themed SVG icon (Z-core) at all sizes, .icns embedded + menu bar template; sibling DNA of Zyquo Cloud
  • Emerald light theme per Phase 4 token spec; dark theme derived (graphite-green) and verified on screen
  • Naming coherent: Zyquo Local user-facing, com.zyquo.local, ZyquoLocal target/data folder
  • Phase 7 table fully green; docs/MODELS.md ↔ ModelCatalog in sync
  • Mandatory header on every code file (repository-wide sweep green)
  • docs/PLAN.md complete with per-phase checkpoints