SPB Git

spb/zyquo-mlx Public MIT

The local MLX foundry for your Mac — run, fine-tune, quantize, and ship models. Nothing leaves your machine.

Swift 93.4% Python 3.8% Makefile 2.2% Shell 0.5%
16.7 KB

# Zyquo MLX — Execution Plan

Strict phase order 0 → 8. One phase at a time. Each phase ends with a checkpoint (build, run, zero warnings, 3–5 line summary here) before the next begins.


# Phase 0 — Mandatory Intensive Web Research ✅ (completed 2026-07-30)

  • 0.A docs/MLX-RESEARCH.md — MLX framework deep dive
    • Core primitives: arrays, lazy eval / eval(), unified memory, streams/devices, autodiff
    • Swift vs. Python honest capability split (current, verified against live repos)
    • Model types & current MLX model zoo (LLM / VLM / embeddings / speech / image-gen)
    • Inference API specifics (Swift-first: loading, tokenizers, streaming, params, KV cache, memory release)
    • Quantization & conversion (tools, exact commands, on-disk format)
    • Build & execution strategy resolved (Metal toolchain question TESTED locally)
    • Memory & performance (RAM estimation for inference vs. training)
  • 0.B docs/TRAINING-RESEARCH.md — fine-tuning on MLX
    • LoRA / QLoRA exact workflow, hyperparameters, adapter save/fuse
    • Full fine-tuning feasibility & memory cost
    • Dataset formats (chat / prompt-completion / text), split, templating, validation
    • Training observability: metrics, checkpoint cadence, resume
    • Evaluation: held-out loss/perplexity, base vs. tuned comparison
  • 0.C docs/MODELS.md — Hub integration + curated catalog
    • Hugging Face Hub HTTP API (search, info, tree, resolve, LFS, token)
    • Featured catalog across types/sizes, live-verified mlx-community repo IDs (43 repos, 0 failures)
    • RAM table (8–128 GB) for inference AND LoRA fine-tuning
  • docs/BUILD.md — no-Xcode-IDE build recipe incl. Metal (tested on this Mac)
  • Phase checkpoint: docs complete, traceable, committed

Phase 0 summary: Verified the mid-2026 MLX ecosystem live (mlx 0.32.0, mlx-swift 0.31.6, mlx-swift-lm 3.31.4, mlx-lm 0.31.3): LM libraries moved to ml-explore/mlx-swift-lm 3.x, which now natively covers LLM/VLM/embeddings inference, LoRA/QLoRA training, and safetensors conversion/quantization; the Python bridge is required only for rich-dataset training, full FT, speech, FLUX, advanced quant, and evaluation. Locally proved CLT-only builds fail at Metal kernel compilation → strategy: full Xcode as toolchain, CLI-only builds, mlx-swift_Cmlx.bundle shipped in the app. Catalog: 43 models live-verified.

# Local ground truth (recorded 2026-07-30)

  • Host: Apple M5 Max, 48 GB RAM, macOS 27.0 (25A5388g toolchain era), arm64
  • Swift 6.4 (swiftlang-6.4.0.25.4), Command Line Tools only at /Library/Developer/CommandLineToolsxcodebuild unavailable, no metal compiler in CLT
  • TESTED 2026-07-30 (clean run, /tmp/zyquo-mlx-buildtest, mlx-swift 0.31.x): swift build with CLT only compiles all of Cmlx C++/Swift, then fails at CompileMetalFile steel_attention.metal with error: unable to spawn process 'metal' (No such file or directory). → CLT-only build is IMPOSSIBLE; a real Metal toolchain (full Xcode, CLI-only usage) is required. Per charter 0.A.6 the rule adapts to: command-line swift build/xcodebuild only, no Xcode IDE, no hand-authored .xcodeproj.
  • No Xcode.app present anywhere on this machine yet → Phase 1 must provision the Xcode toolchain (download strategy documented in docs/BUILD.md)
  • Python 3.14.4 (Homebrew) + uv available; rsvg-convert + iconutil present for the Phase 5 icon pipeline

# Phase 1 — Project Setup ✅ (completed 2026-07-30)

  • Xcode 26.6 toolchain provisioned (CLI-only; Metal toolchain 17F109 installed)
  • Package.swift — target ZyquoMLX, deps per MLX-RESEARCH.md §8 (resolve exact: mlx-swift 0.31.6, mlx-swift-lm 3.31.4, swift-transformers 1.3.3, swift-huggingface 0.9.0)
  • Makefile — build / app / dev / run / icon / clean (+ release stub for Phase 8)
  • Info.plist — Zyquo MLX, com.zyquo.mlx, macOS 14+, arm64, developer-tools
  • @main SwiftUI entry point + Apple-Silicon gate + terminal activation
  • Python bootstrap plan wired (first-run venv per BUILD.md §3.4 — code lands Phase 3)
  • README.md (build instructions)
  • Phase checkpoint: build green (0 warnings), Zyquo MLX.app assembled, launched, window verified by screenshot

Phase 1 summary: Xcode 26.6 installed as toolchain-only; resolved the build definitively — classic SwiftPM silently skips Metal (broken GPU path), but swift build --build-system swiftbuild -c release compiles all kernels and emits mlx-swift_Cmlx.bundle/default.metallib (~90 s clean, 0 warnings), so no xcodebuild is needed. make app assembles an ad-hoc-signed Zyquo MLX.app with all resource bundles; app launches with the Apple-Silicon gate in place. BUILD.md §3.1 updated with the resolved recipe.

# Phase 2 — Architecture + Inference POC ✅ (completed 2026-07-30)

  • Folder architecture per charter (App/ Models/ Engine/ Training/ Hub/ Services/ established with real code; Convert/ Data/ PyBridge/ arrive with Phase 3, DesignSystem/ ViewModels/ Views/ with Phase 4 — no dead placeholder files)
  • Models/ domain types: LocalModel, ModelType, Dataset, TrainingRun, Checkpoint, Job (+ HyperParams mirroring mlx-lm defaults)
  • Engine/: InferenceEngine actor + ModelTypeAdapters (LLM/VLM/Embeddings via LLMModelFactory/VLMModelFactory/EmbedderModelFactory) + GenerationParams + MemoryAdvisor (device-derived working-set gating)
  • Hub/ModelStore: local library scan/validate (config.json, shard-index consistency, quantization metadata, param-count derivation)
  • CLI POC mode (ZyquoMLX --infer <dir> / --embed <dir>) before UI
  • PHASE GATE: LLM (Qwen3-0.6B-4bit) streamed at 603.7 tok/s, TTFT 0.83 s, unload freed 319.8 MB (verified); embeddings (all-MiniLM-L6-v2-4bit) returned 384-dim L2-normed vectors with correct similarity ordering (0.86 related vs 0.72 unrelated)
  • Phase checkpoint: build green, 0 warnings, arm64-only

Phase 2 summary: Domain models, InferenceEngine actor with typed adapters, ModelStore validation, and MemoryAdvisor are in place; every dependency API was verified against the checked-out mlx-swift-lm 3.31.4 sources (no guessed names — #huggingFaceTokenizerLoader macro requires import Tokenizers at the expansion site; non-Sendable UserInput goes through perform(nonSendable:)). CLI POC proves the two-model-type gate with real downloads in the app's Models library. Build is arm64-only via --arch arm64 (swiftbuild otherwise builds universal and x86_64 fails).

# Phase 3 — Training, Quantization & Conversion ✅ (completed 2026-07-30)

  • PyBridge/PythonRunner — Process wrapper over the venv, JSON-lines progress protocol, cancellation
  • PyBridge/PythonEnvironment — uv-provisioned pinned venv in App Support (idempotent adopt/verify/repair; mlx-lm 0.31.3 now, vlm/whisper/audio on demand)
  • PyBridge/scripts/ — zyquo_train.py (TrainingCallback → JSON; bypasses lora.run() which drops the callback in 0.31.3), zyquo_fuse.py, zyquo_convert.py
  • Data/DatasetService + DatasetFormats — import/validate JSONL (chat/completions/text), deterministic split, malformed-row report with fixes, token stats, preview
  • Training/TrainingService + RunStore + MetricsStream — cancellable runs, persisted state + metrics.jsonl, numbered checkpoints, warm-start resume, MemoryAdvisor gating
  • Convert/ConversionService + QuantConfig — Swift-native affine/mxfp4 quant (size preview accurate to 0.1%); Python bridge for fuse/convert
  • CLI: --train, --fuse, --quantize, --validate-dataset (UI in Phase 6)
  • PHASE GATE: QLoRA on Qwen3-0.6B-4bit, 48-row chat dataset — live loss 4.62→0.10 (val 6.31→0.14) at ~2,300 tok/s, checkpoints at 40/80/120, adapter fused (auto-dequantize), fused model answers in the trained "⚒ From the forge:" style
  • Phase checkpoint: build green, 0 warnings

Phase 3 summary: Full foundry core working end-to-end from the app. Two upstream landmines found and handled: (1) mlx-lm 0.31.3's lora.run() silently discards the caller's TrainingCallback — our driver replicates run()'s flow and calls train_model directly; (2) fusing into a quantized base re-quantizes and rounds away small LoRA deltas — Zyquo defaults to dequantize-on-fuse, recommends adapter-attached inference, and warns on re-quantization (documented in TRAINING-RESEARCH.md §2.2). Swift-native quantization verified: 1.19 GB fp16 → 335.5 MB 4-bit, predicted 335.3 MB.

# Phase 4 — Design System & UI ✅ (completed 2026-07-30)

  • DesignSystem/ZyquoTheme — copper-on-slate tokens (light flagship + derived dark), typography (SF Mono for metrics), 4pt spacing grid, radii, 0.5pt hairlines, motion; zero raw hex in views (hex lives only inside the token table)
  • Workbench shell: 240pt translucent navigator (sections + counts, footer: settings gear, active-job spinner, live RAM readout), 52pt contextual header, 1360×880 / min 1080×700
  • Screens: Models (Installed rows w/ type badge + quant + size + RAM verdict + reveal-in-Finder; Discover state designed), Datasets (import JSONL w/ validation flow), Train (run history w/ status pills), Convert / Playground / Evaluate intentional empty states
  • Settings window (native tabs: General, Compute, Python, Storage, Appearance — HF token vault + Shortcuts + Advanced land with their Phase 6 features)
  • Quality gate: light + dark reviewed via screenshots — copper identity reads clearly in both, cards/hairlines/baselines aligned
  • Phase checkpoint: build green, 0 warnings

Phase 4 summary: The token system (ZyquoTheme) is the single source of truth for every color/font/spacing value; shared components (ZyquoCard, StatusPill, TypeBadge, VerdictBadge, EmptyStateView) established. The workbench shell renders the real Phase 3 artifacts (4 models incl. the fused forge model, 1 dataset, 3 runs) with live MemoryAdvisor verdicts. Run configurator, live loss charts, playground interactions, and Hub Discover are Phase 6 features and will be built on this spec.

# Phase 5 — App Icon ✅ (completed 2026-07-30)

  • Two SVG directions rendered and compared: Z-anvil/spark chosen (stronger forge story via glow + anvil); Z-core-node kept in assets for reference
  • Refined: crisp geometric anvil (horn/face/waist/base + hot copper line), Z lifted for breathing room, restrained base glow; reviewed at 512 + zoom
  • Simplified small variant (Z only, thicker rim) for 16/32 px
  • make icon (scripts/make-icon.sh): rsvg-convert → full iconset (16→1024 incl @2x, small variant ≤32) → iconutil → AppIcon.icns; embedded via make app
  • Phase checkpoint: size-ladder review 16→256 sharp; verified in the real Dock — unmistakably the copper Zyquo sibling

# Phase 6 — Features ✅ (completed 2026-07-30)

  • Hub/HubService (live mlx search + tree sizes + resolve, HF token via Keychain HFTokenStore), Hub/DownloadManager (Range-resumable, verified end-to-end: 351 MB repo downloaded with progress → auto-installed), Services/Catalog (16 Featured entries from MODELS.md w/ RAM verdicts)
  • Models › Discover: Featured catalog + debounced live search, download/pause with progress, installed detection
  • Playground: streaming chat verified in-UI (578 tok/s · TTFT 1.47 s shown live), VLM image attach, embeddings inspector w/ vectors + ranked cosine similarities, load/unload with freed-bytes readout
  • Train: configurator sheet (RAM gating + MemoryAdvisor suggestions inline, method/hyperparams with help text) + run detail verified in-UI (copper/slate loss chart from persisted metrics, stat strip, SF Mono console w/ autoscroll, checkpoints, cancel/warm-resume)
  • Convert: Quantize (Swift-native, live size preview) / Fuse (de-quantize smart default + explanation) / Convert-from-HF job cards with progress
  • Evaluate: base-vs-candidate side-by-side compare with per-side tok/s + TTFT (sequential load/unload)
  • Polish: HF token in Keychain (Settings › Hugging Face), Foundry menu with ⌘L/⌘D/⌘R/⌘U/⌘I/⌘E section shortcuts
  • Coherence sweep: headers ✓, naming ✓, no TODO/dead code ✓, zero raw hex in views ✓

Phase 6 summary: All foundry features are wired into the workbench UI on top of the Phase 3 services. Live verification on this Mac: run detail renders the real QLoRA run's curves; Playground streamed a real reply with stats; the Hub pipeline searched, downloaded, and installed a real repo with resumable progress. One heuristic fix surfaced by verification: decoder-style embedding repos (Qwen3-Embedding ships a Qwen3ForCausalLM config) are detected by name — the downloaded embedder then produced correct 1024-dim vectors.

# Phase 7 — Verification ✅ (completed 2026-07-30)

  • Speech pipeline added (mlx-whisper via PyBridge + Playground panel + CLI)
  • Inference matrix green: LLM ×3 (578–630 tok/s), embeddings ×2 (incl. decoder-style), VLM + image (correctly read the icon's Z; 97 tok/s), speech (perfect 1.0 s transcript); memory release verified per type (0.3–2.9 GB freed)
  • Training: QLoRA + LoRA to completion (loss 4.62→0.10 / val 6.31→0.14), live metrics, checkpoints, cancel → cancelled state (fix), warm resume provably loads adapters (resume VAL = 0.136), fuse w/ auto-dequantize, behavior change verified; OOM config blocked with suggestions
  • Convert/quantize: HF→MLX Python path (331 MB, runs at 630 tok/s; snapshot-quirk bypass documented) + Swift-native quantize (size prediction accurate to 0.1 %)
  • Catalog: 16/16 Featured repos live-green
  • Python env: bootstrap, healthy-venv adopt, on-demand extras, JSON protocol
  • docs/VERIFICATION.md green table (image-gen: documented deferral — upstream pipelines are script-only)

Phase 7 summary: Full lifecycle proven end-to-end on this Mac across four model types plus training/convert/hub/python-env. Three real-world defects were found by verification and fixed: cancelled runs mis-recorded as completed, decoder-style embedding repos mis-typed as LLM, and stale sharded indexes in upstream repos rejected by validation. Results in VERIFICATION.md.

# Phase 8 — Signing & Notarization ✅ (completed 2026-07-30)

  • zyquo-term setup inspected: identity Developer ID Application: Simon-Pierre Boucher (3YM54G49SN) + profile MacLustr-Notarize reused; identity verified present in keychain (no secrets printed/committed)
  • Support/entitlements.plist — Hardened Runtime, minimal posture (no sandbox, no allow-jit — MLX's Metal runtime compile is GPU shader compilation, not process JIT)
  • make release: nested-first signing → ditto zip → notarytool submit --wait → staple → verify
  • Signed + notarized + stapled DMG (make dmg)
  • PHASE GATE GREEN on first submission: spctl -a -vvaccepted, source=Notarized Developer ID; stapler validate OK; notarized app launches
  • Final coherence sweep: headers ✓, naming ✓, no TODO/dead code ✓, zero warnings ✓

Phase 8 summary: make release produces a Developer ID-signed, notarized, stapled Zyquo MLX.app (plus DMG) entirely from the command line — no Xcode IDE, no .xcodeproj. Notarization passed on the first attempt with the minimal entitlements posture, confirming BUILD.md §4's analysis. The bundled-Python notarization minefield never arises because the venv lives in Application Support, outside the signed bundle.


# Definition of Done — final review (2026-07-30)

  • make release → Developer ID-signed, notarized, stapled app, no Xcode IDE
  • ✅ Inference across MLX-supported types: LLM/VLM/embeddings/speech verified with streaming + memory release (image-gen: documented deferral — upstream pipelines are script-only; Swift StableDiffusion lib is the follow-up path)
  • ✅ Real LoRA + QLoRA end-to-end: live metrics, checkpoints, cancel, warm resume, fuse, export — with two upstream landmines found and handled
  • ✅ Conversion + quantization produce valid, running MLX models (Swift-native
    • Python paths); export lands in the shared Models library
  • ✅ Datasets import/validate/split/preview; Hub browse + resumable downloads + curated catalog live-verified (16/16 + 43-repo sweep)
  • ✅ MemoryAdvisor: device-derived gating, verdicts on every surface, size prediction measured at 0.1 % accuracy; blocked-config UX verified
  • ✅ Copper-on-slate icon (Z dominant, anvil/spark story), SVG source of truth, Dock-verified; light theme flagship + derived dark, token-only styling
  • ✅ Naming coherent (Zyquo MLX / com.zyquo.mlx / ZyquoMLX); headers on every code file (swept); zero warnings; phase-prefixed commit history
  • ✅ All Phase 0 research docs complete and traceable into the implementation