Zyquo MLX — Execution Plan
Strict phase order 0 → 8. One phase at a time. Each phase ends with a checkpoint (build, run, zero warnings, 3–5 line summary here) before the next begins.
Phase 0 — Mandatory Intensive Web Research ✅ (completed 2026-07-30)
- 0.A
docs/MLX-RESEARCH.md— MLX framework deep dive- Core primitives: arrays, lazy eval /
eval(), unified memory, streams/devices, autodiff - Swift vs. Python honest capability split (current, verified against live repos)
- Model types & current MLX model zoo (LLM / VLM / embeddings / speech / image-gen)
- Inference API specifics (Swift-first: loading, tokenizers, streaming, params, KV cache, memory release)
- Quantization & conversion (tools, exact commands, on-disk format)
- Build & execution strategy resolved (Metal toolchain question TESTED locally)
- Memory & performance (RAM estimation for inference vs. training)
- Core primitives: arrays, lazy eval /
- 0.B
docs/TRAINING-RESEARCH.md— fine-tuning on MLX- LoRA / QLoRA exact workflow, hyperparameters, adapter save/fuse
- Full fine-tuning feasibility & memory cost
- Dataset formats (chat / prompt-completion / text), split, templating, validation
- Training observability: metrics, checkpoint cadence, resume
- Evaluation: held-out loss/perplexity, base vs. tuned comparison
- 0.C
docs/MODELS.md— Hub integration + curated catalog- Hugging Face Hub HTTP API (search, info, tree, resolve, LFS, token)
- Featured catalog across types/sizes, live-verified
mlx-communityrepo IDs (43 repos, 0 failures) - RAM table (8–128 GB) for inference AND LoRA fine-tuning
-
docs/BUILD.md— no-Xcode-IDE build recipe incl. Metal (tested on this Mac) - Phase checkpoint: docs complete, traceable, committed
Phase 0 summary: Verified the mid-2026 MLX ecosystem live (mlx 0.32.0,
mlx-swift 0.31.6, mlx-swift-lm 3.31.4, mlx-lm 0.31.3): LM libraries moved to
ml-explore/mlx-swift-lm 3.x, which now natively covers LLM/VLM/embeddings
inference, LoRA/QLoRA training, and safetensors conversion/quantization; the
Python bridge is required only for rich-dataset training, full FT, speech,
FLUX, advanced quant, and evaluation. Locally proved CLT-only builds fail at
Metal kernel compilation → strategy: full Xcode as toolchain, CLI-only builds,
mlx-swift_Cmlx.bundle shipped in the app. Catalog: 43 models live-verified.
Local ground truth (recorded 2026-07-30)
- Host: Apple M5 Max, 48 GB RAM, macOS 27.0 (25A5388g toolchain era), arm64
- Swift 6.4 (swiftlang-6.4.0.25.4), Command Line Tools only at
/Library/Developer/CommandLineTools—xcodebuildunavailable, nometalcompiler in CLT - TESTED 2026-07-30 (clean run,
/tmp/zyquo-mlx-buildtest, mlx-swift 0.31.x):swift buildwith CLT only compiles all of Cmlx C++/Swift, then fails atCompileMetalFile steel_attention.metalwitherror: unable to spawn process 'metal' (No such file or directory). → CLT-only build is IMPOSSIBLE; a real Metal toolchain (full Xcode, CLI-only usage) is required. Per charter 0.A.6 the rule adapts to: command-lineswift build/xcodebuildonly, no Xcode IDE, no hand-authored.xcodeproj. - No Xcode.app present anywhere on this machine yet → Phase 1 must provision the
Xcode toolchain (download strategy documented in
docs/BUILD.md) - Python 3.14.4 (Homebrew) +
uvavailable;rsvg-convert+iconutilpresent for the Phase 5 icon pipeline
Phase 1 — Project Setup ✅ (completed 2026-07-30)
- Xcode 26.6 toolchain provisioned (CLI-only; Metal toolchain 17F109 installed)
-
Package.swift— targetZyquoMLX, deps per MLX-RESEARCH.md §8 (resolve exact: mlx-swift 0.31.6, mlx-swift-lm 3.31.4, swift-transformers 1.3.3, swift-huggingface 0.9.0) -
Makefile— build / app / dev / run / icon / clean (+ release stub for Phase 8) -
Info.plist— Zyquo MLX, com.zyquo.mlx, macOS 14+, arm64, developer-tools -
@mainSwiftUI entry point + Apple-Silicon gate + terminal activation - Python bootstrap plan wired (first-run venv per BUILD.md §3.4 — code lands Phase 3)
-
README.md(build instructions) - Phase checkpoint: build green (0 warnings),
Zyquo MLX.appassembled, launched, window verified by screenshot
Phase 1 summary: Xcode 26.6 installed as toolchain-only; resolved the build
definitively — classic SwiftPM silently skips Metal (broken GPU path), but
swift build --build-system swiftbuild -c release compiles all kernels and
emits mlx-swift_Cmlx.bundle/default.metallib (~90 s clean, 0 warnings), so
no xcodebuild is needed. make app assembles an ad-hoc-signed Zyquo MLX.app
with all resource bundles; app launches with the Apple-Silicon gate in place.
BUILD.md §3.1 updated with the resolved recipe.
Phase 2 — Architecture + Inference POC ✅ (completed 2026-07-30)
- Folder architecture per charter (App/ Models/ Engine/ Training/ Hub/ Services/ established with real code; Convert/ Data/ PyBridge/ arrive with Phase 3, DesignSystem/ ViewModels/ Views/ with Phase 4 — no dead placeholder files)
-
Models/domain types:LocalModel,ModelType,Dataset,TrainingRun,Checkpoint,Job(+HyperParamsmirroring mlx-lm defaults) -
Engine/:InferenceEngineactor +ModelTypeAdapters(LLM/VLM/Embeddings via LLMModelFactory/VLMModelFactory/EmbedderModelFactory) +GenerationParams+MemoryAdvisor(device-derived working-set gating) -
Hub/ModelStore: local library scan/validate (config.json, shard-index consistency, quantization metadata, param-count derivation) - CLI POC mode (
ZyquoMLX --infer <dir>/--embed <dir>) before UI - PHASE GATE: LLM (Qwen3-0.6B-4bit) streamed at 603.7 tok/s, TTFT 0.83 s, unload freed 319.8 MB (verified); embeddings (all-MiniLM-L6-v2-4bit) returned 384-dim L2-normed vectors with correct similarity ordering (0.86 related vs 0.72 unrelated)
- Phase checkpoint: build green, 0 warnings, arm64-only
Phase 2 summary: Domain models, InferenceEngine actor with typed
adapters, ModelStore validation, and MemoryAdvisor are in place; every
dependency API was verified against the checked-out mlx-swift-lm 3.31.4
sources (no guessed names — #huggingFaceTokenizerLoader macro requires
import Tokenizers at the expansion site; non-Sendable UserInput goes
through perform(nonSendable:)). CLI POC proves the two-model-type gate with
real downloads in the app's Models library. Build is arm64-only via
--arch arm64 (swiftbuild otherwise builds universal and x86_64 fails).
Phase 3 — Training, Quantization & Conversion ✅ (completed 2026-07-30)
-
PyBridge/PythonRunner— Process wrapper over the venv, JSON-lines progress protocol, cancellation -
PyBridge/PythonEnvironment— uv-provisioned pinned venv in App Support (idempotent adopt/verify/repair; mlx-lm 0.31.3 now, vlm/whisper/audio on demand) -
PyBridge/scripts/— zyquo_train.py (TrainingCallback → JSON; bypasseslora.run()which drops the callback in 0.31.3), zyquo_fuse.py, zyquo_convert.py -
Data/DatasetService+DatasetFormats— import/validate JSONL (chat/completions/text), deterministic split, malformed-row report with fixes, token stats, preview -
Training/TrainingService+RunStore+MetricsStream— cancellable runs, persisted state + metrics.jsonl, numbered checkpoints, warm-start resume, MemoryAdvisor gating -
Convert/ConversionService+QuantConfig— Swift-native affine/mxfp4 quant (size preview accurate to 0.1%); Python bridge for fuse/convert - CLI:
--train,--fuse,--quantize,--validate-dataset(UI in Phase 6) - PHASE GATE: QLoRA on Qwen3-0.6B-4bit, 48-row chat dataset — live loss 4.62→0.10 (val 6.31→0.14) at ~2,300 tok/s, checkpoints at 40/80/120, adapter fused (auto-dequantize), fused model answers in the trained "⚒ From the forge:" style
- Phase checkpoint: build green, 0 warnings
Phase 3 summary: Full foundry core working end-to-end from the app. Two
upstream landmines found and handled: (1) mlx-lm 0.31.3's lora.run()
silently discards the caller's TrainingCallback — our driver replicates
run()'s flow and calls train_model directly; (2) fusing into a quantized
base re-quantizes and rounds away small LoRA deltas — Zyquo defaults to
dequantize-on-fuse, recommends adapter-attached inference, and warns on
re-quantization (documented in TRAINING-RESEARCH.md §2.2). Swift-native
quantization verified: 1.19 GB fp16 → 335.5 MB 4-bit, predicted 335.3 MB.
Phase 4 — Design System & UI ✅ (completed 2026-07-30)
-
DesignSystem/ZyquoTheme— copper-on-slate tokens (light flagship + derived dark), typography (SF Mono for metrics), 4pt spacing grid, radii, 0.5pt hairlines, motion; zero raw hex in views (hex lives only inside the token table) - Workbench shell: 240pt translucent navigator (sections + counts, footer: settings gear, active-job spinner, live RAM readout), 52pt contextual header, 1360×880 / min 1080×700
- Screens: Models (Installed rows w/ type badge + quant + size + RAM verdict + reveal-in-Finder; Discover state designed), Datasets (import JSONL w/ validation flow), Train (run history w/ status pills), Convert / Playground / Evaluate intentional empty states
- Settings window (native tabs: General, Compute, Python, Storage, Appearance — HF token vault + Shortcuts + Advanced land with their Phase 6 features)
- Quality gate: light + dark reviewed via screenshots — copper identity reads clearly in both, cards/hairlines/baselines aligned
- Phase checkpoint: build green, 0 warnings
Phase 4 summary: The token system (ZyquoTheme) is the single source of
truth for every color/font/spacing value; shared components (ZyquoCard,
StatusPill, TypeBadge, VerdictBadge, EmptyStateView) established. The
workbench shell renders the real Phase 3 artifacts (4 models incl. the fused
forge model, 1 dataset, 3 runs) with live MemoryAdvisor verdicts. Run
configurator, live loss charts, playground interactions, and Hub Discover are
Phase 6 features and will be built on this spec.
Phase 5 — App Icon ✅ (completed 2026-07-30)
- Two SVG directions rendered and compared: Z-anvil/spark chosen (stronger forge story via glow + anvil); Z-core-node kept in assets for reference
- Refined: crisp geometric anvil (horn/face/waist/base + hot copper line), Z lifted for breathing room, restrained base glow; reviewed at 512 + zoom
- Simplified small variant (Z only, thicker rim) for 16/32 px
-
make icon(scripts/make-icon.sh): rsvg-convert → full iconset (16→1024 incl @2x, small variant ≤32) →iconutil→ AppIcon.icns; embedded viamake app - Phase checkpoint: size-ladder review 16→256 sharp; verified in the real Dock — unmistakably the copper Zyquo sibling
Phase 6 — Features ✅ (completed 2026-07-30)
-
Hub/HubService(live mlx search + tree sizes + resolve, HF token via KeychainHFTokenStore),Hub/DownloadManager(Range-resumable, verified end-to-end: 351 MB repo downloaded with progress → auto-installed),Services/Catalog(16 Featured entries from MODELS.md w/ RAM verdicts) - Models › Discover: Featured catalog + debounced live search, download/pause with progress, installed detection
- Playground: streaming chat verified in-UI (578 tok/s · TTFT 1.47 s shown live), VLM image attach, embeddings inspector w/ vectors + ranked cosine similarities, load/unload with freed-bytes readout
- Train: configurator sheet (RAM gating + MemoryAdvisor suggestions inline, method/hyperparams with help text) + run detail verified in-UI (copper/slate loss chart from persisted metrics, stat strip, SF Mono console w/ autoscroll, checkpoints, cancel/warm-resume)
- Convert: Quantize (Swift-native, live size preview) / Fuse (de-quantize smart default + explanation) / Convert-from-HF job cards with progress
- Evaluate: base-vs-candidate side-by-side compare with per-side tok/s + TTFT (sequential load/unload)
- Polish: HF token in Keychain (Settings › Hugging Face), Foundry menu with ⌘L/⌘D/⌘R/⌘U/⌘I/⌘E section shortcuts
- Coherence sweep: headers ✓, naming ✓, no TODO/dead code ✓, zero raw hex in views ✓
Phase 6 summary: All foundry features are wired into the workbench UI on
top of the Phase 3 services. Live verification on this Mac: run detail renders
the real QLoRA run's curves; Playground streamed a real reply with stats; the
Hub pipeline searched, downloaded, and installed a real repo with resumable
progress. One heuristic fix surfaced by verification: decoder-style embedding
repos (Qwen3-Embedding ships a Qwen3ForCausalLM config) are detected by
name — the downloaded embedder then produced correct 1024-dim vectors.
Phase 7 — Verification ✅ (completed 2026-07-30)
- Speech pipeline added (mlx-whisper via PyBridge + Playground panel + CLI)
- Inference matrix green: LLM ×3 (578–630 tok/s), embeddings ×2 (incl. decoder-style), VLM + image (correctly read the icon's Z; 97 tok/s), speech (perfect 1.0 s transcript); memory release verified per type (0.3–2.9 GB freed)
- Training: QLoRA + LoRA to completion (loss 4.62→0.10 / val 6.31→0.14), live metrics, checkpoints, cancel → cancelled state (fix), warm resume provably loads adapters (resume VAL = 0.136), fuse w/ auto-dequantize, behavior change verified; OOM config blocked with suggestions
- Convert/quantize: HF→MLX Python path (331 MB, runs at 630 tok/s; snapshot-quirk bypass documented) + Swift-native quantize (size prediction accurate to 0.1 %)
- Catalog: 16/16 Featured repos live-green
- Python env: bootstrap, healthy-venv adopt, on-demand extras, JSON protocol
-
docs/VERIFICATION.mdgreen table (image-gen: documented deferral — upstream pipelines are script-only)
Phase 7 summary: Full lifecycle proven end-to-end on this Mac across four model types plus training/convert/hub/python-env. Three real-world defects were found by verification and fixed: cancelled runs mis-recorded as completed, decoder-style embedding repos mis-typed as LLM, and stale sharded indexes in upstream repos rejected by validation. Results in VERIFICATION.md.
Phase 8 — Signing & Notarization ✅ (completed 2026-07-30)
- zyquo-term setup inspected: identity
Developer ID Application: Simon-Pierre Boucher (3YM54G49SN)+ profileMacLustr-Notarizereused; identity verified present in keychain (no secrets printed/committed) -
Support/entitlements.plist— Hardened Runtime, minimal posture (no sandbox, no allow-jit — MLX's Metal runtime compile is GPU shader compilation, not process JIT) -
make release: nested-first signing →dittozip →notarytool submit --wait→ staple → verify - Signed + notarized + stapled DMG (
make dmg) - PHASE GATE GREEN on first submission:
spctl -a -vv→accepted, source=Notarized Developer ID;stapler validateOK; notarized app launches - Final coherence sweep: headers ✓, naming ✓, no TODO/dead code ✓, zero warnings ✓
Phase 8 summary: make release produces a Developer ID-signed, notarized,
stapled Zyquo MLX.app (plus DMG) entirely from the command line — no Xcode
IDE, no .xcodeproj. Notarization passed on the first attempt with the
minimal entitlements posture, confirming BUILD.md §4's analysis. The
bundled-Python notarization minefield never arises because the venv lives in
Application Support, outside the signed bundle.
Definition of Done — final review (2026-07-30)
- ✅
make release→ Developer ID-signed, notarized, stapled app, no Xcode IDE - ✅ Inference across MLX-supported types: LLM/VLM/embeddings/speech verified with streaming + memory release (image-gen: documented deferral — upstream pipelines are script-only; Swift StableDiffusion lib is the follow-up path)
- ✅ Real LoRA + QLoRA end-to-end: live metrics, checkpoints, cancel, warm resume, fuse, export — with two upstream landmines found and handled
- ✅ Conversion + quantization produce valid, running MLX models (Swift-native
- Python paths); export lands in the shared Models library
- ✅ Datasets import/validate/split/preview; Hub browse + resumable downloads + curated catalog live-verified (16/16 + 43-repo sweep)
- ✅ MemoryAdvisor: device-derived gating, verdicts on every surface, size prediction measured at 0.1 % accuracy; blocked-config UX verified
- ✅ Copper-on-slate icon (Z dominant, anvil/spark story), SVG source of truth, Dock-verified; light theme flagship + derived dark, token-only styling
- ✅ Naming coherent (
Zyquo MLX/com.zyquo.mlx/ZyquoMLX); headers on every code file (swept); zero warnings; phase-prefixed commit history - ✅ All Phase 0 research docs complete and traceable into the implementation