SPB Git

spb/zyquo-mlx Public MIT

The local MLX foundry for your Mac — run, fine-tune, quantize, and ship models. Nothing leaves your machine.

macosswiftaimlxfine-tuning
Swift 93.4% Python 3.8% Makefile 2.2% Shell 0.5%

phase7: verification complete — full matrix green (LLM/VLM/embeddings/speech, training incl cancel+resume, convert/quantize, hub, py-env); stale-index + VLM fixes; docs/VERIFICATION.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simon-pierre boucher committed 11 days ago (Jul 31, 2026) parent c91e2cd

Showing 4 changed files with +101 and −10

modified Sources/ZyquoMLX/Hub/ModelStore.swift +4 −1
@@ -96,7 +96,10 @@ actor ModelStore {
96 96 let referenced = Set(weightMap.values)
97 97 let present = Set(weightFiles.map { ($0 as NSString).lastPathComponent })
98 98 let missing = referenced.subtracting(present).sorted()
99 − if !missing.isEmpty {
99 + // Some mlx-community repos ship a consolidated
100 + // model.safetensors next to a stale sharded index (seen live:
101 + // Qwen3-VL-4B-Instruct-4bit) — treat that as valid.
102 + if !missing.isEmpty && !present.contains("model.safetensors") {
100 103 throw ModelValidationError.inconsistentShards(missing: missing)
101 104 }
102 105 }
modified Sources/ZyquoMLX/PyBridge/scripts/zyquo_convert.py +10 −0
@@ -43,6 +43,16 @@ def main():
43 43
44 44 from mlx_lm import convert
45 45
46 + # mlx-lm 0.31.3 first downloads weights with allow_patterns, then its
47 + # copy step demands the FULL snapshot with local_files_only=True and
48 + # trips IncompleteSnapshotError on aux files (.gitattributes, LICENSE…).
49 + # Pre-fetch the complete snapshot for remote repos to sidestep it.
50 + import os
51 + if not os.path.isdir(args.hf_path):
52 + from huggingface_hub import snapshot_download
53 + emit({"event": "stage", "stage": "downloading"})
54 + snapshot_download(repo_id=args.hf_path)
55 +
46 56 kwargs = {
47 57 "hf_path": args.hf_path,
48 58 "mlx_path": args.mlx_path,
modified docs/PLAN.md +15 −9
@@ -162,13 +162,19 @@ Hub pipeline searched, downloaded, and installed a real repo with resumable
162 162 progress. One heuristic fix surfaced by verification: decoder-style embedding
163 163 repos (Qwen3-Embedding ships a `Qwen3ForCausalLM` config) are detected by
164 164 name — the downloaded embedder then produced correct 1024-dim vectors.
165 −## Phase 7 — Verification (in progress)
166 −
167 −- [ ] Speech pipeline added (mlx-whisper via PyBridge) — closes the model-type matrix gap
168 −- [ ] Inference matrix: LLM (streaming/cancel/memory), embeddings ×2, VLM + image, speech transcription; tok/s + TTFT recorded
169 −- [ ] Training: LoRA + QLoRA to completion, live metrics, checkpoints, warm resume, fuse, behavior change; OOM config blocked by MemoryAdvisor
170 −- [ ] Convert/quantize: HF→MLX (Python path) + Swift-native quantize; outputs load and run; sizes correct
171 −- [ ] Catalog dry-verify: all Featured repo IDs live-checked against the Hub
172 −- [ ] Python env: bootstrap/adopt/repair verified
173 −- [ ] Green results table in docs/VERIFICATION.md (image-gen: documented deferral — upstream FLUX is script-only, no packaged pipeline)
165 +## Phase 7 — Verification ✅ (completed 2026-07-30)
166 +
167 +- [x] Speech pipeline added (mlx-whisper via PyBridge + Playground panel + CLI)
168 +- [x] Inference matrix green: LLM ×3 (578–630 tok/s), embeddings ×2 (incl. decoder-style), VLM + image (correctly read the icon's Z; 97 tok/s), speech (perfect 1.0 s transcript); memory release verified per type (0.3–2.9 GB freed)
169 +- [x] Training: QLoRA + LoRA to completion (loss 4.62→0.10 / val 6.31→0.14), live metrics, checkpoints, **cancel → cancelled state** (fix), **warm resume provably loads adapters** (resume VAL = 0.136), fuse w/ auto-dequantize, behavior change verified; OOM config blocked with suggestions
170 +- [x] Convert/quantize: HF→MLX Python path (331 MB, runs at 630 tok/s; snapshot-quirk bypass documented) + Swift-native quantize (size prediction accurate to 0.1 %)
171 +- [x] Catalog: 16/16 Featured repos live-green
172 +- [x] Python env: bootstrap, healthy-venv adopt, on-demand extras, JSON protocol
173 +- [x] `docs/VERIFICATION.md` green table (image-gen: documented deferral — upstream pipelines are script-only)
174 +
175 +**Phase 7 summary:** Full lifecycle proven end-to-end on this Mac across four
176 +model types plus training/convert/hub/python-env. Three real-world defects
177 +were found *by* verification and fixed: cancelled runs mis-recorded as
178 +completed, decoder-style embedding repos mis-typed as LLM, and stale sharded
179 +indexes in upstream repos rejected by validation. Results in VERIFICATION.md.
174 180 ## Phase 8 — Signing & Notarization (not started)
added docs/VERIFICATION.md +72 −0
@@ -0,0 +1,72 @@
1 +<!--
2 + VERIFICATION.md
3 + Zyquo MLX
4 +
5 + Author: Simon-Pierre Boucher
6 + Mail: contact@spboucher.ai
7 +-->
8 +
9 +# Zyquo MLX — Phase 7 Verification Results
10 +
11 +All tests executed on the dev machine (Apple M5 Max, 48 GB, macOS 27.0)
12 +on 2026-07-30, driven through the app (CLI mode and/or UI — both use the same
13 +services). ✅ = green.
14 +
15 +## 1. Inference matrix
16 +
17 +| Capability | Model | Result | Metrics / notes |
18 +|---|---|---|---|
19 +| Text LLM, streaming | mlx-community/Qwen3-0.6B-4bit | ✅ | 603.7 tok/s, TTFT 0.83 s; stop-reason + unload freed 319.8 MB (verified via `Memory.snapshot` delta) |
20 +| Text LLM (converted in-app) | qwen3-0.6b-converted-4bit | ✅ | 630.2 tok/s, TTFT 0.03 s, coherent output |
21 +| Text LLM in UI (streaming chat) | zyquo-forge-qwen3-0.6b-4bit | ✅ | 578.2 tok/s, TTFT 1.47 s shown live in Playground header |
22 +| Embeddings | mlx-community/all-MiniLM-L6-v2-4bit | ✅ | 384-dim L2-normed; similarity ranks correctly (0.86 related > 0.72 unrelated); 3 texts in 1.70 s |
23 +| Embeddings (decoder-style) | mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ | ✅ | 1024-dim; 2 texts in 0.36 s; required name-based type detection (config says `Qwen3ForCausalLM`) |
24 +| VLM + image | mlx-community/Qwen3-VL-4B-Instruct-4bit | ✅ | Correctly described the app icon ("the letter Z"); 97.2 tok/s, TTFT 1.95 s, unload freed 2.89 GB |
25 +| Speech (STT) | mlx-community/whisper-large-v3-turbo | ✅ | Perfect transcript of a 7 s English clip in 1.0 s (incl. "Zyquo"); language auto-detect verified (fr voice → fr) |
26 +| Image generation | — | ⚠️ deferred | Upstream FLUX/SD Python pipelines are script-only (mlx-examples, no PyPI package); Swift `StableDiffusion` lib available as a follow-up. Documented deferral, not a regression. |
27 +
28 +## 2. Training
29 +
30 +| Test | Result | Notes |
31 +|---|---|---|
32 +| QLoRA to completion (4-bit base) | ✅ | Qwen3-0.6B-4bit, 48-row chat dataset, 120 iters: train loss **4.62 → 0.10**, val **6.31 → 0.14**, ~2,300 tok/s, peak 0.8 GB |
33 +| LoRA (fp16 base) | ✅ | zyquo-forge fp16 base, 400 iters at ~3,300 tok/s, peak 1.6 GB |
34 +| Live metrics | ✅ | JSON-lines callback protocol → charted in UI (verified screenshot); stdout never parsed |
35 +| Checkpoints | ✅ | `0000040/0000080/0000120_adapters.safetensors` on cadence |
36 +| Cancel | ✅ | Mid-run cancel at iter ~435/5000 → state `cancelled`, checkpoints intact (fix landed: cancelled stream no longer reports `completed`) |
37 +| Warm resume | ✅ | Resume of the cancelled run starts at VAL 0.136 (== trained level, not the fresh 6.31) — adapter weights provably loaded |
38 +| Adapter fuse | ✅ | Fused model answers in the trained "⚒ From the forge:" style; **auto-dequantize on quantized bases** (without it, re-quantization rounds small adapters away — verified both ways) |
39 +| OOM gating | ✅ | full FT @ batch 256 blocked pre-run by MemoryAdvisor with actionable suggestions; batch-64 borderline config surfaced a runtime failure with persisted `failed` state |
40 +
41 +## 3. Convert / quantize
42 +
43 +| Test | Result | Notes |
44 +|---|---|---|
45 +| HF → MLX + 4-bit quant (Python path) | ✅ | `Qwen/Qwen3-0.6B` → 331 MB, "4.501 bits per weight", loads + generates at 630 tok/s. Driver pre-fetches the full snapshot (mlx-lm 0.31.3 `IncompleteSnapshotError` quirk documented in the script) |
46 +| Swift-native quantization | ✅ | fp16 fused model 1.19 GB → 335.5 MB actual vs 335.3 MB predicted (0.1 % accuracy) |
47 +| Validation of outputs | ✅ | Both outputs pass `ModelStore` validation and run |
48 +
49 +## 4. Hub & catalog
50 +
51 +| Test | Result | Notes |
52 +|---|---|---|
53 +| Featured catalog live-verified | ✅ | 16/16 repos HTTP 200 on 2026-07-30 (plus the 43-repo sweep in MODELS.md) |
54 +| Live search | ✅ | `HubService.search` (mlx filter, downloads sort) feeding Discover |
55 +| Resumable downloads | ✅ | 3 repos downloaded via `DownloadManager` (0.35–3.1 GB) with byte-level progress; per-file Range resume logic; pause keeps partial bytes |
56 +| Stale-index repos | ✅ | Qwen3-VL ships a consolidated `model.safetensors` + stale sharded index — validator accepts consolidated layout (fix landed) |
57 +
58 +## 5. Python environment
59 +
60 +| Test | Result | Notes |
61 +|---|---|---|
62 +| Bootstrap (uv, Python 3.12, pinned mlx-lm 0.31.3) | ✅ | Provisioned into `~/Library/Application Support/ZyquoMLX/py/venv` |
63 +| Adopt existing healthy venv | ✅ | Marker deleted → next Python use verified imports and re-adopted without reinstall |
64 +| On-demand extras | ✅ | `mlx-whisper==0.4.3` installed automatically at first transcription |
65 +| JSON progress protocol | ✅ | All pipelines (train/fuse/convert/transcribe) speak JSON lines; two upstream landmines bypassed and documented (callback drop in `lora.run()`, incomplete-snapshot error) |
66 +
67 +## Artifacts kept for development
68 +
69 +`Models/`: Qwen3-0.6B-4bit (346 MB), all-MiniLM (14 MB), Qwen3-Embedding-DWQ
70 +(351 MB), whisper-large-v3-turbo (1.6 GB), Qwen3-VL-4B (3.1 GB), forge fp16 +
71 +4-bit variants, converted Qwen3. `Datasets/zyquo-forge-style`, `Runs/` with
72 +metrics history. Clean up via Settings › Storage when desired.
73