spb/zyquo-mlx Public MIT

1<!--2  VERIFICATION.md3  Zyquo MLX45  Author: Simon-Pierre Boucher6  Mail: contact@spboucher.ai7-->89# Zyquo MLX — Phase 7 Verification Results1011All tests executed on the dev machine (Apple M5 Max, 48 GB, macOS 27.0)12on 2026-07-30, driven through the app (CLI mode and/or UI — both use the same13services). ✅ = green.1415## 1. Inference matrix1617| Capability | Model | Result | Metrics / notes |18|---|---|---|---|19| Text LLM, streaming | mlx-community/Qwen3-0.6B-4bit | ✅ | 603.7 tok/s, TTFT 0.83 s; stop-reason + unload freed 319.8 MB (verified via `Memory.snapshot` delta) |20| Text LLM (converted in-app) | qwen3-0.6b-converted-4bit | ✅ | 630.2 tok/s, TTFT 0.03 s, coherent output |21| Text LLM in UI (streaming chat) | zyquo-forge-qwen3-0.6b-4bit | ✅ | 578.2 tok/s, TTFT 1.47 s shown live in Playground header |22| Embeddings | mlx-community/all-MiniLM-L6-v2-4bit | ✅ | 384-dim L2-normed; similarity ranks correctly (0.86 related > 0.72 unrelated); 3 texts in 1.70 s |23| Embeddings (decoder-style) | mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ | ✅ | 1024-dim; 2 texts in 0.36 s; required name-based type detection (config says `Qwen3ForCausalLM`) |24| VLM + image | mlx-community/Qwen3-VL-4B-Instruct-4bit | ✅ | Correctly described the app icon ("the letter Z"); 97.2 tok/s, TTFT 1.95 s, unload freed 2.89 GB |25| Speech (STT) | mlx-community/whisper-large-v3-turbo | ✅ | Perfect transcript of a 7 s English clip in 1.0 s (incl. "Zyquo"); language auto-detect verified (fr voice → fr) |26| Image generation | — | ⚠️ deferred | Upstream FLUX/SD Python pipelines are script-only (mlx-examples, no PyPI package); Swift `StableDiffusion` lib available as a follow-up. Documented deferral, not a regression. |2728## 2. Training2930| Test | Result | Notes |31|---|---|---|32| QLoRA to completion (4-bit base) | ✅ | Qwen3-0.6B-4bit, 48-row chat dataset, 120 iters: train loss **4.62 → 0.10**, val **6.31 → 0.14**, ~2,300 tok/s, peak 0.8 GB |33| LoRA (fp16 base) | ✅ | zyquo-forge fp16 base, 400 iters at ~3,300 tok/s, peak 1.6 GB |34| Live metrics | ✅ | JSON-lines callback protocol → charted in UI (verified screenshot); stdout never parsed |35| Checkpoints | ✅ | `0000040/0000080/0000120_adapters.safetensors` on cadence |36| Cancel | ✅ | Mid-run cancel at iter ~435/5000 → state `cancelled`, checkpoints intact (fix landed: cancelled stream no longer reports `completed`) |37| Warm resume | ✅ | Resume of the cancelled run starts at VAL 0.136 (== trained level, not the fresh 6.31) — adapter weights provably loaded |38| Adapter fuse | ✅ | Fused model answers in the trained "⚒ From the forge:" style; **auto-dequantize on quantized bases** (without it, re-quantization rounds small adapters away — verified both ways) |39| OOM gating | ✅ | full FT @ batch 256 blocked pre-run by MemoryAdvisor with actionable suggestions; batch-64 borderline config surfaced a runtime failure with persisted `failed` state |4041## 3. Convert / quantize4243| Test | Result | Notes |44|---|---|---|45| HF → MLX + 4-bit quant (Python path) | ✅ | `Qwen/Qwen3-0.6B` → 331 MB, "4.501 bits per weight", loads + generates at 630 tok/s. Driver pre-fetches the full snapshot (mlx-lm 0.31.3 `IncompleteSnapshotError` quirk documented in the script) |46| Swift-native quantization | ✅ | fp16 fused model 1.19 GB → 335.5 MB actual vs 335.3 MB predicted (0.1 % accuracy) |47| Validation of outputs | ✅ | Both outputs pass `ModelStore` validation and run |4849## 4. Hub & catalog5051| Test | Result | Notes |52|---|---|---|53| Featured catalog live-verified | ✅ | 16/16 repos HTTP 200 on 2026-07-30 (plus the 43-repo sweep in MODELS.md) |54| Live search | ✅ | `HubService.search` (mlx filter, downloads sort) feeding Discover |55| Resumable downloads | ✅ | 3 repos downloaded via `DownloadManager` (0.35–3.1 GB) with byte-level progress; per-file Range resume logic; pause keeps partial bytes |56| Stale-index repos | ✅ | Qwen3-VL ships a consolidated `model.safetensors` + stale sharded index — validator accepts consolidated layout (fix landed) |5758## 5. Python environment5960| Test | Result | Notes |61|---|---|---|62| Bootstrap (uv, Python 3.12, pinned mlx-lm 0.31.3) | ✅ | Provisioned into `~/Library/Application Support/ZyquoMLX/py/venv` |63| Adopt existing healthy venv | ✅ | Marker deleted → next Python use verified imports and re-adopted without reinstall |64| On-demand extras | ✅ | `mlx-whisper==0.4.3` installed automatically at first transcription |65| JSON progress protocol | ✅ | All pipelines (train/fuse/convert/transcribe) speak JSON lines; two upstream landmines bypassed and documented (callback drop in `lora.run()`, incomplete-snapshot error) |6667## Artifacts kept for development6869`Models/`: Qwen3-0.6B-4bit (346 MB), all-MiniLM (14 MB), Qwen3-Embedding-DWQ70(351 MB), whisper-large-v3-turbo (1.6 GB), Qwen3-VL-4B (3.1 GB), forge fp16 +714-bit variants, converted Qwen3. `Datasets/zyquo-forge-style`, `Runs/` with72metrics history. Clean up via Settings › Storage when desired.73