# Zyquo MLX — Phase 7 Verification Results All tests executed on the dev machine (Apple M5 Max, 48 GB, macOS 27.0) on 2026-07-30, driven through the app (CLI mode and/or UI — both use the same services). ✅ = green. ## 1. Inference matrix | Capability | Model | Result | Metrics / notes | |---|---|---|---| | Text LLM, streaming | mlx-community/Qwen3-0.6B-4bit | ✅ | 603.7 tok/s, TTFT 0.83 s; stop-reason + unload freed 319.8 MB (verified via `Memory.snapshot` delta) | | Text LLM (converted in-app) | qwen3-0.6b-converted-4bit | ✅ | 630.2 tok/s, TTFT 0.03 s, coherent output | | Text LLM in UI (streaming chat) | zyquo-forge-qwen3-0.6b-4bit | ✅ | 578.2 tok/s, TTFT 1.47 s shown live in Playground header | | Embeddings | mlx-community/all-MiniLM-L6-v2-4bit | ✅ | 384-dim L2-normed; similarity ranks correctly (0.86 related > 0.72 unrelated); 3 texts in 1.70 s | | Embeddings (decoder-style) | mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ | ✅ | 1024-dim; 2 texts in 0.36 s; required name-based type detection (config says `Qwen3ForCausalLM`) | | VLM + image | mlx-community/Qwen3-VL-4B-Instruct-4bit | ✅ | Correctly described the app icon ("the letter Z"); 97.2 tok/s, TTFT 1.95 s, unload freed 2.89 GB | | Speech (STT) | mlx-community/whisper-large-v3-turbo | ✅ | Perfect transcript of a 7 s English clip in 1.0 s (incl. "Zyquo"); language auto-detect verified (fr voice → fr) | | Image generation | — | ⚠️ deferred | Upstream FLUX/SD Python pipelines are script-only (mlx-examples, no PyPI package); Swift `StableDiffusion` lib available as a follow-up. Documented deferral, not a regression. | ## 2. Training | Test | Result | Notes | |---|---|---| | QLoRA to completion (4-bit base) | ✅ | Qwen3-0.6B-4bit, 48-row chat dataset, 120 iters: train loss **4.62 → 0.10**, val **6.31 → 0.14**, ~2,300 tok/s, peak 0.8 GB | | LoRA (fp16 base) | ✅ | zyquo-forge fp16 base, 400 iters at ~3,300 tok/s, peak 1.6 GB | | Live metrics | ✅ | JSON-lines callback protocol → charted in UI (verified screenshot); stdout never parsed | | Checkpoints | ✅ | `0000040/0000080/0000120_adapters.safetensors` on cadence | | Cancel | ✅ | Mid-run cancel at iter ~435/5000 → state `cancelled`, checkpoints intact (fix landed: cancelled stream no longer reports `completed`) | | Warm resume | ✅ | Resume of the cancelled run starts at VAL 0.136 (== trained level, not the fresh 6.31) — adapter weights provably loaded | | Adapter fuse | ✅ | Fused model answers in the trained "⚒ From the forge:" style; **auto-dequantize on quantized bases** (without it, re-quantization rounds small adapters away — verified both ways) | | OOM gating | ✅ | full FT @ batch 256 blocked pre-run by MemoryAdvisor with actionable suggestions; batch-64 borderline config surfaced a runtime failure with persisted `failed` state | ## 3. Convert / quantize | Test | Result | Notes | |---|---|---| | HF → MLX + 4-bit quant (Python path) | ✅ | `Qwen/Qwen3-0.6B` → 331 MB, "4.501 bits per weight", loads + generates at 630 tok/s. Driver pre-fetches the full snapshot (mlx-lm 0.31.3 `IncompleteSnapshotError` quirk documented in the script) | | Swift-native quantization | ✅ | fp16 fused model 1.19 GB → 335.5 MB actual vs 335.3 MB predicted (0.1 % accuracy) | | Validation of outputs | ✅ | Both outputs pass `ModelStore` validation and run | ## 4. Hub & catalog | Test | Result | Notes | |---|---|---| | Featured catalog live-verified | ✅ | 16/16 repos HTTP 200 on 2026-07-30 (plus the 43-repo sweep in MODELS.md) | | Live search | ✅ | `HubService.search` (mlx filter, downloads sort) feeding Discover | | Resumable downloads | ✅ | 3 repos downloaded via `DownloadManager` (0.35–3.1 GB) with byte-level progress; per-file Range resume logic; pause keeps partial bytes | | Stale-index repos | ✅ | Qwen3-VL ships a consolidated `model.safetensors` + stale sharded index — validator accepts consolidated layout (fix landed) | ## 5. Python environment | Test | Result | Notes | |---|---|---| | Bootstrap (uv, Python 3.12, pinned mlx-lm 0.31.3) | ✅ | Provisioned into `~/Library/Application Support/ZyquoMLX/py/venv` | | Adopt existing healthy venv | ✅ | Marker deleted → next Python use verified imports and re-adopted without reinstall | | On-demand extras | ✅ | `mlx-whisper==0.4.3` installed automatically at first transcription | | JSON progress protocol | ✅ | All pipelines (train/fuse/convert/transcribe) speak JSON lines; two upstream landmines bypassed and documented (callback drop in `lora.run()`, incomplete-snapshot error) | ## Artifacts kept for development `Models/`: Qwen3-0.6B-4bit (346 MB), all-MiniLM (14 MB), Qwen3-Embedding-DWQ (351 MB), whisper-large-v3-turbo (1.6 GB), Qwen3-VL-4B (3.1 GB), forge fp16 + 4-bit variants, converted Qwen3. `Datasets/zyquo-forge-style`, `Runs/` with metrics history. Clean up via Settings › Storage when desired.