spb/zyquo-local Public MIT
Native macOS AI chat that runs LLMs 100% locally on Apple Silicon with MLX — no cloud, no API keys.
Swift 97.2%
Shell 1.8%
Makefile 1%
1<div align="center">23<img src="assets/icon/app-icon.png" width="180" alt="Zyquo Local icon">45# Zyquo Local67### Every token stays on your Mac. 🍏⚡89**A legendary native macOS AI chat client that runs large language models 100 % locally on Apple Silicon with [MLX](https://github.com/ml-explore/mlx-swift) — no API keys, no cloud, no data ever leaving your machine.**1011<br>1213[](https://github.com/spboucher-ai/zyquo-local/releases/latest)14[](https://github.com/spboucher-ai/zyquo-local/releases/latest)15[](https://support.apple.com/en-us/116943)16[](https://swift.org)17[](https://github.com/ml-explore/mlx-swift)18<br>19[](LICENSE)20[](https://github.com/spboucher-ai/zyquo-local/releases)21[](https://github.com/spboucher-ai/zyquo-local/releases/latest)2223<br>2425**[⬇️ Download the latest DMG](https://github.com/spboucher-ai/zyquo-local/releases/latest/download/ZyquoLocal.dmg)** · Developer ID signed · Notarized by Apple · Stapled2627</div>2829---3031## 📑 Table of Contents3233- [Why Zyquo Local?](#-why-zyquo-local)34- [Features](#-features)35 - [Private, local inference](#-private-local-inference)36 - [Browse & download models inside the app](#-browse--download-models-inside-the-app)37 - [A chat experience that respects your hardware](#-a-chat-experience-that-respects-your-hardware)38 - [Power tools](#-power-tools)39 - [macOS-native polish](#-macos-native-polish)40- [Screenshots](#-screenshots)41- [Verified performance](#-verified-performance)42- [Installation](#-installation)43- [Requirements](#-requirements)44- [Usage](#-usage)45- [Building from source](#%EF%B8%8F-building-from-source)46- [Architecture](#%EF%B8%8F-architecture)47- [Roadmap](#-roadmap)48- [Contributing](#-contributing)49- [License](#-license)50- [Author](#-author)5152---5354## 🔒 Why Zyquo Local?5556Cloud chatbots read your prompts. **Zyquo Local doesn't have a server to send them to.**5758Inference runs entirely on your Mac's unified memory and GPU through Apple's MLX framework. The only network calls the app ever makes are the ones **you** trigger to download model weights from Hugging Face — and even those are resumable, verified, and stored in a folder you control. Everything else — every conversation, every token, every system prompt — lives in `~/Library/Application Support/ZyquoLocal/` and nowhere else.5960---6162## ✨ Features6364### 🔐 Private, local inference6566- **100 % on-device generation** via [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm) — no API keys, no accounts, no telemetry, works fully offline once a model is downloaded.67- **57 supported architectures** out of the box: Llama, Qwen 2/3/3.5/3.6, Mistral/Mixtral, Gemma 1–4, Phi-3/4, DeepSeek-V3 distills, gpt-oss, GLM-4, SmolLM3, LFM2, OpenELM, StarCoder2, Granite, and more — with a compatibility check *before* you download anything.68- **True token streaming** with a live tokens/second ticker, cancellable mid-generation — the stop button actually stops the GPU loop.69- **Reasoning display**: `<think>…</think>` output from thinking models (DeepSeek-R1 distills, Qwen3 thinking mode, QwQ-class) streams into a collapsible *Thought process* section, parsed incrementally and robust to tags split across token chunks.70- **First-class statistics** under every response: `⚡ tokens/sec · token count · time-to-first-token`, plus peak-memory tracking per generation.71- **Multi-turn context management**: KV-cache reuse across turns (no re-prefill), automatic oldest-turn truncation when the context window fills (the system prompt always survives), and a live context-usage bar.72- **Verifiable memory hygiene**: one model loaded at a time, explicit Load/Unload, and unloading provably returns memory to the OS (gigabytes → kilobytes, measured via MLX memory APIs).7374### 🧲 Browse & download models inside the app7576- **Live Hugging Face search** without leaving the app: scopes for the curated *Featured* catalog, the `mlx-community` organization, or all MLX-tagged repos; sort by downloads/likes/newest; filter by size class; gated-repo and unsupported-architecture warnings right on the card.77- **A hand-picked Featured catalog of 30 models** — tiny (Qwen3-0.6B, Llama-3.2-1B, SmolLM3-3B…), mid (Qwen3-8B/14B, Gemma-3-12B, Phi-4, gpt-oss-20b…), large (Mistral-Small-24B, Gemma-3-27B, Qwen3-30B-A3B MoE, Llama-3.3-70B…), coding (Qwen2.5-Coder, Qwen3-Coder-30B-A3B) and reasoning (R1 distills, Qwen3-Thinking) — every repo ID and download size verified against the live Hub.78- **Industrial-grade download manager**: pause / resume / cancel per model, **HTTP Range resume that survives app restarts**, automatic retry with backoff on transient network drops, 2 parallel file streams, live speed + ETA, atomic `.partial → final` completion, per-file size verification, and a disk-space pre-check before a single byte is fetched.79- **RAM verdicts everywhere**: a `MemoryAdvisor` reads *your* Mac's physical memory and stamps every model — installed or discoverable — with **Fits / Tight / Too large**, so you never download something your machine can't run.80- **Model library**: sizes on disk, architecture and quantization badges, context-window length, last-used dates, per-model default generation parameters, Reveal in Finder, and delete-with-reclaimed-space confirmation.81- **Optional Hugging Face token** (Settings → Models & Storage, masked input) for gated repos like Llama and Gemma — sent exclusively to huggingface.co, stripped from cross-host CDN redirects.8283### 💬 A chat experience that respects your hardware8485- **Multi-conversation sidebar** with full-text search across titles *and* message content, pinning, renaming, date grouping (Pinned / Today / Yesterday / Previous 7 Days / Older), and per-conversation model badges.86- **Per-conversation everything**: model, system prompt, temperature, top-p, repetition penalty, max tokens, and seed — with plain-language explanations and app-wide defaults in Settings.87- **Model switcher in the chat header**: click the model chip to swap among downloaded models (RAM verdicts included) with inline load progress; each conversation remembers its model.88- **Message actions**: copy, edit & resend (rewinds the conversation), regenerate, quote-reply, delete.89- **Full Markdown rendering** (swift-markdown): headings, lists, quotes, tables, inline styles — and **syntax-highlighted code blocks with one-click copy**.90- **Attachments**: drag & drop or attach text files (`txt`, `md`, code, `csv`, `json`…) — contents are injected into your message as labeled fenced blocks.91- **Auto-titled conversations** using the loaded model itself after the first exchange.92- **Onboarding that doesn't waste your time**: first launch offers starter models *sized for your specific Mac* with one-click download and inline progress.9394### 🛠 Power tools9596- **⚡ Quick Chat** (global `⌥Space`): a Spotlight-style floating panel for one-shot questions to the loaded model, from anywhere in macOS — no accessibility permissions needed.97- **🆚 Compare mode**: run **two models side-by-side** on the same prompt with independent streaming and stats — RAM-gated so you can't load a pair your memory can't hold.98- **📚 Prompt library**: 56 built-in templates across 8 categories (writing, coding, analysis, learning, productivity, communication, language, creative) with `{{variable}}` placeholders — plus your own saved templates.99- **🎭 Personas**: reusable system prompt + preferred model + parameter bundles (4 thoughtful built-ins included).100- **📤 Export** any conversation to **Markdown or PDF**; bulk export/import of all conversations as JSON.101- **📊 Live resource footer**: currently loaded model, real-time MLX active-memory readout, and a colored pressure dot.102103### 🍎 macOS-native polish104105- **Real Mac app**: SwiftUI, `NavigationSplitView`, native Settings (6 tabs), translucent sidebar, hover states, snappy popovers, 0.5 pt hairlines — designed, not generated.106- **Menu bar extra** (toggleable) with loaded-model status, download progress at a glance, and Quick Chat access.107- **Keyboard-first**: `⌘N` new chat · `⌘K` model switcher · `⌘L` library · `⌘F` search · `⌘↩` send · `⇧⌘E` export · `⌥Space` Quick Chat.108- **Light & dark themes** from a strict design-token system (emerald-on-graphite identity), five accent colors, adjustable chat text size with live preview.109- **Signed & notarized**: Developer ID, hardened runtime, stapled ticket — Gatekeeper installs it without warnings.110111---112113## 📸 Screenshots114115<div align="center">116117**Chat — streaming, thinking disclosure, per-response stats (dark theme)**118119<img src="assets/screenshots/chat-dark.png" width="820" alt="Chat in dark mode with thought process disclosure and generation statistics">120121<br><br>122123**Discover — the curated catalog with RAM verdicts for *your* Mac**124125<img src="assets/screenshots/discover.png" width="820" alt="Discover tab with featured models, sizes and memory verdicts">126127<br><br>128129**Library — installed models with architecture, quantization and context badges**130131<img src="assets/screenshots/library.png" width="820" alt="Installed models with badges and actions">132133</div>134135---136137## 🏁 Verified performance138139Every release is validated by an end-to-end harness (`ZyquoLocal --verify`) that downloads real models through the app's own pipeline, then checks loading, deterministic generation, multi-turn recall, streaming cancellation, and memory release. Measured on an Apple M5 Max (48 GB):140141| Model | Load | Generate | Multi-turn | Cancel | Unload | tok/s | TTFT |142|---|:-:|:-:|:-:|:-:|:-:|--:|--:|143| Qwen3-0.6B-4bit | ✅ | ✅ | ✅ | ✅ | ✅ | 109.4 | 0.08 s |144| Llama-3.2-1B-Instruct-4bit | ✅ | ✅ | ✅ | ✅ | ✅ | 222.8 | 0.74 s |145| SmolLM3-3B-4bit | ✅ | ✅ | ✅ | ✅ | ✅ | 97.4 | 0.11 s |146| gemma-3-4b-it-qat-4bit | ✅ | ✅ | ✅ | ✅ | ✅ | 58.3 | 1.08 s |147| Qwen2.5-Coder-7B-Instruct-4bit | ✅ | ✅ | ✅ | ✅ | ✅ | 112.5 | 0.42 s |148| DeepSeek-R1-0528-Qwen3-8B-4bit | ✅ | ✅ | ✅ | ✅ | ✅ | 61.3 | 0.08 s |149150Full methodology and the 30/30 catalog verification: [`docs/VERIFICATION.md`](docs/VERIFICATION.md).151152---153154## 📦 Installation1551561. **[Download `ZyquoLocal.dmg`](https://github.com/spboucher-ai/zyquo-local/releases/latest/download/ZyquoLocal.dmg)** from the latest release.1572. Open the DMG and drag **Zyquo Local** into **Applications**.1583. Launch it. The app is Developer ID signed and notarized by Apple — no Gatekeeper hoops.1594. Pick a starter model sized for your Mac and click **Download**. That's it — chat away, even offline.160161> 💡 Models are stored in `~/Library/Application Support/ZyquoLocal/Models/` and can be deleted from the in-app Library at any time (the app shows exactly how much space you'll reclaim).162163---164165## 🧰 Requirements166167| | Minimum |168|---|---|169| **Mac** | Apple Silicon (M1 or later) — MLX requires it; Intel Macs are not supported |170| **macOS** | 14.0 (Sonoma) or later |171| **Memory** | 8 GB runs ≤4 B models comfortably · 16 GB → 7–14 B · 32 GB → 24–32 B · 64 GB → 70 B |172| **Disk** | 0.35 GB (smallest model) → 40 GB (Llama-3.3-70B); the app pre-checks free space |173| **Network** | Only for downloading models; inference is fully offline |174175---176177## 🚀 Usage178179| I want to… | Do this |180|---|---|181| Get my first model | Launch → pick a starter card → **Download** → it auto-appears in the model chip |182| Chat | Type and hit `⌘↩`; press the stop button to halt generation instantly |183| See how a model thinks | Reasoning models stream into the collapsible **Thought process** section |184| Switch models | Click the model chip in the header (or `⌘K`) — verdicts tell you what fits |185| Tune generation | The sliders icon in the input bar (per-conversation) or Settings → Inference (defaults) |186| Ask from anywhere | `⌥Space` — Quick Chat floats above whatever you're doing |187| Race two models | Menu **Model → Compare Models…**, pick two, broadcast one prompt |188| Reuse a great prompt | The templates icon in the input bar — 56 built-ins with fillable variables |189| Share a conversation | Header export menu → Markdown or PDF |190| Use gated models (Llama, Gemma) | Settings → Models & Storage → add your Hugging Face token |191192---193194## 🏗️ Building from source195196Zyquo Local is plain **Swift Package Manager** — no `.xcodeproj`, and the Xcode IDE is never required. Two prerequisites beyond the Command Line Tools:1971981. **Apple's Metal Toolchain** on `PATH` — MLX compiles its GPU kernels at build time. On a machine with Xcode 26+: `xcodebuild -downloadComponent MetalToolchain`; a standalone `Metal.xctoolchain` on `PATH` also works.1992. **SDK pin** on macOS 27 CLT setups: `SDKROOT=MacOSX26.5.sdk` (SwiftUI macros are Xcode-only plugins in the 27.x CLT SDKs).200201```sh202git clone https://github.com/spboucher-ai/zyquo-local.git203cd zyquo-local204205make build # debug build206make dev # debug bundle (ad-hoc signed) + launch207make poc MODEL=<model-dir> PROMPT="Hello" # CLI inference proof-of-concept208make icon # regenerate AppIcon.icns from the SVG source209make release # Developer ID signed + notarized + stapled app & DMG210```211212The full recipe, the Metal-toolchain findings, and the research that pinned every dependency version live in [`docs/BUILD.md`](docs/BUILD.md) and [`docs/MLX-RESEARCH.md`](docs/MLX-RESEARCH.md).213214---215216## 🏛️ Architecture217218```219Sources/ZyquoLocal/220├── App/ @main entry, scenes, CLI modes (--poc, --hub-poc, --verify)221├── DesignSystem/ ZyquoTheme — every color, font, spacing and radius token222├── Models/ Conversation, Message, LocalModel, DownloadTask, Persona223├── Engine/ InferenceEngine (actor) · ChatSession · MemoryAdvisor · GenerationParams224├── Hub/ HubService · DownloadManager · FileTransfer · ModelStore225├── Services/ Persistence · ModelCatalog · PromptLibrary · Personas · Export · Settings226├── ViewModels/ AppModel · ChatController (streaming + <think> parser)227└── Views/ Chat · Library · Settings · QuickChat · Compare · Components228```229230- **`InferenceEngine` is a Swift actor** — all inference behavior lives behind it; states flow `unloaded → loading → ready ⇄ generating`, generation is an `AsyncThrowingStream` of token/stats/finish events, and cancellation genuinely stops the loop.231- **Dependencies** (pinned, researched, minimal): `mlx-swift-lm` · `swift-transformers` (tokenization) · `swift-huggingface` · `swift-markdown`. Downloads use a custom `URLSession` transport — delegate-backed, chunked, Range-resumable.232- **7 300+ lines of Swift 6**, zero compiler warnings, every file headed and every screen built from design tokens.233234---235236## 🗺 Roadmap237238- [ ] Multi-model residency (keep 2+ small models warm simultaneously)239- [ ] Prompt-cache persistence across launches (MLX `saveCache`) for instant long-context reload240- [ ] Vision-language models (MLXVLM) — image input for Gemma-3 / Qwen-VL class models241- [ ] Tool calling / MCP bridge for local agents242- [ ] Speculative decoding with paired draft models243- [ ] Sparkle-based in-app updates244- [ ] Localization (French first 🇫🇷)245246---247248## 🤝 Contributing249250Issues and pull requests are welcome!2512521. Fork, then `git clone` your fork.2532. Follow [Building from source](#%EF%B8%8F-building-from-source) — `make build` must pass with **zero warnings**.2543. Match the existing style: design tokens only in views, inference logic only in `Engine/`, mandatory file headers.2554. `ZyquoLocal --verify` should stay green if you touch the engine or Hub layers.2565. Open a PR with a clear description of *what* and *why*.257258---259260## 📄 License261262Released under the **[MIT License](LICENSE)** — © 2026 Simon-Pierre Boucher.263264---265266## 👤 Author267268<div align="center">269270**Simon-Pierre Boucher**271272📫 Contact: [contact@spboucher.ai](mailto:contact@spboucher.ai)273274<br>275276*Zyquo Local is part of the Zyquo family of native macOS AI apps.*277278⭐ If Zyquo Local is useful to you, a star helps more people run their AI locally.279280</div>281