SPB Git

Repository Public

README.md 36.0 KB 413 lines markdown

1<!--2Local Chat3File: README.md4Author: Simon-Pierre Boucher5Contact: contact@spboucher.ai6-->7<div align="center">89<img src="docs/brand/logo.svg" alt="Local Chat logo" width="560">1011<br><br>1213<img src="docs/brand/icon.png" alt="Local Chat icon" width="112" height="112">1415# Local Chat1617**Private local AI for Mac, iPhone and iPad — real models running on Apple silicon with Apple's MLX framework.**<br>18One Xcode workspace, two native apps, one shared Swift package. No web view, no Electron, nothing leaves the device by default.1920**Author: [Simon-Pierre Boucher](https://www.spboucher.ai)**2122![version](https://img.shields.io/badge/version-1.0.0%20(1)-6E7BFF?style=for-the-badge)23![macOS](https://img.shields.io/badge/macOS-15+%20·%20notarized-0A0B0E?style=for-the-badge&logo=apple&logoColor=white)24![iOS · iPadOS](https://img.shields.io/badge/iOS%20·%20iPadOS-18+%20·%20TestFlight-0A0B0E?style=for-the-badge&logo=apple&logoColor=white)25![license](https://img.shields.io/badge/license-All%20rights%20reserved-555555?style=for-the-badge)26![author](https://img.shields.io/badge/author-Simon--Pierre%20Boucher-0A66C2?style=for-the-badge)2728![Swift](https://img.shields.io/badge/Swift-6.0%20(toolchain%206.4)-F05138?style=flat-square&logo=swift&logoColor=white)29![SwiftUI](https://img.shields.io/badge/SwiftUI-native-0A84FF?style=flat-square&logo=swift&logoColor=white)30![Xcode](https://img.shields.io/badge/Xcode-27.2-147EFB?style=flat-square&logo=xcode&logoColor=white)31![MLX Swift LM](https://img.shields.io/badge/MLX%20Swift%20LM-3.33.0-8E4DF2?style=flat-square)32![MLX Swift](https://img.shields.io/badge/MLX%20Swift-0.33-8E4DF2?style=flat-square)33![swift-transformers](https://img.shields.io/badge/swift--transformers-1.4.0-FFD21E?style=flat-square&logo=huggingface&logoColor=white)34![GRDB](https://img.shields.io/badge/GRDB-7.12.0-003B57?style=flat-square&logo=sqlite&logoColor=white)35![SwiftMath](https://img.shields.io/badge/SwiftMath-1.7.3-43D9E3?style=flat-square)36![swift-markdown](https://img.shields.io/badge/swift--markdown-0.7.3-43D9E3?style=flat-square)37![XcodeGen](https://img.shields.io/badge/XcodeGen-project.yml-333333?style=flat-square)3839[![commits](https://git.spboucher.ai/badge/local-chat/commits.svg?style=flat-square)](https://git.spboucher.ai/local-chat/commits)40[![last-commit](https://git.spboucher.ai/badge/local-chat/last-commit.svg?style=flat-square)](https://git.spboucher.ai/local-chat/commits)41[![activity](https://git.spboucher.ai/badge/local-chat/activity.svg?style=flat-square)](https://git.spboucher.ai/local-chat/insights)42[![language](https://git.spboucher.ai/badge/local-chat/language.svg?style=flat-square)](https://git.spboucher.ai/local-chat)43[![languages](https://git.spboucher.ai/badge/local-chat/languages.svg?style=flat-square)](https://git.spboucher.ai/local-chat/insights)44[![size](https://git.spboucher.ai/badge/local-chat/size.svg?style=flat-square)](https://git.spboucher.ai/local-chat)45[![branches](https://git.spboucher.ai/badge/local-chat/branches.svg?style=flat-square)](https://git.spboucher.ai/local-chat/branches)46[![tags](https://git.spboucher.ai/badge/local-chat/tags.svg?style=flat-square)](https://git.spboucher.ai/local-chat/tags)47[![clones](https://git.spboucher.ai/badge/local-chat/clones.svg?style=flat-square)](https://git.spboucher.ai/local-chat)4849![Swift files](https://img.shields.io/badge/Swift%20files-142-6E7BFF?style=flat-square)50![Swift lines](https://img.shields.io/badge/Swift%20lines-26%20034-6E7BFF?style=flat-square)51![shared modules](https://img.shields.io/badge/shared%20modules-13-6E7BFF?style=flat-square)52![app targets](https://img.shields.io/badge/app%20targets-2-6E7BFF?style=flat-square)53![tests](https://img.shields.io/badge/tests-200%20passing-3DD68C?style=flat-square)54![real GPU inference tests](https://img.shields.io/badge/real%20GPU%20inference%20tests-9-3DD68C?style=flat-square)55![MLX architectures](https://img.shields.io/badge/MLX%20architectures-69%20text%20·%2024%20vision%20·%2012%20embedding-8E4DF2?style=flat-square)56![device classes](https://img.shields.io/badge/device%20classes-15-43D9E3?style=flat-square)57![measured on M5 Max](https://img.shields.io/badge/measured%20on%20M5%20Max-95.6%20tok/s%20·%20Qwen3----0.6B----4bit-F5B041?style=flat-square)5859![bundle id](https://img.shields.io/badge/bundle%20id-ai.spboucher.localchat-0A0B0E?style=flat-square)60![signing](https://img.shields.io/badge/signing-Developer%20ID%20·%20sandbox%20·%20hardened%20runtime-0A0B0E?style=flat-square)61![notarization](https://img.shields.io/badge/notarization-accepted%20·%20stapled-3DD68C?style=flat-square)62![App Store Connect](https://img.shields.io/badge/App%20Store%20Connect-6837210574-0A84FF?style=flat-square)63![TestFlight](https://img.shields.io/badge/TestFlight-1.0.0%20(1)%20·%20internal%20group-0A84FF?style=flat-square)64![on-device iPhone test](https://img.shields.io/badge/on--device%20iPhone%20test-pending-F5B041?style=flat-square)6566</div>6768---6970## Table of contents7172- [Overview](#overview) · [Screenshots](#screenshots) · [Features](#features) · [Models by device](#models-by-device)73- [Architecture](#architecture) · [Stack](#stack) · [Repository layout](#repository-layout)74- [Build](#build) · [Test](#test) · [Release](#release) · [Configuration](#configuration)75- [Screens](#screens) · [Local API](#local-api) · [Data model](#data-model) · [Security & privacy](#security--privacy)76- [Gotchas](#gotchas) · [Known limitations](#known-limitations) · [Documentation](#documentation) · [Author](#author) · [License](#license)7778## Overview7980Local Chat downloads open models from Hugging Face and runs them **on the device** with [MLX](https://github.com/ml-explore/mlx), Apple's machine-learning framework for Apple silicon. The same Swift code drives a Mac app and an iPhone / iPad app.8182| | macOS | iOS · iPadOS |83|---|---|---|84| Target | `Apps/LocalChatMac` | `Apps/LocalChatIOS` |85| Minimum OS | macOS 15, Apple silicon | iOS 18 |86| Distribution | Developer ID, App Sandbox, Hardened Runtime, notarized DMG | App Store Connect app `6837210574`, TestFlight internal group |87| Layout | sidebar + detail, side inspector, command palette (⌘K), menu bar item, multiple windows | tabs on iPhone, sidebar on iPad |88| Inference | MLX on the Mac's GPU (verified) | MLX on device (built, **not yet run on a physical device**), or a Mac you trust as a remote node |8990Related repositories on spbgit: [sp-chat](https://git.spboucher.ai/sp-chat) and [sp-chat-ios](https://git.spboucher.ai/sp-chat-ios) are clients of a hosted service; Local Chat is independent and needs no server.9192## Screenshots9394Real captures of the Debug build on a MacBook Pro M5 Max and of the iPhone 17 Pro Simulator.9596| Chat — real MLX answer with LaTeX, code and measured metrics | Models → By device — live list from Hugging Face |97|---|---|98| <img src="docs/screenshots/macos-chat.png" width="460" alt="Chat on macOS"> | <img src="docs/screenshots/macos-models-by-device.png" width="460" alt="Models by device"> |99100| Inference Observatory — measured memory and device facts | iPhone (Simulator) — conversation list |101|---|---|102| <img src="docs/screenshots/macos-observatory.png" width="460" alt="Observatory"> | <img src="docs/screenshots/ios-chats.png" width="200" alt="iPhone"> |103104## Features105106<details open>107<summary><b>💬 Chat</b></summary>108109- Real token streaming, stop at any time, measured tokens/s, time to first token and cached prompt tokens under every answer.110- Conversation **tree**: editing a message or regenerating an answer creates a sibling branch; nothing is overwritten (`‹ 2 / 3 ›` switcher), fork a conversation from any message.111- Folders, tags, favorites, archive, **full-text search** over every message (SQLite FTS5).112- System prompts, prompt templates with `{variables}`, inference presets (Fast, Balanced, Reasoning, Maximum Quality) and every sampling parameter.113- Markdown, syntax-highlighted code, tables and **LaTeX** rendered natively (SwiftMath).114- Attachments: images (vision models), PDF / DOCX / CSV / Markdown / source code extracted on device.115- **Context-window bar** (system, summary, knowledge, conversation, attachments, reserved output) with exact token counts from the loaded tokenizer; configurable compaction that summarizes older turns **without touching the original messages**.116- Model switching inside a conversation; each answer shows which model and backend produced it.117- Dictation (system speech recognizer, on device when the OS supports the language) and read-aloud.118</details>119120<details>121<summary><b>📦 Models</b></summary>122123- **By device**: MLX models proposed for every Apple device class, from the live `mlx-community` listing (see [Models by device](#models-by-device)).124- Hugging Face search (MLX only), repository detail with files, quantization, parameters, license, gated state, estimated memory and the compatibility verdict for this device.125- Resumable multi-file downloads with SHA-256 verification of every LFS file, pause / resume / cancel, measured speed and ETA.126- Local library: load / unload, measured speed on this device, import of a local MLX folder, delete.127- Device profiler (chip, memory, storage, thermal state, low power) and recommendations per task; optional router that picks a model per message (always overridable).128</details>129130<details>131<summary><b>⚖️ Model Arena · 📈 Inference Observatory</b></summary>132133- Arena: 2–4 models on the same prompt, side by side, blind mode with vote, fixed seed for repeatable runs, automatic sequential execution when the models do not fit in memory together, Elo and win-rate leaderboard, export.134- Observatory: live generation, loaded models, process and MLX memory, device facts, last generation, history charts and aggregates (median / p10 / p90), benchmark suite. Only measured values — no GPU utilisation, temperature or wattage tiles, because they are not measured.135</details>136137<details>138<summary><b>📚 Knowledge Studio · ✨ Agent Studio</b></summary>139140- On-device RAG: PDF (with optional OCR), DOCX, CSV, Markdown, TXT, source code → structure-aware chunking → local embeddings → exact vector search (100 000 chunks × 384 dimensions in 8–12 ms on an M5 Max) fused with BM25; numbered sources under answers; real deletion.141- Agents: tool registry with explicit permissions, built-in tools (calculator, date, file read / list / search / write inside granted folders, knowledge search, URL fetch with SSRF guard, shell on macOS), human approval checkpoints, step / tool-call / time limits, run logs, research mode with citation checking.142</details>143144<details>145<summary><b>🌐 Local API · remote nodes · sync · security</b></summary>146147- OpenAI-compatible HTTP server on the Mac (`/v1/models`, `/v1/chat/completions` with streaming, `/v1/embeddings`), bound to 127.0.0.1 by default, API keys stored as hashes, rate limiting, request diagnostics, built-in docs page.148- Remote nodes: an iPhone can route a conversation to a trusted Mac (or any OpenAI-compatible endpoint); the conversation shows that it is remote.149- Optional end-to-end encrypted sync of conversations, templates, preferences and agent configurations through a folder or a Mac relay. Model weights are never synchronized.150- Keychain for secrets, optional Face ID / Touch ID lock, App Sandbox.151</details>152153## Models by device154155Local Chat proposes **MLX models only**, taken live from [huggingface.co/mlx-community](https://huggingface.co/mlx-community). A model is proposed for a device class when MLX Swift implements its architecture and its estimated memory (weights + KV cache for 4,096 tokens + 350 MB) stays under 70 % of what a model may use there: 65 % of RAM on a Mac under 32 GB, 75 % from 32 GB, 55 % on iPhone and iPad.156157Snapshot of 2026-11-20 — regenerate with `python3 Scripts/models-by-device.py`; the app refreshes its own list daily (Models → By device). Sizes are estimates; **nothing in the iPhone and iPad rows was run on a physical device yet.**158159| Device | Memory for a model | Chat | Vision | Chat models that fit |160|---|---|---|---|---|161| **iPhone · 6 GB**<br><sub>iPhone 14 Pro, iPhone 15</sub> | 3.3 GB | [`gemma-2-2b-it-4bit`](https://huggingface.co/mlx-community/gemma-2-2b-it-4bit) · ≈ 2.3 GB<br>[`MiniCPM5-2B-mlx-4Bit`](https://huggingface.co/mlx-community/MiniCPM5-2B-mlx-4Bit) · ≈ 2.2 GB<br>[`Qwen3.5-2B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-2B-OptiQ-4bit) · ≈ 1.7 GB<br>[`LFM3-1.7B-OptiQ-4bit`](https://huggingface.co/mlx-community/LFM3-1.7B-OptiQ-4bit) · ≈ 1.6 GB | [`SmolVLM-Instruct-4bit`](https://huggingface.co/mlx-community/SmolVLM-Instruct-4bit) · ≈ 2.0 GB<br>[`Qwen3.5-2B-4bit`](https://huggingface.co/mlx-community/Qwen3.5-2B-4bit) · ≈ 2.0 GB | 50 |162| **iPhone · 8 GB**<br><sub>iPhone 15 Pro, iPhone 16, iPhone 17</sub> | 4.4 GB | [`granite-4.1-3b-4bit`](https://huggingface.co/mlx-community/granite-4.1-3b-4bit) · ≈ 2.8 GB<br>[`Llama-3.2-3B-Instruct-4bit`](https://huggingface.co/mlx-community/Llama-3.2-3B-Instruct-4bit) · ≈ 2.7 GB<br>[`granite-4.0-h-micro-4bit`](https://huggingface.co/mlx-community/granite-4.0-h-micro-4bit) · ≈ 2.7 GB<br>[`OpenCodeInterpreter-SC2-3B-4bit`](https://huggingface.co/mlx-community/OpenCodeInterpreter-SC2-3B-4bit) · ≈ 2.7 GB | [`SmolVLM-Instruct-4bit`](https://huggingface.co/mlx-community/SmolVLM-Instruct-4bit) · ≈ 2.0 GB<br>[`Qwen3.5-2B-6bit`](https://huggingface.co/mlx-community/Qwen3.5-2B-6bit) · ≈ 2.5 GB | 56 |163| **iPhone · 12 GB**<br><sub>iPhone 17 Pro, iPhone 17 Pro Max, iPhone Air</sub> | 6.6 GB | [`gemma-3n-E2B-it-lm-4bit`](https://huggingface.co/mlx-community/gemma-3n-E2B-it-lm-4bit) · ≈ 3.6 GB<br>[`Qwen3.5-4B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-OptiQ-4bit) · ≈ 3.4 GB<br>[`Qwen3.5-4B-3D-Coder-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-3D-Coder-OptiQ-4bit) · ≈ 3.4 GB<br>[`Qwen3-4B-Instruct-2507-4bit`](https://huggingface.co/mlx-community/Qwen3-4B-Instruct-2507-4bit) · ≈ 3.3 GB | [`gemma-4-e2b-it-4bit`](https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit) · ≈ 4.1 GB<br>[`gemma-3-4b-it-qat-4bit`](https://huggingface.co/mlx-community/gemma-3-4b-it-qat-4bit) · ≈ 4.0 GB | 77 |164| **iPad · 8 GB**<br><sub>iPad Air (M2, M3), iPad mini (A17 Pro), iPad Pro M4 256 / 512 GB</sub> | 4.4 GB | [`granite-4.1-3b-4bit`](https://huggingface.co/mlx-community/granite-4.1-3b-4bit) · ≈ 2.8 GB<br>[`Llama-3.2-3B-Instruct-4bit`](https://huggingface.co/mlx-community/Llama-3.2-3B-Instruct-4bit) · ≈ 2.7 GB<br>[`granite-4.0-h-micro-4bit`](https://huggingface.co/mlx-community/granite-4.0-h-micro-4bit) · ≈ 2.7 GB<br>[`OpenCodeInterpreter-SC2-3B-4bit`](https://huggingface.co/mlx-community/OpenCodeInterpreter-SC2-3B-4bit) · ≈ 2.7 GB | [`SmolVLM-Instruct-4bit`](https://huggingface.co/mlx-community/SmolVLM-Instruct-4bit) · ≈ 2.0 GB<br>[`Qwen3.5-2B-6bit`](https://huggingface.co/mlx-community/Qwen3.5-2B-6bit) · ≈ 2.5 GB | 56 |165| **iPad · 12 GB**<br><sub>iPad Pro M5 256 / 512 GB</sub> | 6.6 GB | [`gemma-3n-E2B-it-lm-4bit`](https://huggingface.co/mlx-community/gemma-3n-E2B-it-lm-4bit) · ≈ 3.6 GB<br>[`Qwen3.5-4B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-OptiQ-4bit) · ≈ 3.4 GB<br>[`Qwen3.5-4B-3D-Coder-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-3D-Coder-OptiQ-4bit) · ≈ 3.4 GB<br>[`Qwen3-4B-Instruct-2507-4bit`](https://huggingface.co/mlx-community/Qwen3-4B-Instruct-2507-4bit) · ≈ 3.3 GB | [`gemma-4-e2b-it-4bit`](https://huggingface.co/mlx-community/gemma-4-e2b-it-4bit) · ≈ 4.1 GB<br>[`gemma-3-4b-it-qat-4bit`](https://huggingface.co/mlx-community/gemma-3-4b-it-qat-4bit) · ≈ 4.0 GB | 77 |166| **iPad · 16 GB**<br><sub>iPad Pro M4 / M5 1 TB and 2 TB</sub> | 8.8 GB | [`Qwen2.5-Coder-7B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen2.5-Coder-7B-Instruct-4bit) · ≈ 5.9 GB<br>[`Qwen2.5-7B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen2.5-7B-Instruct-4bit) · ≈ 5.9 GB<br>[`Qwen2.5-7B-Instruct-Uncensored-4bit`](https://huggingface.co/mlx-community/Qwen2.5-7B-Instruct-Uncensored-4bit) · ≈ 5.9 GB<br>[`Qwen2.5-Coder-7B-4bit`](https://huggingface.co/mlx-community/Qwen2.5-Coder-7B-4bit) · ≈ 5.9 GB | [`Ternary-Bonsai-2-9B-2bit`](https://huggingface.co/mlx-community/Ternary-Bonsai-2-9B-2bit) · ≈ 5.1 GB<br>[`NVIDIA-Nemotron-3.5-Bolt-E4B-OptiQ-4bit`](https://huggingface.co/mlx-community/NVIDIA-Nemotron-3.5-Bolt-E4B-OptiQ-4bit) · ≈ 5.8 GB | 82 |167| **Mac · 8 GB**<br><sub>MacBook Air M1 – M3 base</sub> | 5.2 GB | [`gemma-3n-E2B-it-lm-4bit`](https://huggingface.co/mlx-community/gemma-3n-E2B-it-lm-4bit) · ≈ 3.6 GB<br>[`Qwen3.5-4B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-OptiQ-4bit) · ≈ 3.4 GB<br>[`Qwen3.5-4B-3D-Coder-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-3D-Coder-OptiQ-4bit) · ≈ 3.4 GB<br>[`Qwen3-4B-Instruct-2507-4bit`](https://huggingface.co/mlx-community/Qwen3-4B-Instruct-2507-4bit) · ≈ 3.3 GB | [`Qwen3-VL-4B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen3-VL-4B-Instruct-4bit) · ≈ 3.6 GB<br>[`Qwen2.5-VL-3B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen2.5-VL-3B-Instruct-4bit) · ≈ 3.1 GB | 57 |168| **Mac · 16 GB**<br><sub>MacBook Air, Mac mini, MacBook Pro base</sub> | 10.4 GB | [`gemma-2-9b-it-4bit`](https://huggingface.co/mlx-community/gemma-2-9b-it-4bit) · ≈ 7.1 GB<br>[`Qwen3.5-9B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.5-9B-OptiQ-4bit) · ≈ 6.9 GB<br>[`MiMo-V2.6-Distill-Qwen-9B-OptiQ-4bit`](https://huggingface.co/mlx-community/MiMo-V2.6-Distill-Qwen-9B-OptiQ-4bit) · ≈ 6.9 GB<br>[`Ornith-1.5-9B-OptiQ-4bit`](https://huggingface.co/mlx-community/Ornith-1.5-9B-OptiQ-4bit) · ≈ 6.9 GB | [`Ternary-Bonsai-2-9B-2bit`](https://huggingface.co/mlx-community/Ternary-Bonsai-2-9B-2bit) · ≈ 5.1 GB<br>[`Qwen3.5-9B-4bit`](https://huggingface.co/mlx-community/Qwen3.5-9B-4bit) · ≈ 7.2 GB | 91 |169| **Mac · 24 GB**<br><sub>MacBook Air / Pro, Mac mini</sub> | 15.6 GB | [`gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B`](https://huggingface.co/mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B) · ≈ 10.9 GB<br>[`gemma-4-12b-coder-fable5-composer2.5-4bit`](https://huggingface.co/mlx-community/gemma-4-12b-coder-fable5-composer2.5-4bit) · ≈ 9.1 GB<br>[`gemma-4-12B-it-qat-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-12B-it-qat-OptiQ-4bit) · ≈ 9.1 GB<br>[`gemma-4-12B-it-OptiQ-4bit`](https://huggingface.co/mlx-community/gemma-4-12B-it-OptiQ-4bit) · ≈ 9.1 GB | [`gemma-3-12b-it-qat-4bit`](https://huggingface.co/mlx-community/gemma-3-12b-it-qat-4bit) · ≈ 9.1 GB<br>[`gemma-4-12B-it-4bit`](https://huggingface.co/mlx-community/gemma-4-12B-it-4bit) · ≈ 9.1 GB | 105 |170| **Mac · 32 GB**<br><sub>MacBook Pro, Mac Studio</sub> | 24.0 GB | [`Qwen3-Coder-30B-A3B-Instruct-3bit`](https://huggingface.co/mlx-community/Qwen3-Coder-30B-A3B-Instruct-3bit) · ≈ 16.2 GB<br>[`Mistral-Small-3.2-24B-Instruct-2506-4bit`](https://huggingface.co/mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit) · ≈ 16.1 GB<br>[`Devstral-Small-2-24B-Instruct-2512-OptiQ-4bit`](https://huggingface.co/mlx-community/Devstral-Small-2-24B-Instruct-2512-OptiQ-4bit) · ≈ 16.1 GB<br>[`gpt-oss-20b-OptiQ-4bit`](https://huggingface.co/mlx-community/gpt-oss-20b-OptiQ-4bit) · ≈ 14.7 GB | [`Devstral-Small-2-24B-Instruct-2512-4bit`](https://huggingface.co/mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit) · ≈ 16.4 GB<br>[`mistralai_Devstral-Small-2-24B-Instruct-2512-MLX-4Bit`](https://huggingface.co/mlx-community/mistralai_Devstral-Small-2-24B-Instruct-2512-MLX-4Bit) · ≈ 16.4 GB | 130 |171| **Mac · 48 GB**<br><sub>MacBook Pro M4 / M5 Pro and Max</sub> | 36.0 GB | [`KAT-Coder-V2.5-Dev-OptiQ-4bit`](https://huggingface.co/mlx-community/KAT-Coder-V2.5-Dev-OptiQ-4bit) · ≈ 21.9 GB<br>[`Qwen3.6-35B-A3B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit) · ≈ 21.9 GB<br>[`DeepSeek-R1-Distill-Qwen-32B-4bit`](https://huggingface.co/mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bit) · ≈ 20.9 GB<br>[`Qwen2.5-Coder-32B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen2.5-Coder-32B-Instruct-4bit) · ≈ 20.9 GB | [`Qwen3.6-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit) · ≈ 22.2 GB<br>[`Qwen3.5-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.5-35B-A3B-4bit) · ≈ 22.2 GB | 135 |172| **Mac · 64 GB**<br><sub>MacBook Pro Max, Mac Studio</sub> | 48.0 GB | [`Llama-3.3-70B-Instruct-3bit`](https://huggingface.co/mlx-community/Llama-3.3-70B-Instruct-3bit) · ≈ 32.5 GB<br>[`KAT-Coder-V2.5-Dev-OptiQ-4bit`](https://huggingface.co/mlx-community/KAT-Coder-V2.5-Dev-OptiQ-4bit) · ≈ 21.9 GB<br>[`Qwen3.6-35B-A3B-OptiQ-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit) · ≈ 21.9 GB<br>[`DeepSeek-R1-Distill-Qwen-32B-4bit`](https://huggingface.co/mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bit) · ≈ 20.9 GB | [`Qwen3.6-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit) · ≈ 22.2 GB<br>[`Qwen3.5-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.5-35B-A3B-4bit) · ≈ 22.2 GB | 147 |173| **Mac · 96 GB**<br><sub>Mac Studio M3 Ultra</sub> | 72.0 GB | [`Mistral-Small-4-119B-2603-OptiQ-2bit`](https://huggingface.co/mlx-community/Mistral-Small-4-119B-2603-OptiQ-2bit) · ≈ 38.4 GB<br>[`Qwen3-Coder-Next-4bit`](https://huggingface.co/mlx-community/Qwen3-Coder-Next-4bit) · ≈ 45.5 GB<br>[`Qwen3-Next-80B-A3B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen3-Next-80B-A3B-Instruct-4bit) · ≈ 45.5 GB<br>[`Kimi-Dev-72B-4bit-DWQ`](https://huggingface.co/mlx-community/Kimi-Dev-72B-4bit-DWQ) · ≈ 41.9 GB | [`Qwen2.5-VL-72B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen2.5-VL-72B-Instruct-4bit) · ≈ 42.2 GB<br>[`Qwen3.6-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit) · ≈ 22.2 GB | 150 |174| **Mac · 128 GB**<br><sub>MacBook Pro M4 / M5 Max, Mac Studio</sub> | 96.0 GB | [`Mistral-Small-4-119B-2603-OptiQ-2bit`](https://huggingface.co/mlx-community/Mistral-Small-4-119B-2603-OptiQ-2bit) · ≈ 38.4 GB<br>[`Qwen3-Coder-Next-4bit`](https://huggingface.co/mlx-community/Qwen3-Coder-Next-4bit) · ≈ 45.5 GB<br>[`Qwen3-Next-80B-A3B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen3-Next-80B-A3B-Instruct-4bit) · ≈ 45.5 GB<br>[`Kimi-Dev-72B-4bit-DWQ`](https://huggingface.co/mlx-community/Kimi-Dev-72B-4bit-DWQ) · ≈ 41.9 GB | [`Qwen2.5-VL-72B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen2.5-VL-72B-Instruct-4bit) · ≈ 42.2 GB<br>[`Qwen3.6-35B-A3B-4bit`](https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit) · ≈ 22.2 GB | 152 |175| **Mac · 192 GB**<br><sub>Mac Studio M2 Ultra</sub> | 144.0 GB | [`Mistral-Small-4-119B-2603-OptiQ-2bit`](https://huggingface.co/mlx-community/Mistral-Small-4-119B-2603-OptiQ-2bit) · ≈ 38.4 GB<br>[`Qwen3-Coder-Next-4bit`](https://huggingface.co/mlx-community/Qwen3-Coder-Next-4bit) · ≈ 45.5 GB<br>[`Qwen3-Next-80B-A3B-Instruct-4bit`](https://huggingface.co/mlx-community/Qwen3-Next-80B-A3B-Instruct-4bit) · ≈ 45.5 GB<br>[`Kimi-Dev-72B-4bit-DWQ`](https://huggingface.co/mlx-community/Kimi-Dev-72B-4bit-DWQ) · ≈ 41.9 GB | [`Mistral-Medium-3.5-128B-4bit`](https://huggingface.co/mlx-community/Mistral-Medium-3.5-128B-4bit) · ≈ 70.7 GB<br>[`Qwen3.5-122B-A10B-4bit`](https://huggingface.co/mlx-community/Qwen3.5-122B-A10B-4bit) · ≈ 68.0 GB | 156 |176177Full document with embedding models: [`Documentation/MODELS_BY_DEVICE.md`](Documentation/MODELS_BY_DEVICE.md).178179## Architecture180181```mermaid182flowchart TD183    Mac[Apps/LocalChatMac] --> UI184    IOS[Apps/LocalChatIOS] --> UI185    UI[LocalChatUI<br>AppModel · ChatController · screens · design system]186    UI --> INF[LocalChatInference<br>engine · MLX · Apple Foundation Models]187    UI --> HUB[LocalChatModelHub<br>Hugging Face · downloads · device catalog]188    UI --> CONV[LocalChatConversations<br>SQLite · branches · FTS5]189    UI --> KNOW[LocalChatKnowledge<br>extraction · vectors · search]190    UI --> AG[LocalChatAgents<br>tools · permissions · loop]191    UI --> BEN[LocalChatBenchmarks<br>metrics · arena]192    UI --> SYNC[LocalChatSync]193    UI --> REM[RemoteInference]194    UI --> API[LocalAPIServer]195    INF --> CORE196    HUB --> CORE197    CONV --> CORE198    KNOW --> CORE199    AG --> CORE200    BEN --> CORE201    SYNC --> CORE202    REM --> CORE203    API --> CORE204    CORE[LocalChatCore<br>types and contracts · + Security · Networking]205```206207```mermaid208sequenceDiagram209    participant V as Chat view210    participant C as ChatController211    participant S as ConversationStore212    participant K as KnowledgeEngine213    participant E as InferenceEngine214    participant M as MLX runtime215    V->>C: send()216    C->>S: append user message217    C->>K: search attached collections218    C->>C: ContextAssembler (token budget)219    C->>E: generate(model, request)220    E->>M: load if needed (memory check), reuse KV cache221    M-->>C: text / reasoning / tool calls / metrics222    C->>S: save answer + measured metrics223    C->>E: title, compaction (summary record)224```225226Feature modules depend on `LocalChatCore` and never on each other; only `LocalChatUI` knows them all. One on-device generation runs at a time; the engine loads on demand, unloads the least recently used model when memory is needed and releases idle models on memory pressure. More in [`Documentation/ARCHITECTURE.md`](Documentation/ARCHITECTURE.md).227228## Stack229230| Layer | Technology | Version | Role |231|---|---|---|---|232| Language | Swift | language mode 6, strict concurrency | everything |233| UI | SwiftUI, Swift Charts | system | both apps |234| Inference | [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm) | 3.33.0 | LLM, VLM, embedders, tool-call and reasoning parsers |235| Inference | [mlx-swift](https://github.com/ml-explore/mlx-swift) | 0.33.x | arrays, Metal kernels, memory accounting |236| Tokenizers | [swift-transformers](https://github.com/huggingface/swift-transformers) | 1.4.0 | tokenizers and chat templates |237| System model | FoundationModels | OS 26+ | Apple Intelligence backend |238| Storage | [GRDB.swift](https://github.com/groue/GRDB.swift) | 7.12.0 | SQLite, FTS5, observation |239| Rendering | swift-markdown 0.7.3, SwiftMath 1.7.3 | | Markdown AST, LaTeX |240| Networking | URLSession, Network.framework | system | Hub client, API server (no third-party server) |241| Security | CryptoKit, LocalAuthentication, Security | system | AEAD, key derivation, app lock, Keychain |242| Project | XcodeGen | `project.yml` | generates `LocalChat.xcodeproj` |243| Optional | llama.cpp b12359 via llama.swift 3.0.0 | macOS only | imported GGUF files (not proposed in the UI) |244245## Repository layout246247```248LocalChat/249├── Package.swift               single manifest for the 13 shared modules250├── project.yml                 XcodeGen: LocalChatMac, LocalChatIOS, UI tests, LocalChatKitTests251├── LocalChat.xcworkspace252├── Apps/253│   ├── LocalChatMac/           scenes, commands, menu bar item, entitlements254│   ├── LocalChatIOS/           scene, memory-warning handling, entitlements255│   └── Shared/                 AppBootstrap256├── Packages/257│   ├── LocalChatCore/          1 559 lines — models, generation, device, context, routing, contracts258│   ├── LocalChatInference/     1 185 — engine, MLX and system backends259│   ├── LocalChatModelHub/      1 963 — Hub API, downloads, library, device catalog260│   ├── LocalChatConversations/   966 — store, search, export, sync records261│   ├── LocalChatKnowledge/     1 389 — extraction, chunking, vector index262│   ├── LocalChatAgents/        1 468 — tools, permissions, runner, research263│   ├── LocalChatBenchmarks/      693 — metrics, arena, benchmark suite264│   ├── LocalChatSecurity/        405 — Keychain, crypto, lock, path sandbox265│   ├── LocalChatSync/            555 — encrypted sync266│   ├── LocalChatNetworking/      909 — SSE, OpenAI wire types, URL policy267│   └── LocalChatUI/            9 003 — design system and every screen268├── Services/269│   ├── LocalAPIServer/           964 — OpenAI-compatible server270│   └── RemoteInference/          419 — remote nodes271├── Tests/                      23 files, 4 381 lines272├── Scripts/                    release, notarization, App Store Connect, icon, models-by-device273└── Documentation/              architecture, compatibility, limitations, test report, models by device274```275276## Build277278Requirements: Xcode 27, XcodeGen (`brew install xcodegen`).279280```bash281xcodegen generate                 # creates LocalChat.xcodeproj (LocalChat.xcworkspace references it)282open LocalChat.xcworkspace        # schemes: LocalChatMac, LocalChatIOS, LocalChatKitTests283```284285Command line:286287```bash288xcodebuild build -workspace LocalChat.xcworkspace -scheme LocalChatMac -destination 'platform=macOS,arch=arm64' -skipMacroValidation289xcodebuild build -workspace LocalChat.xcworkspace -scheme LocalChatIOS -destination 'generic/platform=iOS Simulator' -skipMacroValidation290```291292If SwiftPM hangs while resolving packages, it is waiting for a Keychain prompt: use `swift package resolve --disable-keychain`.293294## Test295296```bash297LC_NO_MLX=1 swift test --disable-keychain   # pure-Swift modules, seconds298swift test --disable-keychain               # everything, including real inference on this Mac's GPU299```300301- Real MLX tests use weights already on disk (`~/.cache/huggingface/hub/models--mlx-community--Qwen2.5-1.5B-Instruct-bf16`), the GGUF test uses `~/Library/Caches/LocalChatTests/SmolLM2-360M-Instruct-Q4_K_M.gguf`; each prints `SKIPPED` when its weights are missing.302- `LocalChatAppTests` downloads `mlx-community/Qwen3-0.6B-4bit` (351 MB) from Hugging Face once, then chats, branches and retrieves through the same controller the views use.303- `LC_LIVE=1` enables the live Hugging Face tests of the model hub.304- UI smoke tests: schemes `LocalChatMac` / `LocalChatIOS` (`xcodebuild test`), on a throw-away store (`-LCEphemeral`).305306Totals on 2026-11-20 (MacBook Pro M5 Max): **200 tests, 0 failures** — 190 in the pure-Swift modules, 9 real GPU inference tests, 1 end-to-end test through the application layer. Details and what was *not* executed: [`Documentation/TEST_REPORT.md`](Documentation/TEST_REPORT.md).307308## Release309310```bash311Scripts/release-mac.sh && Scripts/notarize-mac.sh      # dist/Local Chat.app + dist/LocalChat-<version>.dmg (keychain profile MacLustr-Notarize)312Scripts/release-ios.sh all && Scripts/release-ios.sh upload   # archive, IPA, TestFlight (App Store Connect API key on this Mac)313```314315`Scripts/asc.py` is a minimal App Store Connect API client (`asc.py GET /v1/apps?filter[bundleId]=ai.spboucher.localchat`).316317| Artifact | State on 2026-11-20 |318|---|---|319| `dist/Local Chat.app`, `dist/LocalChat-1.0.0.dmg` | signed, notarized, stapled (`spctl`: accepted, Notarized Developer ID) |320| iOS 1.0.0 (1) | uploaded, processed (`VALID`), available to the internal TestFlight group |321322## Configuration323324No environment variable is needed to run the apps. Scripts read:325326| Name | Used by | Meaning |327|---|---|---|328| `NOTARY_PROFILE` | `Scripts/notarize-mac.sh` | notarytool keychain profile (default `MacLustr-Notarize`) |329| `ASC_KEY_ID`, `ASC_ISSUER_ID` | `Scripts/asc.py`, `Scripts/release-ios.sh` | App Store Connect API key identifiers (the private key stays in `~/.appstoreconnect/private_keys`) |330| `LC_NO_MLX` | `Package.swift` | `1` leaves MLX and the UI out of the package for fast tests |331| `LC_LIVE` | tests | `1` enables tests that call huggingface.co |332333Debug builds accept launch arguments for visual checks: `-LCEphemeral` (throw-away store), `-LCSection <name>`, `-LCAutoSend "<prompt>"`, `-LCInspector`.334335## Screens336337| Section | Entry point | What it does |338|---|---|---|339| Chats | `ChatScreen`, `ConversationSidebar`, `ChatInspector` | conversations, transcript, composer, parameters |340| Models | `ModelLibraryScreen` | By device · Installed · Discover · Downloads · Recommended |341| Arena | `ArenaScreen` | side-by-side comparison, blind vote, leaderboard |342| Observatory | `ObservatoryScreen` | live and historical measured telemetry, benchmark |343| Knowledge | `KnowledgeScreen` | collections, documents, indexing, search playground, research mode |344| Agents | `AgentStudioScreen` | agents, tool permissions, playground with approvals, run history |345| Settings | `SettingsScreen` | general, privacy and lock, remote nodes, local API server, sync, storage, about |346347## Local API348349Served by the Mac app when enabled in Settings (default `http://127.0.0.1:8765`).350351| Method | Path | Notes |352|---|---|---|353| GET | `/health` | liveness |354| GET | `/v1/models` | installed and remote models |355| POST | `/v1/chat/completions` | streaming (SSE) and non-streaming, tools, usage, `reasoning_content` |356| POST | `/v1/embeddings` | embedding models |357| GET | `/`, `/docs` | built-in documentation page |358| GET / PUT | `/lc/v1/sync/…` | encrypted sync relay (opaque batches) |359360## Data model361362| Store | File | Content |363|---|---|---|364| Conversations | `Data/localchat.sqlite` | conversations, messages (`parentID` tree), folders, templates, workspaces, agents, preferences, FTS5 index, tombstones for sync |365| Knowledge | `Data/knowledge.sqlite` | collections, documents, chunks, Float32 vectors, FTS5 |366| Metrics | `Data/metrics.sqlite` | one record per generation, arena runs and votes |367| Models | `Models/<org>/<repo>/` + `library.json` | weights (excluded from backups) |368| Secrets | Keychain | node API keys, Hugging Face token, sync key |369370## Security & privacy371372- Nothing leaves the device by default. Browsing or downloading models contacts huggingface.co; a remote node receives the conversations that use it; sync is off until configured and encrypted before upload.373- The API server listens on 127.0.0.1; LAN exposure forces authentication and is refused without an active key. Plain HTTP only (no TLS).374- Tools never run unless enabled for the agent; file tools are confined to granted folders (symlink escapes refused); shell commands always need approval and destructive patterns are refused.375- App Sandbox and Hardened Runtime on macOS; standard Apple cryptography only (`ITSAppUsesNonExemptEncryption = false`).376377## Gotchas378379- SwiftPM can hang on a Keychain prompt while downloading a binary artifact: always pass `--disable-keychain`.380- `LC_NO_MLX=1` removes the MLX modules from the manifest; `swift test` without it builds the MLX Metal library and runs real inference.381- MLX does not run in the iOS Simulator (no real GPU): the app says so and offers remote nodes.382- SwiftUI's `.inspector` enters an AppKit constraint loop while text streams; the chat inspector is a plain side panel.383- Database observation events can arrive late: `ChatController` always re-reads the store instead of trusting the event payload.384- The "Increased Memory Limit" and "Extended Virtual Addressing" capabilities are not in the public App Store Connect API; automatic signing (`-allowProvisioningUpdates` with the API key) enables them.385386## Known limitations387388- **Never run on a physical iPhone or iPad** (no device was available): on-device MLX inference, memory behaviour, Face ID, dictation and background downloads are unverified.389- Arena, Knowledge, Agents and Settings screens compile on both platforms but were not clicked through by hand; XCUITest suites are written and were never executed.390- No Share-sheet extension, no structured-output enforcement, no TLS on the local API server, no automatic update on macOS.391- No comparison with other local AI apps was measured; no superiority is claimed.392393Full list: [`Documentation/LIMITATIONS.md`](Documentation/LIMITATIONS.md).394395## Documentation396397- [`Documentation/MODELS_BY_DEVICE.md`](Documentation/MODELS_BY_DEVICE.md) — MLX models per device class (generated)398- [`Documentation/COMPATIBILITY.md`](Documentation/COMPATIBILITY.md) — backend × platform × capability399- [`Documentation/ARCHITECTURE.md`](Documentation/ARCHITECTURE.md)400- [`Documentation/LIMITATIONS.md`](Documentation/LIMITATIONS.md)401- [`Documentation/TEST_REPORT.md`](Documentation/TEST_REPORT.md)402403## Author404405<div align="center"><img src="docs/brand/icon.png" width="64" height="64" alt=""><br>406407**Simon-Pierre Boucher** — design, development, operations<br>408[www.spboucher.ai](https://www.spboucher.ai) · [git.spboucher.ai](https://git.spboucher.ai) · contact@spboucher.ai</div>409410## License411412All rights reserved © 2026 Simon-Pierre Boucher. Third-party packages keep their own licenses (MLX and MLX Swift LM: MIT; GRDB: MIT; swift-transformers: Apache-2.0; SwiftMath: MIT; swift-markdown: Apache-2.0; llama.cpp: MIT).413
Files main
Menu

Theme