Local Chat
Private local AI for Mac, iPhone and iPad — real models running on Apple silicon with Apple's MLX framework.
One Xcode workspace, two native apps, one shared Swift package. No web view, no Electron, nothing leaves the device by default.
Author: Simon-Pierre Boucher
Table of contents
- Overview · Screenshots · Features · Models by device
- Architecture · Stack · Repository layout
- Build · Test · Release · Configuration
- Screens · Local API · Data model · Security & privacy
- Gotchas · Known limitations · Documentation · Author · License
Overview
Local Chat downloads open models from Hugging Face and runs them on the device with MLX, Apple's machine-learning framework for Apple silicon. The same Swift code drives a Mac app and an iPhone / iPad app.
| macOS | iOS · iPadOS | |
|---|---|---|
| Target | Apps/LocalChatMac |
Apps/LocalChatIOS |
| Minimum OS | macOS 15, Apple silicon | iOS 18 |
| Distribution | Developer ID, App Sandbox, Hardened Runtime, notarized DMG | App Store Connect app 6837210574, TestFlight internal group |
| Layout | sidebar + detail, side inspector, command palette (⌘K), menu bar item, multiple windows | tabs on iPhone, sidebar on iPad |
| Inference | MLX on the Mac's GPU (verified) | MLX on device (built, not yet run on a physical device), or a Mac you trust as a remote node |
Related repositories on spbgit: sp-chat and sp-chat-ios are clients of a hosted service; Local Chat is independent and needs no server.
Screenshots
Real captures of the Debug build on a MacBook Pro M5 Max and of the iPhone 17 Pro Simulator.
| Chat — real MLX answer with LaTeX, code and measured metrics | Models → By device — live list from Hugging Face |
|---|---|
![]() |
![]() |
| Inference Observatory — measured memory and device facts | iPhone (Simulator) — conversation list |
|---|---|
![]() |
![]() |
Features
💬 Chat
- Real token streaming, stop at any time, measured tokens/s, time to first token and cached prompt tokens under every answer.
- Conversation tree: editing a message or regenerating an answer creates a sibling branch; nothing is overwritten (
‹ 2 / 3 ›switcher), fork a conversation from any message. - Folders, tags, favorites, archive, full-text search over every message (SQLite FTS5).
- System prompts, prompt templates with
{variables}, inference presets (Fast, Balanced, Reasoning, Maximum Quality) and every sampling parameter. - Markdown, syntax-highlighted code, tables and LaTeX rendered natively (SwiftMath).
- Attachments: images (vision models), PDF / DOCX / CSV / Markdown / source code extracted on device.
- Context-window bar (system, summary, knowledge, conversation, attachments, reserved output) with exact token counts from the loaded tokenizer; configurable compaction that summarizes older turns without touching the original messages.
- Model switching inside a conversation; each answer shows which model and backend produced it.
- Dictation (system speech recognizer, on device when the OS supports the language) and read-aloud.
📦 Models
- By device: MLX models proposed for every Apple device class, from the live
mlx-communitylisting (see Models by device). - Hugging Face search (MLX only), repository detail with files, quantization, parameters, license, gated state, estimated memory and the compatibility verdict for this device.
- Resumable multi-file downloads with SHA-256 verification of every LFS file, pause / resume / cancel, measured speed and ETA.
- Local library: load / unload, measured speed on this device, import of a local MLX folder, delete.
- Device profiler (chip, memory, storage, thermal state, low power) and recommendations per task; optional router that picks a model per message (always overridable).
⚖️ Model Arena · 📈 Inference Observatory
- Arena: 2–4 models on the same prompt, side by side, blind mode with vote, fixed seed for repeatable runs, automatic sequential execution when the models do not fit in memory together, Elo and win-rate leaderboard, export.
- Observatory: live generation, loaded models, process and MLX memory, device facts, last generation, history charts and aggregates (median / p10 / p90), benchmark suite. Only measured values — no GPU utilisation, temperature or wattage tiles, because they are not measured.
📚 Knowledge Studio · ✨ Agent Studio
- On-device RAG: PDF (with optional OCR), DOCX, CSV, Markdown, TXT, source code → structure-aware chunking → local embeddings → exact vector search (100 000 chunks × 384 dimensions in 8–12 ms on an M5 Max) fused with BM25; numbered sources under answers; real deletion.
- Agents: tool registry with explicit permissions, built-in tools (calculator, date, file read / list / search / write inside granted folders, knowledge search, URL fetch with SSRF guard, shell on macOS), human approval checkpoints, step / tool-call / time limits, run logs, research mode with citation checking.
🌐 Local API · remote nodes · sync · security
- OpenAI-compatible HTTP server on the Mac (
/v1/models,/v1/chat/completionswith streaming,/v1/embeddings), bound to 127.0.0.1 by default, API keys stored as hashes, rate limiting, request diagnostics, built-in docs page. - Remote nodes: an iPhone can route a conversation to a trusted Mac (or any OpenAI-compatible endpoint); the conversation shows that it is remote.
- Optional end-to-end encrypted sync of conversations, templates, preferences and agent configurations through a folder or a Mac relay. Model weights are never synchronized.
- Keychain for secrets, optional Face ID / Touch ID lock, App Sandbox.
Models by device
Local Chat proposes MLX models only, taken live from huggingface.co/mlx-community. A model is proposed for a device class when MLX Swift implements its architecture and its estimated memory (weights + KV cache for 4,096 tokens + 350 MB) stays under 70 % of what a model may use there: 65 % of RAM on a Mac under 32 GB, 75 % from 32 GB, 55 % on iPhone and iPad.
Snapshot of 2026-11-20 — regenerate with python3 Scripts/models-by-device.py; the app refreshes its own list daily (Models → By device). Sizes are estimates; nothing in the iPhone and iPad rows was run on a physical device yet.
Full document with embedding models: Documentation/MODELS_BY_DEVICE.md.
Architecture
flowchart TD
Mac[Apps/LocalChatMac] --> UI
IOS[Apps/LocalChatIOS] --> UI
UI[LocalChatUI<br>AppModel · ChatController · screens · design system]
UI --> INF[LocalChatInference<br>engine · MLX · Apple Foundation Models]
UI --> HUB[LocalChatModelHub<br>Hugging Face · downloads · device catalog]
UI --> CONV[LocalChatConversations<br>SQLite · branches · FTS5]
UI --> KNOW[LocalChatKnowledge<br>extraction · vectors · search]
UI --> AG[LocalChatAgents<br>tools · permissions · loop]
UI --> BEN[LocalChatBenchmarks<br>metrics · arena]
UI --> SYNC[LocalChatSync]
UI --> REM[RemoteInference]
UI --> API[LocalAPIServer]
INF --> CORE
HUB --> CORE
CONV --> CORE
KNOW --> CORE
AG --> CORE
BEN --> CORE
SYNC --> CORE
REM --> CORE
API --> CORE
CORE[LocalChatCore<br>types and contracts · + Security · Networking]
sequenceDiagram
participant V as Chat view
participant C as ChatController
participant S as ConversationStore
participant K as KnowledgeEngine
participant E as InferenceEngine
participant M as MLX runtime
V->>C: send()
C->>S: append user message
C->>K: search attached collections
C->>C: ContextAssembler (token budget)
C->>E: generate(model, request)
E->>M: load if needed (memory check), reuse KV cache
M-->>C: text / reasoning / tool calls / metrics
C->>S: save answer + measured metrics
C->>E: title, compaction (summary record)
Feature modules depend on LocalChatCore and never on each other; only LocalChatUI knows them all. One on-device generation runs at a time; the engine loads on demand, unloads the least recently used model when memory is needed and releases idle models on memory pressure. More in Documentation/ARCHITECTURE.md.
Stack
| Layer | Technology | Version | Role |
|---|---|---|---|
| Language | Swift | language mode 6, strict concurrency | everything |
| UI | SwiftUI, Swift Charts | system | both apps |
| Inference | mlx-swift-lm | 3.33.0 | LLM, VLM, embedders, tool-call and reasoning parsers |
| Inference | mlx-swift | 0.33.x | arrays, Metal kernels, memory accounting |
| Tokenizers | swift-transformers | 1.4.0 | tokenizers and chat templates |
| System model | FoundationModels | OS 26+ | Apple Intelligence backend |
| Storage | GRDB.swift | 7.12.0 | SQLite, FTS5, observation |
| Rendering | swift-markdown 0.7.3, SwiftMath 1.7.3 | Markdown AST, LaTeX | |
| Networking | URLSession, Network.framework | system | Hub client, API server (no third-party server) |
| Security | CryptoKit, LocalAuthentication, Security | system | AEAD, key derivation, app lock, Keychain |
| Project | XcodeGen | project.yml |
generates LocalChat.xcodeproj |
| Optional | llama.cpp b12359 via llama.swift 3.0.0 | macOS only | imported GGUF files (not proposed in the UI) |
Repository layout
LocalChat/
├── Package.swift single manifest for the 13 shared modules
├── project.yml XcodeGen: LocalChatMac, LocalChatIOS, UI tests, LocalChatKitTests
├── LocalChat.xcworkspace
├── Apps/
│ ├── LocalChatMac/ scenes, commands, menu bar item, entitlements
│ ├── LocalChatIOS/ scene, memory-warning handling, entitlements
│ └── Shared/ AppBootstrap
├── Packages/
│ ├── LocalChatCore/ 1 559 lines — models, generation, device, context, routing, contracts
│ ├── LocalChatInference/ 1 185 — engine, MLX and system backends
│ ├── LocalChatModelHub/ 1 963 — Hub API, downloads, library, device catalog
│ ├── LocalChatConversations/ 966 — store, search, export, sync records
│ ├── LocalChatKnowledge/ 1 389 — extraction, chunking, vector index
│ ├── LocalChatAgents/ 1 468 — tools, permissions, runner, research
│ ├── LocalChatBenchmarks/ 693 — metrics, arena, benchmark suite
│ ├── LocalChatSecurity/ 405 — Keychain, crypto, lock, path sandbox
│ ├── LocalChatSync/ 555 — encrypted sync
│ ├── LocalChatNetworking/ 909 — SSE, OpenAI wire types, URL policy
│ └── LocalChatUI/ 9 003 — design system and every screen
├── Services/
│ ├── LocalAPIServer/ 964 — OpenAI-compatible server
│ └── RemoteInference/ 419 — remote nodes
├── Tests/ 23 files, 4 381 lines
├── Scripts/ release, notarization, App Store Connect, icon, models-by-device
└── Documentation/ architecture, compatibility, limitations, test report, models by deviceBuild
Requirements: Xcode 27, XcodeGen (brew install xcodegen).
xcodegen generate # creates LocalChat.xcodeproj (LocalChat.xcworkspace references it)
open LocalChat.xcworkspace # schemes: LocalChatMac, LocalChatIOS, LocalChatKitTestsCommand line:
xcodebuild build -workspace LocalChat.xcworkspace -scheme LocalChatMac -destination 'platform=macOS,arch=arm64' -skipMacroValidation
xcodebuild build -workspace LocalChat.xcworkspace -scheme LocalChatIOS -destination 'generic/platform=iOS Simulator' -skipMacroValidationIf SwiftPM hangs while resolving packages, it is waiting for a Keychain prompt: use swift package resolve --disable-keychain.
Test
LC_NO_MLX=1 swift test --disable-keychain # pure-Swift modules, seconds
swift test --disable-keychain # everything, including real inference on this Mac's GPU- Real MLX tests use weights already on disk (
~/.cache/huggingface/hub/models--mlx-community--Qwen2.5-1.5B-Instruct-bf16), the GGUF test uses~/Library/Caches/LocalChatTests/SmolLM2-360M-Instruct-Q4_K_M.gguf; each printsSKIPPEDwhen its weights are missing. LocalChatAppTestsdownloadsmlx-community/Qwen3-0.6B-4bit(351 MB) from Hugging Face once, then chats, branches and retrieves through the same controller the views use.LC_LIVE=1enables the live Hugging Face tests of the model hub.- UI smoke tests: schemes
LocalChatMac/LocalChatIOS(xcodebuild test), on a throw-away store (-LCEphemeral).
Totals on 2026-11-20 (MacBook Pro M5 Max): 200 tests, 0 failures — 190 in the pure-Swift modules, 9 real GPU inference tests, 1 end-to-end test through the application layer. Details and what was not executed: Documentation/TEST_REPORT.md.
Release
Scripts/release-mac.sh && Scripts/notarize-mac.sh # dist/Local Chat.app + dist/LocalChat-<version>.dmg (keychain profile MacLustr-Notarize)
Scripts/release-ios.sh all && Scripts/release-ios.sh upload # archive, IPA, TestFlight (App Store Connect API key on this Mac)Scripts/asc.py is a minimal App Store Connect API client (asc.py GET /v1/apps?filter[bundleId]=ai.spboucher.localchat).
| Artifact | State on 2026-11-20 |
|---|---|
dist/Local Chat.app, dist/LocalChat-1.0.0.dmg |
signed, notarized, stapled (spctl: accepted, Notarized Developer ID) |
| iOS 1.0.0 (1) | uploaded, processed (VALID), available to the internal TestFlight group |
Configuration
No environment variable is needed to run the apps. Scripts read:
| Name | Used by | Meaning |
|---|---|---|
NOTARY_PROFILE |
Scripts/notarize-mac.sh |
notarytool keychain profile (default MacLustr-Notarize) |
ASC_KEY_ID, ASC_ISSUER_ID |
Scripts/asc.py, Scripts/release-ios.sh |
App Store Connect API key identifiers (the private key stays in ~/.appstoreconnect/private_keys) |
LC_NO_MLX |
Package.swift |
1 leaves MLX and the UI out of the package for fast tests |
LC_LIVE |
tests | 1 enables tests that call huggingface.co |
Debug builds accept launch arguments for visual checks: -LCEphemeral (throw-away store), -LCSection <name>, -LCAutoSend "<prompt>", -LCInspector.
Screens
| Section | Entry point | What it does |
|---|---|---|
| Chats | ChatScreen, ConversationSidebar, ChatInspector |
conversations, transcript, composer, parameters |
| Models | ModelLibraryScreen |
By device · Installed · Discover · Downloads · Recommended |
| Arena | ArenaScreen |
side-by-side comparison, blind vote, leaderboard |
| Observatory | ObservatoryScreen |
live and historical measured telemetry, benchmark |
| Knowledge | KnowledgeScreen |
collections, documents, indexing, search playground, research mode |
| Agents | AgentStudioScreen |
agents, tool permissions, playground with approvals, run history |
| Settings | SettingsScreen |
general, privacy and lock, remote nodes, local API server, sync, storage, about |
Local API
Served by the Mac app when enabled in Settings (default http://127.0.0.1:8765).
| Method | Path | Notes |
|---|---|---|
| GET | /health |
liveness |
| GET | /v1/models |
installed and remote models |
| POST | /v1/chat/completions |
streaming (SSE) and non-streaming, tools, usage, reasoning_content |
| POST | /v1/embeddings |
embedding models |
| GET | /, /docs |
built-in documentation page |
| GET / PUT | /lc/v1/sync/… |
encrypted sync relay (opaque batches) |
Data model
| Store | File | Content |
|---|---|---|
| Conversations | Data/localchat.sqlite |
conversations, messages (parentID tree), folders, templates, workspaces, agents, preferences, FTS5 index, tombstones for sync |
| Knowledge | Data/knowledge.sqlite |
collections, documents, chunks, Float32 vectors, FTS5 |
| Metrics | Data/metrics.sqlite |
one record per generation, arena runs and votes |
| Models | Models/<org>/<repo>/ + library.json |
weights (excluded from backups) |
| Secrets | Keychain | node API keys, Hugging Face token, sync key |
Security & privacy
- Nothing leaves the device by default. Browsing or downloading models contacts huggingface.co; a remote node receives the conversations that use it; sync is off until configured and encrypted before upload.
- The API server listens on 127.0.0.1; LAN exposure forces authentication and is refused without an active key. Plain HTTP only (no TLS).
- Tools never run unless enabled for the agent; file tools are confined to granted folders (symlink escapes refused); shell commands always need approval and destructive patterns are refused.
- App Sandbox and Hardened Runtime on macOS; standard Apple cryptography only (
ITSAppUsesNonExemptEncryption = false).
Gotchas
- SwiftPM can hang on a Keychain prompt while downloading a binary artifact: always pass
--disable-keychain. LC_NO_MLX=1removes the MLX modules from the manifest;swift testwithout it builds the MLX Metal library and runs real inference.- MLX does not run in the iOS Simulator (no real GPU): the app says so and offers remote nodes.
- SwiftUI's
.inspectorenters an AppKit constraint loop while text streams; the chat inspector is a plain side panel. - Database observation events can arrive late:
ChatControlleralways re-reads the store instead of trusting the event payload. - The "Increased Memory Limit" and "Extended Virtual Addressing" capabilities are not in the public App Store Connect API; automatic signing (
-allowProvisioningUpdateswith the API key) enables them.
Known limitations
- Never run on a physical iPhone or iPad (no device was available): on-device MLX inference, memory behaviour, Face ID, dictation and background downloads are unverified.
- Arena, Knowledge, Agents and Settings screens compile on both platforms but were not clicked through by hand; XCUITest suites are written and were never executed.
- No Share-sheet extension, no structured-output enforcement, no TLS on the local API server, no automatic update on macOS.
- No comparison with other local AI apps was measured; no superiority is claimed.
Full list: Documentation/LIMITATIONS.md.
Documentation
Documentation/MODELS_BY_DEVICE.md— MLX models per device class (generated)Documentation/COMPATIBILITY.md— backend × platform × capabilityDocumentation/ARCHITECTURE.mdDocumentation/LIMITATIONS.mdDocumentation/TEST_REPORT.md
Author
Simon-Pierre Boucher — design, development, operations
www.spboucher.ai · git.spboucher.ai · contact@spboucher.ai
License
All rights reserved © 2026 Simon-Pierre Boucher. Third-party packages keep their own licenses (MLX and MLX Swift LM: MIT; GRDB: MIT; swift-transformers: Apache-2.0; SwiftMath: MIT; swift-markdown: Apache-2.0; llama.cpp: MIT).



