Compatibility matrix
Status words: verified = executed in this repository's tests or by hand on the stated hardware; built = compiles and is wired, not executed; no = not available. Verification hardware: MacBook Pro M5 Max, 48 GB, macOS 27.0. No physical iPhone or iPad was available.
Backends by platform
| Backend | Formats | macOS (Apple silicon) | iPhone / iPad (device) | iOS Simulator |
|---|---|---|---|---|
| MLX Swift (mlx-swift-lm 3.33.0) | MLX repositories (safetensors + config) | verified | built — never run on a device | no (MLX needs a real GPU; the app says so) |
| llama.cpp b12359 | GGUF | verified | no in this build (the dependency is linked for macOS only; the upstream XCFramework has no Simulator slice) | no |
| Apple Foundation Models | system model | built — not exercised | built — not exercised | listed as available, generation fails (error shown) |
| Remote node | any OpenAI-compatible server | client ↔ server verified in tests over loopback | built — same code, never run on a device | built |
Capabilities by backend
| Capability | MLX | llama.cpp (GGUF) | Apple Foundation Models | Remote node |
|---|---|---|---|---|
| Token streaming | verified | verified | built | verified (SSE) |
| Cancellation | verified | built | built | verified |
| KV-cache reuse across turns | verified (33 cached prompt tokens on turn 2) | verified (prefix reuse, 47 tokens) | managed by the OS | server side |
| Sampling: temperature, top-p, top-k, min-p, repetition penalty, seed | passed to the backend | passed to the backend | temperature and max tokens only | temperature, top-p, max tokens, seed, stop |
| Context length / max tokens | yes (maxKVSize) |
yes (context recreated when the length changes) | max tokens only | max tokens |
| KV-cache quantization | yes (4 / 8 bit) | no | no | no |
| Reasoning separated from the answer | yes (model's reasoning protocol) | yes (<think> parser) |
no | yes (reasoning_content) |
| Tool calling | verified (native parsers of mlx-swift-lm) | built: Jinja template + <tool_call> text parser |
no | built (OpenAI tool_calls) |
| Vision (image input) | built: VLM factory + image attachments; no VLM was run | no (projector files are detected, not loaded) | no | built (image_url parts) |
| Embeddings | built (MLXEmbedders); the Knowledge Studio was verified with Apple sentence embeddings | no | no | built (/embeddings) |
| Structured output (JSON schema) | no (schema is accepted by the API and ignored) | no | no | forwarded to the server |
| Exact token counts | yes (model tokenizer) | yes | yes on OS 27 | no (estimated, labelled ≈) |
| Measured metrics | TTFT, prompt and generation time, tokens, peak / active MLX memory | TTFT, prompt and generation time, tokens, process footprint | wall-clock only | TTFT, total time, server-reported usage |
Models
- MLX architectures are those implemented by mlx-swift-lm 3.33.0 (Llama, Qwen 2 / 3 / 3.5, Gemma 2 / 3 / 3n / 4, Phi, Mistral, DeepSeek, GLM, LFM2, SmolLM3, Granite… and the VLMs it lists). A repository whose
model_typeis not implemented fails at load with an explicit message; Local Chat never claims a model is compatible before it loads. - GGUF: whatever llama.cpp b12359 loads. Chat needs a
tokenizer.chat_templatein the file. - Tested for real:
mlx-community/Qwen2.5-1.5B-Instruct-bf16,mlx-community/Qwen3-0.6B-4bit,bartowski/SmolLM2-360M-Instruct-GGUF(Q4_K_M).
Memory rule
A model is offered when its estimated need (weights + KV cache at the chosen context + 350 MB) fits the share of RAM Local Chat allows a model: 65–75 % of physical memory on a Mac, at most 55 % on iPhone / iPad (and never more than what iOS reports as available). The estimate is replaced by the measured figure once the model is loaded.