Repository Public
1<!--2Local Chat3File: Documentation/COMPATIBILITY.md4Author: Simon-Pierre Boucher5Contact: contact@spboucher.ai6-->7# Compatibility matrix89Status words: **verified** = executed in this repository's tests or by hand on the stated hardware; **built** = compiles and is wired, not executed; **no** = not available.10Verification hardware: MacBook Pro M5 Max, 48 GB, macOS 27.0. No physical iPhone or iPad was available.1112## Backends by platform1314| Backend | Formats | macOS (Apple silicon) | iPhone / iPad (device) | iOS Simulator |15|---|---|---|---|---|16| MLX Swift (mlx-swift-lm 3.33.0) | MLX repositories (safetensors + config) | **verified** | built — never run on a device | no (MLX needs a real GPU; the app says so) |17| llama.cpp b12359 | GGUF | **verified** | no in this build (the dependency is linked for macOS only; the upstream XCFramework has no Simulator slice) | no |18| Apple Foundation Models | system model | built — not exercised | built — not exercised | listed as available, generation fails (error shown) |19| Remote node | any OpenAI-compatible server | client ↔ server **verified** in tests over loopback | built — same code, never run on a device | built |2021## Capabilities by backend2223| Capability | MLX | llama.cpp (GGUF) | Apple Foundation Models | Remote node |24|---|---|---|---|---|25| Token streaming | verified | verified | built | verified (SSE) |26| Cancellation | verified | built | built | verified |27| KV-cache reuse across turns | verified (33 cached prompt tokens on turn 2) | verified (prefix reuse, 47 tokens) | managed by the OS | server side |28| Sampling: temperature, top-p, top-k, min-p, repetition penalty, seed | passed to the backend | passed to the backend | temperature and max tokens only | temperature, top-p, max tokens, seed, stop |29| Context length / max tokens | yes (`maxKVSize`) | yes (context recreated when the length changes) | max tokens only | max tokens |30| KV-cache quantization | yes (4 / 8 bit) | no | no | no |31| Reasoning separated from the answer | yes (model's reasoning protocol) | yes (`<think>` parser) | no | yes (`reasoning_content`) |32| Tool calling | verified (native parsers of mlx-swift-lm) | built: Jinja template + `<tool_call>` text parser | no | built (OpenAI `tool_calls`) |33| Vision (image input) | built: VLM factory + image attachments; no VLM was run | no (projector files are detected, not loaded) | no | built (`image_url` parts) |34| Embeddings | built (MLXEmbedders); the Knowledge Studio was verified with Apple sentence embeddings | no | no | built (`/embeddings`) |35| Structured output (JSON schema) | no (schema is accepted by the API and ignored) | no | no | forwarded to the server |36| Exact token counts | yes (model tokenizer) | yes | yes on OS 27 | no (estimated, labelled ≈) |37| Measured metrics | TTFT, prompt and generation time, tokens, peak / active MLX memory | TTFT, prompt and generation time, tokens, process footprint | wall-clock only | TTFT, total time, server-reported usage |3839## Models4041- MLX architectures are those implemented by mlx-swift-lm 3.33.0 (Llama, Qwen 2 / 3 / 3.5, Gemma 2 / 3 / 3n / 4, Phi, Mistral, DeepSeek, GLM, LFM2, SmolLM3, Granite… and the VLMs it lists). A repository whose `model_type` is not implemented fails at load with an explicit message; Local Chat never claims a model is compatible before it loads.42- GGUF: whatever llama.cpp b12359 loads. Chat needs a `tokenizer.chat_template` in the file.43- Tested for real: `mlx-community/Qwen2.5-1.5B-Instruct-bf16`, `mlx-community/Qwen3-0.6B-4bit`, `bartowski/SmolLM2-360M-Instruct-GGUF` (Q4_K_M).4445## Memory rule4647A model is offered when its estimated need (weights + KV cache at the chosen context + 350 MB) fits the share of RAM Local Chat allows a model: 65–75 % of physical memory on a Mac, at most 55 % on iPhone / iPad (and never more than what iOS reports as available). The estimate is replaced by the measured figure once the model is loaded.48