SPB Git

Repository Public

COMPATIBILITY.md 3.9 KB

# Compatibility matrix

Status words: verified = executed in this repository's tests or by hand on the stated hardware; built = compiles and is wired, not executed; no = not available. Verification hardware: MacBook Pro M5 Max, 48 GB, macOS 27.0. No physical iPhone or iPad was available.

# Backends by platform

Backend Formats macOS (Apple silicon) iPhone / iPad (device) iOS Simulator
MLX Swift (mlx-swift-lm 3.33.0) MLX repositories (safetensors + config) verified built — never run on a device no (MLX needs a real GPU; the app says so)
llama.cpp b12359 GGUF verified no in this build (the dependency is linked for macOS only; the upstream XCFramework has no Simulator slice) no
Apple Foundation Models system model built — not exercised built — not exercised listed as available, generation fails (error shown)
Remote node any OpenAI-compatible server client ↔ server verified in tests over loopback built — same code, never run on a device built

# Capabilities by backend

Capability MLX llama.cpp (GGUF) Apple Foundation Models Remote node
Token streaming verified verified built verified (SSE)
Cancellation verified built built verified
KV-cache reuse across turns verified (33 cached prompt tokens on turn 2) verified (prefix reuse, 47 tokens) managed by the OS server side
Sampling: temperature, top-p, top-k, min-p, repetition penalty, seed passed to the backend passed to the backend temperature and max tokens only temperature, top-p, max tokens, seed, stop
Context length / max tokens yes (maxKVSize) yes (context recreated when the length changes) max tokens only max tokens
KV-cache quantization yes (4 / 8 bit) no no no
Reasoning separated from the answer yes (model's reasoning protocol) yes (<think> parser) no yes (reasoning_content)
Tool calling verified (native parsers of mlx-swift-lm) built: Jinja template + <tool_call> text parser no built (OpenAI tool_calls)
Vision (image input) built: VLM factory + image attachments; no VLM was run no (projector files are detected, not loaded) no built (image_url parts)
Embeddings built (MLXEmbedders); the Knowledge Studio was verified with Apple sentence embeddings no no built (/embeddings)
Structured output (JSON schema) no (schema is accepted by the API and ignored) no no forwarded to the server
Exact token counts yes (model tokenizer) yes yes on OS 27 no (estimated, labelled ≈)
Measured metrics TTFT, prompt and generation time, tokens, peak / active MLX memory TTFT, prompt and generation time, tokens, process footprint wall-clock only TTFT, total time, server-reported usage

# Models

  • MLX architectures are those implemented by mlx-swift-lm 3.33.0 (Llama, Qwen 2 / 3 / 3.5, Gemma 2 / 3 / 3n / 4, Phi, Mistral, DeepSeek, GLM, LFM2, SmolLM3, Granite… and the VLMs it lists). A repository whose model_type is not implemented fails at load with an explicit message; Local Chat never claims a model is compatible before it loads.
  • GGUF: whatever llama.cpp b12359 loads. Chat needs a tokenizer.chat_template in the file.
  • Tested for real: mlx-community/Qwen2.5-1.5B-Instruct-bf16, mlx-community/Qwen3-0.6B-4bit, bartowski/SmolLM2-360M-Instruct-GGUF (Q4_K_M).

# Memory rule

A model is offered when its estimated need (weights + KV cache at the chosen context + 350 MB) fits the share of RAM Local Chat allows a model: 65–75 % of physical memory on a Mac, at most 55 % on iPhone / iPad (and never more than what iOS reports as available). The estimate is replaced by the measured figure once the model is loaded.

Files main
Menu

Theme