SPB Git

Repository Public MIT

MODEL_COMPATIBILITY.md 14.1 KB

# Model compatibility

Generated by Scripts/verify-models.sh from the records in build/verify/. Every number below was measured by running the model; nothing is copied from a model card.

All measurements were taken on a Mac: Mac17,6 (Apple M5 Max, 48 GB), macOS 27.0.0, with mlx-swift-lm 3.33.0 (release build), on 2026-11-21 to 2026-11-22. They are not iPhone measurements. No iPhone was attached to this Mac; see "Not yet verified on a physical iPhone" below.

One-minute load average during the run: 8 to 16 on 18 cores.

What "pass" means: the repository's current commit was resolved from the Hugging Face API, every file was downloaded through AstraLocal's DownloadCoordinator and its SHA-256 recomputed from disk (and matched against the hash the Hub publishes for LFS files), the model was loaded by AstraLocal's MLX runtime, answered a two-turn conversation with the second turn reusing the KV cache, stopped within 3 s of a mid-stream cancellation with AstraError.cancelled, loaded again from disk and answered, and completed the fixed four-prompt benchmark. Speed is the median decode rate of the three warm prompts; time to first token is the warm median; peak memory is the highest MLX allocator peak during the benchmark (weights included, short prompts).

# Results

Model (repository) Result Revision Quantization Download Speed TTFT warm / cold Peak memory In catalog
mlx-community/Qwen3-0.6B-4bit pass 73e3e38d981303bc594367cd910ea6eb48349da8 4-bit 0.33 GB (9 files) 423.3 tok/s 0.013 s / 0.011 s 683 MB yes
mlx-community/Qwen3-1.7B-4bit pass 3b1b1768f8f8cf8351c712464f906e86c2b8269e 4-bit 0.92 GB (9 files) 347.3 tok/s 0.018 s / 0.022 s 1470 MB yes
mlx-community/Qwen3-4B-4bit pass 4dcb3d101c2a062e5c1d4bb173588c54ea6c4d25 4-bit 2.12 GB (9 files) 176.9 tok/s 0.036 s / 0.027 s 2747 MB yes
mlx-community/Qwen2.5-0.5B-Instruct-4bit pass a5339a4131f135d0fdc6a5c8b5bbed2753bbe0f3 4-bit 0.27 GB (9 files) 562.3 tok/s 0.007 s / 0.009 s 641 MB yes
mlx-community/gemma-3-1b-it-qat-4bit pass 15fed4eafb456c6fcb2a1165f19ac609670ed14b 4-bit (QAT) 0.72 GB (8 files) 330.9 tok/s 0.015 s / 0.025 s 856 MB yes
mlx-community/gemma-3-4b-it-qat-4bit pass 3d9ef289111449933c22761961f16a5df237ce2a 4-bit (QAT) 2.83 GB (12 files) 172.5 tok/s 0.038 s / 0.036 s 3316 MB no
mlx-community/Llama-3.2-1B-Instruct-4bit pass 08231374eeacb049a0eade7922910865b8fce912 4-bit 0.66 GB (6 files) 537.4 tok/s 0.011 s / 0.016 s 1139 MB yes
mlx-community/Llama-3.2-3B-Instruct-4bit pass 7f0dc925e0d0afb0322d96f9255cfddf2ba5636e 4-bit 1.70 GB (6 files) 227.7 tok/s 0.031 s / 0.031 s 2344 MB yes
mlx-community/Phi-4-mini-instruct-4bit pass ac1c269cb4222a4e136a3d09edad301056c1f36a 4-bit 2.03 GB (12 files) 186.9 tok/s 0.022 s / 0.025 s 2723 MB yes
mlx-community/SmolLM2-135M-Instruct pass 422de227b90002f443a21a58b1087f6ee7632731 none (16-bit float) 0.26 GB (8 files) 399.2 tok/s 0.008 s / 0.008 s 397 MB yes
mlx-community/SmolLM2-135M-Instruct-8bit pass 0f0d9b8218915bc34d401e1a340b8c049d300d5e 8-bit 0.14 GB (8 files) 501.3 tok/s 0.008 s / 0.009 s 326 MB yes
mlx-community/SmolLM2-360M-Instruct pass 5d5e6fee5beb5d4d4d85e2203ef23e1230a9aac7 none (16-bit float) 0.68 GB (8 files) 312.8 tok/s 0.012 s / 0.015 s 1010 MB yes
mlx-community/SmolLM2-360M-Instruct-6bit pass 642affd1f9e387d1b56c745894afc83795aebe1d 6-bit 0.28 GB (8 files) 507.0 tok/s 0.011 s / 0.014 s 597 MB yes
mlx-community/SmolLM2-1.7B-Instruct pass 464b438b25894353a54836ec5d5a09157e62aa71 none (16-bit float) 3.19 GB (8 files) 115.7 tok/s 0.020 s / 0.073 s 3603 MB yes
mlx-community/SmolLM2-1.7B-Instruct-4bit fail (resolve) – – – – – – no
mlx-community/SmolLM2-360M-Instruct-4bit fail (resolve) – – – – – – no

# Licenses and re-hosting

redistributable in the catalog is true only for "Yes". The others are downloaded from Hugging Face only, pending the owner's decision.

Model License May the owner re-host the weights? Conditions (summary, not legal advice) License tag on the Hub card
Qwen3-0.6B-4bit Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
Qwen3-1.7B-4bit Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
Qwen3-4B-4bit Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
Qwen2.5-0.5B-Instruct-4bit Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
gemma-3-1b-it-qat-4bit Gemma Terms of Use Conditional Redistribution is allowed only if recipients are bound by the same use restrictions (Prohibited Use Policy), receive a copy of the Gemma Terms, and modified files carry a notice. Served from Hugging Face only until the owner decides. gemma
gemma-3-4b-it-qat-4bit Gemma Terms of Use Conditional Redistribution is allowed only if recipients are bound by the same use restrictions (Prohibited Use Policy), receive a copy of the Gemma Terms, and modified files carry a notice. Served from Hugging Face only until the owner decides. other
Llama-3.2-1B-Instruct-4bit Llama 3.2 Community License Conditional Redistribution is allowed only with a copy of the license, the “Built with Llama” attribution shown prominently, the notice file kept, and the Acceptable Use Policy respected. Served from Hugging Face only until the owner decides. llama3.2
Llama-3.2-3B-Instruct-4bit Llama 3.2 Community License Conditional Redistribution is allowed only with a copy of the license, the “Built with Llama” attribution shown prominently, the notice file kept, and the Acceptable Use Policy respected. Served from Hugging Face only until the owner decides. llama3.2
Phi-4-mini-instruct-4bit MIT Yes Keep the copyright and permission notice with the files. mit
SmolLM2-135M-Instruct Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
SmolLM2-135M-Instruct-8bit Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
SmolLM2-360M-Instruct Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
SmolLM2-360M-Instruct-6bit Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0
SmolLM2-1.7B-Instruct Apache-2.0 Yes Keep the license text and any NOTICE with the files; state changes if the weights are modified. apache-2.0

# What each repository actually is

Read from the downloaded config.json and from the Hub API, not assumed from the name.

Model model_type Context (config) Vision tower Base model on the Hub card Recalled the fact in turn 2 Thinking block Tool call Image input Load time Weights in memory
Qwen3-0.6B-4bit qwen3 40960 no Qwen/Qwen3-0.6B yes yes yes not run 0.14 s 320 MB
Qwen3-1.7B-4bit qwen3 40960 no Qwen/Qwen3-1.7B yes yes yes not run 0.10 s 923 MB
Qwen3-4B-4bit qwen3 40960 no Qwen/Qwen3-4B yes yes yes not run 0.12 s 2160 MB
Qwen2.5-0.5B-Instruct-4bit qwen2 32768 no Qwen/Qwen2.5-0.5B yes not run no not run 0.11 s 268 MB
gemma-3-1b-it-qat-4bit gemma3_text 32768 no google/gemma-3-1b-it yes not run not run not run 0.26 s 702 MB
gemma-3-4b-it-qat-4bit gemma3 131072 yes OpenGVLab/InternVL3-1B-Instruct yes not run not run yes 0.28 s 2860 MB
Llama-3.2-1B-Instruct-4bit llama 131072 no mlx-community/Llama-3.2-1B-Instruct-bf16 yes not run yes not run 0.13 s 663 MB
Llama-3.2-3B-Instruct-4bit llama 131072 no mlx-community/Llama-3.2-3B-Instruct yes not run yes not run 0.14 s 1724 MB
Phi-4-mini-instruct-4bit phi3 131072 no microsoft/Phi-4-mini-instruct yes not run no not run 0.18 s 2058 MB
SmolLM2-135M-Instruct llama 8192 no HuggingFaceTB/SmolLM2-135M-Instruct no not run not run not run 0.03 s 258 MB
SmolLM2-135M-Instruct-8bit llama 8192 no mlx-community/SmolLM2-135M-Instruct no not run not run not run 0.04 s 138 MB
SmolLM2-360M-Instruct llama 8192 no HuggingFaceTB/SmolLM2-360M-Instruct yes not run not run not run 0.04 s 691 MB
SmolLM2-360M-Instruct-6bit llama 8192 no HuggingFaceTB/SmolLM2-360M-Instruct yes not run not run not run 0.05 s 284 MB
SmolLM2-1.7B-Instruct llama 8192 no HuggingFaceTB/SmolLM2-1.7B-Instruct yes not run not run not run 0.15 s 3264 MB

"not run" in the three probe columns means the chat template does not declare the feature (or there is no vision tower), so nothing was attempted. "no" means it was attempted and did not work.

# Failures and exclusions

  • mlx-community/gemma-3-4b-it-qat-4bit: ran correctly but is kept out of the catalog. Provenance unclear: the Hub card declares license “other” and base model “OpenGVLab/InternVL3-1B-Instruct”, which does not match a Gemma 3 model (config model_type “gemma3”).
  • mlx-community/SmolLM2-1.7B-Instruct-4bit: failed at resolve: Hugging Face has no public repository named “mlx-community/SmolLM2-1.7B-Instruct-4bit” (HTTP 401).
  • mlx-community/SmolLM2-360M-Instruct-4bit: failed at resolve: Hugging Face has no public repository named “mlx-community/SmolLM2-360M-Instruct-4bit” (HTTP 401).

# Sample output

First turn of the verification chat ("My name is Sam and my favourite number is 42. Greet me by name in one short sentence."), temperature 0.

  • Qwen3-0.6B-4bit: Hello, I'm Sam, and my favorite number is 42! 😊
  • Qwen3-1.7B-4bit: Hello, Sam! You love the number 42.
  • Qwen3-4B-4bit: Hello, Sam! It's great to see you. Your favorite number is 42.
  • Qwen2.5-0.5B-Instruct-4bit: Hello, my dear Sam! Welcome to my home, and I am glad to meet you.
  • gemma-3-1b-it-qat-4bit: Sam, hello!
  • gemma-3-4b-it-qat-4bit: Hello Sam, it’s great to meet you – and a big fan of 42 too!
  • Llama-3.2-1B-Instruct-4bit: Hello Sam.
  • Llama-3.2-3B-Instruct-4bit: Hi Sam, nice to meet you, and I see you're a fan of the iconic number 42!
  • Phi-4-mini-instruct-4bit: Hello Sam!
  • SmolLM2-135M-Instruct: Sam, welcome to my little corner of the world. I'm glad you're here.
  • SmolLM2-135M-Instruct-8bit: Sam, welcome to my little corner of the world. I'm glad you're here.
  • SmolLM2-360M-Instruct: Hello, my name is Sam.
  • SmolLM2-360M-Instruct-6bit: Sam
  • SmolLM2-1.7B-Instruct: "Hello Sam, nice to meet you!"

# Not yet verified on a physical iPhone

Nothing in this document was run on an iPhone or iPad. In particular, the following are unknown until a device run:

  • Whether each model loads and generates on iOS at all (MLX does not run in the iOS Simulator, so there is no simulator result either).
  • Tokens per second, time to first token and peak memory on any iPhone. The Mac figures above come from a machine with far more memory bandwidth and must not be shown as iPhone figures.
  • Behaviour under the iOS memory limit (jetsam): the minimumDeviceMemoryGB and estimatedRuntimeMemoryBytes fields of the catalog are estimates computed from the weight size, a KV-cache formula at 4 096 tokens and a fixed overhead. They are not measurements.
  • Thermal behaviour during long generations, and the effect of the thermal policy.
  • Downloads on a cellular connection, and resuming after iOS suspends the app (downloads use a foreground session).
  • Whether the larger models (2 GB and more of weights) are usable on 8 GB devices in practice.

# Apple Foundation Models (system model): verified on this Mac only

  • macOS, verified. On this Mac (macOS 27.0) the live test appleSystemModelStreamsAMultiTurnAnswer (ASTRA_APPLE_FM_TESTS=1) ran on 2026-11-22: a three-turn conversation replayed through a Transcript streamed an answer that recalled a fact from the first turn, ended with one .messageCompleted, and the framework reported token usage (86 input, 7 output), which the adapter passed on. Cancellation with the system model was not exercised.
  • iOS Simulator, failed. A generation was attempted in the iOS Simulator (iOS 26.5 runtime, Mac host on macOS 27) on 2026-11-21. SystemLanguageModel.default.availability reported .available, yet the generation failed with LanguageModelSession.GenerationError code -1. Availability in the Simulator is therefore not a reliable signal; the app now says so in its error message when an unrecognised failure happens there.
  • iPhone / iPad, NOT verified. Nothing was run on an Apple Intelligence-eligible device. It needs one.

The verified record of each catalog entry says "platform": "macOS" for this reason. It should be replaced by an iOS record only after a real run on the device it names.

Files main
Menu

Theme