spb/zyquo-mlx Public MIT

1<!--2  MODELS.md3  Zyquo MLX45  Author: Simon-Pierre Boucher6  Mail: contact@spboucher.ai7-->89# Zyquo MLX — Hub Integration & Curated Model Catalog1011> Phase 0.C research document. Every catalog row was **live-verified against the12> Hugging Face Hub API on 2026-07-30** (HTTP 200, sizes summed from real tree13> listings). Anything computed rather than measured is marked *estimated*.1415---1617## 1. Hugging Face Hub HTTP API1819### 1.1 Search — `GET https://huggingface.co/api/models`2021Verified working parameter set:2223```24?author=mlx-community&pipeline_tag=text-generation&filter=mlx25&search=Qwen3&sort=downloads&direction=-1&limit=100&full=true&config=true26```2728| Param | Behavior (verified) |29|---|---|30| `search` | substring match on repo name |31| `author` | org/user scope (`mlx-community`, `lmstudio-community`, …) |32| `filter` | tag filter; `filter=mlx` matches the MLX library tag across all orgs |33| `pipeline_tag` | `text-generation`, `image-text-to-text` (VLM), `feature-extraction` (embeddings), `automatic-speech-recognition`, `image-to-image` |34| `sort` + `direction=-1` | `downloads`, `likes`, `lastModified`, `trendingScore` (default) |35| `limit` | up to **1000** per page; pagination via `Link: <…cursor=…>; rel="next"` response header (cursor-based) |36| `full=true` | adds `siblings` (filenames only — **no sizes**) + `sha` |37| `config=true` | adds `config` incl. `quantization_config` (bits/group_size), `architectures`, `model_type` |3839List-item fields: `id, author, downloads (30-day rolling), likes, tags, gated,40lastModified, createdAt, pipeline_tag, library_name, private`.41`safetensors` parameter counts are **not** in search results — only on the42per-model endpoint.4344**Rate limits (anonymous, from live response headers):**45`ratelimit-policy: "fixed window";"api";q=500;w=300` → **500 API requests per465 minutes**; `ratelimit: "api";r=…;t=…` gives remaining/reset. Design47`HubService` to batch (one 1000-item search call) and cache.4849### 1.2 Model info & file listing5051- `GET /api/models/{repo_id}` → `sha` (commit, for revision pinning),52  `siblings`, `usedStorage`, `safetensors: {parameters: {BF16: n, U32: n}, total}`,53  `gated` (`false` | `"manual"`), `cardData`.54- ⚠️ **Quantized-parameter quirk (verified):** MLX packs 4-bit weights as U32,55  so `safetensors.total` undercounts ~8×. Real params ≈ `BF16 + U32 × 8`56  (Qwen3-4B-4bit: 125.9M BF16 + 502.8M U32 → ≈ 4.1B ✓). `ModelStore` must use57  this formula or parse the repo name.58- `GET /api/models/{repo_id}/tree/main?recursive=true` → per-file59  `{type, path, size, lfs: {oid, size, pointerSize}}`.60  - **Not recursive by default** — always pass `recursive=true` (FLUX repos61    keep components in subdirectories).62  - `size` is always the true byte size; `lfs.oid` is the file's SHA-256 —63    use it as the integrity check after download.64  - Revision pinning: `/tree/{commit_sha}?recursive=true`. 1000 entries per65    page, `Link rel="next"` beyond.6667### 1.3 Download / resolve (drives `DownloadManager`)6869- `GET https://huggingface.co/{repo}/resolve/{revision}/{filename}` → **302**70  to a signed, expiring CDN URL (Xet bridge, `us.aws.cdn.hf.co/xet-bridge-us/…`).71  Small files (e.g. `config.json`) → **307** to `/api/resolve-cache/…`.72- `HEAD` on the resolve URL (before redirect) yields: `x-linked-size` (exact73  size), `x-linked-etag` (content SHA-256), `x-repo-commit` (resolved commit —74  **pin all subsequent files of a download to it** for a consistent snapshot),75  `accept-ranges: bytes`.76- **Resume (verified):** `Range: bytes={downloaded}-` on a *fresh* resolve77  request (CDN URLs expire — never persist them) → HTTP 206 with78  `content-range`. This is the resumable-download contract.79- **Auth / gated repos:** `Authorization: Bearer hf_…` works on both `/api/*`80  and `/resolve/*`. Gated repos report `gated: "manual"` and return 40181  anonymously (verified with `meta-llama/Llama-3.1-8B-Instruct`). All82  `mlx-community` catalog repos verified `gated: false`.8384### 1.4 MLX ecosystem conventions (census of top-1000 mlx-community repos)8586- Tags: library tag `mlx`, region tags `4-bit`/`8-bit`,87  `base_model:quantized:{original-repo}`.88- Name-suffix census (top-1000 by downloads): `-4bit` ×400, `-8bit` ×176,89  `-bf16` ×104, `-mlx`/`-MLX` ×99, `-6bit` ×70, `-qat` ×50, `mxfp4` ×33,90  `-5bit` ×24, `-3bit` ×22, `-DWQ` ×18, `-fp16` ×16, whisper-style `-q4/-q8` ×17.91- `config.json` `"quantization"`: `{bits, group_size, mode}` — 2026 repos say92  `"affine"`; `"mxfp4"` (gpt-oss, group_size 32) and mixed recipes93  (`MXFP4-Q8`, `DQ4plus-q8`) also occur. `-DWQ` = distilled-weight quantization94  (same size, better quality).95- Ecosystem state mid-2026 (from live download charts): current generations96  are **Qwen3.5 / Qwen3.6, gemma-4, GLM-4.7/5.2, Kimi-K2.5/K2.6, DeepSeek-V4,97  Mistral-Small-4, FLUX.2-Klein, Qwen3-TTS/ASR, parakeet v3**. No Llama-4 in98  mlx-community. Many new text models carry `pipeline_tag: image-text-to-text`99  (natively multimodal).100101---102103## 2. Featured Catalog (live-verified 2026-07-30)104105Disk = exact sum of weight files from the live tree listing.106Min RAM = weights + ~20% overhead + OS headroom, rounded to a Mac tier107(*estimated — calibrate in Phase 7*). Downloads = 30-day count at verification.108109### 2.1 Text LLMs — small (0.5–3B)110111| Repo | Params | Quant | Disk | Min RAM | DLs |112|---|---|---|---|---|---|113| `mlx-community/Qwen3-0.6B-4bit` | 0.6B | 4-bit gs64 | 0.34 GB | 8 GB | 49.9k |114| `mlx-community/Llama-3.2-1B-Instruct-4bit` | 1B | 4-bit gs64 | 0.70 GB | 8 GB | 48.3k |115| `mlx-community/gemma-3-1b-it-qat-4bit` | 1B | 4-bit gs64 QAT | 0.73 GB | 8 GB | 33.2k |116| `mlx-community/Qwen3-1.7B-4bit` | 1.7B | 4-bit gs64 | 0.97 GB | 8 GB | 14.1k |117| `mlx-community/SmolLM3-3B-4bit` | 3B | 4-bit gs64 | 1.73 GB | 8 GB | 2.0k |118| `mlx-community/Llama-3.2-3B-Instruct-4bit` | 3B | 4-bit gs64 | 1.81 GB | 8 GB | 21.1k |119120### 2.2 Text LLMs — mid (4–9B)121122| Repo | Params | Quant | Disk | Min RAM | DLs |123|---|---|---|---|---|---|124| `mlx-community/Qwen3-4B-Instruct-2507-4bit` | 4B | 4-bit gs64 | 2.26 GB | 8 GB | 41.6k |125| `mlx-community/Mistral-7B-Instruct-v0.3-4bit` | 7B | 4-bit gs64 | 4.08 GB | 16 GB | 12.1k |126| `mlx-community/Qwen2.5-Coder-7B-Instruct-4bit` | 7B | 4-bit gs64 | 4.28 GB | 16 GB | 30.7k |127| `mlx-community/Llama-3.1-8B-Instruct-4bit` | 8B | 4-bit gs64 | 4.52 GB | 16 GB | 27.1k |128| `mlx-community/Qwen3-8B-4bit` | 8B | 4-bit gs64 | 4.61 GB | 16 GB | 32.3k |129| `mlx-community/gemma-4-e4b-it-4bit` | ~8B (eff. 4B) | 4-bit gs64 affine | 5.15 GB | 16 GB | 61.6k |130| `mlx-community/Qwen3.5-9B-4bit` | 9B | 4-bit gs64 affine | 5.95 GB | 16 GB | 22.8k |131132### 2.3 Text LLMs — large (14–35B, incl. MoE)133134| Repo | Params | Quant | Disk | Min RAM | DLs |135|---|---|---|---|---|---|136| `mlx-community/Qwen3-14B-4bit` | 14B | 4-bit gs64 | 8.31 GB | 16 GB | 40.3k |137| `mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit` | 14B | 4-bit gs64 | 8.31 GB | 16 GB | 41.8k |138| `mlx-community/gpt-oss-20b-MXFP4-Q8` | 20.9B MoE | MXFP4 gs32 + Q8 | 12.08 GB | 24 GB | 355.3k |139| `mlx-community/Mistral-Small-3.1-24B-Instruct-2503-4bit` | 24B | 4-bit gs64 | 14.09 GB | 24 GB | 27.5k |140| `mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit` | 24B | 4-bit gs64 affine | 15.10 GB | 24 GB | 114.0k |141| `mlx-community/Qwen3.6-27B-4bit` | 27B | 4-bit gs64 affine | 16.05 GB | 32 GB | 50.7k |142| `mlx-community/Qwen3-30B-A3B-Instruct-2507-4bit` | 30B-A3B MoE | 4-bit gs64 | 17.18 GB | 32 GB | 83.8k |143| `mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit` | 30B-A3B MoE | 4-bit gs64 | 17.18 GB | 32 GB | 7.4k |144| `mlx-community/gemma-4-31b-it-4bit` | 31B | 4-bit gs64 affine | 18.41 GB | 32 GB | 47.5k |145| `mlx-community/Qwen3-32B-4bit` | 32B | 4-bit gs64 | 18.43 GB | 32 GB | 2.1k |146| `mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bit` | 32B | 4-bit gs64 | 18.43 GB | 32 GB | 8.2k |147| `mlx-community/Qwen3.6-35B-A3B-4bit` | 35B-A3B MoE | 4-bit gs64 affine | 20.40 GB | 32 GB | 62.0k |148149MoE note: RAM is set by **total** params (all experts resident); speed by150active params. `Qwen3-30B-A3B` needs a 32 GB Mac but generates at ~3B speed —151the ideal recommendation for 32 GB machines.152153### 2.4 Vision-language models154155| Repo | Params | Quant | Disk | Min RAM | DLs |156|---|---|---|---|---|---|157| `mlx-community/Qwen3-VL-4B-Instruct-4bit` | 4B | 4-bit gs64 affine | 3.09 GB | 16 GB | 17.5k |158| `mlx-community/SmolVLM2-2.2B-Instruct-mlx` | 2.2B | bf16 | 4.49 GB | 16 GB | 365 |159| `mlx-community/Qwen2.5-VL-7B-Instruct-4bit` | 7B | 4-bit gs64 | 5.64 GB | 16 GB | 3.6k |160| `mlx-community/Qwen3-VL-8B-Instruct-4bit` | 8B | 4-bit gs64 affine | 5.76 GB | 16 GB | 2.3k |161| `mlx-community/pixtral-12b-4bit` | 12B | 4-bit gs64 | 7.14 GB | 16 GB | 276 |162| `mlx-community/gemma-3-12b-it-qat-4bit` | 12B | 4-bit gs64 QAT | 8.03 GB | 24 GB | 31.8k |163164### 2.5 Embeddings (MLX format)165166| Repo | Params | Quant | Disk | Min RAM | DLs |167|---|---|---|---|---|---|168| `mlx-community/all-MiniLM-L6-v2-4bit` | 22M | 4-bit gs64 | 0.01 GB | 8 GB | 1.7k |169| `mlx-community/nomicai-modernbert-embed-base-bf16` | 149M | bf16 | 0.30 GB | 8 GB | 5.2k |170| `mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ` | 0.6B | 4-bit DWQ | 0.34 GB | 8 GB | 15.3k |171| `mlx-community/bge-m3-mlx-fp16` | 568M | fp16 | 1.14 GB | 8 GB | 11.2k |172| `mlx-community/Qwen3-Embedding-4B-4bit-DWQ` | 4B | 4-bit DWQ | 2.26 GB | 8 GB | 2.4k |173174### 2.6 Speech175176| Repo | Params | Quant | Disk | Min RAM | DLs |177|---|---|---|---|---|---|178| `mlx-community/Qwen3-ASR-0.6B-8bit` | 0.6B | 8-bit gs64 affine | 1.01 GB | 8 GB | 75.3k |179| `mlx-community/whisper-large-v3-turbo` | 0.8B | fp16 | 1.61 GB | 8 GB | 77.8k |180| `mlx-community/parakeet-tdt-0.6b-v3` | 0.6B | bf16 | 2.51 GB | 8 GB | 1.33M |181| `mlx-community/whisper-large-v3-mlx` | 1.5B | fp16 | 3.08 GB | 8 GB | 26.6k |182183### 2.7 Image generation184185| Repo | Type | Quant | Disk | Min RAM | DLs |186|---|---|---|---|---|---|187| `mlx-community/FLUX.2-Klein-4B-4bit` | FLUX.2 Klein 4B | 4-bit (components in subdirs) | 4.61 GB | 16 GB | 575 |188| `argmaxinc/mlx-FLUX.1-schnell-4bit-quantized` | FLUX.1-schnell (DiffusionKit) | 4-bit | 7.03 GB | 16 GB | 7.3k |189| `dhairyashil/FLUX.1-schnell-mflux-4bit` | FLUX.1-schnell (mflux) | 4-bit | 9.61 GB | 24 GB | 606 |190191No stable-diffusion MLX repos exist in `mlx-community` (live search returned192zero). The mflux ecosystem lives under `mflux-community/` — pick per193mflux-version compatibility at implementation time.194195**Verification total: 43 repos checked, 0 failures, all `gated: false`.**196197---198199## 3. RAM Guidance Table (powers `MemoryAdvisor` badges)200201*Estimated — arithmetic + community guidance; calibrate on real runs in Phase 7.*202203Basis: 4-bit gs64 affine = 4 + 32/64 bits/weight = **4.5 bits ≈ 0.5625 B/param**204(confirmed by catalog: Qwen3-14B → 0.561 B/p, Qwen3-32B → 0.562 B/p);205bf16 = 2 B/param. macOS GPU working-set ceiling ≈ 70–75% of unified RAM; keep206model + KV cache + activations ≤ ~(RAM − 5 GB). KV cache ≈ 0.5–2 GB at 8k ctx207for 7–32B GQA models. LoRA (bf16 base) ≈ 2 B/param + activations + adapter208optimizer state. QLoRA (4-bit frozen base) ≈ 0.5625 B/param + activations209(~2–5 GB at batch 1–4, seq 1–2k, grad checkpointing on).210211| Mac RAM | 4-bit inference (realistic max) | LoRA FT (bf16 base) | QLoRA FT (4-bit base) |212|---|---|---|---|213| 8 GB | ≤3B comfortable; 4B tight | ≤0.6B | ≤1.7B |214| 16 GB | 7–9B comfortable; 14B ok w/ modest ctx | ≤3B | ≤7–8B |215| 24 GB | 14B comfortable; 20B MoE & 24B ok | ≤7B (tight) | ≤14B |216| 32 GB | 27–32B dense & 30B/35B-A3B MoE | ≤8–9B | 14B comfortable; 24B tight |217| 48 GB | 32–35B comfortable + long ctx; ~50B-class | ≤14B | ≤32B |218| 64 GB | 70B 4-bit (~39.4 GB; `Llama-3.3-70B-Instruct-4bit` verified to exist) | ≤24B | 32B comfortable |219| 96 GB | 70B comfortable; ~100–120B MoE | ≤32B | ≤70B (tight) |220| 128 GB | 120B-class MoE (e.g. `Mistral-Small-4-119B-2603-4bit`, exists); 70B 8-bit | ≤35B dense | 70B comfortable; ~120B MoE possible |221222**This Mac (dev machine): 48 GB** → 4-bit inference up to ~35B comfortable;223LoRA ≤14B; QLoRA ≤32B.224225Caveats for `MemoryAdvisor`:2261. Min-RAM columns are computed, not measured — Phase 7 calibrates them.2272. `downloads` is a 30-day rolling count (freshness signal, not lifetime).2283. For quantized repos derive real params as `BF16 + U32 × 8` from the229   `safetensors` field, or parse the name suffix.230