spb/zyquo-local Public MIT

1<!--2  MODELS.md3  Zyquo Local45  Author: Simon-Pierre Boucher6  Mail: contact@spboucher.ai7-->89# Hugging Face Hub Integration + Curated Catalog — Phase 0.B1011Researched live against huggingface.co on 2026-07-30. Every catalog size was12verified by summing `/tree/main?recursive=true` (decimal GB, matches Finder).13This file is the single source of truth for `ModelCatalog.swift` — the two14must never drift apart (updated together per methodology rule 7).1516## 1. HF Hub HTTP API contract (implemented by `HubService`)1718Base `https://huggingface.co`, plain `URLSession`, distinctive `User-Agent`19(`ZyquoLocal/<version>`), optional `Authorization: Bearer hf_…`.2021### Search — `GET /api/models`22Params: `author=mlx-community`, `search=<text>`, `pipeline_tag=text-generation`23(filters out whisper/embeddings noise), `filter=mlx`, `sort=downloads|likes|24createdAt|lastModified|trendingScore`, `direction=-1`, `limit≤100`,25`full=true` (adds filename-only siblings), `config=true` (adds26`config.model_type` → architecture-compat check).27Response items: `id`, `likes`, `downloads`, `private`, `gated`28(`false|"auto"|"manual"`), `tags[]`, `pipeline_tag`, `library_name`,29`createdAt` (ISO8601 fractional). **Pagination via `Link` response header30cursor** (`rel="next"`), no page numbers.3132### Model info + file sizes — `GET /api/models/{owner}/{repo}?blobs=true`33Cheapest single call for sizes: each sibling becomes `{rfilename, size,34lfs{sha256,size}?}`. Canonical alternative: `GET35/api/models/{id}/tree/main?recursive=true` (`{type:"file", size, path,36lfs?}`). **Never use `usedStorage`** (counts all revisions — verified 682 MB37reported vs 351 MB actual). `safetensors.total` param count is unreliable for38quantized repos (packed uint32).3940### Download — `GET /{repo}/resolve/main/{path}`41- LFS/large → **302** to signed, time-limited CDN URL; small files → **307**.42  Always follow redirects; re-resolve if a paused download's URL expired.43- Initial response headers (grab via HEAD): `x-linked-size` (real size),44  `x-linked-etag` (sha256 for LFS — used for integrity), `x-repo-commit`,45  `accept-ranges: bytes`.46- **Resume: `Range: bytes=N-` → 206 verified.**47- ⚠️ Swift gotcha: `URLSession` forwards `Authorization` across the cross-host48  redirect — strip it in `willPerformHTTPRedirection` when host changes.4950### Errors / limits51- Anonymous + missing repo → **401** (not 404!). Gated repo: API info 200 with52  `gated` set; `resolve` 401 anonymous / 403 token-without-grant → in-app hint53  to add an HF token in Settings.54- Rate limits (5-min windows): anonymous 500 API / 3 000 resolve per IP; free55  token 1 000 / 5 000. On **429** parse `RateLimit` header, back off exactly.5657## 2. mlx-community naming scheme5859`mlx-community/{BaseModel}-{quant}` — suffix semantics (≈ GB per B params):6061| Suffix | Meaning | ≈GB/B | Notes |62|---|---|---|---|63| `-4bit` | uniform 4-bit (group 64) | 0.57 | standard choice |64| `-5bit`/`-6bit` | uniform 5/6-bit | 0.7/0.8 | near-lossless at 6 |65| `-8bit` | uniform 8-bit | 1.1 | virtually lossless, 2× RAM |66| `-bf16` | unquantized | 2.1 | reference quality |67| `-4bit-DWQ` | distilled weight quant | 0.6 | 4-bit size ≈ 6-bit quality; prefer |68| `-qat-4bit` | vendor QAT (Gemma) | 0.6+ | best 4-bit Gemma quality |69| `OptiQ-4bit` | 2026 mixed 4/8-bit (KL-sensitivity) + often MTP head | 0.65 | ~1.4× decode speedup; prefer on Qwen3.5/3.6, gemma-4 |70| `MXFP4-Q8` | native MXFP4 MoE + 8-bit rest | — | intended format for gpt-oss |7172Weights rule of thumb: **GB ≈ params(B) × bits ÷ 8 × 1.1**.7374## 3. Featured catalog — 30 models, live-verified 2026-07-307576All `gated: false`, all architectures present in `LLMTypeRegistry`77(see MLX-RESEARCH.md §4). Min RAM = smallest tier that runs it comfortably78with useful context.7980### Tiny (≤4B)81| Repo ID | Params | Quant | GB | Min RAM | Tags | One-liner |82|---|---|---|---|---|---|---|83| `mlx-community/Qwen3-0.6B-4bit` | 0.6B | 4bit | 0.35 | 8 | tiny·general | Smallest useful chat model; instant loads. |84| `mlx-community/LFM2.5-1.2B-Instruct-4bit` | 1.2B | 4bit | 0.66 | 8 | tiny·general | Liquid AI's 2026 edge model; punchy and very fast. |85| `mlx-community/Llama-3.2-1B-Instruct-4bit` | 1B | 4bit | 0.71 | 8 | tiny·general | The classic 1B; most-downloaded tiny LLM. |86| `mlx-community/gemma-3-1b-it-qat-4bit` | 1B | QAT-4bit | 0.77 | 8 | tiny·general | Google QAT checkpoint — best quality-per-byte at 1B. |87| `mlx-community/Qwen3-1.7B-4bit` | 1.7B | 4bit | 0.98 | 8 | tiny·general·reasoning | Hybrid thinking modes in under 1 GB. |88| `mlx-community/SmolLM3-3B-4bit` | 3B | 4bit | 1.75 | 8 | tiny·general | HF's fully-open 3B; long context, optional reasoning. |89| `mlx-community/Llama-3.2-3B-Instruct-4bit` | 3B | 4bit | 1.82 | 8 | tiny·general | The default "runs anywhere" pick. |90| `mlx-community/Qwen3-4B-Instruct-2507-4bit` | 4B | 4bit | 2.28 | 8 | tiny·general | 2507 refresh — best ≤4B all-rounder. |91| `mlx-community/gemma-3-4b-it-qat-4bit` | 4B | QAT-4bit | 3.03 | 8 | tiny·general | Vision-capable 4B with QAT quality. |9293### Mid (7–20B)94| Repo ID | Params | Quant | GB | Min RAM | Tags | One-liner |95|---|---|---|---|---|---|---|96| `mlx-community/Llama-3.1-8B-Instruct-4bit` | 8B | 4bit | 4.53 | 16 | mid·general | The reference 8B; huge prompt/finetune ecosystem. |97| `mlx-community/Qwen3-8B-4bit` | 8B | 4bit | 4.62 | 16 | mid·general·reasoning | Best-selling 8B; thinking mode on demand. |98| `mlx-community/gemma-3-12b-it-qat-4bit` | 12B | QAT-4bit | 8.07 | 16 | mid·general | Sweet spot for 16 GB Macs; strong writing. |99| `mlx-community/Qwen3.5-9B-OptiQ-4bit` | 9B | OptiQ-4bit | 8.22 | 16 | mid·general | 2026 Qwen3.5 gen; top mid-size quality. |100| `mlx-community/phi-4-4bit` | 14.7B | 4bit | 8.26 | 16 | mid·general·reasoning | Microsoft dense 14B; excels at math/STEM. |101| `mlx-community/Qwen3-14B-4bit` | 14B | 4bit | 8.32 | 16 | mid·general·reasoning | Stronger sibling of Qwen3-8B; 16 GB flagship. |102| `mlx-community/gpt-oss-20b-MXFP4-Q8` | 20.9B MoE | MXFP4+Q8 | 12.10 | 16 (24 comfy) | mid·general·reasoning | OpenAI open-weights MoE; #1 download in the org. |103104### Large (24B+)105| Repo ID | Params | Quant | GB | Min RAM | Tags | One-liner |106|---|---|---|---|---|---|---|107| `mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit` | 24B | 4bit | 13.28 | 24 | large·general | Fast dense 24B, low hallucination, good tool use. |108| `mlx-community/gemma-3-27b-it-qat-4bit` | 27B | QAT-4bit | 16.87 | 32 | large·general | Gemma 3 flagship with QAT; superb chat quality. |109| `mlx-community/GLM-4.7-Flash-4bit` | 30B MoE | 4bit | 16.87 | 32 | large·general·coding | Zhipu's 2026 fast MoE; strong agentic/coding. |110| `mlx-community/Qwen3-30B-A3B-Instruct-2507-4bit` | 30B-A3B MoE | 4bit | 17.20 | 32 | large·general | 3B active params → big-model quality at small-model speed. |111| `mlx-community/Qwen3-32B-4bit` | 32B | 4bit | 18.45 | 32 | large·general·reasoning | Dense 32B with thinking; slower but deeper than the MoE. |112| `mlx-community/Qwen3.6-27B-OptiQ-4bit` | 27B | OptiQ-4bit | 20.00 | 32 | large·general | 2026 Qwen3.6 dense; MTP head ≈1.4× faster decode. |113| `mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit` | 35B-A3B MoE | OptiQ-4bit | 24.69 | 48 (32 tight) | large·general | 2026 successor to Qwen3-30B-A3B. |114| `mlx-community/Llama-3.3-70B-Instruct-4bit` | 70B | 4bit | 39.71 | 64+ | large·general | The 70B reference; 64 GB+ Macs only. |115116### Coding117| Repo ID | Params | Quant | GB | Min RAM | Tags | One-liner |118|---|---|---|---|---|---|---|119| `mlx-community/Qwen2.5-Coder-7B-Instruct-4bit` | 7B | 4bit | 4.30 | 16 (8 tight) | coding | The default small local code model. |120| `mlx-community/Qwen2.5-Coder-14B-Instruct-4bit` | 14B | 4bit | 8.32 | 16 | coding | Noticeably better completions on 16 GB Macs. |121| `mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit` | 30B-A3B MoE | 4bit | 17.20 | 32 | coding·large | Best local coding model for 32 GB; fast agentic loops. |122123### Reasoning124| Repo ID | Params | Quant | GB | Min RAM | Tags | One-liner |125|---|---|---|---|---|---|---|126| `mlx-community/DeepSeek-R1-0528-Qwen3-8B-4bit` | 8B | 4bit | 4.62 | 16 (8 tight) | reasoning | R1-0528 distill onto Qwen3-8B; `<think>` chain-of-thought. |127| `mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit` | 14B | 4bit | 8.32 | 16 | reasoning | Most-downloaded R1 distill; great math on 16 GB. |128| `mlx-community/Qwen3-30B-A3B-Thinking-2507-4bit` | 30B-A3B MoE | 4bit | 17.20 | 32 | reasoning·large | Current best local reasoner ≤32 GB. |129130Curation notes: `QwQ-32B-4bit` verified but superseded (near-zero recent131downloads); `Ministral-8B` and `R1-Distill-Qwen-7B` dropped as superseded by132newer entries; 100 GB+ 2026 giants (Kimi-K2.5 658 GB, GLM-5.2 418 GB,133DeepSeek-V4-Flash 152 GB) excluded — no consumer Mac fits them.134135## 4. RAM recommendation table (feeds MemoryAdvisor + badges)136137Needed RAM ≈ weights + KV cache (0.5–4 GB by context) + 4–6 GB macOS headroom.138GPU-wired ceiling ≈ 70–75 % of unified RAM → comfortable = weights ≤ ~60 %.139140| Mac RAM | Comfortable @4bit | Tight | Catalog examples |141|---|---|---|---|142| 8 GB | ≤4B (≤2.5 GB) | 7–8B (4.5 GB, short ctx) | Qwen3-4B-2507, Llama-3.2-3B |143| 16 GB | 7–14B (4.3–9 GB) | gpt-oss-20b (12.1 GB) | Qwen3-14B, gemma-3-12b-qat |144| 24 GB | 14B + long ctx; 24B (13.3 GB) | 27–30B (≈17 GB) | Mistral-Small-3.2 |145| 32 GB | 24–32B (13–18.5 GB) | 35B-A3B (24.7 GB) | Qwen3-30B-A3B, gemma-3-27b |146| 48 GB | 32–35B + long ctx | 70B (39.7 GB) | Qwen3.6-35B-A3B |147| 64 GB | 70B (39.7 GB) | — | Llama-3.3-70B |148| 128 GB | 70B @8bit / 100B MoE | ~150 GB repos: no | gpt-oss-120b |149