SPB Git

spb/zyquo-cloud Public MIT

Native macOS AI chat client for 12 cloud providers — your keys, every cloud model, one beautiful chat.

macosswiftaichat
Swift 97.4% Shell 1.7% Makefile 1%

phase7: COMPLETE — 202/202 live tests green; disable Together gemma vision (empty answers); universal build fix for CLT

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simon-pierre boucher committed 11 days ago (Jul 30, 2026) parent 1616978

Showing 5 changed files with +202 and −178

modified Makefile +12 −5
@@ -56,22 +56,29 @@ bundle:
56 56 @for r in Resources/MenuBarIcon.png Resources/MenuBarIcon@2x.png; do [ -f "$$r" ] && cp "$$r" "$(APP_DIR)/Contents/Resources/" || true; done
57 57 scripts/write-info-plist.sh "$(APP_DIR)" "$(APP_NAME)" "$(EXEC_NAME)" "$(BUNDLE_ID)" "$(VERSION)" "$(BUILD_NUM)" "$(MIN_MACOS)"
58 58
59 +# Both arch builds land in .build/out/Products/Release with this build system,
60 +# so each slice is copied aside before lipo. The -U flag allows the
61 +# swiftCompatibility56 force-load symbol (CLT ships it arm64-only; it is never
62 +# referenced at runtime on macOS 13+).
59 63 universal:
60 64 @echo "=== Building universal binary (arm64 + x86_64) ==="
61 − swift build -c release --arch arm64
62 − swift build -c release --arch x86_64
63 65 mkdir -p .build/universal
66 + swift build -c release --arch arm64
67 + cp .build/out/Products/Release/$(EXEC_NAME) .build/universal/$(EXEC_NAME).arm64
68 + swift build -c release --arch x86_64 -Xlinker -U -Xlinker '__swift_FORCE_LOAD_$$_swiftCompatibility56'
69 + cp .build/out/Products/Release/$(EXEC_NAME) .build/universal/$(EXEC_NAME).x86_64
64 70 lipo -create \
65 − .build/arm64-apple-macosx/release/$(EXEC_NAME) \
66 − .build/x86_64-apple-macosx/release/$(EXEC_NAME) \
71 + .build/universal/$(EXEC_NAME).arm64 \
72 + .build/universal/$(EXEC_NAME).x86_64 \
67 73 -output .build/universal/$(EXEC_NAME)
74 + lipo -archs .build/universal/$(EXEC_NAME)
68 75
69 76 release: universal
70 77 @echo "=== Assembling $(APP_DIR) (universal) ==="
71 78 rm -rf "$(APP_DIR)"
72 79 mkdir -p "$(APP_DIR)/Contents/MacOS" "$(APP_DIR)/Contents/Resources"
73 80 cp .build/universal/$(EXEC_NAME) "$(APP_DIR)/Contents/MacOS/$(EXEC_NAME)"
74 − @for b in .build/arm64-apple-macosx/release/*.bundle; do [ -e "$$b" ] && cp -R "$$b" "$(APP_DIR)/Contents/Resources/" || true; done
81 + @for b in .build/out/Products/Release/*.bundle; do [ -e "$$b" ] && cp -R "$$b" "$(APP_DIR)/Contents/Resources/" || true; done
75 82 @if [ -f Resources/AppIcon.icns ]; then cp Resources/AppIcon.icns "$(APP_DIR)/Contents/Resources/AppIcon.icns"; fi
76 83 @for r in Resources/MenuBarIcon.png Resources/MenuBarIcon@2x.png; do [ -f "$$r" ] && cp "$$r" "$(APP_DIR)/Contents/Resources/" || true; done
77 84 scripts/write-info-plist.sh "$(APP_DIR)" "$(APP_NAME)" "$(EXEC_NAME)" "$(BUNDLE_ID)" "$(VERSION)" "$(BUILD_NUM)" "$(MIN_MACOS)"
modified Sources/ZyquoCloud/Services/ModelCatalogData.swift +3 −1
@@ -1059,9 +1059,11 @@ enum ModelCatalogData {
1059 1059 parameterSupport: .openAIDefault
1060 1060 ),
1061 1061 AIModel(
1062 + // vision disabled 2026-07-30: Together's endpoint accepts image
1063 + // input but streams an empty answer (verified live) — text only.
1062 1064 id: "google/gemma-4-31B-it", provider: .together, displayName: "Gemma 4 31B",
1063 1065 contextWindow: 262_144, maxOutputTokens: nil,
1064 − capabilities: ModelCapabilities(vision: true, tools: true, jsonMode: true),
1066 + capabilities: ModelCapabilities(vision: false, tools: true, jsonMode: true),
1065 1067 pricing: ModelPricing(inputPerMTok: 0.39, outputPerMTok: 0.97),
1066 1068 parameterSupport: ParameterSupport(frequencyPenalty: true, presencePenalty: true, requiresStreaming: true)
1067 1069 ),
modified docs/PLAN.md +22 −7
@@ -109,13 +109,28 @@ plugin required by SDK 27's macro-based @State.
109 109 **Phase 6 checkpoint summary:** All Phase 6 features implemented and wired. UI verified running;
110 110 light/dark render from the same tokens. End-to-end chat with live keys is exercised in Phase 7.
111 111
112 −## Phase 7 — API Verification (in progress)
113 −
114 −- [ ] Verify harness: app binary gains `--verify` mode reusing the exact production provider clients;
115 − `zyquo-verify` target execs it (keys via env from .env.keys, never committed)
116 −- [ ] /models diff vs catalog per provider; "Reply with exactly: OK" on every catalog chat model;
117 − streaming test ≥1 model/provider; vision test on 1 vision model/provider
118 −- [ ] Results table in docs/VERIFICATION.md; fix every failure (update PROVIDERS.md + catalog together)
112 +## Phase 7 — API Verification ✅ (completed 2026-07-30 — table fully green)
113 +
114 +- [x] Verify harness: app binary gains `--verify` mode reusing the exact production provider clients;
115 + `zyquo-verify` target execs it (keys via env from .env.keys, never committed); `--load-vault` seeds the GUI vault
116 +- [x] /models diff vs catalog per provider; "OK" completion on every catalog chat model;
117 + streaming test per provider; vision test per vision-capable provider
118 +- [x] **Final result: 202/202 tests passed** (12 providers, 170 catalog models) — docs/VERIFICATION.md
119 +- [x] Every failure across 3 sweeps fixed, PROVIDERS.md + catalog updated together
120 +
121 +**Phase 7 checkpoint summary:** Live verification caught and fixed real bugs: SSE event separators
122 +dropped by AsyncBytes.lines (all streams hung); async @main breaking GUI task scheduling; a --verify
123 +MainActor deadlock; Together's completions-style `choices[].text` deltas; Mistral's high/none-only
124 +reasoning_effort + ThinkChunk content arrays; Gemini's `models/` ID prefixes; deprecated models
125 +(OpenAI chat-latest generations, Gemini 2.0/3-pro-preview, 18 Together dedicated-only, broken
126 +DeepInfra gemma-4-31B) removed; streaming-only models flagged (`requiresStreaming`); 429/5xx
127 +backoff added. End-to-end GUI chat verified live; auto-title now uses cheapest non-reasoning model.
128 +
129 +## Phase 8 — Signing & Notarization (in progress)
130 +
131 +- [ ] `make release`: universal binary (arm64 + x86_64 via lipo; x86_64 needs `-U` for CLT's
132 + arm64-only swiftCompatibility56), Developer ID sign (hardened runtime), notarize
133 + (`MacLustr-Notarize`), staple, `spctl` verify, DMG
119 134 ## Phase 2 — Architecture (pending)
120 135 ## Phase 3 — SecureKeyStore (pending)
121 136 ## Phase 4 — Design System (pending)
modified docs/PROVIDERS.md +2 −0
@@ -1715,3 +1715,5 @@ Live-key verification (Phase 7 harness) against the real APIs produced the follo
1715 1715 broken/cold endpoint, reproduced with curl.
1716 1716 - `gpt-5.3-chat-latest` returned an empty completion once at a 64-token cap; verified fine at 512
1717 1717 (`reasoning_tokens: 0`) — harness bumped its non-reasoning budget to 256.
1718 +- **Together `google/gemma-4-31B-it` vision disabled**: the endpoint accepts image parts but streams
1719 + an empty answer (HTTP 200, zero text — reproduced with curl); flagged text-only in the catalog.
modified docs/VERIFICATION.md +163 −165
@@ -5,213 +5,211 @@
5 5 Mail: contact@spboucher.ai
6 6 -->
7 7
8 −# API Verification Results — 2026-07-30T12:47:26Z
8 +# API Verification Results — 2026-07-30T12:54:13Z
9 9
10 −**204 tests · 199 passed · 5 failed**
10 +**202 tests · 202 passed · 0 failed**
11 11
12 12 | Provider | Model | Test | Result | Latency | Detail |
13 13 |---|---|---|---|---|---|
14 −| Anthropic | `—` | models | ✅ | 0.3s | 11 live |
15 −| Anthropic | `claude-fable-5` | chat | ✅ | 2.5s | |
16 −| Anthropic | `claude-haiku-4-5-20251001` | chat | ✅ | 0.7s | |
14 +| Anthropic | `—` | models | ✅ | 0.2s | 11 live |
15 +| Anthropic | `claude-fable-5` | chat | ✅ | 2.9s | |
16 +| Anthropic | `claude-haiku-4-5-20251001` | chat | ✅ | 0.6s | |
17 17 | Anthropic | `claude-opus-4-1-20250805` | chat | ✅ | 1.9s | |
18 −| Anthropic | `claude-opus-4-5-20251101` | chat | ✅ | 1.3s | |
19 −| Anthropic | `claude-opus-4-6` | chat | ✅ | 2.2s | |
20 −| Anthropic | `claude-opus-4-7` | chat | ✅ | 1.9s | |
21 −| Anthropic | `claude-opus-4-8` | chat | ✅ | 2.8s | |
22 −| Anthropic | `claude-opus-5` | chat | ✅ | 2.4s | |
18 +| Anthropic | `claude-opus-4-5-20251101` | chat | ✅ | 1.4s | |
19 +| Anthropic | `claude-opus-4-6` | chat | ✅ | 12.9s | |
20 +| Anthropic | `claude-opus-4-7` | chat | ✅ | 2.8s | |
21 +| Anthropic | `claude-opus-4-8` | chat | ✅ | 1.1s | |
22 +| Anthropic | `claude-opus-5` | chat | ✅ | 2.2s | |
23 23 | Anthropic | `claude-opus-5` | stream | ✅ | 2.2s | deltas=2 usage=true |
24 24 | Anthropic | `claude-opus-5` | vision | ✅ | 2.6s | Red |
25 −| Anthropic | `claude-sonnet-4-5-20250929` | chat | ✅ | 1.6s | |
26 −| Anthropic | `claude-sonnet-4-6` | chat | ✅ | 1.3s | |
27 −| Anthropic | `claude-sonnet-5` | chat | ✅ | 1.3s | |
28 −| Cerebras | `—` | models | ✅ | 0.2s | 3 live |
25 +| Anthropic | `claude-sonnet-4-5-20250929` | chat | ✅ | 1.7s | |
26 +| Anthropic | `claude-sonnet-4-6` | chat | ✅ | 1.1s | |
27 +| Anthropic | `claude-sonnet-5` | chat | ✅ | 1.4s | |
28 +| Cerebras | `—` | models | ✅ | 0.1s | 3 live |
29 29 | Cerebras | `gemma-4-31b` | chat | ✅ | 0.3s | |
30 −| Cerebras | `gemma-4-31b` | vision | ✅ | 0.3s | Red |
30 +| Cerebras | `gemma-4-31b` | vision | ✅ | 0.1s | Red |
31 31 | Cerebras | `gpt-oss-120b` | chat | ✅ | 0.1s | |
32 −| Cerebras | `gpt-oss-120b` | stream | ✅ | 0.1s | deltas=1 usage=true |
33 −| Cerebras | `zai-glm-4.7` | chat | ✅ | 0.2s | |
34 −| DeepInfra | `—` | models | ✅ | 0.6s | 174 live |
35 −| DeepInfra | `MiniMaxAI/MiniMax-M3` | chat | ✅ | 1.0s | |
36 −| DeepInfra | `Qwen/Qwen3-235B-A22B-Instruct-2507` | chat | ✅ | 0.3s | |
32 +| Cerebras | `gpt-oss-120b` | stream | ✅ | 0.3s | deltas=1 usage=true |
33 +| Cerebras | `zai-glm-4.7` | chat | ✅ | 0.1s | |
34 +| DeepInfra | `—` | models | ✅ | 0.4s | 174 live |
35 +| DeepInfra | `MiniMaxAI/MiniMax-M3` | chat | ✅ | 2.1s | |
36 +| DeepInfra | `Qwen/Qwen3-235B-A22B-Instruct-2507` | chat | ✅ | 0.2s | |
37 37 | DeepInfra | `Qwen/Qwen3-235B-A22B-Thinking-2507` | chat | ✅ | 1.0s | |
38 38 | DeepInfra | `Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo` | chat | ✅ | 0.3s | |
39 39 | DeepInfra | `Qwen/Qwen3-VL-235B-A22B-Instruct` | chat | ✅ | 0.3s | |
40 −| DeepInfra | `Qwen/Qwen3.5-397B-A17B` | chat | ✅ | 3.3s | |
41 −| DeepInfra | `Qwen/Qwen3.7-Max` | chat | ✅ | 5.7s | |
42 −| DeepInfra | `anthropic/claude-fable-5` | chat | ✅ | 2.3s | |
43 −| DeepInfra | `anthropic/claude-fable-5` | stream | ✅ | 1.7s | deltas=4 usage=true |
44 −| DeepInfra | `anthropic/claude-fable-5` | vision | ✅ | 1.8s | Red |
40 +| DeepInfra | `Qwen/Qwen3.5-397B-A17B` | chat | ✅ | 3.0s | |
41 +| DeepInfra | `Qwen/Qwen3.7-Max` | chat | ✅ | 4.0s | |
42 +| DeepInfra | `anthropic/claude-fable-5` | chat | ✅ | 1.9s | |
43 +| DeepInfra | `anthropic/claude-fable-5` | stream | ✅ | 1.6s | deltas=4 usage=true |
44 +| DeepInfra | `anthropic/claude-fable-5` | vision | ✅ | 1.7s | Red |
45 45 | DeepInfra | `anthropic/claude-haiku-4-5` | chat | ✅ | 0.9s | |
46 −| DeepInfra | `anthropic/claude-opus-4-8` | chat | ✅ | 1.9s | |
47 −| DeepInfra | `anthropic/claude-opus-5` | chat | ✅ | 1.7s | |
48 −| DeepInfra | `anthropic/claude-sonnet-5` | chat | ✅ | 1.5s | |
49 −| DeepInfra | `deepseek-ai/DeepSeek-R1-0528` | chat | ✅ | 1.5s | |
46 +| DeepInfra | `anthropic/claude-opus-4-8` | chat | ✅ | 1.6s | |
47 +| DeepInfra | `anthropic/claude-opus-5` | chat | ✅ | 1.5s | |
48 +| DeepInfra | `anthropic/claude-sonnet-5` | chat | ✅ | 1.6s | |
49 +| DeepInfra | `deepseek-ai/DeepSeek-R1-0528` | chat | ✅ | 1.9s | |
50 50 | DeepInfra | `deepseek-ai/DeepSeek-V3.1` | chat | ✅ | 0.4s | |
51 −| DeepInfra | `deepseek-ai/DeepSeek-V4-Flash` | chat | ✅ | 1.4s | |
52 −| DeepInfra | `deepseek-ai/DeepSeek-V4-Pro` | chat | ✅ | 0.4s | |
51 +| DeepInfra | `deepseek-ai/DeepSeek-V4-Flash` | chat | ✅ | 0.7s | |
52 +| DeepInfra | `deepseek-ai/DeepSeek-V4-Pro` | chat | ✅ | 0.5s | |
53 53 | DeepInfra | `google/gemini-2.5-flash` | chat | ✅ | 1.3s | |
54 54 | DeepInfra | `google/gemini-2.5-pro` | chat | ✅ | 5.8s | |
55 −| DeepInfra | `google/gemini-3.1-flash-lite` | chat | ✅ | 1.2s | |
56 −| DeepInfra | `google/gemini-3.1-pro` | chat | ✅ | 3.9s | |
57 −| DeepInfra | `google/gemini-3.5-flash` | chat | ✅ | 2.3s | |
58 −| DeepInfra | `google/gemma-4-31B-it` | chat | ❌ | 121.0s | Network error: The request timed out. |
59 −| DeepInfra | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | chat | ✅ | 0.8s | |
60 −| DeepInfra | `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | chat | ✅ | 0.3s | |
61 −| DeepInfra | `meta-llama/Llama-4-Scout-17B-16E-Instruct` | chat | ✅ | 0.3s | |
62 −| DeepInfra | `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | chat | ✅ | 0.3s | |
63 −| DeepInfra | `mistralai/Mistral-Small-3.2-24B-Instruct-2506` | chat | ✅ | 1.2s | |
64 −| DeepInfra | `moonshotai/Kimi-K2.5` | chat | ✅ | 3.1s | |
65 −| DeepInfra | `moonshotai/Kimi-K2.6` | chat | ✅ | 2.8s | |
66 −| DeepInfra | `moonshotai/Kimi-K2.7-Code` | chat | ✅ | 0.7s | |
67 −| DeepInfra | `nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B` | chat | ✅ | 0.5s | |
68 −| DeepInfra | `openai/gpt-oss-120b` | chat | ✅ | 1.2s | |
69 −| DeepInfra | `openai/gpt-oss-20b` | chat | ✅ | 0.4s | |
70 −| DeepInfra | `zai-org/GLM-4.7` | chat | ✅ | 2.5s | |
71 −| DeepInfra | `zai-org/GLM-5.2` | chat | ✅ | 1.4s | |
55 +| DeepInfra | `google/gemini-3.1-flash-lite` | chat | ✅ | 4.1s | |
56 +| DeepInfra | `google/gemini-3.1-pro` | chat | ✅ | 4.3s | |
57 +| DeepInfra | `google/gemini-3.5-flash` | chat | ✅ | 2.6s | |
58 +| DeepInfra | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | chat | ✅ | 0.4s | |
59 +| DeepInfra | `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | chat | ✅ | 0.2s | |
60 +| DeepInfra | `meta-llama/Llama-4-Scout-17B-16E-Instruct` | chat | ✅ | 0.2s | |
61 +| DeepInfra | `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | chat | ✅ | 0.4s | |
62 +| DeepInfra | `mistralai/Mistral-Small-3.2-24B-Instruct-2506` | chat | ✅ | 0.4s | |
63 +| DeepInfra | `moonshotai/Kimi-K2.5` | chat | ✅ | 1.2s | |
64 +| DeepInfra | `moonshotai/Kimi-K2.6` | chat | ✅ | 3.2s | |
65 +| DeepInfra | `moonshotai/Kimi-K2.7-Code` | chat | ✅ | 1.4s | |
66 +| DeepInfra | `nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B` | chat | ✅ | 3.6s | |
67 +| DeepInfra | `openai/gpt-oss-120b` | chat | ✅ | 0.9s | |
68 +| DeepInfra | `openai/gpt-oss-20b` | chat | ✅ | 0.6s | |
69 +| DeepInfra | `zai-org/GLM-4.7` | chat | ✅ | 0.6s | |
70 +| DeepInfra | `zai-org/GLM-5.2` | chat | ✅ | 3.2s | |
72 71 | DeepSeek | `—` | models | ✅ | 0.4s | 2 live |
73 −| DeepSeek | `deepseek-v4-flash` | chat | ✅ | 0.9s | |
74 −| DeepSeek | `deepseek-v4-flash` | stream | ✅ | 1.3s | deltas=9 usage=true |
75 −| DeepSeek | `deepseek-v4-pro` | chat | ✅ | 1.3s | |
76 −| Google Gemini | `—` | models | ✅ | 0.3s | 57 live |
77 −| Google Gemini | `gemini-2.5-flash` | chat | ✅ | 0.5s | |
78 −| Google Gemini | `gemini-2.5-flash-lite` | chat | ✅ | 0.4s | |
79 −| Google Gemini | `gemini-2.5-pro` | chat | ✅ | 2.0s | |
80 −| Google Gemini | `gemini-3-flash-preview` | chat | ✅ | 0.9s | |
81 −| Google Gemini | `gemini-3.1-flash-lite` | chat | ✅ | 0.7s | |
72 +| DeepSeek | `deepseek-v4-flash` | chat | ✅ | 1.1s | |
73 +| DeepSeek | `deepseek-v4-flash` | stream | ✅ | 0.9s | deltas=9 usage=true |
74 +| DeepSeek | `deepseek-v4-pro` | chat | ✅ | 1.5s | |
75 +| Google Gemini | `—` | models | ✅ | 0.2s | 57 live |
76 +| Google Gemini | `gemini-2.5-flash` | chat | ✅ | 0.6s | |
77 +| Google Gemini | `gemini-2.5-flash-lite` | chat | ✅ | 0.5s | |
78 +| Google Gemini | `gemini-2.5-pro` | chat | ✅ | 2.3s | |
79 +| Google Gemini | `gemini-3-flash-preview` | chat | ✅ | 1.0s | |
80 +| Google Gemini | `gemini-3.1-flash-lite` | chat | ✅ | 1.1s | |
82 81 | Google Gemini | `gemini-3.1-pro-preview` | chat | ✅ | 2.8s | |
83 82 | Google Gemini | `gemini-3.5-flash` | chat | ✅ | 1.3s | |
84 −| Google Gemini | `gemini-3.5-flash-lite` | chat | ✅ | 1.0s | |
85 −| Google Gemini | `gemini-3.6-flash` | chat | ✅ | 1.0s | |
83 +| Google Gemini | `gemini-3.5-flash-lite` | chat | ✅ | 0.6s | |
84 +| Google Gemini | `gemini-3.6-flash` | chat | ✅ | 1.1s | |
86 85 | Google Gemini | `gemini-3.6-flash` | stream | ✅ | 1.3s | deltas=1 usage=true |
87 −| Google Gemini | `gemini-3.6-flash` | vision | ✅ | 1.4s | Red |
88 −| Google Gemini | `gemini-flash-latest` | chat | ✅ | 1.2s | |
89 −| Google Gemini | `gemini-flash-lite-latest` | chat | ✅ | 0.6s | |
90 −| Google Gemini | `gemini-pro-latest` | chat | ✅ | 2.2s | |
91 −| Google Gemini | `gemma-4-26b-a4b-it` | chat | ✅ | 1.8s | |
92 −| Google Gemini | `gemma-4-31b-it` | chat | ✅ | 1.4s | |
93 −| Kimi | `—` | models | ✅ | 0.7s | 12 live |
94 −| Kimi | `kimi-k2.5` | chat | ✅ | 3.8s | |
95 −| Kimi | `kimi-k2.6` | chat | ✅ | 2.5s | |
96 −| Kimi | `kimi-k2.7-code` | chat | ✅ | 1.1s | |
97 −| Kimi | `kimi-k2.7-code-highspeed` | chat | ✅ | 0.8s | |
98 −| Kimi | `kimi-k3` | chat | ✅ | 7.2s | |
99 −| Kimi | `kimi-k3` | stream | ✅ | 2.9s | deltas=9 usage=true |
100 −| Kimi | `kimi-k3` | vision | ✅ | 13.1s | Red |
86 +| Google Gemini | `gemini-3.6-flash` | vision | ✅ | 1.1s | Red |
87 +| Google Gemini | `gemini-flash-latest` | chat | ✅ | 1.1s | |
88 +| Google Gemini | `gemini-flash-lite-latest` | chat | ✅ | 0.7s | |
89 +| Google Gemini | `gemini-pro-latest` | chat | ✅ | 2.3s | |
90 +| Google Gemini | `gemma-4-26b-a4b-it` | chat | ✅ | 1.6s | |
91 +| Google Gemini | `gemma-4-31b-it` | chat | ✅ | 2.0s | |
92 +| Kimi | `—` | models | ✅ | 0.4s | 12 live |
93 +| Kimi | `kimi-k2.5` | chat | ✅ | 6.4s | |
94 +| Kimi | `kimi-k2.6` | chat | ✅ | 4.7s | |
95 +| Kimi | `kimi-k2.7-code` | chat | ✅ | 1.5s | |
96 +| Kimi | `kimi-k2.7-code-highspeed` | chat | ✅ | 0.9s | |
97 +| Kimi | `kimi-k3` | chat | ✅ | 2.7s | |
98 +| Kimi | `kimi-k3` | stream | ✅ | 4.1s | deltas=9 usage=true |
99 +| Kimi | `kimi-k3` | vision | ✅ | 5.0s | Red |
101 100 | Kimi | `moonshot-v1-128k` | chat | ✅ | 0.5s | |
102 101 | Kimi | `moonshot-v1-128k-vision-preview` | chat | ✅ | 0.5s | |
103 −| Kimi | `moonshot-v1-32k` | chat | ✅ | 0.5s | |
102 +| Kimi | `moonshot-v1-32k` | chat | ✅ | 0.7s | |
104 103 | Kimi | `moonshot-v1-32k-vision-preview` | chat | ✅ | 0.5s | |
105 −| Kimi | `moonshot-v1-8k` | chat | ✅ | 0.5s | |
106 −| Kimi | `moonshot-v1-8k-vision-preview` | chat | ✅ | 0.6s | |
104 +| Kimi | `moonshot-v1-8k` | chat | ✅ | 0.7s | |
105 +| Kimi | `moonshot-v1-8k-vision-preview` | chat | ✅ | 0.5s | |
107 106 | Kimi | `moonshot-v1-auto` | chat | ✅ | 0.7s | |
108 −| Mistral | `—` | models | ✅ | 0.4s | 60 live |
109 −| Mistral | `codestral-latest` | chat | ✅ | 0.3s | |
110 −| Mistral | `devstral-latest` | chat | ✅ | 0.4s | |
111 −| Mistral | `magistral-medium-latest` | chat | ✅ | 0.9s | |
107 +| Mistral | `—` | models | ✅ | 0.5s | 60 live |
108 +| Mistral | `codestral-latest` | chat | ✅ | 0.4s | |
109 +| Mistral | `devstral-latest` | chat | ✅ | 0.3s | |
110 +| Mistral | `magistral-medium-latest` | chat | ✅ | 0.8s | |
112 111 | Mistral | `ministral-14b-latest` | chat | ✅ | 0.3s | |
113 112 | Mistral | `ministral-3b-latest` | chat | ✅ | 0.3s | |
114 113 | Mistral | `ministral-8b-latest` | chat | ✅ | 0.3s | |
115 −| Mistral | `mistral-large-latest` | chat | ✅ | 2.3s | |
114 +| Mistral | `mistral-large-latest` | chat | ✅ | 0.4s | |
116 115 | Mistral | `mistral-medium-latest` | chat | ✅ | 0.4s | |
117 −| Mistral | `mistral-medium-latest` | stream | ✅ | 0.8s | deltas=9 usage=true |
118 −| Mistral | `mistral-medium-latest` | vision | ✅ | 1.4s | The image is a solid red color. Therefor |
119 −| Mistral | `mistral-small-latest` | chat | ✅ | 0.7s | |
120 −| Mistral | `open-mistral-nemo` | chat | ✅ | 0.6s | |
121 −| OpenAI | `—` | models | ✅ | 1.1s | 132 live |
122 −| OpenAI | `chat-latest` | chat | ✅ | 1.1s | |
123 −| OpenAI | `gpt-3.5-turbo` | chat | ✅ | 1.2s | |
124 −| OpenAI | `gpt-4` | chat | ✅ | 1.1s | |
116 +| Mistral | `mistral-medium-latest` | stream | ✅ | 0.6s | deltas=9 usage=true |
117 +| Mistral | `mistral-medium-latest` | vision | ✅ | 0.3s | Red |
118 +| Mistral | `mistral-small-latest` | chat | ✅ | 0.5s | |
119 +| Mistral | `open-mistral-nemo` | chat | ✅ | 0.3s | |
120 +| OpenAI | `—` | models | ✅ | 0.7s | 132 live |
121 +| OpenAI | `chat-latest` | chat | ✅ | 0.9s | |
122 +| OpenAI | `gpt-3.5-turbo` | chat | ✅ | 1.1s | |
123 +| OpenAI | `gpt-4` | chat | ✅ | 1.3s | |
125 124 | OpenAI | `gpt-4-turbo` | chat | ✅ | 1.8s | |
126 125 | OpenAI | `gpt-4.1` | chat | ✅ | 0.8s | |
127 −| OpenAI | `gpt-4.1-mini` | chat | ✅ | 1.2s | |
128 −| OpenAI | `gpt-4.1-nano` | chat | ✅ | 0.4s | |
129 −| OpenAI | `gpt-4o` | chat | ✅ | 0.4s | |
126 +| OpenAI | `gpt-4.1-mini` | chat | ✅ | 0.4s | |
127 +| OpenAI | `gpt-4.1-nano` | chat | ✅ | 0.5s | |
128 +| OpenAI | `gpt-4o` | chat | ✅ | 0.6s | |
130 129 | OpenAI | `gpt-4o-mini` | chat | ✅ | 0.5s | |
131 −| OpenAI | `gpt-5` | chat | ✅ | 1.9s | |
132 −| OpenAI | `gpt-5-mini` | chat | ✅ | 1.2s | |
133 −| OpenAI | `gpt-5-nano` | chat | ✅ | 0.9s | |
134 −| OpenAI | `gpt-5.1` | chat | ✅ | 0.8s | |
130 +| OpenAI | `gpt-5` | chat | ✅ | 1.1s | |
131 +| OpenAI | `gpt-5-mini` | chat | ✅ | 1.4s | |
132 +| OpenAI | `gpt-5-nano` | chat | ✅ | 1.4s | |
133 +| OpenAI | `gpt-5.1` | chat | ✅ | 0.7s | |
135 134 | OpenAI | `gpt-5.2` | chat | ✅ | 0.9s | |
136 −| OpenAI | `gpt-5.2-chat-latest` | chat | ✅ | 1.3s | |
137 −| OpenAI | `gpt-5.3-chat-latest` | chat | ❌ | 2.2s | empty response |
138 −| OpenAI | `gpt-5.4` | chat | ✅ | 0.8s | |
139 −| OpenAI | `gpt-5.4-mini` | chat | ✅ | 0.8s | |
140 −| OpenAI | `gpt-5.4-nano` | chat | ✅ | 0.7s | |
141 −| OpenAI | `gpt-5.5` | chat | ✅ | 1.2s | |
142 −| OpenAI | `gpt-5.6-luna` | chat | ✅ | 2.0s | |
143 −| OpenAI | `gpt-5.6-sol` | chat | ✅ | 2.0s | |
144 −| OpenAI | `gpt-5.6-sol` | stream | ✅ | 1.2s | deltas=9 usage=true |
145 −| OpenAI | `gpt-5.6-sol` | vision | ✅ | 1.1s | Red |
146 −| OpenAI | `gpt-5.6-terra` | chat | ✅ | 4.6s | |
147 −| OpenAI | `o1` | chat | ✅ | 1.5s | |
148 −| OpenAI | `o3` | chat | ✅ | 0.9s | |
149 −| OpenAI | `o3-mini` | chat | ✅ | 1.5s | |
150 −| OpenAI | `o4-mini` | chat | ✅ | 1.0s | |
151 −| Perplexity | `sonar` | chat | ✅ | 1.7s | |
152 −| Perplexity | `sonar` | stream | ✅ | 2.1s | deltas=9 usage=true |
135 +| OpenAI | `gpt-5.2-chat-latest` | chat | ✅ | 1.0s | |
136 +| OpenAI | `gpt-5.3-chat-latest` | chat | ✅ | 1.2s | |
137 +| OpenAI | `gpt-5.4` | chat | ✅ | 0.7s | |
138 +| OpenAI | `gpt-5.4-mini` | chat | ✅ | 0.5s | |
139 +| OpenAI | `gpt-5.4-nano` | chat | ✅ | 0.6s | |
140 +| OpenAI | `gpt-5.5` | chat | ✅ | 1.3s | |
141 +| OpenAI | `gpt-5.6-luna` | chat | ✅ | 0.8s | |
142 +| OpenAI | `gpt-5.6-sol` | chat | ✅ | 1.2s | |
143 +| OpenAI | `gpt-5.6-sol` | stream | ✅ | 1.0s | deltas=9 usage=true |
144 +| OpenAI | `gpt-5.6-sol` | vision | ✅ | 1.2s | Red |
145 +| OpenAI | `gpt-5.6-terra` | chat | ✅ | 0.8s | |
146 +| OpenAI | `o1` | chat | ✅ | 1.1s | |
147 +| OpenAI | `o3` | chat | ✅ | 0.8s | |
148 +| OpenAI | `o3-mini` | chat | ✅ | 1.6s | |
149 +| OpenAI | `o4-mini` | chat | ✅ | 0.8s | |
150 +| Perplexity | `sonar` | chat | ✅ | 1.4s | |
151 +| Perplexity | `sonar` | stream | ✅ | 2.2s | deltas=9 usage=true |
153 152 | Perplexity | `sonar-deep-research` | chat | ✅ | — | SKIPPED: multi-minute agentic research runs; verified via docs only |
154 −| Perplexity | `sonar-pro` | chat | ✅ | 2.5s | |
155 −| Perplexity | `sonar-reasoning-pro` | chat | ✅ | 3.5s | |
156 −| Alibaba Qwen | `—` | models | ✅ | 1.0s | 151 live |
157 −| Alibaba Qwen | `deepseek-v4-flash` | chat | ✅ | 1.4s | |
158 −| Alibaba Qwen | `deepseek-v4-pro` | chat | ✅ | 1.6s | |
159 −| Alibaba Qwen | `glm-5.2` | chat | ✅ | 1.8s | |
160 −| Alibaba Qwen | `kimi-k2.7-code` | chat | ✅ | 1.5s | |
161 −| Alibaba Qwen | `qvq-max` | chat | ✅ | 2.3s | |
153 +| Perplexity | `sonar-pro` | chat | ✅ | 1.6s | |
154 +| Perplexity | `sonar-reasoning-pro` | chat | ✅ | 2.8s | |
155 +| Alibaba Qwen | `—` | models | ✅ | 0.9s | 151 live |
156 +| Alibaba Qwen | `deepseek-v4-flash` | chat | ✅ | 1.2s | |
157 +| Alibaba Qwen | `deepseek-v4-pro` | chat | ✅ | 1.8s | |
158 +| Alibaba Qwen | `glm-5.2` | chat | ✅ | 4.3s | |
159 +| Alibaba Qwen | `kimi-k2.7-code` | chat | ✅ | 4.0s | |
160 +| Alibaba Qwen | `qvq-max` | chat | ✅ | 2.4s | |
162 161 | Alibaba Qwen | `qwen-flash` | chat | ✅ | 0.4s | |
163 −| Alibaba Qwen | `qwen-max` | chat | ✅ | 0.6s | |
162 +| Alibaba Qwen | `qwen-max` | chat | ✅ | 0.5s | |
164 163 | Alibaba Qwen | `qwen-plus` | chat | ✅ | 0.7s | |
165 −| Alibaba Qwen | `qwen-turbo` | chat | ✅ | 0.5s | |
164 +| Alibaba Qwen | `qwen-turbo` | chat | ✅ | 0.4s | |
166 165 | Alibaba Qwen | `qwen3-235b-a22b-instruct-2507` | chat | ✅ | 0.6s | |
167 166 | Alibaba Qwen | `qwen3-235b-a22b-thinking-2507` | chat | ✅ | 1.0s | |
168 −| Alibaba Qwen | `qwen3-coder-480b-a35b-instruct` | chat | ✅ | 0.9s | |
169 −| Alibaba Qwen | `qwen3-coder-flash` | chat | ✅ | 0.7s | |
167 +| Alibaba Qwen | `qwen3-coder-480b-a35b-instruct` | chat | ✅ | 1.1s | |
168 +| Alibaba Qwen | `qwen3-coder-flash` | chat | ✅ | 1.0s | |
170 169 | Alibaba Qwen | `qwen3-coder-next` | chat | ✅ | 0.8s | |
171 170 | Alibaba Qwen | `qwen3-coder-plus` | chat | ✅ | 0.8s | |
172 171 | Alibaba Qwen | `qwen3-next-80b-a3b-instruct` | chat | ✅ | 0.5s | |
173 −| Alibaba Qwen | `qwen3-next-80b-a3b-thinking` | chat | ✅ | 1.1s | |
172 +| Alibaba Qwen | `qwen3-next-80b-a3b-thinking` | chat | ✅ | 1.0s | |
174 173 | Alibaba Qwen | `qwen3-vl-235b-a22b-instruct` | chat | ✅ | 0.5s | |
175 −| Alibaba Qwen | `qwen3-vl-235b-a22b-thinking` | chat | ✅ | 6.2s | |
176 −| Alibaba Qwen | `qwen3-vl-flash` | chat | ✅ | 0.7s | |
177 −| Alibaba Qwen | `qwen3-vl-plus` | chat | ✅ | 1.0s | |
178 −| Alibaba Qwen | `qwen3.5-122b-a10b` | chat | ✅ | 1.3s | |
179 −| Alibaba Qwen | `qwen3.5-35b-a3b` | chat | ✅ | 1.5s | |
180 −| Alibaba Qwen | `qwen3.5-397b-a17b` | chat | ✅ | 6.1s | |
181 −| Alibaba Qwen | `qwen3.5-flash` | chat | ✅ | 1.8s | |
182 −| Alibaba Qwen | `qwen3.5-plus` | chat | ✅ | 7.1s | |
183 −| Alibaba Qwen | `qwen3.6-flash` | chat | ✅ | 2.2s | |
184 −| Alibaba Qwen | `qwen3.6-plus` | chat | ✅ | 3.0s | |
185 −| Alibaba Qwen | `qwen3.7-flash` | chat | ✅ | 2.1s | |
186 −| Alibaba Qwen | `qwen3.7-max` | chat | ✅ | 4.9s | |
187 −| Alibaba Qwen | `qwen3.7-max` | stream | ✅ | 3.8s | deltas=3 usage=true |
174 +| Alibaba Qwen | `qwen3-vl-235b-a22b-thinking` | chat | ✅ | 3.1s | |
175 +| Alibaba Qwen | `qwen3-vl-flash` | chat | ✅ | 0.5s | |
176 +| Alibaba Qwen | `qwen3-vl-plus` | chat | ✅ | 0.6s | |
177 +| Alibaba Qwen | `qwen3.5-122b-a10b` | chat | ✅ | 1.1s | |
178 +| Alibaba Qwen | `qwen3.5-35b-a3b` | chat | ✅ | 1.7s | |
179 +| Alibaba Qwen | `qwen3.5-397b-a17b` | chat | ✅ | 4.8s | |
180 +| Alibaba Qwen | `qwen3.5-flash` | chat | ✅ | 1.4s | |
181 +| Alibaba Qwen | `qwen3.5-plus` | chat | ✅ | 7.2s | |
182 +| Alibaba Qwen | `qwen3.6-flash` | chat | ✅ | 1.5s | |
183 +| Alibaba Qwen | `qwen3.6-plus` | chat | ✅ | 3.6s | |
184 +| Alibaba Qwen | `qwen3.7-flash` | chat | ✅ | 2.4s | |
185 +| Alibaba Qwen | `qwen3.7-max` | chat | ✅ | 3.3s | |
186 +| Alibaba Qwen | `qwen3.7-max` | stream | ✅ | 6.5s | deltas=3 usage=true |
188 187 | Alibaba Qwen | `qwen3.7-plus` | chat | ✅ | 4.3s | |
189 −| Alibaba Qwen | `qwen3.7-plus` | vision | ✅ | 3.2s | Red |
190 −| Alibaba Qwen | `qwq-plus` | chat | ✅ | 3.9s | |
191 −| Together AI | `—` | models | ✅ | 0.4s | 270 live |
192 −| Together AI | `MiniMaxAI/MiniMax-M3` | chat | ✅ | 0.6s | |
193 −| Together AI | `Qwen/Qwen3.5-9B` | chat | ❌ | 1.1s | empty response |
194 −| Together AI | `Qwen/Qwen3.6-Plus` | chat | ✅ | 4.6s | |
195 −| Together AI | `Qwen/Qwen3.7-Max` | chat | ✅ | 4.1s | |
196 −| Together AI | `Qwen/Qwen3.7-Plus` | chat | ✅ | 4.5s | |
197 −| Together AI | `deepseek-ai/DeepSeek-V4-Pro` | chat | ✅ | 1.4s | |
198 −| Together AI | `google/gemma-4-31B-it` | chat | ❌ | 121.0s | Network error: The request timed out. |
199 −| Together AI | `google/gemma-4-31B-it` | vision | ❌ | 0.7s | empty response |
200 −| Together AI | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | chat | ✅ | 2.5s | |
201 −| Together AI | `moonshotai/Kimi-K2.6` | chat | ✅ | 1.3s | |
202 −| Together AI | `moonshotai/Kimi-K2.7-Code` | chat | ✅ | 1.2s | |
203 −| Together AI | `moonshotai/Kimi-K3` | chat | ✅ | 6.8s | |
204 −| Together AI | `moonshotai/Kimi-K3` | stream | ✅ | 2.6s | deltas=2 usage=true |
188 +| Alibaba Qwen | `qwen3.7-plus` | vision | ✅ | 2.7s | Red |
189 +| Alibaba Qwen | `qwq-plus` | chat | ✅ | 2.9s | |
190 +| Together AI | `—` | models | ✅ | 0.3s | 270 live |
191 +| Together AI | `MiniMaxAI/MiniMax-M3` | chat | ✅ | 1.9s | |
192 +| Together AI | `Qwen/Qwen3.5-9B` | chat | ✅ | 1.7s | |
193 +| Together AI | `Qwen/Qwen3.6-Plus` | chat | ✅ | 4.2s | |
194 +| Together AI | `Qwen/Qwen3.7-Max` | chat | ✅ | 4.6s | |
195 +| Together AI | `Qwen/Qwen3.7-Plus` | chat | ✅ | 3.8s | |
196 +| Together AI | `deepseek-ai/DeepSeek-V4-Pro` | chat | ✅ | 0.8s | |
197 +| Together AI | `google/gemma-4-31B-it` | chat | ✅ | 18.0s | |
198 +| Together AI | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | chat | ✅ | 3.5s | |
199 +| Together AI | `moonshotai/Kimi-K2.6` | chat | ✅ | 0.5s | |
200 +| Together AI | `moonshotai/Kimi-K2.7-Code` | chat | ✅ | 0.5s | |
201 +| Together AI | `moonshotai/Kimi-K3` | chat | ✅ | 1.3s | |
202 +| Together AI | `moonshotai/Kimi-K3` | stream | ✅ | 1.6s | deltas=2 usage=true |
205 203 | Together AI | `nvidia/nemotron-3-ultra-550b-a55b` | chat | ✅ | 0.4s | |
206 −| Together AI | `openai/gpt-oss-120b` | chat | ✅ | 0.5s | |
207 −| Together AI | `openai/gpt-oss-20b` | chat | ✅ | 0.3s | |
208 −| Together AI | `thinkingmachines/Inkling` | chat | ✅ | 0.5s | |
204 +| Together AI | `openai/gpt-oss-120b` | chat | ✅ | 0.8s | |
205 +| Together AI | `openai/gpt-oss-20b` | chat | ✅ | 0.7s | |
206 +| Together AI | `thinkingmachines/Inkling` | chat | ✅ | 0.6s | |
209 207 | Together AI | `zai-org/GLM-5.2` | chat | ✅ | 0.9s | |
210 −| xAI | `—` | models | ✅ | 0.3s | 10 live |
211 −| xAI | `grok-4.20` | chat | ✅ | 1.2s | |
212 −| xAI | `grok-4.20-non-reasoning` | chat | ✅ | 0.4s | |
213 −| xAI | `grok-4.3` | chat | ✅ | 1.6s | |
208 +| xAI | `—` | models | ✅ | 0.2s | 10 live |
209 +| xAI | `grok-4.20` | chat | ✅ | 0.8s | |
210 +| xAI | `grok-4.20-non-reasoning` | chat | ✅ | 0.3s | |
211 +| xAI | `grok-4.3` | chat | ✅ | 1.1s | |
214 212 | xAI | `grok-4.5` | chat | ✅ | 0.7s | |
215 −| xAI | `grok-4.5` | stream | ✅ | 0.9s | deltas=9 usage=true |
216 −| xAI | `grok-4.5` | vision | ✅ | 1.1s | Red |
217 −| xAI | `grok-code-fast-1` | chat | ✅ | 2.1s | |
213 +| xAI | `grok-4.5` | stream | ✅ | 1.5s | deltas=9 usage=true |
214 +| xAI | `grok-4.5` | vision | ✅ | 1.3s | Red |
215 +| xAI | `grok-code-fast-1` | chat | ✅ | 1.5s | |
218 216