spb/zyquo-agent Public MIT

1<!--2  VERIFICATION.md3  Zyquo Agent45  Author: Simon-Pierre Boucher6  Mail: contact@spboucher.ai7-->89# Provider Tool-Calling Verification — 2026-07-301011Phase 7.1 live sweep (`ZyquoAgent --verify`): every agent-capable model receives12the `list_files` tool schema, must emit a valid tool call (streamed), consume the13threaded `tool_result`, and produce a final answer naming the listed files.1415**80 models tested · 77 green · 3 failed** (all 3 failures are provider-side —16see the classification at the bottom)1718This table merges the full pass-1 sweep with the per-provider re-runs performed19after the two client fixes below (each fixed model was re-verified green live):2021- **OpenAI GPT-5.4/5.5/5.6:** `/chat/completions` rejects function tools combined22  with any `reasoning_effort` other than `"none"` — the client now downgrades23  `reasoning_effort` to `"none"` whenever tools are attached for these models24  (`OpenAICompatibleClient.buildBody`).25- **Gemini 3.x (compat endpoint):** tool calls carry an26  `extra_content.google.thought_signature` that MUST be echoed back verbatim when27  the call is threaded into history, or Gemini rejects the request28  (`INVALID_ARGUMENT: Function call is missing a thought_signature`). `ToolCall`29  now carries the opaque signature; the client captures it from streaming30  fragments / non-streaming entries and re-attaches it on assistant `tool_calls`.3132| Provider | Model | Tool call | Threading | Streaming | Latency | Notes |33|---|---|---|---|---|---|---|34| OpenAI | `gpt-5.6-sol` | ✅ | ✅ | ✅ | 3.0s | pass 1 ❌ (tools+reasoning_effort rejected); green after client fix |35| OpenAI | `gpt-5.6-terra` | ✅ | ✅ | ✅ | 1.4s | pass 1 ❌ (tools+reasoning_effort rejected); green after client fix |36| OpenAI | `gpt-5.6-luna` | ✅ | ✅ | ✅ | 1.6s | pass 1 ❌ (tools+reasoning_effort rejected); green after client fix |37| OpenAI | `gpt-5.5` | ✅ | ✅ | ✅ | 4.1s | pass 1 ❌ (tools+reasoning_effort rejected); green after client fix |38| OpenAI | `gpt-5.4` | ✅ | ✅ | ✅ | 1.8s | pass 1 ❌ (tools+reasoning_effort rejected); green after client fix |39| OpenAI | `gpt-5.4-mini` | ✅ | ✅ | ✅ | 0.9s | pass 1 ❌ (tools+reasoning_effort rejected); green after client fix |40| OpenAI | `gpt-5.2` | ✅ | ✅ | ✅ | 3.3s | |41| OpenAI | `gpt-5.1` | ✅ | ✅ | ✅ | 1.6s | |42| OpenAI | `gpt-5` | ✅ | ✅ | ✅ | 2.4s | |43| OpenAI | `gpt-5-mini` | ✅ | ✅ | ✅ | 2.1s | |44| OpenAI | `o3` | ✅ | ✅ | ✅ | 1.3s | |45| OpenAI | `o4-mini` | ✅ | ✅ | ✅ | 1.2s | |46| Anthropic | `claude-opus-5` | ✅ | ✅ | ✅ | 3.9s | |47| Anthropic | `claude-sonnet-5` | ✅ | ✅ | ✅ | 4.0s | |48| Anthropic | `claude-fable-5` | ✅ | ✅ | ✅ | 7.5s | |49| Anthropic | `claude-opus-4-8` | ✅ | ✅ | ✅ | 3.2s | |50| Anthropic | `claude-opus-4-7` | ✅ | ✅ | ✅ | 3.2s | |51| Anthropic | `claude-opus-4-6` | ✅ | ✅ | ✅ | 6.1s | |52| Anthropic | `claude-sonnet-4-6` | ✅ | ✅ | ✅ | 4.1s | |53| Anthropic | `claude-haiku-4-5-20251001` | ✅ | ✅ | ✅ | 1.6s | |54| xAI | `grok-4.5` | ✅ | ✅ | ✅ | 2.5s | |55| xAI | `grok-4.3` | ✅ | ✅ | ✅ | 4.5s | |56| xAI | `grok-4.20` | ✅ | ✅ | ✅ | 4.9s | |57| xAI | `grok-4.20-non-reasoning` | ✅ | ✅ | ✅ | 1.2s | |58| xAI | `grok-code-fast-1` | ✅ | ✅ | ✅ | 5.3s | |59| Mistral | `mistral-medium-latest` | ✅ | ✅ | ✅ | 1.4s | |60| Mistral | `mistral-large-latest` | ✅ | ✅ | ✅ | 3.7s | |61| Mistral | `mistral-small-latest` | ✅ | ✅ | ✅ | 0.9s | |62| Google Gemini | `gemini-3.6-flash` | ✅ | ✅ | ✅ | 2.1s | pass 1 ❌ (thought_signature missing on threading); green after client fix |63| Google Gemini | `gemini-3.5-flash` | ✅ | ✅ | ✅ | 2.0s | pass 1 ❌ (thought_signature); green after client fix |64| Google Gemini | `gemini-3.5-flash-lite` | ✅ | ✅ | ✅ | 1.0s | pass 1 ❌ (thought_signature); green after client fix |65| Google Gemini | `gemini-3.1-pro-preview` | ✅ | ✅ | ✅ | 3.9s | pass 1 ❌ (thought_signature); green after client fix |66| Google Gemini | `gemini-2.5-pro` | ✅ | ✅ | ✅ | 3.3s | |67| Google Gemini | `gemini-2.5-flash` | ✅ | ✅ | ✅ | 1.3s | |68| Google Gemini | `gemini-pro-latest` | ✅ | ✅ | ✅ | 4.2s | alias of Gemini 3.x; pass 1 ❌ (thought_signature); green after client fix |69| Google Gemini | `gemini-flash-latest` | ✅ | ✅ | ✅ | 1.5s | alias of Gemini 3.x; pass 1 ❌ (thought_signature); green after client fix |70| Alibaba Qwen | `qwen3.7-max` | ✅ | ✅ | ✅ | 3.0s | |71| Alibaba Qwen | `qwen3.7-plus` | ✅ | ✅ | ✅ | 2.6s | |72| Alibaba Qwen | `qwen3.7-flash` | ✅ | ✅ | ✅ | 1.6s | |73| Alibaba Qwen | `qwen3.6-plus` | ✅ | ✅ | ✅ | 2.2s | |74| Alibaba Qwen | `qwen3.5-plus` | ✅ | ✅ | ✅ | 2.0s | |75| Alibaba Qwen | `qwen3-coder-plus` | ✅ | ✅ | ✅ | 2.1s | |76| Alibaba Qwen | `qwen3-coder-flash` | ✅ | ✅ | ✅ | 2.0s | |77| Alibaba Qwen | `qwen3-coder-next` | ✅ | ✅ | ✅ | 2.3s | |78| Alibaba Qwen | `qwen3-coder-480b-a35b-instruct` | ✅ | ✅ | ✅ | 2.6s | |79| Alibaba Qwen | `qwen3.5-397b-a17b` | ✅ | ✅ | ✅ | 1.9s | |80| Alibaba Qwen | `deepseek-v4-pro` | ✅ | ✅ | ✅ | 4.6s | |81| Alibaba Qwen | `glm-5.2` | ✅ | ✅ | ✅ | 2.1s | |82| DeepSeek | `deepseek-v4-flash` | ✅ | ✅ | ✅ | 2.5s | |83| DeepSeek | `deepseek-v4-pro` | ✅ | ✅ | ✅ | 3.2s | |84| Kimi | `kimi-k3` | ❌ | ❌ | ❌ | 100.5s | INFRA: HTTP 429 `engine_overloaded_error` on most requests during the sweep window (reproduced with raw curl in every request shape; intermittent 200s prove the key/tier is fine). One harness attempt DID emit a correct `list_files` call, so the wire format is right — Moonshot capacity issue; re-verify when the engine recovers |85| Kimi | `kimi-k2.7-code` | ✅ | ✅ | ✅ | 2.3s | |86| Kimi | `kimi-k2.7-code-highspeed` | ✅ | ✅ | ✅ | 1.8s | |87| Kimi | `kimi-k2.6` | ✅ | ✅ | ✅ | 5.1s | |88| Kimi | `kimi-k2.5` | ✅ | ✅ | ✅ | 4.8s | |89| Together AI | `moonshotai/Kimi-K3` | ✅ | ✅ | ✅ | 19.0s | slow first token (reasoning) |90| Together AI | `moonshotai/Kimi-K2.7-Code` | ✅ | ✅ | ✅ | 1.6s | |91| Together AI | `deepseek-ai/DeepSeek-V4-Pro` | ✅ | ✅ | ✅ | 1.6s | |92| Together AI | `zai-org/GLM-5.2` | ✅ | ✅ | ✅ | 1.2s | |93| Together AI | `Qwen/Qwen3.7-Max` | ✅ | ✅ | ✅ | 4.6s | |94| Together AI | `openai/gpt-oss-120b` | ✅ | ✅ | ✅ | 2.2s | |95| Together AI | `nvidia/nemotron-3-ultra-550b-a55b` | ✅ | ✅ | ✅ | 1.2s | |96| Together AI | `MiniMaxAI/MiniMax-M3` | ✅ | ✅ | ✅ | 6.1s | |97| DeepInfra | `anthropic/claude-fable-5` | ✅ | ✅ | ✅ | 6.3s | |98| DeepInfra | `anthropic/claude-opus-5` | ✅ | ✅ | ✅ | 5.6s | |99| DeepInfra | `anthropic/claude-sonnet-5` | ✅ | ✅ | ✅ | 7.1s | |100| DeepInfra | `anthropic/claude-opus-4-8` | ✅ | ✅ | ✅ | 4.1s | |101| DeepInfra | `anthropic/claude-haiku-4-5` | ✅ | ✅ | ✅ | 2.3s | |102| DeepInfra | `google/gemini-3.1-pro` | ✅ | ❌ | ❌ | 4.0s | SERVER-SIDE: step 1 works, but DeepInfra never returns Gemini's `thought_signature` and strips `extra_content` from requests (verified with raw probes, incl. Google's documented bypass placeholder) → Google rejects every threaded tool result with INVALID_ARGUMENT. Multi-turn tool use is impossible through DeepInfra — REMOVAL CANDIDATE (the same model is green via the native Gemini provider) |103| DeepInfra | `google/gemini-3.5-flash` | ✅ | ❌ | ❌ | 2.3s | SERVER-SIDE: same as `google/gemini-3.1-pro` — thought_signature dropped by DeepInfra; REMOVAL CANDIDATE |104| DeepInfra | `deepseek-ai/DeepSeek-V4-Pro` | ✅ | ✅ | ✅ | 4.1s | |105| DeepInfra | `deepseek-ai/DeepSeek-V4-Flash` | ✅ | ✅ | ✅ | 3.3s | |106| DeepInfra | `moonshotai/Kimi-K2.7-Code` | ✅ | ✅ | ✅ | 2.2s | |107| DeepInfra | `zai-org/GLM-5.2` | ✅ | ✅ | ✅ | 3.1s | |108| DeepInfra | `Qwen/Qwen3.7-Max` | ✅ | ✅ | ✅ | 4.7s | |109| DeepInfra | `Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo` | ✅ | ✅ | ✅ | 0.8s | |110| DeepInfra | `openai/gpt-oss-120b` | ✅ | ✅ | ✅ | 2.6s | |111| DeepInfra | `MiniMaxAI/MiniMax-M3` | ✅ | ✅ | ✅ | 3.0s | |112| Cerebras | `gpt-oss-120b` | ✅ | ✅ | ✅ | 1.4s | |113| Cerebras | `gemma-4-31b` | ✅ | ✅ | ✅ | 0.6s | |114115## Failure classification116117| Model | Class | Evidence / disposition |118|---|---|---|119| `deepinfra` `google/gemini-3.1-pro` | Broken server-side | DeepInfra never emits `thought_signature` and drops `extra_content` on requests; Gemini 3 requires it for threaded tool calls. No client-side workaround exists (Google's documented placeholder is also stripped). **Recommend removing from the agent-capable set** — the identical models are green via the native Gemini provider. |120| `deepinfra` `google/gemini-3.5-flash` | Broken server-side | Same as above. **Recommend removing from the agent-capable set.** |121| `kimi` `kimi-k3` | Infra (provider capacity) | HTTP 429 `engine_overloaded_error` on most requests during the sweep window, reproduced with raw curl in every request shape; intermittent 200s prove the key/tier is fine and one harness attempt did emit a correct `list_files` call. Not a wire-format or capability failure — keep in the set, re-verify when Moonshot capacity recovers. |122123No model id in the agent-capable set was rejected as nonexistent/deprecated, and124no model failed tool calling because of model capability — every ❌ above is125provider-side.126