# Provider test matrix

Generated by `pnpm providers:matrix` against the REAL provider APIs on 2026-09-08. A cell is ✅ only when the live request succeeded and the assertion passed. “—” = not run (no key).

Test models: openai: gpt-5.4-mini / reasoning gpt-5.4-mini; anthropic: claude-haiku-4-5 / reasoning claude-sonnet-5; gemini: gemini-3.5-flash-lite / reasoning gemini-3.5-flash-lite; xai: grok-4.20-0309-non-reasoning / reasoning grok-4.3; mistral: mistral-small-latest / reasoning magistral-medium-latest; deepseek: deepseek-v4-flash / reasoning deepseek-v4-flash; kimi: kimi-k2.6 / reasoning kimi-k3; openrouter: openai/gpt-5.4-nano / reasoning openai/gpt-5.4-nano; cerebras: gemma-4-31b / reasoning gpt-oss-120b.

| Capability | openai | anthropic | gemini | xai | mistral | deepseek | kimi | openrouter | cerebras |
| --- | :-: | :-: | :-: | :-: | :-: | :-: | :-: | :-: | :-: |
| Auth | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Model list | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Text | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Streaming | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| System | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Vision | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Tools | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Structured output | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Reasoning | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Token usage | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Invalid key | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Error normalization | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Long response | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |

## Notes

- kimi: models: 4 (kimi-k3, kimi-k2.7-code, kimi-k2.7-code-highspeed, kimi-k2.6…)
- kimi: text(kimi-k2.6): "OK" deltas=68 usage={"inputTokens":12,"outputTokens":71,"cachedInputTokens":12,"reasoningTokens":68,"totalTokens":83} finish=stop
- kimi: system: "Arr hello to ye matey."
- kimi: vision: "Red"
- kimi: tools: [{"n":"calculator","a":{"expression":"1234 * 5678"}}] finish=tool-calls
- kimi: structured: {"city":"Ottawa","country":"Canada"}
- kimi: reasoning(kimi-k3): text="16" reasoningChars=709 reasoningTokens=289
- kimi: invalid key → INVALID_API_KEY (401)
- kimi: unknown model → MODEL_NOT_FOUND: Not found the model definitely-not-a-model-xyz or Permission denied
- kimi: long: 410 words, 1641 deltas