# Together AI — Provider Research (Phase 0) Researched 2026-07-30 against official docs (https://docs.together.ai) and a live authenticated `GET /v1/models` probe (`/tmp/zyquo-probe/together.json`, 271 models returned: 164 `chat` + 14 `language` + image/video/audio/embedding/rerank/etc.). The probe is authoritative for model existence, context lengths, and pricing (embedded `pricing` object, USD per 1M tokens). --- ## 1. Base URL & Endpoints - **Base URL:** `https://api.together.xyz/v1` (docs also present `https://api.together.ai/v1` — both resolve to the same API; Zyquo Cloud uses `https://api.together.xyz/v1`) - `POST /v1/chat/completions` — chat (the endpoint Zyquo Cloud uses) - `POST /v1/completions` — legacy text completion - `GET /v1/models` — model listing - `POST /v1/embeddings`, `/v1/images/generations`, `/v1/audio/*`, `/v1/rerank` — other modalities (out of scope) - Docs also mention an optimized inference host `https://api-inference.together.ai/v2` (not needed; standard v1 is the documented OpenAI-compatible surface) Reference: https://docs.together.ai/reference/chat-completions-1 ## 2. Authentication `Authorization: Bearer ` — standard Bearer header. Confirmed by docs and by the live probe. ## 3. Chat Model Catalog (curated serverless chat models) The live `/models` response contains 178 chat/language entries, many of which are dedicated-endpoint-only or free-tier/lora artifacts (pricing `0/0`). The curated list below covers the main **serverless** chat models (cross-checked against https://docs.together.ai/docs/serverless-models). Pricing = USD / 1M tokens from the live probe. Context = `context_length` from the probe. Capabilities legend: **V** vision, **T** tools/function calling, **R** reasoning/thinking output, **J** JSON mode / structured output. All models stream. Capability flags marked * are inferred from model family docs, not individually verified on Together. | Model ID (exact) | Context | $ In | $ Out | $ Cached | Caps | |---|---|---|---|---|---| | `moonshotai/Kimi-K3` | 1,000,000 | 3.00 | 15.00 | 0.30 | T J R | | `moonshotai/Kimi-K2.7-Code` | 262,144 | 0.95 | 4.00 | 0.19 | T J R* | | `moonshotai/Kimi-K2.6` | 262,144 | 1.20 | 4.50 | 0.20 | T J R* | | `moonshotai/Kimi-K2.5-fp4` | 262,144 | 0.50 | 2.80 | — | T* J* | | `deepseek-ai/DeepSeek-V4-Pro` | 512,000 | 1.74 | 3.48 | 0.20 | T J R* | | `deepseek-ai/DeepSeek-V3.1` | 131,072 | 0.60 | 1.70 | — | T* J* R (hybrid) | | `deepseek-ai/DeepSeek-R1-0528` | 163,840 | 3.00 | 7.00 | — | R | | `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | 131,072 | 2.00 | 2.00 | — | R | | `zai-org/GLM-5.2` | 512,000 | 1.40 | 4.40 | 0.26 | T J R* | | `zai-org/GLM-5.1` | 202,752 | 1.40 | 4.40 | 0.26 | T* J* R* | | `zai-org/GLM-5` | 202,752 | 1.00 | 3.20 | — | T* J* R* | | `zai-org/GLM-4.7` | 202,752 | 0.45 | 2.00 | — | T* J* R* | | `zai-org/GLM-4.6` | 202,752 | 0.60 | 2.20 | — | T* J* R* | | `zai-org/GLM-4.5-Air-FP8` | 131,072 | 0.20 | 1.10 | — | T* J* R* | | `Qwen/Qwen3.7-Max` | 1,000,000 | 1.25 | 3.75 | 0.125 | T* J* R* (unverified caps) | | `Qwen/Qwen3.7-Plus` | 1,000,000 | 0.32 | 1.28 | — | T* J* (unverified caps) | | `Qwen/Qwen3.6-Plus` | 1,000,000 | 0.50 | 3.00 | — | T* J* (unverified caps) | | `Qwen/Qwen3.5-397B-A17B` | 262,144 | 0.60 | 3.60 | 0.35 | T* J* R* | | `Qwen/Qwen3.5-9B` | 262,144 | 0.17 | 0.25 | — | T J | | `Qwen/Qwen3-Next-80B-A3B-Instruct` | 262,144 | 0.15 | 1.50 | — | T* J* | | `Qwen/Qwen3-Next-80B-A3B-Thinking` | 262,144 | 0.15 | 1.50 | — | R T* J* | | `Qwen/Qwen3-Coder-Next-FP8` | 262,144 | 0.50 | 1.20 | — | T* J* | | `Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8` | 262,144 | 2.00 | 2.00 | — | T* J* | | `Qwen/Qwen3-VL-32B-Instruct` | 262,144 | 0.50 | 1.50 | — | V T* J* | | `Qwen/Qwen3-VL-8B-Instruct` | 262,144 | 0.18 | 0.68 | — | V T* J* | | `Qwen/QwQ-32B` | 131,072 | 1.20 | 1.20 | — | R | | `Qwen/Qwen2.5-VL-72B-Instruct` | 32,768 | 1.95 | 8.00 | — | V | | `Qwen/Qwen2.5-72B-Instruct-Turbo` | 131,072 | 1.20 | 1.20 | — | T* J* | | `Qwen/Qwen2.5-7B-Instruct-Turbo` | 32,768 | 0.30 | 0.30 | — | T J | | `Qwen/Qwen2.5-Coder-32B-Instruct` | 16,384 | 0.80 | 0.80 | — | — | | `meta-llama/Llama-4-Scout-17B-16E-Instruct` | 1,048,576 | 0.18 | 0.59 | — | V T* J* | | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | 131,072 | 1.04 | 1.04 | — | T J | | `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` | 131,072 | 0.88 | 0.88 | — | T* J* | | `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | 131,072 | 0.18 | 0.18 | — | T* J* | | `meta-llama/Llama-3.1-405B-Instruct` | 4,096 (probe value — likely a listing quirk; flag in Phase 7) | 3.50 | 3.50 | — | T* | | `meta-llama/Llama-3.2-3B-Instruct` | 131,072 | 0.06 | 0.06 | — | — | | `openai/gpt-oss-120b` | 131,072 | 0.15 | 0.60 | — | R T J (`reasoning_effort`) | | `openai/gpt-oss-20b` | 131,072 | 0.05 | 0.20 | — | R T J (`reasoning_effort`) | | `nvidia/nemotron-3-ultra-550b-a55b` | 512,288 | 0.60 | 3.60 | 0.20 | T J R* | | `nvidia/NVIDIA-Nemotron-Nano-9B-v2` | 131,072 | 0.06 | 0.25 | — | R* | | `MiniMaxAI/MiniMax-M3` | 524,288 | 0.30 | 1.20 | 0.06 | T J R* | | `MiniMaxAI/MiniMax-M2.7` | 196,608 | 0.30 | 1.20 | 0.06 | T* J* R* | | `mistralai/Mistral-Small-24B-Instruct-2501` | 32,768 | 0.10 | 0.30 | — | T* J* | | `mistralai/Ministral-3-14B-Instruct-2512` | 262,144 | 0.20 | 0.20 | — | T* J* | | `mistralai/Mixtral-8x7B-Instruct-v0.1` | 32,768 | 0.60 | 0.60 | — | J* | | `google/gemma-4-31B-it` | 262,144 | 0.39 | 0.97 | — | T J V* | | `google/gemma-3n-E4B-it` | 32,768 | 0.06 | 0.12 | — | — | | `thinkingmachines/Inkling` | 524,288 | 1.00 | 4.05 | 0.17 | T J R* | | `deepcogito/cogito-v2-1-671b` | 163,840 | 1.25 | 1.25 | — | R* | | `LiquidAI/LFM2.5-8B-A1B` | 128,000 | 0.03 | 0.12 | — | — | | `arcee-ai/trinity-mini` | 128,000 | 0.045 | 0.15 | — | — | Notes: - The probe also lists `zai-org/GLM-4.7-fp4`, `zai-org/GLM-5-FP4`, `Qwen/Qwen3.6-35B-A3B-FP8`, `Qwen/Qwen3.5-35B-A3B`, `MiniMaxAI/MiniMax-M2`, `pearl-ai/gemma-4-31b-it`, `Qwen/Qwen3-235B-A22B-Instruct-2507-FP8` etc. with pricing `0/0` — these appear to be dedicated-endpoint variants; exclude from the serverless catalog (revisit in Phase 7). - `moonshotai/Kimi-K3` at 1M context and `Qwen/Qwen3.7-Max`/`Qwen3.7-Plus`/`Qwen3.6-Plus` (1M) are the largest-context serverless options. - Vision on Together: `image_url` content parts (OpenAI format); `video_url`/`audio_url`/`input_audio` also accepted on multimodal models (docs). ## 4. Request/Response Format **Fully OpenAI-compatible** (`POST /v1/chat/completions`). Drop-in with OpenAI SDKs — change only base URL + key (https://docs.together.ai/docs/openai-api-compatibility). Deviations to handle: - Extra `finish_reason` value **`eos`** in addition to `stop|length|tool_calls`. - Assistant messages from reasoning models carry a **top-level `reasoning` field** (`choices[].message.reasoning`) with the chain of thought — not OpenAI's shape. Zyquo Cloud's "Thinking…" section should read this field. - Usage token details vary: reasoning models nest cached/reasoning tokens under `usage.prompt_tokens_details` / `usage.completion_tokens_details`; some models return them flat — read both defensively. - `logprobs` returns Together's own richer shape; `seed` is best-effort; `n` (1–128) not supported on all models; `logit_bias` unsupported on most models; `reasoning_effort` only honored on GPT-OSS models; vision `detail` accepted but ignored. - Ignored params: `service_tier`, `store`, `metadata`, `prediction`. - Response may include `warnings: [{message}]` and an echoed `prompt` array. ## 5. Streaming - `stream: true` → SSE, `data: {chat.completion.chunk JSON}` lines, terminated by `data: [DONE]`. - Chunk deltas follow the OpenAI `choices[].delta` shape; reasoning models stream a `reasoning` delta field alongside `content` (mirror of the non-streaming `message.reasoning`). - **Usage in stream:** Together returns a `usage` object on the final chunk(s) by default (observed behavior; `stream_options.include_usage` is accepted for OpenAI compat — verify exact behavior in Phase 7; marked partially unverified). ## 6. Special Parameters (Full list: https://docs.together.ai/reference/chat-completions-1) - `context_length_exceeded_behavior`: `"truncate"` or `"error"` — Together-specific control of over-context prompts (default error). Expose as an Advanced setting. - `safety_model`: model ID of a moderation model (e.g., Llama Guard) run against output tokens. - `repetition_penalty` (number) and `min_p` (0–1) — sampler params beyond the OpenAI set; `top_k` (integer) also supported. - `echo` (bool, include prompt in response), `logprobs` (0–20, Together shape), `seed`, `n` (1–128). - `response_format`: `text` | `json_object` | `json_schema` (structured outputs supported). - `tools` / `tool_choice` (`none`/`auto`/named) — OpenAI shape. Deprecated `function_call` still accepted. - `reasoning_effort` (`low|medium|high`) — GPT-OSS models only; `reasoning` object also documented for reasoning control. - `chat_template_kwargs` (engine-specific), `compliance: "hipaa"`. - Temperature documented range 0–1 on Together (vs 0–2 at OpenAI) — clamp the UI slider per provider. ## 7. Rate Limits & Errors (https://docs.together.ai/docs/rate-limits) - **Dynamic rate limits** — no published fixed RPM/TPM tiers. Limits adapt per organization and per model based on model capacity and your recent sustained usage; steady traffic raises your limit, spikes get throttled. - `429` with error types `"dynamic_request_limited"` / `"dynamic_token_limited"`; `503` when the platform itself is capacity-constrained even under your rate. - Header `x-ratelimit-reset` = seconds to wait before retrying — use it for backoff. - Error body: OpenAI-shaped `{"error": {"message", "type", "param", "code"}}` but with Together-specific `type`/`code` values; branch on HTTP status (400/401/404/429/500/503/504) for portable handling. ## 8. `/models` Listing `GET /v1/models` returns a **bare JSON array** (NOT wrapped in `{"object":"list","data":[...]}` — deviation from OpenAI; confirmed by live probe). Each entry: ```json { "id": "moonshotai/Kimi-K3", "object": "model", "type": "chat", // chat | language | code | image | video | audio | transcribe | embedding | rerank | moderation "display_name": "Kimi K3", "organization": "Moonshot AI", "context_length": 1000000, "config": { "chat_template": null, "stop": [], ... }, "pricing": { "input": 3, "output": 15, "cached_input": 0.3, "hourly": 0, ... } // USD per 1M tokens } ``` Excellent for dynamic refresh: filter `type in {chat, language}`, use `context_length` + `pricing` directly. No capability flags (vision/tools) in the listing — keep those in the built-in catalog.