SPB Git

spb/zyquo-cloud Public MIT

Native macOS AI chat client for 12 cloud providers — your keys, every cloud model, one beautiful chat.

macosswiftaichat
Swift 97.4% Shell 1.7% Makefile 1%
10.7 KB
View source Raw Blame History

# Together AI — Provider Research (Phase 0)

Researched 2026-07-30 against official docs (https://docs.together.ai) and a live authenticated GET /v1/models probe (/tmp/zyquo-probe/together.json, 271 models returned: 164 chat + 14 language + image/video/audio/embedding/rerank/etc.). The probe is authoritative for model existence, context lengths, and pricing (embedded pricing object, USD per 1M tokens).

# 1. Base URL & Endpoints

  • Base URL: https://api.together.xyz/v1 (docs also present https://api.together.ai/v1 — both resolve to the same API; Zyquo Cloud uses https://api.together.xyz/v1)
  • POST /v1/chat/completions — chat (the endpoint Zyquo Cloud uses)
  • POST /v1/completions — legacy text completion
  • GET /v1/models — model listing
  • POST /v1/embeddings, /v1/images/generations, /v1/audio/*, /v1/rerank — other modalities (out of scope)
  • Docs also mention an optimized inference host https://api-inference.together.ai/v2 (not needed; standard v1 is the documented OpenAI-compatible surface)

Reference: https://docs.together.ai/reference/chat-completions-1

# 2. Authentication

Authorization: Bearer <TOGETHER_API_KEY> — standard Bearer header. Confirmed by docs and by the live probe.

# 3. Chat Model Catalog (curated serverless chat models)

The live /models response contains 178 chat/language entries, many of which are dedicated-endpoint-only or free-tier/lora artifacts (pricing 0/0). The curated list below covers the main serverless chat models (cross-checked against https://docs.together.ai/docs/serverless-models). Pricing = USD / 1M tokens from the live probe. Context = context_length from the probe.

Capabilities legend: V vision, T tools/function calling, R reasoning/thinking output, J JSON mode / structured output. All models stream. Capability flags marked * are inferred from model family docs, not individually verified on Together.

Model ID (exact) Context $ In $ Out $ Cached Caps
moonshotai/Kimi-K3 1,000,000 3.00 15.00 0.30 T J R
moonshotai/Kimi-K2.7-Code 262,144 0.95 4.00 0.19 T J R*
moonshotai/Kimi-K2.6 262,144 1.20 4.50 0.20 T J R*
moonshotai/Kimi-K2.5-fp4 262,144 0.50 2.80 — T* J*
deepseek-ai/DeepSeek-V4-Pro 512,000 1.74 3.48 0.20 T J R*
deepseek-ai/DeepSeek-V3.1 131,072 0.60 1.70 — T* J* R (hybrid)
deepseek-ai/DeepSeek-R1-0528 163,840 3.00 7.00 — R
deepseek-ai/DeepSeek-R1-Distill-Llama-70B 131,072 2.00 2.00 — R
zai-org/GLM-5.2 512,000 1.40 4.40 0.26 T J R*
zai-org/GLM-5.1 202,752 1.40 4.40 0.26 T* J* R*
zai-org/GLM-5 202,752 1.00 3.20 — T* J* R*
zai-org/GLM-4.7 202,752 0.45 2.00 — T* J* R*
zai-org/GLM-4.6 202,752 0.60 2.20 — T* J* R*
zai-org/GLM-4.5-Air-FP8 131,072 0.20 1.10 — T* J* R*
Qwen/Qwen3.7-Max 1,000,000 1.25 3.75 0.125 T* J* R* (unverified caps)
Qwen/Qwen3.7-Plus 1,000,000 0.32 1.28 — T* J* (unverified caps)
Qwen/Qwen3.6-Plus 1,000,000 0.50 3.00 — T* J* (unverified caps)
Qwen/Qwen3.5-397B-A17B 262,144 0.60 3.60 0.35 T* J* R*
Qwen/Qwen3.5-9B 262,144 0.17 0.25 — T J
Qwen/Qwen3-Next-80B-A3B-Instruct 262,144 0.15 1.50 — T* J*
Qwen/Qwen3-Next-80B-A3B-Thinking 262,144 0.15 1.50 — R T* J*
Qwen/Qwen3-Coder-Next-FP8 262,144 0.50 1.20 — T* J*
Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 262,144 2.00 2.00 — T* J*
Qwen/Qwen3-VL-32B-Instruct 262,144 0.50 1.50 — V T* J*
Qwen/Qwen3-VL-8B-Instruct 262,144 0.18 0.68 — V T* J*
Qwen/QwQ-32B 131,072 1.20 1.20 — R
Qwen/Qwen2.5-VL-72B-Instruct 32,768 1.95 8.00 — V
Qwen/Qwen2.5-72B-Instruct-Turbo 131,072 1.20 1.20 — T* J*
Qwen/Qwen2.5-7B-Instruct-Turbo 32,768 0.30 0.30 — T J
Qwen/Qwen2.5-Coder-32B-Instruct 16,384 0.80 0.80 — —
meta-llama/Llama-4-Scout-17B-16E-Instruct 1,048,576 0.18 0.59 — V T* J*
meta-llama/Llama-3.3-70B-Instruct-Turbo 131,072 1.04 1.04 — T J
meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo 131,072 0.88 0.88 — T* J*
meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo 131,072 0.18 0.18 — T* J*
meta-llama/Llama-3.1-405B-Instruct 4,096 (probe value — likely a listing quirk; flag in Phase 7) 3.50 3.50 — T*
meta-llama/Llama-3.2-3B-Instruct 131,072 0.06 0.06 — —
openai/gpt-oss-120b 131,072 0.15 0.60 — R T J (reasoning_effort)
openai/gpt-oss-20b 131,072 0.05 0.20 — R T J (reasoning_effort)
nvidia/nemotron-3-ultra-550b-a55b 512,288 0.60 3.60 0.20 T J R*
nvidia/NVIDIA-Nemotron-Nano-9B-v2 131,072 0.06 0.25 — R*
MiniMaxAI/MiniMax-M3 524,288 0.30 1.20 0.06 T J R*
MiniMaxAI/MiniMax-M2.7 196,608 0.30 1.20 0.06 T* J* R*
mistralai/Mistral-Small-24B-Instruct-2501 32,768 0.10 0.30 — T* J*
mistralai/Ministral-3-14B-Instruct-2512 262,144 0.20 0.20 — T* J*
mistralai/Mixtral-8x7B-Instruct-v0.1 32,768 0.60 0.60 — J*
google/gemma-4-31B-it 262,144 0.39 0.97 — T J V*
google/gemma-3n-E4B-it 32,768 0.06 0.12 — —
thinkingmachines/Inkling 524,288 1.00 4.05 0.17 T J R*
deepcogito/cogito-v2-1-671b 163,840 1.25 1.25 — R*
LiquidAI/LFM2.5-8B-A1B 128,000 0.03 0.12 — —
arcee-ai/trinity-mini 128,000 0.045 0.15 — —

Notes:

  • The probe also lists zai-org/GLM-4.7-fp4, zai-org/GLM-5-FP4, Qwen/Qwen3.6-35B-A3B-FP8, Qwen/Qwen3.5-35B-A3B, MiniMaxAI/MiniMax-M2, pearl-ai/gemma-4-31b-it, Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 etc. with pricing 0/0 — these appear to be dedicated-endpoint variants; exclude from the serverless catalog (revisit in Phase 7).
  • moonshotai/Kimi-K3 at 1M context and Qwen/Qwen3.7-Max/Qwen3.7-Plus/Qwen3.6-Plus (1M) are the largest-context serverless options.
  • Vision on Together: image_url content parts (OpenAI format); video_url/audio_url/input_audio also accepted on multimodal models (docs).

# 4. Request/Response Format

Fully OpenAI-compatible (POST /v1/chat/completions). Drop-in with OpenAI SDKs — change only base URL + key (https://docs.together.ai/docs/openai-api-compatibility).

Deviations to handle:

  • Extra finish_reason value eos in addition to stop|length|tool_calls.
  • Assistant messages from reasoning models carry a top-level reasoning field (choices[].message.reasoning) with the chain of thought — not OpenAI's shape. Zyquo Cloud's "Thinking…" section should read this field.
  • Usage token details vary: reasoning models nest cached/reasoning tokens under usage.prompt_tokens_details / usage.completion_tokens_details; some models return them flat — read both defensively.
  • logprobs returns Together's own richer shape; seed is best-effort; n (1–128) not supported on all models; logit_bias unsupported on most models; reasoning_effort only honored on GPT-OSS models; vision detail accepted but ignored.
  • Ignored params: service_tier, store, metadata, prediction.
  • Response may include warnings: [{message}] and an echoed prompt array.

# 5. Streaming

  • stream: true → SSE, data: {chat.completion.chunk JSON} lines, terminated by data: [DONE].
  • Chunk deltas follow the OpenAI choices[].delta shape; reasoning models stream a reasoning delta field alongside content (mirror of the non-streaming message.reasoning).
  • Usage in stream: Together returns a usage object on the final chunk(s) by default (observed behavior; stream_options.include_usage is accepted for OpenAI compat — verify exact behavior in Phase 7; marked partially unverified).

# 6. Special Parameters

(Full list: https://docs.together.ai/reference/chat-completions-1)

  • context_length_exceeded_behavior: "truncate" or "error" — Together-specific control of over-context prompts (default error). Expose as an Advanced setting.
  • safety_model: model ID of a moderation model (e.g., Llama Guard) run against output tokens.
  • repetition_penalty (number) and min_p (0–1) — sampler params beyond the OpenAI set; top_k (integer) also supported.
  • echo (bool, include prompt in response), logprobs (0–20, Together shape), seed, n (1–128).
  • response_format: text | json_object | json_schema (structured outputs supported).
  • tools / tool_choice (none/auto/named) — OpenAI shape. Deprecated function_call still accepted.
  • reasoning_effort (low|medium|high) — GPT-OSS models only; reasoning object also documented for reasoning control.
  • chat_template_kwargs (engine-specific), compliance: "hipaa".
  • Temperature documented range 0–1 on Together (vs 0–2 at OpenAI) — clamp the UI slider per provider.

# 7. Rate Limits & Errors

(https://docs.together.ai/docs/rate-limits)

  • Dynamic rate limits — no published fixed RPM/TPM tiers. Limits adapt per organization and per model based on model capacity and your recent sustained usage; steady traffic raises your limit, spikes get throttled.
  • 429 with error types "dynamic_request_limited" / "dynamic_token_limited"; 503 when the platform itself is capacity-constrained even under your rate.
  • Header x-ratelimit-reset = seconds to wait before retrying — use it for backoff.
  • Error body: OpenAI-shaped {"error": {"message", "type", "param", "code"}} but with Together-specific type/code values; branch on HTTP status (400/401/404/429/500/503/504) for portable handling.

# 8. /models Listing

GET /v1/models returns a bare JSON array (NOT wrapped in {"object":"list","data":[...]} — deviation from OpenAI; confirmed by live probe). Each entry:

json
{
  "id": "moonshotai/Kimi-K3",
  "object": "model",
  "type": "chat",            // chat | language | code | image | video | audio | transcribe | embedding | rerank | moderation
  "display_name": "Kimi K3",
  "organization": "Moonshot AI",
  "context_length": 1000000,
  "config": { "chat_template": null, "stop": [], ... },
  "pricing": { "input": 3, "output": 15, "cached_input": 0.3, "hourly": 0, ... }  // USD per 1M tokens
}

Excellent for dynamic refresh: filter type in {chat, language}, use context_length + pricing directly. No capability flags (vision/tools) in the listing — keep those in the built-in catalog.