SPB Git forge
spboucher.ai ↗

spb/doc-api

Public
2commits 1branches 0releases
15.7 MBsize
maindefault branch
13 days agolast push
Python 88.3% TypeScript 7.6% Shell 4.1%
10.7 KB
View source Raw Blame History

# Feature detection — supports(model, capability) over the atlas registry (4 providers)

Status: DOCUMENTED (design against the model / tool record schemas in CLAUDE.md and the alias conventions of the four models fragments) · offline-tested with tests/shared/fixtures/{models,tools}.json (11 model records across openai/anthropic/xai/gemini; tests/shared/test_supports.py, 12 tests; supports.ts --selftest) Sources: CLAUDE.md record schemas; generated/fragments/models/openai-models.json (snapshots, canonical_model, record_kind), anthropic-models.json (aliases), xai-models.json (kind: model|alias|retired_redirect|legacy, aliases with parenthetical notes, redirect targets in verification.request_note — e.g. grok-3 → grok-4.3 confirmed live 2026-09-19), gemini-models.json (kind: stable|preview|alias|experimental|agent, -latest records with dict aliases {alias, resolves_to_live, history} — gemini-flash-latest → gemini-3.8-flash live 2026-09-19, dict-shaped tools[] {type, category, support}); generated/fragments/tools/{xai,gemini}-tools.json Last verified: 2026-09-19

Implementation: examples/shared/feature-detection/supports.py (stdlib) and supports.ts.

# Why

Hard-coding "model X supports tools" in application code rots within weeks — faster now that four vendors ship monthly. The atlas already answers the question in generated/models.json (capabilities{} with true | false | "unknown", tools[], aliases[], snapshots[]) and generated/tools.json (compatible_models[]). The registry turns those files into three-valued predicates that applications, the comparator and the provider-choosing agent call at runtime, and papers over the four vendors' different alias conventions and capability vocabularies.

# Semantics

Query Returns Rule
supports(model, cap) true / false / "unknown" Precedence: (1) the exact key in capabilities — a literal boolean yields true/false, an explicit "unknown" wins over everything below; (2) a synonym key from CAPABILITY_SYNONYMS with a boolean value; (3) tool inference: tool_<x> → supports_tool(model, x) (full tri-state); a bare name only counts when it positively matches a tool type (web_search on grok-4.3's tools list → true), never a silent false for unknown words. Missing key, non-boolean metadata (lists, notes), unknown model or no capabilities object → "unknown".
supports_tool(model, tool) tri-state tool is a name (web_search, googleSearch) or a versioned type (web_search_20250305). Match = exact, date-suffix-stripped, or name20…/namev… prefix after normalisation (lower-case, punctuation removed → google_search ≡ googleSearch). Gemini dict entries use their support value (true/false/"Supported (Preview)" → true/"Not supported" → false). tools.json compatible_models is a positive-only signal. Non-empty tools[] without a match → false; empty/missing → "unknown".
tools_for(model) list[str] Union of the model's tools[] types (dict entries with support: false dropped) and tool records whose compatible_models include it, de-duplicated by normalised key (urlContext vs url_context).
models_with(*caps, provider=, include_unknown=False) list[str] Canonical ids where all capabilities are true (or true/"unknown" with include_unknown). Skips record_kind: "snapshot", `kind: alias
compare(a, b) {cap: (tri, tri)} Raw provider keys (no synonym folding) — feeds the comparator, which maps synonyms explicitly.
canonical_id(x) str | None Resolution below.
providers() list[str] Distinct providers present.

# Alias, snapshot and redirect resolution

  1. Case-insensitive; openai/, anthropic/, xai/, gemini/ prefixes tolerated; Gemini resource prefixes models/ and tunedModels/ stripped (models/gemini-3.8-flash).
  2. aliases[] and snapshots[] strings map to their record; trailing parenthetical notes are stripped ("grok-voice-latest (routes here since 2026-08-05)" → grok-voice-latest, "grok-4-fast (probed)" → grok-4-fast).
  3. Dict aliases {alias, resolves_to_live, evidence, history} on Gemini -latest records (kind: "alias"): the record id itself resolves to resolves_to_live when that id exists (gemini-flash-latest → gemini-3.8-flash; gemini-pro-latest → gemini-3.1-pro per the 429 quotaDimensions.model; gemini-flash-lite-latest stays a stub when the target is "unknown (not probed)").
  4. Explicit resolves_to / redirects_to / redirect_to / alias_of fields → target.
  5. xAI kind: "retired_redirect" → target parsed from verification.request_note ("GET /v1/models/grok-3 -> 200 with id 'grok-4.3'"): grok-3, grok-4-0709, grok-4-fast-reasoning → grok-4.3. Records probed docs_only have no target and stay unresolved (queries → "unknown").
  6. A record with canonical_model ≠ id (OpenAI snapshot-kind records) defers to the canonical record when its own capabilities is empty; pointer records (kind alias/redirect, empty capabilities) are followed up to 8 hops with cycle protection.
  7. Unlisted date suffixes (-YYYYMMDD or -YYYY-MM-DD) are stripped as a last resort.

# Cross-provider capability vocabulary

Provider agents emit their own keys. CAPABILITY_SYNONYMS (identical in py/ts) folds the common ones when the exact key is absent:

Portable name OpenAI keys Anthropic keys (real fragment: tool_use, structured_outputs_json, …; tools[] are dicts {type, category, beta_header}) xAI keys Gemini keys
structured_outputs structured_outputs structured_outputs_json structured_outputs structured_output
function_calling function_calling tool_use (+ parallel_tool_use, strict_tool_use) function_calling (+ parallel_tool_calls) function_calling (+ parallel_function_calling, tool function_declarations)
web_search tool_web_search / tools web_search web_search + tools web_search_20250305 web_search (explicit "unknown" on grok-4.3: docs list server tools only on the grok-4.6 page) + tools web_search google_search_grounding + tool google_search
prompt_caching prompt_caching prompt_caching (+ prompt_caching_1h_ttl) prompt_caching_automatic context_caching_implicit, context_caching_explicit
reasoning / extended_thinking reasoning thinking_extended_manual_budget, thinking_adaptive, interleaved_thinking reasoning, reasoning_effort_parameter thinking (per-model thinking{} object, not a boolean)
code_execution tools code_interpreter code_execution + tools code_execution_* code_execution (unknown) + tools code_interpreter, code_execution code_execution + tool code_execution
mcp tool_mcp / tools mcp mcp_connector + tool mcp_toolset remote_mcp (unknown) + tool mcp mcp_servers (UNVERIFIED)
computer_use tool_computer_use / tools computer_use_preview computer_use + tools computer_toolset_* — computer_use + tool computer_use ("Supported (Preview)")

Real-data spot check (2026-09-19, Registry.load() on the rebuilt generated/): models_with("web_search", "structured_outputs", "prompt_caching") returns models from all four providers (OpenAI gpt-4.1/gpt-5.x, Anthropic Claude 4.x, xAI grok-4.6, Gemini 2.5/3.x); canonical_id("grok-3") == "grok-4.3", canonical_id("models/gemini-flash-latest") == "gemini-3.8-flash"; gemini-pro-latest stays unresolved because its live target (gemini-3.1-pro, from the 429 quotaDimensions) is not a record id.

Explicit keys always win: supports("grok-4.3", "web_search") is "unknown" (the flag says so) even though supports_tool("grok-4.3", "web_search") is true (the type string is accepted by the API) — surface both to the user.

# Accepted file shapes

A JSON array; {"records": [...]}, {"models": [...]}, {"tools": [...]}; or an object keyed by id. Missing or malformed files load as empty registries (every query → "unknown" / []).

# Usage

python
from supports import Registry
reg = Registry.load()                                   # generated/models.json + generated/tools.json
reg.providers()                                         # ['anthropic', 'gemini', 'openai', 'xai']
reg.canonical_id("grok-3")                              # 'grok-4.3'   (retired redirect)
reg.canonical_id("models/gemini-flash-latest")          # 'gemini-3.8-flash'
reg.supports("gemini-3.8-flash", "structured_outputs")  # True  (← structured_output)
reg.supports("grok-4.3", "web_search")                  # 'unknown' — explicit flag; reg.supports_tool(...) → True
reg.models_with("web_search", "structured_outputs", "prompt_caching")            # cross-provider, synonyms folded
reg.models_with("web_search", "structured_outputs", "prompt_caching", include_unknown=True, provider="xai")
ts
import { Registry } from "./supports.ts";
const reg = Registry.load();
reg.supportsTool("gemini-3.8-flash", "googleSearch");   // true (== "google_search")
reg.modelsWith(["function_calling"], { provider: "gemini" });

CLI: python3 examples/shared/feature-detection/supports.py <model> <capability>.

# Consumers

  • Provider adapters (docs/architecture/multi-provider-abstraction.md): refuse or warn before sending json_schema / tools to a model that returns false; ask for confirmation on "unknown"; pick the Gemini -latest target explicitly when pinning matters (the alias moved three times in 2026).
  • Fallback chains (resilience.md): pick fallback models with models_with(...), per provider, so the fallback actually supports the request.
  • Capability graph exporter (capability-graph.md): same tri-state (SUPPORTS / SUPPORTS_UNKNOWN / no edge) plus ALIAS_OF and REDIRECTS_TO edges built from the same alias rules — but no synonym folding in the graph (query both keys).

# Limitations

  • The synonym table is small and hand-maintained; keys outside it (e.g. Gemini live_api, xAI deferred_completions) are provider-specific by nature.
  • "Supported" ≠ "supported on every endpoint": xAI server tools are Responses-only (Chat Completions → 422), Gemini tools differ between generateContent, Interactions and Live — consult endpoints[] and tools.json.compatible_endpoints.
  • Account restrictions are not capabilities: gemini-3.1-pro-preview supports everything on paper and returns 429 limit: 0 on a free-tier key; xAI grok-embedding-small is ACCOUNT_RESTRICTED. Check status[] (ACCOUNT_RESTRICTED) alongside supports().