SPB Git forge
spboucher.ai ↗

spb/doc-api

Public
2commits 1branches 0releases
15.7 MBsize
maindefault branch
13 days agolast push
Python 88.3% TypeScript 7.6% Shell 4.1%
8.1 KB
View source Raw Blame History

# Capability graph — provider → api_family → endpoint → model → capability / tool (4 providers)

Status: DOCUMENTED · exporter run on the rebuilt merged files on 2026-09-19 (python3 scripts/build_generated.py && python3 scripts/export_capability_graph.py) — 4 provider nodes (anthropic, gemini, openai, xai) · offline-tested against tests/shared/fixtures/{models,endpoints,tools}.json (tests/shared/test_capability_graph.py, 6 tests, fixtures now span all four providers incl. xAI redirects and Gemini dict aliases/tools) Sources: CLAUDE.md record schemas (model, endpoint, tool); generated/fragments/models/{openai,anthropic,xai,gemini}-models.json conventions (record_kind, kind, dict aliases, dict tools, verification.request_note redirects); Mermaid graph LR; Graphviz DOT Last verified: 2026-09-19

Script: scripts/export_capability_graph.py (stdlib, provider-agnostic — every provider value found in the records becomes a node). Outputs: generated/capability-graph.json, generated/capability-graph.mmd, generated/capability-graph.dot.

text
python3 scripts/build_generated.py                                            # merge generated/fragments/** → generated/*.json
python3 scripts/export_capability_graph.py                                    # defaults: generated/{models,endpoints,tools}.json → generated/
python3 scripts/export_capability_graph.py --models generated/models.json --out-dir /tmp/g --max-mermaid-nodes 200

# Node types

type id pattern label attributes carried
provider provider:xai provider —
api_family api_family:gemini/generate_content family provider
endpoint endpoint:gemini/POST /v1beta/models/{model}:generateContent METHOD /path (templated paths kept verbatim) provider, api_family, method, path, status[], streaming, idempotency, placeholder?
model model:xai/grok-4.3 id provider, status[], family, record_kind (model | snapshot | id_only | alias | redirect), context_window, max_output, placeholder?
capability capability:structured_output key as emitted by the provider agent (no synonym folding — structured_outputs and structured_output are two nodes; the registry folds them) shared across providers
tool tool:gemini/google_search exact type string provider, name, category, status[], beta_header, security

record_kind is derived per provider: OpenAI emits it (model, snapshot, id_only, alias); xAI/Gemini use kind — retired_redirect → redirect, alias → alias, everything else → model; Anthropic's kind: "snapshot" denotes a real dated model record and stays model (a bug fixed on 2026-09-19 had excluded them).

# Edge types

edge from → to meaning
HAS_FAMILY provider → api_family from endpoints.json
HAS_ENDPOINT api_family → endpoint
OFFERS provider → model every model record
AVAILABLE_ON model → endpoint model.endpoints[] (placeholder endpoint node when endpoints.json lacks it)
SUPPORTS model → capability capabilities[k] === true
SUPPORTS_UNKNOWN model → capability capabilities[k] === "unknown" (e.g. xAI web_search on grok-4.3, Gemini computer_use on Flash)
(no edge) false, or non-boolean metadata values (lists, notes, Gemini thinking{} objects)
SUPPORTS_TOOL model → tool model.tools[] — strings, or Gemini dicts {type, category, support} where support: false yields no edge and "Supported (Preview)" counts as supported
PROVIDES_TOOL provider → tool every tool record
USABLE_ON tool → endpoint tool.compatible_endpoints[]
COMPATIBLE_WITH tool → model tool.compatible_models[] (placeholder model nodes for unknown ids — 57 of them on Gemini today, mostly retired 2.0/2.5 ids named by tool records)
ALIAS_OF model(alias) → model model.aliases[] strings (parenthetical notes stripped: grok-voice-latest (routes here since …) → grok-voice-latest), and Gemini -latest records whose dict alias {alias: <self>, resolves_to_live: <target>} names an existing id (gemini-flash-latest → gemini-3.8-flash); gemini-pro-latest stays unlinked because its live target gemini-3.1-pro (quota dimension) is not a record id
SNAPSHOT_OF model(snapshot) → model model.snapshots[] and canonical_model
REDIRECTS_TO model(redirect) → model xAI kind: "retired_redirect" with a live target in verification.request_note (grok-3, grok-4-0709, grok-4-fast-reasoning… → grok-4.3; 6 edges)

Edges are de-duplicated; node attributes are merged (first writer wins, later writers fill gaps). meta.providers lists the providers present and meta.per_provider counts real models (not alias/snapshot/redirect/placeholder), endpoints and tools per provider.

# Graceful degradation

meta.inputs.<name>.status is one of loaded | empty | missing | invalid. Missing endpoints.json → endpoint nodes are synthesised from model.endpoints[] (flagged placeholder). Missing tools.json → tool nodes are synthesised from model.tools[] without attributes. Missing models.json → graph contains only providers/families/endpoints/tools. All inputs missing → empty but valid files (graph LR, digraph capability_graph {}), exit code 0.

# Output formats

  • JSON — {meta{generated_at, inputs, providers, per_provider, node_counts, edge_counts, semantics}, nodes[], edges[]}; the registry, the comparator and RAG loaders should read this file, not the Mermaid.
  • Mermaid (graph LR) — alias/snapshot/redirect model nodes are omitted, then the list is truncated at --max-mermaid-nodes (default 400) with a %% N node(s) omitted comment. Shapes: provider [[ ]], family ( ), endpoint [ ], model ([ ]), capability {{ }}, tool > ].
  • DOT — complete graph, rankdir=LR, one shape per node type and a class attribute. Render: dot -Tsvg generated/capability-graph.dot -o graph.svg (or sfdp for the full graph).

# Example queries on the JSON

  • "Which models support structured outputs on all four providers?" → SUPPORTS edges to capability:structured_outputs and capability:structured_output grouped by node provider (or use Registry.models_with("structured_outputs"), which folds the synonym).
  • "Which endpoints can use MCP?" → tool:openai/mcp, tool:anthropic/mcp_toolset, tool:xai/mcp, tool:gemini/mcpServers → USABLE_ON.
  • "Which capabilities are still unknown for model X?" → SUPPORTS_UNKNOWN out-edges — the verification backlog (2 025 edges today, most of them xAI server-tool flags and Gemini per-model feature matrices).
  • "What does grok-3 resolve to?" → follow REDIRECTS_TO; "what is gemini-flash-latest today?" → follow ALIAS_OF (re-run the exporter after each atlas refresh — the alias moved three times in 2026).

# Full run (2026-09-19, after build_generated.py)

Inputs: models 396 records, endpoints 870, tools 69 (all loaded).

anthropic gemini openai xai total
model nodes (real records) 33 93 160 25 311 (+ 191 alias/snapshot/redirect/placeholder nodes = 502)
endpoint nodes 283 143 372 121 919 (870 from endpoints.json + 49 placeholders)
tool nodes 27 18 23 15 83 (69 records + 14 synthesised from model.tools[])

Other node counts: provider 4, api_family 84, capability 191. Edge counts: HAS_FAMILY 84, HAS_ENDPOINT 870, OFFERS 396, AVAILABLE_ON 165, SUPPORTS 3 665, SUPPORTS_UNKNOWN 2 025, SUPPORTS_TOOL 964, PROVIDES_TOOL 69, USABLE_ON 147, COMPATIBLE_WITH 1 065, ALIAS_OF 67, SNAPSHOT_OF 75, REDIRECTS_TO 6.

# Fixture run

tests/shared/fixtures: 11 model records (openai 3, anthropic 2, xai 3 incl. a retired_redirect and a parenthetical alias, gemini 3 incl. a -latest alias record and dict-shaped tools), 11 endpoints, 9 tools → 4 providers, REDIRECTS_TO 1, Gemini support: false tool produces no edge; edge/node assertions live in tests/shared/test_capability_graph.py.