SPB Git forge

spb/doc-api

Public
2commits 1branches 0releases
15.7 MBsize
maindefault branch
13 days agolast push
Python 88.3% TypeScript 7.6% Shell 4.1%
15.4 KB

# Gemini API — Tools (generateContent) and File Search stores

Status: DOCUMENTED + LIVE_VERIFIED (2026-09-18 run — see "Live verification" section at the end)

Sources:

Last verified: 2026-09-18 (docs only)

Machine-readable twins: generated/fragments/tools/gemini-tools.json (11 tool records) · generated/fragments/parameters/gemini-tools.json (83) · generated/fragments/parameters/gemini-file-search.json (41) · generated/fragments/compatibility/gemini-tool-model-matrix.json (470) · tmp/gemini-parts/tools-endpoints.json (12) · tmp/gemini-parts/tools-objects.json (58).

The guide pages exist in two flavours: /gemini-api/docs/<slug> was rewritten for the Interactions API (POST /v1beta/interactions, tools as {"type": "google_search"}), while /gemini-api/docs/generate-content/<slug> keeps the generateContent shapes (tools: [{"googleSearch": {}}]). This section documents generateContent and notes the Interactions twin where it exists.

# 1. Taxonomy — exact tools[] keys

tools[] key (discovery Tool schema) Who executes Category Config object Model → you You → model Page
functionDeclarations[] your code (client) client FunctionDeclaration{name, description, parameters | parametersJsonSchema, response | responseJsonSchema, behavior} functionCall{name,args,id} (+ thoughtSignature) functionResponse{name,response,id,parts[]} function-calling.md
googleSearch Google (server) server {timeRangeFilter?, searchTypes?{webSearch{}, imageSearch{}}} groundingMetadata — google-search-grounding.md
googleSearchRetrieval Google server, LEGACY {dynamicRetrievalConfig{mode, dynamicThreshold}} groundingMetadata.retrievalMetadata — google-search-grounding.md
googleMaps Google server {enableWidget?} + toolConfig.retrievalConfig.latLng groundingMetadata.groundingChunks[].maps, googleMapsWidgetContextToken — google-maps-grounding.md
urlContext Google server {} urlContextMetadata — url-context.md
codeExecution Google sandbox server {} executableCode, codeExecutionResult, inlineData (echo in history) code-execution.md
computerUse your code (client) client {environment*, excludedPredefinedFunctions[], enablePromptInjectionDetection, disabledSafetyPolicies[]} functionCall (predefined actions) functionResponse + screenshot computer-use.md
fileSearch Google server {fileSearchStoreNames[]*, metadataFilter?, topK?} groundingMetadata.groundingChunks[].retrievedContext — file-search.md
mcpServers[] Google → remote MCP server mcp {name, streamableHttpTransport{url, headers, timeout, sseReadTimeout, terminateOnClose}} UNVERIFIED UNVERIFIED mcp.md
(SDK-side) MCP ClientSession / mcpToTool() SDK on your machine mcp config.tools=[session], automatic_function_calling plain functionCall on the wire plain functionResponse mcp.md

Notes

  • imageSearch is not a top-level key: it is googleSearch.searchTypes.imageSearch {} (sibling webSearch {}), documented for the image model gemini-3.1-flash-image (Nano Banana 2).
  • Server-side tool steps become visible as Part.toolCall {toolType, args, id, toolName} / Part.toolResponse {toolType, response, id} when toolConfig.includeServerSideToolInvocations = true (Gemini 3, Preview). ToolType enum: GOOGLE_SEARCH_WEB, GOOGLE_SEARCH_IMAGE, URL_CONTEXT, GOOGLE_MAPS, FILE_SEARCH (SDK also lists MEDIA_PROCESSING).
  • gemini-3.1-pro-preview-customtools is a separate model endpoint tuned for bash + custom tools (same pricing as 3.1 Pro Preview).

# 2. toolConfig

Field Type Values / notes
functionCallingConfig.mode enum MODE_UNSPECIFIED, AUTO (default: call or text), ANY (forced call, constrained decoding; may reject huge/deep schemas), NONE (never call), VALIDATED (call or text with schema-validated calls; default and only mode when built-in tools / structured outputs are combined or includeServerSideToolInvocations=true)
functionCallingConfig.allowedFunctionNames[] string[] Only with ANY or VALIDATED; restricts the callable set
retrievalConfig.latLng{latitude,longitude} WGS84 degrees User location for Google Maps (and search) grounding
retrievalConfig.languageCode BCP-47 User language
includeServerSideToolInvocations bool Gemini 3 only, Preview: returns toolCall/toolResponse parts that must be echoed back; forces VALIDATED

Interactions API twin: generation_config.tool_choice = auto | any | none | validated or {allowed_tools:{mode, tools[]}}.

# 3. Tool × Model matrix (from model pages "Capabilities" rows + tool guides, 2026-09-18)

✔ documented supported · ✘ documented not supported / retired · – not documented (UNVERIFIED) · P = Preview

Model functionDeclarations googleSearch googleMaps urlContext codeExecution computerUse fileSearch Live API
gemini-3.8-flash ✔ ✔ ✔ ✔ ✔ ✔ P (recommended) ✔ ✘
gemini-3.7-flash ✔ ✔ ✔ ✔ ✔ ✔ P ✔ ✘
gemini-3.6-flash ✔ ✔ ✔ ✔ ✔ ✔ P ✔ ✘
gemini-3.5-flash ✔ ✔ ✔ ✔ ✔ ✔ P ✔ ✘
gemini-3.5-flash-lite ✔ ✔ ✔ ✔ ✔ ✔ P ✔ ✘
gemini-3.1-pro-preview (+ -customtools) ✔ ✔ ✔ ✔ ✔ – ✔ (AI Studio only per model page) ✘
gemini-3.1-flash-lite / -preview ✔ ✔ ✔ ✔ ✔ ✘ ✔ ✘
gemini-3-flash-preview ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✘
gemini-3-pro-preview (not in live model list) ✔ ✔ ✘ ✔ ✔ ✘ (changelog said launched — conflict) ✔ ✘
gemini-2.5-pro / -flash / -flash-lite ✔ ✔ ✔ ✔ ✔ ✘ ✔ (2.5 Flash missing from FS guide table, ✔ on model page) ✘
gemini-2.5-computer-use-preview-10-2025 ✔ (custom fns) – – – – ✔ legacy browser set – ✘
gemini-2.0-flash / -lite RETIRED 2026-06-01
gemini-3.8-live ✔ (NON_BLOCKING default) ✔ ✘ ✘ ✘ ✘ ✘ ✔
gemini-3.8-live-extended-thinking ✔ (async only) ✔ ✘ ✘ ✘ ✘ ✘ ✔
gemini-3.1-flash-live-preview ✔ (sync only) ✔ ✘ ✘ ✘ ✘ ✘ ✔
gemini-2.5-flash-native-audio-preview-12-2025 ✔ (sync + async) ✔ ✘ ✘ ✘ ✘ ✘ ✔
gemini-robotics-er-2-preview ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✘
gemini-3.1-flash-image (Nano Banana 2) ✘ ✔ (+ imageSearch) ✘ ✘ ✘ ✘ ✘ ✘
gemini-3-pro-image ✘ ✔ ✘ ✘ ✘ ✘ ✘ ✘
deep-research-* / antigravity-* Interactions API agents only: google_search, url_context, code_execution (default on), mcp_server, file_search

Full per-(tool, model) records incl. combos: generated/fragments/compatibility/gemini-tool-model-matrix.json.

# 4. Multi-tool combination rules

Combination Support Rule
googleSearch + codeExecution 2.5 and 3.x documented since the 2025-05 multi-tool launch
googleSearch + urlContext 2.5 and 3.x search then read pages
googleSearch + googleMaps Gemini 3.5 Flash and later search & maps guides
built-in tools + functionDeclarations Gemini 3 only, Preview set toolConfig.includeServerSideToolInvocations=true; mode VALIDATED (AUTO unsupported); echo all returned parts with id, toolType, thoughtSignature; do not assume functionCall is the last part
structured outputs (responseSchema/responseJsonSchema) + tools Gemini 3 only, Preview with Google Search, URL context, code execution, File Search, function calling
fileSearch + other built-in tools ✘ File Search guide: "cannot be combined with Grounding with Google Search, URL Context, etc." (but Gemini 3 may combine File Search with function calling)
Live API googleSearch + functionDeclarations only Maps, code execution, URL context, File Search unsupported
computerUse + functionDeclarations ✔ exclude predefined actions and add custom ones (HITL yield_to_user)

Context circulation table (tool-combination guide): Google Search / Maps / URL Context / File Search = server-side, circulation supported; Code Execution = server-side via executableCode/codeExecutionResult; Computer Use & custom functions = client-side via functionCall/functionResponse. toolCall/toolResponse parts echoed in requests count toward promptTokenCount (Search excepted: priced per query, not double-charged).

# 5. Pricing (pricing page "Pricing for tools", 2026-09-18)

Tool Free tier Paid tier
Google Search 500 RPD (shared Flash/Flash-Lite); not for Pro Gemini 3.x: 5,000 free search requests / month (shared across Gemini 3 models), then $14 / 1,000 requests, billed per search query executed (Gemini 3 billing started 2026-01-05). Gemini 2.5: 1,500 RPD free, then $35 / 1,000 grounded prompts (per prompt)
Google Image Search (Nano Banana 2) — same $14 / 1,000 requests for text and image grounding
Google Maps 500 RPD; not for Pro tools table: 1,500 RPD free (Flash/Flash-Lite), 10,000 RPD free (Pro), then $25 / 1,000 grounded prompts; Gemini 3 model tables say 5,000 prompts/month free then $14 / 1,000 search queries → inconsistent, DOCUMENTATION_INCOMPLETE
Code execution free no fee; code + results = output tokens when produced, input ("intermediate") tokens when re-read; runtime not billed
URL context free retrieved content billed as input tokens (usageMetadata.toolUsePromptTokenCount)
Computer use not available regular model token pricing (screenshots = image tokens); legacy 2.5 CU model has its own table
File Search free indexing embeddings $0.15 / 1M tokens (gemini-embedding-2 standard text input is listed at $0.20 on the same page — inconsistent); storage free; query-time embeddings free; retrieved chunks billed as normal input tokens
Function calling / MCP — tokens only
gemini-3.1-pro-preview-customtools — same as gemini-3.1-pro-preview

# 6. Limits

Tool Documented limits
functionDeclarations name ≤128 chars [A-Za-z0-9_.:-]; parameter names ≤64; max 512 declarations (SDK); keep 10–20 active; ANY mode may reject deep schemas; Gemini 3 thought-signature validation (400)
googleSearch Search Suggestions must be displayed; timeRangeFilter needs both bounds
googleMaps text only; off by default; not in Live; prohibited territories / high-risk uses
urlContext 20 URLs/request; 34 MB/URL; public URLs only; no YouTube, Workspace files, audio/video, paywalls
codeExecution Python only; 30 s runtime; ≤5 retries; fixed library list; matplotlib-only graphs; input bounded by context window
computerUse Preview; coordinates 0–999; legacy model 128k in / 64k out
fileSearch 100 MB/file; store totals Free 1 GB · Tier 1 10 GB · Tier 2 100 GB · Tier 3 1 TB (backend ≈3× input); keep stores <20 GB; 20 customMetadata/doc; displayName ≤512; not in Live; not with other built-ins; images ≤4K×4K PNG/JPEG only with models/gemini-embedding-2
Live API no automatic tool response handling; behavior/scheduling/willContinue semantics per model (see matrix)

# 7. Minimal request shape

bash
curl -s "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash-lite:generateContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY" -H "Content-Type: application/json" \
  -d '{"contents":[{"parts":[{"text":"Reply with OK."}]}],
       "tools":[{"googleSearch":{}},{"codeExecution":{}}],
       "toolConfig":{"functionCallingConfig":{"mode":"AUTO"}}}'

# Live verification (2026-09-18)

Run of 2026-09-18 on gemini-3.5-flash-lite (≈ 60 REST calls + 12 WebSocket sessions, estimated ≈ $0.10; raw sanitized responses in tmp-live/gemini-tools/, summary _summary.json, log notes gemini-tools:* in reports/live-requests.jsonl).

Tool Result Key observation
functionDeclarations LIVE_VERIFIED functionCall{name,args,id:"call_…"} + thoughtSignature; echoing the model turn without the signature → 400 Function call is missing a thought_signature; parallel turn: 2 calls, signature only on the first; parametersJsonSchema OK; modes ANY/NONE/VALIDATED accepted; behavior: NON_BLOCKING → 400 outside Live
googleSearch ACCOUNT_RESTRICTED 429 RESOURCE_EXHAUSTED on gemini-3.5-flash-lite and gemini-3.5-flash (free-tier project; also via Interactions google_search)
googleMaps LIVE_VERIFIED groundingChunks[].maps{uri,title,text,placeId}, webSearchQueries, groundingSupports; enableWidget returned no googleMapsWidgetContextToken
urlContext LIVE_VERIFIED candidate.urlContextMetadata.urlMetadata[{retrievedUrl, urlRetrievalStatus}] (example.com → URL_RETRIEVAL_STATUS_ERROR), usageMetadata.toolUsePromptTokenCount
codeExecution LIVE_VERIFIED parts executableCode{language,code,id} → codeExecutionResult{outcome,output,id} → text
computerUse (generateContent, 2.5 model) ACCOUNT_RESTRICTED 429 generate_content_free_tier_input_token_count, limit: 0; Interactions {"type":"computer_use","environment":"browser"} accepted (200) on gemini-3.5-flash-lite
fileSearch LIVE_VERIFIED store → resumable upload → operation → STATE_ACTIVE → grounded answer; groundingChunks[].retrievedContext{title,text,fileSearchStore} (absent in 1 of 3 runs); document delete needs ?force=true
googleSearchRetrieval, mcpServers, SDK MCP not tested no live 2.0 target / undocumented / SDK-side

Machine-readable twins updated: generated/fragments/tools/gemini-tools.json (verification blocks), compatibility/gemini-tool-model-matrix.json (LIVE rows), endpoints/gemini-tools-agents-live.json, objects/gemini-tools-agents-objects.json. Examples: examples/gemini/tools/**, examples/shared/tool-loop/gemini_tool_loop.{py,ts}; tests tests/gemini/test_tools.py.