SPB Git

Repository Public

tool-combination.md 21.3 KB 541 lines markdown

1> [!WARNING]2> **Preview:** Built-in and custom tools combinations are in [Preview](https://cloud.google.com/products#product-launch-stages) and supported for [Gemini 3](https://ai.google.dev/gemini-api/docs/models#gemini-3) models only.34Gemini allows the combination of [built-in tools](https://ai.google.dev/gemini-api/docs/tools), such5as `google_search`, and [function calling](https://ai.google.dev/gemini-api/docs/function-calling)6(also known as *custom tools*) in a single generation by preserving and exposing7the context history of tool calls. Built-in and custom tool combinations allow8for complex, agentic workflows where, for example, the model can ground itself9in real-time web data before calling your specific business logic.1011Here's an example that enables built-in and custom tool combinations with12`google_search` and a custom function `getWeather`:1314### Python1516    from google import genai17    from google.genai import types1819    client = genai.Client()2021    getWeather = {22        "name": "getWeather",23        "description": "Gets the weather for a requested city.",24        "parameters": {25            "type": "object",26            "properties": {27                "city": {28                    "type": "string",29                    "description": "The city and state, e.g. Utqiaġvik, Alaska",30                },31            },32            "required": ["city"],33        },34    }3536    # Turn 1: Initial request with Google Search (built-in) and getWeather (custom) tools enabled37    response = client.models.generate_content(38        model="gemini-3.8-flash",39        contents="What is the northernmost city in the United States? What's the weather like there today?",40        config=types.GenerateContentConfig(41            tools=[42                types.Tool(43                    google_search=types.GoogleSearch(),  # Built-in tool44                    function_declarations=[getWeather],  # Custom tool45                ),46            ],47            tool_config=types.ToolConfig(48                include_server_side_tool_invocations=True49            )50        ),51    )52    function_call_id = None53    for part in response.candidates[0].content.parts:54        if part.function_call:55            print(f"Function call: {part.function_call.name} (ID: {part.function_call.id})")56            function_call_id = part.function_call.id5758    # Turn 2: Manually build history to circulate both tool and function context59    history = [60        types.Content(61            role="user",62            parts=[types.Part(text="What is the northernmost city in the United States? What's the weather like there today?")]63        ),64        # Response from Turn 1 includes tool_call, tool_response, and thought_signatures65        response.candidates[0].content,66        # Return the function_response67        types.Content(68            role="user",69            parts=[types.Part(70                function_response=types.FunctionResponse(71                    name="getWeather",72                    response={"response": "Very cold. 22 degrees Fahrenheit."},73                    id=function_call_id # Match the ID from the function_call74                )75            )]76        )77    ]7879    response_2 = client.models.generate_content(80        model="gemini-3.8-flash",81        contents=history,82        config=types.GenerateContentConfig(83            tools=[84                types.Tool(85                    google_search=types.GoogleSearch(),86                    function_declarations=[getWeather]87                ),88            ],89            # This flag needs to be enabled for built-in tool context circulation and tool combination90            tool_config=types.ToolConfig(91                include_server_side_tool_invocations=True92            )93        ),94    )9596    for part in response_2.candidates[0].content.parts:97        if part.text:98            print(part.text)99100### Javascript101102    import { GoogleGenAI } from '@google/genai';103104    const client = new GoogleGenAI({});105106    const getWeather = {107        name: "getWeather",108        description: "Get the weather in a given location",109        parameters: {110            type: "OBJECT",111            properties: {112                location: {113                    type: "STRING",114                    description: "The city and state, e.g. San Francisco, CA"115                }116            },117            required: ["location"]118        }119    };120121    async function run() {122        const tools = [123          { googleSearch: {} },124          { functionDeclarations: [getWeather] }125        ];126        // This flag needs to be enabled for built-in tool context circulation and tool combination127        const toolConfig = { includeServerSideToolInvocations: true };128129        // Turn 1: Initial request with Google Search (built-in) and getWeather (custom) tools enabled130        const response1 = await client.models.generateContent({131            model: "gemini-3.8-flash",132            contents: [{role: "user", parts: [{text: "What is the northernmost city in the United States? What's the weather like there today?"}]}],133            config: {134                tools: tools,135                toolConfig: toolConfig,136            },137        });138139        for (const part of response1.candidates[0].content.parts) {140            if (part.functionCall) {141                console.log(`Function call: ${part.functionCall.name} (ID: ${part.functionCall.id})`);142            }143        }144145       const functionCallId = response1.candidates[0].content.parts.find(p => p.functionCall)?.functionCall?.id;146147        // Turn 2: Manually build history to circulate both tool and function context148        const history = [149            {150                role: "user",151                parts:[{text: "What is the northernmost city in the United States? What's the weather like there today?"}]152            },153            // Response from Turn 1 includes tool_call, tool_response, and thought_signatures154            response1.candidates[0].content,155            // Return the function_response156            {157                role: "user",158                parts: [{159                    functionResponse: {160                        name: "getWeather",161                        response: {response: "Very cold. 22 degrees Fahrenheit."},162                        id: functionCallId // Match the ID from the function_call163                    }164                }]165            }166        ];167168        const response2 = await client.models.generateContent({169            model: "gemini-3.8-flash",170            contents: history,171            config: {172                tools: tools,173                toolConfig: toolConfig,174            },175        });176177        for (const part of response2.candidates[0].content.parts) {178            if (part.text) {179                console.log(part.text);180            }181        }182    }183184    run();185186### Go187188    package main189190    import (191        "context"192        "fmt"193        "log"194        "os"195196        "github.com/google/generative-ai-go/genai"197        "google.golang.org/api/option"198    )199200    func main() {201        ctx := context.Background()202        client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("GEMINI_API_KEY")))203        if err != nil {204            log.Exit(err)205        }206        defer client.Close()207208        getWeather := &genai.FunctionDeclaration{209            Name:        "getWeather",210            Description: "Get the weather in a given location",211            Parameters: &genai.Schema{212                Type: genai.Object,213                Properties: map[string]*genai.Schema{214                    "location": {215                        Type:        genai.String,216                        Description: "The city and state, e.g. San Francisco, CA",217                    },218                },219                Required: []string{"location"},220            },221        }222223        model := client.GenerativeModel("gemini-3.8-flash")224        model.Tools = []*genai.Tool{225            {GoogleSearch: &genai.GoogleSearch{}}, // Built-in tool226            {FunctionDeclarations: []*genai.FunctionDeclaration{getWeather}}, // Custom tool227        }228        ist := true229        model.ToolConfig = &genai.ToolConfig{230            IncludeServerSideToolInvocations: &ist, // This flag needs to be enabled for built-in tool context circulation and tool combination231        }232233        chat := model.StartChat()234235        // Turn 1: Initial request with Google Search (built-in) and getWeather (custom) tools enabled236        prompt := genai.Text("What is the northernmost city in the United States? What's the weather like there today?")237        resp1, err := chat.SendMessage(ctx, prompt)238        if err != nil {239            log.Exitf("SendMessage failed: %v", err)240        }241242        if resp1 == nil || len(resp1.Candidates) == 0 || resp1.Candidates[0].Content == nil {243            log.Exit("empty response from model")244        }245246        var functionCallID string247        for _, part := range resp1.Candidates[0].Content.Parts {248            switch p := part.(type) {249            case genai.FunctionCall:250                fmt.Printf("Function call: %s (ID: %s)\n", p.Name, p.ID)251                if p.Name == "getWeather" {252                    functionCallID = p.ID253                }254            }255        }256257        if functionCallID == "" {258            log.Exit("no getWeather function call in response")259        }260261        // Turn 2: Provide function result back to model.262        // Chat history automatically includes tool_call, tool_response, and thought_signatures from Turn 1.263        fr := genai.FunctionResponse{264            Name: "getWeather",265            ID:   functionCallID,266            Response: map[string]any{267                "response": "Very cold. 22 degrees Fahrenheit.",268            },269        }270271        resp2, err := chat.SendMessage(ctx, fr)272        if err != nil {273            log.Exitf("SendMessage for turn 2 failed: %v", err)274        }275276        if resp2 == nil || len(resp2.Candidates) == 0 || resp2.Candidates[0].Content == nil {277            log.Exit("empty response from model in turn 2")278        }279280        for _, part := range resp2.Candidates[0].Content.Parts {281            if txt, ok := part.(genai.Text); ok {282                fmt.Println(string(txt))283            }284        }285    }286287### REST288289    # Turn 1: Initial request with Google Search (built-in) and getWeather (custom) tools enabled290    curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \291    -H "Content-Type: application/json" \292    -H "x-goog-api-key: $GEMINI_API_KEY" \293    -d '{294      "contents": [{295        "role": "user",296        "parts": [{297          "text": "What is the northernmost city in the United States? What'\''s the weather like there today?"298        }]299      }],300      "tools": [{301        "googleSearch": {}302      }, {303        "functionDeclarations": [{304          "name": "getWeather",305          "description": "Get the weather in a given location",306          "parameters": {307              "type": "OBJECT",308              "properties": {309                  "location": {310                      "type": "STRING",311                      "description": "The city and state, e.g. San Francisco, CA"312                  }313              },314              "required": ["location"]315          }316        }]317      }],318      "toolConfig": {319        "includeServerSideToolInvocations": true320      }321    }'322323    # Turn 2: Manually build history to circulate both tool and function context324    # The following request assumes you have captured candidates[0].content from Turn 1 response,325    # and extracted function_call.id for getWeather.326    # Replace FUNCTION_CALL_ID and insert candidate content from turn 1.327    curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \328    -H "Content-Type: application/json" \329    -H "x-goog-api-key: $GEMINI_API_KEY" \330    -d '{331      "contents": [332        {333          "role": "user",334          "parts": [{"text": "What is the northernmost city in the United States? What'\''s the weather like there today?"}]335        },336        YOUR_CANDIDATE_CONTENT_FROM_TURN_1_RESPONSE,337        {338          "role": "user",339          "parts": [{340            "functionResponse": {341              "name": "getWeather",342              "id": "FUNCTION_CALL_ID",343              "response": {"response": "Very cold. 22 degrees Fahrenheit."}344            }345          }]346        }347      ],348      "tools": [{349        "googleSearch": {}350      }, {351        "functionDeclarations": [{352          "name": "getWeather",353          "description": "Get the weather in a given location",354          "parameters": {355              "type": "OBJECT",356              "properties": {357                  "location": {358                      "type": "STRING",359                      "description": "The city and state, e.g. San Francisco, CA"360                  }361              },362              "required": ["location"]363          }364        }]365      }],366      "toolConfig": {367        "includeServerSideToolInvocations": true368      }369    }'370371## How it works372373Gemini 3 models use *tool context circulation* to enable built-in and custom374tool combinations. Tool context circulation makes it possible to preserve and375expose the context of built-in tools and share it with custom tools in the same376call from turn to turn.377378### Enable tool combination379380- You must set the `include_server_side_tool_invocations` flag to `true` to enable tool context circulation.381- Include the [`function_declarations`](https://ai.google.dev/gemini-api/docs/function-calling#function-declarations), along with the built-in tools you want to use, to trigger the combination behavior.382  - If you don't include `function_declarations`, tool context circulation will still act on the included built-in tools, as long as the flag is set.383384### API returns parts385386In a single response, the API returns the `toolCall` and `toolResponse`387parts for the built-in tool call. For the function (custom tool) call, the API388returns the `functionCall` call part, to which the user provides the389`functionResponse` part in the next turn.390391- `toolCall` and `toolResponse`: The API returns these parts to preserve the context of which tools are run on the server side, and the result of their execution, for the next turn.392- `functionCall` and `functionResponse`: The API sends the function call to the user to fill out, and the user sends the result back in the function response (these parts are standard to all [function calling](https://ai.google.dev/gemini-api/docs/function-calling) in the Gemini API, not unique to the tool combination feature).393- ([Code execution](https://ai.google.dev/gemini-api/docs/code-execution) tool only) `executableCode` and `codeExecutionResult`: When using the Code Execution tool, instead of `functionCall` and `functionResponse`, the API returns `executableCode` (the code generated by the model that's meant to be executed) and `codeExecutionResult` (the result of the executable code).394395You must return all parts, including all the [fields](https://ai.google.dev/gemini-api/docs/generate-content/tool-combination#critical-fields) they396contain, back to the model on each turn to maintain context and enable tool397combinations.398399### Critical fields in returned parts400401Certain [parts returned by the API](https://ai.google.dev/gemini-api/docs/generate-content/tool-combination#api-returns-parts) will include `id`,402`tool_type`, and `thought_signature` fields. These fields are critical to403maintaining tool context (and therefore critical to tool combinations); you need404to return all parts *as given in the response* in your subsequent requests.405406- `id`: A unique identifier that maps a call to its response. `id` is **set on407  all function call responses** , regardless of tool context circulation. You *must* provide the same `id` in the function response that the API provides in the function call. Built-in tools automatically share the `id` between the tool call and tool response.408  - Found in all tool-related parts: `toolCall`, `toolResponse`, `functionCall`, `functionResponse`, `executableCode`, `codeExecutionResult`409- `tool_type`: Identifies the specific tool being used; the literal built-in tool or (e.g. `URL_CONTEXT`) or function (e.g. `getWeather`) name.410  - Found in `toolCall` and `toolResponse` parts.411- `thought_signature`: The actual encrypted context embedded in **each412  part returned by the API** . Context can't be reconstructed without thought signatures; if you don't return the thought signatures for all parts in every turn, the model will error out.413  - Found in *all* parts.414415### Tool-specific data416417Some built-in tools return user-visible data arguments specific to the tool418type.419420| Tool | User visible tool call args (if any) | User visible tool response (if any) |421|---|---|---|422| **GOOGLE_SEARCH** | `queries` | `search_suggestions` |423| **GOOGLE_MAPS** | `queries` | `places` `google_maps_widget_context_token` |424| **URL_CONTEXT** | `urls` URLs to be browsed | `urls_metadata` `retrieved_url`: URLs browsed `url_retrieval_status`: Browse status |425| **FILE_SEARCH** | None | None |426427## Example tool combination request structure428429The following request structure shows the request structure of the prompt: "What430is the northernmost city in the United States? What's the weather like there431today?". It combines three tools: the built-in Gemini tools `google_search`432and `code_execution`, and a custom function `get_weather`.433434    {435      "model": "models/gemini-3.8-flash",436      "contents": [{437        "parts": [{438          "text": "What is the northernmost city in the United States? What's the weather like there today?"439        }],440        "role": "user"441      }, {442        "parts": [{443          "thoughtSignature": "...",444          "toolCall": {445            "toolType": "GOOGLE_SEARCH_WEB",446            "args": {447              "queries": ["northernmost city in the United States"]448            },449            "id": "a7b3k9p2"450          }451        }, {452          "thoughtSignature": "...",453          "toolResponse": {454            "toolType": "GOOGLE_SEARCH_WEB",455            "response": {456              "search_suggestions": "..."457            },458            "id": "a7b3k9p2"459          }460        }, {461          "functionCall": {462            "name": "getWeather",463            "args": {464              "city": "Utqiaġvik, Alaska"465            },466            "id": "m4q8z1v6"467          },468          "thoughtSignature": "..."469        }],470        "role": "model"471      }, {472        "parts": [{473          "functionResponse": {474            "name": "getWeather",475            "response": {476              "response": "Very cold. 22 degrees Fahrenheit."477            },478            "id": "m4q8z1v6"479          }480        }],481        "role": "user"482      }],483      "tools": [{484        "functionDeclarations": [{485          "name": "getWeather"486        }]487      }, {488        "googleSearch": {489        }490      }, {491        "codeExecution": {492        }493      }],494      "toolConfig": {495        "includeServerSideToolInvocations": true496      }497    }498499## Tokens and pricing500501Note that `toolCall` and `toolResponse` parts in requests are counted towards502`prompt_token_count`. Since these intermediate tool steps are now visible and503returned to you, they are part of the conversation history. This is only the504case for *requests* , not *responses*.505506The Google Search tool is an exception to this rule. Google Search already507applies its own pricing model at the query level, so tokens are not508double-charged (see the [Pricing](https://ai.google.dev/gemini-api/docs/pricing) page).509510Read the [Tokens](https://ai.google.dev/gemini-api/docs/tokens) page for more information.511512## Limitations513514- Default to `VALIDATED` mode (`AUTO` mode is not supported) when `include_server_side_tool_invocations` flag is enabled515- Built-in tools like `google_search` rely on location and current time information, so if your `system_instruction` or `function_declaration.description` has conflicting location and time information, the tool combination feature might not work well.516517## Supported tools518519Standard tool context circulation applies to server-side (built-in) tools.520Code Execution is also a server-side tool, but has its own built-in solution to521context circulation. Computer Use and function calling are client-side tools,522and also have built-in solutions to context circulation.523524| Tool | Execution side | Context Circulation Support |525|---|---|---|526| [Google Search](https://ai.google.dev/gemini-api/docs/google-search) | Server-side | Supported |527| [Google Maps](https://ai.google.dev/gemini-api/docs/maps-grounding) | Server-side | Supported |528| [URL Context](https://ai.google.dev/gemini-api/docs/url-context) | Server-side | Supported |529| [File Search](https://ai.google.dev/gemini-api/docs/file-search) | Server-side | Supported |530| [Code Execution](https://ai.google.dev/gemini-api/docs/code-execution) | Server-side | Supported (built in, uses `executableCode` and `codeExecutionResult` parts) |531| [Computer Use](https://ai.google.dev/gemini-api/docs/computer-use) | Client-side | Supported (built in, uses `functionCall` and `functionResponse` parts) |532| [Custom functions](https://ai.google.dev/gemini-api/docs/function-calling) | Client-side | Supported (built in, uses `functionCall` and `functionResponse` parts) |533534## What's next535536- Learn more about [Function calling](https://ai.google.dev/gemini-api/docs/function-calling) in the Gemini API.537- Explore the supported tools:538  - [Google Search](https://ai.google.dev/gemini-api/docs/google-search)539  - [Google Maps](https://ai.google.dev/gemini-api/docs/maps-grounding)540  - [URL Context](https://ai.google.dev/gemini-api/docs/url-context)541  - [File Search](https://ai.google.dev/gemini-api/docs/file-search)
Files main
Menu

Theme