Repository Public
1# Async tool calling23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45Async tool calling lets the model continue working after it calls a tool, without waiting for that tool's result. Use it to start slow lookup requests early, answer independent parts of a request, and provide results when your application has them.67## How async tools work89A normal [function call](https://developers.openai.com/api/docs/guides/function-calling) pauses the model's turn to wait for a tool response. Set `async: true` on a function or custom tool definition to let the model continue working after issuing that call, before your application returns the output.1011Your application still executes the tool. Async tools don't move execution to12 OpenAI or manage your background jobs.1314This differs from [Background mode](https://developers.openai.com/api/docs/guides/background), which runs response generation asynchronously. Async tool calling lets the model continue working while your application runs a tool.1516When a job finishes, include its output in a later Responses request. Use the original API `call_id` to match the result to its call:1718| Tool type | Call item | Output item |19| --------- | ------------------ | ------------------------- |20| Function | `function_call` | `function_call_output` |21| Custom | `custom_tool_call` | `custom_tool_call_output` |2223## Call an async tool2425Add `async: true` to the tool definition. The corresponding call items in `response.output` include `async: true`.2627Run a weather lookup in the background2829```javascript30import OpenAI from "openai";3132const client = new OpenAI();33const model = "gpt-6-astra";3435const tools = [36 {37 type: "function",38 name: "get_weather",39 description: "Read a demo weather snapshot for a city.",40 async: true,41 strict: true,42 parameters: {43 type: "object",44 properties: { city: { type: "string" } },45 required: ["city"],46 additionalProperties: false,47 },48 },49];50async function getWeather(city) {51 const snapshots = {52 Paris: {53 city: "Paris",54 temperature_c: 22,55 condition: "Clear",56 source: "demo weather snapshot",57 },58 };59 if (typeof city !== "string" || !Object.hasOwn(snapshots, city)) {60 throw new Error(`No demo weather snapshot for ${city}.`);61 }62 return snapshots[city];63}64const instructions =65 "Start the weather lookup and answer the independent packing question " +66 "without waiting. Use the demo weather result when it arrives; never invent it.";6768let response = await client.responses.create({69 model,70 tools,71 instructions,72 input:73 "Check the demo weather snapshot for Paris. Meanwhile, " +74 "list three essentials for any city trip.",75});7677const call = response.output.find((item) => item.type === "function_call");78if (!call || call.name !== "get_weather") {79 throw new Error("The response did not include a weather call.");80}81const { city } = JSON.parse(call.arguments);82let latestResponseId = response.id;8384// Calling an async function starts the application's job immediately.85const job = getWeather(city).catch((error) => ({ error: error.message }));86if (!call.async) {87 // Ordinary synchronous calls must finish before the model resumes.88 await job;89}90console.log(response.output);91// Independent work or conversation turns can happen here.92// Update latestResponseId after each continuation.93const result = await job;94response = await client.responses.create({95 model,96 tools,97 instructions,98 previous_response_id: latestResponseId,99 input: [100 {101 type: "function_call_output",102 call_id: call.call_id,103 output: JSON.stringify(result),104 },105 ],106});107latestResponseId = response.id;108console.log(response.output);109```110111```python112import json113from concurrent.futures import ThreadPoolExecutor114115from openai import OpenAI116from openai.types.responses import FunctionToolParam117118119def get_weather(city):120 # Demo data. Replace this function with your weather service.121 weather = {122 "Paris": {123 "city": "Paris",124 "temperature_c": 22,125 "condition": "Clear",126 "source": "demo weather snapshot",127 }128 }129 return weather[city]130131132worker = ThreadPoolExecutor()133134135def main():136 client = OpenAI()137 model = "gpt-6-astra"138 tools: list[FunctionToolParam] = [139 {140 "type": "function",141 "name": "get_weather",142 "description": "Read the demo weather snapshot for a city.",143 "async": True,144 "strict": True,145 "parameters": {146 "type": "object",147 "properties": {"city": {"type": "string"}},148 "required": ["city"],149 "additionalProperties": False,150 },151 },152 ]153154 instructions = (155 "Start the weather lookup and answer the independent packing "156 "question without waiting. Use the actual tool result when it "157 "arrives; never invent it. Identify the weather as demo data."158 )159 response = client.responses.create(160 model=model,161 tools=tools,162 instructions=instructions,163 input=(164 "Check the demo weather in Paris. Meanwhile, "165 "list three essentials for any city trip."166 ),167 )168169 call = next(item for item in response.output if item.type == "function_call")170 arguments = json.loads(call.arguments)171 if call.name != "get_weather" or arguments != {"city": "Paris"}:172 raise ValueError("Expected a weather lookup for Paris")173174 latest_response_id = response.id175 if call.async_:176 job = worker.submit(get_weather, **arguments)177 print(response.output_text)178 # Independent work or conversation turns can happen here.179 # Update latest_response_id after each continuation.180 result = job.result()181 else:182 result = get_weather(**arguments)183184 response = client.responses.create(185 model=model,186 tools=tools,187 instructions=instructions,188 previous_response_id=latest_response_id,189 input=[190 {191 "type": "function_call_output",192 "call_id": call.call_id,193 "output": json.dumps(result),194 },195 ],196 )197 print(response.output_text)198199200if __name__ == "__main__":201 try:202 main()203 finally:204 worker.shutdown(wait=True)205```206207```go208package main209210import (211 "context"212 "encoding/json"213 "fmt"214215 "github.com/openai/openai-go/v3"216 "github.com/openai/openai-go/v3/responses"217)218219type weatherArguments struct {220 City string `json:"city"`221}222223type weatherSnapshot struct {224 City string `json:"city"`225 TemperatureC int `json:"temperature_c"`226 Condition string `json:"condition"`227 Source string `json:"source"`228}229230func getWeather(city string) weatherSnapshot {231 // Demo data. Replace this function with your weather service.232 if city != "Paris" {233 panic("No demo weather snapshot for " + city)234 }235 return weatherSnapshot{236 City: city, TemperatureC: 22, Condition: "Clear", Source: "demo weather snapshot",237 }238}239240func main() {241 client := openai.NewClient()242 ctx := context.Background()243 tool := responses.ToolParamOfFunction("get_weather", map[string]any{244 "type": "object",245 "properties": map[string]any{"city": map[string]string{"type": "string"}},246 "required": []string{"city"},247 "additionalProperties": false,248 }, true)249 tool.OfFunction.Description = openai.String("Read the demo weather snapshot for a city.")250 tool.OfFunction.Async = openai.Bool(true)251 tools := []responses.ToolUnionParam{tool}252 instructions := "Start the weather lookup and answer the independent packing question " +253 "without waiting. Use the actual tool result when it arrives; never invent it. " +254 "Identify the weather as demo data."255 response, err := client.Responses.New(ctx, responses.ResponseNewParams{256 Model: "gpt-6-astra",257 Tools: tools,258 Instructions: openai.String(instructions),259 Input: responses.ResponseNewParamsInputUnion{OfString: openai.String("Check the demo weather in Paris. Meanwhile, list three essentials for any city trip.")},260 })261 if err != nil {262 panic(err)263 }264 var call responses.ResponseFunctionToolCall265 for _, item := range response.Output {266 if item.Type == "function_call" && item.AsFunctionCall().Name == "get_weather" {267 call = item.AsFunctionCall()268 break269 }270 }271 if call.CallID == "" {272 panic("The response did not include a weather call.")273 }274 var arguments weatherArguments275 if err := json.Unmarshal([]byte(call.Arguments), &arguments); err != nil {276 panic(err)277 }278 latestResponseID := response.ID279 var result weatherSnapshot280 if call.Async {281 job := make(chan weatherSnapshot, 1)282 go func() { job <- getWeather(arguments.City) }()283 fmt.Println(response.OutputText())284 // Independent work or conversation turns can happen here.285 // Update latestResponseID after each continuation.286 result = <-job287 } else {288 result = getWeather(arguments.City)289 }290 output, err := json.Marshal(result)291 if err != nil {292 panic(err)293 }294 functionOutput := responses.ResponseInputItemParamOfFunctionCallOutput(string(output))295 functionOutput.OfFunctionCallOutput.CallID = openai.String(call.CallID)296 response, err = client.Responses.New(ctx, responses.ResponseNewParams{297 Model: "gpt-6-astra",298 Tools: tools,299 Instructions: openai.String(instructions),300 PreviousResponseID: openai.String(latestResponseID),301 Input: responses.ResponseNewParamsInputUnion{OfInputItemList: responses.ResponseInputParam{functionOutput}},302 })303 if err != nil {304 panic(err)305 }306 fmt.Println(response.OutputText())307}308```309310```java311import com.fasterxml.jackson.annotation.JsonProperty;312import com.fasterxml.jackson.databind.ObjectMapper;313import com.openai.core.JsonValue;314import com.openai.models.responses.FunctionTool;315import com.openai.models.responses.Response;316import com.openai.models.responses.ResponseCreateParams;317import com.openai.models.responses.ResponseFunctionToolCall;318import com.openai.models.responses.ResponseInputItem;319import java.util.List;320import java.util.Map;321import java.util.concurrent.CompletableFuture;322323record WeatherArguments(String city) {}324325record WeatherSnapshot(326 String city,327 @JsonProperty("temperature_c") int temperatureC,328 String condition,329 String source) {}330331static WeatherSnapshot getWeather(String city) {332 // Demo data. Replace this function with your weather service.333 if (!city.equals("Paris")) {334 throw new IllegalArgumentException("No demo weather snapshot for " + city);335 }336 return new WeatherSnapshot(city, 22, "Clear", "demo weather snapshot");337}338339FunctionTool tool =340 FunctionTool.builder()341 .name("get_weather")342 .description("Read the demo weather snapshot for a city.")343 .async(true)344 .strict(true)345 .parameters(346 FunctionTool.Parameters.builder()347 .putAdditionalProperty("type", JsonValue.from("object"))348 .putAdditionalProperty(349 "properties", JsonValue.from(Map.of("city", Map.of("type", "string"))))350 .putAdditionalProperty("required", JsonValue.from(List.of("city")))351 .putAdditionalProperty("additionalProperties", JsonValue.from(false))352 .build())353 .build();354String instructions =355 "Start the weather lookup and answer the independent packing question without waiting. Use the actual tool result when it arrives; never invent it. Identify the weather as demo data.";356Response response =357 client358 .responses()359 .create(360 ResponseCreateParams.builder()361 .model("gpt-6-astra")362 .addTool(tool)363 .instructions(instructions)364 .input(365 "Check the demo weather in Paris. Meanwhile, list three essentials for any city trip.")366 .build());367ResponseFunctionToolCall call =368 response.output().stream()369 .flatMap(item -> item.functionCall().stream())370 .filter(item -> item.name().equals("get_weather"))371 .findFirst()372 .orElseThrow(373 () -> new IllegalStateException("The response did not include a weather call."));374WeatherArguments arguments = call.arguments(WeatherArguments.class);375String latestResponseId = response.id();376WeatherSnapshot result;377if (call.async().orElse(false)) {378 CompletableFuture<WeatherSnapshot> job =379 CompletableFuture.supplyAsync(() -> getWeather(arguments.city()));380 System.out.println(response.output());381 // Independent work or conversation turns can happen here.382 // Update latestResponseId after each continuation.383 result = job.join();384} else {385 result = getWeather(arguments.city());386}387response =388 client389 .responses()390 .create(391 ResponseCreateParams.builder()392 .model("gpt-6-astra")393 .addTool(tool)394 .instructions(instructions)395 .previousResponseId(latestResponseId)396 .inputOfResponse(397 List.of(398 ResponseInputItem.ofFunctionCallOutput(399 ResponseInputItem.FunctionCallOutput.builder()400 .callId(call.callId())401 .output(new ObjectMapper().writeValueAsString(result))402 .build())))403 .build());404response.output().stream()405 .flatMap(item -> item.message().stream())406 .flatMap(message -> message.content().stream())407 .flatMap(content -> content.outputText().stream())408 .forEach(text -> System.out.println(text.text()));409```410411```ruby412require "json"413require "openai"414415def get_weather(city)416 # Demo data. Replace this function with your weather service.417 raise "No demo weather snapshot for #{city}" unless city == "Paris"418419 {420 city: city,421 temperature_c: 22,422 condition: "Clear",423 source: "demo weather snapshot"424 }425end426427client = OpenAI::Client.new428tools = [429 OpenAI::Models::Responses::FunctionTool.new(430 name: "get_weather",431 description: "Read the demo weather snapshot for a city.",432 async: true,433 strict: true,434 parameters: {435 type: "object",436 properties: { city: { type: "string" } },437 required: ["city"],438 additionalProperties: false439 }440 )441]442instructions = "Start the weather lookup and answer the independent packing question " \443 "without waiting. Use the actual tool result when it arrives; never invent it. " \444 "Identify the weather as demo data."445response = client.responses.create(446 model: "gpt-6-astra",447 tools: tools,448 instructions: instructions,449 input: "Check the demo weather in Paris. Meanwhile, list three essentials for any city trip."450)451call = response.output.find do |item|452 item.is_a?(OpenAI::Models::Responses::ResponseFunctionToolCall) && item.name == "get_weather"453end454unless call.is_a?(OpenAI::Models::Responses::ResponseFunctionToolCall)455 raise "The response did not include a weather call."456end457458city = JSON.parse(call.arguments).fetch("city")459latest_response_id = response.id460result = if call.async461 job = Thread.new { get_weather(city) }462 puts(response.output_text)463 # Independent work or conversation turns can happen here.464 # Update latest_response_id after each continuation.465 job.value466 else467 get_weather(city)468 end469response = client.responses.create(470 model: "gpt-6-astra",471 tools: tools,472 instructions: instructions,473 previous_response_id: latest_response_id,474 input: [475 OpenAI::Models::Responses::ResponseInputItem::FunctionCallOutput.new(476 call_id: call.call_id,477 output: JSON.generate(result)478 )479 ]480)481puts(response.output_text)482```483484485The response can contain both the async call and an answer. If other conversation turns happen before the job finishes, update `latest_response_id` to continue from the latest response while keeping the original tool `call_id`.486487For earlier dispatch with [streaming](https://developers.openai.com/api/docs/guides/streaming-responses), start the job when its complete call item arrives while you continue consuming the response.488489## Add a wait tool490491A wait tool lets the model choose when it needs a pending result. For example, it can launch two price lookup requests, work on something independent, and wait only when it's ready to compare prices.492493Add a `task_handle` argument to each async tool. The model assigns a handle to each call, and your application binds it to the original API `call_id` and the running job. Keep handles unique throughout the conversation, including completed tasks and repeated lookup requests.494495Define the wait tool as an ordinary synchronous function: omit `async` or set it to `false`. Its schema and behavior belong to your application. `wait_for_tasks` isn't a built-in Responses tool.496497Use these definitions in the request's `tools` array:498499```json500[501 {502 "type": "function",503 "name": "lookup_price",504 "async": true,505 "description": "Look up a product price in the background. Choose a fresh task_handle unique within this conversation, including completed tasks.",506 "strict": true,507 "parameters": {508 "type": "object",509 "properties": {510 "sku": { "type": "string" },511 "task_handle": { "type": "string" }512 },513 "required": ["sku", "task_handle"],514 "additionalProperties": false515 }516 },517 {518 "type": "function",519 "name": "wait_for_tasks",520 "description": "Wait for selected tasks whose results you need. Pass a nonempty list of distinct task_handles from your earlier lookup_price calls. Results arrive on their original calls; this tool returns status only. Do not wait again for results that have already arrived.",521 "strict": true,522 "parameters": {523 "type": "object",524 "properties": {525 "task_handles": {526 "type": "array",527 "items": { "type": "string" }528 }529 },530 "required": ["task_handles"],531 "additionalProperties": false532 }533 }534]535```536537### Register each job538539Register and start each launch before processing a dependent wait. Calls can arrive together or across responses. The following illustrative output items show two launches and a wait that depends on both:540541```json542[543 {544 "type": "function_call",545 "name": "lookup_price",546 "async": true,547 "call_id": "call_widget",548 "arguments": "{\"sku\":\"WIDGET\",\"task_handle\":\"widget_price_1\"}"549 },550 {551 "type": "function_call",552 "name": "lookup_price",553 "async": true,554 "call_id": "call_gadget",555 "arguments": "{\"sku\":\"GADGET\",\"task_handle\":\"gadget_price_1\"}"556 },557 {558 "type": "function_call",559 "name": "wait_for_tasks",560 "call_id": "call_wait",561 "arguments": "{\"task_handles\":[\"widget_price_1\",\"gadget_price_1\"]}"562 }563]564```565566Your application's registry binds each handle to its original call and running job:567568| Task handle | Original call ID | Job |569| ---------------- | ---------------- | ------------------- |570| `widget_price_1` | `call_widget` | WIDGET price lookup |571| `gadget_price_1` | `call_gadget` | GADGET price lookup |572573Keep the registry for the entire conversation to prevent reuse of a completed task's handle.574575### Deliver results before wait status576577Resolve the requested handles in the registry and await only those jobs. Return each newly completed result on its original `call_id`, then return status on the wait call's own `call_id`. This order gives the model the results when it resumes.578579For example, send these output items in the next request's `input` array. The prices are illustrative:580581```json582[583 {584 "type": "function_call_output",585 "call_id": "call_widget",586 "output": "{\"task_handle\":\"widget_price_1\",\"price_cents\":1200,\"currency\":\"USD\"}"587 },588 {589 "type": "function_call_output",590 "call_id": "call_gadget",591 "output": "{\"task_handle\":\"gadget_price_1\",\"price_cents\":1500,\"currency\":\"USD\"}"592 },593 {594 "type": "function_call_output",595 "call_id": "call_wait",596 "output": "{\"status\":\"completed\",\"completed_task_handles\":[\"widget_price_1\",\"gadget_price_1\"]}"597 }598]599```600601Set `previous_response_id` to the latest response ID, and include the tools and instructions in the continuation request. Your application can also deliver results as they become available, without a wait call. Only use the wait tool when the model's next step depends on results that haven't arrived.602603## Compatibility604605Async tool calling is supported by GPT-6 Astra and later models.606607Async execution applies to function and custom tools that your application runs. It doesn't apply to hosted built-in tools. Use direct tool calls; don't configure async tools for [programmatic tool calling](https://developers.openai.com/api/docs/guides/tools-programmatic-tool-calling).608609In [Multi-agent mode](https://developers.openai.com/api/docs/guides/responses-multi-agent), don't combine async tools with parallel tool calls.