1# Realtime with tools23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45You can attach tools to a Realtime session so the model can look up data, take actions, or call services during a live conversation. Tool configuration uses the same event surface whether your client is using a [WebRTC data channel](https://developers.openai.com/api/docs/guides/voice-webrtc?api=realtime) or a [WebSocket](https://developers.openai.com/api/docs/guides/voice-websockets?api=realtime).67Use function tools when your application should execute the tool and return the result. Use MCP tools when the Realtime API should connect to a remote tool server for you.89## Choose a tool type1011| Tool type | Use when | Who executes it |12| ------------------------- | ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |13| `function` | Your application owns the business logic, approval checks, or private system access. | Your client or server receives a function call and returns `function_call_output`. |14| `mcp` with `server_url` | You want the model to call tools exposed by a remote MCP server. | The Realtime API calls the remote MCP server. |15| `mcp` with `connector_id` | You use a legacy built-in connector with an existing model. | The Realtime API calls the connector with the authorization you provide. |1617Add tools in **one of two places**:1819- At the **session level** with `session.tools` in [`session.update`](https://developers.openai.com/api/reference/resources/realtime), if you want the tool available for the full session.20- At the **response level** with `response.tools` in [`response.create`](https://developers.openai.com/api/reference/resources/realtime), if you only need the tool for one turn.2122## Configure a function tool2324Function tools are the right default when the tool should run in your application. The model emits function call arguments, your code executes the action, and your code sends the result back with a `function_call_output` item.2526Configure a function tool with session.update2728```javascript29const event = {30 type: "session.update",31 session: {32 type: "realtime",33 model: "gpt-realtime-2.1",34 tools: [35 {36 type: "function",37 name: "lookup_order",38 description: "Look up an order by its order number.",39 parameters: {40 type: "object",41 properties: {42 order_number: {43 type: "string",44 description: "The customer-facing order number.",45 },46 },47 required: ["order_number"],48 },49 },50 ],51 tool_choice: "auto",52 },53};5455ws.send(JSON.stringify(event));56```5758```python59event = {60 "type": "session.update",61 "session": {62 "type": "realtime",63 "model": "gpt-realtime-2.1",64 "tools": [65 {66 "type": "function",67 "name": "lookup_order",68 "description": "Look up an order by its order number.",69 "parameters": {70 "type": "object",71 "properties": {72 "order_number": {73 "type": "string",74 "description": "The customer-facing order number.",75 }76 },77 "required": ["order_number"],78 },79 }80 ],81 "tool_choice": "auto",82 },83}8485ws.send(json.dumps(event))86```8788```ruby89connection.session.update(90 type: :realtime,91 model: "gpt-realtime-2.1",92 tools: [93 {94 type: :function,95 name: "lookup_order",96 description: "Look up an order by its order number.",97 parameters: {98 type: "object",99 properties: {100 order_number: {101 type: "string",102 description: "The customer-facing order number."103 }104 },105 required: ["order_number"]106 }107 }108 ],109 tool_choice: :auto110)111```112113114When the model calls the function, listen for the function call item, run your application logic, then send the output back:115116Send function call output117118```javascript119const event = {120 type: "conversation.item.create",121 item: {122 type: "function_call_output",123 call_id: functionCall.call_id,124 output: JSON.stringify({125 status: "shipped",126 delivery_date: "2026-05-09",127 }),128 },129};130131ws.send(JSON.stringify(event));132ws.send(JSON.stringify({ type: "response.create" }));133```134135```python136event = {137 "type": "conversation.item.create",138 "item": {139 "type": "function_call_output",140 "call_id": function_call["call_id"],141 "output": json.dumps(142 {143 "status": "shipped",144 "delivery_date": "2026-05-09",145 }146 ),147 },148}149150ws.send(json.dumps(event))151ws.send(json.dumps({"type": "response.create"}))152```153154```ruby155connection.conversation.items.create(156 type: :function_call_output,157 call_id: call_id,158 output: JSON.generate(status: "shipped", delivery_date: "2026-05-09")159)160connection.response.create(tool_choice: :none)161```162163164For a full event-by-event walkthrough of function calling, see [Managing conversations](https://developers.openai.com/api/docs/guides/realtime-conversations#function-calling).165166## Configure an MCP tool167168MCP tools are useful when the tool already exists behind a remote MCP server, or when an existing model uses a legacy built-in connector. Unlike function tools, MCP tools are executed by the Realtime API itself.169170In Realtime, the MCP tool shape is:171172- `type: "mcp"`173- `server_label`174- One of `server_url` or `connector_id`175- Optional `authorization` and `headers`176- Optional `allowed_tools`177- Optional `require_approval`178- Optional `server_description`179180This example makes a docs MCP server available for the full session:181182Configure an MCP tool with session.update183184```javascript185const event = {186 type: "session.update",187 session: {188 type: "realtime",189 model: "gpt-realtime-2.1",190 output_modalities: ["text"],191 tools: [192 {193 type: "mcp",194 server_label: "openai_docs",195 server_url: "https://developers.openai.com/mcp",196 allowed_tools: ["search_openai_docs", "fetch_openai_doc"],197 require_approval: "never",198 },199 ],200 },201};202203ws.send(JSON.stringify(event));204```205206```python207event = {208 "type": "session.update",209 "session": {210 "type": "realtime",211 "model": "gpt-realtime-2.1",212 "output_modalities": ["text"],213 "tools": [214 {215 "type": "mcp",216 "server_label": "openai_docs",217 "server_url": "https://developers.openai.com/mcp",218 "allowed_tools": ["search_openai_docs", "fetch_openai_doc"],219 "require_approval": "never",220 }221 ],222 },223}224225ws.send(json.dumps(event))226```227228```ruby229connection.session.update(230 type: :realtime,231 model: "gpt-realtime-2.1",232 output_modalities: [:text],233 tools: [234 {235 type: :mcp,236 server_label: "openai_docs",237 server_url: "https://developers.openai.com/mcp",238 allowed_tools: ["search_openai_docs", "fetch_openai_doc"],239 require_approval: :never240 }241 ]242)243```244245246### Legacy connectors247248`connector_id` is deprecated for models released after September 1,249 2026. Use `server_url` to connect to a remote MCP server, or 250 `tunnel_id` to connect to a local MCP server through 251 [Secure MCP Tunnel](https://developers.openai.com/api/docs/guides/secure-mcp-tunnels). Existing252 models retain connector support. The example below uses253 `gpt-realtime-1.5`, which predates the cutoff.254255Built-in connectors use the same MCP tool shape, but pass `connector_id`256instead of `server_url`. For example, Google Calendar uses257`connector_googlecalendar`. In Realtime, use these built-in connectors for read258actions, such as searching or reading events or emails. Pass the user's OAuth259access token in `authorization`, and narrow the tool surface with260`allowed_tools` when possible:261262Configure a Google Calendar connector263264```javascript265const event = {266 type: "session.update",267 session: {268 type: "realtime",269 model: "gpt-realtime-1.5",270 output_modalities: ["text"],271 tools: [272 {273 type: "mcp",274 server_label: "google_calendar",275 connector_id: "connector_googlecalendar",276 authorization: "<google-oauth-access-token>",277 allowed_tools: ["search_events", "read_event"],278 require_approval: "never",279 },280 ],281 },282};283284ws.send(JSON.stringify(event));285```286287```python288import os289290connector_authorization = os.environ["OPENAI_CONNECTOR_AUTHORIZATION"]291292event = {293 "type": "session.update",294 "session": {295 "type": "realtime",296 "model": "gpt-realtime-1.5",297 "output_modalities": ["text"],298 "tools": [299 {300 "type": "mcp",301 "server_label": "google_calendar",302 "connector_id": "connector_googlecalendar",303 "authorization": connector_authorization,304 "allowed_tools": ["search_events", "read_event"],305 "require_approval": "never",306 }307 ],308 },309}310311ws.send(json.dumps(event))312```313314```ruby315access_token = ENV.fetch("OPENAI_MCP_ACCESS_TOKEN")316317connection.session.update(318 type: :realtime,319 model: "gpt-realtime-1.5",320 output_modalities: [:text],321 tools: [322 {323 type: :mcp,324 server_label: "google_calendar",325 connector_id: "connector_googlecalendar",326 authorization: access_token,327 allowed_tools: ["search_events", "read_event"],328 require_approval: :never329 }330 ]331)332```333334335Remote MCP servers 336 **don't automatically receive the full conversation context**,337 but **they can see any data the model sends in a tool call**.338 **Keep the tool surface narrow** with `allowed_tools`,339 and require approval for any action you would not auto-run.340341## Realtime MCP flow342343Unlike Realtime `function` tools, remote MCP tools are **executed by the Realtime API itself**. **Your client doesn't run the remote tool** and return a `function_call_output`. Instead, your client configures access, listens for MCP lifecycle events, and optionally sends an approval response if the server asks for one.344345A typical flow looks like this:3463471. You send `session.update` or `response.create` with a `tools` entry whose `type` is `mcp`.3481. The server begins importing tools and emits `mcp_list_tools.in_progress`.3491. While listing is still in progress, the model can't call a tool that hasn't loaded yet. If you want to wait before starting a turn that depends on those tools, listen for [`mcp_list_tools.completed`](https://developers.openai.com/api/reference/resources/realtime). The [`conversation.item.done`](https://developers.openai.com/api/reference/resources/realtime) event whose `item.type` is `mcp_list_tools` shows which tool names were actually imported. If import fails, you will receive [`mcp_list_tools.failed`](https://developers.openai.com/api/reference/resources/realtime).3501. The user speaks or sends text, and a response is created, either by your client or automatically by the session configuration.3511. If the model chooses an MCP tool, you will see `response.mcp_call_arguments.delta` and `response.mcp_call_arguments.done`.3521. **If approval is required**, the server adds a conversation item whose `item.type` is `mcp_approval_request`. Your client must answer it with an `mcp_approval_response` item.3531. Once the tool runs, you will see `response.mcp_call.in_progress`. On success, you will later receive a [`response.output_item.done`](https://developers.openai.com/api/reference/resources/realtime) event whose `item.type` is `mcp_call`; on failure, you will receive [`response.mcp_call.failed`](https://developers.openai.com/api/reference/resources/realtime).3541. `response.done` for a response can arrive before its MCP calls finish. After the response is done and all of its MCP calls have finished, send another [`response.create`](https://developers.openai.com/api/reference/resources/realtime) event to let the model use the results and proceed with the conversation. Repeat this step if the model makes additional MCP calls. The Realtime API doesn't create these follow-up responses automatically.355356This event handler logs the main MCP lifecycle events; it doesn't manage follow-up responses:357358Listen for MCP events during a Realtime session359360```javascript361function parseRealtimeEvent(rawMessage) {362 if (typeof rawMessage === "string") {363 return JSON.parse(rawMessage);364 }365366 if (typeof rawMessage?.data === "string") {367 return JSON.parse(rawMessage.data);368 }369370 return JSON.parse(rawMessage.toString());371}372373function getOutputText(item) {374 if (item.type !== "message") return "";375376 return (item.content ?? [])377 .filter((part) => part.type === "output_text")378 .map((part) => part.text)379 .join("");380}381382ws.on("message", (rawMessage) => {383 const event = parseRealtimeEvent(rawMessage);384385 switch (event.type) {386 case "mcp_list_tools.in_progress":387 console.log("Listing MCP tools for item:", event.item_id);388 break;389390 case "mcp_list_tools.completed":391 console.log("MCP tool listing complete for item:", event.item_id);392 break;393394 case "mcp_list_tools.failed":395 console.error("MCP tool listing failed for item:", event.item_id);396 break;397398 case "conversation.item.done":399 if (event.item.type === "mcp_list_tools") {400 const names = event.item.tools.map((tool) => tool.name).join(", ");401 console.log(`MCP tools ready on ${event.item.server_label}: ${names}`);402 }403404 if (event.item.type === "mcp_approval_request") {405 console.log(406 "Approval required for:",407 event.item.name,408 event.item.arguments409 );410 }411 break;412413 case "response.mcp_call_arguments.done":414 console.log("Final MCP call arguments:", event.arguments);415 break;416417 case "response.mcp_call.in_progress":418 console.log("Running MCP tool for item:", event.item_id);419 break;420421 case "response.mcp_call.failed":422 console.error("MCP tool call failed for item:", event.item_id);423 break;424425 case "response.output_item.done":426 if (event.item.type === "mcp_call") {427 console.log(428 `MCP output from ${event.item.server_label}.${event.item.name}:`,429 event.item.output430 );431 }432433 if (event.item.type === "message") {434 console.log("Assistant:", getOutputText(event.item));435 }436 break;437438 case "response.done":439 console.log("Realtime turn complete.");440 break;441 }442});443```444445```python446def on_message(ws, message):447 event = json.loads(message)448 event_type = event["type"]449450 if event_type == "mcp_list_tools.in_progress":451 print("Listing MCP tools for item:", event["item_id"])452 return453454 if event_type == "mcp_list_tools.completed":455 print("MCP tool listing complete for item:", event["item_id"])456 return457458 if event_type == "mcp_list_tools.failed":459 print("MCP tool listing failed for item:", event["item_id"])460 return461462 if event_type == "conversation.item.done":463 item = event["item"]464465 if item["type"] == "mcp_list_tools":466 names = ", ".join(tool["name"] for tool in item["tools"])467 print(f"MCP tools ready on {item['server_label']}: {names}")468 return469470 if item["type"] == "mcp_approval_request":471 print("Approval required for:", item["name"], item["arguments"])472 return473474 if event_type == "response.mcp_call_arguments.done":475 print("Final MCP call arguments:", event["arguments"])476 return477478 if event_type == "response.mcp_call.in_progress":479 print("Running MCP tool for item:", event["item_id"])480 return481482 if event_type == "response.mcp_call.failed":483 print("MCP tool call failed for item:", event["item_id"])484 return485486 if event_type == "response.output_item.done":487 item = event["item"]488489 if item["type"] == "mcp_call":490 print(491 f"MCP output from {item['server_label']}.{item['name']}:",492 item.get("output"),493 )494 return495496 if item["type"] == "message":497 text_parts = [498 part["text"]499 for part in item.get("content", [])500 if part["type"] == "output_text"501 ]502 print("Assistant:", "".join(text_parts))503 return504505 if event_type == "response.done":506 print("Realtime turn complete.")507```508509```ruby510connection.each do |event|511 case event512 when OpenAI::Realtime::McpListToolsInProgress513 puts("Listing MCP tools for item: #{event.item_id}")514 when OpenAI::Realtime::McpListToolsFailed515 warn("MCP tool listing failed for item: #{event.item_id}")516 break517 when OpenAI::Realtime::McpListToolsCompleted518 puts("MCP tools ready for item: #{event.item_id}")519 connection.response.create(520 output_modalities: [:text],521 input: [522 {523 type: :message,524 role: :user,525 content: [526 {527 type: :input_text,528 text: "Which Realtime API transport should browser clients use?"529 }530 ]531 }532 ],533 tool_choice: :required534 )535 when OpenAI::Realtime::ConversationItemDone536 item = event.item537 case item538 when OpenAI::Realtime::RealtimeMcpListTools539 names = item.tools.map(&:name).join(", ")540 puts("MCP tools ready on #{item.server_label}: #{names}")541 when OpenAI::Realtime::RealtimeMcpApprovalRequest542 puts("Approval required for: #{item.name} #{item.arguments}")543 end544 when OpenAI::Realtime::ResponseMcpCallArgumentsDone545 puts("Final MCP call arguments: #{event.arguments}")546 when OpenAI::Realtime::ResponseMcpCallInProgress547 puts("Running MCP tool for item: #{event.item_id}")548 when OpenAI::Realtime::ResponseMcpCallCompleted549 puts("MCP tool call completed: #{event.item_id}")550 when OpenAI::Realtime::ResponseMcpCallFailed551 warn("MCP tool call failed: #{event.item_id}")552 break553 when OpenAI::Realtime::ResponseOutputItemDoneEvent554 item = event.item555 case item556 when OpenAI::Realtime::RealtimeMcpToolCall557 puts("MCP output from #{item.server_label}.#{item.name}: #{item.output}")558 when OpenAI::Realtime::RealtimeConversationItemAssistantMessage559 text = item.content.filter_map do |content|560 content.text if content.type == :output_text561 end.join562 puts("Assistant: #{text}")563 end564 when OpenAI::Realtime::RealtimeErrorEvent565 warn("Realtime API error: #{event.error.message}")566 break567 when OpenAI::Realtime::ResponseDoneEvent568 puts("Realtime turn complete.")569 break570 end571end572```573574575## Common failures576577- [`mcp_list_tools.failed`](https://developers.openai.com/api/reference/resources/realtime): the Realtime API couldn't import tools from the remote server or connector. Check `server_url` or `connector_id`, authentication, server connectivity, and any `allowed_tools` names you specified.578- [`response.mcp_call.failed`](https://developers.openai.com/api/reference/resources/realtime): the model selected a tool, but the tool call didn't complete. Inspect the event payload and the later `mcp_call` item for MCP protocol, execution, or transport errors.579- `mcp_approval_request` with no matching `mcp_approval_response`: the tool call can't continue until your client explicitly approves or rejects it.580- A turn starts while `mcp_list_tools.in_progress` is still active: only tools that have already finished loading are eligible for that turn.581- A response uses `tool_choice: "required"` but no tools are currently available: the model has nothing eligible to call. Wait for `mcp_list_tools.completed`, confirm that at least one tool was imported, or use a different `tool_choice` for turns that don't require a tool.582- MCP tool definition validation fails before import starts: common causes are a duplicate `server_label` in the same `tools` array, setting both `server_url` and `connector_id`, omitting both of them on the initial session creation request, using an invalid `connector_id`, or sending both `authorization` and `headers.Authorization`. For connectors, don't send `headers.Authorization` at all.583584## Approve or reject MCP tool calls585586If a tool requires approval, the Realtime API inserts an `mcp_approval_request` item into the conversation. **To continue**, send a new [`conversation.item.create`](https://developers.openai.com/api/reference/resources/realtime) event whose `item.type` is `mcp_approval_response`.587588Approve an MCP request589590```javascript591function approveMcpRequest(approvalRequestId) {592 const event = {593 type: "conversation.item.create",594 item: {595 id: `mcp_approval_${approvalRequestId}`,596 type: "mcp_approval_response",597 approval_request_id: approvalRequestId,598 approve: true,599 },600 };601602 ws.send(JSON.stringify(event));603}604```605606```python607# Use the ID from the received MCP approval-request item.608def approve_mcp_request(ws, approval_request_id):609 event = {610 "type": "conversation.item.create",611 "item": {612 "id": f"mcp_approval_{approval_request_id}",613 "type": "mcp_approval_response",614 "approval_request_id": approval_request_id,615 "approve": True,616 },617 }618619 ws.send(json.dumps(event))620```621622```ruby623# Use the ID from the received MCP approval-request item.624approval_request_id = item.id625626connection.conversation.items.create(627 type: :mcp_approval_response,628 id: "mcp_approval_#{approval_request_id}",629 approval_request_id: approval_request_id,630 approve: true631)632```633634635If you reject the request, set `approve` to `false` and optionally include a `reason`.636637## Use MCP for one response only638639If MCP should **only be available for a single turn**, attach the same MCP tool object to `response.tools` instead of `session.tools`:640641Add MCP tools on one response642643```javascript644const event = {645 type: "response.create",646 response: {647 output_modalities: ["text"],648 input: [649 {650 type: "message",651 role: "user",652 content: [653 {654 type: "input_text",655 text: "Which transport should I use for browser clients in the Realtime API?",656 },657 ],658 },659 ],660 tools: [661 {662 type: "mcp",663 server_label: "openai_docs",664 server_url: "https://developers.openai.com/mcp",665 allowed_tools: ["search_openai_docs", "fetch_openai_doc"],666 require_approval: "never",667 },668 ],669 },670};671672ws.send(JSON.stringify(event));673```674675```python676event = {677 "type": "response.create",678 "response": {679 "output_modalities": ["text"],680 "input": [681 {682 "type": "message",683 "role": "user",684 "content": [685 {686 "type": "input_text",687 "text": "Which transport should I use for browser clients in the Realtime API?",688 }689 ],690 }691 ],692 "tools": [693 {694 "type": "mcp",695 "server_label": "openai_docs",696 "server_url": "https://developers.openai.com/mcp",697 "allowed_tools": ["search_openai_docs", "fetch_openai_doc"],698 "require_approval": "never",699 }700 ],701 },702}703704ws.send(json.dumps(event))705```706707```ruby708connection.response.create(709 output_modalities: [:text],710 input: [711 {712 type: :message,713 role: :user,714 content: [715 {716 type: :input_text,717 text: "Which Realtime API transport should browser clients use?"718 }719 ]720 }721 ],722 tools: [723 {724 type: :mcp,725 server_label: "openai_docs",726 server_url: "https://developers.openai.com/mcp",727 allowed_tools: ["search_openai_docs", "fetch_openai_doc"],728 require_approval: :never729 }730 ]731)732```733734735This is useful when only one response needs external context, or when different turns should use different MCP servers.736737## Reuse a previously defined server label738739`server_label` is the stable handle for a tool definition in the current740Realtime session. After you define a server or connector once with741`server_label` plus `server_url` or `connector_id`, later `session.update` or742`response.create` events can reference only that same `server_label`, and the743Realtime API will reuse the earlier definition instead of requiring you to send744the full tool object again.745746Reuse a previously defined connector747748```javascript749const event = {750 type: "response.create",751 response: {752 output_modalities: ["text"],753 input: [754 {755 type: "message",756 role: "user",757 content: [758 {759 type: "input_text",760 text: "Check my schedule for this afternoon.",761 },762 ],763 },764 ],765 // Reuses the google_calendar connector defined earlier in this session.766 tools: [767 {768 type: "mcp",769 server_label: "google_calendar",770 },771 ],772 },773};774775ws.send(JSON.stringify(event));776```777778```python779event = {780 "type": "response.create",781 "response": {782 "output_modalities": ["text"],783 "input": [784 {785 "type": "message",786 "role": "user",787 "content": [788 {789 "type": "input_text",790 "text": "Check my schedule for this afternoon.",791 }792 ],793 }794 ],795 # Reuses the google_calendar connector defined earlier in this session.796 "tools": [797 {798 "type": "mcp",799 "server_label": "google_calendar",800 }801 ],802 },803}804805ws.send(json.dumps(event))806```807808```ruby809connection.response.create(810 output_modalities: [:text],811 input: [812 {813 type: :message,814 role: :user,815 content: [816 {817 type: :input_text,818 text: "Check my schedule this afternoon."819 }820 ]821 }822 ],823 tools: [824 {825 type: :mcp,826 server_label: "google_calendar"827 }828 ]829)830```831832833This reuse is session-scoped. If you start a new Realtime session, send the834full MCP definition again so the server can import its tool list.