Repository Public
1# Delegation and tools in GPT-Live23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45GPT-Live delegates reasoning and tool use to a backend while it manages the spoken conversation. Backend work can run through the configured Responses model or, with client delegation, any model, agent, or service your application operates. In either mode, your application owns permissions, confirmations, business records, and task state.67Read more about [steering the live model for delegation and tools](https://developers.openai.com/api/docs/guides/live-prompting#delegation) in the prompting guide.89The event examples on this page use `connection`, a connected primary Live WebSocket or sideband from the [connection guides](https://developers.openai.com/api/docs/guides/voice-websockets?api=live). On a primary connection, wait for `session.started` before calling an event helper. An attached sideband already belongs to a running session.101112131415## Choose a delegation mode1617With **[Responses delegation](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=responses#configure-responses-delegation)**, GPT-Live calls the Responses model you choose, supplies conversation context, and returns backend results to the live conversation. With **[client delegation](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=client#configure-client-delegation)**, your application prepares the context, runs an agent or workflow, and sends results back to GPT-Live.1819Start with Responses delegation if you want GPT-Live to manage requests. Choose client delegation when you need to run your own workflow or review results before sending them to GPT-Live.2021222324| Consideration | Favor Responses delegation when… | Favor client delegation when… |25| ----------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |26| **Implementation effort** | You want GPT-Live to prepare backend requests, manage connections, and return results to the conversation. | You want to build and operate those pieces yourself. |27| **Reviewing backend results** | Backend output can return directly to GPT-Live. | Your application must validate, redact, combine, or discard results before they reach GPT-Live. |28| **Backend capabilities** | Your workflow fits the Responses settings and tools supported by GPT-Live. | You need another backend, multiple models, or API capabilities beyond the managed configuration. |29| **Context ownership** | The conversation context supplied by GPT-Live fits your application. | You need to choose exactly which history, memory, and application state each backend request receives. |30| **Execution policy** | A configured model and tool loop fits the task. | You need custom routing between code and models, fallbacks, checkpoints, or budgets across backend steps. |3132333435For example, a travel assistant can send flight-status questions to an airline service and itinerary changes to a separate planning agent. The application chooses which backend to call and what verified result to return to GPT-Live.3637In both modes, your application tracks task progress and checks permissions and required user confirmations before running custom tools. GPT-Live can continue speaking while your application reviews a backend result. If your application must control when the user hears audio, add [playback controls](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live#control-playback-when-needed).3839Client delegation also requires your application to [maintain conversation context](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=client#keep-the-conversation-context-in-your-application). The delegation event contains metadata, not task text; use transcript events and application state to prepare the backend request.4041Compare latency, task success, and cost on your own workload when [evaluating your voice agent](https://developers.openai.com/cookbook/examples/audio/voice_agent_evaluation). For guidance specific to your existing architecture, see [Migrate to GPT-Live](https://developers.openai.com/api/docs/guides/live-migration#choose-your-delegation-mode).4243Choose the mode when you create the session; to change modes, start a new session.4445## Configure Responses delegation4647Add this delegation configuration when [creating your Live session](https://developers.openai.com/api/docs/guides/live). Choose the Responses model independently of the voice model:4849```javascript50```5152```python53from openai.types.live.session_config_param import SessionConfigParam5455session: SessionConfigParam = {56 "model": "gpt-live-1",57 "delegation": {58 "type": "responses",59 "responses": {60 "model": "gpt-5.6-terra",61 "instructions": "[Your backend prompt]",62 },63 },64}65```666768Start with [GPT-5.6 Terra](https://developers.openai.com/api/docs/models/gpt-5.6-terra), or try [GPT-5.6 Luna](https://developers.openai.com/api/docs/models/gpt-5.6-luna) for cost-sensitive workloads. Compare answer quality and latency on your tasks before choosing a backend model.6970Register supported tools in `delegation.responses.tools`. Set `delegation.responses.tool_choice` to `"auto"` to let the backend choose a tool, `"required"` to require a tool call, or `"none"` to disable tool calls. You can also select a named function.7172Set `delegation.responses.parallel_tool_calls` to `true` to allow multiple tool calls in a response, or `false` for sequential calls. Your application executes custom functions and checks their dependencies and required approvals. These settings apply after GPT-Live delegates; use the [live prompt](https://developers.openai.com/api/docs/guides/live-prompting#delegation) to guide when it should delegate.7374The Responses configuration requires a backend `model` at creation. It supports `function` definitions and `web_search` entries in `tools`. It also exposes `max_output_tokens` (at least 16 when set), `service_tier`, and the `reasoning` and `text` settings supported by the selected backend model. See [Reduce backend latency](#reduce-backend-latency) for settings you can tune.7576If [Fast mode](https://developers.openai.com/api/docs/guides/fast-mode) is available for your model and project, consider it for latency-sensitive calls. For GPT-Live, select it with `delegation.responses.service_tier: "priority"`.7778Send `session.update` with changes in `session.delegation.responses` to update the backend model, instructions, tools, `tool_choice`, or other supported settings during the conversation. Omitted settings keep their current values.7980To switch between Responses and client delegation, create a new Live session. Updating `delegation` to `null` selects client mode, so sending it to a running Responses session fails with `immutable_field_update`.8182These settings use familiar Responses concepts, but Live supports a subset of the standalone Responses API. Live supplies conversation context and initiates delegated work. Configure the backend through the session; the Live `response.create` command uses that configuration and does not accept a standalone Responses request body.8384## Steer the live conversation from your application8586Responses delegation manages the backend workflow, but your application can still send context directly to the GPT-Live model. If you monitor the call through a sideband WebSocket or the main event connection, you can use `session.instructions.append`, `session.thinking.append`, or `session.commentary.append` with `delegation_id: null`. For example, a transcript-based guardrail can append an instruction to redirect the conversation. This steers the live model; it does not change the Responses backend prompt or cancel work already in progress.8788## Handle Responses delegation8990For Responses-backed work, `session.delegation.created` has `target: "responses"` and a `response_id`. Subsequent Responses events arrive inside a `response.event` envelope:9192```json93{94 "type": "response.event",95 "event_id": "event_response_1",96 "delegation_id": "item_9tA2cB6n2V8c4X1z7Q5r9",97 "event": {98 "type": "response.output_text.delta",99 "sequence_number": 4,100 "item_id": "msg_123",101 "output_index": 0,102 "content_index": 0,103 "delta": "The forecast is",104 "logprobs": []105 }106}107```108109When the top-level event type is `response.event`, dispatch on `envelope.event.type`. Save the outer `delegation_id` to associate the backend event with its delegation. Your handler may also receive nested Responses lifecycle events beyond those shown here.110111Live speech and delegated work continue independently. A completed backend response does not itself mean the user heard the answer. Use the Live output transcript and audio for the spoken part of the interaction.112113114115116117### Run a custom function and return its result118119Read completed function calls from nested `response.output_item.done` events. The finished function item contains `call_id`, `name`, and `arguments`; an arguments-done event alone is not sufficient to identify the call.120121Track the response ID from nested `response.created` alongside the outer `delegation_id`. Collect the response's function calls from `response.output_item.done`, and use that collection to determine which tool results to submit before continuing.122123The forwarded lifecycle events, including `response.completed`, contain `response.output: []` even when function calls need results. These events also have an empty `tools` array, `instructions: null`, and no `input` field. Read the individual output-item events for the function calls.124125After executing the authorized operation, append the result as a Responses item:126127```javascript128export function sendUpdate(connection) {129 connection.send({130 type: "response.item.create",131 event_id: "tool_result_1",132 item: {133 type: "function_call_output",134 call_id: "call_123",135 output: '{"status":"confirmed","order_id":"order_123"}',136 },137 });138}139```140141```python142from openai.resources.live.live import AsyncLiveConnection143from openai.resources.live.sideband import AsyncSidebandConnection144from openai.types.responses.response_input_item_param import ResponseInputItemParam145146147async def send_update(148 connection: AsyncLiveConnection | AsyncSidebandConnection,149) -> None:150 item: ResponseInputItemParam = {151 "type": "function_call_output",152 "call_id": "call_123",153 "output": '{"status":"confirmed","order_id":"order_123"}',154 }155 await connection.response.item.create(156 event_id="tool_result_1",157 item=item,158 )159```160161162Then explicitly continue the response:163164```javascript165export function sendUpdate(connection) {166 connection.send({167 type: "response.create",168 event_id: "continue_1",169 });170}171```172173```python174from openai.resources.live.live import AsyncLiveConnection175from openai.resources.live.sideband import AsyncSidebandConnection176177178async def send_update(179 connection: AsyncLiveConnection | AsyncSidebandConnection,180) -> None:181 await connection.response.create(182 event_id="continue_1",183 )184```185186187Send a `response.item.create` result for every pending function call, then send `response.create` to continue the backend response. `response.item.create` has no separate success acknowledgment; keep processing errors and nested Responses lifecycle events.188189Both commands require Responses delegation. The Live `response.create` command uses the backend configuration stored in the session. Use the event payload shown above, and configure the backend model and other settings through the session.190191 192193 194195196## Configure client delegation197198Set `delegation` when [creating your Live session](https://developers.openai.com/api/docs/guides/live):199200```javascript201```202203```python204from openai.types.live.session_config_param import SessionConfigParam205206session: SessionConfigParam = {"model": "gpt-live-1", "delegation": {"type": "client"}}207```208209210Your application configures and runs the backend: choose its model or service, instructions, tools, and routing. If the backend uses the Responses API, set its model and tools in your application's Responses requests.211212When GPT-Live requests help, build the backend request from your saved conversation history and current task state. Check permissions and required confirmations, run the work, and choose which results to return.213214## Keep the conversation context in your application215216For client delegation, **collect transcripts and keep the current task state yourself**.217218Listen for `session.input_transcript.delta` and `session.output_transcript.delta`. These events contain transcript text in `delta`, along with `start_ms` and `end_ms` timestamps. Keep enough history to understand short replies such as “yes,” corrections such as “Thursday, not Friday,” and details supplied earlier. A transcript fragment is not a complete user turn, and transcripts may contain mistakes.219220The separate `session.delegation.created` event contains an `offset_ms` timestamp and delegation metadata, including `delegation.id` and `delegation.target`. It does **not** contain the user's utterance or task text. Use the transcript events and application state to work out what the user wants. Save `delegation.id` so you can match updates to that request.221222Keep long records and full tool output in the backend. If you create a replacement session, restore the relevant context from your application and check which actions already ran before repeating any work.223224### Receive a client delegation225226`session.delegation.created` identifies a delegation:227228```json229{230 "type": "session.delegation.created",231 "event_id": "event_delegation",232 "offset_ms": 1000,233 "delegation": {234 "id": "item_9tA2bF3h7K9m2P5q8R1s4",235 "type": "delegation",236 "target": "client"237 }238}239```240241Save `event.delegation.id` and include it unchanged in updates about this task.242243Return a result using that ID:244245```javascript246export function sendUpdate(connection) {247 connection.send({248 type: "session.commentary.append",249 event_id: "result_123",250 delegation_id: "item_9tA2bF3h7K9m2P5q8R1s4",251 content: "The order shipped today and should arrive tomorrow.",252 });253}254```255256```python257from openai.resources.live.live import AsyncLiveConnection258from openai.resources.live.sideband import AsyncSidebandConnection259260261async def send_update(262 connection: AsyncLiveConnection | AsyncSidebandConnection,263) -> None:264 await connection.session.commentary.append(265 event_id="result_123",266 delegation_id="item_9tA2bF3h7K9m2P5q8R1s4",267 content="The order shipped today and should arrive tomorrow.",268 )269```270271272Use `session.commentary.append` for results GPT-Live should say aloud; it is trained to paraphrase the text. Use `session.thinking.append` for facts or progress that it can use in later replies without saying them when they arrive. You can send multiple updates with the same client delegation ID.273274See [Send the right kind of update](#send-the-right-kind-of-update) for content limits, required fields, and acknowledgment timing.275276 277278279280281## Start with your existing backend prompt282283Use your existing text-agent prompt as a starting point. Keep its task instructions and business rules with the backend, and adapt instructions that assume a text chat or direct control of speech. Explain how to handle voice transcripts and return useful results. Enforce permissions and required confirmations in your application.284285```text286## Voice conversation context287You are helping an assistant in a live voice conversation. Transcripts288can contain mistakes, unfinished phrases, and later corrections. Use289the latest context and verified records. If a needed detail is still290unclear, ask for that detail instead of guessing.291292## Task instructions293[Your task instructions, business rules, available tools,294and confirmation requirements.]295296## Return the result297Return the relevant facts, the task's current status, and the next step.298Report an action as complete after the tool or service confirms success.299If the outcome is unclear, state that and explain what needs to be checked.300```301302Keep large structured payloads, lengthy tool output, and Markdown intended for display in the backend. Give GPT-Live the relevant facts and let it choose how to say them. A concise tool result doesn't need an additional model call to rewrite it for speech.303304With client delegation, [return the result directly to GPT-Live](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=client#receive-a-client-delegation). With Responses delegation, follow the [function-result flow](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=responses#complete-a-client-actionable-function-call) to continue backend work.305306## Send the right kind of update307308Choose an event based on how GPT-Live should use the content:309310| What you want to send | Event |311| ----------------------------------------------------------------------------------------------------------- | ----------------------------- |312| System-level instructions for the live model, such as a greeting, disclosure, or direction to stop speaking | `session.instructions.append` |313| Information for internal reasoning, not spoken on append but usable for relevant user questions | `session.thinking.append` |314| Information the model should speak aloud, paraphrasing the appended text | `session.commentary.append` |315316All three use a plain-string `content`, limited to 500 tokens per append. Include `delegation_id`: use the original client delegation ID for an update about that task, or `null` for general session context. A non-null ID must identify a known client delegation. Instructions still apply to the live session; an ID does not turn them into a separate backend prompt.317318An appended instruction can interrupt the model's current speech or behavior. Use it when the application needs to redirect the conversation; enforce any related tool or action block in application state.319320For quiet progress during a client-managed task:321322```javascript323export function sendUpdate(connection) {324 connection.send({325 type: "session.thinking.append",326 event_id: "availability_progress",327 delegation_id: "item_123",328 content: "Checking Thursday availability. No appointment has been booked.",329 });330}331```332333```python334from openai.resources.live.live import AsyncLiveConnection335from openai.resources.live.sideband import AsyncSidebandConnection336337338async def send_update(339 connection: AsyncLiveConnection | AsyncSidebandConnection,340) -> None:341 await connection.session.thinking.append(342 event_id="availability_progress",343 delegation_id="item_123",344 content="Checking Thursday availability. No appointment has been booked.",345 )346```347348349For a confirmed booking, send the result the user should hear:350351```javascript352export function sendUpdate(connection) {353 connection.send({354 type: "session.commentary.append",355 event_id: "appointment_result",356 delegation_id: "item_123",357 content: "Your appointment is confirmed for Thursday at 2:00 PM",358 });359}360```361362```python363from openai.resources.live.live import AsyncLiveConnection364from openai.resources.live.sideband import AsyncSidebandConnection365366367async def send_update(368 connection: AsyncLiveConnection | AsyncSidebandConnection,369) -> None:370 await connection.session.commentary.append(371 event_id="appointment_result",372 delegation_id="item_123",373 content="Your appointment is confirmed for Thursday at 2:00 PM",374 )375```376377378Only send that result after the booking has actually succeeded. For a session-wide instruction, use `session.instructions.append` with `delegation_id: null`.379380For example, after your application blocks a request under its guardrails, you can redirect the conversation:381382```javascript383export function sendUpdate(connection) {384 connection.send({385 type: "session.instructions.append",386 event_id: "guardrail_block_17",387 delegation_id: null,388 content:389 "Stop speaking about that request. Briefly explain that you cannot help with it, then wait for the user.",390 });391}392```393394```python395from openai.resources.live.live import AsyncLiveConnection396from openai.resources.live.sideband import AsyncSidebandConnection397398399async def send_update(400 connection: AsyncLiveConnection | AsyncSidebandConnection,401) -> None:402 await connection.session.instructions.append(403 event_id="guardrail_block_17",404 delegation_id=None,405 content=(406 "Stop speaking about that request. Briefly explain that you cannot help "407 "with it, then wait for the user."408 ),409 )410```411412413The instruction does not cancel backend work. [Block the affected action and handle any work already running](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live#apply-conversation-guardrails) in your application.414415The corresponding acknowledgements are `session.thinking.appended`, `session.commentary.appended`, and `session.instructions.appended`. Match their `client_event_id` to your outgoing `event_id`. The acknowledgment waits for estimated context injection, not for speech or playback to finish. See [when context reaches the model](https://developers.openai.com/api/docs/guides/live-conversations#understand-when-context-reaches-the-model) for timing and error handling.416417Send facts and brief progress summaries that GPT-Live can use in the conversation. Content sent with `session.thinking.append` can influence later spoken replies; keep secrets and private backend reasoning in your application.418419## Keep updates accurate and useful420421During longer tasks, send an update when something useful changes: a step finishes, a delay matters, or the user needs to answer a question.422423Use `session.thinking.append` for background progress in client mode. Use `session.commentary.append` when the update is useful to say aloud.424425For spoken updates, send `session.commentary.append` with content that matches the task's verified state:426427| State | Example content |428| ---------------------- | ------------------------------------------ |429| Still working | “I'm checking the available appointments.” |430| Completed | “You're booked for Thursday at 2:00 PM.” |431| Failed | “That time is no longer available.” |432| Cancellation confirmed | “Your appointment has been canceled.” |433434When the user changes a request, update the active task in your application. For example, if they change Friday to Thursday, use Thursday for subsequent work and ignore results from the outdated Friday request. Manage any cancellation in the backend, and confirm it succeeded before telling the user that the work was canceled. Interrupting the spoken conversation leaves backend work running.435436Before retrying a failed tool call, check whether the original action already happened. For example, a lost response should not cause a second booking. If the outcome is unclear, say so and offer the next useful step.437438## Share UI context439440Give GPT-Live a concise summary of the current page or task, relevant selections, and facts that help interpret references such as “this option.” Build the summary directly from application state; no extra model call is needed to format it.441442Send UI context at session start and when relevant state changes. Skip unchanged updates and combine rapid changes into a short summary of the latest state. Make changes to previous selections explicit:443444- **Initial context:** “The user is reviewing a restaurant reservation: August 6 at 7 PM, two guests. No reservation has been made.”445- **Correction:** “The selected time is now 8 PM; the previous selection was 7 PM.”446447In either delegation mode, use `session.thinking.append` with `delegation_id: null` for [background context updates](https://developers.openai.com/api/docs/guides/live-conversations#add-context-during-the-conversation). Keep full HTML, DOM trees, large JSON payloads, and interaction logs in your application or backend. Treat page content as reference data, not instructions.448449### Accept typed input450451Send typed values, such as order numbers, to the backend as user-provided data so it can use the exact text.452453454455 456457458With Responses delegation, queue a user message for the backend:459460```javascript461export function sendUpdate(connection) {462 connection.send({463 type: "response.item.create",464 event_id: "typed_order_number",465 item: {466 type: "message",467 role: "user",468 content: [469 {470 type: "input_text",471 text: "My order number is A0042.",472 },473 ],474 },475 });476}477```478479```python480from openai.resources.live.live import AsyncLiveConnection481from openai.resources.live.sideband import AsyncSidebandConnection482from openai.types.responses.response_input_item_param import ResponseInputItemParam483484485async def send_update(486 connection: AsyncLiveConnection | AsyncSidebandConnection,487) -> None:488 item: ResponseInputItemParam = {489 "type": "message",490 "role": "user",491 "content": [{"type": "input_text", "text": "My order number is A0042."}],492 }493 await connection.response.item.create(494 event_id="typed_order_number",495 item=item,496 )497```498499500Send `response.create` when ready to run or continue the backend. If it is waiting for function results, return all required results first. Queuing text does not itself cancel work already running.501502 503504 505506507With client delegation, send the typed value directly to the backend that handles the conversation. If it corrects a running task, update that task instead of starting the same work again. You can mirror a short factual summary into the live session with `session.thinking.append`, or use `session.commentary.append` for a result the user should hear.508509 510511512513514## Add images and visual context515516To help a caller discuss a photo or screen, send the image and relevant context from your application to a vision-capable backend. The backend interprets the image and returns relevant text for GPT-Live to use in conversation. The Live audio frontend does not accept images directly.517518519520 521522523With Responses delegation, configure a vision-capable backend model. Queue a supported Responses image input item with `response.item.create`, then send `response.create` to run or resume backend work. Return all required pending function results before continuing. See [Handle Responses delegation](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=responses#handle-responses-delegation).524525 526527 528529530With client delegation, send visual input to the backend that handles delegated requests, alongside the relevant conversation and application state. Return concise findings using the [client result flow](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=client#receive-a-client-delegation).531532 533534535536537Keep backend image input separate from `session.input`, which seeds the Live frontend with text history at startup. See [Images and vision](https://developers.openai.com/api/docs/guides/images-vision) for supported image formats and model limitations.538539## Reduce backend latency540541Reduce the time between a request for backend work and a useful result for the conversation. Measure [latency at each stage](https://developers.openai.com/api/docs/guides/voice-agents#measure-latency) to locate delays. Compare useful spoken response time and task success on the same scenarios, and see the [voice agent evaluation Cookbook](https://developers.openai.com/cookbook/examples/audio/voice_agent_evaluation) for evaluation guidance.542543544545 546547548### Responses delegation549550Live manages persistent WebSocket connections to Responses and prepares known request configuration in advance. It can also reuse prior response state when the active connection and state support it. Measure response time and reported cache usage on your workload to see the effect.551552Tune the backend through `delegation.responses`:553554- `model`: choose the model that handles reasoning and tool selection independently of the voice model.555- `reasoning.effort`: balance reasoning time and task quality using values supported by that model.556- `service_tier`: use `auto`, `default`, `flex`, or `priority`, subject to model support and project access. `auto` follows the project's configuration. Evaluate the performance and cost of the tier you choose.557558Update supported settings during the session with `session.update`. Your custom tools still run in your application, so slow service calls, queues, and tool-result buffering can delay the answer even when Live manages the Responses connection. Return each required tool result promptly and [continue the backend response](https://developers.openai.com/api/docs/guides/live-delegation?delegation-mode=responses#complete-a-client-actionable-function-call).559560 561562 563564565### Client delegation566567Your application owns the path from delegation receipt to returning a result. Prepare that path while the voice session runs:568569- **Reuse backend connections.** Keep the API client and its connection pool alive across delegations. For repeated Responses calls, consider a persistent [Responses WebSocket](https://developers.openai.com/api/docs/guides/websocket-mode).570- **Prepare known configuration.** Initialize instructions, tools, and connections before the first request needs them. Responses WebSocket mode also supports warming up known request state before generation; follow its [setup guidance](https://developers.openai.com/api/docs/guides/websocket-mode#connect-and-create-responses).571- **Stream useful results.** Return coherent, verified chunks with `session.commentary.append`. Use `session.thinking.append` for quiet progress. Preserve the client delegation ID and the 500-token limit per append. Keep private reasoning in the backend and confirm actions before announcing success.572- **Keep reusable input stable.** Preserve instructions, tool definitions and ordering, and unchanged history prefixes. Append new information after reusable content when your backend supports caching and continuation.573- **Forward complete, useful updates.** Send each result as soon as you have enough text to understand it on its own. Have your backend label updates as progress or results so your application can choose the appropriate append event. If it marks these categories with a text prefix, wait for the full prefix before forwarding the update.574575Measure the first useful spoken answer when comparing this path with Responses delegation.576577 578579580581582### React to transcript fragments583584Processing transcript fragments in your application is optional and works with either delegation mode. User and assistant [transcript fragments](https://developers.openai.com/api/docs/guides/live-conversations#transcript-deltas) arrive over WebSocket or the WebRTC data channel. You can process them with application logic or a lightweight model to start work before a delegation event arrives, or use the transcript itself to trigger application-owned work.585586Use this pattern to:587588- **Reduce waiting.** Start a speculative lookup when enough information is available—for example, checking availability while the user continues describing their preferences.589- **Run guardrails.** Check the growing transcript for requests or responses that need intervention. See [Apply conversation guardrails](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live#apply-conversation-guardrails).590- **Adapt the conversation.** Look for wording that suggests confusion or frustration, then adjust the experience or send a focused instruction.591- **Update the interface.** Highlight relevant controls, populate suggested fields, or show results as they become available.592593For browser applications, use the WebRTC data channel for captions and local UI updates. When transcript processing runs on your server—for guardrails, lightweight model checks, or speculative tool calls—use a [sideband WebSocket](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live#decide-whether-you-need-a-sideband) to receive events and steer the same GPT-Live session directly.594595Process accumulated text when meaningful new information arrives. A fragment may be incomplete, and later speech can change the request. Discard outdated results, coordinate with subsequent delegated work to avoid duplicate actions, and apply your usual permission and confirmation checks before consequential actions.596597Send findings or instructions back to GPT-Live using the [append event that matches the update](#send-the-right-kind-of-update). For work started outside a client delegation, use `delegation_id: null`. Your application applies UI changes and manages tool execution and cancellation.598599### Shared optimizations600601Both delegation modes benefit from the same backend improvements:602603- **Choose the model and reasoning effort for the task.** Compare configurations that meet your accuracy requirements. Use lower reasoning effort when it completes the task reliably.604- **Keep answers concise.** Return the facts and status GPT-Live needs to continue the conversation. Avoid long explanations and extra model calls just to rewrite results for speech.605- **Reduce tool delays and unnecessary calls.** Start authorized work when its inputs are ready, reuse results while they remain valid, and avoid repeating a completed lookup.606- **Run independent work concurrently.** Independent lookup calls can run together. Respect dependencies and required confirmations for actions. `parallel_tool_calls` lets a model request multiple calls; your application still schedules and executes its custom functions.607608See [Latency optimization](https://developers.openai.com/api/docs/guides/latency-optimization) for general Responses guidance and [Prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching) for reusing stable input.609610## Verify the complete interaction611612Verify that the backend completed the intended action and that the client played the expected spoken result. For example, check both the booking record and the audio played after a successful reservation. The backend can finish while the spoken answer is interrupted, so test these outcomes separately.613614Track each application action with its own operation ID and each changed request with a task revision, alongside the GPT-Live delegation ID. Use those records to recognize completed work after reconnects or retries and to discard results for outdated requests.615616Use [Evaluating voice agents](https://developers.openai.com/cookbook/examples/audio/voice_agent_evaluation) for repeatable tests. For an existing Realtime tool loop or chained backend, follow [Migrate to GPT-Live](https://developers.openai.com/api/docs/guides/live-migration).