spb/doc-api

1# Getting started with the Realtime API23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45Build a speech-to-speech voice agent with the Realtime API. The model works directly with audio, maintains conversation state, and can call tools. This guide starts with the Agents SDK for a browser application; use the lower-level connection guides when you need direct control.67For full-duplex conversations with a separate delegated backend, see [GPT-Live](https://developers.openai.com/api/docs/guides/live). To compare voice architectures and chained pipelines, see [Voice agents](https://developers.openai.com/api/docs/guides/voice-agents).89## Build a speech-to-speech voice agent1011Use the Realtime API when the interaction should feel conversational and immediate. This is the best starting point for voice agents that need barge-in, low first-audio latency, natural turn taking, and realtime tool use.1213The usual browser flow is:14151. Your application server creates an ephemeral client secret for the Realtime session.162. Your frontend creates a `RealtimeSession`.173. The session connects over WebRTC in the browser or WebSocket on the server.184. The agent handles audio turns, tools, interruptions, and handoffs inside that session.1920Start a realtime voice session2122```javascript23import { RealtimeAgent, RealtimeSession } from "@openai/agents/realtime";2425const agent = new RealtimeAgent({26  name: "Assistant",27  instructions: "You are a helpful voice assistant.",28});2930const session = new RealtimeSession(agent, {31  model: "gpt-realtime-2.1",32});3334await session.connect({35  apiKey: "ek_...(ephemeral key from your server)",36});37```383940From there, attach tools, handoffs, and guardrails to the `RealtimeAgent` the same way you would attach them to a text agent. Keep audio transport concerns in the session layer, and keep business logic in the agent definition.4142Start with the transport docs when you need lower-level control:4344- [Audio and voice overview](https://developers.openai.com/api/docs/guides/audio)45- [Realtime API with WebRTC](https://developers.openai.com/api/docs/guides/voice-webrtc?api=realtime)46- [Realtime API with WebSocket](https://developers.openai.com/api/docs/guides/voice-websockets?api=realtime)4748## Safety identifiers4950If your application identifies individual end users, include a [safety identifier](https://developers.openai.com/api/docs/guides/safety-best-practices#implement-safety-identifiers) with Realtime API requests. OpenAI recommends safety identifiers but doesn't require them. They help OpenAI detect harmful behavior and target enforcement to an individual user rather than your entire organization. Use a stable, privacy-preserving value, such as a hashed internal user ID.5152For Realtime API requests, send the identifier in the `OpenAI-Safety-Identifier` header. When using ephemeral tokens, set the header on the server-side request that creates the client secret to associate the identifier with the session. When connecting from a trusted server with WebSocket or the unified WebRTC interface, set the header on the connection request.5354Safety identifiers don't carry over from Responses API requests or other sessions. If you use the Responses API `safety_identifier` parameter elsewhere in your application, pass the same stable value when you create or connect each Realtime session.5556## Beta to GA migration5758If you still have a beta Realtime integration, migrate it to the GA interface before moving forward with new work. The most important changes are:5960- Remove the `OpenAI-Beta: realtime=v1` header when calling the GA interface.61- Use [`POST /v1/realtime/client_secrets`](https://developers.openai.com/api/reference/resources/realtime/subresources/client_secrets/methods/create) to create ephemeral credentials for browser or mobile clients.62- Use `/v1/realtime/calls` when establishing WebRTC sessions.63- Update session and event shapes for the GA interface. In particular, set `session.type`, move output audio configuration under `session.audio.output`, and use the newer response event names like `response.output_text.delta`, `response.output_audio.delta`, and `response.output_audio_transcript.delta`.64- If you are moving a speech-to-speech app forward, start from the [browser example](#build-a-speech-to-speech-voice-agent). If you are moving a transcription workflow forward, use [Realtime transcription](https://developers.openai.com/api/docs/guides/realtime-transcription).6566See the [Realtime client events reference](https://developers.openai.com/api/reference/resources/realtime/client-events), [Realtime sessions reference](https://developers.openai.com/api/reference/resources/realtime/subresources/client_secrets), and [browser example](#build-a-speech-to-speech-voice-agent) for the current GA flow.676869707172## Next steps7374- [Managing conversations](https://developers.openai.com/api/docs/guides/realtime-conversations): Configure sessions and handle audio, text, and events.75- [Voice activity detection](https://developers.openai.com/api/docs/guides/realtime-vad): Configure automatic turn detection.76- [Tools and MCP](https://developers.openai.com/api/docs/guides/realtime-mcp): Add functions, MCP servers, and connectors.77- [Prompting voice models](https://developers.openai.com/api/docs/guides/voice-prompting): Use the guide for your Realtime model.78- [Cost optimization](https://developers.openai.com/api/docs/guides/voice-latency-cost?api=realtime): Understand Realtime accounting and caching.79- [Server-side controls](https://developers.openai.com/api/docs/guides/voice-server-controls?api=realtime): Keep tool execution and session control on your server.8081828384858687888990919293949596979899100101102103## Other audio workflows104105The workflow chooser and shared audio vocabulary now live in [Audio and voice](https://developers.openai.com/api/docs/guides/audio). For continuous translation, use [Live translation](https://developers.openai.com/api/docs/guides/realtime-translation). For live captions, use [Live transcription](https://developers.openai.com/api/docs/guides/realtime-transcription); for recorded audio, use [File transcription](https://developers.openai.com/api/docs/guides/speech-to-text).