SPB Git

Repository Public

voice-sip.md 21.2 KB 465 lines markdown

1# Telephony and SIP23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45Choose your API to see its connection steps and session events.6789## Choose a telephony connection1011A phone call can reach GPT-Live through a SIP trunk or through an application that relays audio. Choose the path that fits your existing phone system and where your application needs to process audio.1213| Connection          | Audio and application responsibilities                                                                                                                   |14| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |15| Direct SIP          | The provider exchanges call audio with OpenAI. Your application handles webhooks, session configuration, call decisions, and business logic.             |16| Server audio bridge | Your application relays provider or room audio to GPT-Live over WebSocket. It manages both connections, event translation, playback, and call lifecycle. |1718A provider's connection to your application and your application's connection to OpenAI are separate. For example, a caller can join a room through SIP while an agent in that room connects to GPT-Live over WebSocket.1920Using Twilio, Telnyx, LiveKit, or Daily/Pipecat? See [GPT-Live partner integrations](https://developers.openai.com/api/docs/guides/live-partner-integrations) for provider-specific guides.2122### Direct SIP2324Direct SIP keeps call audio on the provider-to-OpenAI media path. SIP signaling uses TLS, and GPT-Live requires SRTP for call audio. Your backend still owns the incoming-call decision, session configuration, authorization, and business logic.2526Use a [sideband connection](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live) when your backend needs to receive session events or send commands. It attaches to the existing conversation while SIP carries the audio. Assign one handler to each action so that duplicate webhook deliveries or events observed on multiple connections don't execute tools twice.2728### Handle the call lifecycle2930Confirm that GPT-Live SIP support is enabled for your project and that your31  provider's SIP trunk is routed to that project before using this flow.3233#### Receive the incoming call3435Configure your project's [webhook endpoint](https://developers.openai.com/api/docs/guides/webhooks) for `live.transport.incoming`. Verify the webhook signature and deduplicate deliveries, then accept or reject the call.3637The webhook identifies a SIP call with `data.type: "sip"` and provides `data.session_id`. Use that session ID unchanged for every Live call action. Treat `data.sip_headers` as untrusted caller metadata, not authorization.3839Existing integrations may still receive the deprecated `live.call.incoming` event, which has no `data.type`. During migration, handle both names and retain the old subscription until legacy deliveries and retries have drained. The same pending call can also emit a Realtime webhook; assign one handler to the accept/reject decision rather than accepting through both APIs.4041#### Accept or reject the call4243Apply your application's authorization and routing rules. To [accept the call](https://developers.openai.com/api/reference/resources/live/subresources/sessions/methods/accept), send an authenticated `POST /v1/live/sessions/{session_id}/accept` request with a top-level `session` object:4445```json46{47  "session": {48    "type": "live",49    "model": "gpt-live-1",50    "instructions": "You are answering an inbound support call.",51    "audio": { "output": { "voice": "marin" } },52    "delegation": { "type": "client" }53  }54}55```5657Use `Authorization: Bearer $OPENAI_API_KEY` from your trusted backend for call-control requests. Choose the voice and delegation mode at acceptance. SIP negotiates the audio format, so omit `audio.format`. The example selects client delegation; your backend must handle delegated work. See [Delegation and tools](https://developers.openai.com/api/docs/guides/live-delegation) for client and Responses configurations.5859A successful acceptance returns `200 OK` with an empty body after session initialization. Handle HTTP errors before treating the call as accepted.6061To [reject the call](https://developers.openai.com/api/reference/resources/live/subresources/sessions/methods/reject), send `POST /v1/live/sessions/{session_id}/reject` with a SIP status, such as `{ "status_code": 486 }` for busy. The status must be an integer from 300 through 699. The first accept or reject decision wins; a later competing decision returns `decision_already_made`.6263#### Attach your backend6465After acceptance, connect a [sideband WebSocket](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live) at `wss://api.openai.com/v1/live/sessions/{session_id}/attach`. Use the accepted session ID and the same project authentication and connection headers. Do not send `session.start` again.6667SIP carries the call audio. Use the sideband for transcripts, delegation, tools, commands, and reflected audio. Choose one owner for each side effect, even if multiple connections observe an event.6869#### Observe keypad events7071The sideband receives `transport.dtmf.received` when the caller presses a key and `transport.dtmf.send` after a hosted tool successfully sends a tone. Both are notifications only. The `event` field contains one of `0`–`9`, `*`, `#`, or `A`–`D`.7273#### Transfer or end the call7475To [transfer the call](https://developers.openai.com/api/reference/resources/live/subresources/sessions/methods/refer), send `POST /v1/live/sessions/{session_id}/refer` with `{ "target_uri": "sip:agent@example.com" }` for your destination. To [hang up](https://developers.openai.com/api/reference/resources/live/subresources/sessions/methods/hangup), send `POST /v1/live/sessions/{session_id}/hangup` with no request body. Both return `200 OK` with an empty body on success.7677Keep the sideband open until `session.closed` supplies final usage, then release application resources. If the connection drops first, record finalization as incomplete. See [Usage and graceful close](https://developers.openai.com/api/docs/guides/live-conversations#usage-and-graceful-close) for finalization and close reasons.7879This flow accepts inbound calls. Creating an outbound SIP call through `POST /v1/live/sessions` is not supported; use the relevant [partner integration](https://developers.openai.com/api/docs/guides/live-partner-integrations) for provider-owned outbound calling.8081### Server audio bridges8283Use the [GPT-Live WebSocket connection](https://developers.openai.com/api/docs/guides/voice-websockets?api=live) when your application receives an audio stream from a phone provider or an agent framework. The application authenticates both connections, translates their event envelopes, and relays audio in both directions.8485GPT-Live supports raw G.711 μ-law and A-law audio at 8 kHz over WebSocket. When the provider stream uses the same codec, sample rate, and channel count, your application can forward the raw audio bytes without converting them to PCM. Preserve audio order and wrap the audio bytes in the message format required by each connection.8687Have your bridge manage queued audio, interruptions, and call termination. Account for audio buffered by the provider when handling playback. See [Managing sessions](https://developers.openai.com/api/docs/guides/live-conversations) for the Live session lifecycle and [Migrate to GPT-Live](https://developers.openai.com/api/docs/guides/live-migration) for changes to turn-taking and playback control.8889Keep the provider's call or room identifier alongside the OpenAI session ID so you can trace a conversation across both systems.9091## Next steps with GPT-Live9293- [WebSockets](https://developers.openai.com/api/docs/guides/voice-websockets?api=live): connect a server audio stream to GPT-Live.94- [Webhooks and server-side controls](https://developers.openai.com/api/docs/guides/voice-server-controls?api=live): manage a session from your backend.95- [Delegation and tools](https://developers.openai.com/api/docs/guides/live-delegation): connect speech to your reasoning and tool backend.96- [Managing sessions](https://developers.openai.com/api/docs/guides/live-conversations): handle transcripts, session state, and close.979899100101102103[SIP](https://en.wikipedia.org/wiki/Session_Initiation_Protocol) is a104protocol used to make phone calls over the internet. With SIP and the105Realtime API you can direct incoming phone calls to the API.106107## Overview108109If you want to connect a phone number to the Realtime API,110use a SIP trunking provider (e.g., Twilio). This is a service that converts your phone call111to IP traffic. After you purchase a phone number from your SIP trunking112provider, follow the instructions below.113114Start by creating a [webhook](https://developers.openai.com/api/docs/guides/webhooks) for incoming calls, through your **platform.openai.com** [settings](https://platform.openai.com/settings) > Project > **Webhooks**.115Then, point your SIP trunk at the OpenAI SIP endpoint, using the project ID116for which you configured the webhook, e.g., `sip:$PROJECT_ID@sip.api.openai.com;transport=tls`.117For European data residency, use `sip:$PROJECT_ID@sip-eu.api.openai.com;transport=tls` instead.118To find your `$PROJECT_ID`, visit [settings](https://platform.openai.com/settings) > Project > **General**. That page will display the project ID, which119will have a `proj_` prefix.120121When OpenAI receives SIP traffic associated with your project,122your webhook will be fired. The event fired will be a123[`realtime.call.incoming`](https://developers.openai.com/api/reference/resources/webhooks) event,124like the example below:125126```127POST https://my_website.com/webhook_endpoint128user-agent: OpenAI/1.0 (+https://platform.openai.com/docs/webhooks)129content-type: application/json130webhook-id: wh_685342e6c53c8190a1be43f081506c52 # unique id for idempotency131webhook-timestamp: 1750287078 # timestamp of delivery attempt132webhook-signature: v1,K5oZfzN95Z9UVu1EsfQmfVNQhnkZ2pj9o9NDN/H/pI4= # signature to verify authenticity from OpenAI133134{135  "object": "event",136  "id": "evt_685343a1381c819085d44c354e1b330e",137  "type": "realtime.call.incoming",138  "created_at": 1750287018, // Unix timestamp139  "data": {140    "call_id": "some_unique_id",141    "sip_headers": [142      { "name": "From", "value": "sip:+142555512112@sip.example.com" },143      { "name": "To", "value": "sip:+18005551212@sip.example.com" },144      { "name": "Call-ID", "value": "03782086-4ce9-44bf-8b0d-4e303d2cc590"}145    ]146  }147}148```149150From this webhook, you can accept or reject the call, using the `call_id` value from the webhook.151When accepting the call, you'll provide the needed configuration152(instructions, voice, etc) for the Realtime API session.153Once established, you can set up a WebSocket and monitor the session as usual. The APIs to154accept, reject, monitor, refer, and hangup the call are documented below.155156## Accept the call157158Use the [Accept call endpoint](https://developers.openai.com/api/reference/resources/realtime/subresources/calls/methods/accept) to159approve the inbound call and configure the realtime session that will answer it.160Send the same parameters you would send in a161[`create client secret`](https://developers.openai.com/api/reference/resources/realtime/subresources/client_secrets/methods/create)162request, i.e., ensure the realtime model, voice, tools, or instructions are set before bridging the163call to the model.164165```bash166curl -X POST "https://api.openai.com/v1/realtime/calls/$CALL_ID/accept" \167  -H "Authorization: Bearer $OPENAI_API_KEY" \168  -H "Content-Type: application/json" \169  -d '{170        "type": "realtime",171        "model": "gpt-realtime-2.1",172        "instructions": "You are Alex, a friendly concierge for Example Corp."173      }'174```175176177The request path must include the `call_id` from the178[`realtime.call.incoming`](https://developers.openai.com/api/reference/resources/webhooks)179webhook, and every request requires the `Authorization` header shown above. The180endpoint returns `200 OK` once the SIP leg is ringing and the realtime session181is being established.182183## Reject the call184185Use the [Reject call endpoint](https://developers.openai.com/api/reference/resources/realtime/subresources/calls/methods/reject) to186decline an invite when you do not want to handle the incoming call, (e.g., from187an unsupported country code.) Supply the `call_id` path parameter188and an optional SIP `status_code` (e.g., `486` to indicate "busy") in the JSON189body to control the response sent back to the carrier.190191```bash192curl -X POST "https://api.openai.com/v1/realtime/calls/$CALL_ID/reject" \193  -H "Authorization: Bearer $OPENAI_API_KEY" \194  -H "Content-Type: application/json" \195  -d '{"status_code": 486}'196```197198199If no status code is supplied the API uses `603 Decline` by default. A200successful request responds with `200 OK` after OpenAI delivers the SIP201response.202203## Monitor call events204205After you accept a call, open a WebSocket connection to the same session to206stream events and issue realtime commands. Note that when connecting to an existing207call using the `call_id` parameter, the `model` argument is not used (as it has already been configured208via the `accept` endpoint).209210### WebSocket request211212`GET wss://api.openai.com/v1/realtime?call_id={call_id}`213214### Query parameters215216| Parameter | Type   | Description                                           |217| --------- | ------ | ----------------------------------------------------- |218| `call_id` | string | Identifier from the `realtime.call.incoming` webhook. |219220### Headers221222- `Authorization: Bearer YOUR_API_KEY`223224The WebSocket behaves exactly like any other Realtime API connection. Send225[`response.create`](https://developers.openai.com/api/reference/resources/realtime/client-events#response.create),226and other client events to control the call, and listen for server events to227track progress. See [Webhooks and server-side controls](https://developers.openai.com/api/docs/guides/voice-server-controls?api=realtime)228for more information.229230```javascript231import WebSocket from "ws";232233const callId = "rtc_u1_9c6574da8b8a41a18da9308f4ad974ce";234const ws = new WebSocket(`wss://api.openai.com/v1/realtime?call_id=rtc_u1_9c6574da8b8a41a18da9308f4ad974ce`, {235  headers: {236    Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,237  },238});239240ws.on("open", () => {241  ws.send(242    JSON.stringify({243      type: "response.create",244    })245  );246});247```248249250## Redirect the call251252Transfer an active call using the253[Refer call endpoint](https://developers.openai.com/api/reference/resources/realtime/subresources/calls/methods/refer). Provide the254`call_id` as well as the `target_uri` that should be placed in the SIP `Refer-To`255header (for example `tel:+14155550123` or `sip:agent@example.com`).256257```bash258curl -X POST "https://api.openai.com/v1/realtime/calls/$CALL_ID/refer" \259  -H "Authorization: Bearer $OPENAI_API_KEY" \260  -H "Content-Type: application/json" \261  -d '{"target_uri": "tel:+14155550123"}'262```263264265OpenAI returns `200 OK` once the REFER is relayed to your SIP provider. The266downstream system handles the rest of the call flow for the caller.267268## Hang up the call269270End the session with the [Hang up endpoint](https://developers.openai.com/api/reference/resources/realtime/subresources/calls/methods/hangup)271when your application should disconnect the caller. This endpoint can be used to272terminate both SIP and WebRTC realtime sessions.273274```bash275curl -X POST "https://api.openai.com/v1/realtime/calls/$CALL_ID/hangup" \276  -H "Authorization: Bearer $OPENAI_API_KEY"277```278279280The API responds with `200 OK` when it starts tearing down the call.281282<a id="dedicated-sip-ip-ranges"></a>283284## SIP signaling and media IP ranges285286Realtime SIP calls use separate network paths for signaling and media. To ensure proper operation,287configure your network to allow signaling and media traffic as described below.288289### SIP signaling290291`sip.api.openai.com` and `sip-eu.api.openai.com` are GeoIP-routed endpoints. Your network must allow292outbound TCP/TLS traffic to the addresses returned by DNS on port `5061`.293294### SRTP media295296The API specifies a separate media IP address and UDP port in the negotiated SDP. Your network must297allow bidirectional SRTP traffic over UDP to and from the following CIDRs:298299- `13.79.45.80/28`300- `23.98.140.64/28`301- `40.67.149.176/28`302- `40.83.204.240/28`303304## Server examples305306The following is an example of a `realtime.call.incoming` handler. It accepts the call and then logs all the events from307the Realtime API.308309For the Ruby example, set the `OPENAI_API_KEY` and `OPENAI_WEBHOOK_SECRET`310environment variables, then install the required dependencies with311`gem install openai webrick async-websocket`.312313Handle an incoming SIP call314315```python316from flask import Flask, request, Response, jsonify, make_response317from openai import OpenAI, InvalidWebhookSignatureError318import asyncio319import json320import os321import requests322import time323import threading324import websockets325326app = Flask(__name__)327client = OpenAI(webhook_secret=os.environ["OPENAI_WEBHOOK_SECRET"])328329AUTH_HEADER = {"Authorization": "Bearer " + os.environ["OPENAI_API_KEY"]}330331call_accept = {332    "type": "realtime",333    "instructions": "You are a support agent.",334    "model": "gpt-realtime-2.1",335}336337response_create = {338    "type": "response.create",339    "response": {340        "instructions": ("Say to the user 'Thank you for calling, how can I help you'")341    },342}343344345async def websocket_task(call_id):346    try:347        async with websockets.connect(348            "wss://api.openai.com/v1/realtime?call_id=" + call_id,349            additional_headers=AUTH_HEADER,350        ) as websocket:351            await websocket.send(json.dumps(response_create))352353            while True:354                response = await websocket.recv()355                print(f"Received from WebSocket: {response}")356    except Exception as e:357        print(f"WebSocket error: {e}")358359360@app.route("/", methods=["POST"])361def webhook():362    try:363        event = client.webhooks.unwrap(request.data, request.headers)364365        if event.type == "realtime.call.incoming":366            requests.post(367                "https://api.openai.com/v1/realtime/calls/"368                + event.data.call_id369                + "/accept",370                headers={**AUTH_HEADER, "Content-Type": "application/json"},371                json=call_accept,372            )373            threading.Thread(374                target=lambda: asyncio.run(websocket_task(event.data.call_id)),375                daemon=True,376            ).start()377            return Response(status=200)378    except InvalidWebhookSignatureError as e:379        print("Invalid signature", e)380        return Response("Invalid signature", status=400)381382383if __name__ == "__main__":384    app.run(port=8000)385```386387```ruby388require "openai"389require "webrick"390391client = OpenAI::Client.new(webhook_secret: ENV.fetch("OPENAI_WEBHOOK_SECRET"))392server = WEBrick::HTTPServer.new(393  BindAddress: "127.0.0.1",394  Port: Integer(ENV.fetch("OPENAI_WEBHOOK_PORT", "8000")),395  Logger: WEBrick::Log.new($stderr, WEBrick::BasicLog::WARN),396  AccessLog: []397)398sideband_workers = []399400server.mount_proc("/webhook") do |request, response|401  if request.request_method != "POST"402    response.status = 405403    next404  end405406  headers = request.header.transform_values(&:first)407  event = client.webhooks.unwrap(request.body, headers)408409  if event.is_a?(OpenAI::Models::Webhooks::RealtimeCallIncomingWebhookEvent)410    call_id = event.data.call_id411    sideband_workers.select!(&:alive?)412    sideband_workers << Thread.new(call_id) do |active_call_id|413      client.realtime.calls.accept(414        active_call_id,415        type: :realtime,416        model: "gpt-realtime-2.1",417        instructions: "You are a helpful support agent."418      )419420      client.realtime.connect_to_call(call_id: active_call_id) do |connection|421        connection.response.create(422          instructions: "Thank the caller and ask how you can help."423        )424        connection.each do |server_event|425          puts "Realtime event: #{server_event.type}"426        end427      end428    end429  end430431  response.status = 200432  response.body = "ok"433rescue OpenAI::Errors::InvalidWebhookSignatureError, ArgumentError434  response.status = 400435  response.body = "Invalid signature"436ensure437  server.shutdown if ENV["OPENAI_WEBHOOK_EXIT_AFTER_REQUEST"] == "1"438end439440Signal.trap("INT") do441  sideband_workers.each(&:kill)442  server.shutdown443end444port = server.listeners.first.addr[1]445puts "Webhook server listening on http://127.0.0.1:#{port}/webhook"446$stdout.flush447server.start448sideband_workers.each(&:join)449```450451452## Next steps453454Now that you've connected over SIP, use the left navigation or click into these pages to start building your realtime application.455456- [Realtime prompting guide](https://developers.openai.com/api/docs/guides/voice-prompting)457- [Managing conversations](https://developers.openai.com/api/docs/guides/realtime-conversations)458- [Webhooks and server-side controls](https://developers.openai.com/api/docs/guides/voice-server-controls?api=realtime)459- [Managing costs](https://developers.openai.com/api/docs/guides/voice-latency-cost?api=realtime)460- [Realtime transcription](https://developers.openai.com/api/docs/guides/realtime-transcription)461462### Additional Resources463464- [JavaScript demo](https://hello-realtime.val.run/)465- [Connect the Realtime SIP Connector to Twilio Elastic SIP Trunking](https://www.twilio.com/en-us/blog/developers/tutorials/product/openai-realtime-api-elastic-sip-trunking)
Files main
Menu

Theme