Repository Public
1# Realtime transcription23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45Use realtime transcription when your application needs text from a microphone, call, or other live audio stream without a spoken assistant response. The recommended model returns transcript deltas as speech arrives and a final transcript when your application commits each audio turn.67Start with [`gpt-live-transcribe`](https://developers.openai.com/api/docs/models/gpt-live-transcribe). Use [file transcription](https://developers.openai.com/api/docs/guides/speech-to-text) if your audio is already recorded, or see the [Transcription overview](https://developers.openai.com/api/docs/guides/transcription) to compare the workflows.89## Create a transcription session1011Create a session with `type: "transcription"` and select `gpt-live-transcribe`. Connect with [WebSocket](https://developers.openai.com/api/docs/guides/voice-websockets?api=realtime) for server-side audio pipelines or [WebRTC](https://developers.openai.com/api/docs/guides/voice-webrtc?api=realtime) for browser audio.1213```json14{15 "type": "session.update",16 "session": {17 "type": "transcription",18 "audio": {19 "input": {20 "format": {21 "type": "audio/pcm",22 "rate": 2400023 },24 "transcription": {25 "model": "gpt-live-transcribe"26 },27 "turn_detection": null28 }29 }30 }31}32```333435This example uses 24 kHz PCM audio and disables automatic turn detection so that you can explicitly commit each turn. For the full session configuration, see the [Realtime sessions reference](https://developers.openai.com/api/reference/resources/realtime/subresources/client_secrets).3637## Stream audio3839Send audio chunks with `input_audio_buffer.append`:4041```javascript42ws.send(43 JSON.stringify({44 type: "input_audio_buffer.append",45 audio: base64Pcm16,46 })47);48```495051With automatic turn detection turned off, commit the buffer when you want to finish an audio turn:5253```javascript54ws.send(55 JSON.stringify({56 type: "input_audio_buffer.commit",57 })58);59```606162To let the server detect and commit turn boundaries, configure [voice activity detection](https://developers.openai.com/api/docs/guides/realtime-vad) instead.6364## Handle transcript events6566Listen for incremental transcript deltas and completion events:6768```javascript69ws.on("message", (data) => {70 const event = JSON.parse(data);7172 if (event.type === "conversation.item.input_audio_transcription.delta") {73 process.stdout.write(event.delta);74 }7576 if (event.type === "conversation.item.input_audio_transcription.completed") {77 console.log("\nFinal transcript:", event.transcript);78 }79});80```818283A delta event contains newly available transcript text:8485```json86{87 "type": "conversation.item.input_audio_transcription.delta",88 "item_id": "item_003",89 "content_index": 0,90 "delta": "Hello,"91}92```9394A completion event contains the final transcript for the committed item:9596```json97{98 "type": "conversation.item.input_audio_transcription.completed",99 "item_id": "item_003",100 "content_index": 0,101 "transcript": "Hello, how are you?"102}103```104105Ordering between completion events from different speech turns isn't guaranteed. Use `item_id` to match transcription events to committed input items.106107## Add transcription context108109Add context when the audio contains specialized vocabulary or more than one expected language. Send another `session.update` event to change the transcription configuration during an existing session.110111```json112{113 "type": "session.update",114 "session": {115 "type": "transcription",116 "audio": {117 "input": {118 "format": {119 "type": "audio/pcm",120 "rate": 24000121 },122 "transcription": {123 "model": "gpt-live-transcribe",124 "prompt": "A customer support call about a premium plan and account AC-42.",125 "keywords": ["premium plan", "AC-42", "billing"],126 "languages": ["en", "fr"],127 "delay": "low"128 },129 "turn_detection": null130 }131 }132 }133}134```135136137- Use `prompt` to describe the recording or its setting.138- Use `keywords` for product names, acronyms, and other literal terms that may appear in the audio.139- Use `languages` for expected input languages.140141Supported language-code formats include:142143- ISO 639-1 codes, such as `en`, `es`, and `fr`.144- Selected ISO 639-3 codes, such as `eng`, `spa`, `yue`, and `cmn`.145- Regional `zh` locale codes, such as `zh-cn`, `zh-tw`, and `zh-hk`.146147The Realtime API rejects unsupported or incorrectly formatted language codes.148149Keywords are hints, not required output. Keep each keyword on one line and don't include `<`, `>`, a carriage return, or a line feed. The Realtime API rejects the session update if a keyword contains one of these characters or `prompt` exceeds the model's length limit.150151`gpt-live-transcribe` uses `languages` instead of the singular `language` field. Don't send both.152153## Transcribe a committed turn154155Use `gpt-transcribe` in a Realtime session only when you specifically need transcription to begin after a committed audio turn or need detected-language output. This specialized workflow requires a WebSocket connection.156157When `gpt-transcribe` performs input transcription in a Realtime API session or runs in a dedicated transcription session, it automatically uses earlier transcribed turns as context.158159```json160{161 "type": "session.update",162 "session": {163 "type": "transcription",164 "audio": {165 "input": {166 "format": {167 "type": "audio/pcm",168 "rate": 24000169 },170 "transcription": {171 "model": "gpt-transcribe"172 },173 "turn_detection": null174 }175 }176 }177}178```179180181Append audio and send `input_audio_buffer.commit`. The model can then emit transcript deltas before the final completion event. Its completion event also includes detected languages:182183```json184{185 "type": "conversation.item.input_audio_transcription.completed",186 "item_id": "item_003",187 "content_index": 0,188 "transcript": "Bonjour, pouvez-vous m'entendre ?",189 "languages": [{ "code": "fr" }]190}191```192193When `gpt-transcribe` can't make a reliable language prediction, `languages` is an empty array. `gpt-live-transcribe` doesn't return detected-language predictions.194195## Tune latency and accuracy196197Streaming transcription trades latency for transcript quality. Lower delay settings can produce earlier partial text. Higher delay settings give the model more audio context before emitting text and can improve word error rate.198199Start by setting `audio.input.transcription.delay` and testing against your real audio. Useful starting points are:200201- `minimal` for the most latency-sensitive interactions;202- `low` for low-latency live captions;203- `medium` for a balanced latency/accuracy tradeoff;204- `high` when accuracy matters more than immediate display;205- `xhigh` when your workflow can tolerate the most delay for more context.206207The exact delay in milliseconds can vary by model configuration, so benchmark with representative audio instead of assuming a fixed timing per level.208209Don't choose a setting from synthetic audio alone. Test with representative microphones, telephony audio, accents, background noise, code-switching, domain vocabulary, and long sessions.210211## Handle confidence, timestamps, and speaker labels212213`gpt-live-transcribe` doesn't return word-level timestamps, speaker labels, or transcription confidence scores. If your application requires timestamps or speaker labels, use a compatible [file transcription](https://developers.openai.com/api/docs/guides/speech-to-text) model or add an application-level fallback.214215## Production checklist216217- Pick a target latency and accuracy threshold before tuning.218- Test against real production audio, not only clean samples.219- Test each target language.220- Include numbers, dates, currency, email addresses, product names, and domain terms in your eval set.221- Track empty, truncated, and delayed transcripts apart from word error rate.222- Decide how your UI should revise partial text when later deltas correct earlier text.223- Use `item_id` to order and reconcile final transcripts.224- Keep a fallback path for unsupported timestamps, speaker labels, or confidence fields.225226## Related guides227228[Realtime and audio overview229230231232 Compare voice-agent, translation, and transcription sessions.](https://developers.openai.com/api/docs/guides/realtime)233234[Realtime translation235236237238 Translate live speech with a dedicated translation session.](https://developers.openai.com/api/docs/guides/realtime-translation)239240[WebSocket connection241242243244 Stream raw audio through a server-side media pipeline.](https://developers.openai.com/api/docs/guides/voice-websockets?api=realtime)245246[Voice activity detection247248249250 Configure turn detection for live audio streams.](https://developers.openai.com/api/docs/guides/realtime-vad)