# OpenAI Audio API — text-to-speech, speech-to-text, translation, custom voices, audio in Chat Completions

Status: LIVE_VERIFIED (/v1/audio/speech incl. SSE, /v1/audio/transcriptions ×5 variants incl. streaming/diarization/logprobs, /v1/audio/translations) · DOCUMENTED/UNVERIFIED for voices & voice consents (list returned 404 for our key — likely access-gated) · LIVE_DISCOVERED for Chat Completions audio (400 shape captured). Sources: https://developers.openai.com/api/docs/guides/audio · …/text-to-speech · …/speech-to-text · …/transcription · …/custom-voices · …/audio-chat-completions · https://developers.openai.com/api/reference/resources/audio/subresources/speech/methods/create · …/transcriptions/methods/create · …/transcriptions/streaming-events · …/translations/methods/create · …/voices/methods/create · …/voice_consents/methods/{create,list,retrieve,update,delete} · model pages (gpt-4o-mini-tts, tts-1, tts-1-hd, gpt-transcribe, gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-transcribe-diarize, whisper-1, gpt-audio-1.5, gpt-audio, gpt-audio-mini, gpt-4o-(mini-)audio-preview) · openapi-master.yaml Last verified: 2026-09-18 Twins: generated/fragments/endpoints/openai-realtime-live-audio.json (api_family audio, chat-audio), generated/fragments/parameters/openai-audio.json (53), generated/fragments/streaming-events/openai-audio-transcription.json (5).

# 1. Workflow chooser (official)

# 2. Model matrix (from model pages, 2026-09-18 — pricing owned by the models agent; cite docs/models/*)

# 3. Text-to-speech — POST /v1/audio/speech (LIVE_VERIFIED)

Observed: {"model":"gpt-4o-mini-tts","input":"OK","voice":"alloy","response_format":"mp3"} → 200, 17,664 bytes (tmp-live/ok.mp3, ≈1.1 s), headers openai-processing-ms: 952, x-ratelimit-limit-requests: 30000. Output is non-deterministic: re-runs of the same request produced 16,128 B (alloy/mp3) and 19,968 B (marin/mp3 + instructions); cedar/wav → 76,844 B. SSE variant (pcm) → 4 × speech.audio.delta (12.8k–25.6k base64 chars each) + speech.audio.done {usage: {input_tokens: 1, output_tokens: 33, total_tokens: 34}} + data: [DONE]. Usage-policy note: disclose to end users that the voice is AI-generated. Languages: follows Whisper's list (Afrikaans … Welsh).

# 4. Speech-to-text — POST /v1/audio/transcriptions (multipart/form-data, LIVE_VERIFIED)

Response shapes (all observed on tmp-live/ok.mp3):

• json (gpt-4o-mini-transcribe): {"text":"OK.","usage":{"type":"tokens","total_tokens":15,"input_tokens":11,"input_token_details":{"text_tokens":0,"audio_tokens":11},"output_tokens":4}}; with include[]=logprobs: + "logprobs":[{"token":"OK","logprob":-0.0095,"bytes":[79,75]},…]. gpt-transcribe adds "languages":[{"code":"en"}] (empty array if unsure; documented, not tested).
• verbose_json (whisper-1, word+segment): {"task":"transcribe","language":"english","duration":1.1,"text":"OK.","segments":[{id, seek, start, end, text, tokens[], temperature, avg_logprob, compression_ratio, no_speech_prob}],"words":[{"word":"OK","start":0.0,"end":0.5}],"usage":{"type":"duration","seconds":2}} (duration usage rounds up to whole seconds).
• diarized_json (gpt-4o-transcribe-diarize): {"text":"Okay.","task":"transcribe","duration":1.104,"segments":[{"type":"transcript.text.segment","id":"seg_0","speaker":"A","start":0.0,"end":0.35,"text":" Okay."}],"usage":{"type":"tokens","total_tokens":87,"input_tokens":11,"output_tokens":76}}.
• stream=true (SSE): transcript.text.delta {delta, logprobs?, segment_id?} × n → transcript.text.done {text, languages?, logprobs?, usage}; diarized streams add transcript.text.segment {id, start, end, text, speaker}. Observed: delta "OK", delta ".", done.

Longer inputs: split ≤25 MB chunks at sentence boundaries (PyDub example in guide). Post-processing with a text model is the documented way around whisper's 224-token prompt.

# 5. Translation — POST /v1/audio/translations (LIVE_VERIFIED)

file, model (whisper-1 only), prompt, response_format json|text|srt|verbose_json|vtt (default json), temperature 0–1. Output is English only. Observed: {"text":"OK."}; verbose_json returns {duration, language: "english", text, segments[]}.

# 6. Custom voices & voice consents (DOCUMENTED; access-gated — "limited to eligible customers")

1. POST /v1/audio/voice_consents multipart name, language (BCP 47, e.g. en-US), recording (≤10 MiB; audio/mpeg, wav, x-wav, ogg, aac, flac, webm, mp4) containing exactly one of the 17 consent phrases (de, en, es, fr, hi, id, it, ja, ko, nl, pl, pt, ru, uk, vi, zh — e.g. en: "I am the owner of this voice and I consent to OpenAI using this voice to create a synthetic voice model.") → {id: "cons_…", object: "audio.voice_consent", name, language, created_at}. One consent can back several voices.
2. POST /v1/audio/voices multipart name, consent (cons_…), audio_sample (≤30 s, ≥5 s speech / ≥15 tokens, ≤10 MiB; same speaker) → {id: "voice_…", object: "audio.voice", name, created_at}. Max 20 voices per org. Browser audio/webm;codecs=opus must be sent as audio/webm.
3. Use voice: {"id": "voice_…"} in /v1/audio/speech, Realtime session.audio.output.voice, Live session.audio.output.voice, or Chat audio.voice.
4. Consents: GET /v1/audio/voice_consents?after&limit(1–100, default 20) → {object: list, data[], first_id, last_id, has_more}; GET|POST(update name)|DELETE /v1/audio/voice_consents/{consent_id} (delete → {id, object, deleted: true}). There is no list/retrieve/delete endpoint for voices in the spec (GET /v1/audio/voices → 404 "Endpoint not found."); voices appear in the platform Audio → Voices tab. Scopes: api.voices.read / api.voices.write.

LIVE 2026-09-18: GET /v1/audio/voice_consents?limit=5 → 404 Endpoint not found. with our key. Recorded as FAILED_VERIFICATION; the most plausible cause is that the custom-voice program is not enabled for this organization (a 404 does not mean the route does not exist). Creation endpoints were deliberately not called.

# 7. Audio in Chat Completions (pointer; endpoint owned by the chat agent)

• Output: {"model":"gpt-audio-mini","modalities":["text","audio"],"audio":{"voice":"alloy","format":"wav|aac|mp3|flac|opus|pcm16"},"messages":[…]} → choices[0].message.audio {id, data (b64), transcript, expires_at}.
• Input: content part {"type":"input_audio","input_audio":{"data":"<base64>","format":"wav|mp3"}}.
• LIVE: gpt-audio-mini, modalities:["text"], text-only prompt → 400 {"code":"invalid_value","param":"model","message":"This model requires that either input content or output modality contain audio."} — the audio models cannot be used as pure text models.

# 8. Live verification log (2026-09-18)

Total audio probe cost ≈ $0.002. Raw sanitized outputs: tmp-live/realtime-audio/*.json, tmp-live/ok.mp3.