SPB Git

Repository Public

text-to-speech.md 13.9 KB 427 lines markdown

1# Text to speech23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45The Audio API provides a [`speech`](https://developers.openai.com/api/reference/resources/audio/subresources/speech/methods/create) endpoint based on our [GPT-4o mini TTS (text-to-speech) model](https://developers.openai.com/api/docs/models/gpt-4o-mini-tts). It comes with 11 built-in voices and can be used to:67- Narrate a written blog post8- Produce spoken audio in multiple languages9- Give realtime audio output using streaming1011Here's an example of the `alloy` voice:1213Our [usage policies](https://openai.com/policies/usage-policies) require you14  to provide a clear disclosure to end users that the TTS voice they are hearing15  is AI-generated and not a human voice.1617## Quickstart1819The `speech` endpoint takes three key inputs:20211. The [model](https://developers.openai.com/api/reference/resources/audio/subresources/speech/methods/create#audio-createspeech-model) you're using221. The [text](https://developers.openai.com/api/reference/resources/audio/subresources/speech/methods/create#audio-createspeech-input) to be turned into audio231. The [voice](https://developers.openai.com/api/reference/resources/audio/subresources/speech/methods/create#audio-createspeech-voice) that will speak the output2425Here's a simple request example:2627Generate spoken audio from input text2829```javascript30import fs from "fs";31import path from "path";32import OpenAI from "openai";3334const openai = new OpenAI();35const speechFile = path.resolve("./speech.mp3");3637const mp3 = await openai.audio.speech.create({38  model: "gpt-4o-mini-tts",39  voice: "coral",40  input: "Today is a wonderful day to build something people love!",41  instructions: "Speak in a cheerful and positive tone.",42});4344const buffer = Buffer.from(await mp3.arrayBuffer());45await fs.promises.writeFile(speechFile, buffer);46```4748```python49from pathlib import Path50from openai import OpenAI5152client = OpenAI()53speech_file_path = Path(__file__).parent / "speech.mp3"5455with client.audio.speech.with_streaming_response.create(56    model="gpt-4o-mini-tts",57    voice="coral",58    input="Today is a wonderful day to build something people love!",59    instructions="Speak in a cheerful and positive tone.",60) as response:61    response.stream_to_file(speech_file_path)62```6364```go65package main6667import (68	"context"69	"io"70	"os"7172	"github.com/openai/openai-go/v3"73)7475func main() {76	client := openai.NewClient()77	response, err := client.Audio.Speech.New(context.Background(), openai.AudioSpeechNewParams{78		Model:        openai.SpeechModelGPT4oMiniTTS,79		Voice:        openai.AudioSpeechNewParamsVoiceUnion{OfAudioSpeechNewsVoiceString2: openai.String("coral")},80		Input:        "Today is a wonderful day to build something people love!",81		Instructions: openai.String("Speak in a cheerful and positive tone."),82	})83	if err != nil {84		panic(err)85	}86	defer response.Body.Close()8788	file, err := os.Create("speech.mp3")89	if err != nil {90		panic(err)91	}92	if _, err := io.Copy(file, response.Body); err != nil {93		panic(err)94	}95	if err := file.Close(); err != nil {96		panic(err)97	}98}99```100101```java102import com.openai.client.OpenAIClient;103import com.openai.client.okhttp.OpenAIOkHttpClient;104import com.openai.core.http.HttpResponse;105import com.openai.models.audio.speech.SpeechCreateParams;106import java.io.IOException;107import java.nio.file.Files;108import java.nio.file.Path;109import java.nio.file.StandardCopyOption;110111try (HttpResponse audio =112    client113        .audio()114        .speech()115        .create(116            SpeechCreateParams.builder()117                .model("gpt-4o-mini-tts")118                .voice("coral")119                .input("Today is a wonderful day to build something people love!")120                .instructions("Speak in a cheerful and positive tone.")121                .build())) {122  Files.copy(audio.body(), Path.of("speech.mp3"), StandardCopyOption.REPLACE_EXISTING);123}124```125126```csharp127using OpenAI.Audio;128#pragma warning disable OPENAI001129130string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;131string model = "gpt-4o-mini-tts";132AudioClient client = new(model, key);133134BinaryData audio = await client.GenerateSpeechAsync(135    "Today is a wonderful day to build something people love!",136    GeneratedSpeechVoice.Coral,137    new SpeechGenerationOptions138    {139        Instructions = "Speak in a cheerful and positive tone.",140    }141);142143await File.WriteAllBytesAsync("speech.mp3", audio.ToArray());144```145146```ruby147require "openai"148149client = OpenAI::Client.new150audio = client.audio.speech.create(151  model: "gpt-4o-mini-tts",152  voice: "coral",153  input: "Today is a wonderful day to build something people love!",154  instructions: "Speak in a cheerful and positive tone."155)156File.binwrite("speech.mp3", audio.read)157```158159```bash160curl https://api.openai.com/v1/audio/speech \161  -H "Authorization: Bearer $OPENAI_API_KEY" \162  -H "Content-Type: application/json" \163  -d '{164    "model": "gpt-4o-mini-tts",165    "input": "Today is a wonderful day to build something people love!",166    "voice": "coral",167    "instructions": "Speak in a cheerful and positive tone."168  }' \169  --output speech.mp3170```171172```bash173openai audio:speech create \174  --model gpt-4o-mini-tts \175  --voice coral \176  --instructions "Speak in a cheerful and positive tone." \177  --input "Today is a wonderful day to build something people love!" \178  --output speech.mp3179```180181182By default, the endpoint outputs an MP3 of the spoken audio, but you can configure it to output any [supported format](#supported-output-formats).183184### Text-to-speech models185186For intelligent realtime applications, use the `gpt-4o-mini-tts` model, our newest and most reliable text-to-speech model. You can prompt the model to control aspects of speech, including:187188- Accent189- Emotional range190- Intonation191- Impressions192- Speed of speech193- Tone194- Whispering195196Our other text-to-speech models are `tts-1` and `tts-1-hd`. The `tts-1` model provides lower latency, but at a lower quality than the `tts-1-hd` model.197198### Voice options199200The TTS endpoint provides 13 built‑in voices to control how speech is rendered from text. **Hear and play with these voices in [OpenAI.fm](https://openai.fm), our interactive demo for trying the latest text-to-speech model in the OpenAI API**. Voices are currently optimized for English.201202- `alloy`203- `ash`204- `ballad`205- `coral`206- `echo`207- `fable`208- `nova`209- `onyx`210- `sage`211- `shimmer`212- `verse`213- `marin`214- `cedar`215216For best quality, we recommend using `marin` or `cedar`.217218Voice availability depends on the model. The `tts-1` and `tts-1-hd` models support a smaller set: `alloy`, `ash`, `coral`, `echo`, `fable`, `onyx`, `nova`, `sage`, and `shimmer`.219220If you're using the [Realtime API](https://developers.openai.com/api/docs/guides/realtime), note that the set of available voices is slightly different—see the [realtime conversations guide](https://developers.openai.com/api/docs/guides/realtime-conversations#voice-options) for current realtime voices.221222### Streaming realtime audio223224The Speech API provides support for realtime audio streaming using [chunk transfer encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding). This means the audio can be played before the full file is generated and made accessible.225226Stream spoken audio from input text directly to your speakers227228```javascript229import OpenAI from "openai";230import { playAudio } from "openai/helpers/audio";231232const openai = new OpenAI();233234const response = await openai.audio.speech.create({235  model: "gpt-4o-mini-tts",236  voice: "coral",237  input: "Today is a wonderful day to build something people love!",238  instructions: "Speak in a cheerful and positive tone.",239  response_format: "wav",240});241242await playAudio(response);243```244245```python246import asyncio247248from openai import AsyncOpenAI249from openai.helpers import LocalAudioPlayer250251openai = AsyncOpenAI()252253254async def main() -> None:255    async with openai.audio.speech.with_streaming_response.create(256        model="gpt-4o-mini-tts",257        voice="coral",258        input="Today is a wonderful day to build something people love!",259        instructions="Speak in a cheerful and positive tone.",260        response_format="pcm",261    ) as response:262        await LocalAudioPlayer().play(response)263264265if __name__ == "__main__":266    asyncio.run(main())267```268269```go270package main271272import (273	"context"274	"io"275	"os"276277	"github.com/openai/openai-go/v3"278)279280func main() {281	client := openai.NewClient()282	response, err := client.Audio.Speech.New(context.Background(), openai.AudioSpeechNewParams{283		Model:          openai.SpeechModelGPT4oMiniTTS,284		Voice:          openai.AudioSpeechNewParamsVoiceUnion{OfAudioSpeechNewsVoiceString2: openai.String("coral")},285		Input:          "Today is a wonderful day to build something people love!",286		Instructions:   openai.String("Speak in a cheerful and positive tone."),287		ResponseFormat: openai.AudioSpeechNewParamsResponseFormatWAV,288	})289	if err != nil {290		panic(err)291	}292	defer response.Body.Close()293	if _, err := io.Copy(os.Stdout, response.Body); err != nil {294		panic(err)295	}296}297```298299```java300import com.openai.client.OpenAIClient;301import com.openai.client.okhttp.OpenAIOkHttpClient;302import com.openai.core.http.HttpResponse;303import com.openai.models.audio.speech.SpeechCreateParams;304import java.io.IOException;305import java.io.OutputStream;306import java.nio.file.Files;307import java.nio.file.Path;308import javax.sound.sampled.AudioFormat;309import javax.sound.sampled.AudioSystem;310import javax.sound.sampled.LineUnavailableException;311import javax.sound.sampled.SourceDataLine;312313try (HttpResponse audio =314    client315        .audio()316        .speech()317        .create(318            SpeechCreateParams.builder()319                .model("gpt-4o-mini-tts")320                .voice("coral")321                .input("Today is a wonderful day to build something people love!")322                .instructions("Speak in a cheerful and positive tone.")323                .responseFormat(SpeechCreateParams.ResponseFormat.PCM)324                .streamFormat(SpeechCreateParams.StreamFormat.AUDIO)325                .build())) {326  AudioFormat format = new AudioFormat(24_000, 16, 1, true, false);327  String outputPath = System.getenv("OPENAI_EXAMPLE_AUDIO_OUTPUT_PATH");328  if (outputPath == null || outputPath.isBlank()) {329    try (SourceDataLine speakers = AudioSystem.getSourceDataLine(format)) {330      speakers.open(format);331      speakers.start();332      byte[] chunk = new byte[1024];333      int bytesRead;334      while ((bytesRead = audio.body().read(chunk)) != -1) {335        speakers.write(chunk, 0, bytesRead);336      }337      speakers.drain();338    }339  } else {340    try (OutputStream output = Files.newOutputStream(Path.of(outputPath))) {341      long bytes = audio.body().transferTo(output);342      System.out.println(bytes + " audio bytes");343    }344  }345}346```347348```ruby349require "openai"350351client = OpenAI::Client.new352audio = client.audio.speech.create(353  model: "gpt-4o-mini-tts",354  voice: "alloy",355  input: "Welcome to the OpenAI API.",356  response_format: :pcm,357  stream_format: :audio358)359while (chunk = audio.read(1_024))360  puts(chunk.bytesize)361end362```363364```bash365curl https://api.openai.com/v1/audio/speech \366  -H "Authorization: Bearer $OPENAI_API_KEY" \367  -H "Content-Type: application/json" \368  -d '{369    "model": "gpt-4o-mini-tts",370    "input": "Today is a wonderful day to build something people love!",371    "voice": "coral",372    "instructions": "Speak in a cheerful and positive tone.",373    "response_format": "wav"374  }' | ffplay -i -375```376377378For the fastest response times, we recommend using `wav` or `pcm` as the response format.379380## Supported output formats381382The default response format is `mp3`, but other formats like `opus` and `wav` are available.383384- **MP3**: The default response format for general use cases.385- **Opus**: For internet streaming and communication, low latency.386- **AAC**: For digital audio compression, preferred by YouTube, Android, iOS.387- **FLAC**: For lossless audio compression, favored by audio enthusiasts for archiving.388- **WAV**: Uncompressed WAV audio, suitable for low-latency applications to avoid decoding overhead.389- **PCM**: Similar to WAV but contains the raw samples in 24kHz (16-bit signed, low-endian), without the header.390391## Supported languages392393The TTS model generally follows the Whisper model in terms of language support. Whisper [supports the following languages](https://github.com/openai/whisper#available-models-and-languages) and performs well, despite voices being optimized for English:394395Afrikaans, Arabic, Armenian, Azerbaijani, Belarusian, Bosnian, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Maori, Nepali, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tagalog, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Welsh.396397You can generate spoken audio in these languages by providing input text in the language of your choice.398399## Custom voices400401Create an approved custom voice from a speaker's consent recording and matching402audio sample. See [Custom voices](https://developers.openai.com/api/docs/guides/custom-voices) for eligibility,403recording requirements, consent phrases, and API requests.404405#### Creating a voice406407Follow [Create a custom voice](https://developers.openai.com/api/docs/guides/custom-voices#creating-a-voice).408409#### Using a voice during speech generation410411Pass the created voice ID when generating speech. See the412[speech generation examples](https://developers.openai.com/api/docs/guides/custom-voices#using-a-voice-during-speech-generation).413414## Related guides415416[Realtime and audio overview417418419420      Choose the right path for voice agents, translation, transcription, and421    speech generation.](https://developers.openai.com/api/docs/guides/realtime)422423[Audio and speech concepts424425426427      Review audio modalities, speech tasks, streaming, and request-based APIs.](https://developers.openai.com/api/docs/guides/audio)
Files main
Menu

Theme