SPB Git

Repository Public

speech-generation.md 40.5 KB 1,041 lines markdown

1<br />23The Gemini API can transform text input into single speaker or multi-speaker4audio using Gemini text-to-speech (TTS) generation capabilities.5Text-to-speech (TTS) generation is *[controllable](https://ai.google.dev/gemini-api/docs/speech-generation#controllable)* ,6meaning you can use natural language to structure interactions and guide the7*style* , *accent* , *pace* , and *tone* of the audio.89The TTS capability differs from speech generation provided through the10[Live API](https://ai.google.dev/gemini-api/docs/live), which is designed for interactive,11unstructured audio, and multimodal inputs and outputs. While the Live API excels12in dynamic conversational contexts, TTS through the Gemini API13is tailored for scenarios that require exact text recitation with fine-grained14control over style and sound, such as podcast or audiobook generation.1516This guide shows you how to generate single-speaker and multi-speaker audio from17text.1819> [!WARNING]20> **Preview:** Gemini text-to-speech (TTS) is in [Preview](https://ai.google.dev/gemini-api/docs/models#preview).2122## Before you begin2324Ensure you use a Gemini 2.5 model variant with Gemini text-to-speech (TTS)25capabilities, as listed in the [Supported models](https://ai.google.dev/gemini-api/docs/speech-generation#supported-models)26section. For optimal results, consider which model best fits your specific27use case.2829You may find it useful to [test the Gemini TTS models in AI Studio](https://aistudio.google.com/generate-speech) before you start building.3031> [!NOTE]32> **Note:** TTS models accept text-only inputs and produce audio-only outputs. For a complete list of restrictions specific to TTS models, review the [Limitations](https://ai.google.dev/gemini-api/docs/speech-generation#limitations) section.3334## Single-speaker TTS3536To convert text to single-speaker audio, set the response modality to "audio",37and pass a `speech_config` object with a voice name.38You'll need to choose a voice name from the prebuilt [output voices](https://ai.google.dev/gemini-api/docs/speech-generation#voices).3940This example saves the output audio from the model in a wave file:4142### Python4344    from google import genai45    import wave46    import base644748    def wave_file(filename, pcm, channels=1, rate=24000, sample_width=2):49        with wave.open(filename, "wb") as wf:50            wf.setnchannels(channels)51            wf.setsampwidth(sample_width)52            wf.setframerate(rate)53            wf.writeframes(pcm)5455    client = genai.Client()5657    interaction = client.interactions.create(58        model="gemini-3.1-flash-tts-preview",59        input="Say cheerfully: Have a wonderful day!",60        response_format={"type": "audio"},61        generation_config={62            "speech_config": [63                {"voice": "Kore"}64            ]65        }66    )6768    wave_file('out.wav', base64.b64decode(interaction.output_audio.data))6970### JavaScript7172    import {GoogleGenAI} from '@google/genai';73    import wav from 'wav';7475    async function saveWaveFile(76       filename,77       pcmData,78       channels = 1,79       rate = 24000,80       sampleWidth = 2,81    ) {82       return new Promise((resolve, reject) => {83          const writer = new wav.FileWriter(filename, {84                channels,85                sampleRate: rate,86                bitDepth: sampleWidth * 8,87          });8889          writer.on('finish', resolve);90          writer.on('error', reject);9192          writer.write(pcmData);93          writer.end();94       });95    }9697    async function main() {98       const client = new GoogleGenAI({});99100       const interaction = await client.interactions.create({101          model: "gemini-3.1-flash-tts-preview",102          input: "Say cheerfully: Have a wonderful day!",103          response_format: { type: 'audio' },104          generation_config: {105             speech_config: [106                { voice: 'Kore' }107             ]108          },109        });110111       const audioBuffer = Buffer.from(interaction.output_audio.data, 'base64');112113       await saveWaveFile('out.wav', audioBuffer);114    }115    await main();116117### Java118119    import com.google.genai.Client;120    import com.google.genai.gaos.models.interactions.AudioResponseFormat;121    import com.google.genai.gaos.models.interactions.CreateModelInteraction;122    import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;123    import com.google.genai.gaos.models.interactions.GenerationConfig;124    import com.google.genai.gaos.models.interactions.Interaction;125    import com.google.genai.gaos.models.interactions.InteractionsInput;126    import com.google.genai.gaos.models.interactions.Model;127    import com.google.genai.gaos.models.interactions.ResponseFormat;128    import com.google.genai.gaos.models.interactions.SpeechConfig;129    import com.google.genai.gaos.models.interactions.SpeechConfigUnion;130    import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;131    import java.io.ByteArrayInputStream;132    import java.io.File;133    import java.util.Arrays;134    import java.util.Base64;135    import javax.sound.sampled.AudioFileFormat;136    import javax.sound.sampled.AudioFormat;137    import javax.sound.sampled.AudioInputStream;138    import javax.sound.sampled.AudioSystem;139140    Client client = new Client();141142    SpeechConfig speechConfig = SpeechConfig.builder().voice("Kore").build();143    GenerationConfig generationConfig =144        GenerationConfig.builder()145            .speechConfig(SpeechConfigUnion.of(Arrays.asList(speechConfig)))146            .build();147148    CreateModelInteraction params =149        CreateModelInteraction.builder()150            .model(Model.of("gemini-3.1-flash-tts-preview"))151            .input(InteractionsInput.of("Say cheerfully: Have a wonderful day!"))152            .responseFormat(153                CreateModelInteractionResponseFormat.of(154                    ResponseFormat.of(AudioResponseFormat.builder().build())))155            .generationConfig(generationConfig)156            .build();157158    Interaction interaction =159        client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();160161    if (interaction.outputAudio().isPresent() && interaction.outputAudio().get().data().isPresent()) {162      byte[] pcmBytes = Base64.getDecoder().decode(interaction.outputAudio().get().data().get());163      AudioFormat format = new AudioFormat(24000, 16, 1, true, false);164      AudioInputStream audioInputStream =165          new AudioInputStream(166              new ByteArrayInputStream(pcmBytes), format, pcmBytes.length / format.getFrameSize());167      AudioSystem.write(audioInputStream, AudioFileFormat.Type.WAVE, new File("out.wav"));168    }169170### REST171172    curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \173      -H "x-goog-api-key: $GEMINI_API_KEY" \174      -H "Content-Type: application/json" \175      -d '{176        "model": "gemini-3.1-flash-tts-preview",177        "input": "Say cheerfully: Have a wonderful day!",178        "response_format": {179           "type": "audio"180         },181        "generation_config": {182          "speech_config": [183            { "voice": "Kore" }184          ]185        }186      }'187188You can retrieve generated audio data by using the `interaction.output_audio`189property, which returns the last generated audio block. For details on190convenience properties, see the191[Interactions overview](https://ai.google.dev/gemini-api/docs/interactions-overview#convenience-properties).192193## Multi-speaker TTS194195For multi-speaker audio, you'll need a `multi_speaker_voice_config` object with196each speaker (up to 2) configured as a `speaker_voice_config`.197You'll need to define each `speaker` with the same names used in the198[prompt](https://ai.google.dev/gemini-api/docs/speech-generation#controllable):199200### Python201202    from google import genai203    import wave204    import base64205206    def wave_file(filename, pcm, channels=1, rate=24000, sample_width=2):207       with wave.open(filename, "wb") as wf:208          wf.setnchannels(channels)209          wf.setsampwidth(sample_width)210          wf.setframerate(rate)211          wf.writeframes(pcm)212213    client = genai.Client()214215    prompt = """TTS the following conversation between Joe and Jane:216             Joe: How's it going today Jane?217             Jane: Not too bad, how about you?"""218219     interaction = client.interactions.create(220         model="gemini-3.1-flash-tts-preview",221         input=prompt,222         response_format={"type": "audio"},223         generation_config={224             "speech_config": [225                 {"speaker": "Joe", "voice": "Kore"},226                 {"speaker": "Jane", "voice": "Puck"}227             ]228         }229     )230231    wave_file('out.wav', base64.b64decode(interaction.output_audio.data))232233### JavaScript234235    import {GoogleGenAI} from '@google/genai';236    import wav from 'wav';237238    async function saveWaveFile(239       filename,240       pcmData,241       channels = 1,242       rate = 24000,243       sampleWidth = 2,244    ) {245       return new Promise((resolve, reject) => {246          const writer = new wav.FileWriter(filename, {247                channels,248                sampleRate: rate,249                bitDepth: sampleWidth * 8,250          });251252          writer.on('finish', resolve);253          writer.on('error', reject);254255          writer.write(pcmData);256          writer.end();257       });258    }259260    async function main() {261       const client = new GoogleGenAI({});262263       const prompt = `TTS the following conversation between Joe and Jane:264             Joe: How's it going today Jane?265             Jane: Not too bad, how about you?`;266267       const interaction = await client.interactions.create({268          model: "gemini-3.1-flash-tts-preview",269          input: prompt,270          response_format: { type: 'audio' },271          generation_config: {272             speech_config: [273                { speaker: 'Joe', voice: 'Kore' },274                { speaker: 'Jane', voice: 'Puck' }275             ]276          },277       });278279       const audioBuffer = Buffer.from(interaction.output_audio.data, 'base64');280281       await saveWaveFile('out.wav', audioBuffer);282    }283284    await main();285286### Java287288    import com.google.genai.Client;289    import com.google.genai.gaos.models.interactions.AudioResponseFormat;290    import com.google.genai.gaos.models.interactions.CreateModelInteraction;291    import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;292    import com.google.genai.gaos.models.interactions.GenerationConfig;293    import com.google.genai.gaos.models.interactions.Interaction;294    import com.google.genai.gaos.models.interactions.InteractionsInput;295    import com.google.genai.gaos.models.interactions.Model;296    import com.google.genai.gaos.models.interactions.ResponseFormat;297    import com.google.genai.gaos.models.interactions.SpeechConfig;298    import com.google.genai.gaos.models.interactions.SpeechConfigUnion;299    import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;300    import java.io.ByteArrayInputStream;301    import java.io.File;302    import java.util.Arrays;303    import java.util.Base64;304    import javax.sound.sampled.AudioFileFormat;305    import javax.sound.sampled.AudioFormat;306    import javax.sound.sampled.AudioInputStream;307    import javax.sound.sampled.AudioSystem;308309    Client client = new Client();310311    String prompt =312        "TTS the following conversation between Joe and Jane:\n"313            + "Joe: How's it going today Jane?\n"314            + "Jane: Not too bad, how about you?";315316    SpeechConfig joeConfig = SpeechConfig.builder().speaker("Joe").voice("Kore").build();317    SpeechConfig janeConfig = SpeechConfig.builder().speaker("Jane").voice("Puck").build();318319    GenerationConfig generationConfig =320        GenerationConfig.builder()321            .speechConfig(SpeechConfigUnion.of(Arrays.asList(joeConfig, janeConfig)))322            .build();323324    CreateModelInteraction params =325        CreateModelInteraction.builder()326            .model(Model.of("gemini-3.1-flash-tts-preview"))327            .input(InteractionsInput.of(prompt))328            .responseFormat(329                CreateModelInteractionResponseFormat.of(330                    ResponseFormat.of(AudioResponseFormat.builder().build())))331            .generationConfig(generationConfig)332            .build();333334    Interaction interaction =335        client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();336337    if (interaction.outputAudio().isPresent() && interaction.outputAudio().get().data().isPresent()) {338      byte[] pcmBytes = Base64.getDecoder().decode(interaction.outputAudio().get().data().get());339      AudioFormat format = new AudioFormat(24000, 16, 1, true, false);340      AudioInputStream audioInputStream =341          new AudioInputStream(342              new ByteArrayInputStream(pcmBytes), format, pcmBytes.length / format.getFrameSize());343      AudioSystem.write(audioInputStream, AudioFileFormat.Type.WAVE, new File("out.wav"));344    }345346### REST347348    curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \349      -H "x-goog-api-key: $GEMINI_API_KEY" \350      -H "Content-Type: application/json" \351      -d '{352      "model": "gemini-3.1-flash-tts-preview",353      "input": "TTS the following conversation between Joe and Jane: Joe: Hows it going today Jane? Jane: Not too bad, how about you?",354      "response_format": {355           "type": "audio"356         },357      "generation_config": {358        "speech_config": [359          { "speaker": "Joe", "voice": "Kore" },360          { "speaker": "Jane", "voice": "Puck" }361        ]362      }363    }'364365## Control speech style with prompts366367You can control style, tone, accent, and pace using natural language prompts368for both single- and multi-speaker TTS.369For example, in a single-speaker prompt, you can say:370371    Say in an spooky whisper:372    "By the pricking of my thumbs...373    Something wicked this way comes"374375In a multi-speaker prompt, provide the model with each speaker's name and376corresponding transcript. You can also provide guidance for each speaker377individually:378379    Make Speaker1 sound tired and bored, and Speaker2 sound excited and happy:380381    Speaker1: So... what's on the agenda today?382    Speaker2: You're never going to guess!383384Try using a [voice option](https://ai.google.dev/gemini-api/docs/speech-generation#voices) that corresponds to the style or emotion you385want to convey, to emphasize it even more. In the previous prompt, for example,386*Enceladus* 's breathiness might emphasize "tired" and "bored", while387*Puck*'s upbeat tone could complement "excited" and "happy".388389> [!TIP]390> **Tip:** The \[Voice Library\] applet in Google AI Studio is a great way to try out speech styles and voices with Gemini TTS.391392## Generate a prompt to convert to audio393394The TTS models only output audio, but you can use395[other models](https://ai.google.dev/gemini-api/docs/models) to generate a transcript first,396then pass that transcript to the TTS model to read aloud.397398### Python399400    from google import genai401402    client = genai.Client()403404    transcript_interaction = client.interactions.create(405       model="gemini-3.8-flash",406       input="""Generate a short transcript around 100 words that reads407                like it was clipped from a podcast by excited herpetologists.408                The hosts names are Dr. Anya and Liam."""409    )410    transcript = transcript_interaction.output_text411412    tts_interaction = client.interactions.create(413       model="gemini-3.1-flash-tts-preview",414       input=transcript,415       response_format={"type": "audio"},416       generation_config={417          "speech_config": [418             {"speaker": "Dr. Anya", "voice": "Kore"},419             {"speaker": "Liam", "voice": "Puck"}420          ]421       }422    )423424### JavaScript425426    import { GoogleGenAI } from "@google/genai";427428    const client = new GoogleGenAI({});429430    async function main() {431432    const transcriptInteraction = await client.interactions.create({433       model: "gemini-3.8-flash",434       input: "Generate a short transcript around 100 words that reads like it was clipped from a podcast by excited herpetologists. The hosts names are Dr. Anya and Liam.",435       })436437    const ttsInteraction = await client.interactions.create({438       model: "gemini-3.1-flash-tts-preview",439       input: transcriptInteraction.output_text,440       response_format: { type: 'audio' },441       generation_config: {442          speech_config: [443             { speaker: "Dr. Anya", voice: "Kore" },444             { speaker: "Liam", voice: "Puck" }445          ]446       }447      });448    }449450    await main();451452### Java453454    import com.google.genai.Client;455    import com.google.genai.gaos.models.interactions.AudioResponseFormat;456    import com.google.genai.gaos.models.interactions.CreateModelInteraction;457    import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;458    import com.google.genai.gaos.models.interactions.GenerationConfig;459    import com.google.genai.gaos.models.interactions.Interaction;460    import com.google.genai.gaos.models.interactions.InteractionsInput;461    import com.google.genai.gaos.models.interactions.Model;462    import com.google.genai.gaos.models.interactions.ResponseFormat;463    import com.google.genai.gaos.models.interactions.SpeechConfig;464    import com.google.genai.gaos.models.interactions.SpeechConfigUnion;465    import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;466    import java.util.Arrays;467468    Client client = new Client();469470    CreateModelInteraction transcriptParams =471        CreateModelInteraction.builder()472            .model(Model.of("gemini-3.8-flash"))473            .input(474                InteractionsInput.of(475                    "Generate a short transcript around 100 words that reads "476                        + "like it was clipped from a podcast by excited herpetologists. "477                        + "The hosts names are Dr. Anya and Liam."))478            .build();479480    Interaction transcriptInteraction =481        client482            .interactions483            .create(CreateInteractionRequestBody.of(transcriptParams))484            .interaction()485            .get();486487    String transcript = transcriptInteraction.outputText().orElse("");488489    SpeechConfig anyaConfig = SpeechConfig.builder().speaker("Dr. Anya").voice("Kore").build();490    SpeechConfig liamConfig = SpeechConfig.builder().speaker("Liam").voice("Puck").build();491492    GenerationConfig generationConfig =493        GenerationConfig.builder()494            .speechConfig(SpeechConfigUnion.of(Arrays.asList(anyaConfig, liamConfig)))495            .build();496497    CreateModelInteraction ttsParams =498        CreateModelInteraction.builder()499            .model(Model.of("gemini-3.1-flash-tts-preview"))500            .input(InteractionsInput.of(transcript))501            .responseFormat(502                CreateModelInteractionResponseFormat.of(503                    ResponseFormat.of(AudioResponseFormat.builder().build())))504            .generationConfig(generationConfig)505            .build();506507    Interaction ttsInteraction =508        client.interactions.create(CreateInteractionRequestBody.of(ttsParams)).interaction().get();509510## Streaming speech generation511512You can stream the generated audio as it is being generated by the model by setting `stream: true`.513514> [!NOTE]515> **Note:** Streaming is supported for Text-to-Speech (TTS) models starting with version 3.1 (including `gemini-3.1-flash-tts-preview`).516517### Python518519    from google import genai520    import base64521522    client = genai.Client()523524    stream = client.interactions.create(525        model="gemini-3.1-flash-tts-preview",526        input="Say cheerfully: Have a wonderful day!",527        response_format={"type": "audio"},528        generation_config={529            "speech_config": [530                {"voice": "Kore"}531            ]532        },533        stream=True534    )535536    for event in stream:537        if event.event_type == "step.delta":538            if event.delta.type == "audio":539                audio_data = base64.b64decode(event.delta.data)540                # Process the audio chunk (e.g. play it or write to a file)541542### JavaScript543544    import {GoogleGenAI} from '@google/genai';545546    async function main() {547       const client = new GoogleGenAI({});548549       const stream = await client.interactions.create({550          model: "gemini-3.1-flash-tts-preview",551          input: "Say cheerfully: Have a wonderful day!",552          response_format: { type: 'audio' },553          generation_config: {554             speech_config: [555                { voice: 'Kore' }556             ]557          },558          stream: true559       });560561       for await (const event of stream) {562          if (event.event_type === 'step.delta') {563             if (event.delta.type === 'audio') {564                const audioBuffer = Buffer.from(event.delta.data, 'base64');565                // Process the audio buffer566             }567          }568       }569    }570    await main();571572### Java573574    import com.google.genai.Client;575    import com.google.genai.gaos.models.interactions.AudioDelta;576    import com.google.genai.gaos.models.interactions.AudioResponseFormat;577    import com.google.genai.gaos.models.interactions.CreateModelInteraction;578    import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;579    import com.google.genai.gaos.models.interactions.GenerationConfig;580    import com.google.genai.gaos.models.interactions.InteractionSSEEvent;581    import com.google.genai.gaos.models.interactions.InteractionSSEStreamEvent;582    import com.google.genai.gaos.models.interactions.InteractionsInput;583    import com.google.genai.gaos.models.interactions.Model;584    import com.google.genai.gaos.models.interactions.ResponseFormat;585    import com.google.genai.gaos.models.interactions.SpeechConfig;586    import com.google.genai.gaos.models.interactions.SpeechConfigUnion;587    import com.google.genai.gaos.models.interactions.StepDelta;588    import com.google.genai.gaos.models.interactions.StepDeltaData;589    import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;590    import com.google.genai.gaos.models.operations.CreateInteractionResponse;591    import com.google.genai.gaos.utils.EventStream;592    import java.util.Arrays;593    import java.util.Base64;594595    Client client = new Client();596597    SpeechConfig speechConfig = SpeechConfig.builder().voice("Kore").build();598    GenerationConfig generationConfig =599        GenerationConfig.builder()600            .speechConfig(SpeechConfigUnion.of(Arrays.asList(speechConfig)))601            .build();602603    CreateModelInteraction params =604        CreateModelInteraction.builder()605            .model(Model.of("gemini-3.1-flash-tts-preview"))606            .input(InteractionsInput.of("Say cheerfully: Have a wonderful day!"))607            .responseFormat(608                CreateModelInteractionResponseFormat.of(609                    ResponseFormat.of(AudioResponseFormat.builder().build())))610            .generationConfig(generationConfig)611            .stream(true)612            .build();613614    CreateInteractionResponse response =615        client.interactions.create(CreateInteractionRequestBody.of(params));616617    try (EventStream<InteractionSSEStreamEvent> events = response.events()) {618      for (InteractionSSEStreamEvent streamEvent : events) {619        InteractionSSEEvent event = streamEvent.data().orElse(null);620        if (event instanceof StepDelta) {621          StepDeltaData deltaData = ((StepDelta) event).delta().orElse(null);622          if (deltaData instanceof AudioDelta) {623            AudioDelta audioDelta = (AudioDelta) deltaData;624            if (audioDelta.data().isPresent()) {625              byte[] audioData = Base64.getDecoder().decode(audioDelta.data().get());626              // Process the audio chunk (e.g. play it or write to a file)627            }628          }629        }630      }631    }632633### REST634635    curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions"       -H "x-goog-api-key: $GEMINI_API_KEY"       -H "Content-Type: application/json"       -H "Api-Revision: 2026-05-20"       --no-buffer       -d '{636        "model": "gemini-3.1-flash-tts-preview",637        "input": "Say cheerfully: Have a wonderful day!",638        "response_format": {639          "type": "audio"640        },641        "generation_config": {642          "speech_config": [643            { "voice": "Kore" }644          ]645        },646        "stream": true647      }'648649## Voice options650651TTS models support the following 30 voice options in the `voice_name` field:652653|---|---|---|654| **Zephyr** -- *Bright* | **Puck** -- *Upbeat* | **Charon** -- *Informative* |655| **Kore** -- *Firm* | **Fenrir** -- *Excitable* | **Leda** -- *Youthful* |656| **Orus** -- *Firm* | **Aoede** -- *Breezy* | **Callirrhoe** -- *Easy-going* |657| **Autonoe** -- *Bright* | **Enceladus** -- *Breathy* | **Iapetus** -- *Clear* |658| **Umbriel** -- *Easy-going* | **Algieba** -- *Smooth* | **Despina** -- *Smooth* |659| **Erinome** -- *Clear* | **Algenib** -- *Gravelly* | **Rasalgethi** -- *Informative* |660| **Laomedeia** -- *Upbeat* | **Achernar** -- *Soft* | **Alnilam** -- *Firm* |661| **Schedar** -- *Even* | **Gacrux** -- *Mature* | **Pulcherrima** -- *Forward* |662| **Achird** -- *Friendly* | **Zubenelgenubi** -- *Casual* | **Vindemiatrix** -- *Gentle* |663| **Sadachbia** -- *Lively* | **Sadaltager** -- *Knowledgeable* | **Sulafat** -- *Warm* |664665You can hear all the voice options in [AI Studio](https://aistudio.google.com/generate-speech).666667## Supported languages668669The TTS models detect the input language automatically. The following languages670are supported:671672| Language | BCP-47 Code | Language | BCP-47 Code |673|---|---|---|---|674| Arabic | ar | Filipino | fil |675| Bangla | bn | Finnish | fi |676| Dutch | nl | Galician | gl |677| English | en | Georgian | ka |678| French | fr | Greek | el |679| German | de | Gujarati | gu |680| Hindi | hi | Haitian Creole | ht |681| Indonesian | id | Hebrew | he |682| Italian | it | Hungarian | hu |683| Japanese | ja | Icelandic | is |684| Korean | ko | Javanese | jv |685| Marathi | mr | Kannada | kn |686| Polish | pl | Konkani | kok |687| Portuguese | pt | Lao | lo |688| Romanian | ro | Latin | la |689| Russian | ru | Latvian | lv |690| Spanish | es | Lithuanian | lt |691| Tamil | ta | Luxembourgish | lb |692| Telugu | te | Macedonian | mk |693| Thai | th | Maithili | mai |694| Turkish | tr | Malagasy | mg |695| Ukrainian | uk | Malay | ms |696| Vietnamese | vi | Malayalam | ml |697| Afrikaans | af | Mongolian | mn |698| Albanian | sq | Nepali | ne |699| Amharic | am | Norwegian, Bokmål | nb |700| Armenian | hy | Norwegian, Nynorsk | nn |701| Azerbaijani | az | Odia | or |702| Basque | eu | Pashto | ps |703| Belarusian | be | Persian | fa |704| Bulgarian | bg | Punjabi | pa |705| Burmese | my | Serbian | sr |706| Catalan | ca | Sindhi | sd |707| Cebuano | ceb | Sinhala | si |708| Chinese, Mandarin | cmn | Slovak | sk |709| Croatian | hr | Slovenian | sl |710| Czech | cs | Swahili | sw |711| Danish | da | Swedish | sv |712| Estonian | et | Urdu | ur |713714## Supported models715716| Model | Single speaker | Multispeaker |717|---|---|---|718| [Gemini 3.1 Flash TTS Preview](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-tts-preview) | ✔️ | ✔️ |719| [Gemini 2.5 Flash Preview TTS](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-tts-preview) | ✔️ | ✔️ |720| [Gemini 2.5 Pro Preview TTS](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-pro-preview-tts) | ✔️ | ✔️ |721722## Prompting guide723724The **Gemini Native Audio Generation Text-to-Speech (TTS)** model differentiates725itself from conventional TTS models by using a large language model that726knows ***not only what to say, but also how to say it***.727728You can think of an advanced prompt as a system instruction for the model to729follow. It's a way to give the model more context and control over the730performance.731732To unlock this capability, users can think of themselves as directors setting a733scene for a virtual voice talent to perform. To craft a prompt, we recommend734considering the following components: an **Audio Profile** that defines the735character's core identity and archetype; a **Scene description** that736establishes the physical environment and emotional "vibe"; and **Director's737Notes** that offer more precise performance guidance regarding style, accent and738pace control.739740By providing nuanced instructions such as a precise regional accent, specific741paralinguistic features (e.g. breathiness), or pacing, users can leverage the742model's context awareness to generate highly dynamic, natural and expressive743audio performances. For optimal performance, we recommend the **Transcript** and744directorial prompts align, *so that "who is saying it"* matches with *"what is745said"* and *"how it is being said."*746747The purpose of this guide is to offer fundamental direction and spark ideas when748developing audio experiences using Gemini TTS audio generation. We are excited749to witness what you create!750751### Audio tags752753Tags are inline modifiers like `[whispers]` or `[laughs]` that give you granular754control over the delivery. You can use them to change the tone, pace, and755emotional vibe of a line or section of the transcript. You can also use them to756add interjections and a few other non-verbal sounds to the performance, like757`[cough]`, `[sighs]` or `[gasp]`.758759There is no exhaustive list on what tags do and don't work, we recommend760experimenting with different emotions and expressions to see how the output761changes.762763If your transcript is not in English, for best results we recommend that you764still use English audio tags.765766**Be creative with audio tags**767768To show the kind of variability you can get with audio tags, here are a set of769examples that each say the same thing, but the delivery changes based on the770tags used.771772You can change the emphasis of the delivery by adding tags at the start of a773line to make the speaker excited, bored, or reluctant:774775- `[excitedly]` Hey there, I'm a new text to speech model, and I can say things in many different ways. How can I help you today?776- `[bored]` Hey there, I'm a new text to speech model...777- `[reluctantly]` Hey there, I'm a new text to speech model...778779Tags can also be used to change the pace of the delivery, or to combine pace780with emphasis:781782- `[very fast]` Hey there, I'm a new text to speech model...783- `[very slow]` Hey there, I'm a new text to speech model...784- `[sarcastically, one painfully slow word at a time]` Hey there, I'm a new text to speech model...785786You also have precise control over specific sections, meaning you can whisper787one part and shout another.788789- `[whispers]` Hey there, I'm a new text to speech model, `[shouting]` and I can say things in many different ways. `[whispers]` How can I help you today790791You can also experiment with any creative idea you want:792793- `[like a cartoon dog]` Hey there, I'm a new text to speech model...794- `[like dracula]` Hey there, I'm a new text to speech model...795796Commonly used tags include:797798|---|---|---|---|799| `[amazed]` | `[crying]` | `[curious]` | `[excited]` |800| `[sighs]` | `[gasp]` | `[giggles]` | `[laughs]` |801| `[mischievously]` | `[panicked]` | `[sarcastic]` | `[serious]` |802| `[shouting]` | `[tired]` | `[trembling]` | `[whispers]` |803804Tags give quick control over the delivery of your transcript. For even805more control, you can combine them with a context prompt to set the overall tone806and vibe of the performance.807808### Prompting structure809810A robust prompt ideally includes the following elements that come together to811craft a great performance:812813- **Audio Profile** - Establishes a persona for the voice, defining a character identity, archetype and any other characteristics like age, background etc.814- **Scene** - Sets the stage. Describes both the physical environment and the "vibe".815- **Director's Notes** - Performance guidance where you can break down which instructions are important for your virtual talent to take note of. Examples are style, breathing, pacing, articulation and accent.816- **Sample context** - Gives the model a contextual starting point, so your virtual actor enters the scene you set up naturally.817- **Transcript** - The text that the model will speak out. For best performance, remember that the transcript topic and writing style should correlate to the directions you are giving.818- **Audio tags** - Modifiers you can put into a transcript to change how that part of the text is delivered, such as `[whispers]` or `[shouting]`.819820> [!NOTE]821> **Note:** Have Gemini help you build your prompt, just give it a blank outline of the following format and ask it to sketch out a character for you.822823Example full prompt:824825    # AUDIO PROFILE: Jaz R.826    ## "The Morning Hype"827828    ## THE SCENE: The London Studio829    It is 10:00 PM in a glass-walled studio overlooking the moonlit London skyline,830    but inside, it is blindingly bright. The red "ON AIR" tally light is blazing.831    Jaz is standing up, not sitting, bouncing on the balls of their heels to the832    rhythm of a thumping backing track. Their hands fly across the faders on a833    massive mixing desk. It is a chaotic, caffeine-fueled cockpit designed to wake834    up an entire nation.835836    ### DIRECTOR'S NOTES837    Style:838    * The "Vocal Smile": You must hear the grin in the audio. The soft palate is839    always raised to keep the tone bright, sunny, and explicitly inviting.840    * Dynamics: High projection without shouting. Punchy consonants and elongated841    vowels on excitement words (e.g., "Beauuutiful morning").842843    Pace: Speaks at an energetic pace, keeping up with the fast music.  Speaks844    with A "bouncing" cadence. High-speed delivery with fluid transitions - no dead845    air, no gaps.846847    Accent: Jaz is from Brixton, London848849    ### SAMPLE CONTEXT850    Jaz is the industry standard for Top 40 radio, high-octane event promos, or any851    script that requires a charismatic Estuary accent and 11/10 infectious energy.852853    #### TRANSCRIPT854    Yes, massive vibes in the studio! You are locked in and it is absolutely855    popping off in London right now. If you're stuck on the tube, or just sat856    there pretending to work... stop it. Seriously, I see you. Turn this up!857    We've got the project roadmap landing in three, two... let's go!858859### Detailed Prompting Strategies860861Break down each element of the prompt as follows:862863#### Audio Profile864865Briefly describe the persona of the character.866867- **Name.** Giving your character a name helps ground the model and tight performance together, Refer to the character by name when setting the scene and context868- **Role.** Core identity and archetype of the character that's playing out in the scene. e.g., Radio DJ, Podcaster, News reporter etc.869870Examples:871872    # AUDIO PROFILE: Jaz R.873    ## "The Morning Hype"874875<br />876877    # AUDIO PROFILE: Monica A.878    ## "The Beauty Influencer"879880#### Scene881882Set the context for the scene, including location, mood, and environmental883details that establish the tone and vibe. Describe what is happening around the884character and how it affects them. The scene provides the environmental context885for the entire interaction and guides the acting performance in a subtle886organic way.887888Examples:889890    ## THE SCENE: The London Studio891    It is 10:00 PM in a glass-walled studio overlooking the moonlit London skyline,892    but inside, it is blindingly bright. The red "ON AIR" tally light is blazing.893    Jaz is standing up, not sitting, bouncing on the balls of their heels to the894    rhythm of a thumping backing track. Their hands fly across the faders on a895    massive mixing desk. It is a chaotic, caffeine-fueled cockpit designed to896    wake up an entire nation.897898<br />899900    ## THE SCENE: Homegrown Studio901    A meticulously sound-treated bedroom in a suburban home. The space is902    deadened by plush velvet curtains and a heavy rug, but there is a903    distinct "proximity effect."904905#### Directors notes906907This critical section includes specific performance guidance. You can skip all908the other elements, but we recommend you include this element.909910Define only what's important to the performance, being careful to not911overspecify. Too many strict rules will limit the models' creativity and may912result in a worse performance. Balance the role and scene description with the913specific performance rules.914915The most common directions are **Style, Pacing and Accent**, but the model is916not limited to these, nor requires them. Feel free to include custom917instructions to cover any additional details important to your performance, and918go into as much or as little detail as necessary.919920For example:921922    ### DIRECTOR'S NOTES923924    Style: Enthusiastic and Sassy GenZ beauty YouTuber925926    Pacing: Speaks at an energetic pace, keeping up with the extremely fast, rapid927    delivery influencers use in short form videos.928929    Accent: Southern california valley girl from Laguna Beach |930931**Style:**932933Sets the tone and Style of the generated speech. Include things like upbeat,934energetic, relaxed, bored etc. to guide the performance. Be descriptive and935provide as much detail as necessary: *"Infectious enthusiasm. The listener936should feel like they are part of a massive, exciting community event."* works937better than saying *"energetic and enthusiastic".*938939You can even try terms that are popular in the voiceover industry, like "vocal940smile". You can layer as many style characteristics as you want.941942Examples:943944Simple Emotion945946    DIRECTORS NOTES947    ...948    Style: Frustrated and angry developer who can't get the build to run.949    ...950951More depth952953    DIRECTORS NOTES954    ...955    Style: Sassy GenZ beauty YouTuber, who mostly creates content for YouTube Shorts.956    ...957958Complex959960    DIRECTORS NOTES961    Style:962    * The "Vocal Smile": You must hear the grin in the audio. The soft palate is963    always raised to keep the tone bright, sunny, and explicitly inviting.964    *Dynamics: High projection without shouting. Punchy consonants and965    elongated vowels on excitement words (e.g., "Beauuutiful morning").966967**Accent:**968969Describe the selected accent. The more specific you are, the better the970results are. For example use "*British English accent as heard in Croydon,971England* " versus "*British Accent*".972973Examples:974975    ### DIRECTORS NOTES976    ...977    Accent: Southern california valley girl from Laguna Beach978    ...979980<br />981982    ### DIRECTORS NOTES983    ...984    Accent: Jaz is a from Brixton, London985    ...986987**Pacing:**988989Overall pacing and pace variation throughout the piece.990991Examples:992993Simple994995    ### DIRECTORS NOTES996    ...997    Pacing: Speak as fast as possible998    ...9991000More Depth10011002    ### DIRECTORS NOTES1003    ...1004    Pacing: Speaks at a faster, energetic pace, keeping up with fast paced music.1005    ...10061007Complex10081009    ### DIRECTORS NOTES1010    ...1011    Pacing: The "Drift": The tempo is incredibly slow and liquid. Words bleed into each other. There is zero urgency.1012    ...10131014### Give it a try10151016You can try these examples yourself by using the [Voice Library](https://aistudio.google.com/apps/bundled/voice-library?showPreview=true). Use the1017following guidelines to make great vocal performances:10181019- Remember to keep the entire prompt coherent -- the script and direction go hand in hand in creating a great performance.1020- Don't feel you have to describe everything, sometimes giving the model space to fill in the gaps helps naturalness. (Just like a talented actor)1021- If you ever are feeling stuck, have Gemini lend you a hand to help you craft your script or performance.10221023## Limitations10241025- TTS models can only receive text inputs and generate audio outputs.1026- A TTS session has a [context window](https://ai.google.dev/gemini-api/docs/long-context) limit of 32k tokens.1027- Review [Languages](https://ai.google.dev/gemini-api/docs/speech-generation#languages) section for language support.1028- TTS does not support streaming, except when using `gemini-3.1-flash-tts-preview`.10291030The following constraints apply specifically when using the Gemini 3.1 Flash1031TTS Preview model for speech generation:10321033- **Voice inconsistency with prompt instructions:** The model's output may not always strictly match the selected speaker, causing the audio to sound different than expected. To avoid mismatched tones (such as a deep male voice attempting to speak like a young girl), ensure your prompt's written tone and context align naturally with the selected speaker's profile.1034- **Quality of longer outputs:** Speech quality and consistency may begin to drift with generated outputs that are longer than a few minutes. We recommend splitting your transcripts into smaller chunks.1035- **Occasional text token returns:** The model occasionally returns text tokens instead of audio tokens, causing the server to fail the request with a `500` error. Because this occurs randomly in a very small percentage of requests, you should implement automated retry logic in your application to handle these.1036- **Prompt classifier false rejections:** Vague prompts may fail to trigger the speech synthesis classifier, resulting in a rejected request (`PROHIBITED_CONTENT`) or causing the model to read your style instructions and director's notes aloud. Validate your prompts by adding a clear preamble instructing the model to synthesize speech, and explicitly label where the actual spoken transcript begins.10371038## What's next10391040- Gemini's [Live API](https://ai.google.dev/gemini-api/docs/live) offers interactive audio generation options you can interleave with other modalities.1041- For working with audio *inputs* , visit the [Audio understanding](https://ai.google.dev/gemini-api/docs/audio) guide.
Files main
Menu

Theme