SPB Git

Repository Public

audio.md 26.8 KB 911 lines markdown

1Gemini can analyze audio input and generate text responses.23### Python45    from google import genai67    client = genai.Client()89    myfile = client.files.upload(file="path/to/sample.mp3")1011    response = client.models.generate_content(12        model="gemini-3.8-flash", contents=["Describe this audio clip", myfile]13    )1415    print(response.text)1617### JavaScript1819    import {20      GoogleGenAI,21      createUserContent,22      createPartFromUri,23    } from "@google/genai";2425    const ai = new GoogleGenAI({});2627    async function main() {28      const myfile = await ai.files.upload({29        file: "path/to/sample.mp3",30        config: { mimeType: "audio/mp3" },31      });3233      const response = await ai.models.generateContent({34        model: "gemini-3.8-flash",35        contents: createUserContent([36          createPartFromUri(myfile.uri, myfile.mimeType),37          "Describe this audio clip",38        ]),39      });40      console.log(response.text);41    }4243    await main();4445### Go4647    package main4849    import (50        "context"51        "fmt"52        "os"53        "google.golang.org/genai"54    )5556    func main() {57        ctx := context.Background()58        client, err := genai.NewClient(ctx, nil)59        if err != nil {60            log.Fatal(err)61        }6263        localAudioPath := "/path/to/sample.mp3"64        uploadedFile, _ := client.Files.UploadFromPath(65            ctx,66            localAudioPath,67            nil,68        )6970        parts := []*genai.Part{71            genai.NewPartFromText("Describe this audio clip"),72            genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),73        }74        contents := []*genai.Content{75            genai.NewContentFromParts(parts, genai.RoleUser),76        }7778        result, _ := client.Models.GenerateContent(79            ctx,80            "gemini-3.8-flash",81            contents,82            nil,83        )8485        fmt.Println(result.Text())86    }8788### REST8990    AUDIO_PATH="path/to/sample.mp3"91    MIME_TYPE=$(file -b --mime-type "${AUDIO_PATH}")92    NUM_BYTES=$(wc -c < "${AUDIO_PATH}")93    DISPLAY_NAME=AUDIO9495    tmp_header_file=upload-header.tmp9697    # Initial resumable request defining metadata.98    # The upload url is in the response headers dump them to a file.99    curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \100      -H "x-goog-api-key: $GEMINI_API_KEY" \101      -D upload-header.tmp \102      -H "X-Goog-Upload-Protocol: resumable" \103      -H "X-Goog-Upload-Command: start" \104      -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \105      -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \106      -H "Content-Type: application/json" \107      -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null108109    upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")110    rm "${tmp_header_file}"111112    # Upload the actual bytes.113    curl "${upload_url}" \114      -H "Content-Length: ${NUM_BYTES}" \115      -H "X-Goog-Upload-Offset: 0" \116      -H "X-Goog-Upload-Command: upload, finalize" \117      --data-binary "@${AUDIO_PATH}" 2> /dev/null > file_info.json118119    file_uri=$(jq ".file.uri" file_info.json)120    echo file_uri=$file_uri121122    # Now generate content using that file123    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \124        -H "x-goog-api-key: $GEMINI_API_KEY" \125        -H 'Content-Type: application/json' \126        -X POST \127        -d '{128          "contents": [{129            "parts":[130              {"text": "Describe this audio clip"},131              {"file_data":{"mime_type": "${MIME_TYPE}", "file_uri": '$file_uri'}}]132            }]133          }' 2> /dev/null > response.json134135    cat response.json136    echo137138    jq ".candidates[].content.parts[].text" response.json139140## Overview141142Gemini can analyze and understand audio input and generate text responses to it,143enabling use cases like the following:144145- Describe, summarize, or answer questions about audio content.146- Provide a transcription and translation of the audio (speech to text).147- Detect emotion in speech and music.148- Analyze specific segments of the audio, and provide timestamps.149150As of now the Gemini API doesn't support real-time transcription use cases.151For real-time voice and video interactions refer to the [Live API](https://ai.google.dev/gemini-api/docs/live).152For dedicated speech to text models with support for real-time transcription,153use the [Google Cloud Speech-to-Text API](https://cloud.google.com/speech-to-text).154155## Transcribe speech to text156157This example application shows how to prompt the Gemini API to transcribe,158translate, and summarize speech, including timestamps and emotion detection159using [structured outputs](https://ai.google.dev/gemini-api/docs/structured-output).160161### Python162163    from google import genai164    from google.genai import types165166    client = genai.Client()167168    YOUTUBE_URL = "https://www.youtube.com/watch?v=ku-N-eS1lgM"169170    def main():171      prompt = """172        Process the audio file and generate a detailed transcription.173174        Requirements:175        1. Provide accurate timestamps for each segment (Format: MM:SS).176        2. Detect the primary language of each segment.177        3. If the segment is in a language different than English, also provide the English translation.178        4. Identify the primary emotion of the speaker in this segment. You MUST choose exactly one of the following: Happy, Sad, Angry, Neutral.179        5. Provide a brief summary of the entire audio at the beginning.180      """181182      response = client.models.generate_content(183        model="gemini-3.8-flash",184        contents=[185          types.Content(186            parts=[187              types.Part(188                file_data=types.FileData(189                  file_uri=YOUTUBE_URL190                )191              ),192              types.Part(193                text=prompt194              )195            ]196          )197        ],198        config=types.GenerateContentConfig(199          response_format={"text": {"mime_type": "application/json"}},200          response_schema=types.Schema(201            type=types.Type.OBJECT,202            properties={203              "summary": types.Schema(204                type=types.Type.STRING,205                description="A concise summary of the audio content.",206              ),207              "segments": types.Schema(208                type=types.Type.ARRAY,209                description="List of transcribed segments with timestamp.",210                items=types.Schema(211                  type=types.Type.OBJECT,212                  properties={213                    "timestamp": types.Schema(type=types.Type.STRING),214                    "content": types.Schema(type=types.Type.STRING),215                    "language": types.Schema(type=types.Type.STRING),216                    "language_code": types.Schema(type=types.Type.STRING),217                    "translation": types.Schema(type=types.Type.STRING),218                    "emotion": types.Schema(219                      type=types.Type.STRING,220                      enum=["happy", "sad", "angry", "neutral"]221                    ),222                  },223                  required=["timestamp", "content", "language", "language_code", "emotion"],224                ),225              ),226            },227            required=["summary", "segments"],228          ),229        ),230      )231232      print(response.text)233234    if __name__ == "__main__":235      main()236237### JavaScript238239    import {240      GoogleGenAI,241      Type242    } from "@google/genai";243244    const ai = new GoogleGenAI({});245246    const YOUTUBE_URL = "https://www.youtube.com/watch?v=ku-N-eS1lgM";247248    async function main() {249      const prompt = `250          Process the audio file and generate a detailed transcription.251252          Requirements:253          1. Provide accurate timestamps for each segment (Format: MM:SS).254          2. Detect the primary language of each segment.255          3. If the segment is in a language different than English, also provide the English translation.256          4. Identify the primary emotion of the speaker in this segment. You MUST choose exactly one of the following: Happy, Sad, Angry, Neutral.257          5. Provide a brief summary of the entire audio at the beginning.258        `;259260      const Emotion = {261        Happy: 'happy',262        Sad: 'sad',263        Angry: 'angry',264        Neutral: 'neutral'265      };266267      const response = await ai.models.generateContent({268        model: "gemini-3.8-flash",269        contents: {270          parts: [271            {272              fileData: {273                fileUri: YOUTUBE_URL,274              },275            },276            {277              text: prompt,278            },279          ],280        },281        config: {282          responseFormat: { text: { mimeType: "application/json" } },283          responseSchema: {284            type: Type.OBJECT,285            properties: {286              summary: {287                type: Type.STRING,288                description: "A concise summary of the audio content.",289              },290              segments: {291                type: Type.ARRAY,292                description: "List of transcribed segments with timestamp.",293                items: {294                  type: Type.OBJECT,295                  properties: {296                    timestamp: { type: Type.STRING },297                    content: { type: Type.STRING },298                    language: { type: Type.STRING },299                    language_code: { type: Type.STRING },300                    translation: { type: Type.STRING },301                    emotion: {302                      type: Type.STRING,303                      enum: Object.values(Emotion)304                    },305                  },306                  required: ["timestamp", "content", "language", "language_code", "emotion"],307                },308              },309            },310            required: ["summary", "segments"],311          },312        },313      });314      const json = JSON.parse(response.text);315      console.log(json);316    }317318    await main();319320### REST321322    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \323        -H "x-goog-api-key: $GEMINI_API_KEY" \324        -H 'Content-Type: application/json' \325        -X POST \326        -d '{327          "contents": [328            {329              "parts": [330                {331                  "file_data": {332                    "file_uri": "https://www.youtube.com/watch?v=ku-N-eS1lgM",333                    "mime_type": "video/mp4"334                  }335                },336                {337                  "text": "Process the audio file and generate a detailed transcription.\n\nRequirements:\n1. Provide accurate timestamps for each segment (Format: MM:SS).\n2. Detect the primary language of each segment.\n3. If the segment is in a language different than English, also provide the English translation.\n4. Identify the primary emotion of the speaker in this segment. You MUST choose exactly one of the following: Happy, Sad, Angry, Neutral.\n5. Provide a brief summary of the entire audio at the beginning."338                }339              ]340            }341          ],342          "generation_config": {343            "response_format": {"text": {"mime_type": "application/json"}},344            "response_schema": {345              "type": "OBJECT",346              "properties": {347                "summary": {348                  "type": "STRING",349                  "description": "A concise summary of the audio content."350                },351                "segments": {352                  "type": "ARRAY",353                  "description": "List of transcribed segments with timestamp.",354                  "items": {355                    "type": "OBJECT",356                    "properties": {357                      "timestamp": { "type": "STRING" },358                      "content": { "type": "STRING" },359                      "language": { "type": "STRING" },360                      "language_code": { "type": "STRING" },361                      "translation": { "type": "STRING" },362                      "emotion": {363                        "type": "STRING",364                        "enum": ["happy", "sad", "angry", "neutral"]365                      }366                    },367                    "required": ["timestamp", "content", "language", "language_code", "emotion"]368                  }369                }370              },371              "required": ["summary", "segments"]372            }373          }374        }' 2> /dev/null > response.json375376    cat response.json377    echo378379    jq ".candidates[].content.parts[].text" response.json380381You can prompt [AI Studio Build](https://aistudio.google.com/apps?e=0) to create382an app just like383[this example transcription app](https://aistudio.google.com/apps/bundled/echoscript)384, with the click of a button.385386![A multilingual audio transcription Gemini app](https://ai.google.dev/static/gemini-api/docs/images/audio_understanding_demo.gif)387388## Input audio389390You can provide audio data to Gemini in the following ways:391392- [Upload an audio file](https://ai.google.dev/gemini-api/docs/generate-content/audio#upload-audio) before making a request to `generateContent`.393- [Pass inline audio data](https://ai.google.dev/gemini-api/docs/generate-content/audio#inline-audio) with the request to `generateContent`.394395To learn about other file input methods, see the396[File input methods](https://ai.google.dev/gemini-api/docs/file-input-methods) guide.397398### Upload an audio file399400You can use the [Files API](https://ai.google.dev/gemini-api/docs/files) to upload an audio file.401Always use the Files API when the total request size (including the files, text402prompt, system instructions, etc.) is larger than 20 MB.403404The following code uploads an audio file and then uses the file in a call to405`generateContent`.406407### Python408409    from google import genai410411    client = genai.Client()412413    myfile = client.files.upload(file="path/to/sample.mp3")414415    response = client.models.generate_content(416        model="gemini-3.8-flash", contents=["Describe this audio clip", myfile]417    )418419    print(response.text)420421### JavaScript422423    import {424      GoogleGenAI,425      createUserContent,426      createPartFromUri,427    } from "@google/genai";428429    const ai = new GoogleGenAI({});430431    async function main() {432      const myfile = await ai.files.upload({433        file: "path/to/sample.mp3",434        config: { mimeType: "audio/mp3" },435      });436437      const response = await ai.models.generateContent({438        model: "gemini-3.8-flash",439        contents: createUserContent([440          createPartFromUri(myfile.uri, myfile.mimeType),441          "Describe this audio clip",442        ]),443      });444      console.log(response.text);445    }446447    await main();448449### Go450451    package main452453    import (454      "context"455      "fmt"456      "os"457      "google.golang.org/genai"458    )459460    func main() {461      ctx := context.Background()462      client, err := genai.NewClient(ctx, nil)463      if err != nil {464          log.Fatal(err)465      }466467      localAudioPath := "/path/to/sample.mp3"468      uploadedFile, _ := client.Files.UploadFromPath(469          ctx,470          localAudioPath,471          nil,472      )473474      parts := []*genai.Part{475          genai.NewPartFromText("Describe this audio clip"),476          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),477      }478      contents := []*genai.Content{479          genai.NewContentFromParts(parts, genai.RoleUser),480      }481482      result, _ := client.Models.GenerateContent(483          ctx,484          "gemini-3.8-flash",485          contents,486          nil,487      )488489      fmt.Println(result.Text())490    }491492### REST493494    AUDIO_PATH="path/to/sample.mp3"495    MIME_TYPE=$(file -b --mime-type "${AUDIO_PATH}")496    NUM_BYTES=$(wc -c < "${AUDIO_PATH}")497    DISPLAY_NAME=AUDIO498499    tmp_header_file=upload-header.tmp500501    # Initial resumable request defining metadata.502    # The upload url is in the response headers dump them to a file.503    curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \504      -H "x-goog-api-key: $GEMINI_API_KEY" \505      -D upload-header.tmp \506      -H "X-Goog-Upload-Protocol: resumable" \507      -H "X-Goog-Upload-Command: start" \508      -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \509      -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \510      -H "Content-Type: application/json" \511      -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null512513    upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")514    rm "${tmp_header_file}"515516    # Upload the actual bytes.517    curl "${upload_url}" \518      -H "Content-Length: ${NUM_BYTES}" \519      -H "X-Goog-Upload-Offset: 0" \520      -H "X-Goog-Upload-Command: upload, finalize" \521      --data-binary "@${AUDIO_PATH}" 2> /dev/null > file_info.json522523    file_uri=$(jq ".file.uri" file_info.json)524    echo file_uri=$file_uri525526    # Now generate content using that file527    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \528        -H "x-goog-api-key: $GEMINI_API_KEY" \529        -H 'Content-Type: application/json' \530        -X POST \531        -d '{532          "contents": [{533            "parts":[534              {"text": "Describe this audio clip"},535              {"file_data":{"mime_type": "${MIME_TYPE}", "file_uri": '$file_uri'}}]536            }]537          }' 2> /dev/null > response.json538539    cat response.json540    echo541542    jq ".candidates[].content.parts[].text" response.json543544To learn more about working with media files, see545[Files API](https://ai.google.dev/gemini-api/docs/files).546547### Pass audio data inline548549Instead of uploading an audio file, you can pass inline audio data in the550request to `generateContent`:551552### Python553554    from google import genai555    from google.genai import types556557    with open('path/to/small-sample.mp3', 'rb') as f:558        audio_bytes = f.read()559560    client = genai.Client()561    response = client.models.generate_content(562      model='gemini-3.8-flash',563      contents=[564        'Describe this audio clip',565        types.Part.from_bytes(566          data=audio_bytes,567          mime_type='audio/mp3',568        )569      ]570    )571572    print(response.text)573574### JavaScript575576    import { GoogleGenAI } from "@google/genai";577    import * as fs from "node:fs";578579    const ai = new GoogleGenAI({});580    const base64AudioFile = fs.readFileSync("path/to/small-sample.mp3", {581      encoding: "base64",582    });583584    const contents = [585      { text: "Please summarize the audio." },586      {587        inlineData: {588          mimeType: "audio/mp3",589          data: base64AudioFile,590        },591      },592    ];593594    const response = await ai.models.generateContent({595      model: "gemini-3.8-flash",596      contents: contents,597    });598    console.log(response.text);599600### Go601602    package main603604    import (605      "context"606      "fmt"607      "os"608      "google.golang.org/genai"609    )610611    func main() {612      ctx := context.Background()613      client, err := genai.NewClient(ctx, nil)614      if err != nil {615          log.Fatal(err)616      }617618      audioBytes, _ := os.ReadFile("/path/to/small-sample.mp3")619620      parts := []*genai.Part{621          genai.NewPartFromText("Describe this audio clip"),622        &genai.Part{623          InlineData: &genai.Blob{624            MIMEType: "audio/mp3",625            Data:     audioBytes,626          },627        },628      }629      contents := []*genai.Content{630          genai.NewContentFromParts(parts, genai.RoleUser),631      }632633      result, _ := client.Models.GenerateContent(634          ctx,635          "gemini-3.8-flash",636          contents,637          nil,638      )639640      fmt.Println(result.Text())641    }642643A few things to keep in mind about inline audio data:644645- The maximum request size is 20 MB, which includes text prompts, system instructions, and files provided inline. If your file's size will make the *total request size* exceed 20 MB, then use the Files API to [upload an audio file](https://ai.google.dev/gemini-api/docs/generate-content/audio#upload-audio) for use in the request.646- If you're using an audio sample multiple times, it's more efficient to [upload an audio file](https://ai.google.dev/gemini-api/docs/generate-content/audio#upload-audio).647648## Get a transcript649650To get a transcript of audio data, just ask for it in the prompt:651652### Python653654    from google import genai655656    client = genai.Client()657    myfile = client.files.upload(file='path/to/sample.mp3')658    prompt = 'Generate a transcript of the speech.'659660    response = client.models.generate_content(661      model='gemini-3.8-flash',662      contents=[prompt, myfile]663    )664665    print(response.text)666667### JavaScript668669    import {670      GoogleGenAI,671      createUserContent,672      createPartFromUri,673    } from "@google/genai";674675    const ai = new GoogleGenAI({});676    const myfile = await ai.files.upload({677      file: "path/to/sample.mp3",678      config: { mimeType: "audio/mpeg" },679    });680681    const result = await ai.models.generateContent({682      model: "gemini-3.8-flash",683      contents: createUserContent([684        createPartFromUri(myfile.uri, myfile.mimeType),685        "Generate a transcript of the speech.",686      ]),687    });688    console.log("result.text=", result.text);689690### Go691692    package main693694    import (695      "context"696      "fmt"697      "os"698      "google.golang.org/genai"699    )700701    func main() {702      ctx := context.Background()703      client, err := genai.NewClient(ctx, nil)704      if err != nil {705          log.Fatal(err)706      }707708      localAudioPath := "/path/to/sample.mp3"709      uploadedFile, _ := client.Files.UploadFromPath(710          ctx,711          localAudioPath,712          nil,713      )714715      parts := []*genai.Part{716          genai.NewPartFromText("Generate a transcript of the speech."),717          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),718      }719      contents := []*genai.Content{720          genai.NewContentFromParts(parts, genai.RoleUser),721      }722723      result, _ := client.Models.GenerateContent(724          ctx,725          "gemini-3.8-flash",726          contents,727          nil,728      )729730      fmt.Println(result.Text())731    }732733## Refer to timestamps734735You can refer to specific sections of an audio file using timestamps of the form736`MM:SS`. For example, the following prompt requests a transcript that737738- Starts at 2 minutes 30 seconds from the beginning of the file.739- Ends at 3 minutes 29 seconds from the beginning of the file.740741### Python742743    # Create a prompt containing timestamps.744    prompt = "Provide a transcript of the speech from 02:30 to 03:29."745746### JavaScript747748    // Create a prompt containing timestamps.749    const prompt = "Provide a transcript of the speech from 02:30 to 03:29."750751### Go752753    package main754755    import (756      "context"757      "fmt"758      "os"759      "google.golang.org/genai"760    )761762    func main() {763      ctx := context.Background()764      client, err := genai.NewClient(ctx, nil)765      if err != nil {766          log.Fatal(err)767      }768769      localAudioPath := "/path/to/sample.mp3"770      uploadedFile, _ := client.Files.UploadFromPath(771          ctx,772          localAudioPath,773          nil,774      )775776      parts := []*genai.Part{777          genai.NewPartFromText("Provide a transcript of the speech " +778                                "between the timestamps 02:30 and 03:29."),779          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),780      }781      contents := []*genai.Content{782          genai.NewContentFromParts(parts, genai.RoleUser),783      }784785      result, _ := client.Models.GenerateContent(786          ctx,787          "gemini-3.8-flash",788          contents,789          nil,790      )791792      fmt.Println(result.Text())793    }794795## Count tokens796797Call the `countTokens` method to get a count of the number of tokens in an798audio file. For example:799800### Python801802    from google import genai803804    client = genai.Client()805    response = client.models.count_tokens(806      model='gemini-3.8-flash',807      contents=[myfile]808    )809810    print(response)811812### JavaScript813814    import {815      GoogleGenAI,816      createUserContent,817      createPartFromUri,818    } from "@google/genai";819820    const ai = new GoogleGenAI({});821    const myfile = await ai.files.upload({822      file: "path/to/sample.mp3",823      config: { mimeType: "audio/mpeg" },824    });825826    const countTokensResponse = await ai.models.countTokens({827      model: "gemini-3.8-flash",828      contents: createUserContent([829        createPartFromUri(myfile.uri, myfile.mimeType),830      ]),831    });832    console.log(countTokensResponse.totalTokens);833834### Go835836    package main837838    import (839      "context"840      "fmt"841      "os"842      "google.golang.org/genai"843    )844845    func main() {846      ctx := context.Background()847      client, err := genai.NewClient(ctx, nil)848      if err != nil {849          log.Fatal(err)850      }851852      localAudioPath := "/path/to/sample.mp3"853      uploadedFile, _ := client.Files.UploadFromPath(854          ctx,855          localAudioPath,856          nil,857      )858859      parts := []*genai.Part{860          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),861      }862      contents := []*genai.Content{863          genai.NewContentFromParts(parts, genai.RoleUser),864      }865866      tokens, _ := client.Models.CountTokens(867          ctx,868          "gemini-3.8-flash",869          contents,870          nil,871      )872873      fmt.Printf("File %s is %d tokens\n", localAudioPath, tokens.TotalTokens)874    }875876## Supported audio formats877878Gemini supports the following audio format MIME types:879880- WAV - `audio/wav`881- MP3 - `audio/mp3`882- AIFF - `audio/aiff`883- AAC - `audio/aac`884- OGG - `audio/ogg`885- FLAC - `audio/flac`886- MPEG - `audio/mpeg`887- M4A - `audio/m4a`888- L16 - `audio/l16`889- Opus - `audio/opus`890- ALAW - `audio/alaw`891- MULAW - `audio/mulaw`892- WebM - `audio/webm`893894For the complete list of supported MIME types and parameter schemas, see the [Interactions API reference](https://ai.google.dev/api/interactions-api#Resource:Content).895896## Technical details about audio897898- Gemini represents each second of audio as 32 tokens; for example, one minute of audio is represented as 1,920 tokens.899- Gemini can "understand" non-speech components, such as birdsong or sirens.900- The maximum supported length of audio data in a single prompt is 9.5 hours. Gemini doesn't limit the *number* of audio files in a single prompt; however, the total combined length of all audio files in a single prompt can't exceed 9.5 hours.901- Gemini downsamples audio files to a 16 Kbps data resolution.902- If the audio source contains multiple channels, Gemini combines those channels into a single channel.903904## What's next905906This guide shows how to generate text in response to audio data. To learn more,907see the following resources:908909- [File prompting strategies](https://ai.google.dev/gemini-api/docs/files#prompt-guide): The Gemini API supports prompting with text, image, audio, and video data, also known as multimodal prompting.910- [System instructions](https://ai.google.dev/gemini-api/docs/text-generation#system-instructions): System instructions let you steer the behavior of the model based on your specific needs and use cases.911- [Safety guidance](https://ai.google.dev/gemini-api/docs/safety-guidance): Sometimes generative AI models produce unexpected outputs, such as outputs that are inaccurate, biased, or offensive. Post-processing and human evaluation are essential to limit the risk of harm from such outputs.
Files main
Menu

Theme