Repository Public
1Gemini can analyze audio input and generate text responses.23### Python45 from google import genai6 import base6478 client = genai.Client()910 uploaded_file = client.files.upload(file="path/to/sample.mp3")1112 interaction = client.interactions.create(13 model="gemini-3.8-flash",14 input=[15 {"type": "text", "text": "Describe this audio clip"},16 {17 "type": "audio",18 "uri": uploaded_file.uri,19 "mime_type": uploaded_file.mime_type20 }21 ]22 )23 print(interaction.output_text)2425### JavaScript2627 import { GoogleGenAI } from "@google/genai";2829 const client = new GoogleGenAI({});3031 const uploadedFile = await client.files.upload({32 file: "path/to/sample.mp3",33 config: { mime_type: "audio/mp3" }34 });3536 const interaction = await client.interactions.create({37 model: "gemini-3.8-flash",38 input: [39 {type: "text", text: "Describe this audio clip"},40 {41 type: "audio",42 uri: uploadedFile.uri,43 mime_type: uploadedFile.mimeType44 }45 ]46 });47 console.log(interaction.output_text);4849### Java5051 import com.google.genai.Client;52 import com.google.genai.gaos.models.interactions.AudioContent;53 import com.google.genai.gaos.models.interactions.AudioContentMimeType;54 import com.google.genai.gaos.models.interactions.Content;55 import com.google.genai.gaos.models.interactions.CreateModelInteraction;56 import com.google.genai.gaos.models.interactions.Interaction;57 import com.google.genai.gaos.models.interactions.InteractionsInput;58 import com.google.genai.gaos.models.interactions.Model;59 import com.google.genai.gaos.models.interactions.TextContent;60 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;61 import com.google.genai.types.File;62 import com.google.genai.types.UploadFileConfig;63 import java.util.Arrays;64 import java.util.List;6566 Client client = new Client();6768 File uploadedFile =69 client.files.upload(70 "path/to/sample.mp3", UploadFileConfig.builder().mimeType("audio/mp3").build());7172 Content textContent = TextContent.builder().text("Describe this audio clip").build();73 Content audioContent =74 AudioContent.builder()75 .uri(uploadedFile.uri().get())76 .mimeType(AudioContentMimeType.of(uploadedFile.mimeType().get()))77 .build();7879 List<Content> contents = Arrays.asList(textContent, audioContent);8081 CreateModelInteraction params =82 CreateModelInteraction.builder()83 .model(Model.of("gemini-3.8-flash"))84 .input(InteractionsInput.ofContent(contents))85 .build();8687 Interaction interaction =88 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();8990 System.out.println(interaction.outputText().orElse(""));9192### REST9394 # First upload the file, then use the URI:95 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \96 -H "x-goog-api-key: $GEMINI_API_KEY" \97 -H 'Content-Type: application/json' \98 -d '{99 "model": "gemini-3.8-flash",100 "input": [101 {"type": "text", "text": "Describe this audio clip"},102 {103 "type": "audio",104 "uri": "YOUR_FILE_URI",105 "mime_type": "audio/mp3"106 }107 ]108 }'109110## Overview111112Gemini can analyze and understand audio input and generate text responses,113enabling use cases like:114115- Describe, summarize, or answer questions about audio content116- Transcription and translation (speech to text)117- Speaker diarization (identifying different speakers)118- Emotion detection in speech and music119- Analyzing specific segments with timestamps120121For real-time voice and video interactions, see the122[Live API](https://ai.google.dev/gemini-api/docs/live).123For dedicated speech to text models with support for real-time transcription,124use the [Google Cloud Speech-to-Text API](https://cloud.google.com/speech-to-text).125126## Transcribe speech to text127128This example shows how to transcribe, translate, and summarize speech with129timestamps, speaker diarization, and emotion detection using130[structured outputs](https://ai.google.dev/gemini-api/docs/structured-output).131132### Python133134 from google import genai135136 client = genai.Client()137138 YOUTUBE_URL = "https://www.youtube.com/watch?v=ku-N-eS1lgM"139140 prompt = """141 Process the audio file and generate a detailed transcription.142143 Requirements:144 1. Identify distinct speakers (e.g., Speaker 1, Speaker 2).145 2. Provide accurate timestamps for each segment (Format: MM:SS).146 3. Detect the primary language of each segment.147 4. If not English, provide the English translation.148 5. Identify the primary emotion: Happy, Sad, Angry, or Neutral.149 6. Provide a brief summary at the beginning.150 """151152 response_schema = {153 "type": "object",154 "properties": {155 "summary": {"type": "string"},156 "segments": {157 "type": "array",158 "items": {159 "type": "object",160 "properties": {161 "speaker": {"type": "string"},162 "timestamp": {"type": "string"},163 "content": {"type": "string"},164 "language": {"type": "string"},165 "emotion": {166 "type": "string",167 "enum": ["happy", "sad", "angry", "neutral"]168 }169 },170 "required": ["speaker", "timestamp", "content", "emotion"]171 }172 }173 },174 "required": ["summary", "segments"]175 }176177 interaction = client.interactions.create(178 model="gemini-3.8-flash",179 input=[180 {"type": "video", "uri": YOUTUBE_URL, "mime_type": "video/mp4"},181 {"type": "text", "text": prompt}182 ],183 response_format=response_schema,184 )185186 print(interaction.output_text)187188### JavaScript189190 import { GoogleGenAI } from "@google/genai";191192 const client = new GoogleGenAI({});193194 const YOUTUBE_URL = "https://www.youtube.com/watch?v=ku-N-eS1lgM";195196 const prompt = `197 Process the audio file and generate a detailed transcription.198199 Requirements:200 1. Identify distinct speakers (e.g., Speaker 1, Speaker 2).201 2. Provide accurate timestamps for each segment (Format: MM:SS).202 3. Detect the primary language of each segment.203 4. If not English, provide the English translation.204 5. Identify the primary emotion: Happy, Sad, Angry, or Neutral.205 6. Provide a brief summary at the beginning.206 `;207208 const responseSchema = {209 type: "object",210 properties: {211 summary: { type: "string" },212 segments: {213 type: "array",214 items: {215 type: "object",216 properties: {217 speaker: { type: "string" },218 timestamp: { type: "string" },219 content: { type: "string" },220 language: { type: "string" },221 emotion: {222 type: "string",223 enum: ["happy", "sad", "angry", "neutral"]224 }225 },226 required: ["speaker", "timestamp", "content", "emotion"]227 }228 }229 },230 required: ["summary", "segments"]231 };232233 const interaction = await client.interactions.create({234 model: "gemini-3.8-flash",235 input: [236 { type: "video", uri: YOUTUBE_URL, mime_type: "video/mp4" },237 { type: "text", text: prompt }238 ],239 response_format: responseSchema,240 });241242 console.log(JSON.parse(interaction.output_text));243244### Java245246 import com.google.genai.Client;247 import com.google.genai.gaos.models.interactions.Content;248 import com.google.genai.gaos.models.interactions.CreateModelInteraction;249 import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;250 import com.google.genai.gaos.models.interactions.Interaction;251 import com.google.genai.gaos.models.interactions.InteractionsInput;252 import com.google.genai.gaos.models.interactions.Model;253 import com.google.genai.gaos.models.interactions.ResponseFormat;254 import com.google.genai.gaos.models.interactions.TextContent;255 import com.google.genai.gaos.models.interactions.VideoContent;256 import com.google.genai.gaos.models.interactions.VideoContentMimeType;257 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;258 import java.util.Arrays;259 import java.util.List;260 import java.util.HashMap;261 import java.util.Map;262263 Client client = new Client();264265 String youtubeUrl = "https://www.youtube.com/watch?v=ku-N-eS1lgM";266267 String prompt =268 "Process the audio file and generate a detailed transcription.\n\n"269 + "Requirements:\n"270 + "1. Identify distinct speakers (e.g., Speaker 1, Speaker 2).\n"271 + "2. Provide accurate timestamps for each segment (Format: MM:SS).\n"272 + "3. Detect the primary language of each segment.\n"273 + "4. If not English, provide the English translation.\n"274 + "5. Identify the primary emotion: Happy, Sad, Angry, or Neutral.\n"275 + "6. Provide a brief summary at the beginning.";276277 Map<String, Object> emotionProp = new HashMap<>();278 emotionProp.put("type", "string");279 emotionProp.put("enum", Arrays.asList("happy", "sad", "angry", "neutral"));280281 Map<String, Object> stringType = new HashMap<>();282 stringType.put("type", "string");283284 Map<String, Object> segmentProps = new HashMap<>();285 segmentProps.put("speaker", stringType);286 segmentProps.put("timestamp", stringType);287 segmentProps.put("content", stringType);288 segmentProps.put("language", stringType);289 segmentProps.put("emotion", emotionProp);290291 Map<String, Object> segmentItem = new HashMap<>();292 segmentItem.put("type", "object");293 segmentItem.put("properties", segmentProps);294 segmentItem.put("required", Arrays.asList("speaker", "timestamp", "content", "emotion"));295296 Map<String, Object> segmentsProp = new HashMap<>();297 segmentsProp.put("type", "array");298 segmentsProp.put("items", segmentItem);299300 Map<String, Object> properties = new HashMap<>();301 properties.put("summary", stringType);302 properties.put("segments", segmentsProp);303304 Map<String, Object> responseSchema = new HashMap<>();305 responseSchema.put("type", "object");306 responseSchema.put("properties", properties);307 responseSchema.put("required", Arrays.asList("summary", "segments"));308309 Content videoContent =310 VideoContent.builder()311 .uri(youtubeUrl)312 .mimeType(VideoContentMimeType.VIDEO_MP4)313 .build();314 Content textContent = TextContent.builder().text(prompt).build();315316 List<Content> contents = Arrays.asList(videoContent, textContent);317318 CreateModelInteraction params =319 CreateModelInteraction.builder()320 .model(Model.of("gemini-3.8-flash"))321 .input(InteractionsInput.ofContent(contents))322 .responseFormat(323 CreateModelInteractionResponseFormat.of(ResponseFormat.of(responseSchema)))324 .build();325326 Interaction interaction =327 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();328329 System.out.println(interaction.outputText().orElse(""));330331### REST332333 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \334 -H "x-goog-api-key: $GEMINI_API_KEY" \335 -H 'Content-Type: application/json' \336 -d '{337 "model": "gemini-3.8-flash",338 "input": [339 {340 "type": "video",341 "uri": "https://www.youtube.com/watch?v=ku-N-eS1lgM",342 "mime_type": "video/mp4"343 },344 {345 "type": "text",346 "text": "Transcribe with speaker diarization and emotion detection."347 }348 ],349 "response_format": {350 "type": "object",351 "properties": {352 "summary": {"type": "string"},353 "segments": {354 "type": "array",355 "items": {356 "type": "object",357 "properties": {358 "speaker": {"type": "string"},359 "timestamp": {"type": "string"},360 "content": {"type": "string"},361 "emotion": {"type": "string", "enum": ["happy", "sad", "angry", "neutral"]}362 }363 }364 }365 }366 }367 }'368369370371## Input audio372373You can provide audio data in the following ways:374375- [Upload an audio file](https://ai.google.dev/gemini-api/docs/audio#upload-audio) before making a request.376- [Pass inline audio data](https://ai.google.dev/gemini-api/docs/audio#inline-audio) with the request.377378### Upload an audio file379380Use the [Files API](https://ai.google.dev/gemini-api/docs/files) for files larger than 20 MB.381382### Python383384 from google import genai385386 client = genai.Client()387388 uploaded_file = client.files.upload(file="path/to/sample.mp3")389390 interaction = client.interactions.create(391 model="gemini-3.8-flash",392 input=[393 {"type": "text", "text": "Describe this audio clip"},394 {395 "type": "audio",396 "uri": uploaded_file.uri,397 "mime_type": uploaded_file.mime_type398 }399 ]400 )401 print(interaction.output_text)402403### JavaScript404405 import { GoogleGenAI } from "@google/genai";406407 const client = new GoogleGenAI({});408409 const uploadedFile = await client.files.upload({410 file: "path/to/sample.mp3",411 config: { mimeType: "audio/mp3" }412 });413414 const interaction = await client.interactions.create({415 model: "gemini-3.8-flash",416 input: [417 {type: "text", text: "Describe this audio clip"},418 {419 type: "audio",420 uri: uploadedFile.uri,421 mime_type: uploadedFile.mimeType422 }423 ]424 });425 console.log(interaction.output_text);426427### Java428429 // Upload an audio file using the Files API (recommended for files > 20 MB)430 import com.google.genai.Client;431 import com.google.genai.gaos.models.interactions.AudioContent;432 import com.google.genai.gaos.models.interactions.AudioContentMimeType;433 import com.google.genai.gaos.models.interactions.Content;434 import com.google.genai.gaos.models.interactions.CreateModelInteraction;435 import com.google.genai.gaos.models.interactions.Interaction;436 import com.google.genai.gaos.models.interactions.InteractionsInput;437 import com.google.genai.gaos.models.interactions.Model;438 import com.google.genai.gaos.models.interactions.TextContent;439 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;440 import com.google.genai.types.File;441 import com.google.genai.types.UploadFileConfig;442 import java.util.Arrays;443 import java.util.List;444445 Client client = new Client();446447 File uploadedFile =448 client.files.upload(449 "path/to/sample.mp3", UploadFileConfig.builder().mimeType("audio/mp3").build());450451 Content textContent = TextContent.builder().text("Describe this audio clip").build();452 Content audioContent =453 AudioContent.builder()454 .uri(uploadedFile.uri().get())455 .mimeType(AudioContentMimeType.of(uploadedFile.mimeType().get()))456 .build();457458 List<Content> contents = Arrays.asList(textContent, audioContent);459460 CreateModelInteraction params =461 CreateModelInteraction.builder()462 .model(Model.of("gemini-3.8-flash"))463 .input(InteractionsInput.ofContent(contents))464 .build();465466 Interaction interaction =467 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();468469 System.out.println(interaction.outputText().orElse(""));470471### REST472473 # First upload the file using the Files API, then use the URI:474 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \475 -H "x-goog-api-key: $GEMINI_API_KEY" \476 -H 'Content-Type: application/json' \477 -d '{478 "model": "gemini-3.8-flash",479 "input": [480 {"type": "text", "text": "Describe this audio clip"},481 {482 "type": "audio",483 "uri": "YOUR_FILE_URI",484 "mime_type": "audio/mp3"485 }486 ]487 }'488489### Pass audio data inline490491For small audio files under 20MB total request size:492493### Python494495 from google import genai496 import base64497498 client = genai.Client()499500 with open('path/to/small-sample.mp3', 'rb') as f:501 audio_bytes = f.read()502503 interaction = client.interactions.create(504 model="gemini-3.8-flash",505 input=[506 {"type": "text", "text": "Describe this audio clip"},507 {508 "type": "audio",509 "data": base64.b64encode(audio_bytes).decode('utf-8'),510 "mime_type": "audio/mp3"511 }512 ]513 )514 print(interaction.output_text)515516### JavaScript517518 import { GoogleGenAI } from "@google/genai";519 import * as fs from "node:fs";520521 const client = new GoogleGenAI({});522523 const audioData = fs.readFileSync("path/to/small-sample.mp3", {524 encoding: "base64"525 });526527 const interaction = await client.interactions.create({528 model: "gemini-3.8-flash",529 input: [530 {type: "text", text: "Describe this audio clip"},531 {532 type: "audio",533 data: audioData,534 mime_type: "audio/mp3"535 }536 ]537 });538 console.log(interaction.output_text);539540### Java541542 import com.google.genai.Client;543 import com.google.genai.gaos.models.interactions.AudioContent;544 import com.google.genai.gaos.models.interactions.AudioContentMimeType;545 import com.google.genai.gaos.models.interactions.Content;546 import com.google.genai.gaos.models.interactions.CreateModelInteraction;547 import com.google.genai.gaos.models.interactions.Interaction;548 import com.google.genai.gaos.models.interactions.InteractionsInput;549 import com.google.genai.gaos.models.interactions.Model;550 import com.google.genai.gaos.models.interactions.TextContent;551 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;552 import java.nio.file.Files;553 import java.nio.file.Paths;554 import java.util.Arrays;555 import java.util.Base64;556 import java.util.List;557558 Client client = new Client();559560 byte[] audioBytes = Files.readAllBytes(Paths.get("path/to/small-sample.mp3"));561 String base64Audio = Base64.getEncoder().encodeToString(audioBytes);562563 Content textContent = TextContent.builder().text("Describe this audio clip").build();564 Content audioContent =565 AudioContent.builder()566 .data(base64Audio)567 .mimeType(AudioContentMimeType.AUDIO_MP3)568 .build();569570 List<Content> contents = Arrays.asList(textContent, audioContent);571572 CreateModelInteraction params =573 CreateModelInteraction.builder()574 .model(Model.of("gemini-3.8-flash"))575 .input(InteractionsInput.ofContent(contents))576 .build();577578 Interaction interaction =579 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();580581 System.out.println(interaction.outputText().orElse(""));582583### REST584585 AUDIO_PATH="path/to/sample.mp3"586587 if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then588 B64FLAGS="--input"589 else590 B64FLAGS="-w0"591 fi592593 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \594 -H "x-goog-api-key: $GEMINI_API_KEY" \595 -H 'Content-Type: application/json' \596 -d '{597 "model": "gemini-3.8-flash",598 "input": [599 {"type": "text", "text": "Describe this audio clip"},600 {601 "type": "audio",602 "data": "'$(base64 $B64FLAGS $AUDIO_PATH)'",603 "mime_type": "audio/mp3"604 }605 ]606 }'607608Notes on inline audio data:609\* Maximum request size is 20 MB total (including prompts and all files)610\* For reuse, [upload the file](https://ai.google.dev/gemini-api/docs/audio#upload-audio) instead611612## Get a transcript613614To get a transcript, ask for it in the prompt:615616### Python617618 interaction = client.interactions.create(619 model="gemini-3.8-flash",620 input=[621 {"type": "text", "text": "Generate a transcript of the speech."},622 {623 "type": "audio",624 "uri": uploaded_file.uri,625 "mime_type": uploaded_file.mime_type626 }627 ]628 )629 print(interaction.output_text)630631### JavaScript632633 const interaction = await client.interactions.create({634 model: "gemini-3.8-flash",635 input: [636 { type: "text", text: "Generate a transcript of the speech." },637 {638 type: "audio",639 uri: uploadedFile.uri,640 mime_type: uploadedFile.mimeType641 }642 ]643 });644 console.log(interaction.output_text);645646### Java647648 import com.google.genai.Client;649 import com.google.genai.gaos.models.interactions.AudioContent;650 import com.google.genai.gaos.models.interactions.AudioContentMimeType;651 import com.google.genai.gaos.models.interactions.Content;652 import com.google.genai.gaos.models.interactions.CreateModelInteraction;653 import com.google.genai.gaos.models.interactions.Interaction;654 import com.google.genai.gaos.models.interactions.InteractionsInput;655 import com.google.genai.gaos.models.interactions.Model;656 import com.google.genai.gaos.models.interactions.TextContent;657 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;658 import com.google.genai.types.File;659 import com.google.genai.types.UploadFileConfig;660 import java.util.Arrays;661 import java.util.List;662663 Client client = new Client();664665 File uploadedFile =666 client.files.upload(667 "path/to/sample.mp3", UploadFileConfig.builder().mimeType("audio/mp3").build());668669 Content textContent = TextContent.builder().text("Generate a transcript of the speech.").build();670 Content audioContent =671 AudioContent.builder()672 .uri(uploadedFile.uri().get())673 .mimeType(AudioContentMimeType.of(uploadedFile.mimeType().get()))674 .build();675676 List<Content> contents = Arrays.asList(textContent, audioContent);677678 CreateModelInteraction params =679 CreateModelInteraction.builder()680 .model(Model.of("gemini-3.8-flash"))681 .input(InteractionsInput.ofContent(contents))682 .build();683684 Interaction interaction =685 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();686687 System.out.println(interaction.outputText().orElse(""));688689## Refer to timestamps690691Use `MM:SS` format to reference specific sections:692693### Python694695 interaction = client.interactions.create(696 model="gemini-3.8-flash",697 input=[698 {"type": "text", "text": "Provide a transcript from 02:30 to 03:29."},699 {700 "type": "audio",701 "uri": uploaded_file.uri,702 "mime_type": uploaded_file.mime_type703 }704 ]705 )706707### JavaScript708709 const interaction = await client.interactions.create({710 model: "gemini-3.8-flash",711 input: [712 { type: "text", text: "Provide a transcript from 02:30 to 03:29." },713 { type: "audio", uri: uploadedFile.uri, mime_type: "audio/mp3" }714 ]715 });716717### Java718719 import com.google.genai.Client;720 import com.google.genai.gaos.models.interactions.AudioContent;721 import com.google.genai.gaos.models.interactions.AudioContentMimeType;722 import com.google.genai.gaos.models.interactions.Content;723 import com.google.genai.gaos.models.interactions.CreateModelInteraction;724 import com.google.genai.gaos.models.interactions.Interaction;725 import com.google.genai.gaos.models.interactions.InteractionsInput;726 import com.google.genai.gaos.models.interactions.Model;727 import com.google.genai.gaos.models.interactions.TextContent;728 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;729 import com.google.genai.types.File;730 import com.google.genai.types.UploadFileConfig;731 import java.util.Arrays;732 import java.util.List;733734 Client client = new Client();735736 File uploadedFile =737 client.files.upload(738 "path/to/sample.mp3", UploadFileConfig.builder().mimeType("audio/mp3").build());739740 Content textContent =741 TextContent.builder().text("Provide a transcript from 02:30 to 03:29.").build();742 Content audioContent =743 AudioContent.builder()744 .uri(uploadedFile.uri().get())745 .mimeType(AudioContentMimeType.of(uploadedFile.mimeType().get()))746 .build();747748 List<Content> contents = Arrays.asList(textContent, audioContent);749750 CreateModelInteraction params =751 CreateModelInteraction.builder()752 .model(Model.of("gemini-3.8-flash"))753 .input(InteractionsInput.ofContent(contents))754 .build();755756 Interaction interaction =757 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();758759 System.out.println(interaction.outputText().orElse(""));760761## Count tokens762763Count tokens in an audio file:764765### Python766767 response = client.models.count_tokens(768 model="gemini-3.8-flash",769 contents=[uploaded_file]770 )771 print(response)772773### JavaScript774775 const response = await client.models.countTokens({776 model: "gemini-3.8-flash",777 contents: [778 { fileData: { fileUri: uploadedFile.uri, mimeType: uploadedFile.mimeType } }779 ]780 });781 console.log(response.totalTokens);782783### Java784785 import com.google.genai.Client;786 import com.google.genai.types.Content;787 import com.google.genai.types.CountTokensResponse;788 import com.google.genai.types.File;789 import com.google.genai.types.Part;790 import com.google.genai.types.UploadFileConfig;791 import java.util.Arrays;792793 Client client = new Client();794795 File uploadedFile =796 client.files.upload(797 "path/to/sample.mp3", UploadFileConfig.builder().mimeType("audio/mp3").build());798799 CountTokensResponse response =800 client.models.countTokens(801 "gemini-3.8-flash",802 Arrays.asList(803 Content.fromParts(804 Part.fromUri(uploadedFile.uri().get(), uploadedFile.mimeType().get()))),805 null);806807 System.out.println(response);808809## Supported audio formats810811Gemini supports the following audio format MIME types:812813- WAV - `audio/wav`814- MP3 - `audio/mp3`815- AIFF - `audio/aiff`816- AAC - `audio/aac`817- OGG - `audio/ogg`818- FLAC - `audio/flac`819- MPEG - `audio/mpeg`820- M4A - `audio/m4a`821- L16 - `audio/l16`822- Opus - `audio/opus`823- ALAW - `audio/alaw`824- MULAW - `audio/mulaw`825- WebM - `audio/webm`826827For the complete list of supported MIME types and parameter schemas, see the [Interactions API reference](https://ai.google.dev/api/interactions-api#Resource:Content).828829## Technical details about audio830831- **Tokens**: 32 tokens per second of audio (1 minute = 1,920 tokens)832- **Non-speech**: Gemini understands non-speech sounds (birdsong, sirens, etc.)833- **Max length**: 9.5 hours of audio per prompt834- **Resolution**: Downsampled to 16 Kbps835- **Channels**: Multi-channel audio combined to single channel836837## What's next838839- [Files API](https://ai.google.dev/gemini-api/docs/files): Upload and manage audio files840- [System instructions](https://ai.google.dev/gemini-api/docs/text-generation#system-instructions): Customize model behavior841- [Structured output](https://ai.google.dev/gemini-api/docs/structured-output): Get transcription results in JSON format