Repository Public
1The Gemini API converts speech in audio files into text using the Gemini 3.5 Transcribe model (`gemini-3.5-transcribe`). Based on Gemini's audio understanding capabilities, it delivers accurate transcription with automatic language identification, speaker diarization, word-level timestamps, and custom vocabulary hints. It also provides a [smart transcription](https://ai.google.dev/gemini-api/docs/transcribe#transcription-modes) mode featuring disfluency removal and smart formatting.23To transcribe an audio file, upload the audio and pass it to `gemini-3.5-transcribe`:45### Python67 from google import genai89 client = genai.Client()1011 audio_file = client.files.upload(file="path/to/sample.mp3")1213 interaction = client.interactions.create(14 model="gemini-3.5-transcribe",15 input=[16 {17 "type": "audio",18 "uri": audio_file.uri,19 "mime_type": audio_file.mime_type,20 }21 ],22 )2324 print(interaction.output_text)2526### JavaScript2728 import { GoogleGenAI } from "@google/genai";2930 const client = new GoogleGenAI({});3132 const audioFile = await client.files.upload({33 file: "path/to/sample.mp3",34 config: { mime_type: "audio/mp3" },35 });3637 const interaction = await client.interactions.create({38 model: "gemini-3.5-transcribe",39 input: [40 {41 type: "audio",42 uri: audioFile.uri,43 mime_type: audioFile.mimeType,44 },45 ],46 });4748 console.log(interaction.output_text);4950### REST5152 # First upload the file via the Files API, then pass its URI:53 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \54 -H "x-goog-api-key: $GEMINI_API_KEY" \55 -H "Content-Type: application/json" \56 -d '{57 "model": "gemini-3.5-transcribe",58 "input": [59 {60 "type": "audio",61 "uri": "YOUR_FILE_URI",62 "mime_type": "audio/mp3"63 }64 ]65 }'6667> [!NOTE]68> **Note:** For real-time, low-latency streaming speech recognition from a microphone or live audio stream, see [Live transcription](https://ai.google.dev/gemini-api/docs/live-api/live-transcribe) using the Live API and `gemini-3.5-transcribe-live`.6970## Overview7172Gemini 3.5 Transcribe is optimized for speech-to-text tasks. It handles diverse accents, background noise, and multi-language conversations.7374Key capabilities include:7576- **Automatic speech recognition (ASR):** Automatically detects languages across [85+ locales](https://ai.google.dev/gemini-api/docs/transcribe#supported-languages). Handles intra-sentence and inter-sentential code-switching without manual configuration.77- **Custom vocabulary:** Biases recognition toward domain-specific terms, acronyms, and proper names by passing up to 1,000 phrases.78- **Speaker diarization:** Distinguishes between multiple speakers and attributes spoken segments to distinct labels.79- **Word-level timestamps:** Generates precise start and end time offsets for each recognized word.80- **Smart transcription:** Cleans up disfluencies, filler words, repetitions, and applies structured formatting.81- **Formatting and normalization:** Applies capitalization, punctuation, and inverse text normalization, such as converting "twenty six million dollars" to "$26M".8283For general audio reasoning or question answering over audio content, use [Audio understanding](https://ai.google.dev/gemini-api/docs/audio). For text-to-speech audio synthesis, use [Text-to-speech](https://ai.google.dev/gemini-api/docs/speech-generation).8485## Language detection and hints8687By default, the model detects the spoken language automatically. It switches between languages dynamically when speakers code-switch.8889To use automatic detection, omit `language_codes` or provide an empty list:9091### Python9293 interaction = client.interactions.create(94 model="gemini-3.5-transcribe",95 input=[96 {97 "type": "audio",98 "uri": audio_file.uri,99 "mime_type": audio_file.mime_type,100 }101 ],102 generation_config={103 "transcription_config": {104 "language_codes": [],105 }106 },107 )108109### JavaScript110111 const interaction = await client.interactions.create({112 model: "gemini-3.5-transcribe",113 input: [114 {115 type: "audio",116 uri: audioFile.uri,117 mime_type: audioFile.mimeType,118 },119 ],120 generation_config: {121 transcription_config: {122 language_codes: [],123 },124 },125 });126127### REST128129 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \130 -H "x-goog-api-key: $GEMINI_API_KEY" \131 -H "Content-Type: application/json" \132 -d '{133 "model": "gemini-3.5-transcribe",134 "input": [135 {136 "type": "audio",137 "uri": "YOUR_FILE_URI",138 "mime_type": "audio/mp3"139 }140 ],141 "generation_config": {142 "transcription_config": {143 "language_codes": []144 }145 }146 }'147148If you know the language in advance, specify BCP-47 language codes in `language_codes` to improve transcription accuracy (see [Supported languages](https://ai.google.dev/gemini-api/docs/transcribe#supported-languages)):149150### Python151152 generation_config = {153 "transcription_config": {154 "language_codes": ["es-ES"],155 }156 }157158### JavaScript159160 const generationConfig = {161 transcription_config: {162 language_codes: ["es-ES"],163 },164 };165166### REST167168 {169 "generation_config": {170 "transcription_config": {171 "language_codes": ["es-ES"]172 }173 }174 }175176## Custom vocabulary177178You can steer the speech model toward uncommon words, technical jargon, brand names, or proper nouns. Supply up to 1,000 terms in the `custom_vocabulary` array (best results are typically achieved with up to 100 terms):179180> [!NOTE]181> **Note:** You cannot combine `custom_vocabulary` with speaker diarization or word-level timestamps. The API rejects requests that include `custom_vocabulary` with either feature.182183### Python184185 interaction = client.interactions.create(186 model="gemini-3.5-transcribe",187 input=[188 {189 "type": "audio",190 "uri": audio_file.uri,191 "mime_type": audio_file.mime_type,192 }193 ],194 generation_config={195 "transcription_config": {196 "custom_vocabulary": ["Gemini", "Kubernetes", "BigQuery"],197 }198 },199 )200201### JavaScript202203 const interaction = await client.interactions.create({204 model: "gemini-3.5-transcribe",205 input: [206 {207 type: "audio",208 uri: audioFile.uri,209 mime_type: audioFile.mimeType,210 },211 ],212 generation_config: {213 transcription_config: {214 custom_vocabulary: ["Gemini", "Kubernetes", "BigQuery"],215 },216 },217 });218219### REST220221 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \222 -H "x-goog-api-key: $GEMINI_API_KEY" \223 -H "Content-Type: application/json" \224 -d '{225 "model": "gemini-3.5-transcribe",226 "input": [227 {228 "type": "audio",229 "uri": "YOUR_FILE_URI",230 "mime_type": "audio/mp3"231 }232 ],233 "generation_config": {234 "transcription_config": {235 "custom_vocabulary": ["Gemini", "Kubernetes", "BigQuery"]236 }237 }238 }'239240## Speaker diarization241242Speaker diarization identifies different voices in the recording and tags each segment with a speaker identifier like `spk_1` or `spk_2`. Up to 8 speakers are supported (attribution for 3 or more speakers is experimental).243244Enable diarization by configuring `diarization_mode` within `mode`:245246### Python247248 interaction = client.interactions.create(249 model="gemini-3.5-transcribe",250 input=[251 {252 "type": "audio",253 "uri": audio_file.uri,254 "mime_type": audio_file.mime_type,255 }256 ],257 generation_config={258 "transcription_config": {259 "mode": {260 "type": "verbatim",261 "diarization_mode": "speaker",262 },263 }264 },265 )266267### JavaScript268269 const interaction = await client.interactions.create({270 model: "gemini-3.5-transcribe",271 input: [272 {273 type: "audio",274 uri: audioFile.uri,275 mime_type: audioFile.mimeType,276 },277 ],278 generation_config: {279 transcription_config: {280 mode: {281 type: "verbatim",282 diarization_mode: "speaker",283 },284 },285 },286 });287288### REST289290 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \291 -H "x-goog-api-key: $GEMINI_API_KEY" \292 -H "Content-Type: application/json" \293 -d '{294 "model": "gemini-3.5-transcribe",295 "input": [296 {297 "type": "audio",298 "uri": "YOUR_FILE_URI",299 "mime_type": "audio/mp3"300 }301 ],302 "generation_config": {303 "transcription_config": {304 "mode": {305 "type": "verbatim",306 "diarization_mode": "speaker"307 }308 }309 }310 }'311312## Word-level timestamps313314Word-level timestamps provide exact start and end offsets for every recognized word in the audio stream.315316> [!NOTE]317> **Note:** Enabling word-level timestamps may degrade overall transcription accuracy.318319Enable timestamps by configuring `timestamp_granularities` within `mode`:320321### Python322323 interaction = client.interactions.create(324 model="gemini-3.5-transcribe",325 input=[326 {327 "type": "audio",328 "uri": audio_file.uri,329 "mime_type": audio_file.mime_type,330 }331 ],332 generation_config={333 "transcription_config": {334 "mode": {335 "type": "verbatim",336 "timestamp_granularities": ["word"],337 },338 }339 },340 )341342### JavaScript343344 const interaction = await client.interactions.create({345 model: "gemini-3.5-transcribe",346 input: [347 {348 type: "audio",349 uri: audioFile.uri,350 mime_type: audioFile.mimeType,351 },352 ],353 generation_config: {354 transcription_config: {355 mode: {356 type: "verbatim",357 timestamp_granularities: ["word"],358 },359 },360 },361 });362363### REST364365 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \366 -H "x-goog-api-key: $GEMINI_API_KEY" \367 -H "Content-Type: application/json" \368 -d '{369 "model": "gemini-3.5-transcribe",370 "input": [371 {372 "type": "audio",373 "uri": "YOUR_FILE_URI",374 "mime_type": "audio/mp3"375 }376 ],377 "generation_config": {378 "transcription_config": {379 "mode": {380 "type": "verbatim",381 "timestamp_granularities": ["word"]382 }383 }384 }385 }'386387You can combine `diarization_mode` and `timestamp_granularities` in `mode` to receive both speaker labels and word timestamps:388389### Python390391 generation_config = {392 "transcription_config": {393 "mode": {394 "type": "verbatim",395 "diarization_mode": "speaker",396 "timestamp_granularities": ["word"],397 },398 }399 }400401### JavaScript402403 const generationConfig = {404 transcription_config: {405 mode: {406 type: "verbatim",407 diarization_mode: "speaker",408 timestamp_granularities: ["word"],409 },410 },411 };412413### REST414415 {416 "generation_config": {417 "transcription_config": {418 "mode": {419 "type": "verbatim",420 "diarization_mode": "speaker",421 "timestamp_granularities": ["word"]422 }423 }424 }425 }426427## Transcription modes428429Gemini 3.5 Transcribe supports two transcription modes via the `mode` parameter:430431- **`verbatim` (default)** : Returns an exact word-for-word transcript of everything spoken, preserving raw filler words ("um", "uh", "like", "you know"), repetitions, pauses, and false starts. Timestamps and speaker diarization are configured within this mode (`{"type": "verbatim", ...}`).432- **`smart` (Smart transcription)** : Optimizes the transcript for reading by applying intelligent post-processing:433 - **Disfluency removal**: Strips conversational filler words, stuttering, and false starts.434 - **Inline self-corrections** : Resolves spoken corrections directly (for example, *"Let's meet on Tuesday, actually no, Wednesday at two"* becomes *"Let's meet on Wednesday at 2:00 PM"*).435 - **Automatic structured formatting**: Automatically structures spoken thoughts into paragraphs, numbered lists, bullet points, formatted dates, currencies, and numbers.436 - **Grammatical cleanup**: Applies natural punctuation, sentence casing, and flow.437438| Spoken audio | `verbatim` output | `smart` (Smart transcription) output |439|---|---|---|440| "Um, so for the meeting, I think we should, uh, invite Alice and, wait no, Bob and Carol." | "Um so for the meeting I think we should uh invite Alice and wait no Bob and Carol." | "For the meeting, I think we should invite Bob and Carol." |441| "First item review budget second item finalize timeline third item send recap" | "first item review budget second item finalize timeline third item send recap" | "1. Review budget 2. Finalize timeline 3. Send recap" |442443### Python444445 interaction = client.interactions.create(446 model="gemini-3.5-transcribe",447 input=[448 {449 "type": "audio",450 "uri": audio_file.uri,451 "mime_type": audio_file.mime_type,452 }453 ],454 generation_config={455 "transcription_config": {456 "mode": "smart",457 }458 },459 )460 print(interaction.output_text)461462### JavaScript463464 const interaction = await client.interactions.create({465 model: "gemini-3.5-transcribe",466 input: [467 {468 type: "audio",469 uri: audioFile.uri,470 mime_type: audioFile.mimeType,471 },472 ],473 generation_config: {474 transcription_config: {475 mode: "smart",476 },477 },478 });479 console.log(interaction.output_text);480481### REST482483 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \484 -H "x-goog-api-key: $GEMINI_API_KEY" \485 -H "Content-Type: application/json" \486 -d '{487 "model": "gemini-3.5-transcribe",488 "input": [489 {490 "type": "audio",491 "uri": "YOUR_FILE_URI",492 "mime_type": "audio/mp3"493 }494 ],495 "generation_config": {496 "transcription_config": {497 "mode": "smart"498 }499 }500 }'501502> [!NOTE]503> **Note:** Smart transcription (`"smart"`) is incompatible with `timestamp_granularities` and `diarization_mode`. If you need word timestamps or speaker diarization, configure `mode` with `{"type": "verbatim", ...}`.504505## Parsing transcription output506507The complete transcript text is returned in `interaction.output_text`.508509When `timestamp_granularities` or `diarization_mode` is enabled, the API also returns detailed word-level annotations attached to the interaction content.510511Here is how to extract and iterate over word timestamps and speaker turns:512513### Python514515 def extract_word_annotations(interaction):516 words = []517 for step in getattr(interaction, "steps", []) or []:518 for content in getattr(step, "content", []) or []:519 for annotation in getattr(content, "annotations", []) or []:520 if getattr(annotation, "type", None) == "word_info":521 words.append(annotation)522 return words523524 words = extract_word_annotations(interaction)525526 for w in words:527 speaker = f"[{w.speaker}] " if getattr(w, "speaker", None) else ""528 start = getattr(w, "start_offset", "")529 end = getattr(w, "end_offset", "")530 timing = f"({start} -> {end}) " if start and end else ""531 print(f"{speaker}{timing}{w.text}")532533### JavaScript534535 function extractWordAnnotations(interaction) {536 const words = [];537 for (const step of interaction.steps ?? []) {538 for (const content of step.content ?? []) {539 for (const annotation of content.annotations ?? []) {540 if (annotation.type === "word_info") {541 words.push(annotation);542 }543 }544 }545 }546 return words;547 }548549 const words = extractWordAnnotations(interaction);550551 for (const w of words) {552 const speaker = w.speaker ? `[${w.speaker}] ` : "";553 const timing = (w.start_offset && w.end_offset) ? `(${w.start_offset} -> ${w.end_offset}) ` : "";554 console.log(`${speaker}${timing}${w.text}`);555 }556557### REST558559 {560 "id": "interactions/abc123xyz",561 "status": "completed",562 "steps": [563 {564 "id": "step_001",565 "type": "model_output",566 "content": [567 {568 "type": "text",569 "text": "Hello world",570 "annotations": [571 {572 "type": "word_info",573 "text": "Hello",574 "speaker": "spk_1",575 "start_offset": "0.100s",576 "end_offset": "0.450s"577 },578 {579 "type": "word_info",580 "text": "world",581 "speaker": "spk_1",582 "start_offset": "0.500s",583 "end_offset": "0.850s"584 }585 ]586 }587 ]588 }589 ]590 }591592## Supported languages593594The following languages and BCP-47 language codes are supported for Gemini 3.5 Transcribe:595596| Language | BCP-47 Code | Language | BCP-47 Code |597|---|---|---|---|598| Afrikaans | `af-ZA` | Japanese | `ja-JP` |599| Amharic | `am-ET` | Javanese | `jv-ID` |600| Arabic (Egypt) | `ar-EG` | Kabuverdianu | `kea-CV` |601| Armenian | `hy-AM` | Kannada | `kn-IN` |602| Assamese | `as-IN` | Kazakh | `kk-KZ` |603| Azerbaijani | `az-AZ` | Korean | `ko-KR` |604| Belarusian | `be-BY` | Kyrgyz | `ky-KG` |605| Bengali (Bangladesh) | `bn-BD` | Latvian | `lv-LV` |606| Bengali (India) | `bn-IN` | Lingala | `ln-CD` |607| Bosnian | `bs-BA` | Lithuanian | `lt-LT` |608| Bulgarian | `bg-BG` | Macedonian | `mk-MK` |609| Bulgarian (Aromanian) | `rup-BG` | Malay | `ms-MY` |610| Burmese | `my-MM` | Malayalam | `ml-IN` |611| Cantonese (Traditional) | `yue-Hant-HK` | Maltese | `mt-MT` |612| Catalan | `ca-ES` | Mandarin Chinese (Simplified) | `cmn-Hans-CN` |613| Cebuano | `ceb` | Marathi | `mr-IN` |614| Central Khmer | `km-KH` | Mongolian | `mn-MN` |615| Croatian | `hr-HR` | Nepali | `ne-NP` |616| Czech | `cs-CZ` | Norwegian | `nb-NO` |617| Danish | `da-DK` | Oriya | `or-IN` |618| Dutch | `nl-NL` | Polish | `pl-PL` |619| English (Great Britain) | `en-GB` | Portuguese (Brazil) | `pt-BR` |620| English (India) | `en-IN` | Portuguese (Portugal) | `pt-PT` |621| English (United States) | `en-US` | Punjabi | `pa-IN` |622| Estonian | `et-EE` | Punjabi (Gurmukhi script) | `pa-Guru-IN` |623| Farsi | `fa-IR` | Romanian | `ro-RO` |624| Filipino | `fil-PH` | Russian | `ru-RU` |625| Finnish | `fi-FI` | Serbian | `sr-RS` |626| French | `fr-FR` | Sindhi (Arabic script) | `sd-Arab-IN` |627| Galician | `gl-ES` | Slovak | `sk-SK` |628| Georgian | `ka-GE` | Slovenian | `sl-SI` |629| German | `de-DE` | Spanish (Latin America) | `es-419` |630| Greek | `el-GR` | Spanish (United States) | `es-US` |631| Gujarati | `gu-IN` | Swahili (Kenya) | `sw-KE` |632| Hausa | `ha-NG` | Swedish | `sv-SE` |633| Hebrew | `he-IL` | Tajik | `tg-TJ` |634| Hindi | `hi-IN` | Telugu | `te-IN` |635| Hungarian | `hu-HU` | Thai | `th-TH` |636| Icelandic | `is-IS` | Turkish | `tr-TR` |637| Indian English | `en-IN` | Ukrainian | `uk-UA` |638| Indonesian | `id-ID` | Uzbek | `uz-UZ` |639| Italian | `it-IT` | Vietnamese | `vi-VN` |640641## Supported audio formats642643Gemini 3.5 Transcribe supports the following audio format MIME types:644645- WAV - `audio/wav`646- MP3 - `audio/mp3`647- AIFF - `audio/aiff`648- AAC - `audio/aac`649- OGG - `audio/ogg`650- FLAC - `audio/flac`651- MPEG - `audio/mpeg`652- M4A - `audio/m4a`653- L16 - `audio/l16`654- Opus - `audio/opus`655- ALAW - `audio/alaw`656- MULAW - `audio/mulaw`657- WebM - `audio/webm`658659For the complete list of supported MIME types and parameter schemas, see the [Interactions API reference](https://ai.google.dev/api/interactions-api#Resource:Content).660661## Parameter reference662663Configure transcription by setting fields within the `transcription_config` object in `generation_config`:664665| Field | Type | Description |666|---|---|---|667| `language_codes` | Array of strings | BCP-47 language codes (e.g., `["en-US"]`). If omitted or empty (`[]`), the model automatically detects the language and handles code-switching. |668| `custom_vocabulary` | Array of strings | Up to 1,000 custom terms, acronyms, or proper names to bias speech recognition. Incompatible with speaker diarization and word-level timestamps. |669| `mode` | Object or String | Transcription mode configuration. Accepts `"smart"` or a verbatim mode object (`{"type": "verbatim", ...}`). Defaults to verbatim transcription. |670| `mode.type` | String | *(Verbatim mode only)* Mode identifier. Always set to `"verbatim"`. |671| `mode.timestamp_granularities` | Array of strings | *(Verbatim mode only)* Granularity of timestamps to return. Pass `["word"]` to enable word start and end offsets. Incompatible with custom vocabulary. |672| `mode.diarization_mode` | String | *(Verbatim mode only)* Diarization mode. Pass `"speaker"` to identify and label distinct speakers. Incompatible with custom vocabulary. |673674## Best practices675676- **Provide clean audio:** Ensure audio recordings have clear voice separation and avoid severe clipping.677- **Provide language hints when known:** If you know the audio language in advance, specify `language_codes` to maximize accuracy.678- **Target custom vocabulary:** Include only distinct domain terms, brand names, or proper nouns in `custom_vocabulary` rather than common everyday words.679- **Use the Files API for large recordings:** For files longer than a few seconds, upload the file using `client.files.upload` and pass the returned file URI to the model.680681## Limitations682683- **Audio duration:** Standard unary requests support audio files up to 1 hour. Audio processing is limited to 30 minutes when features like speaker diarization or word-level timestamps are enabled.684- **Word-level timestamps:** Enabling word-level timestamps may degrade overall transcription accuracy.685- **Speaker diarization:** Speaker diarization supports up to 8 speakers. Speaker attribution for 3 or more speakers is experimental.686- **Custom vocabulary:** You can provide up to 1,000 terms in `custom_vocabulary`, but best results are typically achieved with up to 100 terms. You cannot combine `custom_vocabulary` with speaker diarization or word-level timestamps; the API rejects requests that specify `custom_vocabulary` alongside either feature.687- **Mode compatibility:** Smart transcription (`"smart"`) cannot be combined with `timestamp_granularities` or `diarization_mode`.688689## What's next690691- Stream real-time audio with the [Live transcription guide](https://ai.google.dev/gemini-api/docs/live-api/live-transcribe) using the Live API.692- Explore [Audio understanding](https://ai.google.dev/gemini-api/docs/audio) to analyze, summarize, or query audio content.693- Learn how to synthesize audio from text using [Text-to-speech](https://ai.google.dev/gemini-api/docs/speech-generation).694- Check the [Pricing page](https://ai.google.dev/gemini-api/docs/pricing#gemini-3.5-transcribe) for model pricing and token limits.695- Check the [Files API](https://ai.google.dev/gemini-api/docs/files) guide for details on uploading and managing media files.