Repository Public
1The Gemini API converts speech in audio files into text using the Gemini 3.5 Transcribe model (`gemini-3.5-transcribe`). Based on Gemini's audio understanding capabilities, it delivers accurate transcription with automatic language identification, speaker diarization, word-level timestamps, and custom vocabulary hints. It also provides a [smart transcription](https://ai.google.dev/gemini-api/docs/generate-content/transcribe#transcription-modes) mode featuring disfluency removal and smart formatting.23To transcribe an audio file, upload the audio and pass it to `gemini-3.5-transcribe`:45### Python67 from google import genai89 client = genai.Client()1011 audio_file = client.files.upload(file="path/to/sample.mp3")1213 response = client.models.generate_content(14 model="gemini-3.5-transcribe",15 contents=[audio_file],16 )1718 print(response.text)1920### JavaScript2122 import { GoogleGenAI } from "@google/genai";2324 const ai = new GoogleGenAI({});2526 const audioFile = await ai.files.upload({27 file: "path/to/sample.mp3",28 mimeType: "audio/mp3",29 });3031 const response = await ai.models.generateContent({32 model: "gemini-3.5-transcribe",33 contents: [audioFile],34 });3536 console.log(response.text);3738### REST3940 # First upload the file via the Files API, then pass its URI:41 curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \42 -H "x-goog-api-key: $GEMINI_API_KEY" \43 -H "Content-Type: application/json" \44 -d '{45 "contents": [46 {47 "parts": [48 {49 "fileData": {50 "fileUri": "YOUR_FILE_URI",51 "mimeType": "audio/mp3"52 }53 }54 ]55 }56 ]57 }'5859> [!NOTE]60> **Note:** For real-time, low-latency streaming speech recognition from a microphone or live audio stream, see [Live transcription](https://ai.google.dev/gemini-api/docs/live-api/live-transcribe) using the Live API and `gemini-3.5-transcribe-live`.6162## Overview6364Gemini 3.5 Transcribe is optimized for speech-to-text tasks. It handles diverse accents, background noise, and multi-language conversations.6566Key capabilities include:6768- **Automatic speech recognition (ASR):** Automatically detects languages across [85+ locales](https://ai.google.dev/gemini-api/docs/generate-content/transcribe#supported-languages). Handles intra-sentence and inter-sentential code-switching without manual configuration.69- **Custom vocabulary:** Biases recognition toward domain-specific terms, acronyms, and proper names by passing up to 1,000 phrases.70- **Speaker diarization:** Distinguishes between multiple speakers and attributes spoken segments to distinct labels.71- **Word-level timestamps:** Generates precise start and end time offsets for each recognized word.72- **Smart transcription:** Cleans up disfluencies, filler words, repetitions, and applies structured formatting.73- **Formatting and normalization:** Applies capitalization, punctuation, and inverse text normalization, such as converting "twenty six million dollars" to "$26M".7475For general audio reasoning or question answering over audio content, use [Audio understanding](https://ai.google.dev/gemini-api/docs/generate-content/audio). For text-to-speech audio synthesis, use [Text-to-speech](https://ai.google.dev/gemini-api/docs/generate-content/speech-generation).7677## Language detection and hints7879By default, the model detects the spoken language automatically. It switches between languages dynamically when speakers code-switch.8081To use automatic detection, omit `language_codes` or provide an empty list:8283### Python8485 from google import genai86 from google.genai import types8788 client = genai.Client()8990 response = client.models.generate_content(91 model="gemini-3.5-transcribe",92 contents=[audio_file],93 config=types.GenerateContentConfig(94 audio_transcription_config=types.AudioTranscriptionConfig(95 language_codes=[],96 )97 ),98 )99100### JavaScript101102 const response = await ai.models.generateContent({103 model: "gemini-3.5-transcribe",104 contents: [audioFile],105 config: {106 audioTranscriptionConfig: {107 languageCodes: [],108 },109 },110 });111112### REST113114 curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \115 -H "x-goog-api-key: $GEMINI_API_KEY" \116 -H "Content-Type: application/json" \117 -d '{118 "contents": [119 {120 "parts": [121 {122 "fileData": {123 "fileUri": "YOUR_FILE_URI",124 "mimeType": "audio/mp3"125 }126 }127 ]128 }129 ],130 "generationConfig": {131 "audioTranscriptionConfig": {132 "languageCodes": []133 }134 }135 }'136137If you know the language in advance, specify BCP-47 language codes in `language_codes` to improve transcription accuracy (see [Supported languages](https://ai.google.dev/gemini-api/docs/generate-content/transcribe#supported-languages)):138139### Python140141 config = types.GenerateContentConfig(142 audio_transcription_config=types.AudioTranscriptionConfig(143 language_codes=["es-ES"],144 )145 )146147### JavaScript148149 const config = {150 audioTranscriptionConfig: {151 languageCodes: ["es-ES"],152 },153 };154155### REST156157 {158 "generationConfig": {159 "audioTranscriptionConfig": {160 "languageCodes": ["es-ES"]161 }162 }163 }164165## Custom vocabulary166167You can steer the speech model toward uncommon words, technical jargon, brand names, or proper nouns. Supply up to 1,000 terms in the `custom_vocabulary` array (best results are typically achieved with up to 100 terms):168169> [!NOTE]170> **Note:** You cannot combine `custom_vocabulary` with speaker diarization or word-level timestamps. The API rejects requests that include `custom_vocabulary` with either feature.171172### Python173174 from google import genai175 from google.genai import types176177 client = genai.Client()178179 response = client.models.generate_content(180 model="gemini-3.5-transcribe",181 contents=[audio_file],182 config=types.GenerateContentConfig(183 audio_transcription_config=types.AudioTranscriptionConfig(184 custom_vocabulary=["Gemini", "Kubernetes", "BigQuery"],185 )186 ),187 )188189### JavaScript190191 const response = await ai.models.generateContent({192 model: "gemini-3.5-transcribe",193 contents: [audioFile],194 config: {195 audioTranscriptionConfig: {196 customVocabulary: ["Gemini", "Kubernetes", "BigQuery"],197 },198 },199 });200201### REST202203 curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \204 -H "x-goog-api-key: $GEMINI_API_KEY" \205 -H "Content-Type: application/json" \206 -d '{207 "contents": [208 {209 "parts": [210 {211 "fileData": {212 "fileUri": "YOUR_FILE_URI",213 "mimeType": "audio/mp3"214 }215 }216 ]217 }218 ],219 "generationConfig": {220 "audioTranscriptionConfig": {221 "customVocabulary": ["Gemini", "Kubernetes", "BigQuery"]222 }223 }224 }'225226## Speaker diarization227228Speaker diarization identifies different voices in the recording and tags each segment with a speaker identifier like `spk_1` or `spk_2`. Up to 8 speakers are supported (attribution for 3 or more speakers is experimental).229230Enable diarization by setting `diarization` to `True`:231232### Python233234 from google import genai235 from google.genai import types236237 client = genai.Client()238239 response = client.models.generate_content(240 model="gemini-3.5-transcribe",241 contents=[audio_file],242 config=types.GenerateContentConfig(243 audio_transcription_config=types.AudioTranscriptionConfig(244 diarization=True,245 )246 ),247 )248249### JavaScript250251 const response = await ai.models.generateContent({252 model: "gemini-3.5-transcribe",253 contents: [audioFile],254 config: {255 audioTranscriptionConfig: {256 diarization: true,257 },258 },259 });260261### REST262263 curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \264 -H "x-goog-api-key: $GEMINI_API_KEY" \265 -H "Content-Type: application/json" \266 -d '{267 "contents": [268 {269 "parts": [270 {271 "fileData": {272 "fileUri": "YOUR_FILE_URI",273 "mimeType": "audio/mp3"274 }275 }276 ]277 }278 ],279 "generationConfig": {280 "audioTranscriptionConfig": {281 "diarization": true282 }283 }284 }'285286## Word-level timestamps287288Word-level timestamps provide exact start and end offsets for every recognized word in the audio stream.289290> [!NOTE]291> **Note:** Enabling word-level timestamps may degrade overall transcription accuracy.292293Enable timestamps by setting `word_timestamp` to `True`:294295### Python296297 from google import genai298 from google.genai import types299300 client = genai.Client()301302 response = client.models.generate_content(303 model="gemini-3.5-transcribe",304 contents=[audio_file],305 config=types.GenerateContentConfig(306 audio_transcription_config=types.AudioTranscriptionConfig(307 word_timestamp=True,308 )309 ),310 )311312### JavaScript313314 const response = await ai.models.generateContent({315 model: "gemini-3.5-transcribe",316 contents: [audioFile],317 config: {318 audioTranscriptionConfig: {319 wordTimestamp: true,320 },321 },322 });323324### REST325326 curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \327 -H "x-goog-api-key: $GEMINI_API_KEY" \328 -H "Content-Type: application/json" \329 -d '{330 "contents": [331 {332 "parts": [333 {334 "fileData": {335 "fileUri": "YOUR_FILE_URI",336 "mimeType": "audio/mp3"337 }338 }339 ]340 }341 ],342 "generationConfig": {343 "audioTranscriptionConfig": {344 "wordTimestamp": true345 }346 }347 }'348349You can combine `diarization` and `word_timestamp` in a single request to receive both speaker labels and word timestamps:350351### Python352353 config = types.GenerateContentConfig(354 audio_transcription_config=types.AudioTranscriptionConfig(355 diarization=True,356 word_timestamp=True,357 )358 )359360### JavaScript361362 const config = {363 audioTranscriptionConfig: {364 diarization: true,365 wordTimestamp: true,366 },367 };368369### REST370371 {372 "generationConfig": {373 "audioTranscriptionConfig": {374 "diarization": true,375 "wordTimestamp": true376 }377 }378 }379380## Transcription modes381382Gemini 3.5 Transcribe supports two transcription modes via the `mode` parameter:383384- **`VERBATIM` (default)**: Returns an exact word-for-word transcript of everything spoken, preserving raw filler words ("um", "uh", "like", "you know"), repetitions, pauses, and false starts. Required when using timestamps or speaker diarization.385- **`SMART` (Smart transcription)** : Optimizes the transcript for reading by applying intelligent post-processing:386 - **Disfluency removal**: Strips conversational filler words, stuttering, and false starts.387 - **Inline self-corrections** : Resolves spoken corrections directly (for example, *"Let's meet on Tuesday, actually no, Wednesday at two"* becomes *"Let's meet on Wednesday at 2:00 PM"*).388 - **Automatic structured formatting**: Automatically structures spoken thoughts into paragraphs, numbered lists, bullet points, formatted dates, currencies, and numbers.389 - **Grammatical cleanup**: Applies natural punctuation, sentence casing, and flow.390391| Spoken audio | `VERBATIM` output | `SMART` (Smart transcription) output |392|---|---|---|393| "Um, so for the meeting, I think we should, uh, invite Alice and, wait no, Bob and Carol." | "Um so for the meeting I think we should uh invite Alice and wait no Bob and Carol." | "For the meeting, I think we should invite Bob and Carol." |394| "First item review budget second item finalize timeline third item send recap" | "first item review budget second item finalize timeline third item send recap" | "1. Review budget 2. Finalize timeline 3. Send recap" |395396### Python397398 from google import genai399 from google.genai import types400401 client = genai.Client()402403 response = client.models.generate_content(404 model="gemini-3.5-transcribe",405 contents=[audio_file],406 config=types.GenerateContentConfig(407 audio_transcription_config=types.AudioTranscriptionConfig(408 mode="SMART",409 )410 ),411 )412 print(response.text)413414### JavaScript415416 const response = await ai.models.generateContent({417 model: "gemini-3.5-transcribe",418 contents: [audioFile],419 config: {420 audioTranscriptionConfig: {421 mode: "SMART",422 },423 },424 });425 console.log(response.text);426427### REST428429 curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-transcribe:generateContent" \430 -H "x-goog-api-key: $GEMINI_API_KEY" \431 -H "Content-Type: application/json" \432 -d '{433 "contents": [434 {435 "parts": [436 {437 "fileData": {438 "fileUri": "YOUR_FILE_URI",439 "mimeType": "audio/mp3"440 }441 }442 ]443 }444 ],445 "generationConfig": {446 "audioTranscriptionConfig": {447 "mode": "SMART"448 }449 }450 }'451452> [!NOTE]453> **Note:** Smart transcription (`mode: "SMART"`) is incompatible with `word_timestamp` and `diarization`. If you need word timestamps or speaker diarization, use `VERBATIM` mode (default).454455## Parsing transcription output456457The complete transcript text is returned in `response.text`.458459When `word_timestamp` or `diarization` is enabled, the API also returns detailed word-level annotations and speaker labels attached to the candidate parts.460461Here is how to extract and iterate over word timestamps and speaker turns:462463### Python464465 def extract_word_transcriptions(response):466 words = []467 for candidate in getattr(response, "candidates", []) or []:468 content = getattr(candidate, "content", None)469 for part in getattr(content, "parts", []) or []:470 transcription = getattr(part, "audio_transcription", None)471 if transcription:472 speaker = getattr(transcription, "speaker_label", "")473 for word_info in getattr(transcription, "words", []) or []:474 word = getattr(word_info, "word", "")475 start = getattr(word_info, "start_offset", "")476 end = getattr(word_info, "end_offset", "")477 words.append({478 "word": word,479 "speaker": speaker,480 "start_offset": start,481 "end_offset": end,482 })483 return words484485 words = extract_word_transcriptions(response)486487 for w in words:488 speaker = f"[{w['speaker']}] " if w["speaker"] else ""489 timing = f"({w['start_offset']} -> {w['end_offset']}) " if w["start_offset"] and w["end_offset"] else ""490 print(f"{speaker}{timing}{w['word']}")491492### JavaScript493494 function extractWordTranscriptions(response) {495 const words = [];496 for (const candidate of response.candidates ?? []) {497 for (const part of candidate.content?.parts ?? []) {498 const transcription = part.audioTranscription;499 if (transcription) {500 const speaker = transcription.speakerLabel ?? "";501 for (const wordInfo of transcription.words ?? []) {502 words.push({503 word: wordInfo.word ?? "",504 speaker: speaker,505 startOffset: wordInfo.startOffset ?? "",506 endOffset: wordInfo.endOffset ?? "",507 });508 }509 }510 }511 }512 return words;513 }514515 const words = extractWordTranscriptions(response);516517 for (const w of words) {518 const speaker = w.speaker ? `[${w.speaker}] ` : "";519 const timing = (w.startOffset && w.endOffset) ? `(${w.startOffset} -> ${w.endOffset}) ` : "";520 console.log(`${speaker}${timing}${w.word}`);521 }522523### REST524525 {526 "candidates": [527 {528 "content": {529 "parts": [530 {531 "audioTranscription": {532 "speakerLabel": "spk_1",533 "words": [534 {535 "word": "Hello",536 "startOffset": "0.100s",537 "endOffset": "0.450s"538 },539 {540 "word": "world",541 "startOffset": "0.500s",542 "endOffset": "0.850s"543 }544 ]545 }546 }547 ],548 "role": "model"549 },550 "finishReason": "STOP"551 }552 ]553 }554555## Supported languages556557The following languages and BCP-47 language codes are supported for Gemini 3.5 Transcribe:558559| Language | BCP-47 Code | Language | BCP-47 Code |560|---|---|---|---|561| Afrikaans | `af-ZA` | Japanese | `ja-JP` |562| Amharic | `am-ET` | Javanese | `jv-ID` |563| Arabic (Egypt) | `ar-EG` | Kabuverdianu | `kea-CV` |564| Armenian | `hy-AM` | Kannada | `kn-IN` |565| Assamese | `as-IN` | Kazakh | `kk-KZ` |566| Azerbaijani | `az-AZ` | Korean | `ko-KR` |567| Belarusian | `be-BY` | Kyrgyz | `ky-KG` |568| Bengali (Bangladesh) | `bn-BD` | Latvian | `lv-LV` |569| Bengali (India) | `bn-IN` | Lingala | `ln-CD` |570| Bosnian | `bs-BA` | Lithuanian | `lt-LT` |571| Bulgarian | `bg-BG` | Macedonian | `mk-MK` |572| Bulgarian (Aromanian) | `rup-BG` | Malay | `ms-MY` |573| Burmese | `my-MM` | Malayalam | `ml-IN` |574| Cantonese (Traditional) | `yue-Hant-HK` | Maltese | `mt-MT` |575| Catalan | `ca-ES` | Mandarin Chinese (Simplified) | `cmn-Hans-CN` |576| Cebuano | `ceb` | Marathi | `mr-IN` |577| Central Khmer | `km-KH` | Mongolian | `mn-MN` |578| Croatian | `hr-HR` | Nepali | `ne-NP` |579| Czech | `cs-CZ` | Norwegian | `nb-NO` |580| Danish | `da-DK` | Oriya | `or-IN` |581| Dutch | `nl-NL` | Polish | `pl-PL` |582| English (Great Britain) | `en-GB` | Portuguese (Brazil) | `pt-BR` |583| English (India) | `en-IN` | Portuguese (Portugal) | `pt-PT` |584| English (United States) | `en-US` | Punjabi | `pa-IN` |585| Estonian | `et-EE` | Punjabi (Gurmukhi script) | `pa-Guru-IN` |586| Farsi | `fa-IR` | Romanian | `ro-RO` |587| Filipino | `fil-PH` | Russian | `ru-RU` |588| Finnish | `fi-FI` | Serbian | `sr-RS` |589| French | `fr-FR` | Sindhi (Arabic script) | `sd-Arab-IN` |590| Galician | `gl-ES` | Slovak | `sk-SK` |591| Georgian | `ka-GE` | Slovenian | `sl-SI` |592| German | `de-DE` | Spanish (Latin America) | `es-419` |593| Greek | `el-GR` | Spanish (United States) | `es-US` |594| Gujarati | `gu-IN` | Swahili (Kenya) | `sw-KE` |595| Hausa | `ha-NG` | Swedish | `sv-SE` |596| Hebrew | `he-IL` | Tajik | `tg-TJ` |597| Hindi | `hi-IN` | Telugu | `te-IN` |598| Hungarian | `hu-HU` | Thai | `th-TH` |599| Icelandic | `is-IS` | Turkish | `tr-TR` |600| Indian English | `en-IN` | Ukrainian | `uk-UA` |601| Indonesian | `id-ID` | Uzbek | `uz-UZ` |602| Italian | `it-IT` | Vietnamese | `vi-VN` |603604## Supported audio formats605606Gemini 3.5 Transcribe supports the following audio format MIME types:607608- WAV - `audio/wav`609- MP3 - `audio/mp3`610- AIFF - `audio/aiff`611- AAC - `audio/aac`612- OGG - `audio/ogg`613- FLAC - `audio/flac`614- MPEG - `audio/mpeg`615- M4A - `audio/m4a`616- L16 - `audio/l16`617- Opus - `audio/opus`618- ALAW - `audio/alaw`619- MULAW - `audio/mulaw`620- WebM - `audio/webm`621622For the complete list of supported MIME types and parameter schemas, see the [Interactions API reference](https://ai.google.dev/api/interactions-api#Resource:Content).623624## Parameter reference625626Configure transcription by setting fields within the `audio_transcription_config` object in `GenerateContentConfig`:627628| Field | Type | Description |629|---|---|---|630| `language_codes` | Array of strings | BCP-47 language codes (e.g., `["en-US"]`). If omitted or empty (`[]`), the model automatically detects the language and handles code-switching. |631| `custom_vocabulary` | Array of strings | Up to 1,000 custom terms, acronyms, or proper names to bias speech recognition. Incompatible with speaker diarization and word-level timestamps. |632| `word_timestamp` | Boolean | Set to `True` to include word start and end offsets. If omitted or `False`, no word timestamps are returned. Incompatible with custom vocabulary. |633| `diarization` | Boolean | Set to `True` to identify and label distinct speakers. Incompatible with custom vocabulary. |634| `mode` | String | Transcription mode. Supported values: `"VERBATIM"` (default) and `"SMART"`. Incompatible with timestamps and diarization. |635636## Best practices637638- **Provide clean audio:** Ensure audio recordings have clear voice separation and avoid severe clipping.639- **Provide language hints when known:** If you know the audio language in advance, specify `language_codes` to maximize accuracy.640- **Target custom vocabulary:** Include only distinct domain terms, brand names, or proper nouns in `custom_vocabulary` rather than common everyday words.641- **Use the Files API for large recordings:** For files longer than a few seconds, upload the file using `client.files.upload` and pass the returned file to the model contents.642643## Limitations644645- **Audio duration:** Standard unary requests support audio files up to 1 hour. Audio processing is limited to 30 minutes when features like speaker diarization or word-level timestamps are enabled.646- **Word-level timestamps:** Enabling word-level timestamps may degrade overall transcription accuracy.647- **Speaker diarization:** Speaker diarization supports up to 8 speakers. Speaker attribution for 3 or more speakers is experimental.648- **Custom vocabulary:** You can provide up to 1,000 terms in `custom_vocabulary`, but best results are typically achieved with up to 100 terms. You cannot combine `custom_vocabulary` with speaker diarization or word-level timestamps; the API rejects requests that specify `custom_vocabulary` alongside either feature.649- **Mode compatibility:** Smart transcription (`mode: "SMART"`) cannot be combined with `word_timestamp` or `diarization`.650651## What's next652653- Stream real-time audio with the [Live transcription guide](https://ai.google.dev/gemini-api/docs/live-api/live-transcribe) using the Live API.654- Explore [Audio understanding](https://ai.google.dev/gemini-api/docs/generate-content/audio) to analyze, summarize, or query audio content.655- Learn how to synthesize audio from text using [Text-to-speech](https://ai.google.dev/gemini-api/docs/generate-content/speech-generation).656- Check the [Pricing page](https://ai.google.dev/gemini-api/docs/pricing#gemini-3.5-transcribe) for model pricing and token limits.657- Check the [Files API](https://ai.google.dev/gemini-api/docs/files) guide for details on uploading and managing media files.