Repository Public
1# Video understanding23> [!NOTE]4> **Note** : This version of the page covers the new [Interactions API](https://ai.google.dev/gemini-api/docs/interactions/interactions-overview), which is currently in Beta. 5> For stable production deployments, we recommend you continue to use the `generateContent` API. You can use the toggle on this page to switch between the versions.67> To learn about video generation, see the [Veo](https://ai.google.dev/gemini-api/docs/video) guide.89Gemini models can process videos, enabling many frontier developer use cases10that would have historically required domain specific models.11Some of Gemini's vision capabilities include the ability to: describe, segment,12and extract information from videos, answer questions about video content, and13refer to specific timestamps within a video.1415You can provide videos as input to Gemini in the following ways:1617| Input method | Max size | Recommended use case |18|---|---|---|19| [File API](https://ai.google.dev/gemini-api/docs/interactions/video-understanding#upload-video) | 20GB (paid) / 2GB (free) | Large files (100MB+), long videos (10min+), reusable files. |20| [Cloud Storage Registration](https://ai.google.dev/gemini-api/docs/file-input-methods#registration) | 2GB (per file, no storage limits) | Large files (100MB+), long videos (10min+), persistent, reusable files. |21| [Inline Data](https://ai.google.dev/gemini-api/docs/interactions/video-understanding#inline-video) | \< 100MB | Small files (\<100MB), short duration (\<1min), one-off inputs. |22| [YouTube URLs](https://ai.google.dev/gemini-api/docs/interactions/video-understanding#youtube) | N/A | Public YouTube videos. |2324> **Note:** The [File API](https://ai.google.dev/gemini-api/docs/interactions/video-understanding#upload-video) is recommended for most use cases, especially for files larger than 100MB or when you want to reuse the file across multiple requests.2526To learn about other file input methods, such as using external URLs or files27stored in Google Cloud, see the28[File input methods](https://ai.google.dev/gemini-api/docs/interactions/file-input-methods) guide.2930### Upload a video file3132The following code downloads a sample video, uploads it using the [Files API](https://ai.google.dev/gemini-api/docs/interactions/files),33waits for it to be processed, and then uses the uploaded file reference to34summarize the video.3536### Python3738 from google import genai39 import base6440 import time4142 client = genai.Client()4344 myfile = client.files.upload(file="path/to/sample.mp4")4546 while not myfile.state or myfile.state.name != "ACTIVE":47 print("Processing video...")48 time.sleep(5)49 myfile = client.files.get(name=myfile.name)5051 interaction = client.interactions.create(52 model="gemini-3.5-flash",53 input=[54 {"type": "video", "uri": myfile.uri, "mime_type": myfile.mime_type},55 {"type": "text", "text": "Summarize this video. Then create a quiz with an answer key based on the information in this video."}56 ]57 )5859 print(interaction.output_text)6061### JavaScript6263 import { GoogleGenAI } from "@google/genai";6465 const ai = new GoogleGenAI({});6667 async function main() {68 const myfile = await ai.files.upload({69 file: "path/to/sample.mp4",70 config: { mimeType: "video/mp4" },71 });7273 let getFile = await ai.files.get({ name: myfile.name });74 while (getFile.state === 'PROCESSING') {75 getFile = await ai.files.get({ name: myfile.name });76 console.log(`current file status: ${getFile.state}`);77 console.log('File is still processing, retrying in 5 seconds');7879 await new Promise((resolve) => {80 setTimeout(resolve, 5000);81 });82 }83 if (getFile.state === 'FAILED') {84 throw new Error('File processing failed.');85 }8687 const interaction = await ai.interactions.create({88 model: "gemini-3.5-flash",89 input: [90 { type: "video", uri: myfile.uri, mime_type: myfile.mimeType },91 { type: "text", text: "Summarize this video. Then create a quiz with an answer key based on the information in this video." }92 ],93 });94 console.log(interaction.output_text);95 }9697 await main();9899### REST100101 VIDEO_PATH="path/to/sample.mp4"102 MIME_TYPE=$(file -b --mime-type "${VIDEO_PATH}")103 NUM_BYTES=$(wc -c < "${VIDEO_PATH}")104 DISPLAY_NAME=VIDEO105106 tmp_header_file=upload-header.tmp107108 echo "Starting file upload..."109 curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \110 -H "x-goog-api-key: $GEMINI_API_KEY" \111 -D ${tmp_header_file} \112 -H "X-Goog-Upload-Protocol: resumable" \113 -H "X-Goog-Upload-Command: start" \114 -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \115 -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \116 -H "Content-Type: application/json" \117 -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null118119 upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")120 rm "${tmp_header_file}"121122 echo "Uploading video data..."123 curl "${upload_url}" \124 -H "Content-Length: ${NUM_BYTES}" \125 -H "X-Goog-Upload-Offset: 0" \126 -H "X-Goog-Upload-Command: upload, finalize" \127 --data-binary "@${VIDEO_PATH}" 2> /dev/null > file_info.json128129 file_uri=$(jq -r ".file.uri" file_info.json)130 file_name=$(jq -r ".file.name" file_info.json)131 echo file_uri=$file_uri132133 echo "File uploaded successfully. File URI: ${file_uri}"134135 # Polling loop136 echo "Waiting for file to be processed..."137 while true; do138 curl -s "https://generativelanguage.googleapis.com/v1beta/${file_name}" \139 -H "x-goog-api-key: $GEMINI_API_KEY" > file_status.json140 state=$(jq -r ".state" file_status.json)141 echo "Current state: $state"142 if [ "$state" == "ACTIVE" ]; then143 break144 elif [ "$state" == "FAILED" ]; then145 echo "File processing failed."146 exit 1147 fi148 sleep 5149 done150151 echo "Generating content from video..."152 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \153 -H "x-goog-api-key: $GEMINI_API_KEY" \154 -H "Api-Revision: 2026-05-20" \155 -H 'Content-Type: application/json' \156 -d '{157 "model": "gemini-3.5-flash",158 "input": [159 {"type": "video", "uri": "'${file_uri}'", "mime_type": "'${MIME_TYPE}'"},160 {"type": "text", "text": "Summarize this video. Then create a quiz with an answer key based on the information in this video."}161 ]162 }' 2> /dev/null > response.json163164 jq ".steps[].content[0].text" response.json165166Always use the Files API when the total request size (including the file, text167prompt, system instructions, etc.) is larger than 20 MB, the video duration is168significant, or if you intend to use the same video in multiple prompts.169The File API accepts video file formats directly.170171To learn more about working with media files, see172[Files API](https://ai.google.dev/gemini-api/docs/interactions/files).173174### Pass video data inline175176Instead of uploading a video file using the File API, you can pass smaller177videos directly in the request. This is suitable for178shorter videos under 20MB total request size.179180Here's an example of providing inline video data:181182### Python183184 from google import genai185 import base64186187 video_file_name = "/path/to/your/video.mp4"188 video_bytes = open(video_file_name, 'rb').read()189190 client = genai.Client()191 interaction = client.interactions.create(192 model='gemini-3.5-flash',193 input=[194 {"type": "text", "text": "Please summarize the video in 3 sentences."},195 {196 "type": "video",197 "data": base64.b64encode(video_bytes).decode('utf-8'),198 "mime_type": "video/mp4"199 }200 ]201 )202 print(interaction.output_text)203204### JavaScript205206 import { GoogleGenAI } from "@google/genai";207 import * as fs from "node:fs";208209 const ai = new GoogleGenAI({});210 const base64VideoFile = fs.readFileSync("path/to/small-sample.mp4", {211 encoding: "base64",212 });213214 const interaction = await ai.interactions.create({215 model: "gemini-3.5-flash",216 input: [217 { type: "text", text: "Please summarize the video in 3 sentences." },218 {219 type: "video",220 data: base64VideoFile,221 mime_type: "video/mp4",222 }223 ],224 });225 console.log(interaction.output_text);226227### REST228229> [!NOTE]230> **Note:** If you get an `Argument list too long` error, the base64 encoding of your file might be too long for the curl command line. Use the File API method instead for larger files.231232 VIDEO_PATH=/path/to/your/video.mp4233234 if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then235 B64FLAGS="--input"236 else237 B64FLAGS="-w0"238 fi239240 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \241 -H "x-goog-api-key: $GEMINI_API_KEY" \242 -H "Api-Revision: 2026-05-20" \243 -H 'Content-Type: application/json' \244 -d '{245 "model": "gemini-3.5-flash",246 "input": [247 {"type": "text", "text": "Please summarize the video in 3 sentences."},248 {249 "type": "video",250 "data": "'$(base64 $B64FLAGS $VIDEO_PATH)'",251 "mime_type": "video/mp4"252 }253 ]254 }' 2> /dev/null255256### Pass YouTube URLs257258> [!WARNING]259> **Preview:** The YouTube URL feature is in preview and is available at no charge. Pricing and rate limits are likely to change.260261You can pass YouTube URLs directly to Gemini API as part of your request as follows:262263### Python264265 from google import genai266267 client = genai.Client()268 interaction = client.interactions.create(269 model='gemini-3.5-flash',270 input=[271 {"type": "text", "text": "Please summarize the video in 3 sentences."},272 {273 "type": "video",274 "uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg"275 }276 ]277 )278 print(interaction.output_text)279280### JavaScript281282 import { GoogleGenAI } from "@google/genai";283284 const ai = new GoogleGenAI({});285286 const interaction = await ai.interactions.create({287 model: "gemini-3.5-flash",288 input: [289 { type: "text", text: "Please summarize the video in 3 sentences." },290 {291 type: "video",292 uri: "https://www.youtube.com/watch?v=9hE5-98ZeCg",293 }294 ],295 });296 console.log(interaction.output_text);297298### REST299300 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \301 -H "x-goog-api-key: $GEMINI_API_KEY" \302 -H "Api-Revision: 2026-05-20" \303 -H 'Content-Type: application/json' \304 -d '{305 "model": "gemini-3.5-flash",306 "input": [307 {"type": "text", "text": "Please summarize the video in 3 sentences."},308 {309 "type": "video",310 "uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg"311 }312 ]313 }' 2> /dev/null314315**Limitations:**316317- For the free tier, you can't upload more than 8 hours of YouTube video per day.318- For the paid tier, there is no limit based on video length.319- For models prior to Gemini 2.5, you can upload only 1 video per request. For Gemini 2.5 and later models, you can upload a maximum of 10 videos per request.320- You can only upload public videos (not private or unlisted videos).321322## Refer to timestamps in the content323324You can ask questions about specific points in time within the video using325timestamps of the form `MM:SS`.326327### Python328329 prompt = "What are the examples given at 00:05 and 00:10 supposed to show us?"330331### JavaScript332333 const prompt = "What are the examples given at 00:05 and 00:10 supposed to show us?";334335### REST336337 PROMPT="What are the examples given at 00:05 and 00:10 supposed to show us?"338339## Extract detailed insights from video340341Gemini models offer powerful capabilities for understanding video content by342processing information from both the **audio and visual** streams. This lets you343extract a rich set of details, including generating descriptions of what is344happening in a video and answering questions about its content.345346For visual descriptions, the model samples the video at a rate of **1 frame347per second** (FPS). This default sampling rate works well for most content, but348note that it may miss details in videos with rapid motion or quick scene changes.349350### Python351352 prompt = "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments."353354### JavaScript355356 const prompt = "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments.";357358### REST359360 PROMPT="Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments."361362## Supported video formats363364Gemini supports the following video format MIME types:365366- `video/mp4`367- `video/mpeg`368- `video/mov`369- `video/avi`370- `video/x-flv`371- `video/mpg`372- `video/webm`373- `video/wmv`374- `video/3gpp`375376## Technical details about videos377378- **Supported models \& context** : All Gemini can process video data.379 - Models with a 1M context window can process videos up to 1 hour long at default media resolution or 3 hours long at low media resolution.380- **File API processing** : When using the File API, videos are stored at 1 frame per second (FPS) and audio is processed at 1Kbps (single channel). Timestamps are added every second.381 - These rates are subject to change in the future for improvements in inference.382- **Token calculation** : Each second of video is tokenized as follows:383 - Individual frames (sampled at 1 FPS):384 - If `media_resolution` is set to low, frames are tokenized at 66 tokens per frame.385 - Otherwise, frames are tokenized at 258 tokens per frame.386 - Audio: 32 tokens per second.387 - Metadata is also included.388 - Total: Approximately 300 tokens per second of video at default media resolution, or 100 tokens per second of video at low media resolution.389- **Medial resolution** : Gemini 3 introduces granular control over multimodal390 vision processing with the `media_resolution` parameter. The391 `media_resolution` parameter determines the392 **maximum number of tokens allocated per input image or video frame.**393 Higher resolutions improve the model's ability to read fine text or identify394 small details, but increase token usage and latency.395396 For more details on token calculations, see the [tokens](https://ai.google.dev/gemini-api/docs/interactions/tokens) guide.397- **Timestamp format** : When referring to specific moments in a video within your prompt, use the `MM:SS` format (e.g., `01:15` for 1 minute and 15 seconds).398399- **Best practices**:400401 - Use only one video per prompt request for optimal results.402 - If combining text and a single video, place the text prompt *after* the video part in the `input` array.403 - Be aware that fast action sequences might lose detail due to the 1 FPS sampling rate. Consider slowing down such clips if necessary.404405## What's next406407This guide shows how to upload video files and generate text outputs from video408inputs. To learn more, see the following resources:409410- [System instructions](https://ai.google.dev/gemini-api/docs/interactions/text-generation#system-instructions): System instructions let you steer the behavior of the model based on your specific needs and use cases.411- [Files API](https://ai.google.dev/gemini-api/docs/interactions/files): Learn more about uploading and managing files for use with Gemini.412- [File prompting strategies](https://ai.google.dev/gemini-api/docs/interactions/files#prompt-guide): The Gemini API supports prompting with text, image, audio, and video data, also known as multimodal prompting.413- [Safety guidance](https://ai.google.dev/gemini-api/docs/safety-guidance): Sometimes generative AI models produce unexpected outputs, such as outputs that are inaccurate, biased, or offensive. Post-processing and human evaluation are essential to limit the risk of harm from such outputs.