Repository Public
1> To learn about video generation, see the [Gemini Omni Flash](https://ai.google.dev/gemini-api/docs/omni) guide.23Gemini models can process videos, enabling many frontier developer use cases4that would have historically required domain specific models.5Some of Gemini's vision capabilities include the ability to: describe, segment,6and extract information from videos, answer questions about video content, and7refer to specific timestamps within a video.89You can provide videos as input to Gemini in the following ways:1011| Input method | Max size | Recommended use case |12|---|---|---|13| [File API](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#upload-video) | 20GB (paid) / 2GB (free) | Large files (100MB+), long videos (10min+), reusable files. |14| [Cloud Storage Registration](https://ai.google.dev/gemini-api/docs/file-input-methods#registration) | 2GB (per file, no storage limits) | Large files (100MB+), long videos (10min+), persistent, reusable files. |15| [Inline Data](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#inline-video) | \< 100MB | Small files (\<100MB), short duration (\<1min), one-off inputs. |16| [YouTube URLs](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#youtube) | N/A | Public YouTube videos. |1718> **Note:** The [File API](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#upload-video) is recommended for most use cases, especially for files larger than 100MB or when you want to reuse the file across multiple requests.1920To learn about other file input methods, such as using external URLs or files21stored in Google Cloud, see the22[File input methods](https://ai.google.dev/gemini-api/docs/file-input-methods) guide.2324### Upload a video file2526The following code downloads a sample video, uploads it using the [Files API](https://ai.google.dev/gemini-api/docs/files),27waits for it to be processed, and then uses the uploaded file reference to28summarize the video.2930### Python3132 from google import genai3334 client = genai.Client()3536 myfile = client.files.upload(file="path/to/sample.mp4")3738 response = client.models.generate_content(39 model="gemini-3.8-flash", contents=[myfile, "Summarize this video. Then create a quiz with an answer key based on the information in this video."]40 )4142 print(response.text)4344### JavaScript4546 import {47 GoogleGenAI,48 createUserContent,49 createPartFromUri,50 } from "@google/genai";5152 const ai = new GoogleGenAI({});5354 async function main() {55 const myfile = await ai.files.upload({56 file: "path/to/sample.mp4",57 config: { mimeType: "video/mp4" },58 });5960 const response = await ai.models.generateContent({61 model: "gemini-3.8-flash",62 contents: createUserContent([63 createPartFromUri(myfile.uri, myfile.mimeType),64 "Summarize this video. Then create a quiz with an answer key based on the information in this video.",65 ]),66 });67 console.log(response.text);68 }6970 await main();7172### Go7374 uploadedFile, _ := client.Files.UploadFromPath(ctx, "path/to/sample.mp4", nil)7576 parts := []*genai.Part{77 genai.NewPartFromText("Summarize this video. Then create a quiz with an answer key based on the information in this video."),78 genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),79 }8081 contents := []*genai.Content{82 genai.NewContentFromParts(parts, genai.RoleUser),83 }8485 result, _ := client.Models.GenerateContent(86 ctx,87 "gemini-3.8-flash",88 contents,89 nil,90 )9192 fmt.Println(result.Text())9394### REST9596 VIDEO_PATH="path/to/sample.mp4"97 MIME_TYPE=$(file -b --mime-type "${VIDEO_PATH}")98 NUM_BYTES=$(wc -c < "${VIDEO_PATH}")99 DISPLAY_NAME=VIDEO100101 tmp_header_file=upload-header.tmp102103 echo "Starting file upload..."104 curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \105 -H "x-goog-api-key: $GEMINI_API_KEY" \106 -D ${tmp_header_file} \107 -H "X-Goog-Upload-Protocol: resumable" \108 -H "X-Goog-Upload-Command: start" \109 -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \110 -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \111 -H "Content-Type: application/json" \112 -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null113114 upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")115 rm "${tmp_header_file}"116117 echo "Uploading video data..."118 curl "${upload_url}" \119 -H "Content-Length: ${NUM_BYTES}" \120 -H "X-Goog-Upload-Offset: 0" \121 -H "X-Goog-Upload-Command: upload, finalize" \122 --data-binary "@${VIDEO_PATH}" 2> /dev/null > file_info.json123124 file_uri=$(jq -r ".file.uri" file_info.json)125 echo file_uri=$file_uri126127 echo "File uploaded successfully. File URI: ${file_uri}"128129 # --- 3. Generate content using the uploaded video file ---130 echo "Generating content from video..."131 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \132 -H "x-goog-api-key: $GEMINI_API_KEY" \133 -H 'Content-Type: application/json' \134 -X POST \135 -d '{136 "contents": [{137 "parts":[138 {"file_data":{"mime_type": "'"${MIME_TYPE}"'", "file_uri": "'"${file_uri}"'"}},139 {"text": "Summarize this video. Then create a quiz with an answer key based on the information in this video."}]140 }]141 }' 2> /dev/null > response.json142143 jq -r ".candidates[].content.parts[].text" response.json144145To optimize token efficiency and performance, consider using146[Agentic video processing](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#agentic-video-understanding).147148Always use the Files API when the total request size (including the file, text149prompt, system instructions, etc.) is larger than 20 MB, the video duration is150significant, or if you intend to use the same video in multiple prompts.151The File API accepts video file formats directly.152153To learn more about working with media files, see154[Files API](https://ai.google.dev/gemini-api/docs/files).155156### Pass video data inline157158Instead of uploading a video file using the File API, you can pass smaller159videos directly in the request to `generateContent`. This is suitable for160shorter videos under 20MB total request size.161162Here's an example of providing inline video data:163164### Python165166 from google import genai167 from google.genai import types168169 # Only for videos of size <20Mb170 video_file_name = "/path/to/your/video.mp4"171 video_bytes = open(video_file_name, 'rb').read()172173 client = genai.Client()174 response = client.models.generate_content(175 model='gemini-3.8-flash',176 contents=types.Content(177 parts=[178 types.Part(179 inline_data=types.Blob(data=video_bytes, mime_type='video/mp4')180 ),181 types.Part(text='Please summarize the video in 3 sentences.')182 ]183 )184 )185 print(response.text)186187### JavaScript188189 import { GoogleGenAI } from "@google/genai";190 import * as fs from "node:fs";191192 const ai = new GoogleGenAI({});193 const base64VideoFile = fs.readFileSync("path/to/small-sample.mp4", {194 encoding: "base64",195 });196197 const contents = [198 {199 inlineData: {200 mimeType: "video/mp4",201 data: base64VideoFile,202 },203 },204 { text: "Please summarize the video in 3 sentences." }205 ];206207 const response = await ai.models.generateContent({208 model: "gemini-3.8-flash",209 contents: contents,210 });211 console.log(response.text);212213### REST214215> [!NOTE]216> **Note:** If you get an `Argument list too long` error, the base64 encoding of your file might be too long for the curl command line. Use the File API method instead for larger files.217218 VIDEO_PATH=/path/to/your/video.mp4219220 if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then221 B64FLAGS="--input"222 else223 B64FLAGS="-w0"224 fi225226 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \227 -H "x-goog-api-key: $GEMINI_API_KEY" \228 -H 'Content-Type: application/json' \229 -X POST \230 -d '{231 "contents": [{232 "parts":[233 {234 "inline_data": {235 "mime_type":"video/mp4",236 "data": "'$(base64 $B64FLAGS $VIDEO_PATH)'"237 }238 },239 {"text": "Please summarize the video in 3 sentences."}240 ]241 }]242 }' 2> /dev/null243244### Pass YouTube URLs245246> [!WARNING]247> **Preview:** The YouTube URL feature is in preview and is available at no charge. Pricing and rate limits are likely to change.248249You can pass YouTube URLs directly to Gemini API as part of your request as follows:250251### Python252253 from google import genai254 from google.genai import types255256 client = genai.Client()257 response = client.models.generate_content(258 model='gemini-3.8-flash',259 contents=types.Content(260 parts=[261 types.Part(262 file_data=types.FileData(file_uri='https://www.youtube.com/watch?v=9hE5-98ZeCg')263 ),264 types.Part(text='Please summarize the video in 3 sentences.')265 ]266 )267 )268 print(response.text)269270### JavaScript271272 import { GoogleGenAI } from "@google/genai";273274 const ai = new GoogleGenAI({});275276 const contents = [277 {278 fileData: {279 fileUri: "https://www.youtube.com/watch?v=9hE5-98ZeCg",280 },281 },282 { text: "Please summarize the video in 3 sentences." }283 ];284285 const response = await ai.models.generateContent({286 model: "gemini-3.8-flash",287 contents: contents,288 });289 console.log(response.text);290291### Go292293 package main294295 import (296 "context"297 "fmt"298 "os"299 "google.golang.org/genai"300 )301302 func main() {303 ctx := context.Background()304 client, err := genai.NewClient(ctx, nil)305 if err != nil {306 log.Fatal(err)307 }308309 parts := []*genai.Part{310 genai.NewPartFromText("Please summarize the video in 3 sentences."),311 genai.NewPartFromURI("https://www.youtube.com/watch?v=9hE5-98ZeCg","video/mp4"),312 }313314 contents := []*genai.Content{315 genai.NewContentFromParts(parts, genai.RoleUser),316 }317318 result, _ := client.Models.GenerateContent(319 ctx,320 "gemini-3.8-flash",321 contents,322 nil,323 )324325 fmt.Println(result.Text())326 }327328### REST329330 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \331 -H "x-goog-api-key: $GEMINI_API_KEY" \332 -H 'Content-Type: application/json' \333 -X POST \334 -d '{335 "contents": [{336 "parts":[337 {"text": "Please summarize the video in 3 sentences."},338 {339 "file_data": {340 "file_uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg"341 }342 }343 ]344 }]345 }' 2> /dev/null346347**Limitations:**348349- For the free tier, you can't upload more than 8 hours of YouTube video per day.350- For the paid tier, there is no limit based on video length.351- For models prior to Gemini 2.5, you can upload only 1 video per request. For Gemini 2.5 and later models, you can upload a maximum of 10 videos per request.352- You can only upload public videos (not private or unlisted videos).353354## Agentic video understanding355356By default, video inputs use static processing (extracting frames at 1 FPS).357Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite models also support358**agentic video understanding**, where the model dynamically explores the video359timeline, selectively inspecting transcripts and adaptively adjusting frame360rates and resolution on the fly based on the prompt.361362| **Mode** | **Description** | **Supported models** |363|---|---|---|364| **Static** (default) | Extracts frames at a fixed rate (1 FPS) and places them into context in a single pass. Works well for short clips. | All Gemini models |365| **Agentic** | The model dynamically navigates the video timeline, loading only the content it needs based on the prompt. Up to 88% more token-efficient and \~7% higher quality on long-form content. | Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, 3.5 Flash Lite |366367### Choose a processing mode368369As a general guideline, start with **agentic** mode, especially when optimizing370for response quality or token efficiency.371372- **Agentic:** Long-form videos or queries targeting specific moments. The model dynamically navigates the timeline to target contextually relevant information without filling the context window.373- **Static:** Latency-sensitive queries on short clips (under 5 minutes), or cases where frame-level precision across the entire clip is needed.374375> **Note:** For long videos or complex prompts where agentic processing takes376> more time, use streaming (`client.models.generate_content_stream`). This keeps377> the connection active, surfaces intermediate reasoning steps, and avoids378> connection or authentication timeouts.379380### Set the processing mode381382### Python383384 import time385 from google import genai386 from google.genai import types387388 client = genai.Client()389390 video_file = client.files.upload(file="path/to/lecture.mp4")391392 while video_file.state.name == "PROCESSING":393 time.sleep(2)394 video_file = client.files.get(name=video_file.name)395396 response = client.models.generate_content(397 model="gemini-3.8-flash",398 contents=[399 types.Part.from_uri(400 file_uri=video_file.uri,401 mime_type=video_file.mime_type,402 media_processing="AGENTIC",403 ),404 "What are the three main arguments presented?",405 ],406 )407 print(response.text)408409### JavaScript410411 import { GoogleGenAI } from "@google/genai";412413 const ai = new GoogleGenAI({});414415 let videoFile = await ai.files.upload({416 file: "path/to/lecture.mp4",417 config: { mimeType: "video/mp4" },418 });419420 while (videoFile.state === "PROCESSING") {421 await new Promise((resolve) => setTimeout(resolve, 2000));422 videoFile = await ai.files.get({ name: videoFile.name });423 }424425 const response = await ai.models.generateContent({426 model: "gemini-3.8-flash",427 contents: [428 {429 role: "user",430 parts: [431 {432 fileData: {433 fileUri: videoFile.uri,434 mimeType: videoFile.mimeType,435 },436 mediaProcessing: "AGENTIC",437 },438 { text: "What are the three main arguments presented?" },439 ],440 },441 ],442 });443 console.log(response.text);444445### Go446447 uploadedFile, _ := client.Files.UploadFromPath(ctx, "path/to/lecture.mp4", nil)448 parts := []*genai.Part{449 {450 FileData: &genai.FileData{451 FileURI: uploadedFile.URI,452 MIMEType: uploadedFile.MIMEType,453 },454 MediaProcessing: genai.MediaProcessingAgentic,455 },456 genai.NewPartFromText("What are the three main arguments presented?"),457 }458 contents := []*genai.Content{459 genai.NewContentFromParts(parts, genai.RoleUser),460 }461 result, _ := client.Models.GenerateContent(462 ctx,463 "gemini-3.8-flash",464 contents,465 nil,466 )467 fmt.Println(result.Text())468469### REST470471 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent?key=$GEMINI_API_KEY" \472 -H 'Content-Type: application/json' \473 -d '{474 "contents": [{475 "parts": [476 {477 "file_data": {478 "file_uri": "'${file_uri}'",479 "mime_type": "video/mp4"480 },481 "media_processing": "AGENTIC"482 },483 {"text": "What are the three main arguments presented?"}484 ]485 }]486 }'487488> **Note:** To verify that agentic processing was used, inspect `response.candidates[0].content.parts`. The presence of `tool_call` and `tool_response` parts with the `MEDIA_PROCESSING` tool type indicates that the model dynamically navigated the video.489> **Note:** Unlike other server-side tools (such as Google Search or URL490> context), agentic video does not require setting491> `include_server_side_tool_invocations=True` in `ToolConfig` for tool calls and492> results to be returned or streamed. The `tool_call` and `tool_response` parts493> for video navigation are returned automatically when494> `media_processing="AGENTIC"` is set on any input part.495496### Response structure497498When agentic processing is enabled, the response includes additional parts that expose the internal navigation trace:499500- `tool_call` **parts** (`tool_type: "MEDIA_PROCESSING"`): emitted each time the model requests a video segment or audio transcript.501- `tool_response` **parts** (`tool_type: "MEDIA_PROCESSING"`): the result of each load operation.502503You do not need to handle or reply to these parts manually: pass the full response back as conversation history and they are handled automatically.504505If `include_thoughts=True` is set in `ThinkingConfig`, reasoning steps appear as `thought: true` parts interleaved with the tool call/response pairs. With thoughts disabled, thought text is omitted but tool parts are still present.506507The following example shows the response payload with interleaved tool call and response parts:508509 {510 "candidates": [511 {512 "content": {513 "role": "model",514 "parts": [515 {516 "thought": true,517 "text": "Inspecting transcript for key discussion topics..."518 },519 {520 "thought_signature": "sig_A",521 "tool_call": {522 "tool_type": "MEDIA_PROCESSING"523 }524 },525 {526 "thought_signature": "sig_B",527 "tool_response": {528 "tool_type": "MEDIA_PROCESSING"529 }530 },531 {532 "thought": true,533 "text": "Loading visual frames to verify slide content..."534 },535 {536 "thought_signature": "sig_C",537 "tool_call": {538 "tool_type": "MEDIA_PROCESSING"539 }540 },541 {542 "thought_signature": "sig_D",543 "tool_response": {544 "tool_type": "MEDIA_PROCESSING"545 }546 },547 {548 "thought": true,549 "text": "Synthesizing answer from gathered evidence..."550 },551 {552 "text": "The three main arguments presented in the lecture are...",553 "thought_signature": "sig_E"554 }555 ]556 }557 }558 ]559 }560561### Mix processing modes across videos562563You can set different processing modes for each video Part in the same request:564565### Python566567 from google import genai568 from google.genai import types569570 client = genai.Client()571572 lecture = client.files.upload(file="path/to/long-lecture.mp4")573 experiment = client.files.upload(file="path/to/short-experiment.mp4")574575 response = client.models.generate_content(576 model="gemini-3.8-flash",577 contents=[578 types.Part.from_uri(579 file_uri=lecture.uri,580 mime_type=lecture.mime_type,581 media_processing="AGENTIC", # Use agentic video understanding582 ),583 types.Part.from_uri(584 file_uri=experiment.uri,585 mime_type=experiment.mime_type,586 media_processing="STATIC", # Use static processing587 ),588 "Compare the lecture content with the experiment results.",589 ],590 )591 print(response.text)592593### JavaScript594595 import { GoogleGenAI } from "@google/genai";596597 const ai = new GoogleGenAI({});598599 const lecture = await ai.files.upload({600 file: "path/to/long-lecture.mp4",601 config: { mimeType: "video/mp4" },602 });603 const experiment = await ai.files.upload({604 file: "path/to/short-experiment.mp4",605 config: { mimeType: "video/mp4" },606 });607608 const response = await ai.models.generateContent({609 model: "gemini-3.8-flash",610 contents: [611 {612 role: "user",613 parts: [614 {615 fileData: {616 fileUri: lecture.uri,617 mimeType: lecture.mimeType,618 },619 mediaProcessing: "AGENTIC", // Use agentic video understanding620 },621 {622 fileData: {623 fileUri: experiment.uri,624 mimeType: experiment.mimeType,625 },626 mediaProcessing: "STATIC", // Use static processing627 },628 { text: "Compare the lecture content with the experiment results." },629 ],630 },631 ],632 });633 console.log(response.text);634635### Go636637 lecturePart := &genai.Part{638 FileData: &genai.FileData{639 FileURI: lectureFile.URI,640 MIMEType: lectureFile.MIMEType,641 },642 MediaProcessing: genai.MediaProcessingAgentic, // Use agentic643 }644 experimentPart := &genai.Part{645 FileData: &genai.FileData{646 FileURI: experimentFile.URI,647 MIMEType: experimentFile.MIMEType,648 },649 MediaProcessing: genai.MediaProcessingStatic, // Use static650 }651 parts := []*genai.Part{652 lecturePart,653 experimentPart,654 genai.NewPartFromText("Compare the lecture content with the experiment results."),655 }656 contents := []*genai.Content{657 genai.NewContentFromParts(parts, genai.RoleUser),658 }659 result, _ := client.Models.GenerateContent(ctx, "gemini-3.8-flash", contents, nil)660 fmt.Println(result.Text())661662### REST663664 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent?key=$GEMINI_API_KEY" \665 -H 'Content-Type: application/json' \666 -d '{667 "contents": [{668 "parts": [669 {670 "file_data": {671 "file_uri": "'${lecture_uri}'",672 "mime_type": "video/mp4"673 },674 "media_processing": "AGENTIC"675 },676 {677 "file_data": {678 "file_uri": "'${experiment_uri}'",679 "mime_type": "video/mp4"680 },681 "media_processing": "STATIC"682 },683 {"text": "Compare the lecture content with the experiment results."}684 ]685 }]686 }'687688## Use context caching for long videos689690For videos longer than 10 minutes, or when you plan to make multiple requests691against the same video file, use [context caching](https://ai.google.dev/gemini-api/docs/caching) to692reduce costs and improve latency. Context caching lets you process the video693once and reuse the tokens for subsequent queries, making it ideal for chat694sessions or repeated analysis of long-form content.695696## Refer to timestamps in the content697698You can ask questions about specific points in time within the video using699timestamps of the form `MM:SS`.700701### Python702703 response = client.models.generate_content(704 model="gemini-3.8-flash",705 contents=[706 myfile,707 "What are the examples given at 00:05 and 00:10 supposed to show us?",708 ],709 )710 print(response.text)711712### JavaScript713714 const response = await ai.models.generateContent({715 model: "gemini-3.8-flash",716 contents: [717 myfile,718 "What are the examples given at 00:05 and 00:10 supposed to show us?",719 ],720 });721 console.log(response.text);722723### Go724725 parts := []*genai.Part{726 genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),727 genai.NewPartFromText("What are the examples given at 00:05 and 00:10 supposed to show us?"),728 }729730 result, _ := client.Models.GenerateContent(731 ctx,732 "gemini-3.8-flash",733 []*genai.Content{genai.NewContentFromParts(parts, genai.RoleUser)},734 nil,735 )736 fmt.Println(result.Text())737738### REST739740 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \741 -H "x-goog-api-key: $GEMINI_API_KEY" \742 -H 'Content-Type: application/json' \743 -X POST \744 -d '{745 "contents": [{746 "parts":[747 {"file_data": {"file_uri": "'"${file_uri}"'", "mime_type": "'"${MIME_TYPE}"'"}},748 {"text": "What are the examples given at 00:05 and 00:10 supposed to show us?"}749 ]750 }]751 }' 2> /dev/null752753## Extract detailed insights from video754755Gemini models offer powerful capabilities for understanding video content by756processing information from both the **audio and visual** streams. This lets you757extract a rich set of details, including generating descriptions of what is758happening in a video and answering questions about its content.759760For visual descriptions, the model samples the video at a rate of **1 frame761per second** (FPS). This default sampling rate works well for most content, but762note that it may miss details in videos with rapid motion or quick scene changes.763For such high-motion content, consider [setting a custom frame rate](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#custom-frame-rate).764765### Python766767 response = client.models.generate_content(768 model="gemini-3.8-flash",769 contents=[770 myfile,771 "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments.",772 ],773 )774 print(response.text)775776### JavaScript777778 const response = await ai.models.generateContent({779 model: "gemini-3.8-flash",780 contents: [781 myfile,782 "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments.",783 ],784 });785 console.log(response.text);786787### Go788789 parts := []*genai.Part{790 genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),791 genai.NewPartFromText("Describe the key events in this video, providing both audio and visual details. " +792 "Include timestamps for salient moments."),793 }794795 result, _ := client.Models.GenerateContent(796 ctx,797 "gemini-3.8-flash",798 []*genai.Content{genai.NewContentFromParts(parts, genai.RoleUser)},799 nil,800 )801 fmt.Println(result.Text())802803### REST804805 curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \806 -H "x-goog-api-key: $GEMINI_API_KEY" \807 -H 'Content-Type: application/json' \808 -X POST \809 -d '{810 "contents": [{811 "parts":[812 {"file_data": {"file_uri": "'"${file_uri}"'", "mime_type": "'"${MIME_TYPE}"'"}},813 {"text": "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments."}814 ]815 }]816 }' 2> /dev/null817818## Customize video processing819820You can customize video processing in the Gemini API by setting clipping821intervals or providing custom frame rate sampling. These customization options822are only supported when processing the video in `"static"` mode.823824### Set clipping intervals825826You can clip video by specifying `videoMetadata` with start and end offsets.827828### Python829830 from google import genai831 from google.genai import types832833 client = genai.Client()834 response = client.models.generate_content(835 model='models/gemini-3.8-flash',836 contents=types.Content(837 parts=[838 types.Part(839 file_data=types.FileData(file_uri='https://www.youtube.com/watch?v=XEzRZ35urlk'),840 video_metadata=types.VideoMetadata(841 start_offset='1250s',842 end_offset='1570s'843 )844 ),845 types.Part(text='Please summarize the video in 3 sentences.')846 ]847 )848 )849850### JavaScript851852 import { GoogleGenAI } from '@google/genai';853 const ai = new GoogleGenAI({});854 const model = 'gemini-3.8-flash';855856 async function main() {857 const contents = [858 {859 role: 'user',860 parts: [861 {862 fileData: {863 fileUri: 'https://www.youtube.com/watch?v=9hE5-98ZeCg',864 mimeType: 'video/*',865 },866 videoMetadata: {867 startOffset: '40s',868 endOffset: '80s',869 }870 },871 {872 text: 'Please summarize the video in 3 sentences.',873 },874 ],875 },876 ];877878 const response = await ai.models.generateContent({879 model,880 contents,881 });882883 console.log(response.text)884885 }886887 await main();888889### Set a custom frame rate890891You can set custom frame rate sampling by passing an `fps` argument to892`videoMetadata`.893894### Python895896 from google import genai897 from google.genai import types898899 # Only for videos of size <20Mb900 video_file_name = "/path/to/your/video.mp4"901 video_bytes = open(video_file_name, 'rb').read()902903 client = genai.Client()904 response = client.models.generate_content(905 model='models/gemini-3.8-flash',906 contents=types.Content(907 parts=[908 types.Part(909 inline_data=types.Blob(910 data=video_bytes,911 mime_type='video/mp4'),912 video_metadata=types.VideoMetadata(fps=5)913 ),914 types.Part(text='Please summarize the video in 3 sentences.')915 ]916 )917 )918919### JavaScript920921 import { GoogleGenAI } from "@google/genai";922923 const ai = new GoogleGenAI({});924925 const myfile = await ai.files.upload({926 file: "path/to/sample.mp4",927 mimeType: "video/mp4",928 });929930 const response = await ai.models.generateContent({931 model: "gemini-3.8-flash",932 contents: [933 {934 fileData: {935 fileUri: myfile.uri,936 mimeType: myfile.mimeType,937 },938 videoMetadata: {939 fps: 5,940 },941 },942 "Please summarize the video in 3 sentences.",943 ],944 });945946 console.log(response.text);947948By default 1 frame per second (FPS) is sampled from the video. You might want to949set low FPS (\< 1) for long videos. This is especially useful for mostly static950videos (e.g. lectures). Use a higher FPS for videos requiring granular temporal951analysis, such as fast-action understanding or high-speed motion tracking.952953## Supported video formats954955Gemini supports the following video format MIME types:956957- `video/mp4`958- `video/mpeg`959- `video/quicktime`960- `video/avi`961- `video/x-flv`962- `video/mpg`963- `video/webm`964- `video/wmv`965- `video/3gpp`966967## Technical details about videos968969- **Supported models and context** : All Gemini models can process video data.970 - Models with a 1M context window can process videos up to 3 hours long by default (at low media resolution), or up to 1 hour long at high media resolution.971- **Processing modes** : Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, 3.5 Flash Lite, and later models support two video processing modes:972 - **Static**: Frames are extracted at 1 FPS and placed into context (default for all models). Audio is processed at 1Kbps (single channel). Timestamps are added every second. Best for short clips or when every frame matters (such as frame-by-frame inspection). Note that fast action sequences might lose detail due to the 1 FPS sampling rate.973 - **Agentic** : The model dynamically navigates the video, loading transcript and/or frames and/or audio on demand. This uses up to 88% fewer tokens for long-form content, though navigation may slightly increase Time to First Token (TTFT) on short clips (\<5 minutes) due to internal reasoning and tool round-trips before generation begins. Responses include `MEDIA_PROCESSING` tool call and response parts to preserve reasoning context across turns. Best for long-form videos to optimize token costs and response quality. Supported on Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite. See [Agentic video understanding](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#agentic-video-understanding) for details.974- **Token calculation (static mode)** : Each second of video is tokenized as follows:975 - Individual frames (sampled at 1 FPS):976 - If `media_resolution` is set to low, frames are tokenized at 66 tokens per frame.977 - Otherwise, frames are tokenized at 258 tokens per frame.978 - Audio: 32 tokens per second.979 - Metadata is also included.980 - Total: Approximately 100 tokens per second of video at default (low) media resolution, or approximately 300 tokens per second of video at high media resolution.981- **Token calculation (agentic mode)** : Token usage varies based on content complexity and the model's navigation strategy. Navigation reasoning tokens generated during video exploration are accounted as **thinking tokens** (`thoughts_token_count`), while frames, audio, and transcript loaded on demand are accounted as tool prompt tokens (`tool_use_prompt_token_count`). Agentic processing typically uses up to 88% fewer total tokens than static processing for long-form content because the model loads only the transcript and/or frames and/or audio it needs to answer the prompt (see the [tokens guide](https://ai.google.dev/gemini-api/docs/generate-content/tokens#video-token-usage)).982- **Media resolution** : Gemini 3 introduces granular control over multimodal vision processing with the `media_resolution` parameter. The `media_resolution` parameter determines the **maximum number of tokens983 allocated per input image or video frame.** Higher resolutions improve the model's ability to read fine text or identify small details, but increase token usage and latency. The `media_resolution` and `media_processing` parameters are independent: you can set both on the same video Part.984985For more details on token calculations, see the986[tokens](https://ai.google.dev/gemini-api/docs/generate-content/tokens) guide.987988- **Timestamp format** : When referring to specific moments in a video within your prompt, use the `MM:SS` format (e.g., `01:15` for 1 minute and 15 seconds).989- **Prompt placement** : If combining text and a single video, place the text prompt *after* the video part in the `contents` array.990- **Timeouts for long requests** : For videos that require extended processing time or complex multi-step reasoning, use streaming (`client.models.generate_content_stream`). Synchronous, non-streaming requests that experience backend retries under high demand can exceed connection or authentication token validity windows, which may surface as unexpected `401 Unauthorized` or timeout errors. Streaming keeps the connection active and surfaces intermediate reasoning and tool call progress.991992## What's next993994- [Media resolution](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution): Control the resolution of video frames to balance quality and token usage.995- [Tokens](https://ai.google.dev/gemini-api/docs/generate-content/tokens): Understand how video content is tokenized in both static and agentic processing modes.996- [System instructions](https://ai.google.dev/gemini-api/docs/generate-content/text-generation#system-instructions): System instructions let you steer the behavior of the model based on your specific needs and use cases.997- [Files API](https://ai.google.dev/gemini-api/docs/files): Learn more about uploading and managing files for use with Gemini.998- [File prompting strategies](https://ai.google.dev/gemini-api/docs/files#prompt-guide): The Gemini API supports prompting with text, image, audio, and video data, also known as multimodal prompting.999- [Safety guidance](https://ai.google.dev/gemini-api/docs/safety-guidance): Sometimes generative AI models produce unexpected outputs, such as outputs that are inaccurate, biased, or offensive. Post-processing and human evaluation are essential to limit the risk of harm from such outputs.