Repository Public
1> To learn about video generation, see the [Gemini Omni Flash](https://ai.google.dev/gemini-api/docs/omni) guide.23Gemini models can process videos, enabling many frontier developer use cases4that would have historically required domain specific models.5Some of Gemini's vision capabilities include the ability to: describe, segment,6and extract information from videos, answer questions about video content, and7refer to specific timestamps within a video.89You can provide videos as input to Gemini in the following ways:1011| Input method | Max size | Recommended use case |12|---|---|---|13| [File API](https://ai.google.dev/gemini-api/docs/video-understanding#upload-video) | 20GB (paid) / 2GB (free) | Large files (100MB+), long videos (10min+), reusable files. |14| [Cloud Storage Registration](https://ai.google.dev/gemini-api/docs/file-input-methods#registration) | 2GB (per file, no storage limits) | Large files (100MB+), long videos (10min+), persistent, reusable files. |15| [Inline Data](https://ai.google.dev/gemini-api/docs/video-understanding#inline-video) | \< 100MB | Small files (\<100MB), short duration (\<1min), one-off inputs. |16| [YouTube URLs](https://ai.google.dev/gemini-api/docs/video-understanding#youtube) | N/A | Public YouTube videos. |1718> **Note:** The [File API](https://ai.google.dev/gemini-api/docs/video-understanding#upload-video) is recommended for most use cases, especially for files larger than 100MB or when you want to reuse the file across multiple requests.1920To learn about other file input methods, such as using external URLs or files21stored in Google Cloud, see the22[File input methods](https://ai.google.dev/gemini-api/docs/file-input-methods) guide.2324### Upload a video file2526The following code downloads a sample video, uploads it using the [Files API](https://ai.google.dev/gemini-api/docs/files),27waits for it to be processed, and then uses the uploaded file reference to28summarize the video.2930### Python3132 from google import genai33 import time3435 client = genai.Client()3637 myfile = client.files.upload(file="path/to/sample.mp4")3839 while not myfile.state or myfile.state.name != "ACTIVE":40 print("Processing video...")41 time.sleep(5)42 myfile = client.files.get(name=myfile.name)4344 interaction = client.interactions.create(45 model="gemini-3.8-flash",46 input=[47 {"type": "video", "uri": myfile.uri, "mime_type": myfile.mime_type},48 {"type": "text", "text": "Summarize this video. Then create a quiz with an answer key based on the information in this video."}49 ]50 )5152 print(interaction.output_text)5354### JavaScript5556 import { GoogleGenAI } from "@google/genai";5758 const ai = new GoogleGenAI({});5960 async function main() {61 const myfile = await ai.files.upload({62 file: "path/to/sample.mp4",63 config: { mimeType: "video/mp4" },64 });6566 let getFile = await ai.files.get({ name: myfile.name });67 while (getFile.state === 'PROCESSING') {68 getFile = await ai.files.get({ name: myfile.name });69 console.log(`current file status: ${getFile.state}`);70 console.log('File is still processing, retrying in 5 seconds');7172 await new Promise((resolve) => {73 setTimeout(resolve, 5000);74 });75 }76 if (getFile.state === 'FAILED') {77 throw new Error('File processing failed.');78 }7980 const interaction = await ai.interactions.create({81 model: "gemini-3.8-flash",82 input: [83 { type: "video", uri: myfile.uri, mime_type: myfile.mimeType },84 { type: "text", text: "Summarize this video. Then create a quiz with an answer key based on the information in this video." }85 ],86 });87 console.log(interaction.output_text);88 }8990 await main();9192### Java9394 import com.google.genai.Client;95 import com.google.genai.gaos.models.interactions.Content;96 import com.google.genai.gaos.models.interactions.CreateModelInteraction;97 import com.google.genai.gaos.models.interactions.Interaction;98 import com.google.genai.gaos.models.interactions.InteractionsInput;99 import com.google.genai.gaos.models.interactions.Model;100 import com.google.genai.gaos.models.interactions.TextContent;101 import com.google.genai.gaos.models.interactions.VideoContent;102 import com.google.genai.gaos.models.interactions.VideoContentMimeType;103 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;104 import com.google.genai.types.File;105 import com.google.genai.types.FileState;106 import com.google.genai.types.UploadFileConfig;107 import java.util.Arrays;108 import java.util.List;109110 Client client = new Client();111112 File myfile =113 client.files.upload(114 "path/to/sample.mp4", UploadFileConfig.builder().mimeType("video/mp4").build());115116 while (!myfile.state().isPresent()117 || myfile.state().get().knownEnum() != FileState.Known.ACTIVE) {118 System.out.println("Processing video...");119 Thread.sleep(5000);120 myfile = client.files.get(myfile.name().get(), null);121 }122123 Content videoContent =124 VideoContent.builder()125 .uri(myfile.uri().get())126 .mimeType(VideoContentMimeType.of(myfile.mimeType().get()))127 .build();128 Content textContent =129 TextContent.builder()130 .text(131 "Summarize this video. Then create a quiz with an answer key based on the information in this video.")132 .build();133134 List<Content> contents = Arrays.asList(videoContent, textContent);135136 CreateModelInteraction params =137 CreateModelInteraction.builder()138 .model(Model.of("gemini-3.8-flash"))139 .input(InteractionsInput.ofContent(contents))140 .build();141142 Interaction interaction =143 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();144145 System.out.println(interaction.outputText().orElse(""));146147### REST148149 VIDEO_PATH="path/to/sample.mp4"150 MIME_TYPE=$(file -b --mime-type "${VIDEO_PATH}")151 NUM_BYTES=$(wc -c < "${VIDEO_PATH}")152 DISPLAY_NAME=VIDEO153154 tmp_header_file=upload-header.tmp155156 echo "Starting file upload..."157 curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \158 -H "x-goog-api-key: $GEMINI_API_KEY" \159 -D ${tmp_header_file} \160 -H "X-Goog-Upload-Protocol: resumable" \161 -H "X-Goog-Upload-Command: start" \162 -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \163 -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \164 -H "Content-Type: application/json" \165 -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null166167 upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")168 rm "${tmp_header_file}"169170 echo "Uploading video data..."171 curl "${upload_url}" \172 -H "Content-Length: ${NUM_BYTES}" \173 -H "X-Goog-Upload-Offset: 0" \174 -H "X-Goog-Upload-Command: upload, finalize" \175 --data-binary "@${VIDEO_PATH}" 2> /dev/null > file_info.json176177 file_uri=$(jq -r ".file.uri" file_info.json)178 file_name=$(jq -r ".file.name" file_info.json)179 echo file_uri=$file_uri180181 echo "File uploaded successfully. File URI: ${file_uri}"182183 # Polling loop184 echo "Waiting for file to be processed..."185 while true; do186 curl -s "https://generativelanguage.googleapis.com/v1beta/${file_name}" \187 -H "x-goog-api-key: $GEMINI_API_KEY" > file_status.json188 state=$(jq -r ".state" file_status.json)189 echo "Current state: $state"190 if [ "$state" == "ACTIVE" ]; then191 break192 elif [ "$state" == "FAILED" ]; then193 echo "File processing failed."194 exit 1195 fi196 sleep 5197 done198199 echo "Generating content from video..."200 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \201 -H "x-goog-api-key: $GEMINI_API_KEY" \202 -H 'Content-Type: application/json' \203 -d '{204 "model": "gemini-3.8-flash",205 "input": [206 {"type": "video", "uri": "'${file_uri}'", "mime_type": "'${MIME_TYPE}'"},207 {"type": "text", "text": "Summarize this video. Then create a quiz with an answer key based on the information in this video."}208 ]209 }' 2> /dev/null > response.json210211 jq ".steps[].content[0].text" response.json212213To optimize token efficiency and performance, consider using214[Agentic video processing](https://ai.google.dev/gemini-api/docs/video-understanding#agentic-video-understanding).215216Always use the Files API when the total request size (including the file, text217prompt, system instructions, etc.) is larger than 20 MB, the video duration is218significant, or if you intend to use the same video in multiple prompts.219The File API accepts video file formats directly.220221To learn more about working with media files, see222[Files API](https://ai.google.dev/gemini-api/docs/files).223224### Pass video data inline225226Instead of uploading a video file using the File API, you can pass smaller227videos directly in the request. This is suitable for228shorter videos under 20MB total request size.229230Here's an example of providing inline video data:231232### Python233234 from google import genai235 import base64236237 video_file_name = "/path/to/your/video.mp4"238 video_bytes = open(video_file_name, 'rb').read()239240 client = genai.Client()241 interaction = client.interactions.create(242 model='gemini-3.8-flash',243 input=[244 {"type": "text", "text": "Please summarize the video in 3 sentences."},245 {246 "type": "video",247 "data": base64.b64encode(video_bytes).decode('utf-8'),248 "mime_type": "video/mp4"249 }250 ]251 )252 print(interaction.output_text)253254### JavaScript255256 import { GoogleGenAI } from "@google/genai";257 import * as fs from "node:fs";258259 const ai = new GoogleGenAI({});260 const base64VideoFile = fs.readFileSync("path/to/small-sample.mp4", {261 encoding: "base64",262 });263264 const interaction = await ai.interactions.create({265 model: "gemini-3.8-flash",266 input: [267 { type: "text", text: "Please summarize the video in 3 sentences." },268 {269 type: "video",270 data: base64VideoFile,271 mime_type: "video/mp4",272 }273 ],274 });275 console.log(interaction.output_text);276277### Java278279 import com.google.genai.Client;280 import com.google.genai.gaos.models.interactions.Content;281 import com.google.genai.gaos.models.interactions.CreateModelInteraction;282 import com.google.genai.gaos.models.interactions.Interaction;283 import com.google.genai.gaos.models.interactions.InteractionsInput;284 import com.google.genai.gaos.models.interactions.Model;285 import com.google.genai.gaos.models.interactions.TextContent;286 import com.google.genai.gaos.models.interactions.VideoContent;287 import com.google.genai.gaos.models.interactions.VideoContentMimeType;288 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;289 import java.nio.file.Files;290 import java.nio.file.Paths;291 import java.util.Arrays;292 import java.util.Base64;293 import java.util.List;294295 String videoFileName = "/path/to/your/video.mp4";296 byte[] videoBytes = Files.readAllBytes(Paths.get(videoFileName));297 String base64Video = Base64.getEncoder().encodeToString(videoBytes);298299 Client client = new Client();300301 Content textContent =302 TextContent.builder().text("Please summarize the video in 3 sentences.").build();303 Content videoContent =304 VideoContent.builder()305 .data(base64Video)306 .mimeType(VideoContentMimeType.VIDEO_MP4)307 .build();308309 List<Content> contents = Arrays.asList(textContent, videoContent);310311 CreateModelInteraction params =312 CreateModelInteraction.builder()313 .model(Model.of("gemini-3.8-flash"))314 .input(InteractionsInput.ofContent(contents))315 .build();316317 Interaction interaction =318 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();319320 System.out.println(interaction.outputText().orElse(""));321322### REST323324> [!NOTE]325> **Note:** If you get an `Argument list too long` error, the base64 encoding of your file might be too long for the curl command line. Use the File API method instead for larger files.326327 VIDEO_PATH=/path/to/your/video.mp4328329 if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then330 B64FLAGS="--input"331 else332 B64FLAGS="-w0"333 fi334335 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \336 -H "x-goog-api-key: $GEMINI_API_KEY" \337 -H 'Content-Type: application/json' \338 -d '{339 "model": "gemini-3.8-flash",340 "input": [341 {"type": "text", "text": "Please summarize the video in 3 sentences."},342 {343 "type": "video",344 "data": "'$(base64 $B64FLAGS $VIDEO_PATH)'",345 "mime_type": "video/mp4"346 }347 ]348 }' 2> /dev/null349350### Pass YouTube URLs351352> [!WARNING]353> **Preview:** The YouTube URL feature is in preview and is available at no charge. Pricing and rate limits are likely to change.354355You can pass YouTube URLs directly to Gemini API as part of your request as follows:356357### Python358359 from google import genai360361 client = genai.Client()362 interaction = client.interactions.create(363 model='gemini-3.8-flash',364 input=[365 {"type": "text", "text": "Please summarize the video in 3 sentences."},366 {367 "type": "video",368 "uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg"369 }370 ]371 )372 print(interaction.output_text)373374### JavaScript375376 import { GoogleGenAI } from "@google/genai";377378 const ai = new GoogleGenAI({});379380 const interaction = await ai.interactions.create({381 model: "gemini-3.8-flash",382 input: [383 { type: "text", text: "Please summarize the video in 3 sentences." },384 {385 type: "video",386 uri: "https://www.youtube.com/watch?v=9hE5-98ZeCg",387 }388 ],389 });390 console.log(interaction.output_text);391392### Java393394 import com.google.genai.Client;395 import com.google.genai.gaos.models.interactions.Content;396 import com.google.genai.gaos.models.interactions.CreateModelInteraction;397 import com.google.genai.gaos.models.interactions.Interaction;398 import com.google.genai.gaos.models.interactions.InteractionsInput;399 import com.google.genai.gaos.models.interactions.Model;400 import com.google.genai.gaos.models.interactions.TextContent;401 import com.google.genai.gaos.models.interactions.VideoContent;402 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;403 import java.util.Arrays;404 import java.util.List;405406 Client client = new Client();407408 Content textContent =409 TextContent.builder().text("Please summarize the video in 3 sentences.").build();410 Content videoContent =411 VideoContent.builder()412 .uri("https://www.youtube.com/watch?v=9hE5-98ZeCg")413 .build();414415 List<Content> contents = Arrays.asList(textContent, videoContent);416417 CreateModelInteraction params =418 CreateModelInteraction.builder()419 .model(Model.of("gemini-3.8-flash"))420 .input(InteractionsInput.ofContent(contents))421 .build();422423 Interaction interaction =424 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();425426 System.out.println(interaction.outputText().orElse(""));427428### REST429430 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \431 -H "x-goog-api-key: $GEMINI_API_KEY" \432 -H 'Content-Type: application/json' \433 -d '{434 "model": "gemini-3.8-flash",435 "input": [436 {"type": "text", "text": "Please summarize the video in 3 sentences."},437 {438 "type": "video",439 "uri": "https://www.youtube.com/watch?v=9hE5-98ZeCg"440 }441 ]442 }' 2> /dev/null443444**Limitations:**445446- For the free tier, you can't upload more than 8 hours of YouTube video per day.447- For the paid tier, there is no limit based on video length.448- For models prior to Gemini 2.5, you can upload only 1 video per request. For Gemini 2.5 and later models, you can upload a maximum of 10 videos per request.449- You can only upload public videos (not private or unlisted videos).450451## Agentic video understanding452453By default, video inputs use static processing (extracting frames at 1 FPS).454Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite models also support455**agentic video understanding**, where the model dynamically explores the video456timeline, selectively inspecting transcripts and adaptively adjusting frame457rates and resolution on the fly based on the prompt.458459| **Mode** | **Description** | **Supported models** |460|---|---|---|461| **Static** (default) | Extracts frames at a fixed rate (1 FPS) and places them into context in a single pass. Works well for short clips. | All Gemini models |462| **Agentic** | The model dynamically navigates the video timeline, loading only the content it needs based on the prompt. Up to 88% more token-efficient and \~7% higher quality on long-form content. | Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, 3.5 Flash Lite |463464### Choose a processing mode465466As a general guideline, start with **agentic** mode, especially when optimizing467for response quality or token efficiency.468469- **Agentic:** Long-form videos or queries targeting specific moments. The model dynamically navigates the timeline to target contextually relevant information without filling the context window.470- **Static:** Latency-sensitive queries on short clips (under 5 minutes), or cases where frame-level precision across the entire clip is needed.471472> **Note:** For long videos or complex prompts where agentic processing takes473> more time, use streaming (`stream=True`) or background execution474> (`background=True`). This keeps the connection active, surfaces intermediate475> reasoning steps, and avoids connection or authentication timeouts.476477### Set the processing mode478479### Python480481 import time482 from google import genai483484 client = genai.Client()485486 # Upload a long video487 video_file = client.files.upload(file="path/to/lecture.mp4")488489 while video_file.state.name == "PROCESSING":490 time.sleep(2)491 video_file = client.files.get(name=video_file.name)492493 # Use agentic processing494 interaction = client.interactions.create(495 model="gemini-3.8-flash",496 input=[497 {498 "type": "video",499 "uri": video_file.uri,500 "mime_type": video_file.mime_type,501 "processing": "agentic"502 },503 {"type": "text", "text": "What are the three main arguments presented?"}504 ]505 )506 print(interaction.output_text)507508### JavaScript509510 import { GoogleGenAI } from "@google/genai";511512 const ai = new GoogleGenAI({});513514 // Upload a long video515 let videoFile = await ai.files.upload({516 file: "path/to/lecture.mp4",517 config: { mimeType: "video/mp4" }518 });519520 while (videoFile.state === "PROCESSING") {521 await new Promise((resolve) => setTimeout(resolve, 2000));522 videoFile = await ai.files.get({ name: videoFile.name });523 }524525 // Use agentic processing526 const interaction = await ai.interactions.create({527 model: "gemini-3.8-flash",528 input: [529 {530 type: "video",531 uri: videoFile.uri,532 mime_type: videoFile.mimeType,533 processing: "agentic"534 },535 { type: "text", text: "What are the three main arguments presented?" }536 ]537 });538 console.log(interaction.output_text);539540### REST541542 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \543 -H "x-goog-api-key: $GEMINI_API_KEY" \544 -H 'Content-Type: application/json' \545 -d '{546 "model": "gemini-3.8-flash",547 "input": [548 {549 "type": "video",550 "uri": "'${file_uri}'",551 "mime_type": "video/mp4",552 "processing": "agentic"553 },554 {"type": "text", "text": "What are the three main arguments presented?"}555 ]556 }' 2> /dev/null557558> **Note:** To verify that agentic processing was used, inspect `interaction.steps`. The presence of `processing_call` and `processing_result` indicates that the model dynamically navigated the video.559560### Response steps561562Agentic processing adds two new step types to the `steps` array:563564- `processing_call`: the model requested a video segment or audio transcript, identified by `id`.565- `processing_result`: the result of that load, linked by `call_id`.566567These appear interleaved with `thought` steps (when summaries are enabled) and precede the final `model_output` step. They can be used to show a progress trace in your UI but do not require a response.568569The following example shows the response payload with interleaved processing steps:570571 {572 "steps": [573 {574 "type": "thought",575 "signature": "sig_thought_1",576 "summary": [577 {578 "type": "text",579 "text": "Inspecting transcript for key discussion topics..."580 }581 ]582 },583 {584 "type": "processing_call",585 "id": "call_01",586 "signature": "sig_call_01"587 },588 {589 "type": "processing_result",590 "call_id": "call_01",591 "signature": "sig_result_01"592 },593 {594 "type": "thought",595 "signature": "sig_thought_2",596 "summary": [597 {598 "type": "text",599 "text": "Loading visual frames to verify slide content..."600 }601 ]602 },603 {604 "type": "processing_call",605 "id": "call_02",606 "signature": "sig_call_02"607 },608 {609 "type": "processing_result",610 "call_id": "call_02",611 "signature": "sig_result_02"612 },613 {614 "type": "thought",615 "signature": "sig_thought_3",616 "summary": [617 {618 "type": "text",619 "text": "Synthesizing answer from gathered evidence..."620 }621 ]622 },623 {624 "type": "model_output",625 "content": [626 {627 "type": "text",628 "text": "The three main arguments presented in the lecture are..."629 }630 ]631 }632 ]633 }634635### Mix processing modes across videos636637You can set different processing modes for each video in the same request:638639### Python640641 from google import genai642643 client = genai.Client()644645 lecture = client.files.upload(file="path/to/long-lecture.mp4")646 experiment = client.files.upload(file="path/to/short-experiment.mp4")647648 interaction = client.interactions.create(649 model="gemini-3.8-flash",650 input=[651 {652 "type": "video",653 "uri": lecture.uri,654 "mime_type": lecture.mime_type,655 "processing": "agentic" # Use agentic video understanding656 },657 {658 "type": "video",659 "uri": experiment.uri,660 "mime_type": experiment.mime_type,661 "processing": "static" # Use static processing662 },663 {"type": "text", "text": "Compare the lecture content with the experiment results."}664 ]665 )666 print(interaction.output_text)667668### JavaScript669670 import { GoogleGenAI } from "@google/genai";671672 const ai = new GoogleGenAI({});673674 const lecture = await ai.files.upload({675 file: "path/to/long-lecture.mp4",676 config: { mimeType: "video/mp4" }677 });678 const experiment = await ai.files.upload({679 file: "path/to/short-experiment.mp4",680 config: { mimeType: "video/mp4" }681 });682683 const interaction = await ai.interactions.create({684 model: "gemini-3.8-flash",685 input: [686 {687 type: "video",688 uri: lecture.uri,689 mime_type: lecture.mimeType,690 processing: "agentic" // Use agentic video understanding691 },692 {693 type: "video",694 uri: experiment.uri,695 mime_type: experiment.mimeType,696 processing: "static" // Use static processing697 },698 { type: "text", text: "Compare the lecture content with the experiment results." }699 ]700 });701 console.log(interaction.output_text);702703### REST704705 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \706 -H "x-goog-api-key: $GEMINI_API_KEY" \707 -H 'Content-Type: application/json' \708 -d '{709 "model": "gemini-3.8-flash",710 "input": [711 {712 "type": "video",713 "uri": "'${lecture_uri}'",714 "mime_type": "video/mp4",715 "processing": "agentic"716 },717 {718 "type": "video",719 "uri": "'${experiment_uri}'",720 "mime_type": "video/mp4",721 "processing": "static"722 },723 {"type": "text", "text": "Compare the lecture content with the experiment results."}724 ]725 }' 2> /dev/null726727### Multi-turn video conversations728729Video context is preserved across turns in a conversation. When using agentic730processing:731732- **Stateful mode** (using `previous_interaction_id`): The server retains the video context. No additional handling is needed.733- **Stateless mode** (using `step_list`): In stateless mode, the response includes `processing_call` and `processing_result` steps that encode the video context. You must include all steps from the response in your next request's `step_list` to preserve video context. While omitting them does not currently return an API error, the video context is lost, significantly reducing response quality on follow-up questions. Note that returned steps sent in subsequent requests contribute to input token counts.734735## Refer to timestamps in the content736737You can ask questions about specific points in time within the video using738timestamps of the form `MM:SS`.739740### Python741742 prompt = "What are the examples given at 00:05 and 00:10 supposed to show us?"743744### JavaScript745746 const prompt = "What are the examples given at 00:05 and 00:10 supposed to show us?";747748### Java749750 String prompt = "What are the examples given at 00:05 and 00:10 supposed to show us?";751752### REST753754 PROMPT="What are the examples given at 00:05 and 00:10 supposed to show us?"755756## Extract detailed insights from video757758Gemini models offer powerful capabilities for understanding video content by759processing information from both the **audio and visual** streams. This lets you760extract a rich set of details, including generating descriptions of what is761happening in a video and answering questions about its content.762763For visual descriptions, the model samples the video at a rate of **1 frame764per second** (FPS). This default sampling rate works well for most content, but765note that it may miss details in videos with rapid motion or quick scene changes.766767### Python768769 prompt = "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments."770771### JavaScript772773 const prompt = "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments.";774775### Java776777 String prompt =778 "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments.";779780### REST781782 PROMPT="Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments."783784## Customize video processing785786You can customize video processing in the Gemini API by setting clipping787intervals or providing custom frame rate sampling. These customization options788are only supported when processing the video in `"static"` mode.789790### Set clipping intervals791792You can clip video by specifying `start_offset` and `end_offset` in the `processing` configuration object.793794### Python795796 interaction = client.interactions.create(797 model="gemini-3.8-flash",798 input=[799 {800 "type": "video",801 "uri": video_file.uri,802 "mime_type": video_file.mime_type,803 "processing": {804 "type": "static",805 "start_offset": 1200,806 "end_offset": 1500,807 },808 },809 {"type": "text", "text": "Summarize this section of the video."},810 ],811 )812 print(interaction.output_text)813814### JavaScript815816 const interaction = await ai.interactions.create({817 model: "gemini-3.8-flash",818 input: [819 {820 type: "video",821 uri: videoFile.uri,822 mime_type: videoFile.mimeType,823 processing: {824 type: "static",825 start_offset: 1200,826 end_offset: 1500,827 },828 },829 { type: "text", text: "Summarize this section of the video." },830 ],831 });832 console.log(interaction.output_text);833834### REST835836 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \837 -H "x-goog-api-key: $GEMINI_API_KEY" \838 -H 'Content-Type: application/json' \839 -d '{840 "model": "gemini-3.8-flash",841 "input": [842 {843 "type": "video",844 "uri": "'${file_uri}'",845 "mime_type": "video/mp4",846 "processing": {847 "type": "static",848 "start_offset": 1200,849 "end_offset": 1500850 }851 },852 {"type": "text", "text": "Summarize this section of the video."}853 ]854 }' 2> /dev/null855856### Set a custom frame rate857858You can set custom frame rate sampling by passing an `fps` argument in the `processing` configuration object.859860### Python861862 interaction = client.interactions.create(863 model="gemini-3.8-flash",864 input=[865 {866 "type": "video",867 "uri": video_file.uri,868 "mime_type": video_file.mime_type,869 "processing": {870 "type": "static",871 "fps": 0.5, # Sample 1 frame every 2 seconds872 },873 },874 {"type": "text", "text": "Describe the scene changes in this video."},875 ],876 )877 print(interaction.output_text)878879### JavaScript880881 const interaction = await ai.interactions.create({882 model: "gemini-3.8-flash",883 input: [884 {885 type: "video",886 uri: videoFile.uri,887 mime_type: videoFile.mimeType,888 processing: {889 type: "static",890 fps: 0.5, // Sample 1 frame every 2 seconds891 },892 },893 { type: "text", text: "Describe the scene changes in this video." },894 ],895 });896 console.log(interaction.output_text);897898### REST899900 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \901 -H "x-goog-api-key: $GEMINI_API_KEY" \902 -H 'Content-Type: application/json' \903 -d '{904 "model": "gemini-3.8-flash",905 "input": [906 {907 "type": "video",908 "uri": "'${file_uri}'",909 "mime_type": "video/mp4",910 "processing": {911 "type": "static",912 "fps": 0.5913 }914 },915 {"type": "text", "text": "Describe the scene changes in this video."}916 ]917 }' 2> /dev/null918919## Supported video formats920921Gemini supports the following video format MIME types:922923- `video/mp4`924- `video/mpeg`925- `video/mov`926- `video/avi`927- `video/x-flv`928- `video/mpg`929- `video/webm`930- `video/wmv`931- `video/3gpp`932933## Technical details about videos934935- **Supported models and context** : All Gemini models can process video data.936 - Models with a 1M context window can process videos up to 3 hours long by default (at low media resolution), or up to 1 hour long at high media resolution.937- **Processing modes** : Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, 3.5 Flash Lite, and later models support two video processing modes:938 - **Static**: Frames are extracted at 1 FPS and placed into context (default for all models). Audio is processed at 1Kbps (single channel). Timestamps are added every second. Best for short clips or when every frame matters (such as frame-by-frame inspection). Note that fast action sequences might lose detail due to the 1 FPS sampling rate.939 - **Agentic** : The model dynamically navigates the video, loading transcript and/or frames and/or audio on demand. This uses up to 88% fewer tokens for long-form content, though navigation may slightly increase Time to First Token (TTFT) on short clips (\<5 minutes) due to internal reasoning and tool round-trips before generation begins. Best for long-form videos to optimize token costs and response quality. Supported on Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite. See [Agentic video understanding](https://ai.google.dev/gemini-api/docs/video-understanding#agentic-video-understanding) for details.940- **Token calculation (static mode)** : Each second of video is tokenized as follows:941 - Individual frames (sampled at 1 FPS):942 - If `media_resolution` is set to low, frames are tokenized at 66 tokens per frame.943 - Otherwise, frames are tokenized at 258 tokens per frame.944 - Audio: 32 tokens per second.945 - Metadata is also included.946 - Total: Approximately 100 tokens per second of video at default (low) media resolution, or approximately 300 tokens per second of video at high media resolution.947- **Token calculation (agentic mode)** : Token usage varies based on content complexity and the model's navigation strategy. Navigation reasoning tokens generated during video exploration are accounted as **thought tokens** (`total_thought_tokens`), while frames, audio, and transcript loaded on demand are accounted as tool use tokens (`total_tool_use_tokens`). Agentic processing typically uses up to 88% fewer total tokens than static processing for long-form content because the model loads only the transcript and/or frames and/or audio it needs to answer the prompt (see the [tokens guide](https://ai.google.dev/gemini-api/docs/tokens#video-token-usage)).948- **Media resolution** : Gemini 3 introduces granular control over multimodal vision processing with the `media_resolution` parameter. The `media_resolution` parameter determines the **maximum number of tokens949 allocated per input image or video frame.** Higher resolutions improve the model's ability to read fine text or identify small details, but increase token usage and latency. The `media_resolution` and `processing` parameters are independent: you can set both on the same video input.950951For more details on token calculations, see the952[tokens](https://ai.google.dev/gemini-api/docs/tokens) guide.953954- **Timestamp format** : When referring to specific moments in a video within your prompt, use the `MM:SS` format (e.g., `01:15` for 1 minute and 15 seconds).955- **Prompt placement** : If combining text and a single video, place the text prompt *after* the video part in the `input` array.956- **Timeouts for long requests** : For videos that require extended processing time or complex multi-step reasoning, use streaming (`stream=True`) or background execution (`background=True`). Synchronous, non-streaming requests that experience backend retries under high demand can exceed connection or authentication token validity windows, which may surface as unexpected `401 Unauthorized` or timeout errors. Streaming keeps the connection active and surfaces intermediate reasoning and tool call progress.957958## What's next959960- [Media resolution](https://ai.google.dev/gemini-api/docs/media-resolution): Control the resolution of video frames to balance quality and token usage.961- [Tokens](https://ai.google.dev/gemini-api/docs/tokens): Understand how video content is tokenized in both static and agentic processing modes.962- [System instructions](https://ai.google.dev/gemini-api/docs/text-generation#system-instructions): System instructions let you steer the behavior of the model based on your specific needs and use cases.963- [Files API](https://ai.google.dev/gemini-api/docs/files): Learn more about uploading and managing files for use with Gemini.964- [File prompting strategies](https://ai.google.dev/gemini-api/docs/files#prompt-guide): The Gemini API supports prompting with text, image, audio, and video data, also known as multimodal prompting.965- [Safety guidance](https://ai.google.dev/gemini-api/docs/safety-guidance): Sometimes generative AI models produce unexpected outputs, such as outputs that are inaccurate, biased, or offensive. Post-processing and human evaluation are essential to limit the risk of harm from such outputs.