Repository Public
1Gemini Omni Flash (`gemini-omni-1.1-flash`) is a high-performance multimodal2model designed for high-speed video generation, editing, and cinematic control.3Gemini Omni is built on the following core capabilities that distinguish it from4previous video models:56- **Native multimodality:** it processes text, image, audio, and video simultaneously, giving you more cohesive, consistent, and controllable output.7- **Conversational editing:** enabled by the [Interactions8 API](https://ai.google.dev/gemini-api/docs/interactions-overview), it lets you iteratively refine and edit your videos through natural language conversation. Describe what you want to change, and the model applies the edit while preserving the parts of the video you want to keep.9- **World knowledge:** Gemini Omni combines an understanding of physics with Gemini's knowledge of history, science, and cultural context, bridging the gap from photorealism to meaningful storytelling.1011## Text to video generation1213Generate a video from a text prompt. The model generates a video with audio14based on your text description. Write prompts with details like scene description,15camera movement, lighting and mood for best results.1617### Python1819 import base6420 from google import genai2122 client = genai.Client()2324 interaction = client.interactions.create(25 model="gemini-omni-1.1-flash",26 input="A marble rolling fast on a chain reaction style track, continuous smooth shot."27 )28 with open("marble.mp4", "wb") as f:29 f.write(base64.b64decode(interaction.output_video.data))3031### JavaScript3233 import { GoogleGenAI } from '@google/genai';34 import * as fs from 'fs';35 const ai = new GoogleGenAI({});3637 const interaction = await ai.interactions.create({38 model: 'gemini-omni-1.1-flash',39 input: 'A marble rolling fast on a chain reaction style track, continuous smooth shot.',40 });4142 if (interaction.output_video?.data) {43 fs.writeFileSync('marble.mp4', Buffer.from(interaction.output_video.data, 'base64'));44 }4546### Java4748 import com.google.genai.Client;49 import com.google.genai.gaos.models.interactions.CreateModelInteraction;50 import com.google.genai.gaos.models.interactions.Interaction;51 import com.google.genai.gaos.models.interactions.InteractionsInput;52 import com.google.genai.gaos.models.interactions.Model;53 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;54 import java.nio.file.Files;55 import java.nio.file.Paths;56 import java.util.Base64;5758 Client client = new Client();5960 CreateModelInteraction params =61 CreateModelInteraction.builder()62 .model(Model.of("gemini-omni-1.1-flash"))63 .input(64 InteractionsInput.of(65 "A marble rolling fast on a chain reaction style track, continuous smooth shot."))66 .build();6768 Interaction interaction =69 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();7071 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {72 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());73 Files.write(Paths.get("marble.mp4"), videoBytes);74 }7576### REST7778 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \79 -H "Content-Type: application/json" \80 -d '{81 "model": "gemini-omni-1.1-flash",82 "input": "A marble rolling fast on a chain reaction style track, continuous smooth shot."83 }'8485[Video](https://www.youtube.com/watch?v=6DRkXO9nM7g)8687### REST response schema8889The convenience field `interaction.output_video` is **SDK-only** .90Get the video output from the `steps` array when using the REST API directly.9192**Raw REST JSON structure:**9394 {95 "steps": [96 { "type": "user_input", "content": [{"type": "text", "text": "..."}] },97 { "type": "thought", "content": [{"text": "...", "type": "thought"}] },98 {99 "type": "model_output",100 "content": [101 {102 "type": "video",103 "mime_type": "video/mp4",104 "data": "AAAAIGZ0eXBpc29t..." // Base64 encoded video data105 }106 ]107 }108 ],109 "id": "v1_...",110 "status": "completed",111 "model": "gemini-omni-1.1-flash",112 "object": "interaction"113 }114115### Control aspect ratio116117Set the `aspect_ratio` to `"9:16"` to create portrait videos. Landscape (16:9)118is the default.119120### Python121122 import base64123 from google import genai124125 client = genai.Client()126127 interaction = client.interactions.create(128 model="gemini-omni-1.1-flash",129 input="A futuristic city with neon lights and flying cars, cyberpunk style",130 response_format={131 "type": "video", # optional132 "aspect_ratio": "9:16" # Supported values: "9:16", "16:9"133 }134 )135 with open("example.mp4", "wb") as f:136 f.write(base64.b64decode(interaction.output_video.data))137138### JavaScript139140 import { GoogleGenAI } from '@google/genai';141 import * as fs from 'fs';142 const ai = new GoogleGenAI({});143144 const interaction = await ai.interactions.create({145 model: 'gemini-omni-1.1-flash',146 input: 'A futuristic city with neon lights and flying cars, cyberpunk style',147 response_format: {148 type: 'video', // optional149 aspect_ratio: '9:16' // Supported values: '9:16', '16:9'150 },151 });152153 if (interaction.output_video?.data) {154 fs.writeFileSync('example.mp4', Buffer.from(interaction.output_video.data, 'base64'));155 }156157### Java158159 import com.google.genai.Client;160 import com.google.genai.gaos.models.interactions.CreateModelInteraction;161 import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;162 import com.google.genai.gaos.models.interactions.Interaction;163 import com.google.genai.gaos.models.interactions.InteractionsInput;164 import com.google.genai.gaos.models.interactions.Model;165 import com.google.genai.gaos.models.interactions.ResponseFormat;166 import com.google.genai.gaos.models.interactions.VideoResponseFormat;167 import com.google.genai.gaos.models.interactions.VideoResponseFormatAspectRatio;168 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;169 import java.nio.file.Files;170 import java.nio.file.Paths;171 import java.util.Base64;172173 Client client = new Client();174175 VideoResponseFormat videoFormat =176 VideoResponseFormat.builder()177 .aspectRatio(VideoResponseFormatAspectRatio.of("9:16"))178 .build();179180 CreateModelInteraction params =181 CreateModelInteraction.builder()182 .model(Model.of("gemini-omni-1.1-flash"))183 .input(184 InteractionsInput.of(185 "A futuristic city with neon lights and flying cars, cyberpunk style"))186 .responseFormat(CreateModelInteractionResponseFormat.of(ResponseFormat.of(videoFormat)))187 .build();188189 Interaction interaction =190 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();191192 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {193 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());194 Files.write(Paths.get("example.mp4"), videoBytes);195 }196197### REST198199 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \200 -H "Content-Type: application/json" \201 -d '{202 "model": "gemini-omni-1.1-flash",203 "input": "A futuristic city with neon lights and flying cars, cyberpunk style",204 "response_format": {205 "type": "video",206 "aspect_ratio": "9:16"207 }208 }'209210### Output resolution211212Control the output resolution of your generated video using the `resolution`213parameter in `response_format`. The default resolution is 720p.214215| Value | Description |216|---|---|217| `360p` | 360p output resolution |218| `720p` | 720p output resolution (default) |219| `1080p` | 1080p output (upscaled) |220| `4k` | 4K output (upscaled) |221222### Python223224 import base64225 from google import genai226227 client = genai.Client()228229 interaction = client.interactions.create(230 model="gemini-omni-1.1-flash",231 input="A drone shot of a mountain landscape at sunrise.",232 response_format={233 "type": "video",234 "resolution": "1080p",235 },236 )237 with open("hires.mp4", "wb") as f:238 f.write(base64.b64decode(interaction.output_video.data))239240### JavaScript241242 import { GoogleGenAI } from '@google/genai';243 import * as fs from 'fs';244 const ai = new GoogleGenAI({});245246 const interaction = await ai.interactions.create({247 model: 'gemini-omni-1.1-flash',248 input: 'A drone shot of a mountain landscape at sunrise.',249 response_format: {250 type: 'video',251 resolution: '1080p',252 },253 });254255 if (interaction.output_video?.data) {256 fs.writeFileSync('hires.mp4', Buffer.from(interaction.output_video.data, 'base64'));257 }258259### Java260261 import com.google.genai.Client;262 import com.google.genai.gaos.models.interactions.CreateModelInteraction;263 import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;264 import com.google.genai.gaos.models.interactions.Interaction;265 import com.google.genai.gaos.models.interactions.InteractionsInput;266 import com.google.genai.gaos.models.interactions.Model;267 import com.google.genai.gaos.models.interactions.Resolution;268 import com.google.genai.gaos.models.interactions.ResponseFormat;269 import com.google.genai.gaos.models.interactions.VideoResponseFormat;270 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;271 import java.nio.file.Files;272 import java.nio.file.Paths;273 import java.util.Base64;274275 Client client = new Client();276277 VideoResponseFormat videoFormat =278 VideoResponseFormat.builder()279 .resolution(Resolution.of("1080p"))280 .build();281282 CreateModelInteraction params =283 CreateModelInteraction.builder()284 .model(Model.of("gemini-omni-1.1-flash"))285 .input(InteractionsInput.of("A drone shot of a mountain landscape at sunrise."))286 .responseFormat(CreateModelInteractionResponseFormat.of(ResponseFormat.of(videoFormat)))287 .build();288289 Interaction interaction =290 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();291292 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {293 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());294 Files.write(Paths.get("hires.mp4"), videoBytes);295 }296297### REST298299 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \300 -H "Content-Type: application/json" \301 -d '{302 "model": "gemini-omni-1.1-flash",303 "input": "A drone shot of a mountain landscape at sunrise.",304 "response_format": {305 "type": "video",306 "resolution": "1080p"307 }308 }'309310Your browser does not support the video tag.311312## Image to video generation313314You can provide a reference image with your text prompt. Depending on your315prompt, the model will decide316how to use the image. This is useful for bringing product shots, illustrations,317or photographs to life.318319The following example shows how to use the reference image of a drawing of a320fish jumping out of water:321322323With the following prompt:324325 turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video326327To generate a realistic video of the drawing.328329### Python330331 import base64332 from google import genai333334 client = genai.Client()335336 interaction = client.interactions.create(337 model="gemini-omni-1.1-flash",338 input=[339 {"type": "image", "data": base64_image, "mime_type": "image/jpeg"},340 {"type": "text", "text": "turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video"}341 ],342 )343 with open("clownfish.mp4", "wb") as f:344 f.write(base64.b64decode(interaction.output_video.data))345346### JavaScript347348 import { GoogleGenAI } from '@google/genai';349 import * as fs from 'fs';350 const ai = new GoogleGenAI({});351352 const interaction = await ai.interactions.create({353 model: 'gemini-omni-1.1-flash',354 input: [355 { type: 'image', data: base64Image, mime_type: 'image/jpeg' },356 { type: 'text', text: 'turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video' }357 ]358 });359360 if (interaction.output_video?.data) {361 fs.writeFileSync('clownfish.mp4', Buffer.from(interaction.output_video.data, 'base64'));362 }363364### Java365366 import com.google.genai.Client;367 import com.google.genai.gaos.models.interactions.Content;368 import com.google.genai.gaos.models.interactions.CreateModelInteraction;369 import com.google.genai.gaos.models.interactions.ImageContent;370 import com.google.genai.gaos.models.interactions.ImageContentMimeType;371 import com.google.genai.gaos.models.interactions.Interaction;372 import com.google.genai.gaos.models.interactions.InteractionsInput;373 import com.google.genai.gaos.models.interactions.Model;374 import com.google.genai.gaos.models.interactions.TextContent;375 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;376 import java.nio.file.Files;377 import java.nio.file.Paths;378 import java.util.Arrays;379 import java.util.Base64;380 import java.util.List;381382 Client client = new Client();383384 byte[] imageBytes = Files.readAllBytes(Paths.get("drawing.jpg"));385 String base64Image = Base64.getEncoder().encodeToString(imageBytes);386387 Content imageContent =388 ImageContent.builder()389 .data(base64Image)390 .mimeType(ImageContentMimeType.IMAGE_JPEG)391 .build();392393 Content textContent =394 TextContent.builder()395 .text(396 "turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video")397 .build();398399 List<Content> contents = Arrays.asList(imageContent, textContent);400401 CreateModelInteraction params =402 CreateModelInteraction.builder()403 .model(Model.of("gemini-omni-1.1-flash"))404 .input(InteractionsInput.ofContent(contents))405 .build();406407 Interaction interaction =408 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();409410 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {411 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());412 Files.write(Paths.get("clownfish.mp4"), videoBytes);413 }414415### REST416417 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \418 -H "Content-Type: application/json" \419 -d '{420 "model": "gemini-omni-1.1-flash",421 "input": [422 {"type": "image", "data": "'"$BASE64_IMAGE"'", "mime_type": "image/jpeg"},423 {"type": "text", "text": "turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video"}424 ]425 }'426427[Video](https://www.youtube.com/watch?v=zTNIIZ0X7mA)428429> [!NOTE]430> **Note:** For best results with image-to-video, use high-resolution images and provide specific motion descriptions. Vague prompts like "make it move" produce less compelling results than detailed descriptions of the camera movement, subject motion, and environmental effects.431432### First and last frame interpolation433434Gemini Omni Flash supports video interpolation, allowing you to generate a435video that transitions smoothly between a starting image (first frame) and an436ending image (last frame).437438Provide two images in the `input` list and describe the desired transition in439your prompt. The model will animate the scene from the first frame to the440ending frame.441442### Python443444 import base64445 from google import genai446447 client = genai.Client()448449 interaction = client.interactions.create(450 model="gemini-omni-1.1-flash",451 input=[452 {"type": "image", "data": first_frame_b64, "mime_type": "image/jpeg"},453 {"type": "image", "data": last_frame_b64, "mime_type": "image/jpeg"},454 {"type": "text", "text": "A smooth cinematic transition from a lush green forest at sunrise to a snowy forest under a starry night sky."}455 ],456 )457 with open("interpolation.mp4", "wb") as f:458 f.write(base64.b64decode(interaction.output_video.data))459460### JavaScript461462 import { GoogleGenAI } from '@google/genai';463 import * as fs from 'fs';464 const ai = new GoogleGenAI({});465466 const interaction = await ai.interactions.create({467 model: 'gemini-omni-1.1-flash',468 input: [469 { type: 'image', data: firstFrameB64, mime_type: 'image/jpeg' },470 { type: 'image', data: lastFrameB64, mime_type: 'image/jpeg' },471 { type: 'text', text: 'A smooth cinematic transition from a lush green forest at sunrise to a snowy forest under a starry night sky.' }472 ]473 });474475 if (interaction.output_video?.data) {476 fs.writeFileSync('interpolation.mp4', Buffer.from(interaction.output_video.data, 'base64'));477 }478479### Java480481 import com.google.genai.Client;482 import com.google.genai.gaos.models.interactions.Content;483 import com.google.genai.gaos.models.interactions.CreateModelInteraction;484 import com.google.genai.gaos.models.interactions.ImageContent;485 import com.google.genai.gaos.models.interactions.ImageContentMimeType;486 import com.google.genai.gaos.models.interactions.Interaction;487 import com.google.genai.gaos.models.interactions.InteractionsInput;488 import com.google.genai.gaos.models.interactions.Model;489 import com.google.genai.gaos.models.interactions.TextContent;490 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;491 import java.nio.file.Files;492 import java.nio.file.Paths;493 import java.util.Arrays;494 import java.util.Base64;495 import java.util.List;496497 Client client = new Client();498499 String firstFrameB64 =500 Base64.getEncoder().encodeToString(Files.readAllBytes(Paths.get("first_frame.jpg")));501 String lastFrameB64 =502 Base64.getEncoder().encodeToString(Files.readAllBytes(Paths.get("last_frame.jpg")));503504 Content firstFrame =505 ImageContent.builder()506 .data(firstFrameB64)507 .mimeType(ImageContentMimeType.IMAGE_JPEG)508 .build();509510 Content lastFrame =511 ImageContent.builder()512 .data(lastFrameB64)513 .mimeType(ImageContentMimeType.IMAGE_JPEG)514 .build();515516 Content prompt =517 TextContent.builder()518 .text(519 "A smooth cinematic transition from a lush green forest at sunrise to a snowy forest under a starry night sky.")520 .build();521522 List<Content> contents = Arrays.asList(firstFrame, lastFrame, prompt);523524 CreateModelInteraction params =525 CreateModelInteraction.builder()526 .model(Model.of("gemini-omni-1.1-flash"))527 .input(InteractionsInput.ofContent(contents))528 .build();529530 Interaction interaction =531 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();532533 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {534 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());535 Files.write(Paths.get("interpolation.mp4"), videoBytes);536 }537538### REST539540 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \541 -H "Content-Type: application/json" \542 -d '{543 "model": "gemini-omni-1.1-flash",544 "input": [545 {"type": "image", "data": "'"$FIRST_FRAME_B64"'", "mime_type": "image/jpeg"},546 {"type": "image", "data": "'"$LAST_FRAME_B64"'", "mime_type": "image/jpeg"},547 {"type": "text", "text": "A smooth cinematic transition from a lush green forest at sunrise to a snowy forest under a starry night sky."}548 ]549 }'550551Your browser does not support the video tag.552553### Subject reference554555You can generate a video incorporating specific subjects provided as reference images.556For example, the following code shows how to provide 2 images of a cat and yarn557to generate a video of the cat playing with the yarn.558559### Python560561 import base64562 from google import genai563564 client = genai.Client()565566 interaction = client.interactions.create(567 model="gemini-omni-1.1-flash",568 input=[569 {"type": "image", "data": cat_b64, "mime_type": "image/png"},570 {"type": "image", "data": yarn_b64, "mime_type": "image/png"},571 {"type": "text", "text": "A cat playfully batting at a ball of yarn."}572 ],573 )574 with open("cat.mp4", "wb") as f:575 f.write(base64.b64decode(interaction.output_video.data))576577### JavaScript578579 import { GoogleGenAI } from '@google/genai';580 import * as fs from 'fs';581 const ai = new GoogleGenAI({});582583 const interaction = await ai.interactions.create({584 model: 'gemini-omni-1.1-flash',585 input: [586 { type: 'image', data: catData, mime_type: 'image/png' },587 { type: 'image', data: yarnData, mime_type: 'image/png' },588 { type: 'text', text: 'A cat playfully batting at a ball of yarn.' }589 ]590 });591592 if (interaction.output_video?.data) {593 fs.writeFileSync('cat.mp4', Buffer.from(interaction.output_video.data, 'base64'));594 }595596### Java597598 import com.google.genai.Client;599 import com.google.genai.gaos.models.interactions.Content;600 import com.google.genai.gaos.models.interactions.CreateModelInteraction;601 import com.google.genai.gaos.models.interactions.ImageContent;602 import com.google.genai.gaos.models.interactions.ImageContentMimeType;603 import com.google.genai.gaos.models.interactions.Interaction;604 import com.google.genai.gaos.models.interactions.InteractionsInput;605 import com.google.genai.gaos.models.interactions.Model;606 import com.google.genai.gaos.models.interactions.TextContent;607 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;608 import java.nio.file.Files;609 import java.nio.file.Paths;610 import java.util.Arrays;611 import java.util.Base64;612 import java.util.List;613614 Client client = new Client();615616 String catB64 = Base64.getEncoder().encodeToString(Files.readAllBytes(Paths.get("cat.png")));617 String yarnB64 = Base64.getEncoder().encodeToString(Files.readAllBytes(Paths.get("yarn.png")));618619 Content catImage =620 ImageContent.builder()621 .data(catB64)622 .mimeType(ImageContentMimeType.IMAGE_PNG)623 .build();624625 Content yarnImage =626 ImageContent.builder()627 .data(yarnB64)628 .mimeType(ImageContentMimeType.IMAGE_PNG)629 .build();630631 Content textContent =632 TextContent.builder()633 .text("A cat playfully batting at a ball of yarn.")634 .build();635636 List<Content> contents = Arrays.asList(catImage, yarnImage, textContent);637638 CreateModelInteraction params =639 CreateModelInteraction.builder()640 .model(Model.of("gemini-omni-1.1-flash"))641 .input(InteractionsInput.ofContent(contents))642 .build();643644 Interaction interaction =645 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();646647 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {648 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());649 Files.write(Paths.get("cat.mp4"), videoBytes);650 }651652### REST653654 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \655 -H "Content-Type: application/json" \656 -d '{657 "model": "gemini-omni-1.1-flash",658 "input": [659 {"type": "image", "data": "'"$CAT_B64"'", "mime_type": "image/png"},660 {"type": "image", "data": "'"$YARN_B64"'", "mime_type": "image/png"},661 {"type": "text", "text": "A cat playfully batting at a ball of yarn."}662 ]663 }'664665### Tasks parameter666667Use the `task` parameter in the `video_config` to explicitly specify the intended668behavior, for example if you want the model to generate a video from an image,669you can set the parameter to `image_to_video`. If this is not set, the670model will infer what you want from the prompt.671672> [!TIP]673> **Tip:** We recommend relying primarily on prompting and using the `task` parameter only when prompting doesn't work and you need to help the model understand which mode it should use, as using the `task` field adds some constraints.674675The following are allowed values:676677- `text_to_video`678- `image_to_video`679- `reference_to_video`680- `edit`681- `extend`682683The following example shows how to set this for the previously shown image684to video example.685686### Python687688 import base64689 from google import genai690691 client = genai.Client()692693 interaction = client.interactions.create(694 model="gemini-omni-1.1-flash",695 input=[696 {"type": "image", "data": base64_image, "mime_type": "image/jpeg"},697 {"type": "text", "text": "turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video"}698 ],699 generation_config={700 "video_config": {701 "task": "image_to_video",702 }703 },704 )705 with open("example.mp4", "wb") as f:706 f.write(base64.b64decode(interaction.output_video.data))707708### JavaScript709710 import { GoogleGenAI } from "@google/genai";711 import * as fs from 'fs';712 const ai = new GoogleGenAI({});713714 const interaction = await ai.interactions.create({715 model: 'gemini-omni-1.1-flash',716 input: [717 { type: 'image', data: base64Image, mime_type: 'image/jpeg' },718 { type: 'text', text: 'turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video' }719 ],720 generationConfig: {721 videoConfig: {722 task: 'image_to_video',723 }724 }725 });726727 if (interaction.output_video?.data) {728 fs.writeFileSync('example.mp4', Buffer.from(interaction.output_video.data, 'base64'));729 }730731### Java732733 import com.google.genai.Client;734 import com.google.genai.gaos.models.interactions.Content;735 import com.google.genai.gaos.models.interactions.CreateModelInteraction;736 import com.google.genai.gaos.models.interactions.GenerationConfig;737 import com.google.genai.gaos.models.interactions.ImageContent;738 import com.google.genai.gaos.models.interactions.ImageContentMimeType;739 import com.google.genai.gaos.models.interactions.Interaction;740 import com.google.genai.gaos.models.interactions.InteractionsInput;741 import com.google.genai.gaos.models.interactions.Model;742 import com.google.genai.gaos.models.interactions.Task;743 import com.google.genai.gaos.models.interactions.TextContent;744 import com.google.genai.gaos.models.interactions.VideoConfig;745 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;746 import java.nio.file.Files;747 import java.nio.file.Paths;748 import java.util.Arrays;749 import java.util.Base64;750 import java.util.List;751752 Client client = new Client();753754 byte[] imageBytes = Files.readAllBytes(Paths.get("drawing.jpg"));755 String base64Image = Base64.getEncoder().encodeToString(imageBytes);756757 Content imageContent =758 ImageContent.builder()759 .data(base64Image)760 .mimeType(ImageContentMimeType.IMAGE_JPEG)761 .build();762763 Content textContent =764 TextContent.builder()765 .text(766 "turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video")767 .build();768769 List<Content> contents = Arrays.asList(imageContent, textContent);770771 GenerationConfig generationConfig =772 GenerationConfig.builder()773 .videoConfig(VideoConfig.builder().task(Task.IMAGE_TO_VIDEO).build())774 .build();775776 CreateModelInteraction params =777 CreateModelInteraction.builder()778 .model(Model.of("gemini-omni-1.1-flash"))779 .input(InteractionsInput.ofContent(contents))780 .generationConfig(generationConfig)781 .build();782783 Interaction interaction =784 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();785786 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {787 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());788 Files.write(Paths.get("example.mp4"), videoBytes);789 }790791### REST792793 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \794 -H "x-goog-api-key: $GEMINI_API_KEY" \795 -H "Content-Type: application/json" \796 -d '{797 "model": "gemini-omni-1.1-flash",798 "input": [799 {800 "type": "image",801 "data": "'"$BASE64_IMAGE"'",802 "mime_type": "image/jpeg"803 },804 {805 "type": "text",806 "text": "turn this into realistic footage, using the drawing only as a guide for movement, do not show the drawing in the final video"807 }808 ],809 "generation_config": {810 "video_config": {811 "task": "image_to_video"812 }813 }814 }'815816## Stateful video editing817818Generate a video and edit it iteratively using follow-up prompts. Each turn819builds on the previous result. The model remembers the video context, applying820your changes while preserving elements you did not mention. Use the821`previous_interaction_id` to track the conversation history and the generated822video state without re-uploading the previous video.823824> [!NOTE]825> **Note:** See [Limitations](https://ai.google.dev/gemini-api/docs/omni#limitations) for detailed list of restrictions for video editing.826827The following example demonstrates how to generate a first video then edit it:828829### Python830831 import base64832 from google import genai833834 client = genai.Client()835836 # Turn 1: Generate initial video837 res1 = client.interactions.create(model="gemini-omni-1.1-flash", input="A woman playing violin outdoors.")838839 # Turn 2: Edit the previous video840 res2 = client.interactions.create(841 model="gemini-omni-1.1-flash",842 previous_interaction_id=res1.id,843 input="Make the violin invisible."844 )845 with open("example.mp4", "wb") as f:846 f.write(base64.b64decode(res2.output_video.data))847848### JavaScript849850 import { GoogleGenAI } from '@google/genai';851 import * as fs from 'fs';852 const ai = new GoogleGenAI({});853854 // Turn 1: Generate initial video855 const res1 = await ai.interactions.create({856 model: 'gemini-omni-1.1-flash',857 input: 'A woman playing violin outdoors.',858 });859860 // Turn 2: Edit the previous video861 const res2 = await ai.interactions.create({862 model: 'gemini-omni-1.1-flash',863 previous_interaction_id: res1.id,864 input: 'Make the violin invisible.',865 });866867 if (res2.output_video?.data) {868 fs.writeFileSync('example.mp4', Buffer.from(res2.output_video.data, 'base64'));869 }870871### Java872873 import com.google.genai.Client;874 import com.google.genai.gaos.models.interactions.CreateModelInteraction;875 import com.google.genai.gaos.models.interactions.Interaction;876 import com.google.genai.gaos.models.interactions.InteractionsInput;877 import com.google.genai.gaos.models.interactions.Model;878 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;879 import java.nio.file.Files;880 import java.nio.file.Paths;881 import java.util.Base64;882883 Client client = new Client();884885 // Turn 1: Generate initial video886 CreateModelInteraction turn1Params =887 CreateModelInteraction.builder()888 .model(Model.of("gemini-omni-1.1-flash"))889 .input(InteractionsInput.of("A woman playing violin outdoors."))890 .build();891892 Interaction res1 =893 client.interactions.create(CreateInteractionRequestBody.of(turn1Params)).interaction().get();894895 // Turn 2: Edit the previous video896 CreateModelInteraction turn2Params =897 CreateModelInteraction.builder()898 .model(Model.of("gemini-omni-1.1-flash"))899 .previousInteractionId(res1.id().get())900 .input(InteractionsInput.of("Make the violin invisible."))901 .build();902903 Interaction res2 =904 client.interactions.create(CreateInteractionRequestBody.of(turn2Params)).interaction().get();905906 if (res2.outputVideo().isPresent() && res2.outputVideo().get().data().isPresent()) {907 byte[] videoBytes = Base64.getDecoder().decode(res2.outputVideo().get().data().get());908 Files.write(Paths.get("example.mp4"), videoBytes);909 }910911### REST912913 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \914 -H "Content-Type: application/json" \915 -d '{916 "model": "gemini-omni-1.1-flash",917 "previous_interaction_id": "'"$PREVIOUS_ID"'",918 "input": "Make the violin invisible."919 }'920921Example of an initial video:922923[Video](https://www.youtube.com/watch?v=9yk4HuZmF0g)924925Example of an edited video:926927[Video](https://www.youtube.com/watch?v=U5GXcvq6GZY)928929Each turn in the conversation produces a new video. The model understands930context from prior turns, letting you make incremental changes like adjusting931lighting, and swapping backgrounds, without re-describing the entire scene.932933### Edit your own videos934935Upload your videos using the [Files API](https://ai.google.dev/gemini-api/docs/files) to edit them936with Gemini Omni Flash.937938> [!NOTE]939> **Note:** Editing uploaded videos is not available in all regions. See [Limitations](https://ai.google.dev/gemini-api/docs/omni#limitations) section for details.940941The following example shows how to edit the following original video:942943[Video](https://www.youtube.com/watch?v=lw9MJDx9Zas)944945### Python946947 import time948 import base64949 from google import genai950951 client = genai.Client()952953 # Upload video using the file API954 video_file = client.files.upload(file="Video.mp4")955956 while video_file.state == "PROCESSING":957 print('Waiting for video to be processed.')958 time.sleep(10)959 video_file = client.files.get(name=video_file.name)960961 if video_file.state == "FAILED":962 raise ValueError(video_file.state)963 print(f'Video processing complete: ' + video_file.uri)964965 # Edit your video966 interaction = client.interactions.create(967 model="gemini-omni-1.1-flash",968 input=[969 {"type": "video", "uri": video_file.uri},970 {"type": "text", "text": "When the person touches the mirror, make the mirror ripple beautifully like liquid, and the person's arm turns into reflective mirror material"}971 ],972 )973 with open("example.mp4", "wb") as f:974 f.write(base64.b64decode(interaction.output_video.data))975976### JavaScript977978 import { GoogleGenAI } from '@google/genai';979 import * as fs from 'fs';980 const ai = new GoogleGenAI({});981982 // Upload video using the file API983 let videoFile = await ai.files.upload({984 file: 'Video.mp4',985 });986987 while (videoFile.state === 'PROCESSING') {988 console.log('Waiting for video to be processed.');989 await new Promise(r => setTimeout(r, 10000));990 videoFile = await ai.files.get({ name: videoFile.name });991 }992993 if (videoFile.state === 'FAILED') {994 throw new Error(videoFile.state);995 }996 console.log('Video processing complete: ' + videoFile.uri);997998 // Edit your video999 const interaction = await ai.interactions.create({1000 model: 'gemini-omni-1.1-flash',1001 input: [1002 { type: 'video', uri: videoFile.uri },1003 { type: 'text', text: "When the person touches the mirror, make the mirror ripple beautifully like liquid, and the person's arm turns into reflective mirror material" }1004 ],1005 });10061007 if (interaction.output_video?.data) {1008 fs.writeFileSync('example.mp4', Buffer.from(interaction.output_video.data, 'base64'));1009 }10101011### Java10121013 import com.google.genai.Client;1014 import com.google.genai.gaos.models.interactions.Content;1015 import com.google.genai.gaos.models.interactions.CreateModelInteraction;1016 import com.google.genai.gaos.models.interactions.Interaction;1017 import com.google.genai.gaos.models.interactions.InteractionsInput;1018 import com.google.genai.gaos.models.interactions.Model;1019 import com.google.genai.gaos.models.interactions.TextContent;1020 import com.google.genai.gaos.models.interactions.VideoContent;1021 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;1022 import com.google.genai.types.File;1023 import com.google.genai.types.FileState;1024 import com.google.genai.types.UploadFileConfig;1025 import java.nio.file.Files;1026 import java.nio.file.Paths;1027 import java.util.Arrays;1028 import java.util.Base64;1029 import java.util.List;10301031 Client client = new Client();10321033 // Upload video using the file API1034 File videoFile =1035 client.files.upload("Video.mp4", UploadFileConfig.builder().mimeType("video/mp4").build());10361037 while (videoFile.state().isPresent()1038 && videoFile.state().get().knownEnum() == FileState.Known.PROCESSING) {1039 System.out.println("Waiting for video to be processed.");1040 Thread.sleep(10000);1041 videoFile = client.files.get(videoFile.name().get(), null);1042 }10431044 if (videoFile.state().isPresent()1045 && videoFile.state().get().knownEnum() == FileState.Known.FAILED) {1046 throw new IllegalStateException("Video processing failed: " + videoFile.state().get());1047 }1048 System.out.println("Video processing complete: " + videoFile.uri().orElse(""));10491050 // Edit your video1051 Content videoContent = VideoContent.builder().uri(videoFile.uri().get()).build();1052 Content textContent =1053 TextContent.builder()1054 .text(1055 "When the person touches the mirror, make the mirror ripple beautifully like liquid, and the person's arm turns into reflective mirror material")1056 .build();10571058 List<Content> contents = Arrays.asList(videoContent, textContent);10591060 CreateModelInteraction params =1061 CreateModelInteraction.builder()1062 .model(Model.of("gemini-omni-1.1-flash"))1063 .input(InteractionsInput.ofContent(contents))1064 .build();10651066 Interaction interaction =1067 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();10681069 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {1070 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());1071 Files.write(Paths.get("example.mp4"), videoBytes);1072 }10731074### REST10751076 #!/bin/bash1077 VIDEO_B64=$(encode_file "$VIDEO_FILE")10781079 curl -sS -w "\n[HTTP %{http_code}]\n" "https://generativelanguage.googleapis.com/v1beta/interactions" \1080 -H "x-goog-api-key: ${API_KEY}" \1081 -H "Content-Type: application/json" \1082 -d @- <<EOF > video_editing_response.json1083 {1084 "model": "gemini-omni-1.1-flash",1085 "input": [1086 {1087 "type": "user_input",1088 "content": [1089 {1090 "type": "video",1091 "mime_type": "video/mp4",1092 "data": "$VIDEO_B64"1093 },1094 {1095 "type": "text",1096 "text": "When the person touches the mirror, make the mirror ripple beautifully like liquid, and the person's arm turns into reflective mirror material"1097 }1098 ]1099 }1100 ],1101 "response_format": { "type": "video" }1102 }1103 EOF11041105Example of an edited video:11061107[Video](https://www.youtube.com/watch?v=Bn-6vCl1wgQ)11081109> [!NOTE]1110> **Note:** You can also pass the data directly in base64, but since videos can be quite big we recommend using the File API.11111112## Retrieving videos with an URI11131114Use the `delivery="uri"` parameter in1115`response_format` to retrieve generated videos that are larger than 4MB.1116This returns a Google-hosted URI that you can poll until the1117video is `ACTIVE` before downloading.11181119### Python11201121 import time1122 from google import genai11231124 client = genai.Client()11251126 # 1. Request video via URI delivery1127 interaction = client.interactions.create(1128 model="gemini-omni-1.1-flash",1129 input="A beautiful sunset.",1130 response_format={"type": "video", "delivery": "uri"}1131 )11321133 # 2. Extract file name and poll for ACTIVE state1134 video_output = interaction.output_video1135 file_name = video_output.uri.split("/")[-1] # Extract ID11361137 print("Waiting for video processing...")1138 while True:1139 f_info = client.files.get(name=f"files/{file_name}")1140 if f_info.state.name == "ACTIVE":1141 break1142 elif f_info.state.name == "FAILED":1143 raise RuntimeError("Generation failed.")1144 time.sleep(5)11451146 # 3. Download the final video1147 client.files.download(file=video_output.uri, destination="output.mp4")11481149### JavaScript11501151 import { GoogleGenAI } from '@google/genai';1152 const ai = new GoogleGenAI({});11531154 // 1. Request video via URI delivery1155 const interaction = await ai.interactions.create({1156 model: 'gemini-omni-1.1-flash',1157 input: 'A beautiful sunset.',1158 response_format: { type: 'video', delivery: 'uri' },1159 });11601161 // 2. Extract file name and poll for ACTIVE state1162 const videoOutput = interaction.output_video;1163 const fileId = videoOutput.uri.match(/files\/([a-zA-Z0-9]+)/)[1];1164 const name = `files/${fileId}`;11651166 console.log("Waiting for video processing...");1167 while (true) {1168 const fInfo = await ai.files.get({ name });1169 if (fInfo.state.name === 'ACTIVE') break;1170 if (fInfo.state.name === 'FAILED') throw new Error("Generation failed.");1171 await new Promise(r => setTimeout(r, 5000));1172 }11731174 // 3. Download the final video1175 await ai.files.download({1176 file: videoOutput,1177 downloadPath: 'output.mp4',1178 });1179 console.log("💾 Saved video to output.mp4");11801181### Java11821183 import com.google.genai.Client;1184 import com.google.genai.gaos.models.interactions.CreateModelInteraction;1185 import com.google.genai.gaos.models.interactions.CreateModelInteractionResponseFormat;1186 import com.google.genai.gaos.models.interactions.Interaction;1187 import com.google.genai.gaos.models.interactions.InteractionsInput;1188 import com.google.genai.gaos.models.interactions.Model;1189 import com.google.genai.gaos.models.interactions.ResponseFormat;1190 import com.google.genai.gaos.models.interactions.VideoContent;1191 import com.google.genai.gaos.models.interactions.VideoResponseFormat;1192 import com.google.genai.gaos.models.interactions.VideoResponseFormatDelivery;1193 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;1194 import com.google.genai.types.File;1195 import com.google.genai.types.FileState;11961197 Client client = new Client();11981199 // 1. Request video via URI delivery1200 VideoResponseFormat videoFormat =1201 VideoResponseFormat.builder()1202 .delivery(VideoResponseFormatDelivery.URI)1203 .build();12041205 CreateModelInteraction params =1206 CreateModelInteraction.builder()1207 .model(Model.of("gemini-omni-1.1-flash"))1208 .input(InteractionsInput.of("A beautiful sunset."))1209 .responseFormat(CreateModelInteractionResponseFormat.of(ResponseFormat.of(videoFormat)))1210 .build();12111212 Interaction interaction =1213 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();12141215 // 2. Extract file name and poll for ACTIVE state1216 VideoContent videoOutput = interaction.outputVideo().get();1217 String uri = videoOutput.uri().get();1218 String[] parts = uri.split("/");1219 String fileName = parts[parts.length - 1];12201221 System.out.println("Waiting for video processing...");1222 while (true) {1223 File fileInfo = client.files.get("files/" + fileName, null);1224 if (fileInfo.state().isPresent()1225 && fileInfo.state().get().knownEnum() == FileState.Known.ACTIVE) {1226 break;1227 } else if (fileInfo.state().isPresent()1228 && fileInfo.state().get().knownEnum() == FileState.Known.FAILED) {1229 throw new RuntimeException("Generation failed.");1230 }1231 Thread.sleep(5000);1232 }12331234 // 3. Download the final video1235 client.files.download(uri, "output.mp4", null);12361237### REST12381239 #!/bin/bash12401241 # 1. Initial request to generate the video1242 RESPONSE=$(curl -s -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" \1243 -H "Content-Type: application/json" \1244 -d '{1245 "model": "gemini-omni-1.1-flash",1246 "input": "A beautiful sunset over a calm ocean.",1247 "response_format": {"type": "video", "delivery": "uri"}1248 }')12491250 # Extract FILE_ID from the URI (e.g., "files/abc-123" -> "abc-123")1251 FILE_URI=$(echo $RESPONSE | jq -r '.output_video.uri')1252 FILE_ID=$(echo $FILE_URI | cut -d'/' -f2)12531254 echo "Video requested (ID: $FILE_ID). Waiting for processing..."12551256 # 2. Polling loop1257 while true; do1258 # Get current file status1259 STATUS_JSON=$(curl -s -X GET "https://generativelanguage.googleapis.com/v1beta/files/$FILE_ID?key=$API_KEY")1260 STATE=$(echo $STATUS_JSON | jq -r '.state')12611262 if [ "$STATE" == "ACTIVE" ]; then1263 echo "Processing complete! Downloading..."1264 break1265 elif [ "$STATE" == "FAILED" ]; then1266 echo "Error: Generation failed."1267 exit 11268 else1269 echo "Current state: $STATE... (waiting 5s)"1270 sleep 51271 fi1272 done12731274 # 3. Final download1275 curl -L -X GET "https://generativelanguage.googleapis.com/v1beta/files/$FILE_ID:download?alt=media&key=$API_KEY" \1276 --output "output.mp4"12771278 echo "Done! Video saved to output.mp4"12791280**Raw REST JSON structure (URI):**12811282 {1283 "steps": [1284 { "type": "user_input", "content": [{"type": "text", "text": "..."}] },1285 { "type": "thought", "content": [{"text": "...", "type": "thought"}] },1286 {1287 "type": "model_output",1288 "content": [1289 {1290 "type": "video",1291 "mime_type": "video/mp4",1292 "uri": "https://generativelanguage.googleapis.com/v1beta/files/...:download?alt=media"1293 }1294 ]1295 }1296 ],1297 "id": "v1_...",1298 "status": "completed",1299 "model": "gemini-omni-1.1-flash",1300 "object": "interaction"1301 }13021303<br />13041305> [!NOTE]1306> **Note:** Currently, calling `GET /v1beta/interactions/{id}` returns the video as inline base64 data in the `data` field, even if the interaction was originally created with `delivery: "uri"`. The `uri` field is only guaranteed to be present in the initial creation response or Server-Sent Events (SSE) stream.13071308## Video extension13091310Extend an existing video by generating a seamless continuation at the tail end1311of the clip. Describe how you want the video to continue in your prompt, for example1312`"Extend this video"` or `"Continue the scene: the camera pans across the mountains"`.1313The model analyzes the input video to generate a 3--10 second continuation.13141315You can extend:13161317- **Videos generated by the model (multi-turn)** : Extend a previously generated video by referencing its `previous_interaction_id`.1318- **Uploaded videos**: Provide an uploaded video file (via the Files API) along1319 with your extension prompt.13201321### Python13221323 import base641324 from google import genai13251326 client = genai.Client()13271328 # Upload your video using the Files API1329 video_file = client.files.upload(file="my_video.mp4")13301331 # Extend the video using prompt-based extension1332 interaction = client.interactions.create(1333 model="gemini-omni-1.1-flash",1334 input=[1335 {"type": "video", "uri": video_file.uri},1336 {"type": "text", "text": "Continue the scene."}1337 ],1338 )1339 with open("extended.mp4", "wb") as f:1340 f.write(base64.b64decode(interaction.output_video.data))13411342### JavaScript13431344 import { GoogleGenAI } from '@google/genai';1345 import * as fs from 'fs';1346 const ai = new GoogleGenAI({});13471348 // Upload your video using the Files API1349 let videoFile = await ai.files.upload({1350 file: 'my_video.mp4',1351 });13521353 while (videoFile.state === 'PROCESSING') {1354 await new Promise(r => setTimeout(r, 10000));1355 videoFile = await ai.files.get({ name: videoFile.name });1356 }13571358 // Extend the video using prompt-based extension1359 const interaction = await ai.interactions.create({1360 model: 'gemini-omni-1.1-flash',1361 input: [1362 { type: 'video', uri: videoFile.uri },1363 { type: 'text', text: 'Continue the scene.' }1364 ],1365 });13661367 if (interaction.output_video?.data) {1368 fs.writeFileSync('extended.mp4', Buffer.from(interaction.output_video.data, 'base64'));1369 }13701371### Java13721373 import com.google.genai.Client;1374 import com.google.genai.gaos.models.interactions.Content;1375 import com.google.genai.gaos.models.interactions.CreateModelInteraction;1376 import com.google.genai.gaos.models.interactions.Interaction;1377 import com.google.genai.gaos.models.interactions.InteractionsInput;1378 import com.google.genai.gaos.models.interactions.Model;1379 import com.google.genai.gaos.models.interactions.TextContent;1380 import com.google.genai.gaos.models.interactions.VideoContent;1381 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;1382 import com.google.genai.types.File;1383 import com.google.genai.types.UploadFileConfig;1384 import java.nio.file.Files;1385 import java.nio.file.Paths;1386 import java.util.Arrays;1387 import java.util.Base64;1388 import java.util.List;13891390 Client client = new Client();13911392 // Upload your video using the Files API1393 File videoFile =1394 client.files.upload(1395 "my_video.mp4", UploadFileConfig.builder().mimeType("video/mp4").build());13961397 // Extend the video using prompt-based extension1398 Content videoContent = VideoContent.builder().uri(videoFile.uri().get()).build();1399 Content textContent = TextContent.builder().text("Continue the scene.").build();14001401 List<Content> contents = Arrays.asList(videoContent, textContent);14021403 CreateModelInteraction params =1404 CreateModelInteraction.builder()1405 .model(Model.of("gemini-omni-1.1-flash"))1406 .input(InteractionsInput.ofContent(contents))1407 .build();14081409 Interaction interaction =1410 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();14111412 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {1413 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());1414 Files.write(Paths.get("extended.mp4"), videoBytes);1415 }14161417### REST14181419 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" -H "Content-Type: application/json" -d '{1420 "model": "gemini-omni-1.1-flash",1421 "input": [1422 {"type": "video", "uri": "'"$VIDEO_URI"'"},1423 {"type": "text", "text": "Continue the scene."}1424 ]1425 }'14261427Your browser does not support the video tag. Your browser does not support the video tag.14281429### Extending with reference media14301431You can provide reference images in the `input` array along with your prompt to1432introduce new characters or elements into the extended video:14331434### Python14351436 import base641437 from google import genai14381439 client = genai.Client()14401441 # Upload base video and reference image using the Files API1442 video_file = client.files.upload(file="my_video.mp4")1443 character_img = client.files.upload(file="character.png")14441445 # Extend the video while introducing the reference character1446 interaction = client.interactions.create(1447 model="gemini-omni-1.1-flash",1448 input=[1449 {"type": "video", "uri": video_file.uri},1450 {"type": "image", "uri": character_img.uri},1451 {"type": "text", "text": "Extend this video: have the character shown in <IMAGE_REF_0> enter the scene and wave."}1452 ],1453 )1454 with open("extended_with_character.mp4", "wb") as f:1455 f.write(base64.b64decode(interaction.output_video.data))14561457### JavaScript14581459 import { GoogleGenAI } from '@google/genai';1460 import * as fs from 'fs';1461 const ai = new GoogleGenAI({});14621463 // Upload base video and reference image using the Files API1464 let videoFile = await ai.files.upload({ file: 'my_video.mp4' });1465 let characterImg = await ai.files.upload({ file: 'character.png' });14661467 while (videoFile.state === 'PROCESSING' || characterImg.state === 'PROCESSING') {1468 await new Promise(r => setTimeout(r, 10000));1469 videoFile = await ai.files.get({ name: videoFile.name });1470 characterImg = await ai.files.get({ name: characterImg.name });1471 }14721473 // Extend the video while introducing the reference character1474 const interaction = await ai.interactions.create({1475 model: 'gemini-omni-1.1-flash',1476 input: [1477 { type: 'video', uri: videoFile.uri },1478 { type: 'image', uri: characterImg.uri },1479 { type: 'text', text: 'Extend this video: have the character shown in <IMAGE_REF_0> enter the scene and wave.' }1480 ],1481 });14821483 if (interaction.output_video?.data) {1484 fs.writeFileSync('extended_with_character.mp4', Buffer.from(interaction.output_video.data, 'base64'));1485 }14861487### Java14881489 import com.google.genai.Client;1490 import com.google.genai.gaos.models.interactions.Content;1491 import com.google.genai.gaos.models.interactions.CreateModelInteraction;1492 import com.google.genai.gaos.models.interactions.ImageContent;1493 import com.google.genai.gaos.models.interactions.Interaction;1494 import com.google.genai.gaos.models.interactions.InteractionsInput;1495 import com.google.genai.gaos.models.interactions.Model;1496 import com.google.genai.gaos.models.interactions.TextContent;1497 import com.google.genai.gaos.models.interactions.VideoContent;1498 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;1499 import com.google.genai.types.File;1500 import com.google.genai.types.UploadFileConfig;1501 import java.nio.file.Files;1502 import java.nio.file.Paths;1503 import java.util.Arrays;1504 import java.util.Base64;1505 import java.util.List;15061507 Client client = new Client();15081509 // Upload base video and reference image using the Files API1510 File videoFile =1511 client.files.upload(1512 "my_video.mp4", UploadFileConfig.builder().mimeType("video/mp4").build());1513 File characterImg =1514 client.files.upload(1515 "character.png", UploadFileConfig.builder().mimeType("image/png").build());15161517 // Extend the video while introducing the reference character1518 Content videoContent = VideoContent.builder().uri(videoFile.uri().get()).build();1519 Content imageContent = ImageContent.builder().uri(characterImg.uri().get()).build();1520 Content textContent =1521 TextContent.builder()1522 .text(1523 "Extend this video: have the character shown in <IMAGE_REF_0> enter the scene and wave.")1524 .build();15251526 List<Content> contents = Arrays.asList(videoContent, imageContent, textContent);15271528 CreateModelInteraction params =1529 CreateModelInteraction.builder()1530 .model(Model.of("gemini-omni-1.1-flash"))1531 .input(InteractionsInput.ofContent(contents))1532 .build();15331534 Interaction interaction =1535 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();15361537 if (interaction.outputVideo().isPresent() && interaction.outputVideo().get().data().isPresent()) {1538 byte[] videoBytes = Base64.getDecoder().decode(interaction.outputVideo().get().data().get());1539 Files.write(Paths.get("extended_with_character.mp4"), videoBytes);1540 }15411542### REST15431544 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions?key=$API_KEY" -H "Content-Type: application/json" -d '{1545 "model": "gemini-omni-1.1-flash",1546 "input": [1547 {"type": "video", "uri": "'$VIDEO_URI'"},1548 {"type": "image", "uri": "'$CHARACTER_IMG_URI'"},1549 {"type": "text", "text": "Extend this video: have the character shown in <IMAGE_REF_0> enter the scene and wave."}1550 ]1551 }'15521553Your browser does not support the video tag.15541555> [!TIP]1556> **Tip:** Optionally, you can set `"task": "extend"` in `video_config` under `generation_config` if prompting alone does not produce the expected extension mode. However, we recommend relying primarily on prompting first, as using the `task` field adds strict constraints to the model.15571558### Extension constraints and guidelines15591560Keep the following rules and constraints in mind when extending videos:15611562- **Spoken dialogue on uploaded videos**: Currently, you cannot extend an uploaded video where someone is talking to add additional dialogue (it is supported if the character remains silent or if the prompt does not add dialogue).1563- **Multi-turn voice extension** : Generating spoken dialogue or speech is supported when extending previously generated videos via multi-turn (`previous_interaction_id`).1564- **End-of-clip only**: Extension is limited to appending to the end of the video. You cannot prepend content or extend the middle of a clip.1565- **Duration limit**: Input videos for extension must be 10 seconds or less in length when uploading (unless using multi-turn).1566- **Regional availability**: Extending uploaded videos is not currently available for users in the European Economic Area (EEA), Switzerland, and the United Kingdom (extending videos generated by the model is supported in all available regions).15671568## Best practices15691570- **Use URI delivery for large videos:** For videos larger than 4MB (\>720p when available), use `delivery="uri"` in `response_format` to avoid payload size limits.1571- **Optimized performance:** Set `background=false`, `store=false`, and `stream=false` for faster, synchronous unary generation. Note that setting `store=false` means the generated video won't be editable in subsequent turns using the `previous_interaction_id`.1572- **Prompt precision:** See [prompt guidance](https://ai.google.dev/gemini-api/docs/omni#prompt-guide) section for details.15731574## Limitations15751576- Uploading and editing images containing minors is not supported in European Economic Area, Switzerland, and the United Kingdom.1577- Uploading and editing images containing certain recognizable people is not supported.1578- Editing or extending uploaded videos is not currently available for users in the European Economic Area (EEA), Switzerland, and the United Kingdom (editing or extending videos generated by the model is supported).1579- Input videos for editing and extension must be 10 seconds or less when uploading (unless extending videos generated by the model in multi-turn).1580- Video extension is limited to appending to the end of a video; prepending or extending the middle of a clip is not supported.1581- You cannot extend an uploaded video where someone is talking to add additional dialogue (characters can remain silent, or multi-turn extension with `previous_interaction_id` can be used).1582- Voice editing is not supported.1583- Uploading audio references is unsupported in the current version of the API.1584- Video references work best with likenesses; any audio in a video reference is ignored. Video references support a maximum of 3 clips, up to 3 seconds each.1585- Referencing or reasoning across multiple videos is not supported. Attempting multi-video prompting may result in degraded model performance or unexpected outputs.1586- Provisioned throughput is not supported.1587- System instructions, temperature, `top_p`, stop sequences, and negative prompts are not supported (you can put your negatives in the regular prompt: e.g., "Do not do X").1588- Using YouTube videos as a media source is not supported.15891590## Technical details15911592- All generated videos include SynthID watermarking, which is invisible to viewers but can be detected programmatically for provenance verification.1593- Video generation times vary based on duration, resolution, and current API load. Longer and higher-resolution videos take more time to generate.1594- Omni applies content safety filters to both input prompts and generated video (which vary by region). Prompts that violate usage policies are blocked.1595- English (EN) is fully supported, but other languages have not been evaluated, so they may work but results can vary.15961597## Gemini Omni Flash prompt guide15981599This section contains tips and examples on how to prompt Gemini Omni Flash effectively.16001601### Single scene16021603By default Omni Flash will try to create a video with a few different shots.1604It'll attempt to craft an interesting narrative based on the prompt.16051606If you need the output video to contain a single scene, you must prompt for that:16071608- In a single unbroken scene1609- In a single continuous shot1610- No scene cuts16111612For example:16131614 Continuous, unbroken handheld shot of a fluffy tabby cat sitting on a sunny windowsill, looking out into a leafy garden. The cat's tail twitches slowly, and its ears rotate slightly toward ambient noises. Sunbeams illuminate dust motes in the air. Sound design: Gentle breeze, distant bird chirps. No dialogue.16151616### Removing unwanted elements16171618If the generated video contains things you don't want, include simple negative prompts to avoid them:16191620- No dialogue1621- No embellishments1622- No extra sound effects16231624### Prompts for editing16251626Simple prompts work best for video editing. Overly descriptive prompts can lead to unintended changes.16271628The following are more examples of simple editing prompts:16291630- Make this video anime1631- Put a fashionable hat on this person1632- Change the lighting to be more dramatic1633- Change the text on the sign to say "Omni Flash"16341635When editing a specific aspect of the video, include `"Keep everything else the same"` to maintain visual consistency.16361637The following are some examples to show how to apply this technique:16381639- **Avoid:** `In the video of the man sitting on the sofa, please add a small1640 black cat that runs from the right side of the screen, jumps onto his lap,1641 and then he starts to stroke its head while looking down.`1642 - **Simplify:** `Add a cat that jumps onto his lap, he begins to pet it.1643 Keep everything else the same.`1644- **Avoid:** `Please remove the cell phone that the person is holding in1645 their hand and fill in the background so it looks like they are just holding1646 their hand empty.`1647 - **Simplify:** `Make the phone invisible. Keep everything else the1648 same.`16491650### Prompting the audio16511652By default the model will try to generate an appropriate audio track for a1653video. This might not always be what you want. You can use your prompt to1654describe the type of audio you want. This is especially important if you want1655music in your video:16561657- Include calm background music1658- The video has a high energy techno beat1659- The audio is a low tinny radio broadcast in the background, playing a song16601661### Timing events16621663You can prompt for things to happen at specific times in the video, there is no1664precise syntax needed and you can use natural language. This is especially1665useful in creating your own scene cuts, rhythm or rapid fire sequences.1666See the following for examples:16671668- After 3 seconds, a woman enters the scene.1669- At 5s the chorus starts in the background audio.1670- Every 2s cut to a new frame.1671- In a rapid fire sequence, every half a second (12 frames at 24fps) change the scene to a new location.16721673You can also use a timecode syntax:16741675 [0-3s] A person is walking1676 [3-6s] They stop and turn around1677 [6-10s] They start running16781679### Meta prompting16801681You can ask Gemini Omni Flash to pay attention to general qualities or1682principles of video generation:16831684- Consider micro-detail, expression and timing to create a very rich, detailed but entirely natural scene.1685- Be extremely detailed in your descriptions of characters and environments. Apply costume design principles to characters. Be very specific about the people, items and objects in the scene.1686- Include plenty of appropriate detail in the background elements to make the scene feel realistic and natural.1687- Make a rapid fire video that shows a different rare `[thing]` every 1s, upbeat music, include text to label the thing.16881689### Text in videos16901691You can prompt to include text in your video and Gemini Omni will render in a1692way that is correct and readable. If there will be naturally occurring text in1693your video, even in background elements, it can help to define what it should1694say.16951696- One word on the screen at a time: "did, you, know, that, Omni, can, do, awesome, text?" Each word appears for 1s with a different animated style. No dialogue.1697- There is a street sign that says: "This is an AI generation by Omni", there is a storefront that says: "All you need AI", there's a car with the number plate: "OMNI1.1"16981699### Prompts for extending a video17001701With Gemini Omni 1.1 Flash you can extend videos with prompts like, `"Extend this video"` or `"The scene continues"`. You can extend videos by 10s, up to a total length of 40s.17021703Omni creates an extension that keeps video, motion, characters and audio coherent by using the last 10s of your original video as context. Some of the final frames in your input video will be edited to make the transition seamless.17041705When extending, all of this guide's Omni prompting tips still apply:17061707- Describe the audio in your extended scene, especially if you need it to change: `"The music continues into the chorus"`1708- Describe if the scene continues, or if there is a shot cut to a new scene (perhaps with the same characters): `"Show the same characters in the next scene"`1709- Include images and videos as references when extending to help keep your outputs accurate, or to introduce new characters: `"The person shown in the reference image enters the scene"`, `"The dog in the reference video <VIDEO_REF_0> jumps onto the sofa"`1710- If using timestamps or a timecode syntax, 0s refers to the beginning of the extended part of the video. If extending a 10s video, the scene cut in this prompt will happen after 12s: `"After 2s cut to a new scene with the same characters"`17111712### Using tags in prompts to set image and video roles17131714You can use tags to bind uploaded media to specific generation roles. This lets1715you specify whether each image or video is a starting frame, a final frame, or a1716reference.17171718#### 1. Simple tags (recommended)17191720For simple cases where media roles are clear from the prompt, you can bind1721images and videos to roles directly:17221723- **`<FIRST_FRAME>`** : use the image as the starting frame of the video, for example: `<FIRST_FRAME> a woman is walking`1724- **`<LAST_FRAME>`** : use the image as the final frame of the video to transition to. Must be used with `<FIRST_FRAME>`, for example: `<FIRST_FRAME> <LAST_FRAME> a woman is walking`1725- **`<IMAGE_REF_N>`** : use the image as a reference, for example: `in the1726 style of <IMAGE_REF_0> a woman <IMAGE_REF_1> is walking` (combines style reference from the first image and subject reference from the second image). Image references start from 0.1727- **`<VIDEO_REF_N>`** : use the video as a character or object reference, for example: `the person in <VIDEO_REF_0> is playing the violin`. Video references also start from 0.17281729The following is an example with 6 reference images:17301731 [0-3s] A studio fashion sequence. Starting with woman <IMAGE_REF_0>, she is holding <IMAGE_REF_1>1732 [3-6s] Then we see the man <IMAGE_REF_2> holding <IMAGE_REF_3>1733 [6-10s] And finally another woman <IMAGE_REF_4> who is holding <IMAGE_REF_5> while walking.17341735#### 2. Declaring sources and references17361737For more complex cases with multiple media inputs and multiple roles, you can use1738explicit prefix tags paired with natural language instructions. You should1739declare these sources and references at the start of your prompt.17401741- `[# Sources <FIRST_FRAME>@Image1]` will use the first image as the starting frame.1742- `[# Sources <FIRST_FRAME>@Image1 <LAST_FRAME>@Image2]` will use the first image as the starting frame and the second image as the final frame.1743- `[# Sources <FIRST_FRAME>@Image1 <LAST_FRAME>@Image1]` will use the first image as both the first frame and the last frame, creating a video that loops.1744- `[# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2]` will use the first image as the starting frame and the second image as a reference.1745- `[# Sources <VIDEO_0>@Video1]` will use the video as the primary source video to edit or modify.1746- `[# Sources <PREVIOUS_VIDEO>@Video1]` will use the video from the previous turn to extend.1747- `[# References <IMAGE_REF_0>@Image1]` will use the first image as a reference.1748- `[# References <IMAGE_REF_1>@Image2]` will use the second image as a reference.1749- `[# References <IMAGE_REF_0>@Image1 <IMAGE_REF_1>@Image2]` will use both images as references.1750- `[# References <VIDEO_REF_0>@Video1]` will use the first video as a reference.1751- `[# References <IMAGE_REF_0>@Image1 <VIDEO_REF_0>@Video1]` will use both an image and a video as a reference.17521753Add guiding instructions at the end of your prompt:17541755- For a starting frame: `"Use this image as the starting frame."`1756- For a looping video via start and end frames: `"Use this image as the first frame and the last frame."`1757- For reference images: `"Use the given image(s) as references for video generation. The images should not be used as literal initial frames."`1758- For reference videos: `"Use the given video(s) as references. Do not use them as a source for video editing."`17591760Some examples of prompts with source and reference declarations:17611762**Starting frame combined with a reference image:**17631764 [# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2] a woman <IMAGE_REF_0> is walking. Use Image1 as the starting frame. Use Image2 as a reference for the video generation.17651766**Character reference video combined with an object reference image:**17671768 [# References <IMAGE_REF_0>@Image1 <VIDEO_REF_0>@Video1] The woman in <VIDEO_REF_0> is playing the violin shown in <IMAGE_REF_0>. Use Video1 as a character reference and Image1 as an object reference.17691770## What's next17711772- Get started with the Gemini Omni Flash by experimenting in the [Omni Quickstart Colab](https://colab.sandbox.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Get_started_Omni.ipynb).1773- Learn how to write even better prompts with our [Introduction to prompt design](https://ai.google.dev/gemini-api/docs/prompting-intro).