SPB Git

Repository Public

media-resolution.md 12.2 KB 275 lines markdown

1The `media_resolution` parameter controls how the Gemini API processes media2inputs like images, videos, audio, and PDF documents by determining the3**maximum number of tokens** allocated for media inputs, allowing you to4balance response quality against latency and cost. While visual and document5inputs scale token allocation based on the resolution setting, audio inputs6are tokenized at a fixed rate per second across all resolution levels. For7different settings, default values and how they correspond to tokens, see the8[Token counts](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution#token-counts) section.910You can configure media resolution in two ways:1112- [Per part](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution#per-part-media-resolution) (Gemini 3 only)1314- [Globally](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution#global-media-resolution) for an entire `generateContent` request (all multimodal models)1516## Per-part media resolution (Gemini 3 only)1718Gemini 3 lets you set media resolution for individual media objects within your19request, offering fine-grained optimisation of token usage. You can mix20resolution levels in a single request. For example, using high resolution for a21complex diagram and low resolution for a contextual image. This setting22overrides any global configuration for a specific part. For default settings,23see the [Token counts](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution#token-counts) section.2425> [!NOTE]26> **Note:** Per-part media resolution is an experimental feature.2728### Python2930    from google import genai31    from google.genai import types3233    # The media_resolution parameter for parts is available in the v1beta API version.34    client = genai.Client(35      http_options={36          'api_version': 'v1beta',37      }38    )3940    # Replace with your image data41    with open('path/to/image1.jpg', 'rb') as f:42        image_bytes_1 = f.read()4344    # Create parts with different resolutions45    image_part_high = types.Part.from_bytes(46        data=image_bytes_1,47        mime_type='image/jpeg',48        media_resolution=types.MediaResolution.MEDIA_RESOLUTION_HIGH49    )5051    model_name = 'gemini-3.1-pro-preview'5253    response = client.models.generate_content(54        model=model_name,55        contents=["Describe these images:", image_part_high]56    )57    print(response.text)5859### Javascript6061    // Example: Setting per-part media resolution in JavaScript62    import { GoogleGenAI, MediaResolution, Part } from '@google/genai';63    import * as fs from 'fs';64    import { Buffer } from 'buffer'; // Node.js6566    const ai = new GoogleGenAI({ httpOptions: { apiVersion: 'v1beta' } });6768    // Helper function to convert local file to a Part object69    function fileToGenerativePart(path, mimeType, mediaResolution) {70        return {71            inlineData: { data: Buffer.from(fs.readFileSync(path)).toString('base64'), mimeType },72            mediaResolution: { 'level': mediaResolution }73        };74    }7576    async function run() {77        // Create parts with different resolutions78        const imagePartHigh = fileToGenerativePart('img.png', 'image/png', Part.MediaResolutionLevel.MEDIA_RESOLUTION_HIGH);79        const model_name = 'gemini-3.1-pro-preview';80        const response = await ai.models.generateContent({81            model: model_name,82            contents: ['Describe these images:', imagePartHigh]83            // Global config can still be set, but per-part settings will override84            // config: {85            //   mediaResolution: MediaResolution.MEDIA_RESOLUTION_MEDIUM86            // }87        });88        console.log(response.text);89    }90    run();9192### REST9394    # Replace with paths to your images95    IMAGE_PATH="path/to/image.jpg"9697    # Base64 encode the images98    BASE64_IMAGE1=$(base64 -w 0 "$IMAGE_PATH")99100    MODEL_ID="gemini-3.1-pro-preview"101102    echo '{103        "contents": [{104          "parts": [105            {"text": "Describe these images:"},106            {107              "inline_data": {108                "mime_type": "image/jpeg",109                "data": "'"$BASE64_IMAGE1"'",110              },111              "media_resolution": {"level": "MEDIA_RESOLUTION_HIGH"}112            }113          ]114        }]115      }' > request.json116117    curl -s -X POST \118      "https://generativelanguage.googleapis.com/v1beta/models/${MODEL_ID}:generateContent" \119      -H "x-goog-api-key: $GEMINI_API_KEY" \120      -H "Content-Type: application/json" \121      -d @request.json122123## Global media resolution124125You can set a default resolution for all media parts in a request using the126`GenerationConfig`. This is supported by all multimodal models. If a request127includes both global and [per-part settings](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution#per-part-media-resolution), the128per-part setting takes precedence for that specific item.129130### Python131132    from google import genai133    from google.genai import types134135    client = genai.Client()136137    # Prepare standard image part138    with open('image.jpg', 'rb') as f:139        image_bytes = f.read()140    image_part = types.Part.from_bytes(data=image_bytes, mime_type='image/jpeg')141142    # Set global configuration143    config = types.GenerateContentConfig(144        media_resolution=types.MediaResolution.MEDIA_RESOLUTION_HIGH145    )146147    response = client.models.generate_content(148        model='gemini-3.8-flash',149        contents=["Describe this image:", image_part],150        config=config151    )152    print(response.text)153154### Javascript155156    import { GoogleGenAI, MediaResolution } from '@google/genai';157    import * as fs from 'fs';158159    const ai = new GoogleGenAI({ });160161    async function run() {162       // ... (Image loading logic) ...163164       const response = await ai.models.generateContent({165          model: 'gemini-3.8-flash',166          contents: ["Describe this image:", imagePart],167          config: {168             mediaResolution: MediaResolution.MEDIA_RESOLUTION_HIGH169          }170       });171       console.log(response.text);172    }173    run();174175### REST176177    # ... (Base64 encoding logic) ...178179    curl -s -X POST \180      "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \181      -H "x-goog-api-key: $GEMINI_API_KEY" \182      -H "Content-Type: application/json" \183      -d '{184        "contents": [...],185        "generation_config": {186          "media_resolution": "MEDIA_RESOLUTION_HIGH"187        }188      }'189190## Available resolution values191192The Gemini API defines the following levels for media resolution:193194- `MEDIA_RESOLUTION_UNSPECIFIED`: The default setting. The token count for this level varies significantly between Gemini 3 and earlier Gemini models.195- `MEDIA_RESOLUTION_LOW`: Lower token count, resulting in faster processing and lower cost, but with less detail.196- `MEDIA_RESOLUTION_MEDIUM`: A balance between detail, cost, and latency.197- `MEDIA_RESOLUTION_HIGH`: Higher token count, providing more detail for the model to work with, at the expense of increased latency and cost.198- `MEDIA_RESOLUTION_ULTRA_HIGH` (Per part only): Highest token count, required for specific use cases such as [computer use](https://ai.google.dev/gemini-api/docs/computer-use).199200Note that `MEDIA_RESOLUTION_HIGH` provides the optimal performance for most use201cases.202203The exact number of tokens generated for each of these levels depends on both204the **media type** (Image, Video, Audio, PDF) and the **model version**.205206## Token counts207208The tables below summarize the approximate token counts for each209`media_resolution` value and media type per model family.210211**Gemini 3 models**212213|---|---|---|---|---|214| **MediaResolution** | **Image** | **Video** | **Audio** | **PDF** |215| `MEDIA_RESOLUTION_UNSPECIFIED` (Default) | 1120 | 70 | 25 (per second) | 560 |216| `MEDIA_RESOLUTION_LOW` | 280 | 70 | 25 (per second) | 280 + Native Text |217| `MEDIA_RESOLUTION_MEDIUM` | 560 | 70 | 25 (per second) | 560 + Native Text |218| `MEDIA_RESOLUTION_HIGH` | 1120 | 280 | 25 (per second) | 1120 + Native Text |219| `MEDIA_RESOLUTION_ULTRA_HIGH` | 2240 | N/A | N/A | N/A |220221**Gemini 2.5 models**222223|---|---|---|---|---|---|224| **MediaResolution** | **Image** | **Video** | **Audio** | **PDF (Scanned)** | **PDF (Native)** |225| `MEDIA_RESOLUTION_UNSPECIFIED` (Default) | 256 + Pan \& Scan (\~2048) | 256 | 32 (per second) | 256 + OCR | 256 + Native Text |226| `MEDIA_RESOLUTION_LOW` | 64 | 64 | 32 (per second) | 64 + OCR | 64 + Native Text |227| `MEDIA_RESOLUTION_MEDIUM` | 256 | 256 | 32 (per second) | 256 + OCR | 256 + Native Text |228| `MEDIA_RESOLUTION_HIGH` | 256 + Pan \& Scan | 256 | 32 (per second) | 256 + OCR | 256 + Native Text |229230## Choosing the right resolution231232- **Default (`UNSPECIFIED`):** Start with the default. It's tuned for a good balance of quality, latency, and cost for most common use cases.233- **`LOW`:** Use for scenarios where cost and latency are paramount, and fine-grained detail is less critical.234- **`MEDIUM` / `HIGH`:** Increase the resolution when the task requires understanding intricate details within the media. This is often needed for complex visual analysis, chart reading, or dense document comprehension.235- **`ULTRA HIGH`** - Only available for per part setting. Recommended for specific use cases such as computer use or where testing shows a clear enhancement over `HIGH`.236- **Per-part control (Gemini 3):** Optimizes token usage. For example, in a prompt with multiple images, use `HIGH` for a complex diagram and `LOW` or `MEDIUM` for simpler contextual images.237238**Recommended settings**239240The following lists the recommended media resolution settings for each241supported media type.242243|---|---|---|---|244| **Media Type** | **Recommended Setting** | **Max Tokens** | **Usage Guidance** |245| **Images** | `MEDIA_RESOLUTION_HIGH` | 1120 | Recommended for most image analysis tasks to ensure maximum quality. |246| **PDFs** | `MEDIA_RESOLUTION_MEDIUM` | 560 | Optimal for document understanding; quality typically saturates at `medium`. Increasing to `high` rarely improves OCR results for standard documents. |247| **Video** (General) | `MEDIA_RESOLUTION_LOW` (or `MEDIA_RESOLUTION_MEDIUM`) | 70 (per frame) | **Note:** For video, `low` and `medium` settings are treated identically (70 tokens) to optimize context usage. This is sufficient for most action recognition and description tasks. |248| **Video** (Text-heavy) | `MEDIA_RESOLUTION_HIGH` | 280 (per frame) | Required only when the use case involves reading dense text (OCR) or small details within video frames. |249| **Audio** | `MEDIA_RESOLUTION_UNSPECIFIED` (Default) | 25 (per second) for Gemini 3; 32 (per second) for Gemini 2.5 | Audio is tokenized at a fixed rate per second across all supported resolution settings (`unspecified`, `low`, `medium`, and `high`). |250251Always test and evaluate the impact of different resolution settings on your252specific application to find the best trade-off between quality, latency, and253cost.254255## Relationship with video processing modes256257The `media_resolution` and processing parameters control different aspects of video input:258259- `media_resolution` controls the **resolution** of each frame (number of tokens per frame).260- `processing` / `media_processing` controls **which content from the video** is loaded into context.261262You can set both on the same video input. For example, you might use agentic processing with low media resolution to minimize total token usage for a long video.263264For details on video processing modes, see the [Agentic video understanding](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding#agentic-video-understanding) guide.265266## Version compatibility summary267268- The `MediaResolution` enum is available for all models supporting media input.269- The token counts associated with each enum level **differ** between Gemini 3 models and earlier Gemini versions.270- Setting `media_resolution` on individual `Part` objects is **exclusive to271  Gemini 3 models**.272273## Next steps274275- Learn more about the multimodal capabilities of Gemini API in the [image understanding](https://ai.google.dev/gemini-api/docs/generate-content/image-understanding), [video understanding](https://ai.google.dev/gemini-api/docs/generate-content/video-understanding), [audio understanding](https://ai.google.dev/gemini-api/docs/generate-content/audio), and [document understanding](https://ai.google.dev/gemini-api/docs/generate-content/document-processing) guides.
Files main
Menu

Theme