SPB Git

Repository Public

image-understanding.md 25.8 KB 860 lines markdown

1Gemini models are built to be multimodal from the ground up, unlocking a wide2range of image processing and computer vision tasks including but not limited to3image captioning, classification, and visual question answering without having4to train specialized ML models.56In addition to their general multimodal capabilities, Gemini models offer7**enhanced accuracy** for specific use cases like [object detection](https://ai.google.dev/gemini-api/docs/generate-content/image-understanding#object-detection), through additional8training.910## Passing images to Gemini1112You can provide images as input to Gemini using two methods:1314- [Passing inline image data](https://ai.google.dev/gemini-api/docs/generate-content/image-understanding#inline-image): Ideal for smaller files (total request size less than 20MB, including prompts).15- [Uploading images using the File API](https://ai.google.dev/gemini-api/docs/generate-content/image-understanding#upload-image): Recommended for larger files or for reusing images across multiple requests.1617### Passing inline image data1819You can pass inline image data in the20request to `generateContent`. You can provide image data as Base64 encoded21strings or by reading local files directly (depending on the language).2223The following example shows how to read an image from a local file and pass24it to `generateContent` API for processing.2526### Python2728      from google import genai29      from google.genai import types3031      with open('path/to/small-sample.jpg', 'rb') as f:32          image_bytes = f.read()3334      client = genai.Client()35      response = client.models.generate_content(36        model='gemini-3.8-flash',37        contents=[38          types.Part.from_bytes(39            data=image_bytes,40            mime_type='image/jpeg',41          ),42          'Caption this image.'43        ]44      )4546      print(response.text)4748### JavaScript4950    import { GoogleGenAI } from "@google/genai";51    import * as fs from "node:fs";5253    const ai = new GoogleGenAI({});54    const base64ImageFile = fs.readFileSync("path/to/small-sample.jpg", {55      encoding: "base64",56    });5758    const contents = [59      {60        inlineData: {61          mimeType: "image/jpeg",62          data: base64ImageFile,63        },64      },65      { text: "Caption this image." },66    ];6768    const response = await ai.models.generateContent({69      model: "gemini-3.8-flash",70      contents: contents,71    });72    console.log(response.text);7374### Go7576    bytes, _ := os.ReadFile("path/to/small-sample.jpg")7778    parts := []*genai.Part{79      genai.NewPartFromBytes(bytes, "image/jpeg"),80      genai.NewPartFromText("Caption this image."),81    }8283    contents := []*genai.Content{84      genai.NewContentFromParts(parts, genai.RoleUser),85    }8687    result, _ := client.Models.GenerateContent(88      ctx,89      "gemini-3.8-flash",90      contents,91      nil,92    )9394    fmt.Println(result.Text())9596### REST9798    IMG_PATH="/path/to/your/image1.jpg"99100    if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then101    B64FLAGS="--input"102    else103    B64FLAGS="-w0"104    fi105106    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \107    -H "x-goog-api-key: $GEMINI_API_KEY" \108    -H 'Content-Type: application/json' \109    -X POST \110    -d '{111        "contents": [{112        "parts":[113            {114                "inline_data": {115                "mime_type":"image/jpeg",116                "data": "'"$(base64 $B64FLAGS $IMG_PATH)"'"117                }118            },119            {"text": "Caption this image."},120        ]121        }]122    }' 2> /dev/null123124You can also fetch an image from a URL, convert it to bytes, and pass it to125`generateContent` as shown in the following examples.126127### Python128129    from google import genai130    from google.genai import types131132    import requests133134    image_path = "https://goo.gle/instrument-img"135    image_bytes = requests.get(image_path).content136    image = types.Part.from_bytes(137      data=image_bytes, mime_type="image/jpeg"138    )139140    client = genai.Client()141142    response = client.models.generate_content(143        model="gemini-3.8-flash",144        contents=["What is this image?", image],145    )146147    print(response.text)148149### JavaScript150151    import { GoogleGenAI } from "@google/genai";152153    async function main() {154      const ai = new GoogleGenAI({});155156      const imageUrl = "https://goo.gle/instrument-img";157158      const response = await fetch(imageUrl);159      const imageArrayBuffer = await response.arrayBuffer();160      const base64ImageData = Buffer.from(imageArrayBuffer).toString('base64');161162      const result = await ai.models.generateContent({163        model: "gemini-3.8-flash",164        contents: [165        {166          inlineData: {167            mimeType: 'image/jpeg',168            data: base64ImageData,169          },170        },171        { text: "Caption this image." }172      ],173      });174      console.log(result.text);175    }176177    main();178179### Go180181    package main182183    import (184      "context"185      "fmt"186      "os"187      "io"188      "net/http"189      "google.golang.org/genai"190    )191192    func main() {193      ctx := context.Background()194      client, err := genai.NewClient(ctx, nil)195      if err != nil {196          log.Fatal(err)197      }198199      // Download the image.200      imageResp, _ := http.Get("https://goo.gle/instrument-img")201202      imageBytes, _ := io.ReadAll(imageResp.Body)203204      parts := []*genai.Part{205        genai.NewPartFromBytes(imageBytes, "image/jpeg"),206        genai.NewPartFromText("Caption this image."),207      }208209      contents := []*genai.Content{210        genai.NewContentFromParts(parts, genai.RoleUser),211      }212213      result, _ := client.Models.GenerateContent(214        ctx,215        "gemini-3.8-flash",216        contents,217        nil,218      )219220      fmt.Println(result.Text())221    }222223### REST224225    IMG_URL="https://goo.gle/instrument-img"226227    MIME_TYPE=$(curl -sIL "$IMG_URL" | grep -i '^content-type:' | awk -F ': ' '{print $2}' | sed 's/\r$//' | head -n 1)228    if [[ -z "$MIME_TYPE" || ! "$MIME_TYPE" == image/* ]]; then229      MIME_TYPE="image/jpeg"230    fi231232    # Check for macOS233    if [[ "$(uname)" == "Darwin" ]]; then234      IMAGE_B64=$(curl -sL "$IMG_URL" | base64 -b 0)235    elif [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then236      IMAGE_B64=$(curl -sL "$IMG_URL" | base64)237    else238      IMAGE_B64=$(curl -sL "$IMG_URL" | base64 -w0)239    fi240241    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \242        -H "x-goog-api-key: $GEMINI_API_KEY" \243        -H 'Content-Type: application/json' \244        -X POST \245        -d '{246          "contents": [{247            "parts":[248                {249                  "inline_data": {250                    "mime_type":"'"$MIME_TYPE"'",251                    "data": "'"$IMAGE_B64"'"252                  }253                },254                {"text": "Caption this image."}255            ]256          }]257        }' 2> /dev/null258259> [!NOTE]260> **Note:** Inline image data limits your total request size (text prompts, system instructions, and inline bytes) to 20MB. For larger requests, [upload image files](https://ai.google.dev/gemini-api/docs/generate-content/image-understanding#upload-image) using the File API. Files API is also more efficient for scenarios that use the same image repeatedly.261262### Uploading images using the File API263264For large files or to be able to use the same image file repeatedly, use the265Files API. The following code uploads an image file and then uses the file in a266call to `generateContent`. See the [Files API guide](https://ai.google.dev/gemini-api/docs/files) for267more information and examples.268269### Python270271    from google import genai272273    client = genai.Client()274275    my_file = client.files.upload(file="path/to/sample.jpg")276277    response = client.models.generate_content(278        model="gemini-3.8-flash",279        contents=[my_file, "Caption this image."],280    )281282    print(response.text)283284### JavaScript285286    import {287      GoogleGenAI,288      createUserContent,289      createPartFromUri,290    } from "@google/genai";291292    const ai = new GoogleGenAI({});293294    async function main() {295      const myfile = await ai.files.upload({296        file: "path/to/sample.jpg",297        config: { mimeType: "image/jpeg" },298      });299300      const response = await ai.models.generateContent({301        model: "gemini-3.8-flash",302        contents: createUserContent([303          createPartFromUri(myfile.uri, myfile.mimeType),304          "Caption this image.",305        ]),306      });307      console.log(response.text);308    }309310    await main();311312### Go313314    package main315316    import (317      "context"318      "fmt"319      "os"320      "google.golang.org/genai"321    )322323    func main() {324      ctx := context.Background()325      client, err := genai.NewClient(ctx, nil)326      if err != nil {327          log.Fatal(err)328      }329330      uploadedFile, _ := client.Files.UploadFromPath(ctx, "path/to/sample.jpg", nil)331332      parts := []*genai.Part{333          genai.NewPartFromText("Caption this image."),334          genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),335      }336337      contents := []*genai.Content{338          genai.NewContentFromParts(parts, genai.RoleUser),339      }340341      result, _ := client.Models.GenerateContent(342          ctx,343          "gemini-3.8-flash",344          contents,345          nil,346      )347348      fmt.Println(result.Text())349    }350351### REST352353    IMAGE_PATH="path/to/sample.jpg"354    MIME_TYPE=$(file -b --mime-type "${IMAGE_PATH}")355    NUM_BYTES=$(wc -c < "${IMAGE_PATH}")356    DISPLAY_NAME=IMAGE357358    tmp_header_file=upload-header.tmp359360    # Initial resumable request defining metadata.361    # The upload url is in the response headers dump them to a file.362    curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \363      -H "x-goog-api-key: $GEMINI_API_KEY" \364      -D upload-header.tmp \365      -H "X-Goog-Upload-Protocol: resumable" \366      -H "X-Goog-Upload-Command: start" \367      -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \368      -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \369      -H "Content-Type: application/json" \370      -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null371372    upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")373    rm "${tmp_header_file}"374375    # Upload the actual bytes.376    curl "${upload_url}" \377      -H "x-goog-api-key: $GEMINI_API_KEY" \378      -H "Content-Length: ${NUM_BYTES}" \379      -H "X-Goog-Upload-Offset: 0" \380      -H "X-Goog-Upload-Command: upload, finalize" \381      --data-binary "@${IMAGE_PATH}" 2> /dev/null > file_info.json382383    file_uri=$(jq -r ".file.uri" file_info.json)384    echo file_uri=$file_uri385386    # Now generate content using that file387    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \388        -H "x-goog-api-key: $GEMINI_API_KEY" \389        -H 'Content-Type: application/json' \390        -X POST \391        -d '{392          "contents": [{393            "parts":[394              {"file_data":{"mime_type": "'"${MIME_TYPE}"'", "file_uri": "'"${file_uri}"'"}},395              {"text": "Caption this image."}]396            }]397          }' 2> /dev/null > response.json398399    cat response.json400    echo401402    jq ".candidates[].content.parts[].text" response.json403404## Prompting with multiple images405406You can provide multiple images in a single prompt by including multiple image407`Part` objects in the `contents` array. These can be a mix of inline data408(local files or URLs) and File API references.409410### Python411412    from google import genai413    from google.genai import types414415    client = genai.Client()416417    # Upload the first image418    image1_path = "path/to/image1.jpg"419    uploaded_file = client.files.upload(file=image1_path)420421    # Prepare the second image as inline data422    image2_path = "path/to/image2.png"423    with open(image2_path, 'rb') as f:424        img2_bytes = f.read()425426    # Create the prompt with text and multiple images427    response = client.models.generate_content(428429        model="gemini-3.8-flash",430        contents=[431            "What is different between these two images?",432            uploaded_file,  # Use the uploaded file reference433            types.Part.from_bytes(434                data=img2_bytes,435                mime_type='image/png'436            )437        ]438    )439440    print(response.text)441442### JavaScript443444    import {445      GoogleGenAI,446      createUserContent,447      createPartFromUri,448    } from "@google/genai";449    import * as fs from "node:fs";450451    const ai = new GoogleGenAI({});452453    async function main() {454      // Upload the first image455      const image1_path = "path/to/image1.jpg";456      const uploadedFile = await ai.files.upload({457        file: image1_path,458        config: { mimeType: "image/jpeg" },459      });460461      // Prepare the second image as inline data462      const image2_path = "path/to/image2.png";463      const base64Image2File = fs.readFileSync(image2_path, {464        encoding: "base64",465      });466467      // Create the prompt with text and multiple images468469      const response = await ai.models.generateContent({470471        model: "gemini-3.8-flash",472        contents: createUserContent([473          "What is different between these two images?",474          createPartFromUri(uploadedFile.uri, uploadedFile.mimeType),475          {476            inlineData: {477              mimeType: "image/png",478              data: base64Image2File,479            },480          },481        ]),482      });483      console.log(response.text);484    }485486    await main();487488### Go489490    // Upload the first image491    image1Path := "path/to/image1.jpg"492    uploadedFile, _ := client.Files.UploadFromPath(ctx, image1Path, nil)493494    // Prepare the second image as inline data495    image2Path := "path/to/image2.jpeg"496    imgBytes, _ := os.ReadFile(image2Path)497498    parts := []*genai.Part{499      genai.NewPartFromText("What is different between these two images?"),500      genai.NewPartFromBytes(imgBytes, "image/jpeg"),501      genai.NewPartFromURI(uploadedFile.URI, uploadedFile.MIMEType),502    }503504    contents := []*genai.Content{505      genai.NewContentFromParts(parts, genai.RoleUser),506    }507508    result, _ := client.Models.GenerateContent(509      ctx,510      "gemini-3.8-flash",511      contents,512      nil,513    )514515    fmt.Println(result.Text())516517### REST518519    # Upload the first image520    IMAGE1_PATH="path/to/image1.jpg"521    MIME1_TYPE=$(file -b --mime-type "${IMAGE1_PATH}")522    NUM1_BYTES=$(wc -c < "${IMAGE1_PATH}")523    DISPLAY_NAME1=IMAGE1524525    tmp_header_file1=upload-header1.tmp526527    curl "https://generativelanguage.googleapis.com/upload/v1beta/files" \528      -H "x-goog-api-key: $GEMINI_API_KEY" \529      -D upload-header1.tmp \530      -H "X-Goog-Upload-Protocol: resumable" \531      -H "X-Goog-Upload-Command: start" \532      -H "X-Goog-Upload-Header-Content-Length: ${NUM1_BYTES}" \533      -H "X-Goog-Upload-Header-Content-Type: ${MIME1_TYPE}" \534      -H "Content-Type: application/json" \535      -d "{'file': {'display_name': '${DISPLAY_NAME1}'}}" 2> /dev/null536537    upload_url1=$(grep -i "x-goog-upload-url: " "${tmp_header_file1}" | cut -d" " -f2 | tr -d "\r")538    rm "${tmp_header_file1}"539540    curl "${upload_url1}" \541      -H "Content-Length: ${NUM1_BYTES}" \542      -H "X-Goog-Upload-Offset: 0" \543      -H "X-Goog-Upload-Command: upload, finalize" \544      --data-binary "@${IMAGE1_PATH}" 2> /dev/null > file_info1.json545546    file1_uri=$(jq ".file.uri" file_info1.json)547    echo file1_uri=$file1_uri548549    # Prepare the second image (inline)550    IMAGE2_PATH="path/to/image2.png"551    MIME2_TYPE=$(file -b --mime-type "${IMAGE2_PATH}")552553    if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then554      B64FLAGS="--input"555    else556      B64FLAGS="-w0"557    fi558    IMAGE2_BASE64=$(base64 $B64FLAGS $IMAGE2_PATH)559560    # Now generate content using both images561    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \562        -H "x-goog-api-key: $GEMINI_API_KEY" \563        -H 'Content-Type: application/json' \564        -X POST \565        -d '{566          "contents": [{567            "parts":[568              {"text": "What is different between these two images?"},569              {"file_data":{"mime_type": "'"${MIME1_TYPE}"'", "file_uri": '$file1_uri'}},570              {571                "inline_data": {572                  "mime_type":"'"${MIME2_TYPE}"'",573                  "data": "'"$IMAGE2_BASE64"'"574                }575              }576            ]577          }]578        }' 2> /dev/null > response.json579580    cat response.json581    echo582583    jq ".candidates[].content.parts[].text" response.json584585## Object detection586587Models are trained to detect objects in an588image and get their bounding box coordinates. The coordinates, relative to image589dimensions, scale to \[0, 1000\]. You need to descale these coordinates based on590your original image size.591592### Python593594    from google import genai595    from google.genai import types596    from PIL import Image597    import json598599    client = genai.Client()600    prompt = "Detect the all of the prominent items in the image. The box_2d should be [ymin, xmin, ymax, xmax] normalized to 0-1000."601602    image = Image.open("/path/to/image.png")603604    config = types.GenerateContentConfig(605      response_mime_type="application/json"606      )607608    response = client.models.generate_content(model="gemini-3.8-flash",609                                              contents=[image, prompt],610                                              config=config611                                              )612613    width, height = image.size614    bounding_boxes = json.loads(response.text)615616    converted_bounding_boxes = []617    for bounding_box in bounding_boxes:618        abs_y1 = int(bounding_box["box_2d"][0]/1000 * height)619        abs_x1 = int(bounding_box["box_2d"][1]/1000 * width)620        abs_y2 = int(bounding_box["box_2d"][2]/1000 * height)621        abs_x2 = int(bounding_box["box_2d"][3]/1000 * width)622        converted_bounding_boxes.append([abs_x1, abs_y1, abs_x2, abs_y2])623624    print("Image size: ", width, height)625    print("Bounding boxes:", converted_bounding_boxes)626627### JavaScript628629    import { GoogleGenAI } from "@google/genai";630    import * as fs from "node:fs";631632    const ai = new GoogleGenAI({});633    const base64ImageFile = fs.readFileSync("/path/to/image.png", {634      encoding: "base64",635    });636637    const response = await ai.models.generateContent({638      model: "gemini-3.8-flash",639      contents: [640        {641          inlineData: {642            mimeType: "image/png",643            data: base64ImageFile,644          },645        },646        "Detect the all of the prominent items in the image. The box_2d should be [ymin, xmin, ymax, xmax] normalized to 0-1000."647      ],648      config: {649        responseMimeType: "application/json",650      },651    });652653    const boundingBoxes = JSON.parse(response.text);654    console.log(boundingBoxes);655    // To convert normalized coordinates to absolute pixels:656    // const absY1 = (boundingBoxes[0].box_2d[0] / 1000) * imageHeight;657    // const absX1 = (boundingBoxes[0].box_2d[1] / 1000) * imageWidth;658659### Go660661    package main662663    import (664      "context"665      "encoding/json"666      "fmt"667      "image"668      _ "image/png" // Register PNG decoder669      "log"670      "os"671672      "google.golang.org/genai"673    )674675    type BoundingBox struct {676      Box2D []int  `json:"box_2d"`677      Label string `json:"label"`678    }679680    func main() {681      ctx := context.Background()682      client, err := genai.NewClient(ctx, nil)683      if err != nil {684        log.Fatal(err)685      }686687      imagePath := "/path/to/image.png"688689      // Open the image to get dimensions690      file, err := os.Open(imagePath)691      if err != nil {692        log.Fatal(err)693      }694      defer file.Close()695696      imgConfig, _, err := image.DecodeConfig(file)697      if err != nil {698        log.Fatal(err)699      }700      width := imgConfig.Width701      height := imgConfig.Height702703      // Read image bytes704      imageBytes, err := os.ReadFile(imagePath)705      if err != nil {706        log.Fatal(err)707      }708709      prompt := "Detect the all of the prominent items in the image. The box_2d should be [ymin, xmin, ymax, xmax] normalized to 0-1000."710711      parts := []*genai.Part{712        genai.NewPartFromBytes(imageBytes, "image/png"),713        genai.NewPartFromText(prompt),714      }715716      contents := []*genai.Content{717        genai.NewContentFromParts(parts, genai.RoleUser),718      }719720      config := &genai.GenerateContentConfig{721        ResponseMIMEType: "application/json",722      }723724      result, err := client.Models.GenerateContent(725        ctx,726        "gemini-3.8-flash",727        contents,728        config,729      )730      if err != nil {731        log.Fatal(err)732      }733734      var boundingBoxes []BoundingBox735      err = json.Unmarshal([]byte(result.Text()), &boundingBoxes)736      if err != nil {737        log.Fatal(err)738      }739740      fmt.Printf("Image size: %d %d\n", width, height)741      fmt.Println("Bounding boxes:")742      for _, box := range boundingBoxes {743        if len(box.Box2D) == 4 {744          absY1 := int(float64(box.Box2D[0]) / 1000.0 * float64(height))745          absX1 := int(float64(box.Box2D[1]) / 1000.0 * float64(width))746          absY2 := int(float64(box.Box2D[2]) / 1000.0 * float64(height))747          absX2 := int(float64(box.Box2D[3]) / 1000.0 * float64(width))748          fmt.Printf("- %s: [%d, %d, %d, %d]\n", box.Label, absX1, absY1, absX2, absY2)749        }750      }751    }752753### REST754755    IMG_PATH="/path/to/image.png"756757    if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then758      B64FLAGS="--input"759    else760      B64FLAGS="-w0"761    fi762763    curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \764      -H "x-goog-api-key: $GEMINI_API_KEY" \765      -H 'Content-Type: application/json' \766      -X POST \767      -d '{768        "contents": [{769          "parts":[770            {771              "inline_data": {772                "mime_type":"image/png",773                "data": "'"$(base64 $B64FLAGS $IMG_PATH)"'"774              }775            },776            {"text": "Detect the all of the prominent items in the image. The box_2d should be [ymin, xmin, ymax, xmax] normalized to 0-1000."}777          ]778        }],779        "generationConfig": {780          "responseMimeType": "application/json"781        }782      }' 2> /dev/null783784> [!NOTE]785> **Note:** The model also supports generating bounding boxes based on custom instructions, such as: "Show bounding boxes of all green objects in this image". It also support custom labels like "label the items with the allergens they can contain".786787For more examples, check following notebooks in the [Gemini Cookbook](https://github.com/google-gemini/cookbook):788789- [2D spatial understanding notebook](https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Spatial_understanding.ipynb)790- [Experimental 3D pointing notebook](https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/Spatial_understanding_3d.ipynb)791792## Supported image formats793794Gemini supports the following image format MIME types:795796- PNG - `image/png`797- JPEG - `image/jpeg`798- WEBP - `image/webp`799- HEIC - `image/heic`800- HEIF - `image/heif`801802To learn about other file input methods, see the803[File input methods](https://ai.google.dev/gemini-api/docs/file-input-methods) guide.804805## Capabilities806807All Gemini model versions are multimodal and can be utilized in a wide range of808image processing and computer vision tasks including but not limited to image captioning,809visual question and answering, image classification, and object detection.810811Gemini can reduce the need to use specialized ML models depending on your quality and performance requirements.812813The latest model versions are specifically trained improve accuracy of814specialized tasks in addition to generic capabilities, like enhanced815[object detection](https://ai.google.dev/gemini-api/docs/generate-content/image-understanding#object-detection).816817## Limitations and key technical information818819### File limit820821Gemini models support a maximum of 3,600 image files per request.822823### Token calculation824825- 258 tokens if both dimensions \<= 384 pixels. Larger images are tiled into 768x768 pixel tiles, each costing 258 tokens.826827A rough formula for calculating the number of tiles is as follows:828829- Calculate the crop unit size which is roughly: floor(min(width, height) / 1.5).830- Divide each dimension by the crop unit size and multiply together to get the number of tiles.831832For example, for an image of dimensions 960x540 would have a crop unit size833of 360. Divide each dimension by 360 and the number of tile is 3 \* 2 = 6.834835### Media resolution836837Gemini 3 introduces granular control over multimodal vision processing with the838`media_resolution` parameter. The `media_resolution` parameter determines the839**maximum number of tokens allocated per input image or video frame.**840Higher resolutions improve the model's ability to841read fine text or identify small details, but increase token usage and latency.842843For more details about the parameter and how it can impact token calculations,844see the [media resolution](https://ai.google.dev/gemini-api/docs/generate-content/media-resolution) guide.845846## Tips and best practices847848- Verify that images are correctly rotated.849- Use clear, non-blurry images.850- When using a single image with text, place the text prompt *after* the image part in the `contents` array.851852## What's next853854This guide shows you how to upload image files and generate text outputs from image855inputs. To learn more, see the following resources:856857- [Files API](https://ai.google.dev/gemini-api/docs/files): Learn more about uploading and managing files for use with Gemini.858- [System instructions](https://ai.google.dev/gemini-api/docs/text-generation#system-instructions): System instructions let you steer the behavior of the model based on your specific needs and use cases.859- [File prompting strategies](https://ai.google.dev/gemini-api/docs/files#prompt-guide): The Gemini API supports prompting with text, image, audio, and video data, also known as multimodal prompting.860- [Safety guidance](https://ai.google.dev/gemini-api/docs/safety-guidance): Sometimes generative AI models produce unexpected outputs, such as outputs that are inaccurate, biased, or offensive. Post-processing and human evaluation are essential to limit the risk of harm from such outputs.
Files main
Menu

Theme