Repository Public
1<!-- source: https://docs.x.ai/developers/model-capabilities/images/understanding (from https://docs.x.ai/llms-full.txt, retrieved 2026-09-19T05:28:55Z) -->2#### Model Capabilities34# Image Understanding56> [!WARNING]7>8> When sending images, it is advised to not store request/response history on the server. Otherwise the request may fail.9> See .1011Some models allow images in the input. The model will consider the image context when generating the response.1213## Constructing the message body - difference from text-only prompt1415The request message to image understanding is similar to a text-only prompt. The main difference is that instead of text input:1617```json18[19 {20 "role": "user",21 "content": "What is in this image?"22 }23]24```2526We send in `content` as a list of objects:2728```json29[30 {31 "role": "user",32 "content": [33 {34 "type": "input_image",35 "image_url": "data:image/jpeg;base64,<base64_image_string>"36 },37 {38 "type": "input_text",39 "text": "What is in this image?"40 }41 ]42 }43]44```4546The `image_url` value can also be a public URL on the Internet instead of a base64 data URL.4748### Image understanding example4950```python customLanguage="pythonXAI"51import os52from xai_sdk import Client53from xai_sdk.chat import user, image5455client = Client(56 api_key=os.getenv("XAI_API_KEY"),57 management_api_key=os.getenv("XAI_MANAGEMENT_API_KEY"),58 timeout=3600,59)6061image_url = "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png"62chat = client.chat.create(model="grok-4.6")63chat.append(64 user(65 "What's in this image?",66 image(image_url=image_url, detail="high"),67 )68)6970response = chat.sample()71print(response)7273# The response ID that can be used to continue the conversation later7475print(response.id)76```7778```python customLanguage="pythonOpenAISDK"79import os80import httpx81from openai import OpenAI8283client = OpenAI(84 api_key="<YOUR_XAI_API_KEY_HERE>",85 base_url="https://api.x.ai/v1",86 timeout=httpx.Timeout(3600.0), # Override default timeout with longer timeout for reasoning models87)88image_url = (89 "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png"90)9192response = client.responses.create(93 model="grok-4.6",94 input=[95 {96 "role": "user",97 "content": [98 {99 "type": "input_image",100 "image_url": image_url,101 "detail": "high",102 },103 {104 "type": "input_text",105 "text": "What's in this image?",106 },107 ],108 },109 ],110)111112print(response)113114# The response ID that can be used to continue the conversation later115116print(response.id)117```118119```javascript customLanguage="javascriptOpenAISDK"120import OpenAI from "openai";121122const client = new OpenAI({123 apiKey: "<api key>",124 baseURL: "https://api.x.ai/v1",125 timeout: 360000, // Override default timeout with longer timeout for reasoning models126});127128const image_url =129 "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png";130131const response = await client.responses.create({132 model: "grok-4.6",133 input: [134 {135 role: "user",136 content: [137 {138 type: "input_image",139 image_url: image_url,140 detail: "high",141 },142 {143 type: "input_text",144 text: "What's in this image?",145 },146 ],147 },148 ],149});150151console.log(response);152153// The response ID that can be used to recall the conversation later154console.log(response.id);155```156157```javascript customLanguage="javascriptAISDK"158import { xai } from '@ai-sdk/xai';159import { generateText } from 'ai';160161const { text, response } = await generateText({162 model: xai.responses('grok-4.6'),163 messages: [164 {165 role: 'user',166 content: [167 {168 type: 'image',169 image: new URL('https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png'),170 },171 {172 type: 'text',173 text: "What's in this image?",174 },175 ],176 },177 ]178});179180console.log(text);181182// The response ID can be used to continue the conversation183console.log(response.id);184```185186```bash187curl https://api.x.ai/v1/responses \188 -H "Content-Type: application/json" \189 -H "Authorization: Bearer $XAI_API_KEY" \190 -m 3600 \191 -d '{192 "model": "grok-4.6",193 "input": [194 {195 "role": "user",196 "content": [197 {198 "type": "input_image",199 "image_url": "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png",200 "detail": "high"201 },202 {203 "type": "input_text",204 "text": "What'\''s in this image?"205 }206 ]207 }208 ]209 }'210```211212### Image input general limits213214* Maximum image size: `20MiB`215* Maximum number of images: No limit216* Supported image file types: `jpg/jpeg` or `png`.217* Any image/text input order is accepted (e.g. text prompt can precede image prompt)218