xAI image understanding — direct image input vs. view_image / view_x_video sub-tools
Status: DOCUMENTED · LIVE_VERIFIED for direct image input (2026-09-19, grok-4.3: 32×32 PNG data URL on Chat (image_url, image_tokens: 3), Responses (input_image) and Messages (image url + base64)); enable_image_understanding flags accepted; no view_image call triggered.
Sources: https://docs.x.ai/developers/model-capabilities/images/understanding · https://docs.x.ai/developers/model-capabilities/legacy/chat-completions#image-understanding · https://docs.x.ai/developers/tools/web-search#enable-image-understanding · https://docs.x.ai/developers/tools/x-search · https://docs.x.ai/developers/pricing#tools-pricing
Last verified: 2026-09-19
Direct image input (not a tool)
| API | Part | Live |
|---|---|---|
| Responses | {"type":"input_image","image_url":"https://…" | "data:image/png;base64,…","detail":"auto|low|high"} |
✅ (file_id variant → 400 "image_url must either be a base64-encoded image or a URL") |
| Chat Completions | {"type":"image_url","image_url":{"url":…,"detail":…}} |
✅ |
| Messages | {"type":"image","source":{"type":"url","url"} | {"type":"base64","media_type","data"}} |
✅ |
Limits: jpg/jpeg/png; ≤20 MiB; unlimited count; any text/image order; ≥512 total pixels (live 400 invalid_image for 8×8); detail low = fewer tokens, high = more. Docs advise store:false when sending images to Responses. All live text models list input_modalities: ["text","image"].
Media understanding during search (server sub-tools)
web_search.enable_image_understanding: true→ agent may callview_imageon pages it browses (also enables it for x_search in the same request).web_search.enable_image_search: true→search_images, results embedded asmarkdown + annotations.x_search.enable_image_understanding/enable_video_understanding→view_image/view_x_videoon posts.- Usage appears in the xai-sdk
server_side_tool_usagemap asSERVER_SIDE_TOOL_VIEW_IMAGE,SERVER_SIDE_TOOL_VIEW_X_VIDEO,SERVER_SIDE_TOOL_IMAGE_SEARCH. Billing: no invocation fee — image tokens only (image search billed as web search).
Examples: examples/xai/chat/image_input.py, examples/xai/responses/image_input.sh.