Repository Public
1---2title: Computer use tool3url: https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool4description: Give Claude screenshot, mouse, and keyboard control of a desktop environment with the computer use tool, the computer_toolset_20260801 client toolset.5---67## Compatibility8- [ZDR](https://platform.claude.com/docs/en/manage-claude/api-and-data-retention): eligible (excludes [Covered Models](https://platform.claude.com/docs/en/manage-claude/api-and-data-retention#model-specific-data-retention-requirements))9- Supported models: `claude-fable-5-1`, `claude-mythos-5-1`, `claude-fable-5`, `claude-mythos-5`, `claude-opus-5`, `claude-sonnet-5`, `claude-opus-4-8`10- Platforms: Claude API, Claude Platform on AWS (beta), Amazon Bedrock (beta), Google Cloud, Microsoft Foundry (beta)11- Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 4.6, and Claude Opus 4.5 support computer use only through the earlier `computer_20251124` tool version, which requires a beta header; see [Earlier tool versions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#earlier-tool-versions).12- Platforms other than the Claude API and Google Cloud currently offer only the [earlier beta tool versions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#earlier-tool-versions).1314Claude can interact with computer environments through the computer use tool, which provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction.1516The computer use tool is an Anthropic-defined [client toolset](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference#client-toolsets): one `{"type": "computer_toolset_20260801"}` entry in `tools` gives Claude 17 member tools such as `screenshot`, `left_click`, `type`, and `zoom`, and your application runs every call in an environment you control. It isn't currently available in [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/tools). Claude's calls are `tool_use` blocks whose `name` is the member and which carry `"toolset_name": "computer"`, often several per turn (a [batch action](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions)).1718For tasks that stay inside webpages, the [browser use tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/browser-use-tool) is the closer fit: its member tools read and act on the page itself, and it doesn't need a full desktop environment.1920<Note>21 Computer use is available on the Claude API and [Google Cloud](https://platform.claude.com/docs/en/build-with-claude/claude-on-vertex-ai) as the `computer_toolset_20260801` toolset; see [Compatibility](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#compatibility) for the supported models.2223 Existing `computer_20251124` integrations keep working, and earlier tool versions remain available in beta for models and platforms that don't support the toolset. See [Migrate from `computer_20251124`](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#migrate-from-computer-20251124) to upgrade, or [Earlier tool versions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#earlier-tool-versions) for the beta headers.24</Note>2526## Security considerations2728Computer use has unique risks distinct from standard API features. These risks are heightened when interacting with the internet.2930<Warning>31 To minimize risks, consider taking precautions such as:3233 1. Using a dedicated virtual machine or container with minimal privileges to prevent direct system attacks or accidents.34 2. Avoiding giving the model access to sensitive data, such as account login information, to prevent information theft.35 3. Limiting internet access to an allowlist of domains to reduce exposure to malicious content.36 4. Asking a human to confirm decisions that might result in meaningful real-world consequences and any tasks requiring affirmative consent, such as accepting cookies, completing financial transactions, or agreeing to terms of service.37</Warning>3839In some circumstances, Claude will follow commands found in content even when they conflict with your instructions. For example, instructions on webpages or contained in images might override your instructions or cause Claude to make mistakes. Take precautions to isolate Claude from sensitive data and actions to avoid risks related to prompt injection.4041Anthropic has trained the model to resist these prompt injections and has added an extra layer of defense. If you use the computer use tools, classifiers will automatically scan what the tools return, such as screenshots, to flag potential prompt injections. When these classifiers identify a potential prompt injection, they will automatically steer the model to check whether the instruction really came from you before acting on it.4243This extra protection won't be ideal for every use case (for example, use cases without a human in the loop), so if you'd like to opt out and turn it off, [contact support](https://support.claude.com/en/). The precautions above remain important even with these classifiers in place.4445Inform end users of relevant risks and obtain their consent prior to enabling computer use in your own products.4647## Quick start4849Add the computer use toolset to the `tools` array of a [Messages API](https://platform.claude.com/docs/en/api/messages/create) request as `{"type": "computer_toolset_20260801"}`. The request needs no beta header. This example also declares the [text editor tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool) and [bash tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/bash-tool), which Claude typically uses alongside computer use:5051<CodeGroup>52 ```bash cURL53 curl https://api.anthropic.com/v1/messages \54 -H "content-type: application/json" \55 -H "x-api-key: $ANTHROPIC_API_KEY" \56 -H "anthropic-version: 2023-06-01" \57 -d '{58 "model": "claude-opus-5",59 "max_tokens": 1024,60 "tools": [61 {62 "type": "computer_toolset_20260801"63 },64 {65 "type": "text_editor_20250728",66 "name": "str_replace_based_edit_tool"67 },68 {69 "type": "bash_20250124",70 "name": "bash"71 }72 ],73 "messages": [74 {75 "role": "user",76 "content": "Save a picture of a cat to my desktop."77 }78 ]79 }'80 ```8182 ```bash CLI83 ant messages create <<'YAML'84 model: claude-opus-585 max_tokens: 102486 tools:87 - type: computer_toolset_2026080188 - type: text_editor_2025072889 name: str_replace_based_edit_tool90 - type: bash_2025012491 name: bash92 messages:93 - role: user94 content: Save a picture of a cat to my desktop.95 YAML96 ```9798 ```python Python99 client = anthropic.Anthropic()100101 response = client.messages.create(102 model="claude-opus-5",103 max_tokens=1024,104 tools=[105 {"type": "computer_toolset_20260801"},106 {"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"},107 {"type": "bash_20250124", "name": "bash"},108 ],109 messages=[{"role": "user", "content": "Save a picture of a cat to my desktop."}],110 )111 print(response)112 ```113114 ```typescript TypeScript115 const client = new Anthropic();116117 const response = await client.messages.create({118 model: "claude-opus-5",119 max_tokens: 1024,120 tools: [121 {122 type: "computer_toolset_20260801"123 },124 {125 type: "text_editor_20250728",126 name: "str_replace_based_edit_tool"127 },128 {129 type: "bash_20250124",130 name: "bash"131 }132 ],133 messages: [{ role: "user", content: "Save a picture of a cat to my desktop." }]134 });135136 console.log(response);137 ```138139 ```csharp C#140 var client = new AnthropicClient();141142 var parameters = new MessageCreateParams143 {144 Model = Model.ClaudeOpus5,145 MaxTokens = 1024,146 Tools =147 [148 new ComputerToolset20260801(),149 new ToolTextEditor20250728(),150 new ToolBash20250124(),151 ],152 Messages =153 [154 new MessageParam155 {156 Role = Role.User,157 Content = "Save a picture of a cat to my desktop.",158 },159 ],160 };161162 var response = await client.Messages.Create(parameters);163 Console.WriteLine(response);164 ```165166 ```go Go167 client := anthropic.NewClient()168169 response, err := client.Messages.New(context.TODO(), anthropic.MessageNewParams{170 Model: anthropic.ModelClaudeOpus5,171 MaxTokens: 1024,172 Tools: []anthropic.ToolUnionParam{173 {OfComputerToolset20260801: &anthropic.ComputerToolset20260801Param{}},174 {OfTextEditor20250728: &anthropic.ToolTextEditor20250728Param{}},175 {OfBashTool20250124: &anthropic.ToolBash20250124Param{}},176 },177 Messages: []anthropic.MessageParam{178 anthropic.NewUserMessage(anthropic.NewTextBlock("Save a picture of a cat to my desktop.")),179 },180 })181 if err != nil {182 log.Fatal(err)183 }184 fmt.Println(response.RawJSON())185 ```186187 ```java Java188 import com.anthropic.models.messages.ComputerToolset20260801;189 // ...190 import com.anthropic.models.messages.ToolBash20250124;191 import com.anthropic.models.messages.ToolTextEditor20250728;192193 void main() {194 AnthropicClient client = AnthropicOkHttpClient.fromEnv();195196 MessageCreateParams params = MessageCreateParams.builder()197 .model(Model.CLAUDE_OPUS_5)198 .maxTokens(1024L)199 .addTool(ComputerToolset20260801.builder().build())200 .addTool(ToolTextEditor20250728.builder().build())201 .addTool(ToolBash20250124.builder().build())202 .addUserMessage("Save a picture of a cat to my desktop.")203 .build();204205 Message response = client.messages().create(params);206 IO.println(response);207 }208 ```209210 ```php PHP211 $client = new Client();212213 $response = $client->messages->create(214 maxTokens: 1024,215 messages: [216 ['role' => 'user', 'content' => 'Save a picture of a cat to my desktop.'],217 ],218 model: 'claude-opus-5',219 tools: [220 ['type' => 'computer_toolset_20260801'],221 [222 'type' => 'text_editor_20250728',223 'name' => 'str_replace_based_edit_tool',224 ],225 [226 'type' => 'bash_20250124',227 'name' => 'bash',228 ],229 ],230 );231232 echo $response;233 ```234235 ```ruby Ruby236 client = Anthropic::Client.new237238 response = client.messages.create(239 model: "claude-opus-5",240 max_tokens: 1024,241 tools: [242 { type: "computer_toolset_20260801" },243 {244 type: "text_editor_20250728",245 name: "str_replace_based_edit_tool"246 },247 {248 type: "bash_20250124",249 name: "bash"250 }251 ],252 messages: [253 { role: "user", content: "Save a picture of a cat to my desktop." }254 ]255 )256257 puts response258 ```259</CodeGroup>260261When Claude acts on the desktop, the response has a `stop_reason` of `tool_use` and contains one or more member `tool_use` blocks, each naming a member tool and carrying `"toolset_name": "computer"`. Partway through this task, after Claude has seen a screenshot of the desktop, a response might look like this:262263```json Output264{265 "id": "msg_01UZ3bXcQH8mTqNhVfL9eK2p",266 "type": "message",267 "role": "assistant",268 "model": "claude-opus-5",269 "content": [270 {271 "type": "text",272 "text": "I'll open the web browser to find a picture of a cat."273 },274 {275 "type": "tool_use",276 "id": "toolu_01WkoTUvSHDzTBu2xnGk8Ep8",277 "name": "left_click",278 "toolset_name": "computer",279 "input": { "coordinate": [512, 742] }280 },281 {282 "type": "tool_use",283 "id": "toolu_017nJn3RgSCkTMwuZDb4uUov",284 "name": "screenshot",285 "toolset_name": "computer",286 "input": {}287 }288 ],289 "stop_reason": "tool_use",290 "stop_sequence": null291}292```293294Your application runs each call in order in your own environment, returns one `tool_result` block per `tool_use` block, and calls the API again; [How computer use works](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#how-computer-use-works) describes that loop, and the rest of this page shows how to implement it.295296***297298## How computer use works299300<Steps>301 <Step title="Provide Claude with the computer use tool and a user prompt" icon="tool">302 * Add the computer use toolset (and optionally other tools) to the `tools` array of your API request.303 * Include a user prompt that requires desktop interaction, for example, "Save a picture of a cat to my desktop."304 </Step>305306 <Step title="Claude responds with member tool calls" icon="wrench">307 * Claude assesses whether acting on the desktop can help with the user's query.308 * If so, Claude responds with one or more member `tool_use` blocks, such as `screenshot`, `left_click`, or `type`, each carrying `"toolset_name": "computer"`. A response with several of these blocks is a [batch action](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions).309 * The API response has a `stop_reason` of `tool_use`, signaling a tool use request.310 </Step>311312 <Step title="Run the calls in order and return results" icon="computer">313 * Iterate over every `tool_use` block in the response, in order. For each one, dispatch on the member `name` together with `toolset_name`, and perform that action with the block's `input` on your container or virtual machine.314 * Continue the conversation with a new `user` message that contains one `tool_result` block per `tool_use` block, matched by `tool_use_id` and each echoing `"toolset_name": "computer"`. Return an image for `screenshot` and `zoom`; a short text such as `OK` is enough for the other actions.315 * If an action fails, return `is_error: true` for that block and answer the rest of the batch as described in [Batch actions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions).316 </Step>317318 <Step title="Claude continues until the task is complete" icon="arrows-clockwise">319 * Claude analyzes the tool results to determine if more actions are needed or the task has been completed.320 * If Claude determines more actions are needed, it responds with another `tool_use` `stop_reason` and you should return to step 3.321 * Otherwise, it returns a text response to the user.322 </Step>323</Steps>324325The repetition of steps 3 and 4 without user input is referred to as the "agent loop" (that is, Claude responding with a tool use request and your application responding to Claude with the results of evaluating that request).326327### Batch actions328329Claude can plan a short sequence of actions, such as click, type, and then take a screenshot, and return them together in one response. This is called a batch action; it uses the same response shape as [parallel tool use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use) with one difference: you run the blocks in order rather than concurrently.330331A response with a three-action batch looks like this:332333```json334{335 "role": "assistant",336 "content": [337 {338 "type": "tool_use",339 "id": "toolu_01HqCF3nJ4Vzr8sTkPZ2wxYA",340 "name": "left_click",341 "toolset_name": "computer",342 "input": { "coordinate": [640, 60] }343 },344 {345 "type": "tool_use",346 "id": "toolu_01Ppr3sZ3TnE9m6VUu4RyH2K",347 "name": "type",348 "toolset_name": "computer",349 "input": { "text": "pictures of cats" }350 },351 {352 "type": "tool_use",353 "id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",354 "name": "screenshot",355 "toolset_name": "computer",356 "input": {}357 }358 ]359}360```361362Return one `tool_result` block for each `tool_use` block, matched by `tool_use_id`, all in the next `user` message. Every result for a member tool must carry `"toolset_name": "computer"`; a result that omits it, or that names a different toolset than its `tool_use` block, is rejected. Only `screenshot` and `zoom` results need an image; for the other members, a short text acknowledgment such as `OK` is enough (`cursor_position` returns the coordinates as text):363364```json365{366 "role": "user",367 "content": [368 {369 "type": "tool_result",370 "tool_use_id": "toolu_01HqCF3nJ4Vzr8sTkPZ2wxYA",371 "toolset_name": "computer",372 "content": [{ "type": "text", "text": "OK" }]373 },374 {375 "type": "tool_result",376 "tool_use_id": "toolu_01Ppr3sZ3TnE9m6VUu4RyH2K",377 "toolset_name": "computer",378 "content": [{ "type": "text", "text": "OK" }]379 },380 {381 "type": "tool_result",382 "tool_use_id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",383 "toolset_name": "computer",384 "content": [385 {386 "type": "image",387 "source": {388 "type": "base64",389 "media_type": "image/png",390 "data": "iVBORw0KGgo..."391 }392 }393 ]394 }395 ]396}397```398399**Run blocks in order and stop at the first failure.** Later actions in a batch usually depend on earlier ones: the `type` in this example enters text into whatever the preceding click focused. Run the blocks sequentially in the order they appear in `content`, and if one fails, don't run the rest. Every `tool_use` block still needs a `tool_result`, so answer the batch as follows:400401* For each action that succeeded, return its normal result.402* For the action that failed, return `is_error: true` with a text description of what went wrong.403* For every later action in the batch, return `is_error: true` with exactly this text (the browser use tool uses its own [halt text](https://platform.claude.com/docs/en/agents-and-tools/tool-use/browser-use-tool#batch-actions)):404405```json406{407 "type": "tool_result",408 "tool_use_id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",409 "toolset_name": "computer",410 "is_error": true,411 "content": "Not executed: an earlier computer action in this turn failed."412}413```414415Claude then sees which actions succeeded, which one failed, and which were skipped, and replans on its next turn. A request that leaves any `tool_use` block in the batch unanswered is rejected with an `invalid_request_error`, so an agent loop that reads only the first block fails on its next call. If your application asks a human to confirm consequential actions, make that check before each block runs, because a batch can complete a multistep action within one turn.416417Claude typically finishes a batch with `screenshot` so it can observe the outcome before deciding what to do next. When a batch doesn't end with one, your application can attach a screenshot as an extra `image` block on the last result in the batch so that Claude always sees the current state of the screen, which saves a round trip compared with waiting for Claude to ask. You can also prompt Claude to end every batch with a screenshot (see [Optimize model performance with prompting](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#optimize-model-performance-with-prompting)).418419### The computing environment420421Computer use requires a sandboxed computing environment where Claude can safely interact with applications and the web. This environment includes:4224231. **Virtual display:** A virtual X11 display server (using Xvfb) that renders the desktop interface Claude will see through screenshots and control with mouse/keyboard actions.4244252. **Desktop environment:** A lightweight UI with window manager (Mutter) and panel (Tint2) running on Linux, which provides a consistent graphical interface for Claude to interact with.4264273. **Applications:** Pre-installed Linux applications such as Firefox, LibreOffice, text editors, and file managers that Claude can use to complete tasks.4284294. **Tool implementations:** Integration code that translates Claude's abstract tool requests (such as "move mouse" or "take screenshot") into actual operations in the virtual environment.4304315. **Agent loop:** A program that handles communication between Claude and the environment, sending Claude's actions to the environment and returning the results (screenshots, command outputs) back to Claude.432433When you use computer use, Claude doesn't directly connect to this environment. Instead, your application:4344351. Receives Claude's tool use requests4362. Translates them into actions in your computing environment4373. Captures the results (such as screenshots and command outputs)4384. Returns these results to Claude439440For security and isolation, the reference implementation runs all of this inside a Docker container with appropriate port mappings for viewing and interacting with the environment.441442***443444## How to implement computer use445446Upgrading an existing `computer_20251124` integration? Start with [Migrate from `computer_20251124`](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#migrate-from-computer-20251124); the rest of this section applies to both new and migrated integrations.447448<Tip>449 The [computer use reference implementation](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo) is a complete working example: a [containerized environment](https://github.com/anthropics/anthropic-quickstarts/blob/main/computer-use-demo/Dockerfile) suitable for computer use, implementations of [the computer use tools](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo/computer_use_demo/tools), an [agent loop](https://github.com/anthropics/anthropic-quickstarts/blob/main/computer-use-demo/computer_use_demo/loop.py) that calls the Claude API and runs the tools, and a web interface for the container, loop, and tools.450</Tip>451452### Understand the agent loop453454The core of computer use is the "agent loop": a cycle where Claude requests tool actions, your application runs them, and returns results to Claude. The loop uses the client you created in the [Quick start](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#quick-start), a `tools` array that declares only the computer use toolset, and the tool-call processing helper under [Implement the computer use tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#implement-the-computer-use-tool). If you also declare other tools, such as the Quick start's bash and text editor tools, dispatch their `tool_use` blocks in the same pass; the helper answers only computer use member calls, and the loop treats a turn with no answered calls as finished. Here's a simplified example:455456<CodeGroup exclude="shell">457 ```python Python458 def sampling_loop(model: str, messages: list[MessageParam], max_iterations: int = 10):459 """460 Run the computer-use agent loop until Claude stops requesting tools461 or the iteration limit is reached.462 """463 for _ in range(max_iterations):464 response = client.messages.create(465 model=model,466 max_tokens=4096,467 messages=messages,468 tools=TOOLS,469 )470471 # Add Claude's response to the conversation history472 messages.append({"role": "assistant", "content": response.content})473474 # Run the actions Claude requested, in order, and collect the results475 tool_results = process_tool_calls(response)476 if not tool_results:477 return messages # No more tool use; task complete478479 # Send every result back to Claude in a single user message480 messages.append({"role": "user", "content": tool_results})481482 return messages483 ```484485 ```typescript TypeScript486 async function samplingLoop(487 model: string,488 messages: Anthropic.MessageParam[],489 maxIterations = 10,490 ): Promise<Anthropic.MessageParam[]> {491 // Run the computer-use agent loop until Claude stops requesting tools492 // or the iteration limit is reached.493 for (let i = 0; i < maxIterations; i++) {494 const response = await client.messages.create({495 model,496 max_tokens: 4096,497 messages,498 tools,499 });500501 // Add Claude's response to the conversation history502 messages.push({ role: "assistant", content: response.content });503504 // Run any tools Claude requested and collect results505 const toolResults = processToolCalls(response);506 if (toolResults.length === 0) {507 return messages; // No more tool use; task complete508 }509510 // Send tool results back to Claude for the next iteration511 messages.push({ role: "user", content: toolResults });512 }513514 return messages;515 }516 ```517518 ```csharp C#519 async Task<List<MessageParam>> SamplingLoop(520 Model model,521 List<MessageParam> messages,522 int maxIterations = 10523 )524 {525 // Run the computer-use agent loop until Claude stops requesting tools526 // or the iteration limit is reached.527 for (var i = 0; i < maxIterations; i++)528 {529 var response = await client.Messages.Create(530 new MessageCreateParams531 {532 Model = model,533 MaxTokens = 4096,534 Messages = messages,535 Tools = tools,536 }537 );538539 // Add Claude's response to the conversation history540 messages.Add(541 new()542 {543 Role = Role.Assistant,544 Content = response545 .Content.Select(block => new ContentBlockParam(block.Json))546 .ToList(),547 }548 );549550 // Run any tools Claude requested and collect results551 var toolResults = ProcessToolCalls(response);552 if (toolResults.Count == 0)553 {554 return messages; // No more tool use; task complete555 }556557 // Send tool results back to Claude for the next iteration558 messages.Add(new() { Role = Role.User, Content = toolResults });559 }560561 return messages;562 }563 ```564565 ```go Go566 // samplingLoop runs the computer-use agent loop until Claude stops567 // requesting tools or the iteration limit is reached.568 func samplingLoop(ctx context.Context, model anthropic.Model, messages []anthropic.MessageParam, maxIterations int) ([]anthropic.MessageParam, error) {569 for range maxIterations {570 response, err := client.Messages.New(ctx, anthropic.MessageNewParams{571 Model: model,572 MaxTokens: 4096,573 Messages: messages,574 Tools: tools,575 })576 if err != nil {577 return nil, err578 }579580 // Add Claude's response to the conversation history581 messages = append(messages, response.ToParam())582583 // Run the actions Claude requested, in order, and collect the results584 toolResults := processToolCalls(response)585 if len(toolResults) == 0 {586 return messages, nil // No more tool use; task complete587 }588589 // Send every result back to Claude in a single user message590 messages = append(messages, anthropic.NewUserMessage(toolResults...))591 }592 return messages, nil593 }594595 ```596597 ```java Java598 /**599 * Run the computer-use agent loop until Claude stops requesting tools600 * or the iteration limit is reached.601 */602 List<MessageParam> samplingLoop(Model model, List<MessageParam> messages, int maxIterations) {603 for (int i = 0; i < maxIterations; i++) {604 Message response = client.messages().create(MessageCreateParams.builder()605 .model(model)606 .maxTokens(4096)607 .messages(messages)608 .addTool(COMPUTER_TOOLSET)609 .build());610611 // Add Claude's response to the conversation history612 messages.add(MessageParam.builder()613 .role(MessageParam.Role.ASSISTANT)614 .contentOfBlockParams(response.content().stream().map(ContentBlock::toParam).toList())615 .build());616617 // Run any tools Claude requested and collect results618 List<ContentBlockParam> toolResults = processToolCalls(response);619 if (toolResults.isEmpty()) {620 return messages; // No more tool use; task complete621 }622623 // Send tool results back to Claude for the next iteration624 messages.add(MessageParam.builder()625 .role(MessageParam.Role.USER)626 .contentOfBlockParams(toolResults)627 .build());628 }629 return messages;630 }631 ```632633 ```php PHP634 /**635 * Run the computer-use agent loop until Claude stops requesting tools636 * or the iteration limit is reached.637 */638 function samplingLoop(string $model, array $messages, int $maxIterations = 10): array639 {640 global $client, $tools;641642 for ($i = 0; $i < $maxIterations; $i++) {643 $response = $client->messages->create(644 model: $model,645 maxTokens: 4096,646 messages: $messages,647 tools: $tools,648 );649650 // Add Claude's response to the conversation history651 $messages[] = MessageParam::with(role: Role::ASSISTANT, content: $response->content);652653 // Run any tools Claude requested and collect results654 $toolResults = processToolCalls($response);655 if ($toolResults === []) {656 return $messages; // No more tool use; task complete657 }658659 // Send tool results back to Claude for the next iteration660 $messages[] = MessageParam::with(role: Role::USER, content: $toolResults);661 }662663 return $messages;664 }665 ```666667 ```ruby Ruby668 # Run the computer-use agent loop until Claude stops requesting tools669 # or the iteration limit is reached.670 def sampling_loop(model, messages, max_iterations: 10)671 max_iterations.times do672 response = CLIENT.messages.create(673 model: model,674 max_tokens: 4096,675 messages: messages,676 tools: TOOLS677 )678679 # Add Claude's response to the conversation history680 messages << { role: "assistant", content: response.content }681682 # Run the actions Claude requested, in order, and collect the results683 tool_results = process_tool_calls(response)684 return messages if tool_results.empty? # No more tool use; task complete685686 # Send every result back to Claude in a single user message687 messages << { role: "user", content: tool_results }688 end689690 messages691 end692 ```693</CodeGroup>694695The loop continues until either Claude responds without requesting any tools (task completion) or the maximum iteration limit is reached. This safeguard prevents potential infinite loops that could result in unexpected API costs.696697### Optimize model performance with prompting6986991. Specify simple, well-defined tasks and provide explicit instructions for each step.7002. Claude sometimes assumes outcomes of its actions without explicitly checking their results. To prevent this you can prompt Claude with `After each step, take a screenshot and carefully evaluate if you have achieved the right outcome. Explicitly show your thinking: "I have evaluated step X..." If not correct, try again. Only when you confirm a step was executed correctly should you move on to the next one.`7013. Some UI elements (such as dropdowns and scrollbars) might be tricky for Claude to manipulate using mouse movements. If you experience this, try prompting the model to use keyboard shortcuts.7024. For repeatable tasks or UI interactions, include example screenshots and tool calls of successful outcomes in your prompt.7035. If you need the model to log in, provide it with the username and password in your prompt inside XML tags such as `<robot_credentials>`. Using computer use within applications that require login increases the risk of bad outcomes as a result of prompt injection. Review [Mitigate jailbreaks and prompt injections](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks) before providing the model with login credentials.7046. When constructing a user turn's `content` array, place the instruction text *before* the screenshot image. Providing the target description before the image is processed improves click accuracy.7057. Claude uses the `zoom` action to inspect a region at full resolution when asked about small text or specific UI elements that aren't legible at the screenshot's default resolution, such as file names in a sidebar, tab titles, status-bar text, line numbers, or button labels. If Claude isn't zooming when you expect, ask about a specific region or element rather than the screen as a whole.7068. If you want every [batch action](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions) to end with a screenshot, say so in the system prompt, for example, `End each group of actions with a screenshot so you can verify the result before continuing.`707708<Tip>709 If you repeatedly encounter a clear set of issues or know in advance the tasks Claude will need to complete, use the system prompt to provide Claude with explicit tips or instructions on how to do the tasks successfully.710</Tip>711712<Tip>713 For agents that span multiple sessions, run end-to-end verification at the start of each session, not only after implementation. Browser-based checks catch regressions from prior sessions that code-level review alone misses. See [Effective harnesses for long-running agents](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents) for details.714</Tip>715716### System prompts717718When you include the computer use tool in a request, the API generates a computer use-specific system prompt. It's similar to the [tool use system prompt](https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools#tool-use-system-prompt) but starts with:719720> You have access to a set of functions you can use to answer the user's question. This includes access to a sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external resources, except by invoking the below functions.721722As with regular tool use, the user-provided `system` parameter is still respected and used in the construction of the combined system prompt.723724### Available actions725726Each action is a member tool of the computer use toolset: Claude names the member in a `tool_use` block that carries `"toolset_name": "computer"`, and the block's `input` holds only that member's parameters, with no `action` field. The toolset has 17 member tools:727728| Member | Input | Description |729| ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |730| `screenshot` | None (`{}`) | Capture the full display and return it as an image. |731| `zoom` | `region`: `[x0, y0, x1, y1]`, the top-left and bottom-right corners of the area to inspect | Capture only that region of the display at full resolution and return it as an image, scaled to fit within your usual screenshot dimensions with its aspect ratio preserved. This lets Claude read small text or dense UI that isn't legible in a downscaled full screenshot. |732| `left_click` | `coordinate` (optional): `[x, y]`; `text` (optional): modifier keys to hold during the click: `shift`, `ctrl`, `alt`, `super` (the Command or Windows key), or a `+`-joined combination such as `ctrl+shift` | Click the left mouse button at `coordinate`, or at the current cursor position when `coordinate` is omitted. |733| `right_click`, `middle_click`, `double_click`, `triple_click` | Same as `left_click` | Other mouse buttons and multiple clicks. |734| `left_click_drag` | `start_coordinate`: `[x, y]`; `coordinate`: `[x, y]`; `text` (optional): modifier keys | Press at `start_coordinate`, drag to `coordinate`, and release. |735| `mouse_move` | `coordinate`: `[x, y]` | Move the cursor without clicking, for example, to hover. |736| `left_mouse_down`, `left_mouse_up` | None (`{}`) | Press or release the left mouse button at the current cursor position, for drags that `left_click_drag` can't express. Move the cursor with `mouse_move` first. |737| `cursor_position` | None (`{}`) | Report the cursor's current `[x, y]` position as text. |738| `scroll` | `scroll_direction`: `"up"`, `"down"`, `"left"`, or `"right"`; `scroll_amount`: number of scroll-wheel clicks; `coordinate` (optional): `[x, y]`; `text` (optional): modifier keys | Scroll at `coordinate`, or at the current cursor position. |739| `type` | `text`: the string to type | Type literal text at the current keyboard focus. |740| `key` | `text`: a key or a `+`-joined combination such as `"Return"`, `"ctrl+s"`, or `"alt+Tab"`; `repeat` (optional): 1 to 100, default 1 | Press a key or key combination, `repeat` times. |741| `hold_key` | `text`: a key or combination; `duration`: seconds, up to 300 | Hold a key down for the given duration. |742| `wait` | `duration`: seconds, up to 300 | Pause before the next action, for example, while an application loads. |743744Keep the following in mind when implementing the members:745746* **Coordinates are in screenshot pixels.** Every `coordinate`, `start_coordinate`, and `region` value, and the position that `cursor_position` reports, is in the pixel space of the full-display screenshots you return, with the origin at the top left. Zoom images don't change this: after a `zoom`, Claude still expresses coordinates in the full screenshot's space, never relative to the zoomed image. If you scale screenshots down before returning them, scale Claude's coordinates back up before applying them to the real display (see [Size screenshots to fit image limits](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions)).747* **All members are enabled by default, including `zoom`.** If your environment can't produce zoom images, withhold the member with `configs` (see [Tool parameters](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#tool-parameters)) rather than leaving it enabled and returning errors. If Claude calls a member that you have withheld or don't implement, return a `tool_result` with `is_error: true` for that block.748* **Dispatch on the pair (`toolset_name`, `name`).** `toolset_name` is what marks a block as a computer action: a custom tool in the same request can share a member's name, and a later toolset version can add members (see [Client toolsets](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference#client-toolsets)).749750<Accordion title="Example actions">751 Each example is a complete `tool_use` block as it appears in Claude's response.752753 Shift+click at a position, for example, to extend a selection. Unlike `hold_key`, `text` holds the modifiers only for the duration of that click or scroll:754755 ```json756 {757 "type": "tool_use",758 "id": "toolu_01Qg8m3XqC5aRy7tD2eS4jUg",759 "name": "left_click",760 "toolset_name": "computer",761 "input": { "coordinate": [500, 300], "text": "shift" }762 }763 ```764765 Drag from one point to another:766767 ```json768 {769 "type": "tool_use",770 "id": "toolu_01Ed6j9VnA3yPw5rB8cQ2gSe",771 "name": "left_click_drag",772 "toolset_name": "computer",773 "input": {774 "start_coordinate": [200, 300],775 "coordinate": [600, 300]776 }777 }778 ```779780 Scroll down three clicks of the wheel:781782 ```json783 {784 "type": "tool_use",785 "id": "toolu_01Yc5h8UmZ2xNv4qA7bP9fRd",786 "name": "scroll",787 "toolset_name": "computer",788 "input": {789 "coordinate": [500, 400],790 "scroll_direction": "down",791 "scroll_amount": 3792 }793 }794 ```795796 Press Tab four times:797798 ```json799 {800 "type": "tool_use",801 "id": "toolu_01Sb4g7TkY9wLu3pX6zM8eQc",802 "name": "key",803 "toolset_name": "computer",804 "input": { "text": "Tab", "repeat": 4 }805 }806 ```807808 Zoom in to inspect a region at full resolution:809810 ```json811 {812 "type": "tool_use",813 "id": "toolu_01Kf7k2WpB4zQx6sC9dR3hTf",814 "name": "zoom",815 "toolset_name": "computer",816 "input": { "region": [100, 200, 400, 350] }817 }818 ```819820 Report the cursor position. Answer this call with a short text result that gives the position in screenshot pixels, for example, `X=512, Y=384`:821822 ```json823 {824 "type": "tool_use",825 "id": "toolu_01Ekh3vqB6yTs2mNc4Rw8pLd",826 "name": "cursor_position",827 "toolset_name": "computer",828 "input": {}829 }830 ```831</Accordion>832833### Tool parameters834835The toolset entry in the `tools` array accepts four parameters; the rules they share with the browser use toolset are listed under [Client toolsets](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference#client-toolsets).836837| Parameter | Required | Description |838| ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |839| `type` | Yes | `computer_toolset_20260801` |840| `configs` | No | Per-member settings keyed by member name; each member accepts `enabled` (default `true` for all 17, including `zoom`) and `defer_loading` (default `false`, for [tool search](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool#deferred-tool-loading)), and members you omit keep their defaults. |841| `cache_control` | No | [Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) breakpoint at the toolset definition; entry only. A breakpoint on any `tool_use` or `tool_result` block in a batch takes effect at the end of that batch; see [Tool use with prompt caching](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-use-with-prompt-caching#cache-control-on-tool-definitions). |842| `allowed_callers` | No | `["direct"]` only. |843844For example, this entry withholds `zoom` for an environment that doesn't implement it and sets a cache breakpoint at the toolset definition:845846```json847{848 "type": "computer_toolset_20260801",849 "configs": {850 "zoom": { "enabled": false }851 },852 "cache_control": { "type": "ephemeral" }853}854```855856If your agent loop can run only one action per round trip, set `disable_parallel_tool_use` to `true` in `tool_choice`; Claude then returns at most one member `tool_use` block per turn (see [Disable parallel tool use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use#disable-parallel-tool-use)).857858The entry rejects these parameters from earlier tool versions, and a request that includes any of them returns an `invalid_request_error`:859860* `name`: member names are fixed by the toolset version.861* `display_width_px`, `display_height_px`, and `display_number`: coordinates are always in the pixel space of the screenshots you return.862* `enable_zoom`: zoom is a member tool that you control through `configs`.863864The entry also can't be declared in the same request as a `computer_20251124` entry or another tool named `computer`. For `strict`, `input_examples`, `defer_loading` placement, `tool_choice`, streaming, and caller restrictions, see [Client toolsets](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference#client-toolsets).865866### Combining with thinking867868To combine computer use with thinking, see [Thinking](https://platform.claude.com/docs/en/build-with-claude/thinking).869870<Tip>871 For the earlier `computer_20251124` tool, internal benchmarking on the models that use it suggests these `effort` settings:872873 * **Claude Opus 4.7:** use `high` as the default; use `low` for high-throughput or cost-sensitive workloads.874 * **Claude Sonnet 4.6 and Claude Opus 4.6:** use `medium` as the default (best accuracy-to-cost ratio). Avoid `max`, which adds token cost without improving accuracy on UI tasks. On these models, `low` uses *fewer* output tokens than disabling thinking entirely (fewer mistakes mean fewer retries), making it a strong option for cost-sensitive loops.875</Tip>876877### Augmenting computer use with other tools878879To add other tools alongside computer use, include them in the same `tools` array. The [Quick start](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#quick-start) section shows this pattern with the [bash tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/bash-tool) and [text editor tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool). You can add your own [custom tool definitions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools) the same way.880881For tasks that stay inside webpages, you can also [declare the browser use tool in the same request](https://platform.claude.com/docs/en/agents-and-tools/tool-use/browser-use-tool#combine-with-other-tools): the two toolsets work independently, each in its own coordinate frame, and calls to members that share a name, such as `screenshot` or `key`, are told apart by `toolset_name`.882883### Build a custom computer use environment884885The [reference implementation](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo) is meant to help you get started with computer use. It includes all of the components needed to have Claude use a computer. However, you can build your own environment for computer use to suit your needs. You'll need:886887* A virtualized or containerized environment suitable for computer use with Claude888* An implementation of the computer use tool's actions889* An agent loop that interacts with the Claude API and runs the `tool_use` results using your tool implementations890* An API or UI that allows user input to start the agent loop891892### Implement the computer use tool893894The computer use tool is implemented as a schema-less tool. When using this tool, you don't need to provide an input schema as with other tools; the schema is built into Claude's model and can't be modified.895896<Steps>897 <Step title="Set up your computing environment">898 Create a virtual display or connect to an existing display that Claude will interact with. This typically involves setting up Xvfb (X Virtual Framebuffer) or similar technology.899 </Step>900901 <Step title="Implement action handlers">902 Create functions to handle each action type that Claude might request:903904 <CodeGroup exclude="shell">905 ```python Python906 # Placeholder image data; a real executor captures the screen and returns the PNG bytes907 PLACEHOLDER_PNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="908909910 def capture_screenshot() -> list[ImageBlockParam]:911 # screenshot answers with an image block rather than text: return the result content list912 return [913 {914 "type": "image",915 "source": {"type": "base64", "media_type": "image/png", "data": PLACEHOLDER_PNG},916 }917 ]918919920 def click(coordinate=None):921 if coordinate is None:922 return "clicked at current cursor"923 x, y = coordinate924 return f"clicked at ({x}, {y})"925926927 def type_text(text):928 return f"typed: {text}"929930931 def handle_computer_action(name, tool_input):932 match name:933 case "screenshot":934 return capture_screenshot()935 case "left_click":936 # coordinate is optional; without it, click where the cursor already is937 return click(tool_input.get("coordinate"))938 case "type":939 return type_text(tool_input["text"])940 # Handle other actions as needed941 raise ValueError(f"Unknown or unimplemented member: {name}")942 ```943944 ```typescript TypeScript945 // Placeholder image data; a real executor captures the screen as PNG bytes946 const PLACEHOLDER_PNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==";947948 function captureScreenshot(): Anthropic.ImageBlockParam[] {949 // screenshot answers with an image block rather than text950 return [951 {952 type: "image",953 source: {954 type: "base64",955 media_type: "image/png",956 data: PLACEHOLDER_PNG,957 },958 },959 ];960 }961962 function clickAt(x: number, y: number): string {963 return `clicked at (${x}, ${y})`;964 }965966 function clickAtCursor(): string {967 return "clicked at the current cursor position";968 }969970 function typeText(text: string): string {971 return `typed: ${text}`;972 }973974 function handleComputerAction(975 action: string,976 input: unknown,977 ): string | Anthropic.ImageBlockParam[] {978 const params: object =979 typeof input === "object" && input !== null ? input : {};980 switch (action) {981 case "screenshot":982 return captureScreenshot();983 case "left_click":984 // coordinate is optional on the toolset; without one, click at the cursor985 if ("coordinate" in params && Array.isArray(params.coordinate)) {986 const [x, y] = params.coordinate;987 return clickAt(x, y);988 }989 return clickAtCursor();990 case "type":991 if ("text" in params) {992 return typeText(String(params.text));993 }994 break;995 }996 // Handle other actions as needed997 throw new Error(`Unknown or unimplemented member: ${action}`);998 }999 ```10001001 ```csharp C#1002 // Placeholder image data; a real executor captures the screen and returns the PNG bytes1003 const string PlaceholderPng = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==";10041005 // screenshot answers with an image block rather than text: return the result content list1006 List<Block> CaptureScreenshot() =>1007 [1008 new ImageBlockParam(1009 new Base64ImageSource { Data = PlaceholderPng, MediaType = MediaType.ImagePng }1010 ),1011 ];10121013 string ClickAt(int x, int y) => $"clicked at ({x}, {y})";10141015 string ClickAtCursor() => "clicked at the current cursor position";10161017 string TypeText(string text) => $"typed: {text}";10181019 ToolResultBlockParamContent HandleComputerAction(1020 string action,1021 IReadOnlyDictionary<string, JsonElement> input1022 ) =>1023 action switch1024 {1025 "screenshot" => CaptureScreenshot(),1026 // coordinate is optional on click members; without it, click where the cursor is1027 "left_click" when input.TryGetValue("coordinate", out var xy) => ClickAt(1028 xy[0].GetInt32(),1029 xy[1].GetInt32()1030 ),1031 "left_click" => ClickAtCursor(),1032 "type" => TypeText(input["text"].GetString()!),1033 // Handle other actions as needed1034 _ => throw new NotSupportedException($"Unknown or unimplemented member: {action}"),1035 };1036 ```10371038 ```go Go1039 // placeholderPNG stands in for a real capture: an executor returns the1040 // screen as base64-encoded PNG data.1041 const placeholderPNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="10421043 // captureScreenshot returns an image block rather than text.1044 func captureScreenshot() []anthropic.ToolResultBlockParamContentUnion {1045 return []anthropic.ToolResultBlockParamContentUnion{{1046 OfImage: &anthropic.ImageBlockParam{1047 Source: anthropic.ImageBlockParamSourceUnion{1048 OfBase64: &anthropic.Base64ImageSourceParam{1049 MediaType: anthropic.Base64ImageSourceMediaTypeImagePNG,1050 Data: placeholderPNG,1051 },1052 },1053 },1054 }}1055 }10561057 // textContent wraps text as tool_result content.1058 func textContent(text string) []anthropic.ToolResultBlockParamContentUnion {1059 return []anthropic.ToolResultBlockParamContentUnion{1060 {OfText: &anthropic.TextBlockParam{Text: text}},1061 }1062 }10631064 func clickAt(x, y int) string {1065 return fmt.Sprintf("clicked at (%d, %d)", x, y)1066 }10671068 func clickAtCursor() string {1069 return "clicked at the current cursor position"1070 }10711072 func typeText(text string) string {1073 return fmt.Sprintf("typed: %s", text)1074 }10751076 func handleComputerAction(action string, params map[string]any) ([]anthropic.ToolResultBlockParamContentUnion, error) {1077 switch action {1078 case "screenshot":1079 return captureScreenshot(), nil1080 case "left_click":1081 // coordinate is optional; without it, click where the cursor already is1082 coord, ok := params["coordinate"].([]any)1083 if !ok {1084 return textContent(clickAtCursor()), nil1085 }1086 if len(coord) == 2 {1087 x, xok := coord[0].(float64)1088 y, yok := coord[1].(float64)1089 if xok && yok {1090 return textContent(clickAt(int(x), int(y))), nil1091 }1092 }1093 case "type":1094 if text, ok := params["text"].(string); ok {1095 return textContent(typeText(text)), nil1096 }1097 // Handle other actions as needed1098 default:1099 return nil, fmt.Errorf("unknown or unimplemented member: %s", action)1100 }1101 // Reached when a member's input is missing a field or a field has the wrong type1102 return nil, fmt.Errorf("invalid input for %s", action)1103 }11041105 ```11061107 ```java Java1108 /** Placeholder pixels; a real executor captures the screen and base64-encodes the PNG. */1109 static final String PLACEHOLDER_PNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==";11101111 ToolResultBlockParam.Content captureScreenshot() {1112 ImageBlockParam image = ImageBlockParam.builder()1113 .source(Base64ImageSource.builder()1114 .mediaType(Base64ImageSource.MediaType.IMAGE_PNG)1115 .data(PLACEHOLDER_PNG)1116 .build())1117 .build();1118 return ToolResultBlockParam.Content.ofBlocks(1119 List.of(ToolResultBlockParam.Content.Block.ofImage(image)));1120 }11211122 String clickAt(long x, long y) {1123 return "clicked at (" + x + ", " + y + ")";1124 }11251126 String clickAtCursor() {1127 return "clicked at current cursor";1128 }11291130 String typeText(String text) {1131 return "typed: " + text;1132 }11331134 /** Runs one computer toolset member; {@code action} is the tool_use block's name. */1135 ToolResultBlockParam.Content handleComputerAction(String action, Map<String, JsonValue> input) {1136 if (action.equals("screenshot")) {1137 return captureScreenshot(); // the one member here that answers with an image block1138 }1139 String output = switch (action) {1140 case "left_click" -> {1141 JsonValue coordinate = input.get("coordinate"); // optional on the toolset1142 if (coordinate == null) {1143 yield clickAtCursor();1144 }1145 List<JsonValue> point = (List<JsonValue>) coordinate.asArray().get();1146 long x = ((Number) point.get(0).asNumber().get()).longValue();1147 long y = ((Number) point.get(1).asNumber().get()).longValue();1148 yield clickAt(x, y);1149 }1150 case "type" -> typeText(input.get("text").asStringOrThrow());1151 // Handle other actions as needed1152 default -> throw new UnsupportedOperationException("Unknown or unimplemented member: " + action);1153 };1154 return ToolResultBlockParam.Content.ofString(output);1155 }1156 ```11571158 ```php PHP1159 // Stand-in for real PNG bytes; a real executor captures the screen1160 const PLACEHOLDER_PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==';11611162 function captureScreenshot(): array1163 {1164 // screenshot answers with an image block rather than text, so return the result content list1165 $image = [1166 'type' => 'image',1167 'source' => ['type' => 'base64', 'media_type' => 'image/png', 'data' => PLACEHOLDER_PNG],1168 ];11691170 return [$image];1171 }11721173 function clickAt(?array $coordinate): string1174 {1175 // left_click may omit coordinate, in which case the click lands where the cursor already is1176 if ($coordinate === null) {1177 return 'clicked at current cursor';1178 }1179 [$x, $y] = $coordinate;11801181 return "clicked at ({$x}, {$y})";1182 }11831184 function typeText(string $text): string1185 {1186 return "typed: {$text}";1187 }11881189 function handleComputerAction(string $name, array $input): string|array1190 {1191 return match ($name) {1192 'screenshot' => captureScreenshot(),1193 'left_click' => clickAt($input['coordinate'] ?? null),1194 'type' => typeText($input['text']),1195 // Handle other actions as needed1196 default => throw new RuntimeException("Unknown or unimplemented member: {$name}"),1197 };1198 }1199 ```12001201 ```ruby Ruby1202 # Stand-in image data; a real executor captures the screen as a PNG.1203 PLACEHOLDER_PNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg=="12041205 # screenshot answers with an image block rather than text1206 def capture_screenshot1207 [1208 {1209 type: "image",1210 source: { type: "base64", media_type: "image/png", data: PLACEHOLDER_PNG }1211 }1212 ]1213 end12141215 def click(coordinate = nil)1216 return "clicked at current cursor" if coordinate.nil?12171218 x, y = coordinate1219 "clicked at (#{x}, #{y})"1220 end12211222 def type_text(text)1223 "typed: #{text}"1224 end12251226 def handle_computer_action(name, input)1227 case name1228 when "screenshot"1229 capture_screenshot1230 when "left_click"1231 # coordinate is optional; without it, click where the cursor already is1232 click(input[:coordinate])1233 when "type"1234 type_text(input[:text])1235 # Handle other actions as needed1236 else1237 raise ArgumentError, "Unknown or unimplemented member: #{name}"1238 end1239 end1240 ```1241 </CodeGroup>1242 </Step>12431244 <Step title="Process Claude's tool calls">1245 Extract and run tool calls from Claude's responses:12461247 <CodeGroup exclude="shell">1248 ```python Python1249 NOT_EXECUTED = "Not executed: an earlier computer action in this turn failed."125012511252 def process_tool_calls(response: Message) -> list[ToolResultBlockParam]:1253 """1254 Run the computer actions in Claude's response in order and answer each1255 one. After the first failure the rest are skipped, because Claude planned1256 them assuming the earlier actions succeeded.1257 """1258 tool_results: list[ToolResultBlockParam] = []1259 failed = False1260 for block in response.content:1261 # Only the computer toolset is declared; route other tools here if you add them1262 if block.type != "tool_use" or block.toolset_name != "computer":1263 continue1264 result: ToolResultBlockParam = {1265 "type": "tool_result",1266 "tool_use_id": block.id,1267 "toolset_name": "computer",1268 }1269 if failed:1270 result["content"] = NOT_EXECUTED1271 result["is_error"] = True1272 else:1273 try:1274 # A string, or a list of content blocks such as the screenshot image1275 result["content"] = handle_computer_action(block.name, block.input)1276 except Exception as err:1277 result["content"] = f"Error: {err}"1278 result["is_error"] = True1279 failed = True1280 tool_results.append(result)1281 return tool_results1282 ```12831284 ```typescript TypeScript1285 const HALT_TEXT =1286 "Not executed: an earlier computer action in this turn failed.";12871288 function computerResult(1289 toolUseId: string,1290 content: string | Anthropic.ImageBlockParam[],1291 isError?: boolean,1292 ): Anthropic.ToolResultBlockParam {1293 return {1294 type: "tool_result",1295 tool_use_id: toolUseId,1296 toolset_name: "computer",1297 content,1298 is_error: isError,1299 };1300 }13011302 function processToolCalls(1303 response: Anthropic.Message,1304 ): Anthropic.ToolResultBlockParam[] {1305 const toolResults: Anthropic.ToolResultBlockParam[] = [];1306 let failed = false;1307 for (const block of response.content) {1308 if (block.type !== "tool_use") {1309 continue;1310 }1311 if (block.toolset_name !== "computer") {1312 // This example declares only the computer toolset; route other tools1313 // here if you add them.1314 continue;1315 }1316 if (failed) {1317 // A batch stops at its first failure; answer later actions unexecuted1318 toolResults.push(computerResult(block.id, HALT_TEXT, true));1319 continue;1320 }1321 try {1322 // A string, or the image block list that screenshot returns1323 const result = handleComputerAction(block.name, block.input);1324 toolResults.push(computerResult(block.id, result));1325 } catch (error) {1326 failed = true;1327 const message = error instanceof Error ? error.message : String(error);1328 toolResults.push(computerResult(block.id, `Error: ${message}`, true));1329 }1330 }1331 return toolResults;1332 }1333 ```13341335 ```csharp C#1336 const string HaltText = "Not executed: an earlier computer action in this turn failed.";13371338 List<ContentBlockParam> ProcessToolCalls(Message response)1339 {1340 List<ContentBlockParam> toolResults = [];1341 var failed = false;1342 foreach (var block in response.Content)1343 {1344 if (!block.TryPickToolUse(out var toolUse))1345 {1346 continue;1347 }13481349 if (toolUse.ToolsetName != "computer")1350 {1351 // This example declares only the computer toolset; route other tools1352 // here if you add them.1353 continue;1354 }13551356 if (failed)1357 {1358 // A batch stops at its first failure; answer later actions without running them1359 toolResults.Add(1360 new ToolResultBlockParam(toolUse.ID)1361 {1362 Content = HaltText,1363 IsError = true,1364 ToolsetName = "computer",1365 }1366 );1367 continue;1368 }13691370 try1371 {1372 // A string, or the image block list that screenshot returns1373 var result = HandleComputerAction(toolUse.Name, toolUse.Input);1374 toolResults.Add(1375 new ToolResultBlockParam(toolUse.ID) { Content = result, ToolsetName = "computer" }1376 );1377 }1378 catch (Exception e)1379 {1380 failed = true;1381 toolResults.Add(1382 new ToolResultBlockParam(toolUse.ID)1383 {1384 Content = $"Error: {e.Message}",1385 IsError = true,1386 ToolsetName = "computer",1387 }1388 );1389 }1390 }1391 return toolResults;1392 }1393 ```13941395 ```go Go1396 const notExecuted = "Not executed: an earlier computer action in this turn failed."13971398 // computerToolResult builds the result for one computer action. Unlike an1399 // ordinary tool result, it must echo the toolset name.1400 func computerToolResult(toolUseID string, content []anthropic.ToolResultBlockParamContentUnion, isError bool) anthropic.ContentBlockParamUnion {1401 result := anthropic.ToolResultBlockParam{1402 ToolUseID: toolUseID,1403 ToolsetName: anthropic.String("computer"),1404 Content: content,1405 }1406 if isError {1407 result.IsError = anthropic.Bool(true)1408 }1409 return anthropic.ContentBlockParamUnion{OfToolResult: &result}1410 }14111412 // processToolCalls runs the computer actions in Claude's response in order and1413 // builds one tool_result per tool_use block. After the first failure it skips1414 // the rest: Claude planned them assuming the earlier actions succeeded.1415 func processToolCalls(response *anthropic.Message) []anthropic.ContentBlockParamUnion {1416 var toolResults []anthropic.ContentBlockParamUnion1417 failed := false1418 for _, block := range response.Content {1419 switch variant := block.AsAny().(type) {1420 case anthropic.ToolUseBlock:1421 // This example declares only the computer toolset; route other tools here if you add them.1422 if variant.ToolsetName != "computer" {1423 continue1424 }1425 if failed {1426 toolResults = append(toolResults, computerToolResult(variant.ID, textContent(notExecuted), true))1427 continue1428 }1429 var input map[string]any1430 var content []anthropic.ToolResultBlockParamContentUnion1431 err := json.Unmarshal(variant.Input, &input)1432 if err == nil {1433 // Text, or the image block that screenshot returns1434 content, err = handleComputerAction(variant.Name, input)1435 }1436 if err != nil {1437 failed = true1438 content = textContent("Error: " + err.Error())1439 }1440 toolResults = append(toolResults, computerToolResult(variant.ID, content, err != nil))1441 }1442 }1443 return toolResults1444 }14451446 ```14471448 ```java Java1449 /** The exact text the toolset contract prescribes for member calls skipped after a failure. */1450 static final String HALT_TEXT = "Not executed: an earlier computer action in this turn failed.";14511452 /** Every result answering a computer toolset member echoes toolset_name. */1453 ToolResultBlockParam.Builder computerResult(ToolUseBlock toolUse) {1454 return ToolResultBlockParam.builder()1455 .toolUseId(toolUse.id())1456 .toolsetName("computer");1457 }14581459 /**1460 * Run the computer actions in Claude's response in order and build one1461 * tool_result per tool_use block. After the first failure, skip the rest:1462 * Claude planned them assuming the earlier actions succeeded.1463 */1464 List<ContentBlockParam> processToolCalls(Message response) {1465 List<ContentBlockParam> toolResults = new ArrayList<>();1466 boolean failed = false;1467 for (ContentBlock block : response.content()) {1468 // This example declares only the computer toolset; route other tools here if you add them.1469 if (!block.isToolUse() || !block.asToolUse().toolsetName().equals(Optional.of("computer"))) {1470 continue;1471 }1472 ToolUseBlock toolUse = block.asToolUse();1473 ToolResultBlockParam result;1474 if (failed) {1475 result = computerResult(toolUse).content(HALT_TEXT).isError(true).build();1476 } else {1477 try {1478 Map<String, JsonValue> input =1479 (Map<String, JsonValue>) toolUse._input().asObject().get();1480 // A string, or the image block that screenshot returns1481 ToolResultBlockParam.Content output = handleComputerAction(toolUse.name(), input);1482 result = computerResult(toolUse).content(output).build();1483 } catch (RuntimeException e) {1484 failed = true;1485 result = computerResult(toolUse).content("Error: " + e.getMessage()).isError(true).build();1486 }1487 }1488 toolResults.add(ContentBlockParam.ofToolResult(result));1489 }1490 return toolResults;1491 }1492 ```14931494 ```php PHP1495 const HALT_TEXT = 'Not executed: an earlier computer action in this turn failed.';14961497 function processToolCalls(Message $response): array1498 {1499 $toolResults = [];1500 $failed = false;1501 foreach ($response->content as $block) {1502 // This example declares only the computer toolset; route other tools here if you add them.1503 if (!($block instanceof \Anthropic\Messages\ToolUseBlock) || $block->toolsetName !== 'computer') {1504 continue;1505 }1506 $result = ['type' => 'tool_result', 'tool_use_id' => $block->id, 'toolset_name' => 'computer'];1507 if ($failed) {1508 // A batch stops at its first failure; the remaining actions are answered without running1509 $toolResults[] = [...$result, 'content' => HALT_TEXT, 'is_error' => true];1510 continue;1511 }1512 try {1513 // A string, or the image block list that screenshot returns1514 $toolResults[] = [...$result, 'content' => handleComputerAction($block->name, $block->input)];1515 } catch (Throwable $e) {1516 $failed = true;1517 $toolResults[] = [...$result, 'content' => 'Error: ' . $e->getMessage(), 'is_error' => true];1518 }1519 }15201521 return $toolResults;1522 }1523 ```15241525 ```ruby Ruby1526 NOT_EXECUTED = "Not executed: an earlier computer action in this turn failed."15271528 # Run the computer actions in Claude's response in order and build one1529 # tool_result per tool_use block. After the first failure, skip the rest:1530 # Claude planned them assuming the earlier actions succeeded.1531 def process_tool_calls(response)1532 tool_results = []1533 failed = false1534 response.content.each do |block|1535 # This example declares only the computer toolset; route other tools here1536 # if you add them.1537 next unless block.type == :tool_use && block.toolset_name == "computer"15381539 result = { type: "tool_result", tool_use_id: block.id, toolset_name: "computer" }1540 if failed1541 result.update(content: NOT_EXECUTED, is_error: true)1542 else1543 begin1544 # A String, or the image content blocks that screenshot returns1545 result[:content] = handle_computer_action(block.name, block.input)1546 rescue => e1547 result.update(content: "Error: #{e.message}", is_error: true)1548 failed = true1549 end1550 end1551 tool_results << result1552 end1553 tool_results1554 end1555 ```1556 </CodeGroup>1557 </Step>15581559 <Step title="Implement the agent loop">1560 Wrap the two previous steps in a loop that sends the results back and repeats until Claude returns no member tool calls; [Understand the agent loop](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#understanding-the-agentic-loop) shows this loop in each language.1561 </Step>1562</Steps>15631564### Handle errors15651566Report a failed action to Claude as a `tool_result` with `is_error: true` and a short description, and include `"toolset_name": "computer"` as on any other member result. If the failed action was part of a [batch action](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions), answer the remaining blocks in the batch with the halt text shown there instead of running them.15671568For example, when screenshot capture fails:15691570```json1571{1572 "role": "user",1573 "content": [1574 {1575 "type": "tool_result",1576 "tool_use_id": "toolu_01A09q90qw90lq917835lq9",1577 "toolset_name": "computer",1578 "content": "Error: Failed to capture screenshot. Display may be locked or unavailable.",1579 "is_error": true1580 }1581 ]1582}1583```15841585Use the same shape for coordinates outside the display bounds and for actions that fail to run, with a message that says what went wrong.15861587### Size screenshots to fit image limits15881589Screenshots and zoom images that you return to the computer use toolset must already fit within your model's [image size limits](https://platform.claude.com/docs/en/build-with-claude/vision#evaluate-image-size): the toolset takes no display dimensions and the API doesn't downscale for you, so an oversized `tool_result` image is rejected with a validation error. Because Claude returns coordinates in the pixel space of the image it sees, keep the scale factor you used so you can map those coordinates back to your screen.15901591<Note>1592 Limits vary by model. Claude Opus 4.7 and later models, including every model that supports `computer_toolset_20260801`, accept up to 2576 pixels on the long edge and 4784 visual tokens total (`⌈width / 28⌉ × ⌈height / 28⌉`, approximately 3.75 megapixels); earlier models accept up to 1568 pixels on the long edge and approximately 1.15 megapixels total (see [Resolution and token cost](https://platform.claude.com/docs/en/build-with-claude/vision#evaluate-image-size) for each model's tier). The following example uses the earlier-model 1568 px / 1.15 MP limits. For a high-resolution-tier model, size to the visual-token limit rather than a pixel total, for example with the resize helper in [Resize your image before uploading](https://platform.claude.com/docs/en/build-with-claude/vision-coordinates#resize-your-image-before-uploading).1593</Note>15941595If your screen is larger than the limit, resize each screenshot before returning it and scale Claude's returned coordinates back to the original screen space. Because the toolset takes no display dimensions, the resize and the coordinate scaling in your application code are all you need:15961597<CodeGroup exclude="shell">1598 ```python Python1599 import math16001601 screen_width, screen_height = 1512, 982160216031604 def get_scale_factor(width, height):1605 """Calculate scale factor to meet API constraints."""1606 long_edge = max(width, height)1607 total_pixels = width * height16081609 long_edge_scale = 1568 / long_edge1610 total_pixels_scale = math.sqrt(1_150_000 / total_pixels)16111612 return min(1.0, long_edge_scale, total_pixels_scale)161316141615 # When capturing screenshot1616 scale = get_scale_factor(screen_width, screen_height)1617 scaled_width = int(screen_width * scale)1618 scaled_height = int(screen_height * scale)16191620 # Resize image to scaled dimensions before sending to Claude1621 screenshot = capture_and_resize(scaled_width, scaled_height)162216231624 # When handling Claude's coordinates, scale them back up1625 def execute_click(x, y):1626 screen_x = x / scale1627 screen_y = y / scale1628 perform_click(screen_x, screen_y)1629 ```16301631 ```typescript TypeScript1632 const screenWidth = 1512;1633 const screenHeight = 982;1634 const MAX_LONG_EDGE = 1568;1635 const MAX_PIXELS = 1_150_000;16361637 function getScaleFactor(width: number, height: number): number {1638 const longEdge = Math.max(width, height);1639 const totalPixels = width * height;16401641 const longEdgeScale = MAX_LONG_EDGE / longEdge;1642 const totalPixelsScale = Math.sqrt(MAX_PIXELS / totalPixels);16431644 return Math.min(1.0, longEdgeScale, totalPixelsScale);1645 }16461647 // When capturing screenshot1648 const scale = getScaleFactor(screenWidth, screenHeight);1649 const scaledWidth = Math.floor(screenWidth * scale);1650 const scaledHeight = Math.floor(screenHeight * scale);16511652 // Resize image to scaled dimensions before sending to Claude1653 const screenshot = captureAndResize(scaledWidth, scaledHeight);16541655 // When handling Claude's coordinates, scale them back up1656 function executeClick(x: number, y: number): void {1657 const screenX = x / scale;1658 const screenY = y / scale;1659 performClick(screenX, screenY);1660 }1661 ```16621663 ```csharp C#1664 int screenWidth = 1512, screenHeight = 982;16651666 double GetScaleFactor(int width, int height)1667 {1668 // Calculate scale factor to meet API constraints.1669 int longEdge = Math.Max(width, height);1670 int totalPixels = width * height;16711672 double longEdgeScale = 1568.0 / longEdge;1673 double totalPixelsScale = Math.Sqrt(1_150_000.0 / totalPixels);16741675 return Math.Min(1.0, Math.Min(longEdgeScale, totalPixelsScale));1676 }16771678 // When capturing screenshot1679 double scale = GetScaleFactor(screenWidth, screenHeight);1680 int scaledWidth = (int)(screenWidth * scale);1681 int scaledHeight = (int)(screenHeight * scale);16821683 // Resize image to scaled dimensions before sending to Claude1684 var screenshot = CaptureAndResize(scaledWidth, scaledHeight);16851686 // When handling Claude's coordinates, scale them back up1687 void ExecuteClick(int x, int y)1688 {1689 double screenX = x / scale;1690 double screenY = y / scale;1691 PerformClick(screenX, screenY);1692 }1693 ```16941695 ```go Go1696 func getScaleFactor(width, height int) float64 {1697 longest := float64(max(width, height))1698 area := float64(width * height)1699 return min(1.0, 1568/longest, math.Sqrt(1_150_000/area))1700 }17011702 // ...1703 screenWidth, screenHeight := 1512, 98217041705 // When capturing screenshot1706 scale := getScaleFactor(screenWidth, screenHeight)1707 scaledWidth := int(float64(screenWidth) * scale)1708 scaledHeight := int(float64(screenHeight) * scale)17091710 // Resize image to scaled dimensions before sending to Claude1711 screenshot := captureAndResize(scaledWidth, scaledHeight)17121713 // When handling Claude's coordinates, scale them back up1714 executeClick := func(x, y int) {1715 performClick(float64(x)/scale, float64(y)/scale)1716 }1717 ```17181719 ```java Java1720 static double getScaleFactor(int width, int height) {1721 return Math.min(1722 1.0,1723 Math.min(1724 1568.0 / Math.max(width, height),1725 Math.sqrt(1_150_000.0 / (width * height))1726 )1727 );1728 }17291730 void main() {1731 int screenWidth = 1512, screenHeight = 982;17321733 // When capturing screenshot1734 double scale = getScaleFactor(screenWidth, screenHeight);1735 int scaledWidth = (int)(screenWidth * scale);1736 int scaledHeight = (int)(screenHeight * scale);17371738 // Resize image to scaled dimensions before sending to Claude1739 var screenshot = captureAndResize(scaledWidth, scaledHeight);17401741 // When handling Claude's coordinates, scale them back up1742 BiConsumer<Integer, Integer> executeClick =1743 (x, y) -> performClick(x / scale, y / scale);1744 // ...1745 }1746 ```17471748 ```php PHP1749 function getScaleFactor(int $width, int $height): float1750 {1751 return min(1752 1.0,1753 1568 / max($width, $height),1754 sqrt(1_150_000 / ($width * $height)),1755 );1756 }17571758 $screenWidth = 1512;1759 $screenHeight = 982;17601761 // When capturing screenshot1762 $scale = getScaleFactor($screenWidth, $screenHeight);1763 $scaledWidth = (int)($screenWidth * $scale);1764 $scaledHeight = (int)($screenHeight * $scale);17651766 // Resize image to scaled dimensions before sending to Claude1767 $screenshot = captureAndResize($scaledWidth, $scaledHeight);17681769 // When handling Claude's coordinates, scale them back up1770 $executeClick = fn(int $x, int $y) => performClick($x / $scale, $y / $scale);1771 ```17721773 ```ruby Ruby1774 def get_scale_factor(width, height)1775 [1.0, 1568.0 / [width, height].max, Math.sqrt(1_150_000.0 / (width * height))].min1776 end17771778 screen_width, screen_height = 1512, 98217791780 # When capturing screenshot1781 scale = get_scale_factor(screen_width, screen_height)1782 scaled_width = (screen_width * scale).to_i1783 scaled_height = (screen_height * scale).to_i17841785 # Resize image to scaled dimensions before sending to Claude1786 screenshot = capture_and_resize(scaled_width, scaled_height)17871788 # When handling Claude's coordinates, scale them back up1789 execute_click = ->(x, y) { perform_click(x / scale, y / scale) }1790 ```1791</CodeGroup>17921793<Note>1794 **macOS Retina displays** capture screenshots at a device pixel ratio of 2, so the image is twice the resolution of the logical screen coordinates. Either downscale the screenshot by 2x before sending, or halve the coordinates Claude returns before issuing the click.1795</Note>17961797When you choose a display resolution and return screenshots:17981799* For general desktop tasks, use 1024x768 or 1280x720; for web applications, use 1280x800 or 1366x768.1800* Avoid resolutions above 1920x1080 to prevent performance issues.1801* Encode screenshots as base64 PNG or JPEG, and consider compressing large screenshots to improve performance.1802* Include relevant metadata such as timestamp or display state.1803* If you use higher resolutions, ensure coordinates are accurately scaled.18041805### Manage screenshot history18061807Long agent loops accumulate screenshots quickly (roughly 1,000–1,800 input tokens each). The API's [request limits](https://platform.claude.com/docs/en/build-with-claude/vision#request-limits) also apply. Once a single request carries more than 20 images, every image in it is held to a stricter per-side limit. A loop that keeps its screenshot history reaches that count within a few dozen turns, so either resize each screenshot so that neither side exceeds 2000 px or prune older screenshots to keep 20 or fewer in the request.18081809To keep [Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) effective while bounding context:18101811* Place one `cache_control` breakpoint after the system prompt and tool definitions, and up to three more on the last `tool_result` block of each of the most recent turns, advancing them each turn. Within a [batch action](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions), markers on several blocks act as a single breakpoint but each still counts toward the limit of four, so use one per turn.1812* Prune old screenshots in *batches*, not one each turn. Dropping a screenshot every turn changes the prefix every turn and invalidates the cache. A reasonable default is to keep the last three screenshots and prune every 25 turns, so the prefix stays byte-identical between prune events; if your screenshots exceed 2000 px on either side, choose an interval that keeps each request at 20 or fewer images.1813* On Claude Fable 5.1, avoid pruning on the client: removing an earlier screenshot [invalidates every later thinking block](https://platform.claude.com/docs/en/build-with-claude/thinking#preserved-in-conversation) in every request that still carries those turns. Resize screenshots to 2000 px or less per side instead, and use server-side [tool result clearing](https://platform.claude.com/docs/en/build-with-claude/context-editing#tool-result-clearing) to drop old ones from the context. If you must prune, keep [`prefix_mismatch_behavior: "drop_block"`](https://platform.claude.com/docs/en/build-with-claude/thinking#preserved-thinking-controls) set from then on; after each prune, Claude continues without the thinking produced since the pruned screenshot, on that request and every later one.18141815### Diagnose click issues18161817If clicks miss their targets, the cause is usually one of the following:18181819| Symptom | Likely cause | Try |1820| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |1821| Clicks consistently offset in one direction | Claude's coordinates, which are in the pixel space of the screenshots you return, are being applied to a display of a different size without scaling | Scale each coordinate by the ratio of your screen size to your screenshot size before clicking (see [Size screenshots to fit image limits](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions)); on macOS Retina displays, account for the 2x device pixel ratio |1822| Clicks land in the right area but miss the target | Target is very small, detail was lost downscaling a 4K+ source, or aspect ratio was distorted | Keep the `zoom` member enabled and implement it so Claude can inspect the region at full resolution; capture at lower DPI or crop to the relevant region; preserve aspect ratio when resizing |1823| Claude clicks the wrong element entirely | Ambiguous instruction, or visually similar elements nearby | Use positional prompts ("the blue Submit button in the bottom-right"); break the interaction into smaller steps |1824| Accuracy is consistently poor | Resolution too low | Try 1280x720 as a baseline |18251826<Tip>1827 **Model choice affects click precision.** Among the models that use the earlier `computer_20251124` tool, Claude Sonnet 4.6 is more mechanically precise at clicking than Claude Opus 4.6 and is more robust when screenshots require heavy downscaling. Claude Opus 4.7 narrows that gap: its click precision is roughly comparable to Sonnet 4.6, and its higher resolution limit means less downscaling is needed.1828</Tip>18291830### Follow implementation best practices18311832<AccordionGroup>1833 <Accordion title="Add action delays">1834 Some applications need time to respond to actions:18351836 <CodeGroup exclude="shell">1837 ```python Python1838 def click_and_wait(x, y, wait_time=0.5):1839 click_at(x, y)1840 time.sleep(wait_time) # Allow UI to update1841 ```18421843 ```typescript TypeScript1844 async function clickAndWait(x: number, y: number, waitMs = 500): Promise<void> {1845 clickAt(x, y);1846 await setTimeout(waitMs); // Allow UI to update1847 }1848 ```18491850 ```csharp C#1851 static void ClickAndWait(int x, int y, double waitSeconds = 0.5)1852 {1853 ClickAt(x, y);1854 Thread.Sleep(TimeSpan.FromSeconds(waitSeconds)); // Allow UI to update1855 }1856 ```18571858 ```go Go1859 func clickAndWaitFor(x, y int, wait time.Duration) {1860 clickAt(x, y)1861 time.Sleep(wait) // Allow UI to update1862 }18631864 func clickAndWait(x, y int) {1865 clickAndWaitFor(x, y, 500*time.Millisecond)1866 }1867 ```18681869 ```java Java1870 void clickAndWait(int x, int y) throws InterruptedException {1871 clickAndWait(x, y, 500);1872 }18731874 void clickAndWait(int x, int y, long waitTimeMillis) throws InterruptedException {1875 clickAt(x, y);1876 Thread.sleep(waitTimeMillis); // Allow UI to update1877 }1878 ```18791880 ```php PHP1881 function clickAndWait(int $x, int $y, float $waitSeconds = 0.5): void1882 {1883 clickAt($x, $y);1884 usleep((int) ($waitSeconds * 1_000_000)); // Allow UI to update1885 }1886 ```18871888 ```ruby Ruby1889 def click_and_wait(x, y, wait_time: 0.5)1890 click_at(x, y)1891 sleep(wait_time) # Allow UI to update1892 end1893 ```1894 </CodeGroup>1895 </Accordion>18961897 <Accordion title="Validate actions before running them">1898 Check that requested actions are safe and valid:18991900 <CodeGroup exclude="shell">1901 ```python Python1902 display_width, display_height = 1024, 768190319041905 def validate_action(action_type, params):1906 if action_type == "left_click" and "coordinate" in params:1907 x, y = params["coordinate"]1908 if not (0 <= x < display_width and 0 <= y < display_height):1909 return False, "Coordinates out of bounds"1910 return True, None1911 ```19121913 ```typescript TypeScript1914 const displayWidth = 1024;1915 const displayHeight = 768;19161917 interface ActionParams {1918 coordinate?: [number, number];1919 }19201921 function validateAction(actionType: string, params: ActionParams): [boolean, string | null] {1922 if (actionType === "left_click" && params.coordinate) {1923 const [x, y] = params.coordinate;1924 if (!(x >= 0 && x < displayWidth && y >= 0 && y < displayHeight)) {1925 return [false, "Coordinates out of bounds"];1926 }1927 }1928 return [true, null];1929 }1930 ```19311932 ```csharp C#1933 const int DisplayWidth = 1024;1934 const int DisplayHeight = 768;1935 // ...1936 static (bool IsValid, string? Error) ValidateAction(string actionType, IReadOnlyDictionary<string, JsonElement> parameters)1937 {1938 if (actionType == "left_click" && parameters.TryGetValue("coordinate", out JsonElement coordinate))1939 {1940 int x = coordinate[0].GetInt32();1941 int y = coordinate[1].GetInt32();1942 if (x is < 0 or >= DisplayWidth || y is < 0 or >= DisplayHeight)1943 {1944 return (false, "Coordinates out of bounds");1945 }1946 }1947 return (true, null);1948 }1949 ```19501951 ```go Go1952 const (1953 displayWidth = 10241954 displayHeight = 7681955 )19561957 func validateAction(actionType string, params map[string]any) (bool, string) {1958 raw, hasCoordinate := params["coordinate"]1959 if actionType == "left_click" && hasCoordinate {1960 coord, ok := raw.([]any)1961 if !ok || len(coord) != 2 {1962 return false, "Invalid coordinate"1963 }1964 x, y := int(coord[0].(float64)), int(coord[1].(float64))1965 if !(0 <= x && x < displayWidth && 0 <= y && y < displayHeight) {1966 return false, "Coordinates out of bounds"1967 }1968 }1969 return true, ""1970 }1971 ```19721973 ```java Java1974 static final int DISPLAY_WIDTH = 1024;1975 static final int DISPLAY_HEIGHT = 768;19761977 record Validation(boolean valid, String error) {}19781979 Validation validateAction(String actionType, Map<String, JsonValue> params) {1980 if (actionType.equals("left_click") && params.containsKey("coordinate")) {1981 List<JsonValue> coord = (List<JsonValue>) params.get("coordinate").asArray().get();1982 long x = ((Number) coord.get(0).asNumber().get()).longValue();1983 long y = ((Number) coord.get(1).asNumber().get()).longValue();1984 if (!(0 <= x && x < DISPLAY_WIDTH && 0 <= y && y < DISPLAY_HEIGHT)) {1985 return new Validation(false, "Coordinates out of bounds");1986 }1987 }1988 return new Validation(true, null);1989 }1990 ```19911992 ```php PHP1993 const DISPLAY_WIDTH = 1024;1994 const DISPLAY_HEIGHT = 768;19951996 /** @return array{bool, ?string} */1997 function validateAction(string $actionType, array $params): array1998 {1999 if ($actionType === 'left_click' && isset($params['coordinate'])) {2000 [$x, $y] = $params['coordinate'];2001 if (!(0 <= $x && $x < DISPLAY_WIDTH && 0 <= $y && $y < DISPLAY_HEIGHT)) {2002 return [false, 'Coordinates out of bounds'];2003 }2004 }2005 return [true, null];2006 }2007 ```20082009 ```ruby Ruby2010 DISPLAY_WIDTH = 10242011 DISPLAY_HEIGHT = 76820122013 def validate_action(action_type, params)2014 if action_type == "left_click" && params.key?(:coordinate)2015 x, y = params[:coordinate]2016 unless (0...DISPLAY_WIDTH).cover?(x) && (0...DISPLAY_HEIGHT).cover?(y)2017 return [false, "Coordinates out of bounds"]2018 end2019 end2020 [true, nil]2021 end2022 ```2023 </CodeGroup>2024 </Accordion>20252026 <Accordion title="Log actions for debugging">2027 Keep a log of all actions for troubleshooting:20282029 <CodeGroup exclude="shell">2030 ```python Python2031 import logging203220332034 def log_action(action_type, params, result):2035 logging.info(f"Action: {action_type}, Params: {params}, Result: {result}")2036 ```20372038 ```typescript TypeScript2039 function logAction(actionType: string, params: unknown, result: unknown): void {2040 console.error(2041 `Action: ${actionType}, Params: ${JSON.stringify(params)}, Result: ${JSON.stringify(2042 result2043 )}`2044 );2045 }2046 ```20472048 ```csharp C#2049 static void LogAction(string actionType, object? parameters, object? result)2050 {2051 Console.Error.WriteLine($"Action: {actionType}, Params: {parameters}, Result: {result}");2052 }2053 ```20542055 ```go Go2056 func logAction(actionType string, params map[string]any, result any) {2057 log.Printf("Action: %s, Params: %v, Result: %v", actionType, params, result)2058 }2059 ```20602061 ```java Java2062 import static java.lang.System.Logger.Level.INFO;20632064 static final System.Logger LOGGER = System.getLogger("computer-use");20652066 void logAction(String actionType, Object params, Object result) {2067 LOGGER.log(INFO, "Action: {0}, Params: {1}, Result: {2}", actionType, params, result);2068 }2069 ```20702071 ```php PHP2072 function logAction(string $actionType, array $params, mixed $result): void2073 {2074 error_log(sprintf(2075 'Action: %s, Params: %s, Result: %s',2076 $actionType,2077 json_encode($params),2078 json_encode($result),2079 ));2080 }2081 ```20822083 ```ruby Ruby2084 require "logger"20852086 LOGGER = Logger.new($stderr)20872088 def log_action(action_type, params, result)2089 LOGGER.info("Action: #{action_type}, Params: #{params}, Result: #{result}")2090 end2091 ```2092 </CodeGroup>2093 </Accordion>2094</AccordionGroup>20952096***20972098## Migrate from `computer_20251124`20992100Upgrading from `computer_20251124` to the toolset is optional: the models listed for `computer_20251124` under [Earlier tool versions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#earlier-tool-versions) keep accepting it with its beta header, so an existing integration keeps working until you change it. To upgrade, make the following changes together:210121021. **Remove the beta header.** Drop `anthropic-beta: computer-use-2025-11-24` from your requests. In the SDKs, remove the `betas` parameter and call the Messages API through the standard client rather than the beta namespace.21032. **Change the `tools` entry.** Set `type` to `computer_toolset_20260801` and delete `name`, `display_width_px`, `display_height_px`, `display_number`, and `enable_zoom`. The toolset rejects each of these fields.21043. **Choose whether to keep zoom enabled.** Zoom is enabled by default on the toolset, whereas `enable_zoom` defaults to `false`. If your environment doesn't implement zoom, add `"configs": {"zoom": {"enabled": false}}` to keep the previous behavior; otherwise implement it (see [Available actions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#available-actions)).21054. **Handle every block in a turn.** Update your agent loop to iterate over every `tool_use` block in a response rather than reading only the first, and to dispatch on the block's `name` together with `toolset_name` instead of on `input.action`. Member inputs no longer contain an `action` field; the remaining fields are unchanged.21065. **Run blocks in order and use the halt text.** Run the blocks sequentially, stop at the first failure, and answer the remaining blocks with `Not executed: an earlier computer action in this turn failed.` as described in [Batch actions](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#batch-actions). If your loop can't run batches yet, [Tool parameters](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#tool-parameters) explains how to limit Claude to one action per turn.21076. **Echo `toolset_name` on results.** Add `"toolset_name": "computer"` to every `tool_result` that answers a member call. Results may contain only `text` and `image` content.21087. **Support `repeat` on `key`.** The `key` member accepts an optional `repeat` count from 1 to 100. A handler that ignores unrecognized fields would press the key once, so make your `key` handler honor `repeat`.21098. **Resize screenshots yourself.** The toolset rejects a screenshot or zoom image that exceeds the model's image limits instead of downscaling it. Resize before returning the image and keep scaling coordinates as described in [Size screenshots to fit image limits](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#handle-coordinate-scaling-for-higher-resolutions).21109. **Remove unsupported options.** Move any `defer_loading` from the entry into `configs`, with the same value on every enabled member. The other options not supported on toolset entries are listed under [Client toolsets](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference#client-toolsets).21112112This is the `tools` entry before the change, sent with the `anthropic-beta: computer-use-2025-11-24` header:21132114```json2115{2116 "type": "computer_20251124",2117 "name": "computer",2118 "display_width_px": 1024,2119 "display_height_px": 768,2120 "display_number": 12121}2122```21232124This is the `tools` entry after the change, sent with no beta header. The `configs` object keeps zoom off to match the earlier entry, which doesn't set `enable_zoom`; omit `configs` entirely to accept the default and let Claude zoom:21252126```json2127{2128 "type": "computer_toolset_20260801",2129 "configs": {2130 "zoom": { "enabled": false }2131 }2132}2133```21342135The following pair shows a `tool_use` block before and after the change. The action name moves from `input.action` to `name`, and the block gains `toolset_name`:21362137```json2138{2139 "type": "tool_use",2140 "id": "toolu_01A9r5kQm2LxWc7vT3nZ4bJs",2141 "name": "computer",2142 "input": { "action": "left_click", "coordinate": [500, 300] }2143}2144```21452146```json2147{2148 "type": "tool_use",2149 "id": "toolu_01A9r5kQm2LxWc7vT3nZ4bJs",2150 "name": "left_click",2151 "toolset_name": "computer",2152 "input": { "coordinate": [500, 300] }2153}2154```21552156## Earlier tool versions21572158Two earlier versions of the computer use tool remain available in beta for existing integrations, for models that don't support the toolset, and on platforms where the toolset isn't currently available. Each requires its [beta header](https://platform.claude.com/docs/en/api/beta-headers) on every request, and their parameters are documented in the [beta Messages API reference](https://platform.claude.com/docs/en/api/beta/messages/create). In the SDKs, pass the header through the `betas` parameter and use the beta namespace; only the computer use tool needs the header, not the bash or text editor tools in the same request.21592160| Tool version | Beta header | Use with | Parameters |2161| ------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |2162| `computer_20251124` | `computer-use-2025-11-24` | Claude Fable 5.1, Claude Mythos 5.1, Claude Fable 5, Claude Mythos 5, Claude Opus 5, Claude Sonnet 5, Claude Opus 4.8, Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 4.6, and Claude Opus 4.5 | [API reference](https://platform.claude.com/docs/en/api/beta/messages/create) |2163| `computer_20250124` | `computer-use-2025-01-24` | Claude Sonnet 4.5, Claude Haiku 4.5, Claude Opus 4.1 ([retired, except on Bedrock and Google Cloud](https://platform.claude.com/docs/en/about-claude/model-deprecations)), Claude Sonnet 4 ([retired, except on Bedrock and Google Cloud](https://platform.claude.com/docs/en/about-claude/model-deprecations)), and Claude Opus 4 ([retired, except on Google Cloud](https://platform.claude.com/docs/en/about-claude/model-deprecations)) | [API reference](https://platform.claude.com/docs/en/api/beta/messages/create) |21642165***21662167## Limitations216821691. **Latency:** The current computer use latency for human-AI interactions might be too slow compared to regular human-directed computer actions. Focus on use cases where speed isn't critical (for example, background information gathering, automated software testing) in trusted environments.21702. **Computer vision accuracy and reliability:** Claude might make mistakes or hallucinate when outputting specific coordinates while generating actions. Claude's [summarized thinking](https://platform.claude.com/docs/en/build-with-claude/thinking#summarized-thinking) output can help you understand the model's reasoning and identify potential issues; set `display: "summarized"` on the thinking configuration, because the models that support the toolset omit thinking text by default.21713. **Tool selection accuracy and reliability:** Claude might make mistakes or hallucinate when selecting tools while generating actions or take unexpected actions to solve problems. Additionally, reliability might be lower when interacting with niche applications or multiple applications at once. Prompt the model carefully when requesting complex tasks.21724. **Scrolling reliability:** The scroll action supports direction control (up, down, left, right) and a specified amount. In applications where scrolling doesn't take effect, keyboard alternatives such as Page Down can help.21735. **Spreadsheet interaction:** Use the fine-grained mouse control actions (`left_mouse_down`, `left_mouse_up`) and modifier-key combinations to select individual cells. Complex spreadsheet operations might still require multiple attempts.21746. **Account creation and content generation on social and communications platforms:** Although Claude visits websites, its ability to create accounts, generate and share content, or otherwise engage in human impersonation across social media websites and platforms is limited.21757. **Vulnerabilities:** Jailbreaks and prompt injection can affect computer use as they can any frontier AI system, including through instructions embedded in webpages or images; apply the precautions in [Security considerations](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool#security-considerations).21768. **Inappropriate or illegal actions:** Under Anthropic's Terms of Service, you must not employ computer use to violate any laws or the Acceptable Use Policy.21772178Always carefully review and verify Claude's computer use actions and logs. Do not use Claude for tasks requiring perfect precision or sensitive user information without human oversight.21792180## Data retention21812182Computer use is a client-side tool. All screenshots, mouse actions, keyboard inputs, and any files involved in a session are captured and stored in your environment, not by Anthropic. Anthropic processes the screenshot images and action requests in real time as part of the API call. Retention for those API requests is governed by [API and data retention](https://platform.claude.com/docs/en/manage-claude/api-and-data-retention).21832184Because your application controls where and how computer use data is stored, computer use is ZDR eligible. For ZDR eligibility across all features, see [API and data retention](https://platform.claude.com/docs/en/manage-claude/api-and-data-retention).21852186## Pricing21872188Computer use follows the standard [tool use pricing](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview#pricing). When using the computer use tool:21892190**Toolset definition overhead:** Declaring `computer_toolset_20260801` with its default members adds about 4,500 input tokens to a request (about 4,520 on Claude Fable 5, Claude Mythos 5, Claude Opus 5, and Claude Opus 4.8, and about 4,590 on Claude Sonnet 5), which covers the member tool definitions and the tool use system prompt. Disabling `zoom` with `configs` removes about 410 of those tokens. The exact count for a request is reported in the response `usage`, and you can estimate it in advance with the [token counting endpoint](https://platform.claude.com/docs/en/build-with-claude/token-counting).21912192**Earlier tool versions:** The following figures apply to the `computer_20251124` and `computer_20250124` tool versions, not to `computer_toolset_20260801`:21932194* System prompt overhead: 466–499 tokens added to the system prompt2195* Tool definition: about 735 input tokens per tool definition (measured with `computer_20250124`)21962197**Additional token consumption:**21982199* Screenshot and zoom images returned in tool results, billed as image input (see [Vision pricing](https://platform.claude.com/docs/en/build-with-claude/vision#evaluate-image-size))2200* Tool execution results returned to Claude22012202<Note>2203 If you're also using bash or text editor tools alongside computer use, those tools have their own token costs as documented in their respective pages.2204</Note>22052206## Next steps22072208<CardGroup cols={2}>2209 <Card title="Troubleshooting tool use" icon="wrench" href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/troubleshooting-tool-use">2210 Fix the most common tool-use errors with symptom-to-fix diagnostic tables.2211 </Card>22122213 <Card title="Reference implementation" icon="github-logo" href="https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo">2214 Get started with the complete Docker-based implementation2215 </Card>22162217 <Card title="Tool use with Claude" icon="tool" href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview">2218 Connect Claude to external tools and APIs. See where tools execute, when Claude calls them, and which tool fits your task.2219 </Card>22202221 <Card title="Best practices in detail" icon="book" href="https://claude.com/blog/best-practices-for-computer-and-browser-use-with-claude">2222 Benchmarked recommendations for resolution, thinking effort, and context management2223 </Card>22242225 <Card title="Browser use tool" icon="browser" href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/browser-use-tool">2226 Let Claude navigate, read, and interact with webpages in your own browser environment, for tasks that stay inside the browser.2227 </Card>2228</CardGroup>2229