SPB Git forge

spb/doc-api

Public
2commits 1branches 0releases
15.7 MBsize
maindefault branch
13 days agolast push
Python 88.3% TypeScript 7.6% Shell 4.1%
3.3 KB

# Computer use (computer_use_preview, computer) and the computer-use-preview model

Status: computer_use_preview DOCUMENTED · PREVIEW · ACCOUNT_RESTRICTED (2026-09-18: POST /v1/responses with model: computer-use-preview → HTTP 404 model_not_found "does not exist or you do not have access") · computer DOCUMENTED · UNVERIFIED Sources: https://developers.openai.com/api/docs/guides/tools-computer-use · https://developers.openai.com/api/docs/guides/tools-computer-use-integration · https://developers.openai.com/api/docs/models/computer-use-preview · openapi-master.yaml ComputerUsePreviewTool, ComputerTool, ComputerToolCall, ComputerToolCallOutput, ToolChoiceTypes Last verified: 2026-09-18

# Two integration styles (docs)

  1. Code execution (recommended for GPT-6 Astra): a function tool that accepts a script (PyAutoGUI/Playwright) run in your isolated environment; returns text + screenshots (detail: "original" to preserve pixels).
  2. The computer tool: model returns structured actions; you execute them and send back a screenshot.

# Tool definitions

Tool Fields Notes
{"type":"computer_use_preview","environment":"windows"|"mac"|"linux"|"ubuntu"|"browser","display_width":int,"display_height":int} all required historically with model computer-use-preview (snapshot computer-use-preview-2025-03-11, Responses only, truncation: "auto" required, $3 / $12 per 1M tokens)
{"type":"computer"} none current tool; model pages list computer_use for gpt-5.4, 5.4-mini, 5.4-pro, 5.5, 5.6-, gpt-6-astra, daybreak-

tool_choice hosted forms: {"type":"computer"}, {"type":"computer_use_preview"}, {"type":"computer_use"} (spec enum).

# Loop items

  • Output computer_call {id, call_id, action, actions[] (batched, computer_use), pending_safety_checks[] {id, code, message}, status}. action.type ∈ click {button: left|right|wheel|back|forward, x, y, keys?}, double_click {x,y,keys}, drag {path[{x,y}], keys?}, keypress {keys[]}, move {x,y}, screenshot, scroll {x,y,scroll_x,scroll_y}, type {text}, wait.
  • Input computer_call_output {call_id, output: {type:"computer_screenshot", image_url | file_id}, acknowledged_safety_checks[]}; continue with previous_response_id until no more computer_call.
  • include: ["computer_call_output.output.image_url"] returns screenshot URLs when listing items.

# Safety (docs, mandatory)

Isolated browser/VM + site allow-list · treat screen text as untrusted (cannot grant permission) · confirm at the point of risk and require explicit consent before transmitting sensitive data · stop/escalate on suspected prompt injection · bound steps/time/cost, support cancellation, verify outcomes · acknowledge pending_safety_checks deliberately.

# Live evidence

Probe Status Result
computer-use-preview + computer_use_preview tool + 1×1 PNG input_image 404 invalid_request_error, code: model_not_found → ACCOUNT_RESTRICTED (a 404 does not mean the model is gone; the model page is current)
computer tool on gpt-5.4 not run cost/safety; DOCUMENTED

Examples: examples/openai/tools/computer-use/ (sh/py/ts reproduce the 404). Migration notes: docs section "Migration from computer-use-preview" in the integration guide.