From seeing a screen to writing an interaction procedure
OpenAI's GPT‑6 Astra computer-use guide defines two integration patterns. In code execution, the model writes Playwright or PyAutoGUI code and the application runs it. In the computer tool pattern, the model emits structured mouse and keyboard actions for the application to translate into input. Both remain supported, but OpenAI recommends code execution for Astra.
This is more than an interface preference. A click-oriented loop commonly observes the screen and returns a small action. Code can combine multiple actions, loops and conditional branches in one call. That is useful when a browser test must iterate over similar records, continue only after an element appears, or verify completion against page state.
The model does not own the computer
In the documented architecture, the developer's application is still the executor. It supplies an isolated browser or desktop, runs the generated code and returns observations such as screenshots. Keeping the session alive across calls lets Astra build on prior state. Existing function or remote MCP tools that expose UI operations can remain in place.
The claim that Astra can freely control a user's machine is therefore misleading. The integration controls accounts, network access, permissions and execution limits.
The important shift is the unit of work
Code execution makes grouped actions and verification easier to express, but it does not make them safe by default. Production systems should allow-list destinations and capabilities, cap time and steps, inspect the screen and real state after short action groups, and require approval before payment, transmission or destructive changes.
The innovation is a shift from selecting the next coordinate toward an execution agent that observes, writes a small program and verifies its outcome. Teams should measure completion, unintended changes, recovery, approvals and total cost—not only whether a polished demo succeeds once.