GitHub Copilot can now interact with desktop apps with computer use
GitHub Copilot has introduced 'computer use' capabilities in public preview for the CLI and desktop applications on macOS and Windows. This feature enables the AI to actively interact with desktop software on behalf of the user.
Verified State Diff
Impact & Verification Analysis
Developers and enterprise users utilizing GitHub Copilot on macOS and Windows platforms.
This marks a transition toward autonomous agentic workflows, significantly expanding the scope of AI assistance from code generation to cross-application task automation and system-level operations.
Full Fact Overview
The integration of 'computer use' represents a shift from text-based code generation to agentic UI interaction, allowing Copilot to perform tasks across the operating system environment. By leveraging desktop-level access, the model can manipulate GUI elements, potentially automating workflows that were previously restricted to terminal or IDE-based interactions. This capability suggests the use of vision-language models capable of interpreting screen states and executing mouse or keyboard inputs.