Skip to content

About

Desktop app and CLI for AI agents to control computers and physical devices. https://machinecontrol.dev

Resources

Security policy

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

Machine Control

Let AI agents use your computers and devices—with access you control.

Machine Control is a desktop app for macOS, Windows, and Linux that gives agents access to applications, browser tabs, screenshots, keyboard, and mouse. Choose what they can access, pause control, and review activity. It also provides a CLI for automation and remote targets, including physical iPhones, Chromebooks, Android devices, and more.

Download · Desktop guide · CLI guide · Device setup

Machine Control on macOS, showing access scopes, Pause access, and the Permissions, Activity, and Settings tabs.

Machine Control on macOS. Choose access scopes and pause control at any time.

What you can do

  • Use desktop applications: inspect windows and UI elements, click controls, enter text, and capture screenshots.
  • Control the browser: inspect and operate tabs through the optional Chrome extension, including supported CDP and page JavaScript.
  • Work with real devices: install and exercise apps on a physical iPhone, operate a Chromebook's desktop and browser, or deploy to Android and Quest.
  • Test locally or remotely: drive your own computer, a VM, or a configured physical device from the agent's existing session.
  • Prepare test environments: bring up VMs, check readiness, acquire isolated workspaces, and clean up through the same target-selecting CLI.
  • Keep access visible: select access and duration in the desktop app, pause supported control sessions, stop access, and inspect activity.

The unified Desktop/Browser permissions experience is implemented in the unreleased Windows, macOS and Linux adapters over one shared policy core. Their bounded appliance records cover ordinary control, profiles and platform-specific protected/locked behavior. The linked topic records exact candidate evidence and remaining qualification limits.

Available controls depend on the platform and selected permissions. Machine Control combines native UI information with screenshots and input, so agents can inspect what a button means as well as where it appears.

Machine Control is independent of the agent's model provider. Its Computer Use positioning and roadmap separates the complete package's strengths from measured GUI-control gaps and the narrower coverage of the experimental persistent Python interface. For a concise proposed agent entry prompt, see the Machine Control entry guide. For the optional genuine Sky route from Claude, see the repo-local guide and skill.

Get the desktop app

Download for Mac, Windows, or Linux. The app includes the control CLI and its Python runtime; using the installed CLI does not require a source checkout or a separate Python installation.

  1. Install and open Machine Control.
  2. Follow Permissions to enable the platform's required access and optional browser integration.
  3. Choose access scopes and duration, or approve an agent's request.

The app runs in the menu bar or tray. Closing its settings window keeps it running; Quit ends it. Access starts off, and you can stop it from the app. Grants normally apply to callers running as your user; an agent with an unrestricted shell under that same account is not contained by those grants. See security and authorization for the boundaries.

What works today

Current: desktop packages are available for both architectures on all three platforms. The app is actively used and developed; the release remains pre-1.0, and testing coverage varies by platform and environment.

Platform Desktop app availability
macOS Apple silicon and Intel; native app control, capture/input, browser integration, and signed updates. Recorded execution coverage is strongest on Apple silicon.
Windows x64 and ARM64; native app control, capture/input, browser integration, and signed updates, with installed GUI acceptance on both architectures. Optional UAC and locked-screen helpers require separate setup.
Linux x64 and ARM64 Debian/AppImage packages for Ubuntu GNOME Wayland. App control uses accessibility; screen/input sharing uses visible portal consent. Broader Linux desktops remain open.

The desktop acceptance matrix records exact package, architecture, VM, and physical-machine coverage. Setup and feature limits live in the desktop guide and Linux installation guide. Release changes live in the changelog.

Physical devices and remote targets

Machine Control also works beyond the computer running the desktop app. These routes use platform-specific setup and native runners through the common CLI; they are not additional desktop app installers.

  • Physical iOS devices: control a real iPhone from an authorized Mac using CoreDevice and XCTest. Install and launch apps, inspect and act on UI elements, take screenshots, and collect app/system logs, crash reports, and app-container files. Developer Mode, pairing, and runner signing are part of setup; protected authentication retains its native limits.
  • ChromeOS: control a dedicated developer-mode Chromebook's desktop accessibility tree, browser tabs, screenshots, and input remotely, with the controls running on the Chromebook.
  • Android and Quest: native device discovery, deployment, lifecycle, capture/input, and diagnostics through their ADB-based routes.
  • Steam Deck: device administration and session-aware native operations.

For desktop VMs and remote computers, see the platform guides and controller-host support. Ordinary VM control runs inside the guest and does not require focusing its hypervisor window.

Command-line automation

The CLI supports desktop operations, physical devices, remote targets, and headless deployments. From a source checkout, start with read-only discovery:

git clone https://github.com/kzahel/machine-control.git
cd machine-control
bin/machine-control targets

On Windows, use py -3 bin/machine-control targets. The source client requires Python 3.10 or later. Follow the CLI guide for target setup, desktop and iOS examples, scoped tasks, and VM workspaces. Before operating an accepted VM, run its read-only doctor and acquire an exclusive target-use claim; scoped tasks manage claim renewal and cleanup.

North Star

Decision: an authorized agent should be able to select a supported computer or device and use its richest practical native controls, whether the agent runs locally or elsewhere. Desktop control executes in the target OS; constrained devices use their strongest native runner and authorized device host. Another agent inside the target is optional. Hypervisor consoles and external KVM/input are explicit bootstrap and recovery routes.

The project owns a common control experience over replaceable providers, while preserving platform capabilities and limits. Results distinguish action delivery from observed effects. See architecture, inner-first routing, and capabilities and results.

Current, unreleased desktop/reference implementation: independent Desktop and Browser Off, Ask first and Automatic modes share one permission/handoff engine, including takeover warnings, deliberate-input interruption and queued retry. Browser includes supported CDP and page JavaScript. Broad review and bounded unsigned Windows ARM64 candidate acceptance have passed. macOS and Linux now adopt that same core; their native/package records are linked from the shared-resident topic. Signed publication and broader hardware remain separate. The permissions and handoff contract and implementation record own the contract, evidence and limits. These modes have not been published.

Documentation and development

Run python3 bin/check --portable for dependency-light checks that do not contact configured targets. Use python3 bin/check --native for applicable platform builds and static validation; deeper suites are in the platform guides and client tests.

License

MIT.

About

Desktop app and CLI for AI agents to control computers and physical devices. https://machinecontrol.dev

Resources

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages