agent-desktop: developer tools for small teams
Official site link, not an affiliate link. We do not have an approved referral link for agent-desktop.
agent-desktop helps you build software by letting an AI agent read and operate desktop apps through the OS accessibility tree. It is best for developers whose AI agent needs to click and type in desktop apps on the same computer. People also use it to automate repetitive work. A free plan is available.
What it does
agent-desktop is a command-line tool and a C library, published by Vercel Labs, that lets an AI agent read and operate desktop applications through the operating system's accessibility tree. The README marks it as a Vercel Labs product, and the repository is in the vercel-labs organization. It does not name a particular agent product. The bundled skill docs are what an agent reads before the first use: agent-desktop skills get desktop, then agent-desktop skills get platform.
Official docs: agent-browser.dev/desktop. Repository: vercel-labs/agent-desktop. Releases: GitHub Releases. License: Apache-2.0. Open-source tools on this site: GitHub. Another page in that section: PromptDirector.
Editorial note
agent-desktop is a free command-line tool and C library from Vercel Labs. It reads a desktop app through the accessibility tree, acts on a ref, and takes another snapshot to check. The FAQ says the license is Apache-2.0 for personal and commercial use.
Who it’s for
- For: Developers whose AI agent needs to click and type in desktop apps on the same computer.
- Not ideal for: someone who wants a chat window and no terminal. It is a command-line tool and a C library.
- Not ideal for: someone waiting on a Homebrew or Scoop install. Those packages are still being submitted and are not in the default catalogs.
Key use cases
- Build apps & code
- Automate work
What you can do with it
- Free access: The FAQ says agent-desktop is Apache-2.0 licensed for personal and commercial use. This is an ongoing free license, not a trial and not a one-time credit grant.
- Refs, not pixels: A snapshot returns JSON with a ref for every element an agent can act on. The agent acts on a ref and snapshots again to check. The README says this uses no pixel matching. The FAQ says agents do not need to guess from screenshots. A screenshot command still exists for when you want one.
- Three desktop systems: The README lists macOS, Windows, and Linux, with native binaries for x64 and ARM64. GitHub Releases also publishes native archives for those platforms.
- A C library as well as the command: The README calls it a CLI and a C library. The FAQ says that library is a C interface that Python, Swift, Go, Ruby, Node, or C can load, so a program can call it without starting the command each time.
- Current releases: GitHub Releases lists v1.0.0 and v1.0.1 on 28 October 2026 (UTC) and v1.0.2 on 30 October 2026 (UTC). v1.0.2 is the newest release as of October 31, 2026. Earlier 0.x releases are listed on the same page.
How to use agent-desktop: step-by-step tutorial
Five to eight steps from the account to a first useful result. Where the official guide names a control, the step uses that name. Where it does not, the step stays general.
- Open agent-browser.dev and create an account on the free plan. You can reach a first result before you choose a paid plan.
- Install the command with npm install -g agent-desktop, then run agent-desktop version. The README says this works on macOS, Windows, and Linux, and that the package includes native binaries for x64 and ARM64. You can instead download the matching native archive from GitHub Releases and keep its helper beside the executable.
- Allow the operating system to share app structure. On macOS, run agent-desktop permissions --guide and turn on Accessibility. On Windows, the install guide says no extra grant is needed for apps at your terminal's integrity level. On Linux, use your graphical session with at-spi2-core installed.
- Take a snapshot. The README's example is agent-desktop snapshot --app Finder -i. The JSON includes a ref for every element an agent can act on.
- Act on one ref from that snapshot. The README's example is agent-desktop click followed by the ref. A ref belongs to that snapshot. If the screen has changed, the README says to take a new snapshot and use its refs.
- Snapshot the same app again and read the new JSON. The README says the agent acts on a ref and snapshots again to check, with no pixel matching.
These steps follow official guides: agent-desktop README; Installation and permissions. Checked October 31, 2026.
Frequently asked questions
Is agent-desktop free?
Yes. The FAQ says it is Apache-2.0 licensed for personal and commercial use. That is an ongoing free license, not a free trial. Checked October 31, 2026.
Does it guess from screenshots?
The FAQ says no. The core workflow reads the accessibility tree and uses refs. Screenshots exist as a separate command, and agents do not need them to click, type, or move through a window.
Which systems does it support?
macOS, Windows, and Linux, per the README. Native binaries and release archives cover x64 and ARM64 on each.
Does the command call a remote model?
The FAQ says the native CLI does not require a remote model service. An optional helper in the repository, which the FAQ calls the JEV helper, does send goals and screen descriptions to a configured provider.
Which release is current?
GitHub Releases lists v1.0.0 and v1.0.1 on 28 October 2026 (UTC) and v1.0.2 on 30 October 2026 (UTC). v1.0.2 is the newest release as of October 31, 2026. Earlier 0.x releases are on the same page.
Pricing
Prices checked on the official pricing page.
- Open source
-
Free
Apache-2.0 for personal and commercial use. Install with npm install -g agent-desktop, or use a native archive from GitHub Releases.
- Private desktop content: The FAQ says snapshot trees, window titles, clipboard reads, notifications, screenshots, and command results contain the desktop content you asked to inspect. Keep those files, and transcripts that include them, private.
- Windows elevation: The install guide says no grant is needed for apps at your terminal's integrity level. An app running as administrator can be read, but only an elevated agent can send it input.
- macOS screenshots: Accessibility is the permission the README names for normal use. The install guide says screenshots also need Screen Recording permission.
- Linux session: The README says Linux needs the user's graphical session with at-spi2-core. The install guide names Ubuntu 24.04 or newer with GNOME 46 as the primary target.
- Package managers: The install guide says official Homebrew and Scoop packages are being submitted to Homebrew/homebrew-core and ScoopInstaller/Main. They are not in the default catalogs. The repository no longer keeps a custom tap or bucket.
- Optional helper: The FAQ says the native CLI does not require a remote model service. The optional JEV helper, which lives in the repository only, sends goals and screen descriptions to a provider you configure. Use it only with material you are willing to send there.
Facts
- Access
- Ongoing free plan
- Paid plans from
- Free
- Checked
- Category
- Category: Development tools (31 in the library).
- On the record
- The FAQ says agent-desktop is Apache-2.0 licensed for personal and commercial use. This is an ongoing free license, not a trial.