Peter Steinberger has released Peekaboo as an open-source, macOS-native toolkit that exposes the system accessibility tree directly to automation agents. This allows clicks and other interactions to target real UI elements with IDs instead of relying on pixel coordinates that shift when windows move.
What Peekaboo provides
-
Native, not virtualized: Peekaboo runs as a real macOS process and requires Screen Recording and Accessibility permissions. That lets it drive apps that are often resistant to in-browser or virtualized automation.
-
Structured menu discovery: the peekaboo menu command returns the full menu tree as JSON, so agents can navigate named menu paths like "File → Export → PDF…" rather than pattern-matching on screenshots.
-
Multi-screen and multi-Space awareness: first-class support for moving windows between Spaces, switching desktops, and targeting elements on specific displays.
-
Covers common agent shortcomings: beyond simple click-and-type actions, Peekaboo supports interactions that many agents fail at, such as Dock right-clicks, menu bar extras, file dialogs, and drag-to-Trash operations.
Integrations and how to use it
Peekaboo provides a CLI and an MCP server interface, so it can be used with agent harnesses like Claude Code, Codex, OpenClaw, or Hermes Agent. You can pair it with many models — the announcement mentions Claude, GPT-5.1, Grok 4-fast, and local runs via Ollama.
The project is released under the MIT license and is included in the OpenClaw repository as skills/peekaboo-cli, making it available as an OpenClaw skill alongside more than 5,400 other skills.
Why it matters
By giving agents structured access to macOS accessibility data, Peekaboo aims to make UI automation on macOS more reliable and robust, avoiding brittle screenshot-based heuristics. That improvement matters for workflows that depend on precise menu navigation, multi-display setups, or native dialogs that are hard to control through pixel-based methods.
