Google has introduced Gemini Robotics ER 2, its latest embodied-reasoning model intended to act as a high-level controller for robots operating in physical environments. The model is designed to let robots make decisions and reason at the real-time pace required by the physical world, not just perform static spatial reasoning.
Capabilities and role
Gemini Robotics ER 2 provides high-level planning and dialogue capabilities: robots can converse with humans, interpret the physical environment, plan multi-step tasks, and hand off motor execution to lower-level Vision-Language-Action (VLA) models or other control APIs. The model can natively call external tools such as Google Search or any user-defined function.
The system is designed so the model can continue planning while actions are being executed, avoiding stop-and-think pauses during operation.
Improvements over ER 1.6
Compared with Gemini Robotics ER 1.6, ER 2 brings substantial upgrades. By watching continuous video streams, the model can track task progress, adapt to errors, and determine precisely when to move to the next step. ER 2 also introduces multi-robot collaboration, enabling heterogeneous robots to work together on workflows that a single robot could not complete alone.
Availability and developer access
Gemini Robotics ER 2 is available to developers via the Gemini API and Google AI Studio, and in private preview on the Gemini Enterprise Agent Platform. Google has published examples and configuration guides to help developers integrate the model into physical AI workflows.
Agentic workflows and tool orchestration
ER 2 acts as a physical agent, orchestrating steps for robots, enabling self-correction, and generalizing to novel situations. Developers can declare low-level control interfaces—such as VLA models or navigation APIs—as tools, and stream multimodal video, audio, or text directly into the model.
The update improves tool orchestration and has been evaluated in simulation, with real robot control, and with human teleoperation. The company reports that ER 2 consistently outperforms ER 1.6 across three control modes: real VLA, sim VLA, and human tele-op.
Real-time integration and latency
High-level reasoning in robotics depends on execution speed. Gemini Robotics ER 2 integrates with the Gemini Live API using a bidirectional streaming endpoint optimized for latency-sensitive tasks. This enables fluid orchestration: ER 2 issues commands to action models and robotics APIs to complete multi-step tasks without disruptive pauses.
Google demonstrated a use case with Spot from Boston Dynamics, where ER 2 orchestrates Spot APIs (navigation and manipulator motion) to create an interactive robot that can fetch objects on natural language command. Example code is available on GitHub.
Temporal intelligence and task progress
A core challenge in robotics is determining when a task is finished. ER 2 improves video understanding and progress tracking so that complex tasks (for example, tightening a light bulb or tying a trash bag) are verified as complete to specification before transitioning to the next step.
The release highlights two foundational capabilities:
-
Continuous progress classification: each frame in a video feed is assigned to one of five progress levels (0–20%, 20–40%, 40–60%, 60–80%, 80–100%). Quantifying progress gives robots real-time situational awareness and enables on-the-fly adjustments or retrying failed steps without restarting the entire workflow. Gemini Robotics ER 2 achieves 57.4% accuracy on progress classification tasks, surpassing the previous generation and competing frontier models.
-
Precision moment-finding: this measures the model’s ability to identify the exact frame where a critical event occurs (for example, when to stop pouring coffee). Gemini Robotics ER 2 achieves 91.3% accuracy and a 0.96 second mean absolute distance on moment-finding tasks. According to Google, this performance is competitive with much larger model classes while requiring a fraction of the compute cost and delivering four times the execution speed—sub-second latency that is necessary for safe real-world robotics.
Multi-robot collaboration
ER 2 supports multi-robot collaboration by enabling diverse machines to share semantic understanding, hand off tasks, and jointly complete complex workflows. Google cites examples of Apptronik’s Apollo 2 and the Franka F3 Duo collaborating under ER 2 orchestration.
Improved spatial intelligence
Gemini Robotics ER 2 advances core spatial reasoning across several benchmarks:
- Success/failure detection now operates on raw video feeds rather than static snapshots, allowing detection of mid-execution failures such as spills or misalignments.
- General instrument reading extends capability from circular dials and sight glasses to digital displays, linear scales, rulers, and liquid thermometers across ten instrument types.
- Enhanced spatial Visual Question Answering (VQA) benefits from Gemini’s multimodal understanding improvements.
The model reportedly achieves the highest accuracy across these core capabilities, including success detection (image/video), ERQA, and generalized instrument reading.
Safety advances
Google describes Gemini Robotics ER 2 as its safest robotics model to date, showing substantial gains on Safety Instruction Following and Human Proximity benchmarks. The model can halt a humanoid robot when a person is nearby and resume autonomous operation only once the area is clear.
To further evaluate safety, Google introduced a benchmark that assesses a foundation model’s ability to act as a safe VLA orchestrator by testing enforcement of safety constraints, environment monitoring, assessment of physical feasibility, and requesting human clarification when needed. ER 2 outperforms ER 1.6 and other frontier models on these safety-oriented benchmarks.
Outlook
Google plans to continue advancing these models toward more complex tasks, with the stated goal of accelerating the development of helpful robots and supporting the robotics research and development community.
