2026年7月30日
|
7分钟阅读
Gemini Robotics ER 2 代表了机器人技术在视频理解、任务编排和多机器人协作方面的阶跃式进步——使机器人在物理世界中能够提供更多帮助。
Steven Hansen
高级员工软件工程师
Peng Xu
员工软件工程师
要让机器人在日常环境中协助人类，仅具备精确的空间推理能力是不够的。机器人还必须快速思考，使其决策和推理与物理世界的实时速度同步。
这就是为什么今天我们推出 Gemini Robotics ER 2，这是我们最强大的机器人“具身推理”模型。可以将 Gemini Robotics ER 2 视为机器人的高级大脑。它使机器人能够与人类对话、理解物理世界并规划多步骤任务。然后，它将运动执行交给任何给定的低级视觉-语言-动作（VLA）模型。Gemini Robotics ER 2 还可以原生调用 Google Search 等工具来查找信息，或调用任何其他用户定义的函数。Gemini Robotics ER 2 的设计使机器人能够在执行动作的同时“思考”接下来要做什么。
Gemini Robotics ER 2 相比 Gemini Robotics ER 1.6 是一次重大升级。通过观看连续视频流，机器人现在可以跟踪自己的进度，在出现问题时进行调整，并确切知道何时进入下一步。我们还引入了多机器人协作，使机器人能够在共享空间中协同工作，完成单个机器人无法独立完成的复杂工作流程。
Gemini Robotics ER 2 现已通过 Gemini API、Google AI Studio 向开发者公开提供，并在 Gemini Enterprise Agent Platform 上提供私有预览。为帮助您快速上手，我们分享了 示例，展示如何配置模型并提示它以驱动更有用的物理 AI 任务。
推进物理智能体能力
物理世界中的大多数任务都很复杂，需要多个步骤才能完成。Gemini Robotics ER 2 是一个物理智能体，为机器人编排步骤，使其能够自我纠正，并泛化到更多新颖场景。要构建智能体设置，开发者可以将低级控制接口——如视觉-语言-动作（VLA）模型或导航 API——声明为工具，并将多模态视频、音频或文本直接流式传输到模型中。
Gemini Robotics ER 2 改进了这种工具编排工作流程。我们可以通过模拟环境中的机器人、使用真实世界机器人控制，甚至将其与远程控制机器人的人类配对来评估其性能。
Gemini Robotics ER 2 在三种控制模式（真实 VLA、模拟 VLA 和人类远程操作）下的工具编排性能始终优于 ER 1.6。
在机器人技术中，高级推理取决于执行速度。Gemini Robotics ER 2 集成到 Gemini Live API 中，使用针对延迟敏感任务优化的双向流式端点。结果是流畅的编排：Gemini Robotics ER 2 指挥动作模型和机器人 API 完成多步骤任务，而不会出现突兀的“停下思考”停顿。
为了说明这一点，我们与合作伙伴 Boston Dynamics 的 Spot 构建了一个演示。我们使用 Gemini Robotics ER 2 来编排 Spot APIs，例如导航和机械臂运动，创建一个交互式机器人，为您取回物品。
Gemini Robotics ER 2 驱动的 Boston Dynamics Spot 根据自然语言指令取回爆米花零食。
代码可在 Github 上与其他示例一起获取。
解锁时间智能以实现稳健的任务完成
机器人技术中最困难的挑战之一是知道任务何时完成。Gemini Robotics ER 2 在视频理解和进度跟踪方面带来了阶跃式改进，以验证复杂任务——例如拧紧灯泡或系紧垃圾袋——在切换到下一个任务之前是否按规格完成。
在此次更新中，我们在任务进度理解的两项基础能力上取得了进展：进度分类和时刻定位。
连续进度分类
进度分类是指机器人跟踪任务完成进度的能力。在我们的评估中，我们将视频流中的每一帧分配到五个进度级别（0-20%、20-40%、40-60%、60-80%、80-100%）。通过量化任务进度，Gemini Robotics ER 2 为机器人提供实时态势感知，使其能够即时调整动作或重试失败的步骤，而无需重新启动整个工作流程。
Gemini Robotics ER 2 在进度分类任务上达到 57.4% 的准确率，优于前代模型和竞争性前沿模型。
精确时刻定位
时刻定位衡量模型识别关键事件发生的精确视频帧的能力（例如何时停止向杯中倒咖啡）。Gemini Robotics ER 2 在时刻定位方面取得了显著性能提升，使机器人能够精确地在任务之间切换、验证成功并提出纠正建议。
对于时刻定位任务，Gemini Robotics ER 2 达到 91.3% 的准确率和 0.96 秒的平均绝对距离。它与更大的模型类别竞争激烈，但以极低的计算成本和 4 倍的执行速度提供这种精度——这是在现实世界中安全操作物理机器人所需的亚秒级延迟。
多机器人协作
没有单一机器人适合所有任务——轮式漫游车在室内表现出色，而人形机器人可能在崎岖地形上表现优异。Gemini Robotics 2 实现了多机器人协作，使不同机器能够通过共享语义理解进行通信，以交接并完成复杂任务。查看 Gemini Robotics ER 2 如何使 Apptronik 的 Apollo 2 和 Franka F3 Duo 协作 此处。
提升通用空间智能
Gemini Robotics ER 2 通过以下三项基准测试，推进了我们的核心空间推理能力：
- 成功/失败检测： 现在可基于原始视频流而非静态快照运行，以捕捉执行中途的失败情况，如溢出、滑落或错位。
- 通用仪表读数： 应用范围超越圆形表盘和视镜，扩展至数字显示屏、线性刻度、直尺和液体温度计。我们已在10种不同类型的仪表上进行了测试。
- 增强的空间VQA： 通过Gemini在多模态理解方面的进步，改进了视觉问答能力。
Gemini Robotics ER 2 在所有核心能力上均持续达到最高准确率，亮点包括成功检测（图像/视频）、问答（ERQA）和通用仪表读数。
推进具身智能的安全性
Gemini Robotics ER 2 是我们最安全的模型，在安全指令遵循和人体接近基准测试中取得了显著进步，这些基准评估模型在推理任务中遵循物理约束的能力以及检测人体的空间感知能力。我们发现，Gemini Robotics ER 2 能在人员靠近时成功使人形机器人停止动作，并仅在区域清空后自主恢复工作。为了推进物理智能体的安全性，我们引入了一项基准测试，通过测试基础模型执行安全约束、监控环境、评估物理可行性以及寻求人类澄清的能力，来评估其作为安全VLA编排器的表现。详情请参阅我们的安全技术报告。
Gemini Robotics ER 2 在安全指令遵循和人体接近基准测试上优于 ER 1.6 及其他前沿模型。
展望未来，我们的计划是推动这些模型应对更复杂的任务，以加速开发有用的机器人并支持机器人社区。
Jul 30, 2026
|
7 min read
Gemini Robotics ER 2 represents a step change in powering robots with video understanding, task orchestration, and multi-robot collaboration — making it possible for robots to be more helpful in the physical world.
Steven Hansen
Senior Staff Software Engineer
Peng Xu
Staff Software Engineer
For robots to assist humans in everyday environments, accurate spatial reasoning is not enough. Robots must also think fast, timing their decisions and reasoning with the real-time speed of the physical world.
That’s why today we’re launching Gemini Robotics ER 2, our most capable “embodied reasoning” model for robotics. Think of Gemini Robotics ER 2 as a high-level brain for robots. It allows robots to chat with humans, understand the physical world, and plan multi-step tasks. It then hands off motor execution to any given lower level vision-language-action (VLA) model. Gemini Robotics ER 2 can also natively call tools like Google Search to find information, or any other user-defined function. The design of Gemini Robotics ER 2 allows the robot to “think” about what comes next while simultaneously performing its actions.
Gemini Robotics ER 2 represents a significant upgrade over Gemini Robotics ER 1.6. By watching continuous video feeds, robots can now track their own progress, adapt if something goes wrong, and know exactly when to move on to the next step. We are also introducing multi-robot collaboration, enabling robots to work together in shared spaces and complete complex workflows a single robot could not do alone.
Gemini Robotics ER 2 is now publicly available to developers via the Gemini API, Google AI Studio, and in private preview on Gemini Enterprise Agent Platform. To help you get started, we’re sharing examples of how to configure the model and prompt it to power more useful physical AI tasks.
Advancing physical agentic capabilities
Most tasks in the physical world are complex and require multiple steps to complete. Gemini Robotics ER 2 is a physical agent, orchestrating steps for the robot and enabling it to self-correct, and generalize to more novel situations. To build an agentic setup, developers can declare low-level control interfaces — like Vision-Language-Action (VLA) models or navigation APIs — as tools, and stream multimodal video, audio, or text directly into the model.
Gemini Robotics ER 2 improves this tool orchestration workflow. We can evaluate its performance with robots in simulation, using real-world robot control, and even pair it with a human controlling the robot remotely.
Gemini Robotics ER 2 consistently outperforms ER 1.6 for tool orchestration across three control modes: real VLA, sim VLA, and human tele-op.
In robotics, high-level reasoning depends on execution speed. Gemini Robotics ER 2 integrates into the Gemini Live API, using a bidirectional streaming endpoint optimized for latency-sensitive tasks. The result is fluid orchestration: Gemini Robotics ER 2 commands action models and robotics APIs to complete multi-step tasks without the jarring “stop-and-think” pauses.
To illustrate this, we’ve built a demo with Spot from our partners at Boston Dynamics. We use Gemini Robotics ER 2 to orchestrate Spot APIs, such as navigation and manipulator movement, creating an interactive robot that fetches objects for you.
Gemini Robotics ER 2 powered Boston Dynamic Spot fetches a popcorn snack up on a natural language command.
The code is available on Github with other examples.
Unlocking temporal intelligence for robust task completion
One of robotics’ hardest challenges is knowing when a task is done. Gemini Robotics ER 2 brings a step-change in video understanding and progress tracking to verify that complex tasks — such as tightening a light bulb or tying a trash bag — are complete to specification before switching to the next task.
In this update, we’ve made progress on two foundational capabilities for task progress understanding: progress classification and moment finding.
Continuous progress classification
Progress classification refers to a robot’s ability to track progress towards task completion. In our evaluations, we assign each frame in a video feed into five levels of progress (0-20%, 20-40%, 40-60%, 60-80%, 80-100%). By quantifying task progress, Gemini Robotics ER 2 provides robots with real-time situational awareness, and allows them to adjust actions on the fly or retry failed steps without restarting an entire workflow.
Gemini Robotics ER 2 achieves 57.4% accuracy on progress classification tasks, outperforming previous generation models and competing frontier models.
Precision moment-finding
Moment-finding measures a model's ability to identify the exact video frame where a critical event takes place (i.e. when to stop pouring coffee into a cup). Gemini Robotics ER 2 achieves significant gains in performance on moment finding, enabling robots to precisely switch between tasks, verify success and suggest corrections.
For moment-finding tasks, Gemini Robotics ER 2 achieves 91.3% accuracy and a 0.96s mean absolute distance. It competes closely with much larger model categories, but delivers this precision at a fraction of the compute cost and 4x the execution speed—the sub-second latency actually required to safely operate physical robotics in the real world.
Multi-robot collaboration
No single robot fits every task — a wheeled rover excels indoors, while a humanoid robot may excel at uneven terrain. Gemini Robotics 2 enables multi-robot collaboration, allowing diverse machines to communicate via a shared semantic understanding to handoff and complete complex tasks. See how Gemini Robotics ER 2 enables Apptronik’s Apollo 2 and Franka F3 Duo to collaborate here.
Improving general spatial intelligence
Gemini Robotics ER 2 advances our core spatial reasoning capability, as measured by three benchmarks:
- Success/failure detection: Now operates on raw video feeds rather than static snapshots to catch mid-execution failures like spills, slips, or misalignments.
- General instrument reading: Extends beyond circular dials and sight glasses to include digital displays, linear scales, rulers, and liquid thermometers. We tested it across 10 different types of instruments.
- Enhanced spatial VQA: Improves Visual Question Answering throughGemini’s advancements in multi-modal understanding.
Gemini Robotics ER 2 consistently achieves the highest accuracy across all core capabilities, with highlights including success detection (image/video), Question Answering (ERQA), and generalized instrument reading.
Advancing safety for embodied intelligence
Gemini Robotics ER 2 is our safest model, achieving significant gains on Safety Instruction Following and Human Proximity benchmarks, which evaluate how a model adheres to physical constraints during reasoning tasks and spatial awareness for detecting humans. We found that Gemini Robotics ER 2 successfully halts a humanoid robot when a person is nearby and autonomously resumes work only once the area is clear. To advance safety for physical agents, we’re introducing a benchmark that evaluates a foundation model's ability to act as a safe VLA orchestrator by testing its capacity to enforce safety constraints, monitor the environment, assess physical feasibility, and seek human clarification. For details, see our safety technical report.
Gemini Robotics ER 2 outperforms ER 1.6 and other frontier models on Safety Instruction Following and Human Proximity benchmarks.
Looking ahead, our plans are to push these models towards even more complex tasks to accelerate the development of helpful robots and support the robotics community.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文