Researchers at Stanford University and UC Berkeley have introduced RoboReward, a family of general-purpose vision–language reward models designed to evaluate progress on a wide range of robot tasks. The work was carried out by Tony Lee, Andrew Wagenmaker, Karl Pertsch, and colleagues.
What’s new
RoboReward comes in two sizes: 4 billion and 8 billion parameters. The authors also released a dataset and a benchmark intended to train and evaluate vision–language reward models for robotics.
Key insight: generating validated negative and partial examples
The team observed that common text–video datasets of robot actions predominantly contain successful executions, which makes it difficult for models to learn to distinguish success from failure or partial progress. To address this, they augmented successful demonstrations with carefully generated negative and partial examples. For instance, given a video of a robot placing a spoon in a pot, they could relabel the command to “put the spoon by the pot” to produce a failure example, or truncate the successful video to simulate a partial attempt.
Dataset construction and model training
Each example in their dataset consisted of (1) a command, (2) a video showing a robot attempting the command, and (3) a progress score from 1 (failed) to 5 (completed). Videos were collected from two datasets that include single-arm, dual-arm, and humanoid robots. Task descriptions were standardized, and the data were augmented with negative examples and assigned progress scores.
To produce negative examples, the authors used GPT-5 mini to describe, from a successful video, the scene, the robot’s actions, and the final state. Based on that analysis, Qwen3-4B-Instruct-2507 suggested alternative commands that would make the same video warrant a progress score below 5. GPT-5 mini then inspected those alternatives and discarded ones whose labels did not match the videos. Partial-progress examples were generated by truncating videos of successful executions. GPT-5 mini also assigned progress scores in the 1–4 range to these generated examples.
For model training, the authors fine-tuned Qwen3-VL 4B and Qwen3-VL 8B variants to predict the progress score given a video and a command; this predicted progress score served as the reward signal. They manually verified 2,831 examples to produce a test set called RoboRewardBench.
Results
RoboReward models predicted rewards on RoboRewardBench more accurately than several baseline models, including Gemini Robotics-ER 1.5 and generalist models such as GPT-5.
Performance was measured by mean absolute error (MAE) between model predictions and reference labels (lower is better). RoboReward 8B achieved a MAE of 0.665, outperforming 21 other models. For comparison: GPT-5 mini reached 0.691 MAE, GPT-5 0.811 MAE, and Gemini Robotics-ER 1.5 0.906 MAE. RoboReward 4B obtained 0.845 MAE, placing fourth and surpassing models such as Gemini 3 Pro (0.851 MAE).
The authors also ran real-world robot demonstrations. They trained a diffusion transformer to manipulate a WidowX robot arm using rewards from different models and evaluated task success rates on two example tasks:
- Picking up a toy and placing it on a towel: the model trained with RoboReward 8B rewards had a 50% success rate, the model using Gemini Robotics-ER 1.5 rewards had 10%, and the model trained with human-assigned rewards achieved 75%.
- Opening a drawer: RoboReward 8B-based training yielded 80% success, Gemini Robotics-ER 1.5-based training 45%, and human-assigned rewards 90%.
These results show that RoboReward 8B substantially outperformed prior automated reward models in these real tasks, although human-assigned rewards still led to higher performance.
Why it matters
Vision–language reward models are a promising approach to supervising robot learning because they can generalize across tasks and robot platforms and reduce the need for bespoke, hand-crafted reward functions. By augmenting successful demonstrations with validated failures and partial attempts, the authors trained a general-purpose reward model that better distinguishes degrees of task completion.
Publishing the dataset, benchmark, and pretrained reward models invites the research community to improve reward functions directly, rather than relying on such capabilities to emerge incidentally from larger generalist models.
Mentioned models and resources
- RoboReward (4B and 8B)
- RoboRewardBench (2,831 manually verified test examples)
- Qwen3-VL 4B and Qwen3-VL 8B
- Qwen3-4B-Instruct-2507
- GPT-5 mini and GPT-5
- Gemini Robotics-ER 1.5 and Gemini 3 Pro
- WidowX robot arm
