A current trend in robotics hands control to generative AI models, but those architectures are less predictable than traditional algorithms, creating a challenge: how can developers and deployers be confident a new humanoid robot will behave safely?
Dr. Ding Zhao, director of the Safe AI lab at Carnegie Mellon University, has focused on this problem for most of his career. Together with veteran startup executive Kyle Wong and machine learning engineer Simo Rachidi, he has founded Safeworld to address that challenge.
"The safety challenge that we’re talking about is a combination of, one, really advanced generative AI probabilistic evals — how do you underwrite the risk of a probabilistic system?" Zhao says. "The second part that’s really hard is the trust part, and you need both to deploy a robot."
Safeworld is emerging from stealth today with a seed round of more than $12 million led by Shine Capital and a16z Speedrun, with additional investment from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors and SV Angel.
"The time to build an industry safety standard is now while robots are being designed and deployed," Jonathan Lai, partner at a16z Speedrun, told TechCrunch. "By the time you have robots in households colliding with kids and causing safety incidents, that’s way too late."
Simulations populated with realistic human models
Safeworld’s core offering is to evaluate a robot control system inside simulations populated with realistic human models. This challenge resembles what autonomous vehicle companies face — ensuring systems respond correctly to a wide range of surprising events — but Zhao argues it is tougher for robots because they operate in unstructured environments and each facility may impose different safety requirements.
"One of the most common areas is if there is a blind corner in this particular factory," Wong says. "What is the speed or what is the stopping distance that you need to make sure that this robot will not collide with a particular human? If a human is carrying boxes, for example, will the robot detect the human or not?"
To answer such questions, Safeworld creates a digital replica of that corner in a modeling environment like Genesis or MuJoCo, inserts a simulation of the robot driven by its real software, and runs thousands of scenarios in which human models encounter the robot. Zhao notes this is harder than it sounds because human behavior is unpredictable.
"Tripping and falling is also a good example of something that we do a lot of testing with the simulation," Wong says. "Otherwise, you would have to go and trip and fall for the robot, which is like a hard thing to be doing all the time."
Third-party validation and sharing safety cases
There are clear similarities between Safeworld’s platform and tools used internally by robot makers. The founders believe, however, that manufacturers will want a third-party validator — partly to enable sharing of safety cases and lessons among competitors.
"A lot of people are underestimating one how hard some of these edge cases are going to be to solve," Zhao says. "It is not the robot in the vacuum, in the demo, that we are worried about. It is the robot that is deployed at scale, with people who potentially never operated a robot before."
Industry partner perspective: empirical validation
Vishal Dugar, CTO of Gritt Robotics, whose company builds the AI brain for robots that currently help install photovoltaic panels at industrial-scale solar farms and aim to tackle more complex construction tasks, is partnering with Safeworld on safety simulations.
"The difficulty with most of our systems is it’s very hard to formally prove it by doing some math, writing some equations, and saying yeah, the system is verified to be safe," Dugar says. "It necessarily has to be done empirically."
Gritt’s robots operate alongside human workers, so verifying that a robotic arm will not hit people requires considering many potential scenarios: different body configurations, kneeling, standing, tripping, crouching, running, and a wide range of human appearances and clothing, Dugar notes.
"Humans have many kinds of appearances," he says. "Their bodies can be in different configurations. They could be kneeling, standing. They could be tripping and falling potentially. They could be crouching. They could be running. You have to respond to all these behaviors that humans could potentially exhibit on these sites, along with the variety of variations in human appearance, you know, clothes, size, shape, height, skin color, everything else."
Early stage and business model questions
Safeworld and generative-AI robotics are still in early stages. The company is determining whether to offer a platform for external users or to operate via a services-based model. The founding team is optimistic they are addressing an essential problem for any organization that wants to deploy humanoid robots at scale.
"We’ll probably be the first profitable company in this field," Zhao says. "Because if anyone wants to deploy, they need to pay us to handle the situation."
(Short recap: Safeworld, led by Dr. Ding Zhao with co-founders Kyle Wong and Simo Rachidi, launched publicly with a $12M+ seed round and is building simulation-based, human-populated safety evaluations for generative-AI-driven robots, collaborating with companies such as Gritt Robotics.)
