Waymo says it has created a better benchmark for comparing robotaxis to humans

Waymo has created a new computer model designed to more accurately answer an important question: how does its autonomous driving software interact with people?

Alphabet subsidiary robotaxi, which developed a computer model of human driving skills in collaboration with TU Delft, published a research paper about it in Nature Communications on Wednesday.

Waymo said it expects the new model to be more accurate than the previous version it has been using for the past few years. The new model is built using a framework called active inference — the theory that the driver is always thinking about possible futures and taking steps to reach the safest, most predictable one.

Waymo said the new model will help it better understand how people behave in the crash situations they encounter with the robot.

“For decades, the auto industry has used physical and virtual crash dummies to test a vehicle’s safety features, including its hardware and structural integrity,” Waymo wrote in a blog post Wednesday. The new model, Waymo said, “changes this concept, serving as a behavioral benchmark for autonomous driving systems that can realistically represent the expectations of how a careful and skilled human driver reacts to road conflicts.”

A more accurate model of human driving behavior is table stakes for autonomous vehicle companies that need to understand and measure the performance of their robotaxis in accidents. And it comes at a critical moment for Waymo, which is ramping up in more cities and facing increased scrutiny from regulators and the public.

In January, when a Waymo robot hit a child near a school in Santa Monica, California, the company relied on its earlier computer model to say that a careful human driver would have made the impact at about 14 miles per hour. A Waymo robot hit a child at 6 miles per hour, after slowing down from 17 miles per hour, and the company said it suffered minor injuries. (The accident is still under investigation by the National Highway Traffic Safety Administration and the National Transportation Safety Board.)

The main difference between this new model – which Waymo calls the Reference Driver – and its predecessor is that it is able to reproduce the behavior of a human driver during an accident. Previously, Waymo’s models (as well as other models in the industry) focused on replicating a person’s “secondary, active” ways, according to the company.

The Reference Driver, on the other hand, “can measure the internal ‘surprise’ the driver feels during a collision, providing a human-like benchmark for autonomous driving systems that were previously impossible to change at scale,” Arkady Zgonnikov, assistant professor at TU Delft, said in a statement.

Waymo says this new driver model can be adapted to match “many types of road user behavior beyond collision avoidance,” and is better equipped for use in “larger test sets with thousands of scenarios.”

“The model can represent and test multiple complex, real-world crashes in a virtual environment, indicating performance improvements at an unprecedented speed,” the company wrote.

Waymo wants others to collaborate on driving the Reference Driver, too. The company said Wednesday it made the model’s research code available under an academic, non-commercial license that allows it to be used for research, teaching, personal testing and scientific publication.