Skip to main content

Waymo rules and Apple trails in California self-driving car benchmarks

Before they are let loose on public streets and highways, autonomous vehicles will need to prove their capability and safety. Absent a universal measure of self-driving technology performance, autonomous mode disengagement is the current benchmark. In self-driving operations, disengagement refers to any event when a human driver turns off (disengages) a vehicle’s autonomous mode to take control of the vehicle.

Each year the State of California Department of Motor Vehicles releases reports of self-driving car disengagements for all companies that tested autonomous vehicles on public roads. This year’s reports from 48 companies are included in the Autonomous Vehicle Disengagement Reports 2018.

Recommended Videos

Based on the most recent California data, Waymo had the lowest disengagement rate — the best performance — of all companies that reported. GM Cruise had the second best performance. Apple self-driving program, however, reported the highest rate of disengagements, trailing all other reporting companies.

California’s DMV doesn’t rate or grade autonomous vehicle company disengagement rates; it just reports the numbers provided. Based on those numbers, Apple’s self-driving test vehicles disengaged every 1.1 miles. GM Cruise reported disengagement every 5,204.9 miles, and Waymo more than doubled GM Cruise’s distance, driving a whopping 11,017 miles between disengagements.

The definition of disengagement is fuzzy. California’s DMV defines disengagement as “deactivation of the autonomous mode when a failure of the autonomous technology is detected or when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle,” according to The Robot Report.

Even if the definition of disengagement was airtight, however, is easy to find fault with the report’s validity when comparing companies. First off, the companies self-report, which opens questions of company interpretation, application, control, and enforcement.

Second, the reported total miles driven in each company’s testing revealed a huge range. Waymo’s test cars recorded 1,271,587 miles and GM Cruise’s autonomous test cars drove 447,621 miles in 2o18. Among other companies reporting, Mercedes-Benz vehicles drove 1,749 miles, Toyota 381 miles, Honda 168 miles, BMW 41 miles, and Telenav only 30 miles — the lowest of all companies. Comparing company technology based on hundreds of thousands and even more than a million miles with companies with just a few thousand miles or less defies logic. Apple didn’t put together a massive driving log, with 79,745 miles in 2018, the third highest miles driven, after Waymo and GM Cruise.

A quick look at the data might suggest that miles alone are the deciding factor in disengagement frequency, but that’s still not the whole story. Driver experience level, the types of roads traveled and the time of day and distance per trip could all affect the results, but the reported data lacks those levels of granularity.

Regardless of the actual validity of comparing the disengagement reports from various companies, distance traveled does affect data reliability — meaning how likely the same vehicles would perform in the same manner with more miles.

Waymo and GM Cruise continue their testing. Apple, however, always secretive about products and programs in development, recently laid off 200 employees from its mysterious Titan self-driving car program.

Bruce Brown
Bruce Brown Contributing Editor   As a Contributing Editor to the Auto teams at Digital Trends and TheManual.com, Bruce…
An Apple car won’t be coming in 2026 after all
The Apple logo is displayed at the Apple Store June 17, 2015 on Fifth Avenue in New York City

It feels like rumors about Apple building a car have been swirling forever.

We know that it’s been working on an automobile because of various leaks about the project, as well as news of auto-related hires over the years, but the company has never made any official announcements about the initiative.

Read more
Cruise’s robotaxi service suspended by California regulator
A Cruise autonomous car.

Autonomous car startup Cruise has run into trouble in California after the state’s Department of Motor Vehicles (DMV) said Tuesday it was suspending its deployment and driverless permits with immediate effect.

The dramatic intervention comes just a couple of months after General Motors-owned Cruise was given permission to operate robotaxi services around the clock, but also follows a number of troubling incidents involving self-driving Cruise cars on the streets of San Francisco, where it’s been carrying out tests on public roads in recent years.

Read more
Dubai Police to deploy driverless patrol cars with AI smarts
Dubai's autonomous patrol car.

While U.S. firms like Waymo and Cruise focus on ridesharing services with their autonomous vehicles, the United Arab Emirates' coastal city of Dubai is aiming to take the technology to another level by deploying it in police patrol cars.

Dubai Police recently announced plans to use fully electric, self-driving patrol cars in residential areas, local media reported this week.

Read more