How simulation turns one flashing yellow light into thousands of hours of experience

Each day, as many as 25,000 Waymo self-driving cars drive 8 million miles in our virtual world, testing out new skills and refining old ones.

By James Stout, lead software engineer

At the corner of South Longmore Street and West Southern Avenue in Mesa, Arizona, there’s a flashing yellow arrow that permits cars to turn left. Navigating this type of intersection can be tricky for humans and self-driving cars alike — drivers must carefully move into a five-lane intersection and then find a gap in oncoming traffic. Turning left too soon may cause a driving hazard for oncoming traffic; making the move too late may mean frustrated drivers behind.

While this type of traffic signal is rare in our hometown of Mountain View, CA, it’s become a common sight at intersections across Metro Phoenix, where we recently launched our early rider program. Since we first encountered this intersection, we’ve been teaching our self-driving cars to handle these types of traffic signals smoothly and confidently — just as an experienced driver would.

Just like for human drivers, the key to learning is practice. That’s where our simulator comes in. Waymo’s simulator is a realistic virtual world where we can recreate every real-world mile we’ve driven. Each day, as many as 25,000 virtual Waymo self-driving cars drive up to 8 million miles in simulation, testing out new skills and refining old ones. Like athletes visualizing the playing field, our virtual cars envision various scenarios and practice maneuvers that help them safely navigate the real world.

With simulation, we can turn a single real-world encounter — such as a flashing yellow left turn — into thousands of opportunities to practice and master a skill. Here’s how it works:

Our first step is to make our virtual world super-realistic. Using a powerful suite of custom-built sensors, we can build a virtual replica of this complicated left turn in Mesa, complete with identical dimensions, lanes, curbs, and traffic lights. One of the key advantages of simulation is that you can focus on the most interesting interactions — flashing yellow signals, wrong-way drivers, or nimble pedestrians and cyclists — rather than monotonous highway miles.

We can recreate a highly-detailed, realistic virtual version of the East Valley

Step 2: Drive, drive, and redrive.

With this flashing yellow left turn now digitized in our virtual world, our software can practice this scenario thousands of times over. Every time we tweak the software, we can instantaneously test the change at the same intersection in identical driving conditions. That’s how we were able to teach our cars to naturally inch forward at that flashing yellow light, and slot in after oncoming traffic. What’s more, in simulation we can practice this new skill on every flashing yellow arrow we’ve ever come across (at last count there’s more than 60 of these intersections in Chandler, AZ alone). This means our software can get smarter, faster.

In simulation, we can practice driving the same intersection, in the same driving conditions thousands of times

Step 3: Create thousands of variations.

Next, we can take this one tricky left turn and multiply it to explore thousands of variable scenarios and “what ifs?” Through a process called fuzzing, we can alter the speed of oncoming cars and the timing of traffic lights to make sure our vehicles can still find the right gap in traffic to make a safe and smooth turn. We can also make the street scene busier and more complex by adding vehicles, pedestrians and cyclists that never existed in the original scene. With fuzzing, we can even simulate motorcycles “splitting the lane,” or joggers zig-zagging across the street to see how that might change our driving on the road.

To make a scene more complex, we can add vehicles, pedestrians and cyclists that never existed in the original scene

Through a process called fuzzing, we can alter the speed, trajectory and position of objects on these virtual streets

Step 4: Verify, validate, iterate.

Success: our self-driving car has learned how to confidently turn at a flashing yellow arrow. That new skill then becomes part of our car’s permanent knowledge base, and will be shared with every vehicle across the fleet. In turn, we’ll use real-world driving and our private test track to verify and validate our experience in simulation. And then the cycle begins again.

Success: our self-driving cars inch smoothly into the intersection and turn left at the flashing yellow light

Today the vast majority of improvements and refinements we make to our software are based on things we pick up in simulation. Waymo cars drove over 2.5 billion simulated miles in 2016 — miles far richer and more densely packed with interesting scenarios than the average mile of driving.

Every mile brings us closer to our goal of putting self-driving cars on the road, helping people, and saving lives. And each of these eventful simulator miles are guiding us to what everyone wants: billions of serenely uneventful miles in the real world.