Video games save lives

computing

Video games save lives

Hardware and software technologies developed to destroy animated aliens in computer games are protecting drivers and speeding automotive development.

Robert Schoenberger

Every time a car automatically hits its brakes because its driver failed to notice an obstruction, thank queasy computer gamers.

The technology underpinning advanced driver assistance systems (ADAS) and autonomous cars got its start solving a much more pedestrian problem – giving gamers fast-moving images of three-dimensional (3D) environments without making them vomit.

In 1992, id Software launched Wolfenstein 3D, a game that let players sneak through a German castle in World War II and fight Nazi soldiers. The game was an instant success, but many players struggled with the first-person view of the 3D environment – getting nauseated almost immediately.

The problem was frame rates – computers couldn’t render 3D environments fast enough for the human eye. Even the best processors of the early 1990s weren’t up to the task, resulting in jerky, stuttering images. Dozens of companies formed to develop new ways for computers to create virtual, 3D environments. Nvidia, one of the most successful of those startups and a company that has since become a major supplier to the automotive industry, formed in 1993.

Nvidia’s Drive PX Pegasus computing platform is a graphics processing system designed to support autonomous driving.

Graphics processing

Graphics processing unit (GPU) chips that began popping up in the late 1990s took virtual 3D environments and translated them onto two-dimensional (2D) computer screens (see sidebar, right). ADAS and autonomous driving systems face the opposite problem – taking 2D images from cameras and other sensors to create a 3D computer model that tells cars what obstacles surround them.

“We essentially have to recreate the 3D world around us as we drive. So, sensors, the cameras, the radar, and the lidar help us build an understanding of that environment,” says Daniel Shapiro, senior director of automotive at Nvidia. “Inside the brain of the car, it starts to look like a 3D video game. It’s the same type of math from video games taking place inside the car. Except now, it’s mission critical.”

ADAS and autonomous systems use 3D data as well, such as radar and laser scanning, but the bulk of data comes in two dimensions.

Nvidia chips began appearing in automotive infotainment systems more than a decade ago. Any system with a screen benefits from graphics acceleration, so those early automotive systems were basic image accelerators, not the more-sophisticated systems used today to manage ADAS.

“Early on, our customers taught us how to be automotive grade,” Shaprio says. “The design processes, manufacturing processes, the quality assurances processes are all proven in the automotive industry.”

Early this year, the chip maker announced plans to focus more on autonomous and ADAS technology, saying its automotive growth would come from advanced computing, not colorful touch screens. In July, the company partnered with German automotive supplier Bosch and Mercedes-Benz owner Daimler to develop a self-driving taxi fleet with testing set for California next year.

While the Nvidia Drive systems the company provides to automakers are designed for harsh environments – servers running in data centers don’t experience the heat and vibration the typical car does – the underlying chip structure is nearly identical to the ones that power Nintendo’s Switch game console.

GPU vs. CPU

Central processing units (CPUs) are the main processors for computers. Powerful, multitasking chips run the operating system, manage data storage, and control peripheral equipment. They process data serially – four-to-eight computing cores on each chip complete one task at a time before moving to the next.

Graphics processing units (GPUs) are lower-powered chips that work in parallel – thousands of computing cores work on simple math problems simultaneously.

Computers see three-dimensional (3D) shapes as a series of polygons – simple two-dimensional (2D) shapes stacked on top of each other. GPUs can interpret all polygons at once instead of stitching the image together, one 2D shape at a time.

As one researcher put it, because of the high speed of its processor, a CPU could multiply three complex numbers faster than a GPU, then move on to its next task. However, if you want to multiply three complex numbers 1 million times, the GPU’s parallel nature would be faster.

As with home computers, cars will need CPUs and GPUs – multitasking CPUs to handle the various computer-controlled systems in the vehicle, and single- purpose GPUs to collect huge amounts of sensor data to quickly render into a 3D image of the environment surrounding the vehicle.

Teaching cars to drive

The convergence of video game graphics processing and autonomous driving isn’t limited to the chips; software tools designed to simulate spectacular car crashes when vehicles hit each other in games are also teaching self-driving cars to avoid such collisions.

“Companies developing these autonomous technologies have to train the systems to react to real-world problems – a child running into the street to chase a ball,” Heermann says. “It’s a huge application for the technology. They can test scenarios quickly, change variables, and test again to see how systems react.”

Shapiro adds that video-game derived graphics technologies are getting more sophisticated every year – modeling how shadows change depending on cloud cover and time of day. Bizarre scenarios – such as light glinting off a window into a car’s ADAS camera, creating a confused image – can be tested because video game designers wanted to create more immersive experiences for players.

“We can then put those simulations through our Nvidia Drive system to see how it responds in real time,” Shapiro says. “We can simulate 60,000 miles of driving in one hour. That gives us the ability to drive every road in the U.S. in two days.”

Design tools

Using GPUs to power safety systems is the most direct way video games have influenced advanced vehicles, but the industries have other ties – one of the biggest being they both use many of the same software.

Computer-aided design (CAD) systems predate modern 3D video games by decades, so action-oriented games did not lead directly to industrial design systems. However, gaming technology has greatly influenced and advanced the tools that designers use to imagine vehicles and every component used in them, Heermann says.

“Making visual decisions is super important. Designers want to have as much realism in the decision making as possible. Tools like VRED, our visualization tool, are pushing the limits on representation, especially in real time,” Heermann adds.

Decades ago, CAD data could only be depicted as a series of geometric shapes – useful for determining specifications and basic characteristics, but not great for evaluating the look or feel of a product. Shading, coloring, and texturing tools developed for computer games and animated movies create human skin that looks like skin and clothes that move in the breeze. Motor vehicle designers are using those tools to more fully simulate how a vehicle will look at different times of day or in different weather conditions.

“You’ll have a complete car with lots of variation,” Heermann explains. “The goal is to look like the real car on the street, with the ray tracing (for lighting effects), with ambient shadows on it. That’s very similar to what you’re seeing from games’ engines.”

Speeding development

While clay models are still common, many design studios first create vehicle concepts in digital environments. Some are even using virtual reality (VR) headsets to walk around the 3D models and view them from different angles. In late July, Autodesk announced a partnership with Prague, Czech Republic-based VRgineers Inc. to make Autodesk VRED data viewable on VR headsets. Heermann says multiple designers in studios scattered around the world will be able to examine and alter the same 3D model in a virtual space, collaborating in real time – much in the same way Fortnite and Call of Duty players team up in games.

“The same interactivity from games, designers want to have that in the studio. Realism is very important,” Heermann says. “The big difference is speed. On the game, you’re optimizing your representation for speed. No one’s really judging the car in a game for its perfection. The auto guys want to see everything in as much detail as possible.”

Because of increased competition, automakers are releasing more versions of vehicles, faster than ever before. Historically, it could take as many as seven years to go from initial sketch to a vehicle on the road. Today, launches can happen in as few as three years. CAD tools are responsible for much of that increase, but Heermann says better visualization and design tools, coupled with the ability to collaborate across continents, are playing a big role in getting products to market faster.

German automotive supplier Bosch has designed a mobile computing system for self-driving vehicles, powered by Nvidia graphics chips.

Virtual designs become real-world vehicles

Sony Interactive’s Gran Turismo racing games are so popular that several automakers have designed vehicles strictly for the game. Well, that was the concept at least. The virtual cars became fan favorites, prompting several automakers to create physical models of vehicles that had been intended only for the screen.

In 2012, Sony invited companies to design their ultimate street-racing vehicles for the game, and several responded, including Honda, Peugeot, McLaren, Dodge, Hyundai, Chevrolet, Mazda, and Lexus. Mockups of those cars are common at auto shows and consumer electronics exhibitions.

Audi Chief Designer Marc Lichte says, “Although the design of a virtual vehicle allows much greater freedom and the creation of concepts which are only hard to implement in reality, we did not want to put a purely fictitious concept on wheels. Our aim was a fully functional car.”

The electric racecar features three 200kW motors – two for the rear axle, one for the front – for a combined 600kW (815hp) of output. Those are real-world, tested numbers, not statistics for a video game. Using parts from the upcoming e-tron electric crossover, the racecar accelerates from 0mph to 62mph (100kph) in less than 2.5 seconds.

Coming full circle

To a large degree, the video game industry’s impact on the automotive world is payback. The visual 3D rendering tools that id Software engineers used more than 25 years ago for Wolfenstein 3D owe a lot to CAD. Software companies developed CAD to help automotive engineers and other industrial designers better visualize physical objects in a digital world, and game developers adapted those wire-frame tools to create entire worlds.

It’s possible that many visualization technologies would have developed without research and development from video game hardware and software companies, but that movement of the technology from the industrial world to the consumer market and back again has certainly accelerated development.

So, next time someone complains about video games rotting players’ brains or that violent games contribute nothing to society, think about cars avoiding accidents – all because a handful of gamers got sick to their stomachs 30 years ago.

About the author: Robert Schoenberger is the editor of TMV and can be reachedvat 216.393.0271 or rschoenberger@gie.net.

Augmented reality

As the automotive and video game markets continue to develop, augmented reality (AR) could become another technology used by both. Apple and Google, for iOS and Android respectively, have released tools that allow app developers to superimpose digital images or data onto real pictures.

For example, an app game developed for Jurassic World: Fallen Kingdom allows players to superimpose dinosaurs onto photos and videos shot on the phones. In the industrial world, developers have AR tools that project technical images on items being assembled – guiding employees to make the correct connections on wiring harnesses for example.

Autodesk Senior Director of Automotive and Conceptual Design Thomas Heermann says he expects automakers to use AR for sales soon – allowing prospective buyers to see vehicles in their driveways and pick color schemes that match their houses.