Tesla Hardware 3.0 Uses Custom AI Chips to Power Self-Driving

For years it was rumored that Tesla might be building their own AI hardware to power their self-driving technology after hiring several prominent chip architects, including Jim Keller from AMD (who has since left). Finally, after the Q3 2018 earnings call, we had confirmation that Tesla will indeed be launching their own AI hardware starting in 2019, called Hardware 3.0.

Current Hardware Powered by NVIDIA

Currently, the latest generation of Tesla vehicles, including the Model 3, S and X all use NVIDIA hardware to power the neural network used for Enhanced Autopilot self-driving features, specifically the NVIDIA DRIVE PX 2 AI computing platform. This was introduced in late 2016 and coined, ‘Hardware 2.0’ or AP2 (there was a small update since then that some call ‘Hardware 2.5’, but Tesla disputes its significance). Prior to AP2, the original ‘Hardware 1.0’ AP1 cars with first-generation Autopilot capabilities utilized the Mobileye EyeQ3 platform. Learn more about Autopilot features in this article.

Swappable Computers

Tesla wisely made the computing controller swappable for future upgrades since the chip technology is rapidly advancing. Tesla’s new custom AI chips are made specifically to run its software and particular neural network configuration. The memory and processing capabilities are optimized in the chip itself rather than needing work through a CPU to GPU connection as is the case with the current NVIDIA setup. As Elon Musk said, it’s essentially a “matrix multiplier” with local memory.

“Accelerated Mathematical Engine”

Patents filed by Tesla around an “Accelerated Mathematical Engine” say that the hardware required to power self-driving systems require “high-computational-throughput systems and methods that can perform matrix mathematical operations quickly and efficiently.” This is important to ensure the neural network used by Tesla to perform self driving, called a convolutional neural network (CNN), can run efficiently.

Hardware 3.0 improves the processing speed by 10 times, from 200 frames per second to 2,000 frames per second from the car’s onboard cameras. This is critical as Tesla has opted to use a vision-based camera system rather than lidar (see lidar vs cameras) to power its self-driving system. While cameras are cheaper to deploy and easier to integrate in the vehicle, they require much more processing power to understand the images and environment, which is one of the reasons Tesla had to create their own custom AI hardware. Tesla is pushing the boundaries of AI and self-driving and some consider it more of a software company than a car company in that regard, so to move quickly, in many cases they need to build versus buy technology.

If you want to have a complex neural network, you need to have a combination of software and hardware. And your software needs to be that much better in order to compensate for hardware’s (limitations). Sort of like, you have video games and how they’ve progressed — it’s a combination of software and hardware. No amount of clever software could produce a video game on old hardware that you have today. It doesn’t matter, you know. It’s the same thing with neural nets.

Elon also elaborated in that same interview:

So right now, we can process on the order of 100 frames a second and we really need to do a lot of work in terms of cropping the frames, and sort of bending the pixels, and not going to full resolution on all cameras, that kind of thing with the current hardware. We’re at full frames, full resolution with the Tesla hardware. All cameras, at full resolution, full frames, and it still hasn’t tapped out.

Same Cost to Produce

Tesla said that the cost of the new hardware is the same as the previous version despite the huge processing gains. Once Hardware 3.0 comes arrives in mid-2019, owners of older models can swap out for new computers if they have paid for the Full Self-Driving option. It’s a relatively simple process to update the boards as all the connectors are the same.

Secondary Radar?

The only potentially significant change according to leaked computer diagrams of Hardware 3 is the addition of a “secondary” radar input, perhaps a rear-facing radar in future vehicles.

Our Take

We’re excited to see Tesla continuing to innovate and develop novel software and hardware solution in order to bring the vision of true Full Self-Driving to consumers. 2019 should be an exciting year for Tesla self-driving capabilities!