Pixel Visual Core: Google’s custom-made image processing chip

While Google’s Pixel phones have had very impressive scores on DxOMark’s benchmarks, it is perhaps even more impressive that they are able to accomplish that much for very little hardware. Much of the magic happens on the software side, with the use of algorithms and machine learning. Those, however, don’t just magically happen and incur a penalty on the phone’s processor. That is why Google made its own Pixel Visual Core system-on-chip specifically and only for image processing and machine learning related to that. And soon, even third party developers will have access to it.

More than just being a dedicated image processing chip, the Pixel Visual Core is special for being Google’s first ever, custom-designed SoC or system-on-chip. It still relies on the Snapdragon 835 for all other operations, but, being a general purpose application processor, there’s only so much it can do for very specific use cases.

In contrast, the Pixel Visual Core is dedicated to, and therefore optimized, for one task and one task alone: image processing, specifically HDR+ processing. The system-on-chip contains 8 Google custom-designed Image Processing Unit (IPU) cores, each with 512 arithmetic logic units (ILU). Despite all that processing power, the chip sips only very little power. This delivers a 5x improvement in speed and 1/10th energy consumption when it comes to HDR+ processing.

Curiously, the Pixel 2 already has the Visual Core chip inside it, but it’s dormant. The impressive HDR+ processing it does still happens on the Snapdragon 835. Imagine, then, the benefits of doing it in the background, with a dedicated image processor built for speed and power efficiency. That frees up the main application processor for others things, like running the camera app, while HDR+ happens in parallel, making HDR+ shots even faster and better.

That magic is coming soon. Starting in an upcoming developer preview for Android Oreo 8.1, Pixel Visual Core will become a developer option. Following that, it will be open to all third-party apps using Android’s Camera API. This means that, when that promised day does come, third party camera apps will have access to the same HDR+ processing feature and, hopefully, the same high-quality output. What’s more, since the Pixel Visual Core was designed to be a programmable chip, it can go beyond just HDR+ processing and be used for more applications. And all it will take is a single firmware update.