Google has Released the Deep Learning Model used to Create Pixel 2’s Portrait Mode

Overview

Called DeepLab-V3+, it’s built on top of a convolutional neural network

Works by assigning semantic labels and then categorizing and classifying objects

Introduction

We have covered Google’s releases quite a lot recently on AVBytes and that’s because they continue to open source their releases. They want the community to benefit from their resources.

Keeping that going, Google has now released the deep learning model it used to create the Portrait Mode in it’s signature Pixel 2 phone. It is based on semantic image segmentation.

The deep learning model assigns semantic labels to each pixel in the image. Then, categorization classifies the objects in the image, like animal, human, sky, road, etc. It basically recognises the foreground and background in that particular image.

The latter feature was used to develop the Pixel 2’s portrait mode for “shallow depth of field effects with only one physical lens”. It’s main use is in recognizing the outline of objects in the image, or being able to successfully determine where an object ends and the background begins.

The code open sourced by Google is named DeepLab -V3+. It’s an image segmentation tool built on top of a convolutional neural network (CNN).

Check out the underlying structure used by Google’s team to build the model:

This has been implemented in TensorFlow (of course, it’s Google!) and the release includes the training and evaluation code.

Our take on this

Modern image segmentation models built on top of CNNs are scaling never before seen accuracy levels. This latest release will again help the deep learning community understand and replicate (if not improve) what Google have done in the photography field.

Editor at Analytics Vidhya.
Data visualization and Six Sigma practitioner who loves reading and delving deeper into the data science and machine learning arts. Always looking for new ways to improve processes using ML and AI.