Google unveils super-resolution ‘zoom-and-enhance’ tech

One thing every futuristic
detective show has in common is an amazing tool that reconstructs
detailed images from unbelievably small, blurry images. In real life,
various techniques have been used to restore detail to images, but so
far they have tended to work with what’s there. Some use
machine-learning techniques, but typically only to extract as much as
possible from what’s there. Now, some Google Brain researchers have gone
one step further, using a reference library of potentially similar
images to enhance the blurry image into not what it was, but what it
might have been.

That’s
where the interesting part of Google’s new algorithm comes in. It uses a
pair of neural networks (ResNet and PixelCNN) that are jointly trained
to match the low-resolution image to likely higher-resolution versions,
and then fill in details that might be suitable from those images.

On the right, you can see the original “ground
truth” images in the right column. They are down-sampled to 8 x 8 pixel
versions that you can see in the left column. Then Google’s algorithm
is run on the 8 x 8 versions, creating 32 x 32 pixel estimates, shown in
the center column.

Because there is so little detail in the down
sampled image, the system needs a powerful hint to get started. In this
case, it needs to be told that the image is a face. It also can only
restore detail when it has been fed a library of reference images
(faces, in this case) to use as needed.

No, this won’t work for CSI

You can see by looking at the test images
below how the software can create scenes that are similar to the
original, but not the same. The top four rows are the images that human
judges thought the most accurate, and the bottom four rows the ones that
scored lowest. The algorithm has been run on faces (the left two
columns) and bedrooms (the right two columns).

Since many of the reference images are of
popular stars, it’s possible that you may come out looking better than
you do in real life, but of course it won’t really be you. So this
technology hopefully won’t find its way into CSI, although maybe it can
help police sketch artists get a head start on likenesses based on poor
original images. More likely, this may become the modern equivalent of
the fun house mirrors found in arcades.