Google and MIT Demonstrate Reflection Removal Technology from Images

Researchers from MIT in cooperation with Google demonstrated a technique for removing unwanted reflections and obstructions from photos using series of images captured on a mobile device.

The new techniqe which uses parallax effect and is demonstrated in the video above (as well as in a technical article – see link below) works on a series of images (basically a video which is captured in a way similar to the way panorama images are captured by many modern cameras) and using an algorithm searches for edges in the scene, to determine which objects are close and which objects are far away from the camera. In this way they can separate the two and either remove or display each of them (in the demo video you can see an image free of the reflections or an image with only the reflections).

The same technique can be applied to scenes with fences, windows with water on them and many other obstructions which are in our way when trying to shoot an image. In case you were wondering, most of the images for this project were captured using HTC One M8 and Galaxy 4 and rendered using an 8 core XEON system with 64GB memory (although the actual memory acquirement was much lower).

Currently it is unclear when the technology will be available for the general public but given Google’s involvement – we are hoping that pretty soon (although the computational power needed – especially if you are looking at full size images – isn’t small). We will also be looking for a demo of a video rendered in HD using this technology in the future.

If you want to learn more about the new research (and you are O.K. with lots of technical details) you can find it here (PDF).

You can find more photography related technology videos on our photo-tech section here on LensVid.