READ NEXT

Google's new algorithm edits your photos in the blink of an eye

ByElizabeth Stinson

It is very common for objects to be resized or have their positions within the frame altered slightly within the 2D image plane, but the new tool will let people turn or flip objects, which will show bits of them that weren't even captured by the camera.

The secret to this is that the software uses publicly available 3D models of objects to inform the editing software how to complete the geometry and the parts of the object not on show. By studying the structure and symmetry of an object, the software can fill in the blanks to recreate the object in its entirety -- or at least make a best guess and what it would be like. "In the real world, we're used to handling objects -- lifting them, turning them around or knocking them over," said Natasha Kholgade, from Carnegie Mellon Robotics Institute and lead author of the paper on the development of the tool. "We've created an environment that gives you that same freedom when editing a photo."

Originally the system was designed to work using digital imagery, but the researchers have since found that it can also be used to manipulate images within historical photos and even paintings. They have also discovered that it is possible to use this technique to animate photos and have created an example in which an origami bird takes off, turns around and flies off down a corridor.

The main weakness of the method using publicly available models is that most of the models inevitably don't quite match the photo exactly. Variations are often caused inside the photos by ageing, weathering or lighting. The researchers did develop a technique that means the model is semi-automatically aligned with the geometry of the object in question. This allows the software to estimate the environmental illumination on the parts of the object that cannot be seen in the photo and replicate them.

ADVERTISEMENT

While there are not models available for every object online at the moment, the selection is only getting better and better thanks to 3D scanning becoming more widespread.

The software will be shown off at the Siggraph graphics conference in Vancouver next week.