Using information about light reflections, MIT’s artificial intelligence system is able to measure the speed and trajectory of hidden objects in real time using footage from smartphone cameras.

“The technology has a range of applications, from firefighters finding people in burning buildings to self-driving cars detecting pedestrians in their blind spots,” an MIT spokesperson tells Newsweek. “What’s impressive is that this approach works using footage from a smartphone camera, such as an iPhone 8.”

“Even though those objects aren’t actually visible to the camera, we can look at how their movements affect the penumbra to determine where they are and where they’re going,” said Ph.D. graduate Katherine Bouman, lead author on the paper detailing the system.

“In this way, we show that walls and other obstructions with edges can be exploited as naturally occurring ‘cameras’ that reveal the hidden scenes beyond them,” she said.

The CornerCameras system will soon be tested on moving objects. MIT CSAIL

Bouman, who co-wrote the paper with MIT professors Fredo Durand, Bill Freeman, Antonio Torralba and Greg Wornell, will present the latest work at the International Conference on Computer Vision, in Venice later this month.

To the surprise of the research team, the CornerCameras AI system was tested in rainy weather and continued to work.

“Given that the rain was literally changing the color of the ground, I figured that there was no way we’d be able to see subtle differences in light on the order of a tenth of a percent,” Bouman said. “But because the system integrates so much information across dozens of images, the effect of the raindrops averages out, and so you can see the movement of the objects even in the middle of all that activity.”

Despite the success in poor weather, the system displayed limitations when used in low-light conditions. The researchers hope to overcome this through further research, while also adapting the system for use on moving objects.

A roof-mounted camera and radar system is shown on a self-driving car during a demonstration in Pittsburgh on September 13, 2016. Aaron Josefczyk/ REUTERS

The first step will be to test it on a wheelchair, with the eventual goal of testing it on cars and other vehicles.

“The notion to even try to achieve this is innovative in and of itself, but getting it to work in practice shows both creativity and adeptness,” said professor Marc Christensen, dean of the Lyle School of Engineering at Southern Methodist University, who was not involved in the research.

“This work is a significant step in the broader attempt to develop revolutionary imaging capabilities that are not limited to line-of-sight observation,” he said.