Deep Angel: The AI behind the aesthetics of absence

In this Project:

Project Contact:

Groups

Deep Angel is an artificial intelligence that erases objects from photographs. The algorithm is hosted on http://deepangel.media.mit.edu, which enables anyone to interact with the AI and explore what it can disappear.

Part philosophy, part technology, and part art, Deep Angel is designed to spark a series of conversations on technology in our daily lives and AI and media manipulation.

Deep Angel draws from Walter Benjamin's description of Paul Klee's Angelus Novus, the angel of history who has clairvoyance into the dark side of what appears to be progress. The angel sees the unravelling of all that matters in the world and would like to alert the world about his vision, but he's caught in the storm of progress and can't communicate any messages. The images that Deep Angel generates are intended to deliver the message that Angelus Novus would have sent if he could.

The algorithm applies computer vision techniques to automatically (1) detect and outline objects in images, (2) remove the outlines object from the image, and (3) imagine what the image would look like if that outlined object we… View full description

Deep Angel is an artificial intelligence that erases objects from photographs. The algorithm is hosted on http://deepangel.media.mit.edu, which enables anyone to interact with the AI and explore what it can disappear.

Part philosophy, part technology, and part art, Deep Angel is designed to spark a series of conversations on technology in our daily lives and AI and media manipulation.

Deep Angel draws from Walter Benjamin's description of Paul Klee's Angelus Novus, the angel of history who has clairvoyance into the dark side of what appears to be progress. The angel sees the unravelling of all that matters in the world and would like to alert the world about his vision, but he's caught in the storm of progress and can't communicate any messages. The images that Deep Angel generates are intended to deliver the message that Angelus Novus would have sent if he could.

The algorithm applies computer vision techniques to automatically (1) detect and outline objects in images, (2) remove the outlines object from the image, and (3) imagine what the image would look like if that outlined object were removed from the image. Any image uploaded and transformed by Deep Angel can be published on the Deep Angel website by clicking the "Publish to Deep Angel" button.

The AI's performance varies across photographs. Sometimes, it's impossible to tell what has been disappeared. Other times, the images appear similar to the images from Adrian Piper's Everything series. The more people interact with the algorithm, the more attuned people will be to the potential and limitations of modern AI to manipulate the media. It's now possible to automate the vanishing commissar in Soviet photography, but the AI is not yet perfect. Below are two examples of the Deep Angel AI effect: (1) a gif generated by Deep Angel showing a father and daughter disappearing in the wilderness and (2) two images showing the before and after of Deep Angel peering into a photo of a professional surfer.

As Angelus Novus must have foreseen, technology mediates our daily lives. Unfortunately, it's not always for the best. There's a corporate competition over humankind's attention. Media (social, Youtube, television) is often designed to capture our attention rather than expand our world view. In a showcase of Deep Angel's artificial intelligence with the intention of holding a mirror to technology with technology, Scalable Cooperation produced a short film, The Broken Flaneur.

Technology has the power to disappear the world in front of our eyes. The Broken Flaneur is a short film shot and produced in less than 24 hours with special effects that are enabled by a generative convolutional neural network.

From a technical perspective, Deep Angel is an end-to-end targeted object removal pipeline consisting of three interfacing neural networks: an object mask generator (G), a generative inpainter (I), and a local discriminator (D). For every input image and class label pair, Deep Angel generates an object mask using G, which is combined with the image and fed to the inpainting network that produces the generated image. The inpainter, I, is trained from the loss of the discriminator, D, following the typical GAN pipeline. Below is an illustration of the Deep Angel neural network architecture. Of course, there are many additional optimizations that enable running this neural network in a production environment.

More ways to explore

Except for papers, external publications, and where otherwise noted, the content on this website is licensed under a Creative Commons Attribution 4.0 International license (CC BY 4.0). This also excludes MIT’s rights in its name, brand, and trademarks. For papers and external publications included on this website, please contact the author(s) or publisher(s) directly for licensing information.