Using Data, Brand New Portrait Was Generated in Style of the Dutch Painter

By Alexandra Jardine.Published on Apr 05, 2016

Editor's Pick

"The Next Rembrandt," an ING campaign that used data to create an entirely new painting in the style of the old master, has won two Grand Prix at the Cannes Lions International Festival of Creativity, for Creative Data and Cyber.

In the Creative Data category, jury president Tash Whitmey, CEO at Havas helia, called the campaign "a beacon for creativity" because it takes data, which some find a bit "scary," and turns it into something beautiful that makes people feel comfortable with.

In the Cyber category, the project, by JWT Amsterdam, won one of two Grand Prix awarded. The other went to "Justino," a touching multi-platform story promoting the Spanish Lottery. Jury President Chloe Gottlieb praised "The Next Rembrandt," for how its creative emerged from the data."The data is not an output from the creativity. The data was the beginning, the source for the creativity. It's something coming from the digital world creating for the physical world, so in a sense, it's the opposite trajectory from the work we were seeing in Cyber just a few years ago."

A new portrait of a man in black 17th-century clothing with a white collar and a hat was created using data from Rembrandt's total body of work, using "deep learning" algorithms and facial recognition techniques. The portrait consists of over 148 million pixels, based on 168,263 painting fragments from Rembrandt's art.

ING commissioned agency JWT Amsterdam to create the project to showcase what data and technology are capable of creating. Microsoft supported the project with its software and advisors from Delft University of Technology, The Mauritshuis and Museum Het Rembrandthuis also worked on the portrait.

The process involved analyzing all 346 of Rembrandt's paintings using high resolution 3D scans and digital files, and working with Rembrandt experts to determine the subject. After this, the subject's features were generated in the style of Rembrandt. A facial recognition algorithm identified and classified the most typical geometric patterns used by Rembrandt to paint human features, then generated new facial features for the painting. Next, these individual features were assembled into a fully formed face and bust according to Rembrandt's use of proportions.

When the 2D version of the painting was ready depth and texture were added. Finally, to bring the painting to life, an advanced 3D printer specially designed to make high-end reproductions of existing artwork was used.