Teaching a Computer to Paint from Life Like a Monet

A neural network-powered algorithm can take any old photo and turn it into a work of art befitting an old master. Alex J. Champandard of AI Game Dev built his art algorithm off the Deep Forger software architecture, a 19 layer neural network called VGG (short for visual geometry group.)

Essentially, the VGG allows Champanard to meld qualities of two images into a third output. One of the images for his project comes from a library of classic art. When a user's photo is put in, it finds a close compositional match and creates a new output, refining its algorithm via Twitter response. (Likes and retweets serve as a positive reinforcement mechanism.)

Advertisement - Continue Reading Below

"Another very interesting aspect is that machine learning can eventually capture, for example, the craft of Rembrandt or Mary Cassatt: the grains of their canvas, the strokes of the paint, their common elements, and aspects of the painting composition," Champandard said in an email interview.

The users also have filters that work similar to boolean operators, helping steer it to a particular artist if they desire. Champanard said in that, in part, this was a project about making AI art projects more fun and accessible to the layman. (Deep Forger is designed to be much easier to work with than, say, the char-rnn module used by many programmers.)

Most Popular

Of the process, Champanard said: "You send the bot your photo along with a description of what you want, and it'll return a painting that it generated! By default, it will pick randomly one of the famous painting it knows from its database, but you can also use commands like "!sketch" "!stylized" or "!abstract" to commission a particular type of painting."

This means that a photo of, say, a horse butt can turn into a mimic of a Monet painting (and has!). But it also provides a tool for an artist to imitate their own style, and add on to existing works or fill in the gaps. It could also help to visualize a painting before it comes to life and realize a better composition by having a computer generated rough draft.

But it also speaks to the increasing sophistication of machine learning, neural nets, and complex algorithms, which can recognize not just objects, but entire compositional techniques.

"The whole machine learning community is booming right now and the field of machine learning for generative art is just emerging," he says. "A few months ago it was Deep Dreaming, then applications in Recurrent Neural Networks, but with the recent publication of the "Neural Style" there's more potential for the tools to be helpful for artists—and usable by general users too."