Art-ificial Intelligence? Algorithm Sorts Paintings Like a Person

Below:

Next story in Science

From assembly-line work to self-driving cars, computers are
taking over many tasks once performed by humans. Artistic jobs,
however, have been relatively safe — until now.

A team of researchers has developed an
artificial intelligence (AI) program that can classify famous
works of art based on their style, genre or artist — tasks that
normally require a professional art historian.

"We're definitely not
replacing art historians, but with a growing number of
paintings in online collections, we need an automatic tool" for
organizing them, said study researcher Babak Saleh, a computer
scientist at Rutgers University in New Brunswick, New Jersey.

The field of computer vision has advanced significantly in recent
years, but AI still lags far behind humans in basic tasks. A
human can look at a painting and easily draw inferences from it,
such as whether it's a portrait or a landscape, whether the style
is impressionist or abstract, or who the artist was.

"The average person can tell these things, but that's very
challenging when it comes to a machine," said study researcher
Ahmed Elgammal, who is also a computer scientist at Rutgers. "Our
goal is to push what machine intelligence can do."

To create a machine capable of classifying art, Saleh and
Elgammal used a database of more than 80,000 paintings by more
than a 1,000 artists across 15 centuries, spanning 27 different
styles.

The researchers used a variety of machine-learning algorithms to
pick out particular features in a subset of the paintings,
including low-level attributes, such as colors and edges, as well
as more abstract ones, such as what an object is — whether it's a
horse or a human, for example. One approach they used is known as
deep learning, a method employed by Google and other
companies in image searches and translation tools.

Then, the researchers tested their algorithm on a set of
paintings the machine had never seen, and it performed remarkably
well. The program was 63 percent accurate at identifying the
artist, about 60 percent accurate at figuring out the genre and
about 45 percent accurate at determining the style.

It's difficult to compare the AI's performance to that of an art
historian, because the historian has a lot of prior knowledge,
Elgammal said. However, he estimated the algorithms would "do
much better than the average human," though "not as good as an
expert."

In addition, the paintings the algorithm had trouble categorizing
offered insight into the influences different painters may have
had on each other. For example, the algorithm had difficulty
distinguishing between a painting by the 18th-century Danish
painter Christoffer Wilhelm Eckersberg in the neoclassical style
and one by the early 19th-century Dutch painter Cornelis
Vreedenburgh in the
impressionist style.

These parallels are no surprise to art historians, but are
nevertheless impressive for a computer program, the researchers
said.