Can an algorithm tell us who influenced an artist?

Computer scientists at Rutgers University are developing an algorithm that picks up similarities between images of paintings, based on visual elements such as composition color. It found similar elements in Vincent van Gogh’s Old Vineyard with Peasant Woman 1890 (left) and Joan Miro’s The Farm 1922 (right). (Comparison courtesy of Rutgers University's Department of Computer Science./Comparison courtesy of Rutgers University's Department of Computer Science.)

At Rutgers University in New Jersey, scientists are training a computer to do instantly what might take art historians years: analyze thousands of paintings to understand which artists influenced others.

The software scans digital images of paintings, looking for common features — composition, color, line and objects shown in the piece, among others. It identifies paintings that share visual elements, suggesting that the earlier painting’s artist influenced the later one’s.

The project is part of a broader effort at Rutgers to apply computer science techniques to the humanities. This year, the university established a Digital Humanities Lab, based in its Computational Biomedicine Imaging and Modeling Center. The art application is among its first projects.

The field is growing. The Getty Foundation in Los Angeles provides grants to researchers in digital art history; George Mason University’s Roy Rosenzweig Center for History and New Media, one of the recipients, received $155,000.

And Washington’s Folger Shakespeare Library recently was awarded a grant to digitize its collection of manuscripts and artwork. The goal is to let outside researchers download the entire database and analyze it, according to Michael Witmore, the library’s director.

But some art historians — including Lisa Strong, director of Georgetown University’s art and museum studies program — are skeptical about visual algorithms such as the one in development at Rutgers.

“You can’t really impose a scientific framework so profitably on an exercise like painting analysis,” she said. “It’s not something where raw data tells you something. It’s all subjective.”

Still, the software has revealed some connections that art historians had not — at least, according to the team’s survey of existing art history literature, said Ahmed Elgammal, an associate professor of computer science, who has been working on the project for about three years. “The advantage is it can easily mine thousands and millions of art works in a very [efficient] way.”

For instance, after churning through a database of 1,700 pieces created between the 15th and 20th centuries, its visual algorithm zeroed in on American artist Norman Rockwell’s “Shuffleton’s Barbershop,” completed in 1950, and French impressionist painter Frederic Bazille’s 1870 “Bazille’s Studio; 9 rue de la Condamine.”

“The painting might not look similar at the first glance, however, a closer look reveals striking similarity in composition and subject matter, that is detected by our automated methodology,” Elgammal’s team wrote in a paper outlining their research, called “Toward Automated Discovery of Artistic Influence.”

Both Rockwell’s depiction of a barbershop, seen through a window, and Bazille’s painting of his studio have heating stoves on the right side; roughly where Bazille’s has a window, Rockwell has placed a door; and the composition of objects in each painting creates a triangular space in the lower left corner. (The software was trained to identify certain objects in photographs by analyzing Google images.)

Although detecting similarities between paintings can help art historians discover possible influences, the software cannot definitively establish a connection between two artists, according to the Rutgers team.

“Our final goal is not to get a final answer,” Elgammal said. Rather, it is “to be a tool to art historians, so it can help them do their job.”

Even if the visual software calls attention to two paintings that had never been compared before, only an art historian can analyze that comparison within the social context of two painters, Strong said.

“When you’re crafting an argument about a painting, you might want to say, ‘Norman Rockwell looked at Bazille, because he wanted to burnish his reputation and make his art more acceptable to art [critics]’ . . . but it’s not important in and of itself.”

She said that a computer program analyzing two-dimensional photographs of paintings cannot make all the observations that a historian might make while standing in front of the paintings. “You could have an artist who is heavily influenced by Monet’s [thick application of paint], and the algorithm might not catch that when it’s looking straight on.”