Traditional occidental painting techniques like watercolor or oil build an image from many layered brush strokes. You don’t usually notice the individual strokes unless you stand very close. But in traditional oriental ink painting, called sumi-e, the brush strokes are the painting. Each stroke is made to capture maximum information about the subject.

Example brush strokes in typical shapes, from Xie et al.

In its simplicity, sumi-e poses a challenge to computer-generated artistry. The computer can’t simply add and layer a large number of standard strokes, because the shape of each brush stroke is entirely dependent on the subject. And subjects can be infinite.

Now three computer science researchers from the Tokyo Institute of Technology have cracked the sumi-e puzzle by training a digital ink brush with a technique called reinforcement learning.

Iterations in the learning process of the digital brush, from Xie et al.

They trained their model brush not on entire images (like a bird) but only on individual brush strokes (like the curve of the bird’s belly). They digitized eighty natural brush strokes and gave them to the model to mimic, along with a function that rewarded smoother shapes. (To your correspondent’s dismay, “reward functions” in computer science don’t actually hand out cookies.)

After the sumi-e brush program had been trained, the researchers tested it with a greater variety of natural brush strokes. Finally, they used it to create full paintings from photos–though they had to lend a helping hand, by manually drawing contours onto the photos for the program to find and fill.

Some digital painting programs attempt to satisfy artists by simulating all the physics of the painting process, from the exact angle of the brush to the absorption of ink into paper. Xie et al.’s sumi-e program is from a different family of methods, called “stroke-based rendering,” whose aim isn’t to create a believable experience for the user, but merely a believable end product. It’s hard to argue with the results.

Author

Danna Staaf

Danna Staaf is a marine biologist, science writer, novelist, artist, and educator. She holds a PhD in Squid Babies from Stanford and a BA in Biology from the College of Creative Studies at the University of California, Santa Barbara. She helped found the outreach program Squids4Kids, illustrated The Game of Science, and blogs at Science 2.0. She lives in San Jose with her husband, daughter, and cats.

About KQED

QUEST is supported by:

The National Science Foundation

Funding for KQED Learning is provided by the Koret Foundation, the Cisco Foundation, David Bulfer and Kelly Pope, the Horace W. Goldsmith Foundation, the Mary A. Crocker Trust, and the members of KQED.

Support for KQED Science is provided by HopeLab, the S. D. Bechtel, Jr. Foundation, The David B. Gold Foundation, The Dirk and Charlene Kabcenell Foundation, The Vadasz Family Foundation, the John S. and James L. Knight Foundation, Gordon and Betty Moore Foundation, the Smart Family Foundation and the members of KQED.