February 16, 2016

Artist Uses 3D Facial Projection Technology to Create Digital Skin

Using computer graphics and facial projection and tracking technology, Nobumichi Asai creates a new art form of facial tattoos, where digital makeup transforms a human face into a masterpiece of computer graphics.

Makeup has the transformative ability to make a face beautiful, alluring, sexy, clownish or creepy. As the art of makeup moves into the digital age, it won’t be long before people can use 3D facial projection technology to paint faces with anything from tiger stripes and leathery snakeskin, to exploding waterfalls and erupting bouquets of flowers.

The experimental art of facial projection by Japanese artist and scientist Nobumichi Asai turns the human face into a canvas of the imagination. The curves and features of the face become an irresistible landscape for animated, kaleidoscopic images.

Based in Japan and London, Asai and his team from WOW worked with Intel to demonstrate how Asai’s artistry, combined with a lot of computer processing power, could push the limits of facial projection.

“The most interesting thing about technology is that it makes the impossible, possible,” said Asai. “When I start creating, I first feel something that excites me.”

Asai uses the human face much like a filmmaker uses the big screen. By tracking the exact contours of a model’s nose, mouth, eye sockets, chin and face shape, Asai and his team use the model’s face as a canvas for what is essentially very intricate animated make-up.

By knowing the mathematical contours of the face, Asai can map his digital animations so they fit perfection onto facial features. A face can become a tribal mask, a waterfall of tears, a collage of stone — any kind of programmable digital skin.

“Since the beginning of time, humans have been decorating their faces, enhancing their beauty using art, colors and textures,” said Paul Tapp, Director of Technology for Intel Marketing.

Asai’s background in science pushes his curiosity for how technology can bring his ideas to life.

“He uses advanced technology to decorate the face in an unconstrained and unique manner. He has full freedom of expression, using the face as a three-dimensional canvas which he can adorn at a level of detail which was previously impossible,” Tapp continued.

“Perhaps even more incredibly, for the first time, he is able to animate his canvas,” Trapp said. “He couples this with advanced facial-tracking techniques to ensure that the animations are perfectly congruent with the face, no matter what position it’s in or expression it is wearing.”

Tapp said Asai first takes a 3D scan of a model’s face. He uses makeup to create different looks and then captures the looks using a technique called photogrammetry. He uses those photographs as the basis for creating 3D models.

It’s important to understand the full 3D detail of the face, said Tapp, otherwise, if the content is designed in two dimensions, it will stretch and look unpleasant when it is projected. Asai then places a few infrared markers on the (now makeup free) model and then an array of five infrared cameras can see and track those points on her face.

An ultra-fast computer — using an Intel 6th Gen Core i7-6700K processor — combines the tracking data so the digital skin can be crafted to fit the exact position and expression of the model’s face.

“The level of accuracy we can calculate from the tracking algorithms, combined with the responsiveness of the PC to generate the right image instantaneously, means that we end up being able project these inert ‘virtual tattoos’ to the face with an almost skin-like authenticity,” said Tapp.

For Asai, integrating technology into art is an innovative and essential way to bring nature to life.

“My favorite thing is the joy and excitement people get from my work,” said Asai. “It makes me want to create something even better.”

Editor’s Note: In this Experience Amazing series, iQ explores how computer technology inside is enabling incredible experiences outside. We look at how computer technology powers new experiences and discoveries in science, the maker movement, fashion, sports and entertainment. To learn more about the tech behind these stories, visit Experience Amazing.