Mixed Reality Museum in Kyoto: A unique insight into centuries-old Japanese artwork

Jeff Hansen

Little can compare to getting a personal tour of one of Japan’s national treasures by a very personable Zen Buddhist monk, while standing in an amazing, beautiful Zen temple in Japan.

Today, I’m proud to share some of the innovative work of hakuhodo-VRAR on an extraordinary mixed reality experience available to visitors of Kyoto’s Kennin-ji, the oldest Zen temple in Japan. In partnership with the Kyoto National Museum, hakuhodo-VRAR’s “MR Museum in Kyoto” applies a mixed reality experience to Kennin-ji’s beautiful and sacred artwork The Folding Screen of Fujin and Raijin (Wind God and Thunder God Screens) by Tawaraya Sotatsu, painted over 400 years ago.

The 10-minute experience provides a dynamic, holographic narrative that helps temple visitors better understand Tawaraya Sotatsu’s vision for The Folding Screen of Funin and Raijin. A holographic Kennin-ji monk guides guests through the experience as contextual, digital descriptions populate within the dedicated temple room. One of the most compelling experiences is seeing the world’s first 3D renditions of works from the same Fujin and Raijin theme by Ogata Korin, Sakai Hoitsu and other artists of the Rinpa school who were inspired by Tawaraya Sotatsu. Each was produced 100 years apart and they are physically in several separate museums, but in this experience, you get to see them all side by side in very high resolution and in compelling 3D holographic images.

The result is a remarkable blend of historical artifacts with bleeding-edge mixed reality technology that provides an all-new, engaging way to appreciate and understand one of Japan’s national treasures.

“MR Museum in Kyoto” solves several challenges faced by traditional museums. Through HoloLens, the process of learning about The Folding Screen of Funin and Raijin becomes experiential and interactive. Historical components are given new context—in full, volumetric 3D—within the physical temple, enabling a more effective communication of detailed or complex concepts than traditional text or video can deliver.

In addition, hakuhodo-VRAR’s experience is the first Japanese HoloLens project to take advantage of our Mixed Reality Capture Studios. Like many other mixed reality content creators from around the world, hakuhodo-VRAR visited one of our studios to record volumetric, holographic video of dynamic people and performances; in this case, you can see how they captured the Kennin-ji monk in the video below. I remarked in front of the press and TV stations gathered at the announcement event, that Asano-san is now likely the world’s first 3D monk. Viewers can experience content captured at the studios in a variety of ways, from Microsoft HoloLens, the world’s first fully self-contained holographic computer, right through to 2D, mobile-phone displays.

While it was a profound privilege to stand in a temple dating from 1202 AD and experience the cutting-edge technology of HoloLens and mixed reality to take a museum experience to an entirely new level of immersion, “MR Museum in Kyoto” is far from the first example of HoloLens applied to culture and the arts. Back in December, my friend and colleague Lorraine Bardeen sharedsome fascinating projects from Case Western Reserve University, Cornish College of the Arts and a mixed reality art installation from Artsy called “Concrete Storm.”

Along with commercial and entertainment applications, the variety of these experiences further underscores the transformative nature of mixed reality through every facet of our lives. We’re proud to support this momentum through 2018 and beyond as the era of mixed reality continues to take shape.