Windows Insiders are using HoloLens to train future doctors

We’re thrilled to introduce the winner of the Insiders in Action Contest! We asked Windows Insiders to enter the contest by showing us how Windows helps them do what they love. Beerend Hierck’s team atLeiden University Medical Center was selected as a winner out of nearly 350 applications received from over 75 countries. Thank you, Windows fans!

Mixed reality is one of most exciting innovations gaining traction today. Also known as MR or augmented reality, it has the potential to dramatically improve the way we learn and work. Seeing this potential, Beerend Hierck and his team at Leiden University Medical Center in the Netherlands have developed an app called DynamicAnatomy that opens up new pathways for medical students and doctors to gain an in-depth understanding of the human body.

Wearing a HoloLens loaded with the app, medical students can learn with the help of a three dimensional (3D) hologram. For now, the app is focused on the ankle and foot area. The user can walk around an enlarged 3D model of the lower leg while toggling the display of bones, muscles, ligaments, blood vessels, nerves, and more. The life-like model lifts, tilts, and stretches realistically, making it easy to learn the complex functions of the ankle and foot joints.

Having taught at medical school for more than 25 years, Beerend has long sought a better way to enable students to learn in 3D. “Typically, for the first three years of study, students learn the anatomy of the human body essentially in 2D—from books, computers, and tablets. They learn a bit from each other, and there are 3D specimens in the dissection hall, but that’s limited, because it’s time consuming and expensive,” he explained.

“Later, students come into contact with patients, who are far from 2D of course. We saw that they had to relearn their anatomy lessons to some extent, which was essentially a waste of time. So this project is an exploration of how to improve learning and teaching using 3D,” Beerend said.

Using mixed reality and the HoloLens is an especially immersive way to learn. Students can work in groups and each wear a HoloLens unit while interacting with the same hologram. And when DynamicAnatomy is paired with a motion capture suit, students can even flex and rotate their own ankles and watch the hologram’s various parts moving and responding in real time. The result is being able to feel the motion that you are initiating, which adds a sensory and kinesthetic component to learning.

“A lot of people are working on haptics,” Beerend said. (Haptic holograms simulate the sensation of touch when you interact with them.) “We took it from the other side and used our own body or our own movement to move the hologram. That helps the students to learn from their own bodies. For example, not every student is capable of stretching their foot a certain way. And they can discuss why one can stretch their foot more than the other, because—perhaps one played soccer and was kicked in the ankle and injured, etc. They really love seeing exactly what is happening inside.”

To secure funding for the project, Beerend’s team at the medical school collaborated with Leiden University’s Centre for Innovation to apply for a grant from the Netherlands. After securing the grant, the team was able to join forces with a software company to develop the user interface and use HoloLens to bring their vision to life. The software development process took about six months.

Being part of the Windows Insider program helped Beerend’s team gain early access to updates, including the Fall Creators update, which was essential for HoloLens developers. They also relied on the Insider community for feedback and help with troubleshooting. “It’s such fun to work not only with such a cool device but also with the whole community,” Beerend said. “The community organized within the Insider program is helping each other a lot with implementation questions—working out things that almost everybody suffers from when they’re developing the big stuff. And so, both the Insider Program’s community and the early access to software improvements were very important to our process.”

Students have been overwhelmingly enthusiastic about using the app as part of their learning. To quantify the app’s advantages, the team is officially studying how augmented reality and the app improve the educational experience. “I love cutting edge technology, but only if it helps me do my job better,” Beerend added. “So, we are doing all kinds of experiments to prove how augmented reality is valuable for improving student learning. We’re studying how it relates to 3D cognition, motivation, and knowledge retention.”

Beerend’s colleagues in the medical field are already excited by the possibilities of deploying augmented reality in a variety of ways. For example, using 3D modeling for MRI data could help both doctors and patients better understand what’s happening with their bodies. Augmented reality could help doctors vividly demonstrate to one another how they’ve performed complex procedures, and it could be used to overlay holographic information during surgical operations.

Today, the DynamicAnatomy team includes medical students who are helping improve the app and expand its capabilities to other parts of the human body. “We’re working now with students to develop a lower-tech solution, because not everybody has a motion capture suit,” Beerend said. “As we speak, these students are working on the algorithms to make that happen.” Students are also creating features that gamify learning through the app. Beerend is happy to point out that as co-creators, the students keep him “very sharp” and inspired with fresh ideas.

Intrigued yet? DynamicAnatomy is available to download for free at the Microsoft Store. You can follow the app’s progress here or connect with Beerend on LinkedIn here. Once again, congrats to Beerend and his team for winning the inaugural Insiders in Action contest and having their photo and story featured on our home page! Would you like your work profiled on the Windows Insider home page? Enter the next Insiders in Action Contest, to be announced next month!