Users of the HoloLens headset gesture to click open a schematic or make a Skype call.Photo source: Building Operating Management

Thyssenkrupp Elevator has announced its use of Microsoft HoloLens technology in its elevator service operations worldwide. The company is the first to apply Microsoft’s new “mixed reality” wearable computer headset to empower elevator service technicians to do their jobs more safely and efficiently. A program for thyssenkrupp’s more than 24,000 technicians is currently in the pilot phase, and the technology was demonstrated for journalists in September.

Using Microsoft HoloLens, which is a wearable holographic computer that Microsoft introduced to developers this spring, thyssenkrupp service technicians will be able to visualize and identify problems with elevators ahead of a job, view schematics and other site-specific data, and have remote, hands-free conversations with technical experts while on site.

“With more than 12 million elevators transporting more than 1 billion people each day, the elevator service industry has a critical role to play in keeping cities moving,” said Andreas Schierenbeck, CEO of thyssenkrupp Elevator, at the company’s first demonstration of the technology, which was held at One World Trade Center (WTC 1) in New York City. Thyssenkrupp is one of the world’s largest elevator, escalator and moving sidewalk makers in the world, and its equipment is used throughout the landmark building. The WTC 1 complex boasts 12 escalators and 71 elevators, including the fastest elevator in the Americas, he added.

“We remain focused on leading the transformation in this industry: introducing the latest technologies, processes and training to enable technicians to do a better job with less stress and more fun,” said Schierenbeck. “Our goal is to dramatically increase efficiency, raise elevator uptimes and speed up service interventions to ensure mobility equipment is always running as it should, providing each passenger with the safest and most comfortable travel experience possible."

The thyssenkrupp HoloLens service solution follows the successful launch of MAX, the company’s cloud-connected predictive maintenance software that is already in use with thousands of elevators, including those at One World Trade Center. MAX is Internet of Things enabled, and the first step in thyssenkrupp’s journey to transform its business. By combing the power of the Building Internet of Things with solutions like MAX and HoloLens, thyssenkrupp intends to “step further into the digital era and transform the way the elevator industry offers maintenance services,” said Schierenbeck.

“This application of HoloLens in our service operations was made possible through our collaboration with Microsoft. The close integration with their software experts has allowed us to explore dedicated HoloLens apps to meet thyssenkrupp’s specific requirements,” he added.

At the press conference, Sam George, partner director with Microsoft Azure IoT, on which MAX is based, said, “Predictive maintenance, powered by Microsoft Azure IoT and enabled by thyssenkrupp, offers time savings to elevator passengers worldwide equivalent to 95 million hours of new availability per year of operation. Today, we are proud to have once again collaborated with thyssenkrupp to bring another game-changing solution to market together.”

Launched in 2015, MAX is already connected with thousands of units in the pilot countries of U.S., Germany and Spain, and is on track to be connected with 180,000 units by the end of 2017. The successful implementation in the pilot countries has also set the stage for rollouts in additional countries by the end of this year.

A wearable holographic computer

The Microsoft HoloLens headset is a hands-free laptop on your head that responds to gestures the user makes in front of it. The six cameras on the device see surfaces and depth, and gyroscopes monitor the movement of the wearer’s head. It is not the “person-locked” immersive virtual reality functionality typical of gaming systems, but is instead projecting a hologram onto what the user is seeing, similar to the augmented reality functionality of heads-up automobile displays, said Scott Erickson, general manager for Microsoft HoloLens.

That hologram projection—which can be, for example, schematics of the mechanisms behind the wall or inside the elevator shaft—is “world-locked” so as the technician moves around the space, the projected hologram adjusts, explained Erickson. He called this technology “mixed reality” and said HoloLens is the first such mixed-reality, fully self-contained holographic computer running Windows 10.

The HoloLens developer kit, which has been out since March, is being used by NASA on the Mars Rover program and by Case Western to train medical students. The thyssenkrup application combines elevator schematics and MAX predictive maintenance data with the headset’s out-of-the-box Skype experience to enable technicians to transmit what they are seeing and what the headset is projecting to experts located off-site.

Said Erickson, “Triaging service requests ahead of the visit and getting hands-free remote holographic guidance when on site has reduced the average length of thyssenkrupp’s service calls by up to four times” in tests.

Schierenbeck said that the more than 2,000 types of elevators in use around the world lead to complexity that slows technician response and increases system downtime. In the U.S. alone, there are 150,000 emergency calls in the U.S. per year. Worldwide, “the average elevator stops working four to six times every year and accounts for a total of 190 million hours of downtime. In the auto industry, no one would accept that,” he added.

HoloLens and MAX follow several other technological innovations from thyssenkrupp, including TWIN, with two elevator cabins per shaft providing up to 30 percent higher transport capacity and 27 percent less energy consumption; MULTI, an elevator that moves horizontally as well as vertically, providing a 50 percent increase in transport capacity and a reduced elevator footprint in buildings; and ACCEL, a moving walkway that uses linear magnetic technology to reach speeds of up to 12km/h.

To see a video of the HoloLens technology, visit the thyssenkrupp blog at http://blog.thyssenkruppelevator.com/content/elevator-maintenance-got-way-cooler-today