The term “mixed reality” has been thrown about a lot as of late, but pinning down a precise definition has proven elusive. After spending the day at Stanford University’s SCIEN-hosted workshop, which was full of augmented reality (AR) and mixed reality (MR) luminaries, it appears the one thing everyone can agree on is there isn’t a standard definition for either term. Let’s see if we can do a bit better.

AR now covers a broad spectrum, from Instagram Stickers and Pokemon Go, to sophisticated onsite walkthroughs of rooms of virtual furnishings. Technically, MR covers that, plus virtualized surroundings as context for physical objects–for example, visiting an appliance showroom and seeing an oven nestled in a model of your kitchen.

For now, though, high-end AR companies have adopted MR as a marketing buzzword to describe Head Mounted Displays (HMDs), which present an integrated view of digital content and the real world as seen through the mostly transparent goggles. Loosely speaking, they are using MR to mean incorporating the best bits of both AR and VR in a single device. After a day of talks and demos, it’s clear they’re still short of that goal.

Augmented Reality already has traction for vertical applications

In the marketplace, AR is almost certainly already bigger than VR. It has dozens of uses in industry, some of which we covered at Augmented World Expo. There were well over 100 exhibitors with commercial and industrial applications already in the marketplace, as well as several high-end military products, and even many vendors providing infrastructure support for AR rollouts. We’ll be back at AWE this year and let you know how things have progressed.

Just like VR, though, AR and MR have had a hard time getting traction outside of niche markets, and the staggering but unique success of Pokemon Go. As far as AR-friendly hardware, the best known devices are based on Google’s Project Tango. Google only made it available to developers, but Lenovo has packaged it in a consumer phone–the Phab2 Pro. Unfortunately, it’s bulky and not an especially great phone, so it hasn’t taken the world by storm.

Asus has announced an impressive new model, the Zenphone AR, which supports both Project Tango for AR and Google’s Daydream for VR. It has impressive specs, but no price yet. If it is a great phone, it should be a better test of how popular Tango’s AR features — like building a 3D model of any space you walk through — are with consumers.

Introducing mixed reality

At the extreme, MR builds on AR by adding elements of Virtual Reality (VR), as exemplified by Microsoft’s HoloLens demo of a NASA simulation of walking on the surface of Mars. Microsoft uses the term MR to imply a mash up of the best of AR and VR, which of course makes the HoloLens one of the leading products in the space. But MR is definitely catching on with a large number of players in AR and VR. Even Oculus, best known for its VR headsets, had two speakers at the workshop talking about its work in AR and MR.

Reality roadmap according to Intel, which prefers Merged Reality to Mixed Reality

Where mixed reality falls short

Mixed reality demos shown on a big screen in front of a roomful of eager conference participants, like those at the SCIEN-hosted workshop at Stanford, are amazing. For example, the Mars experience for the HoloLens makes it look like you are practically in the Star Trek Holodeck. Similarly, watching Meta’s CEO Meron Gribetz throw windows around with his hands like the conductor of a symphony orchestra is compelling.

When you actually put on a HoloLens, however, you realize the field of view is tiny. You need to keep moving your head around to see any of the augmented or virtual effects. It is a definite stretch to imagine yourself transported anywhere, like you can with some of the best pure VR demos.

Likewise, the actual hands-on demos of Meta we were given were simply a poor rehash of the typical VR demos of looking inside a car or spinning a globe. There was no integration with anything in the real world. Meta is promising to formally unveil its spatial user interface demo at AWE at the end of this month, so we’ll soon know if that’s a major step forward.

Magic Leap has already been heavily criticized for over-hyping its efforts using demo videos created with special effects. I also got a quick demo of Intel’s Alloy platform, that relies heavily on its RealSense technology for tracking, but it was actually a pure VR demo, so I don’t know how well they have solved the problem of linking real and virtual objects.

What it will take for mixed reality to move beyond niche applications

A large part of the Stanford workshop was spent agonizing over where the MR industry needs to go from here. At a minimum, several inter-related technical challenges need to be solved.

The first is device Field of View (FOV). It needs to be competitive with VR, at something over 100 degrees. Unfortunately, that requires more power and more bandwidth, which is at odds with the second problem, which is tethering. So far, large FOV devices all need to be tethered to a high-powered computer. Third is alignment with, and tracking of, the real world surroundings. Without this, AR and MR wind up looking like parlor tricks or a watered down version of VR.

Perhaps most importantly, as we heard from Google’s Johnny Lee, is the need for inventing new kinds of storytelling that can leverage the strengths of AR and MR. His own Project Tango is a good example. There are one or two impressive applications–mostly creating a model of a home for realtors, and visualizing how furniture will fit in your home or apartment–but they haven’t been enough to get consumers to want Tango enough to buy a different phone.

Lee, and some of the other speakers, openly wondered how the AR/MR industry would sustain the investment levels needed to solve all the technical challenges without additional mainstream acceptance of the technologies. For now, that isn’t a problem, as billions of dollars are being poured into R&D in the area. As with VR, though, it looks like we are some years away from truly broad adoption of either augmented or mixed reality products.