Magic Leap finally arrives – Augmented Reality for real?

It’s been a long while coming, but the $2.3bn investment in Magic Leap has finally seen its first Augmented Reality headset ship, the Magic Leap One Creator’s edition. On 8th August 2018 it became a real thing, that real people could get hold of, almost. I say almost, because this developer edition of the headset is only available in a select few US cities, so over here in the UK it’s tricky to join in. Having officially been a corporate and independent metaverse evangelist, and still very much one unofficially, I would be really interested in getting my hands on one, but instead I am able to explore it at arm’s length through the wonders of the interwebs.

I have often alluded to what Magic Leap is in my AR, VR work at 451 Research but we prefer to talk about companies we have managed to talk to or engaged with in some capacity, and I don’t seem to have unlocked that path yet. The BBC had a piece on Magic Leap, with a not overly impressed presenter, to which I had to reply and remind the world that AR is the user interface for IoT. It was nice to see the founder and head of Magic Leap Rony Abovitz (@Rabovitz) reply “We are seeing early developers begin to link @magicleap to IOT. Will be really interesting.” He has been diligently following social media and reminding the World this is not the end point for AR but it’s a starting point. The problem is this new product set comes into a world with 2.3 billion high expectations.

Real AR

Magic Leap One is a set of glasses, cyberpunk bug eye looking ones, and an accompanying computing device, a puck shaped and ready to be hung on a belt and a wireless hand controller. The headset is laden with sensors to detect the world around it and a set of translucent displays that allow images to be pumped to them in such a way that they blend with the user’s physical view of the world. These waveguide lens layers the company refers to as photonic light fields.

As the device can determine what is in the physical world it can use that when rendering content, so digital things can bump into or rest on surfaces or disappear behind other real-world objects. Spatial audio and digital rendered images combined with the real world make for a compelling concept. The Lumin SDK allows both Unity and Unreal developers to build environments and experiences, including simulating the device for those who don’t yet have one.

Fictional AR

In Cont3xt (pronounced Context) my follow up sci-fi novel to Reconfigure I extended the phone and tablet AR experience lead character Roisin has with a fictional AR device the eyeBlend. This is what I would expect from an AR device (in the near future).

She moved the glasses up to her face and slid the arms over and past her ears. She pushed the ear buds down a little on their hinged lever. She felt the gel like bumps make contact. Their surface tension stuck them to her exposed skin. She could see the lounge around her, it was only gently tinted now by the layers in front of her eyes. The lens was almost all encompassing. There was no perceivable frame in her field of view.

A triple ascending beep, like a text message being delivered, sounded in her ears. As it did so directly in front of her, floating in space, was a slowly spinning version of the eye, World and lashes logo. She turned her head to look back out to the bay window. The logo stayed where it was. She could still see it out of the corner of her eye. She turned back to admire the logo. It was a shiny blue metallic version. The globe in the pupil was now slowly rotating, showing all the countries of the World. It was swirling with the sea blue and dark green of the land. As the sun blasted into the room, from the large window, Roisin noticed the logo seemed even more real, as it was casting a shadow on the wall and also, she could see it reflected in the marble floor too. She stepped closer to it. The globe rotated slowly, and she moved her head nearer to it to get a closer look. Cloud systems swirled around on the globe. She reached her hand out to cup the World. She saw her hands obscure her view of the object and then disappear behind it. The projected image was completely blended with her view of the World. Only the strange concept, of a spinning logo in midair, made this feel anything other than totally real. She stepped back out and around the logo as it hovered and still gently rotated like her childhood bedroom mobile.

“Is it as good as they say it is?” She heard from behind the logo. Roisin could just make out Alex pouring some more coffee in the kitchen.

“As a boot up logo, this pretty much wins, and then some.” Roisin replied.

So it’s actually pretty close at a certain level to what the real world has to offer, except the eyeBlend does a lot more (no spoilers here).

Fact meets Fiction

Magic leap, like Microsoft’s HoloLens do indeed offer an augmented or blended reality experience but not quite to the level one might expect. As Magic Leap appeared it had not really had that much hype, obviously raising shed loads of money does attract attention and there was a significant wired article but it was not like the company was asking everyone to believe a marketing story.

Yes, it’s true the first website and video showed a large blue Whale crashing into an imaginary ocean in the middle of a school hall with hundreds looking on, but I think we all knew, in the business anyway, that was aspirational. A subsequent demo showed an office worker’s view with emails, calendar and then shooting things coming out of the walls. Another showed a working solar system.

Here is where the reality bites though and where some people report disappointment. With both systems the area of digital projection does not fill your field of view, you are looking through a lens within a lens. Our brains only really get the detail from central focus of our eyes but he brain is very aware of things in its peripheral vision. In VR this is less of a problem, everything you see is generated, some system use foveated rendering to provide intricate detail precisely where you are looking but still render the outer blur.

The real world, the canvas for AR, is not so forgiving. Objects will disappear or be cut in half as they drop out of view. This is one reason that Magic Leap is being slowly rolled out as a