The best travel apps for a smart, stress-free holiday

Inside Magic Leap, the startup creating a new type of reality

This article was first published in the June 2016 issue of WIRED magazine. Be the first to read WIRED's articles in print before they're posted online, and get your hands on loads of additional content by subscribing online.

There is something special happening in a generic office park in an uninspiring suburb near Fort Lauderdale, Florida.

Inside, amid the low grey cubicles, clustered desks and empty swivel chairs, an impossible 20cm robot drone from an alien planet hovers chest-high in front of a row of potted plants. It is steampunk-cute, minutely detailed. I can walk around it and examine it from any angle. I can squat to look at its ornate underside. Bending closer, I bring my face right up close to inspect its tiny pipes and protruding armatures. I can see polishing swirls where the metallic surface was "milled". When I raise a hand, it approaches and extends a glowing appendage to touch my fingertip. I reach out and move it around. I step back across the room to view it from afar.

Advertisement

All the while it hums and slowly rotates above a desk. It looks as real as the lamps and computer monitors around it. It's not. I'm seeing all this through a synthetic-­reality headset. Intellectually, I know this drone is an elaborate simulation, but as far as my eyes are concerned it's really there, in that ordinary office. It is a virtual object, but there is no evidence of pixels or digital artifacts in its three-dimensional fullness. If I reposition my head just so, I can get the virtual drone to line up in front of a bright office lamp and perceive that it is faintly transparent, but that hint does not impede the strong sense of it being present.

This, of course, is one of the great promises of artificial reality – either you get teleported to magical places or magical things get teleported to you. And in this prototype headset, created by the much-speculated-about, ultra-secretive company called Magic Leap, this alien drone certainly does seem to be transported to this office in Florida – and its reality is stronger than I thought possible.

Wednesday briefing: Samsung reveals its Oculus Rift rival

I saw other things with these magical goggles. I saw human-sized robots walk through the actual walls of the room. I could shoot them with power blasts from a prop gun I really held in my hands. I watched miniature humans wrestle each other on a real tabletop, almost like a Star Wars holographic chess game. These tiny people were obviously not real, despite their photographic realism, but they were really present - in a way that didn't seem to reside in my eyes alone; I almost felt their presence.

Virtual reality overlaid on the real world in this manner is called mixed reality (MR). The goggles are semi-transparent, allowing you to see your actual surroundings. It is more difficult to achieve than the classic fully immersive virtual reality (VR), where all you see are synthetic images, and in many ways MR is the more powerful of the two technologies.

Advertisement

Magic Leap is not the only company creating mixed-reality technology, but right now the quality of its virtual visions exceeds all others. Because of this lead, money is pouring into this Florida office park. Google was one of the first to invest. Andreessen Horowitz, Kleiner Perkins and others followed. In the past year, executives from most major media and tech companies have made the pilgrimage to Magic Leap's office park to experience for themselves its futuristic synthetic reality. At the beginning of this year, the company completed what may be the largest C-round of financing in history: $793 million (£556m). To date, investors have funnelled $1.4 billion into it.

That astounding sum is especially noteworthy because Magic Leap has not released a beta version of its product, not even to developers. Aside from potential investors and advisers, few people have been allowed to see the gear in action, and the combination of funding and mystery has fuelled curiosity. But to understand what's happening at Magic Leap, you need to also understand the tidal wave surging through the entire tech industry.

Read next

Apple's ARKit demos give a trippy taste of our mixed reality future

ByAndy Vandervell

All the major players – Facebook, Google, Apple, Amazon, Microsoft, Sony, Samsung – have whole groups dedicated to artificial reality, and they're hiring more engineers daily. Facebook alone has more than 400 people working on VR. Then there are some 230 other companies, such as Meta, the Void, Atheer, Lytro and 8i, working on hardware and content for this new platform. To fully appreciate Magic Leap's gravitational pull, you must see this emerging industry – every virtual-reality and mixed-reality headset, every VR camera technique, all the novel VR applications, beta-version VR games, every prototype VR social world. As I did – over the past five months.

Then you will understand just how fundamental VR technology will be, and why businesses such as Magic Leap have an opportunity to become some of the largest ever created.

Advertisement

Even if you've never tried virtual reality, you probably possess a vivid expectation of what it will be like. It's The Matrix, a reality of such convincing verisimilitude that you can't tell if it's fake. It will be the Metaverse in Neal Stephenson's rollicking 1992 novel, Snow Crash, an urban reality so enticing that some people never leave it. It will be the Oasis in the 2011 best-­selling story Ready Player One, a vast planet-scale virtual reality that is the centre of school and work. VR has been so fully imagined for so long, in fact, that it seems overdue.

I first put my head into virtual reality in 1989. Before even the web existed, I visited an office in Northern California whose walls were covered with neoprene surfing suits embroidered with wires, large gloves festooned with electronic components and rows of modified swimming goggles. My host, Jaron Lanier, sporting ­shoulder-length blond dreadlocks, handed me a black glove and placed a set of home-made goggles secured by a web of straps on to my head. The next moment I was in an entirely different place. It was an airy, cartoony block world, not unlike the Minecraft universe. There was another avatar sharing this small world (the size of a large room) with me - Lanier.

We explored this magical artificial landscape together, which Lanier had created just hours before. Our gloved hands could pick up and move virtual objects. It was Lanier who named this new experience "virtual reality". It felt unbelievably real. In that short visit I knew I had seen the future. The following year I organised the first public hands-on exhibit (called Cyberthon), which premiered two dozen experimental VR systems from the US military, universities and Silicon Valley. For 24 hours in 1990, anyone who bought a ticket could try virtual reality. The quality of the VR experience at that time was primitive but still pretty good. All the key elements were there: head-mounted display, glove tracking, multi-person social immersion.

Read next

Intel's 'merged reality' headset arrives this year

ByMatt Burgess

But the arrival of mass-market VR wasn't imminent. The gear cost many scores of thousands of dollars. Over the following decades, inventors were able to improve the quality, but they were unable to lower the cost.

Twenty-five years later a most unlikely savior emerged - the smartphone! Its runaway global success drove the quality of tiny hi-res screens way up and their cost way down. Gyroscopes and motion sensors embedded in phones could be borrowed by VR displays to track head, hand and body positions for pennies. And the processing power of a modern phone's chip was equal to an old supercomputer, streaming movies on the tiny screen with ease. The cheap ubiquity of screens and chips allowed a teenage Palmer Luckey to gaffer-tape together his first VR headset prototypes, launching a Kickstarter campaign for the Oculus Rift in 2012. And the Rift was the starting signal that many entrepreneurs were waiting for. (Facebook bought the company for $2 billion in 2014.)

All of today's head-mounted VR displays are built out of this cheap phone technology. Lanier, who has contributed to Microsoft's HoloLens MR system, estimates it would have cost more than $1 million in 1990 to achieve the results that phone-inserted headsets like the Samsung Gear or Google Cardboard do today.

Engineer Eric Browy working in the Magic Leap optics lab

Peter Yang

One of the first things I learned from my recent tour of the synthetic-­reality waterfront is that virtual reality is creating the next internet. Today the internet is a network of information. It contains 60 trillion web pages, remembers four zettabytes of data, transmits millions of emails per second, all interconnected by sextillions of transistors. Our lives and work run on this internet of information. But what we are building with artificial reality is an internet of experiences. What you share in VR or MR gear is an experience. What you encounter when you open a magic window in your living room is an experience. What you join in a mixed-reality teleconference is an experience. To a remarkable degree, all these technologically enabled experiences will rapidly intersect and inform one another.

Read next

Hands-on with Microsoft's HoloLens as it lands in Europe

ByVictoria Woollaston

The recurring discovery I made in each virtual world I entered was that although every one of these environments was fake, the experiences I had in them were genuine. VR does two important things: one, it generates an intense and convincing sense of what is generally called presence. Virtual landscapes, virtual objects and virtual characters seem to be there - a perception that is not so much a visual illusion as a gut feeling. That's magical. But the second thing it does is more important. The technology forces you to be present – in a way flatscreens do not – so that you gain authentic experiences, as authentic as in real life. People remember VR experiences not as a memory of something they saw but as something that happened to them.

Experience is the new currency in VR and MR. Technologies such as Magic Leap's will enable us to generate, transmit, quantify, refine, personalise, magnify, discover, share, reshare and overshare experiences. This shift from the creation, transmission, and consumption of information to the creation, transmission and consumption of experience defines this new platform. As Magic Leap founder Rony Abovitz puts it: "Ours is a journey of inner space. We are building the internet of presence and experience."

We haven't yet fully absorbed the enormous benefit that the internet of information has brought to the world. And yet we are about to recapitulate this accomplishment with the advent of synthetic realities. With a VR platform we will create a Wikipedia of experiences, potentially available to anyone, anywhere, anytime. Travel experiences – terror at the edge of an erupting volcano, wonder at a walking tour of the pyramids – once the luxury of the rich (like books in the old days) will be accessible to anyone with a VR rig. Or experiences to be shared: marching with protesters in Iran; dancing with revellers in Malawi; how about switching genders? Experiences that no humans have had: exploring Mars; living as a lobster; experiencing a close-up of your own beating heart, live.

You've seen a lot of this in movies and on TV or read about it in books. But you haven't experienced it, felt it below your intellect, had it lodge in your being in a way that you can call your own. Kent Bye, founder of the podcast Voices of VR, has conducted more than 400 interviews with the people creating VR and has seen almost every possible prototype of VR there is. "VR talks to our subconscious mind like no other media," he says.

Magic Leap's user experience team creates guides to new universes

Peter Yang

Read next

Intel launches wireless Project Alloy headset for 'merged reality'

ByMatt Burgess

The most complete sense of subconscious presence that I experienced occurred with a system called the Void, which debuted at the 2016 TED conference in Vancouver. The Void isn't as advanced as Magic Leap technologically, but it integrates the best off-the-shelf parts available with custom gear to create an unforgettable experience. For several hours I watched a line of people enter the Void. Almost every person squealed with delight and staggered away asking for more.

The Void grew out of stage magic, a theme park and a haunted house. Every year, Ken Bretschneider, one of the three co-founders, stages a gonzo haunted house in Utah that draws 10,000 people in two days. It occurred to him that he could amplify the interactions of his house with VR. Curtis Hickman, the second co-founder, is a professional illusionist and a visual-effects producer. The third, James Jensen, started out developing special effects for film and unique experiences for theme parks. He came up with the idea of layering VR over a physical playground. The common factor among the three was their realisation that VR was a new way to trick the mind.

The Void takes place in a large room. You wear a 5kg vest that carries batteries, a processor board, and 22 haptic patches that vibrate and shake you at the right moments. Your headset and earphones are connected to your vest, so you're free to roam without a cord or tripping over a cable. That relief heightens the effect of being present in the VR. Inside, you navigate an Indiana Jones-like adventure that seems to take place over a large territory. The illusion of unbounded space, or, as Hickman describes it, "a magical space bigger inside than it is outside", is achieved by a trick called redirected walking.

As an example, whenever you turn 90 degrees your VR will show you the room turning only 80 degrees. You don't notice the difference, but the VR accumulates those small ten-degree cheats until it redirects your route or even gets you to walk in a circle while making you think you've walked a kilometre in a straight line. Redirected touching does a similar trick. A room could contain one real block but display three virtual blocks on a shelf. You see your hand grab a block, but the VR system will direct your hand to touch the only real block.

It's astounding how those tiny misdirections fool you into believing that what you're seeing is real. Stairs can be made to feel endless if they drop down as you walk upward. In fact, at one point in the Void a decaying floor collapses while you're walking across it, and you see, hear, and feel a plunge down to the floor below. But in fact the real floor only sinks 15cm.

Read next

Magic Leap shows off top-secret 'mixed reality' tech

ByMatt Burgess

Seeing, it turns out, is not believing. We use all our senses to gauge reality. Most of the high-end VR rigs on sale this year include dynamic binaural – that is, 3D – audio. This is more than just stereo, which is fixed in space. To be persuasive, the apparent location of a sound needs to shift as you move your head. Deep presence includes the sensations of motion from your inner ear; if the two are out of sync with what you see, you get motion sickness.

Good VR also includes touch. Jason Jerald, a professor at the Waterford Institute of Technology in Ireland, claims that much of our sense of presence in VR comes from our hands. Gloves are still not consumer-ready, so hardware makers are using controllers with a few easily operated buttons. When you wave them, their positions are tracked, so you can manipulate virtual objects. As primitive as these stick-hands are, they double the sense of being present. Touch, vision and sound form the essential trinity of VR.

Although Magic Leap has yet to achieve the immersion of the Void, it is still, by far, the most impressive on the visual front - the best at creating the illusion that virtual objects truly exist. The founder of Magic Leap, Rony Abovitz, is the perfect misfit to invent this superpower. As a kid growing up in south Florida, he was enthralled by science fiction and robots. Abovitz gravitated towards robots as a career and got a degree in biomedical engineering from the University of Miami. While he was a student, he started a company that built robots for surgery.

Abovitz is heavy-set, bespectacled and smiley. He hums with ideas. Overflowing. One idea unleashes two more. Most of his ideas seem to combine physics and biology. In his Twitter bio, Abovitz describes himself as a "friend of ­people, animals, and robots", which is pretty accurate. In his conversation and his work he exhibits a rare sensitivity to both the logic of machines and the soul of biology. If you're making robot arms that help human doctors carve into living flesh, you have to obey the laws of physics, the laws of biology and the minds of humans. Abovitz has a knack for all three, and his surgery robots sold well. In 2008, his company, Mako, went public. It was sold in 2013 for $1.65bn.

That success sparked a new idea. Could you make a virtual knee to help repair a real knee? Could you augment a knee operation with an overlay of a virtual knee? Abovitz began thinking about the technology that could match virtual worlds with complex real-life surgery. At the same time he began to create a graphic novel.

Read next

Sensory hacking: perfume-infused dreams and virtual intimacy

ByKatie Collins

Abovitz has a deep love of science fiction, and he invented a whole world on another planet - flying whales, men in dragonfly gear, a young girl with a pet monkey-bat and an invading army of robots. Flush with cash from his robotics company, he hired Weta Workshop, the New Zealand special-effects house co-owned by movie director Peter Jackson, to create a detailed realisation of that world. For Abovitz they designed his world, called Hour Blue, and it quickly mutated from graphic novel into virtual-reality precursor. Because what alien world would not be better experienced in immersive 3D? Abovitz was already pioneering MR for doctors; this would be an extension of his ideas.

Two robots, Click and Clack, test and calibrate prototype hardware

Peter Yang

The company Abovitz set up to develop this immersive world was Magic Leap. Its logo would be his totem animal, the leaping whale. The hardware to create the MR would have to be invented. By this time, 2012, the Oculus Kickstarter campaign had launched and other prototypes with similar phone-based technology were in the works. Here Abovitz deviated off the path. Because of his work in biomedicine, he realised that VR is the most advanced technology in the world where humans are still an integral part of the hardware. To function, VR and MR must use biological circuits as well as silicon chips. The sense of presence you feel in these headsets is created not by the screen but by your neurology. "If you give the mind and body what they want, they'll give you back much more," Abovitz says.

Artificial reality exploits peculiarities in our senses. It effectively hacks the human brain to create what can be called a chain of persuasion. In a movie, our brains perceive real motion in a sequence of still images. In the same way, you can scan a blue whale from many angles and then render it as a 3D volumetric image that can be displayed on a headset screen and viewed from any position. Even if we know the object isn't real, we feel subconsciously that its presence is real.

But if even one small thing is misaligned, that discrepancy can break the illusion of presence. Something as simple as worrying about tripping over a cable can seed our unconsciousness with doubt. It might look like it's there, but it won't feel there.

Following his hunch to exploit human biology, Abovitz set off to make an artificial-reality display in a more symbiont way. The phonelike screens used in the majority of head-mounted displays created a problem: they were placed right next to your eyeballs. If the device is generating the illusion of a whale 30 metres away, your eyes should be focused 30 metres away. But they're not; they're focused on the tiny screen 2cm away. Likewise, when you look at a virtual jellyfish floating 15cm from your face, your eyes are not crossed as they would be in real life but staring ahead. No one is conscious of this optical mismatch, but over long use it could weaken the chain of persuasion. Magic Leap's solution is an optical system that creates the illusion of depth so that your eyes focus far for far things and near for near, and will converge or diverge at the correct distances.

In trying out Magic Leap's prototype, I found that it worked amazingly well close up, which was not true of many of the other mixed- and virtual-reality systems I used. I also found that the transition back to the real world while removing the Magic Leap's optics was effortless, as comfortable as slipping off sunglasses, which I also did not experience in other systems.

Magic Leap's competition is formidable. Microsoft is selling development versions of its MR visor called the HoloLens. The technology is unique in that the entire contraption - processor, optics and battery - is contained in the visor; it is truly untethered. Meta, another startup, has released an MR device that began, like Oculus, with a Kickstarter campaign. The headset is tethered to a computer and dev kits should hit the market this autumn - likely well before Magic Leap.

All three major MR headsets rely on images that are projected edgeways on to a semi-transparent material - usually glass with a coating of nanoscale ridges. The user sees the outside world through the glass, while the virtual elements are projected from a light source at the edge of the glass and then reflected into the user's eyes by the beam-splitting nano-ridges. Magic Leap claims that its device is unique in the way it beams light, though the company declines to explain it further for now.

However Magic Leap works, its advantage is that pixels disappear. Most screen-based, head-mounted VR displays exhibit a faint "screen door" effect that comes from a visible grid of pixels. Magic Leap's virtual images, by contrast, are smooth and incredibly realistic. But in truth, the quality of displays in all alternative-reality gear is improving rapidly. Within two decades, when you look into a virtual-reality display, your eye will be fooled into thinking you're looking through a real window into a real world.

Once this small display perfects realism, it becomes the one display to rule them all. If a near-eye screen offers sufficient resolution, brightness, breadth and colour richness, it can display any number of virtual screens, of any size, inside it. While I was wearing the photonic spectacles of Magic Leap, I watched an HD movie on a virtual movie screen. It looked as bright and crisp as my TV at home. With Microsoft's HoloLens on, I watched a live football game on a virtual screen, alongside a few other virtual screens. I could fill my office with as many screens as I wanted. I could click for a screen overlaid anywhere in the real world.

One of Microsoft's ambitions for the HoloLens is to replace all the screens in a typical office with wearable devices. The company's demos envision workers moving virtual screens around or clicking to be teleported to a 3D conference room with a dozen co-workers in different cities. At Magic Leap, the dev team will soon abandon desktop screens in favour of virtual displays. Meron Gribetz, founder of Meta, says that its new Meta 2 mixed-reality glasses will replace monitors in his company within a year. It's no great leap to imagine such glasses also replacing the small screens we all keep in our pockets. In other words, this is a technology that can upend desktop PCs, laptops and phones. No wonder Apple, Samsung and everyone else is paying attention. This is what disruption on a vast scale looks like.

Peter Jackson agrees. The director strides into a sunny room in his film studio outside Wellington, New Zealand. Jackson says he is less than excited about making movies these days; not the content but the process. He sees artificial reality as virgin territory for telling stories and creating new worlds. Jackson serves on an advisory panel for Magic Leap, and his company will produce content for the new gear. "This mixed reality is not an extension of 3D movies. It's different," he says. "Once you can create the illusion of objects anywhere you want, you create new entertainment opportunities."

Jackson has been inspired by working with prototypes of the Magic Leap glasses. "I find mixed reality much more exciting than VR," he says. "MR doesn't take you out of this world. Instead, it adds elements to our real world. And it has great flexibility. You can add as little as you want - a tiny figure on this table talking to us - or you can replace the walls of this room with a skyscape so we're sitting watching clouds float by. With Magic Leap glasses on, you can look up at the Empire State Building and watch it being built in the early 1930s but sped up. It could be a form of education, entertainment and tourism. In ten years, I expect that MR technology like Magic Leap will be used as much as, if not more than, smartphones."

Weta Workshop's skill is in making imaginary worlds believable by attending to the details. Blockbuster MR and VR worlds will require the highest level of building. The freedom of the audience to move around, to peek at the underside of things, to linger and appreciate the details, means that great effort and skill will be needed to preserve the chain of persuasion for all the things that make up that world.

Weta is working with Magic Leap to develop a virtual world called Dr Grordbort's, based on sculpted ray guns. Leading this effort is Richard Taylor, who has been building worlds, often with Jackson, for nearly 30 years. Taylor has been a sculptor all his life and the move to virtuality is a big step for him. "I was not prepared for the emotional impact of Magic Leap," he says. "I could not have thought I would crave to be in a world with virtual artefacts and characters. But once I got over the surprise that this really works, I've had to rein in my ideas."

The team uses Beam robots to collaborate – for now

Peter Yang

Artificial reality will need world builders like Taylor and Jackson to invent the grammar of VR and MR. It's already clear that the language of experiences is different from what's come before. One example: first-person point of view is the default for many of the video-game franchises dominating bestseller lists. Among them is Minecraft, which is played by more than 100 million people on the screens of PCs, tablets and phones. Inside the game you see your hand or a pick. But in the VR version of Minecraft that Microsoft is building, the experience of holding the pick and chopping the blocks is so immediate that the player's own presence is amplified. Their sense of being shifts inward. In tests with volunteers, Minecraft developers discovered that performing the same role in VR feels more intimate than it does in first-person on a flatscreen. We might call this new immersive VR view the "you-person", because it's the position of feeling rather than observing.

Researchers found that the you-person view that VR creates is so intense that it's emotionally taxing. The degree of presence can be so strong in VR that you have to tone down the evocation of base emotions and the depiction of brute force. The usual gore and mayhem of a first-person shooter doesn't work as well in VR. Exaggerated scenarios that are merely compelling in a flat world can be overwhelming when you're immersed in them.

All that said, it was not the reality of artificial reality that surprised me most. It was how social it is. The best experiences I had in VR or MR involved at least one other person. It's a network effect: the joy of VR is proportional to the square of the number of people sharing it. That means VR will be the most social medium yet. More social than social media is today.

One of my first tests for the quality of virtual reality was something I call the bat-flinch test. If you stood next to someone who was holding a virtual baseball bat and they swung the bat, would you duck? Only if you believed in it. Otherwise you'd just laugh or maybe wait to see what getting hit "felt" like. You'd never wait to get hit in real life. But a better test is the poker-game test. Do the avatars sitting across from you convey sufficient subtle eye contact, body language and social presence that you can tell if they're bluffing?

I visited an Oculus demo at Facebook's campus and Palmer Luckey, Oculus' creator, joined in. We shared a virtual playground. In real life, Luckey is exuberant. He likes to bounce. He pumps his arms as he speaks. That body language crossed over into VR. Even though our avatars did not map our outside visual features, Luckey's avatar moved just like him. He was playfully throwing blocks at me. They passed the bat-flinch test because I was ducking. Luckey was an expert in lighting virtual firecrackers and fireworks and tossing them my way too. Their explosions were real enough that I needed to back away. His enthusiasm was contagious, so I tried to blow him up with a blaster, but I missed and knocked down a tower. Although the physics of this demo, called Toybox, were remarkable, the toys felt real because we could pass them around, share them and collaborate on moving them. My experience was not with toys but with another person.

"Our goal is to make virtual communication even better than real-world communication," Luckey said. "Virtual reality is the only thing that will get us there."

The time is coming when, if someone says, "Let's meet," everyone will know that means let's meet in virtual reality. The default mode of VR is "together".

Very soon, perhaps in five years, the bounded worlds within virtual reality will begin to be networked together into distributed virtual worlds. When you're wearing the visor of an ­augmented- or mixed-reality system such as Magic Leap, HoloLens or Meta, it maps the local environment. To make, say, a virtual teacup appear on your real table, it needs to know where your table is. The visor uses outward-facing cameras and sensors to scan your environment to create this map. Magic Leap (among others) is working on protocols that save a mapped place in the cloud so it doesn't have to be remapped for each encounter. Your unit merely needs to register and update any changes in the space. This in turn will let you share virtual objects across different surroundings, even if participants are in distant places. Someone in Barcelona can drop a virtual flower into your virtual vase in Chicago. Because artificial reality is inherently social, its environments will be inherently social and networked.

That's not to say this will be easy. Don't let the relatively portable size of VR and MR wearables fool you. As they get smaller and lighter, the infrastructure behind them must grow larger and larger. The scale of the servers, bandwidth, processing, storage and cleverness required to run networked virtual places at the scale of the planet for billions of people is beyond big data. It is ginormous data.

Which raises another issue: every virtual world is potentially a total surveillance state. By definition, everything inside a VR or MR world is tracked. After all, the more comprehensively your behaviour is tracked, the better your experience will be.

During a virtual journey, whether it lasts two minutes or two hours, the things your gaze lingers on, the places you visit, how you interact with others could all be captured to customise the experiences to your preferences. But many other uses for this data are obvious. It could be used to sell you things, to compile a history of your interests, to quantify your actions for self-improvement, to personalise the next scene, and so on.

As far as I can tell, there are no VR systems that currently store the data they track or do anything with it. While companies are aware of this potential, they are simply too consumed with getting the virtual worlds to work to bother with exploiting the data feed. Inevitably, however, some will graduate to view this immense trove of personalised data as a commercial treasure. The puzzles of its legal status, who has access to it, what government claims apply and what can be done with it will occupy us as a society in the future. It's easy to imagine a company that succeeds in dominating the VR universe quickly stockpiling intimate data on not just what you and three billion other people favourite but what you do on weekends, what people you pay attention to, whether you're depressed, and so on. To do that in real life would be expensive and intrusive. To do it in VR will be invisible and cheap.

The creation of global artificial reality is an enormous project, and its adoption will start slowly. In every VR demo I tried in the past few months, I needed assistance to get the gear on and adjust the fit. Most demos required spotters to watch me. There were straps to deal with, cords to trip over, furniture to avoid. The software was glitchy. "Right now VR systems, particularly the tracking, don't work without constant technical maintenance," says Jeremy Bailenson, who directs the Virtual Human Interaction Lab at Stanford. "I've been running VR for 20 years, and the bane of my existence is driver updates."

Some of these problems are the growing pains of the prototype phase. But there are also some fundamental features missing. Chris Dixon, a partner in venture capital firm Andreessen Horowitz, who led his company's early investment in Magic Leap, thinks VR will follow the flywheel effect: sluggish to start, its momentum slowly compounding until it's unstoppable.

As the flywheel begins to turn, friction will hinder its rotation. But those friction points are also fresh opportunities. These are problems whose solutions will enable many other innovations. Any of the following pain points might be the opening that produces the first VR billionaire:

The dork factor. There's no getting around the fact that everyone looks like a dork wearing a head-mounted display. The failure of Google Glass was in large part due to the fact that you could not pass the cool test wearing one. The form factors of VR and MR have a way to go before they become culturally invisible.

Safety. I nearly fell in a recent VR journey because I tried to jump into a pit that wasn't really there. Oculus weirdly warns its users to "remain seated at all times". The problem is, if you're present - really present - in an alternative place, you're absent from the place your body is. That's a recipe for accidents. Mixed reality, where the room you're actually in remains visible, can diminish the clumsiness between realms but doesn't eliminate it. Then there is our ignorance of the long-term effects of fooling your mind and body. This is so new we don't even know yet what questions to ask.

Inadequate interface. At this moment in its development, VR is at the same infant stage as early PCs that required a command-line input. There are no intuitive tools for easy creation. The VR industry is waiting for its Doug Engelbart to invent the equivalent of the mouse. This is perhaps the most critical missing piece preventing a rapid take off. Without an interface that anyone can grasp in minutes, content can be made only by the truly dedicated.Nearly all of the non-movie VR experiences uploaded to date were created using a computer-game engine from either Unity or Unreal.

All these first-generation experiences were created with 2D tools - screen, windows, mouse. But VR cannot reach ubiquity until the tools for VR creation live in VR itself. The first steps towards native tools were announced this spring. Both Unity and Unreal have demoed a VR version that permits users to make VR in VR. However, to foster a smooth transition, the VR versions of both creation engines import 2D metaphors (like menus) - the equivalent of a command line - into VR. Still missing is the breakthrough insight that takes advantage of VR's peculiarities to deal with VR's complexities.

I had an aha moment inside a VR app called Tilt Brush that was purchased by Google. I was using a brush to paint with light in three dimensions. I was inside my creation, moving around with my body, working up a sweat. It was the most fun I've had in VR. And it's not just for fun. Trials at Google revealed Tilt Brush could be an ideal prototyping tool. In a few minutes, an untrained person could sketch a design and you would instantly see it. My aha was that, at its root, VR is as much a creation tool as a consumption tool. VR is in line to reduce the barriers to creation even further.

Fame awaits the genius who figures out the elegant VR interface for VR creation. The tools would allow you to manipulate 3D space with minimal gestures, voice and gaze. I suspect there would be a beauty in watching a skilled creator work in VR, much like in watching a woodworker or dancer.

Narrow field of view. Right now the view in MR devices is too narrow. Of the current crop of MR spectacles, Meta 2's field of vision is the widest, but even its coverage is inadequate. Virtual objects that are located in front of you, within the coverage of the screen, appear present. But when you turn your gaze away, they disappear from your peripheral vision. This breaks the chain of persuasion. Fully enclosed VR devices don't suffer the same drawback; because you see nothing in your peripheral vision (only blackness), you don't get contradictory information. Objects disappear when you turn, but the background area does too.

All MR systems labour under a second challenge that VR systems don't: ideally, in a mixed reality, the virtual teacup you see on your desk would be lit in the same way as your real desk. The mismatch in the lighting is another weak link in the chain of persuasion.

Tethers. It's hard to overstate the benefit of a lightweight device that allows you to roam freely. Screens and processors can be made much smaller to fit into glasses, but batteries are the problem. It'll be a long time before a day's worth of power can be squeezed into the frames of glasses. For now they will be wired to a battery in your pocket.

The co-evolution of science fiction and innovation is slowly being recognised as a paramount cultural force. Talk long enough to any engineer working on VR and they will eventually mention the book Snow Crash. Among the first people Abovitz hired at Magic Leap was Neal Stephenson, its author. He wanted Stephenson to be Magic Leap's chief futurist because "he has an engineer's mind fused with that of a great writer". Abovitz wanted him to lead a small team developing new forms of narrative. "We've maxed out what we can do with 2D screens," Stephenson says. "Now it's time to unleash what is possible in 3D, and that means redefining the medium from the ground up." He compared the challenge to crossing a treacherous valley to reach new heights. He admires Abovitz because he is willing to "slog through that valley".

It's too early to know what virtual reality is or what it will be. Abovitz believes that synthetic reality is the ultimate human medium because it is so directly wired to our brains. "Our brain is an amazing sensory computer. Magic Leap is just the tool for a power that people have had brewing in them since people first appeared." One thing we do know: the evolution of technology can take curious turns. Cell phones started out so bulky they needed their own luggage. It was easy to imagine them getting smaller. But they did not merely shrink into miniature versions of themselves. As it got smaller, the mobile phone lost its keypad, gained a hi-res colour screen, started to grow in size again and stopped being used as a phone. It evolved into something unexpected. VR will surprise us too.

Not immediately, but within 15 years, the bulk of our work and play time will touch the virtual to some degree. Systems for delivering these shared virtual experiences will become the largest enterprises we have ever made. Fully immersive VR worlds already generate and consume gigabytes of data per experience. In the next ten years, the scale will increase from gigabytes per minute to terabytes per minute.

Advertisement

The global technology industry will all struggle to handle the demands of this vast system. The bigger the virtual society becomes, the more attractive it is. And the more attractive, the bigger yet it becomes. These artificial-reality winners will become the largest companies in history.

I don't know if Magic Leap will be one of those companies. It's not going to win the race, but none of the current titans were first to their markets. Magic Leap has filed more than 150 patents, but it has not yet publicly demoed a prototype. Most important, we still don't know enough about human perception to know what will work in virtual domains; it'll take more VR to figure that out. We must navigate the treacherous valley before reaching new heights. Yet something has happened. A threshold has been crossed. After a long gestation, VR is good enough to improve quickly. It's real.

Kevin Kelly is senior maverick of WIRED's US edition and the author of The Inevitable, out in June

This article was first published in the June 2016 issue of WIRED magazine