Microsoft Hololens – the augmented reality headset that can turn your living room into Minecraft. But are developers impressed?
Photograph: Microsoft

When Microsoft unveiled Windows 10 last Wednesday, the company had something else to show the assembled press – just in case a new operating system wasn’t exciting enough. This was the Hololens, a stand-alone augmented reality heardset, capable of projecting computer generated objected into your real-world environment. Look through its futuristic screen, and you may see a dinosaur on your coffee table, or a monster on your sofa.

The device seemingly uses an array of motion senses, as well as a Kinect-style camera so that it knows where the wearer is in the room, and can also recognise objects. Imagine, for example, a horror game where a zombie could crash through your own living room window. Add in both gesture and voice controls, and you have both a compelling alternative to fully immersive virtual reality headsets like Oculus Rift and Sony’s Project Morpheus, and the Magic Leap augmented reality headset that counts Google among its many excited investors.

So what do seasoned virtual reality developers make of the new technology? We asked a selection of experienced VR experts about this new AR interloper – and the future of a platform that promises to bring the world – or even the universe – into your living room.

For a prototype it certainly seems like they’ve done a very good job on producing the illusion of a solid object appearing before you — and that’s really exciting. It’s one of the things that good AR systems need to be able to do well, and has been very hard to achieve.

We see Hololens as a benefit for the VR industry — if only because there’s so much crossover in the technologies that power augmented reality and virtual reality systems. Both have to deal with similar performance and design concerns in terms of user experience and interface. An easy to understand example is how users want to be able to reach out and interact with objects like you would in the real world. To do that you need a specialised combination of hardware and software that tracks the user’s hands and understand how virtual objects should behave when they’re interacted with. Natural user input is one of the key areas where consumer virtual reality systems need to improve, so hopefully we’re going see faster advances as more hardware manufacturers enter this space.

Nick Pittom, VR developer, FirePanda

It seems like a natural extension of Microsoft’s Kinect research in many ways: sort of like sticking a Kinect on your head – but you are not tracked, your room is. If they can also solve the “transparency” issue (images appearing ghostly rather than solid) and get a bigger field of view then it could really be quite special. By all accounts the tracking is excellent.

There’s a lot of potential to deliver “turn your living room into Mario World” type AR games, where characters run about on your sofa, or cars race on your desk, or mini planes dog fight in the air. Table-top experiences where players can lean into them to get a different perspective could also be exciting. Sim City as a mini-city on your living room floor is an enticing thought. Microsoft will need to learn the lessons of Kinect and ensure the software is there to justify the existence of the hardware. Skype and Minecraft will only get them so far.

Katie Goode, developer, Triangular Pixels

The lack of a controller has its pros and cons. In an ideal world, players will not only be able to pull off all the organic interactions they want to do, but also feel a feedback response. The Hololens concepts I’ve seen so far are missing any sort of touch feedback, so developers will have to work especially hard compensating with visual and audio clues. Also, users with movement difficulties may struggle with applications that only use just gesture control.

Turning any room into a virtual environment will be incredible. I can’t wait to be able to use the device as my virtual projector, have my e-wallpaper update with movies and photos, or play games where my favourite characters stand full height inside my room. AR interactive stories will start to become a thing, where you can sit in your living room and have characters act out scenes in front of you – your own personal theatre production. And AR board games! Having pieces come to life will be magical.

There’s a lot to think about though. How many different room shapes are there in the world? The player may not have the space to move at all. Different real world lighting, materials, colours can affect the art style of the game. What if the players have guests or pets? A lot more variables come into play and it’s going to be a challenge just to think of them all, as well as come up for solutions for them.

Augmented reality is less imposing that virtual reality: and I think VR is harder to do in the living room; it’s more vague how it will reach consumers. And my mom still thinks VR is scary. Microsoft is probably trying to reach the audience Nintendo left behind when it went from the Wii to the Wii U.

The biggest plus AR has, is that it can be a social experience. Imagine a dad playing Minecraft with his kids – with AR, he’d be able to interact directly with them and help them to build a castle. You could have a game of Dungeons and Dragons where your environment changes before your eyes on the whim of the Game Master. You could go on a quest around your home to find pokemon, or play a real-time-strategy game where each player sees a “fog of war” on their opponent’s side of the room. The possibilities are endless.

Henrique Olifiers, co-founder, Bossa Studios

Microsoft got it spot on with the form factor. Moving the processing into the HMD itself, making it wireless, is something we’ve been keen to see introduced into VR for some time now. Finding design and technical solutions to deal with the weight, heat and power issues related to this approach is a big challenge, so it’s very exciting to see they’re tackling it with Hololens.

VR is extremely processing intensive, requiring a lot of CPU and GPU power. AR, on the other hand, can be done with lighter hardware. So if you’re going for a wireless, self-contained HMD, AR is the best option with the current technology. On the surface, AR also enables more social interactions as illustrated by the Hololens Skype demos, while VR’s strengths lie in full immersion at the expense of the reality around you. From a content point of view, AR also makes it much easier to deal with issues such as motion sickness by keeping real reference points in view, so developers may enjoy more freedom as creators.

It seems like Microsoft has cracked one of the key challenges with augmented reality by providing a headset that understands the user’s environment.

I’m very much of the mindset that VR and AR will complement each other beautifully. Seeing the recently-revealed Leap Motion quick-switch, which allows users to switch between VR and reality, had my imagination running wild, and not being tethered to a chair is clearly useful.

In terms of device specifics, I’m really interested in the holographic display itself, and would love to know a little more about what kind of technology it’s using. I’m sure details on the GPU, CPU, etc, are a long way away.

There’s huge potential for gaming, engineering, planners, architects and a ton of other industries. Opposable Games is already working with engineering companies on augmented reality so we can see the benefits it has to offer first-hand.

I predict we’ll see a swathe of social gaming experiences ranging from animated 3D board games, to truly interactive Warhammer fantasy games, moba titles and, what I’m personally looking forward to the most: strategy titles like Command and Conquer or Supreme Commander. Directing tiny armies and spaceships around your living room is going to be tremendous fun.