Game development, technology, and Japan.

Playing with the HoloLens

At an extremely spendy price of $3,000 I picked up a HoloLens. Why so much more than a Rift or Vive?

Well, the biggest difference is instead of tethering to your computer, this thing IS a computer+kinect small enough to wear.

Welcome to the world of mixed reality! I hope I’m keeping up with my jargon correctly

Mobile VR is trash, HoloLens is different

The current mobile VR solutions (GearVR, Google Daydream, etc) are garbage, you know why? They can’t track positional movement. Take a step sideways or forward – in the game nothing happens.

This means there is an entire category of games they won’t work with – basically nothing where you move around a room naturally.

HoloLens is different (well, it’s AR/MR and not VR for one, but that’s not the point here), it can fully track position/rotation/acceleration, you can even do something a Vive can’t – you can walk BETWEEN ROOMS and it knows.

Without any complicated setup, you can plop this on someone’s head and it just works, anywhere. Oh, and it’s fast. As for tracking your hand.. well, not so fast, we’ll get into that later.

Akiko knows kung-fu

It uses multiple cameras (normal and infrared) to figure out where your walls, table, and chairs are – and is capable of knowing exactly where your head is located in space.

It’s quick enough to feel nearly as smooth as say, a Vive. I think it’s using acceleration/rotation sensors to do accurate predictions while constantly correcting things with the camera based space-mapping but it works well despite the occasional glitch.

I didn’t expect it to track outside (it must use walls to calculate head position, right?) but it worked fine! I guess it’s scanning the ground or something.

Cool, but here are the problems

At 1268×720 resolution per eye, the objects you’ll see overlaid look great – but that’s mostly because the DPI (dots per inch) are so tiny.

If you hold a piece of A4 paper at arm’s length you’ll get an idea of how small the rendering area is.

The actual screen overlay area is tiny

This means when a spaceship tries to run away in a game, it just gets cut-off unless you turn your head to follow it. A workaround that games are using is they will pop up an arrow “<– It’s over there!” to help you find it again. Not great, but hey.

It also isn’t able to overlay graphics too close, like if you set your GL near plane too far away. You can see the image break apart in the gif below.

Click this gif to view it in action. Don’t blame me if you get sick

Kinect delay is back, baby

It can sort of track one of your hands – but it’s too laggy to be of much use for anything. It’s a better experience to move your head around to put a centered crosshair on an option, then use the included remote’s satisfying clicker to select something.

Random free idea that makes no sense due to the hardware requirements: Use with a Vive hand control to make a spray paint simulator so you can tag up your house.

Seth’s verdict

The good:

The wall mapping and fast, accurate positional tracking is amazing. This is a hard nut to crack and probably the most impressive thing about the entire HoloLens project

It barely works, but it can sort of do smart occlusion around physical objects (if a real chair is in front of the hologram, it won’t render the hologram there) – its 3d scanning is too rough but .. it’s still tantalizing us with what the future will hold

The bad:

Costly, it’s squarely in “developers only” territory right now. It’s no surprise it’s only sold “thousands”

The tiny video overlay area is very limiting

Laggy-ass gesture controls are bad for most gaming

Can’t walk up close enough to objects, they disappear

UE4 doesn’t support HoloLens as an export target, I don’t think there is a way to build it without fighting with Microsoft’s hacked up version of UE4 they did a while back, no thanks

There are rumors that Microsoft has canceled the 2nd generation HoloLens and are skipping straight to a 3rd gen version slated for release in 2019. Interesting.