Examining the HoloLens hardware

- Let me introduce you to the HoloLens.As you can see, it resembles a somewhatheavy pair of ski goggles.You may have used other virtual reality devices beforein which case you'll notice one of the biggest differences.You can actually look through the displayand see the outside world.It's translucent with a bit of a shade on it.Let me talk you through the physical parts.First of all, you'll notice these cameras.There are two on this side and two on this.These cameras are there to tell the deviceof any movement the user has made.

It does so by taking snapshots of the environmentand comparing those with previous-made snapshots.By doing all sorts of calculations,it can figure out if and how you moveand adjust the image the device is showing you accordingly.The net result is that when you move to one side,the image appears to stay in the same place.Now, this is where the display is projected.However, it's very hard to see from the outside.Other people won't be able to see what you see,we have to look at the inside of the device.

There's a light engine here at the top of the visorthat beams the image to these two displays.Now, since there are two displays, one for each eye,we can project a stereoscopic or three dimensional image.The sensors and the displays togethermake it so we can achieve true 3D.What do I mean by that?Well, imagine you watch a 3D movie in a theater.At some point, something is sticking outof the screen towards you.

If you move your head, the object sticking outmoves with you.Now, this makes sense in a movie theater.Every audience member needs to havethe same kind of experience.It would be weird if people sittingin the middle of the theater see the object straight aheadwhile the people sitting near the aislessee it from the side.But this is exactly how it would look likeif there was actually something sticking out of the screen.People sitting at the edges would seethe object from the side.Now, your mind recognizes this.

It sees the effect, but it also noticesthat it's moving with you as you move your head,so your brain will reject it.It will probably like the experience,but it will not really accept it as a real 3D experience.The HoloLens is different.You can watch something going towards you in 3D,but when you step to the side,it will go passed you instead of at you.This time, your brain will tell you,"Hey, this is the real thing!"The result is that your body will reactto objects as if they are real.

These two sensors form the depth sensor.This is an infrared light emitting deviceand a sensor that reads reflections off this infrared light.And by measuring the time the light takesto return to the device, HoloLens can calculatehow far away objects are.We can use this to determine where objects arein the real world.Now, it takes about 0.2 seconds to do a scan,so it does a refresh five times per second.This isn't enough to track moving objects,for instance, people walking by,but it's very useful for detecting floors, walls,ceilings, tables, and so on.

Now, the HoloLens also has got a regular webcam right here.It's more or less hidden behind the visorso you can't see it, but we'll use this later onto stream videos and take pictures.These are the spatial sound emitters.This is what enables the device to let us hear things,and the nice thing is that it is in stereo sound,it's spatial sound, and we can use thisto play sounds in a 3D world.You can actually hear where the sounds are coming from,and you can find objects in your virtual world.

Now the speakers are placed just above your ears.This is done so it doesn't block out the real world sounds.You're still able to hear what's going on,but you get additional sounds.Just like the visor where you can see the real world,but you also have additional images added.Now, in the back, you see a USB port.This is used for charging the deviceand also for uploading your apps as you write them.And that's it, you can't connect a 3D printer for instance.

On the other side, you see a 3.5mm audio jack.You can plug in headphones here if you want to,but I haven't met anyone who actually does that yet.With headphones, you block out the real world soundswhich goes against the principles of the device.Now, the batteries are located here and here,three on each side, so it makes a nicely balanced device.The device is Bluetooth and Wi-Fi on boardwhich we can use to connect to all the devices and networks.Be warned though, the Bluetooth only workswith human input devices such as your keyboardor an Xbox controller.

Again, don't try to hook up a 3D printer to the HoloLens,it won't work.Last but not least, behind this piece here is the computer.Yes, it does contain a fully self-containedWindows 10 device that has a CPU, a GPU, memory, and SSD.It also has something called a HPUwhich stands for holographic processing unit.This custom-made chip is what takes all the datacoming from the sensors and translates thatinto events for the CPU.

This is the big secret.This HPU makes the HoloLens workwithout being connected to an external device.And there you have it!The HoloLens hardware in detail.Now, it's time to investigatewhat we can actually do with it.

Resume Transcript Auto-Scroll

Author

Released

9/22/2017

Augmented reality (AR) is the next big frontier for app developers. The HoloLens is one of the first commercial devices available in AR. Luckily for developers, Visual Studio has built-in support for HoloLens, and the concepts and syntax are similar to developing for other Microsoft platforms. With Windows 10 Pro and a decent graphics card, you can start developing for mixed reality today.

Join instructors Dennis Vroegop and James Ashley for this introduction for HoloLens app development. They show you how to set up your development environment, including Unity and the HoloToolkit; how to deploy to HoloLens and the HoloLens emulator; and how to build apps that accept user input via gazes, tapping, and speech. Then discover how to use spatial mapping to detect your surroundings and analyze your environment with spatial understanding. Learn how to implement spatial sound that adds to the user's existing environment, and synchronize data to create shared experiences. Along the way, learn how it all comes together in two simple 3D apps: a visualization for enterprise business and a platform game.