Concepts

To get started controlling the virtual input to the HoloLens emulator and Windows Mixed Reality simulator, you should first understand a few concepts.

Motion is controlled with both rotation and translation (movement) along three axes.

Yaw: Turn left or right.

Pitch: Turn up or down.

Roll: Roll side-to-side.

X: Move left or right.

Y: Move up or down.

Z: Move forward or backward.

Gesture and motion controller input are mapped closely to how they physical devices:

Action: This simulates the action of pressing the forefinger to the thumb or pulling the action button on a controller. For example, the Action input can be used to simulate the air-tap gesture, to scroll through content, and to press-and-hold.

Bloom or Home: The HoloLens bloom gesture or a controller's Home button is used to return to the shell and to perform system actions.

You can also control the state of simulated sensor input:

Reset: This will return all simulated sensors to their default values.

Tracking: Cycles through the positional tracking modes. This includes:

Default: The OS chooses the best tracking mode based upon the requests made of the system.

Orientation: Forces Orientation-only tracking, regardless of the requests made of the system.

Positional: Forces Positional tracking, regardless of the requests made of the system.

Types of input

The following table shows how each type of input maps to the keyboard, mouse, and Xbox controller. Each type has a different mapping depending on the input control mode; more information on input control modes is provided later in this document.

Keyboard

Mouse

Xbox controller

Yaw

Left / right arrows

Drag Left / Right

Right thumbstick left / right

Pitch

Up / down arrows

Drag up / down

Right thumbstick up / down

Roll

Q / E

DPad left / right

X

A / D

Left thumbstick left / right

Y

Page up / page down

DPad up / down

Z

W / S

Left thumbstick up / down

Action

Enter or space

Right button

A button or either trigger

Bloom

F2 or Windows key (Windows key only works with the HoloLens emulator)

B button

Controller grip button

G (Windows Mixed Reality simulator-only)

Controller menu button

M (Windows Mixed Reality simulator-only)

Controller touchpad touch

U (Windows Mixed Reality simulator-only)

Controller touchpad press

P (Windows Mixed Reality simulator-only)

Reset

Escape key

Start button

Tracking

T or F3

X button

Input control modes

The emulator can be controlled in multiple modes, which impact how the controls are interpreted. The input modes are:

Default mode: The default mode combines the most common operations for ease of use. This is the most commonly used mode.

Hands or controller mode: The HoloLens emulator simulates gesture input with hands, while the Windows Mixed Reality simulator simulates tracked controllers. To enter this mode, press and hold an alt key on the keyboard: use left alt for the left hand/controller, and/or use right alt for the right hand/controller. You can also press and hold a shoulder button on the Xbox controller to enter this mode: press the left shoulder for the left hand/controller, and/or press the right shoulder for the right hand/controller.

Hands are typically not visible to the HoloLens emulator - they are made visible briefly when performing gestures such as air-tap and bloom using the default input mode. This is a difference from tracked controllers in the Mixed Reality simulator. The corresponding Hand is also made visible when you enter hands mode, or when you click "Turn On" in the Simulation tab, which is located in the Additional Tools pane. * Head mode: The head mode applies controls, where appropriate, exclusively to the head. To enter head mode, press and hold the H key on the keyboard.

The following table shows how each input mode maps each type of input:

Default

Hand/controller (Hold alt / shoulder)

Head (Hold H)

Yaw

Turn body left / right

Move hand left / right

Turn head left / right

Pitch

Turn head up / down

Move hand up / down

Turn head Up / down

Roll

Roll head left / right

Roll head left / right

X

Slide body left / right

Move hand/controller left / right

Turn head left / right

Y

Move body up / down

Move hand/controller up / down

Turn head up / down

Z

Move body forward / backward

Move hand/controller forward / backward

Turn head up / down

Action

Perform action

Perform action

Bloom / Home

Perform bloom gesture or Home button press

Perform bloom gesture or Home button press

Reset

Reset to defaults

Reset to defaults

Reset to defaults

Tracking

Cycle tracking

Cycle tracking

Cycle tracking

Controlling an app

This article has described the complete set of input types and input modes that are available in the HoloLens emulator and Windows Mixed Reality simulator. The following set of controls is suggested for day-to-day use: