Interaction Tutorial

Prosper Van
- August 31, 2015 11:56

Basic Concept

Ubi uses Microsoft Kinect,a depth sensor, to scan your finger and hand movement in front of the display. The interaction paradigm is the same as touch screen: when you touch the projected display with your finger or hand, you will trigger a touch down event. When you move your finger or hand across the display and keep them in contact with the display, you will trigger a touch move event (drag). When your finger or hand leaves the display, you will trigger a touch up event. By combining these three basic gestures- touch down, move and touch up- you can perform the standard touch gestures in Windows 8: such as Tap, Press and hold, Slide, Swipe, Turn, Pinch and Stretch, to control any application on Windows 8.

How do I trigger a touch down event?

A touch down event is triggered when your finger (Kinect v2) or hand (Kinect v1) is in contact with or is in close proximity of the display. When your finger or hand is approximately 1-3 inches away from the screen, you will trigger a touch down event.

How do I trigger a drag event?

To trigger a drag/pan, you just need to keep your finger (Kinect v2) or hand (Kinect v1) in contact with the display and move anywhere within the display. Note that if your finger or hand is moving too fast or is blocked by your body, Ubi will fire up a touch up event as the pointer is missing. So please try to make your finger and hand visible to Kinect all the time.

How to tap a object?

A tap gesture is complete when your hand enters and exists the interaction zone. You just tap the object as if you were tapping a touch screen.

Note: Try to tap at different speed to see what speed works the best for you. If you always trigger drag instead of tap, try to leave the display perpendicular to the display.

How do I pinch or zoom?

You can pinch and stretch some objects by moving fingers on two hands (Kinect v2) or two hands (Kinect v1) together or away.

Windows 8 introduces edge gestures to trigger application and system commands. You can swipe in from outside of the display from four edges (bottom, top, left and right) to trigger the edge gestures to bring up application menu, switch between applications, bring up charm bar or even close the application. Moving slower near the edges will help Ubi detect your gestures better.

How do I activate the Windows 8 touch keyboard?

If you want to open the touch keyboard manually (without tapping in a text field or other area where you can type), here's how: