Introduction

With the release of Windows 7, we now have native support for multi-touch in our applications. As long as the hardware can register multiple, simultaneous touches, Windows 7 can handle it. In addition, .NET 4.0 and WPF 4.0 now have built-in support for multi-touch coding. This article will show you how to utilize these features. I wrote a multi-touch robot control app to demonstrate the new features. This was written using Visual Studio 2010 beta 2 and WPF 4.0.

Background

When Visual Studio 2010 beta 2 was first made available, I downloaded it right away and installed it. Then I wondered, what's new that I can do with this? Of course, multi-touch immediately came to mind. I'm excited about the new support for multi-touch. We are just scratching the surface (heh heh) for new app interfaces using multi-touch. I think there are going to be many cool, new user experiences created by developers like you.

Slider Control

For starters, I needed a slider control, so I created a WPF user control called SliderControl.XAML. I gave it a nice, rounded border and a linear gradient fill. Then, I added an ellipse as the control knob for the user to interact with.

TouchDown, TouchUp, and TouchMove Events

The TouchDown event is fired each time your finger initially touches the screen. In TouchEventArgs, you will get a TouchDevice object that contains an ID that the multi-touch hardware assigns to the current touch. You can use this ID in subsequent code to track the touch. However, there are other methods that can track this for you by capturing the touch.

The TouchUp event is fired each time you lift your finger off of the screen. You will get another TouchDevice object so that you can release any capture you have on the current touch.

The TouchMove event is fired many times as you drag your fingers across the screen. A separate event will fire for each finger movement on the screen. You cannot guarantee any order in which the events will arrive. This is the reason that TouchDevice.ID is important.

To detect and capture touch events, we need to first hook the event handlers in the constructor of SliderControl. You will notice that we hook the TouchDown and TouchUp events of the user control, but hook the TouchMove event of the ellipse. This is because the ellipse will eventually capture the TouchMove events and process them.

Once a TouchDown event fires, we want to determine if it occurred over the control knob, in this case, the ellipse. If the TouchDevice.DirectlyOver object returned is the ellipse, then we will capture the touch as long as the ellipse hasn't already captured another touch.

After a TouchDevice has been captured by the ellipse, the TouchMove event handler will receive all the moved data from that touch. So, we simply get the current position, do whatever calculations we want, and wait for the next TouchMove event. I use the position data to set the Top.Property of the ellipse. I also added code to limit how far it can travel on the screen, calculate a normalized value based on the position, and fire an event called ValueChanged.

After we lift one of our fingers off the screen, we will get a TouchUp event. Here, we will determine if that TouchDevice is captured by the ellipse. If so, then release it, center the ellipse on the slider, and send another ValueChanged event.

Using the Code

The whole reason for this app was to control a Lego NXT robot using multi-touch. I wanted two slider controls on the screen that could simultaneously control each motor independently. So, each slider control is wired up to fire a ValueChanged event, and the main app will send serial commands to the NXT robot to move each motor forward or backward depending on the position of the slider control.

If you don't want to control a robot, but you still want multi-touch slider controls for your application, then it's easy enough to pull out the SliderControl.XAML user control and drop it in your own app. You can hook the ValueChanged event and away you go. I did add a throttling mechanism in the SliderControl so that it doesn't send back values too quickly for the NXT robot to process. That's easy enough to remove if you don't want it.

Devices in Windows 7

Windows 7 has made it very easy to connect up devices to your laptop. In this case, I connected a Lego NXT brick using Bluetooth. It was very simple. First, turn on the NXT brick and set it to be visible over Bluetooth. Then, open Devices and Printers in Windows 7. It's located below the Control Panel in the Start menu.

You should see your NXT available in Devices once you type in the correct pass-code. The default for NXT is '1234'.

You can double-click on the NXT icon and see the services that it makes available. In this case, it shows up as a serial port. The app uses the SerialPort class to talk with the NXT robot. You can find all the documentation and direct commands on the Lego NXT website. For this application, the serial port for the specific NXT you want to control is set in the App.Config file. Simply change the COM value to whatever your NXT usesm and run the app.

Conclusion

This article showed you how to easily access the multi-touch capabilities of multi-touch enabled laptops, using Visual Studio 2010 beta 2 and WPF 4.0. I hope that in the future, hardware manufacturers will ship laptops with more simultaneous touch points. I believe that there are still many great multi-touch .NET apps waiting to be written. We need the hardware to keep up with our imaginations.

In the future, I plan on adding more features to this app for even more multi-touch robot control fun.

Comments and Discussions

The robot is built using Lego Mindstorms. The controller is the Lego NXT brick with bluetooth. Very easy to pair with the unit and send commands using the Serial class in .NET.

Currently you can find Lego Mindstorms NXT 2.0 for about $260. It includes sensors and actuators. You get ultrasonic, bump switches, infrared, etc. Cool product for the robot hobbyist. I bought mine at Amazon.

I plan on making my own robot from scratch but need to find the time to do that.

Where do I find the Nxt Library? Did you make this yourself or is this something I can download?

This got me excited because I host Lego Robotics class through 4H and we are just now getting a group of kids that are ready to do some advanced stuff. I'm a C# programmer so I'd prefer to do advanced programs using this tool instead of VPL or the Lego software.

I did roll my own for this article. You're more than welcome to pick it apart and use it for your projects. It's pretty easy. Lego makes all the docs on their communication protocols freely available. You can find the Bluetooth Dev Kit with the docs at : http://mindstorms.lego.com/en-us/support/files/default.aspx[^]

There was an NXT.NET library out on CodePlex but they don't provide source code.

Let's evaluate that again in a couple decades, when robots rule the earth and we are their fluffy meat puppets who get mind-controlled for hunting amusement... *beep* Careful, don't let him walk off the cliff! *beep-beep*