Using canvas and multi touch

09 August 2010

I'm excited about the possibilities that touch screens and multi-touch screens will give us. I'm also interested in HTML 5 and making something that will work accross a variety of devices whether they have a touch screen or just a mouse. To play about with both I'm currently making a colouring book app that will work on an iPad, a desktop computer's browser and hopefully any other touch enabled phone browser.

<Canvas> is a new HTML tag that is part of HTML 5. It acts like a drawing surface that you draw on using JavaScript commands.

In JavaScript you can catch mouse and touch events. At first I thought that an iPad would issue mouse events as well as touch events, but it doesn't. Touch and mouse events are slightly different, but when handled correctly you can make a good interface that works with both.

This is just a simple example that responds to both touch and mouse events to draw a line on the HTML 5 canvas screen. It will work on an iPad, iPhone (and hopefully Android and other touch phones) using touch and touches. It will also work in any modern browser that supports the <canvas> tag.

[Your browser can not show this example.]

To try it out click on the box above and while the mouse is still pressed drag it about. Or, if you have a touch device try touching it with one or more fingers.

The code

It's a lot like an image tag, except it needs the closing tag. The content enclosed between the two canvas tags will only be seen by browsers that don't support canvas, like Internet Explorer 8. The position relative bit is needed to make capturing the mouse position work across different browsers.

The JavaScript for this was a bit more complicated. Drawing to a canvas element is very easy, capturing mouse and touch events was a bit more difficult, but the most difficult bit was working out where the mouse was in a way that worked on all the different browsers.

It's quite easy really. The main things that caught me out were that a line will not appear on screen until you add a stroke to it. And, if you do not close and start a new path the stroke will get applied again to all lines since beginPath was called.

Capturing mouse and multitouch events

I found this article about touching and gesturing on the iPhone very helpful in getting this to work. I couldn't have figured out multi touch events without it. I wanted my example to work with a mouse as well as with touch and this took a little bit more work.

Working out the cursor position relative to the element it clicked on

Getting this to work in all the browsers I tested made me very angry. It really shouldn't have been this hard. There are a few JavaScript libraries that make this easier, but it didn't seem worth it just to get this one simple thing to work.

It still needs some more testing. I've tested it in the latest versions of Firefox, Safari (desktop/iPhone/iPad), Chrome. I've not tested it in IE 8 because this example already needs canvas which IE8 doesn't support. I've got high hopes for IE 9 though.

These little code snippets should give you some idea of how the whole thing works but you really need to refer to the full JavaScript Source to see how it works fully.

After testing,it can not work on the Chrome(12.0.742.112)/Safari(5.0.5)/Firefox(5.0, our test browser is all for desktop. Can you tell me if we need to do some configuration to run the demo successfully?

There seems to be a problem with your coordinate offset detection, its offsetting the offset... Lol...

So the only point which is accurate is the 0,0 coordinate of the element, extreme top left... Everything else is offset by itself, so the coordinate 20,30 is input by the user, but the canvas draws at coordinate 40,60...

Hi, thanks for the great script. It's the best cross platform touch demonstration I have seen. One question tho, I am having trouble detecting a single 'touchup' event. I can see it fire from 2 fingers, when one is lifted, but not one finger when it's lifted. Is there a way to have the equivalent of a mouseup fired for a single touch?

I noticed that when you make the stroke size larger, the stroke is made of rectangles so if the line becomes very choppy. Is there some way to change the stroke to circles (or something else) so the line is smoother at larger widths?

It depends how you've zoomed. You'll probably need to scale your coordinates by the amount you've zoomed and also translate them if the zoomed in section isn't central. It might take a bit of trial and error, but I'm sure you'll figure it out.