Hi all!Does anyone have any experience with coding mouse gestures in Java? I'm think of something along the lines of what you see in Lionhead Studio's Black & White where you can have a reasonably complex shape recognised at any size in any position in the screen. Brief googling doesn't reveal anything much so hopefully someone here has a homebrew they'd like to share or anything at all in the way of advice about writing something like this myself? Thanks in advance!

How complex are the shapes? Some people recognize simple motions in 4 or 8 directions, convert it to a string, then use a regular expression or state machine to recognize certain sequences. ("LULULURD" matches "(LU)+RD") Handwriting recognition is similar type of problem. Many of those libraries use machine learning and use many different factors. There are open source Java libraries for simple gestures recognition and hand writing recognition.

There are medium level complexity algorithms which may be good enough if the shapes all follow a similar pattern or are constrained some how.

Hm, I was after something that could do curves but I suppose that in terms of recognising that an 8 directional implementation could do the trick. I'll also need to check relative lengths of each of the motions in the gesture.

Those gestures will definitely work with the simple method, but you could get false positives and you would need to make sure everything was really different according to the algorithm. Those gestures all have long strokes, so you could use the 8 direction with filtering to ignore very short deviations so jittering would not turn a square into a circle. Or use some other filtering so "88889888" was transformed to "88888888". (See numpad keys.) If you wanted the star to work only if it self-intersected at the correct points I know some techniques, but it's probably overkill.

I also forgot to mention you could try radial menus as an alternative, but that does not seem to be what you wanted.

Let us know how it works, there does not seem to be much online for in game gestures (as opposed non-twitchy to applications like web browsers.)

I've honestly never tried this before, but maybe this is an idea worth mentioning.

You could have the user create a series of points as they attempted to invoke a gesture. Record these points.

Your gestures could be complex geometry shapes. Essentially what you would do is you would construct the area of that gesture using an Area object and a series of Shapes, then take that Area object, and copy it.

Take the copy, scale it down, place it over the original and subtract from the original. Then you have a shape that outlines this gesture with a thickness proportional to however much you scaled the copy.

Now you can see if the user's points are contained by the gesture. Ofcourse, the problem is scaling the gesture shape properly. I haven't quite figured out how you would scale it...

Thanks for the great ideas guys!I guess at this point I should elaborate on what I'm trying to do here, which is using mouse gestures for side-scrolling melee combat, and Black and White was the best example of mouse gestures in a game that came to mind. My reason for this is to try and keep the combat more skill based (as in, the human player's prowess) as opposed to being level/gear based which I figure could make for an interesting multiplayer scenario or even as a single player campaign - also it is theoretically more immersive. It's very much a WIP so I'm trying the gestures as a proof of concept... If it doesn't work I'll try something else but that is for another thread.In any case, sounds like there's definitely some scope here for experimentation and maybe a chance to make an open source complex-gestures library of some sort while I'm at it If there are any further ideas or interesting links do post them here! I'll go away and see what I can come up with.

Having never attempted anything even remotely like this, I'd go with something like:

- express each target gesture as a sequence of joining angles, and relative line distances (so the overall scale of what is drawn doesn't matter).- record the points as a path- apply a flatten filter to the recorded path- compare the relative line distances and joining angles of the flattened path to each of your target gestures, applying some kind of fuzzy fitness algorithm.- tune the parameters of the path flattening, the fitness algorithm, and the design of your gestures until it works to your satisfaction.

Though as I said, I've no experience in the matter, so this approach could be completely wrong.

This isn't a java specific question so don't let that limit your searches. One technique I'm familiar with, which is straightforward and great for shape recognition, is by chaining balls together that move in 8 directions. You can vary the frequency of the balls and the radius of balls and/or zoom in/out the whole mesh to get the amount of accuracy you need.

Thanks for the link Nate, I'll check it out.It sounds like there are a few techniques out there - really depends on what sort of gestures need to be recognised. It may not be a one-fits-all solution if I want to keep it super efficient (kinda necessary for live combat). I'll try to find the time over the next few nights to mock something up and post it here!

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org