Manouse -- Webcam mouse

Inspiration

One of our team members wanted to make an alternative mouse input to play osu!, a rhythm game, without a mouse. We figured the webcam would be a fun place to start!

Manouse is a portmanteau of "manus", the Latin word for hand, and "mouse."

What it does

Manouse uses computer vision technology to map hand gestures to mouse control.
We use OpenCV to create an outline of the hand, find it's position, and count the number of fingers. The position and number of fingers data can be mapped to various actions. Currently the position moves the mouse and finger count toggles clicking.

How we built it

We used OpenCV, win32api (to move the mouse), and Python

Challenges we ran into

We had to filter out faulty readings of the number of fingers, and develop robust algorithms and functionalities to compensate for inaccuracies.

Accomplishments that we're proud of

We are all first year Cal students, with the majority of us having little to no programming experience before this Fall.

What we learned

We learned how to use OpenCV to process frames captured by a computer's webcam, and using powerful algorithms, resolve meaningful information from the frames. We also learned how to use Python's win32api to control mouse position, mouse movement, and other core PC functionalities.

What's next for Manouse -- Webcam mouse

We look forward to expanding it's functionality, making a more friendly user interface, and improving it's hand detection and background filtering solutions.

Built With

Try it out

Submitted to

Created by

I worked on using OpenCV to find the most appropriate/efficient method to filter out the background, extract the hand, and count the number of fingers from each of the webcam's frames. This functionality was then sent to our other teammates to implement other features (mouse control, demo game, etc.)

I applied our sensor algorithm to create a basic rock, paper, scissors game. The traditional gestures used in the game were recognized by the algorithm and OpenCV was also used to modify and display victory/defeat images.