Leap Mesh Painter Demo

A while back I posted a video on youtube of me painting on a 3D mesh inside a webbrowser using a Leap Motion. This was just a quick demo I had made "for the fun of it". However, I got great reactions on that video. Even Leap Motion contacted me to ask whether they could add the demo to their leapjs example page. Of course they can! So today I release Leap Mesh Painter.

First, let's go back to where it all began. With this video:

The video shows me, interacting with a model of a female pelvis (no giggling please) using a Leap Motion. The most notable type of interaction is the drawing on the mesh. I will release a post soon where I describe in detail how this drawing works. It has changed drastically since the first version.

Actually, pretty much everything about the demo has changed. I plan to write more posts on it, explaining several components in detail. The first I published a week ago about the Leap Motion Colorpicker. The biggest change was to incorporate works of others. I'd like to highlight threeleapcontrols. I found this amazingly helpful library a while back. Basically it contains two plug-and-play controller for three.js to use with a Leap Motion. One for camera interaction and one to interact with a model. Setting it up is super easy and it allows you to use the Leap's input for other things as well. For example, the zoom, pan and rotation interactions are all handled by threeleapcontrols, while the painting and gestures are custom built.

Also, the model has changed due to licencing. The current model is a male pelvs. However, you can use your own models as well. If you have a model encoded in three.js json format you can simply drag that file onto the droparea in the lower right and Leap Mesh Painter loads your model without textures or colors so you can paint on it freely. You can download your creation as a png image by clicking saveAsPNG in the control panel.

I've made a video that explains how everything works:

If you don't have a Leap Motion you can still use the demo. Simply select useMouseControl in the control panel on the right and follow instruction on the left.

Without further ado, click the link below to try it out yourself. Let me know what you think. Please remember that this is a proof of concept, so things might not work optimal in every case. Let me know if you have any problems. Also, if you'd like me to explain certain detail, leave your request in the comments.

Comments

Wow!
The end result is great. I didn't knew about three.js json format before.

#2 | suvir | Tuesday, February 11, 2014 7:22 PM

How to convert model to three.js json format?
Is there any .stl to three.js json format converter?

#3 | SeeDoubleYou | Thursday, February 27, 2014 6:28 PM

There are several ways to convert models to the json format. Take a look at the Utils section on the three.js Github page: github.com/mrdoob/three.js/tree/master/utils. There are exporters for blender, maya and max as well as some converters.

Thanks for sharing your amazing work. I'm however having a problem. The page is loading endlessly. I've tried different browsers. Also, is there a way to get it as a desktop application? The app would be very useful to medical students, I'd very much appreciate your help.