Friday, December 21, 2007

A lot of very smart people have written in questions and suggestions and I think a lot of them are very clever and worth sharing... and I'd like to add some of my own expansions on what I showed in the video. Like they say, 2 brains are better than 1.... it turns out several thousand brains are even better. This post will be REGULARLY EDITTED to add new ideas as they come in. I know it breaks the RSS model, but I don't want to make a new post for every new idea. Sorry.------------------------------------My Bluetooth adapter and LEDs - Though it is reported to work with many bluetooth adapters and chipsets, I've been using this one sold by SparkFun electronics. The IR LEDs I use are Vishay TSAL6400s running at 100mA available from Mouser.

Tip switch - Instead of pressing the side button to illuminate the LED, a tip swtich could be added to illuminate when pressed on a surface. Though, it's mechanically harder to make and finding the right level of pressure for activation can be hard, especially if your surface varies (ex: whiteboard vs carpet)

Can you do this with a webcam? - The wii remote contains a 1024x768 camera with built-in hardware for IR blob tracking of upto 4 points at 100Hz. This significantly outperforms any $40 webcam I'm aware of. It'll work with a webcam, just not as well and not as easily.

Rear-Projection - using a rear projected surface, works great. The software will automatically work as-is so long as your pen is bright enough to shine through the projection surface. The wiimote gets a unobstructed view of the screen which eliminates occlusion problems and maximizes tracking resolution. Though, actually building a rear-projection wall or table can be a little difficult if you don't already have one.

Pressure sensitivity - wacom tablets give nice pressure sensitive behavior. Doing this with the wiimote whiteboard will be tough, at least through the camera data. The brightness of the LED varies significantly depending on angle, surface reflectivity, and distance. Any pressure data based off birghtness will not be reliable. The data could be transmitted via a high-frequency IR channel (like remote contol data) to an IR reciever mounted on the wiimote. (see "Right click" for more)

Right Click - any right click signal based off brightness is not likely going to be reliable. Additionally, using multiple LEDs (1 - left click, 2- right click) won't work at all distances and orientations as the LEDs get too close for the wiimote to distinguish them. The RIGHT way to do this is to use high-frequency IR data transmission (like remote control data) to an IR reciever. This will also give you a way of potentially getting pen ID - which is VERY powerful and may be a future project. Though, this requires quite a bit more hardware.

Multiple wiimotes for better tracking - if you have a large surface, you could have multiple wiimotes covering different areas to keep tracking reslution high, or have the wiimotes cover the same area to avoid occlusion problems, or do some interpolation of the data for higher accuracy. All possible, just with the smarter software. Doing this myself is probably not going to happen, but the source is freely available.

257 comments:

If you want a touch screen then yes you do have to touch it (or you could just move the pen really close to the screen and just light up the IR LED). Basically you can map any surface to act as the area where you can manipulate your mouse cursor (think Wacom tablets).

As for using webcams, I think you would have to replace the IR filter in one with a normal light filter. This is how it's usually done with DIY multi-touch surfaces.

I wonder if you could configure you pen such that it flashed the ir light in a reliable manner depending on which button pressed (for right click or left click) and have the software translate flahses within a certain time frame as identification rather than more clicks

Maybe the IR LED should just move the cursor, and separate buttons on the pen would act as click button; of course this would require some mouse guts. Imagine if you used a laser instead of a LED; you'd get a pointer mouse instead of a touchscreen.

Also, perhaps a well structured and moderated wiki would work better than updating a post when new ideas need to be added. The special category of recent changes in the wiki would easily allow people to see when new things are added.

Just some thoughts:Using two Wii Controllers you can have stereoscopic tracking in 3d space. Two eyes = depth of field.

On losing finger track:To resynchronize finger tracking how about a resync motion - for instance bring all four fingers together as one point, first finger to go...upper right = right index fingerlower right = right thumbupper left = left index fingerlower left = left thumb

About the right click - Wouldn't it be possible to modulate the IR signal quite easily? by using a small microcontroller and some switching it should be possible to implement almost any sequence into the IR pen. It would be quite nice to have, say, 2 buttons on the pen to send 'a modulated sequence blob' to the wiimote..

Instead of using a pen, why not put a bit of reflector on you fingernail and use the IR emitter array from the multi-touch project. This gives you the mouse tracking, but you still need some way to do mouse clicks. Maybe by bending your finger so the reflector disappears?

Nice work, I'm looking forward to create an IR pen to try that myself :)I'm interested in how this is working and looked at the code. The important part is the warp matrix construction done using the computeSquareToQuad and computeQuadToSquare functions, but I don't understand them. Can you do an explanation or give some references about that?

I tryed it today with my laptop and my wiimote. It is not acurate enough. Maybe because I was using an IR LED from a TV remote? And my swithch was a little wierd. I just didn't work as it shoud. I will go to electronic store and buy new LEDs and switch.What is the correct type for IR LEDs?

I found this amazing, and had to try, mounted a ir pen, it worked but when I draw ie Paint the drawings are very noisy and squared, not round and soft like in the video, tried many angles and distances and cant solve, what I'm doing wrong? Can my wiimote bad? thanks. (PS: im using a CRT monitor)

Expo pens are PERFECT for wiipens. AA batteries slide right in and the IR LEDs also fit very well out of the tip, but need a little cushioning for snugness. However, the end of an Expo parker is hard to get off, but you can probably replace that with a different cap (cut top of plasstic bottle off, glue onto the end of marker and you're done).

i figured out one of the reasons that the remote can have bad tracking sometimes. the wiimote is sensitive enough that it will actually pick up the light emitted directly from the led AND the light reflected off of the screen. i noticed that whenever the remote was very jumpy it was switching between seeing one and two points. i found that using a flat surface, in my case a wooden cabinet door just to test, the resolution was much better. i wonder if maybe diffusing the led with some sandpaper or something of that sort would improve tracking on a screen.

Despite my electrical ineptitude, I managed to hack an IR LED pen together today. I used an Avery Hi-liter pen (Expo dry-erase pens were more expensive), and everything else came from Radio Shack: 1.2v/100mA 45deg 5mm IR LED, mini momentary switch, N-sized battery holder, and N-size batteries (the AA battery holder doesn't fit and I don't have the skill yet to do this without ready-made holders). Drilled a hole towards the tip of the pen for the switch, glued the IR LED at the tip, threaded the wires into the pen and out the switch hole, soldered them together, and crammed them back in. It feels quite solid, though the button isn't great.

Tested it on my 24" LCD monitor and it works, but all the lines are jaggy. The battery's 1.5v, and I didn't get a 3.3 ohm resistor, so the LED is being overdriven, which may be creating too much reflection. Can't wait until work's off winter break so I can try it with our projectors.

I really want to try to get an active tip (push the LED in to activate it), but I'm not sure how to go about that.

Hello, your project is very interesting and I will try to make it in this holiday. By the way, can you post the picture of your a simple schematic of the light pen ? Since the link before its not work.

Here's the Expo pen I made. The switch I used was a little too big so I had to cut a hole in the bottom to fit it all.

http://i130.photobucket.com/albums/p253/chrisluedtke/102_0129.jpg

http://i130.photobucket.com/albums/p253/chrisluedtke/102_0135.jpg

Yes, those holes were made with my hot solder iron.

I haven't tested anything yet because it turns out my dongle isn't detected by Bluesoleil. You can go here http://www.wiili.org/forum/bluesoleil-solution-for-not-supported-adapters-t794.html if you want to help me out. I'd appreciate it.

Hi,We can imagine an other application of your idea in an other way : domotic !Imagine a room with 4 walls equiped by wiimote. The person in the room wear a glove with IR transmiter on each finger. We can have a TV without any button, or a cooker without any button. A computer calcul the position 3D in the room an can turn on the TV because the finger simule the pression on a button. Sorry, i'm not clear because i'm not very good in english. I hope you understand what i mean.

What with the holidays and all, I haven't had a chance to try building a light-pen yet, but I see lots of comments from people having problems. I would advise holding the pen so that the LED is on the end away from the board. As long as your grip on the pen is consistent, the quad-to-square transform should be able to compensate. Also, if you are right-handed, place the camera to look over your right shoulder, as shown in this image.

If this isn't enough, you can then put a sleeve over the LED, so it won't reflect off the whiteboard, but will still be visible from the Wiimote.

I just whipped up a quick tute for anyone wanting a simple design for an IR marker. http://users.bigpond.net.au/barsdell/irmarker/I haven't actually tried this thing out yet, but shall tomorrow at my friend's place, where there be wiimotes :)

I would like to have the tip switch in the Wiipen I made, but I have no clue what I need, how I need to add it, and Where to buy the part. Has anyone completed a pen with the pressure-sensitive LED that turns on when touching a hard surface and off when it is not?

to use a web cam, you would have to have a web cam that sees only IR, or mod a cam. Its easy to mod, just take it apart, remove the IR filter (that most have and is usually a greenish tint) and replace it with a filter that blocks all other light and lets IR in. An exposed piece of film works good and is cheap. Put the web cam back together and done.

full instructions with pics herehttp://www.instructables.com/id/Infrared-IR-Webcam/

Note: I am not responsible for you breaking your web cam or anything else.

I wondered long time where to get IR led, then i got it, from old remote control. It works fine with 1.5V AA battery, that needed only some wires. That works fine with my PC, but my laptot keeps saying "the application failed to initialize properly (0xc0000135). Click on OK to terminate the application.Is there any way to get that to work? Laptop is Hp 530 Notebook, and there is Windows XP professional with SP3.

Aren't you better off having the tip of the pen as the button (quite a smooth one so as not to scratch the screen) and mount the ir led slightly above it. As long as the pen is held the same way round (with the led pointing at the wiimote) then it should track just fine. It may even track better seeing as the led will point towards the sensor rather than the screen (which also causes reflection problems).

On second thoughts I'm wondering whether it's best to have two types of pen available. Maybe one like I described in my post above for simply prodding the screen, ie for use as a touch screen media player using media center. Most of the time you're simply prodding the screen so a button on the end would be good. But if you want to write on the screen or scroll then the standard design of pen is probably better, if the button was on the end it is more likely to scratch the screen as a reasonable amount of pressure would be needed to ensure the button stayed pressed in. If you didn't put enough pressure then the button would more than likely not be pressed all the time you were writing resulting in large gaps in your text!

Also thinking you don't necessarily have to map the touch screen part to the display. ie you could have the projector (or tv) displaying the film somewhere in your lounge and you could have the wiimote pointing at a wall or the table for example. You could have the play/pause buttons mapped out on your wall and simply touch the wall with the pen to play/pause the film etc. Of course it's no real use but it'd be great to show off to your friends, no one would believe that you could control a film by touching one of your walls!

Johnny, great job! I have only seen the video, but would love to test it out.

If you could really think how to create a right click - I understand this is not supported - then it would be even better. My guess would be to light up both tip and end of the pen for a click if the wiimote can register this (ofcourse not very far away).

For the pressure activated LED tip - just use simple coil spring under the LED tip and let it move inside the pen when pressed against surface... then the coil spring connects say negative current and you leave the other connection so that it connects the positive of a battery only when you push the led a bit inside... then when you realease pressure, the positive connection is lost and thus the LED turns off.. This is mechanical so only works on hard-enough surfaces that you can really press against, but there might be some extra sensitive approaches too..

The right click surely has a simple option, basically make it like a pda or a tablet. ie press (or light on) for one second = left click, press (light on) for 2 seconds = right click. the click would have to happen when the light goes off I guess, not sure how that would work with writing on the screen but it works like that ok on a pda or tablet so I guess it would be able to work in this situation, although I know very little about programming.

As for the solution above about putting a spring mechanism in for the button on the end, I personally don't think that any of that is necessary, just switch the button and the led round, with the led mounted as near to the tip as possible. You'd have to make sure that the led faces the wiimote when using it though.

Pressure sensitive touch detection, like the Wacom, could be done by connecting your IR pen back to the Wiimote via the nunchuck port.

This would have the benefit of also powering your IR pen from the Wiimote batteries.

This could be done by adding a cheap pressure sensor and a cheap PIC or AVR micro.

A pressure sensor fitted into the pen just behind the IR led in it's tip could give an analog pressure value that you feed into a PIC or AVR micro ADC input. Some simple C code would then encode this value into the I2C message that the Wiimote understands, and sends this via a wired connection into the Wiimote nunchuck expansion port.

See the Wii Linux Wiki for details on the Wiimote I2C comms protocol.http://www.wiili.org

Excellent set of projects JCLee, an inspiration to us all, keep up the good work!

And definately concentrate on the augmented reality projects. This is the real technology of the next gen consoles, peeps are finally starting to realise that more horsepower and sharper graphics just doesn't cut the mustard any more!

^^^ A massive wire would be needed and some sort of fixing to make sure a slight tug on the wire wouldn't move the wiimote as even a slight movement would make the screen out of sync. Other than that a very good idea.

A cable can be your friend too!But you're right, a big part of the Wii concept is wireless.You could use bluetooth in the IR pen instead, and there's already talk of N1ntend0 releasing a wireless nunchuk.So there could soon be an even simpler solution..junk the AVR/PIC micro idea and just hijack one of the analog accelerometer inputs of a wireless nunchuk to monitor the IR pen pressure.

I finally got to do some testing (aka, playing) with the projectors at work.

It works pretty well at 'long range', with the Wiimote set well back from the screen (10+ feet wasn't a problem). At close range, the Wiimote may pick up two IR sources from one pen: the LED itself and the IR spotlight, which contributes to jaggies. When you're using it in an actual whiteboard setup, the Wiimote will only see the spotlight.I want to get a wide-angle IR LED (the ones I bought were 60 degrees, I think) to see how I can play with mid-air shadow-based pointing, which could be useful for large projections (although that will only work if the Wiimote is right on top of the projector).

I've also given thought to the way the Wiimote Whiteboard software works and how it could be changed to increase functionality.One of the problems is that it treats the momentary LED as a mouse click. You cannot do any 'mouseover' stuff without clicking on it. Instead of trying to make the pen increasingly complicated by adding in rapid flashers or something, can't the software be modified to use the IR blobs better? Detect one blob: mouse cursor, no click. Detect two blobs, the whiteboard software recognizes this as a left click at current position. Detect 3 blobs, software sends a right click. 4 blobs, a both-button/middle button click/scrollmode, whatever. It would require the software to recognize which blob is the 'cursor' blob so it doesn't get confused when more blobs are added, moved, and removed.Of course, this does increase pen complexity by adding switches and LEDs, but I think it'd be easier than adding flash pattern hardware. Though, now that I've done this stream-of-consciousness pontificating, I'm not quite sure how to design the pen to properly accomodate all of these extra IR sources, at least if you want to make it as orientation-free as the original design.

I suppose I can try and find a cheap bluetooth presenter mouse and try to hack it and the IR pen together...

Buy one of these ^ for about £25 (think that's $50) and mount the led at the front and the battery near the bottom and have a switch in between. Simply turn the led on when using it and turn it off when finished. The program just needs to be changed to use the led as the cursor rather than a click. Now you have full cursor control (useful for hover over and making sure you're in exactly the right place before you click etc) and you have the left click on the trigger of the above product and scroll and right click on the top. This means no clicking needs to be programmed into the device, it's already there!

So you can get up close and write on the screen by holding the trigger down.

And you can use it from far away rather than having to stand infront of the screen and clock people from seeing.

ie if you want to get up and point at something or write/draw on the screen you can. If you want people to be able to see and just want to do usual things like scroll windows etc you can do it from wherever you want in the room.

All from one device!!!

Changing the led to cursor control rather than clicks also has the added advantage of meaning you can easily change the wiimote to point at anything to be able to draw. The problem with drawing on something that wasn't the screen before was that you couldn't tell where the cursor was before you clicked by which time it's too late as you've started drawing. This way you could turn a table for example into a massive graphics tablet being displayed on a big screen as you'd have cursor control to be able to see where you're about to draw.

Is anyone having probelms with Office 2007 Apps? For example, in powerpoint i select the feltpen option. When i use the IR pen, i can make dots on the screen when i first turn on the IR LED (click the button). Then the "dot" is tracked around the screen along with the pen. This tells me that the IR LED is on and is being tracked. The issue is that no line is being drawn on the screen. I can use all other programs without any issues such as MS paint. Any Ideas?

^^ Basically the same thing as I've posted above. That one looks to connect to the pc via infrared rather than 2.4ghz wireless so would require line of sight.

It's probably a little nicer to hold in your hand but lacks a trackball so you couldn't use it from afar too and the buttons look like they'd need remapping as they're for page up/down etc. I guess that shouldn't be a problem but a little more work.

I think the one I posted is probably a little easier to hold the button down for a long period. The button on the pen would have to be held using the tip of your finger whereas the trigger on my one would be held by squeezing your hand. Easier for long periods IMO.

Yes, your changes would be necessary, and I think the "pen" format is easier than the "pistol grip" for me, at least.

I was thinking of placing a sleeve over the pen and sliding it over the laser button when I wanted it on. The buttons should work up to about 65ft., but I am not sure how coherent the IR laser (pointer relpacement) would be.

Not sure how this sleeve would work? I assume it's got to be tight enough to keep the button pressed? Might be easier just to add a switch rather than keep a sleeve tight on a button? IMO holding the button to left click whilst writing for long periods is going to hurt your hand more than my method. But of course that's down to perosnal choice.

An RF version would be better.

The trackball (I assume you know what a trackball is, sort of like an upside down mouse ball. Basically provides the mouse movement function). The advantage of this is that you can use the device from anywhere in the room. ie with your pen you'd need to be right at the screen to do anything whereas with mine you would be at the screen to write/draw but you could do lots of things from wherever, it sitting down or the other side of the room so as to not block the view of the screen.

marco, the pen that john speaks about should work fine. But it relies on the laser being replaced with an ir led and the rest of the pen working with it. The voltage may be too high/low and the sleeve might not be easy to make.

Personally whichever solution you go with, the bits should simply be added on top. That way you don't rely on the rest of the electronics working with it. All you should need to add to the device is a battery, switch and the ir led. You may need a resistor, I'm not sure.

marco,No. I just read benjimoron's comments and found the pointer -- all today. I have used a sleeve on an regular pen-shaped laser pointer to keep the button pressed. I used a few mm of plastic tubing (just bigger in inside diameter than the diameter of the pointer).

The mod is similar to the ones proposed before, so it "should work". If I can find an affordable RF pen-shaped pointer (best I have found is USD55), I'll try that.

benjimoron,My goal is to have the presenter in front of the screen. Now I see why you want the trackball -- it is probably good for your use. I already use something else to allow remote screen drawing.

Piece those together, if wynn could do the honours that would be fantastic.

http://img144.imageshack.us/img144/4082/

trackballal1.jpg

That's basically the same thing but people may prefer it to the pistol version. The left click on this one is controlled by pressing the left button with your thumb. Bit more natural to write with I think but the left click and the trackball are both operated by the thumb rather than thumb and forefinger (I think that's the one next to the thumb!) in the pistol version. Makes clicking and dragging when not at the screen difficult.

My aim is the same as yours, to have the device right up at the screen to effectively make a touch screen. But at the same time to add left and right click, hence why using a wireless trackball. The advantage of which is that you can use it at the screen and also from a distance. 2 birds, 1 stone!

This requires a bit of extra co-ordination so I'm not sure that it'd be that great but I thought I'd throw it into the mixing pot anyway.

You could have the pen in your right hand, led permenantly on and acting as the mouse cursor. And in your left hand have a wireless trackball which is controlling the clicks.

Like I said this requires a bit more co-ordination and would take a while to get used to. But it does mean you can use a very simple pen for controlling the cursor and allows you to left/right click too.

Gesturing in the air is nothing new, you can do it with the wiimote on its own rather than using leds or you could just buy a gyration mouse. These have been around for years and basically do the same thing. Look at software like streetdock if you want good gesture control. Only problem with waving in the air is you need to know where in the air the screen is aligned to in order to make good use of it.

Hey, This is some pretty neat stuff! in not actually having a wii i had no idea this was going on until someone sent me the Gizmodo link.

I got to playing with you whiteboard app and was unimpressed when i had to install some other application (bluesoliel) to use it so i got to tinkering around with brian peeks library on coding4fun.net and found your connect methods are mostly the same with the exception of the "if (!IsRemoteConnected((int)index))" part. once i changed your project to have the same connect as the 10.22.2007 wii-lib release, it works now with my linksys BT device and vista drivers! you just have to pair them. Just letting you know. Thanks for all the awesome projects! and keep up the good work! your stuff is amazing and my office loves the new electronic whiteboard, heh.

I've had some problems with cheaper IR leds (only 20mA), which did not have the juice to get detected, unless they were pointed at the wiimote... kind of defeats the purpose, just be aware and buy some powerful leds instead.

I've noticed someone challenging the 100Hz sampling rate and the resolution. It would be nice to have some confirmation, that these are indeed correct.

If they are, then I suspect we could build a device that would allow for 5 different programmable functions to be recognized - with relative safety - by the software.

Suppose:

F1: Led on.F2: Led flashed at 20Hz.F3: Led flashes at 40Hz.F4: Led flashed at 60Hz.F5: Led flashed at 80Hz.

These are sufficiently spaced apart, for software to be able to distinguish between them, after calibration (pen would need to be in sync). It could allow different actors to be differentiated in multi-user scenarios, or the "pen" could perform different actions: Left Click, Right Click, Paint, Erase, Brush, etc...

One more question: Why on earth does the wiimote track 4 blobs? Any ideas? It seems completely a waste, since the "sensor bar" has only two blobs on it...

I found a perfect ir pen at Walgreen today. An "LED Pen Light / With Pocket Clip", item #561434, from East West Distributing Co., in Deerfield, IL 60015.

It's a standard pen-shaped pocket flashlight job with 2 AA batteries in the case, an on-off switch on top, and a white LED poking out the bottom, but the chip the LED's soldered to sits loosely in the case. I broke apart an old keyboard, cut out the thick centers of two of the rubber nipples used to make the keys pop back up, cut a small hole in the center of each, and inserted them between the end of the battery and the chip. It provides just enough resistance to break the circuit unless you press on the LED.

The pen itself was $5 and is stainless steel; actually quite a nice pen for the price. The rest was just junk, of course, and free. I'm able to vary the pressure at which the point reacts by trimming the height of one of the nipples. I'm certain you all could make much more elegant solutions, but it works very well, took a lot less work than diying the whole thing, and looks really good. Might keep an eye out for the flashlight.

Would anyone like to use my website for discussion about these projects? I am not doing it for ads or site traffic, I just want to help the users here communicate with each other as best they can, and give them a great platform to do it.

Just had a thought about my idea to use multiple IR blobs to differentiate clicks.

Clicks, unlike the cursor, don't necessarily need to be IN the IR pen.Simply hook up what amounts to a Wii sensor bar to the back of your pen. Put the bar somewhere stationary by your screen, but out of the calibrated screen area. Run a power wire out to the sensor bar hooked up to extra switches in your pen. Push the extra buttons on the pen, and the sensor bar lights up.Much simpler than trying to cram more lights or more sophisticated stuff into an Expo pen or cheap LED flashlight.

Now...anyone have any ideas for powering a Wiimote directly from USB power?

I know about those, but they're not what I have in mind. Those are chargers with a replacement battery pack.

I'm talking about something that lets me wire the wiimote directly into a USB cable, without a $20 possibly-defective (Nyko's charging station has been causing fires) charging unit between it.

Some Sony cameras at work came with wall adapters that were basically paired-AA-shaped shells with the contacts that had a wire going out to an external power brick that attached to the wall. That's what I want, but USB.I considered getting some of those charging units to steal their battery shell and door, but I didn't want to spend $10 on the off chance I could do something with it.

If USB can't supply the required amperage (it tops out at 100mA, 500mA with some sort of negotiated handshake), some other connector would work. Just something that goes from batteries to external wires.

Looks like that could be the bit of software to do it, I've posted before about changing it so that a 1 sec click is left and a 2 sec click is right just like a pda or tablet works. Not sure how that would work with drawing, would it realise that you wanted to hold the left down for drawing rather than thinking you were right clicking? A pda or tablet tells the difference by whether you're moving the cursor, ie if you're moving then you're drawing but if you're still then you're right clicking. Not sure if the software can handle that but it's certainly one of the solutions.

Only problem with it is that you don't get a cursor, you just get touch screen. Not a massive issue but a cursor is very handy sometimes.

On the subject of right clicking, I think someone mentioned a little earlier about lighting up more leds to mean different clicks. This could be a very good idea if the wiimote doesn't get confused as to which one is the cursor.

Suppose you had the main led on the tip of the pen permenantly on (with a switch to turn it off when not in use) and the wiimote tracks that as the cursor. You then have two more leds somewhere, doesn't have to be on the pen but probably easier to. Now you have two buttons on the pen, one for left and one for right click. If you click the left button then one of the leds light up, there are now two leds lit, one is the cursor, the other is for a left click. The wiimote should now report two leds on, this should work as long as the wiimote doesn't get confused as to which one's which. I think the wiimote recognises which one is which doesn't it as in the video with the blue and green pens and the swirly pattern on the table. The location of the second led is irrelevant, all the software would need to know is that there are now two co-ordinates being reported so that means left click. Same with the right click, 3 leds would now be lit (they need to be fairly seperate so they don't collide) therefore that means right click.

Must admit that I don't fully understand your question but I have a funny feeling that the wiimote is going to get confused by the pulsing lights, for example 20hz is repeated in 40,60 and 80hz and 40 is repeated in 80hz too so I guess at 80 they could all be flashing and I'm not too sure how it would tell the difference then anyway? Can't it tell the difference anyway as in the example I gave above, ie the video with the blue and green pen on the desk with swirly patterns?

I assume the wiimote tracks 4 as that was cheaper than making one that only tracked two. ie the manufacturer probably already made loads of units that tracked 4 therefore they work out cheaper.

Hey teamJust tried this software I posted with my in progress whiteboard, and it works fine for a actual MOUSE - but not for the wiimote! You can move the pointer and so forth, but holding down a 'eleft click' doesnt do anything, yet the mouse does.

I think its the software, knowing to look for a HUD mouse, while the wiimote is classed as a gaming device! Damn!

I think you may have a valid point about possibly confusing different cycles, but that's a software problem. The wiimote would not have to interpret that. It would just give you blinking "blobs" at specific co-ordinates. You can then handle those from your code. Even the slowest of blinks can probably be profiled in about a quarter of a second, (5 flashes in .25sec means 20Hz, 10 means 40... the problem is, if the wiimote loses sight of the pointer for a short while, and then the software needs to figure out, just what's happening. If you cut down the number of functions to 4, then you will have more separation between the blinks, thus safer operation. We never know if it works, until someone tries it :)

One more question pls: I have now built two pens, each with a single 20mA 850nm led, but I am having trouble getting the wiimote to notice it. I am probably using the wrong led, with too little power, or too narrow a viewing cone. Any suggestions on which type work best?

@tabGabi got mine from radioshack. they are the only IR leds they have that i found and they are 100ma.alternatively, i have a mostly see through pen that i used to make my pointer, and the pen has a moslty see through but still fogged up cap. when i put that on my led, it shows up way better. try and find something like that, or just something that will light up with the IR light so the wiimote can see it when it is not directly aimed at it.

First off, this is by far one of the coolest thing I've seen to do with the wiimote! This and the head tracking are fantastic!

Regarding the Right Click problem: why not sync a second wiimote that is configured with macros, held in your other hand? For example, hold down the trigger to simulate the right click function, (a la the Ctrl click on macs) press the 1 button to toggle the on screen keyboard, the arrow keys switch between pen colors, etc.

This would eliminate the hassle of trying to configure the whiteboard program to recognize different LED frequencies, which may be troublesome given the multitude of whiteboard setups. It also adds a lot more functionality to the whiteboard.

Of course, I don't have the programing knowledge to pull this off, but this is merely a suggestion. I'm pretty sure you could use GlovePie or something to create this control scheme. Let me know what you think.

as soon as i saw this i wanted to make it, and one link leads to another... I found some things of interest.

http://www.whitenoiseaudio.com/touchlib/

http://wearables.unisa.edu.au/mpx/?q=mpx

thats for us linux users. The first link is Touchlib, which seems to be a library that supports tracking "blobs" of infrared as an input device. The second link is Multi-Point X Server (MPX), which allows you to use multiple input devices. It is pretty awesome what you can do with this. What if we combined the two? :D

http://www.instructables.com/id/Interactive-Multitouch-Display/

this looks like it may be combinable with your wii-mote obsession. Although Im not sure if I understand how this one works. Does the camera record infrared reflected off of your finger, or does the camera detect your finger by seeing where it blocks the infrared? What exactly does the acrylic do to the leds around the edges?

A thought occured to me about multi-object tracking and labeling. First, on labeling, I have an ugly hack to differentiate between multiple IR sources - you could shield them with polarizers, and polarize multiple wiimotes. So, the vertical polarized wiimote sees light only from the vertival polarized IR, and the horizontal only from the horizontal. They both see the unpolarized one. So, with two wiimotes, you can uniquely identify three light sources, as long as they maintain their relative rotations.

Second, I'm envisioning a table workspace with tactile objects you want to track - chess pieces, for instance. Instead of a constant IR light, put a pulsing IR light that turns on and off in a recognizable pattern. It might not work great, but there's my thought. As long as only four of the pieces are on at any given moment, the wiimote can track all of their lights, and some other software can track their overall positions and their bit codes, and decode their unique ID. The problem here is synchronizing the lights to all turn on and off in the proper sequence. I imagine either radio control, or them all watching a clocked IR light near the wiimote to synchronize them.

I was wondering if anyone had the same problem as I did :I first bought some IR Leds rated at 840nm. Performance was pretty terrible, at first I thought it was the angle that was to narrow. I then bought some 940nm leds that seemed less powerful when checking with a webcam, however, they work much better with the wii remote. Could it be that the wii remote doesn't work with IR Leds that are below 940nm ?

Chase: It has been my experience that you don't necessarily have to touch the screen to make it work. But the further you get back from the surface, the more awkward it gets to use. You also lose some ability to tell where the pointer is going to go. Your best bet would probably be to discipline yourself enough to not press on the screen too hard.

hi:I saw your demo "Low-Cost Multi-point Interactive Whiteboards Using the Wiimote" and I was wondering on 2:51 what software is that? It seems many stars suddenly appear when you press the screen. Is that kind of interactive software? Thanks

Im particularly interested in the video where you demonstrated calibration of two projector side by side.

I have a project of creating a small cinema room in the small town where my father lives but the cost of a projector that will five me a decent size/quality image is way too high for my pocket.

Your experiment shows that it might be possible to calibrate maybe 4 projectors together and get a decent size/quality image that might allow for the creation of a small size projection room where I might be able to fit about 50 people.

Is there any more info published by you on this project?

Do the images look seamless when the projectors run together side by side?

is there an IR camera that I could use in stead of the wiimote? I am trying to make a more permanent white board and dont want to have to change batteries or power on the wiimote, just a camera that is connected via usb to my computer.

I'm thinking the same thing, and here's what I've come up with so far:For pairing the wiimote, I have a rubber band holding a plastic block over buttons 1 and 2. I can leave it there all day and all night.

For the batteries, I am going to feed a 3V power supply to it constantly. That way I'll never have to deal with it at all.

One way to eliminate having to hold the side switch, is have an on/off switch (not momentary) that activate an electronic circuit. LEDs can also be used to sense light, so a circuit can be built to switch between photosensing and light emitting modes using a small microcontroller. When the circuit is on, and the pen tip is not near a surface, it intermittently outputs a pulse of light, switches into sensing mode to check the amount of reflected light and then goes idle for a period. When the pen is brought near a surface, the amount of reflected light during the sense phase changes, so the micro switches into operating mode. In this mode, it is mostly emitting light, and periodically switching into sense mode just long enough to check if it is time to switch back into its idle mode.

I worked the code a bit and added a right click at 1 second. (like a PDA) For some bizzare reason, it works perfectly everywhere except MS Excel. I think there must be some kind of extra activity that happens on right click in Windows whenever Excel is the active app.

John:please can any one HELP me!!I downloaded the C# source code from this board, and when I attempt to run the program in debug mode I get this error:

A call to PInvoke function 'WiimoteWhiteboard!WiimoteWhiteboard.Form1::mouse_event' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

on this line:

mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);

the first time I attempt to click with the ir pen when calibrating.

and if I uncheck the cursor option, and try it I get:

cross-thread operation not valid: Control 'CalibrationForm' accessed from a thread other than the thread it was created on.

on this line:

base.Dispose(disposing);

immediately after the last calibration point. I'm using visual c# 2005 express edition on a windows xp home machine. I can run the precompiled application, and my wiimote connects fine etc. Is there a certain configuration I need to be compiling in to get this to work or any additional files I need? I haven't put very much time into researching this problem, so it may be trivial to fix, but I figured it was worth posting anyways for anyone who might be as surprised as I was when the original source did not compile. Any help would be appreciated.

John:I downloaded the C# source code from this board, and when I attempt to run the program in debug mode I get this error:

A call to PInvoke function 'WiimoteWhiteboard!WiimoteWhiteboard.Form1::mouse_event' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

on this line:

mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);

the first time I attempt to click with the ir pen when calibrating.

and if I uncheck the cursor option, and try it I get:

cross-thread operation not valid: Control 'CalibrationForm' accessed from a thread other than the thread it was created on.

on this line:

base.Dispose(disposing);

immediately after the last calibration point. I'm using visual c# 2005 express edition on a windows xp home machine. I can run the precompiled application, and my wiimote connects fine etc. Is there a certain configuration I need to be compiling in to get this to work or any additional files I need? I haven't put very much time into researching this problem, so it may be trivial to fix, but I figured it was worth posting anyways for anyone who might be as surprised as I was when the original source did not compile. Any help would be appreciated.

Check out motion tracking for webcams on google. there are very many solutions out there. but to convert a webcam to do the same thing, you would need to write all the processing of the brightest dots. the wiimote has all that built in so you would have to take it one step further by creating your own tracking portion before you ever got to the normal whiteboard part

Please can any one helpI downloaded the C# source code from this board, and when I attempt to run the program in debug mode I get this error:

A call to PInvoke function 'WiimoteWhiteboard!WiimoteWhiteboard.Form1::mouse_event' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

on this line:

mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);

the first time I attempt to click with the ir pen when calibrating.

and if I uncheck the cursor option, and try it I get:

cross-thread operation not valid: Control 'CalibrationForm' accessed from a thread other than the thread it was created on.

on this line:

base.Dispose(disposing);

immediately after the last calibration point. I'm using visual c# 2005 express edition on a windows xp home machine. I can run the precompiled application, and my wiimote connects fine etc. Is there a certain configuration I need to be compiling in to get this to work or any additional files I need? I haven't put very much time into researching this problem, so it may be trivial to fix, but I figured it was worth posting anyways for anyone who might be as surprised as I was when the original source did not compile. Any help would be appreciated.

Fantastic work. I'm no geek, but had no trouble cannibalizing an IR LED from an old Tivo remote to replace the LED in a $3 pen lite. Works like a charm on my notebook LCD!I wanted to demonstrate this at work where our meeting rooms are equipped with large screens/projectors. Just one problem: I can't reach high enough to calibrate the top corners. This had me thinking that a useful feature for the next release would be the option to calibrate to full screen (default) or only a portion of it (i.e., bottom half, or lower left quadrant, or an arbitrary height/width + 0,0 axis for placement). Is there an aspect to how the software must work that would make this impossible? It would really help for those working in front of medium-to-large scale projections.

May I just say this is fantastic work and it will really do amazing amounts of good for schools in countries where the economy is still developing.

Please do keep up the good work and continue to update and improve this wonderful idea as I'm sure that as word of this creeps round the internet your idea will become more and more popular and more and more widely used.

RIGHT MOUSE CLICK & WHEEL. Modify the Wiimote Whiteboard software so you can leave the pen IR LED light on all the time and just use it for mouse tracking, no clicking. Use a bluetooth mouse for left/right click and wheel. Hold the mouse in your other hand or combine the pointer and mouse into one unit. You need to cover the optical sensor on the mouse with electrical tape so it doesnt move you're pointer.

I am using Vista 64 bit and I am able to connect the Wii to my computer via blue tooth. When i run the Wiiwhiteboard.exe my computer crashes and goes to blue screen. I am a teacher in Phx, AZ and really want to do this. Please help!

It would be possible to send a radio signal with some freq. when you activate the IR LED for left-click, and another freq. for right click (2 buttons). In this way, not only do you implement the right-click, but also make it non-responsive to parasite IR sorces.

P.S. I have built the steadycam, and i'm just a paint job away from testing it ;)

you could distinguish between a right or left pointer (or the front or back of a glove) by filtering the range of infrared light reflected from the reflected surface. The software could be written to recognize the different wave length variation thus giving an "a" and "b" input. If you can build it and if it works tell me about it. my e-mail is mwhitaker@innovative-ehs.com

I just purchased a TRENDnet TBW-101UB Bluetooth adapter and was able to get Wiimote to be recognized. I ran the wiimotewhiteboard and it comes up with "Exeption: wiimote not found in HID device list". I am running Windows XP. I cannot seem to find anything wrong as I do have things in my HID device list.Any help would be greatly appreciated.

Hey I have a Sports related question. It seems that if these remotes can be used to track movement a system could be set up to anylize and track the movement of an athlete to better understand his/her performance. Is possible to have more than one remote in different locations in order to pick up many points on an athlete?

I have an HP Laptop running Vista 64-bit and am having alittle problem getting this to work. It detacts the wii controller via Bluetooth, I open the Calibration software, select my points....with the software always seeing an IR point......when I try to move anything, nothing happens?

i have a question, i have come to realize that for some odd reason the wiimote is "dectecting infrared motion from itself, i'm able to move, "but very limited."the wiimote as mouse after calibration, since the wiimote has an ir all in itself, is it possible to use a SECOND wiimote as the pen in itself, thanks if you can answer!!!

Great Job. I was impressed by seeing your whiteboard project.And try to do the same with different hardware.

I have some device apart from wii mote which detects IR or light source only. I can connect that device through USB and read detected light source as data. I am trying to connect my USB HID device by changing VID and PID with your software. Its not working & shows that device cannot found. When i start debugging ur wiimote project code, at one point i realize program flow and it is not going inside this condition.“if(HIDImports.HidD_GetAttributes(mHandle.DangerousGetHandle(), ref attrib))”

Could you please help me in sorting out the mentioned issue.Advance thanks for your help!

Great Job. I was impressed by seeing your whiteboard project.And try to do the same with different hardware.

I have some device apart from wii mote which detects IR or light source only. I can connect that device through USB and read detected light source as data. I am trying to connect my USB HID device by changing VID and PID with your software. Its not working & shows that device cannot found. When i start debugging ur wiimote project code, at one point i realize program flow and it is not going inside this condition.“if(HIDImports.HidD_GetAttributes(mHandle.DangerousGetHandle(), ref attrib))”

Could you please help me in sorting out the mentioned issue.Advance thanks for your help!

I'm in a classroom. My projector projects onto a screen beyond my reach, with my whiteboard below it. Will this still work? I write on the whiteboard and it projects up on the screen? Sort of like a giant tablet.

Hi, I am using the wiimote whiteboard for graphic design. The problem i face is that the mouse cursor does not move with the pen when the IR LED is not on (of course). I wonder since the mouse works when the whiteboard program is running, can you remove the mouse depression command when the IR LED is lit? So that the LED is on all the time and the position of the LED yields the position of the cursor. Then I can hack a mouse to remove the right and left click buttons, connect them to the pen with two new momentary switches. That way the cursor moves around with the pen without clicking on anything. When i push down on one of the switches connected to the mouse, it sends the mouse depression command to the PC.

Hello, after connecting the remote via blue tooth and downloading and extracting the software, it blocked my usage of the program saying: "Exception: wiimote not found in HID device list" . Any help? I'm using windows XP.Thanks -- Joshua

hi i want to try and use this on my computer screen like in one the videos with the laptop..but i have a a CRT monitor and in the vid it said for any "liquid crystal display"... does this work on CRT monitors?

I havent tried this yet, but i am trying to find a bluetooth adapter, then go make the ir pens, but i think the calibration scheme should be changed

i would expect that the calibration starts at the top left, then goes to bottom left, top right, bottom right, as this would be more natural, and might be easy to implement

Would more points of calibration make this more accurate, less jagged or something, because the smartbard had 9 points of calibration. Maybe you could add a calibration point, right in the center. The pressure tip may be usefull, but you will scratch up your led, and it might not work as well. you could wrap in the a cloth with lots of wholes, and make the led really powerful, as the cloth will limit the brightness, and this would stop the relfection prob some ppl are experencing

By the way, i love this idea, i think another guy made this with 2 wii remotes, so if you cover one, the other one picks up, but could you implement a shutdown, as you click a button, the remote turns off, and then you could press the button a, and it would reconect, and work

One thing that would be truly useful would be a modification that hooked the Wiimote directly to the computer via a USB connection (bypassing the Bluetooth).

This would be for:1. Powering the Wiimote via USB instead of batteries2. Eliminating the need to start up a Bluetooth connection

This would be advantageous to folks who would prefer to mount the whiteboard Wiimote up high (ie. ceiling mount). It would also be excellant for folks doing Wiimote-based headtracking. Instead of mounting a Wiimote on the top edge of a display, It could be mounted directly over the user, thereby minimizing the chances of the head blocking a view of the IR LEDS. Great work and promotion going on here. Thanks!!

I have installed everything and followed all the steps, however everytime i try the ir pen on my lcd screen the wii remote goes crazy. I really don't know why it seems to detect 2 or 3 signals. I use only one ir pen and the pointer goes crazy on the screen.

How much advanced programming (and/or hardware) are we talking about if we used your whiteboard project to emulate Arcstream Living Image floor projection system? For example, people walking across the illuminated floor, creating "water ripples" or "chasing fish away".

I imagine it's more involved with precise IR tracking system? Along with some fancy coding for FX?

What is the mathematics behind the calibration? I coudln't find anything useful on google... I just considered the vectors made by the theorical coordinates of the calibration points and the actual coordinates read by the wiimote, then compute the polar coordinates of those vectors and compute 2 maps, one for r and one for theta, using the bilinear interpolation algorithm. It works ok when the deformation is small, but very bad whith big deformation.

Do you have any pointers to some of the theory and math behind your calib?

1.Our Website - is a great way to find new friends or partners, for fun, dating and long term relationships. Meeting and socializing with people is both fun and safe.

2.Common sense precautions should be taken however when arranging to meet anyone face to face for the first time.

3.You4Dating Free Online Dating ,You4Dating is a Free 100% Dating Site, There are No Charges ever. We allow You to Restrict who can Contact You, and Remove those unfit to Date.

4. You4Dating is Responsible for Creating Relationships per Year proving it is possible to Find Love Online. It will Quickly become a Leader in the Internet Dating Industry because of its Advanced Features and matching Systems,and most of all,Because is a 100% Free-There are No Charges Ever.

5. You4Dating is an International Dating Website Serving Single Men and Single Women Worldwide. Whether you're seeking Muslim,Christian,Catholic, Singles Jewish ,Senor Dating,Black Dating, or Asian Dating,You4Dating is a Right Place for Members to Browse through, and Potentially Find a Date.Meet more than 100000 Registred Users

A study last yearnike tn, the author in the essay read NIKE , a reporter at the Shanghai headquarters in an interview, nike chaussuressee a pr in high school to read a league plans, employees interviewed told reporters in Beijing's streets, they children to interview, the children said, "truly understand them." tn chaussures These words, if the author touches product function, brand spirit and culture is to become part of the consumer of two basic methods

I am planning on aiming the Wii camera at my coffee table from directly above. It will take a little trial and area to do the initial calibration as I will not be able to see the calibration points but once done will be laughing. Reason being I am going to use hand gestures over my coffee table to control media centre PC. Instead of a pen I am making in IR finger attachment with a tilt activated switch. eg. To change channel I will move my hand over the coffee table, guide the curser to the channel button and title my finger.

A study last yearnike tn, the author in the essay read NIKE , a reporter at the Shanghai headquarters in an interview, nike chaussuressee a pr in high school to read a league plans, employees interviewed told reporters in Beijing's streets, they children to interview, the children said, "truly understand them." tn chaussures These words, if the author touches product function, brand spirit and culture is to become part of the consumer of two basic methods

There are spyder jackets 10 pcs of excellent Ralph Lauren multi-colored stripe big pony cheap columbia jackets. These products are more soft and comfortable for being made of breathable and durable mesh cotton .cheap polo shirtsthere are lots of colors and sizes for your choice, just pick up one you like ,it is absolutely value for money.

These projects are awesome. I recreated the whiteboard and all applications I tried work fine for now.

Only the programming language Scratch doesn't work correct. When I want to drag and drop a block I only have a right click event. All other apps use the normal left click. Why does that happen? How can I solve the problem?

Wow. I remember you heptic pen. There should be a way to adapt it to the Wiimote Whiteboard. That could then use pressure sensitivity, so you could implement it into doing things like the pressure slider you posted (from that other team) and stuff like that.This could easily make a great DIY package. I mean, you could sell, even in the spirit of just taking the technology everywhere, packages for doing this project. Something in the way of self calibrating projector image would be great too. Concerning the right click, what about the approach the touchscreen phones have? Just maintaining the pen in one spot for a while. This would be great for schools that need the tech but don't have the cash.

Hello I tell you that your project seems great, I feel the responsibility to inform you that your project was abducted by some persons they wish to present as their own when they are not even credit you as the person who developed the link to review the information is here http://itsanjuan.edu.mx/index.php?option=com_content&view=article&id=107:alumnos-que-obtuvieron-el-1er-lugar-en-el-xxiv-evento-nacional-de-creatividad-en-su-fase-regional-con-el-proyecto-pantalla-multiproposito&catid=9:infogralinstitucional

Subscribe To

About Me

I am currently a Technical Program Lead in the Advanced Technology and Projects (ATAP) group at Google. Previously, I worked at Google X, and Microsoft Applied Sciences as core contributor to the Xbox Kinect. In 2008, I graduated from Carnegie Mellon University with a PhD in Human-Computer Interaction. My research interests are in exploring novel interface technology that can influence the lives of many people. My main website can be found at johnnylee.net