Photo (CC) moonpie, who while I live in New York, lives in the original York.

It’s just US$40, and it’s your best ticket to creating your own computer vision and augmented reality projects, imagining stuff before big game console makers do. It’s the Sony PlayStation 3 Eye. Sony intended it to be used with their flagship game console. (The guy working at my local GameStop looked emotionally dejected when I told him I don’t actually own a PS3.) But thanks to community-developed, open-source drivers for all three platforms, the PS3 Eye has become a big choice for interactive applications, from tracking fingers and objects on tables to projection mapping and augmented reality.

Why choose the PS3 Eye over another webcam? Because it was built for CV applications, the camera performs well in variable lighting, has rock-solid, low-latency USB performance, and is capable of high framerates (60-75 fps at normal resolution or even 125-150 fps if you can sacrifice resolution, which might be okay for tracking).

If you don’t already own a PS3 Eye, you should. Go get one — I’ll wait. Now, the hackers among you aren’t done yet. You’ll want to trick out the PS3 Eye to make it really yours, with drivers, infrared-tracking capability, and a beautiful, new custom housing.

Hack the Case, Remove the IR Blocking Filter

Peau Productions has been one of the best sources for information on working with the PS3 Eye, and the man himself – Nolan – has done some great tutorials.

Whether you want to use IR or not, you will likely want to hack Sony’s case. The case works fine when the camera is upright, but it’s a bit awkward, and it can’t sit horizontally for use in multitouch tables and installations and such. And it just looks, you know, like a gaming accessory. So I recommend the tutorial here even if only to figure out how to open the case. (Yes, I did in fact manage to cut myself the first time I tried. I’m a complete klutz.)

There are various reasons you might want to use infrared light only. You might be doing tracking on top of a projection, as on many multitouch tables. In this case, blocking visible light will eliminate the image of the projection (digital projectors don’t emit much IR-spectrum light), while infrared light can still provide tracking information. You might want light sources people can’t see; that is, IR LEDs or broad-spectrum lights with filters. You can also easily track IR emitters or markers using this method. Like most webcams, the Sony normally detects visible light and blocks IR light. Here’s how to hack that on the OEM-shipped lens.

Add a Case, Custom Lenses

Nolan has also started a little store with some very cool accessories for your PS3 Eye. The lenses, when combined with his custom mount, give you the option of using visible light or infrared light easily – just screw in the lens you need. Different focal lengths make installations and interactive tables more practical. But my favorite creation of Nolan’s is his custom-designed case. I just bought a case after seeing one at digital magician Marco Tempest‘s studio.

The case is really beautiful. The camera fits snugly inside after you’ve modded it. You can still place the camera vertically – it balances atop my older Apple Cinema Display. But you can set it horizontally for use in a table, too, or easily add it to an installation.

You can’t use the built-in lens with the case, so you will need the mount and some lenses, too. That makes your US$40 camera a bit more expensive, but on the other hand, you’re still well, well below the cost of other cameras and you have a setup that’s road-ready and worthy of a variety of different projects. Nolan is also friendly with customer service, helped me late on a Saturday night, and shipped incredibly fast.

I’ve already sold a couple of these over IM. Nolan, I hope you’re ready to ship.

Here’s what the PS3 Eye looks like once you’ve taken it out of its stock Sony case.

Use it with your OS – and try Linux

Early on, PS3 Eye support was a bit shaky, but thanks to the power of communities using these and helping with the projects, I can now wholeheartedly recommend all three desktop operating systems. And Linux isn’t a second-class citizen this time, either – it’s actually what you should consider as your first choice. I’ll go into the most detail on Linux because, while there are a few steps involved, I’ve actually found it to be the best-performing. (In other words, don’t assume just because that section is the longest means that it’s the biggest pain!)

Recommended drivers:

Mac: Grab the macam drivers. 0.9.2 is the latest version as I write this, but the revision you want was February 6, 2009, which currently is only in the CVS repository.

Windows: AlexP aka Alex Popovich has written DirectShow drivers which perform really well on Windows. Early versions were a bit unstable, but the revision listed below for me works (just keep scrolling to the end of the post, before comments). The one trick is that the video driver will be listed under “AlexP Devices” and not in the video group in Device Manager. This could confuse some software.

Linux: Normally, this is where you’d expect to hear that Linux causes the most trouble and requires the most tweaking, etc., etc. – not the case. I’ve been talking to ReacTable co-creator Martin Kaltenbrunner, and he and I have both been happiest with the drivers available for Linux, thanks to work by him and multitouch NUI Group contributor kaswy. First, you’ll want to download the drivers – be sure to use version 0.5, revised April 23, 2009 (or a newer version if available):

These work on 9.04, too, and I suspect Hardy/8.04, as well. Here’s a quick summary:

These drivers can be built from source. You may need to have your kernel headers available so that the drivers can be built for your Linux kernel. If you’re using the default Ubuntu kernel, for instance, you can simply choose “sudo apt-get install build-essential” for everything you need. If, like me, you built your kernel from source (which I did to get a newer real-time kernel), then that source folder should still be available. Clarification: Apparently the required module is built into many kernels, so if you haven’t modified your kernel, you may be okay. I still recommend having kernel source and a package like build-essentials handy, however, especially if you’re using the real-time kernel for audio. And you will still need to invoke modprobe below to switch video modes.

Installing the driver assuming that kernel is available is then simply a matter of changing to your extracted directory, entering “make,” then “sudo make install.”

If you have trouble with Linux installation, there are lots of tips in the following thread on NUI Group. I don’t recommend their installation steps at the beginning, because the driver above (updated after the original post) works much better. But you will see some common troubleshooting techniques.

Normally, cross-platform compatibility with cameras means recommending UVC (Universal Video Class) cameras. Those have become more common – in fact, any camera with a Windows Vista logo on it ought to be class-compliant and driver free, because Microsoft’s certification requires it. But the PS3 Eye trumps them on performance/price, and these drivers provide options you likely won’t get out of the class drivers.

Use Your PS3 Eye with Processing

The PS3 Eye has excellent support under ReacTIVision and several of the multitouch tracking/tangible projects. I’ve been really happy working with it so far in GSVideo.

The one decision the Processing developers have made that I strongly disagree with is shipping QTJava with Processing as a core library. It’s often disastrously bad on Mac and unusable on Windows. It’s completely incompatible with Linux. It’s proprietary, officially deprecated by Apple… mostly what it does is convince people not to ever touch video or use Processing for video. Okay, I’ll stop.

Fortunately, there’s a terrific alternative that works in Windows, Linux, and Mac that builds on the superior GStreamer framework and gstreamer-java bindings (which, incidentally, is your strongest capture and playback choice for Java projects, too).

Install is absurdly easy on Linux. On Ubuntu, I do like to go ahead and install “libgstreamer0.10-dev.” Other than that, just install the library normally in Processing, and try out the simple Linux capture example. For my laptop, /dev/video0 is the built-in webcam, so I just select /dev/video1.

More tips?

We’d love to hear them. I’m just getting into the PS3 Eye and I’m sure that folks from the NUI community and elsewhere know plenty more I’ve missed.

just bought one the other day – soon to be several more. Working with OpenCCV and processing (initially, migrating to oFx even as I type) to create very large multitouch display window…
i really love this thing – ran perfectly on OSX with maccam drivers – wish i had bought a case for testing tho – but seemed impractical as i need the camera(s) to have smallest physical footprint possible.

for the record, the bases of the ps3 cameras have a metal plate in them. if you run a 1/4 20 tap (or whatever size you want) through it you can make it fit a tripod mount screw, which can be useful for certain situations.

alternatively you can just drill a hole though it to accept a bolt for more permanent mounting situations.

this can be really helpful if youve had issues with mounting the camera.

Thanks for the great tutorial. Do I still need to patch the recent kernel (2.6.32-22-generic) which is released with ubuntu 10.04? I tried plugging the ps3 and it worked right out of the box with cheese. I can get about 30fps with reacTIVision. Image quality is great. But I dont know how to get more than 30fps.

jiajun

very nice blog! thanks!

Have anyone successfully make it to work on Ubuntu 8.04 (kernel 2.6.24)?

I can build and install the driver, but the driver_test and vlc program failed to capture the frame.

does anyone know if you can control the auto-exposure (or at least turn it off) on this cam using a Mac? or any uvc web cam for that matter?

http://djtutorial.org/2010/11 dj tutorial

I don’t even know the way I finished up here, but I believed this post was great. I don’t recognise who you might be however definitely you are going to a well-known blogger if you happen to aren’t already. Cheers!

how do I take photos with the PS3 eye it wont let me.. PRograms wont recongize there is a camera saying “No camera found”

angeriod88

I have a PS3 eye camera and the video works on the CL test and When I skype I can do video and Audio. But when I try to take a photo or record a video it wont work. But I am able to record audio though. Please help; I have installed the new DRive correctly. I have plugged in the Ps3 eye camera in different slots in my computer. SLot 1 or 2 lets me have video on CL Test. Thanks alot