At the heart of the system is the computer turntable control system Ms. Pinky, as seen here previously powering vibrating furniture and hiding out in tree trunks. Jesse uses Max/MSP to generate the scratching sounds, then hooks up a second laptop to output visuals. Processing, the open-source software that promises easy coding even for artists, handles the eye candy. You could use the same approach to generate sound, however; Processing can work as a synthesis toolkit with the help of the free library JSyn. (Or, alternatively, Max/MSP/Jitter could work on visuals — though you may still need two laptops to catch up.)

Go check out the video for an explanation of how it all works. Thanks, Jesse! Great project, and nice scratching, to boot!

Great work. I have been intrested in doing osme interactive work like this but I have been a bit overwhealmed by visualisation software. I'm downloading 'processing' to try out, because you say even artists can use it!

I'm more keen in making visualisaions from camera inputs, but we'll see.