Student Programs Motion-Tracking Musical Instrument

Profile

Quick Links

What do Iron Man, Minority Report, and musical conductors have in common? They’ve all been used to describe the open-source software experiment IRP, short for Infrared Performer, developed by Audio Technology Program senior Ben Mangold. The software allows you to be a DJ, manipulate digital audio, and play software instruments with gestural control simply by moving your hands before a camera affixed to a laptop.

IRP is a musical control surface that uses open-source software developed by audio technology professor William Brent. It communicates with an infrared camera (with an attached infrared light) to track “blobs” of infrared light reflected back at the camera.

By wearing reflector clips on the thumb and forefinger (which are tracked by the camera), the user can pinch and move to “click and drag” across a fully customizable layout of buttons, toggles, sliders, and interactive shapes and instruments.

The buttons and other components of IRP can send MIDI data out of the program, which allows the user to control musical content and interact with digital audio workstations (DAWs), an integral tool for both the recording studio and any musical performance utilizing a computer. In conjunction with Ableton Live, a DAW built specifically for live performance and DJing, Mangold has used the tool he developed to create an innovative audiovisual performance.

The project’s beta was conceived, designed, and completed over the course of two years through classes at AU in digital instrument design, software engineering, and an audio technology capstone under the advice of Brent. In fall 2012, IRP was entered in the Student Design Competition and Exhibition at the 133rd Audio Engineering Society convention in San Francisco. It was selected into the competition through a blind peer review process from a panel of faculty members and professionals from around the world.