It’s something we take for granted: listen to a track, and it starts at the beginning and goes to the end in a fixed length of time. Wonderful things can be done with music that way, and it’s the traditional model of composition and recording. But the equally old, if not older, tradition of improvisation suggests that music doesn’t always have to be linear. It can be specific to a place, a time, a mood.

Now that the technologies that power music creation can fit on a standard mobile device, listeners could have music that’s as pliable when they listen through headphones as it is in a studio when it’s created. Music could respond to the environment you’re in, and sound different each time you plug in your earbuds. That presents new challenges for the people making the music, but it could be an entirely new medium.

The team behind RjDj, a reactive and interactive music platform for mobile devices, don’t just want to wait around for this to happen. They’ve got it up and running right now, in a just-released application for iPhone. I spoke via Skype to the team in Vienna as a crowd of enthusiastic programmers and volunteers hacked away in a massive patching and music-making fest they call a “sprint.” More sprints are planned around the world, and the entire project is being built with the open-source visual patching environment for multimedia, Pd (Pure Data), cousin to Max/MSP.

Hackers work away in a “sprint” in Vienna. Photo by jennifereight; used with permission.

If you’re ready to geek out with Pd, in fact, you can have at the patches yourself. But even if you’re just an interested musician, there’s plenty to watch here. It’s about more than just the software (Pd) or device (iPhone) – indeed, this app alone is likely to extend to other devices. What it’s really about is a new approach to how to listen to music, how to develop musical tools, and how communities own and share that work.

And, oh, by the way, team members have been behind everything from the port of Pd to Linux to the launch of Last.fm – the latter sold to CBS as one of the hottest musical properties on the Web, and a personal fave among the CDM team. So don’t doubt for a second that this group can drive some serious change.

If you watch just one video, check out the one above – especially about halfway in, as it starts to get juicy. Even for someone who’s been doing this for a while, watching a tiny device respond to the environment is magical.

Gunter Geiger is a technologist and advocate of free software. He puts his code where his mouth is: he ported the multimedia tools Pd and GEM to Linux a decade ago, helping launch the free community around them. Now he’s harnessing Pd again – but it’s not just about the software, he says.

Gunter: It’s not about if it’s Pd or not. The idea is to be able to create music in a different way. Instead of doing a fixed track, you do something interactive. These kinds of programs have been around for ages, but it really didn’t catch up on the music market.

The important thing is to get momentum behind it — not just one guy doing this thing. [And] it’s not only having people to create things, but [expanding] the audience, which is very small. What we really want to create is some momentum, and a scene. We hope that we get artists who make new [work].

You start to create different forms of music. Some of them are more like classical interactive things. Others are using the sound input a lot. It’s really a very open world, and the good thing about using Pd in there is that basically you can do everything. It’s really so open that we don’t know what’s coming out of it. We’re just trying to improve the things, and all the people working here are constantly changing their scenes and making them better.

I asked specifically about whether they were working to standardize these interactive structures, but Gunter emphasized they’re mainly keeping it open. And that’s important to note here – the actual “scenes” are completely open-ended, limited only by what you can do with the target hardware and the objects in Pd approved for the project.

You have a sort of chicken and egg problem. It’s really hard to make a structure before you know what these things look like.

What he could promise was growth – and on more devices than just the iPhone.

Now it’s the iPhone. In a year, I hope … more. There are sprints happening everywhere.

Michael Breidenbruecker initiated the project, now joined by a team of musical and technological thinkers and coders, with a select group of backers with experience in new Web projects for music. As one of the original co-founders of Last.fm, Michael is familiar with what a platform can do for music listening. He’s committed not only to the free, open source model for the project, but to transforming the way people think about music making – even those who aren’t musicians themselves.

Michael: I think we are all just starting at this, in a way. The scenes that we have right now have a [deep] effect. If you’re producing music, maybe you remember the first time you played with an echo or with a delay. At least for me, I spent ages pushing the button and going "poo, poo." For many people on the street, or what I experienced at Burning Man [with the RjDj], people were really going crazy because it was the first time they had this interactive or reactive experience of music. Music was not just something fixed or something they could consume, but something they could influence.

Ever since Burning Man, I’ve known we have a reason to be on the planet, to do what we’re doing.

When you write about this or talk about this, it’s really hard for people to understand what it is. As soon as you put headphones on them, they actually get it.

Michael says that to make that connection with listeners, they first have to connect with artists – which means their challenge is not only evangelizing interactive and reactive music, but on the tool side, making Pd’s power more accessible.

Michael: The big task now for us … the couple of sprints we’ve had, and the people we have involved already, is just blowing my mind. And that’s something that we really actively want to push. In the next couple of months, we’ll have to do a lot of work on the composing interface. Pd is a bit abstract for people who are used to other production software. So that’s our job in the end.

There are a lot of people standing behind Pd, but in the art scene it’s totally … inadequate. If RjDj can bring the whole idea of Pd and interactive music closer to the market, that would be really great.

We are trying to keep it as free as possible. It makes a lot of sense to use and reuse things. All the stuff that’s done should be provided to the community. We have it all on a public SVN [Subversion, a free, standard server tool for tracking changes to code and collaborating on projects]. All we can say to the artists is, if you don’t want to share it, don’t put it up there now.

Artists selling RjDj scenes could be very possible in the future – and wouldn’t necessarily conflict with providing open-source patches for those savvy enough to run Pd. But so far, Michael says the project is driven by imagining a new shift in music more than a new business model. And, interestingly, the ideas behind RjDj predate the now wildly-successful Last.fm, which was acquired last year by CBS.

I had this idea for a project ages ago. I started to work on this thing in 99. In 2000/2001, I started up Last.fm. When I saw what was happening on the iPhone, I said maybe it’s time to start [this concept] up again. I tried to get a bit of structure, all of our investors.

Michael: To be honest with you, including the investors we haven’t yet said, this is our business model, not at all. We just know we’re working on something new which we think has potential for the future. We’d [be happy to] manage to get the idea of reactive music booted, in two, three, four years even, to see a shift in the music market. So people who are now listening to MP3 songs could also be listening to reactive music, and something that’s customizable, highly dynamic, and personal. We would certainly try to be the driving force in that development, that market. Right now, all we can do is try to make the product as good as possible, that the person from the street would be able to listen to it and enjoy it, and artists would enjoy doing scenes.

I can tell you how the idea was born. It was actually one of these stupid things. In the 90s, people started to wear earplugs to raves because they were so loud. They had to protect their ears. Then I saw people who actually had microphones on their ears, and I thought, wow, that’s crazy. They have a microphone and a headphone, so what they hear is filtered. I found out that’s not what it was; it was a binaural microphone. I thought it was like sound glasses. I thought that was great. Eyeglasses for your ears.

Changing the medium, Michael notes, does transform what music can be – for musicians, as well. They have hooked up RjDj to a P.A. at parties, taking care to avoid feedback since RjDj scenes often make use of the microphone as an input. Even networking is potentially on the table, for collaborative scenes, though no development has taken place yet. (Pd supports networking, so that’s definitely something that could happen, with control data beamed between different devices running RjDj.) In the meantime, RjDj poses problems you might not even have imagined.

There is another interesting topic which we haven’t solved yet. You have the RjDj scene, and your sound experience is in the boundaries of that scene, but what you’re actually hearing is totally individual. That’s something that you can record on the rjdj. What do the artists — if a listener makes a recording of his scene which is very private, it’s his voice, his environment, what about that? Who’s the owner of that?

It’s scalable uniqueness — the RjDj scene, you can copy it a trillion times, it’s still the same, it’s a copy, but the individual experience listening to it. and tha’ts something traditional music is fighting. You have a digital copy of a recorded track. The musical industry wanted that scalable; that’s why they made that digital format, the digital CD. So they had this tremendous scalability, but then they started to realize that the uniqueness [is lost]. That was one reason why we did Last.fm at that time.

[Then] people started to realize they make music with objects. An instrument, it’s an object. But with digital music, music in a way became totally objectless. Look at the iPhone – in the end, it’s so miniaturized. RjDj is really bringing it back to the object. You know how this glass sounds [if you strike it], but with RjDj it sounds different. People begin to experience objects in a different way.

RjDj received its first official release today on the iTunes App Store. Software is available for free, or as an “album” for US$2.99.

How to Create Scenes (And incidentally, you can work on scenes with a laptop even if you don’t own an iPhone. Testing on the device is, of course, very nice – fellow iPod touch users, I’m working on finding out how that mic solution is coming for us!)

So, for the record, *strong* recommendation: if you run apps on your Apple mobile, delay OS upgrades just as you would on a computer, until you're confirmed compatibility.

And I find it frustrating that, because of Apple's policy for rolling out firmware changes and for having to approve any upgrades via the store, fixes are happening way too slowly.

Anyway, that rant out of the way, this particular problem — Michael says the devs are looking into it. Stay tuned. Apparently the crashes occur on some systems that for others are fine, so perhaps not *all* 2.1 firmware users.

We'll be sure to post an update when it works. Life with software.

PeterPan

Pardon my ignorance, but i thought you could only code an iPhone application using Objective-C as a language ? How can u develop with PD ?

Also, how do you download the upcoming scenes in the application (once they become available) ?

They ported Pd to the iPhone platform. I think more accurately, you need to use Objective-C to access iPhone-specific APIs. The machine itself will run any code compiled for the platform.

Scenes: my understanding is that you'll purchase/download via the store, as with the Single version here (or perhaps more Albums for multiple scenes)

Glad to hear it's working for you under 2.1! It sounds as though there are bunch of new testers thanks to the popularity of the downloads, so that should help iron out these bugs.

Peter

me

I had the crash issue on a 1st gen. iphone, and did a restore of the iPhone software and the rjdj started working.

http://virb.com/mapmap marc

i just tried the same thing and it now loads up fine…

but the volume is so low.

i can't win.

Eoin Rossney

I read the title as Exclusive RDJ (as in Richard D James) interview

But this is cool too!

http://www.barrythrew.com barry threw

"team members have been behind everything from the port of Pd to Linux"

that was Miller's first platform, no port necessary

http://www.rjdj.me Frank Barknecht

@all_who_experience_crashes: We received some feedback now from users where the RjDj app wasn't working or crashing. Our (small) team is working on a solution, but it would help a lot if you could post some info (as iPhone version, fw etc.) to the ticket here: https://trac.rjdj.me/ticket/32

Some users were successful in fixing the issue by doing a restore of their Phone.

http://www.davidkanaga.com David

It pains me that I only have the Touch and can't get this!

That said, from watching the video, it looks like some of the scenes (i.e. Amen Shake) use just the accelerometer…

Does anyone know if Touch users can download the app and have access to scenes that don't require the mic?

http://Hundertmarknow.bplaced.net HUNDERTMARK BLOG

Incredible app. as an beta tester i did not have to pay this best app. whats definitly missing, is an export/mailing feature, but i think they'll fix it.

HUNDERTAMARK

Visittt my Blog

http://virb.com/mapmap marc

does anyone know if it only really works with the iphone headphone/headset? the built in mic (at the bottom of the iphone) is so quiet.

For those having issues with rjdj crashing don't download this app directly to your phone through a cell phone network(edge,3g) I did this and rjdj would crash upon launch. Download directly to your computer then transfer app to phone via sync. I first deleted the program from the phone. Then downloaded via wifi to computer than synced. No need to do a full restore. As for low volume it seems this program only works well with the apple supplied headset with built in mic.

http://www.createdigitalmusic.com Peter Kirn

@SUN1: interesting. Yeah, I heard the low volume issue on a friend's device. It was almost entirely inaudible. He was in fact not using the Apple-supplied headset. Now, of course, theoretically that shouldn't matter, so not certain what the issue is here. Stay tuned…

Josh Peterson

LOL

"and that's something verrry similar to…the effect of drugs

its so inspiring to know that people who do drugs can still develop groundbreaking software…

http://www.lmd64.com Lmd64

Just to give hope to other installers, I've a 16GB iPhone 3G running v2.1 OS, was able to download RjDj over the cell network, up and running without any problems whatsoever, works like a charm, and an incredibly impressive application it is too. My ukelele has never sounded so interesting through it

Now, all I want is some kind of multi-way cable with a stereo jack out (or better yet, phonos) for hooking up to a PA system, and a jack socket for cable input from a microphone or similar. It could make the iPhone an interesting part of a live performance rig, basically a ultra-portable Pd-based audio processor

This app is fantastic. I had this idea a while ago with distorted video feedback goggles as well but it never occurred to me to do it with the iphone. I was a little disappointed with Echolon at first but I bought the full album and listened to Eargasm while walking around Chicago and it made my whole day better. WorldQuantizer is great too. My favorite part was walking past people talking and having them continue to talk musically in my ears. It makes interacting with people interesting too like you always have your own inside joke. I also enjoyed having my friends put on the headphones and then say something, they were always like "what is this?…Woah".

I definitely want to start developing for this and it's great that you can with PD. Something integrating video would be cool too, like using these overpriced ipod glasses http://www.myvu.com/ , though you'd have to hold the iphone camera in front of you the whole time.

sonny reese

Is it possible to get rjdj 'songs' off my iphone and onto my mac, and how? thanks. this is a great live looper/recorder. now you too can make your very own nurse with wound track. i hope this is a first step in somethign more like a workstation, maybe some drum/trigger pads on the touch screen…

Ok, this looks and sounds great. I want to start working on an idea. I know a bit of Pd but i can't get anything to work. I think i'm missing the Rjlib but i simply can't find any download link. Am i missing something here? I must be getting old…