You Can Take It With You

I've always wondered just what went into creating mobile audio and how to get into the development community. Mobile audio these days is very similar to computer audio nearly 20 years ago. There are hundreds of different kinds of devices, each providing different audio capabilities — from beeps to MP3s. Considering that ringtones take up about 10 percent of the world's music-publishing market, according to recent reports, and that there are more than 30 million iPods and that nearly every videogame publisher has a mobile department, this is an area worth exploring. People are taking music with them in ways that the original Walkman couldn't have portended.

And why shouldn't they? Quality is improving and you can't beat the convenience. But let's get real: With this unprecedented freedom comes a market scrambling for standardization. The Highlander phrase applies now just as much as it did with VHS and Beta: “There can be only one.” The closest thing to the “one” at the moment is the ubiquitous MP3 (though iTunes, the world's largest digital music store, uses AAC and its copy-protected variants), but under the hood of most cell phones resides a widely varying set of playback mechanisms requiring a lot of patience to navigate. When you consider that there are more than 15 ringtone formats alone, we're talking a lot of patience.

Don't panic — getting into ringtones isn't too difficult if you combine knowledge of MIDI and creating stereo digital files (and 5.1, with the Samsung SCH-B570 cell phone) with a middleware tool like Ringtone Creator Pro from unwiredtec or mobileBAE from Beatnik. Considering the size of the ringtone market, this is a pretty low barrier to entry, and if you become an expert at converting music into the best possible sound over such tiny speaker systems, then the biggies at major record companies might just take you up on converting part of their enormous back catalog into ringtones.

While ringtones have captured the imagination of the national consumer media, they are only part of the mobile audio picture. Game development and manipulation of audio interactively on phones is a more complex task. And the market is only going to get bigger. To help answer some questions on this front, we visisted with Phil Sorger, president of Sandcastle Studios (Carlsbad, Calif.), one of the country's biggest mobile game developers.

Sandcastle Studios develops mobile games for a wide range of phones and handheld devices appearing on North and South America's largest carriers, including Verizon, Sprint, Cingular, T-Mobile and Vivo. Publishing and technology partners include MLB.com, Electronic Arts, Blaze, Qualcomm and Sony Pictures Digital.

“For mobile audio,” Sorger says, “Sandcastle starts with assets provided by the publisher or commissioned to external composers and sound designers, and then converts those assets as needed to the target platform. For Qualcomm chipset devices, Sandcastle uses the BREW SDK and CMX Studio to prepare sounds for the appropriate chipset. For Java-based phones, we use device manufacturer's tools — like those available from Sony Ericsson and Nokia — to convert and create game-ready files. Since most devices still support General MIDI, Sandcastle applies MIDI composition and sequencing tools to maximize the emotional impact and still work within each device's audio constraints.”

But not every project requires pushing the limits of a device's audio capabilities. Madden 2006 for EA Mobile blended voice-overs from John Madden and Al Michaels with grunts, groans and tackle sounds, as well as a MIDI menu score to create a rich multimedia experience. At the same time, CBS SportsLine Fantasy Companion has no sound at all.

FORMATS, PLEASEEither way, sound or no sound, the need for standardization is accelerating. Kids on Sprint want to play with kids on Verizon. And music publishers would obviously like to control the quality of ringtone downloads across all formats to keep the buyers coming back. So just who is handling formats for the numerous phone types? You can't just send a WAV file to a phone and have it play back. You need to send several dozen different kinds of WAV files using more than just Pro Tools or Cubase, as Sorger has demonstrated with the use of BREW and CMX Studio.

To help tackle the format issue, the Interactive Audio Special Interest Group (www.iasig.org) has formed a Mobile Audio Working Group, with representation from companies such as Philips, Dolby and QSound. To discuss the IA-SIG's effort, we talked to Brad Fuller, Steve Horowitz and Jim Rippie of MAWG. The organization has two goals: to discuss methods of interactive composition on mobile devices and to publish recommendations and guidelines for production techniques.

But MAWG isn't a standards group, Fuller cautions. It's more of a recommendations body. The group plans to release a document soon, stating problems with mobile technology and recommended solutions. For instance, looping standards for MIDI don't exist, and MAWG members have established desired criteria for a looping standard.

“There are many parallels to early computer audio,” says Horowitz. “Just like back in the '80s, it's common to have a processor that specializes in digital signal processing to support sufficient audio features. Now and in the recent past, we're pretty accustomed to having plenty of horsepower on the CPU to direct to audio features. And in gaming, even the processor budgets dedicated to audio processing have relaxed a bit in recent years.” Mobile terminals are still in a conspicuous “do more with less” mode, and this is made even more interesting by the aggressive power-consumption techniques employed by handset OEMs and component manufacturers.

There's another element of “early computer audio” that's often overlooked: You hear your carefully crafted audio on miserably underspecified speakers, frequently in mono. Even worse, because handset designs change frequently, manufacturers sometimes swap a more capable speaker for a cheaper one, either for cost savings or because they've completely switched suppliers. These switches sometimes push all audio on the device through a brute-force compression to guarantee loudness.

Still, development continues and quality will only get better, as evidenced by some of the high-end phones now on the market. According to Horowitz, “Each carrier has different high-end phones like the LG8000. T-Mobile and Cingular both carry those, but every six months this changes. The way the carriers make money is air time and the additional services, but the churn rate on phones is six months to one-and-a-half years, so they won't make money on the phone.”

At the same time, Fuller cautions not to place your bets on ringtones. “I think ringtone work will eventually go away for audio developers,” he says. “Most people are going to want the latest pop tune, and phones will be more capable of playing MP3s and the like. Many studios did very well porting ringtones to MIDI, but this is going to fade.

“[Mobile audio] is definitely tough. There are a lot of musicians that know MIDI and know how to use General MIDI. But there is a problem with mobile GM: It often isn't a complete set in many mobile phones. There are subsets of GM that you need to know, like guitar, for instance. Many phones have just one guitar instrument rather than the other GM guitars, so your composition has to take that into account.

“Nokia has several phones that contain a subset of general MIDI,” he continues. “So if you want guitar, you need to write for the single guitar to account for the subsets. You can't necessarily write a rock tune with distortion guitar that has a lot of bends in it because some of those bends are subsets, as well, and the range of the bends differ from subset to subset.”

If you want to get into mobile audio the easy way, wait until streamed and compressed audio becomes commonplace in cell phones and other embedded systems. (Yes, even GPS requires audio.) But for now, there are solutions for the large number of formats. As Fuller puts it, “They're formats. They shouldn't stand in your way.”

Alexander Brandon is the audio director for Midway Home Entertainment in San Diego, Calif.