Network broadcasting: streaming audio with RealServer and Icecast

If you've been trying out the examples from part one, then you should have a good idea about what's already out there on the Internet radio waves. But perhaps you're thinking that you could do a much better show than anything you've heard so far -- if only you knew where to start. Well, let me show you how you can begin your career as the next famous Internet DJ with some help from RealServer and Icecast.

RealServer

The complete version of RealServer (RealServer Plus) for Linux is commercially available from RealNetworks.com. It's a proprietary closed source Internet broadcasting solution with a fairly hefty price tag. However, bear in mind that RealServer Plus is industrial-strength software and is certainly more than you are likely to need for desktop network broadcasting.

Thankfully, RealNetworks has provided RealServer Basic for evaluation purposes. This basic server is an excellent package for personal broadcasting, but it's not an all-in-one solution for streaming RealAudio. You'll also need the RealProducer content-creation software in order to make your RealAudio files from pre-existing sound files or prepare live netcasts in RealAudio streams. Again, Real has graciously made available a free basic version of RealProducer. We will use the free versions of both RealProducer and RealServer for our examples here.

Download RealProducer Basic (the link to the basic version is at the bottom of that page) into a temporary installations directory and unpack it with tar xzvf realproducer_8.5_linux.tar.gz. Read the README file, then run ./install to set up RealProducer Basic for your system. Once RealProducer is installed, you can start converting or creating your own content in RealAudio and RealVideo formats. We'll return to RealProducer, but now let's get a server to send our masterpieces out to the networked world.

Retrieve the basic version of the server from the RealServer download site. The software arrives in a binary installation bundle without execution permission. Run this command to make the installation binary executable:

chmod +x rs8-0-linux-libc6.bin

Next run (as root) ./rs8-0-linux-libc6.bin, follow the instructions at the prompt, and within a few moments your RealServer Basic will be fully installed and ready to use.

Now let's create some content to stream from our new RealAudio server. You can use RealProducer to convert existing soundfiles with this simple command:

realproducer -i first.wav -o first.ra

which will create an RA file with default values for a variety of other options. Here is a more complex command sequence (with line breaks added for clarity) along with its output diagnostics:

The output message reveals the obvious meanings of the string options; run realproducer --help for a complete list of the available command options.

Note that the encoding took five minutes, achieving a 3:1 reduction in file size for a 6.5 minute high-quality stream. RealProducer's efficiency may be subjectively evaluated by this comparison of input and output file sizes:

You can also set up RealProducer for live broadcasting in streaming RealAudio. This very simple command line does the trick:

realproducer -l 2:0

where -l indicates a live feed and n:n selects the audio capture source. RealProducer handles live input from WAV or AU soundfiles, Quicktime MOV files, and input directed to /dev/dsp. The example above uses this last method, setting input to the microphone input to your soundcard. 2:1 selects the Line In of your card, and 2:2 chooses your CD player for input.

Finally, let's assemble a list of RA files to be streamed by RealServer. Open your favorite text editor with a file named something like my_mix.smil. SMIL (Syncronized Multimedia Integration Language) files are composed in a simple mark-up language and are easy to write. Here's a basic example using the loopback adress:

Note that you can symlink your RA files into the RealServer Content directory; you don't need to copy or move them.

This example simply plays the files in sequence, but SMIL allows much greater flexibility in the layout, timing, and even bandwidth choices for each audio source. Alas, we haven't the time to get further into SMIL, but readers interested in knowing more about SMIL can consult the excellent documentation in RealAdministrator. We need to keep moving, we're almost ready for our first Internet broadcast in RealAudio ...

Become root user (su root), go to your RealServer installation directory (/usr/local/realserver on my system), and verify that the installation process created the server configuration file, an XML textfile named rmserver.cfg. Accept the configuration defaults for now, and start the server with the following command:

Bin/rmserver rmserver.cfg &

A series of messages will scroll by. They are reports from various self-checks the server will make before startup, such as license verification and restrictions, the number of CPUs on the host, and a list of the media format modules loaded.

The start-up messages also report any errors from a misconfigured rmserver.cfg file. Configuration details are available in the RealServer documentation (in RealAdministrator/Docs) and in the RealNetworks release guide. That information can be most helpful: Thanks to the error report and the documentation, I learned that I needed to add this IP binding before the server could start:

That setting required a command-line flag, as did a bypass for an error with my /etc/hosts file. At last, I had my working command sequence to launch RealServer:

Bin/rmserver rmserver.cfg --sct --iehp &

Once your server is running, point your browser to the RealServer administration account via the loopback URL:

http://localhost.localdomain:26122/admin/index.html

The admin port number is a random number assigned during installation. See your rmserver.cfg file for the correct value for your admin port.

From here you simply point and click your way through the server's administrative tools. The GUI is well-designed (Figure 9), help files are a click away, and the experience is totally painless. Alas, we don't have the space here to fully explore RealAdministrator, but I must commend RealNetworks for the obvious effort put into this excellent tool.

Figure 9. RealAdministrator in Netscape.

Fortunately, you can safely accept most of RealAdministrator's default values and move on towards our goal of producing and transmitting a streaming RealAudio broadcast; however, you should look at and understand the information regarding the server's mount points. Under the Configure topics, click on General Setup, then click on Mount Points (Figure 10).

Figure 10. Mount point description in RealAdministrator.

The base path indicated here is the path to your SMIL files, the point for requests from network clients for the audio stream. You can now verify the server's operation by opening RealPlayer and entering this URL in the File/Open Location dialog box:

rtsp://127.0.0.1/my_mix.smil

This URL assumes the existence of my_mix.smil at the mount point seen in Figure 10 and tests it via your local host loopback address (127.0.0.1). If the requested file is valid and is located at the configured mount point (/usr/local/realserver/Content is the default path), RealPlayer will pop up and begin streaming the first audio source in the SMIL list.

Once you have verified the server's correct operation, you're ready for network broadcasting. Open rmserver.cfg and change all references to localhost.localdomain (or 127.0.0.1) to your actual IP address. Do the same for your SMIL files, then restart your server with the new configuration. For my first network broadcasts, I changed the audio source settings in my_mix.smil from this address:

You will want the value for the inet addr, not for P-t-P. Also, note that I would now open my RealAdministrator with this URL:

http://64.108.112.104:26122/admin/index.html

Obviously, a static IP address is preferable: You enter the address once and leave it. Unfortunately , my address is dynamically assigned, so I need to change it in my server configuration and my SMIL files. I suppose a nice PERL script could take care of all that automatically, but I'll leave that problem for the more resourceful reader.

Regardless of how you update your IP address, you still need a way for people to connect with your server. Easily accomplished by adding this kind of tag to your Web page:

RAM files should be located at the same site as the Web pages containing them. Now clients can connect to your server via Web links or by entering URLs into RealPlayer's File/Open Location dialog box. You can also configure your links to call and stream to a browser's RealPlayer plug-in, but I leave that procedure as another exercise for the diligent student. If you made it this far, then congratulations are in order: You have successfully installed, configured, and deployed RealServer for streaming your media to the Internet.

Icecast

Icecast is a free open source streaming server licensed under the GPL. It is easy to install, configure, and use, making it possible for anyone with sufficient software and hardware resources to become an instant Internet broadcaster. Icecast 1 supports streaming MP3s and M3U playlists, and version 2 supports streaming Vorbis files. That news should come as a welcome relief to broadcasters concerned about the eventuality of having to pay Fraunhofer for streaming MP3s over the Internet. Vorbis players and plug-ins exist now, with many more on the way, but Icecast 2 is the first streaming Vorbis server. We'll look at both versions of Icecast, starting with version 1.

Streaming MP3 with Icecast 1

Download the latest version of Icecast 1 from the link above into your $HOME directory and unpack it with tar xzvf icecast-1.3.8.beta2. Move into your new Icecast directory and run ./configure --help to look at your compile-time configuration options. Finally, run ./configure with your selected options, then make and (as root) make install. You will also want to download and build the shout utility available from the Icecast site. Unpack and build it following the directions for Icecast.

The Icecast installation will create a /usr/local/icecast hierarchy for its default directory path. The conf and etc directories are of particular importance: They are the default locations for your server and streamer configuration files and your M3U playlists. As with RealServer, your configuration files will need such information as your IP address, server port number, connection limits, and so forth. Complete configuration instructions can be found in the README and INSTALL files for shout and Icecast.

Let's prepare an M3U playlist for the shout streamer. M3U files are simple text files listing absolute paths to your MP3s. For instance, here's a section from my test playlist for this example:

The playlist is declared in the conf/icecast.conf file. When you have edited your playlist and configuration files, you can start the server by running 'icecast' in an open xterm. You'll see some messages like these:

Now fire up XMMS and point it to http://127.0.0.1:8000 to hear the stream. When you're ready for the network, simply change the loopback references in your Icecast and shout configuration files to your actual IP address, just as we did with RealServer, and you're set to rock the net. Of course, you want to make it easy for people to connect to your MP3 stream, so you might want to add a link like this one to a Web page:

<a href=http://connection.m3u>My Streaming MP3 Show</a>

where the connection.m3u file contains a single line redirecting the request to the actual mount point:

http://64.108.112.104:8000

Make sure that you place the connection.m3u file on the same web site as the page in which the link appears. Web surfers can then simply click on the link to listen to the sounds of your personal MP3 stream.

This introduction touches only the basic points of getting Icecast up and running. The system also handles live broadcasts, and you can reflect your broadcasts to a larger audience using the public server at yp.icecast.org. There's much more to Icecast, but I'll leave that exploration to the industrious student. Let's move further on and take a look at Icecast 2, the next generation. ...

Streaming Vorbis with Icecast 2

Icecast 2 is currently available only in a BitKeeper repository, which means you'll have to obtain and install the BitKeeper software. It is available in source code, but I recommend using the easy-to-install pre-compiled binary. Follow the download directions on the BitKeeper page to obtain your user ID and instructions. Log on to the download site, then retrieve the correct binary for your system (glibc 2.0 or glibc 2.1) and move it to a temporary directory ($HOME/tmp is good). Go into the holding directory, become the root user (su root) and run sh x86-glibc21-linux.bin to install the BitKeeper software.

You must also have the OggVorbis libraries correctly installed on your system. If you ran the OggVorbis examples earlier in this article, then you've already installed the necessary libraries. If not, head over to the OggVorbis nightly CVS, download the package, and install the software (as described in this article). Now you're ready to compile and install Icecast.

Note: You must use OggVorbis 1.0 beta 4 or higher to build and run the following examples.

Icecast developer Jack Moffit has placed online a set of instructions for downloading and compiling Icecast 2. The process of retrieving files from a BitKeeper repository is simple, but be sure to strictly follow the instructions on that page. Download the sources for Icecast and libshout, then follow Jack's directions for compiling them (./autogen.sh; make; make install should do it).

Before running the server we need to have some content to stream, so let's create a playlist of pieces in OGG format.

Open your favorite text editor and create a new file named "ogglist." This text file lists the locations of the OGG files you wish to play, as in this example:

Now go to your Icecast home directory and edit (as root) the conf/icecast.xml file to reflect your location, server name, connection port, and so forth. While still in the Icecast home top directory, start the server with this command line:

icecast -c conf/icecast.xml

You will see the XML file displayed, along with some debugging messages; then your Icecast2 server is ready to work.

Icecast will eventually use the ices utility to send Vorbis streams out to the server, but the current BitKeeper libshout package includes a prototype streamer (appropriately named "example") in the examples directory. Open the example.c program in a text editor and edit the following lines to reflect your build configuration:

These settings will set up the example binary to stream content to a virtual mount point named shouter.ogg at the loopback address. You can name the mount point anything you like, as long as it ends with .ogg. The password must match the password in the icecast.xml configuration file.

Run make to compile the example, then pass your Ogg playlist to the streamer with this command string:

cat ogglist | xargs cat | example 8000

The pipes route the file names from ogglist to xargs, which parses and cats each file to the example streamer at port number 8000. The server window will report on the connection from the streamer:

Now bring up XMMS (the CVS version described above) or ogg123 and point it to this address:

xmms http://127.0.0.1:8000/shouter.ogg

If you use ogg123 make sure you indicate an output device:

ogg123 -d oss http://127.0.0.1:8000/shouter.ogg

FreeAmp is another excellent open source streaming audio player. As of release candidate 2.1, Freeamp includes built-in support for streaming Vorbis; unfortunately, I did not have time to test it, but developer Robert Kaye kindly sent a screenshot of FreeAmp rendering a Vorbis stream (Figure 11). With the addition of FreeAmp, Linux users now have their choice of three streaming Vorbis players, and more are certainly on the way. But let's get back to your server ...

Figure 11. FreeAmp playing a Vorbis stream.

The server streams the OGG files from your playlist to the mount point, where the requesting client will receive the stream. Again, the server will report on the client connection:

Meanwhile, your player should note the connection, pre-buffer some of the stream (you can set the pre-buffering value in XMMS by configuring the Ogg-Vorbis input plug-in), and start playing. Behold the wonder of streaming Vorbis! (Sorry, I couldn't contain my awe... :)

Acknowledgements

The author thanks Jack Moffitt for his extensive and invaluable assistance with this article, particularly regarding the material on Icecast 1 and 2. Thanks also to my broadband neighbors Jack Paxton and Josh Flechtner and to the members of the Icecast and Vorbis mail-lists for their help with my real-world tests of RealServer and Icecast.

The developers of Icecast 2 are working hard to put the finishing touches on the public distribution package. Much of what I have presented here will likely change (i.e., get easier), but the example presented here delivers the proof of concept. High-quality streaming audio with Vorbis is a reality, and that is great news for anyone involved with network broadcasting.

Final thoughts

The pace of development is fast and furious in the world of streaming audio. Access to broadband is becoming more common, content creation is booming, and viable alternatives to patented and licensed streaming formats have begun to appear. Whether on the client side or the server side, Linux is poised to become the platform of choice for listeners and producers of streaming audio. The major streaming formats are already supported by Linux players, utilities, and servers, and even better things are coming our way fast. As we have seen in this article, the future of streaming audio is in good company with Linux: It's just a natural match.