Real-time mp3 recording, part 2

Abstract:

This article is the follow up of an earlier publication concerning
making real-time mp3 recordings of e.g. radio broadcasts. In the
previous paper, it is demonstrated how a large mp3-file is
obtained, e.g. 650 MB for 12 hours or 1.3 gigabyte of 24 hours of
music approaching CD quality (128 kbit/s; stereo). How to handle
such a large file? Is it possible to make even longer recordings or
to compress the music even better? What about the quality of the
mp3 recordings? We will focus on answering the questions above.

Editing mp3-files

Assume that you made a beautiful recording of 24 hours, e.g. the Eighties Top 300
as broadcasted by Loostad
radio, and you want to split this file in smaller, thus more
suitable, files. During the splitting process, you also want to
remove advertisements and news broadcasts. How can you do this
best? One way to do it would be:

Decode the mp3-file to the wave-format.

Use a wave-editor for the better 'cut and paste' job.

Encode the edited wave-file back to an mp3-file.

This procedure would theoretically work and as opposed to common
believe the re-encoding of a decoded mp3 signal with the same mp3
algorithm does not result in a loss of quality. It has however
in practice a few problems:

Decoding an 650 MB mp3-file to the wave-format results in a
6.5 gigabyte file (compression is about a factor 10). I am not
even discussing the required processor time!

During decoding, the original 650 MB file remains. For
editing the decoded wave-file, some more disk-space is needed. If
you want to cut the whole file in parts of one hour, you will
need 2x6.5=13 gigabyte!

In short, you need a program that is capable of editing the
mp3-files THEMSELVES! Such programs do exist, but not (for the
moment) for Linux. Quite a number of such Windows programs are
available, but most of these programs have one big problem: they
cannot handle an mp3-file with a size of 650 MB! I have found one
program that is able to do the job, if you are patient:
'Manns' mp3 edit'. I use this program in Windows98. In the
graphical interface, you can indicate the begin and the end of the
desired piece from the original mp3-file. You can copy this piece
to a new file with the 'trim' command. This program is
'donationware', which means that you are not obliged to pay for it,
but you are kindly asked to do so if the program suits your needs.
This program has two drawbacks: it is awful SLOW on a 650 MB file
and it can only handle CONSTANT bit-rates!!! Searching the beginning
of the final hour, e.g. in an mp3-recording of 24 hours, takes a
long time and includes heavy disk access. Trimming the desired
piece of mp3-file, on the other hand, is very fast (without losing
mp3-quality). Until now, I have not encountered any problem with
mp3-file playback after using this editor. Nevertheless, I always
check the final trimmed files by loading them in the mp3-editor. If
loading works correctly, the trimmed files are O.K. During my first
experiments with this editor, I already clicked on the
'trim'-button while still heavy disk access occurred. This resulted
in damaged mp3-files that cannot be played back! To give you an
indication: editing a 650 MB file takes me roughly 1 hour, mainly
caused by waiting (which can be quite frustrating sometimes)!

Making long mp3-recordings (>24 hours)

It is no problem to make 24 hours recordings from a radio
broadcast. The file remains reasonable small (e.g. 1.3 gigabyte for
24 hours) and can easily be stored on the harddisk. This
problem changes if you want to make a recording for e.g. for a week.
Presumable, it won't fit on your harddisk and you want to overcome
this problem. I encountered this problem with the broadcast of the
'Alltimes Top 2000' during Christmas and new year 2000 (144 hours
of music!), broadcasted by a Dutch national and public
radio-station: Radio 2. My
harddisk was on beforehand far too small to store all recordings,
so I had to solve the matter in a different way. Here you have the
recipe:

Mount this file (or more precisely: this image), so it can be
accessed as a separate Linux disk.

Make an 12 hour mp3-recording on this image (128 kbit/s;
stereo; about 670 MB).

Unmount this image, mount a second image, continue the
recording on this second image.

Copy the first image to CD (using the CD recorder). This is
not any problem during real-time mp3-recording on my computer, a
Pentium II 450 MHz (no XWindows running).

If the CD has been created successfully, delete the mp3-file
(in the image; not the image itself!). This image can be used to
make a new recording.

Repeat this procedure until the recording is finished.

I have used the above procedure to record the whole Top2000
completely and without any problem! I used 12 backup-CDs that will
never be used anymore, but you have to make some expenses to get
these kind of successful recordings. How does it all work? I
obtained the idea reading the CD-Burning-Howto
and the necessary steps are explained below:

I created 4 images on a separate partition. I happened to
have 2.9 gigabyte free.

I created 4 images, respectively called
image_1, image_2, image_3, and image_4:

dd if=/dev/zero of="image_1" bs=1024k count=700

I repeated this command for all 4 images. Creating the images
takes some time!

Note: the ext2-filesystem also uses some disk space, so you do
not have the full 700 MB available for storing the
mp3-recording!

To mount the image, you need to use the loopback device. This
requires support for the loopback device in the kernel (or
making the loopback device available as a module).

mount -t ext2 -o loop=/dev/loop1 image_1 /mnt

You can access the image via the /mnt-directory. Now make any
recording you like using this image!

Unmounting the image and creating the CD:

umount /mnt
cdrecord -v speed=2 dev=0,1,0 -data image_1

Please note that the command 'dev=0,1,0' depends on your system
configuration and might be different for your particular
situation (cdrecord -scanbus, will give you a clue what it should be).
The image is being copied to CD. In the mean-time,
you are already continuing the mp3-recording on the second
image (hopefully).

Testing of the created CD:

mount -t ext2 /dev/scd0 /cdrom

If everything went right, you can see the recorded mp3-file via
the /cdrom directory! I want to remark that I encountered some
problems with kernels 2.4.0 to 2.4.2. I was unable to mount
these CDs (with the ext2-filesystem) and I encountered problems
with the loopback device. In kernel 2.4.3, all these problems
are properly solved and everything functions correctly (as is
the case with the 2.2.x-kernels).

The benefit of this approach is the limited processor usage,
needed for creating the mp3-CDs. The images already exist! You can
automate the whole procedure by using a script: mounting the
images, starting the mp3-recording, finishing the mp3-recording and
unmounting of the image after 12 hours of recording, mounting a
second image and continuing the mp3-recording, and so on. I burned
the CDs by hand, just in case problems occured (which was true
once, due to faulty media). For this reason, I created 4 images: 3
spare images in case I encountered CD burning problems (giving me
maximally 36 hours to solve the problem). The script I used looks
like this:

I did everything WITHOUT XWindows, using the Bash-prompt.
I loaded the sound support by hand, as well as the
OSS emulation (necessary to get correctly functioning ALSA-drivers)
and the mixer driver (of course you can automate this by using a
pre-install/post-install trigger in /etc/conf.modules). T
The 4th line configures the mixer correctly: recording
source is 'line-in' and the volume has been maximally opened. After
setting these options, the script handles the mounting and
unmounting of the images.
Concerning the mpegrec options: '-mj' indicates a joined stereo
recording. I did this because I expected smaller mp3-files using
this option (it should all fit on 670-680 MB). This appeared to be
untrue: it does not matter if you use stereo or joined-stereo
recordings! When I make new recordings, I will use ordinary stereo,
The 'mpegrec'
program should stop after 12 hours. To achieve this task, I adapted
the source code and changed the maximally 24 hours (which is simply
stated in the source code) to 12 hours. Unfortunately, this
approach did not work: the program continued recording after 12
hours! I had to break the recording every 12 hours (you won't get
it for free...). The script took care for the remaining parts. I do
not understand a thing from 'mpegrec' anymore: I should study the
source code carefully...

Note:
The most recent version of cdrecord (cdrecord-1.9) caused me
problems with creating the CDs. The previous version,
cdrecord-1.8.1, operated without any problem. I solved these
problems by upgrading the firmware of my CD recorder, a Philips CDD
3610, from version 3.02 to version 3.09. Initially, I expected that
the CD burning problems were caused by kernel 2.4.3, but this
appeared to be untrue!

The quality of the mp3-recordings

Due to the feedback on my previous article via the English Talkback
page, I received a link to a website containing interesting
information concerning mp3-recording. Especially the mp3 quality
and the connection with the mp3-bitrate is excellently explained,
focusing on the program 'lame'. The address is: http://www.r3mix.net/.

Summarizing the most important facts:

A sampling rate of 128 kbit/s is NO CD-quality. A listening
panel could successfully detect all 128 kbit/s mp3-files. For
making recordings of radio-broadcasts, this should not be any
problem: radio broadcasts are not in CD quality and 128 kbit/s is
more than sufficient for this specific situation!!!

A sampling rate of 256 kbit/s gives you REAL CD-quality, in
other words: it is not possible to separate a 256 kbit/s
mp3-stream from the original CD. Despite this fact, it is still
possible to detect some compression artifacts, but only if you
know which music-aspects to consider (as an audiophile)!

Lame recordings with a variable bitrate (VBR) are
recommended. The standard bitrate of 192 kbit/s (a value which I
considered 'weird' in my previous publication) is a good starting
value. Depending on the 'difficulty' of the song, the sampling
rate is increased or decreased. A second benefit is the
application of filters, giving a better mp3-file compared to the
256 kbit/s recording (without these aforementioned filters). It
is indicated that Philips
specified these filters during the development of the CD
recording technique and that these filters can be used without
any problem!

All this is discussed in detail in the above mentioned website!

Creating as small as possible mp3-recordings

Below, I will discuss a script that has been submitted by a reader
(via email). The script looks as follows:

The script's main goal is to obtain
understandable radio recordings that only need 3.5 megabyte/hour
diskspace! With understandable I mean that one should be able to
interpret the recorded speech. Off course, the quality of the
music is awful! The first line in the script configures the mixer
and starts a radio-card program and 'sox'. The 'trick' resides in
the 'lame'-options, which we will investigate now:

--resample 16 -b 8 -a -m m -h --lowpass 4 --highpass .12 --voice

--resample 16

Redefine the sampling frequency from the output file from
44 kHz to 16 kHz.

-b 8

Set the sampling rate on 8 kbit/s (gasp!).

-a

Combine the stereo stream to a mono recording.

-m m

Force mono mp3 encoding.

-h

An option to use high-quality mp3-encoding (slower).

--lowpass 4

Set the lowpass filter on a value of 4 kHz. All signals
(frequencies) higher than 4 kHz are removed.

--highpass .12

Set the highpass filter on a value of .12 kHz. All signals
(frequencies) below .12 kHz are removed.

--voice

Use an mp3-compression technique optimized for speech.

It is clear that a high compression can be obtained for speech.
Most of the times, this will be undesired. Now, at least the
possibility is indicated!

A standard script to create mp3-recordings

This script was sent via the TalkBack pages
(I am grateful for that; everyone can profit from this
information!). This is the large benefit of the TalkBack pages that
are made available for the previous article (and also for the
current article). Everyone can post his or her experiences, ask
advice to other visitors if troubles arise, in short: a tremendous
amount of information is brought together and is (hopefully) made
available for everyone! The script, called 'mp3_record', looks as
follows:

The option '-l 5400' sets the recording time on 90 minutes. All
options after '-x' are directly piped to 'lame' (lines 4 to 9) and
the '-o' option saves the file as 'date_and_time.mp3'. These lines
can be adapted as one likes. For this purpose, it might be
useful to consider the 'lame'-options more closely. The current
options are explained in the table below:

--preset fm

Lame knows some standard settings. 'fm' is meant for making
fm-broadcasts recordings, but I dislike the recording-quality
compared to the standard sampling rate of 128 kbit/s. The main
cause are the applied filters in 'lame'.

--tt "Computers in Your Life"

'--tt' define the title of the recording (30 characters
max).

--tl "Open Line with Tom Wieble"

'--tl': album, or: original source (30 characters
max).

--ta "Tom Wieble" --ty `date +%Y`

'--ta': artist (30 characters max).

--tg "Speech"

'--tg': genre (name or number).

--tc "Copyright (c) 1997 - `date +%Y` WOSU"
-c"

'--tc': comment (additional information; 30 characters
max).

In short: a nice way of creating an mp3-file, including
recording date and time and incorporating all kinds of additional
information. It requires some time to write everything down into
a script, but it is nice to see that all these things are
possible!

Talkback form for this article

Every article has its own talkback page. On this page you can submit a comment or look at comments from other readers: