The research of WAV/PCM compression performance is initiated by studying
the LAME MP3 codec. The feedback let us determine the software, its
options and other stuff. As a result, we decided to extend the test
technique at the expense of new formats and codecs and develop a script
(primitive but efficient, bat) to automate testing. I must say that
many users out there underestimate the "small scripting" based
on bat files in the Windows. As a result, we have selected the following
codecs and formats which are more or less popular:

Formats, codecs and software

LAME 3.93 is one of the most popular MP3 codecs. We used the one from SMPMAN.
It's one of the fastest though it's not that popular among Windows users
as the version from
MP3-Tech.org.

The GOGO 3.12 is less popular but it's also a very efficient and well optimized
MP3 codec (it supports MMX, 3DNow!, E3DNow!, SSE, SSE2 instruction sets
and SMP systems). Unfortunately, the developer makes available only
source
texts, and the version we used before can now be downloaded from our
site.

The OGG coder is a dynamically developing and efficient OGG
Vorbis format from the Vorbis
Tools 1.0.1. This standard delivers high audio quality at a relatively
low bitrate; besides, it's open and free.

We shouldn't forget about Microsoft's formats, and the one we selected
is Windows
Media Audio (latest 9th
version); it works in two modes: VBR (by the way, the Windows Media
Encoder 9 is the only software that we know that supports two-pass (!)
VBR audio encoding), and Lossless Compression.

Encoding options

We tried and discussed a lot of various LAME presets and
came to the conclusion that tastes do differ :). Some prefer altpreset,
others r3mix and so on... what should we do? We decided that the final
set of parameters should be as simple as possible: <-m j -q 0 -V4
--silent>. Joint Stereo mode, CBR high quality, VBR average quality,
and no diagnostic messages (it can also affect the speed...). The codec can
choose the most optimal bitrate from the range of 32 to 320. The histogram
obtained with these parameters looks ordinary: as soon as the codec gets
a change to choose a bitrate itself it chooses the golden mean - the most
part of the file is encoded at 160 Kbps:

It was much simpler to choose options for the GOGO as
it's based on the LAME code. So, we took the same settings, except minor
details (small "v" instead of the capital one, and one minus instead of
two before "silent"): <-m j -q 0 -v 4 -silent>. The speed and
the size are as expected different as the code wasn't just copied.

The OGG Vorbis codec has one interesting peculiarity -
in case of the default encoding it works in the VBR mode (or it's rather
closer to ABR). The developers recommend setting the desirable quality
instead of the bitrate. We followed their advice, and now the options look
even simpler than before: <-q 5 --quiet> - average quality and
no diagnostic messages.

Windows Media Audio is a kind of a terra incognita for us, that
is why we chose the known bitrate, sampling rate, bits... so that it looks
like the good old MP3 :). We didn't use a command line in this case as
the Windows Media Encoder has a convenient GUI configurator for creation
of profiles which can be then given to the encoder in a compression command.
Here are the screenshots:

VBR encoding settings

Lossless compression settings

About test script

The test script and the respective software which allow measuring the compression
time automatically with all the codecs described above can be downloaded
free of charge. The distribution packet (one bat file and several executable)
includes the EXE file in the form of a self-extracting 7-zip file. When
started, it can be unpacked into any directory, into the folder named audio.
The root directory gets !audio.bat which, if started without arguments
in the command line or with arguments unknown to the script, displays information
about them.

The distribution packet doesn't contain the following software. First
of all, it doesn't include the test WAV file because you would hardly be
glad to download the test technique with this weighty 146MB WAV file. Besides,
it might overload the server if there are too many users willing to download
it. But if you want to place this file on your own server and make it freely
available, email us. We can suggest
that you make a WAV file entirely identical to ours yourselves: just grab
from an audio CD the composition named "O`malley`s bar" from Murder Ballads
album (performers are Nick Cave & The Bad Seeds). In our case at the
sample rate of 44100 Hz and 16bit sampling the file is 146 MB (153275180
bytes), the play time is 14 min. 28 sec.

Otherwise, you can just place into the root directory (where !audio.bat
is) any WAV file of the PCM format and name it TEST.WAV. The results
will hardly be the same even if the hardware coincides, but if you want
to compare different systems, the outcome and conclusion must be the same.
However, some believe that contents of the source file also influences
the speed - well, if someone proves it, it will be very interesting to
us to know the details.

The distribution packet doesn't include the Windows Media Encoder 9
required for testing the Windows Media Audio encoding performance (only
for these tests!). You can download it yourselves - it's freely available
on the Microsoft
site. You can change settings of the Windows Media Encoder at your
own risk, but we didn't change any, that is why we do not guarantee flawless
operation of the script in other cases.

All other codecs are included into the distribution packet in the version
necessary for running the tests.

For aesthetes...

...If you are interested in the script source texts and commands, we will
briefly explain functions of some utilities and some operations in the
script which do not look obvious.

Contig
is a freely available defragmentor which allows defragmenting certain files
and/or folders and it runs from the command line. Although this Mark
Russinovich's utility is pretty old, it successfully works even in
the Windows XP. Contig uses standard functions of the Defragmentation API
of NT based systems, that is why it's useless for the Windows 9x. We hope
it's clear what defragmentation is needed for.

Sync
is another Mark Russinovich's utility and is an alternative of the well-known
unix utility with the same name. Sync resets the disc cache and does it
before the script "forcedly" caches useful information (see further about
copy ... nul).

Timer is Igor
Pavlov's utility which measures time of operation of any program (the
way it can be used is clear from the bat file). This is one of the best
programs of this kind, it displays not only the running time but a lot
of other useful information.

copy /y test.wav space.tmp followed by defragmentation and deletion
of space.tmp file is needed to clear a part of the defragmented space for
files created while the script is running. The space freed is not small,
but more is better than less...

copy test.wav nul caches test.wav before running the compression
speed test so that it can be read quicker by the encoding program. It's
necessary to weaken the influence of the disc subsystem on the outcome.
The file is copied to nowhere because copying doesn't matter - what is
important is to have the file read before starting the test.

While the test is running no encoded files are created, because the
encoders are output to nul in order to lessen the effect the disc subsystem
has on the outcome. If you are interested in the real outcome just replace
nul with a certain file name in the respective script line.

Timer (in the logs) reminds Process Time = 66.296 = 00:01:06.296
= 128% i.e. it indicates that the process time exceeds the overall
time of the program execution, it testifies that you have an SMP system
and the application has created several streams. In this case you can see
the total time of execution of all streams, and if there are more than
one stream, the time exceeds 100%. Such situation is typical of GOGO and
Windows Media Encoder on systems based on two CPUs or on the Pentium 4
+ Hyper-Threading because these application can use more than one CPU.

About support

Undoubtedly, we are interested in your testing this packet. And we welcome
all facts about bugs, errors etc., as well as your suggestions about options,
software, versions etc. But do not expect all errors to be corrected at
once, because the test packet is meant primarily for testing audio compression
performance in our lab, and the most important for us it to have this packet
running correctly in our lab.

It doesn't mean we refuse to provide support, but it will depend
on the workload we have. Anyway, you use this script at your own risk
and we are not responsible for the outcome (though we do our best
to prevent any problems). Here you can always
find news, changes and updates.

Using the test packet by other mass media and commercial organizations

Of course, everyone can use it. But if you decide that it's simpler to
use a finished program than to develop your own software, you should mention
the developer's name -iXBT.com.
Firstly, you express your thanks, and secondly, it can be useful for the
readers as they can always compare a lot of results obtained within the
open test technique by us, you and the readers themselves, and for that
they must know what technique is used.

The only limitation is that we do not recommend making the test packet available
for download on your own sites - you'd better give the link to the
official support page. Undoubtedly, there will be bug-reports and
suggestions about modification of the test technique, that is why
it's better to let everyone have the latest version.