Lossless audio formats comparison

This is page two of a three-page article. This page covers lossless audio formats. Page one is the introduction page, which sets out the test method and lists the CDs used in the comparison. Page three covers lossy formats.

Lossless codecs and settings

This comparison examines six lossless audio formats, five of which use lossless compression. Compression modes were chosen based on recommendation by the codec's creator. Failing such a recommendation, I looked for the consensus on audio discussion forums. Failing that, I settled on the mode I felt produced the best compromise between speed and resultant file size.

Wave (.wav)

Uncompressed Wave files are some people's idea of an ideal lossless format when archiving their CD collection digitally. Because the audio files are uncompressed, the file size is larger, but the rip-and-encode process should take less time.

FLAC is an open source codec that produces lossless compressed files. It has been around a while, and it's the format I use when I'm archiving my CD collection. Even though lossless files are relatively large, there are a small number of portable music players that support playback of FLAC files. There is also support for FLAC in several digital hi-fi systems. FLAC files support tagging to store track details.

When I published my last audio format comparison, I was worried that development on Monkey's Audio had dried up. It turns out I was wrong; the Monkey's Audio codec has seen version updates twice this year, so it's still very much in the game. Monkey's Audio files use their own APE tagging system to store details about the audio file.

Performance tables seemed to put this codec's speed in the same neighbourhood as FLAC's, and I wanted to see for myself. WavPack is also an open source codec.

Raw data

Rip-and-encode times

Firstly, the time taken to rip each CD album and encode it using each lossless codec. The time recorded was that reported by dBpowerAMP after the rip-and-encode process of each album was complete.

Time in seconds needed to rip and encode each album with each codec

Album

Wave

FLAC

WavPack

Shorten

Monkey's Audio

OptimFROG

album mean average

Life of Destructor

103

101

119

142

130

252

141.2

Exit Planet Dust

107

105

122

150

137

267

148.0

Demon Days

109

108

125

153

141

273

151.5

The W

123

123

151

176

162

313

174.7

Endtroducing. . . . .

126

127

153

179

176

328

181.5

The Best of Nick Cave...

136

146

192

203

204

381

210.3

Superunkown

140

146

187

208

209

381

211.8

Barrio Fino

140

150

198

209

212

392

216.8

Resident Evil: Apocalypse

135

151

196

211

211

398

217.0

The Essential Classics Collection

146

200

173

203

278

372

228.7

format mean average

126.5

135.7

161.6

183.4

186.0

335.7

188.2

Lossless file sizes

Next, the total file size of each album stored in each lossless format.

File size in bytes of each album encoded with each codec

Album

Monkey's Audio

OptimFROG

FLAC

WavPack

Shorten

Wave

album mean average

Life of Destructor

300,483,192

301,805,204

316,803,249

319,344,246

335,870,042

487,835,816

343,690,292

Exit Planet Dust

314,072,235

314,773,558

331,367,604

335,924,925

371,818,203

523,075,876

365,172,067

The Essential Classics Collection

282,403,706

285,372,817

303,111,263

300,311,766

310,475,191

742,460,908

370,689,275

Demon Days

321,027,835

321,722,606

343,758,092

344,219,955

371,766,015

537,879,540

373,395,674

The W

352,109,763

353,662,913

372,014,851

373,411,869

434,884,909

626,462,828

418,757,856

Endtroducing. . . . .

357,560,360

359,459,746

382,116,673

383,168,158

420,184,040

667,505,360

428,332,390

Superunknown

500,597,960

503,973,822

522,006,755

525,194,210

557,709,496

779,745,152

564,871,233

The Best of Nick Cave...

492,129,385

498,955,693

519,281,439

526,254,421

552,536,492

800,224,016

564,896,908

Barrio Fino

544,066,012

546,697,374

569,206,029

570,641,928

593,960,352

808,338,768

605,485,077

Resident Evil: Apocalypse

554,499,706

559,473,643

576,383,453

582,654,498

609,737,886

806,715,624

614,910,802

format mean average

401,895,015

404,589,738

423,604,941

426,112,598

455,894,263

678,024,389

465,020,157

Lossless compression rates

Finally, a look at how well each album compressed on average. This is measured by dividing the average file size of each album encoded to a compressed lossless format (so we aren't including the uncompressed Wave format) and dividing it by the number of seconds in that album's running time. This gives us the average lossless compression rate for each album, in bytes per second.

Compressed file size in bytes per second of album running time

Album

Average compressed size

Running time (hh:mm:ss)

Average bytes per second

The Essential Classics Collection

296,334,949

01:04:50

76,179

Endtroducing. . . . .

380,497,795

01:02:59

100,687

Demon Days

340,498,901

00:50:43

111,896

Exit Planet Dust

333,591,305

00:49:19

112,738

Life of Destructor

314,861,187

00:46:00

114,080

The Best of Nick Cave...

517,831,486

01:15:29

114,337

Superunknown

521,896,449

01:13:33

118,263

Barrio Fino

564,914,339

01:16:10

123,614

Resident Evil: Apocalypse

576,549,837

01:16:06

126,270

The W

377,216,861

00:49:04

128,131

Analysis of the data

Average rip-and-encode times

Overall, Wave is still the fastest lossless format to rip and encode a CD to. But I was shocked to see FLAC finish faster than Wave on three of the albums. FLAC took only seven percent longer than Wave, on average. A great improvement over the performance I saw from FLAC in my previous audio format comparison.

WavPack was next fastest, but couldn't keep up with FLAC. An extra twenty-five seconds taken for each album isn't too bad, unless you've got a big stack you want to rip and encode all in one go.

Shorten came in next, at an average of over twenty seconds longer than WavPack.

Monkey's Audio came in fifth, taking almost fifty percent longer to rip and encode the albums than it took to rip and encode to uncompressed Wave.

OptimFROG took an awful long time to encode each album. It was the slowest on every album tested, and slowest by quite some time. On average, it took OptimFROG two-and-a-half minutes longer to rip and encode a CD than Monkey's Audio, and over two-and-a-half times the amount of time taken to rip and encode to Wave. If OptimFROG had offered a faster compression mode, I'd have taken it. But Fast mode is the fastest on offer. Five-and-a-half minutes feels like a long time to rip and encode a single album.

Average album sizes

Monkey's Audio produced the smallest lossless file size for every one of the ten albums tested. And its performance seems to have improved a little since my previous format comparison. On average, an album encoded into Monkey's Audio takes up only fifty-nine percent of the space that the same album encoded into uncompressed Wave would.

OptimFROG comes in second, producing average album sizes only a little larger than Monkey's Audio, at sixty percent of the size used by Wave.

FLAC comes in third, and also seems to have improved its ability to compress since my last format comparison, this time producing an average album file size only sixty-two percent that of uncompressed Wave.

WavPack is very close behind FLAC, producing an average file size of sixty-three percent that of Wave. One interesting observation is that WavPack files took up slightly more space than FLAC for every album except The Essential Classics Collection. The smooth nature of classical music perhaps agrees more with WavPack than it does FLAC. The difference was still slight, however.

Shorten offered the least effective compression in this comparison. On average, the albums compressed with Shorten took up sixty-seven percent of the space that Wave used. This is roughly the performance that FLAC turned in during my last comparison.

Wave, being uncompressed, is not surprisingly the least space-efficient format. The albums tested in this comparison ended up using an average 678MB per album when stored in uncompressed Wave format, which would take up almost seventy percent of a 1GB flash disk in a portable music player. (If anyone were mad enough to use uncompressed Wave on a portable player.)

Each format held the same order of effectiveness for every album except The Essential Classics Collection, which caused WavPack to be slightly more effective than FLAC for just that one album.

Average compression rates

The Essential Classics Collection (Volume 1) is by far the most successfully compressed of the ten albums. Its average lossless compression rate is 76KB per second of the album. I assume this is because classical music is much slower and smoother than modern studio music, and because classical music features much more silence and near-silence. Silence ought to be extremely easy to compress for any codec.

Endtroducing is not as compressible as the classics, but it still needs fewer bytes per second than the other eight albums, and by some margin. This album features fairly orderly and laid-back tracks, as well as sparse verbal tracks that ought to be easy to compress.

Most of the remaining albums loiter in the 110–120 kilobytes per second range. But three albums demand more. Barrio Fino, Resident Evil: Apocalypse, and The W all need over 123KB per second. These albums all feature tracks that are full of noise that doesn't let up. Barrio Fino is thick with dance sounds, Resident Evil: Apocalypse vibrates with wailing guitars, and The W is an album throbbing with varying basslines and samples. Such densely-packed audio is apparently difficult to compress losslessly.

Conclusion

In my opinion, it's a choice between FLAC and Monkey's Audio. And the choice depends on whether you value time or disk space more.

FLAC is now so fast that the time saving you'd make by ripping and encoding to uncompressed Wave is barely worth thinking about. FLAC took the same amount of time or less than Wave on four of the ten albums. Combine that with the fact that encoding to FLAC will save you almost forty percent of the space that Wave uses, and you'd be mad to keep using Wave to rip and encode CDs on the basis of saving time.

If disk space is more important to you, then Monkey's Audio may be the codec for you. Ripping and encoding to Monkey's Audio will take you a third longer than doing so with FLAC, but Monkey's Audio produced the smallest file sizes for each of the ten albums tested here. The space saving is only five percent of the space taken by FLAC files, an average of 21MB per album, but if you're on a tight disk budget, or if you want to serve the files from a web server, twenty megabytes per album might mean a lot to you.

OptimFROG and WavPack show promise. OptimFROG is clearly capable of compression of the same strength as Monkey's Audio, but it needs to become a lot faster before it will look attractive. WavPack offers the same sort of compression strength as FLAC, but isn't as fast. The speed of a software process usually improves as the software gets developed, as optimisations allow the same work to be done more efficiently. So these lossless codecs may well offer faster ripping and encoding times in the future.

Shorten performed, in my opinion, the least impressively of all the compressed lossless codecs, being the slowest and least effective at reducing file size. To all the people who trade in live music recordings stored in the Shorten format, I recommend they take a look at FLAC or Monkey's Audio, both of which will save space and bandwidth compared to SHN files. [Shorten's creators, SoftSound, now seem to have gone offline.]

Of course, if you need even bigger space savings, and couldn't care less how long it takes to encode your audio, then you should give the slower compression modes of some of the codecs a try. In this comparison I've gone with the compression mode I think will give the best mix of speed and size for each codec, but some people will care only about size.

Whichever codec you choose to encode your music with, make sure you test a few codecs for yourself first. That way you can decide which one produces the best compromise between speed and size.

For me, though, it's going to continue to be FLAC that I use to rip and encode albums to.

Evidence that conversions really are lossless

To make sure that the audio stored in each lossless format really is the same as the original audio source, I used an uncompressed Wave file of Canon in D major from The Essential Classics Collection. This Wave file was examined using the handy HashTab Shell Extension, and the MD5, SHA1 and CRC-32 hash values for the file were noted down, as was the size of the Wave file in bytes. This Wave file was converted using dBpowerAMP into a FLAC file (using the same compression settings as used in the big comparison above), and the original Wave file was deleted. The FLAC file was converted into a Monkey's Audio file, which was converted into an OptimFROG file, which was converted into a Shorten file, which was converted into a WavPack file. Finally, the WavPack file was converted into an uncompressed Wave file.

The resulting Wave file was exactly the same size as the original Wave file, and the MD5, SHA1 and CRC-32 hash values matched exactly with the original. This is very strong evidence that each of the lossless formats tested preserves the original audio perfectly.

Note that this process only succeeds because Wave does not support meta tags, it only stores the audio in the track. The conversion process above will change meta tags (that describe the track name, year the track was released, etc), so these are not necessarily preserved exactly between lossless formats. This is because not all audio file formats support the same meta tags. However, as far as audio is concerned, these formats really are lossless.