Royalty-free codec boasts best-in-class quality for both voice and music.

The IETF has standardized the Opus lossy audio codec as RFC 6716. While most audio codecs aim to solve specific problems—relatively high bit-rate music reproduction such as AAC and MP3, low latency voice reproduction such as Speex and the AMR family—Opus provides a "one size fits all" option which, according to its developers, provides best-in-class quality in almost all applications.

The codec was primarily developed by Mozilla and Xiph.org, with contributions from Skype/Microsoft and Broadcom. Opus weds Xiph.org's low-latency high bitrate music-oriented CELT algorithm to Skype's low bitrate speech-oriented SILK. The codec switches between algorithms depending on the bandwidth available, while boasting real-time latencies across the full bitrate/quality spectrum.

The Opus working group's own quality comparison suggests that, between about 12kbit/s and 128kbit/s at least, Opus boasts better quality than any other codec on the market. It loses out slightly at very low bitrates to the AMR-WB and AMR-NB codecs used by GSM telephones, but these codecs require the payment of royalties.

Enlarge/ Opus spans the entire range from low bitrate narrow band (speech) to high bitrate wideband (music) with low latency across the board.

The CELT portions of the codec are covered by patents from Broadcom and Xiph.org; the SILK portions by patents from Skype/Microsoft and Huawei. However, all the patent holders have committed to making those patents royalty-free now that Opus is an IETF standard. The codec also includes an open source, BSD-licensed implementation.

This freedom from royalties is important for one of Opus' motivating usage scenarios. Opus is likely to be made a mandatory part of the WebRTC ("Web Real Time Communication") specification. WebRTC defines ways for browsers to access webcams and microphones to create audio and video streams suitable for building real-time voice and video calling apps that work without any plugins or extra software.

The group developing WebRTC is keen to avoid the fighting about codecs that surrounded HTML5's <video> element. HTML5 doesn't require any specific codec, and while H.264 is supported by most browsers and arguably has most support from media producers, it's not royalty-free, which in particular is a problem for Mozilla. The organization has considered supporting H.264, at least on mobile devices, as a pragmatic necessity.

The flexibility and quality of the codec means that it may see adoption even beyond this role; the low latencies make it attractive for streaming, but the high quality means that even latency insensitive applications, such as music storage, are a good fit for Opus.

CELT is/was a fantastic codec; I've been a huge proponent ever since its experimental inclusion in early versions of the Mumble voice chat client. It's bandwidth-efficient, sounds great with both speech and general-purpose audio, and has class-leading low-latency. I'm pretty happy to see it standardized and expanded upon as Opus; I just hope people adopt it and it doesn't get mired up in some kind of legal quagmire because somebody wants to try and squeeze some bucks out of their frivolous patent.

Opus is a lossy file format that's new wrinkle is that it's scalable ---- so no, actually, that doesn't fill every role nor offer best-in-class quality.

Yeah I'm disappointed in the reporting quality here too. It's arguably suitable to fill most if not all low to medium bit-rate roles; and I'd've expected Ars to have gotten the distinction between that and "every role" right.

There are probably a billion devices that, right now, run AAC and MP3, as lossy codecs. There have been tens of billions of songs downloaded in those formats.

Unless companies that sell music in those formats, and stream it, decide to rewrite their software to accept it, and offer to re-encode all their music in it, this will be difficult for anyone to use. Considering that it doesn't seem to offer any advantages in higher bitrate encoding, I don't see why they would want to do that, considering how much it will cost.

So, maybe for audio in a browser, it might work, but otherwise, it will have a very steep mountain to climb.

There are probably a billion devices that, right now, run AAC and MP3, as lossy codecs. There have been tens of billions of songs downloaded in those formats.

Unless companies that sell music in those formats, and stream it, decide to rewrite their software to accept it, and offer to re-encode all their music in it, this will be difficult for anyone to use. Considering that it doesn't seem to offer any advantages in higher bitrate encoding, I don't see why they would want to do that, considering how much it will cost.

So, maybe for audio in a browser, it might work, but otherwise, it will have a very steep mountain to climb.

It's actually designed for streaming and real-time voice communication, it just happens that it does other stuff well, too.

And the reason it doesn't do lossless is because FLAC already does it so well and doesn't need to be replaced.

There are probably a billion devices that, right now, run AAC and MP3, as lossy codecs. There have been tens of billions of songs downloaded in those formats.

Unless companies that sell music in those formats, and stream it, decide to rewrite their software to accept it, and offer to re-encode all their music in it, this will be difficult for anyone to use. Considering that it doesn't seem to offer any advantages in higher bitrate encoding, I don't see why they would want to do that, considering how much it will cost.

So, maybe for audio in a browser, it might work, but otherwise, it will have a very steep mountain to climb.

Will it replace AAC or MP3 as an archival format? Probably not. Will it become the de facto standard for interactive streaming audio in bandwidth constrained applications? We should hope so.

I just hope that sometime I can use my phone's HSPA+ capabilities to use a higher fidelty speech codec like Opus here for example. Its appalling that I can watch movies, browse the internet etc etc on my phone but I still have calls that sound the same quality as they have for the last decade.

(Though to give credit where credit is due, the new noise rejection capabilities are really awesome)

There are probably a billion devices that, right now, run AAC and MP3, as lossy codecs. There have been tens of billions of songs downloaded in those formats.

Unless companies that sell music in those formats, and stream it, decide to rewrite their software to accept it, and offer to re-encode all their music in it, this will be difficult for anyone to use. Considering that it doesn't seem to offer any advantages in higher bitrate encoding, I don't see why they would want to do that, considering how much it will cost.

So, maybe for audio in a browser, it might work, but otherwise, it will have a very steep mountain to climb.

There necessarily will be a transitional phase before any new codecs are adopted. Look at H.264, which has caught on fairly well, but by no means universal. Opus at least removes many of the barriers to adoption. It would be dumb to adopt it exclusively at this point but if people start adding it to applications and devices now, at some point in the future we can have nice things.

I still have devices that don't support AAC, so I'm less enthusiastic about the present situation than you are....

I just hope that sometime I can use my phone's HSPA+ capabilities to use a higher fidelty speech codec like Opus here for example. Its appalling that I can watch movies, browse the internet etc etc on my phone but I still have calls that sound the same quality as they have for the last decade.

(Though to give credit where credit is due, the new noise rejection capabilities are really awesome)

That was pretty much the motivating thought that Monty, of Xiph, gave for development of CELT, which became opus.

160 kBPS does not qualify as "high-bitrate." How does Opus stack up against MP3/AAC at 256 or 320 kBPS?

Any encode of MP3/AAC/Vorbis from a modern encoder is indistinguishable from reference at those bitrates. It is the same for Opus. Did you see the graph where the codecs come together around 128 kbps? Nobody even does listening tests above that bitrate anymore because the results are uninteresting.

160 kBPS does not qualify as "high-bitrate." How does Opus stack up against MP3/AAC at 256 or 320 kBPS?

I don't think that's really the right question to ask: what really matters is what is the noise like (PSNR or some other metric) at a given bitrate across the different codecs. It's entirely possible that a codec could be written that gives the same PSNR at 160kbps that MP3 gives at 256 kbps. We have already seen this in the image world where JPEG2000/JPEGXR offer *considerably* higher PSNR when compressed compared to a JPEG image compressed at the same bitrate (or conversely, images compressed at the same PSNR have very different bitrates).

Hah, I was about to ask Ars for a write-up on Opus ever since I noticed it in the latest Firefox release notes. Wikipedia has some more information, and comparisons with other lossy codecs. It seems to be high-performance across a broad range of bit rates, so you probably COULD use it for music storage if you wanted. It's more useful for streams, though, because it can handle both voice and music well at those bitrates. It also has a low latency emphasis.

So is this truly a new codec or does it merely swap through existing codecs depending on bitrate?

since it switches on the fly I think it can be considered as a new one, although it uses two tweaked pre-existing codecs

Bad Monkey! wrote:

Will it replace AAC or MP3 as an archival format? Probably not. Will it become the de facto standard for interactive streaming audio in bandwidth constrained applications? We should hope so.

Those two format are not adequate for "archival" purposes, they are lossy. Opus is superior on both of them at 128 kb/s

DeComposer wrote:

160 kBPS does not qualify as "high-bitrate." How does Opus stack up against MP3/AAC at 256 or 320 kBPS?

I'm wondering, have you ever tested encoding of normal music at 160 kb/s and 320 kb/s blindfolded? (i.e. doing ABC tests)I'm asking because most people don't tell the difference, there is usually too much noise in the environment to pick those differences. Since these formats aren't lossless it's a waste of space encoding at such high bitrates if you can encode it at half the rate and not tell the difference, for archival purposes, uses flac or another lossless format.

160 kBPS does not qualify as "high-bitrate." How does Opus stack up against MP3/AAC at 256 or 320 kBPS?

Can you blind ABX separate MP3 and AAC VBR implementations at anywhere near 256 or 320kbps? Everyone really needs to just try this for myself. I'm blown away that on my decent home stereo or with headphones I cannot tell the difference between FLAC and 96-128kbps AAC or OGG on the vast vast majority of my music.

Foobar makes this test very easy (as long as you start with a lossless file, and encode yourself to the test formats). I highly recommend it.

So, h.264 may not be open source, but it is free to use for these kinds of things for the future. There are no royalties to pay. And there never will be.

And just like that standard Google was pushing a while back, do we really know it's free of copyright and royalty challenges? Unless it has been vetted by a boat load of IP lawyers and been challenged in court by some copyright troll, that is yet to be determined. It would hardly be the first time something claimed to be royalty-free which later turned out to be the opposite.

So is this truly a new codec or does it merely swap through existing codecs depending on bitrate?

A little of both actually. Opus is composed of modified and improved versions of two codecs, SILK (a low latency speech codec) and CELT (a low latency transform codec similar to AAC/WMA/Vorbis). However, both are heavily modified to allow scalable latency and a wider range of quality/compression levels.

DeComposer wrote:

160 kBPS does not qualify as "high-bitrate." How does Opus stack up against MP3/AAC at 256 or 320 kBPS?

In this context it does. Opus is efficient enough that there is no quality reason to encode stereo audio at such bitrates (multichannel is another issue).

nebrius wrote:

I don't think that's really the right question to ask: what really matters is what is the noise like (PSNR or some other metric) at a given bitrate across the different codecs.

Noise in the sense that you're thinking is not really defined for audio codecs because of auditory masking:

So you can't really ask that question to any meaningful degree. The best you can do is assemble a body of samples and have people compare them to uncompressed versions and see if they can spot a difference, and if they can, how annoying it is.

Opus is a lossy file format that's new wrinkle is that it's scalable ---- so no, actually, that doesn't fill every role nor offer best-in-class quality.

Yeah I'm disappointed in the reporting quality here too. It's arguably suitable to fill most if not all low to medium bit-rate roles; and I'd've expected Ars to have gotten the distinction between that and "every role" right.

I think you've misapprehended something about the codec; based on the chart and the reading I've done, you can use Opus at high bitrate too, with quality comparable if not better than other codecs at equal bitrates. Why do you think it's only suitable for the low-medium bitrate applications?

Will it replace AAC or MP3 as an archival format? Probably not. Will it become the de facto standard for interactive streaming audio in bandwidth constrained applications? We should hope so.

Those two format are not adequate for "archival" purposes, they are lossy.

I'm glad you put archival in quotes, because obviously that word means different things in different contexts. Here, it implies a format suitable for non-interactive transport and storage. You can have the lossy vs. lossless argument with someone else, if you please.

Quote:

Opus is superior on both of them at 128 kb/s

And Opus is also vastly inferior in terms of actual deployment to consumer applications, which was what melgross was pointing out. Doesn't matter if mp3 or AAC are inferior to lossless codecs for "archival" purposes, or inferior or superior to Opus for any application, they're the defacto standard for consumer music sales and storage, and Opus probably isn't going to change that, just as FLAC hasn't changed it.

Oh yeah, and it might be worth mentioning that Rockbox is already adding support for Opus. So if you decided to go nuts and rip your CDs to Opus then you could play it on your Rockbox'd iPod mini or something.

And Opus is also vastly inferior in terms of actual deployment to consumer applications, which was what melgross was pointing out. Doesn't matter if mp3 or AAC are inferior to lossless codecs for "archival" purposes, or inferior or superior to Opus for any application, they're the defacto standard for consumer music sales and storage, and Opus probably isn't going to change that, just as FLAC hasn't changed it.

Indeed, the point of Opus is not to have a marginal couple percentage point improvement over AAC or MP3 in terms of compression efficiency, it is to allow AAC/MP3 quality audio and compression in low latency situations where AAC/MP3 simply cannot be used at all. If people choose to use it as their primary archive format, it will work very well for that as well, but that is not the main purpose of the format.

So, h.264 may not be open source, but it is free to use for these kinds of things for the future. There are no royalties to pay. And there never will be.

Unless something changed, MPEG-LA still has the ability to start charging royalties for decoding of H.264. And I don't think they ever offered to stop charging royalties for encoding. These are completely different situations.

Quote:

And just like that standard Google was pushing a while back, do we really know it's free of copyright and royalty challenges? Unless it has been vetted by a boat load of IP lawyers and been challenged in court by some copyright troll, that is yet to be determined. It would hardly be the first time something claimed to be royalty-free which later turned out to be the opposite.

How does it compare to mp3/aac/etc in higher rate/non-streaming contexts? eg 192/256/320(/higher?) bitrates?

The answer: small and moderate-sized listening tests find no statistically significant ability to distinguish any of the above codecs from the original at those bitrates, Opus included. Assuming of course, you're using a good encoder for each, eg, not Blade for mp3....

You could possibly eventually reach statistical significance, but the tests would have to be huge, and they're exhausting for participants, because the vast majority of trials are indistiguishable. People burn out quickly listening for differences that are inaudible.

You could use pathological samples designed to break the codecs, but we test against those kinds of samples too, so whenever we find one, we study it and improve the codec until that kind of sample can't break it anymore.

HA has done such listening tests between codecs at high rates. The results were uninteresting (N codecs, N-way tie). There's a reason all the quality lines converge.