If I told you that Google had helped fund an ARM code optimised version of the Theora video codec, most people’s reaction would be immediately to skip forward to the next blog entry. Audio and video codecs are the classic example of things that no one cares about, until they don’t work.

Ask most computer users what their preferred video codec is and they’ll look at you as if you asked what sort of motor they’d prefer in their washing machine. “We just want it to work!” they say. In this regard, it’s exactly the same for content creators and publishers. Every visitor to a website that can’t view a video is one set of eyeballs less for a message to get through to. It doesn’t matter how clever the advertising is, how much time is spent honing the message or how many clever viral tricks are deployed to attract surfers to the site, the moment the page opens up with a big blank box where the content should be, all that has been in vain.

So, publish video so it plays back on everything

Nice idea - but far from simple to achieve. At the moment there is no standard way to deploy video on the web. Some sites use Flash, but this restricts them to a viewing audience that have Flash players, instantly excluding most phones. Some use embedded Java players, but this restricts you to a viewing audience running on powerful enough devices to be able to decode video and audio in a virtual machine, excluding anything slower than a laptop. Still others rely on embedded native players (such as Windows Media Player), excluding every platform other than the intended one. Finally some sites just offer videos as links and farm the job out to whatever video playing software the user has (hopefully!) installed on his machine.

None of these meets the seamless “it just works” goal - and none of them looks like it will ever do so in future. Like it or not, the profusion of different web access devices out there means that this is only going to get harder and harder. Once it was enough to make sure your video was viewable on both PCs and Macs. Now you have Android, ChromeOS, iPhoneOS, Linux, Maemo, Symbian and umpteen others. Not only that, you have to cope with a range of processing powers, from desktops to laptops, to netbooks, PDAs and phones. The problem is exploding, not shrinking.

Fortunately, there is some good news in the form of HTML 5. This new version of HTML (the basic language used to write webpages) introduces a video element.

This will allow people to write their websites specifying the appearance of videos in a standard way. How the individual browsers choose to implement the playback is then up to them - whether they handle movies themselves or farm them out to embedded/external players is a decision made by the viewer, not forced back onto the content creator. The even better news is that support for this is already arriving - Firefox, Opera, Chrome, and Safari have already rolled out HTML 5 support and other browsers won’t be far behind.

So, problem solved?

Well, sadly, no. Having a consistent way of publishing video is a great start, but there is still the issue of what format to use. There is no “one size fits all”. Are we surfing the site on a phone with a small screen? Or with a netbook? A desktop? Or on our new 150 inch QuadHD 3D LCD TV? Screen size, connection size and processing power all affect the decision here. In the same way that we’ve seen home video quality improve from VHS to DVD to BluRay, video on the web is going to get better and better. And that’s fine: existing web server technologies can tailor the video tags used to the browser/devices in use.

What is clear though, is that we need a baseline to work from - one standard format that (if all else fails) everything can fall back to. This doesn’t need to be the most complex format, or the most advertised format, or even the format with the most companies involved in its creation. All it needs to do is to be available, everywhere. The codec in the frame for this is Ogg Theora, a spin off of the VP3 codec released into the wild by On2 a couple of years ago. It scores quite well on both the quality and compression fronts, standing up respectably against it’s more popular rivals such as MPEG4, while actually being much simpler to decode. The overwhelming feature that makes it stand out from its rivals is the fact it’s free. Really free. Not just “free to use in decoders," or “free to use if you agree to this complicated license agreement," but really, honestly, genuinely, 100% free. The specification for the stream and encoder/decoder source is available for public download and can be freely used/modified by anyone. Theora was designed and is maintained with the overriding goal of avoiding patents. No other codec can come even close to claiming to be as patent or royalty free as Theora can, whilst still holding a candle to the alternatives.

So, what’s missing?

Video decode is a pretty CPU intensive task. In order to fulfill the dream of being able to work on every device some painstaking effort is required. The complexity of Theora is considerably less than that of many of its peers; other codecs often require dedicated hardware in devices to help achieve performance targets, but with careful coding Theora can be made to run without this. In fact, on desktop/laptops realtime decode can be managed by an embedded Java player (such as the excellent free Cortado), enabling video playback on browsers still waiting to have video tag support added. For the increasing range of PDAs, phones, netbooks, web tablets and media players out there though, this isn’t an option. Rather than having typical power hungry desktop processors in, these devices tend to be built using the much more frugal ARM processors. While these have increased in power in leaps and bounds in recent years, they still can’t compare with their larger cousins for raw computational grunt. These ARM based devices represent the single biggest class of devices still needing work for decent Theora playback. Any efficiency savings we can make feed back directly into being able to cope with larger screen sizes or giving longer battery life.

This is where Google's grant comes in - by helping fund the development of TheorARM (a free optimised ARM version of Theora), they are helping to hasten the day when video works everywhere on the web, for everyone. That’s got to be something to be pleased about. And now you can flick forward to the next blog post.

46 comments:

Yes, yes, that's all very necessary and impressive, but what about the larger issue of Google's supposed purchase of On2's IP assets, including their more recent and efficient codecs, in the scheme of HTML5?

It is really awesome to see Google pushing at least some support for Ogg Theora! Congratulations on that. It would be even more awesome if Google could use its acquisition of On2 to clear up any concerns that Google itself and Apple have over using ogg theora as the HTML5 baseline codec.

http://lwn.net/Articles/372566/

Are you going to help revert the bad decision that was made regarding this?

AWESOME! This is EXACTLY what the world needs. Now if only you guys at Google would quit working with the various fascist governments like the US government. I applaud you for the China pull out, but you should probably consider a US pullout too. This country has been just as evil since they shot JFK in the face.

Seriously though, supporting Theora is 100% the right move! I'll keep my gmail account another year at least! Thanks Google!

Innovators like Google were able to come into existence because they could build their service on the open technology of the web without going and asking the incumbent technology companies for permission.

We're at a crossroads now where web technology can continue its commitment to openness—or we can adopt Apple's model of closed systems, privately owned formats, and innovation by permission only.

This is really the crux of the web video debate—should the right to publish video that the world can view be controlled by a handful of large corporations? Or should this ability be freely available to anyone who has ideas and initiative?

It's really fantastic news to see Google putting a little more weight behind truly open systems… and that they aren't afraid that the next innovator might not be something that they control. Fantastic news, and a refreshing break from some of their peers in industry.

And, besides, 1280x720 video on a mobile phone processor— without any special hardware support beyond color-space conversion— is just freaking cool.

If Google makes the bold move and start serving their YouTube content in Ogg Theora will start real revolution. I know it's going to cost big money to Google, but social benefit would be enormous. Then Google could change their corporate slogan from "Don't be evil" to "Not only: Don't be evil, but actually be Good."That would be real evolution for the businesses and the world.

Chris DiBona of the Google open source group claimed that "If [youtube] were to switch to theora and maintain even a semblance of the current youtube quality it would take up most available bandwidth across the Internet."

Great news, so will google be supporting the free Theora format on HTML5 Youtube soon? Or sticking to the non-free H264? If there isn't consistency within one company then setting a standard is gonna be a struggle.

Theora can already playback in real time on an iPhone 3GS at native screen resolution, 25 fps, without any optimizations except for the final YUV2RGB stage.

Hardware decode support is a red herring. I've brought this up many times with various organizations, and once you demonstrate that decoding on a portable can in fact already be done, they admit that it's actually a political issue. It's just a convenient excuse to hide the real reasons.

Still, optimizing decode will reduce power consumption somewhat (but not a massive factor vs everything else in the system e.g backlight), will give some headroom, and is a laudable thing to do in any case.

You know what, this is dandy. Google just up its money where it's mouth is. It's also a sign that ARM is in Android's future. Good signs...

But, the elephant in the room is content. And with that in mind YouTube is king, if this is a sweeping policy within Google then YouTube has to tow the line... like it or not. And so far, I don't think that they have been willing to do so.So, that is the problem now... how to get Youtube to change tune on Ogg. Because as it stands, it's Youtube (and in part Google itself as the owners of YouTube) that stand in the way.

I know this will take a while, but like the other here, I would implore you to do it....

Because Flash is still not proliferating onto many ARM devices, it would be a great situation if YouTube had a fallback option of Theora and we could play video back on ARM based devices, given this work that you're funding.

I wholeheartedly welcome any move to kill any platform-centric technology. The sad fact is that the other king of the Net, Apple, is moving to the opposite direction. I hope Google and others do not give up. I might use Apple products but I am ready to ditch any of their service, while I cannot do the same for Google.

The FFmpeg Theora/VP3 decoder is, I'm told, faster than TheorARM on any reasonably new ARM device.This is actually the decoder shipped in Chrome, not any derivative of the official libtheora, so I wonder if anyone will really get much use out of this one, besides those that need a BSD-like license.

"Some use embedded Java players, but this restricts you to a viewing audience running on powerful enough devices to be able to decode video and audio in a virtual machine, excluding anything slower than a laptop"

but it's great with Theora, right?"In fact, on desktop/laptops realtime decode can be managed by an embedded Java player (such as the excellent free Cortado), enabling video playback on browsers still waiting to have video tag support added."

Apple's Maciej Stachowiak has some legal questions about the VP3 patent grant which she posted on the Xiph mailing list. It's regarding some of the subtle wording in the license text.

Now that Google owns On2, it would be a tremendous boost to simply publish a watertight patent license for VP3 (i.e. Theora) - that might well help get Apple to support it, and at the least it would help remove this argument against Theora.

Just a simple worldwide/non-exclusive/irrevocable/etc license, along the lines of Sun with the ODF format.

With all that I have said put to text, this does make me wonder if Google still has plans to open-source VP8, or if that was the plan at all.

@Chris DiBona:

You know, I have much respect for you but what you are saying on this subject seems baseless at the moment.

I know you have seen the internal workings of YouTube and how their infrastructure is handled. But this is the second time that I have seen you post "compared to later codecs, Theora still uses a lot more bandwidth for a similar quality stream."With very little evidence to that.

So I will ask, if it is in your power to do so, please... show us what evidence you have to back up your claim. If you can't and you can explain why you can't, that is understandable. Because as it stands from where I am sitting, you seem to be pluckling that from thin air every time that the subject comes up.

It seems to me, in retrospect, that Google bought On2 as part of a patent armoury to protect Theora when it's adopted for YouTube. Google's a big target and did not, up to that point, have a decent set of video patents.

In fact, there were probably 2 options on the table when On2 was acquired:

1. "Is VP8 unencumbered enough, or does it have a low-enough infringement surface area, that we could open-source it?"2. "Failing (1), do we gain enough IP with the acquisition of On2 to protect Theora?"

I'm guessing that (1) failed. Google's sole motivation here is to increase the usefulness of the web across every device.

I think we'll see some input of improvements from VP8 into Theora and then a YouTube conversion to that format.

is this just me who notice license change from GPL to BSD-style? original post makes it almost poetic, while those who stay longer in this business know that BSD is used often to create proprietary chunks inside, leave alone it makes possible to stop providing source code at certain point just on deliberate decision by yet-another-manager...