Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

An anonymous reader writes "Matthew Gregan is working on bringing David Schleef's DSP accelerated port of Theora to Firefox Mobile. He writes on his blog: 'The C64x+ DSP is often found in systems built upon TI's OMAP3 SoC, such as the Palm Pre, Motorola Droid, and Nokia N900. Last year, Mozilla funded a port, named Leonora, of Xiph's Theora video codec to the TI C64x+ DSP. David Schleef conducted the port impressively quickly and published his results. The intention of this project was to provide a high-quality set of royalty-free media codecs for a common mobile computing platform. The initial focus is Firefox Mobile on the N900, so I am working on integrating David's work into Firefox. To experiment with other facilities Firefox could use to accelerate video playback, and test integration, I've been hacking on a branch of a stand-alone Ogg Theora and Vorbis player originally written by Chris Double called plogg.'"

In my mind, hardware accelerated means using fixed-function, special purpose functional blocks in silicon. Saying that using a DSP is hardware acceleration for video codecs is like saying that software is hardware accelerated by the execution units of a CPU.

Lets look at graphics cards. A DSP is the same "thing" as a GPU is, in that it's a special purpose processing unit (ie, Graphics Processing Unit, Digital Signal Processor...). Rendering with a GPU is a hell of a lot faster than with a general CPU. Likewise, processing digital signals is a hell of a lot faster in a DSP than with a general CPU.

One of the blobs on the OMAP chip's block diagram is the IVA (Image, Video, Audio) coprocessor. This includes a DSP (the C64x), a DMA engine, and a couple of other features. All of the video CODECs that are hardware accelerated on the OMAP (H.264, RealVideo, and WMV) use the IVA part, and now Theora does too.

The C64x is found in most of TI's ARM SoCs, being omitted from only the very cheapest. This means that the implementation of Theora there could relatively easily be deployed on any handheld comput

It's great seeing the benchmarks showing the CPU usage dropping from 99% to 1%, but at the same time the DSP and GPU usage is going up by some unstated amount. It would be really great to see some comparison of how this effects the battery life. Playing MP3s on the C64x can be done in a bit under 15mW, but Theora is a lot more complex and doing the colourspace conversions and compositing on the GPU is going to add a bit too. I'd expect the power usage to be lower than doing it on the CPU, but maybe not by a huge amount.

Either way, it's great that they can free up the CPU to do other stuff. 0.4% of the ARM core isn't really enough to run the scripts that typically accompany a web page that uses the video tag, 99% almost certainly is.

This is a good example how open systems drive innovation. Allowing people to tinker with the device (root access, access to the DSP) attracts hackers, which in this case lead to DSP accelerated Theora video decoding.
It's quite fitting that Apple is resisting Theora in HTML 5, mainly because their devices don't have accelerated Theora decoding (this is what I assume). While at the same time the restrictions imposed by Apple make it impossible to develop something like this for their iCrap devices (apart from not having root access, no API for accessing the DSP, it would also violate the developer agreement.. since you need some DSP assembly, which is not an approved language).

Apple has kind of a tough time there, the way that they keep things stable is by not having people screwing around with the internals and drivers, but in order to do something like this one has to screw around with the internals. Meaning that they've bet on stability and reliability at a cost of having this sort of community involvement. Hard to say whether they'll come out ahead or behind in the long run.

Because I have yet to see a Theora video site. I say that the price of the additional data traffic costs to get the same quality as H.264 is still more than the license.I”m all for an open codec. But everybody knows that unless it”s also a *better* codec, we won”t see it being used. Companies have no interest in paying more and don—t care as much about open source as they should for their own good.File sharers don’t care about licenses anyway.So we”re left with the tiny s

Why would they need to drop Theora to support H.264? The situation isn't either or. They don't support H.264 because they are concerned about potential legalities or licensing costs (not just now, but in the future).

And then theres the point of one actually caring about licensing. Honestly I dont thing the MPEG will ever sue.

Pity that argument doesn't work with your average corporate lawyer, isn't it?

So I will use H.264 even without a license, just as I did with GIF, JPEG, etc, etc, etc.Because I think we are stronger, and the cant ever hurt us. Instead of backing down like a submissive beta-human loser.

Ohh, you're so *manly* when you infringe on a big company's patents! Take me, Hurricane, take me hard!

Don't be so childish and ridiculous. You're content with committing patent infringement and forcing everybody else to either do the same or pay MPEG-LA's extortion fees, and all that why? because you're a lazy fuck. You're not an alpha male, you're not part of La Resistance, you're just a nerd hiding behind anonymity to

I have no idea why Firefox still fights against the giant and loses money and time on it. Great, they got hw-accelerated Theora to one single mobile phone. What about all the televisions, other mobile phones, computers, airplanes, PS3, 360, and everything else under the sun that has H.264 hw supported? It's a lost battle.

If I were them, I would seriously start concentrating on the next generation of video codecs. It might be closed H.264 for now, but if you want to get an open source product out there, you

We need a backup in the case H.264 patent owners start collecting some money when H.264 is spread everywhere. If I understood right, the current permission to use H.264 freely in free web content is just temporary and ending December 31 2010! Developing and optimizing these codecs take time, so better start working now. In fact we should hurry!

Also the situation is very similar to using OpenOffice-threat to blackmail discounts for Microsoft Office. So even if you happen to like H.264 more than Theora, this

(DENVER, CO, US – 2 February 2010) – MPEG LA announced today that its AVC Patent Portfolio License will continue not to charge royalties for Internet Video that is free to end users (known as Internet Broadcast AVC Video) during the next License term from January 1, 2011 to December 31, 2015. Products and services other than Internet Broadcast AVC Video continue to be royalty-bearing, and royalties to apply during the next term will be announced before the end of 2010.

So how about instead of touting a codec thats already outdated use h.264 for now and use the next 5 years to develop a codec that is actually better in some way.

If all internet use h.264 now, will still be using, and in far more ways, in 5. You now can move away with not so big effort, in 5 years will be impossible, and most of the internet content will be tied to the will of a single company wanting that you and everyone else in the planet pays them for every device and app that potentially connects to internet. Support it "by now",and will never be pressure to change till will be too late.

And open source somewhat follows the "shoulders of giants" idea, all have the source, so the experience could be used with another future open source codec if necessary.

Internet is a bit bigger than it was by the time of gif vs png problem, and there werent so much commercial interest around as now big media companies could have over anything related with video in internet.

I don't see how that would matter. MP3s are still ubiquitous, and the format was finalized in 1991. Of course, they weren't used in earnest until the mid-90s, but that's still almost 20 years of them being around.

Umm, Flash is exactly why this needs to not happen. Flash isn't well supported on _any_ platform. Even on Windows which is the primary focus it sucks. The problem is that people don't all want to use Windows or Mac, even adding in Linux users you still don't hit everybody.

If Firefox requires Flash to play h.264, then it's not supported. I can't install flash properly on my FreeBSD install without a lot of shenanigans, even then it doesn't really work very well.

No, the name and icon are not closed-source, they are trademarked. There is a difference between the two. Linux is trademarked by Linus Torvalds, but to claim that makes it closed-source is patently ridiculous. Trademark only means that other parties cannot use your brand to advance their own products without permission, aka diluting your brand. The fact that you can point to Libre derivative forks of Firefox disproves your claim that Firefox is closed-source.

The fact that you can point to Libre derivative forks of Firefox disproves your claim that Firefox is closed-source.

The fact that the GP only said that the name and the icon are closed invalidates your entire comment. Open Source means that you can get the source code, that is it. Free Software means that the software provides the user with certain freedoms, like the freedom to modify and redistribute. You don't have the right to redistribute your changed version as Firefox, therefore it can be argued that it is not Free Software. Nowhere did the GP claim that Firefox's code was not Open Source.

Excuse me... Video in HTML5 is supposed to get rid of that plugin thinking in the first place.

It's not that you can do that- HELL, you can do that NOW.

The real problem is that while it's implemented and "legitimized", each and every use requires a payment and a sign off against a license you may/may not agree with. Even to provide the content on the Internet will eventually require royalty payments with it. You won't be able to legally implement ANYTHING with video without their permission- and they ca

Waste your points mods, waste them all away...it doesn't change the facts: FACT- HTML V5 IS NOT just a generic container, so sayeth Wikipedia [wikipedia.org] and I quote "User agents should support Ogg Theora video and Ogg Vorbis audio, as well as the Ogg container format."

FACT- Theora has NO hardware acceleration on most platforms. FACT- Theora runs like hammered shit on anything less than a P4 3.2GHz, which means you just cut out most netbooks/nettops, older PCs, and pretty much every mobile device except the ONE CHIP th

While streaming H.264 is currently royalty free, distributing a codec for it is NOT. So how about it moneybags, would you care to foot the bill for a worldwide unlimited license from MPEG-LA?

NO? That's what everyone else says, and there's your answer. Mozilla made the choice they did because that's the one they are legally able to make. If you find it that objectionable, perhaps you can help Mozilla get a deal from MPEG-LA?

``Open source has some advantages, but if it's technically lesser and doesn't work with companies, it's not going to win.''

I've emphasized the part that I think is the main point here. It's not about quality. It's not about cost. It's about how businesses work.

I don't know this for sure, but I would be surprised if H.264 weren't pushed through the normal corporate channels. If you run a company that makes H.264 decoders, you are going to have some of your people meet with people from other companies to try

One big thing getting in the way of Theora is the lack of easy encoders.

Have you seen what they call easy? Two lines of cryptic text commands - commands that change from version to version, and aren't listed in the documentation.

With H.264, you can pick up a GUI for free, drag a slider to "Maximum", and wait 2-3 hours. Then you have a wondrous quality video, ready to be streamed, and hardware accelerated on a lot of platforms. There's even websites that will generate all the code for you, so plunking it in

I have no idea why Firefox still fights against the giant, emm, I mean Microsoft and loses money and time on it. I mean, great, geeks have nice browser on Linux, but what about Windows 98, Windows 2000, Windows XP, Windows Vista, etc. and everything else...emmm there is no anything else that has Internet Explorer support? It's a lost battle.

If I were them...ok, I can't continue this with straight face.

I have question - why do you ask? Mozilla can't distribute H.264, period. Not in this, nor in parallel univ

I have no idea why Firefox still fights against the giant and loses money and time on it. Great, they got hw-accelerated Theora to one single mobile phone. What about all the televisions, other mobile phones, computers, airplanes, PS3, 360, and everything else under the sun that has H.264 hw supported? It's a lost battle.

This is/. so how dare you try to use an argument that may or may not use facts and or logic in it. Plus I ain't clicking on any fancy links just because you want to blow steam off on the internet. Remember winning an argument on the internet is like winning the special olympics, your still retarded after you get that medal.

The codec was implemented against the C64x+ on a Linux based target on OMAP3.

Each phone isn't magically different in that respect. In fact, TI provides a platform SDK to use the DSP and everybody using Linux as their base OS is using the same SDK and pathways to get the data in/out of the engine and merely specifying their own get/on get/off points, which are actually submitted to the DSP via the API TI provides to each and every customer of their hardware.

What does this mean?

It means that the DSP work done now is available, largely ready for use on:

Seriously. Done once, able to be used elsewhere as long as you provide consistent interface rules. You might need to adjust the place you get and put the data for the operation, but the algorithm and the vast bulk of the DSP code doesn't change from device to device.

More to the point, you're going to find that with a bit of tweakage, the work can probably be implemented on OMAP2 platforms such as the Nokia N800/N810 as there's more than enough headroom on the C64x+ to lead me to believe that there's a possibility of doing it on a C55x class DSP- not that someone would go to that trouble unless there was a business case for it.

Now, this doesn't mean everybody gets "hardware" Theora yet- you still need to implement for at least Blackfin and Snapdragon as well for the bulk of the mobile device space. The main big deal here is that it has now been shown that it CAN be done and done fairly easily.

and before you remark... iPhone uses a similar DSP edge so my offhand remark about iPhone alongside with "anything else Linux" shouldn't nullify the argument... I added iPhone at the last moment and didn't qualify it this way.

the rest of the world plays grande ball with the H.264 format and the superior encoders available for it

Up until Dec 31, 2010 when the patent holders have stated that they're ending the royalty-free period and it becomes the GIF of the video world.

You can bet that with all the money youtube loses now, Google ain't going to pay for millions of H.264 videos and H.264 will literally disappear overnight. I'm sure Google is working overtime on getting that shiny new codec they just bought into chrome, if not fi

That is not correct.http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/226/n-10-02-02.pdf

(DENVER, CO, US – 2 February 2010) – MPEG LA announced today that its AVC Patent Portfolio License will continue not to charge royalties for Internet Video that is free to end users (known as Internet Broadcast AVC Video) during the next License term from January 1, 2011 to December 31, 2015. Products and services other than Internet Broadcast AVC Video continue to be royalty-bearing, and royalties to apply during the next term will be announced before the end of 2010.

I would argue that Youtube qualifies. But let's see. Also this claim is not binding. In fact, MP3 patent holders had a public pledge of not enforcing patent fees from free software/open source mp3 players, but later changed their mind. So I wouldn't sleep on it.

Just... let it die. I know free/open is awesome, but Theora is just bad bad bad on top of awesome. If you refuse to look towards H.264, then at least now you can look towards Google and VP8 now. Bury Theora.

That's not Google's view, though. That's just a single guy, who's working independently from Google, but they finance his work. Their official stance with respect to backing H.264 for HTML5 didn't change [slashdot.org] - they're still opposed to standardizing on Theora.

And VP8 rumors are just that, rumors - for now. We shall see later. In any case, if it goes ahead as rumored, it would be very different from Theora (which may in fact be why Google opposes standardizing HTML5 on the latter!).

That's not Google's view, though. That's just a single guy, who's working independently from Google, but they finance his work. Their official stance with respect to backing H.264 for HTML5 didn't change - they're still opposed to standardizing on Theora.

Theora isn't bad. It's better than MPEG-1 and a MPEG-2, and similar quality to MPEG-4 part 2. It's not as good as H.264, VC-1, or Dirac, but that's not the same as being bad. It's perfectly acceptable for a lot of uses.

You make an interesting point. Theora is particularly well suited for high bitrates, so DVDs would actually be a fine use. It's trivial for any modern device to decode theora in hardware - the software just has to exist, and be bundled in the firmware of the device.

I feel H.264 or a codec like VP8 should win for streaming, simply because of the reduced bitrate. H.264 does exceptionally well at super low bitrates(256-512kbit), where other codecs would turn into a blurry mess. Because of the low bitrate, it's

Mozilla and wider Open Source world has three options: (not mutually exclusive)

You missed a 4th option:

Drop Firefox and switch to Chrome

With support for H.264, Theora, (and hopefully soon VP8) all the bases are covered. This is the option that gets my vote, and now I can relax and enjoy the web as it's meant to be enjoyed, without having to worry or whinge about anything.

Can you legally do all the things that open source allows you to do _and_ support H.264? And before anyone says "yes, by using the H.264 support from some other component", I'm asking about whichever component contains the support for H.264, not the browser per se.

I'll admit that Theora isn't the greatest video codec. But it's been explained many times that Mozilla simply can't use MPEG-4 AVC/h.264 because of the patents involved.

And has been explained many times before, that's complete, utter bullshit.

All Firefox needs to do is use a generic video decoding pipeline for it's backend. GStreamer and DirectShow come to mind (interesting sidebar, they're *already doing this with Fennec*, they just refuse to backport the changes to Firefox for idiotic political reasons).

If Firefox uses something like DirectShow on Windows: 1/ they will need to have codec support for theora (not necessarily installed);
2/ the H.264 codec is only available on Windows 7 and later, and they cannot legally ship a H.264 codec for XP/Vista;
3/ they need to ensure that the DirectShow output works with the layers/svg/smil infrastructure as well as DirectWrite/Direct2D on Windows 7 -- e.g. does having a rotating, scaling video in a smil/svg file work;

1/ they will need to have codec support for theora (not necessarily installed);

User problem.

2/ the H.264 codec is only available on Windows 7 and later, and they cannot legally ship a H.264 codec for XP/Vista

User problem.

3/ they need to ensure that the DirectShow output works with the layers/svg/smil infrastructure as well as DirectWrite/Direct2D on Windows 7 -- e.g. does having a rotating, scaling video in a smil/svg file work;

Yup, it's a technical challenge. But don't tell me it's a blocker, as other bro

they need to ensure that the DirectShow output works with the layers/svg/smil infrastructure as well as DirectWrite/Direct2D on Windows 7 -- e.g. does having a rotating, scaling video in a smil/svg file work;

Yup, it's a technical challenge. But don't tell me it's a blocker, as other browsers are already using this approach.

It's not even a challenge. In both GStreamer and DirectShow, the codec just outputs an image (usually YUV). The browser can scale/rotate/resize it as it sees fit. Yes, those frameworks also offer facilities for hardware-accelerated rendering of those images, but it doesn't have to be used (or it can only be used when image isn't rotated or partially obscured, etc).

Heck, Opera already does it with GStreamer - cross-platform, and they support arbitrary rotation/translation/scaling of page elements or the enti

That sounds like the worst possible solution, because then you'd have Firefox working differently on different platforms. I sure don't want to go back to the Bad Old Days where FF on Linux couldn't view the same media as FF on Windows.

I sure don't want to go back to the Bad Old Days where FF on Linux couldn't view the same media as FF on Windows.

That's currently the situation with plugins. They're pieces of object code, so you're dependent on the plugin developer to provide you with a version for your OS/architecture. For example, Flash (something that HTML 5 and the video tag could replace) is (AFAIK) only available on three mainstream platforms - Win/x86, OSX/x86, and Linux/x86. There's a beta version for x86-64 and special versions (Lite?) for embedded devices, but that's pretty much it. As for the Quicktime plugin, there's no Linux or *BSD support at all (mainly because there's no QT on those platforms), although there's plugins that use VLC or totem, etc instead.

Now, lets look at Firefox. The Debian iceweasel package [debian.org] (Firefox without the branding) is listed as being available for 14 architectures (not counting the unofficial 68k port that languishes at 2.0). That includes x86-64 for your latest AMD64/Intel64 machines, armel for your portable ARM devices, powerpc for your old G3/G4/G5 PowerMacs, and a bunch of other architectures too. None of those are supported by Adobe or the many other companies that have, over the years, tried to lock us into their proprietary software with binary-only plugins. So I'm siding with Mozilla for some very practical reasons. Theora's not the best out there, but I'd prefer the web to be open to all.

H.264 is open and yet proprietary. The patent holders own H.264. It's not Free, nor is it free, although various entities will pay on your behalf if you are a member of one of several common classes of computer user.

No. It is not proprietary in any sense of the word. You really need to look up the definition before you speak.

Theora has more claim to being proprietary, as one single organization defined it and continues to change it however they see fit. That has been expressed as one of the reasons companies refuse to accept Theora... Not that the quality and installed-base issue isn't overwhelmingly paramount, though.

I'm not implying anything. I'm *saying* that defenders of Mozilla's ridiculous stance have tried to claim:

1) Mozilla simply *can't* support H.264 for legal reasons. This is false, as there is no need for Mozilla to ship any codecs.2) Mozilla simply *can't* support H.264 using OS-provided codecs because it's, like, hard and stuff. This is just BS.3) Mozilla simply *can't* support H.264 using OS-provided codecs because of, like, security and stuff. As if Mozi

In short, I'm saying, not implying, flat out stating, the people defending Mozilla's ridiculous position (either Mozilla staff themselves, or Mozilla fanbois) are now scraping around at the bottom of the barrel to come up with excuses to support their stance, instead of just admitting that the original choice was purely ideological, and their current stance is essentially an unwillingness to simply admit they were wrong.

Not all systems have an h.264 codec installed. Mac OS X does (via QuickTime). Windows 7 has an included h.264 codec, but neither Windows Vista nor Windows XP do. The only available h.264 codecs for Windows Vista and Windows XP are either commercial (which nearly nobody will have installed) or unlicensed. On Linux, there is exactly one licensed h.264 codec (Fluendo's GStreamer plugin), which virtually nobody has installed.

Mozilla can use AVC/H.264. They just need to do it via the OS, like I think IE and Safari will do, and Opera already does. H.264 is already directly supported on Windows 7 and Mac OSX. Linux users need to do their tricks, like with MP3.

They only way they can do as you suggest is by severely compromising Firefox. This has been answered exhaustively many times. See Mike Shaver's blog post about it [off.net]. To quote:

People have raised questions about using existing support for H.264 (or other formats) that may already be installed on the user’s computer. There are issues there related to principle (fragmentation of format under the guise of standardized HTML), to effectiveness (about 60% of our users are on Windows XP, which provides no H.264 codec), to security (exposure of arbitrary codecs to hostile content), and to user experience (mapping the full and growing capabilities of to the system APIs provided); I’ll post next week about those in more detail, if others don’t beat me to it.

Security is not to be underestimated. It's worth noting that IE9 will not support all media formats supported on the host operating system, purely to limit the attack vector surface area available. See this comment by Microsoft's Frank Olivier on the IE blog [msdn.com]. He's talking about image formats but the same applies to video formats:

We've not heard many requests for additional image types - to limit the attack surface in a web browser, it is a good idea to not expose more decoders.

They only way they can do as you suggest is by severely compromising Firefox. This has been answered exhaustively many times. See Mike Shaver's blog post about it [off.net]. To quote:

People have raised questions about using existing support for H.264 (or other formats) that may already be installed on the user’s computer. There are issues there related to principle (fragmentation of format under the guise of standardized HTML), to effectiveness (about 60% of our users are on Windows XP, which provides no H.264 codec), to security (exposure of arbitrary codecs to hostile content), and to user experience (mapping the full and growing capabilities of to the system APIs provided); I’ll post next week about those in more detail, if others don’t beat me to it.

We've all read the arguments (I hope), we just don't buy them. I'm sorry but the OS should be handling video codecs for applications. Too many different applications such as video players, teleconferencing, browsers, and editing software all need to use codecs and the OS handling them on behalf of applications is simply the right way to engineer things. I can use Theora in my browser right now using HTML5 tags because I gave the OS a codec and my browser can use any codec I have installed. In practice I don

They only way they can do as you suggest is by severely compromising Firefox.

Utter garbage. Firefox is *already* subject to any bugs in the underlying OS on an incredibly wide range of issues. Hell, just a few years back, Firefox was compromised thanks to an image rendering bug in GDI on Windows. So, what, *now* they've suddenly decided they're going to reimplement OS-supplied services for fear of security issues? Please.

On the bright side, at least the Mozilla devs *tried* to invent a technical excuse

They don't have to use DirectShow in particular, just any extensible framework.

For example, Opera (10.5+) uses GStreamer on all platforms. On a Unix system, it'll just use the one that is already installed, and all the codecs therein, so adding H.264 support is trivial - you just install GStreamer wrapper for ffmpeg.

On Windows and OS X, they simply package a private version of GStreamer with Opera itself, and add the baseline plugins (in their case, just Theora and Vorbis) there. It does not conflict with s