Sunday, April 3, 2011

Ruminations on Mozilla rapid release, the end of embedding and Firefox 4.0.1

First, old business. Firefox 4.0.1 (this appears to indeed be the version number) is slowly shaping up. Assuming Mozilla is in any way typical, it will probably emerge in late April or early May. There are so far some minor stability updates, nothing especially major. There may or may not be some TenFourFox-specific fixes in there, depending on when I get them complete, but there haven't been many major issues specific to us (good!). The AltiVec WebM accelerator will not make this release; it will be in a separate beta (probably 4.0.2pre). Mozilla has stated they will continue with their "old" maintenance schedule for Mozilla 2.0, which means there will still be stability and security updates for a period of time, which is also good because of what I'll talk about next.

Before we move on, however, there is some somewhat unhappy news that Camino will drop Gecko after they release 2.1 (equivalent to Gecko 1.9.2, i.e., Firefox 3.6). This is sad to hear -- I myself was a Camino user until I started working on TenFourFox, mostly because I like and trust Gecko more than WebKit, but Fx 2.x and 3.x didn't feel like first-class Mac applications by comparison. I'd probably still use Camino if they were planning to support PowerPC, but they too will undoubtedly drop it after they end support for Camino 2.1, which in fairness to them will probably not be for many, many months.

The reason is because Mozilla is dropping embedding, at least for in-process. There is a promise in the mozilla.dev.embedding thread that out-of-process embedding will be reconsidered, but I doubt it, and for a big reason: Gecko really, really sucks to embed. It was always hard for Camino and many bugs were filed on getting it to work right (it still doesn't always), and WebKit's entire existence is owed to the fact Apple also thought Gecko would be a PITA to embed, and instead took KHTML and messed up around with that instead. Part of this is Gecko's scope, because Gecko requires XPCOM, and XPCOM isn't just some internal engine -- it's an entire object framework. But part of it is simply because Mozilla never made it a priority, and now the people who might have kept the embedding aspect alive will probably give up and work on something else. When the time comes for out-of-process embedding, there won't be any browser project around that's interested in it, and Mozilla themselves will have moved on to something else and it will never happen. This is essentially the end of Mozilla as an embedded rendering engine.

For the record, we are not embedded, neither in Classilla or TenFourFox -- we are the browser, not a shell around it, and we are built ultimately on XULRunner (or in Classilla's case, XPFE Apprunner), so this doesn't affect us. It also doesn't affect Songbird, SeaMonkey or anything else that uses a XUL-based front end, but this really sucks for people who are trying to break through XUL's interface limitations. Camino probably will survive the jump when "3.0" emerges with WebKit, but it's a real shame (especially because w/r/t custom WebKits on Mac, OmniWeb is really my personal choice and it will be hard for Camino to compete against Safari and Chrome as well), and it shows that Mozilla's priorities are nowhere near as aspirational as they used to be. Gecko was what distinguished Camino, but no longer. This is part of why WebKit will eventually eat the world, and we will damn it in the same tones we damn Internet Explorer. But I digress.

So, new business. Mozilla, freed from useful things like embedding, is busily working on Firefox 5, which right now they call Firefox 4.2 alpha. The difference in version numbers isn't too odd, as Fx4 was Fx3.7 originally. However, the major difference is more one of process than content: this will be the first release in which Mozilla will use their new "rapid release" framework. And frankly, I have no firm idea what that means, and from idly perusing the Mozilla newsgroups, there are certainly disagreements about this even among Mozilla higher-ups.

Let's review, then, what Mozilla has placed on their wiki and written by one of their key developers. The release schedule is pretty well known: raw work lands on mozilla-central, then stuff that is finished makes the cut to (these are provisional names) fx-experimental, then stuff that is shippable makes the cut to fx-beta, and then the final browser pops out into a release branch. Each stage is intended to last about six weeks, with approximately one week of overlap betwixt. Firefox 5, to jump start the process (and presumably because there is a lot of pent-up work that didn't make Fx4), will have only three weeks in mozilla-central before moving up the release ladder. That, in the words of Larry King, is what we "know-know."

What we don't know-know are several important considerations. The big one will be support intervals. Chemspills will be supported (and necessarily), but it is highly likely that releases will become unsupported much more rapidly. This makes it a lot harder for us to have a stable footing, and may indicate that security aspects discovered "late" in a release's short lifespan may go unrepaired directly by Mozilla if sufficiently late in the next release's cycle (we may have to backport significantly more fixes and forgo others when we lose source parity).

Similarly, we don't really have a handle yet on how updates will be delivered. Google Chrome has a background self-update mechanism, which has its plaudits and its pitfalls, respectively, that it keeps their audience current but also can more widely distribute serious bugs that escape into a release. Mozilla, continuing their Chrome crush, wants to do the same. From a developer perspective, this requires significant back-end infrastructure to distribute partial fixes -- we don't have this kind of back-end even for one architecture, let alone the four builds we release, including for just keeping users in sync, and would greatly complicate testing. Other Mozilla distributors probably have a similar problem. The solution is to simply build snapshots like we do now as the same sort of "big package," but this may require us to maintain a completely different release schedule. Version numbering is related to this problem. Sayre's document alludes at a way to turn automatic beta updates off if this became a reality, and we would probably ship with it hard-wired that way (just notifications, same as now).

In the meantime, I've been watching what's landing in mozilla-central for "Firefox 5" and the major thing concerning us is that IPC is now required. At one stage I did have Chromium IPC building on Tiger, but when built in debug mode, the resulting libxul (which is also now required) was too large to link. To get around this problem (and to avoid maintaining IPC), TenFourFox 4.x is built with both libxul and IPC off, but that will no longer be possible. Fortunately, I have a solution to the linker issue, and I think I can patch enough places in the current Chromium IPC to still get it to build (I'm not sure it will work, but I'm pretty confident I can at least get it to compile). Plus, the three week period makes it unlikely stuff will be "frivolously" removed that we depend on, let alone 10.5 compatibility. Therefore, the chance is not excellent, but it is reasonably good, that we will make the jump to Firefox 5 and still maintain source parity at least through Mozilla 2.2.

Assuming that we do, however, "Firefox 6" will be a major concern. 10.7 will have emerged by then, and it is quite possible that legacy code may be purged during the process of getting Fx6 into Lion. There is even an outside chance that 10.5 might be dropped and that would almost certainly doom source parity as there would be no good reason not to adopt 10.6-specific optimizations like GCD and the like. It's still too early to forecast this, but even if the worst happens early and we drop source parity later this year, like Classilla I still plan to backport security and stability features and make whatever improvements I am able to do so. After all, I still need a browser for this quad.

6 comments:

Did Mozilla change the Firefox version numbering, like 4.2=FF5, 4.3=FF6, 4.4=FF7 and so on, to make the development appear faster to users? Or what's the point?

About the end of embedding for Gecko: That's sad. Gecko is the only capable player besides Webkit on the Mac and it would have been nice to have a choice. The difference is that Apple thinks into the future. iCab dropped its own engine for Webkit (not for Gecko), Omniweb likewise. Who knows what computers will look like in 10 years, and in which applications or gadgets one could use a rendering engine. The latest Safari *is* available for PPC. I know, not for much longer. But Apple is known to change its OS and processor architectures quite casually. And I'm pretty sure for this reason they will keep processor independence, flexibility and portability for Webkit on a much more efficient and fundamental level than Mozilla does with Gecko.

Well, yes and no. WebKit's layout is pretty cross-platform, but I think it's instructive to look at the situation with the modern Amigas, which are our Power Mac's closest present-day relative. PPC-based Amigas have a decent web browser in Origyn, which is a WebKit derivative, but there is no JavaScript acceleration; it's all interpreter, which is still fairly fast as interpreters go, but still an interpreter.

The reason is that WebKit exclusively relies on JIT compilation of individual JavaScript methods. Such a JIT makes very nice code, but makes porting considerably harder, because you must write an entire compiler for each target architecture. Look at, say, V8, and you'll see what I mean -- there is extensive infrastructure for each supported architecture, and the only ones it currently supports are ARM, x86 and x86-64. SquirrelFish Extreme in Safari is the same way. Every other arch runs in the interpreter. For the longest time with V8, there wasn't even an ARM port, and I believe V8 doesn't even *offer* a straight interpreter (it's compiled or nothing). This is probably why no bright soul has ported Chrome to Power Macs.

When Mozilla and Adobe collaborated on the now ill-fated Tamarin project, the lasting improvement was to target JavaScript compilation to a platform-independent intermediate language (LIR) and then write backends that turn the much simpler LIR into machine code. To avoid the overhead of a method-by-method compilation, the tracer notices when code is hot, records the operations being done, and just compiles those. The compilation is simpler, the backend is architecturally independent, and thus porting to new architectures is relatively easy. Trace JIT doesn't handle everything, which is why Mozilla added method JIT compilation for Firefox 4 (and suffers from the same problem as WebKit), but it handles a lot very, very well. While Tamarin foundered in Mozilla, the trace JIT that originated with that project remains as a very important part of TraceMonkey and JaegerMonkey.

As proof, there are Mozilla nanojit backends for SH4, ARM, x86, x86-64, SPARC, MIPS, and PowerPC (the last, of course, being the one that we polished off and got working). All of these, near as I can tell, now actually do work. In contrast, the tracejit only works on x86, x86-64 and ARM, just like the WebKit code it was descended from.

So, in the Amiga world, there is now work to port Mozilla to it and people have been looking at our nanojit to get JavaScript acceleration because it's easier. Nobody wants to write a PowerPC methodjit compiler from scratch; there's not enough bang for that outlay of "time bucks." In the meantime, they are doing their own port of NSPR, XPCOM and the other pieces, and then will use that to deploy Firefox (as "Timberwolf"). Origyn remains important, largely because it's working and it's here now, but I bet the Amiga community will find that certain performance-critical pieces of code are easier to tune up with Gecko.

W/r/t Apple, I wouldn't be surprised if there were an ARM-based "Mac" in a few years; Apple has a lot of in-house experience with ARM and they own PA Semi (bye PWRficient, *sob*), and Apple knows that their being tied to Intel has similar disadvantages to being tied to IBM. But that discussion is orthogonal to whether WebKit is cross-platform or not, because WebKit is already proven on ARM thanks to iOS. They don't have to do any porting, and there is no other architecture in the pipeline that's any competition to ARM, or for that matter, x86. I certainly don't see Steve swallowing his ego to go back to POWER.

Mind you, layout I think is much more important than JavaScript, and I'm tired of pages with expensive scripting just because they can. Layout in WebKit is very, very good, modulo its own bugs and shortcuts. But a JavaScript-heavy environment is the world we live in and the scripting engine IMHO is nearly as important as the rendering engine. That piece is, outside of the target architectures, very lacking in WebKit.

That said, getting WebKit into a project is a lot easier than getting Gecko into a project, regardless of its limitations. This isn't an issue of architectural portability, however; it's an issue of codebase portability. Since most OSes these days are x86 and ARM, there's no reason for WebKit to be particularly well-optimized to running on anything else, and because it can be integrated much more easily into a project, it will eat the world and further marginalize those of us who use minority CPUs. Gecko unfortunately is going the same way, but there is still hope, and it still works acceptably well on minority architectures. It's just too bad that there will probably never be Gecko without the overhead of XUL, which was the beauty of Camino.

Speaking of version numbers, they're now using a harmonized scheme. Mozilla 5 will correspond to Firefox 5, Mozilla 6 to Fx6, and so on. I'll comment more about that shortly.