Is Mozilla’s 2011 roadmap unrealistically ambitious?

Mozilla plans to deliver four major new versions of Firefox in 2011. Ars takes …

Mozilla has published an updated roadmap in which it lays out its plans for 2011. The organization hopes to significantly shorten its release cycle and deliver a total of four major releases during 2011, cranking the browser up to version 7 by the end of the year.

Some of Mozilla's key technical priorities include improving responsiveness, integrating social sharing, refining the user interface, supporting 64-bit Windows and Android tablet form factors, finally delivering process isolation for tabs, and supporting emerging standards like CSS 3D transforms and WebSockets. In terms of features, Mozilla's 2011 roadmap is compelling and achievable. There is room for skepticism, however, about the organization's new release management strategy. Instead of aiming to roll all of this functionality out in a major release next year, Mozilla intends to push it out to users incrementally, using a series of three releases after the upcoming launch of Firefox 4.

The ambitious plan will require Mozilla to completely reinvent its development, testing, and release management practices. The organization currently averages one significant release a year and typically spends months on beta testing and quality assurance before officially issuing a new stable version. Mozilla will face major technical and operational challenges as it attempts to transition to shorter release cycles.

Mozilla started talking about tightening the development process and issuing smaller releases at more regular intervals shortly after the release of Firefox 3 in 2008. The idea seemed particularly promising at the time. The significant architectural overhaul in 3.0 made it much easier to add certain classes of features that simply hadn't been practical before. There were also lots of leftover enhancements that hadn't quite been ready for inclusion in 3.0 but were perfect candidates for subsequent minor point releases.

It was clear that the momentum from the 3.0 release would be able to carry Mozilla forward faster. There were plans in place almost immediately for a 3.1 release with a handful of nice improvements. Of course, there never was a 3.1 release. The pace of development after the 3.0 release was extremely fast, but the development cycle lengthened considerably as the scope of the planned improvements expanded. Mozilla ended up doing the release the following year—in 2009—and decided to rename it 3.5 during the beta testing period, a number meant to reflect the breadth of new functionality incorporated in the update. Firefox 3.6 was released in 2010 with a comparably significant collection of improvements.

As the Firefox 4 release approaches, Mozilla is once again taking the opportunity to explore the possibility of a more incremental development model. There are a lot of good reasons why rapid iteration is desirable, especially now, but the transition would come with a lots of challenges.

Why Firefox would benefit from more frequent releases

Web standards are evolving at a faster pace than they have in the past. We are seeing some high-profile emerging standards like WebGL going from the concept stage to implementation in an astonishingly short time window. Acceleration of the standards draft process has arguably changed the entire culture in which Web standards are formulated and implemented. The recent decision by WHATWG to discard the version number of their draft of the HTML standard reflects the industry-wide shift towards a faster and looser approach to rolling out new Web functionality.

Mozilla is often at the forefront of developing innovative new Web standards, but generally takes longer than some of its competitors to deliver new features in a stable release, and that's due to the protracted nature of the Firefox development cycle. Speeding up the release cycle would make it possible for Mozilla to get new Web development features out to users sooner.

Another key issue is competitiveness, keeping up with other browser vendors. Our readers often cynically characterize version number inflation as a marketing ploy, but that's not really at issue. It's worth noting that Google, for example, doesn't prominently display Chrome's version number anywhere. There is a competitive metric that seems a lot more visible lately, however: JavaScript performance. The leading browser vendors are largely on even footing in JavaScript performance, so the winner of the benchmarks usually ends up being the company that has pushed its latest round of optimizations into a stable release.

If you run JavaScript benchmarks on the latest stable versions of all the mainstream browsers, you will find that Firefox lags far behind the rest of the pack. It's not just dead last—it's pitifully left in the dust. You get a much different picture, however, if you compare the latest pre-release builds, where Firefox's JavaScript performance is fully competitive with that of other browsers.

The amount of time that it takes Mozilla to get their latest and greatest JavaScript engine out to users sometimes puts them at a competitive disadvantage. A shorter cycle could potentially help them get critical performance optimizations to users sooner.

Challenges of releasing more frequently

It's important to understand that we are talking about release management, rather than the actual speed of development. For example, imagine that you have 10 major features that you want to release in one year. In principle, the length of the development cycle determines whether you roll out a few of those features at a time in shorter intervals or implement them all during a longer cycle and issue one big release at the end.

Issuing more frequent releases doesn't imply faster development. In fact, a shorter cycle can actually have a detrimental effect on the pace of development. A quick development cycle can make it challenging to deliver all of the new features that you want and still have sufficient time left over for bugfixing and quality assurance before you ship.

New versions of Firefox have always had extremely long beta test periods, during which the developers fastidiously eliminate regressions and ensure that none of the new features are going to break anything. QA is a notoriously thorny challenge for HTML rendering engines because improvements can lead to all kinds of subtle changes in behavior that look innocuous on the surface but end up breaking the way certain Web pages render. Making matters even harder, HTML rendering engines have to be able to cope with egregiously invalid HTML, JavaScript, and CSS code in predictable and consistent ways.

In order to adopt shorter development cycles, Mozilla will have to completely change its approach to quality assurance. Instead of having a lengthy testing period for each new version, the developers will have to do a massive amount of continuous quality assurance on the tip of the code base and make sure that nothing breaks as the code is landing. Features that are under development will have to be tested more rigorously in external branches before they can be merged. Shifting more resources to stabilization, code review, quality assurance, and deployment could potentially detract from the resources available for actual development.

In the Linux ecosystem, there are a number of large-scale projects that conform with strict time-based release models. It requires an enormous amount of discipline and can sometimes create problems.

One particularly noteworthy example is Ubuntu, a Linux distribution that issues a new version every six months. Ubuntu is generally pretty robust, but Canonical's uncompromising commitment to releasing Ubuntu on time every six months has created an atmosphere where developers are reluctant to treat serious bugs as release blockers—the releases consequently end up going out to users with bugs that get patched later. When software developers adopt shorter release cycles, they need to be careful to not lose sight of quality considerations in the rush to do a "stable" release.

Can Mozilla do it?

Mozilla is tremendously adaptable, has profound engineering skill, and benefits from some really good managers who have a deep understanding of the code. They have achieved seemingly impossible things in recent history—particularly their timely and well-executed move into the mobile browsing market. As such, I wouldn't rule out their ability to do anything.

Despite my faith in Mozilla's development prowess, I lean towards a skeptical view of their proposed roadmap. Unlike WebKit-based browsers, Firefox's code is a bit crufty in places and isn't as conducive to rapid iteration. Shorter cycles are also just not in the organization's DNA. Making Mozilla's release process more agile is going to require a major cultural shift and a departure from the extremely cautious release management strategy that has served it well up to this point.

It's a bit of a stretch to imagine Mozilla sustainably rolling out three or four new versions with major features during a year. Attempting that cadence could possibly erode the QA process and necessitate more bugfixing of stable releases alongside the development of new versions. I think a six-month cycle is a lot more credible and a lot less likely to inflict collateral damage on the browser's quality. If they can successfully deliver three or four releases a year, however, it's definitely going to make Firefox more competitive.