WebKit releases new SunSpider 1.0 benchmark to chart the future of JavaScript performance

Apple's WebKit team has released a revamped new SunSpider 1.0 benchmark suite for profiling the performance of web browser JavaScript execution, replacing the 0.9 version initially released at the end of 2007.

SunSpider was initially released with the goal of measuring how quickly a JavaScript engine can perform a balanced variety of tasks, with the goal of profiling how effective new advances will be.

JavaScript is the programing language of the web, handling tasks ranging from numerical calculations to date and regular expression manipulation to object oriented programming tasks that have, collectively, turned the originally simple web browser into a platform capable of hosting feature-rich web applications.

JavaScript is also used to power interactivity outside of the web browser, such as in iOS' iAd, the interactive widgets used in iBooks and in developing dynamic content in iTunes including iTunes LP albums and iTunes Extras for DVD-like, interactive menus for movies and related bonus material.

In the five and a half years since SunSpider 0.9 was released, it has documented a more than 30x improvement in JavaScript performance from a typical 2.33 GHz computer of 2007 running Safari 3 to a modern 2.2GHz i7 MacBook Pro running Safari 6.

The new 1.0 version just released "fixes a number of bugs and aims to further increase test accuracy and repeatability," taking into consideration the much more sophisticated design of modern JavaScript engines.

Tracking the progress of JavaScript engines to power HTML5

Back in 2007, when the SunSpider benchmark first appeared, the execution of JavaScript hadn't changed dramatically since its introduction by Netscape in 1995 as a way to perform basic calculations within a web browser on the local client side (rather than remotely by the web server).

After SunSpider appeared, an intense new push to drive more programatic sophistication into dynamic web pages resulted in dramatic new leaps in JavaScript technology, first under the term "Ajax" (relating to web pages that can update themselves within the browser without needing to be redrawn by the server) and increasingly related to the promise of "rich web apps" related to HTML5 (see also: "Why Apple is betting on HTML 5: a web history").

In 2008, the WebKit team announced a rewriting of the original JavaScriptCore as a direct-dispatch register based, high-level bytecode virtual machine named "SquirrelFish." Mozilla subsequently announced its TraceMonkey engine for FireFox and Google acquired the V8 engine to accelerate JavaScript performance in Chrome.

Before the year was out, and before SquirrelFish even made it into a shipping version of Safari, the project was enhanced further under the name "SquirrelFish Extreme," which accelerated JavaScript execution using a Just In Time compiler to turn JavaScript into native machine code as an alternative to generating bytecode.

In 2009, Apple applied the enhancements to Safari 4 under the brand "Nitro," noting that the new implementation could run JavaScript up to 4.5 times faster.

The next year, it subsequently released new SquirrelFish Extreme enhancements in Safari 5, boosting JavaScript performance on the Mac another 30 percent over the previous Safari 4. In 2011, the company added Nitro to mobile Safari in iOS 4.3, boosting JavaScript performance by 200 percent on Apple's mobile devices.

A new benchmark to chart the future progress of JavaScript

A year ago, WebKit announced further enhancements to JavaScriptCore that rendered it a "triple tier virtual machine," incorporating an efficient new LLInt (Low Level Interpreter) used by JavaScriptCore to attempt the execution of code before passing it to the standard interpreters, either the bytecode virtual machine or the JIT (Just In Time complier) which builds native machine code on the fly.

The revamped new SunSpider takes into consideration these increasing levels of complexity in modern JavaScript engines. As the WebKit announcement explains, due to this complexity "it makes sense to make our benchmarks perform some validation of correctness. Benchmarks are uniquely capable of stressing the optimizing JIT in a way that conformance tests cannot. This observation comes from our own experience: as we added validation to our benchmarks, we were able to catch bugs sooner."

In addition to new validation checks, the revised benchmark eliminates delays in the original version that were added to allow the browser to complete each portion of the tests before moving to the next. However, these delays started distorting the results because on fast hardware, the test suite's built in delays were allowing hardware power management features to kick in, sporadically creating the false appearance of poor performance.

Competing for the future of HTML5

While Apple and Google have worked together closely for over half a decade to improve WebKit, particularly with the goal of making HTML5 rich apps possible across both desktop browsers and on mobile devices, the two companies are now separating their efforts after Google decided to fork the project and take its own Chromium code solo (under a project now named "Blink").

This allows Google to purge any links to Apple's Nitro engine (which it doesn't use in Chrome), necessitating a similar purge of Chromium code from the main WebKit project. This may likely impact third parties such as Samsung, which uses Apple's WebKit web engine but pairs it with Chromium's V8 JavaScript engine, now part of Chromium.

Somewhat ironically, Google's fragmentation of Apple's WebKit platform (as the platform's largest independent user) is similar to what many expect Samsung to do to Google's Android project, ultimately leaving Google to maintain its platform while Samsung runs with its own copy, following the footsteps of Amazon and a variety of Chinese forks.

At the same time, Google also invested in a project to replace JavaScript with its own new web programming language named Dart, a move that was opposed by Apple, Mozilla and Microsoft.

At the end of 2011, Apple's Oliver Hunt warned that "adding direct and exposed support for a non-standard language [like Dart] is hostile to the open-web by skipping any form 'consensus' driven language development that might happen, and foisting whatever language we want on the web instead. This implicitly puts any browser that supports additional proprietary extensions in the same position as a browser supporting something like [Microsoft's proprietary] vbscript, and has the same effect: breaking the open web by making content that only works effectively in a single product."

Alongside its flirtation with Dash in 2011, Google also advocated Adobe Flash (hardwiring it into its Chrome browser) and worked to derail the H.264 video standard in favor of its own WebM video codec, two other efforts that were clobbered by the reality of mobile devices dominated by iOS. Google has since stated that it does not plan to push adoption of Dart within its own "Blink" Chromium fork.

Apple is expected to introduce new versions of Safari incorporating the latest advances in WebKit for iOS 7 and OS X 10.9 at next month's Worldwide Developer Conference.

At the same time, Google also invested in a project to replace JavaScript with its own new web programming language named Dart, a move that was opposed by Apple, Mozilla and Microsoft.

Isn't Javascript a W3C open standard language? Why would developers want to code in a language that requires a proprietary browser? Microsoft finally got on board with open standards in IE9. Why topple the apple cart when things are just now starting to work across all platforms? The fact that Google's business model is based on getting as many eyeballs on their pages as possible, why would they introduce a platform that immediately reduces their marketshare by being incompatible with the rest of the web?

The fact that Google's business model is based on getting as many eyeballs on their pages as possible, why would they introduce a platform that immediately reduces their marketshare by being incompatible with the rest of the web?

Isn't Javascript a W3C open standard language? Why would developers want to code in a language that requires a proprietary browser? Microsoft finally got on board with open standards in IE9. Why topple the apple cart when things are just now starting to work across all platforms? The fact that Google's business model is based on getting as many eyeballs on their pages as possible, why would they introduce a platform that immediately reduces their marketshare by being incompatible with the rest of the web?

I think Google had an inflated estimation of its own ability to shift the tech landscape. That's been toned down after a series of rather high profile disasters (Buzz, Honeycomb, WebM, G+, Wave). I think Google expected the rest of the web to come rushing to its door to adopt Dart as a superior alternative to JS.

Google acts like one of those freshman Tea Party representatives that campaigned on "changing Washington!!" and has instead realized that a) change comes slow and needs to b) its easier to talk than walk and c) the world does not work anything like it imagines. The Android advocates have all bought into the campaign promises and are about to be upset when they realize it was all BS and all they've accomplished is turning back the clock a bit and wasting tons of billions of dollars.

I think Google had an inflated estimation of its own ability to shift the tech landscape. That's been toned down after a series of rather high profile disasters (Buzz, Honeycomb, WebM, G+, Wave).

... along with some high-profile successes like Android, GMail, Chrome, and Google Maps that did have the ability to "shift the tech landscape".

Not every product a company releases has to change the world for them to be successful. "High-profile disasters" like MacIntosh TV, Pippin and the Newton didn't scare Apple away from those markets nor prevent them from learning and doing much better with their subsequent products. You seem to form some of your opinions with one eye closed.

Personally I don't have any opinion on whether Google should be taking a stab at it. I'm no programmer and have no knowledge of Javascripts limitations if any. Writing DART off as a failure seems a bit premature tho.

Personally I don't have any opinion on whether Google should be taking a stab at it. I'm no programmer and have no knowledge of Javascripts limitations if any. Writing DART off as a failure seems a bit premature tho.

Dart and Go are far from failures, they're very good languages, the Dart VM is quicker than even V8, but for developers I don't think they're compelling enough to switch from JavaScript and C++. V8 is going to be developed and used for a very long time I think...

Apple's WebKit team has released a revamped new SunSpider 1.0 benchmark suite for profiling the performance of web browser JavaScript execution, replacing the 0.9 version initially released at the end of 2007.

SunSpider was initially released with the goal of measuring how quickly a JavaScript engine can perform a balanced variety of tasks, with the goal of profiling how effective new advances will be.

Apple is expected to introduce new versions of Safari incorporating the latest advances in WebKit for iOS 7 and OS X 10.9 at next month's Worldwide Developer Conference.

How do we run Sunspider benchmark tests offline. Like storing the webpage and then running offline without using Internet and displaying the result.