Channels

Services

JavaScript benchmark SunSpider reaches version 1.0

After five years in the public eye, version 1.0 of the SunSpider benchmark for JavaScript implementations has been released. In 2007, the WebKit developers at Apple introduced a new benchmark, SunSpider, to the world and, like all benchmarks, it has been criticised on many fronts and also widely used as baseline for testing performance.

In 2010, it was updated to be more accurate as JavaScript implementations were faster and power management was causing a problem when measuring performance. The same year saw the arrival of Mozilla's Kraken benchmark as competition and in 2012, as the battle for JavaScript performance supremacy heated up, Google introduced Octane, its own benchmark – both offered different more "realistic" scenarios for their testing. Despite that, SunSpider has remained a useful benchmark, focusing on the things such as the Date, String and Regexp performance and on the things like eval and for/in syntax, which makes JavaScript a challenging language to optimise.

With version 1.0, the developers have decided to address some issues that have appeared over the benchmark's lifetime. JavaScript optimisers have reached a point where much code in a benchmark could easily be eliminated as "dead code". To fix that and ensure the entire test is run, 23 of the 26 SunSpider tests have had validation checks added to them so that all of the test's code is run and checked for correct running. The developers have also had to revisit the issue of power management again, this time completely eliminating the delay between tests – that delay had been cut from 500ms to 10ms in SunSpider 0.9.1, but now testers were witnessing power management effects with even that small a delay. With no delay, the test suite runs up to twice as fast.