Biz & IT —

Windows browsers benchmarked: October 2010 edition

Whose browser reigns supreme? We test both current releases and the latest …

Our readers enjoy our browser market share stories, but sometimes complain in the comments that we don't do enough to compare the actual browsers. We've therefore decided to do some performance tests for the top five browsers (stable and beta versions) on Windows. This is not meant to be an exhaustive performance rundown, as we have not tried every test in existence nor did we run them on every browser for Windows. Also, please keep in mind that we have yet to identify a test that measures all the factors that influence the performance of a browser.

Thanks to the frequent release of nightly builds, it's practically impossible to test the latest version of a browser and post the results before they're already outdated, but the data here remains recent.

The setup

We used an Asus laptop for every test. It features an Intel Core 2 Duo T9600 processor at 2.80GHz, 4GB of DDR2 800 RAM, a 320GB SATA hard drive (7200 RPM), an Nvidia GT 240M graphics card with 1GB of DDR3 VRAM, and a 15.6-inch HD LCD display (1366 x768).

The first thing we did was add a brand new 50GB partition for a fresh copy of Windows 7 64-bit Ultimate. Then we let the hundred or so patches come through Windows Update. We made sure all the drivers were up-to-date, especially the graphics card's, and then we went out and downloaded all of IE8's best friends: Firefox 3, Chrome 6, Safari 5, and Opera 10. After running all the tests, we went out and grabbed the IE9 beta, Firefox 4 beta 6, Chrome 7 dev, a WebKit nightly, and Opera 10.70.

As a side note, we tried using the latest Minefield build instead of the latest Firefox build, but we couldn't get a single test to run without a crash. Also, when these tests were run, Chrome 8 was not yet available... but as you'll soon see, that didn't hurt Google's results.

The tests

The first test we performed is known as the SunSpider JavaScript Benchmark, and it's one of the most popular tests for measuring JavaScript performance. Version 0.9 was released by Apple's WebKit team in December 2007. We did not use version 0.9.1 this time because it was causing some of the nightly builds to crash.

The second test we used was the V8 Benchmark Suite, developed by Google. We used version 5 of the test because version 6 was not yet available.

All the tests were run on the latest stable and the recent nightly build of each browser. We ran each test three times and then took the average score.

The Results

Before we jump right in, we want to make a quick note about 64-bit browsers. Internet Explorer is the only browser to have official 64-bit versions, so IE 64-bit is the only one we included in our tests. We don't explicitly compare the 64-bit version to the other browsers, since it's not a logical comparison, but the numbers are there for you to take a look at. We will say that, in the tests we did, 64-bit did not make a huge difference for IE.

Still, we did notice an odd pattern: while IE8 32-bit and 64-bit are pretty much on par in most tests, the 64-bit version of the IE9 beta is much worse than the 32-bit version. Given that a 64-bit version of Flash is on its way, we can expect 64-bit browsers to start arriving soon. Until then, though, we're going to stick to evaluating the 32-bit versions.

In this test, Chrome is the winner, with Opera close behind. The gap is larger between Opera and Safari, even larger between Safari and Firefox, and just huge for Firefox and Internet Explorer.

In this test, Chrome is the clear winner. Opera and Safari are both far behind, placing second and third respectively. Way back in fourth, Firefox isn't even close to competing, and Internet Explorer is not even worth mentioning.

This test is a bit more even, and this time Firefox is the winner. Chrome still manages to take second; Safari is third with Opera just behind it, and once again Internet Explorer takes last place.

Chrome once again destroys the competition. Opera manages to keep pace in second place, and Safari has just over half of Chrome's score at third. Firefox takes fourth and Internet Explorer is dead last with an eighth of Chrome's score.

In this test, once again Chrome is first and Opera is second. When comparing the stable versions to the beta versions, we see that IE has moved from fifth to third, pushing Safari into fourth and Firefox into last. In raw numbers, IE has made the biggest improvement as well, while Opera actually did worse, though it kept its position.

Chrome is significantly ahead in this test again, with Opera in second, and Safari in third with less than half of Chrome's score. When comparing the stable versions to the beta versions, IE has moved into fourth, pushing Firefox into last. In raw numbers, IE has made the biggest improvement again, and the nightly build of Opera actually did worse, though the browser once again kept its position.

This time, Firefox snatches first place and Chrome takes second. When comparing the stable versions to the beta versions, IE has moved from fifth to third, pushing Safari into fourth and Opera into last. In raw numbers, all the unstable browsers did worse than their stable counterparts. IE managed to have the least change for the worse, while Opera saw the biggest decline.

Chrome once again takes first and Opera snatches second. Safari and Firefox are close here but have to settle for third and fourth, respectively. Internet Explorer is in last. When comparing the stable versions to the beta versions, none of the browsers swapped places. In raw numbers, IE and Safari improved the most.

Chrome: it's fast!

Chrome is the obvious winner in these tests. It has a such a significant lead that we doubt it's going to be bumped out of the top spot anytime soon, especially if we take into consideration that the team wants to release a major version every six weeks. Still, competition in the browser market is only getting fiercer, so Chrome's king-of-the-hill status may not last forever.

You see, I would switch over to Chrome, but Opera is just too powerful. The seconds I save on page load and javascript rendering time, I lose when typing out full links. To search wikipedia, I type: w search term. Guitar tabs: u search term. Just those little shortcuts make Opera such a better browsing experience for me despite the lack of support.

So basically, Chrome destroys other browsers. I've been using it since it's first beta release, and every dev. build since. It's by far better than any of the rest, and now even the Firefox fanboys are coming over.

Chrome has always spouted about being a minimalistic browser. For all intents and purposes, they'll never include all the bells and whistles we want in favor of a very fast and speedy browsing experience.

I use chrome exclusively at work for speed, and at home I swap between FF4 beta and Chrome depending on what I want to do.

Not surprising to see Chrome so far ahead of the competition. The browser is built for speed and lightweight. It's like comparing an F1 Racer to a Bentley. They're two different pieces of equipment for two different experiences.

I personally just don't like top tabs. As someone who switches tabs with the mouse, moving from the window area to the top tabs actually increases the amount of time it takes for me to move to a different tab. Also, I like being able to read the entire web site title. Call me old fashioned.

You see, I would switch over to Chrome, but Opera is just too powerful. The seconds I save on page load and javascript rendering time, I lose when typing out full links. To search wikipedia, I type: w search term. Guitar tabs: u search term. Just those little shortcuts make Opera such a better browsing experience for me despite the lack of support.

Chrome allows the exact same thing, completely customizable to your every whim.

I just started using Firefox Sync, and I like how it's maintaining my tabs, bookmarks, etc between browsers (and available on my iPhone, though I don't check it there as often). Looking at these numbers though, I feel like Chrome is the obvious choice - do they have a similar feature to Sync with one of their addons? I'd love to know. That's really the only thing holding me back.

Is it just me, or does this emphasis on browser speed seem somewhat misplaced? My only issue with browsing speed for years now has been the annoyance of waiting for some third-party ad server to respond so that the actual content of the page will display.

Is it just me, or does this emphasis on browser speed seem somewhat misplaced? My only issue with browsing speed for years now has been the annoyance of waiting for some third-party ad server to respond so that the actual content of the page will display.

I think it's important but that it isn't all that matters.

Still, it's nice to see Ars do some benchmarks once in a while, because they give you something concrete to talk about. I think it would be cool to see a few "usability tests" and reviews for people who switch between browsers and stuff but those are pretty resource-intensive.

To search wikipedia, I type: w search term. Guitar tabs: u search term. Just those little shortcuts make Opera such a better browsing experience for me despite the lack of support.

Chrome does the same. By default the "shortcut" words are too long, but I have it set up so that w: (string) searches wikipedia, gg: searches google, gi: searches google images, az: searches amazon.com, etc.

Now if only it was compatible with our Sharepoint 2010 portal/ticketing system/document library, I could have all sorts of neat accelerated searches for work-related stuff too. But no, got to keep a real browser for Ars, and IE7 for Sharepoint.

It would be interesting to see these same test on OSX and Linux, at least for the browsers you could test it with. Maybe on Linux throw a classic like lynx in the mix :-) even though the benchmarks would probably be useless..

I used to use Firefox almost exclusively, until I got my netbook. An Asus Eee 1000h does not have alot of horsepower, and simply put...Firefox is slow on it. Chrome however, is not. While those numbers are close, on a low power system, Chrome just blazes. Nothing else on my netbook is faster, and it convinced me to switch to Chrome on my desktop.

I personally just don't like top tabs. As someone who switches tabs with the mouse, moving from the window area to the top tabs actually increases the amount of time it takes for me to move to a different tab. Also, I like being able to read the entire web site title. Call me old fashioned.

Have fun with FIrefox 4 then - it switches over to top tabs as well.

And I'm going to repeat the first poster's point - could we get some details on what each of these tests actually test for, instead of just the results? Thanks

Chrome has always spouted about being a minimalistic browser. For all intents and purposes, they'll never include all the bells and whistles we want in favor of a very fast and speedy browsing experience..

Minimalistic interface-wise, but certainly not in resource usage. The trouble is (at least on Linux, probably true on Windows), Chrome uses 5x as much CPU and 3x as much memory as any other browser. Fine if that's all you're doing, but put more than one user on a machine and it becomes crippling pretty fast.

You see, I would switch over to Chrome, but Opera is just too powerful. The seconds I save on page load and javascript rendering time, I lose when typing out full links. To search wikipedia, I type: w search term. Guitar tabs: u search term. Just those little shortcuts make Opera such a better browsing experience for me despite the lack of support.

Chrome actually supports...right click the URL bar->Edit Search EnginesYou can setup keyword searches like in Opera and Firefox. The more useful feature IMO is that if you use a site with a search box like wikipedia or netflix, it'll automatically build keyword searches for those sites. To search wikipedia, I just start type wikipedia.org, and by the 2nd or 3rd character I get the option to hit tab to search wikipedia...type in your term and off you go

I mostly use Chrome these days, though I don't care for the UI as was previously mentioned. Honestly, I don't notice a huge difference in rendering speed on the sites I visit, but I've found it to be more stable for me than Firefox. IE 9 is looking interesting though.

It's a bit dated now, but one of the Fedora Project package maintainers (Tom Callaway) had some interesting, and less than complimentary things to say about Chrome. I wonder if this might explain why it's so fast?

Quote:

Google is forking existing FOSS code bits for Chromium like a rabbit makes babies: frequently, and usually, without much thought. Rather than leverage the existing APIs from upstream projects like icu, libjingle, and sqlite (just to name a few), they simply fork a point in time of that code and hack their API to shreds for chromium to use. This is akin to much of the Java methodology, which I can sum up as "I'd like to use this third-party code, but my application is too special to use it as is, so I covered it with Bedazzler Jewels and Neon Underlighting, then bury my blinged out copy in my application.". A fair amount of the upstream Chromium devs seem to have Java backgrounds, which may explain this behavior, but it does not excuse it. This behavior should be a last resort, not a first instinct.

Thanks for the data, but it would have been nice to have at least a little discussion as to what each of the benhmarks was testing...

What I get out of this is Chrome is fast and IE is horrific... Everything else is pretty good.

My primary browser (on Mac though) is definitely still FF but I'm trying to switch to Chrome, the extsions in FF are still better (FireBug, 1password, Greasemonkey, Evernote clipper, instapaper clipper). Even when the extensions exist for Chrome they aren't as robust as on FF

Doesn't change the fact that Chrome has been proven to make your eyes melt and bleed out of your skull from the sheer disaster that is its interface.

care to elaborate with specifics? I find chrome the most minimalistic. I have different themes, but use minimal at work, it's just white. empty space is almost non existent in chrome.

@squidz: What are you smoking? It's minimalistic interface is exactly why I'm using Chrome. Speed being the second. Chrome shows only what is needed, no unnecessary waste of space, has every functionality within reach without any keyboard use. What's not to like?

It would be interesting to see these same test on OSX and Linux, at least for the browsers you could test it with. Maybe on Linux throw a classic like lynx in the mix :-) even though the benchmarks would probably be useless..

Yes, I was planning on doing a Linux version. The main reason why I didn't do both Windows and Linux at the same time was simple: I don't want confusion and the two being compared against each other.

MrBabou wrote:

Based on these results, I'm surprised Opera is not a more popular browser than it is...

Based on these results, I'm surprised Opera is not a more popular browser than it is...

And I'm surprised that anyone uses Firefox at all. I understand IE (inertia, compatibility, security) and I understand Chrome (performance, almost IE8-class security)... but Firefox is last in almost every objective category.

I personally just don't like top tabs. As someone who switches tabs with the mouse, moving from the window area to the top tabs actually increases the amount of time it takes for me to move to a different tab. Also, I like being able to read the entire web site title. Call me old fashioned.

Chrome is missing two things that I rather like about it's slower friends.

1) A tree-tab plugin. I use a lot of tabs. TreeTabs is where it's at on FireFox. I've looked for a plugin for Chrome about a half year ago, and didn't find anything comparable.

2) I can't tell when it's "thinking". Yes it's fast, and thinks a bit less than the others. But the user feedback on "Hold on, I'm rendering" would be nice. The others absolutely need this, but the 2 seconds I'm waiting for Chrome wondering whether or not it has crashed is irritating.

I have a question though: What does Browser Stability mean? And how is it tested?

1. Turn on HTTP pipelining in Firefox.2. Conduct a real browser test by timing the browsers cold loading Ars, quickly scroll furiously to the bottom as the browser rendering engine allows, Click "Wired" under Conde Nast. Repeat this same behaviour fivetimes for other Conde Nast sites and tell me which browser is the fastest to reach the bottom of the 5th site.3. Javascript performance != rendering performance.

Chrome is actually quite unresponsive when it's still trying to render, having no HTTP pipelining doesn't help on sites like Ars either. With Firefox, I can traverse and manipulate the pages while it's rendering.

Sorry, Chrome. I know exactly where and what I am looking for on a website even before you finish loading/rendering it. When you learn to obey my scroll wheel, I'll happily give you my default browser check mark.

Based on these results, I'm surprised Opera is not a more popular browser than it is...

As I pointed out in the MS forum, Opera is the most popular browser in the former Soviet Union, except Russia, where Firefox narrowly beats it (check out Belarus, Opera totally rules there!).

Dunno, the world is weird.

Probably all the browsers are fine in their latest versions, so it boils down to whatever you're used to, and word of mouth/media. I am an Opera user, have been since version 5 or so. But I have to use Firefox often as that's what other people here use, and it's fine. Shrug.

I tried chrome for a while and while it's fast, that speed isn't worth it to me to put up with some (IMO) horrific UI decisions. I've finally gotten used to the whole "tabs on top" thing that browsers are heading to with the Firefox 4 beta, though the inability to open new tabs by double clicking in the wasted space to the left is annoying. The early Safari 4 builds and Chrome put the tabs in the title bar which completely ruins the consistency that applications should have. No amount of speed will tempt me to chrome until they give me the title bar back.

Aside from that, I agree with the idea of summarizing the various benchmarks used at the top of the article.

Chrome is actually quite unresponsive when it's still trying to render, having no HTTP pipelining doesn't help on sites like Ars either. With Firefox, I can traverse and manipulate the pages while it's rendering.

Sorry, Chrome. I know exactly where and what I am looking for on a website even before you finish loading/rendering it. When you learn to obey my scroll wheel, I'll happily give you my default browser check mark.

Opera responds while rendering/waiting for data to arrive, and that can be quite confusing as the page jumps around. It's hard to click a link if something renders that causes the link to suddenly be somewhere else.

I think waiting for the page to render before accepting userinput isn't /that/ bad.

I used to use Firefox almost exclusively, until I got my netbook. An Asus Eee 1000h does not have alot of horsepower, and simply put...Firefox is slow on it. Chrome however, is not. While those numbers are close, on a low power system, Chrome just blazes. Nothing else on my netbook is faster, and it convinced me to switch to Chrome on my desktop.

Almost the same experience. If I need to browse quickly (i.e., open a web page as soon as booting & login is complete) I fire up Chrome. An example: I need to open Gmail quickly to fetch something a colleague has sent me.

BUT

If I want to do a general browsing, e.g., doing some research, I fire up Firefox. I just couldn't live without AdBlockPlus, NoScript, TabMixPlus, and TextLink.

You see, I would switch over to Chrome, but Opera is just too powerful. The seconds I save on page load and javascript rendering time, I lose when typing out full links. To search wikipedia, I type: w search term. Guitar tabs: u search term. Just those little shortcuts make Opera such a better browsing experience for me despite the lack of support.

Chrome allows the exact same thing, completely customizable to your every whim.

Barring making changes to the source code and re- compiling, not really. I know it's kind of dumb, but one of the things I appreciated about Opera was that you could very easily set it up to use "unix keyboard shortcuts", while on chrome doesn't seem to allow any customization of keyboard interaction. (not just shortcuts, the great thing about opera was that it would change key-behavior for text entry). You can get the same functionality in Firefox by adding extensions like "firemacs". But chrome has nothing.

On the other hand, Chrome (or in my case, Chromium), IS notably faster to start up and it renders some pages faster than Opera and Firefox, while I've never seen it render a page slower. Also, on Linux, Chrome has fewer bugs and quirks than Firefox. So even though I don't really like the user interface, for me, the speed gain and lack of bugs make chrome my primary browser.

I would consider Opera, but I'm not sure I trust the Opera distributors, I always feel more comfortable using open-source software.

Based on these results, I'm surprised Opera is not a more popular browser than it is...

And I'm surprised that anyone uses Firefox at all. I understand IE (inertia, compatibility, security) and I understand Chrome (performance, almost IE8-class security)... but Firefox is last in almost every objective category.

I tried Chrome and I still use it occasionally, but Firefox is my number one choice. Extensions and customizability trump any artificial speed tests. I haven't noticed any useful speed differences between Chrome and Firefox on the sites I've used.

A more useful browser speed comparison would have been using some automated scripts to go to the top x websites.