Posted
by
timothy
on Thursday March 17, 2011 @02:08PM
from the yeah-but-those-are-only-real-world-results dept.

bongey writes "In a series of measured real-world web load tests, the Android-based Nexus S phone spanked the iPhone 4. The Android phone and iPhone 4 median load times were 2.144s and 3.254s respectively. The sample size was 45,000 page loads, across 1000 web sites. It also follows rumors that Apple is intentionally slowing down web apps to make their native apps more favorable."

Okay, just so we're all in agreement, when no bad iPhone news happens next week, we're supposed to cycle back to the water sensors getting tripped. We haven't used that one in a while so it'll seem like a fresh complaint. Boy it'd make our jobs easier if the people who actually had the phones would just complain about something.

It was a joke [reference.com]. But I guess you haven't found an app that will give you a sense of humor.;-)

Okay, just so we're all in agreement, when no bad iPhone news happens next week, we're supposed to cycle back to the water sensors getting tripped. We haven't used that one in a while so it'll seem like a fresh complaint. Boy it'd make our jobs easier if the people who actually had the phones would just complain about something.

It was a joke [reference.com]. But I guess you haven't found an app that will give you a sense of humor.;-)

What if, the built-in version proxies requests and doesn't actually serve what you ask for? How do you know what's happening in the browser?

The only reasonable solution to that is to load up an independent app optimized for each of the two platforms. Otherwise, how do you know what the browsers are really doing? If I were building a browser for a phone, I'd do what other mini apps already do and cheat. Not to slow down other apps, like people are hinting Apple may be doing, but to proxy and compress be

I highly doubt that anything remotely like this is happening, and for one simple reason: The proxying you describe (I'm assuming you're talking about Skyfire/Opera Mini type server-side compression?) is dog-slow because it increases the ping-time by a ton. My Desire (old 1st-gen Snapdragon device with a decent helping of RAM) loads sites FASTER without any form of server-side caching and/or compression (I've tried this with Opera Mobile, Opera Mini, Skyfire, Miren Browser...), simply because the response ti

Here, they address this point. First, they compared their app's times with Safari's times, and found no noticeable difference. Second, they point out that javascript performance accounts for a small fraction of the load times (see large yellow box at the top of the page), and if Nitro was not in use, they estimate that using it would improve Safari's load times, but would not dramatically change the results.

Second, they point out that javascript performance accounts for a small fraction of the load times (see large yellow box at the top of the page), and if Nitro was not in use, they estimate that using it would improve Safari's load times, but would not dramatically change the results.

JavaScript is not the only difference between safari and an embedded web renderer. Safari has different caching and multithreading as well.

Here, they address this point. First, they compared their app's times with Safari's times, and found no noticeable difference.

To go to the trouble of testing the thing with their own app, then testing Safari, publishing the numbers for their own app and not publishing the benchmark for Safari seems obtuse in the extreme. Just tell us the numbers you got for the browser.

Second, they point out that javascript performance accounts for a small fraction of the load times (see large yellow box at the top of the page), and if Nitro was not in use,

A web browser renders content and loads it as well as executing stuff; javascript is only one part of the whole operation and only pertains to certain use cases.

they estimate that using it would improve Safari's load times, but would not dramatically change the results.

Why estimate when they can just run a benchmark on the actual browser, instead of handwaving?

Although nowhere near 45,000 tests, Anandtech recently ran a preview of the iPad 2 and did some browsing benchmarks [anandtech.com] to test the CPU where they loaded the pages for the iPad 2, Xoom, and the original iPad. Obviously the two tablets are different animals than the two phones, but given they run essentially the same OS and have beefier CPUs, we should expect similar results.

However, the iPad 2 is clearly faster in 7 of the 8 tests and the same speed as the Xoom in the remaining 1. It's possible that the webs

if Nitro was not in use, they estimate that using it would improve Safari's load times, but would not dramatically change the results.

And that's the best way to run a test. You run a set of well-defined tests and make precise measurements, then you just "estimate" what the real results of a proper test would be...

There's no way around the fact that this test is flawed. "Estimating" and guessing at the results of a proper test is nonsense. Any Slashdotter who has any respect for scientific methodology should be ashamed to be playing so loose here in order to make their favorite product look better than some other. There's a word for this t

Actually, before buying my NEXUS ONE, I looked up quite a few comparison's on youtube. They were pretty much matched, but it some tests the Nexus was faster. In one particular test, by the time the iPhone4 loaded the homepage of the review sites, on the Nexus it was already loaded and a flash video playing. The difference still was just around 1 second, which is not the end of the world of course, but noticeable enough. I concluded that for web browsing, the Nexus is as good or slightly better as the iPhone. And remember, I'm talking about the Nexus One that came out 4 months before the iPhone4. So I do believe there might be something to this... and yeah, I've been a very happy Nexus owner since then. It's longevity is superb - still can't find anything that tops it. I mean yeah, there are better and faster phones out there right now, but I couldn't find a single compelling feature that would prompt me to buy a new phone for the foreseeable future.

No big surprise really. The Nexus One's CPU is 25% faster than the iPhone4's (1GHz vs. 800MHz). The iPhone hardware has always been underpowered compared the Android.

Not at all true –this is one of the first devices to have a more powerful CPU, and the iPhone's GPU is *only just* being beaten by the most recent releases now. Notably, apple is busy replacing that CPU and GPU combination with a CPU and GPU that benchmark about 4 times faster. The iPhone actually has generally had a significant hardware *advantage* compared to android –this is one of many reasons why my company doesn't consider developing games for android –you can't rely on fast hardwa

When it comes to working efficiently, I've always seen that my Nexus S was better than the iPhones that my friends have. This study is just a more methodological and quantitative observation of what I and other Android users already know.

Isn't the iPhone's A4 CPU supposedly some hundred MHz slower than the the one in the Nexus S, giving it better battery life? I don't think this has anything to do with strangling web apps, just different design goals.

Isn't the iPhone's A4 CPU supposedly some hundred MHz slower than the the one in the Nexus S, giving it better battery life? I don't think this has anything to do with strangling web apps, just different design goals.

The iPhone 4 is 777 MHz while the Nexus S is 1 GHz. Both are based on the ARM Corext-A8 and both have 512 MB of RAM. Given the difference in CPU speed, the results of the page load tests don't seem far departed from what would be expected. While the Nexus S is still proportionally a little faster, it isn't so wildly so that it can't be attributed to some minor tweaks in the OS or browser software. Using the term "spanked" seems a bit sensationalist in this instance.

Page load speed, that's their metric? And 50% faster is spanked? We're talking about computers, not 100m dash times - I expect an order of magnitude difference. How is the actual browsing experience - how easy is it to read and navigate on a 4" device?

I will go so far as to quote from TFA:

"Users don’t always notice the speed gap because websites are sometimes tailored to mobile phones, Blaze said. The difference will become more obvious as users demand richer experiences and move to tablet computers with larger screens.

So the metric their using to judge the devices isn't very noticeable, and probably won't matter on a device this size ever. Great. Guess if you have to break out a ruler to feel good about yourself...

Not to mention that they didn't even use the native browser on each platform, but a custom app, which makes this test even more irrelevant. If you want to measure browser performance, then use the bundled browser that the vast majority of users will be using.

Especially since if last week's story about slower JavaScript performance in apps that embed WebKit is correct, that means there's a good chance that the native browser in iOS would spank the Android browser despite being on a slower CPU.

I agree. Unless you are going to run two phones side by side, people will not notice the difference.

My bigger concern is that Safari on the iPhone makes for a poor user experience (at least compared to Opera on my old Nokia Communicator). Opera did some nice reflowing of HTML elements to fit web pages on a small screen. The iPhone makes the virtual screen size default to 920px across and relies on zooming in and out to be able to read things properly. It is particularly bad when reading text on a page that

Now see, I actually like that about the iPhone (and I believe also the default Android) web browser. The way Opera and the default web browser on my Treo both did it was to try to wrap everything to the screen size. It made most sites look awful and poor attempts to wrap around (undersized to the point of near invisibility) graphics often made things hard to read besides. Render the site the way it's meant to be rendered and I'll zoom in and scroll. If I want to look at the graphics I can zoom them triv

You would have been using the crappy Opera Mini, which ran on Java. The one on my Nokia was Opera Mobile (I think that was the name) and it was 1000% better than the one you can download from Opera's site as it had a nice mix of a slight zoom and an intelligent reflow that could fit a site in using the same layout that you see on a PC. It helped that the Nokia was a clamshell design that had an internal screen that was 480px wide.

There was no way of getting the better version except to have it preinstalled

HA! the test is used as evidence on how different devices will behave in an imaginary future where everyone demands rich web apps on future tablets. Extrapolating from this that an android tablet would "spank" the ipad is pure speculation. You don't even need the pretense of science to say you think android tablets in the future could be way faster at everything than an ipad. It's possible, but in that hypothetical future, i'll take the slightly slower ipad that lets me hop on it like the silver surfer's su

From personal observations, I have noticed that transitions are much smoother on iPhones than on comparable Android phones. For example, if I am browsing photos on an iPhone and I swipe left, I see the image smoothly (60fps or more?) move to the left and the new image smoothly move on. By comparison, every Android phone I have seen implements the same effect, but I see artifacts like tearing or skipping of frames. It looks like it goes at 60fps, then drops down to 5, then back up to 60. I tend to be mor

This has a lot to do with hardware acceleration in the GUI, which for the most part isn't there in any Android below 2.3. I bought my Droid 2 last september and noticed exactly what you mention. In 2.3, that's no longer true. It feels MUCH smoother. In fact, my wife went out a month ago and picked up a low end device (with 2.3) that has a much better response rate and feel despite having a processor only half as fast.

I'm not convinced by this. The gallery app and the app drawer are both hard-ware accelerated, and neither is as smooth as the iphone. I think it is more to do with the slowness of java/dalvik, and the garbage collector, which only recently became concurrent.

>> I bought my Droid 2 last september and noticed exactly what you mention. In 2.3, that's no longer true

Good to hear. The jerky UI performance on Android is what prevented me from getting an Android phone. Love my iphone4 right now, but it's definitely nice to hear that Android is catching up and I can easily switch in the future if Apple's restrictions start to bother me.

I absolutely hate the startup screenshot thing in iOS, because when it takes 5+ seconds to load the application I'm mashing the screen wondering when it's going to start. If it said "Loading" or at least looked like a splash screen I'd be less impatient on waiting.

Apple requires apps to include a screenshot to be displayed before the app completely opens, to trick users into thinking the phone is faster than it is. The practical effect is that every single app is 20-100kB larger and starts a fraction of a second slower, but "Apple Users" don't care about that, they just want it to be prettier.

You are misinformed or a liar. I have anapp in the App Store and it has NO STARTUP SCREEN. In fact, since it is version 1.0, it is inefficient in loading up (blank screen before ready!). Apple passed it into the store NO PROBLEM.

From experience, iOS will skip frames or otherwise cut the eye candy animations if it needs to. They have never slowed down the system, as they only use otherwise idle power. It's much harder to see this happen on the newer phones, but I saw it plenty of times back on the iPhone 3G as the OS became more complex. Apple does put a lot of effort into this, complete with dedicated system frameworks to drive the eye candy. Apple's view on the eye candy is that it is there to bring attention to what is occurr

If the animation slows down, that's a good sign that SOMETHING else is going on behind the scenes and the CPU isn't idle. If the CPU was idle then there wouldn't be anything to slow down the animation.

Either that, or it is a sign that the animation code isn't written well. Going back to my original post, I was talking about the effect when moving from one photograph to another in the camera roll. There is no other background task slowing the animation, and the iPhone isn't animating more slowly to make it look smoother. For whatever reason, it seems to be coded to look better. Maybe they are adding a blur effect... or maybe as someone pointed out, they customized the routine for the hardware. But it

Depends on the transitions are very smooth in the 3d gallery of stock android, to bad that most phones use their own gallery instead of the 3d one which is absolutely superior to anything else on any platform, the ios gallery is a joke compared to the android one.

Someone pointed out already that the way they tested is with apps that use the browser engine available to apps. As the second link says in the main story (probably, I'm too lazy to RTFA, I read others already), the iOS browser engine doesn't use the Nitro javascript engine.

I found one link that discusses it, but I'm sure there are better ones:

Where are mod points when you need them? The methodology was flawed since they built a custom app, rather than using the actual browser. Admittedly, there is a bug in iOS at the moment, but most people don't access the web that way.

Personally, I think it's a ploy, scare the consumer into being glad that Apple took mercy on them and "looked out for their interests." (Hah.) But, if not, a 32GB iPhone 4 holds value like nobodies business, so I'll recoup the cost of whatever top of the line model is available for an Android handset. If they do indeed pull it. But I'm not ready to believe they'd take such a step backwards until it's actually gone. Then, I voice my opinion with my dollars.

And I think my local telco now brags about 7 mbps for $19.99. You can do 720p over that. Netflix ahoy.

But yes, I do have the 30 mbps cable line instead. And the cable company is now my phone company. I wish the satellite company didn't have latency issues. Except for its abysmal web channel guide, DirecTV delivers the goods most days.

Oh stop whining. We, as usual, are ignoring both TFA and TFS. We're just happily bouncing our keyboards and gabbing about random things. I'm sure you've noticed that the comments have nothing at all to do with the subject, the article, each other or the laws of Thermodynamics. It's just about Apple and occasionally Microsoft.

âoeWe know thereâ(TM)s no such thing as a perfect Web page load measurement.â

My first thought was, why not have a simple page that grabs the current time, loads a page in the iframe, when the iframe triggers it's ready() event, grab the current time and compare against the start for a load time analysis? the overhead of having it in an iframe can't be *that* bad can it?

This is just like all those articles that say that BrowserX has a javascript engine that is 15% faster than BrowserY. As an engineer that is a tiny bit interesting (only a tiny bit mind you) but as an end user I could not possibly care less. I honestly cannot feel the difference even if it is measurable. Benchmarks hold little fascination for me and are almost always irrelevant to my choice of device. A 2 second versus a 3 second load time? Sure there is a difference but not enough for me to really not

Why is that every time you show some Android product has better feature or performance, call it X, than an competing Apple product the reply from Apple fans follows this logic..

People don't care about X

Eventually Apples popularity will start to fade and people WILL care.
1 second difference can add up to a lot of time if you read many web pages, or you are searching for something. Just do the math. Say 100 modest amount web pages a day , 365 days a year. So you have (100*365)/3600 = 10.13 extra hours spent a year staring at scre

Oh please, if you're so concerned over how much time you've wasted waiting for all of those 1 seconds maybe you should have done the browsing on an actual desktop or laptop computer. Stop making a mountain out of a mole hill.

First, Apple isn't "intentionally slowing down web apps to make their native apps more favorable." They have added a new JS interpreter (actually a just-in-time JS compiler) to Safari, but not to the "normal" web views that other apps can embed. This means only Safari is faster now, others are as fast as before.

Second, this test is flawed since it does not use Safari. It uses a custom app which uses neither the new JS engine nor the better caching of Safari or asynchronous multithreading.

They show that they can beat the iPhone in one discipline (browser speed) with some cheating (custom app, not the default browser). Well, that's not the trick. You have to beat the complete package to be the better phone.

Blaze backed away from its conclusion in light of the new data. Chief Technology Officer Guy Podjarny told CNET in a statement:This test leveraged the embedded browser which is the only available option for iPhone applications. Blaze was under the assumption that Apple would apply the same updates to their embedded browser as they would their regular browser. If this is not the case and according to Apple's response, it's certainly possible the embedded browser might produce different results. If Apple decides to apply their optimizations across their embedded browser as well, then we would be more than willing to create a new report with the new performance results.

I just bought an Android after a year as an Apple guy, and I don't really care how quick it is, I care about the fact that my favourite websites work(lots of flash). An extra tenth of a second isn't going to make or break it.

Browser performance is never "good enough" on degraded platforms like smartphones. On my desktop I'm used to neural-rate browsing. When there's a hitch I send email to ISPs that their routers are fucking up. But on my phone I see every little gear grinding on every badly-constructed javascript botch. One phone that unties the knots 20% faster than another is significantly different from a quality perspective.

If two phones can handle their native apps without breaking, then the one with the best network/