Test results promoted by Blaze Software that purport to prove that Android is much faster at loading web pages than Apple's iOS 4.3 did so using a poorly performing custom iPhone app, rather than using Safari itself.

The results of the test, according to Bloomberg, said that an Android-based Nexus S phone performed 52 percent faster on average after loading more than 45,000 pages from 1,000 websites compared to iPhone 4.

The average speed difference was about a second longer page load on iPhone 4: 2.14 seconds compared to 3.25 seconds. The more complex the page, the greater the performance difference, Blaze reported. Guy Podjarny, the firm's chief technology officer, said "its not that Apple doesnt care about speed, but Google is fanatical about it."

However, while Blaze maintained that its benchmarks used the newly released iOS 4.3, suggesting that it took into account the fast new Safari browser with Apple's new Nitro JavaScript engine, the way it performed the tests completely bypassed those improvements.

Rather than using Apple's Safari browser directly, Blaze tested page loading on iPhone 4 using the company's own proprietary app that did not take advantage of the new improvements in iOS 4.3.

As noted in a previous report by AppleInsider, apps that implement Apple's UIWebView to provide web browsing functions within an app (as Blaze did), in addition to full screen web apps, do not take advantage of the new web acceleration features Apple introduced in iOS 4.3, including Nitro and a variety of other improvements to the mobile Safari browser.

While Apple hasn't officially commented on the disparity between the newly revamped Safari and the features of the UIWebView framework, it appears that the difference relates to both to the fact that Apple wanted to rapidly roll out new WebKit features quickly to mainstream iOS users in Safari (and simply didn't have time to retrofit every other element of the system with the new code), and also to security considerations.

Apple's new Nitro JavaScript engine (originally called Squirrel Fish Extreme) competes against Google's Chrome V8 and Mozilla's FireFox TraceMonkey to speed JavaScript (the programming language behind the web) using various different approaches, each of which has different strengths and advantages.

Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.

Running an automated test on page loading using the actual Safari browser on iPhone 4 would be far more difficult to perform, but would also fail to account for other, likely more important differences between iOS and devices running Android.

These include overall stability and usability of the platform, power management and battery life, hardware quality, and easy access to iTunes music and movie rentals, iBooks, and App Store, three features Apple has started promoting in series of new ads that end with the line, "if you dont have an iPhone, well, you dont have an iPhone.

Any time some study comes out pro or against an Apple product, it inevitably fans flames from all sides. Go have fun Apple Devotes and Android Fans. I want to see a big old fight all over a 1 second difference in web page loading on a mobile device. I'll eat my popcorn.

What I found interesting was this:

Quote:

Originally Posted by AppleInsider

Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as a both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.

Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...

These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.
People relax! Its just an OS.

Indeed! I would be the first to point out that all OSs, regardless of the company attached, have positives and negatives against the other OSs. I love aspects of the webOS, and I love aspects of the iOS, and love aspects of the Android OS (I have not had experience with the newest WinMobile OS, so I can't vouch there). There are also aspects I hate on all. Its a to each their own, and respect the decisions and argument points. Studies like this I feel just increase a "Mine is Better/Bigger than Yours" when in the real world, its all about personal preference. Just cause a study says that the thing you have is now "devalued", doesn't mean it really is. If it works great for you, and you like how it works, then the study doesn't mean anything for you. If you happen to have issues that a study shows, then that's something different.

I think that those "my d*** is longer than yours" contest are stupid so I don't mind if Android Webview loads a page 1 millisecond faster or slower than Apple's UIwebview but: how Nitro JS can make pages load faster. The only thing I whink it can make loading pages faster is asynchonous loading so it's not so flawed the contest.

Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...

Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...

The title of the test is about whose browser is faster. They assumed that the embedded browser and native browser were exactly the same. Security concerns are the reason Apple has not opened Nitro to its embedded browser. They would need to sandbox the process so that it becomes a trusted chain of executable code.

Quote:

Originally Posted by jb510

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps...

Not much of a "clarification" when they say, "We were wrong but we're still right."

Or this bit:

Quote:

Despite this fundamental testing flaw they still only found an average of a second difference in loading Web pages.

We see this is a bad interpretation of our results. First and foremost, our tests were run over networks and conditions more favorable than the average user browsing on his mobile device. Second, on many sites the gap was greater in absolute terms (for example, on wsj.com we saw a 5-10 second gap). The median gap was only one second, thanks in part to the great network conditions.

So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.

So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.

Two seconds vs. three seconds. OMG. I'm Glad I live in a remote area of Alaska and have access to the web using a satellite connection (Starband). They promise that my maximum download speed will not exceed 0.5 Mb per second and in practice I get 0.45 Mb / sec. This is achieved with large downloads such as software updates from Apple. The iOS 4.3 update downloaded in just over 3 hours (actually a total time of 4.5 hours since the first attempt failed halfway through and iTunes started over when I attempted a second download).

The weak link in my setup definitely is not my MacBook. I don't think it will be my refurbished iPad 1 either. Apple Technical Services is sending me a mailer to return the refurbished unit which takes 73 seconds to boot after the white apple appears on the screen. I see that white apple occasionally (actually occasionally x 10, which is frequently, but it's more fun to try to spell occasionally), Usually when I try to type, or when I click on an icon. They sent the mailer a week ago and it should arrive any day. If it's a FED EX mailer I will have to drive 130 miles to Fairbanks to the Fed Ex office which is located about 2000 feet from MacHaus of Fairbanks, the nearest Apple Service Provider.

Not due to differences in "browsers", simply due to network issues. That whole point they tried to make was completely stupid. And, the gap will only be wider under controlled conditions. In the wild, which is what they are talking about, where you can't control conditions, the gap could become smaller or reverse. They're idiots.

Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...

Of course there is a double-standard. That's the advantage of being the builder of the device and the OS. You get to install your apps on the shipping device. You get access to private APIs for your applications that nobody else is allowed to use. It's not right or wrong, it just "is". It's Apple's embeded browser API they are using. They could always build their own browser right into the application. They used the one Apple provided, and regardless of the reasons, Apple decided to not provide that browser the same enhancments Safari got.

Quote:

Originally Posted by TenoBell

The title of the test is about whose browser is faster. They assumed that the embedded browser and native browser were exactly the same. Security concerns are the reason Apple has not opened Nitro to its embedded browser. They would need to sandbox the process so that it becomes a trusted chain of executable code.

Quote:

Originally Posted by anonymouse

It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.

But until the story was reported after 4.3 was released, how many people had the reasonable expectation that the embedded browser used the same engine as Safari? That's the way it's been since the beginning of Safari on both the Mac and iOS, right? Other applications used the same WebKit code that Safari used. Apple didn't tell anyone that in iOS 4.3 it was any different. So how was Blaze supposed to know? They've even put a disclaimer at the very top of their original story. So please, explain to me where the "deception" is.

Now that they know, should they try to repleat the test using Safari? Sure. But as even AI stated, that could be very difficult to execute.

lol this is an easy argument to solve and its why we continue to see these trumpeted stories of superiority from android supporters.
Solution:

An android user and, iphone user walk into a room of 20 people round a large table. Sit, take out your device and place it on the table... now flip it over so the back is showing. As i've tested this, 13 out of 20 ask if thats the iPhone and how do you like it? Out of the remaining 7; 5 Already own an iPhone, . 1 owns a blackberry (haha) and the lone Android user comes to the realization that he doesn't have the cool device.

... But until the story was reported after 4.3 was released, how many people had the reasonable expectation that the embedded browser used the same engine as Safari? That's the way it's been since the beginning of Safari on both the Mac and iOS, right? Other applications used the same WebKit code that Safari used. Apple didn't tell anyone that in iOS 4.3 it was any different. So how was Blaze supposed to know? They've even put a disclaimer at the very top of their original story. So please, explain to me where the "deception" is.

For one thing, their headline is completely dishonest. Secondly, they made a bunch of assumptions that were stupid to make, so that's no defense for their dishonest lead.

They hadn't claimed that they measured Safari and Chrome, they claimed that they measured iOS and Android browsers

"The measurement itself was done using the custom apps, which use the platforms embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone. Manual verification showed that page load performance of the embedded browsers, when properly configured, is effectively identical to the stand-alone browsers. The load times are calculated using the Document Complete callback from the browser, which is a standard way of measuring a web pages load time. As mentioned above, the agents are now a part of a free service available at http://blaze.io/mobile/, and we encourage you to try it out."

And yes, they used the browsers and until iOS 4.3 the performance of embedded browser and Safari browser was the same as it is with android embedded and Chrome browsers.

However, how so? Please explain. If you explain it well enough, I'll change my opinion.

There's no double standard at all. Apple only allows code it has direct control over to run interpreted/downloaded code. It's well known that this is done for security reasons; allowing all apps to run interpreted or other downloaded code makes it impossible to screen them for malicious intent. So, it's not a double standard, it's the only way that they can provide any level of security, and it's a single well understood standard based on well understood reasons.

Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...

There's no double standard at all. Apple only allows code it has direct control over to run interpreted/downloaded code. It's well known that this is done for security reasons; allowing all apps to run interpreted or other downloaded code makes it impossible to screen them for malicious intent. So, it's not a double standard, it's the only way that they can provide any level of security, and it's a single well understood standard based on well understood reasons.

There's no double standard at all. Apple only allows code it has direct control over to run interpreted/downloaded code. It's well known that this is done for security reasons; allowing all apps to run interpreted or other downloaded code makes it impossible to screen them for malicious intent. So, it's not a double standard, it's the only way that they can provide any level of security, and it's a single well understood standard based on well understood reasons.

And what is the difference of running Facebook JS code in Safari and in UIwebview? Or any other web app, for that matter?

Blaze's conclusion was that the embedded browser in Android was faster than the embedded browser in iOS. That conclusion holds up to scrutiny.

Perhaps it escapes AppleInsider, but in the real world of computer performance testing, a fair set of benchmarks compares platforms as equally across the table as possible. It might make AI happy to see a set of unequal comparisons, but that would hardly be considered fair outside the confines of fanboidom.

That is what happens when you make an assumption without doing your due diligence to check. The onus is on the tester to make sure they understand all of the variables of their test. Before they release the results as fact.

Quote:

Originally Posted by Wiggin

But until the story was reported after 4.3 was released, how many people had the reasonable expectation that the embedded browser used the same engine as Safari? That's the way it's been since the beginning of Safari on both the Mac and iOS, right? Other applications used the same WebKit code that Safari used. Apple didn't tell anyone that in iOS 4.3 it was any different. So how was Blaze supposed to know? They've even put a disclaimer at the very top of their original story. So please, explain to me where the "deception" is.

Wow...
Talk about an organization trying to establish itself as a reliable source of testing, Blaze has sure shot itself in the foot.
They're particularly getting hammered in their own comments section under the article for a clearly flawed and misleading 'study'.

For one thing, their headline is completely dishonest. Secondly, they made a bunch of assumptions that were stupid to make, so that's no defense for their dishonest lead.

"Dishonest" and "deceptive" means that they knew and were intentionally misleading in their report. Everytime anyone says anything non-positive, and even sometimes when they say something positive but use a word like "overkill", you instantly assume that they are out to get Apple, that they are Apple haters and have a chip on their shoulder. It's a very paranoid world you live in.

Did they make a mistake? Yes. Did they admit that their test didn't test Safari? Yes. And they were open about it by putting the disclaimer on their orignal story. Let's see AI be as forthcoming about the deceptive writing by DED (deceptive or delusional, take your pick). As of the time I'm writing this, DED still has not acknowledged that Blaze discovered the error in their testing and has added a disclaimer on their story. And he probably never will. He doesn't even included a link to Blaze's orignial article. Why? Oh right, because then people would also see the disclaimer.

Blaze's conclusion was that the embedded browser in Android was faster than the embedded browser in iOS. That conclusion holds up to scrutiny.

Perhaps it escapes AppleInsider, but in the real world of computer performance testing, a fair set of benchmarks compares platforms as equally across the table as possible. It might make AI happy to see a set of unequal comparisons, but that would hardly be considered fair outside the confines of fanboidom.

No, if you want to compare embedded apps against each other, that's fine. Perhaps even of interest.
But they are clearly (let's all say the word together) LYING when they say that their study analyzes BROWSER speed. That would be Safari against whatever browser is built into Android.
Words have meaning.

Blaze's conclusion was that the embedded browser in Android was faster than the embedded browser in iOS. That conclusion holds up to scrutiny.

Perhaps it escapes AppleInsider, but in the real world of computer performance testing, a fair set of benchmarks compares platforms as equally across the table as possible. It might make AI happy to see a set of unequal comparisons, but that would hardly be considered fair outside the confines of fanboidom.

You do realize this "real world of computer performance testing" is different than the "real world," right?

Statistics and the benchmarks generated by performance testing are meaningless without correct analysis and interpretation.

As stated, the article said "iPhone v. Android: 45000 tests prove which browser is faster."
Not which "embedded" browser is faster. And they certainly didn't put it in real world terms, eg, this affects third party browsers, not the safari browser (which is the browser used by the vast majority of users).

Their poor interpretation and presentation did a disservice to the real and useful statistics that they produced.