I'm director of content strategy at startup Exitround. I was previously a staff reporter at Forbes covering start-ups and venture capital. I'm interested in entrepreneurs who want to change the world, or have a point of view or compelling story. Email me at tgeron.news@gmail.com. I was previously a reporter for Dow Jones VentureWire where my work also appeared in the Wall Street Journal. I've also written for Red Herring, the Long Beach Press-Telegram and other outlets. In a former life I was a web developer. Follow me on Twitter tomiogeron, or Facebook , or Google+.

Do iOS Apps Crash More Than Android Apps? A Data Dive

It turns out there are many possible reasons. And it can vary particularly depending on whether you are using an Apple iOS device such as an iPhone or iPad, or an Android device.

One of the reasons for app crashes is the proliferation of mobile operating systems on iOS and Android. As Apple and Google have released more new operating systems, each with multiple updates, app developers face more operating systems to test apps on. In data that mobile app monitoring startup Crittercism compiled for app crashes between December 1 and 15, there were at least 23 different iOS operating systems on which apps had crashed and 33 Android operating systems on which apps had crashed. (See the graphs above.) Note that the graphs that separate out Android and iOS show these number of operating systems and the graph that combines both iOS and Android shows less–22 iOS and 17 Android.

The largest proportion of app crashes from both iOS and Android platforms were on iOS 5.01 with 28.64% of overall crashes (in a normalized data set). That makes sense since iOS 5 was still relatively new at that time and many apps still need to work out the kinks with the new OS. But there are also older iOS versions that have a significant proportion of app crashes. For example, iOS 4.2.10 had 12.64% of app crashes, iOS 4.3.3 had 10.66% and iOS 4.1 had 8.24%. One other point that this made clear to me is that many people apparently take their time updating their iPhone software or never update it at all.

The data raises two main questions for me: why do apps on these operating systems crash so much, and do iOS apps crash more than Android apps? On the first question of why apps crash, the reasons are many, says Crittercism CEO Andrew Levy. This can be due to hardware issues, such as the use of location or GPS services or cameras; it could be due to the Internet connection, that is, how a phone connects to 3G or WiFi, or that the device is not connected to the Internet at a certain moment, or that something happens during the switch between 3G and WiFi. There could also be issues with language support on certain devices. There can also be memory problems if an app uses too much memory.

Problems can also occur with third-party services that developers use in their apps, from analytics to advertising systems. For example, there were reports that Apple’s iAds system gave some developers problems if they did not adhere to certain standards. ”It can be a mix of both hardware and software issues that developers may or may not be responding to,” Levy says.

In addition, developers also constantly create new updates to their apps to create new features or fix bugs. But again, people often don’t update their apps–just as they don’t update their operating system. (Android, unlike iOS, allows users to auto-update their apps, which can eliminate some of the problems.) So developers often test all previous versions of their apps with each version of the different operating systems. ”The permutations go on forever,” Levy says. “That’s a large reason for creating our platform.” Particularly with a new OS, developers have to test their app to make sure it still works on the new OS. Often they will seek to test their apps in a test environment, but often that isn’t possible.

The Apple iOS operating system app crashes accounted for more of the app crashes in Crittercism’s data than did Android-based phones, as mentioned above. In the pie graph “Crashes by OS Version Normalized” you can see that iOS accounted for close to three-fourths of the app crashes, with Android making up the rest. But is that just because Crittercism has more iOS phones in its network? Crittercism parsed some data to answer that question.

Crittercism analyzed a total of more than 214 million app launches from November and December 2011 from apps that use its service (see graph at top of this article). There were about 3 times more app launches for iOS that Crittercism analyzed, about 162 million to 52 million. But the analysis examined app crashes as a percentage of each app launch, so this data takes out the issue of there being more iOS than Android apps. In other words for each iOS app and each Android app how often percentage-wise do they crash?

In the top quartile of apps, Android apps crashed 0.15% of the time they launched, while top quartile iOS apps crashed 0.51% of the time. In the second quartile of apps, Android apps crashed 0.73% of the time and iOS apps crashed 1.47% of their launches. In the third quartile of apps, Android apps crashed 2.97% of the time, while iOS apps crashed 3.66% of the time.

So what does all this data mean? On a basic level, you can see that iOS apps crashed more than Android apps during this time period. But Crittercism’s Levy cautions that this doesn’t necessarily mean that overall iOS apps crash more than Android apps. That’s because Apple had recently released a new version iOS 5 in October. Android’s new Ice Cream Sandwich operating system (Android 4.0), meanwhile, had not been widely released on phones yet at the time of this study. “I expect as Ice Cream Sandwich just launched and the new Nexus S phone launched (during the study), we’ll expect the same situation to occur (with Android) as what happened (with iOS),” Levy says.

Still, the data shows that apps on iOS did crash substantially more than Android apps. Anecdotally, I know that certain apps I use on my iPhone crash and they crash often. Will that change as Ice Cream Sandwich rolls out and as more developers improve on iOS 5? We’ll see.

One other thing about the data: in the best apps, that is, the top quartile, the apps crashed much less than in the third quartile. Levy emphasized that that shows the difference that developers can make with their apps by analyzing the data and improving their apps. Splitting up the data by quartiles also removes apps with massive user bases that can skew the averages.

My own point: when you get to the top apps, Android’s lower app crash rate than iOS makes less of a difference because they are both well below 1%. However, there was a bigger difference between iOS and Android app crashes in the top quartile of apps than in the third quartile. In other words, the best apps in Android crashed about one third as many times as the best iOS apps, while the second best quartile Android apps crashed about half as much as comparable iOS apps, and in the quartile, the difference between the two operatings systems was even less. So the very top Android apps are achieving a crash rate that, at least in this time period, the best iOS apps can’t match. Why that is, I’m not entirely clear.

However, Android, it should be noted, allows developers to push updates faster than Apple. With Android developers can just send an update to its code, which can show up almost in real-time. But for iOS it can takes days or a week for an update to show up. That means there can be more app crashes while those updates are waiting to happen. Whereas with Android, presumably if developers know there’s a bug they can immediately fix it.

One final piece of interesting data from Crittercism: The performance of apps is not only different on various operating systems but also on different devices. About 74.41% of the iOS crashes Crittercism tracked were on the iPhone, 14.81% were on the iPod Touch, and 10.72% were on the iPad.

Crittercism, which is backed by Google Ventures, Kleiner Perkins Caufield & Byers, AngelPad, AOL Ventures, Opus Capital and Shasta Ventures, provides crash reporting to app developers. The company provides developers with a wide range of data besides the mobile device and operating system, including, for example, the length of time between when an app is loaded and when it crashes, or how a user is holding a phone–portrait or landscape–when it crashes. It can also help evaluate whether an app’s own code has caused it to crash or whether a third party service SDK being used is causing the problem. This kind of data is important for helping developers plug all the holes in the landscape of operating systems. Clients using Crittercism include: Aston Martin Explore, Bullet Time and Hipster.

Protecting apps from crashes is not only important for app developers, who by definition live and die by the ability of their apps to work smoothly. Increasingly, many types of companies rely on apps to run their businesses, even if there isn’t a large consumer user base using the apps. Banks have mobile apps that enable check deposits, sales people use apps in the field, and so on. “It’s about protecting your brand,” says Crittercism cofounder Rob Kwok. “More and more business-critical functions are moving to apps.”

Crittercism is signing up a number of large customers to its platform as a result. The company has already ramped way up from the 214 million app launches that it monitored in this data from November and December. If you’re hoping to end those annoying crashes on your phone or tablet, that should be welcome news.

Post Your Comment

Post Your Reply

Forbes writers have the ability to call out member comments they find particularly interesting. Called-out comments are highlighted across the Forbes network. You'll be notified if your comment is called out.

Comments

The title of this article should be: Do iOS Apps Crash more than Android Apps ? Not: Does iOS Crash More Than Android?….. One would expect better reporting from a publication like Forbes ??? The current title would suggest to the uninformed that, the Apple iOS is less stable than the Android OS.

Define stability. How often does an OS itself actually crash? Pretty much never on either of these platforms. Because of this, I’d say that stability SHOULD be measured by how well the apps behave on the OS, which would mean that this study confirms what you said it suggests – Android is more stable. I’m gonna stick with this definition unless you can think of a better way of defining what it means to be stable.

Long ago I saw stability defined by the program crashing less due to its own characteristics than by the operating system that supported it.

That standard may not do much for the horse-race issue that gets the eyeballs, but it DOES point out that many crashes are simply oversights by the developers, and are completely up to their correction. Since the users, types of apps and programming models are so totally different, I don’t know how this data could be very useful to anybody OTHER than the developers who need to track their own work.

Walt, you are right to a degree about the data’s usefulness, you just need to move up another level. This data is probably more useful for contract negotiations with Apple from the mobile providers. The devs probably already knew much of this information from customer feedback. If apps crash more on Apple, then it gives the mobile provider a black eye, so Apple is now worth less for the new contracts. Average users could also use the data as well for picking devices and long term device strategies, if they knowledgeable to know what they’re looking at.

Three things got called out though that were major contributing factors to iOS’s difficulties. A major OS update, the seemingly contradictory issue of customers not upgrading, and the Android Devs having the ability to discover and fix defects in mere hours instead of days with iOS apps. That summarizes the article right there for why Apple is having problems with stable apps and OS both.

Crittercism thought that Android 4′s general release would cause some issues, and maybe it will, but what didn’t get called out is that 4.0 has been in a open testing phase now for months giving devs time get their software fixed well prior to general release. I personally have been seeing a LOT of apps with updates for 4.0 stability in the last couple of months. When 4.0 goes into full release next week, most issues would have already been resolved. This is in stark contrast to Apple’s closed/secretive development cycles which can catch developers flat footed each time, particularly the smaller ones with less resources to respond in a hurry.

The app approval process is easier for Android. Developers issue more updates (with bug fixes) more quickly on the Android platform. In the end, this sort of data never matters to users. A user can’t distinguish these differences.

Perception is debatable…Companies/developers really care about it though– Each basis point can have an incredible economic impact, particularly for a 3rd quartile performing app vs a 1st quartile. The economics couldn’t be more front/center than in game and retail apps.