The next revision of the iPhone OS will remove the rating prompt that users see when they delete an app from their device.

The finding, noted by Arnold Kim of MacRumors, erases a behavior first implemented in iPhone 2.2, shortly after Apple opened the App Store.

The initial intent of the prompt, which asked users to assign a one to five star rating for the app, was to increase the number of feedback ratings in the App Store. Users could also decline to rate the app (as shown below on iPad).

However, developers complained that the prompt negatively skewed their ratings, as it added more low ratings from users who didn't like the app, but didn't do anything to encourage high approval ratings from those who kept the app and used it frequently.

Keeping on top of App Store reviews

A year ago, Apple modified its ratings policies in iTunes to prevent users from entering ratings for apps unless they'd actually downloaded them.

The company subsequently removed large numbers of ratings from people who had never even tried the apps they had rated. Removed ratings were often either from users protesting over pricing or, occasionally, from developers padding their reviews with positive rankings from dummy accounts.

The company followed up a few weeks later by tying reviews to the revision of the app, a feature which allowed users to filter out old complaints related to problems that had since been fixed.

The current way of doing things ABSOLUTELY skews ratings downward into hell, because why else would you be deleting an app unless you didn't like it and didn't want to use it anymore?

The very first time it prompted me to assign a star rating to a program that I had just deleted because I didn't like it, I thought, "Uh-oh. This can't be good for developers' ratings." And then I proceeded to give the app a 1-star rating because it was junk.

Yet the programs that I love, and which have been living happily on my iPhone for months or years, have never prompted me for ratings, which would all get 4 or 5 star ratings from me.

Although I have taken the extra effort of going back to the app store and visiting the apps themselves to give them positive ratings (and reviews).

For some reason, I wasn't even sure the rating was going to iTunes Store, the little box doesn't say so directly. For a little while, I thought it goes back to my iTunes app like how songs are rated for local use. I use those ratings as one of my smart playlist conditionals, which in turn determines how often songs get synced to my phone. That aside, there is no local equivalent ratings, so I had to assume it was being sent.

I think the general adage is that people that had a bad experience tend to put more work into telling others about their experience as it is, which tends to skew the perception of the product. Making it that easy to post a negative probably doubly compounds the problem.

All arguments about statistical accuracy aside, I think this is a bad move.

The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?

Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.

A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.

The company followed up a few weeks later by tying reviews to the revision of the app, a feature which allowed users to filter out old complaints related to problems that had since been fixed.

I'm always going to try to take credit for this change as I emailed Apple suggesting the idea a while before they did it. I don't really care if it was me they listened to or they had the idea independently but it makes me feel better

As for this new change, I quite like it... not so much because the 'rate on delete' system skewed results, but because it would often leave apps with a number of ratings, but few actual reviews stating why the app was so rated. Hopefully this means that the people who are reviewing the apps in the App Store now care enough to tell us why.

I didn't have a problem with the prompt, b/c if I was deleting the app, it's b/c I didn't find it valuable. Hence the (appropriate) lower rating.

And I agree with Apple to make feedback easy. It's obviously critical to App Store, for users to sift through the apps and for developers to keep quality high. It just needs to be both easy and allow for balanced ratings. Maybe an auto-prompt after a week, not just one after deletion?

Personally, there are some apps I just love and use religiously, have yet to give feedback. This is (very) poor on my part, but for some reason there is an energy barrier to logging into iTunes, finding the app, and providing positive comments. Hopefully Apple can implement something that facilitates this. My lazy a$$ would be happy.

Apple should just track total number of hours that apps are in use on everyone's systems and sync that back to the store. Apps used for more hours per day are likely to be better...

Some apps only need to be used for short time but are still good, so this system wouldn't work either. Like asapHockey, i use it JUST to track twitter of the local AHL hockey team around me, and sometimes i can read through those in like 2 minutes, but the app stays on my system b/c it does the job. I use other apps to get the rest of my hockey news.

All arguments about statistical accuracy aside, I think this is a bad move.

The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?

Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.

A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.

diabolical situation you have created
your king may have to settle all this

It seems fine that the system didn't capture positive ratings. It still resulted in lower ratings for apps that weren't as good. Who cares what the average is for all apps as long as the better ones are ranked higher?

The company followed up a few weeks later by tying reviews to the revision of the app, a feature which allowed users to filter out old complaints related to problems that had since been fixed

It's better, but could still be improved. When I post a new release itunes defaults to showing the "most helpful" reviews, since the current version of the app has no reviews. The issue the "most helpful" reviews were from version 1.0 which is almost two years old now and has no baring on the current version of the app. Apple won't remove these ancient reviews, or use any sort of aging so that a helpful review from 2 years ago isn't ranked as as-helpful now.

The other major issue with the review system is that developers still can't respond to customers reviews. Privately would be better than letting devs respond in the review threads so it doesn't turn into a post a review to get support, but letting us privately respond would let us address customers issues or seek clarification on the issue they are having.

It seems fine that the system didn't capture positive ratings. It still resulted in lower ratings for apps that weren't as good. Who cares what the average is for all apps as long as the better ones are ranked higher?

Because apps (as well as many other things) should be judged on both a relative and absolute scale. If best app in a particular category was ranked highest by rating, but only rated out at 3 stars, I'd be slightly hesitant to buy.

All arguments about statistical accuracy aside, I think this is a bad move.

The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?

Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.

A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.

Paraphrasing Henrik Ibsen (An Enemy of the People)::

What is true may not be popular.
What is popular may not be true.

As noted by others, we tend to voice our opinion more so when we have a negative experience or opposing perspective. It is not unusual too that there are those who solicit their friends to get good reviews. This fraud has been documented in Amazon book reviews.

One should trust reviews based mainly from reliable sources, those with credibility to weigh in on the matter.

While the current system probably does skew ratings downward, the new system will likewise skew them upward, which is a bit of a disadvantage for apps that have been out for a while (skewed downward) vs. newer apps (skewed upward). However, it's a very difficult problem to solve. Weighting ratings by absolute numbers might help -- for example, a "3-star" app with 1,000 ratings would rank higher than a 3-star app with 10 ratings. Weighting them by how much time they spend on a users iPhone might also help, but raises privacy issues, and tracking usage (which is happening all the time now) is not appropriate. (I think this should have to be disclosed in the description of each app, if it's tracked, how, and what information is used/taken).

But, in the long-run, this is probably the best way to do this, putting all ratings on an equal footing so that everyone who rated did so through the same mechanism.

The biggest problem with the ratings, from a user perspective, is that you can't sort/search the app store based on them.

The current way of doing things ABSOLUTELY skews ratings downward into hell, because why else would you be deleting an app unless you didn't like it and didn't want to use it anymore?

Or you wanted to download something else to try and want to remove some clutter. I did this all the time,especially when the number of screens was lower. didn't mean the app was junk, I just didn't need it all the time and I either had on my computer or could download it again so no worries.

Just because something isn't your favorite doesn't make it bad.

Also, if I use an app, play a game etc more than 3 times a week consistently I consider it my duty to review it. And if an app is bad, especially if it's also overpriced, I will review it for that also. And say what is bad. So not just 'this app sucks' but "the UI is clunky and cluttered making use slow"

Forcing prompts to rate an app you keep using, is still forcing a pop up right when you are trying to do something else. Minimizing that is a good idea.

Maybe Apple should make an App that is all about feedback, rating and sharing things you like. Nintendo has a decent one in their Nintendo Channel for the Wii. It's novel to go and just check out how long you've spent time in any game, and then you're there, so why not rate it? That app could wait until you've used an App for an hour over at least 2 days or something to make sure you gave it a chance (of course you could still go to the AppStore and rate it)

The App could get away with asking some info in the rating (age, interests, what the app is best for) and then make a local profile compared to a generic profile built around the app. It could then make suggestions of other Apps. This also gives a single place for a user to decide whether they want to share info like that or not, just for the purposes of rating.

Sure, only enthusiasts would spend a lot of time there, but I think on fun experience with it and one good suggestion as a reward would bring people back again and again. I know I'd rather look through the Apps I own to rate them rather than try to remember the name, search for it, and then go through the rating process.

Forcing prompts to rate an app you keep using, is still forcing a pop up right when you are trying to do something else. Minimizing that is a good idea.

What might have been a good idea would have been to keep the rate on delete feature and to use the iAd framework to occasionally display a "rate this app" "ad" for apps in use which the user has not rated.

What might have been a good idea would have been to keep the rate on delete feature and to use the iAd framework to occasionally display a "rate this app" "ad" for apps in use which the user has not rated.

Interesting. That sounds like a good idea, but then paid apps don't get rated unless the buyer chooses to use iAd, which may update buyers who paid money to not be bothered.

Perhaps Apple needs to also use an option that is invisible to the user that anonymously tecords the number of times an app is opened and used so that global stats can be listed on the App Store page.

Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"

Actually, the statistical effect developers have complained about is called a positive skew, not a negative skew, as referenced in the article. I am surprised that Apple didn't consider this before implementing the rating system. Goes to show how easy it is for any company to miss the effects of survey techniques!

Interesting. That sounds like a good idea, but then paid apps don't get rated unless the buyer chooses to use iAd, which may update buyers who paid money to not be bothered.

Perhaps Apple needs to also use an option that is invisible to the user that anonymously tecords the number of times an app is opened and used so that global stats can be listed on the App Store page.

Well, I was thinking a specific method that could be used by the developer to present a rate this app banner at the developers discretion, without having to actually always have ads in the app. Wouldn't have to be part of iAds but, but a similar mechanism, I guess.

I don't think recording app usage stats, even "anonymously" is appropriate, even though many developers are doing just that now. (Well, I'm not sure how "anonymously they are doing it.)

I don't think recording app usage stats, even "anonymously" is appropriate, even though many developers are doing just that now. (Well, I'm not sure how "anonymously they are doing it.)

How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.

PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used. I have apps that I've never used but still get updates for on a regular basis. This causes extra data to DLed for Apple, the ISPs and me, yet I should just be deleting a 2year old app I don't use, but "out of sight, out of mind" means that I don't even bother hunting these down and removing them. I think Windows 95's Program Manager has the right idea.

Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"

As a developer I appreciate this move. From my experience, it hurts free apps harder, because people often download them just to take a look and then may delete them, not necessarily because they are bad, but because they don't need them.

Back in December my son released an app which was calculating the recommended sun exposure time based on actual UV index, which apparently is useful mainly during the summer. He hoped to attract some Australian users but this didn't happen. The majority of the downloads were from the Northern hemisphere. After appearing in the "What's new" category the app was getting a fare number of downloads, and the negative ratings grew first. There were almost no comments to help to identify what customers don't like. With later versions he added an option to send feedback from within the app and got pretty positive feedback from there.

The initial version was free with an in-app purchase to enable the full functionality. After a couple of weeks the full functionality was unlocked and the app was made a paid download, and a separate lite version was released. The lite version followed the same pattern as the first version. Interestingly, the ratings for the paid version were much more generous.

My interpretation is that those who purchased the full version have read the description and made a decision whether they need the app or not, while many downloaded the free version just to take a look and quickly deleted it. The number of the updates suggests that more than 2/3 deleted the lite version.

PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used

You can't just ask customers what they want and then try to give that to them.

It was an obvious terrible idea on paper. Somehow it made it into the OS, where it proved how bad an idea it is. And somehow it lingered through multiple updates anyway. Thank goodness it will be gonebut the truly amazing story is that for now its still in 3.2 on iPad! Bizarre.

Its just that it encourages ratings just from people who dont like an app. Its that it encourages ratings from people who dont even CARE about the app and simply dont have a use for it! If they download a great app (especially a free one) but its not the KIND of app they were after, it gets one star! Want a $1 diving tutor app, but you got a $1 driving tutor instead? Doesnt matter how good that app is, youre into diving, not driving. One star. Now multiply this effect by the number of people who randomly grab anything thats free. Ugh.

We all want a simple answer. Is this app good at what it claims to do? What Apple really needs to do is find a way so that a customer can download any app they want to use without the penalty of paying for it first. Apple would give the user 24hrs to evaluate the app. At the end of the 24hrs, a popup would appear asking them if they want to purchase the app (if its a paid app)? At that time, the user could purchase it or delete it. Let the user decide if an app is good for their purpose, not some stranger.

If the app is free, they can just delete it. No harm, no foul.

I don't want to hi-jack the thread into another subject, but this is going to be even more relevant when iAD ads start. We may see many apps with two versions so that a user has a chance to evaluate an app without paying for it. The user may use the iAD version as a test version to see if they like the app but hate ads. They may delete the iAD version and purchase the paid version. The app store could grow bigger and bigger with this type of logic. It want help the user to see 300,000 apps available when so many have duplicate function because of iAd.

I just think relying on a raw rating system is a flawed system. Let me be the judge of the usefulness of the app. I can tell pretty quickly if its what I want.