User reaction to a lack of ratings and reviews

14 Jul 2009 - 10:32am

Last reply:
5 years ago

10 replies

955 reads

Michael Etgen

2008

Does anyone know of any academic or industry research that addresses
how users react-to/perceive products on a site when there is a
function displayed for ratings/reviews, but none (or maybe just 1)
have been completed?

I've looked through a number of papers/articles and have yet to see
that particular issue addressed.

Comments

15 Jul 2009 - 6:08am

jasonrobb

2009

Micheal,

That's a good question. As far as solid research goes, I haven't
come across it, though I'll keep my eyes open.

Wearing my user hat, if I saw 0 stars/0 reviews everywhere, I would
probably get the impression that this site doesn't have a
community.

One possible solution could be to only show the stars or reviews when
you have 1 or more. As in, don't even show the header "Reviews"
unless you actually have some. That way you can provide value when
you have it, and avoid giving the impression that nobody is home when
you don't have ratings/reviews.

Another option is to have somebody go through and review each item
once. I know it seems disingenuous, but frankly it's the solution most
often used. Hiding the ratings until you have one results in UI
inconsistency. The problem you're having has to do with user
participation -- sometimes you just need to seed the site and userbase.

adrian

> One possible solution could be to only show the stars or reviews when> you have 1 or more. As in, don't even show the header "Reviews"> unless you actually have some. That way you can provide value when> you have it, and avoid giving the impression that nobody is home when> you don't have ratings/reviews.>

15 Jul 2009 - 8:08am

Joshua Porter

2007

As Adrian mentions, seeding is a bit unethical, as the reviews aren't
authentic. (keeping reviews authentic is an ongoing problem all the
time, however)

Another idea is to reward first reviewing behavior.

So, give people a special badge on their profile when they review an
item first.

1) It gets the item in the database (hugely important for user-
generated content sites like Yelp)
2) It gets a review active for that item (thus getting over the hurdle
in question)
3) It rewards users in a unique way (people recognize this distinction
as different than a simple review).

This method is a win-win for both the site and users.

Josh

On Jul 15, 2009, at 9:48 AM, adrian chan wrote:

> Another option is to have somebody go through and review each item> once. I know it seems disingenuous, but frankly it's the solution> most often used. Hiding the ratings until you have one results in UI> inconsistency. The problem you're having has to do with user> participation -- sometimes you just need to seed the site and> userbase.>> adrian>>> One possible solution could be to only show the stars or reviews when>> you have 1 or more. As in, don't even show the header "Reviews">> unless you actually have some. That way you can provide value when>> you have it, and avoid giving the impression that nobody is home when>> you don't have ratings/reviews.>>> ________________________________________________________________> Welcome to the Interaction Design Association (IxDA)!> To post to this list ....... discuss at ixda.org> Unsubscribe ................ http://www.ixda.org/unsubscribe> List Guidelines ............ http://www.ixda.org/guidelines> List Help .................. http://www.ixda.org/help

15 Jul 2009 - 7:15am

C K Vijay Bhaskar

2009

As a user, I typically dont go and start rating - depending on my
mood, time available, usefulness of the content etc on product pages.

Also I have seen many users who do a "random" rating just to
complete a process without giving away the actual feel/usefulness of
the product.
The other way I can think of is to display how many people have
visited the page and the time that they have spent on the page. Even
though they may not have rated the product, this measure could be a
good indicator on how many people visit and time that they may have
spent on the page.
This data can be tinkered to assess the content etc and decide if the
product page, its description etc was in fact readable or useful.
Hope this helps.

Being consistent for the sake of consistency isn't a good reason to
be consistent. Hiding the ratings in this case for unrated shoes
reduces the noise on the page. This is helpful, and well done, I
think.

Not sure I like the zappos approach. Visually, i would prefer to have
the ratings even if they're "blank." For consistency's sake ;-) (If
consistency isn't desired for its own sake, then for what else?) Now
that's just a visual argument. In terms of what it indicates, the lack
of ratings to me indicates the same thing as an unrated rating. So I
dont see how they've solved that in any way: no ratings here, next to
all the other shoes that do have ratings, just says "no rating" ... Or
so it seems to me. To me absence of the ratings can be noise -- when
something seems to be missing, isnt that the same kind of noise as
something that's not yet filled out?

We're splitting hairs, but that's what we do well.

On a side note, this opens a back door to social interaction design
and social usability matters: A ratings system has two social
functions: to encourage the act of rating by user; to display average
ratings. Interestingly, my suggestion favors the former; yours I think
favors the latter. My suggestion is to leave ratings in there -- we
want user to rate -- and if needs be then have one user rate just to
seed the activity. Your suggestion is to remove it because it doesnt
show anything, which is totally valid and true.

How would we design a principle here? If the input element also
provides a social connotation, which function prevails? The call to
action or the display of data?

Personally this is why I think a lot of social design elements
introduce social bias and distortion : input mechanisms are the
display mechanisms also. But that's another topic....

thoughts?

adrian

>> With regards to hiding ratings:>> Zappos, for instance, hides the ratings on the search results page,> until the shoe has been rated.>>http://www.quicksnapper.com/files/1946/5248341084A5DFBFF1CD89_m.png>> Being consistent for the sake of consistency isn't a good reason to> be consistent. Hiding the ratings in this case for unrated shoes> reduces the noise on the page. This is helpful, and well done, I> think.>> Cheers,>> Jason R.

15 Jul 2009 - 2:52pm

Diana Wynne

2008

I'd rather see a blank than read a review that was only written to solve a
UI problem.

Yelp's "firsts" do reward a certain kind of competitive behavior. It doesn't
mean those people's opinions are particularly reliable.

> Jason,>> Not sure I like the zappos approach. Visually, i would prefer to have the> ratings even if they're "blank." For consistency's sake ;-) (If> consistency isn't desired for its own sake, then for what else?) Now that's> just a visual argument. In terms of what it indicates, the lack of ratings> to me indicates the same thing as an unrated rating. So I dont see how> they've solved that in any way: no ratings here, next to all the other shoes> that do have ratings, just says "no rating" ... Or so it seems to me. To me> absence of the ratings can be noise -- when something seems to be missing,> isnt that the same kind of noise as something that's not yet filled out?>> We're splitting hairs, but that's what we do well.>> On a side note, this opens a back door to social interaction design and> social usability matters: A ratings system has two social functions: to> encourage the act of rating by user; to display average ratings.> Interestingly, my suggestion favors the former; yours I think favors the> latter. My suggestion is to leave ratings in there -- we want user to rate> -- and if needs be then have one user rate just to seed the activity. Your> suggestion is to remove it because it doesnt show anything, which is totally> valid and true.>> How would we design a principle here? If the input element also provides a> social connotation, which function prevails? The call to action or the> display of data?>> Personally this is why I think a lot of social design elements introduce> social bias and distortion : input mechanisms are the display mechanisms> also. But that's another topic....>> thoughts?>> adrian>>>> With regards to hiding ratings:>>>> Zappos, for instance, hides the ratings on the search results page,>> until the shoe has been rated.>>>>http://www.quicksnapper.com/files/1946/5248341084A5DFBFF1CD89_m.png>>>> Being consistent for the sake of consistency isn't a good reason to>> be consistent. Hiding the ratings in this case for unrated shoes>> reduces the noise on the page. This is helpful, and well done, I>> think.>>>> Cheers,>>>> Jason R.>>>> ________________________________________________________________> Welcome to the Interaction Design Association (IxDA)!> To post to this list ....... discuss at ixda.org> Unsubscribe ................ http://www.ixda.org/unsubscribe> List Guidelines ............ http://www.ixda.org/guidelines> List Help .................. http://www.ixda.org/help>

15 Jul 2009 - 1:48pm

Michael Etgen

2008

Well, the particular angle I'm thinking about here is more of the
shopper evaluating products versus a customer coming back to
rate/review.

Are there particular behavior patterns we see when a shopper finds a
product with no reviews/ratings that "looks good"?

If the same product has ample ratings/reviews elsewhere but not here,
what is the user's perception (this site stinks or is untrustworthy?)
and how does it affect their behavior? If the same product has few or
no ratings/reviews at all anywhere, then what's my
perception/behavior?

You're right that individual people's opinions aren't particularly
reliable, but the system tends to self correct.

In our work, we've noticed that when people see a review they don't
agree with, they are more motivated to review than if they see reviews
they do agree with.

So, if the first reviewers skew away from the average beliefs, it will
eventually populate in the direction that matches the average.

Jared

16 Jul 2009 - 12:41pm

Jared M. Spool

2003

On Jul 14, 2009, at 8:32 AM, Michael Etgen wrote:

> Does anyone know of any academic or industry research that addresses> how users react-to/perceive products on a site when there is a> function displayed for ratings/reviews, but none (or maybe just 1)> have been completed?

We've been talking about it. I mentioned it in this presentation: http://is.gd/1wndV
(The specifics you're interested start about slide 26.)

Basically, we've found there's a negative reaction that users have
when the review or rating system isn't populated or is only filled
with negative reviews or ratings.

Our current thinking is that you only want to have a review/rating
system if you are sure folks will participate.

When we studied Amazon's reviews, we estimated that for the average
product, it takes 1300 purchases to generate a review. If you want 20
reviews for your most popular products and you run at an average
conversion rate of 2%, we estimate you need 1,300,000 visitors to the
product to succeed.

On low volume sites, you have to resort to other strategies to
encourage reviewing & rating.