Why Ratings Will Ruin Wikipedia

Did you know you can rate pages on Wikipedia now? It’s new. And it’s meant to get you involved, with the ultimate goal of increasing the site’s accuracy, diversity and completeness. It won’t work.

Sure. Wikipedia’s new rate this page feature will drive more engagement. The rating tool is sure to get people interacting with the site more. But more people using the site, on a more casual level, has quite a lot of potential to be a very bad thing.

Ratings would work great if people weren’t, well, people. That is to say, stupid, biased, and lazy.

Wikipedia needs that. It’s quit growing and lacks diversity. They’ve levelled off around 100,000 active editors. They assume (maybe wrongly) that they haven’t hit peak smart people, and so they want to bring in more. And moreover, they want them not to all be white dudes (four out of five Wikipedia editors are male. Four out of five are from the Global North.) This tool, it hopes, will do that.

Get more people involved, the thinking goes, and you’re bound to get more topics covered. Maybe so. But will it really make Wikipedia better? No. No it will not. Ratings are a terrible way to drive engagement.

First problem: Anyone can rate

I don’t have to know anything whatsoever about a subject to rate something. Despite knowing essentially nothing about, say, Californium, under the new system I’ll be able to rate that page. People love to rate things – even when those things are something they know nothing about. (See: app store ratings of apps by people who have not actually used the app.)

That would be totally surmountable, however, were it not for two other problems.

Second problem: People are biased jerks

The Rate this Page feature will be fantastic for most pages on topics that are mundane and non-controversial. It’ll make that pseudomorph page just sing, I’m sure. But it also seems ripe for abuse.

“Many people come to Wikipedia with an agenda,” says Joanna Pearlstein, a senior editor with WIRED magazine in charge of its fact-checking operation. She’s the one who makes sure that nobody who writes for WIRED uses Wikipedia as a source. “The examples of people mucking around with Wikipedia for their own personal purposes is legion. A few years ago, someone with an IP address out of the New York Times offices edited Condaleeza Rice’s page to say she was a concert penis, rather than a concert pianist.”

People are going to give subjects they don’t agree with shitty ratings just to express disapproval, even when the stories are accurate. Which brings me to problem number three.

Third problem: It’s too easy to rate

Wikipedia’s entire premise is that anyone can edit. That’s messy, but it largely works. It works because there is a barrier to entry. You have to go to the trouble to actually click that edit link and change some text. You have to be motivated. People deface it constantly, but it’s easy to revert those edits.

But ratings are something you can basically do on a drive-by basis. Click. Finished. Click. Finished. And there’s no reverting somebody’s opinion.

Not only will this make it easy to axe-grind, but it’s not clear that (especially on controversial subjects) that it will make them any more accurate.

Disagree with something? Just give it a shitty rating. Feel like the evolution article should talk more about God’s role? Slap it with a one-star rating for incompleteness. And then move on!

Imagine people doing this en masse. Imagine if, instead of Steven Colbert sending legions of viewers to Wikipedia to create their own realities, a popular media figure with an agenda – a Glenn Beck or Ed Schultz for example – sends his or her audience to Wikipedia to drive by downvote politically inconvenient articles.

Which matters because other people will come along behind them and see those poor ratings, and assume that stories that may very well be accurate and complete, are not, and vice versa.

If you want to get people engaged, ask them a question. As for their thoughts and their words and their expertise in a direct fashion. If you want to get the lowest common denominator, ask them for stars.

Share

Tags

Discuss

Totally agree with everything, specially the 'People are biased jerks" bit. I just hope 'Wikipedia' realises this error in judgement and goes back to the original system, which is itself flawed but manageable! If it aint broke don't fix it!

Your article strikes me as paradoxical. "Anyone can rate" is exactly the argument that wikipedia opponents use against the online encyclopedia itsself "anyone can edit".
The fact that only a small amount of raters reflects just the fact that wikipedia has a relatively small proportion of editors compared to readers. If a rating process engages those passive consumers in contributing with their opinion, that's a good thing isn't it.
BTW: The lazy biased jerks have created the biggest encyclopedia in human history :-) Please see the comparative study on quality Britannica vs Wikipedia http://en.wikipedia.org/wiki/Reliability_of_Wikipedia#Assessments