Comment voting working, getting expanded

46,000 votes in 6 days, now with more at stake.

It has been a week since we introduced comment voting on Ars, and the consensus among the staff, readers on Twitter, and a handful of people who have reached out to me by e-mail is that it's working. And by "working," I mean "improving discussions on the site." In the time since we rolled out the feature, there have been nearly 46,000 votes cast.

Yes, there have been several instances where we can clearly see someone getting downvoted merely because their innocuous opinion was expressed poorly, or in the wrong context. But what we are generally seeing is that comments are being fairly rated. We've also noted that sincere criticism of our coverage has been preserved, and in many cases voted up vigorously. In short, I do not feel at this time that we run any risk of an "echo chamber" effect, so we're going to continue with our voting system. We're even expanding it.

Today we are introducing thresholds. The concept is simple. When a given post achieves a predetermined number of votes (either negative or positive), it will change the way it is displayed in the thread. Posts with high ratings will receive a green "Reader Fav" badge, to make them easy to spot—and in the future we may call them out in other ways. In the other direction, posts that receive enough negative votes will become faded, and once they cross a certain threshold, they will be collapsed. You will still be able to click to expand the comment.

The goal here is simple. Highlight the content the community finds most valuable, while reprimanding the content that it does not. After just a week of voting, it is very clear to me that readers writing the comments have all the power. It's rarely what one has to say that earns them positive or negative votes, but how they say it.

We are starting with thresholds of 20—both positive and negative—with posts fading at -15, but we may adjust this in the coming days.

318 Reader Comments

Great work Ars staff. I suspect I speak for many when I say that I appreciate the value you're helping us get out of the (many) comments made on popular stories. It's not always possible to locate the gems on pages 7-13...

I've often found a significant dichotomy between the quality of posts on articles as opposed to the content in the "regular forums", as it were. Sounds like this system will help close that gap which is a really good thing, of course.

I've often found a significant dichotomy between the quality of posts on articles as opposed to the content in the "regular forums", as it were. Sounds like this system will help close that gap which is a really good thing, of course.

Absolutely, which is why we treat them rather differently. If you're a front page commenter and you haven't come into the forums you should give it a try, there's a lot more depth there.

That said, we get some really great comments on the front page, it's certainly not all bad by any means. We're just working on the signal/noise ratio a bit. And we'll keep working on it!

I hope this doesn't lead to youtube-style panderings for votes via "up-vote this comment!" or just folks flooding the article with worthless comments trying to be funny for the sake of votes.

edit:

side note, I've found the article-writer's flagging of relevant comments to be quite informative and good. Likewise, I've seen folks down-voting valid comments in a discussion just to weed out dissent, or ignore a valid point that can't get rebutted. The more "social" you make the commenting system, the more good or bad it can become. I have faith in Ars and it's writers/readers, so hopefully this won't turn into a yahoo-style idiocy.

I'm not sure if this is a bug or if it's the way it was intended but it looks like only the first comment to hit +20 gets the "Reader Fav" badge.

Do you have an example where this is happening? Part of the problem is that posts that were over +20 before turning on this feature won't be flagged as a Reader Fav. Once they get another +1 vote the badge should kick in.

In my opinion, posts should never dip below zero, only move above. You can still store the relative up and down votes, but display any negative number as a zero. This helps avoid "unpopular" ideas from being erroneously suppressed but still allows good comments to move upwards.

I'm not sure if this is a bug or if it's the way it was intended but it looks like only the first comment to hit +20 gets the "Reader Fav" badge.

Do you have an example where this is happening? Part of the problem is that posts that were over +20 before turning on this feature won't be flagged as a Reader Fav. Once they get another +1 vote the badge should kick in.

edit: Ah yes, you are correct, if I up/down vote a comment that's a +20 and hit refresh, the badge shows up.

I like the voting. However, I would appreciate a mechanism to prevent it from turning into making the wittiest comments rising to the top (a la reddit, which works there but not here, imho), and favor informed and educated comments being seen first.

I don't know how this would be possible, and I doubt anyone at Ars does, either, but those are my thoughts.

Slashdot-esque slider bars for user-customizable thresholds might be nice instead of these hard-coded threshold values. I've always enjoyed this on Slashdot because I'm only usually there to read particularly insightful or funny comments.

Voting never improves commenting, it only leads to people telling jokes for upvotes such as reddit. I would have rather you learned from the NY times and tabbed the comment interface. Have a tab which shows the comments untouched and another tab where the author highlights post that provide insightful questions for the article.

I like the voting. However, I would appreciate a mechanism to prevent it from turning into making the wittiest comments rising to the top (a la reddit, which works there but not here, imho), and favor informed and educated comments being seen first.

I don't know how this would be possible, and I doubt anyone at Ars does, either, but those are my thoughts.

Are any of the posts actually moving? It would break the flow of the conversation.

It's rarely what one has to say that earns them positive or negative votes, but how they say it.

If it is true that people will tend to get down-voted for poor expression rather than unpopular opinion, I support the change. A little more decorum would go a long way to improve some discussions around here, and make much it easier to respond to trolls without replying to them.

Maybe instead of using absolute values to determine the promotion/demotion threshold it should be a proportion of the overall number of comments? A bell curve of some kind...

What I'm getting at is that the more popular articles will produce more overall votes and crossing the thresholds in either direction will be more easy for comments in those article but at the same time it's going to dilute the value of the badge if too many have it... just a thought!

Please hide vote counts. The quantitative aspect affects voting patterns and reading patterns which are not necessary to achieve the results of getting comments fairly rated. Let individual users see how their personal posts get voted though. It diminishes "I'm voting up this comment because I agree with all these other people who have up voted" vs "I personally think this is a worthwhile post".

The secondary effect is that not knowing vote count will deter from simply voting to cross thresholds.

Finally, I'm disappointed with your trial so far. You described it as an experiment with the look to try different implementations. This hasn't been the case. You've introduced one design and seem to be moving forward missing out on an opportunity before users set in to try different implementations.

I'm sure Ars already thought of this but people could just make new accounts when they want to troll or express an opinion that will not be popular instead of risking the reputation of their account. Do Ars have an rule against multiple accounts ?

Well, my first experience with comments wasn't a positive one. I expressed my experience with Windows 8, from a developers perspective in what I thought was a reasonable, respectful and polite manner.

Yet, only a couple of hours after posting and I'm sitting at "-2" for what I can only think of was because I conveyed a negative experience of Microsoft's direction. I may as well have simpley said "Micro$oft sux man".

There's nothing to encourage a positive rating for a well-written and thoughtful comment or opinion if someone disagrees and decides to down-vote you on that basis alone.

I almost feel that there should be a disclaimer "For entertainment purposes only." on the comments section.

Voting never improves commenting, it only leads to people telling jokes for upvotes such as reddit.

Did you hear the joke about the guy who left a forum because he made an uninformed assumption instead of politely requesting a feature that might already exist?

saturnblackhole wrote:

I would have rather you learned from the NY times and tabbed the comment interface. Have a tab which shows the comments untouched and another tab where the author highlights post that provide insightful questions for the article.

You can see the uncut edition by going to the forums and clicking on the news subforum. If you subscribe, you can make that your default feature as well.

I'm sure Ars already thought of this but people could just make new accounts when they want to troll or express an opinion that will not be popular instead of risking the reputation of their account. Do Ars have an rule against multiple accounts ?

Yes, it is explicitly prohibited in the TOS. If you can't stand by your post under your forum name, you probably shouldn't post it.

Slashdot-esque slider bars for user-customizable thresholds might be nice instead of these hard-coded threshold values. I've always enjoyed this on Slashdot because I'm only usually there to read particularly insightful or funny comments.

We have this idea on a list of things to consider already, but thanks for bringing it up.

Well, my first experience with comments wasn't a positive one. I expressed my experience with Windows 8, from a developers perspective in what I thought was a reasonable, respectful and polite manner.

Yet, only a couple of hours after posting and I'm sitting at "-2" for what I can only think of was because I conveyed a negative experience of Microsoft's direction. I may as well have simpley said "Micro$oft sux man".

Honestly? Real life is ugly and not always fair...and forums often reflect it to a non-insignificant degree.

My advice...don't worry about it. Just keep making high quality posts and you'll win more often than you lose.

Ken Fisher / Ken is the founder & Editor-in-Chief of Ars Technica. A veteran of the IT industry and a scholar of antiquity, Ken studies the emergence of intellectual property regimes and their effects on culture and innovation.