Reputation System Design Rationale

When we came up with the idea behind GameDev.net’s reputation system we decided to take a good hard look at some of the more modern technical resource sites out there and see what they were doing. Now we have been using reputation for years, but a number of sites have expanded well beyond what we were doing.

We focused on three major sites in particular as inspiration for our changes: MSDN Forums, Stackoverflow, and Codeproject. What we liked about the MSDN forums was how they subtly blended a forum-based mode of discussion with a question and answer format. Stackoverflow primarily revolves around a simple question and answer format. Now what you have to understand is that while Stackoverflow is a pure Q&A format and works well.. the pure Q&A format is something we will never switch entirely to because there is benefit in being able to have actual discussions. This is particularly true in situations where there may not be a correct answer to a topic post such as in any art or music-related forum. The bad thing about Stackoverflow Q&A is that it encourages quick answers over continued thoughtful discussion. However, the good thing though is that the end product has more long-term usefulness as an archive.

One of the things sites like Stackoverflow.com have heavily criticized is the cruft inherit in a forum system and how their approach heavily reduces the amount of extraneous “stuff” you see when viewing a particular question (e.g. avatars, member titles, …) We also like the idea of cutting down on non-essential things in topic views while still allowing members to retain their own sense of identity. What we also like about Stackoverflow is the wiki-style ability to revise posts and improve their quality. That’s something that we will be trying to do a lot more of as time goes on with our ability to now store post revisions.

In terms of recognition, we decided to go with a hybrid approach of having members both work to convince others to grant them upvotes as well as giving members the direct ability to gain reputation by participating in the site actively, helping in moderation tasks, and writing tutorials.

How does it work -or- How to be a productive member

We need you to do more than use the community as a resource for your own development. We need you to give back as much as you can by writing up tutorials (even small ones) and sharing your knowledge by answering questions from your fellow members.

Reputation

Reputation is a point value that represents how much the community trusts you and how much you give back to the community. You can accumulate reputation points through answering questions and creating articles/journals. What is cool about reputation is that the more reputation you earn, the more privileges you'll unlock on GameDev.net.

Originally you could only earn reputation by convincing others that you know what you are talking about, but it is far too easy to contribute and not receive proper recognition. We want that to change. Not only are we doing much more to encourage your fellow members to give you recognition for your contributions, but we are putting you in the driver’s seat with a slew of new ways to directly advance your rank within the community.

The new reputation system update is designed to completely turn GameDev.net member participation on it’s head. One of our stated goals for 2012 was to increase member recognition for their contributions and we’re going to change the playing field to put a much bigger spotlight on all those community members who work their butts off to help others.

As a member you’ll be able to earn points for basic site participation activities, for posting helpful replies in the forums, policing the community using moderator actions, and ESPECIALLY for authoring tutorials and helpful blog posts.

Points are split into 4 categories:

Scholar - Any up/down votes you receive from posts in the technical, business, or creative forums will be added to the scholar category.

Author - Points given to authors who write articles, get them peer reviewed, get articles favorited, etc. This is where people can pick up a lot of rep points if they get an article granted with a peer-reviewed status

Participation

These 4 categories will combine to create your overall reputation number on the site.Gamedev.net has employed some form of reputation/rating system for over 10 years now. Pre-2011 we had a lot of success with a very simple rating system that was modeled after the USCF chess ratings calculations (with some modifications). The idea was that a person with higher rating should be considered more experienced than someone with a lower rating. Granted, it takes time for a person to accumulate rating points but in the end if a person is actively participating on the site they should see a higher rating for themselves. The original system had one member directly rating the helpfulness of another member. What was different between our system and some of the others out there was that how much a member could contribute to another member was not only related to how many points the rating member had, but what the DIFFERENCE between ratings was. Basically, if I have a rating of 2000 I was able to have a more profound impact when I rate someone who has 1000 points versus when I rate someone at 1900 points.

In migrating to a different forums we adopted the reputation system built into that native forum. Under this system members gained reputation largely by up/down voting particular posts. This was actually one of the more frequent suggestions we received on how to tweak the old system to make it better. While the system itself is fine, one of the problems we often see is a failure to recognize those who take the time to help each other.

In one of our staff blogs we had posted our top goals for 2012:

GameDev.net will:

Provide an environment that makes it easy for members to publish information on game development

Encourage members to communicate and share information with each other and recognize those who take the time to contribute

Allow members to easily find information by providing a very focused number of topic areas

Allow trustworthy members beyond elected moderators to take ownership of information and participate in the moderation process

Allow employers \ team leaders to connect easier with potential employees or contractors

The second point is clear in that we really want to do a better job to recognize contributions of members on our site. And if you want to continue to see expert members contributing their knowledge it’s important that you take the time to recognize them as well.