I have noticed a couple of kinds of behavior that seem to crop up in the SO community, and it really seems to artificially boost reputation. There may not be a solution (or it may not be a problem!) but here are some thoughts.

The first is the fact that "celebrity" users, or users with lots of reputation, are treated significantly better than others. Take Joel Spolsky's most recent question:

It was closed, opened, closed, and re-opened. And the question has earned Joel - what - 1030 reputation 103 votes, and lots of reputation! Wow! I'm not saying that Joel's question is a bad one. But lets compare to other questions that one might consider to be in the same equivalence class - easily answerable, specific questions, short questions:

If you look at the hightest-voted questions, I can hardly find any which are specific technical questions. So that Joel's question was so upvoted could only mean that:

It was upvoted based on his celebrity

People have been interpreting the questions as a sort of zen-like wakeup call, designed to get people talking and thinking about they way they vote.

I do not mean any disrespect toward Joel, and again, I think his question was a good one. This post isn't about Joel! But the boost in reputation did not (imho) mean "Joel is really good at programming, and he has earned my trust as someone knowledgeable about computers." It may be true that Joel is a proficient computer expert, but not based on the merit of that one question.

Which leads me to an example about myself. I recently gave this answer (130 reputation) to a question about PHP's image-creating libraries. I am very appreciative of the community's support for my answer - an answer which I believe was well-thought and that I put effort into.

At the same time I can't look someone and tell them, with a straight face, that my reputation is a great reflection of my computer-related knowledge. Billions of people on the net have created tutorials for PHP's GD image library - I just happened to be the person to post a few of them. Heck if I know why it got 13 votes! (oh, but please don't downvote it!!)

SO is built by the community - so the community absolutely gets to decide what questions are important, and how they choose to dispense reputation. But in practice, I feel like peoples' reputation would be much lower if the community were more discriminating about the way they voted.

So: am I totally off base here? Is this even something that can be corrected with a different reputation algorithm? Or am I the only one who sees this as a detriment to the SO community?

(And, to be completely clear, I do not intend for this to be an ad hominem, and I am not jealous about not having a higher reputation. But I do take reputation with a grain of salt, in part for these reasons, and I thought I'd bring it up! It is meta, after all!)

The question you use as an example is also highly irregular, as it was specifically mentioned in a podcast multiple times, including it's ID number. Many people sought out the question for days following it actually being posted.
–
wompJul 6 '09 at 20:23

@womp interesting - I did not know that. It would be neat to casually mention some other obscure SO question to see if the same things happens!
–
user3788Jul 6 '09 at 20:48

12 Answers
12

This doesn't seem to be any different than real life. If Steve Jobs says he has an idea for a new application, many more people will "notice" what he has to say than if I come up with the next best iPod killer.

<sarcasm>

All you have to do is become one of the leading software-industry commentators since the earliest days of blogging, co-found a web site that attracts 30-40% of the world's programmers, start a podcast that 1000's of people download weekly, make an off-comment plug for a post you are going to make...

@DrJokepu: Fair question. I'm sure it keeps going up as an incentive for the top users of the site to keep playing.
–
Bill the LizardJun 29 '09 at 18:00

This is true, semantically. So it seems like trust manifests itself in 2 ways: "Trust that your answer is likely to be accurate" - which is a reflection of your computer knowledge. And "Trust to be a moderator," giving access to more moderation tools. But I feel like the intent of the system is that "people who are the most technically apt" are to be the most trusted in both respects - a good computer person, with good and accurate answers, can better tease apart good vs bad questions. No?
–
user3788Jun 29 '09 at 18:08

I've found that by answering tough or obscure questions, I'm lucky to get more than one vote, but by repeatedly answering the easy questions about IDisposable that are asked every day, I can make 220+ in a day pretty reliably - assuming I can find the time (and I get lucky with the timing of accepted questions).

Update: just answered "What's the value of yield return?" again - and have seen it asked and answered by others many, many times.

I think SO rep measures answers more than questions, because repeatedly asking easy questions appears to be quite hard to get away with - whenever someone starts asking 2 stupid questions per day, the community soon gets wise to this and downvotes them. I doubt if you can build up much rep by asking instead of answering.

So it's pretty clear to me that SO rep is nothing more than a measure of how a person likes to spend their spare time helping other people with programming problems.

Nothing to do with how smart they are, or how deep their knowledge goes... just where they focus their available time.

I agree this is something of an issue. Or at least, it is a very good reason to not simply judge people based on the number of their rep. It happens every week with whatever question Joel and Jeff discuss on the podcast. However, I really expect that's nothing that can be done, other than being a wise consumer of the data presented by SO. Your rep is a number, based losely on "how much the system trusts you" and somewhat on "how good a programmer you are". And even, somewhat on "how good a writer you are". The key is, if you're trying to evaluate a user, look at more than just their number, look at how they achieved it.

I have even noticed it with my own reputation, as my rep has gotten higher, I've found it easier to get reputation. Part of that is likely because I've gotten better at clearly stating my opinion/answer. However, I certainly recognize that part of it is that basic psychology plays into how people vote and therefore, they are more likely to vote for an answer by someone who has established reputation.

Of course guys like Jon Skeet attract more rep, because they already earned a lot (emphasize on "earned").
But don't make the exception (Joel asking about turtles) the rule. Joel is very famous in this community and the question was obviously simple. The question got popular because of this contrast. It make people think "What does Joel intend?". And they tried to show their respect.

In addition to being (sort of) a measure of programming skill and writing ability, reputation serves as a metric for how much time and effort you put into the community. Someone like Jon Skeet, who visits SO daily and continues to answer questions even after he's hit the daily cap, should have an enormous rep IMHO. Said another way, SO can only give you what you put into it.

Sure. But that kinda implies that one can "mine for gold" - gain lots of reputation - by the sheer number of hours spent. I'm not sure this is a great idea.
–
user3788Jun 29 '09 at 18:11

@rascher - why not? The site wouldn't be a success without people putting time into it. The rep says "I put effort into this".
–
EarwickerJul 6 '09 at 20:59

@Earwicker - This is a good point. I was originally inclined to compare it to someone who spends a lot of time mining for gold in an MMO - all it takes is time but no game skill. SO is different - it is a time sink (like gold mining), but if you are getting upvoted, it means people are finding your posts to be valuable. I hadn't thought about it that way.
–
user3788Jul 8 '09 at 2:05

I think that there is a valid point in that being a celebrity or at least viewed as a celebrity can help you gain some attention. I think that the LOGO question was interesting because it was used as an example in the podcast which undoubtedly brought in a large audience of viewers.

Since the question was really a discussion around whether or not a question was too simple and easy to be on SO between Joel and Jeff, I think that it hit home with many SO users. Because it hit home, I think people responded by voting up the question and answers.

I also don't want questions/answers to become anonymous because there are some users on the site who have helped me out a great deal and I do feel some responsibility towards trying to assist them in the future. The inverse of this is also true.

Take a look around. Upvoting based on celebrity is not only SO's problem. It concerns many communities. I'm sure such celebs worked hard for their reputation and they deserve it. Well, at least most of them did. It's a common noticeable behavior that celebs tend to lose theirself with time and tons of easily earned votes. Their answers start to sound more general, don't solve particular problem but sound reasonable and on the subject. Neat but not useful for the asker.

It really annoys me when a celeb gives their vague-but-neat answer and his fans like a swarm start the upvoting. I've even herd opinions that such situations scare off people that would place a real problem solving answer: Yay, da celeb again! I'd help the asker but it requires some effort! I'd do the effort even for this tiny shiny vote-up prize. But I won't! This celeb dude put no effort and yet still gets the cream...

However, I do respect those of celebs that still got it and their answers are still sharp.

You are right, I corrected the number, thank you!
–
user3788Jun 29 '09 at 18:01

What? Since when has there been a per-question limit?
–
mmyersJun 29 '09 at 18:07

mmyers: You can only gain 200 reputation points per day, excluding rep from people accepting your answers
–
dbrJun 29 '09 at 18:24

2

According to his Reputation page, Joel garnered 922 points from that question.
–
Shog9♦Jun 29 '09 at 19:02

yes, he got 900+ rep from that question. There is no per question limiter, only a 200 rep per day cap -- so that question was popular for multiple days!
–
Jeff Atwood♦Jun 30 '09 at 10:15

2

full data: 976 rep (as of this comment) for 122 upvotes, so the cap only denied 244 rep. Again, this indicates the Q had some staying power and was getting votes over a series of days.
–
Jeff Atwood♦Jun 30 '09 at 10:17

While it is probably true that a question from Joel will always attract more attention and votes than one from John Stackoverflow Doe, I believe that many people voted for his question because of nostalgia: