How does that data correlate with views per post? I think that would be an important metric. Maybe the real problem is that less eyes are on the posts. (I suspect that may be it)
–
jjnguyFeb 23 '11 at 4:15

@jjnguy I would need help from the stats guy to answer that, we only store total views, not views per-month
–
wafflesFeb 23 '11 at 4:19

@waffles could you please add the total number of users to the stats?
–
systempuntooutFeb 23 '11 at 8:25

That would have to be active users @systempuntoout, because I you can't vote if you don't log in
–
Ivo FlipseFeb 23 '11 at 8:38

Could there be a correlation between users 'flocking' to SE sites in favor of SO and taking their votes with them? Programmers for instance certainly doesn't lack votes...
–
Ivo FlipseFeb 23 '11 at 8:40

+1; I agree with the general idea, but given the number of metrics that make up civic-dutyness I've stuck up a proposal to expand it to include all those other metrics that make a good citizen.
–
user142852Feb 25 '11 at 11:20

The raw number of votes is increasing, so I think what you're seeing is either:

Even though there are more users, they are hitting the 30 vote/day cap.

People are not seeing as many % of the posts as they used to be and what they are seeing is of declining quality (partially supported by the fact that answers/question is on a slow downward trend).

To counteract these:

Raise the number of votes/day to 45 (or some other higher value).

Increase the time between new questions for new and low-rep users. (If we cut down on the low-quality questions, people will see more of the high quality questions, which they are more inclined to vote on because downvoting charges a penalty).

After a low of ~1.9 votes/post in March of 2011, the rate has settled into a range between 2.6 and 2.8.

That got me thinking about the network as a whole. For instance, I looked at the perceived drop in quality on Digital Signal Processing and concluded that a steady drop in voting was the culprit. Either good contributors are less motivated to ask, and answer questions or posts with few votes come off as lower quality. (Or both!)

What's a healthy ratio of votes to post? Should a large drop, such as SO saw from 2009 to 2011, concern us? To answer those questions, I queried our database for votes/post, total views, and total posts. I've included upvotes, downvotes, and accept votes because these have the greatest impact on participation. I'm using views as a loose proxy for success. (That's not really fair to smaller topics, such as Poker, but we take that into account when evaluating sites.)

This graph shows votes/post on the vertical axis and the horizontal axis is log scale views. The bubbles represent total number of posts:

The left side of the graph is populated by sites we closed and the right side is dominated by Stack Overflow. Taking out those exceptional sites we get this:

So there's a big cluster of beta sites that get between 2 1/2 and 6 votes per post. On the right side we have some graduated sites with low voting and a handful (Skeptics, Science Fiction & Fantasy, RPG, Programmers, CS Theory, TeX, MathOverflow, Travel, and Mathematica) with v/p north of 5 1/2. There are also a number of graduated sites that aren't extreme in either direction.

Some of the sites on the high side of this metric (Skeptics, MathOverflow, and CS Theory, in particular) suggest that votes per post is a measure of "degree of difficulty" in writing questions and answers on a particular site. On Skeptics, if you don't do a good job of establishing your claim as notable, your question will be closed. If you don't follow the rules on using sources, your answer will be deleted. It's probably our most demanding site. Therefore, many people vote on the site without contributing much in the way of posts.

On Stack Overflow, we see the problem with success. There are far more posts submitted than anyone could hope to keep up with. At best, you might be able to keep up with the questions on a popular tag, such as python...maybe. So we can easily see why so many posts get no votes at all. Anecdotally, that's a problem for people who started contributing later in the game; only exceptional posts gather many votes.

Conclusion

Tentatively, I suggest that different sites will have different "steady state" levels of voting. Drops in voting, such as SO saw between the start of 2009 and 2011, should be concerning for those of us watching particular sites. When this sort of decline is observed, steps should be taken to reverse the trend.

Now, the question remains: what reversed the trend on Stack Overflow? Some candidate changes:

It's the first graph I find most interesting here. Did you use post date or vote date? If you look at median votes using post date, it looks a lot worse. Median votes by vote time show a pretty stable trend over all time. I don't think things have got better; it just looks like there are more good questions that people are voting on.
–
ben is uǝq backwardsJan 15 '14 at 21:54

To speculate heavily: in the beginning all the "easy" questions got answered and so have lot's of upvotes. Then there was a growing period, while SO got big and a bit of pain as the community cracked down on what it disliked (more deletion etc). Now, the community keeps growing, which keeps the median number of votes per post down but the posts by people who've been here a while are better than they were in 2011, making the average better.
–
ben is uǝq backwardsJan 15 '14 at 21:56

@benisuǝqbackwards: The first graph is from this query. (The graph has the link to the query surrounding it, but I guess I didn't end up liking it in the text somewhere.) I used the time the vote was cast to determine which month it belongs to.
–
Jon Ericson♦Jan 15 '14 at 21:57

1

Some important factors I suspect to lead to the unusal voting on Skeptics are the accessibility and popular appeal of the typical Skeptics question combined with the very low question volume. Our answers are designed to be evaluated by someone without any particular domain knowledge as we rely on citations. And slow sites always seem to have much higher votes than the fast sites.
–
Mad ScientistJan 15 '14 at 22:03

@waxeagle: The graphs aren't working properly, but here is the spreadsheet data. I used the "Bubble chart" to graph the data. I could try zooming in on the cluster of sites in the middle to make them easier to separate, but I'm not sure I'll do that today.
–
Jon Ericson♦Jan 15 '14 at 22:11

1

@MadScientist: Hmmm... could you expand a bit on what you mean by "slow" and "fast" sites? Do you mean the rate at which the front page changes with activity on posts? That's something we can measure.
–
Jon Ericson♦Jan 15 '14 at 22:33

1

March 2011: "Increase in the daily voting limit for questions" + "Free downvotes for questions" => hunters for Civic Duty and Electorate badges push more questions down to and under zero => autodeletion kicks in => more low voted questions go away => average votes go up
–
gnatJan 15 '14 at 22:36

2

@JonEricson I mean the volume of posts, my very subjective impression is that the number of votes per posts decreases significantly if the question volume increases, and each post gets less attention.
–
Mad ScientistJan 15 '14 at 22:50

...(continued from above) in this "chain", badge hunter's role is to educate other users by demonstrating that voting is widespread, normal and expected. Right score is great, wrong score at least shows a clue / prompt to correct it (vote opposite). No score is a dead end, worst case. By the way, hot questions play similar role, especially since these aren't hidden in collider anymore
–
gnatJan 16 '14 at 4:43

@MadScientist less questions also mean individual voting makes substantial impact. Take an example of the site having about 4K questions. Take four top voters who cast about 12K votes. Divide 12K by 1,5-2 to account for deleted posts, get their 6-8K votes spread over 4K questions => this adds solid 1,5-2 votes to average. You see, 4 voters matter there
–
gnatJan 16 '14 at 6:43

1

Is it possible to get something like "average votes/view" by site, for both question/answer? We have been discussing this in Workplace and wonder if this affects the "hot questions" problems we have.
–
enderlandMar 14 '14 at 16:16

Maybe it's because fun/subjective questions are becoming less and less allowed over time, such as this one.

Those kind of questions have probably thousand of times the votes of a normal question (probably more than enough to screw the average votes per post on all questions). Since they are usually CW, maybe you can try to only count non-CW votes in your query and see if it makes a difference?

+1 for re-running the query with CW posts excluded. I'd be interested to see if there's a large difference (though personally, while I do expect it to be measurable, I disagree and and think it will be pretty small)
–
Joel CoehoornFeb 23 '11 at 17:24

Badge

Stealing slightly from jjnguy, we have a badge for hitting the daily vote cap (Suffrage) once, but what about on several days, or lots of days? As such, I propose a silver and a gold badge for voting a lot. Here's an example of what one might look like:

For hitting the daily vote cap on 150 days.

Since badges are designed to encourage positive behaviour on the site, it makes sense to have a badge for voting a lot. I've hit the daily vote cap fairly often.

Potential downsides:

This doesn't guarantee quality voting. That's where you have sportsmanship etc.

This will have the same effect as the electorate badge, where people will vote like crazy on everything in sight.

Computer Index Score -> Stackoverflow citizenship score

This is an extension of what systempuntoout is proposing, sort of, but changed.

Yes, I'm talking about this. Let's face it, the stackoverflow profile is accumulating a lot of statistics and information and there's lots we've all wanted to see in the moderator elections, in that little badge etc.

So, as an idea, I suggest we do a Microsoft: take all these metrics, give them a score out of 10, then take an average (or something) giving a total community rating score. Then display that on the profile. This would take into account all the rewarding badges, like sportsmanship, electorate etc.

In short, it's kinda like a stackoverflow citizenship test. It takes the need for all this extra information in the profile signature box away. You can factor things like accept rate into it too.

This would be in addition to the existing rep/badges. This is because rep/badges are for you, and this is how well you play in the team.

Upsides:

Reduces clutter.

Provides a simplified "niceness" indicator.

You'd still be able to see all the badges, votes, questions etc. Indeed, these stats could go under one of the profile tabs.

This information is being computed anyway, for the badges.

You could use it in moderator elections.

When you have another niceness statistic as demanded by meta, you just factor it in to this one.

Downsides:

Like all such statistics, its a face-value approximation, or a rough guide to how nicely you've played. Just like rep, basically.

Everyone will have an opinion on what should form part of said statistic and how much weight it should hold and nobody will be happy with the result.

I haven't made any proposal as to how these calculations should happen. I've just proposed an idea. So that would need to be worked out.

Probably some more.

The image is badly done and only slightly uses freehand circles. I'm sorry. I couldn't fit it all in in freehand without using a massive image. I hope the badge makes up for it, and my overuse of bold text and obnoxious formatting.

Well, yes... but you already have that. It's harder when you build the total metric from things like #edits compared to #closes, flag weight etc. There's only so much you can automate. And I believe there's an abuse detection system in operation anyway. But you're right, any new system and someone will try to game it.
–
user142852Feb 25 '11 at 13:59

I would look to query figures like views as compared to votes, as well as vote spread - are we seeing a higher percentage of posts not get any votes at all (or more negative votes), skewing the results?

The number of duplicate and off topic posts is increasing as the number of truly new questions goes down (The Simpsons did it). I don't know enough about the workings of SO to know if your community ownership clause is catching those [closed] posts.

The number of Google drive by questions (account with just one question) is rising, contributing to the previous two points?

"So you have a graph that proves that a question that was asked a second ago was viewed less than a question asked 3 years ago. What exactly does that prove?"

and

"I have no problem and any data helps out. However as it stands this proves nothing, if you could come up with an expected trend graph and show where we are dropping from the expected trend, it would be helpful. But as it stands all that this proves is that there is a long tail."

and from Jeff:

"I agree with waffles, this is simply a graph that says "over time, older things are viewed more". It's not particularly useful."

Does the same not apply to votes?

Isn't this just saying that over time posts get more votes? Where is the trend?

sorry, I did not mean to be abrupt. this particular query does not look at views, only votes. I partition it so I only look at votes on a particular post that happened in the month we are analyzing. so this is safe against older stuff being treated unfairly. to do the view analysis properly we would need help from the stats guys.
–
wafflesFeb 23 '11 at 22:43

@waffles - I had a relook at the data as @Kyle Cronin suggested and the long term trend (across three months of data dumps) is only slightly down for views/question, not as drastic as the initial data suggests. I would suggest you do the same with the votes, it gives a much better picture.
–
xiaohouzi79Feb 23 '11 at 23:04

I do that with vote analysis already, it is excluding votes out of the time frame analyzed. so for example when looking a June ... I count the questions in June, Answers in June and votes on those questions and answers that happened in June.
–
wafflesFeb 23 '11 at 23:07

@waffles - What I mean is pick a 3 month period (not the previous month, but the three months before that) and see how that period of 3 months compares across multiple data dumps. So currently for Sep, Oct, Nov you have (1.22, 1.26, 1.29). If you look at the same 3 month periods in previous data dumps do they also trend at the same numbers.
–
xiaohouzi79Feb 23 '11 at 23:31

1

yeah for views we would need to use that approach, but for votes we have a timestamp on each vote
–
wafflesFeb 23 '11 at 23:34

Imagine you read dozens of questions and answers every day. When you're fresh and eager you read each in detail, upvoting those you find helpful. But... as time goes by, days pass, you take less time to read each answer and you don't upvote because you didn't consider it deeply enough.

I'm sure there are people here who are as fresh and upvoters as they used to be years ago when they first joined, but I'm also suspecting there are many others with the above "symptom".

Newcomers are not relevant, they might upvote the answer that solved their problems but it's really rare they check any other questions.

So the problem IMO is with the "inner circle" of members and the solution? Good question. Probably some of those members can answer this better. :)

I have no idea if this applies to the community at large, but as a newer user on the site I'm hesitant to down-vote other users who offer suggestions that may slightly miss the mark but aren't out-and-out wrong (especially if they have been on the site for a while or have high rep). I've also found myself less likely to vote-up an answer that is clearly correct but already has the highest score of the answers listed for a question.

On a related note, when is it appropriate to downvote? If somebody leaves an incorrect answer - not necessarily a "wrong" answer, but just "didn't answer my specific problem", should they be voted down?

I asked a question on SO earlier today. It's only my second question. The first response didn't apply - but I wasn't sure if it was my fault for not explaining well enough, or his fault for not reading thoroughly. Within moments another user had offered a suggest that fixed my problem. I'd upvoted his answer immediately (and later accepted it) but I hadn't considered downvoting the initial wrong answer from the other user.

Mostly I felt like it's sort of a dick move to down-vote a guy who tried to help me, but just missed the mark. He didn't leave bad advice, it just didn't apply to my question as I'd scoped it.

I'm glad to see downvotes. This would most likely be a terrible idea.
–
jjnguyFeb 23 '11 at 4:14

Someone once said on Meta, and I can't remember who, and I'm going to paraphrase, "Reputation is for what others think of you; badges are for what you do". Or something like that. And I agreed, so that's why I personally dislike this.
–
waiwai933Feb 23 '11 at 4:16

@waiwai, I also agree it's a bad idea. I just though it should be brought to the discussion.
–
jjnguyFeb 23 '11 at 4:17

1

Someone else (Jeff) once said, reputation is a measure of community trust, earned via participation and personal investment in the site.
–
Lawrence DolFeb 23 '11 at 7:27

@waiwai When there was a feature request for removing the +2 bonus for acceptance (then the only method contradictory to that philosophy), Robert cited pretty much exactly the first part of that.
–
Grace Note♦Feb 23 '11 at 16:34