Documentation. Also known as a gold mine for people who love to get easy reputation points. Right now it's being (ab)used by many users, some of whom are using it to exponentially increase their reputation points.

Currently, upvotes on examples give all contributors +5 rep. There are no limits on the amount of rep you can gain from an example, except for the standard 200 rep per day limit. This system is severely broken.

For example, I contributed one sentence to the Python 'Hello world' example, to revert an edit made by another user (which should have been rejected) and add a single (relatively unimportant) sentence. Since then, I have received more than 300 reputation, tripling my Stack Overflow reputation points. And I continue to get reputation, even though the sentence I added is no longer there! I don't deserve this.

Why does reputation even matter that much? Surely a bit of inflation caused by documentation can't hurt?

Yes, theoretically reputation is just a form of imaginary Internet points. But it still determines privileges on Stack Overflow. We don't want a bunch of low-rep users with high-rep features, do we? Also, in the professional world, employers do use your Stack Overflow reputation as a rough gauge of your programming ability, but Documentation is making it look worthless.

I think that this system is severely broken at the moment, and we need to introduce some more reputation limits to fix it. I have a few ideas which would reward substantiate edits while reducing the possibility to farm reputation points. (Note that some of these ideas were seeded by Jon Ericson and sokin's answers.)

Idea 1: Only reward editors whose contributions still make up a substantial part of the example.

Basically, an editor should only be rewarded if the content they contributed makes up more than x% of the whole example (e.g. around 10 or 20 percent). How 'their content' is counted is debatable, but possible systems include:

Counting based on whoever last modified a line (e.g. git blame)

Counting based on the number of words which are still part of the example

Whether or not the example creator always receives reputation points or not is also debatable.

Idea 2: Introduce a reputation points cap per-example

There should be a reputation points cap of (e.g.) 50 reputation per example, so single edits won't produce huge amounts of reputation. Subsequent edits could possibly allow a higher cap.

Alternative

The reputation points cap could only be introduced when a large number of people edit an example. E.g. once more than 5 or 10 people edit an example, further editors cannot gain any more than 50 rep for that example. Previous editors can keep any excess reputation points they have accumulated.

Idea 3: Scale reputation points awarded per-upvote, based on number of editors

If an example only has 5 or so editors, each person will be awarded the full 5 reputation for every upvote. But, once an example reaches 10 or so editors, each upvote may only be worth 3 reputation; and when there are 20 editors each upvote will only be worth 1 reputation.

This way, the amount of reputation received from upvotes is proportional to your contribution to the example.

Alternative

Instead, reputation per upvote could be scaled by the amount you have contributed to an example: if you have only contributed a few words, you only get a few reputation points; if you wrote most of the documentation, you will get a larger share of the reputation points awarded. This would probably be a better way to scale reputation points, but is more complex.

Idea 4: Do not award reputation points if your contribution has been overwritten or removed

This bit is obvious - if your contribution isn't part of the example any more, you shouldn't be getting any reputation for it.

Hopefully if some of these suggestions are implemented, reputation points will be a better representation of your activity and helpfulness on Stack Overflow - not some huge number caused by a couple of tiny edits to documentation. Of course, a reputation points recalc will probably be required to bring things back to normal.

"Currently, upvotes on examples give all contributors +5 rep." Are you sure this is still correct? A few days ago I tested by editing this example and have not received any reputation. Maybe they silently fixed this?
– Bjørn-Roger KringsjåJul 28 '16 at 11:44

@Bjørn-RogerKringsjå You have to do a "substantial" edit. I think right now that is 100 characters. Not 100% sure on that number but a simple indentation or small typo fix is not supposed to give you rep anymore.
– NathanOliverJul 28 '16 at 11:48

Plenty of people want the reputation removed from Documentation - I would also approve of such action. Or if these imaginary internet points are so important - let Documentation have separate "reputation" points. This was suggested several times as well. I second each of these options.
– Maria DelevaJul 28 '16 at 12:02

@MariaDeleva If the point are imaginary and not all that important, why so much fuss over them?
– DavidGJul 28 '16 at 13:26

2

The "imaginary internet points" bubble is about to burst, yet life will move on and you'll still wake up in the morning. Try not to let it get to you. EDIT: @DavidG easy for you to say, you're at 30k.
– SharkJul 28 '16 at 13:26

1

@Shark Yes, but I've been here for 4 years. If I had really been bothered, I would have been a lot higher :)
– DavidGJul 28 '16 at 13:30

1

Also could change the incentive method to award points to the act of editing, rather than an edit giving you points till the end of time. This would decrease the payoff of getting a small change in. But we shouldn't remove all incentive from small changes. Sometimes a single character, may in fact, have an impact on code you write.
– Conrad.DeanJul 28 '16 at 13:50

2

The simple solution is to have Q&A rep and Documentation rep be separate totals. This also makes sense, because they're two different things. Competency in one, does not imply competency in the other.
– hatchet - done with SOverflowJul 28 '16 at 13:52

1

@hatchet It's already been said by SO staff that having a second set of rep would have been a huge amount of work so this is highly unlikely to happen.
– DavidGJul 28 '16 at 14:34

3

I'm not really sure what this is contributing that hasn't already been said. Ideas 1 and 3 are covered by my Documentation shares the work make the rewards shared not multiplicative, which you're obviously aware of because you linked to Jon Ericson's answer there. 4 is implicit in 1. An answer there proposing variations or specifics would be welcome. Number 2 has been proposed in comments here and there, but comments don't really count as proposals, so that could make a fine feature request on its own if you were to edit this down.
– jscsJul 28 '16 at 17:30

7

@hatchet High rep in Q/A doesn't apply or even strongly correlate to competency. You can (and some people have) get to 100k answering only very easy question that a student can answer. Its all about when you started and how persistant you are.
– MagischJul 29 '16 at 8:51

I'd just like to share the insane scenario I'm in. In the past week, I've gotten 750 rep. Over the summer, I've been asking and answering to earn rep, and got to around 1k. I was pretty proud of that number, because it doesn't come as easily as it did a few years ago. Now, just by contributing to a couple of docs, I get 750 rep? I wasn't making illegitimate suggestions - they were, in my opinion, useful and needed. But they were so insignificant! This has to be fixed. The lights at the end of the tunnel are the headlamps of an oncoming train.
– AresAug 1 '16 at 16:50

Original answer follows

We're definitely going to fix this. Kevin and others will be discussing the particulars this week, though it may be a bit longer before anything gets rolled out (as the variety of suggested changes here demonstrates, there are a lot of factors that should be taken into consideration). Once changes are made, reputation will be recalculated for anyone affected and we'll post an announcement here on meta.

To be clear, I'm not saying these are all problematic. But they are anomalous; we should examine them carefully when deciding whether or not they embody behavior we should be encouraging here... And then adjust the system as needed.

There will be a post from me, soon, which more clearly lays out what we envisioned with the rep system for docs as it was launched. The goals are really good, balance is tricky. As Shog noted, we're going to have to do a recalc, so we've got to be really careful to not inadvertently make the balance a bigger issue than it is - which means this is going to be at least a few weeks. I'll be posting this week at some point, there's a lot of stones to turn over before we can say we've turned 'em all, and a breadth-first search comes in quite handy there.
– Tim Post♦Aug 1 '16 at 18:02

2

@Tim will you please address one aspect which I haven't really seen discussed yet: will a rep system adjustment be retroactive, meaning undone reputation for the masses? Or only concerning future gains? From "recalc" I'd suspect (and hope for) the former, but it would be nice to have an explicit heads-up about this. Thank you.
– Andras DeakAug 1 '16 at 22:58

7

At this point, changes to the rep system will be retroactive, @Andras. Still a beta, still in flux. That said, rep system changes are never trivial; we'll talk about this a lot more before anything happens.
– Shog9Aug 1 '16 at 23:11

3

Thanks for taking the time to respond - we appreciate the communication.
– angussidneyAug 2 '16 at 0:55

@TimPost, What about, for each N up votes, the rep per up vote is reduced by 1, so after so many up votes additional up votes give no more rep to anyone.
– Ian RingroseAug 2 '16 at 14:27

1

@TimPost I think a big point to to take into account is that a lot of the upvotes happened within the first few days (even hours) of the public beta launch when documentation was new and exciting. Although the rep system still needs tweaks, it seems like the big rush has slowed and people aren't earning crazy amounts of points anymore, at least not any crazier than the people who earn 100s of points daily for old SO questions.
– Tot ZamAug 3 '16 at 2:28

3

@TotZam It reminded me of the earlier days on SO (and every public beta we've ever launched). There's just this initial sort of frenzy and then everything settles down. I actually think the main problem is that the system isn't directing people to move examples, or create new topics entirely, once there's more than [x] examples in a topic. Had 3 of the largest been spread over 4 - 5 each, I don't think the problem would have been nearly as pronounced. We're looking at tweaks there now.
– Tim Post♦Aug 3 '16 at 3:18

Asking good questions and writing great answers is the purpose of Stack Overflow. Seven years on (is it eight? nine? I don't even remember any more) the whole concept is well understood at this point.

Not that writing good examples documentation is not important, but it is a very different animal. Examples Documentation is highly speculative, in a way that traditional Q&A fundamentally is not. In Q&A, some real person asked a question, with our strict SO definition of "useful / researched / clear" -- a fairly high bar relative to the Internet -- and it got answered. This isn't someone guessing "hey someone might want to know X so I'll just type a bunch of stuff here just in case", it is a direct response to a direct question by an actual person.

Documentation, on the other hand, is answering a set of questions that we're not sure anyone is even asking! Now I am not arguing that this is right, wrong, good, bad, or guy with the gun, but that it is just ... different.

To the extent that the differences between examples documentation and Q&A is devaluing rep people earned by asking questions and answering them -- the primary activity in the system, the reason people come to Stack Overflow in the first place -- that is a concern. Wouldn't you be upset, too, if you earned 10k rep through a series of great answers, and then were directly compared to a user who earned 10k rep exclusively through writing examples documentation?

Perhaps merging these reputation systems for fundamentally different activities might be the source of a lot of the current friction?

"Perhaps merging <...> be the source of a lot of the current friction?" You bet. When I see "newbies" getting buckets of rep for trivial edits to documentation, it devalues the hard-earned Q/A rep.
– CerbrusJul 29 '16 at 9:26

15

Thank you!(The author of this comment wishes to apologize for a complete lack of constructivity.)
– TerraPassJul 29 '16 at 9:27

12

Sad part is the disproportionate number of very low rep users doing the guessing of what is needed vs higher rep folks that have a better finger on the pulse for recurring issues
– charlietflJul 29 '16 at 12:30

8

If being on a different site of the network which merely changes the topic is a good enough reason to have a separate bucket of reputation there, why should documentation and Q&A have a shared bucket? Split reputation already! Then figure out how to fix documentation rep before the database explodes from the overflowing reputation column in the users table.
– nullJul 29 '16 at 16:14

11

I agree. I made an innocuous edit to an example in documentation the day it went live, when I didn't really know that there was rep involved. I got the +2 when the edit was approved and though it was weird, but well, it was little rep. But then I started getting rep from upvotes to the example, that's just not right, my edit was minimal, I want to give that rep back
– LamakJul 29 '16 at 16:14

7

I truly truly hope that at least the creator of this dang thing has enough clout to finally put this in motion, as it appears from the lack of inaction that the community sure doesn't. Thank you @JeffAtwood for finally speaking up for the rest of us.
– Paul SamsothaJul 29 '16 at 16:30

17

"Perhaps merging these reputation systems for fundamentally different activities might be the source of a lot of the current friction?" Certainly. But that's not the problem. Or rather, that's not the only problem. Divorcing Docs.SO rep from SO rep will help Q&A, but it still leaves Docs.SO with its own rather ludicrous rep system, where you get rep forever even if your changes have been obliterated. It still encourages making small, trivial changes to important examples rather than making new examples that will never bee seen. It just won't hurt Q&A with its terribleness.
– Nicol BolasJul 29 '16 at 16:38

28

@NicolBolas perhaps, but separating the rep systems makes it easier to revamp one or the other without breaking both.
– Jeff AtwoodJul 29 '16 at 18:14

on the other hand if you make reputation system on the Docs more fair there will be no need to make the SO and docs reputations separate.
– giorgimJul 30 '16 at 18:23

6

Why not just admit that the current reputation system for docs is poorly designed? The best way to earn lots of reputation on docs is making trivial edits to many high-profile examples and that's a problem that exists even if you put it in a different bucket. Not only because those people are rewarded for the behavior, but also because it floods the system with many pointless revisions.
– RadiodefJul 30 '16 at 22:54

3

@Cerbrus Here I disagree. The merging itself isn't the problem. The problem is in merging an unbalanced reputation system. Fix the docs reputation system, not the merge.
– bwoebiJul 31 '16 at 5:00

6

@uoɥʇʎPʎzɐɹC the satisfaction of having Rep in a Technical Writing bucket. So far, I haven't seen much Technical Writing talent. I am sure it is out there, but I haven't stumbled onto it yet.
– DrewAug 1 '16 at 0:03

11

@uoɥʇʎPʎzɐɹC considering that nearly all of your rep appears to have been gained from activities other than asking or answering questions (aka, documentation), I am not surprised at your objections.
– David LAug 1 '16 at 1:59

Docs should fill gaps in official documentation, rather than trying to replace it wholesale. Instead of being the place where anyone can contribute, and farm rep, it should allow intermediate and above users in a field to craft expertly-written content:

Docs contributions are written by people with at least 1k rep (or an appropriate percentage of the rep range the tag has, whichever is lower) in the tag from answers

Docs contributions are worth no rep by themselves

Docs can be edited for technical inaccuracies/omissons, lack of clarity and lack of "flow" and references between sections of documentation

Docs sections can be used directly as answers to SO Q&A questions, or embedded as subsections inside answers. Upvotes to these answers will provide the Docs contributor with Q&A rep

Docs are versioned according to the tech version they refer to, so for example embedding PHP4 documentation in a PHP4 question will keep the same docs embedded, rather than have it change "underneath" the answer, when PHP5 content is added. Edits to that version are applied to the embedded text, of course

Question and embedded Docs answer will be the preferred method of self-answering questions, where possible

Point 1 is too restrictive for small tags. Take ABAP for example, where no users are over 1k rep, and only four of which have over 100, two of which being relatively inactive.
– gkubedAug 2 '16 at 12:45

1

I think the key is that docs are there to answer questions and should only get rep when they do.
– Ian RingroseAug 2 '16 at 14:29

1

@gkubed fair point - perhaps it could scale with the range of rep available - e.g. top X% of rep or 1000 rep, whichever is lower?
– Rob GrantAug 2 '16 at 16:43

If an example only has 5 or so editors, each person will be awarded the full 5 reputation for every upvote. But, once an example reaches 10 or so editors, each upvote may only be worth 3 reputation; and when there are 20 editors each upvote will only be worth 1 reputation.

This way, the amount of reputation received from upvotes is proportional to your contribution to the example.
Alternative

Instead, reputation per upvote could be scaled by the amount you have contributed to an example: if you have only contributed a few words, you only get a few reputation; if you wrote most of the documentation, you will get a larger share of the reputation awarded. This would probably be a better better way to scale rep, but is more complex.

I'd take this even further.

In my opinion, if the documentation reputation were to stay in the same bucket, then a single upvote on an example shouldn't increase the sum of total reputation in the system by more than 5 (or constant N).

What I've seen is that the more editors there is to an example, the worse it usually becomes. It is just sawing between acceptable and unacceptable - invalid edits, and then deletions, yet all of them would continue (as far as I know) receive reputation from upvotes. An example is not made 50 times more reputable by 50 editors "fixing" it again and again than it was if the exact same content were thoughtfully written by one person in one revision.

I was thinking of a quite complicated algorithm that after each edit would simply "git blame" the current revision - count how many characters each editor would have contributed to the current state. Then for each upvote, basically each editor would get lottery tickets based on how much of the example was theirs. This can be made stable so that the reputation recalculation always produces the same result. Then the 5 reputation points from upvotes would be allocated by drawing lots, either all 5 points, or one reputation point a time.

As an example, in revision R, 68 % of the example text is provided by me, and 32 % by others. Upvotes to revision R would either

Precedent: community wiki posts already display a contribution percentage for the original author, though of course it doesn't award rep (maybe that will change if Docs continues to do so?). On the other hand I recall a meta topic where someone complained someone else was getting partial credit by this metric despite their edit having negative value (and getting mostly reverted).
– Jeffrey BosboomAug 1 '16 at 5:24

@JeffreyBosboom my algorithm would calculate each delta in the final result. If someone's edit was mostly reverted but then contains one retained fix, say a to an, then they'd get some fraction of percent of upvote rep, but not 5.
– Antti HaapalaAug 1 '16 at 5:29

1

The lack of reputational enhancement for Community Wiki posts has always been one of the least comprehensible parts of SO - it's like contributions aren't wanted, or something
– holdenwebAug 2 '16 at 8:02

Only reward editors whose contributions still make up a substantial part of the example

Someone creates an awesome example, over the time it evolves somewhat: some sentences are reworded, typos are fixed, variables are renamed etc. Eventually SO's algorithm determines that "nothing is left", although for human readers the example remains essentially the same.

Result: less people want to create new examples and topics, it's more profitable to edit existing topics. This happens already, so the change will just make things worse.

Introduce a rep cap per-example

You'll gain more rep from creating new examples.

Result: less people will be interested in improving existing examples, topics will be flooded by new examples. Cheaters will remove existing topics and create copies of them. It already happens too.

The rep cap could only be introduced when a large number of people edit an example

Again, less motivation for editing examples. Or maybe even finding your enemy's examples and editing them to deny reputation gains. And again, deleting and recreating becomes a profitable strategy.

Scale rep awarded per-upvote, based on number of editors

The same. Though this looks like the most reasonable one. Mostly because it's gradual.

Instead, reputation per upvote could be scaled by the amount you have contributed to an example

Nobody will fix typos. You gain nothing, others lose a lot. Currently only the first part is true.

Also small edits may be rejected - if I notice that somebody tries to edit my topic I will want to avoid losing rep.

(From comments) Plenty of people want the reputation removed from Documentation - I would also approve of such action

Then Documentation will die. It's already flooded by low-quality examples which provide zero value, but even that content will be destroyed. Nobody will care.

The only fair way of giving reputation is manually grading every single edit to every topic by professionals, and this is impossible for obvious reasons.

"Nobody will care." Is it so much worse than everybody caring just because of cheap reputation? I mean, Documentation might die, but at least it won't take down the entire SO rep economy with it. And then, the death of Documentation without rep is not a given, either: there are people, who will contribute regardless of reputation rewards. They might be even more encouraged to do so once we remove the incentive for the swarm of me-too editors, ready to vandalize (I mean, improve) other people's hard work to get on the rep train.
– TerraPassJul 29 '16 at 9:14

1

@TerraPass I doubt SO's devs/owners want Documentation to die, most likely they want more content to be generated as fast as possible to get something to work with. Disabling rep isn't an option because they want vandals (I mean, editors) to generate text. Once that stage is over, maybe something will be improved...
– AthariJul 29 '16 at 9:27

8

"Then Documentation will die." Then Docs.SO deserves to die. If it can't live on the quality of people who do work for free, without reputation, then it deserves to die. Wikipedia manages to work even though it has no rep system. cppreference.org manages to work even though it has no rep system. And so forth. Why does Docs.SO need one?
– Nicol BolasJul 29 '16 at 16:40

7

To me, these are all further arguments for keeping docs/examples rep in a separate bucket, as I covered in my answer.
– Jeff AtwoodJul 29 '16 at 18:20

2

@JeffAtwood Yes. There's no way to fix all this mess without making it even worse (at least I can't come up with any), unless reputations are separated. I'm just afraid it's not an option SO dev/owners consider. :-( My previous post critisized the "solution" which SO devs chose (less rep for edits), and how it didn't actually improve anything. It's also not only about devaluing rep, it's also about losing motivation for answering, something which everyone seem to forget - after literally a couple of edits, you get Jon Skeet's level of passive rep.
– AthariJul 30 '16 at 6:40

@Squidward this is why my answer is the way it is: no Docs rep (or not directly, anyway).
– Rob GrantAug 16 '16 at 21:38

That looks like a user that wasn't active at all before documentation. Without context, this graph says nothing. Heck, for all we know, that rep could come from bounties.
– CerbrusAug 3 '16 at 7:05

8

@Cerbrus Most of the rep comes from "creating and initializing arrays", "using console.log()", "using window.alert()" and "Hello World". None of the rep comes from bounties. The user has been active for over 2 years, posting questions and answers regularly, and gaining rep at a normal pace. In the last 10 days he has gained as much rep as he normally does in a year. I don't want to drag anyone through the mud here, this is not about him/her, it's about the rep system.
– m69 ''snarky and unwelcoming''Aug 3 '16 at 7:19

From the look of that graph, he wasn't active at all in the weeks prior to documentation. How much did he have before docs? Also, the fact that he's consistently passing the rep cap, means he's doing more than just getting upvoted, since example votes can't pass the rep cap.
– CerbrusAug 3 '16 at 7:22

The core problem is that the data model for giving out reputation is fundamentally flawed: Currently reputation is given to all contributors based on votes on the end result. That is not good because it does not differentiate between contributions. Thus

Idea 6:Each contribution/edit should be rated by its own and reputation given to its author based on that only

This could be implemented as when someone votes on an example, a "Please spend a few seconds rating one or more of the edit to this" question appears1, and also possibly for each X-th visit of the page.

The rating could be

Improves the example very much

Improves the example moderately

Improves the example a little

Improves the example insignificantly

Makes the example worse.

Don't know/unsure.

and then some rules for converting such ratings into reputation.

1 Along with rating hint: "Rate for the lowest part, e.g. if the added text is great but the added code allows for SQL injection, rate it as makes it worse".

This would absolutely be the fairest way to do it, but I'm sorry, neither I nor, I strongly suspect, most everyone else, wants to spend hours reading diffs to decide which particular edits deserve votes.
– jscsJul 29 '16 at 22:38

If they were separate who would contribute - the major incentive for getting SO rep is bounties + privileges.

Just because the rep system is broken doesn't mean it can't be fixed. You don't remove it, you fix it. You don't fix a car with a broken engine by converting it to a electric car and buying a charger. Removing it doesn't really address the problem - we got a gas car for a reason - and we don't want to forfeit the benefits.*

I think with the changes suggested in the original question the rep system can be fixed.

If Jeff's answer were posted by someone else I would expect it to be down voted heavily.

A lot of people have gone "Oh, this user got most of his rep from documentation, he's pretty biased." I think the rep I got from editing popular posts is temporary and don't expect it to stay. But look at this: I contributed several original (IMHO - you can judge for yourself) high-quality examples. Because I know in the long term I'd get more reputation from those examples, and the reputation from the popular ones would vanish in a recalculation.

*and unlike in real life, the electric car industry (separating the systems) is a mature technology and was considered as an alternative.
– noɥʇʎԀʎzɐɹƆAug 1 '16 at 0:01

Not a DV from me, cuz I am not in the mood. You asked me to look at your contributions. One of your last Here on main. I am focusing on Q&A this split second. You did a nice job helping the op and clearly explained the situation. Very insightful, brief. I loved it. You allowed yourself to be opened up to scrutiny, field questions, and get married to the problem. Fortunately, no problem. You were willing to enter the arena, you left like a champion. Nice job. Docs is another issue and is no arena. I think the reason why has been explained well by others
– DrewAug 1 '16 at 0:34

I agree boss. You know where to find me if you want any of your high-quality stuff quickly reviewed. The queue can be a hassle otherwise :p
– DrewAug 1 '16 at 16:37

@Drew mostly to the upvoters of your comment and not to you :p
– noɥʇʎԀʎzɐɹƆAug 1 '16 at 16:39

3

It was a cheap shot. I apologize. Understand though the frustration to put it mildly that the new system represents to those that gave so much. We understand gaining rep on Main is difficult. We need some balanced system that allows for new contributors without the whole thing becoming a slot machine.
– DrewAug 1 '16 at 16:42

1

@Drew this... this should be an answer. brief and concise.
– noɥʇʎԀʎzɐɹƆAug 1 '16 at 16:44