Short comments and extended essays about Wikipedia, based on over eight years of participation -- as well as other topics.

Wednesday, September 08, 2010

Pending changes, & how it defines Wikipedia

Well, the English language Wikipedia I mean.

For those of you who aren't that familiar with the latest conflicts at Wikipedia -- which has now come to include a large share of active Wikipedians -- probably the most important is the deadlock over whether or not to continue a test of the "pending changes" feature.

The proposed feature is simple enough. For certain groups of editors -- those editting anonymously or with a new account -- their contributions are sent to a queue where they must first be approved before being made visible to one and all -- such as people who are simply reading Wikipedia, who make up over 90% of its users. Otherwise, the edit is deleted.

This is something that Jimmy Wales has long advocated for (see here for one example), and as a result many Wikipedians consider it a good thing. And maybe it is; I'm undecided only because after a two month test, no one has presented any factual evidence that it solved any problems. I haven't seen any in the lengthy, at times bitter, and for the moment deadlocked discussion over its adoption. But what I have seen are examples of long-existing tensions that exist over the ideals of Wikipedia, but which have not been adequately discussed. (For all of his supposed experience with Wikipedia, I haven't seen Jimmy Wales even acknowledge their existence.)

One of these tensions is presented quite clearly on Wikipedia's front page: "the encyclopedia anyone can edit". This is both the strength and the weakness of Wikipedia -- that anyone can, and does, edit its articles. The fact that something useful has resulted -- let alone a reference work which is as useful as it is -- has been pointed out so many times to become banal. When critics pointed out that allowing "anyone" to edit, this included vandals and malicious individuals, the response was to point out that bad edits were almost always promptly reverted. And one could say it was part of the price for creating an otherwise invaluable resource; after all, when dealing with experts and powerful people, one must put up with their eccentricities to benefit from them.

However, the steady trend since its inception has been to limit the meaning of "anyone" with more and more exceptions in order to improve its quality and usefulness. Some of these exceptions are quite reasonable: vandals, kooks and cranks, people who simply can't play nice with others. Others fall into a more difficult category, such as those who want to advocate for a specific political agenda or for their own economic gain. One can't do the simple thing and exclude them all because in many cases, they are the exact experts Wikipedia needs in order to be a useful reference. So Wikipedians are forced to develop essays such as Tendentious Editting or Civil POV pushing to define the problem and help each other confront it.

And then there are some policies which came about in a quick, knee-jerk reflex whose rationale have never been adequately explained, such as denying anonymous and new editors from creating articles. This came about because of the Seigenthaler incident, where an anonymous editor made a libelous allegation about John Seigenthaler which remained undetected for four months. This denial has remained mostly out of inertia: denying anonymous editors all editting rights is a perennial proposal which, although always rejected, has strong support. And if one creates an account then waits a few days, one will be able to create articles -- so no one sees this as anything more than an inconvenience for new editors. On the other hand, it is not an effective security fix: determined troublemakers need to simply create an account and wait for it to age before abusing it to wreak havoc. This change is only a speed bump, which may discourage more new editors than it justifies in dissuading vandals; no one really knows.

Now keep in mind that I'm not criticizing putting limits on just who edits; all of us want Wikipedia to be as accurate and useful as possible. My point is that there is a tension here, between allowing anyone to edit and creating reliable content, which fails to be considered in these decisions. And when I tried to participate in the discussion on whether to continue this test, I was unpleasantly surprised to find that no one had provided the needed information to show whether this test actually improved content or not. Instead a lot of anecdotes, first impressions and opinions were being tossed around. I don't know whether to be discouraged or just sad at this.

Before I continue, I'd like to also point out that I'm not saying that these tensions are not a weakness; they are simply a fact of Wikipedia's nature. Further, by identifying the existence of these tensions I am not consciously forcing a Hegelian dialectic upon Wikipedian dynamics; rather, I am picturing a statics diagram with opposing forces in an equilibrium. In other words, this is a situation where modifying one force in the tension without sufficient care or information can destroy the existing equilibrium and lead to everything falling apart.

A second tension illustrated in this debate is an old one: the clued versuses the clueless. This tension has existed either explicitly or implicitly since practically the beginning of Wikipedia. Larry Sanger's notorious emphasis on credentials was simply one extreme version of this tendency. Most Wikipedians, whether they are professional experts, serious amateurs, or just individuals with a desire for learning, tend to be elitists; we spent our spare time reading or tinkering in order to understand something better, while everyone else was watching television, playing games, or otherwise frivolously occupied. And we often did this both at our own monetary expense, and at the derision of our peers. So, yes, Wikipedians can be elitist in how they respond to new editors; it's an assumption experienced editors often find themselves fighting against.

In this conflict, over whether to turn this experiment into a full-scale adoption, I see this tension playing out on several levels. One level is the change itself: that new and unregistered editors are somehow less clued than the established ones. While a reasonable assumption, it inadvertently insults those new editors who are knowledgeable, forcing them to run a gauntlet to make even trivial changes of spelling or punctuation. But another, and more serious, level can be seen in how the two camps are discussing this proposal: many of the participants seem from their tone to be one step away from responding to their opposite number with something along the lines of "well, if you were smart and actually knew what was going on, the solution would be obvious to you and you'd agree with me."

And, in my humble opinion, I believe that this elitism is more prevalent on the side which wants to adopt this change and make "Pending changes" permanent.

The problem for anyone not invested heavily in this discussion -- like me -- is that I don't see any proof that this will fix any of the problems it is supposed to fix. No one has provided the evidence showing not only that it will solve the problem of incorrect information being inserted into articles, but to what degree. (This feature can be subverted, some times rather simply. I'll leave that as an exercise for the reader.) And then there is the trade-off of discouraging new editors from further contributions: one of the enticements that is often mentioned by new users is the thrill of seeing one's edit accepted for all to see.

There is the advantage that some articles no longer need to be indefinitely semi-protected or fully protected -- but no one has provided evidence whether applying "Pending changes" to certain contentious articles will work better in protecting content than protecting those articles. However, instead of providing a comparison between these two approaches, those for adopting "Pending changes" simply assert that it is the best approach, and that it should be adopted immediately.

I am all for any change which improves Wikipedia's content, but Wikipedia has grown so large and so complex that I doubt anyone understands how it works comprehensively any more. I used to criticize Jimmy Wales because in many of his responses to its problems it was clear he no longer understood either the community or its dynamics; only those in constant interaction with the community, with editting articles and debating policy would have any chance of knowing. He had been out of touch with the day-to-day activities of Wikipedia for years now. More recently, despite the fact I participate in some way daily on Wikipedia, I've realized that I have grown out of touch. And it is clear that many other core members are too, for people will assume the concensus around a given article or discussion page is identical to its state when they last viewed it, when it may have changed radically in a few months or even weeks.

And here is yet another tension present in Wikipedia: between its constant changing and the need to manage that evolution. Its increasing rate of change, most of the time but not always for the better, has left practically all of its members behind and out of touch. There are no more "clued" Wikipedians -- assuming they existed in the first place. Some may see this as a good thing because it is this uncontrolled evolution which is responsible for Wikipedia's success; but if this true, then why does Wales, a philosophical libertarian, feel compelled to intervene when there is a problem? Wikipedia needs some management, and for management to be successful one needs information.

I like to think that the dynamics of Wikipedia can be controlled, through the same approach which existed in the beginning, through open discussion and building consensuses. It through this discussion and consensus-building that information is shared. The problem is that often, to arrive at a timely decision shortcuts needed to be made. Yes, one last tension that exists in Wikipedia! (I guess now that I've seen a few structural tensions The old shortcuts, however, no longer work; we need to either find and create new ones, or accept the need to laboriously build consensuses at every new change. Because the one thing that will kill Wikipedia is have a single person who makes all of the decisions; Wikipedia has flourished because of its "out of control" nature, not despite it.

The balance of openness that you touch on is similar to a chapter in my (forthcoming book) and I'd appreciate any thoughts you have on it. Also, [[Wikipedia:Pending changes/Metrics/Preliminary Analysis]] shows classes of pages/contributions, including pages that under pending changes had lots of useful anonymous edits. I've also seen stats on such pages that were protected, and hence a productive gain under pending changes.

Seurat, do you care to explain your assertion in further detail? Flinging bumper-sticker messages back & forth has been the crippling weakness of at least this part, if not the entire discussion, over Pending changes.