Wednesday 12 August 2009 08.15 EDT
First published on Wednesday 12 August 2009 08.15 EDT

Yet again, Wikipedia is about to break new ground. The website that has become one of the biggest open repositories of knowledge is due – within the next week or so – to hit the mark of 3m articles in English.

It's all a very long way from January 2001, when Wikipedia launched. Its first million articles took five years to put together, but the second was achieved by 2007. It was not just the number of articles that grew, but also the number of people involved in creating them. During Wikipedia's first burst of activity between 2004 and 2007, the number of active users on the site rocketed from just a few thousand to more than 300,000.

Learning curve

However, statistics released by the site's analytics team suggest Wikipedia's explosive growth is all but finished. The quickening pace that helped the site reach the 2m article milestone just 17 months after breaking the 1m barrier suddenly evaporated: adding the next million has taken nearly two years. While the encyclopedia is still growing overall, the number of articles being added has reduced from an average of 2,200 a day in July 2007 to around 1,300 today.

Elsewhere, the number of active Wikipedians (those contributing to the site in some way) now comes in at just under 500,000. That is a 61% increase in the past two years; hardly shabby, but nowhere near the increases seen in the past. At the same time, however, the base of highly active editors (who contribute new words to the project and marshall the billions of pieces of information the site contains) has remained more or less static.

From the numbers, it looks as though Wikipedia is stagnating. Why?

One of those who has spent his time studying what happens on Wikipedia is Ed H Chi, a scientist who works at the Palo Alto Research Center (Parc) in California. His team, the Augmented Social Cognition group, wanted to understand what was happening on the website in order to build better collaborative software.

"For a long time, the understood model for all kinds of large knowledge systems on the web was that they grow exponentially," he says. "The accepted explanation was that the rich get richer – things that receive a lot of attention end up getting a lot more attention."

Wikipedia fitted that model perfectly in its early days. However, when Chi and his colleagues looked at the recent data, they realised this approach did not fit any more. But with a site as complex and sprawling as Wikipedia, simply crunching the numbers proved a major task in itself.

First they spent a significant amount of time downloading a carbon copy of Wikipedia: every article, every edit and every piece of information ever to cross the site's servers. Even when compressed, the files stretched to an enormous 8 terabytes – the equivalent of more than 1,200 DVDs stuffed with information. Decompressing in preparation for analysis took almost a week. But when the group fed the data into their 60-machine computing cluster, they got some surprising results.

Chi's team discovered that the way the site operated had changed significantly from the early days, when it ran an open-door policy that allowed in anyone with the time and energy to dedicate to the project. Today, they discovered, a stable group of high-level editors has become increasingly responsible for controlling the encyclopedia, while casual contributors and editors are falling away. Wikipedia – often touted as the bastion of open knowledge online – has become, in Chi's words, "a more exclusive place".

One of the measures the Parc team looked at was how often a user's edit succeeds in sticking. "We found that if you were an elite editor, the chance of your edit being reverted was something in the order of 1% – and that's been very consistent over time from around 2003 or 2004," he says.

Meanwhile, for those who did not invest vast amounts of time in editing, the experience was very different. "For editors that make between two and nine edits a month, the percentage of their edits being reverted had gone from 5% in 2004 all the way up to about 15% by October 2008. And the 'onesies' – people who only make one edit a month – their edits are now being reverted at a 25% rate," Chi explains.

In other words, a change by a casual editor is more likely than ever to be overturned, while changes by the elite are rarely questioned. "To power users it feels like Wikipedia operates in the way it always has – but for the newcomers or the occasional users, they feel like the resistance in the community has definitely changed."

While Chi points out that this does not necessarily imply causation, he suggests it is concrete evidence to back up what many people have been saying: that it is increasingly difficult to enjoy contributing to Wikipedia unless you are part of the site's inner core of editors.

Include me out

One person who typifies that feeling is Aaron Swartz, a 22-year-old programmer who lives in Cambridge, Massachusetts. Something of a wunderkind in the software development world, Swartz used to spend a lot of time working on Wikipedia – in 2006 he even stood for election to the Wikimedia Foundation, the organisation behind the site (his bid failed). These days, however, he rarely checks in.

"I used to be one of the top editors …now I contribute things here and there where I see something wrong." The reason, he explains, is that the site feels more insular and exclusive than in the past. "In general, the biggest problem I have with the editors is their attitude," he says. "They say: 'We're not going to explain how we make decisions, we basically talk amongst ourselves.'

"There's no place on Wikipedia that says: 'Want to become a Wikipedia editor? Here's how you do it.' Instead, you basically have to really become part of that community and pick it up through osmosis and have the tradition passed down to you."

Swartz's experience certainly correlates with the figures unearthed by Parc, even if his attitude is not shared by everyone.

Given the history of the online world – where escalating growth can continue for years – it seems unlikely that this gradual slowdown was inevitable. Instead, it could be the end result of a battle between two competing factions of Wikipedia editors.

On one side stand the deletionists, whose motto is "Wikipedia is not a junkyard"; on the other, the inclusionists, who argue that "Wikipedia is not paper".

Deletionists argue for a tightly controlled and well-written encyclopedia that provides valuable information on topics of widespread interest. Why should editors waste time on articles about fly-by-night celebrities or wilfully obscure topics? Inclusionists, on the other hand, believe that the more articles the site has, the better: if they are poorly referenced or badly written, they can be improved – and any article is better than nothing. After all, they say, there is no limit to the size of the site, and no limit to the information that people may want.

Less is more?

The two groups had been vying for control from early on in the site's life, but the numbers suggest that the deletionists may have won. The increasing difficulty of making a successful edit; the exclusion of casual users; slower growth – all are hallmarks of the deletionist approach.

Swartz, an avowed inclusionist, says the deletionists have won – but says he understands their motivation. "When Wikipedia is in the news, it's always because someone found this inaccuracy, or somebody's suing Wikipedia … It's always about how Wikipedia screwed up. So of course what they're going to be worried about is not how to make Wikipedia grow and have more content, it's about how we keep Wikipedia out of trouble and how we stop people from messing it up."

Still, there remain unanswered questions. Could its growth ever halt completely? How big will the site be when the editors decide that the sum of human knowledge is catalogued? Could a new website take Wikipedia's place by toeing an inclusionist line?

Parc's research doesn't give any answers, but Chi has identified one model that Wikipedia's growth pattern matches. "In my experience, the only thing we've seen these growth patterns [in] before is in population growth studies – where there's some sort of resource constraint that results in this model." The site, he suggests, is becoming like a community where resources have started to run out. "As you run out of food, people start competing for that food, and that results in a slowdown in population growth and means that the stronger, more well-adapted part of the population starts to have more power."