Thursday, December 15, 2005

Jimmy Wales' Wikipedia comes close to Britannica in terms of the accuracy of its science entries....[A]n expert-led investigation carried out by Nature — the first to use peer review to compare Wikipedia and Britannica's coverage of science — suggests that such high-profile examples [of Wikipedia errors] are the exception rather than the rule.
The exercise revealed numerous errors in both encyclopaedias, but among 42 entries tested, the difference in accuracy was not particularly great: the average science entry in Wikipedia contained around four inaccuracies; Britannica, about three.
Considering how Wikipedia articles are written, that result might seem surprising. A solar physicist could, for example, work on the entry on the Sun, but would have the same status as a contributor without an academic background. Disputes about content are usually resolved by discussion among users.
But Jimmy Wales, co-founder of Wikipedia and president of the encyclopaedia's parent organization, the Wikimedia Foundation of St Petersburg, Florida, says the finding shows the potential of Wikipedia. "I'm pleased," he says. "Our goal is to get to Britannica quality, or better."...In the study, entries were chosen from the websites of Wikipedia and Encyclopaedia Britannica on a broad range of scientific disciplines and sent to a relevant expert for peer review. Each reviewer examined the entry on a single subject from the two encyclopaedias; they were not told which article came from which encyclopaedia. A total of 42 usable reviews were returned out of 50 sent out, and were then examined by Nature's news team.
Only eight serious errors, such as misinterpretations of important concepts, were detected in the pairs of articles reviewed, four from each encyclopaedia. But reviewers also found many factual errors, omissions or misleading statements: 162 and 123 in Wikipedia and Britannica, respectively....[T]o improve Wikipedia, Wales is not so much interested in checking articles with experts as getting them to write the articles in the first place.
As well as comparing the two encyclopaedias, Nature surveyed more than 1,000 Nature authors and found that although more than 70% had heard of Wikipedia and 17% of those consulted it on a weekly basis, less than 10% help to update it.

In its accompanying editorialNature endorses Wikipedia and asks scientists to help it out:

So can Wikipedia move up a gear and match the quality of rival reference works? Imagine the result if it did: a comprehensive, accurate and up-to-date reference work that can be accessed free from Manhattan to rural Mongolia. To achieve this, Wikipedia's administrators will have to tackle everything from future funding problems — the site is maintained by public donations — to doubts about whether enough new contributors can be found to increase the quality of the mushrooming number of entries. That latter point is critical, and here scientists can make a difference.
Judging by a survey of Nature authors, conducted in parallel with the accuracy investigation, only a small percentage of scientists currently contribute to Wikipedia. Yet when they do, they can make a significant difference. Wikipedia's non-expert contributors are, by and large, dedicated to getting things right on the site. But scientists can bring a critical eye to entries on subjects they study, often highlighting errors and misunderstandings that others have unintentionally introduced. They can also start entries on topics that other users may not want to tackle. It is no surprise, for example, that the entry on 'spin density wave' was originated by a physicist....Nature would like to encourage its readers to help. The idea is not to seek a replacement for established sources such as the Encyclopaedia Britannica, but to push forward the grand experiment that is Wikipedia, and to see how much it can improve. Select a topic close to your work and look it up on Wikipedia. If the entry contains errors or important omissions, dive in and help fix them. It need not take too long. And imagine the pay-off: you could be one of the people who helped turn an apparently stupid idea into a free, high-quality global resource.

If you're an expert on a certain topic, then make sure that Wikipedia includes the fruits of your expertise....You may not have a high opinion of Wikipedia, but there are two reasons not to let that stop you. First, it can become a self-fulfilling prophecy. If experts add or enhance articles to reflect their expertise, then Wikipedia will deserve respect to that extent. Second, Wikipedia is an increasingly common first stop, and probably last stop, for non-academic users looking for information. If you want to be visible to non-academic users, then it's an eyeball destination that you can easily join....Don't give up your standards, but don't judge this resource from mere presumptions without firsthand knowledge.

Update. Wikipedia has a page collecting the independent reviews of its accuracy, and the page now includes the Nature study. Nice touch: the page reports that all the errors noted in the Nature study have been tagged and will soon be corrected. Can Encylopedia Britannica do that?

Posted by
Peter Suber at 12/15/2005 08:34:00 AM.

The open access movement:
Putting peer-reviewed scientific and scholarly literature
on the internet. Making it available free of charge and
free of most copyright and licensing restrictions.
Removing the barriers to serious research.