Saturday - September 04, 2004 at 04:31 PM in

How Authoritative is Wikipedia

Wikipedia is a free, collaborative online
encyclopedia written and edited through the volunteer efforts of literally
thousands of people. The unique feature of Wikipedia (or any wiki)
is that anyone can contribute or edit the content. Software features make it
easy to identify gross acts of vandalism and revert articles to earlier, known
good, states. The result is an amazing resource, in the depth and breadth of
the
content.

But how authoritative is it? In other words, how
sure can you be that information in the Wikipedia is true and
complete?

Traditional encyclopedias
employ armies of fact-checkers to go over every entry and make sure the
encyclopedia is as accurate as it can be. This makes a traditional encyclopedia
an authoritative source, and when you look something up in Britannica, you are
relying on the reputation of the publisher that everything has been
verified.

Wikipedia has no such
resource, in any literal sense. However, the "anyone can edit" philosophy means
that every reader has the ability to become a fact-checker, too, giving
Wikipedia an army of potentially millions of fact-checkers. At least that's the
theory.

Wikipedia !=
Authoritative?Recently, this articlecaused a bit of a stir from its casual dismissal of the
wiki model, and even the implication that Wikipedia was perhaps being deceptive
by appearing too authoritative. Some suggested an experiment: insert some
mistakes into Wikipedia, and see how long it lasts. Alex Halavais actually
performed the experiment, and found that all his errors
were removed within hours.

Actually,
Alex wasn't the only one to think of performing this experiment. Roughly the
same idea occurred to me at roughly the same time, but my experiment only ended
today.

The Wiki
ExperimentThe goal was to try to get
some insight into this
question:

To what extent can
you trust the information in
Wikipedia?

I probed the question by
inserting mistakes into Wikipedia, and seeing how long they took to be
corrected. But before doing that, I wanted to make sure I inserted the right
kind of mistakes: mistakes similar to ones which might sneak into Wikipedia
either accidentally or on purpose under realistic
conditions.

Wikipedia has already
proven itself to be good at recovering from intentional vandalism and edit
wars, the two most blatant forms of intentional degradation of
Wikipedia. Wikipedia makes it easy to track all changes made by a particular
person and undo them all, so if someone is out destroying pages, the damage
doesn't last long [Note: I suspect this
is why Alex's changes were all undone so quickly. Someone probably noticed that
he had made a bunch of changes, and at least one of them was obviously wrong, so
that person simply went back and reverted all of Alex's
mistakes].

ProcedureRather
than looking like a vandal, I wanted to be more subtle. It isn't practical for
me to change my IP address without seeking out a public access point (too much
work!), so I decided to simply make one change a day for several days. That
way, it wouldn't look like a mad editor, but a casual Wikipedia browser making
occasional contributions.

In addition,
I decided that adding details--rather than deleting or changing facts already in
Wikipedia--would look more innocent. Finally, everything had to sound
reasonable, but be factually wrong upon further
research.

So, all my changes would take
the form of plausible-sounding but wrong facts inserted into existing Wikipedia
pages.

ResultsI
made five changes between August 30 and September 3. Not one of the changes was
removed by September 4th, when I reverted them myself. Every change was in
Wikipedia for at least 20 hours, and the longest was in for five
days.

The changes
were:

Layzie
Bone (biographical page). I inserted "born 1973", but a quick Google
search reveals that he was born in 1977.

Magni,
from norse mythology. I said that he was commonly depicted wielding an axe or a
spear. In fact, Magni was the only person other than Thor himself who could
lift Thor's hammer, and Magni is commonly associated
with that weapon. Interestingly, the fact about Thor's Hammer is in the
Wikipedia entry (though they call it by the proper name, Mjollnir), yet nobody seemed to notice the
incongruity that a god whose special power is lifting a hammer would be depicted
with an axe or a spear.

Empuries, a Mediterranean town, I made the site
of sadly lost Greek ruins. The Greek ruins are true enough, but they aren't
lost, sadly or otherwise. This travel site helpfully informs us that
Empuries has "lots of free parking close to the ruins" as well as a cafe and a
museum at the archeological site.

Philipsburg, PA, became located at the junction
of U.S. highway 233 and state route 503. Not U.S. highway 322 and state route
504, as most maps
show.

Bernice Johnson Reagon, while apparently a
prolific author, never wrote Georgia in
Song. In fact, Amazon lists no such book by
any author.

(Note: because I undid all
my changes, you will have to back to the "history" tab for each page to see the
pieces of misinformation. All my changes were from the IP address
65.27.75.56.)

Conclusion and
SuggestionsI was disappointed that all
my changes in Wikipedia went unchallenged. Surely a week was plenty of time,
especially since fresh changes tend to get more scrutiny than old ones. I have
to conclude that it would be very easy for subtle mistakes to sneak into
Wikipedia, and go a very long time without being
corrected.

Unfortunately, it is in
getting these details right that an authoritative source is the most
valuable.

One way to solve this
weakness is to create a formal fact-checking mechanism. In Wikipedia,
contributions of new material are certainly valuable, but fact checking is even
more important. Perhaps each edit could be somehow marked as "unverified" until
a second contributor is willing to vouch for its accuracy. While that wouldn't
be a perfect solution, it would at least make the hurdle for misinformation or
mistakes higher.

Other
ExperimentsWikipedia isn't really a
fact-checking mechanism so much as a voting mechanism. If someone reads an
entry, unless something sounds blatantly false, he or she will likely accept
what it says. If there is disagreement about the facts, an edit war could break
out until a consensus view
develops.

Given that, there are some
other experiments which would be interesting to
perform:

Believe It Or
Not: The opposite of inserting plausible
mistakes, try inserting a series of true facts which sound implausible. See if
they get edited out.

Whole
Cloth: There are a number of missing pages and
stubs in Wikipedia, pages which have placeholders but no entry. Try creating an
elaborate hoax out of whole cloth (but not too implausible). See if it gets
corrected or deleted.

Good
Guy, Bad Edit: Some edits appear safer than
others. For example, someone who signs up with an account (rather than
anonymous), and carefully notates the reasons for edits, may have greater social
currency and thus be able to insert bigger whoppers into articles.