That was the lesson hundreds of Web developers came away with after User Interface Engineering analyst Jared
Spool yesterday presented findings on a Web design and information
retrieval study at the Web Builder
conference in San Francisco.

In a test of ten Web sites conducted last year, each with at least 3,000
pages, study participants rated a site that reportedly cost $300 million
the worst, and a $10,000 site the best.

The bad news: When it came to information retrieval, even the best of the
Web sites were bad. On a scale of one to ten, none rated higher than a 4.5.

The nine sites surveyed included those of industry heavyweights Hewlett-Packard, Disney (which produced the site
that testers said cost $300 million, a figure Disney disputes), and NEWS.COM
publisher and Web Builder conference organizer CNET.

The trouble with these popular destinations on the information
superhighway, according to the study, was that they make it too difficult
to find information.

More coverage on CNET Radio

Researchers tested more than 50 users--all of whom had at least some Web
experience--by asking them to find specific information at particular Web
sites in what Spool termed an "online scavenger hunt." The participants'
activities were monitored for three hours. The users encountered daunting
stumbling blocks.

One problem was that links were not sufficiently descriptive. In an
example from the Disney site, a text link reading 101 Dalmatians
under the subject heading "Movies" might have led to anything from a
description of the movie to a lengthy download of the movie itself.

Disney noted that its site had undergone extensive renovations since the
study. The company also disputed Spool's contention that the company's Web site
had cost $300 million.

"That number is definitely grossly overstated," said Disney spokesperson
Diane Passarelli. But Passarelli declined to disclose how much money Disney
had spent on the site and its relaunches, or how far off the UIE number
was.

Spool later defended the figure, which he characterized as an estimate
based on the reports of engineers who worked on the project and on the
public statements of Disney executives.

In contrast to Disney's multimillion-dollar site, the $10,000 site for Edmunds's Automobile Buyer's
Guides, which also has been redesigned since the study was conducted,
offered unambiguously named links such as "Click here for Car Prices!"

Even if users can't find what they're looking for through well-labeled
links, many companies offer site searching as a shortcut to information.
But the study found that site searching was not only useless but also
detrimental for information gathering: users were 50 percent more likely to
find what they were looking for if they never hit the search button.

Part of the trouble with site searching was that some sites had multiple search
engines that were insufficiently differentiated from one another.

Another problem was that search results were long lists of titles and file
names that did little to describe their documents' contents.

Web designers aiming for clarity with lots of white space and sparse text
are defeating their purposes, according to the report. In what might seem
a paradoxical result, study users said more white space caused sites to be
too complicated, over-detailed, visually confusing, unclear, and "not
enticing." Mistrustful of the results, the researchers
tested the effects of white space five different ways, only to come up with
similar results.

Another surprising finding: users find that the less "readable" a page is,
the more authoritative, clear, and useful it is.

The reason is that readability is concerned with the structure of
sentences, the length of words, and other measures of good English prose.
But Web surfers don't read, concluded the researchers; instead, they skim,
making traditional measures of readability irrelevant to Web information
retrieval.

Another surprising finding for some Web designers was that users found animation and movement irritating, sometimes to the point that they would cover up the offending .gif with their hand.

This observation seemed to clash with studies showing that animation results in twice the click-through rate for Web banners. But the apparent contradiction stems from two disparate groups of Web users: those looking for specific information and those randomly surfing.

"Surfing and information retrieval are two different things," Spool noted. "You need to know what you are designing for."

Web developers agreed.

"This is all pretty common sense advice," said Teri Olsen, Web coordinator
for the University of Utah. "It
reinforced stuff that those of us who aren't doing cutting-edge design
already know, in terms of things like descriptive links and not using
animated .gifs."

Olsen did encounter some surprises, however.

"It was very interesting to see how the average user responded to white
space and site searching," Olsen said. "I'm definitely going to go back and
take a look at how search is serving our users."

Another Web developer applauded the study, but said that in testing average
users it wasn't necessarily measuring the Web's ultimate audience.

"A lot of what the study found is partly a measure of the youth of the
medium," said Scott Jarol, senior project manager for MediaLive'sSurfMonkey site. "When people get
used to the concept of hierarchical categories, some of the things he said
were obstacles will turn out to be beneficial."