New offering from NN/g

Durability of Usability Guidelines

Summary: About 90% of usability guidelines from 1986 are still valid, though several guidelines are less important because they relate to design elements that are rarely used today.

The usability field is getting on in age, and we now have the cumulative wisdom of more than twenty years of user research at our disposal. Some usability enemies like to dismiss established findings, claiming, "they may have been true once, but not any more." Because the computer field does tend to move quickly, it's reasonable to ask whether old usability insights have indeed become obsolete.

From 1984 to 1986, the U.S. Air Force compiled existing usability knowledge into a single, well-organized set of guidelines for its user interface designers. I was one of several people who advised the project (in a small way), and thus received a copy of the final 478-page book in August 1986.

20-Year-Old Guidelines — Past Their Expiration Date?

The 944 guidelines related to military command and control systems built in the 1970s and early 1980s; most used mainframe technology. You might think that these old findings would be completely irrelevant to today's user interface designers. If so, you'd be wrong.

I decided to use the 1986 report to assess the longevity of usability work. Because reassessing all 944 guidelines would require too much effort, I took a shortcut, reviewing ten guidelines from each of the report's six sections, for a total sample of 60 guidelines. (A sidebar reprints these 60 guidelines, so you can judge them for yourself.)

Of those 60 usability guidelines, 54 continue to be valid today. In other words, 90% of the old guidelines are still correct.

What's Changed

Ten percent of the guidelines would have to be retracted or reconsidered for today's world. But even these questionable guidelines are at least partly correct in most cases. In fact, I would deem only two guidelines (3%) completely wrong and harmful to usability if followed.

Guideline 4.2.6 said to provide a unique identification for each display in a consistent location at the top of the display frame. This guideline worked well in the target domain of mainframes: Users typically navigated only a few screens, and having a unique ID let them understand their current location. The IDs also made it easy for manuals and help features to refer to specific screens.

Today, screen identifiers would clutter the screens with irrelevant information. They would not help modern users, who move freely among numerous locations.

Even this invalidated guideline continues to contain a core of truth: it's good for users to know where they are and what they can do on each screen. The current recommendation is to provide a headline or title that concisely summarizes each screen's purpose.

Guideline 3.1.4.13 said to assign a single function key to any continuously available feature. This made sense for mainframe interfaces because they relied extensively on function keys to speed up the interaction. Also, mainframe systems were so heavily moded that very few functions were available across all system areas; the few that were obviously deserved special treatment.

Modern systems attempt to be modeless, so many features have become ubiquitous and accessible from anywhere. Furthermore, function keys are no longer the primary way of operating computers. Given these two changes, it no longer makes sense to assign function keys to constantly available features.

In addition to the invalid guidelines, twenty percent of guidelines are essentially irrelevant today because they relate to rarely used interface technologies.

For example, guideline 1.4.13 discussed how to overtype the field markers (typically underscores) that mainframe systems used to indicate where users could type their input. Today, input fields are almost always denoted by text entry boxes, so knowing how to improve a field marker's usability is largely irrelevant.

What's Still Valid

Of the 944 guidelines from 1986, 70% percent continue to be both correct and relevant today. There is much good advice, for example, on dealing with entry fields and labels on online forms, which have changed little from the dominant mainframe designs of the 1970s.

The guidelines on using business graphics to display different types of data are also highly relevant today. In our recent studies of how investors and financial analysts use the investor relations area of corporate websites, we found many usability problems related to overly complex charts. Following twenty-year-old guidelines for charting numbers would have improved many IR sites considerably.

The guidelines for error messages, system feedback, and login also hold up. It was interesting to see that guideline 6.2.1 recommended single sign-on. In our intranet usability measurements, we found that login difficulties (mainly due to multiple sign-in requirements) accounted for the second-largest difference in employee productivity between intranets with good usability and those with poor usability. (Search usability constituted the biggest difference between good and bad intranets.)

Too bad most systems still lack the login interface recommended in 1986.

Luckily, other aspects of user interface design have advanced. Several of the guidelines for messaging interfaces continue to be both valid and relevant, but are hardly revolutionary today: almost all email systems follow them to the letter. This is an example of usability insights becoming so firmly entrenched that they change from "best practices" to "the way things are always done." If anything, it's sad that so few of the findings from 1986 have made this transition.

Why Usability Guidelines Endure

You would be hard-pressed to find any other Air Force technical manual from 1986 that's 70% correct and relevant today. Whether for pilots, airplane engineers, or programmers, general lessons of the past might continue to apply, but the specific guidelines changed long ago.

Usability guidelines endure because they depend on human behavior, which changes very slowly, if at all. What was difficult for users twenty years ago continues to be difficult today. People can only remember so many things, and we don't get any smarter.

I recently analyzed my own old guidelines for Web usability, as published on the Alertbox and elsewhere in the early days of the Web. Of those early guidelines, 80% continue to be valid and relevant. Of course, my early guidelines are only ten to eleven years old, so it's hardly surprising that they'd score better than twenty-year-old guidelines.

Usability guidelines mainly become obsolete when they're tightly bound to specific technologies. For example, neither the 1986 field marker guidelines nor my 1995 guideline on making hypertext links blue held up. (More recent guidelines for link colors offer updated recommendations.) However, the corresponding underlying usability principles do hold: ensure that users know what they can do and that they can recognize actionable user interface elements.

Posterity vs. the Present

The more permanent guidelines tend to be those that are the most abstracted from technology. I'm nonetheless under strong pressure from readers to make usability guidelines as specific as possible so that they're easier to apply. In resolving the tension between eternal truth and short-term application, it's often tempting to err on the side of serving current rather than future readers.

The lure of the present is especially strong when writing for the Web. In writing a book, I'm highly conscious of people who will be reading my text ten or more years into the future. But when posting to my website, I tend to write for today's readers, even though 80% of the pageviews will occur after an article has passed into the archives. Luckily, most of my old analyses hold up pretty well, and ten-year-old articles continue to be 78% relevant.

However seductive the present might be, writing for the Web is writing for the ages, not just for the moment. (People who post stream-of-consciousness entries in their weblogs, for example, might want to consider that they're also writing for managers who might hire them in twenty years.)

Usability guidelines have proven highly durable, and most hold true over time. Present-day designers should not dismiss old findings because of their age.