Friday, July 1, 2016

A checklist does not an expert make. (David Travis)

Whatever you call it – usability evaluation, expert evaluation, heuristic evaluation – what it is is a particularly cost- and time-effective way to make some improvements to your system. If you think about it, there’s usually enough low-hanging fruit on any system that having someone who knows a little about usability take a quick look at it is never a bad idea. Personally, I’m a huge fan.

But, how exactly to go about doing one? Well, there are two basic approaches. One is to use a checklist; the other is to use an expert. Often, the first one is what you use when you don’t have the second one around.

Over the years, though, I’ve found that there are some definite issues with that first approach. In particular, I’ve found that checklists are either too vague … or too specific.

The vague ones – like the famous 10 from Jakob Nielsen and Rolf Molich – tend to have way too few guidelines. Though these kind of guidelines may apply to many situations, they are typically pitched at such a high level that anyone who is not an expert may have trouble applying them to the very specific situations that are involved in an actual eval. Using a checklist like this, a non-expert is very likely to get lost in the forest.

The specific ones, on the other hand, tend to have way too many guidelines. Though they can be very specific, that same granularity means that it can be very tedious to go through them all, applying them to each screen and interaction in a real system. They also may miss things that involve multiple elements or that occur at a higher level than pixels and code. Using a checklist like this, a non-expert is very likely to get lost in the trees.

So, a true expert review is usually my go-to for these things. Note that that usually means expert in usability and in the particular subject matter domain as well. And all that comes about from running tests, analyzing data, doing interviews and focus groups, and all the many things that separate someone who knows what they’re doing from someone who's just giving it a shot.

David Travis is a consultant based in the UK(and has some other great ideas about evals in this blog post)