Last month I was fortunate enough to present a webinar with Seth Maislin of Earley and Associates and our very own Peter Eckert. If you missed it, you can still access it on the Earley web site (you will need to create a user account).

The overarching topic of the webinar was quick and dirty usability techniques. Registrants were invited to respond to a survey prior to the webinar about their use of some common user feedback techniques. Approximately 25% of the 400 people who registered participated in the survey.

Although it was a fairly small sample, an apparent trend was revealed. Most of the techniques folks said they are using regularly on all of their projects involved subjective opinion gathering: interviews, focus groups, and surveys. None of these techniques involved looking at anyone using an actual product! Less than 10% of the respondents were regularly employing techniques like usability testing, contextual inquiry, and cognitive walkthroughs that involve watching people interacting with a system either live or prototyped.

Why is this? Is it a cost issue? Do companies think they simply can’t afford to actually test their ideas? Gathering opinions is fine, but they are just that: opinions. And basing the design of a complex system on opinions without testing the usability is risky at best and sometimes downright foolish! What users say and what they do are two very different things.

The only true way to understand the usability of your system is to watch people use it. Watching users doesn’t have to be a costly undertaking either. You can gain huge insight from observing just a few users, and at a minimum eliminate usability problems early on, which inevitably will save you money in the long run.

Don’t stop talking to your users, but make sure you watch them using your product too!