March 19, 2010

It's a defect … it's a feature … it's an experience bug

On the software and web projects for which I've done user experience design, I've been fortunate to work with thorough and thoughtful quality assurance (QA) engineers. Frequently these folks will enter tickets for uncertain issues, such as "I expected something to happen, but instead something else did. Is that expected behavior?" In other words, is that a bug or a feature?

There are three typical characteristics of what I call experience bugs. First is the uncertainty in whether an issue must be fixed. Frequently the behavior is as designed, but it just doesn't make sense to the quality assurance engineer. It could be nothing, or it could be a problem that no one noticed before now.

Second is uncertainty in what the fix ought to be. With other software defects, frequently the nature of the fix is self-evident. In a complex system, however, gaps in the user experience many be unanticipated until late in the development process. There might be no fix possible, but often some lateral thinking can save the day.

Third, the call on how to fix something starts with you, the user experience lead. That's because experience bugs require a design solution. The big gotcha here is making the first, quick recommendation that comes to mind, because you're feeling launch deadline pressure. Watch out, because a small fix to one part of the experience can ramify. Instead, test your first idea by returning to your design documentation, talking it through with a coworker, or just walking through the new experience on paper. Think back to a past situation that applies, and run a mental reality check to make sure the solution won't cause problems in other parts of the experience.

It's great to have QA involved and have experience tickets in the bug tracking database. You can, and should, enter them yourself whenever you discover an inconsistency or when a stakeholder has a question about the UI that makes you doubt. But your responsibility doesn't end there. If it's a significant issue, it's critical to get the attention of the development manager, product manger, or whoever's in charge of prioritizing fixes. Experience bug descriptions can be difficult to follow, and likely there are many bugs competing for priority. You may need to telephone or sit down with decisionmakers and help them understand a bug's importance.

You can forestall false experience bugs by supplying QA with all possible information about the final design. A QA engineer's background knowledge about the experience is mostly up to you. If you have access to your engineers before the QA phase of the project, you can walk them through your prototype, specifications, or wireframes to impart that background knowledge. If not, you may be able to capitalize on their first false experience bug as a teachable moment.

What have you learned from trying to fix UX problems in a project's QA phase?