Usability Tests in a Nutshell, Part 2: Recruiting Users

Many of our clients come to us requesting usability testing consulting services. One of the reasons they reach out to us is because they think usability testing must be a complex and scientific process. As a result, they’d prefer to have an outside consulting company conduct their tests.

The first thing I tell people who contact me is that usability testing is not a complicated process. It’s a technique that anyone can learn with training and lots of practice. In this series of articles, I’ll share the essential steps when conducting your first usability tests. In part 1 of the series, I shared the steps for planning your usability tests. In this article, I’ll discuss best practices for recruiting your users.

When considering your first usability tests, you’ll want to find people who are representative of your target audience. You have a couple of options for doing this. With in-house recruitment, you can assign someone internally to find your users. Or, you can outsource to a reputable recruitment agency. We use both approaches at Perfetti Media and they’ve worked well for us. Usability Works and AlphaBuzz are two of the organizations we recommend for their recruitment services.

When thinking about who to recruit for usability tests, many teams start by focusing on demographics, such as age, gender, or ethnicity. Unfortunately, in most cases, recruiting for demographics will be one of the least effective ways to find the most appropriate users for your tests.

For example, if you were recruiting for a usability study of a video game, what demographic would you usually think of? People who play video games are typically thought of as boys in the 13-24 age range. But if we only recruited for that demographic, we may mistakenly recruit boys who don’t actually play video games. Additionally, many women also play video games. We would risk missing their feedback if we only recruited the boys.

At Perfetti Media, we only focus on demographics when it’s a critical component of the target audience, such as when we’re recruiting for a a university portal devoted to college students or a social security benefits web site devoted to elderly users. Rather than focusing on demographics, we focus on the specific behaviors the target audience exhibits and their levels of tool knowledge and domain knowledge.

For any recruitment project, we work with the client to answer the following questions:

What are all of the specific behaviors we’re looking for in our users?

What level of tool knowledge do users need?

What level of domain knowledge do users need?

The first question we ask our clients is, “What our the key behaviors of your audience?” For example, if we are recruiting for a video game system, we’d want to find people who play video games. If we are recruiting for an e-commerce site, we’d want to find people who use the internet and have bought products online before.

Users’ behaviors are also informed by their amount of tool and domain knowledge. The users’ tool knowledge, the level of familiarity users have working with particular tools or products, can have a huge impact on how they behave. We recruit for tool knowledge when the test includes intermediate or advanced features and techniques. For example, if we are recruiting for a video game, we may need to find people who already know how to play with a specific game console, such as the wii.

We recruit for domain experience when specific information is required to use the product. For example, in our recruitment for a video game, if we were testing the adventure game, Myst, we would want to find people who are already familiar with the game and how it’s played.

When finding users, we start by asking clients if they have a database with existing users and prospects. This is always a great place to start. We’ve also had a lot of success with recruitment firms, Craigslist, user groups, temp agencies, real estate agencies, and even friends and family.

Once you’ve found some potential recruits, it’s necessary to screen the people by phone before bringing them into the lab. For our projects, we develop a screener to help us qualify or disqualify potential users. The screener is a script that helps the recruiter apply the requirements we’re seeking in users. The parts of a screener script include:

Greeting and the purpose of the study

Qualifying questions to assess whether we’ve found a user who exhibits the right behaviors