User Research for Taxonomy Part 5: Interview Like an Expert

The following is an excerpt from an ebook I wrote with Rachel Price, User Research for Taxonomy Design. We wrote it as a companion piece to a presentation we gave at Taxonomy Bootcamp 2016, because we think there’s a lack of good information out there about generative qualitative research for taxonomy. In our book, we break it down into simple steps so you can do your own research.

Expert interviewing is an art, and people spend careers perfecting it, but it’s relatively easy to become good enough at it to get useful information. The most important thing, of course, is just to do it. Practicing is the best way to learn, but we also have some guidelines.

Start off right

It doesn’t matter whether you feel nervous; when you’re interviewing, you’re 100% invested in the person you’re talking to. The introduction is by far the most talking you’ll do, and it’s the time to set the tone and make your participant comfortable. Start the interview off by being very clear and reassuring about what you’re doing, because not everybody understands the idea of user research. We always make sure to say:

I’m not here to evaluate you; you can’t make a mistake.

I didn’t build any of this; you won’t hurt my feelings.

This will be confidential; nothing you say will be attributable to you.

I’m going to record this for my notes; is that okay?

Less talking, more listening

Let your participant lead the conversation. Even if they’re veering into a digression, gently steer them back toward what you’re interested in via follow up questions. The key to good data is to keep people talking, and the key to keeping them talking is to make them feel comfortable. They can’t feel comfortable if they feel wrong because you abruptly changed the subject.

Don’t interrupt your participant, even if you think you’re being helpful. If your background is in librarianship, as ours is, it can be excruciating to not help someone who’s struggling or mistaken. Be strong and resist! You’ll learn much more from what they think is going on and how they think it should work than you will from teaching them.

Play the know-nothing. There’s an old trope in comedy teams, where it’s one comedian’s job to not know anything about the situation, just ask questions and react to the answers (the classic Abbott & Costello sketch, “Who’s on First,” is one of these). As an interviewer, you need your participant to explain everything. No, you’ve never used that website before. Wow, is that how you search for something? The more you let them tell you, the more you’ll learn.

Embrace awkward silences. People tend to speak in paragraphs, and if you speak up every time there’s a silence, you’re disrupting their line of thought, and missing out on some valuable insights. We like to leave about ten seconds of quiet before we move them onto the next question.

When you do talk, always ask people to expand, rather than assuming you know what they mean. Our personal favorite phrases to do that are, “Tell me more about that…,” “Unpack that for me…,” and “Tell me what you mean by that.”

As they answer your questions, get your participants to be specific. People will often want to tell you what usually happens, or how it ought to go. The problem with this is that people tend to smooth off the rough edges when they make generalizations, so they’re likely giving you a more logical version of events than is realistic. Get them to show you exactly how they did it last time, or how they’d handle a similar situation again.

In person or remote?

You can get great data from both in-person and remote studies, and which you choose depends on where you and your your users are. In-person user research tends to be thought of as the best option, if you can afford it, and it’s certainly a good option, but don’t discount remote research. On most large projects we do, we choose to do remote research, but you can choose the best option for your needs.

In-person studies:

Pros

You get to see your participants’ facial expressions and develop a rapport with them.

You can observe participants using mobile devices or no devices at all.

Maintaining a conversation is much easier, since you can observe your participant’s body language.

Cons

You need to set up a lab for them to come to or determine how to record in the field.

You’re limited to people who will take time out of their day during working hours, usually.

Any note-taking or observing is much more conspicuous and can make participants feel self-conscious.

Remote studies:

Pros

You can talk to participants anywhere in the world, in their natural context.

You can observe them working on their own equipment and record it more easily.

Usually cheaper and requires less investment in a setup.

Recordings are often easier and higher quality, with less equipment.

Cons

It can be harder to develop a rapport with your participant.

You have to rely on voice cues for how your participant is feeling, which is often difficult.

Technology can be fickle, and interviews can be derailed by connectivity issues, software issues, or recording glitches.

You will only get fairly technology-savvy participants, who own devices and are able to screenshare on them.

Whatever you want participants to interact with needs to be generally available on the internet, which may be impossible for sensitive materials.

Keeping records

No matter how small or informal the study, we recommend capturing the data both as a recording and with notes. Use the notes to do your analysis and come up with your findings and the recordings to supplement your findings with evidence. If it’s at all possible, have another person observe the interview and take notes. It’s much better than doing it all yourself, not only because you get to focus on the participant instead of note taking, but also because they’ll likely notice things you won’t and can contribute to the analysis (discussed in Part 6).

If you can’t have a dedicated note taker, you can take notes for yourself, either in the moment or from a recording. Even if you’re planning to return to the recording later, jot down the big things you notice as the participant talks; it’s hard to keep that fresh perspective after the fact. If you’re interviewing people in person, take notes by hand, as keys clacking and having a screen between you draws too much attention to the fact that the participant is being observed and can make them uncomfortable.

For in-person studies, a smartphone makes an excellent audio recorder. We’ve used the recording function in Evernote very successfully. You’ll usually want participants to show you how they do things, which is a challenge for in-person non-lab-based interviews, and might just require extra-good notes.

For remote studies, most meeting technologies like GoToMeeting, WebEx, and Join.me will let you do the basics of letting a participant share their screen with you and recording it. We’ve explored fancier, research-focused technologies for years and still return to GoToMeeting for sheer convenience.

What makes good notes?

When you’re taking notes on an interview, you want to focus on three key things:

Detailed observations of what the participant actually does. What search terms do they use? Where do they go? Let them show you, not tell you, and look for the implications.

Quotes of what they say, or “verbatims.” Get these as precisely as you can, because evidence in a user’s own words is extremely valuable. These are some of the best notes to sort in a thematic analysis, also.

Larger patterns or themes you notice. These can be paraphrased based on things they say (“Wants to see all possible things at once”), things they do, (“Always searches first”), or things they don’t do (“Doesn’t realize the facets are there.”)

People speak quickly, so focus on getting it all down as efficiently as possible, and don’t worry about typos. The basics of a notetaking document are just:

The participant ID. This can be a number (like we have here) or the participant’s first name. If you’re working with internal participants, anonymizing them as much as possible is a best practice, but you want to be able to track what came from which participant.

The note itself. What’s the observation? Focus on the three types of notes above.

That’s it. If you want to get more detailed, separate out topics or questions and their answers, like we have here. It’s a tiny bit of extra work in the moment for the note taker, but it makes doing a thematic analysis later much easier.