A guide to using screening questions in your survey

We’ll walk you through everything you need to know about screening questions—from the benefits they offer your survey to the best practices in writing them. You can skip to whatever part you’d like, but if you want a brief refresher on screening questions, check out our first section!

01Overview

A review of screening questions

Screening questions (also known as "screeners") either qualify or disqualify respondents from taking your survey—depending on how they answer. They let you decide who takes your survey based on the target audience you want to hear from.

To help visualize what this looks like on an actual survey, let’s review an example:

Imagine you work at a pet food company that offers food for dogs.

Say you want feedback on a new product that’s made for small and medium-sized dogs. This means you only want to survey small and medium dog owners. You can ask the following survey-screening question:

“How would you describe the size of your dog?”

Toy - up to 12 pounds

Small - 13 to 25 pounds

Medium - 26 to 50 pounds

Large - 51 to 100 pounds

Extra Large - over 100 pounds

Respondents who answer "Large" or "Extra Large" aren’t a good match for your business. So you’d disqualify them from your survey while the rest of your respondents go on to the next page.

Now that you know what screening questions are and how they can look, let’s explore the different reasons for using them.

LogoGoldie

Listen to our webinar to learn more

Learn how to use screening questions to reach your ideal market research respondents.

Why screening questions improve your survey

For example, if you want to survey pet owners who’ve purchased pet food in the past 6 months, you’d ask a qualifying question around the last time they purchased pet food.

Also, if you’re sending a survey through a market research panel, like SurveyMonkey Audience, some of the exact targeting options you're looking for may not be available. Screening question(s) can help bridge the gap and confirm that respondents meet all of your criteria.

2) It reduces your survey costs.

When you pay for a panel of respondents, you’ll be charged, in part, by the number of people who take your survey.

A screening question filters out respondents whose responses are relatively less valuable while helping you receive responses from those you value most. As you aren’t charged for disqualified responses when using SurveyMonkey Audience, you’re getting the best bang for your buck.

3) It helps you analyze the response data, faster.

Receiving responses directly from your target audiences saves you from performing extra steps to filter out unqualified responses. You’ll see valuable high-level responses by default, and you can use filters to discover in-depth insights.

4) It contributes to eliminating respondent bias.

People who take surveys aren’t necessarily qualified to do so. They may not have enough knowledge in the topic, or have a background that pressures them to answer in a certain way (e.g. they work in the industry you’re surveying about). Screening questions remove these individuals from taking your survey, and put a higher standard on the quality of responses you receive.

5) It improves the respondent experience.

Put yourself in your respondents’ shoes. If you knew little to nothing about a survey topic, would you still want to take the survey? Probably not. Screening questions help respondents take surveys that are relevant and worth their time.

6) It allows you to confirm your audience.

People’s personal and professional situations can drastically change over time. Maybe they’ve gotten married and had a child. Or perhaps they’ve changed jobs.

Use screening questions to confirm that your targeting criteria is up to date. And if recent changes lead a respondent to no longer match your target audience specifications? Then they won’t take your survey.

03Question types

The two types of screening questions

When it comes to deciding on the type of screener to use, there are two popular types to choose from: Behavioral and industry-specific.

Behavioral screeners are just how they sound. They limit the survey to people who behave a certain way—whether it’s related to how they spend their time or money.

A behavioral screener can look as follows:

“How often do you watch TV?”

Every day

A few times per week

Once per week

Less than once per week

Never

Where if you’re only interested in surveying devoted TV watchers, you can choose to disqualify respondents who select “Never” or “Less than once per week.”

Meanwhile, industry-specific screeners filter out respondents who may be biased in your brand research. This can be the result of respondents working in the same industry you’re doing research in or being close to someone who does. If this is the case, they might be influenced to answer in a certain way.

As an example, let’s assume you want to compare your pet store to competitors. Your industry-specific screener can be:

“Do you or anyone in your family work in the following industries?”

Cosmetics

Pet food

Household goods

Restaurants

Automobiles

Insurance

None of these

To prevent any bias, you disqualify respondents who select the Pet food option.

As you look at both types, you’ll probably conclude that you need to focus on using behavioral screeners. But keep an open mind to using an industry screener as it can help you keep the responses unbiased.

Know which screening questions to ask but don’t have the contacts to ask them to? Use SurveyMonkey Audience to access our consumer panel and survey your target market.

04Question design

Sold on using screeners? Before you do, let’s talk about how to use them in your survey.

To simplify our tips, we’ll use dos and don’ts:

Do put all screening questions at the very beginning of your survey. Otherwise, you’ll take more of the respondents’ time than you need to and collect responses from people you don’t want taking the survey.

Do place a page break right after your last screening question to effectively disqualify and qualify respondents.

Do include enough screeners. It can be worth using more than one screening question if it leads to improving the quality of your responses.

Do be specific. If you keep the screening question(s) too general, you might let more respondents take the survey than you’d like.

﻿Don’t use “Yes” and “No” as answer options. These choices tempt respondents to answer the question less thoughtfully. Also, the acquiescence bias (tendency to answer in the affirmative to appear more agreeable) influences respondents to answer dishonestly.

﻿Don’t write leading questions, such as: “Given our recent award in providing high value products, what do you think of our pet food prices?” Adding information in the question that guides or even pressures respondents to answer a certain way can bias their response.

﻿Don’t make respondents think too hard. Respondents don’t have a perfect memory or the time and patience to recall something. Be reasonable with your ask and when in doubt, err on the side of using a broader question.

﻿Don’t include industry slang. You don’t want respondents to feel stupid and answer your screeners incorrectly. If you decide to use any slang or industry-specific terminology, make sure to define it in the question.

Finally, consider the disqualifying logic in your screeners. If a respondent doesn’t qualify to take your survey, do you want to kick them out without learning anything new about them?

Since you already have their attention, you can use the opportunity to learn more about them—for free. To do so, apply either Question Skip Logic (when using one screener) or Advanced Branching (when using multiple screeners) to the page your screening questions are on. If they get disqualified, you wouldn’t boot them to the disqualification page immediately, but send them to a page that asks a few questions, instead. Just remember to keep the questions on the page relatively broad so that they’re comfortable answering them, and apply Page Skip Logic to it so that respondents get disqualified afterwards.

Here’s an illustration of how this works:

05Incidence rate

How to measure and predict it

Before launching your survey, consider its incidence rate.

The incidence rate is the percentage of respondents who pass your screening question(s) and go on to take your survey.

To help clarify the rate, let’s walk through a quick example for calculating it: Say 100 respondents take your survey. 70 pass your screeners while 30 are disqualified. This gives you a 70/100 x 100 = 70% incidence rate.

So why does the incidence rate matter? We use it to calculate the amount of people to send to your survey in order to give you the number of qualified respondents you need. If your guess is too high, we’ll send less people to take your survey and you’ll collect fewer responses than you ordered. And if your guess is too low, we’ll send more people to take your survey and you’ll collect more responses than you ordered.

Let’s visualize this using numbers:

Assume you’re hoping to collect responses from 100 people who own small or medium-sized dogs.

You estimated a 50% incidence rate but it’s actually 25%. So you overestimated it by 25 percentage points. This means we served your survey to 100/.50 = 200 people but only 50 qualified to take your survey. So you fall 50 completes short of your goal.

Now say that you estimated a 25% incidence rate but it’s actually 50%. So you underestimated it by 25 percentage points. In this case, we served your survey to 100/.25 = 400 people and 200 qualified to take your survey.

Note: When surveying a panel in the U.S., there’s a chance that underestimating your incidence rate results in receiving more completes than you asked. But when surveying an audience internationally, we stop collecting responses once we reach your desired number of completes.

To make an educated guess on your incidence rate, there are a few things you can do:

Check to see if your team has relevant market research on the target audience.

Conduct an online search for outside research, which can also give you valuable insights, especially if the category is well researched.

Run an “incidence rate test” where you create a mini-survey with just your screening questions. You can then launch your survey to 100-200 people and see how people answer the questions. From there, you can calculate your incidence rate, and use a more accurate estimate when launching your survey to the entire audience.

For more help on estimating your incidence rate, check out the tips in this article. And if you’re ready to add your screening questions and set your incidence rate, see how you can do both on SurveyMonkey Audience:

06Analysis

Total vs. qualified completes and its analysis implications

As you look at the number of responses you’ve collected, keep in mind the difference between total responses and complete responses. Total responses include every person who took your survey—even the ones that were disqualified by your screening questions; complete responses only include the people who qualified for your survey and reached the end, so this number will be lower.

When sharing the sample size of responses from non-screening questions, use the complete responses figure. Otherwise, you’d be misrepresenting your survey data.

To prevent including disqualified respondents in your sample size, filter your survey by completeness and select, “Complete responses.” Your resulting data will only include qualified completes.

Knowing when and how to use screeners can help you get the most out of your survey. They let you reach the right audience, collect more valuable responses, and analyze your responses more quickly—all the while giving survey takers a better experience.