Impact of Question Design on Research

Delivered March 15th, 2020. Contributors: Praise O.

Impact of Question Design on Research

The addition of explanations and including a multi-select category in/before a question are two case studies that illustrate the impact of question design on research.

CASE STUDY: ADDITION OF EXPLANATIONS

In this case, survey participants were asked the following question: "please try to recall a recent instance in which you found important information online, information that led to a significant action or decision. Please describe that incident in enough detail so that we can visualize the situation."

The question also defined a significant action or decision as a meaningful change in one's plans, thinking, or actions.

RESULTS

The results showed responses such as "I looked at the weather app on my phone before I left for work in the morning. It said the temperature was colder than I expected it to be. So, I put on a warmer coat and a hat."

Another respondent in search of a job provided details as to how she changed her job application strategy to focus on specific companies due to online information.

IMPACT

The data only revealed results regarding specific changes, even though a change does not necessarily depict a significant action or decision.

The reason for this result was the addition of explanations to what significant in the question entailed, which eventually led to bias in the respondents' responses.

CASE STUDY: INCLUDING A MULTI-SELECT CATEGORY

In this study, the question asked respondents to detail significant activities that they might have carried out online within the past month. However, the question also included a multi-select category before the main question.

The multi-select category asked the respondents to select options for their online activities within the past month, such as "bought something, watched a TV show or movie, planned a vacation, sentanemail, posted on social media (for example, Facebook or Instagram), and researched a topic."

RESULTS

The answers to the question regarding significant activities carried out online by the respondents were very similar to the options provided in the multi-select category, with most of the participants reporting details of research they did online.

Some responses included "researching information about tax base transfer in California" and searching for weight-loss surgery information online.

IMPACT

The multi-select category and, more specifically, the option for researching a topic online streamlined the nature of respondents' answers, as it primed their responses for significant activities to represent researching something serious such as health and sector-related information.

However, significant activities online do not ordinarily refer to only research. As such, the multi-select category had an obvious impact on the responses from the participants.