Using Social Media Intelligence to Design Better Survey Questions

Using Social Media Intelligence to Design Better Survey Questions

Creating a survey that people will actually answer or that provides valuable answers to business questions can definitely be one of those, "sounds pretty easy, but not really" type of deals.

Pew Research says, questionnaire design is perhaps the most important part of the survey process. It is the creation of questions that accurately measure the opinions, experiences and behaviors in public.

If you're having trouble learning the fundamentals of writing survey questions, stop reading and grab a free copy of The Art of Asking Survey Questions from Hubspot and Survey Monkey.

The guide gives you a basic rundown of how you should be asking the right questions and digs into what you should and shouldn't be doing when creating a sound survey.

If you've got the art of asking great survey questions down and want to enhance those skills with social data, continue reading.

Whether you are pushing out surveys in traditional formats, online, or conducting focus group research, use these 3 ways to kick your questions up a notch with social media intelligence.

1. Determine what questions to ask

Word clouds not only help you understand common lingo but it can also help with discovering topics that you could be including/excluding in your survey. Using Fruit Loops, the cereal as an example, let's say we are curious about what Fruit Loops is often being paired with.

The example above shows what other subjects are often mentioned with Fruit Loops (Example Query: (("fruit loops" OR #fruitloops) NEAR/1 (with))).

Surprisingly people often associate Fruit Loops with other foods and drinks such as: marshmallows, green tea matcha, and donuts (or doughnuts). What's also interesting is that those three subjects are more commonly mentioned with Fruit Loops than the word "milk" itself.

If you’re conducting a focus group for taste testing in the future, these examples might be really interesting to investigate further.

2. Discover who to speak to

Who should be surveyed? Social media listening tools that have gender and demographic data can be immensely helpful in this aspect.

Should you be surveying both sexes? By looking at gender distribution you can confirm if topics have a gender bias or are gender neutral.

Taking it one step further:

Aside from just looking at gender, some tools are also capable of displaying U.S. census data too. (Use that to your advantage!)

Based on the Fruit Loops example above, there are 1,218 sources that were identified in the U.S.

In translation, the typical U.S. participant that has talked about "Fruit Loops" in the past year makes about $52,739 annually, has $23,237 in disposable income, lives in a household with at least 2 people and lives in a home valued at $187,844.

If you are trying to set strict parameters around those that are talking about 'Fruit Loops' you might consider implementing an annual income question that moves the consumer to the next question only if they say that they make between $45k to $60k annually.

Or, you could simply use this metric as a comparison to your broad survey results and make additional inquiries afterwards.

Taking it two steps further:

So let's say you already have an idea of who you want to focus on. The next thing to do is to customize and target different segements.

As an example, let’s say you’ve discovered that cannabis users are one of your core targets. You may wonder if you could get even more details about this group (why yes, yes you can):

The demographcic data for cannabis users above who are talking about Fruit Loops are bit different than your general public (jump to previous photo for comparison).

Most cannabis users that talk about Fruit Loops are likely to be female (although not strong enough to rule out males completely) and typically live in a household of more than 2 people on average.

If you were to compare this with the general public, you can see that cannabis users that are discussing Fruit Loops, tend to have less household income, less disposable income and hold less household value in comparison.

Use this kind of data to draw up a criteria for cannabis users you are deciding to survey on!

Whether you are comparing cannabis users to the general population (or whatever you desire), you can turn to these simple pointers to provide a more targeted effort in designing your survey.

3. Help identify keywords and common language

Social media listening platform should have a way for you to understand the common language that's being used around your particular search query. In most cases, this is communicated through a word cloud (much like the one seen above in example #1).

When you are crafting your questions are you using words that a typical person would use when discussing Fruit Loops? Not only that but do you even understand how these different words are being used in context (positive or negative in sentiment)?

As a very basic example, if you were surveying single moms about Fruit Loops and some of the keywords identified in the word clouds were: cereal, kids, breakfast, night and mom -- you probably should stick to those terms in your vocab (assuming that those are terms in your survey design, of course) than terms like: oats, children, morning meal, evening meal or mother.

While this may not seem like a huge difference, I can't tell you how many times where I have taken a survey and will inevitably spend more time than I'd like to on a question because of the simple word choices survey designers use.

Another thing that should be accounted is how certain terms are being used. For example, the word “cheap” can often be considered a negative but if consumers are typically using that term to describe products or experiences in a positive way, that should also be taken into account too when designing.

You want make every piece of your survey as seamless as possible -- Make it easier by using common language.

A recap on how to bump your survey question designing up a notch:

Use word clouds to help your uncover new questions to ask

Use demographic and census data to help tighten up your focus and criteria