The Science of Surveys

I recently came across an odd question in a conference survey. It read: Will you attend the topical dinner? One respondent replied, "Topical dinner? Is that something I can smear all over my body?" It was then I knew data analysis would be a problem.

There's a science to writing surveys. Well-written surveys can yield powerful findings. But poorly written surveys are a big waste of time.

How you ask

Sometimes it's not about what you ask, but how you ask. Say you and a friend go to dinner. You think your friend didn't like their entree but you're not sure. How do you find out? You could ask, "Did you enjoy your entree?" But the response would likely be "yes" regardless of the truth.
"Yes" is the polite response. Most friends don't want to ruin dinner. But asking the question in a different way can change everything.
Asking, "Would you order that entree again?" could uncover an entirely different answer.

Really listening

Questions aren't the only issue.
Response sets can do just as much damage.

I was once called in to investigate survey findings from a Fortune 500 company. Executives couldn't
understand why thousands of customers called to express their dissatisfaction about store service when customers consistently reported being satisfied
in store surveys.

One look at the survey responses explained everything. Customers could select "satisfied," "very satisfied" or
"extremely satisfied" to rate store service. It was impossible for customers to report that they were dissatisfied or even neutral. So, quite
naturally, they didn't.

Distributing well

Writing good questions and response sets is really only half of what you need to collect good data.
Survey distribution is equally important. I once spoke to a call center rep who asked me if I would rate her performance on a survey. I agreed. She
proceeded to administer the survey herself. My ratings were quite positive but, to be honest, I didn't feel that way. I'd have been more comfortable
reporting what I really felt to someone else. But because she administered her own evaluation, her company will never know my true feelings.

Doing
something about it

Once good data have been collected, perhaps the most important step is turning your findings into actionable strategy. Far too
often, findings documentation is a place where survey results go to die. And that's a shame.

When asked to provide suggestions on a "topical
dinner," one survey respondent wrote "Butter, please." Surveys are one of the most common ways of collecting customer feedback.
Unknowingly, many companies design surveys that produce worthless findings. And that means that many of us are completely in the dark about our
customers.

Melissa Read is the director of behavioral research at Spunlogic, an interactive marketing and technology agency. She can be reached at
melissa.read@spunlogic.com.