Do all those website "pre-surveys" really help?

Much hoopla has been made in the not-too-distant past about having a free little questionnaire on your site. It hits you when you enter with a question like this one: “After your visit, would you be willing to answer a few questions?” 4Q from iPerceptions does one, as does Kampyle.

When I spend $150 at usertesting.com to find out how much visitors hated my company’s incredibly old site, I also found out that they hate getting a popup box before they get into the site. (For the record, we use Kampyle.) We had programmed the site so that only 30% of visitors were served the box, and only on the home page, but that meant two out of five testers got it. They both reacted vehemently. So we pulled the questionnaire and just left the ability for someone to comment if they were actually looking for a way to give feedback.

We won’t get as much feedback, but the truth is — we weren’t getting a lot of feedback anyway. And we were turning off a lot of people.

So when I have data on reduced bounce rate (more than a day or two’s worth), I’ll tell you.

Robbin

Receive Blog UpdatesGet notified when blog posts like this one are published!

Our owner and CEO, Robbin Steif, started LunaMetrics twelve years ago. She is a graduate of Harvard College and the Harvard Business School, and has served on the Board of Directors for the Digital Analytics Association. Robbin is a winner of a BusinessWomen First award, as well as a recent Diamond Award for business leadership. You should read her letter before you decide to work with us.

You make some very good points there. When we give sampling guidance to our 4Q online survey users, we generally recommend that they err on the side of caution when setting a sampling rate. The great thing about simple random sampling is that you don’t really need a super-sized sample to be able to make accurate generalizations about your visitor population. Often a few hundred a month will do the trick.

We also recommend that our 4Q survey users avoid intercepting visitors during a traffic acquisition or usability testing phase. Experience has shown that the average usability tester is a far more fastidious individual than the average website visitor. That’s not to say that garden-variety survey respondents never express frustration about being intercepted. It does happen, but we’ve seen time and again that the value of the feedback garnered more than makes up for it.

excellent arguments, in my opninio. We were thinking about employing this kind of feedback mechanism in our subscription shop to get a better qualified idea why people go there without making a purchase. But as your post shows, it’s just like in quantum physics: The observation itself changes the behaviour 🙂

Michael – thanks for your speedy reply. You clearly have great Alerts (certainly better than my Google Alerts) or great ears on the ground. I know those ears …. but seriously, think about what you wrote. “Often a few hundred a month will do the trick. ” Ok, let’s run the numbers.

Our site gets about 9000 visits/week, only about 3/10 of the visits get served the opportunity to comment, I get a comment about once every other week. So that’s one Kampyle “conversion” out of about 5400 opportunities, or a conversion rate of .018%. So to get just a few hundred (let’s say 500), you would need to show your 4Q survey to 2.7M visits. Since you tell people to err on the side of caution wrt sampling rate, maybe they show it to one out of four people. So now you need 10.8 million visits/month to get a few hundred (assuming the same conversion rate I have.)

Those are a lot of assumptions and you are welcome to tell me how wrong I am.

Bjoern, that is such an interesting way of looking at it. I wish I could remember the name of the scientific principle.

Mike, do you think that if we only put it on the thank you page, we skew our results to people who were satisfied enough to buy or give us their name? My guess is that we skew it, but still get some valuable data. Maybe you will tell us what you have learned, if you have?