The Continued Debate Over DIY Market Research

Earlier this week I wrote about one of the most significant debates currently taking place in the market research community: the debate over DIY research. We’ve gotten some great feedback and interesting opinions on both sides of the argument, including some really helpful resources, provided in a comment on the original post by Kathryn Korostoff of Research Rockstar. (Go check those out.)

Indianapolis, Ind.-based Delta Faucet performs its market research in-house for two reasons: to stay close to the customer and to remain nimble.

With DIY research, “we have more control over exactly what’s going on,” Paul Ponsford, Delta’s market research analyst, says in the report The Pros and Cons of DIY Research, from 20/20 Research. “If we want to change the direction of the discussion based on the feedback we’re getting, we can do that easily.” 20/20 Research is a Nashville, Tenn.-based provider of qualitative research fieldwork services and software.

Doing its own research gives Delta a better understanding of the findings, as it is involved in all aspects of the research project. “If we’re the ones running through all of the data, that will help us become better experts in our business,” Ponsford says.

This is definitely the argument we hear most often in favor of a DIY approach. (Well, perhaps second only to the argument that DIY is cheaper.) The idea of directly connecting with your customers, of combing through all the data to gain a deeper understanding of opinions and behavior… I can see how that’s appealing.

But at the same time, those conducting their own research who don’t have a background in marketing and statistics might inadvertently draw the wrong conclusions from their data. (It’s certainly happened before.) They also run the risk of introducing bias into the analysis – discovering the answers they’re hoping for in order to justify their own desires. To further quote Michael’s article:

“Yes, some manufacturing execs do rely on DIY research in an effort to save money; and this method of acquiring critical information can produce disastrous results,” according to Cathy Williams-Owen, president and CFO of Port Washington, N.Y.-based Dri Mark Products, Inc., a manufacturer of writing instruments, security marking systems and inks. “It is somewhat like working in a vortex. The information obtained may not produce the valuable insight that, say, a well-formulated focus group can provide. The conclusions that are reached can skew results with the potential for a disastrous outcome.”

Manfred Bluemel, Ph.D., at Seattle-based Zeitgeist Research, is a proponent of DIY research tools like Survey Monkey, Survey Analytics and Zoomerang, with a caveat: “They work as long as you have a skilled market researcher who knows what to do with those tools.”

So where’s the line? As an industry, how do we strike the balance between delivering quality, value, and a connection between our clients and their subjects? What types of research needs can be met entirely by DIY, and where is professional execution and analysis required? Is there even a right answer?