Innovate Blog

A Course Correction for Conjoint Analysis

An Exploration on the Inclusion of Mobile Audiences and Its Impact on Insights

For several months, I’ve been part of an effort spearheaded by GRBNto address the participant experience – to ensure that not only is it a pleasant and inviting experience, but that more people are enticed to participate.

One of the areas that stands out the most when you look at the experience of survey participants is mobile. Surveys that look like this:

And the litany of other headaches that go with trying to complete surveys designed for desktop users on a mobile device.

To address this, I’ve been part of a team of market research professionals conducting research-on-research (ROR) that brings together clients, agencies, data collectors and technology providers across the globe, with the intent of creating meaningful change.

Looking at the Mobile Experience

Taking what we developed for ENGAGE in the ROR process and applying it to the mobile experience, I was part of a panel recently in which we explored inclusion of mobile audiences and its impact on insights.

The variables that influenced participants’ willingness to sign-up for a yoga class (time of day, day of week, type of yoga, instructor, price)

Variability of results based on device – mobile or desktop

We designed a mobile conjoint to present consumers (N= 1,245 yoga enthusiasts) with several alternative yoga classes including different combinations of these 5 variables. Consumers were asked to choose one offering, or none (if they were not interested in any of the options presented). This task was repeated several times to understand each consumer’s preference level for the various attributes.

Some of our key considerations in this study included the fact that mobile survey participation, while increasing, continues to lag overall. We wanted to know if better design could increase mobile participation and representation, and whether a discrete choice exercise produces comparable results and quality regardless of device. We wanted to know how these discrete choice exercises would render on smaller device screens and what impact we’d see on the data if we excluded mobile participants from the sample.

Key Takeaways

What we were able to take away from this study was that, at a high level when comparing mobile against desktop, consumers did not change their behavior based on price, although there is a group of consumers in this category that are price sensitive.

Discrete choice data provided a deeper look at opportunities and worked well on both mobile and desktop platforms. The consistency of data across platforms and a focus on a good experience for participants opens doors for improved representation among mobile audiences. There was a mobile preference among participants of 48.5% compared to 34.9% for desktop.

What we know is that participants expect mobile opportunities for research and many choice-based designs work well on smaller screens. At the same time, how many surveys are truly designed for a mobile audience? RelevantIDis the leading digital fingerprinting technology in the industry and is widely used by both sample and full-service providers.Based on stats from their system, 40% of inbound traffic going into surveys is mobile.Essentially, by excluding mobile you are cutting your feasibility in half. Some providers are as high as 60% mobile traffic.