Lessons In Data Integration

Editor’s Note: I’ve been lucky enough not just to know Mel Courtright, but to have collaborated with her on a variety of projects over the years. She is, quite simply, brilliant and a joy to work with. I’ve looked for more ways to work together and now I found two: she is presenting at IIeX in 2 weeks on the groundbreaking work Research Now and Experian are doing with single-source analysis and today we have her first blog post here on GreenBook Blog! And what a post it is: data synthesis is one of the most important topics in our industry today and Mel does a great job of sharing the basics from her experience on how to get folks started. This is a solid “meat & potatoes” post for today’s research professional. I think you’ll like it, so enjoy!

By Melanie Courtright

Behavioral data tells us what people are doing, and from evidenced behaviors rather than through recall questioning. Attitudinal research has always helped clarify the reasons behind the behaviors. Historically, these two exercises have been conducted concurrently, but the data was not connected and fused to link specific behaviors with attitudes at the respondent level. This is changing, however! Permission-based tracking of people’s online activities, mobile behaviors and geo-locations, combined with connected survey and profile data, is enabling a new level of analysis and understanding.

With or without the data being linked at a respondent level, the process of tying behavioral data to attitudinal data can be daunting. While I didn’t actually take underwater basket weaving in college, I did do quite a bit of homework on basket weaving, and the concepts are a good parallel for the steps you would take to integrate behavioral data into your survey data.

To enable understanding of the concept/process, we will work through an example of a project where we use disparate data woven together to better understand the automobile shopper…

1. Pick Strong Materials: The stronger the materials, the stronger the product. Work with data that comes from a broad base of people, has reliable data collection processes and delivery, and is a good blend of structured and unstructured data. Too much unstructured data can prove difficult to manipulate, and only including structured data can be harder to mine and investigate. Work with material you can readily access and define.

Ex: For an auto shopper study, we would start with survey data to identify intenders, and then (with permission from the respondent) use mobile geo data to watch visits to dealerships, as well as search and web visitation data to understand information gathering and online shopping activities. Additionally, we would incorporate social media data to monitor brand mentions and sentiment.

2. Lay out the Base: This is really a big data phrase for “start with the objectives” and align the data points accordingly. Create a data usage (or analysis) plan at the beginning of the project that will guide the insights. Additional data can always be brought in to investigate or validate, but you should outline a basic data plan at the beginning of a project that is based on what business question you are attempting to answer. Identify the hooks between each data stream that will tie the data together.

Ex: We plan to tie survey intention data to real behavioral data, both to validate and to understand differences in intent vs. behaviors. We will tie survey brand considerations to social media sentiment and awareness. Etc.

3. Start Weaving: Unfortunately, many people start here and get lost. They look at disparate data and start trying to make the data work together in aggregate. But with this approach, one can easily get lost. If you begin with steps 1 and 2, identifying the correct data stream and the best data within that stream, while also identifying the hooks that will strengthen your understanding of a topic, it’s much easier to start bringing that data into the reports and findings.

Ex: Survey data revealed that 55% of auto intenders planned to visit a dealership over the next 30 days. Review behavioral data for those members, and determine what percent actually visited a dealership along with when, where and how often. Review brands being considered from survey data and compare to social media sentiment scores over the last 12 months. Compare internet shopping and information gathering activity data and enhance the findings with online behavioral data that shows what sites are being visited, how long the visitors stay on the sites, and what is trending in automotive searches.

4. Shape the Corners: As you begin to weave the data together, you will find a lot of data that supports and builds your stories, but you will also encounter curiosities and anomalies. Don’t be afraid of these, but rather explore them. Some of the deepest insights can be found here.

Ex: In one case study, we found that nearly one-third of people planned to consider a specific mega brand and visit dealerships for that brand. As we reviewed behavioral data, we found a smaller number actually followed through with that intention. Investigation revealed a large spike in searches, and site/dealership visits for a competing brand. Social media reviews showed us commentary about a large rebate and deal program, which that brand was using to drive traffic. These intervening events changed the purchase path for the consumer.

5. Practice in the Dark: It’s customary for researchers to want to test and retest methods and insights before offering services to a client for a live research project. Our suggestion is to begin layering data on some of your tracking or ad hoc projects behind the scenes, and share results as they become interesting. Execute internal discovery exercises. Ask your data partners to help you develop products and approaches.

Ex: We took an automotive tracker where we programmed and hosted the data and began watching the behavioral data of members who took the survey, creating comparisons. We also took the brands in that survey and started monitoring them in social media. We did these exercises behind the scenes for a few months before sharing them with the client. Once we were more comfortable with the process, the data and the value, we were able to deliver it in a way that delighted the client.

Data integration is growing and thriving. Clients are excited about what it is bringing to the insights table, especially when combined to answer the, “who, what, when, where and why” of consumer choices. We hope you find the topic as exciting as we do, and that you begin experimenting.

By the way – here’s a picture of my first basket weaving attempt. I’m definitely better at weaving data than I am at weaving baskets! But I’m going to practice and get better at both basket weaving and data layering. And I’ll share my journey along the way.