GreenBook Blog provides original insight into the challenges faced by the market research industry today. Contributors from both sides of the table share their expertise and offer unique perspectives on a wide variety of issues, both strategic and tactical.

We asked market research clients to tell us their most pressing unmet need from insights suppliers. Here is what they told us. Is your firm up to the challenge of solving them?

Update: Since we originally posted this the response has been enormous!

We have about 60 submissions so far and we are reviewing them on a few criteria:

Fit for purpose (How well can they potentially address the issue)

Originality (Is it the same old, same old just repurposed, or something truly new within the MR space)

Overlap with the event agenda (Are others already presenting on similar technologies, approaches, or ideas in regular sessions)

Business model (Is it scalable? Cost effective? Efficient? Reliable?)

The goal is to find 11 companies to match the 11 challenges. Those companies will be invited to the Insight Innovation eXchange event in Philadelphia to give a brief presentation on their proposed solution and meet with prospective clients and partners. All solutions will be published by GreenBook in the weeks after IIeX.

If your company has a solution, please email Leonard Murphy (lmurphy@greenbook.org) by May 20 to tell me about it.

One important note: even if your submission is not selected to present on stage, many of our client-side partners have expressed an interest in still meeting with all companies who submitted possible solutions to their submitted challenges at the IIeX event. We’ll be facilitating those meetings, but you have to submit and be in attendance at the event to be included in the mix!

This is just one more way we are working to for the future growth of the insights industry; we can create the opportunity, but we need your help to fulfill the potential!

Original Post:

A few weeks ago we launched a new initiative as part of the upcoming Insight Innovation Exchange event to really zero in on the whitespace opportunities for suppliers to step-up and deliver solutions to meet the unmet needs of client organizations. We call it the Insight Innovation Challenge.

Over the past two weeks 25 challenges have been submitted and voted on by client-side stakeholders, presenting a unique opportunity to understand what this all important constituency is looking for that they can’t find currently. Think of it as a crowdsourced concept ideation project.

Our goal in doing this is pretty simple; to match supply with demand. We envision the IIeX platformessentially as an ecosystem driven by the basics of a free market. We work with client organizations to understand their needs and areas of interest and then work to identify, engage and support suppliers that can deliver the solutions needed for today and tomorrow.

Since IIeX is an open ecosystem we have multiple partners from across the value chain that we work with to achieve this vision: entrepreneurs, investors,established firms, public sector organizations, media outlets, trade associations and clients. There are a lot of moving parts involved to pull this off, but we think the Insight Innovation Challenge, along with The Insight Innovation Competition (re-launching this week; stay tuned!), to be the heart and soul of our efforts. The research industry simply can’t sustain itself unless it transforms to meet the evolving needs of the folks who pay the bills; many other players are standing by to fill the gap if we don’t.

Identifying needs and supporting innovation to meet them is the only way our market will grow.

Recent reports from Nielsen, WPP, and Ipsos regarding the decline in their traditional research businesses sharply underline this point and should serve as a loud alarm bell to all involved in this industry that the tide is turning rapidly and innovation must be the focus of the future.

So, with all of that in mind, the first phase of the Challenge is over now: we have a strong list of ideas that enterprising suppliers can take and run with , so it’s time to move on to Phase 2: identifying companies that think they have a solution.

Here is what will happen now.

I’ve listed the Top 10 Challenges by vote counts below. Upon reviewing the remaining 15, 14 of them seemed to be variations on a theme from the Top 10, so I’ve added them where I think they best fit as “sub concepts” to the 10 core Challenges. There are some very interesting opportunities here that run the gamut from the traditional research wheelhouse (sampling & targeting models, opportunity assessment, concept testing, field work optimization, etc…) to behavioral economics/unconscious drivers of decisions, media measurement, Big Data, advanced analytics and data visualization. What they all seem to have in common thematically is a need to deliver ROI, increase speed and efficiency, and deliver more value across the client organization, all of which are attributes that traditional MR is perceived to be notoriously weak in.

The final one is a “wildcard” Challenge that didn’t fit with any of the others, and frankly it is fairly niche so it was at the bottom of the league tables, although looking at it objectively it’s a critical issue for a VERY large niche (healthcare) and could be applied in a very impactful way in other sectors as well.

This is where it gets fun! The goal now is to identify companies that think they are up to the challenge of solving these problems. There are a few ways we’re going to do that.

If you know of a company that you think has a solution, either comment on this post or email me at lmurphy@greenbook.org

We’ll also be reaching out to our network to identify firms though our own scouting efforts.

The goal is to find 11 companies to match the 11 Challenges and they will be invited to IIeX in Philly to give a 5 Minute presentation on their proposed solution in front of prospective clients, partners and investors. We’ll be working with all involved to line up any needed follow-up meetings and engagements.

When combined with the already cutting edge agenda topics and presentations scheduled at IIeX, the other initiatives we’ll be launching connected to all of this, as well as the Insight Innovation Competition, we believe we’ll have constructed a comprehensive “virtuous circle” to enable the matching of supply with demand.

Without further ado, here are the Top 25 Unmet Needs Of Insights Clients, in vote order from most to least. Bear in mind the subconcepts did not make the Top 10 themselves, but I think you’ll agree with me that they fit where I’ve put them.

If you’re a supplier, now’s your chance to shine. Good luck!

1. Show me what categories are ripe for disruption and why.

Businesses both large and small are always on the lookout for categories/fields that are ripe for disruption (e.g. Netflix disrupting video rental, Napster/iTunes disrupting the music business, MapQuest disrupting maps, Turbo Tax disrupting tax preparation).Each of the fields that were disrupted likely have some common consumer complaints/frustrations/pain points.What are the categories that are ready to topple next?Why?What are the pain points?I’m looking to understand the deep needs/desires of those categories that make the category ripe for disruption so companies can focus their energy there and ultimately better serve consumers.

Sub Concepts:

1.1 Measure Product Category Disruption Probability

Can someone give us a metric that indicates the ability for a new product to transform an existing category or create a whole new category that doesn’t exist? A few ideas would be:

1. Read reaction to “Just Gotta Have It” discontinuous propositions.

2.Indicators of Category transformation/creation potential

Ideally the approach would give a more accurate read on consumer reactions for new/discontinuous products via spontaneous reactions vs. traditional methods that rely on rational structures.

1.2 Identifying and creating genuinely disruptive innovations.

Chaos is the new reality.Linearity and incrementality is the pathway to doom. Disruption is the norm.In this context, we all need new tools and approaches to creating genuinely disruptive innovation.

2. Reaching and changing the mind of uninterested/uninformed

It’s relatively easy to survey or study the engaged. It is more difficult to engage with the uninterested/uninformed. When they are your target market (or part of your target market), it is difficult to gather an understanding of the barriers to participation. I’m thinking here about surveys, censuses and program participation – largely marketing for government.

Sub Concepts:

2.1 We need smart analytics to identify new donors/customers

In order to reduce costs and enhance traditional direct and digital marketing performance, we need smart analytics that identifies our target donors/customers and delivers our marketing only to these individuals. This involves front end modeling and predictive analytics using “big data” followed by filtering of direct mail and digital marketing to those most likely to respond. The traditional direct and digital marketing worlds need to be more sophisticated and open to these efforts.

2.2 I want to determine tech savvy users

I would like to segment tech savvy users, i.e. they have experience using social media, are comfortable with most websites and applications. Would use social media analysis to find keywords, links, photos etc.

3. System 1 tools and metrics…beyond traditional qual.

With all of the talk of system 1 thinking there has been very little that I have seen from MR suppliers on the subject. Most seem to think they are checking that box by tacking it on somewhere talking about “emotion” but I have seen very few true system 1 tools. Where the heck are they?Further I need those insights quickly…days not weeks or months.Deep psychoanalytic methods are great, but these are time-consuming, expensive, and not usually validated or findings replicable.I would love to see what is out there that could help me shed light on system 1.Help!

Eye tracking in situ is expensive and invasive; we want to understand what draws the eye in POP displays, at the shelf, and in the real world without using traditional eye tracking set-ups. Give me something scalable, inexpensive, and accurate and I’ll give you my business.

4. I need a holistic view of the consumer

Each study and data source provide a single perspective of the consumer.What is needed is a much more integrated view of a SINGLE consumer. If I could have all or most of the following in an easy to understand format – many of the core question for marketing effectiveness would be answered with a unified perspective.

1.Transactions within the category and adjacent categories

2.Attitudes regarding the category and brands.

3.Lifestyle/media consumption

4.Values that allow some foundational understanding of motivations.

5.Point in time emotional drivers.

Sub Concepts:

Single source data across social platforms

As I’m asked by my internal clients to give them analytics across attitudinal, behavioral, transactional & social at the consumer level, I have to explain that no one has offered such a system yet. I can’t engage with individual consumers and understand them across all of those data channels because the data is all housed in many different silos across the organization and with many vendors. The Big Data platforms are good at running models across large data sets, but synthesizing multiple data streams is not easily done. IBM, HP, Oracle, etc… don’t have such a system either. Can someone who understands the marketing value of data please step in?

5. Better concept evaluation methodology (tried them all)

Most of us have tried the standard quant concept evaluations that every vendor offers as unique (but ultimately are similar) and they help us get slightly above average product offerings.Yet, these same concept evaluations often fail badly when we have a innovative product that consumers don’t fully understand yet.I’ve seen $1 Billion+ ideas succeed in market that failed in typical quant evaluations and these stories do terrible things for the research industry’s reputation.So companies instead sometimes pursue the qualitative/focus group route, but it’s painful/scary to ever push forward with products based on a feedback from focus groups where we don’t know “how good is good?”The only other option is going with our gut feel as experienced professionals.There must be a better way, can you create it?

Sub Concepts:

5.1 A solution to radically shorten the product development lifeycle

The current process from ideation, prototype, test, refine, production, market takes waaaay too long. How can we cut down the time it takes on the front end via the insights process to create winning products faster and get them to market sooner?

Also, once we do get into market, it takes too long to enter new markets and we lose competitive advantage to more nimble competitors. How can we measure product performance faster in order to cut down on the in-market development process?

5.2 An agency that will service mobile ethnography ad hoc

Everyone wants to sell me a community with mobile plug-ins; I don’t need a community! I want to conduct ad hoc ethnography with a variety of consumer segments as needed to get “in the moment” feedback. I don’t have time to run it myself though so I don’t want to license the platform: I need an agency that is interested in working with me based on my needs, not trying to sell me some uber-solution that I don’t have budget for. Bring the tech in house and let me come to you as needed; I promise you’ll get my business!

5.3 Speed of turnaround – how to field/report a study in 2-3 wks

A quick survey should take no longer than 2 weeks to plan, field and report if it is a simple concept test, or is limited to an area of questions.However, if you have more than one or two questions and want insightful results, most vendors need at least 4 to 6 weeks or longer to get through their process. My counterparts in the Analytics area can deliver the results of their analysis within hours.I’m at a disadvantage with my set of tools and resources.Who can speed up the survey process?

5.4 We have 3D Printing, why not 3D Imaging?

3D Printing is great for quick product prototyping, but why isn’t there a solution to do rapid scanning of real objects and spaces (like store aisles!) and do rapid 3D virtual mapping? It would be incredibly effective for concept testing in my business.

6. Video analytics: how to process video data like text

We do a ton of video work today; crowd-sourced videos, ethnography, online groups, etc.. plus support fans submitting and making videos about their brand experience on YouTube and other sites. Where is the software to allow us to analyze and manage all of that content like you can do with text using social media and text analysis tools? I know the government has tools like that; why haven’t they hit the market yet?

7. How do we get a representative sample?

With a decreasing number of Americans who have landlines, horrible phone response rates, non-representative online panels, how do we get quantitative measurement that is actually statistically projectable to the population at large without spending outrageous sums?

8. Can Art meet Science: Big Data Visualization

Data seems to be now available from all possible sources – social, mobile, web and behavioral and insights are possibly sitting as power point decks. However one of the biggest challenge is the ability to synthesize this data/insights and present this so that you can get your message across to even a novice user. The need is to have data visualizations move beyond just pie and bar charts, it should be able to make data and not just big data but MR insights tell a story, something like DIY ( Infographic in a Minute). Can the firms/agencies walk this last mile and not leave it to just a presentation?

Sub Concepts:

Strategic vs. Academic?

How can a client and vendor partner so that the first draft of the report hits the mark in a strategic way and represents the business problem/solution vs. just the research reporting? As much as we try clients are continuously re-writing vendor reports so that they are attuned to the executive audience.Are there any reporting best practices, tools or resources that can “come to the rescue”, here?

9. Comprehensive competitive intelligence on marketing strategy

Internal constituents frequently ask me “why did product X do so well with consumers?”I could field a simply study to assess what consumers reacted to, but what my colleagues really want is a comprehensive story of how much they spent on marketing, what channels they bought media on, what creative was used, how they positioned the product to consumers, what drove purchases the most, and what the sales curve looked like.They also want that information even before the product hits the shelves.That can be both time-consuming and very difficult to recreate in many categories.I’d love to see a syndicated offering that analyzes successful campaigns for products as it’s happening (eventually filling in all stages) and in a comprehensive, standardized manner.

Sub Concepts:

9.1 Show me a way to identify key data sources

I need a way to “gin up” relevant data from diverse sources. I want to create a baseline with that data and use that information to create targeted longitudinal research on key issues.

The model would allow for continual monitoring of my transactional/social media/blogs/community and competitor data sources for changes. The model should have the flexibility needed to adjust sources based on “attuned value” (some type of scoring algorithm) defined by scope and the prevalence of mentions combined with hard sales data. Ideally, the model will simplify source analysis of data used by C-level, Social Media Managers, IT and Market Research to create simpler interpretation of positioning thus allowing for a clearer action path. This is in turn should allow our departments to align strategy, create a higher ROI on our data and research expenditures and finally allow us to remove extraneous data sources that don’t provide value. Ultimately it will make our analysis more timely and our reporting more useful to multiple areas within the company.

9.2 Integrating Information into Business Activities

Generating information, collecting information, analyzing, reporting…all not a problem (only that we may do too much of it). Gap is the ability to take what we are generating, collecting, analyzing, and/or reporting and easily integrate it back into the business. Assumption is that the majority of what we produce is not shared with all parties that could derive value from that insight. Vendors continue to focus on the front-end and not this critical back-end.

10. Omni-channel media measurement

How can I activate consumer relationships if I can’t see all of the touchpoints of brand exposure in a single place? I need to measure campaign effectiveness across all media channels in the course of a consumer’s day (social, online, TV, radio, in-store, etc…). The current models just don’t deliver that. Whomever can nail that gets a ton of my business.

Sub Concepts:

Real time, cross channel media exposure measurement

Nielsen, Comscore, Arbitron, etc.. have dropped the ball on cross-channel and cross platform media measurement. Why has no one created a mobile app that will allow real time measurement of media exposure? If Shazam can recognize songs, how hard can it be to recognize brand messaging via digital signals and record online exposure accessed via mobile devices? C’mon people, there is no need to have panels and set-top boxes anymore, let’s use the devices that almost all consumers globally have with them 24/7 and get REAL metrics!

Wildcard: Improve users’ compliance with handling contact lenses

(Editor’s Note: I suggest thinking of this as how to engage patients and reinforce compliance in any healthcare related issue)

Overwear of contact lenses (CL), evenlow cost daily lenses, remains a persistent non-compliance factor causing clinical problems.While reusing CL can be done safely, it requires active participation of the user.The quality of this participation often diminishes over time.The users follow the rules in the beginning, but then they follow the limits.Common examples of non-compliance include

─topping off (the solution in the lens case),

─using old (expired) lens cases,

─skipping the required rubbing when replacing the lens,

─using water instead of formula et al.

What is needed is ideas making compliance with the protocol fool proof.The scope is limited to the lens case and the solution.The lens itself cannot be changed.

About Leonard Murphy

Leonard “Lenny” Murphy has been in the Market Research industry for close to 20 years in various senior level roles, most notably as CEO of full service agency Rockhopper Research, CEO of tech-driven start-up BrandScan360, and Senior Partner of strategic consultancy Gen2 Advisors. A major aspect of his work focuses on collaborating with multiple organizations to help advance innovation and strategic positioning of the market research industry, most prominently as the Editor-in-Chief of the GreenBook Blog and GreenBook Research Industry Trends Report, 2 of the most widely read and influential publications in the global insights industry. Mr. Murphy is a key consultant to numerous insight-focused organizations on both the supplier and client side and an advisor to several technology focused start-ups via his consulting practice. He is involved with multiple organizations including the Insight Innovation Exchange, The ARF, NY AMA, NewMR & others. He sits on the boards of several Academic programs as well as numerous companies. He also Chairs and produces conferences globally and speaks about the future of insights and how to deliver maximum business impact at many events annually. He lives in Atlanta, GA with his 6 (yes, 6!) children, Should- be-Sainted wife, a dog and 3 clinically insane cats.

Seems to me that 5.3 and your #8 subcategory (you didn’t assign a number to this one) don’t require new methods or products or systems. Instead, they require one of two things: 1) clients finding vendors that can better meet their needs, or 2) clients doing a better job of freeing good vendors to meet their needs.

Both speed and report relevance can be a result of vendors not doing a good job, or of clients limiting their vendors or failing to give vendors what they need in order to do a good job for clients. The need for speed can be undermined by slow vendors, or by clients who drag their feet, fail to make decisions, or otherwise delay the process. Similarly, irrelevant reports can again be poor vendors who fail to understand or respond to the business need, or clients that don’t communicate the business need appropriately and instead expect vendors to operate in a vacuum and guess what they need.

If either of those is a consistent problem for you, either be a better client or find better vendors. There’s no need to reinvent the wheel to solve either one of these problems. There’s no magic methodology that will be the solution.

[…] of reviewing almost 70 candidates, the IIeX Advisory Board have chosen the companies to meet The Top 25 Unmet Needs Of Insights Clients onstage at the Insight Innovation Exchange event in Philadelphia next […]