For a long while, the census model was closely mimicked for other research purposes. Want to understand what 1930s mothers felt about the latest detergent? Get them to fill out a piece of paper, hope they send it back to you and dedicate considerable man hours to having the data processed.

In true Mad Men style, much of the insight was ignored in favor of a hunch. After all, who has time to wait for the data to come in when it was telling you something that happened yesterday?

What’s more, the paper-based research that underpinned the marketing industry for so long was not cheap. The sheer man hours it demanded meant a chunky bill at the end of any insight project.

Then emerged the focus group. Individuals selected from a cross section of society being lured to some cheap meeting room on the promise of weak coffee and donuts. Once there, a marketer would begin a four-hour discovery session to truly unearth what people really thought of cashmere cardigans.

While an important step forward, on its own, the focus group was dangerously low on sample size. When blended with quantitative research it could provide the useful context to arbitrary numbers, but it lacked agility.

What’s interesting is despite having more tools at our fingertips than ever before, focus groups, postal research and the good old-fashioned clipboard surveyor are still methods employed today. What’s changed, however, is our ability to blend multiple research disciplines and deploy a variety of tactics to answer more complex and nuanced questions about our audiences.

We can decide whether the insight we need has to be deep and robust, in which case it requires time and probably a well-stocked bank balance, or a quick probe launched into the public’s collective opinion. We are truly in the insight led era.

Automated democracy

From Google Trend Data through to YouGov’s Profiler Tool, we’re now able to take millions of bytes of data and rapidly paint a picture of an audience’s behavior towards a particular brand or understand what part of last night’s reality TV shot hit a chord with the viewers.

What’s more, we can do a lot of this for free. The bourbon-filled Madison Avenue offices of the 50s would be blown away by the insight we can produce today, after a few digital clicks. However, when we start reaching into our pockets, we begin to unlock considerable automated insight power.

Sociabakers, a social media analytics tool, is capable of monitoring millions of live conversations based on a set of keywords – and can determine how to best engage the people doing the talking. Obviously, analytics have been on our radar for some time and are still being perfected, but today’s tools can identify imagery, video content, sentiment, sarcasm and even purchasing intent in order to provide a three dimensional view of a potential customer. While the technology itself is fascinating, what’s more astounding is that the keys to drive the platforms can be handed to a complete novice — and they’d still be able to come out with some useful data.

It’s not enough, however, to just rely on the machines. We still need the clipboards. We still need the focus groups. We still need the humans.

The human touch still matters, even in in the insight led era

Twitter predicted that Britain would remain firmly within the European Union. It wasn’t touch and go, it was a whopping 70% probability that on 23rd June 2016 Britain would vote remain.

Twitter was wrong.

A more mixed prediction emerged around the U.S. presidential election. One Twitter study predicted Hillary would win when the whole population of Twitter was taken into account. Meanwhile, when only looking at influential users, Trump emerged the winner.

In both scenarios, social analytics tools spotted key trends — mostly pointing towards ‘remain’ and Clinton. Yet, everyday people would comment on how their uncle was voting for Donald Trump or how it seemed most people at the pub were keen on breaking free from Europe,.

To read the runes and spot the trends you needed to ditch the automated analytics and settle in to some proper research. It wasn’t enough to rely on the keywords or the algorithms to try and pick out a trend. Instead it needed human ingenuity and reason in order to monitor the conversations. Interestingly, according to a Bloomberg Report, Twitter defied the pollsters and foretold Clinton’s upset because of a downward trend in sentiment.

There will forever be a need to sit people down and ask them what they’re thinking and to monitor their responses. What technology allows us to do is achieve this on a grander scale than ever before. We’re utilizing open platforms to supercharge our human-to-human research.

Quora, Facebook, Reddit and other online forums can provide you with the perfect platform for coordinating well thought out strategic research. They provide us with the opportunity to host healthy discussions and pose useful questions to individuals that we want to speak to on a scale that we thought would never be possible a decade or so ago. It’s the human interaction we need to spot those tiny nuances that can lead to an absolute gem of an insight. But it’s digital.

Does this mean that the ‘real life’ focus group is dead? Absolutely not. The key is to ensure that there is a blend between machine and humans.

What’s next?

With insight, we really are going to be bombarded with more information than ever before. Across all industries.

While local councils will be interested in seeing the energy consumption of their residents through the use of smart metering, haptic technology companies are going to be more curious about how the smart meter’s interface will be used. Products will be developed to satisfy specific interaction habits rather than a one size fits all.

Readership, views and yes even engagement statistics will fade into the background as communications programs will start and finish with one question in mind – what emotion are we generating with this campaign? Emotive and behavioral technological advances will help to feed that information back to brands.

As such, we can expect to see more micro-campaigns launched by brands looking to trigger specific emotive responses to their content. Short, sharp bursts of creative activity in response to changes in mood and behavior will become the norm.

In turn, there will be a graveyard filled with decaying cadavers of communications campaigns that are launched without the necessary grounding in audience understanding. Brands will have to work far harder to truly understand the audience they’re communicating with if they want to succeed.

Equally we’ll also witness campaigns that fall flat on their face due to the over-reliance on automated research. Brands forgetting to blend the human touch with the automated insight will sometimes be led down the rabbit hole and look bemused as their campaigns meet tepid audience reactions.

We can expect to see communications teams become much more comfortable with insight as a whole. Sure, experts will continue to be hired, but we’ll have a far greater understanding and appreciation of insight, irrespective of whether gathering it is your day job or not.

Finally, we’ll see costs continue to drop for automated insight as technology continues to advance and big data becomes the norm. In contrast, real human-to-human qualitative research will rise in price and complexity as the ‘grunt’ work will already have been done.

There’s no doubt we’re in an insight led era of communications, the question is whether the next 12 months is going to see people invest more in the technology or more in the human side of insights and research. We say, hedge your bets and do a bit of both. See more from our Communication Trends Hub.