In a world of ‘alternative facts’, are irrational humans really ready for the world of Big Data?

Sunday, February 5, 2017

Are we really ready for big data? As we watch the world
rapidly change around us with decisions being based on gut rather than facts, I
find myself questioning the intelligence of humanity and its readiness to make
sense of big data.

As a consultant I often talk to business executives about
the need to collect and analyse data. The discussions tend to focus on how to
get the data, store it and make it accessible. When I bring up the question of
whether people are ready to use the data they subsequently get, I often get
told not to worry about that. One manager recently told me, “when we start
getting meaningful data out our people will start to use it… they’re not
stupid”. The manager assumed that people
will naturally look to rely on meaningful facts over gut instincts but is that
true? Let’s look at real examples at the national scale.

Lies, Damn Lies and
Alternative Facts

We’ve just seen President Trump implement an immigration ban
for countries he sees as a threat. Statistically the threat from immigrants
from these countries is so low you’d have a strong case to say it was
non-existent. American toddlers cause more deaths. There’s been mass protests
in the US and various Tweets, Facebook posts and news reports quoting every
stat imaginable about why the ban makes no sense. Another interesting stat
being quoted is that Trump has gained a majority disapproval rating in a record
8 days. However, Trump’s administration has already come out with a statement
to say that the ban “is working”. Clearly, they have “alternative facts” which
tell a different story. Like it or not, the world’s biggest economy is being
directed based on opinion rather that hard data.

The Will of the
People

Of course, maybe Trump is an outlier (a very significant
outlier, but outlier none-the-less). So let’s look at the good old UK. After
all we are supposedly very sensible over here. Let’s examine the well-educated
thinking behind the Brexit vote. Whether the outcome was right or not, it’s
difficult to see how our access to big data and the real facts helped in any
way. The Brexit campaign managed to combat expert statistical arguments put in
its way by stating that Britain had “had enough of experts”. It then put up
statistics which were widely ridiculed by the experts but which seemed to have
a great sway on the voting public. The Brexit campaigners claimed that an exit
would free up £350m a year for the NHS – they even put this number on the side
of a double decker bus they drove around the UK. Three days after the vote a
prominent Brexit campaigner went on live television and stated the £350m “was a
mistake”. I’d call it an outright lie, but in the modern world of alternative
facts the term “mistake” is apparently acceptable. The Remainers weren’t guilt
free either. Toward the end of the campaign we saw them publish some
questionable facts of their own, further clouding the real truth. Is Brexit
really the right answer? Even now we don’t know, as there appears to be very
little data analysis being relied upon. Now parliament has just voted to back
Brexit because “it is the will of the people”, even though few would claim that
the people were properly informed.

So what does this all
tell us?

The success of the Trump and Brexit campaigns helps
illustrate the problem with data. People seem to intrinsically distrust data,
and will always prefer to go with their gut. Business schools around the world
are telling everyone that we are entering the age of the data driven business,
yet the reality is that we human beings do not seem ready for this. Indeed,
there’s even an argument to say that data just gets in the way.

Entrepreneurs often take big risks, defying what the data
tells them about their chances of success. Trump is an entrepreneur so is it a
surprise that he doesn’t use data to make the big calls? Given his successes,
and how the stock market seems to like what he’s doing, is this wrong? Perhaps
the question is not ‘are we ready for big data’, but ‘do we need data at all?’
We certainly don’t seem to need or want it to inform big decisions like how to
run America, or whether the UK should risk everything and leave the EU.

I suspect the truth is that until we learn to
better interpret data and know when to trust and distrust what it tells us, the
preference to go with instinct over analytical direction will always prevail. The Brexit campaigners realised this. In a
world where we have more data than ever at our fingertips, people just don’t
know how to interpret it, and human nature is to distrust what you don’t understand.
What this means for big data and the use of data analytics is that businesses
need to consider how their people will be able to make real use of it. Business
users need to understand how to combine data insights with their internal
decision making. Without this most people will go with what their experiences
tell them, not what the data says.