In a recent QA article, we looked at how organisations need to develop Skills and Capabilities for the Digital Age tobecome the ‘Organisation of the Future’. The first article in this Data Science series looks at how highly trained and motivatedData Scientistsare crucial to meet this need. These organisational development needs are neatly summarised as the ‘New Rules’*. Three of which are particularly relevant:

Proactively leading the market - rather than just responding to the competition

Handling disruption and change – before it happens

Anticipating, meeting and exceeding rising expectations with suggested new and improved customer experiences

How do you ‘learn from data’ - and why are more Data Scientists needed?

Data Scientists, AI and Machine Learning specialists uncover hidden trends to inform business strategy, product development and resource management - as well as wider questions of political, cultural and economic development.

Human decision-making and technology evolved along with the species - and with our cultural development - and history shows, is not always rational. We have now reached a rather curious point where science is able to reliably achieve ‘miracles’ unrecognisable just decades ago, while ordinary citizens decline lifesaving medicine out of superstitious fears and phobias.

In between this spectrum, lies the decision-making approach of the business world. Businesses typically rely on a mixture of:

Intuition (which might be called "reasonable prejudices")

Superstition (which might be called "unreasonable prejudices") and

Empirical data-driven, repeatable and testable deliberation.

Intuition definitely has its place. As Holbrook Jackson said, ‘Intuition is reason in a hurry’ and it is the brilliance of a renowned business leader to infer cultural trends, market wants, and directions (and then make the right call). But wouldn’t it be better for organisations - and for society - if that intuition was supported by more than the Chairman’s whim?

In todays’ Digital Age, clues about the future lie hidden in the patterns that connect our daily financial, medical, social, online, economic, political, cultural and intellectual interactions. ‘What are the people of the United Kingdom concerned with right now?’ is no longer a question for a newspaper editor, but best answered by Google's ‘most searched questions’. Google not only records questions posed, their answers, and their implied preoccupations - but it connects these to, for example: potential disease outbreaks, political and newsworthy stories and worldly events happening everywhere, right now.

This ability to see widely and to see deeply; to turn questions which seem highly speculative, into concretely answerable queries, to know something novel in a moment that no one has known before: this is ‘learning from data’. It is enabled by machines which operate at a vast scale that process or "learn" trends within. It is the brilliance of a Data Scientist to confirm them, to reject them, and to uncover what yet might not have been seen.

Are your leaders still ‘betting the bank’ without using Data Science?

Does your business collect a large amount of data no one is looking at? Do you throw away data you don't need, and miss what's hidden inside? Planning your ownDigital Transformationand wish to make decisions based on real insight? Then consider QA'sUnderstanding Data Science and Big DataandUnderstanding Machine Learningprogrammes to introduce leaders to these concepts; to enable data-driven business thinking; and to train-up your Data Science and Machine Learning experts in their Big-Data implementation.

About the author

Michael began programming as a young child, and after freelancing as a teenager, he joined and ran a web start-up during university. Around studying physics and after graduating, he worked as an IT contractor: first in telecoms in 2011 on a cloud digital transformation project; then variously as an interim CTO, Technical Project Manager, Technical Architect and Developer for agile start-ups and multinationals. His academic work on Machine Learning and Quantum Computation furthered an interest he now pursues as QA's Principal Technologist for Machine Learning. Joining QA in 2015, he authors and teaches programmes on computer science, mathematics and artificial intelligence. He co-owns QA's Data Science and Machine Learning curriculum with his colleague Lianheng Tong. His areas of expertise include: Data Science and Machine Learning; Programming; Data and Big Data Tools; Agile; Project Lifecycles.