An Introduction to the Positive Side of Big Data

There was a time when the phrase “Big Data” didn’t mean anything to me, and I know there are still plenty of people who don’t have any real understanding of it. Now that I need to talk about it in my job, I’m getting used to its first mention causing a lot of blank looks. While big data is getting more and more important in certain sectors of the IT world, it’s just not a topic that comes up regularly in professional conversations in most industries. I’m still occasionally surprised to find fellow IT professionals who are unclear on the concept. It’s high time to change that, as big data is poised to deliver significant benefits to people everywhere.

At the most abstract level, big data is just an accurate descriptive term used to talk about datasets that are exceptionally large and/or complex. Thanks to the unprecedented amount of data being generated by ordinary consumers – in the way they shop, browse, and learn on mobile devices and the internet – there are now vast pools of information available that can teach us a lot about the general public or about specific communities within it.

As an ordinary lay person first starts to learn about big data, the concept may seem a trifle ominous. A concern for privacy is entirely reasonable in the modern electronic age, and it’s a good idea to be concerned about how your data is used, shared, and manipulated. My educational background isn’t in maths or sciences – which hasn’t hindered my understanding of big data – but in law (undergraduate) and human resources (postgraduate). That means I’ve spent a great amount of time learning about the ethical implications of data collection and the importance of confidentiality. The good new on this front is that it’s actually easy to anonymise data generated by individual users as it flows into a big data system. That makes it possible to unlock all the benefits of big data without leaving behind any links between your results and the individuals who contributed their individual data to the data sets.

I’m now working as the Business and Local Government Data Research Centre’s Coordinator of Outreach and Engagement. This has given me some positive first-hand experiences with the possibilities of big data. Perhaps most importantly, big data can lead policymakers toward more accurate, informed, and useful decisions. A big data collection of mobile phone location data, for instance, could provide anonymous traffic information that could be used to plan out new routes for public transit. Big data analysis can help with a number of different economic improvements in the community: Everything from improving High Street businesses to identifying critical skill gaps in the workforce.

In a commercial setting, big data derived from loyalty card usage can help businesses identify trends and create new deals and special offers that benefit both customers and companies. Big data techniques also make supply chain logistics more efficient, reducing transportation time and cost and extending the shelf life of perishable goods. Big data is integral to research and this post from Capita IT Resourcing argues this clearly.

Big data can even have a positive impact on public health and safety by making it easier to identify vulnerable individuals and groups. Social care can be directed to the most effective areas and future needs can be accurately predicted.

The more I learn about big data, the more positive applications I see for it. Gleaning useful information from big data is a formidable analytic challenge that calls for specialized skills. The BLGDRC employs many researchers who have big data experience, and we pass along the benefits of their insight to Local Authorities and businesses. We help to train others in big data analytics as well in order to further expand the positive reach that big data can have.