5 Challenges of Utilizing Big Data in the Public Sector and How to Overcome Them

Smart public sector CIOs know that the effective use of data in a secured, proactive manner can make the difference between good and great service for citizens. It’s because of this awareness that researcher IDC says the federal/central government sector will be one of the top five industries investing in big data through 2017.

Big data in the public sector comes in multiple forms from myriad sources. With the right data platform, your organization can do more with its data without significant cash investments. But as you look to make the most of big data, there are some challenges you’ll need to overcome. Here’s an overview to guide you.

1. Overcoming the Status Quo

In many ways, outlining the positives of moving to a modern big data platform is easy. Most executives recognize the power of digital transformation. What can be tougher is dealing with legacy concerns. Estimates suggest that public CIOs spend 80 percent of their resources (or more) maintaining operational systems that are outdated. Legacy systems, in short, make it challenging to fully engage with a data-led modernization agenda.

Simply ripping and replacing existing systems will not fit with a public sustainability agenda. Your approach to big data in the public sector must be both scalable and sensitive to the ongoing business demands faced by the organization.

2. Beating Cultural Concerns

Over the past decade, the public sector’s preparedness to embrace open source has provided a considerable boon for data-savvy IT leaders. Many concerns about the stability and security of open platforms have now been set aside, so government organizations are using enterprise open source solutions and reaping the benefits.

But while a move to open source has helped bring down some long-standing barriers, you and your team are likely to discover that many people work in silos. This mentality can mean that individuals are wedded to decisions that were made a long time ago—a significant impediment if they also use outdated IT systems.

Rather than starting with technology, ensure your organization has broad support for its data-led changes. Focus first on establishing business objectives, and then demonstrate how big data can help your organization serve its citizens more effectively and efficiently.

3. Avoiding Vendor Lock-In

Like other technology purchases, big data technologies can come with strings attached. Avoid proprietary technologies that bind you to a long-term course of development, because spending decisions that are right today won’t necessarily be right tomorrow.

Rather than a prescribed approach, aim to support openness. Select vendors who allow you to add best-of-breed tools to your big data platform. Such flexibility means that your business can add new analytics tools, which analyst Gartner suggests are helping government staff make better decisions in real time.

Your funding requirements should match your wider big data requirements. What you should look to create is a long-term strategy that helps map big data technologies to clear business objectives, creating both significant savings and performance increases.

4. Prioritizing Security Without Being Prescriptive

Your organization is rightly cautious when it comes to protecting citizen data. While you can establish strong policies for internal employees, you must also ensure that your vendor partners have the right credentials. Look for partners who have formal qualifications, such as the Networthiness Certification Program, to help manage risks.

Always be aware of vulnerabilities, too. The fight against flaws and bugs is a constant battle. Your big data platform provider must have an extensive partner network that consists of companies that take a proactive approach to security vulnerabilities. Rather than waiting for flaws to be exposed, your partners should stay ahead of any threats.

You must also ensure that security policies and procedures don’t become too restrictive. While protection matters, so does enablement. Make sure your approach to security gives employees the assurance to use data safely and change direction when necessary.

5. Presenting a Single View of the Citizen

The strongest use case for big data is the single view of an individual across all public databases and organizations, whether it’s a citizen, patient, or soldier. But bringing this information together from disparate sources is a significant task.

Information often resides in silos. Interoperability can be limited, and the discovery of patterns can be a manually intensive task. Smart big data tools, such as enterprise data warehousing, can help ensure sources are brought together and trends are identified.

To make the most of this single view, your organization must pay attention to the presentation layer. It should draw on a range of partners and tools to make the most of your underlying big data platform. The right presentation layer can provide the compelling differentiator for your organization and its citizens.

The government needs to modernize its legacy infrastructure. A flexible approach to big data in the public sector, starting with the right big data platform, can help your organization update its approach to technology and information. Simply put, you can use a big data platform in ways you hadn’t previously considered, and this can create big benefits across your organization and the wider community.