2013: Big Data’s Big Move from Pilot to Production

Rarely a day goes by that we don’t see a new analyst estimate highlighting the impact of Big Data. Gartner now claims it will drive $34 billion of IT spending in 2013, up from $28 billion in 2012. There’s no doubt big data means big business.

And while the past 12 months have been about pilots, this next year will be about production and rewards.

From business intelligence to product development and marketing, Big Data is driving disruptive change across industries. As more enterprises engage in Big Data projects, and start storing sensitive data, both opportunities and risks will rise to the forefront. Here’s what we’re likely to see in 2013:

The first big data company acquisition will happen, signaling a shift in focus from proof-of-concept projects to high-business-value implementations/rollouts. The first acquisitions will focus on vendors that provide business data analytics.

Vertical line of business applications on top of big data will start to explode, with some early examples already starting to emerge in retail, financial services and oil and gas.

Audit and compliance mandates will be imposed on the “wild west” of big data projects. On a related note, we will see the first big rash of class-action lawsuits related to big data breaches, adding millions of dollars to the costs of these incidents.

What IBM does, others eventually follow. To that end, the industry will see more corporate policy established around cloud storage such as Dropbox – and more technical solutions developed to enforce these policies.

Product differentiation will be focused less on infrastructure and more on application and service layers. As a result, more big brands will start open sourcing their architecture to encourage adoption and attract development talent, similar to what Netflix and Disney are already doing.

A damaging big data breach will cause the market to question holes and vulnerabilities in NoSQL infrastructure.

OpenStack is not overhyped as suggested by Gartner, but will succeed as an alternative to Amazon by providing security features such as default encryption and cloud entropy/randomization, and deliver solutions for companies that want on-premises cloud infrastructure.

As encryption and multi-factor authentication become more broadly adopted for cloud security, key management will become the new challenge. Solutions will need to provide strong policy enforcement and compliance capabilities, while maintaining high levels of flexibility and availability.

ARM servers will finally be deployed in data centers, supporting lower energy consumption and extreme parallel processing of big data. Intel will continue to build lower-power processors, but will be unable to keep up with ARM.

Higher integration of structured and unstructured data will lead to greater insights for organizations of varying sizes and industries and drive even greater adoption of big data.

There’s no question, the cloud, big data and open source markets will continue to mature in 2013, bringing with them an increasingly complex landscape of new business opportunities and challenges. The potential for enterprises to use Big Data to gain a competitive edge is huge. The time to get onboard and convert those pilots to production is now.

Larry Warnock is CEO of Gazzang, responsible for the company’s leadership, operations and strategic direction. He is the former president and CEO of Phurnace Software (acquired by BMC Software in 2009), and also held roles as chief marketing officer at Vignette, and vice president of marketing at OnLink Technologies (acquired by Siebel Systems). He brings more than 27 years of operational expertise working with startups and established technology companies to his role at Gazzang, particularly in the enterprise software and data center tools markets.