What’s Up with 2019? Big Data Predictions

Alex Woodie

(Gearstd/Shutterstock)

As 2018 rolls to a close, it’s time to turn our attention to 2019 and the possibility that it holds. What will happen next year is anybody’s guess, which is half the fun in assembling (and hopefully reading) predictions from leaders and experts in the big data and data science fields.

Machine learning had a good year in 2018. But enterprises will embrace machine learning in new and profound ways in 2019, envisions Hilary Mason, GM of machine learning at Cloudera.

“Next year we’ll see a new step in maturity in the enterprise ML transformation as companies advance from proof-of-concepts to production capabilities,” Mason says. “Enterprise ML adoption will continue as businesses look to automate pattern detection, prediction and decision making to drive transformational efficiency improvement, competitive differentiation and growth. We’ll see infrastructure and tooling evolve around efforts to streamline the process of building and deploying ML apps at enterprise scale, including the rise of cloud-native platforms to enable elastic auto-scaling and multi-cloud portability for end-to-end machine learning workflows.”

“Low-latency, high-bandwidth networks mean more connected things, cars and systems – and a boat load of AI, machine learning and compute happening at the edge, because that’s where all the data will be generated,” the company says. “It won’t be long before we begin to see micro-hubs lining our streets – mini datacenters if you will – that will also give rise to new ‘smart’ opportunities for real-time insights happening on the corner of your street. Cities and towns will become more connected than ever, paving the way for smart cities and digital infrastructure that we predict will be thriving in 2030. And it’ll be a game changer for industries like healthcare or manufacturing, where data and information being generated out in the field can be quickly processed and analyzed in real time – versus having to travel back and forth to a cloud – and then readily shared with those who need it.”

In the market for digital immortality? Then you’re in luck, because next year AI knowledge graphs will begin to make that possible by synthesizing information from books, research papers, notes and media interviews and then resurfacing them in an interactive format, according to Dr. Jans Aasman, CEO of Franz.

“We’ll see the first examples of digital immortality in 2019 in the form of AI digital personas for public figures,” Aasman envisages. “The combination of artificial intelligence and semantic knowledge graphs will be used to transform the works of scientists, technologists, politicians and scholars

Courtesy: Futurama

like Noam Chomsky into an interactive response system that uses the person’s actual voice to answer questions. AI digital personas will dynamically link information from various sources – such as – and turn the disparate information into a knowledge system that people can interact with digitally. These AI digital personas could also be used while the person is still alive to broaden the accessibility of their expertise.”

“Whether affected by GDPR or not (most are), companies should be looking to it as a framework, it’s a good starting point for those building out their processes. It’s important to have something set-up for how data is kept and used. If we want to continue to have personal information protected, we will need to have more regulation. Next year, I believe we’ll see more regulation proposed and/or put in place, like the Consumer Data Privacy Act recently introduced by Oregon Sen. Ron Wyden.”

The decentralization of big data has been occurring for some time, and that trend will continue in 2019, predicts Jack Norris, senior vice president of data and applications at MapR.

“Organizations will save time and money by processing and analyzing data at the edge versus moving it back to a core, storing it and applying traditional analytics. Use cases include anomaly detection (fraud), pattern recognition (predicting failures/maintenance) and persistent streams. Autonomous

vehicles, oil and gas platforms, medical devices are all early examples of this trend that we will see expand in 2019. Cost drivers for this trend are bandwidth (semi-connected environments as well as expensive cellular) considerations and storage (reduce the amount of data

(James Jones Jr./Shutterstock)

sent to the cloud).”

Monte Zweben, the CEO of Splice Machine, has seen his share of ups and downs in the IT market. In 2019, he sees Hadoop struggling to maintain its place.

“New customer growth for Hadoop will dwindle and Hadoop clusters will slow in growth,” while cloud-based SQL data platforms see “massive growth.” Machine learning will emerge from the backroom and become a core operational component of the business, while continued growth of scale-out databases forces Oracle to disclose a risk factor in its financial report.

We’ve been living with distributed data silos for years, and the trend shows no sign of letting up. In 2019, we’ll be better prepared to handle distributed workloads on all that spread-out data, predicts Dan Sommer, a senior director with Qlik.

“One of the biggest unsung megatrends of today is the rise of microservices and Kubernetes. Together, these technologies take what used to be monolithic and disperse it, essentially enabling a new way to scale workloads and a third wave of empowerment. Just like scaling the hardware and scaling the infrastructure before it, scaling workloads will have a quantum-leap effect on spurring innovation. In 2019 the majority of enterprise architects at leading organizations will view microservices and container-orchestration as critical architectural components of BI and analytics platforms.”

Get ready for some of the data platforms you used in the past to begin making their exit from the enterprise, predicts Roman Stanek, CEO at GoodData.

“The modern enterprise will continue to edge out technologies like Hadoop. The merger of Hortonworks and Cloudera was a first look into the projected value for Hadoop in 2019. Technology that was designed twenty years ago in an era of ‘small’ data will no longer support the modern, global, and dynamic

(B-Media/Shutterstock)

enterprise. Data will still require management tools, but the complexity will be eliminated with the rise of artificial intelligence and machine learning.”

AI will go from hype to business impact in 2019, foresees Josh Poduska, the chief data scientist at Domino Data Lab. “The honeymoon is officially over for artificial intelligence. 2019 will be the year that AI will become an organizational reality, rather than experimentation, tinkering, and doubt.”

Machine learning will be integrated into products and services at an even greater clip, predicts Seth DeLand of MathWorks.

“Machine learning is already present in some areas: image processing and computer vision for facial recognition, price and load forecasting for energy production, predicting failures in industrial equipment, and more,” he writes. “In the coming year, it can be expected that machine learning will be increasingly present as more companies are inspired to integrate machine learning algorithms into their products and services by using scalable software tools, including MATLAB.”

It’s important for organizations to get the right balance between two data forces, including “offensive” data capabilities (i.e. exploiting insights in data to boost profits) and “defensive” data capabilities (such as governance and security), says Ron Agresta, Director, Product Management, SAS. There has been a disruption in the force, so expect a rebalancing towards defense in 2019, he writes.

“Extra scrutiny on data collection and usage has put many businesses on defense. Many companies rely almost exclusively on monetizing data relinquished by users, but regulatory attention is increasing in this area. Expect more laws for consumer data protection with the associated changes to technology needed to cope not far behind.”

CIOs will be on the hot seat in 2019, predicts Forrester. “If the CIO cannot deliver, there will be a breakdown in the business relationship and the CEO will look to others to lead the tech agenda,” the tech analyst firm writes. ” However, those CIOs that are seeing success will advance to even more influential C-level roles at their existing organizations or move on to the next IT challenge.”

Complexity has dogged would-be data science practitioners since the big data revolution started more than five years ago, with the Hadoop ecosystem a prime example. That’s why Tripp Smith, the CTO of Clarity Insights, sees a wholehearted embrace of simplicity taking place in 2019.

(Khakimullin Aleksandr/Shutterstock)

“More often than ever companies, are sacrificing performance or cost optimization in order to increase simplicity. Ten years ago, we were all trying to lessen or eliminate the need for internal IT in the enterprise,” he writes. “Today, organizations need to be data-driven and so how that data is analyzed and digested needs to be simple, too; and that means utilizing the cloud. Talent that doesn’t support digital transformation isn’t worth it. The cost of paying more for cloud infrastructure is worth it, freeing up greater cost of finding good engineering resources.”

The emergence of AI has already impacted the workforce, and that trend will continue in 2019, predicts Windy Garrett, vice president of Atos North America.

“AI is a clear need for the coming year and beyond and the collective workforce will keep growing to serve that specific need. In 2019, industry will see a significant uptick in universities ramping up AI programs, as well as businesses reskilling and upskilling their current workforces to remain relevant in competitive markets.”