Data and analytics are fundamental to digital transformation and are playing a broader role in creating business value – How can organizations infuse analytics in every part of the organization?

Data and analytics are no longer peripheral to the business. They are core to how organizations serve their customers, optimize business processes and are the foundation of new transformational business models and revenue streams. But, to achieve broad business impact, data and analytics leaders must extend these investments beyond individual departments and projects to empower everyone in the enterprise and beyond. Only then can you optimize every business moment, action and outcome.

Focused business driven leadership, cultivating a data driven culture, finding the right organizational model will play pivotal roles. It means tighter collaboration than ever with teams of people across the organization and beyond your borders. Growing the size and the reach of your data and analytics teams, bringing more of the right skills into those teams, and engaging more roles in a distributed way across the business to be a virtualized part of those teams will be a necessary condition of success.

Specific investments must be made to create new roles and responsibilities, such as the chief data officer to link data and analytics investments with strategic business outcomes and value. Building new skills competencies in data science and machine learning and AI, data engineering and importantly in data literacy and fluency for everyone in the organization are new cultural imperatives and organizational success factors in the digital era. It will be necessary to establish new ways of working and new data-driven approaches that exploit complex and diverse data assets, as well as diverse thinking and teams to spark creativity and innovation.

Importantly, making analytics pervasive requires applying best practices from initial pockets of success in data and analytics initiatives more broadly to all parts of the business. Investing in data and analytics as a strategic priority aligned to line of business priorities and outcomes and pushed out to all corners of the business is an imperative. Delivering on most digital business goals and objectives will depend on it.

What are the latest trends and some of the new innovations in data and analytics architectures and technologies?

The imperative of building an agile data-centric architecture that augments every aspect of the business and responds to constant change (Gartner calls this the ContinuousNext) has never been more critical to business survival.

The size, complexity, the distributed nature of data, the speed of action and the continuous intelligence required by digital business means that rigid and centralized architectures and tools break down. There’s now too much data, it’s already too distributed, it’s too diverse, it’s too complex.

The path to success requires making the right choices in the face of unprecedented business demands and rapidly changing and complex technology options. We know that a lot of organizations working on data and analytics struggle with balancing between investments that drive innovation and renovating their technology core.

The very challenges created by digital disruption – too much data - have also created an unprecedented opportunity. Leveraging these new vast amounts of data when coupled with increasingly powerful processing capabilities enabled by the cloud (for both data management and data science) makes it now possible to train and execute the algorithms at the very large scale necessary to finally realize the full potential of AI.

We expect to see a growth in the use and application of data science, machine learning and AI across all industries, particularly as AI matures and the democratization of AI accelerates. AI will no longer be for the privileged few. AI-enabled analytics and data management tools will empower the many to process the vast amounts of data needed for advanced analytics at scale. This will make it possible for a broader range of skills, such as citizen data scientist and developer data scientists to create AI-driven insights and embed them in applications used by everyone across the organization. AI as the new UI will feature prominently in this transformation where natural language and conversational interfaces will also open up insights to more people in the organization. Augmented and mixed reality, still in its infancy in the enterprise, will also start to play a role in the enterprise, beyond gaming and entertainment, including in data and analytics. With so much data and insights from diverse sources, data storytelling - part art, part technology-enabled - will be a critical personal and organizational competency necessary to drive optimized actions across the enterprise.

Moreover, the broad distribution of analytics capabilities to everyone in the enterprise is giving less technical individuals across the business the ability to generate value from data assets and analytics. To enable this growth we see considerable adoption of capabilities such as agile data preparation, data cataloging and metadata management technologies, and new adaptive data governance-related processes that are enabling content authors using any tool anywhere in the business to do more in creating and using trusted, high-value data. Having the ability to capture all of the knowledge about what data you have, where it resides, how it’s all related, who uses it, why, when and how — and then using that insight to provide more personalized, automated and properly governed solutions to the business. New roles such as the data engineer and processes supporting dataops and user communities are key enablers of this more distributed, but governed approach.

Capabilities such as digital twins are needed create actionable insight from vast amounts of IoT data while blockchain is maturing to cope with demands for trust in a distributed environment in the face of exponentially increasing data volumes, complexity and pace, and yes, bad actors.

So you want to be a Chief Data Officer? How can you succeed in creating a data-driven culture?

Along with the rise of digital business, the latest CDO survey shows we’re continuing to see the growth and expansion of the chief data officer role as more and more organizations view their data and analytics initiatives as strategic.

The CDO role is not only focused on data and governance, it’s also about owning analytics and driving how data and analytics can be instrumental in achieving organizations’ vision, goals and aspirations. The CDO office should have responsibility for data and analytics strategy, data and analytics governance, data literacy among the workforce, and establishing a data-driven culture. One of the findings from the Gartner CDO survey reveals that two-thirds of CDOs now have responsibility for both data and analytics.

The chief data officer (CDO) is the best role to maximize the data and analytics value in the organization. Not every company needs to adopt the CDO title, but every company does need someone to adopt the tasks of prioritizing or leading its data and analytics strategies.

CDOs most likely to succeed view themselves as champions of change. They’re leading the development of their enterprises’ data and analytics capabilities to innovate and create new data commercialization and monetization opportunities. Those focused solely on the internal, operational benefits of data and analytics are tending to be somewhat less successful than those who diversify their strategy and also drive top-line and transformational benefits.

Successful CDOs tend to be those that have the full support, buy-in and backing of their leadership teams, even to the degree of reporting directly to the CEO. They’re supported with the budget, the people and the resources needed to succeed. This places them in a far stronger position to not only scale data and analytics throughout the enterprise but also to have the headroom and resources needed to innovate and identify new opportunities to commercialize and monetize their data assets.

How can you not get left behind in the AI gold rush?

Digital business and the Internet of Things have fuelled an exponential growth in data and the need to analyse it - instantaneously. AI now holds the promise of being the technology that will transform business, every aspect of our personal lives and the broader society in an even more consequential way than the Internet did now decades ago.

Application- specific AI has now come out of the lab and reached a state where you can get real business value from it. It has reached a level of maturity where it's no longer theoretical, it’s no longer something only for academia or for rocket scientists or even specialist data scientists. Given the advances, most CEOs are desperate not to fall behind.

Leading companies have had early successes at embedding AI in customer facing and back office processes and some in making it central to their products and business models. Enterprise application vendors are embedding AI into core processes used by people across most industries every day. The additional of natural language processing technologies as an interface for interacting with all enterprise data and processes will continue to expand the reach and impact of AI.

Given the limited supply of advanced analytics skills, vendors in the data and analytics market have also begun to inject various kinds of AI into their platforms and technologies to reduce data and analytic complexity and expand insights to more people across the enterprise. AI being leveraged in tools for building analytics content (including analytics and BI platforms and data science and machine learning itself) and in platforms for managing, integrating, cataloguing and storing data. These augmented analytics and augmented data management capabilities are transforming how data is integrated and managed, how analytics content is authored and how insights are generated, shared and consumed. AI-enabled innovations are disrupting all of data and analytics leverage AI to automate tasks once reserved for people with specialized skills, reduce time to insight and bias and improve accuracy over manual, human-only driven approaches. This trend is a key enabler of the democratization of AI.

Establishing a focus on AI with specific leadership and skills and investing to operationalize and scale AI deployments will be key to broader business impact – as will an even greater focus on the data that feeds and trains algorithms. Data and Analytics leaders will play a central role AI success.

How can we build a foundation of trust and respect privacy while making the use of data increasingly central to business success?

Success with data and analytics will depend on building a foundation of trust, accountability, governance and security that respects privacy and promotes digital ethics.

Personalization versus privacy is as old as data and analytics itself. However, with data at the heart of digital business leverage both will be critical to success: How do we achieve more relevant hyper personalization that relies more and more data without being creepy? How can we leverage augmented analytics to autogenerate more accurate and unbiased insights and data science and ML/AI models results that are also transparent and easy to explain? How can we collect and use as much data as possible while also being able to curate it to the point where we can verify our data (avoid dangerous data) in time to better inform the business decisions we make? How can make sure that the algorithms we build or auto generate are ethical and unbiased?

The answer is governance: Resolving these ambiguities requires multiple levels of governance processes: We must be able to trust our data, trust our analytics and trust our algorithms. Trust is always having the ability to provide scrutiny through verification and transparency to better inform the business decisions we make. Trust is about making sure the algorithms we are run our business on are based on diverse data and free from bias. Capabilities such as blockchain, metadata management, knowledge graphs, and data catalogs, augmented data management will form the foundation of an adaptive governance strategy.

This level of transparency and verification is one of the key elements of the General Data Privacy Regulation (GDPR) which is now a reality for most organizations that have EU residents in their customer base, and or, EU employees. So GDPR requirements, like being able to explain why a predictive model treats one customer differently than a customer in another segment, will become common place when conducting digital business.