Numbers talk when it comes to boards and executive leaders, especially when those facts and figures tie to the bottom line. Most organizations are already delivering some reporting and metrics to those running the business, but often the cadence is not regular, the metrics are cumbersome to produce or are not well-defined across the organization. The key for boards and senior leadership is consistent and customized executive-level reporting.

In the Gartner report Innovation Insight for Security Orchestration, Automation and Response (Nov. 2017) they note that the challenges from an increasingly hostile threat landscape, combined with a lack of people, expertise and budget are driving organizations toward security orchestration, automation and response (SOAR) technologies. In that same report, Gartner identifies a dashboard that "provides visualizations and capabilities for collecting and reporting on metrics and other information" as one of the four functional components of SOAR.

Forum was asked to help the CISO of a large technology company develop a strategic-level view of the organization outlining the status of various security initiatives as well as an enterprise rollup view to support board-level discussions.

Data, Data Everywhere – But Insights Tell the Story

As with most organizations, there were thousands of data points around hundreds of metrics, however the client needed to decipher the important metrics to inform critical business decisions. Leveraging our data & analytics expertise and experience with boards of directors and senior executives, we crafted an executive dashboard that enabled the CISO to have real-time information at their fingertips. The automated dashboard facilitates discussion of security initiatives with the board of directors and senior leadership team as well as enables security-related business decisions based on consistent, accurate and trustworthy data.

Metrics are not the story; they are just the tools we use to tell the story

— Brian Hadfield

Formulating the Solution

The first step in providing the best solution was working with the client to define what purpose the dashboard would serve, where the data would come from and what capabilities were needed or not needed. Once we had answers to the high-level questions, we began the process of building the dashboard with the following principles.

Define the important metrics. Determine the measures and initiatives that are important to the audience – at the right level. Boards of directors and senior executives need an enterprise/initiative rollup view.

Define the single source of truth. Data is only valuable when it can be trusted, metrics need to have a consistent nomenclature across teams, a common and agreed upon calculation, and a consistent refresh rate. If a metric is reported in a roll up dashboard, it must be consistent with reporting through other modalities.

Determine how to best render the information. Many of the metrics are reported on individually and not always part of a rollup that provides an enterprise-view.

Uncover insights. Not all metrics could (or should) be included in a dashboard – only those that provide meaningful insights into the business. Find ways to use that data to provide a better understanding of how the organization is really working.

Engage key stakeholders. Metric owners need to be comfortable with their status rolling-up for broader reporting. These metrics may appear on other reports or dashboards but may not have been used as part of a broader conversation at the C-level. Data owners need to be involved and engaged with how this data will be used in the new dashboard.

Build for automation and scale. To drive long-lasting, meaningful value from the dashboard, it must be kept current with a minimum of manual effort. Automating the data collection and metric calculations must be the goal.

Keep the dashboard relevant. The metrics being reported on the dashboard should be reviewed at least quarterly to confirm that they still represent the important key performance indicators (KPIs). A process needs to be developed for adding and removing measures from the dashboard to ensure that the important metrics are represented as priorities change over time.

This dashboard Forum created has put a lot of truth into the room. It is fostering exactly the kinds of conversations we should be having regarding security.

— Security Executive, Global Technology Company

Results

The main goal of this project was to provide a vehicle to provide answers to critical business questions and align the business actions with corporate initiatives. In addition to achieving those goals, we were able to help the customer realize additional benefits.

Linking outcomes to activities. The dashboard provided a mechanism to show progress of an initiative over time with key activities identified in the timeline. This provided a mechanism to quickly evaluate how an activity affected a KPI and provide insights into driving effective outcomes.

Data confidence. Consolidation of the organizations data into a single data lake allows for a single source of truth for the metric calculations to avoid different reporting mechanisms to show different results.

Multi-level reporting. The dashboard delivered in this case includes 3 distinct levels of detail - each providing the right level of information for each audience group:

Top level view – Provides a red/yellow/green ‘at a glance’ view for the CISO to see the condition of the organization and report to the board.

Second level view – The detail behind metrics causing a yellow or red status at the top level to quickly determine which areas need attention.

Third level view – A detailed view of the data and calculations for a given metric enable actionable and immediate actions at a tactical level thereby saving valuable time in addressing critical issues.

Reduction in time. All of the information needed to prepare a board report is now contained in a single dashboard. There is no longer a need to elicit data from multiple sources in order to prepare an accurate report. The CISO and team can spend time addressing critical business aspects versus preparing for meetings.

Bring data together from different sources. To drive unique insights, the dashboard correlated data from multiple sources into a single visualization. This allowed for big-picture views of business initiatives that were not previously easily available.

Metric owner accountability. Having a single source for all data and metric calculations creates transparency to the status of an initiative. The calculations are done automatically by the dashboard based on agreed upon rules to show where each metric stands.

Summary

Dashboards will be the most visible part of a risk management platform. Identifying the security metrics useful to executives accountable for information security and risk management can seem like a daunting task. While there are common security metrics that report on operational health (how the processes, policies and controls are functioning), those metrics often do not provide actionable insights into organizational security practice maturity, return on technology investments or business objectives.

An effective dashboard will be the expression of discussions on KPI measurements, organizational maturity and business objectives. The discussions that lead to these agreements are critical to gaining buy-in from all parties and can serve as an agent of change independent of the dashboard. At first glance, a dashboard may seem like simply a software solution but Involving business owners from the beginning and doing the difficult work of sifting through the mountains of data to find the hidden insights are the keys to success. It is important to remember that the metrics are not the story, they are just the tools we use to tell the story.

Brian Hadfield is an experienced management consultant professional with deep expertise in technology, business strategy, operations, process improvement, user experience and program management. Brian combines his technical knowledge with strong business acumen to help clients build clear operational plans leading to measurable results.

Forum Solutions is a management consulting company that works with Seattle’s business and nonprofit leaders to build and implement effective strategies for transformative growth and sustainable results. Forum offers clients the right skills for every job: strategic expertise, lean execution and agile resourcing – improving businesses at any level – from the executive suite to the individual contributor.