Search This Blog

Tools & Techniques - Key Performance Indicators

Introduction

To date, I've facilitated senior level reporting on the performance of security driven activity in almost every position I've held. For the best part, this has been a green-field requirement which has meant that I've been able to set out and make a case for reporting that drives improvement in the real-world security posture (RWSP) of my charge from the get-go. I believe, no matter what else is going on, that if you can inform the changes and behaviours that lead to your organisation protecting itself, you've had a good day. Secondly to driving improvement, KPIs and metrics provide your leadership team with the insight and visibility into the efforts and valuable work of your security team which might otherwise be hidden from them.

There is no shortage of materials and musings about the importance of security KPIs however, despite this, many organisations struggle in a number of ways to define and/or employ them. I thought then I'd share some thoughts on the subject and set out some simple parameters for defining KPIs.

What is a Key Performance Indicator (KPI)?

Lets stars with a definition:

A Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively your organisation is achieving key objectives. To this end then, organisations can use KPIs to evaluate their success at reaching targets.

A KPI is not the turn-key reporting from every product or service managed by the security team or that the security team has an interest in...

A KPI is not a dashboard representing the collective outputs from every client, server and device across your estate (aka dashboard fever)...

Understanding what a KPI should be and why/how it should be used compared with the raw data that might inform it is vital. KPIs should be specific to your organisation and some time and effort should be needed to design them. For a lot of organisations though, the need for KPIs has become confused with the need for teams, products and services to demonstrate their own intrinsic value. Vendors too recognise the need to demonstrate their value and typically include 'Executive' or 'Management' reporting or dashboards which typically become presented as KPIs.

Getting Started - Defining your Objectives

To develop effective KPIs you first need to identify your objectives. The amount of steer and support you receive from your leadership team is fundamentally important. A lack of management commitment will not only effect your ability to define KPIs but also your ability to make the necessary changes in their pursuit. KPIs should reflect specific business objectives. If you don't have the support of the business then KPIs, and the work that informs them just becomes another *thing* security tries to get people to do.

KPI Parameters

KPIs should comply with these parameters:

KPIs should be aligned with the strategic goals of the business. They should have a clear, documented objective (i.e. "Lowering non-compliant hosts below 1% total hosts will reduce our vulnerable surface area in line with the Executive's risk appetite").

KPIs should be attainable and their pursuit achievable. KPIs should not be aspirational or beyond the reach of your current capabilities.

KPIs should be acute and should help keep everyone on the same page and moving in the same direction. They should be approved by stakeholders before being tracked and reported on.

KPIs should be accurate, based on trustworthy and reliable data sources.

KPIs should be actionable and should provide insight and information into the business it's processes.

KPIs should be alive and reviewed, updated or amended periodically in line with the business objectives or emerging situations.

Developing KPIs

In addition to being customised, KPIs need to be developed over time as your organisation's objectives change and as the pursuit of these effects how security considerations and requirements adapt. Pursuing poorly conceived KPIs, or KPIs that are impossible, unrealistic or outdated can result in significant cost and disruption. Pursuing the wrong KPIs is detrimental and can even instigate 'bad' behaviours. If you don't receive feedback the relevance of KPI sets, ask...

Thoughts on Presentation

Executive level updates are likely to form a small part in a larger set of cross-business updates. To this end, brevity and impact are essential when considering how to effectively communicate with business leaders. Visual aids can be used to convey large amounts of information quickly. Traffic lights, scales and binary measures have always served me well. Know your audience...

Get link

Facebook

Twitter

Pinterest

Email

Other Apps

Popular posts from this blog

I was recently tasked with throwing up a replacement IDS box after an appliance 'died' in not-so-mysterious circumstances during some DC work. The IDS (Suricata) was stipulated, as was the base platform (RHEL 7).

I wanted to share here some of the notes I made during the build and subsequent testing, some useful links as well as one 'gotcha' I encountered along the way. These might cause you headaches in keeping your IDS running.

There are a ton of good articles already around covering how to get Suricata working on CentOS (RHEL's community backed spin off) but special mention has to go out to Daniel Miessler's guide which I've linked to below. In terms of getting Suricata up and running it really covers everything.

That gotchaYou can provide Suricata with parameters around pcap file management if you're capturing full packet and writing it to disk. These parameters are the size limit for each pcap file and the number of files to retain. For example, you c…

There are a couple of reasons why you might want to install Kali linux on an inexpensive hardware platform that you can deploy, abandon or hide somewhere. An obvious use might be to serve as an 'Evil AP' in support of wireless assessments. Kali linux is officially supported on a number of low-cost ARM based devices, with Offensive Security maintaining minimal, streamlined pre-built images which can be copied across to an SD card, installed and then configured with the packages you need for the task you have in mind.

Installing Kali Linux on a Raspberry Pi Offensive Security maintain good documentation here. For the our needs:Download and verify the image from here.$ shasum -a 256 /Volumes/SANDISK/kali-2017.01-rpi2.img.xzDD the image over the SD cards$ sudo dd if=kali-2017.01-rpi2.img of=/dev/disk2 bs=1mInsert the SD cards after the dd has completed and boot the rpi. I had a DHCP reservation set on my router so I knew what IP it would get. I also made sure I plugged in the extr…

Apart from being a
source of all too frequent and embarrassing typos, Splunk is a big data
platform which allows you to interrogate data and present results is a variety
of contexts and visualisations. I've been using it for a little over 12 months,
self teaching or Googleing as I go, predominantly to sift through the
terabytes of logs from various applications and appliances that get generated
in my 9-5 every day.

I've started to pull together all the searches, notes and bits of code into a sort of security cheat sheet which I thought would be a good thing to share as well as
providing some real world examples of how you might use Splunk in a security
context.

I'm actively working back through my notes and adding to this all the time so it might be a good thing to reference via the URL or re-visit from time to time. I'll try to keep this as accessible as possible and base it around real world examples and use cases.