Thursday, November 5, 2009

Time and time again I am challenged by clients and security professionals alike on what is the real benefit of penetration testing. Though this seems like an age old debate with many famous hackers and security professionals weighing in, I am not entirely sure I understand the argument undermining the importance/benefits of penetration testing. Below are arguments from both black and white hat security professionals:

White Hat – “Pen testing can show one of two things: your security sucks or your security is better than your pen tester”

Black Hat– “The very concept of "penetration testing" is fundamentally flawed. The problem with it is that the penetration tester has a limited set of targets they're allowed to attack, while a real attacker can attack anything in order to gain access to the site/box. So if a site on a shared host is being tested, just because site1.com is "secure" that does NOT in any way mean that the server is secure, because site2.com could easily be vulnerable to all sorts of simple attacks. The time constraint is another problem. A professional pentester with a week or two to spend on a client's network may or may not get into everything. A real dedicated hacker making the slog who spends a month of eight hour days WILL get into anything they target. You're lucky if it even takes him that long, really.”Though I understand the point of the above arguments, I believe the logic behind these statements is fundamentally flawed. The point they are trying to make is that an organization is never going to be entirely secure and that an attacker with dedicated time and resources WILL in all cases break in, so performing a pen test is only validating something already known. I see penetration testing differently. Penetration testing is not designed to make an organization 100 percent secure but to make them MORE secure (assuming identified vulnerabilities are remediated) and MORE aware what they were before the penetration assessment was performed. It also is a good means to test current logical and physical security controls. From my experience, many security professionals responsible for an organization’s security do not understand the full ramifications of vulnerabilities and thus can become complacent in fixing them.

Case Study:

A vulnerability scanner returns one vulnerability on Company A’s external presence. The vulnerability identified was Cross Site Scripting or SQL Injection. A report is issued to Company A showing a “High” risk rating based on this vulnerability. Company A may not understand how a single vulnerability can translate into a “High” risk rating and thus chooses to ignore or at least delay remediating this vulnerability until time and resources become available. Does this mean Company A’s security is bad? I would say if that if Company A had an external presence of 50 servers and 10 applications, and only one vulnerability was identified, the answer would be no. Company A may very well have good security, but as with everything else in life, mistakes happen. Now let’s assume a penetration assessment was performed on Company A’s external presence instead. Not only would the penetration assessment identify this vulnerability, it would attempt to exploit it. Let’s go on to say that this vulnerability is in fact exploitable and allows for full system compromise. What is a client going to react to more? A report stating they have one critical vulnerability stating what could happen or a report stating they have one critical vulnerability and, oh yeah, by the way we compromised your entire domain controller? A client who just had their entire domain controller compromised is going to be more inclined to fix the vulnerability in a timely manner than one reading a report stating what could happen if not fixed.

How can one argue that this penetration assessment was not beneficial to Company A? It effectively made Company A more aware as to the dangers associated with a critical vulnerability, which in turn made them take a proactive approach to fixing the problem almost instantaneously, thus reducing their overall risk rating.

This is one simple example. I can go on and on about the benefits of penetration testing. Security is about managing and reducing risk to an acceptable level. A penetration assessment isn’t intended to reduce an organizations risk to zero percent, but then again neither is any security assessment. Any time an organization connects a device to a network it assumes a certain amount of risk. It’s understood that zero-day vulnerabilities will always surface and cannot be prevented. So sure, a dedicated attacker could decide to spend 6 months developing an exploit for an unknown vulnerability, however this is going to take a more sophisticated attacker which makes this a less likely scenario.

A penetration assessment is simply used as a means to identify vulnerabilities and provide proof of concept examples on exploiting these vulnerabilities. By doing so, it effectively better explains ratings associated with vulnerabilities which in turn produce much more conscious/aware security professionals. A much more aware security department will be able to better help reduce the overall risk for an organization.

I don't quite buy it. If the vulnerability is reported as High Risk, yet the client doesn't act to fix it, they are responsible for the misstep. When compromised, the resulting investigation will reveal that the issue wasn't taken seriously, and those responsible will no longer treat a high risk issue so casually.

On the other hand, if a security manager is so under-resourced that a large compromise is only a matter of time, a pen test will open the eyes of those that sign the checks like nothing else can.

I think that the point being made is that Penetration Testing is an attempt to achieve a better balance of being secure, and that's exactly what security is...a balance.

Andrew highlights a very key point, that given enough time, anyone attempting to access a system WILL likely succeed. To fall victim to a known vulnerability is probably not a desirable thing, however security must be balanced with other functions, such as accessibility. If having to take something offline to apply a patch impacts productivity that is a company need, the company must balance the risks, and apply some sort of mitigation (perhaps monitoring access to this particular server and looking for signatures until it can be patched).

Penetration testing, like most other security measures, isn't a 100% solution. As more technologies are introduced and vulnerabilities for current and old technologies are developed, penetration testing becomes a tactically tailored approach to your company. A process of:

1. Here's the technology that your business uses.2. Here's the vulnerabilities that those technologies pose.3. Here's what can be done to mitigate and reduce vulnerabilities, at least for today.

It addresses the technologies you have, and maybe implements ones that work more securely. I'd recommend consistent Penetration Testing as more technology is put in place or changed, to ensure best practices are being followed.

not everyone's ready for pentesting, even though they may be required to do it.

in some cases, pentesting adds so little value to what the customer actually needs, it's a little sad that we're getting paid to do it. I've walked into engagements where goal-oriented pentesting is similar to kicking a baby in the face. Generally not an advisable course of action. You'll do more good to work with them get a VA program in place, or help them them on process of deployment / maintenance or SDLC.

not to quote the DSS, but their prioritized approach (which is actually pretty good) specifies that pentesting be done absolutely LAST in the set of steps that you should take to secure your data. After monitoring. After encryption.

ALSO. PENTESTING DOES SUCK. mostly because it's used at the wrong times, or in the wrong places. or the pentester sucks. or they're damn superstar, but they can't express it in writing - i've see it all. pentesting /requires/ intelligent human input, and all the problems that come with it. (varying skill, <100% coverage, generally erratic behavior)

However, as pentesters, we should be providing as much transparency as possible.

the more assurance you can give that you covered the set of targets to completion, the better off you're going to be. For instance, did you hit all ports, or did you touch all pages in an application (much like a code coverage metric) and did you check for all known weaknesses and check all known attack patterns.This is why we use VA / automation / known-quantity tools to cover basics. pentesting builds on top of that.

Even touching every input vector(for all known weaknesses) isn't going to be sufficient - business logic flaws require intelligence on top of all of that automation. almost a "meta" of all the baseline work. this is why we'll always have pentesting, in some form or another.

on that note, what you DO NOT cover in a pentest is just as important as what you DO cover. you can make statements about what you DO cover & compare with items in your experience. i've seen way too many pentest reports that read like finding,finding,finding, but no indication of what was covered / not covered. i guess that's the idea of the scope, but a set of ips in the scope leaves a lot to be desired in terms of what was touched.