What Is Pentesting?

Since a lot of people are apparently unfamiliar with the concept of penetration tests, or pentests for short, I want to give a short introduction to what we feel defines a good pentest.

In order to give you a good overview, I will go through the following questions.

What exactly is a penetration test?

What types of pentests are there?

What should the scope of a pentests be?

What is included in a typical pentest?

What comes after a pentest?

What exactly is a penetration test?

The term “penetration test” has multiple meanings, in the military jargon for example, it is both used for the process to test the penetrating power of bullets through solid objects and the attempt of special forces to penetrate a secured area or facility. While the first is more a test of power, the later refers to the testing of security measures for weaknesses in a controlled and systematic manner. It is the latter use that has been adopted in the information security field as a term for testing IT systems or applications for potential weaknesses. While “penetration test” is really the task of “getting in”, which would best translate to “getting admin rights” in the IT world, it is often used as a synonym for vulnerability assessments.

There is a distinct difference between pentests and vulnerability assessments, as the second is tasked with finding as many vulnerabilities in a given time as possible, whereas the first is tasked with trying to penetrate the defenses. A penetration tester does not need to look for further potential vulnerabilities once they are “in”, since the assignment has already been a success at that point. However, much like with the terms “hacker” and “cracker”, the difference has been lost to many due to the wrong use in of the terms by the media.

Because of this, LastBreach primarily uses the wider known term pentest for both assignments, since distinguishing between them could be confusing to some of our customers. The exact nature of the assignment is then discussed in our kick off meeting.

What types of pentests are there?

The different types of pentests refer to the target area, although the exact terminology might differ between security firms. If the target is based on web technologies such as websites, web applications or web APIs, then the tests are generally referred to as “Web App Pentest”. Testing a single server is usually called a “Host Based Pentest”, while a whole network is called just that, a “Network Pentest”. However, there are more pentests than just these three. Other areas include physical test assignments to inspect the security of an office building or area, which often goes hand in hand with social engineering pentests, in which the social aspect of security is targeted. Yet another type are reverse engineering assignments, where the task usually lies in trying to obtain sensitive information be dismantling an application down to the very core.

Furthermore, there are different ways to conduct a penetration test based on the goal or the assignment and the provided information. We already covered the goals based differences in the distinction between vulnerability assessments and penetration tests. Based on the amount of information and access we get, we can furthermore distinguish between three categories of penetration tests, white, black and grey box testing.

White box testing refers to an approach, where the tester is presented with a lot, if not all, available information and possibly access accounts. The idea is to let a well versed security professional take a close look at what you’ve created, from your point of view. The black box, on the other hand, aims to simulate an attack on the target. Therefore, the tester is provided with next to no information regarding the target, aside from what they are allowed to target and possibly some restrictions. These restrictions can include certain systems or areas of an application that are not to be tested, or attack scenarios that are to be excluded, such as social engineering. The grey box approach is the way in the middle and can go more into one or the other direction, depending on the requirements of the customer.

What should the scope of a pentest be?

Something that’s always part of the preparation for a penetration test is defining the target scope. This includes creating either a list of targets to attack or not to attack, as well as allowing or forbidding the use of certain tools, attack techniques or other restrictions. In our opinion, the scope of a pentest should always go hand in hand with the goal and purpose of your test.

If you want to do a white box approach, you can set restrictions more easily, as the tester should have enough information about the target without having to actually attack it to verify a vulnerability. If you are using the black box approach, chances are that your goal is to get a glimpse into how vulnerable the application is to an actual attack. If that’s the case, you should remember that an attacker doesn’t have a scope, or rather, that the scope of an attacker is EVERYTHING. By restricting the penetration test to certain techniques, hosts, or otherwise, you are effectively restricting the test results to the same.

Of course there is always the chance that a penetration test can result in damage, but think of it as this. The penetration tester you hired to inspect your application or network, does not want to harm or exploit you. They only want to do a good job. Should there be any event where damage is done during a penetration test, you have all the advantage on your side. The penetration tester will notify you immediately, you can work together with them to find and fix the problem. But above all, any good penetration tester will always proceed with caution if there is any chance that the result may have a negative impact, which means they won’t launch an armada of risky attacks all at once and see what happens, but rather figure out which one is most likely to succeed and then run them one by one, verifying the results each time.

So yes, you can restrict the target scope of your penetration test, but you should weight it against the restrictions it puts on your test results and compare them with your goals.

What is included in a typical pentest?

The scope of a penetration test differs from each security company, which is why I want to address this very thing in this post. In this case, I’m not talking about the target scope, but the scope of the assignment itself, as in, where does the assignment begin and where does it end. I’ve seen pentests that looked similar to this one.

Customer requests penetration test

Consultant starts automatic scanner and sends back generated report

Unnecessary to say, that this is a very bad example of a penetration test. Others I’ve seen where more like this.

While this is worlds better than the first version, it’s still not how I would like a penetration test to go down, because it doesn’t utilize the available time in the best way possible. The first example only ran a automated test and gave the results to the customer without even checking them for false-positives or false-negatives. The second example spends time on unnecessary tasks. I’m not saying that they don’t add value to the overall experience, but as someone who has been customer to penetration tests once, I know that the cost/value ratio isn’t right at that point. So here is how I think a good pentest could look like.

Kick off meeting to clarify the needs of the customer and answer any questions on the topic of the provided service. This should only be done at the customers site, if it is a requirement or part of a long-term engagement.

Any given information or credentials are verified before the assignment starts to avoid problems during the testing phase. The testing phase is usually a fixed time window that is not easily moved back and forth. Problems that arise during testing can sometimes be hard to compensate for.

Penetration test is conducted through both manual techniques, as well as automated tools. The right tool has to be selected for the right job and the combined results are written down in a detailed report. Doing everything manually or automated, either costs to much time or will result in a generic result.

The written report includes a management summary, as well as a description of each of the found vulnerabilities, along with an explanation of the associated risks, detailed steps to reproduce and suggestions on how to mitigate the risks.

Report is handed over and, if required, discussed to answer any questions and clear any uncertainties.

The consultants are available for further questions or, should help be required, to assist in remediation of discovered vulnerabilities.

After all vulnerabilities have been addressed, a retest is done to verify that the risks have been mitigated and no new vulnerabilities have been introduced in the process.

Points 4 to 7 are very important but often overlooked. The explanation of associated risks helps defensive staff to decide whether a vulnerability has to be addressed, or if it can be accepted as a known risk. If it has to be addressed then the steps to reproduce, along with the suggestions on how to mitigate the risks are a great help in fixing the problem. If both is not enough than the customer can always ask the penetration testers for clarification and if even that doesn’t help, the consultants can help finding the right solution for the problem.

While this might look like a bit much on the first glance, you should remember that the result of any penetration test should always be an increase in the customers defense. A penetration test is not there to tell the customer they’re vulnerable, in fact, most already kind of know that. It is there to tell them where exactly they are vulnerable and how to go on fixing it. To achieve that, the penetration testers or consultants must think from an defensive, rather than an offensive point of view, when it comes to presenting the results and, if necessary, be ready to help the customer in implementing the required changes.

What comes after a pentest?

As the previous section already highlighted, a penetration test isn’t over until the targets security has been improved. That is the ultimate goal of a penetration test and it strongly relies on both the quality of the pentest and the customer to take the necessary steps. But what happens after the vulnerabilities have been addressed and after the retest verified that everything is fine now? For one thing, security is an ongoing process, which means that doing a penetration test is a good way of increasing security but a very bad way to “tell you’re safe”. A penetration test can never verify that your application, host, network or office is absolutely safe because there is always a chance that something has been missed. However, security isn’t really about “being safe” and more about raising the bar, which is exactly what pentests do.

But let’s assume for a moment that the last pentest was a year ago and it was a huge success. The internet moved on, new and exciting things got discovered or invented and as a result your applications got updated with a multitude of new features, your infrastructure grew even further and maybe your company even moved to a bigger facility. As you can see, there is always room for improvement and just like that, there is always need for another pentest.

That’s not to say that nothing has changed though. With the increased security of the first pentest, you will soon realize that a much smaller budget might be sufficient for the second assignment, and an even smaller budget might be enough for the one after that. That’s simply because your environment got more secure as a whole and a penetration tester will now find less oddities and anomalies. If your hosts had all ports open during the first pentest than the penetration testers will have undoubtedly spend more time with each and every host. But if it’s obvious that the host only offers a secure version of OpenSSH on port 22 than the amount of time to be spent on that server will likely decrease rapidly.

So in summary there are two things you should do after a successful penetration test. First, think about the other areas that haven’t been addressed yet and try to find a place in your calendar and budget to tackle these areas. Secondly, document which areas have been tested, as well as the results, such as fixed vulnerabilities and accepted risks and determine when the next penetration test should be done. This could be in 12 months, or earlier, depending on how many changes are planned.

Questions and Comments

I hope this sheds some light on penetration testing as a service and the value to its customers. If there are any questions or concerns, please write them in the comment section or send an email to fredericmohr@lastbreach.com and I’ll try to update the post accordingly.