UPDATE: AlientVault service is performed and monitored by third party SOC 2 certified vendor.

Penetration Testing and security event monitoring are both crucial security methods. They can both be very expensive. Considering that AlienVault has built in Security Scanning tools and complex network environment understanding, can AlientVault replace, or mitigate the need for yearly Penetration Testing? It's security features continue to advance to the point where it appears to overlap with Pen Testing, in a manner of speaking.

Before you would recommend doing both every year, please consider the need for a limited budget.

1) Would you use an advanced Security SEIM, like AlienVault, to mitigate your company's desire to perform yearly Pen Testing?

2) If you would reduce the amount of Pen Testing, what would be your minimum baseline, where you would perform Pen Testing.

3) Or, if you have already had recent results from a External\Internet Pen Test, would you consider replacing future Pen Testing, with increased attention to Advanced Security SEIM?

5 Replies

Answer is no. A proper 3rd party penetration test is still a good thing or required thing to have on top of whatever you're doing in-house. Especially in a regulated industry, no matter what I did internally, auditors still need to see that a 3rd party test is done. Reason being is verification. Who's to say you didn't doctor the report or the scanning in a way to make yourself look good? If a 3rd party doesn't do their full job, the hammer is going to drop hard on them.

These tools could make it so you don't have to have a pen test as often, but that depends on what other factors you have, such as any regulations that required a set period.

UPDATE: AlientVault service is performed and monitored by third party SOC 2 certified vendor.

Hmm, that makes it more interesting. Is this 3rd party different from the one who does the pen testing?

Yes, different company. Pen testing is very expensive, so is security monitoring. For SMB under 100 users.

Yes, Pen testing helps to spot any missed vulnerabilities, although the final result is to reduce vulnerabilities. Correct? With today's advanced vulnerability assessments, and real-time reporting, I'm thinking that better real-time monitoring is a better investment for SMB's that Pen testing. In my humble opinion, it seems that Pen testing is not as helpful, when we have more robust, overlapping techniques to reduce vulnerabilities. Does anyone else have the same thoughts? Now, I'm sure I know what the safe answer is, especially if you are a security salesman, but we need to balance a realistic SMB budget.

Not every organization is in a similar situation. There's a saying I like to repeat: Generalizations are generally ineffective.

There's a couple different things to consider:

1. Vulnerability Scanning != Penetration Testing. -Vulnerability Scanning uses automated mechanisms to detect vulnerabilities in the devices it can see. -Penetration Testing can/should be scoped to include more than automated testing of devices they can see.It's true. Both a red team (where a penetration testing service resides) and a blue team (where a SIEM/Vulnerability Scanning service resides) have the goal of finding vulnerabilities and reporting them so they can be remediated. The truth is they go about it very different ways, and they are both needed for different reasons in most circumstances.

In a vulnerability scan and through SIEM configurations, your goal is to detect vulnerabilities so you can remediate them before they get exploited, and if they get exploited, detect those attacks so you can respond the best way possible.

In a penetration test, your goal is to find vulnerabilities using the same techniques malicious hackers use to verify your defenses, find your short-comings, and remediate those short-comings.

For instance, one of the big differences between the two is the use of social engineering (if it's in scope on a penetration test). An automated mechanism can't attack the human element like a human element can. Penetration testing tends to bring to light very different vulnerabilities than an automated vulnerability scan (Typically, a penetration test will use a combination of vulnerability scans and manual testing).

2. You need to consider what environment variables affect your organization's threat landscape. By this, I mean your organization's vertical, your size, your architecture, your compliance requirements, your existing protections, what you have that would be valuable to an attacker, etc. -A large manufacturing company is going to have very different requirements than a local eye care facility. They are going to have very different regulatory requirements most likely, and they are going to face very different threats.

You might be able to say for the eye care provider that has to deal with HIPPA and PCI requirements, an independent 3rd party penetration test might be necessary. You might deem a manufacturing company that produces plastic parts for office supplies might not need a penetration test. If that manufacturer produces small parts for the U.S. military, that's a completely different story, and an external penetration test would be required, and likely the SIEM/Vulnerability scanning as well.

At the end of the day, there are a lot of factors that determine what is right for your business, including the cost/value proposition and anticipated ROI for the particular services.