I just got a quick question for you "seasoned" pentesters. Do you guys stick to Backtrack and other open source tools when performing pentests or do you use a combination of bt and Canvas/D2/Nessus(ProFeed)/CI?

And will there be found exploits added to the commercial applications that can not be found in places such as exploit-db? (Maybe due to some "copyright"? )

Im not sure how much "insight" you get into the code used in the commercial applications. Even though it has a pretty GUI and all you still should be able to see what you are doing.... Not just "point&click"

From the number of penetration tests conducted on systems managed by me, the professional penetration testers have never used live exploits on ANY production systems.

Should the penetration testers raise a potential exploit risk, then I may ask them to run it against a lab environment machine. I'd expect them to understand inside and out what the exploit is, does and what the code does.

I always confirm with external companies what tools and methods they will use to scan and test environments. If any one was stupid enough to start fire exploits without permission - open source or closed - they'd be dismiss and taken to the courts as breach of contract.

Having many fancy tools may make a fun time playing in other networks, doesn't make a useful pentester to companies that want to understand risk in their environment.

My suggestion is that you avoid being focused on exploits and look at the bigger picture in which tools play a much smaller part in a penetration tests, because in the long run that's what sorts the script kiddies out from the experts.

i see exploits as cheating during a pentest. remember that 0day exploits are, ofcourse, not patched and are always a threat. its our duty to expose threats that are there due to misconfiguration, policy, lack of updating systems etc. During this the use of ANY tool is allowed, just think every step through and get the potential damage done by it clear, and from there decide to continue or notify the company.

CISSP, CEH, ECSA, OSCP, OSWP, eCPPT, eWAPT

earning my stripes appears to be a road i must travel alone...with a little help of EH.net

jonas wrote:I just got a quick question for you "seasoned" pentesters. Do you guys stick to Backtrack and other open source tools when performing pentests or do you use a combination of bt and Canvas/D2/Nessus(ProFeed)/CI?

And will there be found exploits added to the commercial applications that can not be found in places such as exploit-db? (Maybe due to some "copyright"? )

Im not sure how much "insight" you get into the code used in the commercial applications. Even though it has a pretty GUI and all you still should be able to see what you are doing.... Not just "point&click"

My answer is going to be quite different from the others in the sense that I WILL and HAVE exploited systems during blackhat/red team pentests. There is a huge difference between a vulnerability assessment and a penetration test. There is no true value in reporting a "suspected vulnerability" as it is wasted money.

Imagine you have the following report: (shortened to make this post bearable)

Port 139, 445, 443, 80, 21 open and running vulnerable versions...

What would you do? As a security manager doing risk assessment, you'd have to put controls in place. Justify an expense: Procure firewall to defend, have engineers spend time locking down... There is a cost Now what would you do with the same information reported only with notes stating although these services and or ports are opened, the are NOT exploitable. It's of no value other than a VULNERABILITY ASSESSMENT to NOT exploit potentially vulnerable services:

"I walked by your house and saw the door opened... I could have robbed your entire house!"

Versus:

"I walked by your house and saw the door opened... I was about to walk in and saw the Rottweiler. There was no way I was going in there..."

Tools are tools are tools are tools... Penetration testing (full exploitation) is the only real way to validate the existent of a hole. What you want to do is be responsible about it and try to make sure the risks don't outweigh the benefits. If you know FACTUALLY that by trying to exploit say IIS will force IIS to crash first, then you want to present the option beforehand so there is awareness. In a responsible pentesting assignment, they may allow you to attempt the exploit at say Sunday 2am where the risks are so low no one would notice. Trying to mimic an environment is not always feasible as there is no way for one to determine patch levels, custom configurations, rules, service packs, etc. It's just not feasible.

In your testing objectives you need to clarify WAY BEFOREHAND what you will and will not do. These need to be understood clearly not only by you but your client. You should not be fearful to outright state: "We will diligently adhere to a strict framework to avoid disaffecting your business. Should we find an instance of a potential SOMETHING we will notify you before we proceed" Carefully spelling things out goes a long way.

The goal of a blackhat pentest is usually adversarial. Therefore all is fair in love and war. Your client should be aware of this. There is no malicious hacker on the planet that will stop and say "Gee, if I run this 0day, it will crash IIS" The same need be understood by your client. This is not to say you will be wreckless. On the contrary, you need to point this out beforehand.

I am with sil on this one. I feel like exploitation is absolutely part of a pen test. This tends to be the more advanced part of the pen test, and where the value of the pen tester is. I feel like the client can probably find someone in-house to just run Nessus / Retina / Qualsys, or whatever else on their own systems. All these tools produce to many false positives and to many false negatives to be taken at face value.

Exploitation confirms that the vulnerability is actually there. Besides, hackers won't stop at a scan. They are not just there to suppose that a vulnerability exists. They want in. It is my opinion that it is our job to emulate that hacker and get as far in as you possibly can, within reason. We also have to compromise as many systems as possible, in a much shorter amount of time. This is where tools help tremendously.

Those are just my 2 cents. I feel like without exploitation, the client is left scratching their heads when they get a report with an MS08-067 vulnerability on their Redhat machine.

To me, that's the value of having professionals that help the client scope the review and can advise the client of risk before pressing a button.

Having all the latest and great tools is way down the list of requirements.A skilled professional explains how exploits can be chained to achieve to target in terms for non-it folk to comprehend, this beats automated tools showing "bad stuff". It's smart people that work out what the exploit can get them, the tool just gets a # prompt.

The goal and reason of a pentest, as a client, is to display risk and offer cost options to mitigate it (or get a tick from the auditors, but that's just too cynical). Yes, these tests just brush the surface, but if that's what the company wants, it's their choice and risk.

Personally, when working with penetration testers we provide a great deal of detail about the network and systems to avoid the time wasting recon phase. I'm not really interested in having you scan all the ports of my class x networks, especially when I can provide the daily nessus/system7/Qualsys ones we do as SOP. We also shipped them off a standard laptop with the SOE on it, as an example of providing a more rounded review.

I've never been able to convince a company, and frankly had no need, to ask for a red team approach. The current pen test reports and a solid verbal break down has lead to a number of proof of concepts/demos and that's what they (the management) are willing to except. They don't want to spend money to fix old software or on increasing the security expenditure. The business has never been interested in taking the risk and especially with the systems that have been highlighted as prime targets - SAP, SCADA, WOPR etc.

This differs with each company and their level of threats so, each to their own. However, I would love to be on the blue team to either of you working as red team, as those times really do sharpen and develop IR skills and drills.

@Chrisj - With the massive event of virtualization, more companies are able to clone production systems in to a stand alone lab environment. This provides a great place for the security team to play bad guys in. You just have to remember to revert the lab back to it's previous state before the infrastructure team have to practice their tweaks and updates :-)

I understand where you are coming from, but there are way to minimize the risk (in my opinion). We discuss the risks of live exploits, we discuss the scope and what's off scope before the test. It is usually very easy to agree on which systems can be tested. Most larger businesses have development boxes that are configured almost identically to the production boxes. Sometimes we test those. I don't have any experience testing SCADA systems, so I can't comment on those.

There are going to be some holes that an automated scanner can identify and there are going to be some public exploits out there that you can easily use to root the box. There is value in stringing together public exploits still, especially for those who undervalue security. To me, the most valuable items on the report are those vulnerabilities that are discovered during the test. I find that the majority of those revolve around the custom applications.

As far as risk, value, cost, etc, that's a given to me. Any decent pen test report should identify the risk value and the cost and time line to remedy. It's also not enough to simply hand over a report. I feel that the pen test team should sit down with the client and go over the findings in great detail.

I think that this is a very important discussion. How each side perceives the value of a pen test is priceless to me.

What90 wrote:Having all the latest and great tools is way down the list of requirements.

You seem to be pointint to/referring to/and or confusing a risk assessment not a penetration test.

What90 wrote: A skilled professional explains how exploits can be chained to achieve to target in terms for non-it folk to comprehend, this beats automated tools showing "bad stuff". It's smart people that work out what the exploit can get them, the tool just gets a # prompt.

This is more along the lines of risk management and security awareness so again I point to the analogy of "telling me the door is open, I could get in by..." versus: "I tried to walk in the door but once inside found out it was a false wall. I couldn't get anywhere..." (https://www.cia.gov/library/center-for- ... ality.html)

Many years ago during the cold war, the CIA would build its building with the first few feet past the main wall with a "false wall" where if someone tried to eavesdrop, they would see/hear nothing. Telling me "I could get in with X" versus "I GOT IN with X" is a whole different ballgame.

What90 wrote: The goal and reason of a pentest, as a client, is to display risk and offer cost options to mitigate it (or get a tick from the auditors, but that's just too cynical). Yes, these tests just brush the surface, but if that's what the company wants, it's their choice and risk.

Personally, when working with penetration testers we provide a great deal of detail about the network and systems to avoid the time wasting recon phase. I'm not really interested in having you scan all the ports of my class x networks, especially when I can provide the daily nessus/system7/Qualsys ones we do as SOP. We also shipped them off a standard laptop with the SOE on it, as an example of providing a more rounded review.

This sound more along the lines of an assessment in fact, the entire NSA IAM and IEM (http://www.isatrp.org/) certifications on built on this type of audit. While there isn't nothing wrong with the approach, it WILL lead to a false sense of security coupled with wasted money. Again pointing to the analogy of my door being opened... "So you're telling me someone can walk in the front door?!" There I go wasting money trying to secure that front door not even knowing if the THEORY holds true that someone can walk in. Versus: "So what happened when they tried to get in? Where they successful? Do I need to spend money locking that door down?"

What90 wrote: I've never been able to convince a company, and frankly had no need, to ask for a red team approach. The current pen test reports and a solid verbal break down has lead to a number of proof of concepts/demos and that's what they (the management) are willing to except. They don't want to spend money to fix old software or on increasing the security expenditure. The business has never been interested in taking the risk and especially with the systems that have been highlighted as prime targets - SAP, SCADA, WOPR etc.

I've always made it a POINT to tell the company that a vulnerability assessment won't yield true results (door analogy) and leave it up to them to make that decision. There would be nothing more embarrassing than telling a company: "Well you have X service running and an attacker could utilize..." To be stopped by an engineer who is aware service X is running and has already placed compensating controls in place. It's a waste of the clients time and money.

What90 wrote: This differs with each company and their level of threats so, each to their own. However, I would love to be on the blue team to either of you working as red team, as those times really do sharpen and develop IR skills and drills.

Blue team, red team. Threats are threats are threats and with client side exploitation ranking high on the totem pole of compromise, I prefer performing both methods of testing. Inside, outside. Both from an adversarial perspective. There is a huge difference in telling someone "I could walk out the door with your server..." versus "here is the copy of the server I walked out the door with." Which do you think holds more weight?

Most companies in my experience that solicit pentesting, vulnerability assessments are already aware that a malicious attacker can perform an attack. Think about this for a moment.

Client thinks to themselves: A malicious attacker can get in the door, let me hire a company to determine whether this is so...

Vendor: A malicious attacker can get in the door

What have you done for them other than tell them what they've already suspected versus:

Client thinks to themselves: A malicious attacker can get in the door, let me hire a company to determine whether this is so...

Vendor: A malicious attacker can get in the door if you had that configured improperly. I tried but couldn't get in....

To each their own. I see more value in pentesting than I do vulnerability assessments and have often had to fight with PCI assessors over this: "Well AppScan says this is insecure" versus "Prove it" There are too many variables to get wrong during a vulnerability assessment and the last thing you'd want AFTER performing an assessment is to find the company was 'owned' because you missed something, overlooked something, relied solely on the output of Nessus/AppScan/etc. Aside from that, the problem with the majority of tools is that... They CAN'T THINK and DON'T THINK like an attacker. They often "THINK" the way an attacker did months or years ago, then created a rule to *try* that vector but an attacker is resourceful and this is what should be tested and defended against.

Fair enough sil,I'd imagine the people that hire you in a vested interest in proving if their security is working as expected. I'd be interested to know how many companies take the extra step and purchase the full package.

Fear factory of exploiting systems The problem with live exploits on live system is that you may break something, even with the upmost care taken. You find we have well known system X, and have exploit Y for it, and get sign off to run it. Me, as the client, didn’t know that the loony coder had added crappy code to the app and your exploit nails our production SQL server. This stops the factory and Z number of million dollars is lost while we scramble to fix the broken system. This would be an excellent way of generating new career “opportunities” at other companies for me... Would a bad guy do this to get the gold – without a doubt - but is a pentest, rather than a code review or assessment, be the safer option to discover this?

Your analogy of the door is very clear, but it highlights the bigger problem of a pentest; it only focuses on what you’re allowed to test. You do a stellar job showing how to get in and how the company can defend it. What about the un-scoped windows, chimney, cellar or garage entry points?

Most companies in my experience that solicit pentesting, vulnerability assessments are already aware that a malicious attacker can perform an attack

Or from what I keep seeing, sadly, is to complete a check box for the auditors. This goes along with the any form of compliancy, firewall -check, logs - check, "pentest" of web sites - check.

[side track]

Take the requirements of a medium size company of 2000 staff with standard internet presence (couple of interactive web sites, vpn, remote email). They don't have a secret sauce and their most important data is spread over file shares, in email and in databases. The have a couple of small offices in remote locations. The IT group consists of 10 people, with one server engineer who's done a couple of security courses, out of personal interest, rather than requirement. They have firewalls, AV and keep system relatively up to date. Security has no priory and availability of internal systems is the number one priority.

Would a company like this be aware of what threat agents that are placed to attack the organization? Would it help to undertake full exploitation testing? Do you think most small to large companies have that level of awareness or care factor about security?

The company would have to have some cost metric to justify the effort, time and expense of a full test. The need to balance this against what they believe a breach will cost them is paramount.

If that test isn't demanded from the board of directors, then do you think it will get anyone from that company any closer to a better security posture?

Showing that you could become a domain admin in 30 minutes or walking out with a server's contents is a huge, in your face, visual statement, but what if only to the IT group or mid-management? The CEO needs to see the results and the effect it can have on his business for real change to be made.

I'm not disputing that a fully scoped pentest of that level has its place, it is just a difficult sell in most companies I had experience with. The pages of the major business news have been full of TJ Max, RBS, the US government and a large pile of fortune 1000 companies which have been successfully attacked. Why isn't there a line of board level execs demanding security and the best validation that the measures actually work?

Is it that they don't care, understand or think it won’t happen to them?

[/End of side track]

Aside from that, the problem with the majority of tools is that... They CAN'T THINK and DON'T THINK like an attacker. They often "THINK" the way an attacker did months or years ago, then created a rule to *try* that vector but an attacker is resourceful and this is what should be tested and defended against.

I think we could spend hours debating whether exploits should be use, over beers preferably, but this goes back to my original response to jonas’ question. I'd rather hire someone who can argue his corner and put across why they should go the distance to the business and can adapt to the situation in hand, rather than someone just having shiny pile of tools.

What90 wrote:Fair enough sil,I'd imagine the people that hire you in a vested interest in proving if their security is working as expected. I'd be interested to know how many companies take the extra step and purchase the full package.

Almost all companies that have met with my managers, etc., almost always (at least 99%) gone through a full blown NETWORK/HOST based penetration test - most opt out for social engineering. When we get RFP's or calls we make them aware of the differences of a vulnerability assessment and a penetration test. What most opt for is a dually visible test (blackbox in, whitebox out). From the outside perspective, it's zero knowledge, from the inside it's whitebox based. This method (whitebox in) allows us to have firsthand knowledge in the event we overlook something.

What90 wrote:Fear factory of exploiting systems The problem with live exploits on live system is that you may break something, even with the upmost care taken. You find we have well known system X, and have exploit Y for it, and get sign off to run it. Me, as the client, didn’t know that the loony coder had added crappy code to the app and your exploit nails our production SQL server. This stops the factory and Z number of million dollars is lost while we scramble to fix the broken system. This would be an excellent way of generating new career “opportunities” at other companies for me... Would a bad guy do this to get the gold – without a doubt - but is a pentest, rather than a code review or assessment, be the safer option to discover this?

The problem with an attacker is that he won't care whether he does. We attempt our best to keep our timing parameters down, to perform extreme testing at low level hours and often we ask beforehand if they can either mirror the EXACT server on a development box to avert this. You state: Would a bad guy do this to get the gold – without a doubt - but is a pentest What is your definition of a blackbox test? In my own testing gigs, I'm always sketchy at running "anything" in fact I can't recall the last time I have run "anything." When I'm attacking a machine from the blackhat level it is a very targeted, structured and well planned out test. There would be no need for me to launch N amount of worthless exploits for the hope of getting in. I focus on specifics which minimize the potential of breaking something. Now, on a whitebox (internal) test, this becomes even more focused as I know the specifics of what I'm after. There is no need for guesswork. The risks are even more minimized for example... Now I won't have to worry about fiddling with attacking say an Apache server as I am now aware that - that Apache server - is nothing more than a proxy. No need to run Apache based tools....

There are plenty of mechanisms to exploit servers without bringing the house down and to be honest, the point you make is sort of moot and I mean this in a respectful way. Go back and take a look at all the Aurora based attacks and let me know at what point in time did any of those companies (GE, Google, Yahoo, Rockwell, General Dynamics, etc.) have their house crumble down even AFTER attackers loaded COCKTAILS of exploits to get in.

What90 wrote:Your analogy of the door is very clear, but it highlights the bigger problem of a pentest; it only focuses on what you’re allowed to test. You do a stellar job showing how to get in and how the company can defend it. What about the un-scoped windows, chimney, cellar or garage entry points?

Isn't this what you specify from the onset. What you will and won't be testing. If a client stated they only wanted this tested that's fine. I would definitely point out that the lack of testing on another door, chimney, garage may still be an entry point and let them make that decision. My bottom line in a pentest:

I make this quite clear to a client and make it quite clear that an attacker may not simply focus on a webserver. As long as a business is online in any shape form or fashion, there are dozens of attack vectors. I offer the low hanging fruit first as that is the obvious and quickest for clients to understand but I won't beat around the bush to them and state: "Well I couldn't compromise your IIS server behind an Apache proxy running on OpenBSD, therefore you're safe!" On the contrary I will let them know that although their forward facing connection is untouchable, there may be other mechanisms to get in." I make them aware and let them choose what to do - it's their business and their money.

Again the analogy: Client wonders after seeing the news and hearing about a hacker... "Am I secure?" The client already is aware that hackers are attacking and that they may be insecure (vulnerable). What service am I giving them in stating: "Well, you MIGHT be vulnerable!" ... "K now pay me" as opposed to the obvious: "I tested the vulnerability and can validate that you're NOT vulnerable. Which one has a deeper/worthwhile benefit.

I will snip out portions to keep this down to a readable/manageable thread so forgive me for just snipping out to the questions and spacing them out for clarity.

What90 wrote:Would a company like this be aware of what threat agents that are placed to attack the organization?

Would it help to undertake full exploitation testing?

Do you think most small to large companies have that level of awareness or care factor about security?

The company would have to have some cost metric to justify the effort, time and expense of a full test. The need to balance this against what they believe a breach will cost them is paramount.

If that test isn't demanded from the board of directors, then do you think it will get anyone from that company any closer to a better security posture?

Is it that they don't care, understand or think it won’t happen to them?

[/End of side track]

My opinionated answer to the first question: The problem with MANY businesses is, they won't know or even care about what threat agents are lurking. Their main concern is about the posture of their security. Can someone GET IN. If so, then how can they get in and HOW do you defend against it. Even experienced security admins and engineers can overlook something/anything. Does this mean they're at fault for something. As a pentester, remember my goal is to get in. There is no defacto "attack vector", it's whatever I choose to use to get in. You can't have automation to defend against a human brain.

My opinionated answer to the second question: Absolutely it would help. You can never (I repeat NEVER) give any tangible results with an assessment. All you're giving is a theory, a hunch.

My opinionated answer to the third question: Most companies won't have that level of awareness and if most understood the true risks associated with NOT understanding, they'd care a hell of a lot more:

A New York marketing firm that as recently as two weeks ago was preparing to be acquired now is facing bankruptcy from a computer virus infection that cost the company more than $164,000.

Karen McCarthy, owner of Merrick, N.Y. based Little & King LLC, a small promotions company, discovered on Monday, Feb. 15 that her firm's bank account had been emptied the previous Friday. McCarthy said she immediately called her bank - Cherry Hill, N.J. based TD Bank - and learned that between Feb. 10 and Feb. 12, unknown thieves had made five wire transfers out of the account to two individuals and two companies with whom the McCarthys had never had any prior business. http://www.krebsonsecurity.com/2010/02/ ... king-loss/

You think that if I showed that to a small to mid sized company to make them aware of the threat that their "care factor" (as you put it) won't go up?

Reponse to your comment: The company would have to have some cost metric to justify the effort, time and expense of a full test The cost metrics come out of the costs of doing business. If you're performing a business impact analysis ESPECIALLY on a mid sized company, A pentest even at say $100,000.00 is peanuts in comparison to a compromise or running out to buy $100,000.00 in firewalls, IPS, IDS, training, etc., only to find out (drum roll): "You got served!" You still were owned because some security company only gave you an assessment using tools that scanned specifics. They didn't know about programZ running deep in the trenches. Cost of compromise in monetary loss, consumer confidence, potential fines (PCI, SOX) is peanuts versus a pentest.

Answer to your fourth question: "If that test ... security posture?" There are still other ways to minimize their risk to acceptable levels. This begins with awareness and training. For example, the cost of me contacting say 3 pentesters for a weeks worth of training their staff to perform in-house pentesting is almost always going to be less than actually hiring a shop to perform a lengthy test. The benefits of this is, your staff will have the capabilities of performing recurring testing at no cost to you. It would be inclusive to their job. I'm sure many security admins and engineers would love to get high level training. So from the cost perspective I have two options immediate... 1) Hire an outside firm to do the work or 2) Spend $4k to send one of my engineers to InfoSec Institute and have them learn the ropes. In fact. here is a game plan as a business manager not wanting to fork out 100,000.00+ on a pentest (your wording seems to imply all pentests are uber expensive... it all depends on the scope)... The game plan;

Take John the sec engineer. Send him to InfoSec Institute for 10 day hacker/advanced hacker training for $4k (hello Jack, Mihn, etc., if you stumble on this) . After he is done with that training, contact Dave Aitel at Immunity for some more training (about another $4k) (sup dave if you stumble upon this) (http://www.immunitysec.com/services-cnop.shtml). Fork out more money for him to go to Blackhat (why not...) Richard Bejtlich's TCP weapons school $2,400.00 As a manager I've now invested $10,000.00 on what I would hope is a clued in intro pentester. Even if I exaggerate the cost and spend $30,000.00 on training, do you think any of my engineers would be upset with me for sending them to school? My benefit, now I have a strong team. I won't need to spend ANYTHING on assessments OR pentesting. Things all boil down to management, perception, understanding and the cost of doing business. $20k for a business is peanuts when they're aware that the costs of NOT doing could force them into bankruptcy, potentially lose money from loss of confidence, etc.

What90 wrote:I think we could spend hours debating whether exploits should be use, over beers preferably, but this goes back to my original response to jonas’ question. I'd rather hire someone who can argue his corner and put across why they should go the distance to the business and can adapt to the situation in hand, rather than someone just having shiny pile of tools.

I think you've confused my use of "performing a pentest" with "using tools" Here are two links to show historically what I think of the usage and reliance on tools

You bring up some great points but I’ll respond with the one reason why full exploitation tests doesn’t happen, in my opinion, in most companies: money.

Despite massive media coverage about online threats, breaches and real world monetary loses, for the majority, security isn’t going to be high on the list of priorities. Human nature or just poor awareness leaves companies with a sense it won’t happen to them. It’s easy to argue that it is a failure of communications or lack of a decent security awareness program to the big wigs or to anyone that will listen.

It’s obvious that IT security is important these days, so I classify it in a same context as why do so many people not buy insurance unless force to. They don’t want to spend the money and justify it in a way it suits them.

Reports from Gartner, Forrester, et all state that businesses will spend 15-18% of their IT operating budget on security this year. Breaking that down, a small company with a IT budget of 50K will use at least half of that as capital spend, that translates around 3,750 for security.

Spin that up to 1mil for a medium company and after the same maths that leaves a respectable 75K. Pull out AV, web filtering and security appliances licences, fees, etc and you’re left with 50K. If other defensive projects are deployed : full disk encryption, HIDS, WAF, NAC or secure remote access, you’d chew up that money without breaking a sweat. Or you could take the approach to hire consultants in to test the a subset of the systems for 5 days.

Let’s say they pick the full exploitation pentest and a detail report is generated while providing actually risk to the business. Clear, reasonable recommendations are handed over on how to fix or mitigate the risk and the consultants move on to the next client.

The company is left with choices on how to handle the recommendations, all of which are cost based. These costs are in time, manpower, staff education, resources and other projects put on hold while the work is completed. Without a senior stake holder to make sure this work get followed through and completed, the other projects will slowly suck time and resources from the remediation work.

My argument is that the business should pick up the bill for security testing. Security is a company concern not just IT, that would then leave IT with money and resources to work on mitigating the findings from a test. That doesn’t happen without the business owner understanding the threat and risk posed to their business. Catch 22 again.

How I’d like to be put on your John the Security engineer’s training path! From my experience, medium size companies, with a team of 10 IT staff, are unlikely to have a full security engineer role. With standard staff training is around 3-5K a year per person, it would take determination to get to that level in under three years. Even if that person was given sec training, the focus would be on the current security devices the company have first. Yes, a pentest may help the training budget increase, but not at the pace to have a skilled employee in under 18 months or before the results of a year pentest. Increased head count to create a sec role is unlikely to happen either.

I don’t believe pentests are super expensive. They are like any other service, so the costs are variable dependant on scope and requirements. If we were to pay for a full exploitation test, I would want the best I could afford, just like any other service. I’d be damn upset if your chimp from the woodshed turned up with his shiny tool kit though. Hidden costs, such as, assigning the resources to a work with the pentesters, to fix what was found and any system outage times are critical to factor in, but pretty darn hard to work out ahead of time.

Business folk have to work out where and how to spend their cash. We as IT sec folk would like them to spend it with us to protect against certain threats. They have to balance that against staff and building insurance, growth opportunity, staff cost, PR issues and a million other things. Anyone of those million things could result in the business going bust. There are many more public cases of business going bust due to more mundane reasons: greed, short sightedness and all-round stupidity. Lehman Brothers, most of the British and American banks just to name a few of these in the last year alone. Is the risk of being compromised any greater or less than these, that’s up to sec professionals provide advice, but, on the business to make the final call.

That’s why I’ve never got a full exploitation test off the ground, I’ve talked to businesses, shown them the news clippings of similar companies that have been attacked and the losses they accrued from those attacks and still only want audited and asset. I know I did my due duty and diligence to inform them. You can with the hand you’ve been dealt with :-)

I definitely hear you. I've been in quite a few situations where I haven't been able to provide full blown penetration tests and I try my best to inform them about the risks of underestimating and overestimating a threat.

When I *do* perform those types of security assessments, I try to follow the IAM/IEM framework and work with the client to understand their overall goal: "Do you want to find out what's potentially vulnerable or do you want to secure your infrastructure".

Many companies will often hire/procure services to determine what's solely vulnerable and are often left without a concise understanding of why they'd need to lock down specifics or minimize access to (need-to-know, etc.). I've found the IAM/IEM approach to be functional in allowing the client to work with me/us(company) in determining a "security strategy" where an assessment and hardening takes place.

There have been times also that I've provided nothing more than a typical "look at the front door" report. I try to embed language to express that these types of reports are not concise and or accurate and the only way they should be taken serious is upon validation/confirmation of the exploitability (if that's even a word) either by their own staff or a third party.The last thing I've ever want to hear is "no one told us..." or "it wasn't emphasized..."

As for costs, that's for the company to assess. A business case analysis when it comes to penetration testing is a difficult one to justify and most companies when made aware of the risks go outside of an IT budget. I always try to emphasize some of the following which I've had bookmarked, printed and saved:

• A penetration test helps organizations to understand their current security posture by identifying gaps in security. This enables organizations to develop an action plan to minimize the threat of attack or misuse.

Can save them many-a-headache-walletache in the future. It also allows them to place controls effectively. Not in places they don't need them. This can actually reduce costs in the long run.

• A well-documented penetration test result, helps managers in creating a strong business case to justify a needed increase in the security budget or make the security message heard at the executive level.

Instead of managers fighting with managers, senior level execs need to know. They're the ones that make decisions financially. Security is a cost of doing business and how you pitch it determines what you get out of it. When you offer the benefits as opposed to the risks, most CEO's understand this. They understand that in the long run they can save money. Not just throw it out the door. There is no way to calculate ROI on a firewall or IPS or any other security technology. Anyone pitching any kind of risk metrics doing so (qualitative/quantitative) is using nothing more than voodoo FUD metrics.

• Security is not a single point solution, but a process that requires due diligence. Security measures need to be examined on a regular basis to discover new threats. A penetration test and an unbiased security analysis enable organizations to focus internal security resources where they are needed most. In addition, the independent security audits are rapidly becoming a requirement for obtaining cyber-security insurance.

Cost beneficial. The cost of a compromise WITHOUT insurance is insane. If you'd like more info on "cyberinsurance" I have plenty of friends in the insurance field (I live in the insurance capital of America ) who would light up one's eyes with the benefits.

• Meeting regulatory and legislative requirements are a must for conducting businesses today. Penetration testing tools help organizations meet these regulatory compliances.

And finally the trump card... It's not that some business *want* to have pentesting done. PCI has a clause that sort of mandates it. The cost of being "non-compliant" is far greater than the cost of a pentest. Most companies are in the business of making money - haven't met a businessman whose goal wasn't to make money. Pentesting need be pitched to businessmen on their own terms. How it SAVES them money in the long run. How it allows customers to remain loyal and trusting. After all, would you want to do business with some business listed on Datalossdb.org or some business listed as sponsoring and or practicing security best practices?

I'm with sil and ketchup; I think exploitation is a crucial aspect of penetration testing. As mentioned earlier, it seems like there is some confusion when it comes to terminology. Identifying and reporting vulnerabilities is much better classified as a vulnerability assessment. It doesn't make sense to label something a penetration test if there are no attempts to penetrate. While there are other ways to compromise a system (i.e. password guessing), exploitation is one of (if not the most) effective methods.

I think some of you are over-hyping exploitation and "automated" tools. While it's always possible that someone may have a severely out-of-date domain controller or a password of "Password1", I've found that it's very rare to own a network with just a few clicks.

Here's a personal example of how exploitation provided much more meaningful results for a test of mine: I found a Linux server that was running exclusively as a web server. It only had the default Apache page accessible, but a reverse DNS lookup returned a meaningful name, so it was clearly being used for something. I was curious, so I busted out DirBuster. A few seconds later I was playing around with some obscure open-source helpdesk/ticketing system. I researched it a bit, and lo-and-behold, there was a publicly available exploit for it.

It provided a shell on the system. The system was locked down for the most part, but I was able to get a MySQL username and password from settings.php. From there, I queried the helpdesk database's users table. The administrator's password hash was cracked in a few seconds, and it got me into a few other systems.

Unfortunately, this all transpired shortly before the end of the engagement, and I wasn't able to completely compromise their network. However, you can see how this provided much more meaningful results than saying, "Your helpdesk web app is vulnerable." No one would have cared about that at all.