Anurag also posed a good question about the Geeks.com incident related to PCI, “Should ScanAlert be revoked of their PCI Scanning abilities?” A fair question and probably one we won’t know the answer to for some time – which in and of itself is an answer. We also don’t know if Geeks.com was “PCI Certified” at the time of the incident, who their auditor was, or anything like that. What we do know is they automatically become a Tier-1 merchant, which carries a certain cost impact with it.

Once Geeks.com gets done with the incident response fire drill (expensive), PCI compliance is going to cost a lot more. Before it was just a quick quarterly scan and a questionnaire. Now their going to have to do a lot more and get it all signed off by a QSA. Incidents like these are going to bite more and more small to midsized merchants hard in the pocket book - especially since PCI compliance really doesn't make a website harder to hack. I don’t think anyone has really done an analysis on the PCI costs after the fact have they?

There is a wider lesson for online merchants here, that "Hacker Safe" banners and even PCI compiance are not the same as security. I referenced your entry on the Higher Education PCI blog: http://treasuryinstitute.org/blog/index.php?itemid=85

This is really a deep problem, and the media certainly does not help. Those of us with a history in security (and some common sense) full well know that security will be broken someday. It is never a sure, 100% safe thing.

Unfortunately, there is a huge disconnect between a great many people on the expectations of security. Media will have a never-ending line of insecurity stories because we'll never be ultimately secure. They need to get over that problem rather than trumpet every security incident around like some "gotcha! badge. People also carry over this idea. One breach means game over? A file got past my AV app. Do I declare it utter useless? Maybe...depending on our definition of 'security' I guess.

Should ScanSafe be banned? That depends...are we also holding up security companies to be perfect? It is a common statement in our world that there is no silver bullet...so when this "law" proves true to a security company/application, do we wring them up for proving us right? It is a common double standard that we say "no silver bullet" but then chastise when one mistake is made.

I'm all for things like PCI that provide less negligence and ignorance. They don't necessarily provide ultimate security.

No, I don't condone the whole HackerSafe theater/fiasco thing. It's marketing and nothing more, and marketing is caught in the same problem that media is caught up in...

Jeremiah, I'm not putting you on the spot or anything, but let's say a customer of yours gets "hacked" to some degree or other after you've scanned them (a week or a year after). Does that mean your company failed? :) How would 100 random people just in our field answer that question? Kinda illustrates a deep problem we have... :) (Something law enforcement has been dealing with for decades and decades...)

Oh cmon, you are putting me on the spot. :) But you know what, it’s a fair question and one I’m happy to answer. WhiteHat sells a top-tier service that’s designed to reduce (not eliminate) the likelihood of a website getting hacked. Other services have other value propositions. There are two potential scenarios when/if a customer’s website gets hacked. 1) Exploited a vulnerability we already reported, but customer didn’t resolve. 2) Exploited a vulnerability we didn’t find, but should have.

In the case of #1, which happens on occasion, the reality is we can’t force a customer to fix anything. Our job is to provide vulnerability intelligence, which we did, and it’s the customer’s responsibility to act … or not.

In case #2, which has never happened to our knowledge, we’d expect to lose that customer, would likely give back the money they paid, and suffer a substantive amount of reputation damage (more lost business).

As you might expect we do everything we can to make sure our service remains of superior quality because we don’t ever want that phone call. As you say no one can be perfect all the time, so odds are we’ll have to deal with it eventually. I’d prefer it to be rare, but for other vendors that you read about it appears painfully common. And it’s THOSE vendors who credibility needs to be questioned.

HACKER SAFE should not get banned, punished, or fined because they are an ASV. ASV's shouldn't assume any responsibility or liability. They already pay in $5k/year just to get tested.

ASV's test against PCI-DSS not PA-DSS, mind you. Whether or not XSS or SQLi (or any other web application vulnerability) should be in PCI-DSS is up for debate still, isn't it?

The problem I see with HACKER SAFE is the name. Similar to TrustWave, HACKER SAFE has a modified version of the Nessus 2 scanner (because using Nessus 3 would violate Tenable's rights). However, they are in violation of the GPL by modifying Nessus 2 and not giving back improvements, but the GPL lawyers can't "get right on that" because they have no proof of this.

Using the name "HACKER SAFE" when all they do is accept payment and run Nessus 2 on an IP is a different issue. This is almost false advertising or bad/immoral business practices, but IANAL. To compare WhiteHatSec in concept would be to say that Jeremiah suddenly freaks out, moves to Belize, joins up with the RBN, and starts putting up phishing sites.

The problem is not with HACKER SAFE. Big firms like TrustWave (who have smart, great guys like Martin McKeay) also have this exact same problem in the ASV space. One could even go on to say that TrustWave (who is also a QSAC and QIRC) is poor at code review or incident response.

Side note here for those of you who do not know the differences between an ASV, QSAC, and QIRC.

1) ASV - Approved Scanning Vendor - company certified by PCI to perform quarterly scans. In order to qualify, someone at the aspiring company (e.g. janitor, secretary, owner's 15 yro son) must run Qualys against a demo site. If they choose not to run Qualys, the tool(s) that they use must still find all the bugs that Qualys would have found. Anyone at the company can run the scan (e.g. the CEO's cousin, who happens to have down-syndrome), and no interpretation is necessary; only a report which is the output of the scanning tool2) QSAC - Qualified Security Assessor Company - a company that retains QSA's. QSA's are people that are certified to perform the PCI-DSS audit along with any necessary code review. I get a little fuzzy on this part, but I think they are required to take a week class, followed by a test (or possibly two tests, I'm not sure). Either way, these are certified individuals. They must work for a QSAC who also has requirements that they have to meet, and lose their certifications if they leave the QSAC until they join back up or with another QSAC. They also deliver reports, but they are custom reports (not output from a tool)3) QIRC - Qualified Incident Response Company - I'm fuzzy on this, too, but these are the people who come in to clean up a breach. I think this is a company that retains some sort of team, but I'm not sure if that team has to be pre-qualified or made up of QSA's or what

Note that the same big company can be ASV, QSAC, and QIRC - but when filling the function of ASV, the same company cannot send a QSA even if they are a QSAC (it must be from a different company). Reversely, a QSA working for a QSAC who performs an audit for a company cannot have their ASV wing perform quarterly scans.

So, if there is a problem - it's not with the ASV, QSAC, or QIRC -- it's a problem with the certification process or the PCI-DSS audit framework itself.

About Me

Jeremiah Grossman's career spans nearly 20 years and has lived a literal lifetime in computer security to become one of the industry's biggest names. He has received a number of industry awards, been publicly thanked by Microsoft, Mozilla, Google, Facebook, and many others for his security research. Jeremiah has written hundreds of articles and white papers. As an industry veteran, he has been featured in hundreds of media outlets around the world. Jeremiah has been a guest speaker on six continents at hundreds of events including many top universities. All of this was after Jeremiah served as an information security officer at Yahoo!