A page to show up #1 on Google when searching for "Jeremiah" (Currently #4). Only the prophet and TV show left! I have the edge, TV show is cancelled and the prophet isn't generating any new content.

The prophet, TV show, and that pesky Owyang guy going down!A page to show up #1 on Google when searching for "Jeremiah Grossman", and it FINALLY has!

Tuesday, November 14, 2006

Web Application Security Professionals Survey (Nov)

Update: Thanks again to all those whom responded to the survey. 48 respondents, doubling my first attempt of 21, and a good representative split between security vendors and enterprise professionals. The results are below. I may make this survey a monthly activity provided the responses is good and people find the data helpful.

My Observations- I was a bit amazed by the significant portion of web application VA’s combining the black box and white box testing methods. I knew black box would be the most common approach, but I would have figured the pure source code reviews and the combo approach would have been statistically swapped. I may need to dig in more here and ask what the benefits people are seeing as a result.

- 73% of those performing web application vulnerability assessments are not using or rarely using commercial scanner products. It’s hard to say if this is good/bad/increasing/decreasing or otherwise. Certainly people want tools. People love their open source tools as a vast majority are using them. Be mindful that open source webappsec tools are mostly productivity tools, not scanners like we asked about in #3, so they’re not opting for one over the other. There is a lot of room to dig in here with future question as to why people use or don’t use certain types of products.

- People see XSS as slightly more dangerous and widespread over SQL Injection. But what’s clear is they find both issues important and weigh them heavily over the rest. Also surprising was prior to the survey, I would have though few assessors would be checking for CSRF issues. The fact is most of them are testing for CSRF at least some portion of the time. And imagine this issue is not on any vulnerability list. This will change soon.

- 3/4 of assessors agree more than 50% of websites have serious vulnerabilities. They also believe it would take them less than a day to find a serious issue in most of them. And 65% of assessors alredy knew of a previously undisclosed incident (web hack) that led to fraud, identify theft, extortion, theft of intellectual property, etc. That’s a sobering trifecta for the state of web application security.

- When asked what activity most improve security - modern software development frameworks, secure software and/or awareness training, and a stronger security presence in the SDLC were evenly split across the range. With the exception of industry regulation which assessors felt was not a driver of security. Interesting.

DescriptionSeveral weeks ago I sent out an informal email survey to several dozen people who work in web application security professional services, an informal email survey consisting of a handful of multiple choice questions designed to help us understand more about the industry. The results were interesting enough to try again, this time with a few more questions and distributed to a wider audience.

If you perform web application vulnerability assessments, whether personally or professionally, then this survey is for you. I know most of us dislike taking surveys, but the more people who respond the more informative and representative the data will be.

The survey should take about 15 minutes to complete. All responses most be sent in by Nov 14.

Guidelines

Should be filled out by those who perform web application vulnerability assessments

Copy-paste the questions below into an email and send your answers to me (jeremiah __at__ whitehatsec.com)

9) How long would it take you find a single serious web application vulnerability in MOST public websites?a) Few minutesb) Hour or twoc) Day and a nightd) A few dayse) Don't know, never tried

A: 23% B: 35% C: 19% D: 2% E: 21%

10) How long after a web application vulnerability assessment are most of the severe issues resolved?a) Within hoursb) The next couple daysc) During the next scheduled software updated) Months from discoverye) Just before the next annual assessment

A: 0% B: 40% C: 30% D: 26% E: 4%

11) What organizational activity MOST improved the security of their websites?a) Using modern software development frameworks (.NET, J2EE, Ruby on Rails, etc)b) Secure software and/or awareness trainingc) A stronger security presence in the SDLCd) Compliance to industry regulationse) Other (please specify)

A: 21% B: 28% C: 21% D: 2% E: 28%

12) Are you privy to any undisclosed (not made public) malicious attacks made against a web application?(fraud, identify theft, extortion, theft of intellectual property, etc.)a) Nob) Onec) A fewd) Too many to count

10 comments:

Interesting stuff! I feel that the "what makes sites more secure" question is a little problematic. Regulations never make sites more secure. They might motivate management to cause their IT folks to use some method to makes their sites more secure, though. It seems like you're conflating two questions:* What organizational activity MOST improved the security of their websites?* What prompts these activities?a) Incidentb) Regulationc) Risk analysis

Or something like that.

Putting on my copy editor hat, you hat the 40% label selected when you took the screen shot for #10, and your legends are terrible. Put the answers in the legend (or at least to the right of the legend), and have the legend be in the same order as the pie slices. I think the best way to do it is to get rid of the a/b/c entirely and just put the answers in the legend, sorted from most to least common.

I criticize because your blog is awesome. If it wasn't I wouldn't care.

Comment #1I think your right on. I was trying to get two answers with one question and it probably didn't come out right. I'll see if I can word this better for the december survery, maybe like web application assessment drivers, and we'll see what the results are.

Comment #2Again, right on. Viewing in the format I supplied is a bit awkard and could use improvement. I'll be making use of your feedback on the next go round. This is a learning process for me as well since this is my first time doing survey reports. :)

It's very interesting! Good work! "Compliance to industry regulations" only got 2% in improving security of web site. I think, from a practical standpoint, it implies the compliance report generated by automatic assessment tool probably doesn't necessary. And the most important thing might be to solve those critical and obvious problems like SQL Injection, XSS, PHP File Inclusion.

Interesting to see a massive chunk of people using their own risk rating systems. What is it that people don't like about DREAD/TRIKE? Where are the weaknesses in using those systems and how can we build upon then?

Putting on the OWASP hat, would people feel happier if we came up with a better rating system which could be adopted by the community and easier for clients to understand?

Walter, thank you, glad you enjoyed it. I think you touched upon the same issue that Jeremiah and Jeremiah (read above) were discussing. I think was trying to get two answers with one question and it became a little muddled. Next time I'll phrase it in such where I can find out what "solution" people feel are working and what "caused" people to implement web application security in the first place. We'll see how that works.

From what I gathered people find the severity of web app sec vulns dependent on the organization they are working with, the website, and the specific vulnerability found. No one has yet made a generic and meaningful method to classify these. Probably because its really hard to do so. I think PCI might be a model worth considering, but I'd also modify their definitions to suit web app sec better. Its the route WhiteHat is traveling down and seems to be doing ok.