Monday, October 5, 2009

Note: The following does not represent the opinion of Mark McKinnon. He merely had the good grace to allow me a forum in which to post it after it was respectfully declined (for obvious reasons) by the SANS Institute'sForensic Blog. I wrote it chiefly because I hadn't seen anything recently, or as I recall, ever, that so much as acknowledged any downside to certification. I respect the pro-certification viewpoint, but I do disagree with it. And so, without further ado...

Folks, this is an opinion piece, and it's going to be a controversial one. Some of you started composing a scathing rebuttal to it as soon as you read the title. Normally I restrict myself to what I hope are useful technical tidbits, but like most of you out there, I'm a forensic practitioner, and I have little patience for time sinks which provide no benefit (no I'm not including the training in that category, save your flames for the end). I've always begrudged the time commitment (over and above what's required to actually take the training and learn the included material) required to attain certifications, despite which I'm in possession of five, soon to be six, not counting my master's degree, so I like to think I speak from some degree of experience.

I do understand the arguments used by the proponents of certification. In essence, they allow people who have no understanding of a technical discipline to discriminate between other people who do and don't have that understanding. At least that's what they're supposed to do. Let me list two of the most egregious counterexamples that I have found in my own personal experience (with no disrespect intended to either Microsoft or the International Information Systems Security Certification Consortium). I have met, in my career, an extraordinarily large number of clueless CISSPs and MCSEs. These are people who were apparently able to pass the test, but who were unable to, respectively, secure or administer their way out of wet paper bags. To state it in more general/inflammatory terms, one problem with certifications is the number of idiots who are in possession of them. On the flip side of this, I personally oversaw the hiring of a system administrator back in 1996 who had nothing but a High School Diploma and a clue. I still work with him on occasion, and his hiring was one of the smartest decisions I ever made.

One logical response to this issue is simply to make certifications more difficult to get, but there we run into a second fundamental problem. When a certification raises its difficulty in order to exclude a certain percentage of unqualified people, they also exclude a certain percentage of qualified people. As the difficulty raises more and more, the incremental number of unqualified people being excluded gets smaller, and the incremental number of qualified people being excluded becomes larger. The amount of work required in order to to pass increases substantially as well. Qualified people get excluded for several reasons. For one, the more difficult a certification, the more training is typically required before attempting the exam. One forensic certification I heard about last week, the one which finally prompted me to write this posting, requires six months of training and six exams. That's a tremendous amount of time committed to obtaining a fancy certificate and some alphabet soup to put on your resume. Don't get me wrong, I'm not saying that training is useless. But what do you do if you're already in possession of 75% of the knowledge this training is intended to pass on? It's in the financial interest of the certification providers to make it more difficult to pass the certification if you haven't attended their custom-designed training program. Review guides may be available, but typically cover more material than the certification vendor's training, without the subtle emphasis often provided by that training. The practical upshot of this is that an individual who may know 75% of the material on the exam off the top of his head, substantially better than a graduate of the certification course will be (probably) after six months or so, may still have to complete a long and expensive training course just to get to a point where he can reliably pass the certification exam. For many of us, it's simply not worth it. We resign ourselves to being filtered out because we don't have the requisite alphabet soup, even though we're otherwise qualified.

You'd think that at some point, an exam would filter out all the idiots, but that's much harder than you'd think. That's why IQ tests have fallen out of vogue, and why an actual interview is still the best way to select a new employee. This brings me to the third reason certifications, or more specifically certification exams are bad. Many standardized tests consist of simple regurgitation of facts. They don't require that the subject really be able to think, just memorize. Personally, I believe that any idiot can pass such a test if they put sufficient time into preparation. It's possible to design questions to test problem solving ability, but it's difficult. One tactic that's often resorted to, and this is a personal hot button of mine, is to provide the subject limited information, allow him to assume the rest, and make him pick the 'most reasonable' or 'best' solution from the list. The problem with this occurs when the test subject is smarter or knows more than the individual who designed the question. I personally have run into this several times on various certification exams (I got a couple of the questions changed), and I find it intensely frustrating.

Finally, certifications are bad because they provide lazy people with a tool that can be easily misused. Rather than read 100 resumes to determine the 15 most qualified for a particular position (which he may lack the expertise to do anyway), an HR person can simply filter out all those lacking a specific certification. If this still results in a number of resumes that is too large, he can filter on another certification. This sort of data reduction can easily remove more qualified people than unqualified. In my opinion, it's better to pass all 100 resumes down to the hiring manager.

Certifications are bad for hiring managers, because they reduce their pool of qualified candidates, and they're bad for the candidates, because they enable those candidates' resumes to be filtered out before the manager sees them. In the end, they provide the most benefit to the vendors who provide them and their associated training, and to HR organizations, who are able to get by with fewer and less expert people.

Once a certification is accepted as required in a certain area, this fact can be used by people who lack training in that area to obtain it. The downside of this is that people who are already qualified sometimes must forgo more advanced training to take training just to get the certification. I'm not suggesting they don't learn anything in this training, but typically it will be much less than they could have learned had they been able to attend training of their choice.

So, you might ask, what's the alternative? Isn't there some other low-overhead way to reliably tell if a candidate knows anything about a given specialty without actually reading his resume or interviewing him? Well, I have a suggestion. Maybe somebody out there can make it work. It's based on word of mouth, and the PGP web of trust. Basically, there are a number of people who's word I trust if they say somebody has a clue. If everybody had one or more PGP keys with a comment that said "I am an expert in X", then people could sign that key, and the subject could publish the result. If Rob Lee, Ed Skoudis, & Josh Wright all say I'm an Uber Geek (and I'd like to think they might), I tend to think most people would buy into it. Maybe we could call this the web of cluefulness.

As always, please feel free to leave commentary if you liked this article or want to call me on the carpet for some inaccuracy.

211 comments:

No flame here. I 100% agree with your points. I have seen it happen in my Department. They added the requirement of needing at least 60 college credits in order to get hired for the job, shutting out qualified HS graduates. People think that just because someone has spent time in college that they are brighter than most. What a bunch of crap! Most of them couldn't police their way out of a paper bag given a chainsaw.

Although I do agree that certifications should play some importance, there must be a way to keep them from making people "Paper Tigers" (I stole that from Chris Nickerson, great analogy though). Experience, as well as certs, must be combined for an overall picture.

I also agree with a lot of what John states. If you have followed some of my tweets you know this as I have had a few rants about the level of some individuals that hold certifications. I have worked with a few individuals that do not have college degrees and run circles around those with degrees, those non-degree individuals I would have on my team any day of the week.

A few years ago as a member of a Database Administration Team we were told we had to hire another DBA. The HR people gave us a while list of certified people (none of the current DBA's were certified at the time and still are not) and when we started asking questions on how they would solve certain problems the answer we got everytime was using a tool. We would thne tell them the only tool we have is SQL*Plus (guess what DB that was) and how would they solve the problem then, all we got were blank stares. Now lets equate this with a lot of current forensic examiners and the "button jockeys" mentality they have, see any similarities?

I do agree that certs show a basic understanding, with emphasis on basic. But there needs to more things taken into consideration like Joe states. Experience as well as understanding of underlying technologies and the ability to learn help to form the complete individual and no cert will ever be able to show this.

I'm going to come at this from a different angle altogether: not as a forensic practitioner but as a marketer/public relations person involved in the space.

Your last paragraph strikes an important vein that crosses a much broader area than just the forensics world. As a general rule, people don't trust what organizations tell them to trust anymore. They trust the word of their friends, people they look up to. Which is why I'm trying to work with clients to be involved in the community, develop that trust.

Of course, given not only the 9th Circuit decision but also the general questioning of all forensic science that has been happening (mainly as concerns capital cases), it will be interesting to see how certifications are treated by the legal system: good enough or not so much?

Christa,One of the main offenders of this, is management. If they don't understand what someone in either Computer Forensics or InfoSec exactly do, they rely on the "alphabet soup" that are certifications to measure someones "hire-ability", regardless of what they truly knowledge.

Unfortunately, that hurts rather than helps the true professionals in their respective fields. It starts to become more important (for some people) to get the certs, rather than get the knowledge & experience they need. The attitude of "as long as I have my paper" starts to loom large.

I completely agree with you. CISSP is actually a big joke.. I know few so called CISSP certified and beleive me they know nothing about the stuff..and one of the major thing they lack is passion and keen interest in security field.

Part of the problem is that what if it does not make it to court? I will pick on LEO's here but it could be an civilian examiner as well so hopefully no one is offended.

What if a LEO examiner states in a report that in a limewire case the t_XXXXXX file was viewed? The LEO examiner has a certain certification, is he to be believed because he has this cert? What if an independent analysis is done and it is shown that the t_XXXXXX file was never previewed but the defense expert does not have a cert? Who's opinion is better? The person with or without the cert? Now if the Defense expert also has a cert whose opinion should win out then? The person with more training on their CV? Logically the defense should but when have the courts/jurys been logical? Now this takes it past certs and training and who is more believeable the LEO or the defense expert, which is another debate.

I will have to say that one of the things I do like about the certs as they are trying to all have a similar code of ethics when it comes to this field which is a good thing.

Never let it be said I was unwilling to correct myself when I misstated something. The cert I was talking about was the MFCE, as discussed by Lee Reiber on episode 21 of Forensic 4-Cast. I listened to it in something of a fog on my way to work one morning last week, and I appear to have gotten a few details incorrect. There aren't six months of classes, but six months is allowed for the process. There are six practical projects and one final exam. I still think this comprises an awful lot of work, but it's not quite as bad as I made it out to be.John

I agree with almost 100% of what John has said with the exception of "that is why an actual interview is still the best way to select a new employee" which I think couldn't be further from the truth.I introduced practical tests about 10 years ago in selection procedures for Fraud Squad and Hi Tech Crime Unit officers and they have proved to be far and above the best method of selection. I have seen someone sell themselves very well in interview only to fail miserably when it came to doing it for real. The tests have to be based on practical scenarios from real life and set in as realistic circumstances as real life. So for example in a Fraud selection you might be given a practical scenario but you will be given a copy of the relevant law and any policies and you have to assimilate the information and make judgements like real life. You get an hour or whatever is needed prior to the interview to consider the scenarios and then you come in and answer questions and give your views.

I have multiple college degrees and computer certifications, and I agree with some points of this article. Certifications are indicators, not validators. People who rely solely upon them for hiring decisions are not properly doing their jobs.

Some certifications are worth it while others are not. I find that the more open and inclusive the certifications process is, the more valuable it tends to be.

For example, CompTIA certs are limited in scope, but anyone can take them and provide feedback to CompTIA, which allows CompTIA to improve them. Certs that are exclusive to LEOs may be nice, but by their very exclusionary nature they have a smaller population from which to draw feedback and make improvements. For the record, I do not work for CompTIA and do not like all of their certs either, but they do allow just about anyone who wants to try to pursue their certifications. Plus, CompTIA is very upfront with letting people know that the cert is just designed to show a level of general knowledge rather than expertise.

Requiring training classes from a specific vendor prior to beginning the certification process is hogwash! People should be allowed to prepare in whatever way works best for them, be it self-study or classroom environments. Not everyone learns and retains things the same way. Plus, I have taken classes from vendor A where I learned more about specific aspects of vendor B's product than I learned originally in vendor B's classes.

I will be even harsher than you - IMHO people do certificates for recruiters or to help their independent business, not for themselves to get smarter; and as for SANS and ISC2 - c'mon, these are cash cows and as long as they need to think of a revenue, it will stay like this; plus, for BSc/MSc you need to really work hard while CISSP can be done in a week; yup, it took me a week to prepare for CISSP and I did it purely for "recruiting" purposes - I was thinking of changing the job at that time and it DID help; a few bright guys I know are CISSPs only either because of "recruiting" purposes or because their companies sent them for bootcamps; after that, they let their CISSP expire (and I consider doing it as well - why paying them money? they keep spamming you with their upselling more than once a week); also, note that for quite some time SANS exams were without a proctor - I saw a guy (no, it was not me) passing such exam with the whole group of his coworkers helping him. To close this rant with a conclusion - cert are like insurance anyone can purchase; it's not a magic pill that will make dumb people smarter; I am waiting for a first person to run a stunt and send non-IT/non-security guy to pass CISSP and one SANS exam in 2 weeks; IMHO very "doable"

No flames here either. I agree with most of what John has stated. I have always struggled with understanding the time spent on certs vs. getting things done.

I think solid work experience AND formal education (B.S./M.S. Comp Sci/Engineering..) is the strongest path to creating a highly competent professional. I have worked with people with experience only, and they seem to lack depth in fundamentals and breadth of knowledge, however very strong with the real world stuff. In contrast, I have also met computer science graduates that needed to be taught how to use FTP on the job. (no lie, true story...). IMHO, without both something is lacking in the long run.

The best candidates are truly ones that balance experience and education over time. I am a reluctant CISSP. I will likely maintain it as it seems to provide monetary value for the filtering process (point already sufficiently beaten to death). I sadly have settled on this to be a necessary evil that has provided the best bang for the buck (my employers buck BTW).

I also feel that when SANS removed the practical requirement from its certifications a number of years ago, that it only hurt the certification process as a whole. They have slightly mitigated the damage with the Gold status, yet it (IMHO) appears as a transparent method to milk cash out of the market. Most certifications are easily passed by a well prepared candidate (a point also previously beaten...). One note on this...the SANS exams...I mean open book, come on! Any person that takes the time to index the material can pass the exam without even knowing any of the concepts.

Another note on workers with 7-10 certifications, if I see that on your resume, then why do you think I should hire you? Based on your resume, you are likely to spend more of your energy obtaining/maintaining the next certificate and not focusing on learning your job and improving the work environment.

Consequently, employer beware. Those that rely on certifications and those that dismiss the value of foundational education are missing the boat.

I think certifications can cause complacency among forensic examiners. As with any professional industry those that make it up must "police" themselves and decide what will and what will not be accepted. Like Mark stated, it is good to see a standard among certifications conforming to a common code of ethics; however, certifications should be a combination of textbook and a hands-on practical approach. There are some very intelligent & brilliant examiners whom do not have a clue in practical application (think I'm more practical than techie at times). John’s article is one that definitely needed to be written and debated. Just my two cents!

I like the idea of a web of cluefulness and I think many good hiring managers have something akin to this, though it may not be perfect. Most jobs I've applied for have required me to provide professional references. Sure, people can and do stack the references, but part of my point is that a good hiring manager isn't going to only consider a single factor in the hiring process.

The person I currently work for considered my resume, my certifications, what my references had to say, what was said and discussed during the interview process and perhaps most importantly, the hands-on portion of the interview that required me to spend time actually demonstrating that I knew the things I claimed to know.

It's true that earning certifications is a burden and that there are many clueless idiots holding them. I've worked with one or two myself and in fact, hold one certification that I think is a joke because it was ridiculously easy to get and another that I think is a joke because maintaining it is more a matter of paying for it than demonstrating that I know anything.

All that said, I used to be in the camp that thought certs were a waste of time and effort. And then I worked with a guy who was pursuing his CCIE. His dedication and knowledge was amazing and he encouraged me to pursue certification for my own work, at the time Oracle database administration.

I went out and bought the books for the OCP exam and started studying. At that time, I'd been working as an Oracle DBA for three years and yes, multiple SQLPlus windows were a fixture on my desktop. I thought I knew Oracle pretty well as I'd exhausted the knowledge of those around me who had been working with Oracle for many more years than I had.

But during the months that I studied for the exam, I learned things about Oracle that I hadn't known. I learned new commands that were built in that could do things I had spent hours writing scripts to accomplish. Studying for the exam made me more efficient and more effective at my job.

I never did take the OCP exam, but just the process of studying for it made me a better DBA and opened my eyes to a benefit of certification that I'd never really considered. The important part is not the piece of paper, but the knowledge one gains along the way. Or as someone more eloquently put it long ago, the journey is its own reward.

Great reply Dave, I would agree with you some of the exam prep material is very good, I have also learned a lot from going over some of that material as well.

A couple of other things I thought of is how many times did it take an individual to pass the exam? Especially after they had received all the training they needed. Also what areas of the test did they fail miserablely at?

One last thought, at what point does the cert you have become obsolete? For example for an OCP how important are knowing how rollback segments work considering Oracle does not want you touching them anymore

Now the other thing no one has touched on is what will the states require of us. In Michigan in order to be a PI I have to have been LEO, Law Degree, Military or a Degree in Security for CF. I only have a BS and MS in Comp Sci. Since I do not fall under those I have to look at the cert. For the certs I have to have a CISSP or equivalent and some CF training/cert (which is not specified which one). Now I do not have a CISSP and was not planning on getting one so I am resigned to the fact I have to be a fingerprinted employee of a PI. There was a board that met to determine this and I wonder who on the board would meet the qualifications (I have sent numerous messages to foinf out who was on the board but no reply back).

Certs are not intended to ensure that someone is awesome at their job, but that they pass the minimal qualifications for someone in the field. Much like basic training teaches you the basics to fight in combat, but hardly makes you an Army Ranger.

For the sake of the profession, something similar to the bar or medical exams has to ensure that a basic set of knowledge exists for an entry level individual. CPAs, Doctors, Lawyers, get better with experience that is clear.

However, in order to even begin the first day, they have to prove that they at least know enough not to make a critical error on day 1.That is the point of certification.Unfortunately, licensing will be barreling down on our profession faster than you think for everyone. There are bills in congress as well as legislative actions that are taking place in many states.

We live in a society where you need to be licensed to cut hair, be a plumber (Joe the plumber was not licensed), or babysit (Michigan). Do you really believe that you will not need a license of any sort to do this job we love?Good certs are needed as a counter to that. CDFS is a part of that as well, but the states want educational/testable proof that someone doing the job has jumped through a couple of hoops so they are not snake oil salesmen.

For the profession overall, certs are needed. Personally, I respect many certifications. EnCE, CCE, and the CFCE. Last year I sent out a Common Body of Knowledge to over 80 practioners, the CBK comment process outline which skills are needed and which skills are “nice to have.” But I based my course development based on these. I do believe that we should, as a profession accept we will need to become tested to perform our work. It is not a matter of “If”, but “when”.

How many professions do not have an entry level test? I personally do not care the color of the certification, but professionals should consider certifications in their profession. Get certified to show we are a true profession.

In less than 10 years... all InfoSec professions will need to get licenses as well to do their jobs. Your call on how they should get that license. Leave it to biased industry groups such as the PI lobby or have professionals like you and I decide what the minimal qualifications are.

Help us decide what the qualifications are needed for a minimally qualified professional in digital forensics. Help mold the future of your profession.

I think in this instance there are valid reasons on both sides. Although Rob does have a good point that the certifications serve more of a foundation than anything else.

The idea of an apprenticeship is a good idea as well, although I think that would be more effective for a college degree requirement than anything else. I believe (and Lee can correct me if I'm wrong) that in the UK most of the university programs have that requirement as part of their program.

The overall issue with our industry is that it's still a young field and does not have any type of oversight like the others that have been mentioned (Law, Accounting, etc). But should we work towards a program that's similar to becoming a Doctor or an Attorney?

Until an organization is formed that can set the bar for what a computer forensic examiner "should be" (even if it's still at a baseline), the certifications that are out there are going to be a factor of the job. It's going to teach you the basics and experience will teach you the rest.

This falls almost into the same argument as LE versus non-LE. Who will be the better examiner? The LE person that has the investigative background or the non-LE person who's been in IT for x amount of years and is familiar with how technology works (not that LE people don't understand technology, but you get my point).

I think the reality is the "good" forensic examiners are going to be those that love the job that they do and want to learn everything they can about it. That want to have these sorts of discussions and not just do it for a paycheck. As with any job that's out there if you don't like/love what you do, you're not going to give it your best. You are just going to do what it takes to get the job done.

BTW, this is a good discussion and has been very insightful to me. There is always value out of multiple opinions and perspectives. Thank you to John McCash for speaking up and putting it on the table and to those willing to share their thoughts.

I would like to respond to Rob's comment regarding certs are to insure minimal qualifications. While this concept is great in theory, the reality is that certification holders (in general) have been driven to expect higher pay for these certificates (minimal qualifications) and not skills and experience. The certification organizations (ISC2/SANS, and others...not intentionally picking on them.) have created this situation by promoting their product as a career booster. If the case were as you say, then most professionals would not hesitate to get them. Yet, the riff arises when the adhoc system is not adhered to by hiring organizations. (I know..back to blaming management again. :)

I am a process oriented person. (some might say I crave it..) So to me, standardization has value and strength beyond its initial objective. But, standardization is like a chain. It's is only as strong as its weakest link.

My input is to either make every practitioner take an exam (the same exam), (i.e. license to drive) or make the certification exams harder so that when someone accomplished a passing grade that it actually means something more than the candidate was able to prepare for and pass a test. Yet the latter does not support most organizations economic drivers. That's why lawyers are paid well, the bar exams are not easy to pass.

I will say, though, that I like the CISSP exam for giving folks new to the industry 'background'. Passing the CISSP exam doesn't mean you can _do_ anything. It just means you studied up on the 10 domains. It means you (should) know a baseline of information for the industry which you're in. I think it's a nice little indoctrination, kind of like getting a bachelor's degree. Nice to have, says you made an effort, but ultimately says nothing about your abilities. And you are supposed to follow the CISSP code of ethics, which is nice in theory.

It doesn't mean anything in regard to a candidate being a good hire.

It does, however, mean I can ream you during an interview when I ask questions about any of the 10 domains when related to more practical matters :-)

I love Kevin's comments. Honestly, we are in an evolution. And it will take time. Hopefully we can all work together to make it happen. We are such a small field and I think that we will eventually need that hard "bar" exam to test everyone.

I had been up and down on commenting to this blog post, but I figure I might as well.

To start off I have my CISSP and CISA. I worked hard studying for both and did not do the "week and out" studying method. Most of my experience has been in Policy, C&A, Crypto etc.. Recently I have moved into the IH and forensics.

Neither of the certs provided much knowledge etc.. in what I do now. That being said they do help when I have to deal with people outside of our team. It does lend some credibility to what I am asking or telling someone who's computer is being investigated.

I compare this conversation to some political conversations. You have your hardliners one way or the other and then you have your uneducated people who just back their favorite public figure. It's your right to have an opinion, but research the facts. If you met one bad CISSP does that make all of them bad?

I've found that most people who bash certs do it for one of two reasons. They got burned by someone with a cert (either lost a job to someone with it or the person was an idiot) or they don't have it and are honestly jealous of someone who does.

I know some people who have been studying for their CISSP for literally years, but just not taken it. When I got mine I got comments from these people along the lines of "doesn't mean you know anything" and "it's a worthless cert anyway". Why study for it for years then?

Certs mean you have a base knowledge and can demonstrate that knowledge. Gets you in the door and an interview. You still have to show yourself in the interview and you still have to prove your competence. If the supervisor hires people because of certs, maybe you shouldn't be working there anyway.

I've known doctors that I would trust to give me a prescription for Tylenol. It's the same with any cert.

The problem with the Info Sec community is we are making it what it is. We B!tch and complain on places like twitter or comments on blogs, but how many people actually try to educate about where the value lies and what to do after you get Cert-X? I've seen tons of blog posts about Cert-X is worthless, Cert-Y is a Joke, etc.., but I've seen very few articles or blog posts along the lines of "You have Cert-X, now what should you do?" or "You have Cert-X that requires CPEs where should you be trying to get them?"

CISSP and CISA are around for a while. SANS will be here for a good bit. Unless we work on improving the community, unless WE make it better, it's just going to get worse. DoD 8570.1 says everyone has to have a level of certification to work at certain level on the network. Have any of you worked with Gov't contractors? If we start giving some of these guys certifications it would be like giving drug dealers the right to handour prescriptions. YES SOME OF THEM ARE THAT BAD!

Rob reposted his first comment here as a SANS blog entry, but he prefaced it with a statement that it was in response to my assertion that "certifications are not useful". This requires that I clarify my position, as I did with the following comment to his posting.

Rob,I suppose I understand your reluctance to use “certifications are evil” in the anchor text of your link, but I wish you hadn’t misquoted me quite so vigorously. I never said certifications weren’t useful. My point was that there are numerous problems with both their implementation and their use, and that in my opinion, these overshadow their benefits I also think these problems are endemic, and can’t be reduced to what I consider an acceptable level without altering the concept of certification completely. In essence, certificatons attempt to balance the mutually incompatible interests of candidates, hiring managers, HR departments, and certification providers. Of these, the ones with the most control of the certification process are HR departments (which tend to determine, with input from the hiring managers, which certs are considered useful to have) and cert providers. As a consequence, the whole process benefits those participants to the detriment of the candidates.John

"Certs are not intended to ensure that someone is awesome at their job, but that they pass the minimal qualifications for someone in the field." Rob Lee

I have made this very point to countless individuals who equate "certification" with "expert".Simply because you have certifications does not mean you can do the work. When someone send me a resume with a dozen letters after their name, my first reaction is "What is this person compensating for"? Usually it's lack of actual experience and depth of understanding on how to do the job.

I would also argue that we need certification because people lie. There are so many unqualified people out there charging crazy rates and getting them because the demand for "experts" is high. We need to have some way to filter the "want-a bes" from the legitimate experts.

Using word of mouth is great. I can go to one happy hour every month and in three hours I have more intel then I could ever use on who's where, who's good and who's bad - works for both companies and people. But it's biased. Word of mouth favors LEOs, it favors the good old boys networks. There are lots of execellent CF people outside those perameters.

I have been within the field for 10years now and I have yet to find a certifying body that does not have some hidden agenda. Generally it's to make a buck. Many groups have tried to set themselves up as a "unifying body" for people to go to to find legitimate experts. All have at some point succumbed to greed. If anyone ever successfully accomplishes this, we may be able to look to certification as a means of identifying competent people. Till then, it's a crap shoot. Hire at your own risk.

My God, John, thank you for this post. You echo my thoughts exactly, and I too am a multiply "certified" long-time practicioner and hiring manager. I've worked with phenomenal examiners with nary a cert to their names, and I've worked with certified people who can't examine evidence right in front of their faces.

It's time the industry realizes what certifications REALLY are at the end of the day -- a revenue generating engine for the companies that offer them.

I agree with your frustration, but disagree with your view on the value of certifications.I've worked as a software developer, yet have little formal education in programming since high school. I learn best on my own from books. My degree is in Physics.I've worked with college interns who couldn't even write a "Hello World" program in any programming language. One of those interns is now president of a company. He isn't smart, is lazy, and good at pulling the wool over people's eyes and taking credit for other people's work. He has a degree in computer science.I still respect a college degree, but know that some college graduates are worthless. So, should I say that college degrees are too easy to get, and that standards should be changed?No, I think a degree and/or certification shows that you were diligent enough to jump through the right hoops and earn proof that you are qualified. It is then my responsibility, as a potential employer, to ask the right questions to make sure that you meet my hiring standards and can fit the position that I am filling.

I agree almost completely with this post. When the computer forensics community in the UK was moving towards using the CRFP (Council for the Registration of Forensic Practitioners - now defunct!) there was some cautious enthusiasm from many quarters that dissolved as soon as the list of judges (or whatever they were dubbed) was announced.

I also give a hearty thumbs-up to your condemnation of exams, and only partially on selfish grounds - I've always been terrible at taking exams, but I consider myself to be good at what I do. As you hint at, exams only test your ability to cram and regurgitate knowledge in a specific way, they don't mirror any real world activity.

There are already plenty of forensic practitioners out there with an Alphabetti Spaghetti of post-nominals (with more available from www.ispn.org.uk!), with varying degrees of competence, and there are plenty of meaningless registers of expert witnesses whose only criteria for membership is a cheque and a reference from a lawyer. I often find that the people with the longest calling card are rarely the most competent.

To echo something else you said: in my experience, from now as well as before I entered forensics, the sysadmins I've had most respect for have always been the ones who've risen through the ranks and been self-taught. Give me a bookcase full of O'Reillys over a wall of certificates any day.

I was looking for something else & Google had me drop in here. Found the original post & comments since to be interesting.

I worked in computers for 20 years. Then the lay-off/downsizing/whatever-the-PC-term-is-now. So I spent a few days looking things over, rather than job hunting. Quickly decided I should get certified (ignore the mental institution jokes). So I decided I should start with the basics, signed up to take my HW & OS tests. Came out saying that should be a C+, not an A+. Because I realized that it was the minimum needed to work in the industry. Since then I have accepted what some others said: the cert is to show that you have an active interest in working in the industry, and have made an attempt to learn about it.I was working on VAXes. Stories were coming out about security issues & I told my boss, several times, that we needed to hire somebody who knew security. One day he said "OK, you're now officially ou security guy."I don't learn well. When PCs began becoming common, I told the same boss we needed to get somebody who knew PCs. One day I was told "Bob's leaving for the Philipines. Tomorrow (Friday) find out what you need to know, because as of Monday, you are now the PC Dept Director."But with all my years of experience (starting with that first 44-lb Mark IV I soldered together), I've learned lots studying for those certs.Are there paper certs? Of course. Visit some doctors, lawyers, scientists,... and you can learn it's true there, too. My undergrad degree is in nuclear chemistry, my master's is in anthropology. But I'm really a teacher. I don't fix what's wrong, I try to get them to fix it. So I've taught people enough to get them certified -- and if they stay where I am, I teach them more. But I realize some aren't capable of getting certified: too dumb, not enough gumption, too lazy, etc. I've seen successes and failures. There's nothing ood - or bad - about certifications, but as humans, we'll always find ways around it. Just look at govt in this country. How often have I seen somebody speed up to be sure to get through the light. In one sense, you could say the law is what caused them to do wrong. And for some, to say the law was wrong in the first place (I'd like to go down that path, but...)

So, the ones that said certs are to show you've learned the minimum necessary to get into the job - or to go on to the next level of the job, I would agree, agreeing that there are people who will cheat.To the ones who said a cert just means I can have a better idea what to interview them on (if I were in HR), I'd agree. To the ones who said they are worthless, I'd disagree, though I'd have to agree that it is possible to get a law degree, or doctorate (medical, science, ...) and have it be meaningless.But I also think if we spend to much time saying they are worthless, then the bureaucracy will decide THEY will be the ones to certify us (aka license) - at which point the cert (license) will be(come) worthless. Because bureaucracy has to pander to the lowest common denominator. (Look at drivers licenses, schools, and anything else our govt controls for examples.)Time to get off my soapbox, too.John

I agree with most of the original author's post and disagree with most of the posts that support certification.

Some of the arguments that have been posted in support of certifications are clever indeed, but fall apart when scrutinized closely.

For example, to use the analogy of doctors and lawyers and what their requirements are is inappropriate.

Computer Technology, and the study of it, is significantly different from other fields of study. And it is an error to say that just because other forms of employment have began to implement certifications that computer forensics will need to as well.

We need to stick to the issues, not blindly implement some rules.

For example, here is a common error people in support of certifications frequently make - they blindly make general statements about what knowledge is required to hold a certification.

A statement I like to make to these people, that usually causes them to spontaneously erupt into a barrage of insults, is that a person needs to know nothing about Windows in order to be a forensic examiner. Nothing about the registry. Nothing about NTFS. Nothing about Internet Explorer. It's usually quite funny to watch their reaction.

After they are done with their assumption-based "rebuttal", I simply respond by saying what about the guy who only offers computer forensic services of Linux and/or Mac computers? Windows knowledge is not relevant for him.

Now take it a step further. Lets say a case involves some proprietary software for which there is no forensic training or certification. Can you honestly tell me you would take the word of a certified forensic examiner over the programmer of that software as to how it works?

Stop trying to equate computer forensics to other unrelated fields. For crying-out-loud, you don't need to hold a certification to write the frigin code we examiners have to examine the results of. So don't tell me I need a certification that the person writing the code doesn't.

Judge a peron's qualifications based on their training and/or experience, but don't try to tell me that because a person passed a test, that they somehow have even a baseline understanding.

Certifications do NOT prove a baseline understanding. That gets into another whole argument that has already been posted on this forum about certification requirements. And, if a certification requires a training course, then why not just accept the fact that person when to the course as proof of qualification?

I certainly hope that if the people supporting certifications win, that they won't have their heads in the sand and try to say that a Windows based certification proves qualifications to examine Linux and Mac systems, too.

I could not agree more...I have been in the IT industry for 15+ years, Law enforcement for more than 8 years most of which is investigator experience including the last 3 years on cyber crimes. I am looking for a full time position in computer forensics and have applied for more than 100 jobs in the last years or so, spoken to recruiters, but never once gotten an interview. Why? I do not have a college degree (instead I choose to work for the same company for 14 years out of high school until getting laid off after 9/11). When HR start that elimination process they drop all of the those resumes with no degree, mine included, and have no criteria for equivalent experience. I have (and are currently) working with people who are HIGHLY educated (BA, MBA, you name it) but can not even install a font or set up a projector!

First, may I suggest that people that don't have the same skill set as you do aren't automatically "idiots".

That being said, I will have to confess that I would be classified as one of those "idiots"...I was laid off from my CAD drafting job and encouraged by the state unemployment office to obtain several CompTIA certifications with the hope of tranistioning into IT as a new career. I am taking the first exam today and will pass it - not because I know the material inside out but because I was able to memorize the material enough to answer the test questions. I know if I take that certification into an employer and am asked to actually DO something, I would not be able to do it.

I am open for suggestions. I am trying to learn the material but am finding it impossible to cram five years of practical work experience and knowledge into five months of certification classes. There are many in my class who DO have the experience and aptitude for this field who have been laid off for whatever reason, and are trying to get certified in an attempt to create some notice from HR departments. I'm just trying to do what I have to do to make a living, and I'm too old to work at the grocery store - too young and broke to retire.

I am a Microsoft Outlook user. Some times I face problem with my Outlook, I am unable to open it or it gives me an error message. I lost all my outlook contacts. I am aware that there are many Repair Outlook PST File tools available in the market. But even repairing our pst file by the software I lost some emails and contacts etc. I remember, in past I had some problem in my Outlook, I purchased one PST repair software, however it just recovered some emails. I lost many emails folder from my outlook, then I purchased one more software but still I lost many important contacts.

Hi, Good work folks! Keep that work great. Computer security certifications are easy to gain but the quality of the certification and its worthiness in the market is important. There are many such computer security certifications and one of them is EC-Council's Certification. For more information go through the web link:http://www.eccouncil.org/certification/why_choose_ec-council_certifications.aspx

Way to go John,no flame from this corner either.However not sure about the ire of some future hiring manager.

Like you I had a similiar hiring experience.The guy showed up 1st,left last,finished his portion of the job 2hrs faster than the more experienced in his team and then asked for more all witin 3 weeks of hire,and he had no degree.Like you,I've also met my share of IT "experts" who have no clue,yet will go down convinced they know better,you gotta remember, "they are the IT leaders".I've also looked at some of the certs and with 15 yrs of IT and post graduate qualifications,I am forced to attempt then only to populate my CV,to satisfy a string search.Aside from that they hold minimal value to me personally,(except for the pain of the cert fee)and teach me nothing.I have learned and continue to do so, from guys and gals who are in and have been in the trenches as well as by those who take the time to write an unbiased white paper or technical publication.

I ran into a forensic guy in the UK who was highlighting hiss 6 mths of continous training to get certified;however my thought was so if the tool broke then what? somewhat like the mechanical engineers I met fresh off university back in my Oil days...98% of them had no clue what a socket wrench was..

I do agree that certs show a basic understanding, with emphasis on basic. But there needs to more things taken into consideration like Joe states. Experience as well as understanding of underlying technologies and the ability to learn help to form the complete individual and no cert will ever be able to show this.Mold Certification

Certifications are credentials. Just like college degrees are credentials. Having a certification doesn't guarantee you're an expert, just like having a college degree doesn't guarantee you're an expert.

I've met many incompetent certified people, and I've also met many incompetent degreed people. Certifications are no less useful than degrees in "validating" technical expertise.

They're just one more data point that a hiring manager can use to determine if a candidate is a good fit. I don't see the problem with certifications from that standpoint, nor do I see a point in opposing them or labeling them as "evil".

The problem discussed in the article is not caused by certifications, but rather by constrained resources. (In terms of HR resources, time to hire, time to evaluate, etc)

If certifications are "evil", then college degrees are "evil" as well.

When it comes to chic style, nothing can rival the distinction of a replica handbags , replica handbags, or Louis Vuitton sunglasses. Diehard brand loyalists think nothing of handing over an entire paycheck just for a small replica bags . Some of the world's most prominent actors, musicians, and models are often photographed carrying the latest designer replica handbags fashions

Honestly I abused the system just as you mentioned. I didnt know didly before starting my career, but did a bunch of studying on technologies I never touched during my internship, I passed the tests, got passed the HR interview, and got my foot in the door.

Once working .. that is when my learing really began. Its sad but that route was most viable to me. I didnt have the means/or money to build a server when I was younger, but I certainly had the means to study a testking/certkiller/etc to get a certification under my belt, without ever administering a server in my life. This "route" I have taken across the different certifications I have.

For me its kind of like, the chicken and the egg. How can you get experience/training to pass a certification without the job, how can you get the job without the experience/training to pass the certification.

The topic here really made me stumble to read it as I think this it's a really controversial topic, I think certification is really important but without work experience it's not worth that much as there's a huge difference in the educational and office environmental. I would say be qualified but not only by certification but by some quality work experience.

"For me its kind of like, the chicken and the egg. How can you get experience/training to pass a certification without the job, how can you get the job without the experience/training to pass the certification."

Owning to the high quality and competitive price, our products had won great popularity all over the world. power balanceare are sweeping the whole nationour Power balance is designed to work with your body's natural energy field. They are come equipped with two visible Power Balance holograms. power balance

I think certification is really important but without work experience it's not worth that much as there's a huge difference in the educational and office environmental. I would say be qualified but not only by certification but by some quality work experience.

I totally totally agree. So often it's not the most qualified person who is the most relevant for the job, but just try and get past an HR department if you don't have the necessary quals. Excellent post.

Having worked in an HR department for years before my current role, I've seen many people's CVs get thrown in the bin just because a few letters or qualification names were missing from their CV. Sad but true. Good post.

I realize this post is old but just wanted to add my opinion. John I tried the MFCE after taking one of the MFI trainings and to be honest, was a waste of my $250. Trying to work on that and do my job at the same time was impossible. On the last project I was tied up so I quickly tried to image the phones and was told to send them on to the next person. I did that but the images in Bitpim were blank and when I tried to get the phones back, I was told no. So basically I was not allowed to complete the exam. I was not the only person that had issues with this process. In my opinion, experience always trumps certification. What people can learn in a process of hands on experience is much better than controlled situations in a classroom. I've learned more about iPhones and Androids by digging for myself than any class could teach me and every day I learn more.

Wow. Fantastic article, it’s so nice and your blog is very good.I've learned a lot from your blog here, you’re provided blog is very super perfect i really like it. Keep it up and Thanks you for sharing......!

Wow. Fantastic article, it’s so nice and your blog is very good.I've learned a lot from your blog here, you’re provided blog is very super perfect i really like it. Keep it up and Thanks you for sharing......!

Wow. Fantastic article, it’s so nice and your blog is very good.I've learned a lot from your blog here, you’re provided blog is very super perfect i really like it. Keep it up and Thanks you for sharing......!

Wow. Fantastic article, it’s so nice and your blog is very good.I've learned a lot from your blog here, you’re provided blog is very super perfect i really like it. Keep it up and Thanks you for sharing......!

Wow. Fantastic article, it’s so nice and your blog is very good.I've learned a lot from your blog here, you’re provided blog is very super perfect i really like it. Keep it up and Thanks you for sharing......!

AMEN brother...I have hacking and Cracking experience. Started fiddling with computers since 1989 and am currently doing Computer Forensics WITHOUT any "forensic Cert's"...and guess what...i get RESULTS! The MAIN thing is...you must become ONE with a PC and the software...and DETERMINATION is the main key ( in MY case anyways )...I love your post...as you are SPOT ON!!!

What i usually tell people is....with PRACTICAL...you ca write THEORETICAL....but with THEORETICAL...you cant always implement PRACTICAL...Where do you guys thing the "books" came from? From folks having PRACTICAL experience regarding Forensics :-)

Marvelous!!! Job you have done. Actually, you are pleasure to know, seriously. Your work is incomparable. It’s a masterpiece.Your article is very resourceful and beneficial. And the work done is appreciable.Laptop Repair

New Aston Martin Rapid, 2013 Hyandai Veloster Turbo Mazda Takeri Concept and Top Companies of Most popular Cars and Vehicles. Total Concept Cars in the World, Latest Strange Vehicles in the World, All Concept Cars and Strange Vehicles Hot pictures and infoworldlatestvehicles.com