Who Needs Them?

Wouldn’t you like tests to go away? Does it seem like they’re more trouble than they’re worth? I just read an article about a 17-year-old high-school girl who took the SAT test twice, scoring 350 points higher the second time. Such a jump was deemed too unlikely by the Educational Testing Service (ETS), which administers the test for the College Board. ETS informed the girl that she had cheated and that they wouldn’t provide her new score to schools.

Months later, after an appeal process, the ETS decision remained and the girl was left explaining the ETS decision to colleges she wanted to attend. Did she actually cheat? Probably, but who knows for sure? Having to take such an important test was hard the first time. Taking it twice doubles the pain and pressure. I’m sure this girl wishes she had never heard of the SAT, the ETS or any other test for that matter.

The same goes for each of you. I’m not just talking about the SAT, although I’m sure many of you have personal experience with that test and many others taken during your school experience. I’m referring to the constant taking and retaking of IT certification exams. When you graduated from high school or college, I’ll bet you thought you were through with tests forever—I know I did—only to find them constantly looming on every horizon. And the way programs are structured, you usually need to take more than one to get certified!

Do program managers delight in such torture? Do they get private and personal glee knowing they are tormenting their candidates on a fairly regular basis? Maybe they don’t know how seriously their exams are considered or the amount of effort, worry and money that goes into them? No, my 14 years of experience with certification program managers assures me that they are certainly aware of the difficulty of the exams and the stress they produce on each of you. It also tells me that they would use some alternate method of certification if one were available.

Hold it right there. Maybe there are alternatives to tests. Let’s pause for a minute or two and consider some options:

Self-evaluation and surveys: In the early ’90s I suggested to a Novell VP who wanted to increase the number of Certified Novell Engineers (CNEs) from 50,000 to 500,000 that we simply have certification candidates sign a document stating that they are competent enough to support Novell’s products. Of course, I wasn’t serious, but he thoughtfully considered the idea.

Submission of work product and resume: Perhaps a candidate could send in a resume of work experience, with phone numbers so a program manager could verify the work. Or they could provide “hard evidence” of work, such as papers written, contracts completed, servers installed, etc.

Ratings by peers and managers: The certification program could poll IT company managers, instructors and others about the knowledge and abilities of each individual candidate. The managers could fill out a detailed survey about the candidate. Higher scores on the survey would indicate greater competence.

Use a test, but provide the answers to the questions beforehand: This would certainly produce a less stressful testing experience. Another version of this alternative would be to create a test with very easy questions.

No test or qualification at all: Just provide an application on the Web and certify the first 1,000 people who apply, with the assumption that early applications come from more motivated and experienced individuals.

You can probably think of others, but these provide a broad range of examples. Now, which ones would be better test alternatives? We have to remember that the major criterion is that the method must identify the most skilled and most qualified individuals. Personally, I don’t like any of the alternatives. None of them are capable of identifying skilled candidates reliably. Their problems are obvious. Of the tools we have today, tests are the best barometer to measure talent and aptitude.

Having said that, I am a firm believer that today’s IT certification tests are not as well designed or as well built as they could be. There are alternatives to multiple-choice questions that should be used. For instance, instead of trying to measure a person’s ability to regurgitate facts, the questions should measure the ability to think logically and solve problems. We should be measuring how well a person can do important job tasks.

Instead of throwing out tests, let’s just throw out bad tests. Maybe what remains won’t be so hard to take every month or so for the rest of our lives.

David Foster, Ph.D., is president of Caveon (www.caveon.com) and is a member of the International Test Commission, as well as several measurement industry boards.