Posts Tagged ‘evaluation’

I hear it all the time when we build training objectives. People will say, “you’ve got to understand the process,” “you need to know the features and benefits of the process,” “you need to be “aware of all the safety hazards.” Words like understand, know and aware or all things that happen inside people’s heads. It’s not a description of what they will actually be doing with this new knowledge, understanding and awareness. In many cases, it’s nothing or something like retain it for a few weeks and then forget it. It’s quite a leap between knowing how to do something and actually doing it. You can know how to make a great presentation to a large group but that doesn’t mean you can. You can know the features and benefits of a product but not be able to use them effectively during a sales presentation with a hostile prospect.

So to write better objectives, I like to ask the question “so what?” If I have an awareness of safety hazards..so what? What does it do for me? How do I put that awareness in action? It’s better to state something like, “recognizes all safety hazards and takes steps to avoid them.” It’s a better sales objective to say, “presents products by describing the appropriate features and benefits to meet the customers needs.”

What this leads to is training the focuses on results and changed behaviors rather than knowledge acquisition. It also leads to different ways to train versus death by PowerPoint and data dumps. This is especially true when subject matter experts deliver training. They tend to “tell” students what they think they should know. They forget the part about how they learned to put that knowledge into action.

Finally cognitive objectives lead to paper and pencil tests..fill in the blanks, multiple choice, and true/false. Easy to score but won’t tell you about what students will do on the job. That’s way there is often little correlation between what happens on the test and what happens on the job. Changing these objectives requires more observation do evaluate what’s been learned.

In education and training we spend a lot of time and effort testing for knowledge. Some of it is even around application of knowledge. But here’s the rub. Knowing and doing are two very different things. I think it goes with the comment, if you’re so smart why aren’t you rich?

We see it all the time we’re someone does good on the test and poorly on the job and vice versa. Many people think this is all about test anxiety. In reality, it’s because of the major difference between knowing and doing.

Also knowledge tests are rarely about speed and fluency. In the classroom you have time to answer questions, on the job you need to respond quickly.

So two suggestions. If you need to do knowledge tests, put a fast time limit on it so it mirrors the pace of the job. Second, dump the knowledge tests and replace them with expert on the job observation.

This is a very popular discussion question. It goes along with how do you actually measure training. This is the first of a series of posts on the topic. In this posts, I want to talk about the evaluation sheets that are passed out after every seminar, lecture or course. It includes questions such as:

How did you like the workshop?

Did you get anything out of it?

How did the instructor do?

Did you like the lunch?

It’s a lot like a customer satisfaction survey. It’s never a bad idea to ask customers what they think.

However, I think the value of these evaluations is very limited. Just liking something doesn’t necessarily lead to learning, skill building or real change. This evaluation really won’t tell you if anything will have a lasting effect a day, a week or a month later. Here’s an example,

One of the training programs that always gets great reviews is style training. Style training comes in lots of different varieties and people like them. Everyone likes to here good things about themselves. However, being able to actually use this information to do anything significant on the job seldom happens. In fact, it’s more likely to be misused.

So, the point is..it’s nice to have happy participants. Much better than unhappy participants. However, don’t confuse happiness with learning.

In order for new learning to happen, you often have to clear out and challenge beliefs and preconceived notion. One very interesting way to do this is a pre-test. I found this one which is really good. The trick here is to forgot about the topic but rather see how the simple test challenges your beliefs and that you may know this already.

True and false, multiple choice, fill in the blank, matching and even essay tests for the most part evaluate knowledge and comprehension. They’re easy to write. Easy to score and easy to compare one student to another.

However, knowing and doing are two very different things. That’s why there is often very little correlation between how employees do on classroom tests and how they perform on the job. Unless you’re training academics, there needs to be a more meaningful way to do testing.

It’s actually fairly easy. You ask an expert to watch the student work or perform a required task. That expert will be able to tell you a lot about what that student knows and doesn’t know. Most experts tell me they’d know in less than five minutes.

You can then ask the expert what he or she is looking at. What are the key inidicators? Now you have a good evaluation checklist that you can use to train others on this type of evaluation.

This type of evaluation doesn’t favor those who are “good test takers.” It also tests the links between a range of tasks and knowledge.

Unfortunately, this type of test isn’t very easy with elearning unless you can do some very sophisticated simulations.

Here’s a reading test that any school system can use that is guaranteed to be better than what they use today. Give a student a book. Say, okay read it to me. After five minutes, ask, “what did you just read?” Any good teacher will know where this student is on their reading ability. You also won’t get anyone through high school who fakes their way through without learning to read.

Welcome!

This blog is focused on anything related to learning faster. From time to time, I'll be posting about my book Learning Paths: Increase profits by reducing the time it takes to get up-to-speed (Pfeiffer 2004).
Email me!