Scenario mistakes to avoid #1: Eager-beaver feedback

What’s the most common mistake scenario designers make? They provide feedback like this:

Incorrect. You should always have the customer wiggle the synderhobble first. Most noise issues are caused by a loose synderhobble.

Or this:

This wasn’t the best choice. If Lars really has witnessed an ethics violation, you need to encourage him to describe it in detail.

Or this:

Great job. You’ve helped the patient manage their anxiety and as a result they’ll be able to describe their symptoms more accurately.

“But that’s good feedback! It’s helpful!”

Yes, that feedback is helpful. It immediately tells me what I’ve done right or wrong and shoves me firmly down the path of good behavior.

But when does that feedback appear? In the middle of a story. Like this:

You’re on a call with a customer whose widget is squealing. The customer is being driven insane by the squeal, but he has to run the widget at top speed to deliver a product that’s due this afternoon. You tell the customer to oil the widget’s cooling fan.

Suddenly a beaver pops its furry head over your cubicle wall. “You shouldn’t have done that!” it squeaks. “You should always have the customer wiggle the synderhobble first!”

Helpful? Sort of. Annoying? Seriously.

The beaver is just eager to help you learn. But has it helped?

For example, how much thinking did you do after you mistakenly told the customer to oil the fan? Zero. You didn’t have a chance to think, because the beaver interrupted and told you what to think.

How much can you learn when a beaver does your thinking for you?

We can be helpful without interrupting

In the real world, we learn from the consequences of our choices. We tell the customer to oil his fan, but the squealing continues, and we realize on our own that the synderhobble might be the culprit. We’ve used some brain cells and effort, and as a result the lesson is memorable.

But in training land, this kind of learning is often considered messy and inefficient. Also, our clients sometimes think that people can’t be trusted to draw the right conclusions.

The real world lets us learn from experience. The training world tells us what to do. Here’s how we can have the best of both worlds:

Pose a realistic challenge.

Let the learner make a decision.

Show the real-world consequence of the decision (the widget continues to squeal).

Optionally offer beaver-style feedback.

In elearning, we can offer optional feedback through a link, like this:

Mr. Jackson puts down the phone and oils the widget fan. You can hear the squealing continue in the background. “That didn’t do a thing,” Mr. Jackson says when he gets back to the phone. “And now the shop stinks of oil.”

Why did this happen?

First, we show the real-world consequence. Then, people who want help from the beaver can click the link. Others can continue to try to diagnose the problem on their own.

If your software allows it, you could keep an eye on people’s choices. If someone repeatedly fails to learn from experience, you could unleash the beaver on them even though they didn’t ask for help.

In live sessions, you could pass out squealing widgets and have people try to fix them on their own. “Let me know if you need help,” you’d say. Then you’d stroll through the room, and if you see someone getting too frustrated, you could step in to nudge them toward the synderhobble or just tell them what to do.

Let a job aid help

Often, we use beaver-style “telling” feedback when a job aid would be better.

For example, the widget exercise could include a link to a diagnostic checklist. The link could appear on the same screen as the question. So when the caller describes the problem and we need to decide what to do first, we can either:

Make a decision based on our pre-existing knowledge, or

Look at the cheat sheet called “How to Fix a Squealing Widget” and then make a decision

We then see the real-world consequence of our decision. We can click “Why did this happen?” for an explanation that confirms or corrects our knowledge. Ideally, the explanation includes a snippet of the job aid that shows the relevant step. This tells the people who skipped the job aid, “Hey, you would have gotten this right if you’d looked at this thing.”

“But they must all be exposed to all of the information!”

Some clients insist that everyone should be “exposed” to the right way to do it, regardless of whether they made the right choice in the scenario. “They might just guess right,” the client says. “You have to tell everyone the right way.”

Here’s an easy fix: Show the real-world consequence as above, and then provide the “telling” feedback. It’s no longer optional. Everyone gets “exposed” to the information, but the exposure happens as feedback to their choices. Instead of staring at a presentation, people have to think, and they see the information when it most matters to them.

Elearning

The elearning version of this approach might look like this:

Mr. Jackson puts down the phone and oils the widget fan. You can hear the squealing continue in the background. “That didn’t do a thing,” Mr. Jackson says when he gets back to the phone. “And now the shop stinks of oil.”

Why this happened: Most noise issues are caused by a loose synderhobble, not the fan. You should have the customer wiggle the synderhobble first to see if it’s loose.

[GRAPHIC] The relevant part of “How to Fix a Squealing Widget” with a big arrow pointing to step 1, “Wiggle the synderhobble.”

You’d create scenarios for other squealing widgets, including one that represents the widgets that squeal for reasons unrelated to their synderhobbles. Widget support people go through several scenarios representing different situations until they’re using the cheat sheet with confidence, just like they should use it on the job. Along the way, they’re “exposed” repeatedly to the correct diagnostic procedures, with no need for a presentation.

Live training

Here’s one way to do it in a live session: Give everyone a squealing widget and the cheat sheet called “How to Fix a Squealing Widget.”

“You’ve got 10 minutes to fix your widget,” you say.

After 10 minutes, stop everyone and see how it went. Some widgets are still squealing. A few are squealing because their repair people didn’t use the cheat sheet, and a few are widgets that represent the minority that don’t have a loose synderhobble.

In the group discussion, confirm what most people learned: Following the cheat sheet will silence most widgets in one step. The remainder need more diagnostics. Using one of the still-squealing widgets, go through the sheet as a group, applying each step until the squealing stops.

What about branching scenarios?

If you’ve got a branching scenario, you might try this:

Plot the scenario with three types of endings in mind: one best ending, a few mediocre ones, and a few poor ones.

Build the plot, letting paths occasionally cross so players can realize they’re heading for a poor ending and get on a better path.

Make any job aid or other reference available at all times.

Have each decision result in a realistic consequence with no “telling” feedback to interrupt the story.

If you have any especially tricky decisions or startling consequences, consider offering an optional hint or explanation at that point, in a link.

At each ending, first show the end of the story. Then (optionally or not) describe how the player got to that point, point out mistakes they made, and offer hints that will help them do better when they play again.

There are lots of other approaches, of course, depending on how much experience people already have, how open they are to this type of activity, and what types of skills you want them to practice.

How does this work with “tell, show, do?”

If your client wants you to expose people to information, they probably expect you to use the “tell, show, do” formula: First you tell everyone what they might need to know, then you show them how you do it, and then finally you let them do it themselves. Kind of like this classic video on how to diagnose sigmoid rumbling in a turbo encabulator.

In a future post in this series, we’ll look at whether “tell, show, do” is always the best approach and consider some alternatives.

“In the real world, we learn from the consequences of our choices. We tell the customer to oil his fan, but the squealing continues, and we realize on our own that the synderhobble might be the culprit. We’ve used some brain cells and effort, and as a result the lesson is memorable.”

My son sent me a link to your article after we discussed the relationships between game play OLT. The opportunity for gathering data had me salivating. Such data could be used to feed a simple AI agent which decides when to “deploy the beaver”. Such data would also be useful in both revising existing material and building future examples. And, of course, academic research would benefit.

Unfortunately, such data gathering is too costly for use in commercial environments – but one can dream.

Thanks for a thought provoking article!

On this site, you’ll find ideas that will help you design lively training for working adults. It’s written by Cathy Moore.