Gone Fish’n: Safety Measurements don’t have to be an Illusion

The measurement revolution began in my prime. I was working with Industrial Engineers, a nice bunch, with the world-view that everything can be designed to run smoothly. We were a part of the Quality Revolution. I had just gone to see W. Edwards Deming for the third time, eager to hear his thoughts on measurement.

Deming was a statistician who, like me, saw the world in terms of variance. We don’t look at absolutes, we look at differences. Therefore, when viewing injury data I look for change over time, variation across work units, comparisons to industry top quartile, and the like. Within the groups of numbers, quite interesting and actionable sources of variation lie.

When someone gets injured, many look at the investigation for answers as to what to change. All well & good. But what about the dozens of stories, the dozens of variations in human and machine behavior that, if measured, would have revealed a basis for action before the injury? A good measurement system can save a world of hurt…literally.

But if you don’t understand variance, your view of the world blinds you to risk. Deming said that one of the greatest threats to organizational quality (and safety, from my perspective) is “single data-point management” too often practiced by managers who don’t understand the concept of variability. Unfortunately, in the safety world I see too many single data-point managers.

I recall a story of a man in South Africa who was caught without his safety gloves by his supervisor who immediately fired him based on this single data point. The safety manager refused to accept the firing because the man had 3 young children. Upon interviewing the man he found out that he had removed his gloves temporarily to clean his safety glasses for better visibility.

FISHING BADLY

I feel bad for my son because he likes fishing and I’m not very competent in the sport. We go to the lake, he throws out his line and waits to hook that one fish of the unseen thousands.

Single data-point management resembles that type of fishing. The manager or supervisor, or you, yourself, have a list of measures consisting of outcome data (such as injuries), or instrumentation data (from machines or via other forms of technology) resembling the kinds collected on fishing trips. Also, walk-arounds are like fishing trips where you’re fishing for violations of rules. You’ve got your line out and are trolling for rule-breaking behavior.

Measurement is often used that way much too often. You don’t pay attention to thousands of safe behaviors, but just hunt for violations. As long as the measures come back clean, you credit an absence of bites. So you continue looking for that instance in which the data gives you what you think is a clear danger signal and you yank on the line to attempt to hook the violation.

Once, my sons and I went fishing with a guide on a chilly day in Florida. The fish were not biting because it was so cold, so the guide threw some “chum” into the water (something like candy for fish), to draw them in, so my young sons could score some catches. And, fortunately for our vacation, we did catch a couple of small hungry fish.

Sometimes when managers see injuries occurring, but lack valid or accurate measurement systems capable of revealing root causes, they may begin throwing safety’s version of “chum.” They walk around trolling frequently observing the smallest fish; that is, catching the smallest violations.

BUT… these apparent safety successes actually are an ILLUSION.

You are made to think you are doing something productive, when in fact you’re actually unintentionally promoting injuries; and diminishing your managerial skills in the process.

THE ILLUSION

Think about it. Early in your career as a lead, supervisor, and/or manager you probably didn’t start your new job thinking: “I’m going to be pissed off all the time and scold employees.” Rather probably you approached your assignment thinking: “I’m going to be the calm, understanding boss.” The measurement illusion you picked up by chumming the waters during your fishing trips turned you into a scolding and ranting supervisor.

The reality is that each of your employees has good moments and bad moments. All have moments of good safety performance, making an effort to keep themselves and others safe. At other times they may have moments when they take risks, intentionally or not. Most of the time employees vary around an average set of behaviors that, for the most part, keep them safe and productive.

Now, as a manager who fails to pay attention to variance, you tend to overlook this average set of behaviors generally responsible for keeping employees safe. What you noticed early in your leadership career on your “fishing expeditions” were the outliers in performance.

You noticed when a particular employee promoted safety in some exceptional way. Perhaps, tired of waiting for an engineered fix, the employee built a guard on a piece of machinery that had worn down and presented a hazard. You noted that behavior and made a clear effort to praise the employee, perhaps even recognizing that effort in front of the work team.

But you didn’t understand variation; you praised a single data point. This exceptional performance was an outlier; not what that employee always did. So the employee’s safety performance regressed back to the employee’s average set of behaviors. You saw the employee’s safety performance drop and didn’t recognize this as a natural occurrence. Perhaps the next day you see the employee take a shortcut by walking over some pipes. You think to yourself “I just praised him and now he’s doing something risky.” You become less likely to use praise in the future because you’ve just been punished for being positive. But your interpretation is an illusion, because it was not your praise that caused the drop. It happened naturally.

Now consider what happens when you go trolling for violations. You search far and wide for risk, ignoring the average safe practices going on all around; or worse, think that if you praise safety, you’ll promote injuries. Then you catch an employee who does something obviously risky. Perhaps, he had his safety glasses around his neck while engaging a machine. He had taken them off to clean them and check some paperwork and forgot to put them back on. So you yanked the line and caught him. You were upset and disappointed in him and let him know it by reprimanding him.

Well, in the past, this employee probably had always worn his Personal Protective Equipment (PPE). This lapse was a rarity. In fact, after this incident, this employee wore his PPE consistently like he always had. You may have perceived “I just scolded him and his safety performance increased”. You become more likely to scold in the future because your use of scolding and/or discipline has just been reinforced. But that is an illusion because your scolding did not produce the improvement; it happened naturally.

Your punitive management style is an insidious superstition because slowly, at first, it shaped your increasing use of negative consequences like scolding and discipline. Eventually your management behavior has become like a fishing expedition– looking for opportunities to use unpleasant consequences — under the illusion that they work. Sometimes that can shape you into quite the grumpy dude. Even worse, you’re not stopping injuries.

THE DAMAGE

Fishing expeditions frighten employees along with the measurement system you use. This damages a safety culture. Think about it from the employees’ perspective. When they see you coming, they anticipate punishment. When they see a measurement taken, they worry it will be used against them. Nothing positive there!

Folks get nervous and distracted. They hide their behaviors. Peers teach one another how to avoid getting caught. (I’d do that if I were a fish who could talk.) The culture deteriorates. Injuries increase.

Don’t be fooled by this illusion. Instead, look for variation, discover its source, and manage it. Better fisher folk than I use sonar in their boats. Sonars bounce sound waves down toward the lake bottom. Schools of fish reflect this wave back to the sensors and show up on the sonar screen, giving you a good idea of where fishing may be productive.

A good measurement system scans your operations objectively, looking for pockets of variation that may reflect hazards and risks. When you find this variance, use your problem solving tools to cast your net over the potential problem and solve it… before harm results.

But you can only look at variation and its sources if employees are participating in safety measurements. Outcome data such as injury rates are not sufficiently sensitive. Additionally, your own walk-arounds probably reflect your personal biases and are confounded, thereby, by your presence. Instead, turn to your employees who should be able to provide the insights into the risky behaviors and conditions that set the stage for them. They know the sources of variance; they can collect those data in a no-name/no-blame system of the type found to be successful in Behavioral Safety systems and they can play a part in reducing that variance.

Timothy Ludwig’s website is Safety-Doc.com where you can read more safety culture stories and contribute your own. Dr. Ludwig is a senior consultant with Safety Performance Solutions (SPS: safetyperformance.com), serves as a commissioner for Behavioral Safety Accreditation at the non-profit Cambridge Center for Behavioral Studies (CCBS: behavior.org) and teaches behavioral psychology at Appalachian State University, in Boone, NC. You can contact him at TimLudwig@Safety-Doc.com.

Tags:

Given that W Edwards Deming was invoked for viewing the world in terms of variance, I was surprised that, although “average” was mentioned, the development of the chart did not include the mean and 3-sigma control limits.

Dr Deming was very clear about two kinds of variation: variation from common causes (random causes that belong to the system) and variation from special causes.

For a system in statistical control, in the absence of other relevant information, we predict, logically, regression to the mean. If the mean and the variation due to common causes are not acceptable, we should work to improve the system. However, a point outside the control limits is a signal, a special cause, which indicates a need for action – on one data point.

If it is a good result we should try to learn from it to replicate the good performance; if it is a bad result we should take action to prevent a recurrence.

As Deming stated, Mistake 2 is “To treat an outcome as if it came from common causes of variation, when actually it came from a special cause”. Depending on where it comes on the chart, an outlier may well warrant special attention.

Nick, Thanks for the thoughtful comments. You make a good point about Deming’s use of control limits to understand the common and special causes of variation. Indeed, rewarding and punishing behaviors within common cause of variation is the biggest fallacy here because they are most likely to succumb to regression to the mean.

Deming taught that events that are special causes of variation (one data point) should be understood. But since it is a special cause, these events quite often unique and passing (e.g., a supply chain disruption due to weather). Managers should seek to understand these events and learn from them, as you suggest. (For other Statistical Process Control special causes I suggest Mainstone, L.E., & Levi, A.S. (1987). Fundamentals of Statistical Process Control. JOBM, 9, 5-21.)

The question to be answered is if the employee’s behavior contributed to the special cause or not. Most special causes can be traced to environmental events and not significant one-time changes in behavior, unless that behavior was influenced but he special environmental event. Therefore, reinforce or punish the employee for something they had no control over may still lead us to the types of problems I described.

Behavioral research uses Statistical Process Control to understand behavior change. We’ve verified the efficacy of behavioral safety processes using SPS. These approaches seek to understand the variation you describe and create stable improvements that can, by themselves, be considered special causes because employees are indeed acting differently (e.g., providing safety feedback, etc.).