No Quick Fix: Safety cannot be Reverse Engineered

No Quick Fix: Safety cannot be Reverse Engineered

In one of my previous lives, I was an instructor at the U.S. Air Force Emergency Management School. At the same time, my wife taught second grade students at the local elementary school. We used to marvel – and laugh – and how similar some of our students were in terms of looking for seemingly quick ways to finish their work, pass their tests and avoid really bearing down and studying hard. Like any group, we both had high achievers – those who would intently focus during class, study at night and deliberately excel on every assignment – low achievers – those who wouldn’t try at all – and everything in between, including those with a ‘7-0, good to go’ mentality (70% was the passing score on exams during the course).

A Lack of Effort

In my wife’s classes, the second graders who didn’t want to study or work hard during class would usually attempt to complete their assignments in as little time as possible so they could get back to not working, even if it meant getting the answers wrong. If the assignment was homework, these students would usually just not do it at all. In either case, these students would usually find themselves in danger of failing at some point during the school year, an issue easily rectified in most cases by simply doing their work. However, even knowing this, many would still wait until they were danger of failing before actually doing anything about the problem. In the case of my Air Force students, the same issue usually existed with those who chose not to study or work hard. They would try to sleep during class, not attend study sessions, not study in their rooms at night and would simply hope to pass their exams on whatever knowledge they had and luck. In many of these cases, these students would find themselves with a failing exam or field demonstration grade (each of which could fail them out of the school) at some point during the 3 ½ month course. When this happened, the failing student’s performance record would be evaluated by the instructor staff and leadership to determine if the student would be allowed to re-test, if the student would be ‘recycled’ back through the course block he or she failed, or if he or she would simply be removed from the course altogether and either send to a different school or send home from the Air Force altogether. Needless to say, for anyone wanting to remain in the Air Force, he or she would need to prove there was a reason for the instructor team to allow a retest or recycle. If the student didn’t show any effort during class, didn’t attend study sessions and, overall, didn’t show any interest in even trying to pass, there would be no reason to allow him or her a second chance to pass.

No Quick Fix

With these heightened stakes, many students who didn’t pass their exams would try to find a way to invalidate test questions or otherwise make up enough points to barely pass their otherwise failed test. For example, if a student made a 68% on an exam requiring a 70% to pass, that student might ‘challenge’ a test question by claiming invalidity so that he or she could get the two points necessary to pass. Usually, the claim was that, ‘If that invalid test question was thrown out, I would have passed.’ In reality, it wasn’t the one test question that made the student fail – that is, if the question was even invalid to begin with – but instead the other 15 questions he or she marked incorrectly. Had the student studied and done his or her diligence, the one question now being claimed to be invalid wouldn’t have made a difference. Instead, the student was attempting to reverse engineer his or her grade, to try to pass the exam by scrounging enough points on the back end to pass.

In another example, for those students who were granted an opportunity to retest, many of them would request a review of their failed exam. During these reviews, some of these students would only show interest in the specific questions they missed. With this, instead of studying the concepts on the exam and seeking a better understanding of all of the material on the exam, these students were hoping that they could simply make up the questions they missed to bring their score up to a passing grade. There were/are a number of issues with this idea; for example, doing this, the student doesn’t actually gain a better understanding of the testable material. Instead, he or she only learns a handful of specific questions in an effort to barely pass the second try at the exam. Additionally, this concept only works if the test remains the same. If any questions change on the second attempt, there’s a significant chance the student will again fail. The only way to become successful in a legitimate, valid and reliable way is to study, learn the concepts and gain a broad understanding of the testable material.

With all of this said, what does this have to do with safety? Safety, like schoolwork and tests, cannot be reverse engineered.

No Safety Management, No Validity or Reliability

Many organizations track lagging indicators in safety to supposedly determine progress. They set targets for their recordable rates, DART rates or other indicators and then report them on a recurring basis, often using spreadsheets and colorful boxes to illustrate ‘good’ and ‘bad’ rates. However, many organizations will not, even while tracking lagging indicators, have a definite safety management process. Ultimately, without a complete package – hazard analysis, hazard controls, an information/communication program, leading indicators to validate hazard control use and target unsafe behaviors and conditions, lagging indicators to track progress after validating hazard control use and investigations to follow up on incidents and ensure preventative and corrective measures – trying to affect lagging indicators is like trying to pass a test without ever studying. I’ve seen many organizations pontificate and argue about why their incident rates are what they are but do so without even having analyzed hazards or implemented hazard controls. I’ve seen organizations argue about why employees were injured but without having ever trained those employees on hazard control use. I’ve seen some organizations argue about why employees were working unsafely without ever having implemented a hazard control for the process that injured the employees. Ultimately, without a safety management program, even talking about lagging indicators is a moot point. Like trying to pass a test without making an effort in class or studying, trying to affect incident rates without a safety management program is a lost cause. Furthermore, should a question of the organization’s diligence come up (as in a deposition or otherwise uncomfortable situation), it’s almost impossible to defend a bad lagging indicator without a safety management program to show an effort. Safety cannot be reverse engineered.

Investigations are Only a Part of the Program

I’ve had some organizations with heightened incident rates ask me for recommendations on how to improve their safety culture. After conferring with them on the need for a hazard analysis, hazard controls, an information program, leading indicators, lagging indicators and investigations, several of these organizations have attempted to forego a full-circle program and simply investigate the most recent incidents that occurred within their organization. In short, instead of trying to truly understand safety by thoroughly identifying, assessing and controlling the hazards in their processes, these organizations were attempting to only identify and correct single unsafe conditions or behaviors – after the incident had occurred. For example, if an employee was injured by not using a hazard control, their intended course of action was to call that employee into a meeting and write up a performance improvement plan. While this is a valid course of action as part of a safety management plan (when applicable), my recommendation to them was to determine what the employee was doing, what hazard control was needed, how to communicate that hazard control expectation to the team and how to validate that all employees are using that hazard control. This, in turn, would allow for safe behaviors and conditions across the whole team instead of simply calling out one employee and hoping that one employee wouldn’t repeat the behavior that he or she was injured while doing, all the while the other employees are still possibly working unsafely. This is exactly like trying to pass an exam on the second try while only studying the missed questions. This doesn’t provide an understanding of the whole situation; it only allows for a few specific examples. Furthermore, there’s a significant, if not absolute chance that the next incident will be totally different than the last. By only investigating specific incidents, in a best-case scenario, this prevents a few specific incidents from being repeated. It does not, however, create continual improvement in hazard identification, assessment and control. Again, safety cannot be reverse engineered.

Don’t Get Stuck in the Past

It’s unfortunate when organizations get stuck in the past and only focus on lagging indicators. Lagging indicators are simply rates of injuries that have already happened. If we know how and why these incidents occurred, we can transfer this knowledge into our continual hazard analysis, improve our hazard controls, communicate them and begin to validate their use. With this, we can work towards continually improving safe behaviors and conditions instead of arguing and pontificating over lagging indicators. I’ve seen many organizations stuck in the rut of trying to make lagging indicators look better by arguing about incident culpability or recordability, all the while without a safety management program to even show due diligence in hazard control. I’ve seen organizations try to improve safety by only following up on specific incidents as if preventing repeats of those exact scenarios will allow for safe behaviors and conditions without ever identifying, assessing and control hazards. Safety cannot be reverse engineered. It has to be a proactive effort stemming from the organization’s leadership and integrated into the whole team’s processes and culture. Otherwise, trying to change a safety culture is just like trying to pass a test without studying; we have to put in the hard work up front to make positive results. It can’t be done after the fact.