Small Bad Jim and RBM: A Cautionary Tale

By Penny Manasco - September 8, 2016

Walking around Denver this weekend, I noticed something new on a street I had traveled many times. It always amazes me when I find something new in place I thought I knew very well. I discovered the plaque pictured beside this blog.

In 1864, James “Small Bad Jim” Clark was the first man to rob the Denver Mint. The gold bars were so heavy that he began dropping them only one mile from the Mint. Six days later, the diminutive desperado was captured 25 miles south of Colorado City. His horse had deserted him and Small Bad Jim was in tears.

Unfortunately, Small Bad Jim did not ask the important RBM and Quality Management questions: “What could go wrong?” and “How will I know?”

If Small Bad Jim had asked, “What could go wrong”, he might have determined the weight of one gold bar and how much weight his horse could carry. Small Bad Jim should have also done some contingency planning since his single point of failure, the capacity of his trusty steed to haul the loot, was critical to his success.

It’s hard to believe this 1864 tale of Small Bad Jim could teach us so much today about RBM and Total Quality Management.

First, the lessons I took from this story were to consider everything that can go wrong, determine how you will identify the issue, and implement plans to monitor whether/when those issues occur.

While the Risk Assessment and Categorization Tool, provided by TransCelerate, was a great contribution to the industry, MANA RBM believes we need to take that document “up a notch”. For instance, it was not enough for Small Bad Jim to identify a risk and categorize the risks, he needed to know how to identify the risk and what to do next if it occurred.

In our clinical research environment, we can review the protocol and determine there are risks around protocol compliance, subject reported outcomes, and dosing errors. The next, important step, which is often missing from the process of risk assessment, is to determine what data we need to collect to identify whether the protocol was conducted as designed, whether the data were collected directly from the subject, and whether the dose was calculated correctly.

This is a big change from traditional data collection for statistical analysis only. By collecting data or designing reports that can confirm protocol compliance, dosing, and subject-reporting, we can continue to develop and refine our methods to deliver a well-run study with protocol compliance and high quality data.

That is the most important reason why MANA RBM has strongly encouraged, cajoled, and pleaded with technology vendors to provide ad hoc reporting tools with its RBM technology. It’s not enough to get general reports, we need the capability to design reports that look at study-specific protocol risks.

It is also why the MANA RBM model includes more remote trial management. When systems are designed to remotely identify high-risk data items related to protocol compliance or subject safety, we are not dependent on a “single point of failure” (e.g., a poor monitor). Instead, we can have multiple eyes reviewing the data critical to study success.

What is your experience seeing something new for the first time in an area you thought you knew so well? Let me know.