Understanding CSA and IRT: A Fleet Manager’s Guide

Building an effective safety culture not only reduces the number of safety events incurred by drivers but also incentivizes them to do their best and empowers others within the fleet to do the same. A pivotal element to judging and improving your safety program is understanding driver performance. Many fleets have relied on the Federal Motor Carrier Safety Administration’s (FMCSA) CSA Program to do so, but have found their results to be difficult to gain value from.

Problems with Current CSA Scoring

In June 2017, the National Academies of Sciences, Engineering, and Medicine (NAS) issued a report that assessed the CSA program and the metrics used to evaluate the safety of truck drivers. As predicted by safety managers, NAS’s results proved that these scores were not always accurate in depicting driver performance as intended and failed to be the predictive indicator it set out to be.

Why exactly is the CSA failing? There are many shortcomings, but one of the most substantial deficits in this scoring system is the failure to recognize that safe drivers and the fleet’s safety culture as a whole take place most often outside of the cab. In addition, current CSA scoring will negatively impact drivers for unavoidable incidents, affecting drivers who aren’t necessarily unsafe. The CSA scoring also doesn’t take into account the enforcement disparities in different regions or the lack of inspection data collected by smaller fleets.

Non-preventable accidents affect CSA scores, even though a driver’s performance usually isn’t the cause. For example, if a truck is legally parked and a passenger vehicle crashes into the truck causing significant damage to both vehicles, the truck driver’s CSA score will suffer, and their reputation will tarnish although there was nothing they could have done to prevent this accident.

A report issued by The American Transportation Research Institute (ATRI) investigated how non-preventable accidents affect a carrier’s overall score. It concluded from over a dozen fleets in ATRI’s analysis that CSA’s Behavior Analysis and Safety Improvement Category (BASIC) dropped nearly 15% once it cut out the “non-preventable crash” data subset.

Disparate State EnforcementGeography can sway CSA Scores

Safety challenges vary from state to state, which can pose issues for some drivers more than others based on their geography alone. The CSA doesn’t acknowledge targeted enforcement zones, so carriers who operate in those zones typically face poorer scores than those who drive elsewhere, resulting in an inability to accurately compare both drivers and fleets. This degrades the effectiveness of the overall program and score.

Amount of Scored CarriersLess than 20% of carriers have enough data for CSA Scoring

CSA today can’t score a carrier with insufficient inspection data, meaning that approximately only 100,000 carriers out of over 500,000 fleets in the US have enough data to receive a CSA Score. Each sector of the BASIC score has a minimum number of safety events (inspections, violations, and crashes) to be eligible, so if a fleet falls below this threshold, they can’t receive a score.

How can a fleet that isn’t collecting several of the required safety events begin to understand driver performance? Some fleets have turned to driver management systems like the Idelic Safety Suite to achieve a true understanding of driver behavior and performance.

What is the IRT Model?

The IRT comprises several mathematical models that drill deeper than the previous CSA methodology by assigning weights and looking for patterns within data. CSA Scores use a “classical test theory” where cumulative scores matter the most, but the IRT model instead focuses on individual items and questions. By analyzing each element, you can determine if a driver’s incident should harm their score and also can assess the event’s significance. This method should more easily identify which events are a result of poor or unsafe driving and help fleets determine which shortcomings your drivers might have.

Although the success of this model is not yet proven in trucking, the IRT model has been successful in various other fields, such as education. A good example of where this is seen in use today is the SAT Exam, where answering pertinent or difficult questions correctly weighs more, which measures students’ success rather than the sheer amount of correct responses.

Will FMCSA Successfully Use the IRT Model in Trucking?

The FMCSA is still developing the IRT Model, and there is no implementation date in effect, but it is anticipated to be tested in September 2019 and looking to be implemented by the middle of 2020. There is also no timeline on when the IRT Model will begin to impact carriers, and some say these changes can come with a lot of unexpected pitfalls and pivots in direction, so the future of IRT remains mostly unknown.

No matter the timeline, FMCSA plans to improve the CSA model by identifying “cultural weaknesses” with the IRT Model. The IRT Model is intended to end the “crash risk” approach by focusing on BASIC category scores in addition to an overall “Safety Culture Score” which incentivizes carriers to improve Safety Programs and driver behavior. Intermittent incidents amongst otherwise safe driving will not impact a driver’s score as much as one with poor scores all around.

FMCSA’s Need for Data

With the IRT model, the FMCSA plans to improve data collection and determine the possibility of adding new data sets, including carrier data such as driver turnover rates, types of cargo hauled, and compensation levels. Using new data sets to better assess driver performance is imperative to a robust model, but it remains uncertain if the FMCSA is capable of gaining access to this fleet data.

Utilizing more data points allows for a more scientific approach to calculating CSA Scores. The hope is for the IRT model to adapt to fluctuating patterns to quickly ingest more data as the model develops. In an industry where technology is rapidly added inside and outside the cab, this new approach might not be able to adapt to the unforeseen future.

Integrating fleets’ mass quantities of data into a form that is digestible and reportable is essential to get the most out of your driver information. The Idelic Safety Suite gathers integrated data into a transparent platform, which unlocks the power of the data that fleets already hold.

How Your Fleet Can Start Right Now

Using the Idelic Safety Suite, safety managers can integrate all their data from third-party systems and gain insight into driver behavior that was never before possible. Unlike the CSA or IRT, which is a static score, the Safety Suite provides a 360-degree understanding of each driver’s behavior. This driver management platform utilizes machine learning predictive analytics allowing fleets to access a holistic view of the safety culture within their fleet and use that newfound understanding to prevent accidents, reduce driver turnover, and lower insurance.

Whether or not the IRT model will revolutionize the industry of trucking like the FMCSA and its advocates’ hope is still uncertain. But, comprehensive driver management and predictive analytics software such as the Idelic Safety Suite grants safety managers the ability to evaluate their drivers on a level that no other system or model can, granting the ability to make data-driven decisions across an entire fleet.