CSA is ‘unreliable’ and here’s why – and what to tell customers, says ATA

CSA'S DATA TRAIL

The article below is part of an ongoing, in-depth series on the U.S. Department of Transportation's Compliance, Safety, Accountability program that analyzes federal inspection, investigation and crash data and offers original reporting. Overdrive and CCJ editors have built a site dedicated to hosting the stories, interactive maps and downloadable data at CCJdigital.com/csa.

Trucking continues to question the accuracy and application of data at the heart of the federal government’s regulatory program, and a powerful political voice for the industry has issued a white paper that spells out the points of contention in a way that customers – and maybe even Congress – can understand.

At issue is the statistical correlation between government-assigned trucking company safety scores, based on industry-wide trends, and whether the math holds true for individual carriers.

The American Trucking Associations on Monday issued a white paper on the reliability of scores from the Federal Motor Carrier Safety Administration’s Compliance, Safety, Accountability program in evaluating the safety of individual trucking companies.

“ATA continues to support the objectives of CSA and to call for improvements to the program,” says ATA President and CEO Bill Graves. “However, data and methodology problems continue to plague the system and the accuracy and reliability of companies’ scores.”

More simply, as an ATA representative told CCJ, the organization expects member carriers to share the report with third parties – such as shippers and insurers – who might be using CSA scores to evaluate trucking company safety performance.

In its paper, ATA examined data and research on the connection between CSA scores and crash risk, as well as how problems with the data and methodology produce “an imperfect and unreliable measure” of a carrier’s safety record.

Researchers have arrived at “mixed conclusions” with respect to the reliability of Safety Management System scores in identifying unsafe, or crash-prone, motor carriers. Some found virtually no correlation between scores and crash rates in any of the measurement categories, the report states.

Most significantly, ATA suggests that the high number of “exceptions,” or carriers that don’t fit the statistical model, “leads to the conclusion that SMS scores alone … are, at a minimum, unreliable.”

The American Transportation Research Institute (ATRI), “using a better prediction model,” found a positive relationship between scores and crash risk in three of the publicly available measurement categories (BASICs) but also found that scores in two others bear an inverse relationship to crash risk.

ATRI, the research arm of ATA, also pointed out that the number of alerts that a carrier has been assigned is a strong indicator of crash risk. However, the strength of the relationship varies depending on the BASICs in which the carrier has alerts, since scores in some BASICs more strongly correlate with crash risk than those in others, according to the paper.

“It may make sense for FMCSA to use scores in those categories that correlate positively with crash risk to prioritize companies for enforcement review,” Graves says. “In the process, FMCSA can verify whether or not the scores paint an accurate picture. But third parties need to know that for the purposes of drawing conclusions about individual carriers, the scores are unreliable.”

ATA continues to point to a number of “data and methodology problems that plague the system”:

– A substantial lack of data, particularly on small carriers who comprise the bulk of the industry, hinders the system’s ability to render meaningful scores of comparative performance.

– Regional enforcement disparities likely cause fleets of all sizes operating in jurisdictions with robust enforcement practices to be perceived as less safe than those operating in other regions.

– The questionable assignment of severity weights to individual violations can skew carriers’ scores; and

– The underreporting of crashes by states, the use of crashes that were not caused by motor carriers, and the increased exposure to crashes experienced by carriers operating in urban environments, all affect the significance of Crash Indicator BASIC scores.

U can have everything working n they can still find a reason to hassel you if they want !!!??? I avoid the scales n DOT like the plague because I have no control over what they do to me at the scalehouse n checkpoints ?? I (just like any other driver) want everything to work and be shiny clean n look good be safe ect ect but some-times no matter how much u do some-thing will quit working while u r moveing down the road (turn light stop light ect ect ) there is no way that u will know this till u stop the next time n check !! But yet they will hang u for it at the scalehouse ect ect ?? HOW R WE SUPPOST TO KNOW WHEN IT WAS WORKING 2 HOURS AGO ?? Then there is this issue of paying all these fuel taxes n road use taxes ect ect only to have this gov issued roadway beat my equipment to death n shake it till breaks wires lights cracks wheels ect ect then they herd me into the scalehouse n ticket n fine n give me points on my score ect ect for things that ""THEIR ROAD"" did to my equipment !!!!!!!!!!!!!

I have a big problem with ATA now coming out and saying they do not like the way FMCSA is grading trucking companies. ATA was a supporter of this program which has hurt all companies except the big four which run the ATA. The last time I took notice of ATA it was the "AMERICAN TRUCKING ASSOCIATION" and a lot of midsize carriers were hurt by this ruling, and we are making do in spite of the ruling. Where was ATA when the rules were being passed.

Kevin Jones is Senior Editor, Trucking Media, and writes from his home in Little Rock, Ark. His Fleet Street blog features whatever strikes his fancy and has at least a little connection to trucks, or drivers, or highways. Or David Allan Coe. (Google "the perfect country and western song" if you're not nearly as old as Kevin is.) You can also keep up with Kevin by following his Twitter feed (@KevinJonesCCJ) or just drop him a line: kevin.jones@randallreilly.com.