Houston Forensic Science Center Slides Blind Testing Into Workload

Two personnel at the Houston Forensic Science Center’s quality division create “blind tests” for the latent print section of the crime lab. It appears that the extent of HFSC’s use of the “blinds” program—which tests workers while also verifying scientific reproducibility—is unique in forensic science. (Credit: Ramit Plushnick-Masti, HFSC)

Can sliding in blind samples amid the workload at a forensic laboratory test how well the work is going? Can it also help raise standards, by keeping analysts on their toes?

The Houston Forensic Science Center says yes. The agency is ramping up its “blinds testing” program amid most of its disciplines this year. The seized drugs, firearms, and other test-case evidence that makes its way through to the analysts are proving that the forensic work is reproducible and accurate, the lab’s leadership told Forensic Magazine.

The analysts also know they’re being watched—and the pressure is on to always perform, according to the HFSC brass.

“Can we use blinds as a way of altering the behavior of the system in the direction that we want to?” said Peter Stout, the HFSC’s chief executive officer. “It’s an interesting psychosocial effect.”

The “blinds program” has sent 329 samples through the normal evidentiary process. That includes firearms, blood samples tainted with alcohol and drugs, DNA and fingerprint samples, as well as digital forensic items, they write.

Among hundreds of test samples, no errors have yet been reported. At the same time, the watchful forensic analysts have caught some of the test cases, due to exacting observations of the deceptive “evidence.”

Creating convincing tests samples is one of the challenges. Human blood with just the right amount of alcohol—near the standard threshold of 0.08 percent—is provided through a deal with the North Carolina-based nonprofit RTI International, according to HFSC leadership. But the HFSC’s quality assurance team has to fashion many of the other complicated examples. That includes making trips to Walmart to buy BB guns, underwear, bandannas, crowbars and other items—even amid suspicious looks from cashiers, officials said.

If enough blind tests are done successfully, Stout says, the laboratory may begin to claim error rates with confidence intervals—bolstering its forensic analyses.

In a twist, the analysts called out approximately a dozen of the blind samples coming through their workflow, for a variety of reasons. The main “tell” so far has been in the routine paperwork. Too often, the spelling is perfect and the handwriting is neat—not the hallmarks of notes that generally come from a crime scene, according to Stout. But other giveaways have been caught by the sharp eyes of the most experienced analysts. One time a clear document could not have come from a defective printer normally used the Houston Police Department’s Narcotics Section. Still another time, a drug expert suspected the true nature of the controlled substances, since one narcotic sample came from a part of town not normally associated with the drug flow there. In yet another instance, one of the quality division members gripped a crowbar in a way a true perpetrator would never have—a detail that was evidence to the trained eye of one of the scientists.

For each catch, Stout doles out a Starbucks gift card for the observant forensic workers.

“If they spot a blind, and it is a blind, I’ve got a Starbucks card for them,” said Stout. “But if they spot one that’s not a blind, they owe me a buck.”

Aside from the dozen or so gift cards, he is up two dollars.

But with each blind, they are attempting to make better examples, so that the forensic scientists simply process it as they would with any other sample.

The blinds program is expected to grow in 2018. They are hoping to target as much as 5 percent of the entire workload—approximately 800 blind tests this year alone. That increase would come with planned spikes in testing among toxicology and controlled substances, and latent prints and biology (DNA) sections.

However, some of those goals may be complicated by the forensic databases themselves. For instance, there are privacy and other concerns about using actual fingerprints, firearms and DNA samples in AFIS, NIBIN and CODIS, respectively. One solution, as far as the genetic samples go, is to keep the blind samples at the local level, and avoid putting them into the national (and searchable) databases.

“There are some real sensitivities; there are some real concerns,” said Stout.

The analysts processing the blinds appear to have accepted the testing in stride, Stout added. Instead of perceiving it as some intrusion into their workflow, it has instead only strengthened their testimony. This is especially true in toxicology cases, where the workers are often challenged in court, he added.

“It gives them the ability to answer (attorneys’) questions appropriately,” said Stout. “They can say they never know whether they’re being tested.”

The National Laboratory Certification Program has had a requirement for “blinds” in drug testing for decades. The idea for instituting blinds in Houston came from Stout’s experience in the military system, which has also been employing such quality control for approximately 30 years.

Some other laboratories have tried to use blind testing to determine accuracy - but to include a fake case in the forensic workpile is "unbelievably difficult," according to Ray Wickenheiser, the president of the American Society of Crime Laboratory Directors.

"Can you really fake a blind test in all its complexity?" said Wickenheiser.

Wickenheiser himself is also director of the New York State Police Crime Laboratory System. He told Forensic Magazine that New York and some other laboratory instead use old cases which they re-test to determine whether the same results are consistent. Houston's model is a "wonderful experiment," and Wickenheiser said he hopes their findings could improve testing for other laboratories, once the results are shared in the community. For now, the cost of running such "blinds" in most labs is prohibitve, he said.

"In an ideal world, all crime labs would be able to do this," said Wickenheiser.

It appears Houston’s use of blinds across disciplines is unique. Forensic Magazine is interested in hearing from other laboratories which may be running their own “blinds program,” at seth.augenstein@advantagemedia.com.