The problem with experts who don't know what they don't know

I recently finished a hearing, which focused on "expert" testimony that had been presented at trial. As with many cases I've handled, the individual was testifying about something they had never done before; in this case, it was a height analysis. When most people hear that, they are surprised, or don't believe me. That's because it's not something you would ever do in real life. Can you imagine going to a dentist who just got out of school and had never worked on someones' mouth? Or a doctor who didn't do an internship in medical school? If you've been following this blog, you know the case I'm talking about is George Powell's case, who was convicted of aggravated robbery despite the fact that he is 6'3", and all the witnesses described the robber as 5'6" to 5'8".

The people who have the hardest time believing this is civil lawyers, who use experts all the time. In a civil case, both sides go out and choose the best expert they can find or afford. Not only do they want the best person, but they also want to make sure they will be allowed to testify, and that the other side won't be able to destroy them through cross-examination. Far too often, the exact opposite happens in a criminal case. The State doesn't go looking for the "best" expert; most of the time they spend very little time vetting potential experts. Instead, they want to know whether the person is willing to help. Many times, the less qualified the individual is, the better off they are. That person will want to help. In contrast, if they go to someone with a lot of experience, they may tell them they can't do what they want. You might find this hard to believe, but I've seen it time and time again. How else do you find someone with no experience, and who hasn't been trained on the software program he is going to use - and more importantly, are willing to go before a jury and claim they are an expert?

The use of forensics in criminal cases has been repeatedly questioned in the last several years; unfortunately, those criticisms have not filtered down to the trial court level. I've spent a lot of time thinking about how to fix this problem. The simple answer is that individuals should refuse to accept cases where they are going to have to "learn on the job." However, that's contrary to human nature. Most people are flattered when someone reaches out for help, especially if they are wanting to get into a new field. They are reluctant to say no - especially to the government.

I think there's a more significant factor though, which I've only recently recognized. The person doesn't know what they don't know. They don't look at themselves as unqualified, but truly believe they can do what they are being asked to do. The best example is the one involving the height estimate; the State's expert believed he was qualified to do that because he believed measuring height was no different than measuring skid marks on the highway. If he had done some limited research, he would find that most experts claim there is a significant difference between measuring height, and measuring other things, and you need a substantial amount of experience before doing it. The FBI has what is probably the leading forensic video lab in the world, and they would never in a million years allow someone whose only experience was in crime scene reconstruction to undertake a height analysis. Of course, you don't know that if you aren't familiar with the field.

So, how do you fix this? The only way is to make sure lawyers and judges are better educated on forensics. More importantly, judges need to be not only better educated but also more involved in the qualification of experts. They are supposed to be the "gatekeepers", and they have done a horrible job of that in criminal cases. Experts with minimal qualifications are routinely allowed to testify in criminal cases - especially when offered by the State. As long as the expert has more knowledge and experience than the average juror, they are probably going to be allowed to testify. Just because you know more than the jurors, though, doesn't mean you know enough to be considered an expert.

Defense lawyers also share a lot of blame for this. Unfortunately, they are often like experts, in that they don't know what they don't know. They are the check on the use of qualified experts though; they must challenge them, and use their own qualified experts. If they don't do that, you can't fault the judges for allowing the testimony.

Finally, the State has a huge part in this. They need to seek out qualified experts, and not simply experts who are willing to help. Just because you can find an expert to testify, doesn't mean you should.

My hope is that by continuing to point out the cases where the system has failed, the better the system will work in the future.