Welcome to Texas justice: You might beat the rap, but you won't beat the ride.

Tuesday, May 06, 2008

Counting the Innocent: A discussion of estmates

As the number of exonerated Texans mount, the media and courts have begun to indulge in speculation as to just how many more actually innocent people might be locked up, both in Texas and elsewhere across the country.

The United States has less than 5% of the planet's population but 25% of the earth's prisoners. When you deal in that kind of unprecedented volume, after a while, mistakes inevitably get made.

The debate was set off officially some time ago when SCOTUS Justice Antonin Scalia cited an estimate by Astoria, Oregon District Attorney Josh Marquis, who calculated in an op ed that the system enjoyed a 99.973 percent accuracy rate. In other words, he estimated the wrong person was convicted only .027% of the time.

See this blog post by Marquis explaining his estimate's methodology, in which he took an academic's running tally of exonerations, multiplied by a factor of ten, then compared it to total felony convictions to get this infinitesimal ratio.

If the error rate were that low, said Scalia, "that rate, he said, is acceptable. 'One cannot have a system of criminal punishment without accepting the possibility that someone will be punished mistakenly,' he wrote. 'That is a truism, not a revelation.'”

But what if the error rate were higher - and it might be - when would we reach an unacceptable threshold of wrongful convictions? Scalia to my knowledge hasn't given us that benchmark, but Marquis has written that if the justice system "is hurling innocents into prison at a rate of 2 or 3 percent I would agree that such a system is dangerously flawed."

This spring in the New York Times, Adam Liptak took Marquis' estimate as a starting point for a discussion of how to estimate the number of innocent people behind bars. He focused on the principal statistical conundrum in this debate, that "there is no obvious control group to measure these exonerations against."

In other words, whatever you decide is the right numerator for the number of innocence cases, it's difficult or impossible to know what denominator to compare it to.

That's why yesterday I felt pretty good about the innocence estimate based on Texas death row inmates, whose final convictions were reversed at about a 1.52% rate since the death penalty in Texas was reinstated. Unlike murders or sexual assaults, capital murder cases constitute a finite enough group to supply reliable denominator, which as Liptak says is the hardest part of making a credible estimate.

Since I was reacting in part to his calculations, I sent that piece to Josh Marquis and he responded suggesting this caveat, which I mostly agree with:

Here's the problem with extrapolating numbers of "innocents" based on such relatively small numbers when doing systems analysis. One can't say that Valujet is a deadly airline because they lost 75 passengers out of 1000 flights the month that plane crashed. We can say how dangerous flying is in America if we take ALL flihts in one yeear and divide that by air crash deaths.

That's true, and it's part of the tradeoff for using a smaller dataset to make the estimate: It makes the denominator easier to identify, but the margin of error for any estimate becomes greater.

That's why I especially appreciated Marquis forwarding me a letter he sent to the New York Times in response to Liptak's article, in which he updated his estimate in light of criticisms of his math, essentially using a more narrowly defined denominator:

In a column by Adam Liptak Professor Samuel Gross criticized the method by which Justice Scalia relied upon in determining that wrongful convictions are exceedingly rare. Gross' study published in 2005 listed just under 400 cases of both rape, murder, and a few other felonies which he claimed were exonerations. Since it was my arithmetic that is under challenge I refined my statistics. Using federal statistics for the relevant time period (1989-2003) of total rapes and non-negligent homicides only (not merely felonies) and still allowing for the assumption that Professor Gross under-reported exonerations by a factor of 10, so that there were in fact 4000, not 400 false convictions, the rate of rightful conviction is still 99.25%.

No-one is claiming the justice system is infallible and it can always use tinkering but is it better for 1000 guilty men go free to spare one innocent man? How many innocent victims are acceptable losses for those who criticize my math?

Joshua MarquisDistrict AttorneyAstoria, OR

The DA's updated estimate that perhaps .75% of people convicted were actually innocent gets us closer to the ballpark of the 1.52% number seen among Texas death row exonerees, especially since, Marquis is correct, both numbers suffer from limits because of their datasets and assumptions.

Applying the .75% figure to Texas' prison population, which is currently around 160,000, give or take, that would mean about 1,200 innocent people are locked up in Texas prisons right now. If my estimate based on death row stats is closer to the truth, double that figure.

Those get to be awfully big numbers pretty quickly, don't they? At what point, I wonder, would Justice Scalia consider the error rate too high?

Marquis is not estimating an "exoneratinon rate," 12:25, he's estimating an "error rate," or a he would put it, an "accuracy rate," and quite reasonably assuming not every error is captured. The 10x factor, though, is certainly the weakest part of his calculation, which is partially why I looked at the Texas DP stats.

I think that 12:59 has a valid point. My recollection of the earlier discussion was that people were willing to plea in order to get out of jail not to go to prison. Although I have heard defense attorneys say their client pleaded guilty because they knew they would be promptly paroled from prison and that was the low cost option.

A 1% error rate for the CJ system is not bad for a non DP case. I would like to see a lower rate for DP cases.

The thing about plea bargains is that they go on your record and eventually you end up in prison because of your record, not because of a crime you actually committed. Actual innocense should be a much greater emphasis of our justice system. Instead, conviction rates rule.

My point - plea bargains are costly on the back end for both individuals and societies stands!

My point - plea bargains are costly on the back end for both individuals and societies stands!

You are mistaken because 95% of the cases plea out. Most of the remainders are dismissed, and only a tiny few proceed to trial.

If there were no plea bargins then what to do with the 19 out of every 20 cases currently disposed of with a guilty plea? Not only would it would require building lots more courthouses and jails, it also means hirinbg tons more judges and prosecutores. Also since policemen would be testifying against everyone they arrested, you're going to massively increase the size of the police force.

The more I think about the idea of getting rid of plea bargins, the worse it seems.

Those DP cases ought to have the lowest numbers, but instead have the highest numbers. In fact it seems to be justice of less than three sigmas (~2.37 sigma). Industry currently tries for 6 sigma in their efforts but the courts can't do even 3 when a life is on the line. What does that say about the American justice system? Three sigma would yield less than 27 reversals per 10K sentences, four less than 64 per 100K, five less than 5 per 10M, and six less than 2 per billion. While the rest of us have been producing higher quality goods our court system has been producing inferior justice. It time we tried for six sigma in our justice system. Judges can't judge well once they become stuck in a particular spot. They begin to know and believe the prosecutors even when they are bringing in lying witnesses or withholding evidence. And lawyers get off with the minimum if anything - see how often one goes to jail if he's caught on a DWI/DUI. The familiarity with the prosecution and the absence of real penalties on lawyers/police/CPS for violating the law is killing the justice system.

I suggest rotating judges between courts and making prosecutors, lawyers, cops, and CPS workers accountable by creating a special court of justice that prosecutes them and allows them to be sued for their acts.

For instance I know of a case where the cop testified that he was waiting to turn left into his parking lot on a dark moonless night for two vehicles to go by. after they went by, he pulled into the lot and heard a vehicle gun his engine and saw the second vehicle pass the first in a no passing zone. Now despite the fact there was a fence, low lying vegetation, signs, a rise in the road, more than a couple of football fields and the fact his vision was extremely poor from having just watched those same vehicles come toward him with their headlights on he claimed he could see the violation. Despite the defendant claiming he had just turned on to the road at a spot well after this point and bringing sworn dispositions testafying that he wasn't on that road at that spot, the judge convicted the defendant. I think that if this case had gone before a jury instead they would never have believed that tale.

Of course innocent people take a plea. As 1st times in the system they are lost and very much afraid. The promise that an example will be made out of those who take their cases to trial has impact. Being told your Judge goes off the deep end adds to the fear. Everyone from the arresting officer forward has determined guilt. A jury of peers is a group of people from a society that is known for locking folks up. A plea can start looking attractive to an innocent person. When the panic wears off it becomes evident it isn't a bargain.If there is a next time it's a whole different story.

I do think that plea bargain's are inappropriately controlled by contract law, which does not account for their coercive nature, but just because I think the process should be reformed doesn't mean I think pleas should be eliminated.

JSN, a 1% error rate is not bad unless you're the lucky one out of 100! The number Scalia thought was okay was 27 times lower than Marquis' updated estimate. I'm not sure even Scalia would think a 1% error rate is acceptable.

I've always thought, FWIW, that many innocents convicted likely wind up on probation, since that's often what happens with weak cases. I agree with the first commenter the rate may be even higher for probation and time-served plea bargains. But frankly, if it were 10% or more, it'd be time to stop blogging and take up arms! That'd be a Kafka-esque, totalitarian nightmare that honestly I don't believe we're even close to right now.

That said, while I don't think the number of wrongful convictions is remotely as high as 10%, neither do I find a 1% rate acceptable.

Thanks, folks, for a good discussion. (REALLY groovin' on the six sigma idea!) best,

That statement sickens me. What are people to him, cattle? I deplore utilitarianism in the application of law. I don't find the conviction of 1 innocent person to be acceptable. How can Scalia, purportedly a practicing Catholic, say such a thing?

"If there were no plea bargins then what to do with the 19 out of every 20 cases currently disposed of with a guilty plea? Not only would it would require building lots more courthouses and jails, it also means hirinbg tons more judges and prosecutores. Also since policemen would be testifying against everyone they arrested, you're going to massively increase the size of the police force."

The fact that more judicial resources might be needed to give real justice is no reason, to me, not to do so. Plea bargains are, in my opinion, a perversion of the system and an encouragement for bearing false witness.

The thing is, I think that the criminal justice system should be reserved for those crimes which actually involve victims. I believe many cases such as criminally negligent homicide or white collar fraud should emphasize restitution for victims (other than through insurance) and not incarceration.

I guess that I am more of a Blackstone/ Old Testament kind of gal than a Benthamite.

I dunno. I think an estimate based on known wrongful convictions in capital cases would be on the low side.

Years ago I researched the California Penal Code on various kinds of theft for a paper I was writing. The thickest section of the Calif Penal Code was capital punishment. And I mean by far.

On the whole it attempted to define the correct application of the death sentence down to the jot and tittle, especially procedures. The feeling I got as a naif reading law was that the powers that be in the state had an awful conscience about it, so got bogged down in details to ensure it was done right if done at all.

I've never checked the Texas Penal Code, but I'd bet without looking at my hole cards that its section on capital punishment is also the thickest part of the code. Maybe not.

Anyhow, of all sentences for all crimes, it is the most carefully scrutinized. So my guess is that the error rate for capital punishment is much lower than other crimes. If you get 1.5 percent, 10 percent generally isn't a stretch.

Plato, there has never been a definitive finding of innocence (e.g., through DNA) AFTER an execution. If DNA evidence were routinely saved historically, though, IMO we'd already have seen a definitive case.

However, there are several cases where most people besides the prosecutor think they were innocent of the crime they were executed for.

There's another recent case in San Antonio, Ruben Cantu, where the sole witness recanted and claimed to have been coerced, and the evidence looks pretty strong he didn't do it.

There are several others over the years where strong actual innocence claims can be made and the only honest conclusion one could reach, at the end of the day, IMO, is "I don't know." Somebody who focuses more on the death penalty like Steve Hall or Scott Cobb might be able to provide more examples.

Let us say there are three type of CJ system errors.1. A guilty person is freed..2. An innocent person is convicted.3. An innocent person pleads guilty.

I doubt the error rate for type 1 is smaller than 10% and I doubt the type 2 error rate is smaller than 1% and I have no idea what the type 3 error rate is (no doubt there are some persons don't think they occur).

What I should have said was if the error rate were as small as 1% it would be a good thing.

FBIwitchHunt.com was created in response to the USgovernment's increasing use of Grand Juries, and other tools of repression, against activists. The government reliesheavily on misinformation and secrecy in its efforts to crushvarious social and political movements. It is our goal toprovide a forum for activists across the country, from allmovements, to relay information about their ongoingstruggles against government repression, and to providesome insight into how to combat this repression.

Green Scarehttp://www.greenscare.org/Excerpt:"Operation Backfire" is the name given by the Federal Bureau of Investigation (FBI) to the ten year old investigation and indictment of fifteen individuals accused of several arsons in the Pacific Northwest between 1996 and 2001 claimed by the Earth Liberation Front (ELF) or the Animal Liberation Front (ALF). It is because of these arrests and the fervor that federal agencies are seeking to impose severe sentences under the statutes of "anti-terrorism" that the term, "Green Scare" has resurfaced and is being used to describe the draconian measures taken against environmental activists in the same way that alleged communists were targeted in the McCarthy-era Red Scare.

These indictments have become the subject of much debate since they were based primarily on the testimony of a heroin addicted self-professed serial arsonist, whose name apparently came up in a grand jury inquisition of a bitter ex-girlfriend.

The savings and loan fraud – which former Attorney General Dick Thornburgh called "the biggest white collar swindle in history" – cost us anywhere from $300 billion to $500 billion.

And then you have your lesser frauds: auto repair fraud, $40 billion a year, securities fraud, $15 billion a year – and on down the list.

Number 19

Corporate crime is often violent crime.

Recite this list of corporate frauds and people will immediately say to you: but you can't compare street crime and corporate crime – corporate crime is not violent crime.

Not true.

Corporate crime is often violent crime.

The FBI estimates that, 16,000 Americans are murdered every year.

Compare this to the 56,000 Americans who die every year on the job or from occupational diseases such as black lung and asbestosis and the tens of thousands of other Americans who fall victim to the silent violence of pollution, contaminated foods, hazardous consumer products, and hospital malpractice.

These deaths are often the result of criminal recklessness. Yet, they are rarely prosecuted as homicides or as criminal violations of federal laws.

1: Trade Secrets,A web page from Bill Moyers Public Broadcasting show, national television.Documentation of over 50 years of corporate cover up, costing an unknownnumber of American lives, with out criminal presecution.http://www.pbs.org/tradesecrets/index.html

2: PDF Skeletons in the Courthouse:Hazards to the Public Remain Secret.April 2002 By Coalition for Consumer Rights,A copy of the "censored" PDF report is re-posted athttp://www.angelfire.com/nm/redcollarcrime/courtsecrets.pdfI had download the report and made back ups . So I and re-posted it.Their web page is gone from the net and the times that I havetried to call the authors (lately) their phone is "being checked for trouble".

3:W.R. Grace to Pay $250 Million for Libby Cleanup,by Liz BorkowskiA 10-count criminal indictment against sevensenior current and former Grace officials allegingconspiracy. For knowing endangerment, obstructionof justice and wire fraud for endangering thepeople of Libby by concealing the well-documentedhazards of the tremolite asbestos.http://thepumphandle.wordpress.com/2008/03/12/wr-grace-to-pay-250-million-for-libby-cleanup

In reference to the above story, the criminalcharges are "pending". National support is neededto see that these criminal charges against theindividuals responsible are in fact heard in a courtof criminal (not civil) court.

The charges have been brought byWilliam W. MercerUnited States Attorneyfor the District of MontanaRegards,

With some of the Judges who occupy seats in District Courts now, it is no wonder people accept a plea deal. Some are so mean and inconsiderate of the lives they are ruinging no wonder there are so many plea deals. The could be the lesser of the two evils, face a crooked DA and Judge and you will lose and if you elect to take a plea deal, you lose the right to appeal.

The CCA will not do the right thing and read both Writs submitted by the Defense attorney and the State, if the Judge signs with the State, they automatically rule against an appeal.

Now is the time to start cleaning out the court system and get rid of the Judges in the District Courts and the Judges who sit on the CCAs. Vote against all incumbents and clean out part of the mess we have in our court system.

A classic "But First Moment";Before the judges can be voted out, campaign finance laws will most probably need to change.

Example in this news article;

Justice for Sale - JS Online: Poll says money, courts atroubling mix... court, and 26% of judges agree, according to the survey, conducted for Justice at Stake, a national non-partisan organization concerned with judicial issues. ... By DAVID DOEGE, of the Journal Sentinel staff, Last Updated: Feb. 17, 2002http://www.jsonline.com/news/metro/feb02/21148.asp?source=tmj4

"I always tell people interested in these issues that your blog is the most important news source, and have had high-ranking corrections officials tell me they read it regularly."

- Scott Medlock, Texas Civil Rights Project

"a helluva blog"

- Solomon Moore, NY Times criminal justice correspondent

"Congrats on building one of the most read and important blogs on a specific policy area that I've ever seen"

- Donald Lee, Texas Conference of Urban Counties

GFB "is a fact-packed, trustworthy reporter of the weirdness that makes up corrections and criminal law in the Lone Star State" and has "shown more naked emperors than Hans Christian Andersen ever did."

-Attorney Bob Mabry, Conroe

"Grits really shows the potential of a single-state focused criminal law blog"

- Corey Yung, Sex Crimes Blog

"I regard Grits for Breakfast as one of the most welcome and helpful vehicles we elected officials have for understanding the problems and their solutions."

Tommy Adkisson,Bexar County Commissioner

"dude really has a pragmatic approach to crime fighting, almost like he’s some kind of statistics superhero"

- Rob Patterson, The Austin Post"Scott Henson's 'Grits for Breakfast' is one of the most insightful blogs on criminal justice issues in Texas."

- Texas Public Policy Foundation

"Nobody does it better or works harder getting it right"

David Jennings, aka "Big Jolly"

"I appreciate the fact that you obviously try to see both sides of an issue, regardless of which side you end up supporting."

Kim Vickers,Texas Commission on Law Enforcement Officer Standards and EducationGrits for Breakfast "has probably broken more criminal justice stories than any TX reporter, but stays under the radar. Fascinating guy."

Maurice Chammah,The Marshall Project"unrestrained and uneducated"

John Bradley,Former Williamson County District Attorney, now former Attorney General of Palau

"our favorite blog"

- Texas District and County Attorneys Association Twitter feed"Scott Henson ... writes his terrific blog Grits for Breakfast from an outhouse in Texas."