In 2012, it was discovered that a chemist working at the Massachusetts state drug lab in Jamaica Plain had been falsifying drug tests (e.g., claiming that samples contained narcotics without testing them and even adding cocaine to samples to get a positive result when prior testing came back negative). She had worked at the lab for nearly a decade, and these revelations called into question the outcomes in tens of thousands of cases.

Obviously, this sort of tampering means there are convictions waiting to be overturned. But two years later, little progress has been made. It isn't that the state is obstructing efforts to make the falsely-convicted whole again (there may be some of that, but Colarusso's post doesn't indicate there is), but that nearly a decade's-worth of bogus lab work potentially infects thousands of convictions. Narrowing down this list to those directly affected is an enormous task, one that Colarusso was tasked with making more manageable. Narrowing down "The List" to a single link in the evidence chain -- the drug receipt -- still returned far too many potential matches to be of use. Additional restrictions trimmed the possible matches a bit more, but still left far too many potential victims of the chemist's work.

Staff attorneys take only a small fraction of indigent cases. The majority are handled by private attorneys. So only a subset of defendants on The List would be in our client files. However, given The List contained nearly 40,000 names, this subset was still rather sizable. So we used some nice open source software to look for matches between our clients’ names and those found on the list (this involved some data wrangling in Python and Pandas along with the creation of a nice IPython Notebook or two). This gave us a rough list of clients on The List, and we used these names to create a list of their co-defendants.

The narrowing of the field only did so much. The List remained sizable, thanks to inconsistencies inherent in the system itself.

This gave us a rough list of clients on The List, and we used these names to create a list of their co-defendants. We then checked The List for the co-defendant names. Unfortunately, a lot of these were missing. If we assumed the same rate of missing names across all cases, it seemed The List was missing somewhere between 0 and 9,600 names. Wait, what? That’s right, thousands of potentially missing names. The uncertainty came from the fact that we had to match names. The List did not come with dates of birth, addresses, or Social Security Numbers—just names. So occasionally, we could not find a name we were looking for because the Commonwealth and CPCS disagreed on the spelling of a name or someone made a transcription error.

By this point in the investigation, the master list was still huge and it was obvious the list itself was missing hundreds of names, which meant hundreds of possibly wrongly-convicted citizens. The reality of this situation was this: to track down those missing names and to finally set the wheels of justice in motion, thousands of police reports would need to be read and cross-referenced against those on the master list. But who will do this? And with what funds? That's still unclear. It all depends on who feels justice should be served and who feels justice should be served, but only up to a certain dollar amount.

So, the problem -- which was one person in one lab falsifying thousands of test results -- has become something so unwieldy that it may never result in the exoneration of everyone chemist Annie Dookhan managed to wrongly put behind bars. The problem is too big to solve, and much of that has to do with the efficiency of drug prosecutions versus the much less efficient wheels of bureaucracy. Data wrangling helped determine the size of the problem and point a way towards a solution, but the solution is still hundreds, if not thousands, of hours away.

But Colarusso points to one aspect that should have been noticed and would have kept this from becoming a 40,000-file catastrophe (and that's without counting the undetermined number of omissions).

The report shows that the Hinton lab leaned heavily on Dookhan’s productivity. Supervisors lauded her work ethic and assigned her an increasing share of tests.

“From January 1, 2004, through December 31, 2011, Dookhan was assigned 25.3% of all analyses in the Drug Lab and completed 21.8% of all tests conducted by staff,” the report said.

There's a point when a problem is still manageable and there's a point when it becomes too big to correct within the confines of the system that helped create it. The drug lab itself pushed it from solvable to impossible. What happened here isn't an isolated incident. Similar abuses have occurred at the nation's top labs -- those belonging to the FBI. Results of forensic testing were overstated to prosecutors, who then put these experts on the stand to help convict hundreds of thousands of people using the questionable results. It's only because of outside pressure that the FBI and DOJ are even looking into this, and it's easy to imagine "The List" of possibly wrongly-convicted persons here far exceeds the 40,000+ Colarusso tangled with.

In addition to the problem begin ignored for two decades, the effort to right the FBI's wrong isn't being pursued with nearly the same enthusiasm as the bogus prosecutions. And yet, there were early warning signs that were ignored. Information about the FBI's evidence issues was passed on to prosecutors -- but this information, including exculpatory evidence, was never passed on to defense attorneys.

The time to handle a problem is before it negatively affects thousands by depriving them of their liberty. But the desire to fight a drug war led to the cover up of exculpatory evidence by US prosecutors working with the FBI and the opportunity to fake even more test results in Annie Dookhan's case. Caught in the middle of all of this are thousands who may never see justice done because the problem was ignored until it no longer could be.

Re: Re:

Really? You can be detained at any point in the borders or airport because reasons (maybe you look too latino, eh?), your belongings can be seized without any warrant or accusation and at any point you may be subject of multiple cavity exams because reasons (or you look to real, historical Jesus). Is it really being childish?

Re: Re:

...and if you don't do drugs, you're still rolling the dice on the possibility that some GED reject with a badge and gun will decide you're "suspicious" and toss you in a cell to "preserve the element of surprise." Until they can figure out what they think you did, and then convict you for it on faulty forensic evidence.

But, at least it's not like someone might just outright fake up that evidence just to get a pat on the head at work... oh, wait.

Re: Re: Re: never set foot in the states

Re: There are LOTS of reasons to never set foot in the States

For one, a huge number of food-service workers don't have low out-of-pocket health insurance (or any health coverage at all), and they don't have paid sick leave, so they try to hide their symptoms and come in to work sick. (A lot of them don't have enough of a financial cushion to be able to take unpaid days off and still make next month's rent.) Bon appétit!

For another, if you come to the US as a visitor and don't have first-class emergency-medical and medevac insurance, you risk being financially ruined if you have a serious accident or fall seriously ill. American medical bills are nothing short of obscene. (Just ask the parents of Canadian skier Sarah Burke.) Hopefully, the worst symptoms of any illness you contract at an American restaurant won't kick in until you're back home.

And you don't want to travel with very much cash -- not because of "bad guys," but because of cops. A lot of American law enforcement agencies will simply take it from you on the theory that it's inherently suspicious, and you will have to hire a lawyer, sue them, and prove that you aren't a drug trafficker in order to get it back. And no, you won't recover your attorney fees.

I don't see how this is unmanageable

It shouldn't be all that hard to determine if this technician processed the samples from a case. If yes, then the prisoner goes free - and hopefully, the prosecutor gets it deducted from his score card. Easy. OR, are they really saying we should accept innocent people being jailed for decades, at taxpayer expense, because it's too hard to let them free?

Re: I don't see how this is unmanageable

That's the whole problem. They don't really have a simple way to create a list of every prisoner convicted on the basis of this technician's work.Some, but not all, defendants are named with the sample tested, many names were left off. The only way to create the complete list is to have a person go to the paper court records scattered all over the place, and read them all.Given the reasons why this is being done, it probably should also involve a second person doing an audit of the first person's work.

ROFL, They built a prison for you and me now stfu and get in your cell!! Really is a gross tragedy, "law enforcement" is nothing but incentive to harm your fellow man. This history is boring and the repeating of mentioned history is fucking tired.

It is NOT too big to solve

We are talking about goddamn human lives unjustly put behind bars here. It does not matter how much it will cost. It is a moral and social obligation of the Government to fix it. If anything, all the convicted should get a lawyer and go question their results in the justice system. This would generate a much bigger problem.

Also, said chemist should be jailed for life. What kind of rotten, sick sociopath deliberately screws people's lives by adding positives where there were negatives? And the lab itself should be prosecuted and closed down and its management along with the power chain that allowed her to work without auditing put behind bars as well.

Re: It is NOT too big to solve

I constant trend in the US and UK is institutional meanness.

If something good is to be handed out by the state (welfare benefits, healthcare, education or relief from wrongful imprisonment) then every possible financial stringency is invoked to avoid doing it on grounds of cost - but if something bad is to be done (arrest, trial, imprisonment etc) then NO EXPENSE WILL BE SPARED.

Re: It is NOT too big to solve

We are talking about goddamn human lives unjustly put behind bars here. It does not matter how much it will cost.

I would certainly agree with that, but I don't know if it's a majority position. I suspect many people would not want a lot of money spent on such a problem, because "most of them are probably guilty of something anyway". I know, it's disgusting. I'm with you though - there should be no consideration of whether to seek justice for the wrongfully convicted based on how much it will cost.

failure to test

I've been asking for years why we don't a control policy at these police-affiliated, District-Attorney-influenced labs:

"You will be analyzing specimens taken from crime scenes and from suspects in custody, but occasionally some of them will actually be control samples prepared by the Public Defender's office. If you return one false positive, your career is over and the Public Defender's office gets your pension."

Re: failure to test

It's pretty unreasonable to expect an analyst to never return a false positive. People are human, even when the stakes are incredibly high.

What really needs to be done (and is generally required for labs to get accreditation) is the expansion of quality control resources in labs. Because one cannot reasonable expect a person to be perfect, put more eyes on it. Some labs I've been to have two people in each examination: one running it and one observing the running to make sure it's done properly. Then it's run again by a different pair. Then the results are reviewed by quality control. It isn't perfect, but it's better than only one person seeing it.

Call it 'incentive'

It's pretty unreasonable to expect an analyst to never return a false positive. People are human, even when the stakes are incredibly high.

When you consider a false positive has the chance to ruin someone for life, no, I don't really consider it unreasonable. The stakes are high, and they're only human as you say, but when screwing up can get someone tossed behind bars for years, decades or longer, 100% accuracy is not only desired, it should be required.

Either that or a positive result should be drastically decreased in evidentury value, such that even a positive result doesn't ensure a conviction, but is only considered a minor bit of supporting evidence, rather than grounds for instant conviction.

Re: Call it 'incentive'

I didn't say that I don't think 100% (or as close to it as possible) is not required. I completely agree with that and, as an examiner, the fear that my work may harm an innocent person is terrifying.

What I said was that it's unreasonable to expect that of any one person. The technician in this instance was a problem, a huge problem, but the lack of a proper quality control was a bigger one. I believe the lab is as much responsible for this as Annie Dookhan. You can't expect someone to be perfect, so you build a way of determining whether they functioned correctly or not. This lab failed at that on a criminal level.

We can push for Annie Dookhan being punished (and she should be, severely) and we can call for the results to be overturned (which they should be), but by not addressing that the laboratory itself is what truly failed, we leave the door open for the next person to do this. As long as we blame the technician and don't use the opportunity to demand the labs themselves to operate in a transparent, controlled, and unbiased way from the very beginning, nothing will change.

As a side note, I also agree that positive results should be less valuable. There is far too much reliance on psuedo-science and incorrect understanding of the statistical results of an examination.

Re: Re: Call it 'incentive'

Huh. I had never read Annie Dookhan's actual sentencing. Adjust my bit about how she should be punished severely and alter it to outrage about it only being 3-5 years and the lack of real punishment for the people who mismanaged the lab and allowed this to happen.

Re: Re: failure to test

Bah, jumped the gun a bit in my comment.

The second half of your comment would certainly help, but even then, I think that a positive result should be drastically reduced in value as evidence, simply because while desired, 100% accuracy is not possible, so the possibility of a false positive should always be taken into consideration. Treat a positive or negative result as a piece of evidence, but not a core piece, such that it's not enough to win a case on it's own without significant supporting evidence.

Re: Re: Re: failure to test

This we can definitely both agree on...and I did when I also jumped the gun and didn't read this response on yours.

There's far too much weight put on this type of evidence and on the testimony of people who run them. I firmly believe that these sorts of examinations should be used exclusively (or at least almost exclusively) for investigative purposes. There should always be more than the words of an expert witness, especially when it is very difficult for courts to determine the true expert-ness of those witnesses. There are systems in place to try, but it takes a lot of effort on all sides (judge included) to determine the true nature of the expert's knowledge. I, unfortunately, have some stories about examiners that are well respected by everyone except the scientific part of the community.

Re: Reminds me of our multiple detection dog programs

I'm pretty sure that without a major reform, we're not going to see redundant tests, given that the prosecutors are going to prefer labs that yield more positive results. Even ones that are false.

This is why seeing those major reformations occur is going to be so difficult. There's really no political incentive to reforming it. If you make it harder to get the positive results (true or false), you look soft on crime. Easier to just throw the tech in prison for a bit, let the lab head resign, and keep going about doing things in a dangerous way (for everyone else.)

A couple years ago, there was a call by NIST to strengthen the scientific ethics of forensic laboratories and better their practices, meaning: training personnel in the ethics of testing and reporting, ensuring that all techs are capable of doing their jobs, detailing false positive rates of testing methods, and so on. The forensic scientist communities and organizations generally regarded this as a good thing. Several government agencies wanted nothing to do with it and flat out ignored several proposals made by NIST and refused to classify certain disciplines as being "sciences" so as to avoid making necessary changes (specifically those revolving around audio and video work.)

Re: Re: failure to test

"It's pretty unreasonable to expect an analyst to never return a false positive. People are human, even when the stakes are incredibly high. "

really? you talk as if he checked the wrong box by mistake or mixed up two samples by accident. He deliberately didn't even check samples and then went even further and deliberately falsified negative samples. that a bit more then a mistake.

Re: failure to test

Previous anonymous commenter here with an issue I forgot to mention: many government controlled labs actively push against accreditation so that they can magically produce whatever result is wanted. This is especially true for three letter agency labs and even further so for those dealing with audio/video analysis.

That, I believe, is the real issue. Labs being run by ritual and wizardry, rather than reproducible science.

Ambrose Bierce said it best

"As records of courts and justice are admissible, it can easily beproved that powerful and malevolent magicians once existed and were ascourge to mankind. The evidence (including confession) upon whichcertain women were convicted of witchcraft and executed was without aflaw; it is still unimpeachable. The judges' decisions based on it weresound in logic and in law. Nothing in any existing court was ever morethoroughly proved than the charges of witchcraft and sorcery for whichso many suffered death. If there were no witches, human testimony andhuman reason are alike destitute of value."

Costs

Leaving aside the absolute moral/ethical/judicial responsibility for a second (as one does) - how much does it cost to incarcerate each of these innocent people? Letting them out will save a bucketful of money - surely enough to pay for a records search. The wrongful convictions and compensation for kids I'd money and earnings would probably be covered too. (Though not the extra millions that are usually thrown around as punitive damages or whatever.)

THE Solution is release ALL prisoners ,in prison for non violent ,drug offenses,eg possession or use of hash,people who sold cocaine etc already had their property, cash taken off them .People who sold cocaine could be put on community service .And it would save money,it costs 30k plus to have 1 per person in prison for 1 year .

Re:

Yes, I see the pattern.

Stupid voters that have no idea what their responsibilities are as a fully informed juror.

Stupid & Ignorant people screwing themselves over by loving the rich an famous for no other reason that the fact they are rich and famous. Politics are practically no different here. The fact that Pelosi and now Paul Ryan can say shit like we have to pass this bill to see what is in it is a damn good sign of how terrible the corruption is while the voters do nothing about anything. Except bicker back and forth on party lines like idiots.

Cynic

Actually, this can be solved easily...and probably will be, ultimately. Congress simply passes a law stating that lab errors are incidental and cannot be reviewed; another law that protects forensics labs from external investigation; and a law that protects prosecutors from penalties for concealing exculpatory evidence. Simple. Done all the time.

After all, we've got to keep those private prison corporations profitable and also protect the reputation of our legal system.

Who will pay for it? That's the easy part.

Annie Dookhan will pay for it. She created the mess. She gets to pay for fixing it. That is the standard applied to people who caused property damage in the commission of a crime. (Yes, this may bankrupt her and/or require forfeiting her house, which was doubtless paid for with proceeds from her crimes.)

Too many?

Narrowing down "The List" to a single link in the evidence chain -- the drug receipt -- still returned far too many potential matches to be of use.

I think one of us misunderstood the article. I don't see anywhere that it states The List (as he calls it) has too many people on it. Rather, the problem he's trying to solve is finding the hundreds or possibly thousands of additional people who should be on the list but aren't. Did I miss something?

falseinfo

I'm not sure I get what Annie did wrong here. She was using drugs and then replacing people's pee with hers?, had bags of heroin, cocaine, meth, to throw pinches in pee jars for the lulz thinking of all the lives, families she would destroy?

It is not about the drugs anymore. It is about the money it can bring to the government. It is an excuse to take money from it's own people. We need to buy all this equipment for millions to fight the drug lords. Yeah! We need to seige your money to stop the drug lords. wink. wink. It is greed.

statistical analysis & quality control

It's long been standard practice for government entities that routinely send out samples for testing -- such as municipal water works and environmental agencies -- to have quality-control checks in place. Testing labs also have their own quality-control practices.

These include methods such as splitting samples and sending them in separately (or to different labs), spiking test samples with measured quantities of adulterants, and other quality-control checks that are used to gauge the accuracy and consistency of the testing facility -- as well as each employee if need be.

Therefore, in any "normal" testing lab situation, it would be virtually impossible for a technician to put out flawed results for years and never get caught.

But apparently police agencies don't believe in the need for any of this, because in their mind (at least as far as the public is concerned) everyone from the arresting deputy all the way down the line is 100% trustworthy and infallible.

Re: statistical analysis & quality control

But apparently police agencies don't believe in the need for any of this, because in their mind (at least as far as the public is concerned) everyone from the arresting deputy all the way down the line is 100% trustworthy and infallible.

The more cynical explanation is that their goal is not accuracy but convictions.

Why don't they tell us his name?

Please print the name of the "rogue chemist." Because it sounds like there are 10,000 people wrongly in prison because of him -- and going after his ill-gotten personal life savings in a court of law is the best way to encourage the next guy to do his job honestly.