The Drug War Is Creating Problems Too Big To Fix

from the in-which-warning-signs-are-greeted-with-praise,-enthusiasm-and-blinders dept

David Colarusso, a public defender turned data scientist, has a fascinating post at Law Technology Today describing the many issues arising from the abusive activities of a single chemist at the Massachusetts state drug lab. The starting point of his post — and his problems — trace back a few years.

In 2012, it was discovered that a chemist working at the Massachusetts state drug lab in Jamaica Plain had been falsifying drug tests (e.g., claiming that samples contained narcotics without testing them and even adding cocaine to samples to get a positive result when prior testing came back negative). She had worked at the lab for nearly a decade, and these revelations called into question the outcomes in tens of thousands of cases.

Obviously, this sort of tampering means there are convictions waiting to be overturned. But two years later, little progress has been made. It isn’t that the state is obstructing efforts to make the falsely-convicted whole again (there may be some of that, but Colarusso’s post doesn’t indicate there is), but that nearly a decade’s-worth of bogus lab work potentially infects thousands of convictions. Narrowing down this list to those directly affected is an enormous task, one that Colarusso was tasked with making more manageable. Narrowing down “The List” to a single link in the evidence chain — the drug receipt — still returned far too many potential matches to be of use. Additional restrictions trimmed the possible matches a bit more, but still left far too many potential victims of the chemist’s work.

Staff attorneys take only a small fraction of indigent cases. The majority are handled by private attorneys. So only a subset of defendants on The List would be in our client files. However, given The List contained nearly 40,000 names, this subset was still rather sizable. So we used some nice open source software to look for matches between our clients’ names and those found on the list (this involved some data wrangling in Python and Pandas along with the creation of a nice IPython Notebook or two). This gave us a rough list of clients on The List, and we used these names to create a list of their co-defendants.

The narrowing of the field only did so much. The List remained sizable, thanks to inconsistencies inherent in the system itself.

This gave us a rough list of clients on The List, and we used these names to create a list of their co-defendants. We then checked The List for the co-defendant names. Unfortunately, a lot of these were missing. If we assumed the same rate of missing names across all cases, it seemed The List was missing somewhere between 0 and 9,600 names. Wait, what? That’s right, thousands of potentially missing names. The uncertainty came from the fact that we had to match names. The List did not come with dates of birth, addresses, or Social Security Numbers—just names. So occasionally, we could not find a name we were looking for because the Commonwealth and CPCS disagreed on the spelling of a name or someone made a transcription error.

By this point in the investigation, the master list was still huge and it was obvious the list itself was missing hundreds of names, which meant hundreds of possibly wrongly-convicted citizens. The reality of this situation was this: to track down those missing names and to finally set the wheels of justice in motion, thousands of police reports would need to be read and cross-referenced against those on the master list. But who will do this? And with what funds? That’s still unclear. It all depends on who feels justice should be served and who feels justice should be served, but only up to a certain dollar amount.

So, the problem — which was one person in one lab falsifying thousands of test results — has become something so unwieldy that it may never result in the exoneration of everyone chemist Annie Dookhan managed to wrongly put behind bars. The problem is too big to solve, and much of that has to do with the efficiency of drug prosecutions versus the much less efficient wheels of bureaucracy. Data wrangling helped determine the size of the problem and point a way towards a solution, but the solution is still hundreds, if not thousands, of hours away.

But Colarusso points to one aspect that should have been noticed and would have kept this from becoming a 40,000-file catastrophe (and that’s without counting the undetermined number of omissions).

The real promise is in catching the next scandal early. This means listening to data and looking for patterns. The rogue chemist had a throughput three times that of the next “most productive” chemist. That should have been a red flag. Data science offers the promise of mining data for signals such as these and sounding the alarm.

But they weren’t looking for a problem at the drug lab. They were looking for productivity. Dookhan’s suspiciously-fast output wasn’t greeted with suspicion. It was greeted with praise and an increased workload.

The report shows that the Hinton lab leaned heavily on Dookhan’s productivity. Supervisors lauded her work ethic and assigned her an increasing share of tests.

“From January 1, 2004, through December 31, 2011, Dookhan was assigned 25.3% of all analyses in the Drug Lab and completed 21.8% of all tests conducted by staff,” the report said.

There’s a point when a problem is still manageable and there’s a point when it becomes too big to correct within the confines of the system that helped create it. The drug lab itself pushed it from solvable to impossible. What happened here isn’t an isolated incident. Similar abuses have occurred at the nation’s top labs — those belonging to the FBI. Results of forensic testing were overstated to prosecutors, who then put these experts on the stand to help convict hundreds of thousands of people using the questionable results. It’s only because of outside pressure that the FBI and DOJ are even looking into this, and it’s easy to imagine “The List” of possibly wrongly-convicted persons here far exceeds the 40,000+ Colarusso tangled with.

In addition to the problem begin ignored for two decades, the effort to right the FBI’s wrong isn’t being pursued with nearly the same enthusiasm as the bogus prosecutions. And yet, there were early warning signs that were ignored. Information about the FBI’s evidence issues was passed on to prosecutors — but this information, including exculpatory evidence, was never passed on to defense attorneys.

The time to handle a problem is before it negatively affects thousands by depriving them of their liberty. But the desire to fight a drug war led to the cover up of exculpatory evidence by US prosecutors working with the FBI and the opportunity to fake even more test results in Annie Dookhan’s case. Caught in the middle of all of this are thousands who may never see justice done because the problem was ignored until it no longer could be.

Filed Under: , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “The Drug War Is Creating Problems Too Big To Fix”

Subscribe: RSS Leave a comment
Richard (profile) says:

Re: drug what?

Actaully it’s not a drug war – it’s a war against the laws of economics.

As such it is impossible to win – and that should have been clear from the very beginning.

The more successful you are in “winning” the war – the more the price of drugs rises – and hence the incentives and financial backing of your opponents grows.

Starke (profile) says:

Re: Re: Re:

…and if you don’t do drugs, you’re still rolling the dice on the possibility that some GED reject with a badge and gun will decide you’re “suspicious” and toss you in a cell to “preserve the element of surprise.” Until they can figure out what they think you did, and then convict you for it on faulty forensic evidence.

But, at least it’s not like someone might just outright fake up that evidence just to get a pat on the head at work… oh, wait.

Anonymous American Coward (profile) says:

Re: There are LOTS of reasons to never set foot in the States

For one, a huge number of food-service workers don’t have low out-of-pocket health insurance (or any health coverage at all), and they don’t have paid sick leave, so they try to hide their symptoms and come in to work sick. (A lot of them don’t have enough of a financial cushion to be able to take unpaid days off and still make next month’s rent.) Bon appétit!

For another, if you come to the US as a visitor and don’t have first-class emergency-medical and medevac insurance, you risk being financially ruined if you have a serious accident or fall seriously ill. American medical bills are nothing short of obscene. (Just ask the parents of Canadian skier Sarah Burke.) Hopefully, the worst symptoms of any illness you contract at an American restaurant won’t kick in until you’re back home.

And you don’t want to travel with very much cash — not because of “bad guys,” but because of cops. A lot of American law enforcement agencies will simply take it from you on the theory that it’s inherently suspicious, and you will have to hire a lawyer, sue them, and prove that you aren’t a drug trafficker in order to get it back. And no, you won’t recover your attorney fees.

But, hey! Some of our national parks are still pretty nice…

scotts13 (profile) says:

I don't see how this is unmanageable

It shouldn’t be all that hard to determine if this technician processed the samples from a case. If yes, then the prisoner goes free – and hopefully, the prosecutor gets it deducted from his score card. Easy. OR, are they really saying we should accept innocent people being jailed for decades, at taxpayer expense, because it’s too hard to let them free?

Chris-Mouse (profile) says:

Re: I don't see how this is unmanageable

That’s the whole problem. They don’t really have a simple way to create a list of every prisoner convicted on the basis of this technician’s work.
Some, but not all, defendants are named with the sample tested, many names were left off. The only way to create the complete list is to have a person go to the paper court records scattered all over the place, and read them all.
Given the reasons why this is being done, it probably should also involve a second person doing an audit of the first person’s work.

Ninja (profile) says:

It is NOT too big to solve

We are talking about goddamn human lives unjustly put behind bars here. It does not matter how much it will cost. It is a moral and social obligation of the Government to fix it. If anything, all the convicted should get a lawyer and go question their results in the justice system. This would generate a much bigger problem.

Also, said chemist should be jailed for life. What kind of rotten, sick sociopath deliberately screws people’s lives by adding positives where there were negatives? And the lab itself should be prosecuted and closed down and its management along with the power chain that allowed her to work without auditing put behind bars as well.

But we all know this only happens in a fair world.

Richard (profile) says:

Re: It is NOT too big to solve

I constant trend in the US and UK is institutional meanness.

If something good is to be handed out by the state (welfare benefits, healthcare, education or relief from wrongful imprisonment) then every possible financial stringency is invoked to avoid doing it on grounds of cost – but if something bad is to be done (arrest, trial, imprisonment etc) then NO EXPENSE WILL BE SPARED.

nasch (profile) says:

Re: It is NOT too big to solve

We are talking about goddamn human lives unjustly put behind bars here. It does not matter how much it will cost.

I would certainly agree with that, but I don’t know if it’s a majority position. I suspect many people would not want a lot of money spent on such a problem, because “most of them are probably guilty of something anyway”. I know, it’s disgusting. I’m with you though – there should be no consideration of whether to seek justice for the wrongfully convicted based on how much it will cost.

Beta (profile) says:

failure to test

I’ve been asking for years why we don’t a control policy at these police-affiliated, District-Attorney-influenced labs:

“You will be analyzing specimens taken from crime scenes and from suspects in custody, but occasionally some of them will actually be control samples prepared by the Public Defender’s office. If you return one false positive, your career is over and the Public Defender’s office gets your pension.”

Anonymous Coward says:

Re: failure to test

It’s pretty unreasonable to expect an analyst to never return a false positive. People are human, even when the stakes are incredibly high.

What really needs to be done (and is generally required for labs to get accreditation) is the expansion of quality control resources in labs. Because one cannot reasonable expect a person to be perfect, put more eyes on it. Some labs I’ve been to have two people in each examination: one running it and one observing the running to make sure it’s done properly. Then it’s run again by a different pair. Then the results are reviewed by quality control. It isn’t perfect, but it’s better than only one person seeing it.

That One Guy (profile) says:

Re: Re: Call it 'incentive'

It’s pretty unreasonable to expect an analyst to never return a false positive. People are human, even when the stakes are incredibly high.

When you consider a false positive has the chance to ruin someone for life, no, I don’t really consider it unreasonable. The stakes are high, and they’re only human as you say, but when screwing up can get someone tossed behind bars for years, decades or longer, 100% accuracy is not only desired, it should be required.

Either that or a positive result should be drastically decreased in evidentury value, such that even a positive result doesn’t ensure a conviction, but is only considered a minor bit of supporting evidence, rather than grounds for instant conviction.

Anonymous Coward says:

Re: Re: Re: Call it 'incentive'

I didn’t say that I don’t think 100% (or as close to it as possible) is not required. I completely agree with that and, as an examiner, the fear that my work may harm an innocent person is terrifying.

What I said was that it’s unreasonable to expect that of any one person. The technician in this instance was a problem, a huge problem, but the lack of a proper quality control was a bigger one. I believe the lab is as much responsible for this as Annie Dookhan. You can’t expect someone to be perfect, so you build a way of determining whether they functioned correctly or not. This lab failed at that on a criminal level.

We can push for Annie Dookhan being punished (and she should be, severely) and we can call for the results to be overturned (which they should be), but by not addressing that the laboratory itself is what truly failed, we leave the door open for the next person to do this. As long as we blame the technician and don’t use the opportunity to demand the labs themselves to operate in a transparent, controlled, and unbiased way from the very beginning, nothing will change.

As a side note, I also agree that positive results should be less valuable. There is far too much reliance on psuedo-science and incorrect understanding of the statistical results of an examination.

That One Guy (profile) says:

Re: Re: failure to test

Bah, jumped the gun a bit in my comment.

The second half of your comment would certainly help, but even then, I think that a positive result should be drastically reduced in value as evidence, simply because while desired, 100% accuracy is not possible, so the possibility of a false positive should always be taken into consideration. Treat a positive or negative result as a piece of evidence, but not a core piece, such that it’s not enough to win a case on it’s own without significant supporting evidence.

Anonymous Coward says:

Re: Re: Re: failure to test

This we can definitely both agree on…and I did when I also jumped the gun and didn’t read this response on yours.

There’s far too much weight put on this type of evidence and on the testimony of people who run them. I firmly believe that these sorts of examinations should be used exclusively (or at least almost exclusively) for investigative purposes. There should always be more than the words of an expert witness, especially when it is very difficult for courts to determine the true expert-ness of those witnesses. There are systems in place to try, but it takes a lot of effort on all sides (judge included) to determine the true nature of the expert’s knowledge. I, unfortunately, have some stories about examiners that are well respected by everyone except the scientific part of the community.

It’s a pretty screwed up system that is unfortunately easy to game.

Uriel-238 (profile) says:

Re: Re: Reminds me of our multiple detection dog programs

Many of which still produce dogs trained to detect substances (narcotics, explosives, whathaveyou) but many are just trained to signal when the handler indicates to do so.

These dogs are derisively called trick horse dogs.

Most detection dogs are used as a pre-warrant search in order to justify a probable cause search, for which trick horse dogs serve better than actual detection dogs.

As a result we have stats like 82% of the positive signals from Chicago police dogs sniffing Hispanics prove to be false.

Even regular detection dogs have about a 58% false positive rate used by US law enforcement.

I’m pretty sure that without a major reform, we’re not going to see redundant tests, given that the prosecutors are going to prefer labs that yield more positive results. Even ones that are false.

Anonymous Coward says:

Re: Re: Re: Reminds me of our multiple detection dog programs

I’m pretty sure that without a major reform, we’re not going to see redundant tests, given that the prosecutors are going to prefer labs that yield more positive results. Even ones that are false.

This is why seeing those major reformations occur is going to be so difficult. There’s really no political incentive to reforming it. If you make it harder to get the positive results (true or false), you look soft on crime. Easier to just throw the tech in prison for a bit, let the lab head resign, and keep going about doing things in a dangerous way (for everyone else.)

A couple years ago, there was a call by NIST to strengthen the scientific ethics of forensic laboratories and better their practices, meaning: training personnel in the ethics of testing and reporting, ensuring that all techs are capable of doing their jobs, detailing false positive rates of testing methods, and so on. The forensic scientist communities and organizations generally regarded this as a good thing. Several government agencies wanted nothing to do with it and flat out ignored several proposals made by NIST and refused to classify certain disciplines as being “sciences” so as to avoid making necessary changes (specifically those revolving around audio and video work.)

kog999 says:

Re: Re: failure to test

“It’s pretty unreasonable to expect an analyst to never return a false positive. People are human, even when the stakes are incredibly high. “

really? you talk as if he checked the wrong box by mistake or mixed up two samples by accident. He deliberately didn’t even check samples and then went even further and deliberately falsified negative samples. that a bit more then a mistake.

Anonymous Coward says:

Re: failure to test

Previous anonymous commenter here with an issue I forgot to mention: many government controlled labs actively push against accreditation so that they can magically produce whatever result is wanted. This is especially true for three letter agency labs and even further so for those dealing with audio/video analysis.

That, I believe, is the real issue. Labs being run by ritual and wizardry, rather than reproducible science.

lemgandi (profile) says:

Ambrose Bierce said it best

“As records of courts and justice are admissible, it can easily be
proved that powerful and malevolent magicians once existed and were a
scourge to mankind. The evidence (including confession) upon which
certain women were convicted of witchcraft and executed was without a
flaw; it is still unimpeachable. The judges’ decisions based on it were
sound in logic and in law. Nothing in any existing court was ever more
thoroughly proved than the charges of witchcraft and sorcery for which
so many suffered death. If there were no witches, human testimony and
human reason are alike destitute of value.”

Digitari says:

first the Military...........

this is NOT new, nor unexpected to anyone that was in the military 30 some years ago.

I knew of someone in the military that was bust in rank and pay for the offense of

“suspicion of being in an area where Marijuana was suspected of being smoked”

he passed the drug screen but was still busted, for being near a place where someone of authority thought they smelled pot (this was in 1982).

I knew then what was coming….

don’t hate the players, hate the Government that set up the game.

(see how much your [informed] vote matters?)

carborundum (profile) says:


Leaving aside the absolute moral/ethical/judicial responsibility for a second (as one does) – how much does it cost to incarcerate each of these innocent people? Letting them out will save a bucketful of money – surely enough to pay for a records search. The wrongful convictions and compensation for kids I’d money and earnings would probably be covered too. (Though not the extra millions that are usually thrown around as punitive damages or whatever.)

Anonymous Coward says:

THE Solution is release ALL prisoners ,
in prison for non violent ,drug offenses,
eg possession or use of hash,
people who sold cocaine etc already had their property, cash taken off them .
People who sold cocaine could be put on community service .
And it would save money,
it costs 30k plus to have 1 per person in prison for 1 year .

Anonymous Coward says:

Re: Re:

Yes, I see the pattern.

Stupid voters that have no idea what their responsibilities are as a fully informed juror.

Stupid & Ignorant people screwing themselves over by loving the rich an famous for no other reason that the fact they are rich and famous. Politics are practically no different here. The fact that Pelosi and now Paul Ryan can say shit like we have to pass this bill to see what is in it is a damn good sign of how terrible the corruption is while the voters do nothing about anything. Except bicker back and forth on party lines like idiots.

Coyne Tibbets (profile) says:


Actually, this can be solved easily…and probably will be, ultimately. Congress simply passes a law stating that lab errors are incidental and cannot be reviewed; another law that protects forensics labs from external investigation; and a law that protects prosecutors from penalties for concealing exculpatory evidence. Simple. Done all the time.

After all, we’ve got to keep those private prison corporations profitable and also protect the reputation of our legal system.

Anonymous Coward says:

Who will pay for it? That's the easy part.

Annie Dookhan will pay for it. She created the mess. She gets to pay for fixing it. That is the standard applied to people who caused property damage in the commission of a crime. (Yes, this may bankrupt her and/or require forfeiting her house, which was doubtless paid for with proceeds from her crimes.)

nasch (profile) says:

Too many?

Narrowing down “The List” to a single link in the evidence chain — the drug receipt — still returned far too many potential matches to be of use.

I think one of us misunderstood the article. I don’t see anywhere that it states The List (as he calls it) has too many people on it. Rather, the problem he’s trying to solve is finding the hundreds or possibly thousands of additional people who should be on the list but aren’t. Did I miss something?

Uriel-238 (profile) says:

We've already long since established that Justice in the US is not served.

Every prisoner in the states is a political prisoner. It’s time for Bastille Day, US.

That some of the people who walk are genuinely monstrous is the price the society pays for letting our justice system rot to ruin.

Of course this won’t happen today. But it will eventually have to happen.

Personanongrata says:

Eliminate State Run Laboratories

It is abundantly clear that many technicians/chemists/scientists working within government run laboratories are biased in favor of the state as are the results of their tests.

It is also abundantly clear that any evidence testing required by law enforcement should only be conducted at independently run laboratories in double-blind fashion.

This simple step will help eliminate any potential bias in the lab or test results.

Anonymous Coward says:

statistical analysis & quality control

It’s long been standard practice for government entities that routinely send out samples for testing — such as municipal water works and environmental agencies — to have quality-control checks in place. Testing labs also have their own quality-control practices.

These include methods such as splitting samples and sending them in separately (or to different labs), spiking test samples with measured quantities of adulterants, and other quality-control checks that are used to gauge the accuracy and consistency of the testing facility — as well as each employee if need be.

Therefore, in any “normal” testing lab situation, it would be virtually impossible for a technician to put out flawed results for years and never get caught.

But apparently police agencies don’t believe in the need for any of this, because in their mind (at least as far as the public is concerned) everyone from the arresting deputy all the way down the line is 100% trustworthy and infallible.

nasch (profile) says:

Re: statistical analysis & quality control

But apparently police agencies don’t believe in the need for any of this, because in their mind (at least as far as the public is concerned) everyone from the arresting deputy all the way down the line is 100% trustworthy and infallible.

The more cynical explanation is that their goal is not accuracy but convictions.

Anonymous Coward says:

Not the real objective.

“But who will do this? And with what funds? That’s still unclear. It all depends on who feels justice should be served and who feels justice should be served, but only up to a certain dollar amount.”

Actually, considering the cost of incarcerating the wrongly convicted, it would probably save money in the end. But that’s not the real objective, is it?

Add Your Comment

Your email address will not be published.

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...