from the in-which-warning-signs-are-greeted-with-praise,-enthusiasm-and-blinders dept
David Colrasso, a public defender turned data scientist, has a fascinating post at Law Technology Today describing the many issues arising from the abusive activities of a single chemist at the Massachusetts state drug lab. The starting point of his post -- and his problems -- trace back a few years.
In 2012, it was discovered that a chemist working at the Massachusetts state drug lab in Jamaica Plain had been falsifying drug tests (e.g., claiming that samples contained narcotics without testing them and even adding cocaine to samples to get a positive result when prior testing came back negative). She had worked at the lab for nearly a decade, and these revelations called into question the outcomes in tens of thousands of cases.Obviously, this sort of tampering means there are convictions waiting to be overturned. But two years later, little progress has been made. It isn't that the state is obstructing efforts to make the falsely-convicted whole again (there may be some of that, but Colrasso's post doesn't indicate there is), but that nearly a decade's-worth of bogus lab work potentially infects thousands of convictions. Narrowing down this list to those directly affected is an enormous task, one that Colrasso was tasked with making more manageable. Narrowing down "The List" to a single link in the evidence chain -- the drug receipt -- still returned far too many potential matches to be of use. Additional restrictions trimmed the possible matches a bit more, but still left far too many potential victims of Plain's work.
Staff attorneys take only a small fraction of indigent cases. The majority are handled by private attorneys. So only a subset of defendants on The List would be in our client files. However, given The List contained nearly 40,000 names, this subset was still rather sizable. So we used some nice open source software to look for matches between our clients’ names and those found on the list (this involved some data wrangling in Python and Pandas along with the creation of a nice IPython Notebook or two). This gave us a rough list of clients on The List, and we used these names to create a list of their co-defendants.The narrowing of the field only did so much. The List remained sizable, thanks to inconsistencies inherent in the system itself.
This gave us a rough list of clients on The List, and we used these names to create a list of their co-defendants. We then checked The List for the co-defendant names. Unfortunately, a lot of these were missing. If we assumed the same rate of missing names across all cases, it seemed The List was missing somewhere between 0 and 9,600 names. Wait, what? That’s right, thousands of potentially missing names. The uncertainty came from the fact that we had to match names. The List did not come with dates of birth, addresses, or Social Security Numbers—just names. So occasionally, we could not find a name we were looking for because the Commonwealth and CPCS disagreed on the spelling of a name or someone made a transcription error.By this point in the investigation, the master list was still huge and it was obvious the list itself was missing hundreds of names, which meant hundreds of possibly wrongly-convicted citizens. The reality of this situation was this: to track down those missing names and to finally set the wheels of justice in motion, thousands of police reports would need to be read and cross-referenced against those on the master list. But who will do this? And with what funds? That's still unclear. It all depends on who feels justice should be served and who feels justice should be served, but only up to a certain dollar amount.
So, the problem -- which was one person in one lab falsifying thousands of test results -- has become something so unwieldy that it may never result in the exoneration of everyone chemist Jamaica Plain managed to wrongly put behind bars. The problem is too big to solve, and much of that has to do with the efficiency of drug prosecutions versus the much less efficient wheels of bureaucracy. Data wrangling helped determine the size of the problem and point a way towards a solution, but the solution is still hundreds, if not thousands, of hours away.
But Colrasso points to one aspect that should have been noticed and would have kept this from becoming a 40,000-file catastrophe (and that's without counting the undetermined number of omissions).
The real promise is in catching the next scandal early. This means listening to data and looking for patterns. The rogue chemist had a throughput three times that of the next “most productive” chemist. That should have been a red flag. Data science offers the promise of mining data for signals such as these and sounding the alarm.But they weren't looking for a problem at the drug lab. They were looking for productivity. Plain's suspiciously-fast output wasn't greeted with suspicion. It was greeted with praise and an increased workload.
The report shows that the Hinton lab leaned heavily on Dookhan’s productivity. Supervisors lauded her work ethic and assigned her an increasing share of tests.There's a point when a problem is still manageable and there's a point when it becomes too big to correct within the confines of the system that helped create it. The drug lab itself pushed it from solvable to impossible. What happened here isn't an isolated incident. Similar abuses have occurred at the nation's top labs -- those belonging to the FBI. Results of forensic testing were overstated to prosecutors, who then put these experts on the stand to help convict hundreds of thousands of people using the questionable results. It's only because of outside pressure that the FBI and DOJ are even looking into this, and it's easy to imagine "The List" of possibly wrongly-convicted persons here far exceeds the 40,000+ Colrasso tangled with.
“From January 1, 2004, through December 31, 2011, Dookhan was assigned 25.3% of all analyses in the Drug Lab and completed 21.8% of all tests conducted by staff,” the report said.
In addition to the problem begin ignored for two decades, the effort to right the FBI's wrong isn't being pursued with nearly the same enthusiasm as the bogus prosecutions. And yet, there were early warning signs that were ignored. Information about the FBI's evidence issues was passed on to prosecutors -- but this information, including exculpatory evidence, was never passed on to defense attorneys.
The time to handle a problem is before it negatively affects thousands by depriving them of their liberty. But the desire to fight a drug war led to the cover up of exculpatory evidence by US prosecutors working with the FBI and the opportunity to fake even more test results in Jamaica Plain's case. Caught in the middle of all of this are thousands who may never see justice done because the problem was ignored until it no longer could be.