DOJ Tells Forensic Experts To Stop Overstating The 'Scientific Certainty' Of Presented Evidence
from the directed-to-use-air-quotes-when-saying-'science'-or-'certainty' dept
The DOJ is finally addressing some long-ignored problems with the forensic evidence its prosecutors rely on. For two decades, FBI forensics experts handed out flawed testimony in hundreds of criminal cases, routinely overstating the certainty of conclusions reached by forensic examination. Of those cases, 28 ended in death penalty verdicts.
An earlier attempt to address issues with flawed science and flawed testimony swiftly ran aground. Federal judge Jed S. Rakoff very publicly resigned from a committee formed to examine these issues after he was informed by the attorney general's office that he wasn't actually supposed to be examining these issues.
Last evening, January 27, 2015, I was telephonically informed that the Deputy Attorney General of the U.S. Department of Justice has decided that the subject of pre-trial forensic discovery -- i.e., the extent to which information regarding forensic science experts and their data, opinions, methodologies, etc., should be disclosed before they testify in court -- is beyond the “scope” of the Commission’s business and therefore cannot properly be the subject of Commission reports or discussions in any respect.
Because I believe that this unilateral decision is a major mistake that is likely to significantly erode the effectiveness of the Commission -- and because I believe it reflects a determination by the Department of Justice to place strategic advantage over a search for the truth -- I have decided to resign from the Commission, effective immediately. I have never before felt the need to resign from any of the many committees on which I have served over the years; but given what I believe is the unsupportable position now taken by the Department of Justice, I feel I have no choice.
Caleb Mason of Brown, White & Osborn (the "White" is Popehat's Ken White) reports that the DOJ appears to be taking these problems more seriously. It has issued a directive [PDF] forbidding forensic experts from making claims about "scientific certainty" when presenting evidence.
Directive Number 1 provides that agencies must now “ensure that forensic examiners are not using the expressions ‘reasonable scientific certainty’ in their reports or testimony.” Yes, you read that right. The Department of Justice is telling its forensic expert witnesses to stop claiming “scientific certainty.” Why? Because for most forensic disciplines, there never was any, and DOJ is—after decades of resistance—admitting it.
One of the forms of evidence is fingerprints, the thing every law enforcement agency makes sure to obtain when booking suspects because it's supposedly so infallible. But like almost everything else law enforcement forensic experts claim are reasonably certain, scientifically-speaking, examination of prints no more guarantees a match than examining bite marks.
Fingerprint examiners look for “matching points” in prints, but believe it or not, there are no general standards for which points to look at, how many points to look at, or even what counts as a “point.” Not only are there no established standards, there isn’t even general agreement within the forensic analysis community. Some people like eight points, others ten, others twelve. Many examiners insist they can make an identification with just a single point.
Even more amazingly, in stark contrast to DNA matching, no one knows what the statistical likelihood is of two fingerprints sharing particular points, or whether that likelihood is different for different regions or features of the print. This is the crucial question for any identification methodology, because while each person’s fingerprints may be unique, the examiner doesn’t look at every molecule—the examiner looks at whatever five (or eight, or ten) “points” he or she chooses to look at.
Why is this process still so vague even after decades of reliance on it for identifying suspects? Well, it's because the DOJ won't allow anyone other than the government to take a look at the collected records. Researchers who may have been able to make better determinations on how many points are needed for more definitive matches (or how often false positives are returned by the database) have been locked out by the DOJ.
But the big fingerprint databases are controlled by DOJ, and DOJ has steadfastly refused to let researchers use them for the types of analyses geneticists do with DNA. That’s what makes print analysis so frustrating: the data exists, so fingerprint analysis could be a genuine scientific discipline, with publicly-available databases, peer-reviewed research, known error rates, and accepted methodologies. It could be a real body of knowledge about the differential rates of occurrence among populations of particular physical features of our fingerprints. Hopefully one day it will be. But it’s not now, as the DOJ directives finally acknowledge.
The DOJ's not offering to open up its fingerprint database for outside examination. But at least it's admitting it hasn't let anyone without a vested interest in successful prosecutions take a good look at the methods used by its forensic examiners or the collected evidence they're working with.
[As a bonus, here's another fantastic read by Caleb Mason: a Constitutional examination of Jay-Z's hit track, entitled "JAY-Z’S 99 PROBLEMS, VERSE 2: A CLOSE READING WITH FOURTH AMENDMENT GUIDANCE FOR COPS AND PERPS."]