Hide Techdirt is off for the long weekend! We'll be back with our regular posts tomorrow.

Scientists Looking To Fix The Many Problems With Forensic Evidence

from the can-you-fix-the-people-performing-the-tests? dept

Everything everyone saw in cop shows as evidence linking people to crimes — the hair left on someone’s clothing, the tire tracks leading out to the road, the shell casings at the scene, etc. — is all proving to be about as factual as the shows themselves.

While much of it is not exactly junk science, much of it has limited worth. What appears to indicate guilt contains enough of a margin of error that it could very easily prove otherwise. Science Magazine is taking a look at the standbys of forensic science and what’s being done to ensure better presentations of evidence in the future.

On a September afternoon in 2000, a man named Richard Green was shot and wounded in his neighborhood south of Boston. About a year later, police found a loaded pistol in the yard of a nearby house. A detective with the Boston Police Department fired the gun multiple times in a lab and compared the minute grooves and scratches that the firing pin and the interior of the gun left on its cartridge casings with those discovered on casings found at the crime scene. They matched, he would later say at a pretrial hearing, “to the exclusion of every other firearm in the world.”


So how could the detective be sure that the shots hadn’t been fired from another gun?

The short answer, if you ask any statistician, is that he couldn’t. There was some unknown chance that a different gun struck a similar pattern. But for decades, forensic examiners have sometimes claimed in court that close but not identical ballistic markings could conclusively link evidence to a suspect—and judges and juries have trusted their expertise. Examiners have made similar statements for other forms of so-called pattern evidence, such as fingerprints, shoeprints, tire tracks, and bite marks.

Six years ago, the National Academy of Sciences found that these forensic standbys had a much larger margin of error than was portrayed in court by detectives and expert witnesses. It recommended the margin of error be delivered along with the testimony to head off future verdicts based on faulty evidence.

To date, not much has changed. While actual junk science like bite marks has largely been discarded by prosecutors, the others remain, even as their reliability has been constantly questioned. The FBI loved hair analysis, right up to the point that it determined its witnesses had overstated test results 90% of the time in the two decades prior to 2000.

Even fingerprints, which have long been considered unassailable because of their supposed uniqueness, aren’t much better. Some of it has to do with the presumption that every fingerprint is so unique even a partial print can eliminate suspects. The rest of its issues lie with those matching the prints.

One study of 169 fingerprint examiners found 7.5% false negatives—in which examiners concluded that two prints from the same person came from different people—and 0.1% false positives, where two prints were incorrectly said to be from the same source. When some of the examiners were retested on some of the same prints after 7 months, they repeated only about 90% of their exclusions and 89% of their individualizations.

The NIST has given $20 million to the Center for Statistics and Applications in Forensic Evidence (CSAFE) to come up with a better way to present this sort of evidence — one that clearly accounts for any uncertainties in the results or processes. CSAFE is still trying to figure out how to present this as a number/rating. But that might not be the only problem. The other issue is that juries and judges may not find specifics about forensic reliability to play much of a part in deciding guilt or innocence.

In a 2013 study, for instance, online participants had to rate the likelihood of a defendant’s guilt in a hypothetical robbery based on different kinds of testimony from a fingerprint examiner. It didn’t seem to matter whether they were simply told that a print at the scene “matched” or was “individualized” to the defendant, or whether the examiner offered further justification—the chance of an error is “so remote that it is considered to be a practical impossibility,” for example. In all those cases, jurors rated the likelihood of guilt at about 4.5 on a 7-point scale. “As a lawyer, I would have thought the specific wording would have mattered more than it did,” Garrett says. But if subjects were told that the print could have come from someone else, they seemed to discount the fingerprint evidence altogether.

The other part of the problem is the people who perform the tests. Multiple incidents where evidence was falsified or not properly tested have been uncovered. The evidence is only as good as the processes, and if steps are skipped because of sloppiness or laziness, the evidence’s credibility becomes highly questionable — not just for the specific instance where results were faked, but for every test this person has touched.

There’s no possible way to eliminate honest errors, much less prevent anyone from falsifying results. In both cases, the problems are caught after the damage has been done. Humans are the most unpredictable part of the chain of evidence but also an irreplaceable part. CSAFE will be working with forensics labs to create best practices, but it can do nothing to prevent the lazy and/or incompetent from completely ignoring the proper steps.

Problems are also present higher up the chain. When bad science or bad practices result in questionable evidence, it’s often extremely difficult to have convictions resulting from them overturned.

What’s troubling, [federal judge Nancy] Gertner says, is that when judges accept junk science, an appeals court rarely overrules them. Attaching a numerical probability to evidence, as CSAFE hopes to do, “would certainly be interesting,” she says. But even a standard practice of critically evaluating evidence would be a step forward. “The pattern now is that the judges who care about these issues are enforcing them, and the judges who don’t care about these issues are not.”

In this way, the courts are no better than labs where shoddy work is done. Variations in personality undermine the dispassionate nature of science, making it susceptible to human prejudices rather than the strength of the evidence itself.

Filed Under: , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Scientists Looking To Fix The Many Problems With Forensic Evidence”

Subscribe: RSS Leave a comment
madasahatter (profile) says:

Re: Goals

The prosecutor’s goal is often a conviction but in some jurisdictions the jury is charged with finding the truth.

Back to the issue in the post, part of the problem is crime scene evidence is often poor quality (smudged fingerprints, partial prints, etc.) that make accurate identification very difficult. Couple this with very sloppy evidence handling procedures (OJ trial when the detective kept the samples in his coat pocket) and there is no valid evidence.

Anonymous Coward says:

Science... Good for only 1 direction

Science is only good for proving that the person IS NOT GUILTY, it is not good enough to PROVE guilt.

We only test a subset of DNA, which means you can get far more false positives than even close to comfortable when you look at the numbers. Additionally with all of the DNA databases now being referenced for criminal searches the police will only see more potential suspects as we move forward. Combine that with the Guilty until proven innocence and ignorant as fuck on science American people and you have a recipe for innocent people going to jail closing the case while the actual criminals walk free!

Possibly moving poor people to jail is the goal - says:

Re: Science... Good for only 1 direction

Many states have “for profit” private prison systems.
These states likely really need to feed more serfs/poor people into the system to meet the contract terms.

The US Gulag – (War on Every Damn Thing) justice system!

Billions for Pretend investigation’s, “Faux forensics” and guaranteed funds to private contractors.
Nowadays mostly the result of of dark money (and future employment promises) fed through the right State/Fed Lobbyists (Definition: purveyor of funds/jobs to corrupt Politicians).

madasahatter (profile) says:

Re: Science... Good for only 1 direction

Good detectives know the limits of the evidence. But one key is amount of evidence present. If the murder weapon is found in your possession, your finger prints are present, your DNA is present, you were seen in the area at time of the crime, etc. you now have some explaining to do. One item is likely not enough to get a conviction.

Also, DNA testing has improved tremendously since it was first introduced to the point it is one the best methods to positively identify a person (except for identical twins) available. Many cold cases have been solved because of DNA evidence.

The point of the scientific evidence (all evidence actually) is not that each bit proves absolutely the guilt of the person but that there is enough evidence that beyond a reasonable doubt the person is guilty. If you apply probability, if each bit is 95% certain the more bits will push the overall certainty much higher.

Anonymous Coward says:

Re: Re: Science... Good for only 1 direction

I said PROOF not good enough for PROOF!

And while the detectives know that limit juries often do not. You will almost never find any jury ever suspect of evidence, far to many of them treat it like evidence is some gospel truth for just exactly the reasons you specified.

It is far to easy to admit a stack questionable evidence that makes someone look damn guilty despite a lot of that evidence being circumstantial and ultimately inconclusive.

A good example is someone not being anywhere near a crime scene but having been accused of lying to the police and going to jail on that alone. Even when they were not lying at all! All it takes is for one officer to lie about you and even worse if they can get a witness to say that they “think” they saw you somewhere else calling your alibi into question and you can go to jail by a just in case you ARE a criminal jury.

We need to seriously stop underestimated the amount of fucks people really do not give. When was the last time you went out and tried to stop someone from being screwed by he government? Now contrast that to all of the specially selected jury’s that are lied to that only want to get back to their lives and you can begin to see how the system has been slowly engineered to produce successful criminal prosecutions! Even when evidence is complete bunk.

They just throw a low of evidence at the jury hoping something sticks to overwhelm them and making them think… there has to be no way for this person to be innocent… look at all of that evidence right?

You should read up on the innocence project and you will quickly see how wrong you are.

In all aspects of law, do not discount the corruption factor multiplied by the worthless and ignorance citizen factor, its a huge factor and often the deciding factor… to hell with any meaningful evidence!

Coyne Tibbets (profile) says:

Hard vs cakewalk

Let’s see…doing it forensic evidence right versus doing a cakewalk to conviction. How do we think real-world CSI’s will decide?

Forensic scientists can debate until they’re blue, but nothing is changing until their testimony starts sinking cases. And since forensic scientists are “free” for testimony at $400/hour or so, guess how much testimony they’ll be giving for average man.

Dr. Glover says:

I have some horror stories...

regarding some forensics science majors I went to college with.

In lab-based biology classes they had absolutely no qualms “cooking the books” to get a desired data outcome, rather than explain why their experiment didn’t work in a scientific manner. Because of that, I refused to work with them on any group projects. And as professionals now, I have little doubt that they are still fudging numbers.

So, think about the issues the FBI has had with this problem and realize the willingness to change data starts early in the careers of these individuals. They somehow manage to not get caught, or professors/instructors don’t give a damn.

So forensics has the problem of not only 1.) scientific validity, but 2.) highly unethical scientists willing to manipulate data.

New Mexico Mark says:

Are grammar police unique?*

It is a good habit to mentally replace the word “unique” with its short definition, “one of a kind”. This prevents misuse with modifiers, and allows poor, neglected words like “unusual” to step up to the plate every now and then.

* http://goo.gl/yMp5eB

Despite my griping, I enjoyed the article. What disturbs me far more than pseudo-scientific bias is the jury selection process itself. Smart, conscientious jurors would help mitigate a lot of these problems. However, in at least one U.S. state jury members are practically guaranteed to be either liars or useless in their role as mediators of justice by one qualifying question along the lines of, “Are you willing to set aside conscience and your own knowledge and decide this case only according to the law and the instructions of the judge?” If laws are always good and judges are always right, why did we need juries in the first place?

Anonymous Coward says:

Re: Are grammar police unique?*

You’re not a grammar Nazi. You just choose to ignore the fact that language is fluid and changes over time.

The earliest meanings of unique when it entered English around the beginning of the 17th century were “single, sole” and “having no equal.” By the mid-19th century unique had developed a wider meaning, “not typical, unusual,” and it is in this wider sense that it is compared: The foliage on the late-blooming plants is more unique than that on the earlier varieties. The comparison of so-called absolutes in senses that are not absolute is standard in all varieties of speech and writing.

Anonymous Coward says:

Re: Re: Re: Are grammar police unique?*

I find it amusing how some words morph onto meaning the exact opposite of what was once the case.

For example, the word literally.
I smirk every time someone uses “literally” in a sentence where they really mean “figuratively”.

That plane was so low that I literally could reach out and touch it.

I would like to see that video.

Uriel-238 (profile) says:

Re: Re: Re:2 Literally offensive

A few published and otherwise well-versed writers started using literally for emphasis, much like seriously or really and it caught on.

I still cringe at it since literally is the last bastion for non-metaphorisms. So that when I say Game of Thrones (or more properly A Song of Ice And Fire) is a literal epic, that’s not saying it’s really awesome (though it’s pretty durned good) but also fits the properties of the epic genre classification.

Quiet Lurcker says:

It would take a ground-shift in how things are done in court, but, how about this idea?

Instead of the lawyers presenting to the judge, they present to the jury (I think it’s supposed to be that way anyway – hence, e.g., jury consultants), but let’s re-emphasize that point.

Then, during the course of the case, as each witness is done testifying for the lawyers, the jury are given an opportunity to present matters on which they want amplification or further information. Basically, give the jury the opportunity to explore the evidence in their own way, and in their own time.

As to the integrity of the forensic labs, maybe require the prosecution to present the full chain of custody, for examination by the defense, at least.

normanjd says:


The problem with many of these test is not uniqueness but how we perform the test.

With fingerprints, you have to find so many points of commonality for a match. (I forget the exact numbers but let’s say 9 for a partial and 15 for an exact match.) One issue is these numbers are probably too low. Also when these techniques were automated, different programs use different algorithms to find what is common and they produce different results. (This is where you get too many false negatives and a few false positives. Double checking by hand should eliminate false positives, but sometimes this is NOT done as the prosecutor wants evidence he can use. Even so, based on numbers in article, odds are only 1 in 1000 cases would have a false positive, and the defense does have access to the fingerprint to manually double check.)

Uriel-238 (profile) says:

No interest in seeking justice.

As JustShutUpAndObey, our courts of law are not interested in seeing justice done, but getting the bad-guy. And every suspect not convicted is a bad-guy who escaped, not an innocent who was exonerated.

This has to change before we consider forensic evidence. Otherwise they can just get a company to build a bad-guy detector which sounds an alarm when you point it at a suspect. Currently, like our detector dogs, that is the height of where forensic science can go.

Right now we’re in a state where forensic labs are not impartial, and will skew their results to implicate, even to the point of committing open fraud.

In this state, the only purpose a forensic lab can serve is the same as detector dogs, to implicate the innocent and force a conviction.

Anonymous Coward says:

Maybe we need “open-source” evidence analysis. They say many eyes on the code is the best quality control for software. Would the same apply to criminal evidence? Would a forensic professional, knowing that his or her methods and conclusions would be re-examined by hundreds of others, be motivated to work to a higher standard of ethics?

Add Your Comment

Your email address will not be published.

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...