Tuesday, July 22, 2008

Every snowflake is different except for the ones that are not: DNA not immune to forensic science critiques

No two snowflakes are alike, we're told, though clearly the possibility that duplicate snowflakes may have existed at some point throughout history remains unmeasured and speculative.

Much of modern forensic science relies on the same unproven assumption. Handwriting analysts think no two people's penmanship are the same. Tires and shoes, we're told, wear and tear differently and may be identified by an "expert" in court. Even earprints have been claimed to be unique.

Ballistics analysts say each gun fired leave unique markings. Fingerprint analysts rely on the notion that no two people could possibly have the same one. And of course, DNA technology seeks to identify us using the ultimate extension of the snowflake theory - the biological roots of human individuality itself.

In recent years significant research has cast doubt on a number of these disciplines that operate under what we'll call the "snowflake assumption."

Handwriting analysis is less individuating than proponents might hope, with experts sometimes misreading their cues to erroneously match different people. Standardized mass production techniques cast significant doubt on the validity of matching to a certainty a bullet to the gun that shot it. Indeed, such comparison based forensics is not based in scientific analysis, but is performed by police technicians whose methods were developed and learned on the job sans external review.

Even fingerprints - the gold standard of evidence produced under the snowflake assumption - may be subject to mismatching. The most prominent recent example had to be:
Brandon Mayfield, a Portland lawyer held for two weeks as a suspect in the Madrid train bombings in 2004. FBI investigators matched prints at the scene to Mayfield, and an independent examiner verified the match. But Spanish National Police examiners insisted the prints did not match Mayfield and eventually identified another man who matched the prints.

The FBI acknowledged the error and Mayfield was released.

Now, according to a story in the Sunday Los Angeles Times, even DNA evidence is coming under fire for potentially failing to accurately differentiate between individuals. Reported the Times ("How reliable is DNA in identifying suspects," July 20):

State crime lab analyst Kathryn Troyer was running tests on Arizona's DNA database when she stumbled across two felons with remarkably similar genetic profiles.

The men matched at nine of the 13 locations on chromosomes, or loci, commonly used to distinguish people.

The FBI estimated the odds of unrelated people sharing those genetic markers to be as remote as 1 in 113 billion. But the mug shots of the two felons suggested that they were not related: One was black, the other white.

In the years after her 2001 discovery, Troyer found dozens of similar matches -- each seeming to defy impossible odds.

As word spread, these findings by a little-known lab worker raised questions about the accuracy of the FBI's DNA statistics and ignited a legal fight over whether the nation's genetic databases ought to be opened to wider scrutiny.

The FBI laboratory, which administers the national DNA database system, tried to stop distribution of Troyer's results and began an aggressive behind-the-scenes campaign to block similar searches elsewhere, even those ordered by courts, a Times investigation found.
I don't have any scientific knowledge to lend to the debate, but the political and legal implications of this discussion are profound. The issue with DNA appears to arise from using a limited number of match points and overstating the likelihoods of a "match" based on limited data. Even if DNA is rich enough for individualized identification, just picking 13 points to check may not be enough detail to justify the astronomical likelihood figures routinely stated in court. Limitations with the databases and the relatively small scope of user inquiries may combine to make it a lot more likely a false match occurs.

Federal courts have only just begun to struggle with the implications of the lack of independent scientific verification on fingerprints' individuality. Clearly DNA evidence, too, will not escape more detailed scrutiny in an era that history may remember as spawning a forensic science revolution.

At the very least, I wouldn't be surprised to see the number of matched points on a DNA strand increased and more restrictions in court on the overwhelming (and perhaps fanciful) mathematical estimates frequently offered to support them.

Convictions based on a single piece of evidence or one witness simply may always risk a wrongful conviction. Eyewitnesses sometimes get it wrong. Forensic analysts sometimes accuse the wrong person, and some forensic science is more accurate than others. We've long known most of it is subjective; now we're learning virtually none of it's definitive - even "gold standard" evidence like eyewitnesses, confessions, or DNA and fingerprint technology.

While it's legally possible for the state to make a case based on one witness or one piece of evidence, the long string of recent DNA-based exonerations shows that prudence may dictate requiring corroboration for even the most damning, one-in-a-billion piece evidence presented to a jury. That is, if the goal is to secure convictions "beyond a reasonable doubt."

Photos from Wilson Bentley, 'The Snowflake Man,' 1902


Wretched said...

Great post Scott. There's not enough discussion, it seems, on the true reliability of these forensic sciences. In Texas state courts, at least, even fingerprints are usually seen as irrefutable.

Anonymous said...

Very interesting article. I wonder how many possible points there are (beyond the current 13 points standard).

"The FBI laboratory, which administers the national DNA database system, tried to stop distribution of Troyer's results and began an aggressive behind-the-scenes campaign to block similar searches elsewhere, even those ordered by courts"

Seems FBI has a long history of not being much interested in securing convictions "beyond a reasonable doubt."

Renowned fingerprint expert A. Nathan Darby in 1998 got a 34 point match on Malcolm "Mac" Wallace's figerprint as reportedly found in the School Book Depostory Bldg 6th floor on Nov. 22, 1963.
"The prints, and Darby's analysis, were submitted to the FBI for evaluation. After 18 months had passed, the Bureau released a simple statement that the print match was in error. No analysis accompanied the statement, and no further comment has been made by the FBI on this issue. I think that this verdict, backed by nothing but the (arguably dubious) history of FBI criminal science, is essentially worthless."*


Anonymous said...

What is really a shame about all of this is, the years spent in prison can never be undone. if these person's harbor anger, the public is shocked as they were ultimately freed. But 15 years, or 27 years is a huge loss of time to just forgive away.

So, why can't we do such things to ensure that no longer will a person be wrongfully convicted?

Anonymous said...

There could be some stringent and reasonable internal controls and independent verifications and truly blind testing of samples. They could set higher standards, more accurate normal values, and other stronger scientific "proofs".

All of this could be done now. We could start cutting way down on wrongful convictions now.

There is some kind of dark philosophy coming from on high these days that pervades all of criminal justice. Wrongful convicitions is simply no longer a strong concern.

When the system makes a mistakes and obliterates 25 years of a person's life, they just say "we're sorry."

Anonymous said...

since the criteria is "beyond a reasonable doubt" then some error is still allowed; hence, "irrefutable" cannot be the sole basis for conviction.

Gritsforbreakfast said...

Requiring corroboration for a conviction does not mean making a claim "irrefutable," just backing it up with more than a single piece of evidence, however compelling on its face.

E.g., if a fingerprint at the scene is all you've got and literally not one other admissible piece of evidence, there's a decent chance they have the wrong guy, anyway. My friend Tom had dinner here recently and I'm sure his fingerprints could be found here and there around the house. If a crime were committed, though, finding his print here wouldn't make him culpable.

Mere corroboration of a claim beyond a single source is not too great a burden.

Anonymous said...

Why are the guys from the Yogurt Shop murders still in jail?
Apparently nothing matched their DNA!

Don Dickson said...

....wondering if you saw this in today's NYTimes....

August 8, 2008

Fingerprint Test Shows Not Only Who but What
With a new analytical technique, a fingerprint can now reveal much more than the identity of a person. It can now also identify what the person has been touching — drugs, explosives or poisons, for example.

Writing in Friday’s issue of the journal Science, R. Graham Cooks, a professor of chemistry at Purdue University, and his colleagues describe how a laboratory technique known as mass spectrometry could find a wider application in crime investigations....