Fallible Fingerprints: The Dustup over Cognitive Bias

I just ran across a lengthy, excellent article on the sources and frequency of error in fingerprint forensics, published in the online magazine MillerMcCune.com last month: "Bias and the Big Fingerprint Dustup" (by Sue Russell, June 18). Here's how the story opens:

In 2004, cognitive neuroscientist Itiel Dror and Dave Charlton, a veteran fingerprint examiner and doctoral candidate, chatted over coffee in a Brighton hotel suite after a gala dinner at the U.K. Fingerprint Society's conference. Charlton was upset. Months earlier, [Dr. Itiel] Dror had designed a study to see if fingerprint examiners' decisions on matches might unconsciously be biased by information they received about a case.

Would examiners change their opinions about prints they'd called matches five years earlier, Dror wondered, if they viewed them again in a different context?

Charlton, supervisor of a U.K. police department's fingerprint lab, editor of the Fingerprint Society's journal Fingerprint Whorld and a true believer, was certain they would not. He urged Dror not to waste his time.

But Dror was insistent: "I said, 'Indulge me! Let's do it.'" And so, five international experts were put to the test covertly, re-examining matched prints from their own old cases while armed with different — and potentially biasing — "case information." They'd agreed to be tested, but they didn't know when — or even if — test prints would cross their desks.

That night in Brighton, the results were in. For Charlton, they were a jaw-dropper.

"Not only some, but most, of the fingerprint examiners changed their minds," said Dror, who was far less surprised by the flip-flopping. As an expert in human thought processes and the hidden power of cognitive bias — an array of distortions in the way humans perceive reality — he had a decided advantage.

Fingerprints have been accepted as unassailable evidence in courts for more than 100 years, but vaunted claims of their uniqueness and infallibility still lack scientific validation. By contrast, the existence of cognitive bias and the subjective nature of human judgment have been thoroughly established by hundreds of scientific studies going back decades.

What's more, the experiment was replicable with different fingerprint examiners using different fact scenarios:

In another study, [Dror's] team had six international experts each view eight latent prints that they'd each previously examined, but now they were accompanied with a new, mundane context — something like, "the suspect has confessed," or, "the suspect is in custody." More expert reversals followed. Four of the six reached different conclusions. One changed his mind three times.

Indeed, this isn't just a bias in experimental situations but occasionally in the real world:

in a landmark U.S. case, Stephan Cowans of Boston became the first person to be convicted on fingerprint evidence, then — after serving six years in prison for shooting a police officer — exonerated by DNA. Two prosecution experts and Cowans' two defense experts (formerly of the same fingerprint unit) had all verified the match. After his 2004 release, Cowans revealed his earlier certainty about fingerprints by saying that on the evidence presented in court, he would have voted to convict himself.

What can be done to reduce cognitive bias among fingerprint examiners and other forensic workers? Dror advocates removing forensics from the purview of law enforcement and severely limiting the amount of contextual information given to forensic workers about individual cases:

A key National Academy of Science report recommendation — to move crime labs out from under law enforcement's wing and create a new national institute of forensic sciences — would surely help impartiality. If lack of funding delays that, "so be it," Dror said. "But you can't have it both ways. If there's no reform, don't say, 'I am 100 percent objective, I make no mistakes, there is no problem.'"

In the interim, some steps can be taken. When further examiners are called on to verify the work of a first, they should always examine the evidence independently without knowing the earlier results.

Efficiency, scientific validity and objectivity could also be dramatically improved for a relatively small financial outlay by establishing and enforcing "best practices" in crime labs (another NAS report recommendation.) Best practices are formally documented standard operating procedures, processes and rules for how to do your work that are specifically designed to make it effective and efficient, and avoid error. Having best practices that all fingerprint examiners everywhere must adhere to would be a big step forward, Dror believes, but only if they are science-based and validated by experts in cognitive neuroscience, psychology and thought processes.

Today, guidelines are provided by Scientific Working Group on Friction Ridge Analysis, Study and Technology. However, these are only guidelines with no mechanism of enforcement. "What is more," Dror said, "none of the current guidelines really directly and adequately deals with confirmation bias and other cognitive issues." ...

He favors the immediate implementation of the practice of withholding all nonessential crime details from forensic scientists. Detectives, investigators, lawyers, judge and jury need to know if fingerprints are related to terrorism or bicycle theft, but for fingerprint examiners counting ridge characteristics, loops, whorls and other minutiae, such context is irrelevant.

"We're not going to send a fingerprint to Interpol if somebody stole a bike," Dror said. "But we need to make sure the fact that the examiner thinks it's a terrorist or the Madrid bomber doesn't cloud their judgment too much." To Dror, it's like his personal physician needing his medical history, while the lab technician counting his white blood cells for a blood test does not.

The issue of cognitive bias comes up again and again in discussions of forensic errors. While it seems like it should be the easiest problem to fix - managing who gets what information - the reaction to such suggestions from forensic workers tend to range from defensive to openly hostile, as this excellent Miller-McCune article demonstrates. In the wake of the National Academy of Sciences report, we're going to see a revamping in the near future of methodologies in many forensic fields, and removing sources of cognitive bias must surely be a key component of any such reforms.