Coming to grips with unscientific forensic practices

The Fort Worth Star Telegram yesterday published a lengthy, remarkable piece from Yamil Berard titled "Stakes are high as doubt is cast on forensic lab techniques." The article is one of the first published in the MSM in Texas to fully explore the implications of conclusions published earlier this year by the National Academies of Science (NAS) that many common forensic techniques had no scientific basis and their validity had never been tested - particularly comparative disciplines where individual technicians seek to match patterns in everything from fingerprints to tire tracks to ballistics to bite marks. Reports Berard:

The Texas Forensic Science Commission had planned this month to begin a series of discussions about a national report that opened a Pandora’s box of questions about crime lab techniques. The National Academy of Sciences — advisers to Congress and the president — reported that conclusions about bullet matching are opinion, not fact. Most other identification methods widely used by forensic scientists, the panel advised, also haven’t been validated.
How did a scientifically unproven method receive the blessing of the FBI and forensic "experts" across the nation and other crime lab methods become so widely accepted?
"In a nutshell, these people aren’t scientists," said Jay A. Siegel, a member of the academy, which was established by President Abraham Lincoln to advise the nation on far-reaching questions of science and technology. "They don’t know what validation is. They don’t know what it means to validate a test."
Bullet matching — a practice that takes place every day in Texas crime labs — isn’t reliable, Siegel said, and no studies have been conducted to prove the extent to which firearms marks are unique.
"It’s not possible to state with any scientific certainty that this bullet came from any weapon in the world," said Siegel, who is the chairman of the department of chemistry and chemical biology at Indiana University-Purdue University Indianapolis and director of the Forensic and Investigative Sciences Program.
Other identification methods — including tying hairs to suspects, guns to criminals and blood spatter to crime scenes — lack protocols and standards that legitimize such practices as "scientific."
Of the methods, only DNA analysis stands up to the test to match an individual to crime evidence, the scientists reported.

One of the Forensic Science Commissioners Gov. Perry recently declined to reappoint told the Startlegram that ""The NAS opened the [Pandora’s] box, and now we just need to face it." Everyone interested should read the whole piece.

Berard predicts that, if the Forensic Science Commission restarts its plans to sponsor discussions around the state about the implications of the NAS report, "criminalists and crime lab directors across Texas will likely blast the academy’s report, 'Strengthening Forensic Science in the U.S.: A Path Forward.'" I've not heard anyone "blast" the findings per se, but maybe that's coming. (A discussion of the NAS report appears on the first day's agenda of an upcoming conference in Fort Worth on "Current Trends in Forensic Science," so it's possible that might be a place where reservations are more frankly expressed.)
Relatedly, the Waco Tribune-Herald had a story yesterday on the current state of arson science and the impact of critiques of outdated forensics in the Todd Willingham case on the field of arson investigation ("Local fire investigators confident in arson training techniques and welcome scrutiny," Nov. 29) According to reporter Cindy Culp:

Critics of Texas’ fire investigation standards worry that some investigators may still cling to the old wives’ tales, however. They also express concern about investigators not being required to have a science background, saying it is needed to truly understand fire behavior.
In Texas, people can become fire investigators as long as they have a high school diploma, complete 150 hours of training and pass a certification exam. To investigate arsons, they also have to become certified as a peace officer, which generally involves going through a police academy.
Central Texas officials who conduct fire investigations say there may be truth to some of the criticism. But, by and large, investigators do a good job, they say. ...
[Killeen Fire Marshal James] Chism said the controversy over the Willingham case is a good chance for the profession to look at old investigations where questionable techniques may have been used. The industry also needs to seize the opportunity to weed out investigators who cling to outdated beliefs. It would be naive to think none exist, he said. “Whether it is an old-school mentality or sheer laziness because it’s what they’ve always done, I still have to think those old wives’ tales are still getting play in the state of Texas,” Chism said.
In the end, Chism predicted, public confidence in fire investigations will increase.

Over the next five to ten years we're going to see an historic, first-of-its-kind evaluation of so-called forensic "science" disciplines that are really subjective, untested and in some cases inherently unscientific.
That's the real concern with John Bradley shutting down the Forensic Science Commission debate over the Todd Willingham case. As the Killeen fire marshal says, it would be naive to believe other investigators aren't still coming up with arson findings based on assumptions that aren't supported by science. But even more importantly, there are many other forensic disciplines that we now know are equally deserving of open, honest scrutiny. Unless Mr. Bradley does an unexpectedly bang-up job that assuages widespread concerns that he's engaged in a coverup, the main entity created by the state to spearhead such evaluations won't have the public confidence required to be seen as a neutral arbiter.

Related Grits posts:

• NAS Report: Many forensic disciplines prone to error
• Justice system commonly relies on shoddy forensic science
• USDOJ should embrace, not fight science-based forensics
• Shoddy forensic science ruins real people's lives
• Background materials on shoddy forensics
• Arson cases may be next venue for innocence claims
• Many arson convictions based on invalid science
• College developing screening processes to vet old arson cases for junk science