Sunday, December 30, 2012

'Make forensic evidence meet standards of science'

A Texas A&M professor and former FBI forensic scientist teamed up to author a column in the Austin Statesman today titled "Make forensic evidence meet standards of science." The article opens:
For decades, largely unrealized by judges, many of the forensic practices admitted into judicial proceedings have been without scientific foundation or any other logically acceptable basis other than observational (inductive) study by experimenters untrained in experimental process.

Most troubling are the declarations of certainty associated with proffered expert opinions in those unfounded practices as expressed by examiners in criminal proceedings, such as in firearm/toolmark identification.

Despite the lack of true scientific foundation, firearm/toolmark examiners typically state to a practical certainty that a defendant’s gun was the only possible firearm that could have fired the fatal bullet in a murder. The identification is made on the basis of scratches (striae) and/or impressions on the bullets and comparisons with defendants’ guns.

When examiners confidently declare individualizations (specific source attributions) between crime scene evidence and evidence seized from an eventual defendant without adequate foundational statistical studies, the testimony constitutes nothing more than intuited opinion or speculation, even if an educated guess, rather than evidence-based testimony.

Largely because there has been little or no extrajudicial interest in most forensic practices in past decades other than DNA, forensic procedures such as hair and fiber analyses, bitemarks and comparative bullet lead analysis (CBLA) were developed through empirical induction (observational study) by the practitioners themselves, almost exclusively nonscientists. As noted by a 2009 National Academy of Science committee report, “The fact is that many forensic tests — such as those used to infer the source of firearms or bitemarks — have never been exposed to stringent scientific scrutiny. Most of these techniques were developed in crime laboratories to aid in the investigation of evidence from a particular crime scene, and researching their limitations and foundations was never a top priority.”

Although observational study (induction) is a useful basis for decision-making in everyday life (e.g., purchasing a vehicle or new shoes based on favorable outcomes of past transactions), it is a particularly tenuous process for forensic experimentation or scientific hypothesis testing without appropriate statistical inference.
Read the rest of this informative column here.


  1. Meet standards of science? How ridiculous can we get, to demand that "scientific" evidence be in fact scientific?

  2. A lot of the comparative disciplines - fingerprints, ballistics matching, hair and fiber matching - is "science" only in a euphemistic sense and really amount to subjective comparisons by individuals in law enforcement that cannot be (or at least have not been) substantiated by the scientific method. There are some actually scientific "forensic sciences," like DNA and toxicology, but a lot of it isn't "science" in the sense any actual scientist would use the word.

  3. FYI, many aspects of non-forensic science are also comparative. Anthropologists make observations based on observable differences in skeletal anatomy to make inferences into population origin. Likewise, they may make observations on individual skeletal remains to make inferences on what the life of the person was like that left the remains. Taxonomists make similar observations based on species morphology and use these differences to classify organisms into different species, genera, etc. Geologists may study the structure and elemental composition of material to determine how to classify a mineral. Many more examples could be supplied...

    The comparison of forensic evidence in an attempt to determine a common source is scientific, in that it tests two competing hypotheses. These two hypotheses are that 1) the questioned material (latent fingerprint, fired bullet/cartridge case, shed hair/fiber recovered from a crime scene) either shares a common origin as the known reference or 2) that they do not (i.e. null hypothesis). The issue with some of these comparative analyses is not that the analyses are "unscientific", but more so in the statistical interpretation (i.e. how much weight should be applied to a result).

  4. 1:00, are you comfortable with forensic evidence as unreliable and full of guesswork as anthropologists' inferences about "what the life of the person was like" based on ancient skeletal remains? Educated guesswork is not evidence and I doubt anthropologists would portray such conclusions as "scientific."

    You seem to be saying that comparative forensic sciences are more like social sciences than the hard sciences. That's fine as far as it goes, but it's not how they're portrayed in court. Plus, unlike anthropology, etc., most forensic methods were not developed by scientists and lack the sort of rigor, peer review, etc., that underlies even the softer social sciences.

  5. Moving beyond the (relatively) hard science of toolmarks and the like, there should be a demand for closer scrutiny of the soft junk science areas of mental health testimony, especially psychiatric predictions of future dangerousness in capital cases. Widely acknowledged in the professions to be wrong most of the time, these quack predictions continue to surface in capital sentencing, notwithstanding the Court of Criminal Appeals decisions in the 2010 Coble case where they finally debunked Dr. Richard Coons, one of the primary purveyors of such predictions. Prosecutors do not seem to identify any ethical problem in presenting junk mental health evidence, and the defense bar are doing a lousy job in that area. Time for the CCA to spend some of that grant money, of which they seem to have so much, to delve into this problem?