DNA mixture software widely used in Texas disallowed by MI federal judge

For some time, Grits has been skeptical whether DNA-mixture software would hold up to "Daubert" scrutiny if judges were to directly evaluate it. In 2015, this blog reluctantly broke the story in Texas about errors and uncertainties regarding how to evaluate DNA mixture evidence. (They were being openly discussed by the Forensic Science Commission and the Court of Criminal Appeals' Criminal Justice Integrity Unit when I worked for the Innocence Project of Texas, but reporters weren't covering the complicated, math-heavy story.) After that, I've followed the issue from afar even after I stopped tracking forensics in any professional capacity.

STR-Mix is one of a handful of companies selling their product as a solution to the conundrum of how to evaluate DNA mixture evidence. Now, a federal district judge in Michigan, Janet Neff, after an exhaustive review of the science, has ruled directly on the Daubert-based admissibility questions surrounding this product that Texas courts punted on. In particular, Judge Neff ruled in favor of defendant David Gissantaner, concluding that

The DNA evidence sought to be admitted in this case—in essence, that it is 49 million times more likely if Daniel Gissantaner is a contributor to the DNA on the gun than if he is not— is not really evidence at all. It is a combination of forensic DNA techniques, mathematical theory, statistical methods (including Monte Carlo-Markov Chain modeling, as in the Monte Carlo gambling venue), decisional theory, computer algorithms, interpretation, and subjective opinions that cannot in the circumstances of this case be said to be a reliable sum of its parts. Our system of justice requires more. 

It is the Court’s hope that this decision brings to light the shortcomings or, at the very least, points of inquiry necessary in evaluating this advancing technology as a tool in forensic DNA analysis. There must be a dialogue among key players in the general interest of the development and refinement of the technology, the software and its application by the individuals charged with its use in the field, rather than post-hoc testing of its reliability in the context of a criminal prosecution where the ultimate question is the freedom and guilt or innocence of the person of interest. 

Here, because the sum of the parts simply does not add up to a reliable whole, the DNA analysis/likelihood ratio resulting from the use of the STRmix probabilistic genotyping software must be excluded. Defendant’s motion to exclude evidence is granted. An Order will enter consistent with this Opinion.

Wow! Obviously, this doesn't apply in Texas nor anywhere in the 5th Circuit. But it's the first time to my knowledge that a federal judge has taken a deep dive into the science behind proprietary DNA mixture software and rejected it based on the Daubert standard. Welcome to the party, Judge! Grits was beginning to feel a bit lonely in that view!

In one-to-one DNA matches, or two-source matches like a rape kit where the DNA of the victim is known, DNA evidence remains the gold standard of forensic science. But analyzing DNA mixture evidence - e.g., "touch DNA" on a doorknob or DNA from a blanket on a couch - remains a more subjective process that's sometimes more of a guess than a "match."

I want to delve into the judge's opinion more deeply before commenting further, but this seems like a big development. We know at least some Texas labs use the same proprietary DNA software the federal judge in Michigan disallowed.

Clarification: More on this soon, but having now more closely read the opinion, Grits should clarify that the court recognized STR-Mix software results could be considered valid for distinguishing up to three-sourced DNA mixture samples, but only if the DNA one is attempting to match makes up at least 20% of the sample; in this case it made up 7%. So saying STR-Mix software was "disallowed" in the headline was too broad. Under this ruling, STR-Mix results for one and two source DNA samples would remain admissible, and for three sources, if the sample one wanted to identify made up at least 20 percent of the total. But it couldn't be used when there were only trace amounts from the third DNA source, or when there are four or more sources. Here is a spot where user expectations for forensics exceed the capabilities of actual science. In many ways, the judge's advice corroborates what experts have said in Texas since this issue first arose: beyond a certain point, DNA mixtures become un-interpretable "crap." You can push the math further, but the evidence doesn't therefore improve.