Saturday, September 05, 2015
A reluctant scoop: Changing intepretations of DNA mixtures vex legal system
DNA has often been referred to as the gold standard of forensic evidence. And when comparing a single sample to a suspect, that's true. When DNA evidence includes mixtures of DNA from multiple sources, however, the scientific community is still figuring things out. And it takes years for new knowledge to pass down from the highest-end basic research to the work tables of practicing DNA analysts at front-line crime labs.
The most recent such development: improvements in DNA science and interpretation techniques have caused practitioners to change how they calculate probabilities when it comes to singling out a defendant based on DNA mixtures (i.e., when biological evidence includes samples from more than one person). In the most extreme instance, the new method reduced a one in a billion probability that evidence matched a particular suspect to around one in 50, the Texas Forensic Science Commission's Lynn Garcia told the Texas Criminal Justice Integrity Unit (TCJIU) on Wednesday. The issue was first publicly revealed in Texas and, to their credit, the courts and relevant agencies have been quick to confront the challenge, even if nobody quite knows yet what to do.
Grits has put off writing about this topic for a while, mainly because I hoped the MSM would pick up the story - it's a big one, with national and even international implications - and thus do me the favor of not having to write up a complicated issue (this blog is a hobby for me, after all). But it doesn't appear anyone else will cover it, so it falls to your correspondent to break the news.
The hows and whys of the shift in DNA analysis are complex and sciency. My own math acumen and knowledge of DNA testing is such that, while I can mostly follow what's being said, and have learned much of the vocabulary, I can't pretend to fully understand. So I won't put on airs by attempting to explain what little I do know. It's damn complicated, which is partly why it's taken so long for the issue to percolate to the fore.
At the TCJIU, Garcia told us that the world's leading experts, five of whom will advise Texas on how to address the matter, were all meeting in Poland last week hashing through many of these same topics. One of those experts, Dr. Bruce Budowle, teaches at the UNT Health Science Center and has been providing technical support to the FSC, the Court of Criminal Appeals, and the state prosecutor and defense lawyer associations (for whom he put on webinars ten days ago). Some of the issues which came up in the impromptu discussion at the TCJIU - e.g., must samples be re-tested or may the math just be re-done - are among those on which the the world's leading scientific minds as yet have no firm opinion, said Garcia. We're entering entirely uncharted waters.
The DNA-mixture issue was first publicly discussed at the August 14th Forensic Science Commission meeting, then formally announced in a blandly titled notice issued Aug. 21st by the Texas Forensic Science Commission which landed like a bombshell within the highest echelons of Texas legal circles. Titled, "Notification re: FBI Allele Frequency Corrections and DNA Mixture Interpretation Issues in Texas," the document's dry language revealed a new bleeding-edge conundrum facing the legal system's use of ostensibly gold-standard forensics.
This all began when the FBI identified a handful of errors in its CODIS DNA database (51 problems out of tens of thousand of entries) and issued a public notice saying that the change in any probabilities affected would be insignificant. Garcia said the largest discrepancy estimated from the data-entry errors would have reduced a likelihood of one in 260 billion that it belonged to another person to one in 225 billion. With seven billion people on the planet, that looked like a yawner.
But some Texas prosecutors asked for their probabilities to be recalculated, anyway, and when they came back, they were affected to a far greater extent than had previously been portrayed. The most radical difference involved the case mentioned above where a one-in-a-billion probability was lowered to around one-in-fifty, said Garcia. At that, prosecutors and the FSC perked up and took notice.
It turned out, the difference stemmed not from the data entry errors but because - in response to all that complex and sciency stuff that I'm not going to attempt to explain here - the FBI had changed its methodology for calculating probabilities in mixed DNA samples and moved to a new method which they considered more accurate. According to the FSC notice, "Changes in mixture interpretation have occurred primarily over the last 5-10 years and were prompted by several factors, including but not limited to mixture interpretation guidance issued in 2010 by the [national] Scientific Working Group on DNA analysis" (see here).
When they changed methodologies, though, scientists didn't think to notify prosecutors, much less the defense bar. Apparently, nobody wondered, "What happens when you apply the new method to old cases?," much less, "What if a defendant included by the old method is excluded or undetermined under the new one"?
Those are the questions the wider legal community is struggling with now, at the moment in Texas but very soon across the country and everywhere on the planet where DNA evidence is used in courts.
So many other questions are raised, it boggles the mind to consider. No one knows how many cases are involved, but the fear is it's a lot: Many more than in the FSC's hair microscopy review, which is itself expected to take years.
Further, Texas' mechanism for notifying defendants about past forensic errors has proven inadequate. And while people may be entitled to court-appointed lawyers to get DNA retested or the results recalculated, the state has no viable mechanism for matching lawyers to defendants and, anyway, few criminal defense lawyers have experience filing habeas corpus writs - a point Court of Criminal Appeals Judge Barbara Hervey emphasized at the TCJIU. Many if not most counties outside the major urban centers don't have anyone in the local bar with significant post-conviction writ experience.
Texas is arguably better situated than most states to address these questions. The Forensic Science Commission provides an important and trusted clearinghouse that stakeholders from both sides of the bar and the scientific community, for the most part, all consider a fair broker. Indeed, that's the main reason why Texas is the first state to address this: The FSC is out there actively scouting these issues, not sweeping them under the rug.
Moreover, our DNA testing statute is one of the most robust in the country. And the state's new junk science writ provides a solid basis for litigating forensics claims, particularly those as in this case where the issue stems from advancements in science rather than errors or misconduct.
Finally, this session the Legislature expanded the jurisdiction of the Office of Capital Writs to make it the Office of Capital and Forensic Writs, presaging exactly this sort of development. (Grits supported this idea, which was dubbed by critics the "OCW Empire Expansion Act.") Regrettably, though, no extra money came with that added mandate. And with the small agency recently changing leadership and facing a series of intense deadlines in numerous capital cases this fall, wizened observers fear handing the agency any new duties related to forensic writs would utterly overwhelm it.
Luckily, the Texas Indigent Defense Commission for the first time this biennium was given a new pot of money which could fund this work through the OCFW if that's what state leaders decide to do. The commission next meets in December. Between them and the Governor's office, which Garcia said had already stepped up to the plate with funding from the Criminal Justice Division to help with the initial reaction, there should be resources to at least triage the issue and create a plan, which would be an improvement from where we are now.
All that to say, Texas has a few more tools in the toolbox than most other states to address this cluster$#@k. But it's still a cluster$#@k.
RELATED: Go here to view Dr. Budowle's presentation to the Forensic Science Commission's Aug. 14th meeting, beginning at the 2:04:30 mark. The Commission's working group on DNA mixtures will meet in Dallas on Sept. 18.
AND MORE: The Marshall Project linked to this post in their 9/8 daily email and referred to one of their own recent articles, "The Surprisingly Imperfect Science of DNA Testing" which hones in on some of these mixture issues with a dramatic case study.
The most recent such development: improvements in DNA science and interpretation techniques have caused practitioners to change how they calculate probabilities when it comes to singling out a defendant based on DNA mixtures (i.e., when biological evidence includes samples from more than one person). In the most extreme instance, the new method reduced a one in a billion probability that evidence matched a particular suspect to around one in 50, the Texas Forensic Science Commission's Lynn Garcia told the Texas Criminal Justice Integrity Unit (TCJIU) on Wednesday. The issue was first publicly revealed in Texas and, to their credit, the courts and relevant agencies have been quick to confront the challenge, even if nobody quite knows yet what to do.
Grits has put off writing about this topic for a while, mainly because I hoped the MSM would pick up the story - it's a big one, with national and even international implications - and thus do me the favor of not having to write up a complicated issue (this blog is a hobby for me, after all). But it doesn't appear anyone else will cover it, so it falls to your correspondent to break the news.
The hows and whys of the shift in DNA analysis are complex and sciency. My own math acumen and knowledge of DNA testing is such that, while I can mostly follow what's being said, and have learned much of the vocabulary, I can't pretend to fully understand. So I won't put on airs by attempting to explain what little I do know. It's damn complicated, which is partly why it's taken so long for the issue to percolate to the fore.
At the TCJIU, Garcia told us that the world's leading experts, five of whom will advise Texas on how to address the matter, were all meeting in Poland last week hashing through many of these same topics. One of those experts, Dr. Bruce Budowle, teaches at the UNT Health Science Center and has been providing technical support to the FSC, the Court of Criminal Appeals, and the state prosecutor and defense lawyer associations (for whom he put on webinars ten days ago). Some of the issues which came up in the impromptu discussion at the TCJIU - e.g., must samples be re-tested or may the math just be re-done - are among those on which the the world's leading scientific minds as yet have no firm opinion, said Garcia. We're entering entirely uncharted waters.
The DNA-mixture issue was first publicly discussed at the August 14th Forensic Science Commission meeting, then formally announced in a blandly titled notice issued Aug. 21st by the Texas Forensic Science Commission which landed like a bombshell within the highest echelons of Texas legal circles. Titled, "Notification re: FBI Allele Frequency Corrections and DNA Mixture Interpretation Issues in Texas," the document's dry language revealed a new bleeding-edge conundrum facing the legal system's use of ostensibly gold-standard forensics.
This all began when the FBI identified a handful of errors in its CODIS DNA database (51 problems out of tens of thousand of entries) and issued a public notice saying that the change in any probabilities affected would be insignificant. Garcia said the largest discrepancy estimated from the data-entry errors would have reduced a likelihood of one in 260 billion that it belonged to another person to one in 225 billion. With seven billion people on the planet, that looked like a yawner.
But some Texas prosecutors asked for their probabilities to be recalculated, anyway, and when they came back, they were affected to a far greater extent than had previously been portrayed. The most radical difference involved the case mentioned above where a one-in-a-billion probability was lowered to around one-in-fifty, said Garcia. At that, prosecutors and the FSC perked up and took notice.
It turned out, the difference stemmed not from the data entry errors but because - in response to all that complex and sciency stuff that I'm not going to attempt to explain here - the FBI had changed its methodology for calculating probabilities in mixed DNA samples and moved to a new method which they considered more accurate. According to the FSC notice, "Changes in mixture interpretation have occurred primarily over the last 5-10 years and were prompted by several factors, including but not limited to mixture interpretation guidance issued in 2010 by the [national] Scientific Working Group on DNA analysis" (see here).
When they changed methodologies, though, scientists didn't think to notify prosecutors, much less the defense bar. Apparently, nobody wondered, "What happens when you apply the new method to old cases?," much less, "What if a defendant included by the old method is excluded or undetermined under the new one"?
Those are the questions the wider legal community is struggling with now, at the moment in Texas but very soon across the country and everywhere on the planet where DNA evidence is used in courts.
So many other questions are raised, it boggles the mind to consider. No one knows how many cases are involved, but the fear is it's a lot: Many more than in the FSC's hair microscopy review, which is itself expected to take years.
Further, Texas' mechanism for notifying defendants about past forensic errors has proven inadequate. And while people may be entitled to court-appointed lawyers to get DNA retested or the results recalculated, the state has no viable mechanism for matching lawyers to defendants and, anyway, few criminal defense lawyers have experience filing habeas corpus writs - a point Court of Criminal Appeals Judge Barbara Hervey emphasized at the TCJIU. Many if not most counties outside the major urban centers don't have anyone in the local bar with significant post-conviction writ experience.
Texas is arguably better situated than most states to address these questions. The Forensic Science Commission provides an important and trusted clearinghouse that stakeholders from both sides of the bar and the scientific community, for the most part, all consider a fair broker. Indeed, that's the main reason why Texas is the first state to address this: The FSC is out there actively scouting these issues, not sweeping them under the rug.
Moreover, our DNA testing statute is one of the most robust in the country. And the state's new junk science writ provides a solid basis for litigating forensics claims, particularly those as in this case where the issue stems from advancements in science rather than errors or misconduct.
Finally, this session the Legislature expanded the jurisdiction of the Office of Capital Writs to make it the Office of Capital and Forensic Writs, presaging exactly this sort of development. (Grits supported this idea, which was dubbed by critics the "OCW Empire Expansion Act.") Regrettably, though, no extra money came with that added mandate. And with the small agency recently changing leadership and facing a series of intense deadlines in numerous capital cases this fall, wizened observers fear handing the agency any new duties related to forensic writs would utterly overwhelm it.
Luckily, the Texas Indigent Defense Commission for the first time this biennium was given a new pot of money which could fund this work through the OCFW if that's what state leaders decide to do. The commission next meets in December. Between them and the Governor's office, which Garcia said had already stepped up to the plate with funding from the Criminal Justice Division to help with the initial reaction, there should be resources to at least triage the issue and create a plan, which would be an improvement from where we are now.
All that to say, Texas has a few more tools in the toolbox than most other states to address this cluster$#@k. But it's still a cluster$#@k.
RELATED: Go here to view Dr. Budowle's presentation to the Forensic Science Commission's Aug. 14th meeting, beginning at the 2:04:30 mark. The Commission's working group on DNA mixtures will meet in Dallas on Sept. 18.
AND MORE: The Marshall Project linked to this post in their 9/8 daily email and referred to one of their own recent articles, "The Surprisingly Imperfect Science of DNA Testing" which hones in on some of these mixture issues with a dramatic case study.
Labels:
DNA,
Forensic Errors,
Forensic Science Commission
Subscribe to:
Post Comments (Atom)
8 comments:
"This finding does not mean laboratories or individual analysts did anything wrong intentionally or even knew the approaches fell outside the bounds of scientific acceptability, but rather the community has progressed over time in its ability to understand and implement this complex area of DNA interpretation appropriately."
This is just poor scientific etiquette. In NO WAY is this the lab analysts fault, and to presume such is poor taste. The lab analyts are simply following "code reds" protocols, poorly written by the DNA lab supervisors.
"The forensic DNA community has been aware of substantial variance in mixture interpretation among laboratories since at least 2005 when the National Institute of Standards and Technology (“NIST”) first described the issue in an international study called MIX05."
And it only took TEN YEARS to formally notify stakeholders? Peerwani, Barnard, DiMaio all run DNA labs -- did they do anything "intentionally wrong" in the past ten years? For those labs that have been using incorrect statistical procedures, they "did something wrong" intentionally and should be held accountable for being lazy and stupid (and training their lab analysts as such).
And where the f*ck is ASCLD/LAB? Where is the DPS? Aren't these Texas labs accredited and considered to have picture perfect protocols (at least that's what the lab analyst testify to in a court room in front of a jury)? How can these DNA labs acquire accreditation when it is known that the statistical protocols are erroneous? Dare I say that the agencies are simply rubber-stamping accreditations for these lab, and collecting the hefty fees (paid by taxpayers)? Has any ever seen an ASCLD/LAB auditor testify as to the work they've done? No? I didn't think so. They don't believe in accountability.
The "Oh, well" attitude among the "elite forensic DNA community" is vomit inducing.
No tummy rubs for those that are "trying".
This is way over my head, but I was still trying to think ab it a few months ago while reading a book about how geneticists were using DNA extracted from very old bones to identify how humans mated in the very distant past with Neanderthls and other hominids.
One geneticist said when he found DNA in one bone, he was skeptical, thinking it might have been some of his own or one of his assistants', for a hair that got mixed in or something.
I thought about how there you had one of the world's foremost geneticists in pristine lab quarters, and he wasn't sure. And the county's crew was probably not exactly Paabo Svante.
This is intriguing and it's really only taken off the past few years. Thanks for posting this!
On a side note, the forensic scientist responsible for helping cops and prosecutors in Oklahoma frame thousands of innocents has died: http://www.okcfox.com/story/29910952/former-forensic-scientist-accused-of-forging-evidence-in-hundreds-of-cases-dies#.VeJL_AHhTJI.twitter
Many of the existing forensic DNA identifications will be undermined in the next 20 years... for reasons not contemplated in this particular blog post.
I am certainly not smart enough to understand the science of it. I have 2 questions:
1. Does this mean hundreds maybe thousands of cases are going to be questioned?
2. Do you have something you want us to print? Agreed we are not MSM but it is a start.
The calculations will have to be done for that many, THC, not sure that many will ultimately be "questioned."
Most of what I know of the story is linked in the post, you're welcome to print what you like.
@Trigger, thanks, I hadn't seen that and had nearly forgotten about her.
@1:29, please elaborate.
This happened in Ohio. The DNA testimony was an inclusion result at trial. At the hearing on the motion for new trial, the DNA testimony was changed to an exclusion result. The trial court and court of appeals both dismissed the incongruent testimony as a change in semantics. State v. Zeddrick Mann, 2015-Ohio-3151 available at: http://www.supremecourt.ohio.gov/rod/docs/pdf/10/2015/2015-Ohio-3151.pdf
9:45 does not correctly represent the findings of the appellate court. The DNA interpretation did not change to exclusion, but to uninterpretable.
Post a Comment