A reluctant scoop: Changing intepretations of DNA mixtures vex legal system

DNA has often been referred to as the gold standard of forensic evidence. And when comparing a single sample to a suspect, that's true. When DNA evidence includes mixtures of DNA from multiple sources, however, the scientific community is still figuring things out. And it takes years for new knowledge to pass down from the highest-end basic research to the work tables of practicing DNA analysts at front-line crime labs.

The most recent such development: improvements in DNA science and interpretation techniques have caused practitioners to change how they calculate probabilities when it comes to singling out a defendant based on DNA mixtures (i.e., when biological evidence includes samples from more than one person). In the most extreme instance, the new method reduced a one in a billion probability that evidence matched a particular suspect to around one in 50, the Texas Forensic Science Commission's Lynn Garcia told the Texas Criminal Justice Integrity Unit (TCJIU) on Wednesday. The issue was first publicly revealed in Texas and, to their credit, the courts and relevant agencies have been quick to confront the challenge, even if nobody quite knows yet what to do.

Grits has put off writing about this topic for a while, mainly because I hoped the MSM would pick up the story  - it's a big one, with national and even international implications - and thus do me the favor of not having to write up a complicated issue (this blog is a hobby for me, after all). But it doesn't appear anyone else will cover it, so it falls to your correspondent to break the news.

The hows and whys of the shift in DNA analysis are complex and sciency. My own math acumen and knowledge of DNA testing is such that, while I can mostly follow what's being said, and have learned much of the vocabulary, I can't pretend to fully understand. So I won't put on airs by attempting to explain what little I do know. It's damn complicated, which is partly why it's taken so long for the issue to percolate to the fore.

At the TCJIU, Garcia told us that the world's leading experts, five of whom will advise Texas on how to address the matter, were all meeting in Poland last week hashing through many of these same topics. One of those experts, Dr. Bruce Budowle, teaches at the UNT Health Science Center and has been providing technical support to the FSC, the Court of Criminal Appeals, and the state prosecutor and defense lawyer associations (for whom he put on webinars ten days ago). Some of the issues which came up in the impromptu discussion at the TCJIU - e.g., must samples be re-tested or may the math just be re-done - are among those on which the the world's leading scientific minds as yet have no firm opinion, said Garcia. We're entering entirely uncharted waters.

The DNA-mixture issue was first publicly discussed at the August 14th Forensic Science Commission meeting, then formally announced in a blandly titled notice issued Aug. 21st by the Texas Forensic Science Commission which landed like a bombshell within the highest echelons of Texas legal circles. Titled, "Notification re: FBI Allele Frequency Corrections and DNA Mixture Interpretation Issues in Texas," the document's dry language revealed a new bleeding-edge conundrum facing the legal system's use of ostensibly gold-standard forensics.

This all began when the FBI identified a handful of errors in its CODIS DNA database (51 problems out of tens of thousand of entries) and issued a public notice saying that the change in any probabilities affected would be insignificant. Garcia said the largest discrepancy estimated from the data-entry errors would have reduced a likelihood of one in 260 billion that it belonged to another person to one in 225 billion. With seven billion people on the planet, that looked like a yawner.

But some Texas prosecutors asked for their probabilities to be recalculated, anyway, and when they came back, they were affected to a far greater extent than had previously been portrayed. The most radical difference involved the case mentioned above where a one-in-a-billion probability was lowered to around one-in-fifty, said Garcia. At that, prosecutors and the FSC perked up and took notice.

It turned out, the difference stemmed not from the data entry errors but because - in response to all that complex and sciency stuff that I'm not going to attempt to explain here - the FBI had changed its methodology for calculating probabilities in mixed DNA samples and moved to a new method which they considered more accurate. According to the FSC notice, "Changes in mixture interpretation have occurred primarily over the last 5-10 years and were prompted by several factors, including but not limited to mixture interpretation guidance issued in 2010 by the [national] Scientific Working Group on DNA analysis" (see here).

When they changed methodologies, though, scientists didn't think to notify prosecutors, much less the defense bar. Apparently, nobody wondered, "What happens when you apply the new method to old cases?," much less, "What if a defendant included by the old method is excluded or undetermined under the new one"?

Those are the questions the wider legal community is struggling with now, at the moment in Texas but very soon across the country and everywhere on the planet where DNA evidence is used in courts.

So many other questions are raised, it boggles the mind to consider. No one knows how many cases are involved, but the fear is it's a lot: Many more than in the FSC's hair microscopy review, which is itself expected to take years.

Further, Texas' mechanism for notifying defendants about past forensic errors has proven inadequate. And while people may be entitled to court-appointed lawyers to get DNA retested or the results recalculated, the state has no viable mechanism for matching lawyers to defendants and, anyway, few criminal defense lawyers have experience filing habeas corpus writs - a point Court of Criminal Appeals Judge Barbara Hervey emphasized at the TCJIU. Many if not most counties outside the major urban centers don't have anyone in the local bar with significant post-conviction writ experience.

Texas is arguably better situated than most states to address these questions. The Forensic Science Commission provides an important and trusted clearinghouse that stakeholders from both sides of the bar and the scientific community, for the most part, all consider a fair broker. Indeed, that's the main reason why Texas is the first state to address this: The FSC is out there actively scouting these issues, not sweeping them under the rug.

Moreover, our DNA testing statute is one of the most robust in the country. And the state's new junk science writ provides a solid basis for litigating forensics claims, particularly those as in this case where the issue stems from advancements in science rather than errors or misconduct.

Finally, this session the Legislature expanded the jurisdiction of the Office of Capital Writs to make it the Office of Capital and Forensic Writs, presaging exactly this sort of development. (Grits supported this idea, which was dubbed by critics the "OCW Empire Expansion Act.") Regrettably, though, no extra money came with that added mandate. And with the small agency recently changing leadership and facing a series of intense deadlines in numerous capital cases this fall, wizened observers fear handing the agency any new duties related to forensic writs would utterly overwhelm it.

Luckily, the Texas Indigent Defense Commission for the first time this biennium was given a new pot of money which could fund this work through the OCFW if that's what state leaders decide to do. The commission next meets in December. Between them and the Governor's office, which Garcia said had already stepped up to the plate with funding from the Criminal Justice Division to help with the initial reaction, there should be resources to at least triage the issue and create a plan, which would be an improvement from where we are now.

All that to say, Texas has a few more tools in the toolbox than most other states to address this cluster$#@k. But it's still a cluster$#@k.

RELATED: Go here to view Dr. Budowle's presentation to the Forensic Science Commission's Aug. 14th meeting, beginning at the 2:04:30 mark. The Commission's working group on DNA mixtures will meet in Dallas on Sept. 18.

AND MORE: The Marshall Project linked to this post in their 9/8 daily email and referred to one of their own recent articles, "The Surprisingly Imperfect Science of DNA Testing" which hones in on some of these mixture issues with a dramatic case study.

DNA Forensics Roundup

Several recent DNA-related items deserved Grits readers attention.

Do DNA exonerations merit death penalty moratorium?
Twenty men exonerated by DNA evidence all gathered in Austin on Friday to call for a moratorium on Texas' death penalty in light of the state's recent slew of long-time inmates proven innocent by applying modern forensics to old evidence, including most recently the exoneration by DNA of Michael Blair who'd been sitting on Texas' death row. Reported the Austin Statesman:

The exonerated men, members of Witness to Innocence, a Philadelphia-based organization that is holding its annual meeting in Austin, want Texas to create a commission to search for wrongful convictions. And while the commission works, they want a moratorium on executions in the busiest death penalty state — with 419 executions since 1982 and six more scheduled this month. ...

Sam Millsap, former Bexar County district attorney, said he slowly came to believe that the death penalty must be abolished because of the growing number of exonerated death row inmates — 130 since 1973, including nine in Texas, according to the Death Penalty Information Center. "I am no longer convinced that our courts will in fact guarantee the protection of the innocent," Millsap said.

Millsap said he has taken responsibility for the 1993 execution of Ruben Cantu , a San Antonio man who Millsap said might have been innocent of a 1984 murder. The conviction was based on one eyewitness who later recanted, and no physical evidence tied Cantu to the crime, he said. "My decision to seek the death penalty was a mistake."

The most recent Texas exoneration was in September , when a Collin County court dismissed the capital murder case against Michael Blair , sentenced to die for the 1993 murder of 7-year-old Ashley Estell.

However, the reporter (perhaps rightly) warned the exonerees:

any bill to halt executions stands no chance of passing the Texas Legislature, [Austin state Representative Elliott] Naishtat said. Capital punishment has substantial support in Texas. The 2007 Texas Crime Poll by Sam Houston State University found 74 percent of Texans support the death penalty. And 66 percent said they were confident that innocent people are protected from execution.

Research underlying DNA forensics began in Houston
The Houston Chronicle published a story commemorating the 20th anniversary of a Harris County case that launched the international boom in the use of DNA in forensic science and interviewing Dr. Tom Caskey, whose patents on the early technology still fund research positions at the Baylor College of Medicine:

"Don't ever say crime doesn't pay. Crime does pay," quipped Caskey, who now directs the Brown Foundation Institute of Molecular Medicine for the Prevention of Human Diseases, a part of the University of Texas Health Science Center at Houston.

Caskey's early work on a Houston homicide case became the foundation for today's national network of criminal DNA sampling, reported the Chronicle's Eric Berger:

Caskey's primary scientific interest at the time involved the identification of genes linked to human disease. But, during the course of his research, he identified short segments of DNA — called short tandem repeats, or STRs — that vary widely from person to person.

His lab developed an STR identification technique that soon became the gold standard for criminal cases.

The method underlies the U.S. Combined DNA Index System, or CODIS, which contains DNA information on more than 6 million offenders. The system also forms the basis of Interpol's criminal database and that of most governments.

A constitutional right to DNA testing?

The US Supreme Court will soon decide whether a federal right exists to have DNA evidence tested that could prove a long-ago convicted defendant's innocence, even if he confessed to the crime:

William Osborne was accused of raping a prostitute at gunpoint, beating her with an ax handle and leaving her for dead in the snow. His lawyer declined a DNA test of the evidence, thinking that it would confirm his guilt.

Osborne was convicted, spent more than a decade in prison and gave a detailed confession to a parole board. But after recanting that confession, the Alaska man won a federal lawsuit seeking new DNA tests that he now says can clear him, a judgment that was affirmed by the U.S. Court of Appeals for the 9th Circuit. It is the first time an appellate court has ruled that an inmate has a federal constitutional right to such testing.

Now, the Supreme Court is being asked to evaluate that ruling in a case that pits the administration of Alaska Gov. Sarah Palin, the Republican vice presidential nominee, against a Republican-appointed judge who accuses her state attorney general of being "obstinate" in blocking Osborne from getting to test the evidence used to convict him. The high court debated Alaska's request to take the case in a private conference on Friday and could announce its decision as early as today.

Touch DNA evidence could overwhelm crime labs
Finally, if you're not yet familiar with the concept of "touch DNA," be sure to check out this piece from the back pages of Scientific American which answers the question:

So what’s touch DNA?

The touch DNA method—named for the fact that it analyzes skin cells left behind when assailants touch victims, weapons or something else at a crime scene—has been around for the last five years. In fact, the prosecutor in the Ramsey case, Boulder County District Attorney Mary Lacy, learned about touch DNA when she attended a course here at the West Virginia University Forensic Science Initiative in the summer of 2007.

The technique has dramatically increased the number of items of evidence that can be used for DNA detection. In the 1980s, in order to perform DNA analysis on a crime scene or victim, forensic investigators needed a blood or semen stain about the size of a quarter. The sample size fell in the 1990s to the size of a dime and then became: “If you can see it, you can analyze it.”

Touch DNA doesn’t require you to see anything, or any blood or semen at all. It only requires seven or eight cells from the outermost layer of our skin.

Here’s how it works: Investigators recover cells from the scene, then use a process called polymerase chain reaction (PCR) to make lots of copies of the genes. Next, scientists mix in fluorescent compounds that attach themselves to 13 specific locations on the DNA and give a highly specific genetic portrait of that person. The whole process takes a few days, and forensic labs are often backed up analyzing data from other cases.

These cautionary asides about touch DNA, backed up labs and evidence retention problems echo concerns voiced recently to the Court of Criminal Appeals "Criminal Justice Integrity Unit that:

The advent of "touch DNA" ... threatened to overwhelm agencies' storage capacity. Potentially lots of new items could be stored for touch-DNA testing, even though labs already have tremendous backlogs. That means long lag times during which the evidence must be securely stored despite limited space.

Texas' DNA labs right now are backed up and understaffed, but the trend will be for their caseloads to dramatically expand in the near term as these techniques become more widely used (especially considering the range of possible uses to which it could be subjected). For these reasons, ironically, demand for DNA lab services will inevitably increase regardless of whether crime rates go up or down.

Forensic science errors and the introduction of junk science as evidence are responsible for up to a quarter of false convictions among Texas DNA exonerees. The focus on DNA forensics in Texas has been to bring existing labs up to snuff quality-wise, but nobody's planning pro-actively, to my knowledge, how the state will make the labs independent, much less eliminate current backlogs, or meet the inevitable, expanded future demand for forensic capacity.

The advent of touch DNA is an amazing and wonderful thing evidence-wise, but it exacerbates an already problematic situation with regard to lab backlogs and DNA-related forensic errors.

Use adversarial system, redundancies to prevent wrongful convictions from faulty forensic science

The main solutions to faulty forensic science causing convictions of innocent people may lie, I've long believed, in using the adversarial system to create redundancies in forensic testing. It's needed because, for the most part, forensic science just isn't as accurate as most watchers of TV crime dramas may believe. A columnist at Forbes earlier this month ("What's wrong with CSI?,"May 2), Roger Koppl, wrote that:

to judge by the most comprehensive study on the reliability of forensic evidence to date, the error rate is more than 10% in five categories of analysis, including fiber, paint and body fluids. (Meaning: When the expert says specimen X matches source Y, there's a 10% probability he's wrong.) DNA and fingerprints are more reliable but still not foolproof. The 1995 study, in the Journal of Forensic Sciences, looked at proficiency tests labs take to see whether their work is sound.

More recent studies have also shown problems. Though a 2005 study in the Journal of Criminal Law & Criminology suggests a fingerprint false-positive rate a bit below 1%, a widely read 2006 experiment shows an alarming 4% false-positive rate.

Yet the public sees errors as gross anomalies. Like the time the FBI wrongly linked an Oregon attorney named Brandon Mayfield to the 2004 Madrid commuter train bombing that killed 200 people. The FBI had claimed a 100% match between fingerprints found at the scene and Mayfield, who was held for two weeks in federal custody. When the Spanish National Police got the real perpetrator, an Algerian named Ouhnane Daoud, the FBI had to admit its mistake. Mayfield accepted a $2 million settlement from the government.

Another debacle, this time involving DNA testing: Josiah Sutton served four and a half years for rape after the Houston Crime Lab tied him to crime-scene DNA. The lab was later found to be rife with problems, including a leaky roof that let rainwater contaminate evidence. Sutton was proclaimed innocent in 2004 and awarded $118,000 in reparations the next year.

Forbes editor William Baldwin (no bleeding heart liberal), declared in response to these data that, "When it comes to science, judges are steeped in a long tradition of welcoming junk into the courtroom." It's a hard point to argue. When even fingerprint matches aren't 100% reliable, and some forensic techniques yield 10% error rates, that means quite a few innocent people may be accused by faulty forensics, allowing actual offenders to go free. Koppl's data about errors in the softer nether regions of forensic science reminded me of this recent assessment:

At the "Actual Innocence" conference in Plano last month [in April 2008], former executive director of the National Forensic Technology Science Center Bill Tilstone told the audience that most "pattern evidence" - handwriting analysis, shoe and tire print comparisons, etc., has no research-based foundation at all. Much of forensic science is "soft" science, he said, that at best has not or even cannot be comprehensively tested for accuracy.

Given the subjectivity of much forensic analysis, IMO the key to minimizing forensic errors lies in redundancy, an idea Mr. Koppl also embraced.

The core problem with the forensic system is monopoly. Once evidence goes to one lab, it is rarely examined by any other. That needs to change. Each jurisdiction should include several competing labs. Occasionally the same DNA evidence, for instance, could be sent to three different labs for analysis.

This procedure may seem like a waste. But such checks would save taxpayer money. Extra tests are inexpensive compared to the cost of error, including the cost of incarcerating the wrongfully convicted.

He's on the right track, but I think Mr. Koppl hasn't quite yet found the best solution. The most efficient way to create redundancy in forensic science is to use the adversarial system. Instead of police sending evidence to three different labs, two is plenty if we simply provide funds for defense experts whenever government crime labs analyze evidence.

As Koppl notes, such redundant checks promote accuracy, justice and save taxpayer money in the long run by reducing overall error. How much would error be reduced? Here's how the math works:

Say a method of forensic analysis yields a 10% error rate, to use an example from the high end. If a second analyst examines the evidence, the redundancy diminishes the probability of error from 1 in 10 to 1 in 100 (.1 x .1 = .01). In the case of fingerprints, if the error rate currently is 1%, allowing redundant defense analysis would reduce the possibility of error to just one-one hundredth of 1% (.01 x .01 = .0001).

What's more, if the defense hires the second expert, it avoids questions of vendors coming up with similar results to cater to the same customer's agenda, making it more likely evidence will be thoroughly vetted and more accurate results discovered.

Recently Sen. John Whitmire identified one of the problems confronting the much-maligned Houston PD crime lab as its lack of neutrality, opining that the forensic lab should be controlled by a "neutral" entity. But forensic science isn't neutral, no matter who performs it. Rather than attempting to create a single neutral entity, IMO it makes a lot more sense to use the existing structure within the adversarial system to mandate redundancies and improve accuracy.

UPDATE: The author of the Forbes article, Roger Koppl, reacts.

See related recent Grits posts:

• Are crime lab reports "testimonial" documents? SCOTUS will decide
• Many arson convictions based on invalid science
• Arson convictions may be next venue for innocence claims
• More from Texas Innocence Summit
• Innocence summit drew officials, opinion leaders from around state
• Washington Post explores frontiers of DNA science
• Proving innocence, solving cold cases frequently depends on DNA collection and preservation
• When DNA exonerations end, innocence problems will continue
• Beyond the iceberg's tip: Dallas to extend innocence investigations beyond DNA cases
  
• CSI-Houston would be dark comedy

Evidence retention failures thwart pursuit of innocence claims

On Tuesday, Texas witnessed its 34th DNA-based exoneration since 2001, this time again from Dallas County which is one of the only jurisdictions in Texas that saved old biological evidence. Such evidence only exists in about 10% of violent crime cases anyway, so DNA alone won't help most innocent offenders. But what about cases where DNA evidence existed but wasn't preserved?

USA Today this week published a pair of interesting stories focused on an issue made more immediate by the large number of recent DNA-based exonerations around the country - the failure of many jurisdictions to preserve DNA evidence ("DNA not saved in half of states," Aug. 6):

Evidence preservation has been the key to freeing more than 200 wrongfully convicted prisoners, says the Innocence Project, a group that works to free the innocent based on DNA testing.

Preserving DNA also has helped secure convictions. "We're becoming more successful in identifying perpetrators in cold cases than we were when we didn't have this technology," says Scott Storey, district attorney in Jefferson County, Colo.

The USA Today piece lists Texas as among states that retain biological evidence, but that's a) relatively new and b) inconsistently applied. District clerks control such evidence in Texas post-conviction, and no analysis of their evidence collection and retention practices has been performed to my knowledge. Who knows how different jurisdictions are handling it? Texas does not require district clerks to keep biological evidence that wasn't admitted in court, which may result in potentially exculpatory evidence being destroyed. And in some places like Harris County, prosecutors make an end run around evidence preservation requirements by insisting on evidence destruction a condition of plea bargains.

A companion story in USA Today ("In lieu of DNA evidence, exoneration proves tougher," Aug. 6) questioned whether the relatively small number of cases with DNA might thwart efforts to prove most innocence cases post-conviction. The story quoted Jeff Blackburn of the Innocence Project of Texas (for whom, I should remind readers unaware of the possible conflict, this writer is a paid consultant):

Jeff Blackburn, chief counsel for the Innocence Project of Texas, fears Texas cases dependent on DNA could "run out" within a year. Of the 700 cases his group believes have potentially credible claims, 225 would be heavily weighted on the outcome of DNA analysis — if the material exists. The rest involve issues such as witness identification problems and coerced confessions.

Some advocates' concerns over the availability of DNA have injected tension into a movement to free the wrongfully convicted.

Barry Scheck, co-director of the Innocence Project, a national group whose work relies almost exclusively on DNA testing, says enough cases exist to sustain a decade's worth of potential exonerations. His group says the DNA caseload has increased from 141 in 2004 to 278 this year.

"These cases are not slowing down," he says, adding his colleagues "are not looking hard enough." He says DNA cases will decline eventually — but not yet.

Just because DNA cases may slow down, though, that doesn't mean false convictions will have ceased, only that their causes will have been exposed by a serendipitous case study generated by this new technology. Unscientific eyewitness ID practices, false confessions, mendacious informants, forensic errors, prosecutors withholding exculpatory evidence, and a handful of other key systemic flaws will continue to convict innocent people after old DNA cases run out if they're not addressed by the Legislature or the courts.

With proper evidence preservation going forward, perhaps some new technology we can't even imagine today will provide another window into justice system errors. Given society's recent experience with DNA proving innocence in old cases, it's worth hanging onto the evidence to find out.

DNA mixture software widely used in Texas disallowed by MI federal judge

For some time, Grits has been skeptical whether DNA-mixture software would hold up to "Daubert" scrutiny if judges were to directly evaluate it. In 2015, this blog reluctantly broke the story in Texas about errors and uncertainties regarding how to evaluate DNA mixture evidence. (They were being openly discussed by the Forensic Science Commission and the Court of Criminal Appeals' Criminal Justice Integrity Unit when I worked for the Innocence Project of Texas, but reporters weren't covering the complicated, math-heavy story.) After that, I've followed the issue from afar even after I stopped tracking forensics in any professional capacity.

STR-Mix is one of a handful of companies selling their product as a solution to the conundrum of how to evaluate DNA mixture evidence. Now, a federal district judge in Michigan, Janet Neff, after an exhaustive review of the science, has ruled directly on the Daubert-based admissibility questions surrounding this product that Texas courts punted on. In particular, Judge Neff ruled in favor of defendant David Gissantaner, concluding that

The DNA evidence sought to be admitted in this case—in essence, that it is 49 million times more likely if Daniel Gissantaner is a contributor to the DNA on the gun than if he is not— is not really evidence at all. It is a combination of forensic DNA techniques, mathematical theory, statistical methods (including Monte Carlo-Markov Chain modeling, as in the Monte Carlo gambling venue), decisional theory, computer algorithms, interpretation, and subjective opinions that cannot in the circumstances of this case be said to be a reliable sum of its parts. Our system of justice requires more. 

It is the Court’s hope that this decision brings to light the shortcomings or, at the very least, points of inquiry necessary in evaluating this advancing technology as a tool in forensic DNA analysis. There must be a dialogue among key players in the general interest of the development and refinement of the technology, the software and its application by the individuals charged with its use in the field, rather than post-hoc testing of its reliability in the context of a criminal prosecution where the ultimate question is the freedom and guilt or innocence of the person of interest. 

Here, because the sum of the parts simply does not add up to a reliable whole, the DNA analysis/likelihood ratio resulting from the use of the STRmix probabilistic genotyping software must be excluded. Defendant’s motion to exclude evidence is granted. An Order will enter consistent with this Opinion.

Wow! Obviously, this doesn't apply in Texas nor anywhere in the 5th Circuit. But it's the first time to my knowledge that a federal judge has taken a deep dive into the science behind proprietary DNA mixture software and rejected it based on the Daubert standard. Welcome to the party, Judge! Grits was beginning to feel a bit lonely in that view!

In one-to-one DNA matches, or two-source matches like a rape kit where the DNA of the victim is known, DNA evidence remains the gold standard of forensic science. But analyzing DNA mixture evidence - e.g., "touch DNA" on a doorknob or DNA from a blanket on a couch - remains a more subjective process that's sometimes more of a guess than a "match."

I want to delve into the judge's opinion more deeply before commenting further, but this seems like a big development. We know at least some Texas labs use the same proprietary DNA software the federal judge in Michigan disallowed.

Clarification: More on this soon, but having now more closely read the opinion, Grits should clarify that the court recognized STR-Mix software results could be considered valid for distinguishing up to three-sourced DNA mixture samples, but only if the DNA one is attempting to match makes up at least 20% of the sample; in this case it made up 7%. So saying STR-Mix software was "disallowed" in the headline was too broad. Under this ruling, STR-Mix results for one and two source DNA samples would remain admissible, and for three sources, if the sample one wanted to identify made up at least 20 percent of the total. But it couldn't be used when there were only trace amounts from the third DNA source, or when there are four or more sources. Here is a spot where user expectations for forensics exceed the capabilities of actual science. In many ways, the judge's advice corroborates what experts have said in Texas since this issue first arose: beyond a certain point, DNA mixtures become un-interpretable "crap." You can push the math further, but the evidence doesn't therefore improve.

Austin PD crime lab DNA section screwed the pooch

Austin has closed the DNA division of its crime lab because of problems with their processes testing DNA mixtures dating to 2010, reported the Austin Statesman:

The Austin Police Department has temporarily suspended operations at its DNA lab because of concerns raised by the Texas Forensic Science Commission, a lack of properly trained supervision and the need to allow staff to learn a new federally required way of verifying evidence, the American-Statesman and KVUE-TV have learned.

The decision means that hundreds of DNA samples — often crucial evidence in crimes such as homicides and sexual assaults — will either be shipped to the Texas Department of Public Safety lab or to private labs for analysis, possibly delaying the outcome of pending cases in already backlogged Travis County criminal courts.

The article declared that the closure "comes at a time when DNA labs nationally are adapting to new federally required procedures." However, when I emailed the FSC's Lynn Garcia, she said she's "not sure what federal guidelines they are referring to." Really APD just screwed up the science, using a method which is "neither scientifically valid nor supported by the forensic DNA community," according to the six-page letter from the FSC to Austin PD critiquing their methods, which Garcia provided to your correspondent.

Embarrassingly, according to the letter, the main problem was that lab personnel didn't understand past federal directives: "Technical Leaders (TLs) and senior analysts in the APD DNA Lab appear to have misunderstood language from the Scientific Working Group on DNA Analysis Methods (SWGDAM) Interpretation Guidelines for Autosomal STR Typing by Forensic DNA Testing Laboratories ... and from Dr. John M. Butler's textbook, 'Advanced Topics in Foensic DNA Typing: Methodology.'"

"Of greater concern, the analysts themselves were aware the [method] was ineffective because they observed ... [problems] in their own casework" and did nothing. They cherrypicked data, choosing which loci to compare based on whichever ones show up in the sample. "The appropriate approach is to decide which locus (or lici) should be used first ... as indicated by the overall analysis of the evidentiary sample, not on which alleles are present or absent based on the victim or suspect known profiles."

The FSC identified at least one case where contamination made APD's results invalid, as confirmed later by a private lab. They also found ten cases of contaminated reagents.

Worryingly, the problems were not revealed by ASCLD/LAB assessments, even though deficiencies were evident in the data they were given.

These observations raise legitimate questions regarding the limits of accreditation and the consistency of assessor teams. Specifically,: (a) Are the scope and limitations of accreditation well understood by the criminal justice community? (b) Do assessors consistently consider whether the laboratory's protocols and underlying validation based on sound scientific principles or do they limit their review solely to determining whether the laboratory has a protocol in place that it follows? (c) Should assessors re-review validation data from prior years considering that validation studies are relied upon to build subsequent protocols?

Most of that detail wasn't included in the Statesman story.

Concluded the paper, "the Austin Police Department’s crime lab, which will have to recalculate statistics on about half of the 1,297 Travis County cases identified so far, is still validating new software and updating its protocols. Meanwhile, the lab’s backlog of cases awaiting DNA analysis has risen to about 1,300, the most in the past five years."

It's a mess, but it's not that different than what most other crime labs in the state are going through, except most of them didn't shut down operations until their staff are retrained. After all, the errors relate only to mixtures. There are still quite a few cases, including most rape kits, where there is only one unknown sample. That part Grits finds odd, but I'm glad they're belatedly getting their act together on this topic.

MORE: From Forensic magazine.

See prior, related Grits posts:

• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

Timothy Cole pardon recommended: How many more false convictions are out there?

The Texas Board of Pardons and Parole has recommended a pardon for Timothy Cole, who died in prison of an asthma attack before DNA could prove he was innocent of the rape for which he was falsely convicted. Reports the Fort Worth Star-Telegram:

It would be the state's first posthumous pardon, and [Governor Rick] Perry has indicated that he would sign an order clearing Cole's name if recommended by the board.

"Gov. Perry looks forward to pardoning Tim Cole pending the receipt of a positive recommendation from the Board of Pardons and Paroles," Perry spokeswoman Allison Castle wrote in an e-mail to The Associated Press on Saturday.

Cory Session, who has been fighting to clear his brother's name for years, said he anticipates that the governor will sign Cole's pardon in March during a ceremony in Fort Worth.

"To say that the wheels of justice turn slowly would be an understatement," Session said Saturday.

"The question is: How many more Tim Coles are out there?"

Excellent question, Cory. Keep asking it!

In partial response to Session's query, Dave Mann at the Texas Observer published a column last week titled "Who gets wrongly convicted and why?," in which he broke down the causes of wrongful convictions among the first 250 DNA exonerations in the United States. (Timothy Cole is not on that list of 250 because his pardon hasn't been finally approved, but he'll be added to it soon.)

In many cases, Mann points out, more than one single error contributed to the erroneous outcome. For example, 78% of cases involved faulty eyewitness identification, but in 38% of cases, two or more witnesses made false identifications that DNA later disproved. Just over half of cases - obviously including many with faulty eyewitnesses - also included forensic errors, and just over a quarter involved false confessions.

Remarkably, seventeen DNA exonerees were released from death row, leading Mann to suggest:

That’s 17 innocent people who would have been executed had DNA testing not cleared them. You have to assume there's been an innocent person somewhere who wasn’t lucky enough to have testable DNA in their case and was wrongly executed in this country—quite possibly in Texas and quite possibly Cameron Todd Willingham.

That's an excellent point about those innocent prisoners "lucky enough to have testable DNA in their case." For a variety of reasons, DNA evidence is usually unavailable. Biological evidence only exists in 10% or less of violent crime cases, meaning that DNA would be unavailable clear innocents in the other 90% of death row inmates and others convicted of violent crimes. It seems nearly inconceivable that the error rate for the other 90% would be any lower than that for those where police happened to collect DNA at the crime scene.

Further expanding the possible number of additional, unidentified innocent prisoners out there, even when biological evidence was gathered, most jurisdictions did not keep DNA from older cases and the lion's share of exonerations (certainly in Texas) come from before DNA testing was widely available.

Even more frustrating, when jurisdictions do find batches of old evidence - as in Bexar County where the San Antonio PD found more than 5,000 untested rape kits languishing in storage - they frequently choose to test only for guilt, but do nothing to identify possible innocence cases. In Dallas, after his predecessor fought DNA testing tooth and nail, District Attorney Craig Watkins famously partnered with the Innocence Project of Texas to vet old cases after they found a similar DNA cache, a move which has afforded him national acclaim. But in Bexar County, DA Susan Reed has so far refused to follow his lead.

To honor the memory of Timothy Cole and the other 250 DNA exonerees - who Mann notes collectively spent 3,160 years in prison for crimes they didn't commit - Reed should insist on vetting cases with recently discovered DNA evidence to search for the innocent as well as the guilty. So far, the San Antonio Police Department is going through the cases looking only for evidence in unsolved stranger rapes. But if prosecutors do the work, they'll have an obligation which the police do not to turn over potentially exculpatory evidence to the defense. That's why the DA, not the cops, should take the lead.

Though Reed is a Republican and Watkins a Democrat, this is not a partisan issue in any way, shape or form. (That said, if the DA remains recalcitrant, she has a Democratic challenger in the fall so I suppose it could become one.) Instead, it's a question of whether the DA will abide by the fundamental tenet of her office: To seek justice, not just convictions. Timothy Cole's memory, much less the sacrifice of thousands of years by those 250 innocent men, demands that the justice system take responsibility for its errors and not sweep them under the rug.

DNA mixture math errors discovered in 2015 weren't the first

Grits this week revisited the 2007 "Bromwich report" regarding what then was the Houston PD crime lab while researching DNA-mixture evidence issues and realized something I hadn't understood at the time - or perhaps just never connected the dots: Errors calculating DNA-mixture probabilities were at the center of the years-long crisis that engulfed the Houston crime lab's DNA section for most of the aughts.

Readers will recall, in 2015, we discovered that all Texas crime labs were miscalculating DNA-mixture probabilities - in one case, a sample jurors were told matched one person out of a billion really matched one out of 50. Well, the same thing was happening at the Houston crime lab. Check out this chart from the Bromwich report identifying cases where DNA analysts vastly over-estimated the probability ratios:

So in Houston, they discovered the crime lab was miscalculating probabilities in the aughts, updated the math, then in 2015 discovered that math was wrong and had to update it again. Wow! This '07 comment from Bromwich about the HPD lab would have applied equally to every Texas crime lab as recently as 2015:

It is clear that DNA analysts in the historical Crime Lab ... did not fully understand the scientific basis for calculating frequency estimates from DNA profiles obtained from evidence samples and that they were not trained in the methods of properly calculating statistics associated with DNA mixture profiles and partial DNA profiles.

Notably, the method Texas labs were advised to shift to in 2015 - "probabilistic genotyping," most commonly using equipment/software from a company called STR-Mix - lately has itself come under fire. A federal judge out of Michigan recently excluded such evidence in a Daubert hearing. So it's possible DNA analysts will need to re-do their math again.

Returning to the Houston PD crime lab example, these are in many cases radical downward adjustments. From one in 6.3 million to one in 30? From a one in 2.9 million chance to one in 5? Would juries have convicted if they'd heard the lower numbers? I wonder if other crime labs were using the same method as HPD's during this period?

Of the cases in this chart, Franklin Alix was executed despite DNA testimony in his case having been declared unreliable. Only Josiah Sutton was ever exonerated; 11 remain incarcerated. By all accounts, the criminal-defense bar in Houston failed to step up in many instances, leaving most of these folks un-or-poorly represented.

It's highly likely there are more actually-innocent people on that list, but we'll never know. The DNA lab's other big problem was that it had a leaky roof and most of the historical DNA evidence was destroyed and couldn't be re-tested.

See prior, related Grits posts:

• Digesting new Daubert opinion on DNA mixtures
• Most crime labs accused innocent person in DNA mixture study
• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• Courts punt on forensics surrounding DNA mixtures
• Black box DNA mixture analysis method used in NYC turned out to be flawed
• Changes to DNA mixture software raise questions about old versions
• News flash: 'Touch DNA' doesn't necessarily require touching
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

Texas should conduct review of hair and fiber forensics comparable to feds

Excellent news for those concerned with the lack of rigor in certain forensic science disciplines, this time hair and fiber analysis. The Washington Post reported last week ("Justice Department, FBI to review use of forensic evidence in thousands of cases," July 10) that:

The Justice Department and the FBI have launched a review of thousands of criminal cases to determine whether any defendants were wrongly convicted or deserve a new trial because of flawed forensic evidence, officials said Tuesday.

The undertaking is the largest post-conviction review ever done by the FBI. It will include cases conducted by all FBI Laboratory hair and fiber examiners since at least 1985 and may reach earlier if records are available, people familiar with the process said.

The review comes after media reports alleging that the agencies had known of flawed hair and fiber analyses and covered it up. Here's a little more detail

The Post reported in April that hair and fiber analysis was subjective and lacked grounding in solid research and that the FBI lab lacked protocols to ensure that agent testimony was scientifically accurate. But bureau managers kept their reviews limited to one agent, even as they learned that many examiners’ “matches” were often wrong and that numerous examiners overstated the significance of matches, using bogus statistics or exaggerated claims.

Details of how the new FBI review will be conducted remain unclear. The exact number of cases that will be reviewed is unknown. The FBI is starting with more than 10,000 cases referred to all hair and fiber examiners. From those, the focus will be on a smaller number of hair examinations that resulted in positive findings and a conviction.

It also is unclear whether the review will focus only on exaggerated testimony by FBI examiners or also on scientifically unfounded statements made by others trained by the FBI, or made by prosecutors. Also unclear is at what point government officials will notify defense attorneys or the Innocence Project.

In past reviews, the department kept results secret and gave findings only to prosecutors, who then determined whether to turn them over to the defense.

To my knowledge the FBI has only done this one other time, and then they didn't do it very well, and certainly not quickly: When actual science demonstrated that FBI experts had been falsely claiming for years that they could differentiate bullets by their lead content (e.g., "this bullet came from the same batch at the factory as the ones found in the possession of the defendant"), they had to retract the testimony and systematically identified every case in which it'd been used, leading to several cases being overturned.

But lead-content testimony was relatively uncommon: The number of times it was used at trial ran to less than 200. Hair and fiber analyses have been much more common for much longer and done at labs across the country, not just at the FBI. Will other crime labs now begin similar reviews? There's no requirement they do so, and really no process for it.

Texas, to its credit, in some ways pioneered the concept that old cases need to be comprehensively reviewed when forensic errors are found. The watershed moment perhaps was Dallas DA Craig Watkins partnering with my employers at the Innocence Project of Texas (IPOT) to review old DNA cases and recommend which ones deserved testing, a process that led to numerous high-profile exonerations and landed Watkins a feature on 60 Minutes and other national acclaim.

Similarly, the Forensic Science Commission and State Fire Marshall have partnered with IPOT to review  arson cases of people currently incarcerated in TDCJ for errors that may have led to false convictions. And after incompetent drug analysts were discovered in El Paso and at a DPS lab in Houston, the agencies systematically notified defense counsel. After the incident in Houston, the DA's Association recommended that "For any case with a bad retest, or cases with now-destroyed evidence, [prosecutors should] request that the court appoint an attorney to take the case through a writ process if appropriate."

Still, though, that's happening only on an ad hoc basis. Fort Bend Sheriff's Deputy Keith Pikett for years used highly suspect techniques in "scent lineups" which Texas courts have now disavowed. Even so, Pikett claimed to have performed scent lineups with his dogs in more than 2,000 cases and testified in court many times, but there's been no official review - by the Fort Bend Sheriff, the Texas Attorney General, the Forensic Science Commission (outside their jurisdiction), or anyone else.

In this case, analysts from Texas crime labs including at DPS testified for years about hair and fiber evidence in similarly overstated ways as the FBI, recently reining in the language they use in court and relying more where possible on much more accurate DNA evidence. Pretty much everybody agrees such overstated analyses were both problematic and used for a long time. But there's no state-level review of hair-and-fiber testimony in past cases comparable to what's happening at the FBI, neither here in Texas nor to my knowledge in other states, even though it's obvious the same issues extend far beyond FBI analysts.

Grits has long insisted there needs to be a mechanism - arguably at the Forensic Science Commission, with expanded jurisdiction - for reviewing old cases when the science behind long-used forensics are called into question or debunked. We're starting to do that when new forensic errors arise, but for the most part haven't addressed the lingering old ones like flawed or overstated hair and fiber testimony. It's time to start.

See prior, related Grits posts:

• Texas needs process to vet cases based on forensic hokum
• What is state duty to inform defendants of flawed forensics in old cases? 
• What is the duty to notify defendants of past crime lab errors?
  
• Forensic commission urges review of old arson cases based on junk science
• The politics of reexamining flawed arson foresics 
• If arson science in Willingham case was 'flawed,' what about other, similar cases?
• How best to vet old arson innocence claims?
• Faulty hair forensics used to convict executed defendant
• Hair analysis technique that was 'judicially accepted for decades' called 'highly unreliable' 

Digesting new Daubert opinion on DNA mixtures

Grits earlier referenced the opinion from federal District Judge Janet Neff, in Michigan's western district, who directly evaluated the most common, commercial DNA-mixture software using probabilistic genotyping under the Daubert standard for whether to allow expert testimony. Texas courts heretofore have let in such evidence but avoided a direct Daubert evaluation.

There was a Texas connection in the case. Dr. Michael Coble, of the University of North Texas Health Science Center, was the expert employed by the prosecution. He has been one of several key advisers to the Texas Forensic Science Commission and the legal system in general as the state has struggled with this DNA-mixture issue. The expert for the defense, whose arguments more frequently prevailed in Neff's opinion, was Dr. Dan Krane, from Wright State in Ohio.

Judge Neff evaluated each prong of the Daubert standard as it related to the STR-Mix DNA mixture-analysis software. This program employs high-end math that few DNA-lab practitioners can actually reproduce called "probabilistic genotyping." But programs are only as good as the assumptions that guide their work, and some of the assumptions in each case - e.g., the number of contributing samples - are inputted by the user. There are subjective judgment calls throughout the process. Further, because the math results from tens of thousands of randomized guesses, the results aren't replicable test to test.

Judge Neff was disquieted that, unlike for one-to-one DNA matching or two-source mixtures, as with a rape kit, there are no controlling standards for interpreting DNA mixtures, especially for samples with very small amounts of DNA - maybe only a few scattered cells.

The judge also made the interesting point that, even if the math behind the STR-Mix software is impeccable, there were international standards for coding such high-end math projects and the STR-Mix software didn't meet them all. Coble was dismissive that other fields might have standards to contribute, but the product in question is so obviously an intersection of an array of multidisciplinary work, the judge didn't buy it.

Further, the judge echoed concern that most validation studies on the software so far have been conducted by company principals. And while there was peer-reviewed support for STR-Mix (again, mostly for studies by people with a financial stake in the project), none of that peer-reviewed work focused on low-level samples as in this case, where just a few cells were found that were said to belong to the defendant.

One point that's always bugged me was raised in an expository section of the opinion but not included in her reasons for her Daubert denial: “[D]ue to the methodologies used in STRMix, '[t]he results of no two analyses will be completely the same',” she wrote. Elsewhere, she quoted an academic article noting that, "for a technique to be broadly applicable, it must be based on measurements that can be replicated." Supporters of the software, however, say replicability isn't necessary and it should be good enough if the results were all in some acceptable range. Regardless, while Judge Neff clearly observed this incongruity, it wasn't included in her final reasons for her ruling.

Clearly the most persuasive piece of evidence for Judge Neff was the 2016 PCAST report which expressly stated that STR-Mix was validated for up to three sources if the sample to be identified is at least 20 percent of the total. She did not find further validation regarding smaller samples, and the very small sample in this case, she believed, merited even greater caution. She essentially treated the threshold cited by PCAST as a bright line regarding the current state of the science. STR-Mix can be used for traditional one or two-source DNA matching, she ruled, and for three-source mixtures if the sample to be matched made up 20 percent or more of the whole. But otherwise, the estimates would be inadmissible.

One small upside for backlogged crime labs using STR-Mix: This would make it easier to screen DNA mixture submissions. Samples with more than 3 contributors should probably be rejected up front as unanalyzable "crap," to use the words of a prominent DNA expert who advised Texas crime labs on the topic.

Errors in such cases can cause false convictions, so Grits is glad to see this increasingly dirt-common evidence more carefully vetted. From the time probabilistic genotyping was first introduced to Texas crime labs in 2015, experts were warning against stretching the limits of the math (i.e., trying to analyze "crap"). Judge Neff would set a bright-line threshold, and given the current state of the literature, it's not an unreasonable choice. In many ways, it's the most defensible (it would probably be easier to get past it by improving the tech than defeating her arguments). But whatever threshold one chooses, her decision highlights that one must exist, even if courts haven't heretofore required it.

I'd quoted it earlier, but I loved Judge Neff's conclusory analysis of current DNA-mixture-evidence practices, so let's give her the final word here:

The DNA evidence sought to be admitted in this case—in essence, that it is 49 million times more likely if Daniel Gissantaner is a contributor to the DNA on the gun than if he is not— is not really evidence at all. It is a combination of forensic DNA techniques, mathematical theory, statistical methods (including Monte Carlo-Markov Chain modeling, as in the Monte Carlo gambling venue), decisional theory, computer algorithms, interpretation, and subjective opinions that cannot in the circumstances of this case be said to be a reliable sum of its parts. Our system of justice requires more.

Anyone interested should definitely read the whole opinion. This summary hardly does it justice.

MORE: From Techdirt.

See prior, related Grits posts:

• Most crime labs accused innocent person in DNA mixture study
• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• Courts punt on forensics surrounding DNA mixtures
• Black box DNA mixture analysis method used in NYC turned out to be flawed
• Changes to DNA mixture software raise questions about old versions
• News flash: 'Touch DNA' doesn't necessarily require touching
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

Prosecutors seek to thwart post-conviction innocence claims

I'm glad to learn that US Attorney General Eric Holder is reviewing a Bush Administration policy of securing plea waivers from federal defendants of their right to post-conviction DNA testing that could prove their innocence. But it's too bad the scope of that review won't extend to Texas prosecutors who seek DNA destruction as part of plea deals. Reports the Washington Post:

Attorney General Eric H. Holder Jr. has ordered a review of a little-known Bush administration policy requiring some defendants to waive their right to DNA testing even though that right is guaranteed in a landmark federal law, officials said.

The practice of using DNA waivers began several years ago as a response to the Innocence Protection Act of 2004, which allowed federal inmates to seek post-conviction DNA tests to prove their innocence. More than 240 wrongly convicted people have been exonerated by such tests, including 17 on death row.

The waivers are filed only in guilty pleas and bar defendants from ever requesting DNA testing, even if new evidence emerges. Prosecutors who use them, including some of the nation's most prominent U.S. attorneys, say people who have admitted guilt should not be able to file frivolous petitions for testing. They say the wave of DNA exonerations has little impact in federal court because all those found to be innocent were state prisoners, and the waivers apply only to federal charges. DNA evidence is used far more frequently in state courts. ...

Interviews and documents show that language allowing for DNA waivers was inserted into the law at the behest of Republican senators and that the Bush Justice Department lobbied against the measure even with the waiver provision. Soon after the law passed with bipartisan support, the department sent a secret memo to the nation's 94 U.S. attorney's offices urging them to use the waivers, several federal officials familiar with the memo said.

Josh Marquis of the national District Attorneys Association told the Washington Post that "he's never heard of DNA waivers in state court and that the organization opposes the concept. 'I think it's important to always leave the door open for actual proof of innocence,' he said." However, this tactic reminds me of a method used by Texas prosecutors to prevent post-conviction DNA claims: Requiring as part of plea deals that DNA evidence be destroyed to prevent future testing.

Indeed, the newly appointed Texas Forensic Science Commission chairman John Bradley has openly argued that destruction of DNA is important as part of a plea deal because "innocence trumps everything" and so what appear to be solid cases might unravel after conviction if the evidence isn't destroyed.

Innocence cases tend to represent gross errors in the system, seldom outright malicious prosecution. But choosing to destroy or limit testing of evidence in order to avoid those errors from being discovered surely borders on malicious: Certainly it's unfair, unjust, and simply bad public policy.

It strikes me that Texas prosecutors appear to be far ahead of their brethren among the feds and in other states at concocting methods to avoid successful innocence claims. Walter Reaves has a good post up about another tactic by Mr. Bradley (who is the Williamson County DA) to thwart post-conviction innocence claims: Prosecuting for aggravated perjury when someone who pled guilty later claims innocence in a habeas corpus writ. Writes Reaves:

Even the most staunch defender of law order might sense some unfairness in this. Other than concerns about decency and fairness, there are also some practical problems with this approach. The first is that just because a defendant pleads guilty doesn't mean he agrees he is guilty.

Long ago the United States Supreme recognized that defendants might who don't believe they are guilty might not want to risk going to trial. Although this can be he subject of a separate post, its basically common sense. Would you rather be innocent and spend 10 years in prison or innocent and spend 50 years in prison. Prosecutors know this, and sometimes make offers to good to be true. So a defendant basically fibs, and admits guilt in return for a favorable outcome.

The Court of Criminal Appeals recognized this a few years ago in DNA cases. Some of the persons who have been exonerated actually plead guilty originally. The court recognized there could be a number of reasons for pleading guilty, and held that would not prevent you from claiming innocence and filing a motion for DNA testing.

The Catch-22 Mr. Bradley would impose on defendants is further evidence of how absurd it is to analyze plea agreements as though they're voluntary contracts.

I don't blame prosecutors for the occasional mistake, but I do blame them when they attempt to cover up errors after the fact or seek to prevent them from ever being discovered. Habeas writs are a safety valve that exists to correct such errors - for example, when science improves to allow better testing or new evidence is discovered after the fact. Just because the law gives prosecutors tools to close off that safety valve using these kind of aggressive tactics doesn't make it wise or moral to do so.

Forensic Roundup

I wanted to point out several stories this morning related to topics in forensic science that deserve Grits readers' attention:

Response published to crime lab criticisms
The organization Crime Lab Report published this week a rebuttal to critics ("The Wrongful Conviction of Forensic Science," pdf) of crime lab errors, re-analyzing data on DNA exonerations first crunched by the Innocence Project in NYC. (Conflict alert: I'm a paid policy consultant for the Innocence Project of Texas.) Some interesting data here, but also some paranoid speculation (e.g., they think the Innocence Project directs when and what stories are published in the New York Times!) and more than a little defensive posturing. For instance, their top finding was that "Of the 32 instances of forensic science malpractice shown above, only 1 was found to have occurred in an accredited laboratory." In Texas, though (I can't speak for other states), accreditation wasn't required until recently, after a slew of errors were discovered, so that observation from my perspective falls into the category of "true but trivial." In addition, they conclude that "forensic malpractice" was the "sole systemic error in only two overturned convictions," but fail to acknowledge that forensic errors and malpractice have more frequently combined with other causes to generate system failure. They also insist that "bad lawyering" is responsible for more wrongful convictions than previously estimated.

Ironically, the sense I get from reading this critique of the Innocence Project's analysis is that everybody admits the system made mistakes, but the lawyers are more likely to blame the scientists and the scientists are more likely to blame the lawyers. Go figure! My take on the (mostly Texas-based) innocence cases I've looked at is that - with the notable exceptions of faulty eyewitness identifications or false confessions - a single error is rarely the sole cause of a wrongful conviction. More frequently outright systems failures occur when errors or neglect compound one another, often coupled with a "team spirit" mentality.

Houston FBI opens new digital forensics lab
Calling the process of removing digital evidence from electronic equipment a "hard science," the FBI this week announced they're opening a new digital forensics lab in Houston. They've begun soliciting requests for assistance from area law enforcement on their website. I'm just a little skeptical, however, of the futuristic concept of "self-serv" forensics. Reports the Houston Chronicle:

The laboratory's operations are overseen by a board of directors that includes investigators and commanders from the Harris County Sheriff's Office, the Houston, Pasadena and Richmond police departments and other area law enforcement agencies.

Participating agencies provide staff members, who process digital evidence from their own departments as well as others.

Forensic examiners also will travel to known or potential crime scenes to examine computers and other equipment.

The lab has four examiners who specialize in examining cell phones or PDAs, said sheriff's detective John Pohutsky, who helped conduct tours of the laboratory.

Such devices provide the "fastest-growing form of digital evidence," he said.

Eventually, the laboratory will offer self-service kiosks to speed up examinations of cell phone data for area law enforcement, Pohutsky said.

"An investigator from an agency would be able to bring a cell phone in that he's acquired as evidence," he said. "With a little bit of help getting started, he can basically process his own phone."

Champagne tastes and a 'near beer track record'
BlogHouston recently criticized HPD Chief Harold Hurtt for proposing ambitious upgrades at the HPD crime lab when they can't get the basics right:

It was just this past January that HPD's crime lab was in the news as the head of DNA testing resigned after a cheating scandal. So now Chief Hurtt wants to go from disgraced and dysfunctional to state-of-the-art and tops-in-the-U.S. I'm guessing Houstonians would settle for just processing DNA samples successfully. Period. You know the old saying, "champagne taste on a beer budget"? Chief Hurtt has champagne taste, with a near beer track record.

National FBI crime lab profiled
CNN offered up a feature on the FBI's much-vaunted national crime lab at Quantico, Virginia, an effort launched in 1932 by J. Edgar Hoover with the hiring of a single handwriting expert. Today more than 500 agents, scientists and staff work at the facility.

TX property room techs have their own association
An ad on page 2 of latest issue of The Property Times, the publication of the Texas Association of Property and Evidence Inventory Technicians, depicts a brutal NASCAR car crash with the bold headline, "Evidence Mismanagement is One of the Leading Career Killers in Law Enforcement."

The group's site links prominently to the Denver Post series on evidence mismanagement, so I'm glad to see recent critiques of property room practices are on their radar screen, though evidence rooms in Texas still have their own problems. I was unaware the organization existed but they've been around for ten years and have a substantial membership for such a narrow field.

Faulty hair forensics used to convict executed defendant

I don't suppose this will change anybody's mind on the death penalty pro or con, but it appears Texas may have executed a defendant based on a bogus forensic hair comparison. The Texas Observer has the scoop in a story which opens:

Claude Jones always claimed that he wasn’t the man who walked into an East Texas liquor store in 1989 and shot the owner. He professed his innocence right up until the moment he was strapped to a gurney in the Texas execution chamber and put to death on Dec. 7, 2000. His murder conviction was based on a single piece of forensic evidence recovered from the crime scene—a strand of hair—that prosecutors claimed belonged to Jones.

But DNA tests completed this week at the request of the Observer and the New York-based Innocence Project show the hair didn’t belong to Jones after all. The day before his death in December 2000, Jones asked for a stay of execution so the strand of hair could be submitted for DNA testing. He was denied by then-Gov. George W. Bush.

A decade later, the results of DNA testing not only undermine the evidence that convicted Jones, but raise the possibility that Texas executed an innocent man.

State Sen. Rodney Ellis issued a press release lamenting that "the General Counsel memo to then President-elect Bush failed to let him know that Jones was seeking DNA testing of the only physical evidence in the case. That is simply unacceptable and raises serious concerns about the post-conviction review process for death row inmates in Texas." [emphasis added] Good point. Between this and the Willingham case, it's clear Texas' clemency process under the last two governors hasn't been adequately vetting capital cases, much like the Court of Criminal Appeals.

However, the Observer stops short of claiming the evidence exonerates Jones:

Anti-death penalty advocates had hoped that the Jones case would provide the first-ever DNA exoneration of an executed person. While quite a few death penalty cases have been called into question, including several in Texas, no executed prisoners have been proven innocent by DNA testing, widely considered the most reliable form of forensic evidence.

Instead, Jones' case now falls into the category of a highly questionable execution—a case that may not have resulted in a conviction were it tried with modern forensic science. In that respect, it's much like the case of Cameron Todd Willingham, executed in 2004 for starting a house fire that killed his three children. Fire scientists now say the arson evidence used to convict Willingham was flawed. (The Forensic Science Commission will continue its investigation of the Willingham case at hearing on Nov. 19.)

Still, the revelations in the Jones case raise more questions about how Texas administers the ultimate form of punishment.

Jones was no innocent - he had a prior murder conviction and in this case the question was whether he was the shooter or the wheelman - but even if he were straight-up falsely accused, like Anthony Graves, I don't believe for a moment that executing the wrong person would magically turn the tide of public opinion against the death penalty. A recent national poll (pdf) found that more than 80% of the public already thinks an innocent person has been executed, and support for the death penalty is higher than ever.

Even so, here's another case where forensic methods portrayed to jurors as reliable in reality were not. Left up to jurors without minimum safeguards or even definitions regarding what constitutes evidence beyond a reasonable doubt, inference too often qualifies as evidence. And gubernatorial clemency - even when stays or pardons are recommended by the Board of Pardons and Paroles -  doesn't happen often and fails to provide adequate remedy for false convictions.

Since Jones is dead, the only way to benefit from re-vetting the facts of the case will come from whatever steps are taken to ensure similar errors don't recur. So what are the takeaway lessons here?

For starters, non-DNA hair comparisons take a big hit: Crime lab analysts matched the hair to a defendant when it really came from the victim, for whom they would/should have had comparison samples. This makes me wonder if all the hair analysts who reviewed the sample universally agreed on the match? Recently I wrote about how fingerprint examiners at DPS do not reveal in their final report when multiple examiners reached different conclusions about fingerprint matches. Later it was pointed out to me (by somebody who'd probably rather remain nameless) that other comparative forensic disciplines at DPS - from tool marks to hair analysis to ballistics - all use the same process of having a second examiner verify results, with supervisors meeting with analysts to reach a conclusion in the event the two disagree. Those disagreements, according to that source, historically aren't recorded in other forensic reports either. So the Brady concerns, in other words, go beyond just DPS' fingerprint division. Given the egregious error - mismatching the victim's hair to a defendant - one wonders if all the analysts on the case concurred,  or if disagreements were just glossed over when the final report was written?

Beyond that, the folks preparing then-Gov. Bush's clemency recommendations failed to tell him about the request for DNA testing. As Sen. Ellis points out, that's a significant oversight. Clemency has never been a big focus of Texas governors in my adult lifetime, but its denial shouldn't become so pro forma that staff doesn't even bother to adequately brief each case.

Finally, once again we see the results of informant testimony allegedly secured through threats from prosecutors. According to Time magazine, "The prosecution's star witness against Jones was a friend of [his alleged accomplice] who later said that prosecutors had coerced him into testifying." Of course, coercion of a witness by prosecutors is also the main cause of the false conviction in Anthony Graves' case. I haven't read the trial transcript, but I wouldn't be surprised if the informant testimony were as big a factor in Jones' conviction as the flawed forensics.

Usually when someone is convicted of something they didn't do, there's not just one error that caused it but multiple, usually well-intentioned sources of error that come together to collectively generate a false conviction. That's not to make excuses for avoidable errors, only to argue in favor of tweaks to the system that would help avoid more of them in the future.

MORE: From Stand Down and Texas Coalition to Abolish the Death Penalty. ALSO: See the national Innocence Project's web page on the Claude Jones case.

'Forensics on Trial': Texas forensic commission at forefront of vetting forensic errors

PBS's NOVA on Wednesday ran a show titled "Forensics on Trial" (you can watch it online) with the following promo description:

There is a startling gap between the glamorous television world of “CSI” and the gritty reality of the forensic crime lab. With few established scientific standards, no central oversight, and poor regulation of examiners, forensics in the U.S. is in a state of crisis. In "Forensics on Trial", NOVA investigates how modern forensics, including the analysis of fingerprints, bite marks, ballistics, hair, and tool marks, can send innocent men and women to prison—and sometimes even to death row. Shockingly, of more than 250 inmates exonerated by DNA testing over the last decade, more than 50 percent of the wrongful convictions stemmed from invalid or improperly handled forensic science. With the help of vivid recreations of actual trials and cases, NOVA will investigate today’s shaky state of crime science as well as cutting-edge solutions that could help investigators put the real criminals behind bars.

Faulty forensics are important and account for a statistically significant proportion of exonerations, but IMO they're a much less common direct cause of false convictions than things like eyewitness error, ineffective defense counsel, and police or prosecutor misconduct. Forensics are usually used as corroborative evidence after an arrest has been made and with a few notable exceptions (e.g.,, DNA and fingerprint matches, DWI evidence) are seldom the sole evidence pointing to a defendant's guilt. Still, it's certainly on the short list and, as the Forensic Science Commission's reviews have shown, there has been a significant amount of error and incompetence to be rooted out in Texas state crime labs. The PBS report doesn't mention Texas, but it should be noted that the Texas FSC represents a pioneering effort at state-level forensic oversight, which has continued even after the cameras stopped rolling in regards to the whole Todd Willingham brouhaha.

True, the FSC's findings have no "teeth" and are sometimes spun rather generously toward the labs. OTOH, labs have been surprisingly compliant with their investigations, and if you agree sunlight is the best disinfectant, then exposing problems in detail serves a significant function. (I've learned a lot.) Plus, assistance by DPS, ASCLD/LAB, the Court of Criminal Appeals' "integrity unit," the state fire marshal, the Innocence Project of Texas, and other entities has given the FSC an extended  reach which would otherwise remain beyond their grasp. No one involved could fail to find fault with various aspects of the Texas FSC's processes, particularly when John Bradley was chair, but also before and after. However, a reform bill carried by state senator Juan "Chuy" Hinojosa, which nearly made it through the process last session and is expected to pass next spring, may ameliorate some of the most pressing limitations on the agency's authority.

Texas has focused its attention on issues surrounding forensic (and for that matter, eyewitness) errors more intensely than almost any other state. Before the NAS report was even begun, Texas labs had to be accredited by 2005, the year the FSC was created. So the state deserves credit for beginning to wrestle with these questions well before the National Academy of Sciences' much-publicized 2009 report on flaws in forensic analysis, though that certainly gave the FSC a much-needed jump start. All this to say, despite the Forensic Science Commission's rather odd makeup and some fixable, structural flaws, its detailed investigations provide much more information about what's going on in state and local crime labs than Texas ever had before the commission ramped up its efforts.

We're only at the beginning of an era when the scientific method is being applied to historically accepted forensic products, and the results so far have been a mixed bag. But at least Texas has created a mechanism capable of documenting crime-lab problems about which, a decade ago, we perhaps wouldn't even be aware. The NOVA special is right that we're in an era when forensic sciences are "on trial." But to the extent forensics is science, it should be constantly put "on trial," and I'm gratified that Texas has created a mechanism, however flawed, for doing so.

MORE: From Boing Boing.