What the jury didn't hear, against SWAT raids for routine search warrants, bail explainers, courthouse architecture, and other stories

Here are a few odds and ends that merit Grits readers' attention:

Margaret Moore, Rosa Jimenez, and what the jury didn't hear
Weird comments from Travis County DA Margaret Moore on the Rosa Jimenez case in The Appeal: “There is an ultimate fact question that was resolved by the 12 men and women who actually saw all the evidence and heard opinion testimony,” Moore told The Appeal. “Everything after that is opinion by people who were not in that courtroom.” But here's the thing: The reason four judges have now said Jimenez is likely innocent and should be released is that the jury heard false, un-rebutted expert testimony that biased their view. When judges looked at all the same evidence, and also evidence to which the jury wasn't privy rebutting junk science in the case, they said Rosa didn't do it. So jurors didn't consider all the evidence. That seems disingenuous. (See prior Grits coverage, and listen to a segment on the case on the latest Reasonably Suspicious podcast, plus coverage from a Travis DA Democratic candidates forum over the weekend.)

Use of SWAT raids for routine search warrants creates needless risk
The practice of using SWAT tactics to execute routine search warrants continues to result in unnecessary deaths. A Waller County man was killed in a SWAT raid by police who wanted to seize a computer (someone else's) over alleged possession of child pornography. Can it really require a no-knock raid to seize a computer? This was unnecessary; the man's death was much more a predictable policy failure than it was an accident.

Fewer inmates beaten up more often at TDCJ
Recent inmate deaths at the hands of guards in Texas prisons highlights that use of force by staff has increased dramatically in recent years, reported the Texas Tribune, even as the number of inmates supervised declined and eight prison units closed.

Whistleblower gaining momentum in Sheriff's race
Liz Donegan, the Austin PD whistleblower who was removed as head of that agency's Sex Crimes unit because she wouldn't improperly classify cases as "cleared," is now running for Travis County Sheriff and, remarkably, earned the Austin Statesman's endorsement. Although Donegan was removed from her Sex Crimes post during Chief Art Acevedo's tenure, current Chief Bryan Manley earned ownership of the topic by blaming data errors on victims when the story came out. Him having her as a Sheriff-to-Chief peer would be deliciously awkward.

Bail explainers
Egged on by police, the Dallas Morning News has been blaming Dallas County DA John Creuzot for failures in the legacy bail system. But when they tried to do that in front of the City Council, staff gave everyone a primer on who is in charge of setting bail in Texas: Judges, not prosecutors. In Harris County, a judge demanded an explanation from prosecutors on why they blamed her in the press for a violent criminal's release when they'd never informed her of the details. Meanwhile, at the Paris News (TX, not France), a local reporter offered better explanatory coverage of the bail system than the Dallas News has yet.

Travis County judges dip toes in bail-reform waters
Travis County judges are saying they want to implement bail reform, including requiring defense attorneys at magistration, despite opposition from Travis County DA Margaret Moore. But the Texas Fair Defense Project and their allies say there would still be too much delay before release under the new proposal, and called for changes to the draft. Still, judges taking leadership on this is heartening news. They'd mostly dug in their heels before now.

No extra prosecutors for you, Kim Ogg
For the Harris County Commissioners Court, turning down District Attorney Kim Ogg when she asks for more prosecutors has become habit forming.

Houston crime lab to use disputed DNA mixture software
The Houston Forensic Science Center has begun using STR-Mix software for analyzing DNA mixture evidence. But last fall, a federal district judge in Michigan excluded such software from evidence after a "Daubert" hearing. DNA mixture analyses have been fraught with error for many years. Under the Michigan judge's ruling, based on recommendations from President Obama's forensics commission, STR-Mix software may be used when a) there are no more than three contributors and b) when DNA from the target makes up at least 20 percent of the sample. No word if HFSC intends to abide by those limitations.

Cherry picking data for scary headlines
The Austin Statesman issued a story with the headline: "Violent crimes with homeless suspects, victims went up in 2019, data show." The big news was that reported violent-crime incidents in the city increased by one percent last year, with a small increase attributable to the city's homeless population. What they didn't say was that Austin's population has been growing by 2-3% annually, so the rate likely decreased! Austinites were less likely to be victimized by violent crime last year than the year before. Why wasn't that the headline?

Defending Austin's federal courthouse architecture
The Department of Justice wants all federal courthouses to look like Roman temples and specifically criticized Austin's federal courthouse as an example of what they don't want. But I really like the federal courthouse in Austin. I was there recently for a hearing in the Rosa Jimenez case, then later to retrieve audio from the clerk. It's incredibly well-designed, with much more natural light and customer-friendly arrangement than most of them. Here's more on the Austin courthouse's architectural approach.

Fines and fees
Two essays on fines and fees for you:

• Marshall Project: The True Cost of Court Debt
• The Hill: Fees, fines, and the ability to pay

'Doing justice isn't left, it's right'
The Texas Public Policy Foundation's Marc Levin thinks progressive prosecutors are mis-labeled.

DNA mixture math errors discovered in 2015 weren't the first

Grits this week revisited the 2007 "Bromwich report" regarding what then was the Houston PD crime lab while researching DNA-mixture evidence issues and realized something I hadn't understood at the time - or perhaps just never connected the dots: Errors calculating DNA-mixture probabilities were at the center of the years-long crisis that engulfed the Houston crime lab's DNA section for most of the aughts.

Readers will recall, in 2015, we discovered that all Texas crime labs were miscalculating DNA-mixture probabilities - in one case, a sample jurors were told matched one person out of a billion really matched one out of 50. Well, the same thing was happening at the Houston crime lab. Check out this chart from the Bromwich report identifying cases where DNA analysts vastly over-estimated the probability ratios:

So in Houston, they discovered the crime lab was miscalculating probabilities in the aughts, updated the math, then in 2015 discovered that math was wrong and had to update it again. Wow! This '07 comment from Bromwich about the HPD lab would have applied equally to every Texas crime lab as recently as 2015:

It is clear that DNA analysts in the historical Crime Lab ... did not fully understand the scientific basis for calculating frequency estimates from DNA profiles obtained from evidence samples and that they were not trained in the methods of properly calculating statistics associated with DNA mixture profiles and partial DNA profiles.

Notably, the method Texas labs were advised to shift to in 2015 - "probabilistic genotyping," most commonly using equipment/software from a company called STR-Mix - lately has itself come under fire. A federal judge out of Michigan recently excluded such evidence in a Daubert hearing. So it's possible DNA analysts will need to re-do their math again.

Returning to the Houston PD crime lab example, these are in many cases radical downward adjustments. From one in 6.3 million to one in 30? From a one in 2.9 million chance to one in 5? Would juries have convicted if they'd heard the lower numbers? I wonder if other crime labs were using the same method as HPD's during this period?

Of the cases in this chart, Franklin Alix was executed despite DNA testimony in his case having been declared unreliable. Only Josiah Sutton was ever exonerated; 11 remain incarcerated. By all accounts, the criminal-defense bar in Houston failed to step up in many instances, leaving most of these folks un-or-poorly represented.

It's highly likely there are more actually-innocent people on that list, but we'll never know. The DNA lab's other big problem was that it had a leaky roof and most of the historical DNA evidence was destroyed and couldn't be re-tested.

See prior, related Grits posts:

• Digesting new Daubert opinion on DNA mixtures
• Most crime labs accused innocent person in DNA mixture study
• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• Courts punt on forensics surrounding DNA mixtures
• Black box DNA mixture analysis method used in NYC turned out to be flawed
• Changes to DNA mixture software raise questions about old versions
• News flash: 'Touch DNA' doesn't necessarily require touching
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

The iceberg's tip: CA man walks free bc of DNA-mixture SNAFU

It's perhaps a testament to the reduction in size of and competition among MSM outlets that nobody has yet covered the problems with DNA-mixture forensics raised by federal District Judge Janet Neff of Michigan's Western District* (discussed by Grits here and on the latest Reasonably Suspicious podcast). That must change; some full-time reporter(s) must step up. These issues deserve high-level coverage and national context that this humble, regional blog cannot provide.

Once you begin to pay attention, these cases are cropping up everywhere. In San Diego, we have the case of Flamencio Dominguez. In 2011, he was convicted of a 2008 murder based on DNA mixture evidence and sentenced to 50 years, reported the San Diego Union Tribune. Months before the trial, the crime lab realized the mathematics behind their old DNA-mixture analyses used an invalid baseline. They decided to abandon their old approach and switch to "probabilistic genotyping" instead.

(Similarly, in Texas in 2015, crime labs discovered every lab in the state was using erroneous math in DNA-mixture analyses in ways that risked falsely accusing innocent people, see coverage here and here. In Travis County alone, 11 cases were found where suspect matches changed to "inconclusive." Our crime labs, too, were advised to switch to probabilistic genotyping.)

When the baseline issue was corrected in Mr. Dominguez's case, the new algorithm went from accusing him to "inconclusive." But prosecutors did not tell Mr. Dominguez's counsel about the change. The defendant was convicted based on what now is admittedly erroneous DNA math, and his lawyer wasn't aware of the flawed DNA-mixture protocols until six years later.

After his lawyer found out about the inaccurate math, Dominguez prevailed in 2017 on a habeas corpus claim and was released from prison. But prosecutors decided to try him again, this time using a tool from a company called STR-Mix based on probabilistic genotyping. That was the forensic tool Judge Neff evaluated in Michigan. The bottom-line assessment in Judge Neff's opinion was that:

The DNA evidence sought to be admitted in this case—in essence, that it is 49 million times more likely if [the defendant] is a contributor to the DNA on the gun than if he is not— is not really evidence at all. It is a combination of forensic DNA techniques, mathematical theory, statistical methods (including Monte Carlo-Markov Chain modeling, as in the Monte Carlo gambling venue), decisional theory, computer algorithms, interpretation, and subjective opinions that cannot in the circumstances of this case be said to be a reliable sum of its parts. Our system of justice requires more.

Courts in California were robbed of their chance to decide whether they agree. San Diego prosecutors essentially let Dominguez plea out to time served because the company STR-Mix would not allow the state courts to examine their source code without a slew of non-disclosure agreements the court deemed inappropriate, reported the Union-Tribune:

court records filed last month show that the company wanted Speredelozzi and his experts to sign a restrictive non-disclosure agreement and abide by other restrictions. on Oct. 23 [Judge Charles G.] Rogers declined to require the [defendant's] lawyer sign the agreement, and warned that if the company failed to comply, he might exclude the DNA evidence all together from the trial. On Nov. 7 a lawyer for the company wrote to Speredelozzi they would not provide the crucial source code for the software without the non-disclosure agreement.

So the company chose to let the case against an alleged murderer fall apart rather than let California courts review its source code. But Judge Neff in Michigan already had experts do just that! Is the company afraid other courts may reach similar conclusions and disallow or limit their product's use? The execs at STR-Mix must really be feeling the heat!

One also wonders if the good folks at the Houston Forensic Science Center, which late last month announced they would begin using the STR-Mix software, might now begin to consider that decision ill-timed? Certainly, after Judge Neff's decision, one would question using it for either mixtures involving more than three sources or three-source mixtures where the target makes up less than 20 percent of the sample. Lots of trace-DNA samples are submitted to crime labs that don't meet those criteria!

On the November episode of the Reasonably Suspicious podcast, my co-host Mandy Marzullo and I discussed Judge Neff's opinion and its implications for crime labs interpreting DNA mixture evidence. I pulled out that segment as a stand-alone; you can listen to it here:

The news about Mr. Dominguez's case hadn't yet come out, so we didn't talk about that. But the segment discussed some of the recent history of ever-changing DNA-mixture math and explored the reasons underlying Judge Neff's decision.

IMO this is going to become a significant story with national and international implications (everybody relies on the same science) and lots of twists and turns over the next few years. We need some journalists with chops to jump on this coverage on the front end. Wrongful convictions have gone unchallenged simply because not one reporter in the whole country is covering this beat.

For more background on this controversy, see:

• National Institute of Standards and Technology: "DNA Mixtures: A Forensic Science Explainer" (April 2019)
• Ryan Forensic: "What in the world is going on with forensic DNA mixture analysis?" (April 2017)

*Except Techdirt, which picked the item up from Grits.

On the dangers of dick-ish drug enforcement, racist cop rose in Austin PD ranks, indigent defense denied in Amarillo, and other stories

Just in time for the drive to Grandma's house, here's the November 2019 episode* of Just Liberty's Reasonably Suspicious podcast covering Texas criminal justice politics and policy, co-hosted with Amanda Marzullo of the Texas Defender Service.


In this month's episode:

Introductory tomfoolery

• ¡Poncho!

Top Stories

• Houston PD narcotics division a black hole of bad practices (2:32)
• Federal judge defines limits of DNA mixture software (8:54)

Fill in the Blank

• Victoria County Jail death from drug withdrawal (16:25)
• Racist Austin PD assistant chief outed, what's next? (21:01)
• Indigent defense shortchanged in Amarillo (24:50)

The Last Hurrah (29:30)

• Denouement of Harris County bail litigation
• Why Greg Abbott owns a homeless camp
• Rodney Reed execution stayed

*N.b. It really is the November episode, despite my embarrassing screw up in the intro to say it's June. 

Find a transcript of this episode below the jump.

Digesting new Daubert opinion on DNA mixtures

Grits earlier referenced the opinion from federal District Judge Janet Neff, in Michigan's western district, who directly evaluated the most common, commercial DNA-mixture software using probabilistic genotyping under the Daubert standard for whether to allow expert testimony. Texas courts heretofore have let in such evidence but avoided a direct Daubert evaluation.

There was a Texas connection in the case. Dr. Michael Coble, of the University of North Texas Health Science Center, was the expert employed by the prosecution. He has been one of several key advisers to the Texas Forensic Science Commission and the legal system in general as the state has struggled with this DNA-mixture issue. The expert for the defense, whose arguments more frequently prevailed in Neff's opinion, was Dr. Dan Krane, from Wright State in Ohio.

Judge Neff evaluated each prong of the Daubert standard as it related to the STR-Mix DNA mixture-analysis software. This program employs high-end math that few DNA-lab practitioners can actually reproduce called "probabilistic genotyping." But programs are only as good as the assumptions that guide their work, and some of the assumptions in each case - e.g., the number of contributing samples - are inputted by the user. There are subjective judgment calls throughout the process. Further, because the math results from tens of thousands of randomized guesses, the results aren't replicable test to test.

Judge Neff was disquieted that, unlike for one-to-one DNA matching or two-source mixtures, as with a rape kit, there are no controlling standards for interpreting DNA mixtures, especially for samples with very small amounts of DNA - maybe only a few scattered cells.

The judge also made the interesting point that, even if the math behind the STR-Mix software is impeccable, there were international standards for coding such high-end math projects and the STR-Mix software didn't meet them all. Coble was dismissive that other fields might have standards to contribute, but the product in question is so obviously an intersection of an array of multidisciplinary work, the judge didn't buy it.

Further, the judge echoed concern that most validation studies on the software so far have been conducted by company principals. And while there was peer-reviewed support for STR-Mix (again, mostly for studies by people with a financial stake in the project), none of that peer-reviewed work focused on low-level samples as in this case, where just a few cells were found that were said to belong to the defendant.

One point that's always bugged me was raised in an expository section of the opinion but not included in her reasons for her Daubert denial: “[D]ue to the methodologies used in STRMix, '[t]he results of no two analyses will be completely the same',” she wrote. Elsewhere, she quoted an academic article noting that, "for a technique to be broadly applicable, it must be based on measurements that can be replicated." Supporters of the software, however, say replicability isn't necessary and it should be good enough if the results were all in some acceptable range. Regardless, while Judge Neff clearly observed this incongruity, it wasn't included in her final reasons for her ruling.

Clearly the most persuasive piece of evidence for Judge Neff was the 2016 PCAST report which expressly stated that STR-Mix was validated for up to three sources if the sample to be identified is at least 20 percent of the total. She did not find further validation regarding smaller samples, and the very small sample in this case, she believed, merited even greater caution. She essentially treated the threshold cited by PCAST as a bright line regarding the current state of the science. STR-Mix can be used for traditional one or two-source DNA matching, she ruled, and for three-source mixtures if the sample to be matched made up 20 percent or more of the whole. But otherwise, the estimates would be inadmissible.

One small upside for backlogged crime labs using STR-Mix: This would make it easier to screen DNA mixture submissions. Samples with more than 3 contributors should probably be rejected up front as unanalyzable "crap," to use the words of a prominent DNA expert who advised Texas crime labs on the topic.

Errors in such cases can cause false convictions, so Grits is glad to see this increasingly dirt-common evidence more carefully vetted. From the time probabilistic genotyping was first introduced to Texas crime labs in 2015, experts were warning against stretching the limits of the math (i.e., trying to analyze "crap"). Judge Neff would set a bright-line threshold, and given the current state of the literature, it's not an unreasonable choice. In many ways, it's the most defensible (it would probably be easier to get past it by improving the tech than defeating her arguments). But whatever threshold one chooses, her decision highlights that one must exist, even if courts haven't heretofore required it.

I'd quoted it earlier, but I loved Judge Neff's conclusory analysis of current DNA-mixture-evidence practices, so let's give her the final word here:

The DNA evidence sought to be admitted in this case—in essence, that it is 49 million times more likely if Daniel Gissantaner is a contributor to the DNA on the gun than if he is not— is not really evidence at all. It is a combination of forensic DNA techniques, mathematical theory, statistical methods (including Monte Carlo-Markov Chain modeling, as in the Monte Carlo gambling venue), decisional theory, computer algorithms, interpretation, and subjective opinions that cannot in the circumstances of this case be said to be a reliable sum of its parts. Our system of justice requires more.

Anyone interested should definitely read the whole opinion. This summary hardly does it justice.

MORE: From Techdirt.

See prior, related Grits posts:

• Most crime labs accused innocent person in DNA mixture study
• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• Courts punt on forensics surrounding DNA mixtures
• Black box DNA mixture analysis method used in NYC turned out to be flawed
• Changes to DNA mixture software raise questions about old versions
• News flash: 'Touch DNA' doesn't necessarily require touching
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

DNA mixture software widely used in Texas disallowed by MI federal judge

For some time, Grits has been skeptical whether DNA-mixture software would hold up to "Daubert" scrutiny if judges were to directly evaluate it. In 2015, this blog reluctantly broke the story in Texas about errors and uncertainties regarding how to evaluate DNA mixture evidence. (They were being openly discussed by the Forensic Science Commission and the Court of Criminal Appeals' Criminal Justice Integrity Unit when I worked for the Innocence Project of Texas, but reporters weren't covering the complicated, math-heavy story.) After that, I've followed the issue from afar even after I stopped tracking forensics in any professional capacity.

STR-Mix is one of a handful of companies selling their product as a solution to the conundrum of how to evaluate DNA mixture evidence. Now, a federal district judge in Michigan, Janet Neff, after an exhaustive review of the science, has ruled directly on the Daubert-based admissibility questions surrounding this product that Texas courts punted on. In particular, Judge Neff ruled in favor of defendant David Gissantaner, concluding that

The DNA evidence sought to be admitted in this case—in essence, that it is 49 million times more likely if Daniel Gissantaner is a contributor to the DNA on the gun than if he is not— is not really evidence at all. It is a combination of forensic DNA techniques, mathematical theory, statistical methods (including Monte Carlo-Markov Chain modeling, as in the Monte Carlo gambling venue), decisional theory, computer algorithms, interpretation, and subjective opinions that cannot in the circumstances of this case be said to be a reliable sum of its parts. Our system of justice requires more. 

It is the Court’s hope that this decision brings to light the shortcomings or, at the very least, points of inquiry necessary in evaluating this advancing technology as a tool in forensic DNA analysis. There must be a dialogue among key players in the general interest of the development and refinement of the technology, the software and its application by the individuals charged with its use in the field, rather than post-hoc testing of its reliability in the context of a criminal prosecution where the ultimate question is the freedom and guilt or innocence of the person of interest. 

Here, because the sum of the parts simply does not add up to a reliable whole, the DNA analysis/likelihood ratio resulting from the use of the STRmix probabilistic genotyping software must be excluded. Defendant’s motion to exclude evidence is granted. An Order will enter consistent with this Opinion.

Wow! Obviously, this doesn't apply in Texas nor anywhere in the 5th Circuit. But it's the first time to my knowledge that a federal judge has taken a deep dive into the science behind proprietary DNA mixture software and rejected it based on the Daubert standard. Welcome to the party, Judge! Grits was beginning to feel a bit lonely in that view!

In one-to-one DNA matches, or two-source matches like a rape kit where the DNA of the victim is known, DNA evidence remains the gold standard of forensic science. But analyzing DNA mixture evidence - e.g., "touch DNA" on a doorknob or DNA from a blanket on a couch - remains a more subjective process that's sometimes more of a guess than a "match."

I want to delve into the judge's opinion more deeply before commenting further, but this seems like a big development. We know at least some Texas labs use the same proprietary DNA software the federal judge in Michigan disallowed.

Clarification: More on this soon, but having now more closely read the opinion, Grits should clarify that the court recognized STR-Mix software results could be considered valid for distinguishing up to three-sourced DNA mixture samples, but only if the DNA one is attempting to match makes up at least 20% of the sample; in this case it made up 7%. So saying STR-Mix software was "disallowed" in the headline was too broad. Under this ruling, STR-Mix results for one and two source DNA samples would remain admissible, and for three sources, if the sample one wanted to identify made up at least 20 percent of the total. But it couldn't be used when there were only trace amounts from the third DNA source, or when there are four or more sources. Here is a spot where user expectations for forensics exceed the capabilities of actual science. In many ways, the judge's advice corroborates what experts have said in Texas since this issue first arose: beyond a certain point, DNA mixtures become un-interpretable "crap." You can push the math further, but the evidence doesn't therefore improve.

Most crime labs accused innocent person in DNA mixture study

A new study described in the New York Times confirms the problems with DNA mixture evidence Grits has described on this blog for several years. Even more disturbing, the study's authors hoped to keep the results secret until they were pressured by other scientists into publishing, and then downplayed their bombshell findings.

It's understandable that they feared their study would undermine the work or practitioners analyzing DNA mixture evidence. Check this out:

Researchers from the National Institute of Standards and Technology gave the same DNA mixture to about 105 American crime laboratories and three Canadian labs and asked them to compare it with DNA from three suspects from a mock bank robbery. 

The first two suspects’ DNA was part of the mixture, and most labs correctly matched their DNA to the evidence. However, 74 labs wrongly said the sample included DNA evidence from the third suspect, an “innocent person” who should have been cleared of the hypothetical felony. ...

One shocking result from the new N.I.S.T. study is that labs analyzing the same evidence calculated vastly different statistics. Among the 108 crime labs in the study, the match statistics varied over 100 trillion-fold. That’s like the difference between soda change and the United States’ gross domestic product. These statistics are important because they are used by juries to consider whether a DNA match is just coincidence.

In other words, more than 2/3 of crime laboratories analyzing the evidence would have falsely accused an innocent person.

Equally disturbing, the authors realized their findings were explosive, and so attempted to scuttle or downplay them. "If some of us had not complained publicly, it may not ever have been published," noted Greg Hampikian, the Boise State biologist whose Times op ed highlighted the study.

Moreover, "Neither the paper’s title nor the abstract mention the shocking findings. And the paper contains an amazing number of disclaimers," including one "apparently intended to block courtroom use."

Hampikian says there have been at least five DNA exonerations based on flawed DNA mixture evidence: "our Boise State University laboratory has re-examined a few select cases and already persuaded courts to overturn a conviction in New Mexico, two in Indiana and two in Montana. We have also helped identify a new suspect in a 23-year-old murder."

Grits considers it flat-out scandalous that NIST withheld publication for so long and then attempted to downplay the results. That behavior smacks of bad faith.

But I'm glad problems with DNA mixture evidence are receiving more public attention. Maybe one day, Texas courts will acknowledge them as well.

MORE: From Forbes, "Framed by our own cells: How DNA evidence imprisons the innocent." 

See prior, related Grits posts:

• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• Courts punt on forensics surrounding DNA mixtures
• News flash: 'Touch DNA' doesn't necessarily require touching
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

Changes to DNA-mixture software raise questions about old versions

STR-Mix, one of the DNA-mixture testing conpanies hoping to prove their algorithms can successfully identify suspects when the DNA is mixed and messy, has issued a new version of its software, a press release informs us.

The biggest difference?

Users can now input a range for the possible number of DNA contributors. In prior versions, they had to guess how many people's DNA is in a sample, even if they didn't and couldn't know.

For example, say an analyst is confronted with a "touch DNA" sample taken from a swab on a doorknob. They know for sure there is DNA there, but how many contributors are represented in the sample? Two, three, six, fourteen? How many people have touched the doorknob since it was last cleaned? And when it was cleaned, was old DNA wiped away or just damaged or deformed by the cleaning product?

Under the old method, analysts had to guess at the number of contributors - say, "5" - then the software spit out a probability based on those assumptions. The software did not adjust for the possibility they guessed wrong. Now it does.

Which raises the question, what happens when one re-runs old tests where analysts guessed the number of contributors? According to the press release, users can use the new software to "calculate multiple" likelihood ratios for old samples based on "multiple reference inputs." So which "likelihood ratio" should courts rely upon if there are multiple choices? How are judicial gatekeepers supposed to evaluate a situation when the original likelihood ratio testified to in court based on their software is now either deemed wrong, or is just one of multiple choices now being offered?

It seems inevitable that the software in those earlier cases overstated the probability that any given DNA belonged to a defendant. In one notable example, STR-Mix's old software accused a defendant while another, competing black-box service, TrueAllele, excluded the defendant as a suspect. One wonders, if they re-ran the test with the new software, using a range of possible contributors to the mixture, might some of STR-Mix's "likelihood ratios" now agree with TrueAllele's exclusion?

Grits does not believe any black-box software whose inner workings are not publicly available for peer review by opposing experts - right down to the all-important coding language - should be used in court to interpret DNA mixtures. Too many well meaning people keep getting the math wrong.

See related Grits posts:

• Courts punt on forensics surrounding DNA mixtures
• A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
• Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
• Resources on DNA mixtures
• The challenge of notifying defendants in large-scale forensic error cases
• DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims

Newsflash: "Touch DNA" doesn't necessarily require touching

Grits has discussed problems with DNA mixture evidence - particularly regarding so-called "touch DNA" - at some length. But conversations in Texas about the math have ignored an even bigger problem raised in this Wired magazine article: Touch DNA from an individual can be transferred to places to which the suspect has never been. 

The featured case involved an innocent man whose DNA wound up under the fingernails of a murder victim. But the crime was committed while he was hospitalized and could not have committed the offense. It turned out, the same paramedics treated the suspect and responded to the murder scene, somehow transferring his DNA in the process.

This may be more common than anyone - even innocence advocates - have understood. It turns out, for example, about one in five people "walk around with traces of other people's DNA on our fingernails," a study found. 

Or consider: Scientists have determined that a man who shakes hands with another person then goes to the restroom may end up with their DNA on his penis. Or, according to this Canadian study, a father's DNA may frequently wind up on his daughter's underwear because it "migrates there in the wash." (One wonders how often that latter circumstance may have contributed to false convictions in child molestation cases?)

The emerging questions around touch DNA - both the math surrounding the analysis and the assumptions surrounding what the results mean - are a lot more complex than anyone could have imagined five or ten years ago. Unfortunately, the criminal-court judges charged with sorting out the mess are ill-equipped and unprepared to do so.

Indeed, if these questions are resolved a decade from now, Grits will be pleasantly but seriously surprised. The Trump Administration shut down forensic reform efforts soon after Jeff Sessions became Attorney General, and it's doubtful state-level activities like the review at the Texas Forensic Science Commission can forge national or (really) international standards, which is what's ultimately needed.

Myths and dilemmas surrounding DNA testing backlogs

The Fair Punishment Project has a good roundup in its In Justice Today Texas newsletter of stories on Texas' efforts to reduce the so-called "rape kit backlog," and I was pleased to see among them this item identifying "5 myths" surrounding the controversy over un-tested rape kits from the co-founders of People for Enforcement of Rape Laws.

I agree with most of that commentary, but there's one other "myth" they didn't cover: That every un-tested rape kit has the potential to identify a criminal. Many times, the reason rape kits go un-tested is that the identity of the alleged assailant isn't the issue. Rather, the issue is whether a sexual act was consensual, and the existence of DNA doesn't prove culpability.

For more context, check out the lab director from the Los Angeles Sheriff's Office discussing the cost-benefit issues surrounding testing of rape-kit backlogs at a National Institute of Justice event in 2010. They spent $1.7 million to analyze their rape-kit backlog to find just two viable suspects.

In Houston, by contrast, they found many more un-solved cases when old rape kits were tested. IMO that's because HPD was doing a much poorer job of investigating sexual assaults than the Los Angeles Sheriff. This speaks to one of the main critiques of the authors of the "5 myths" article:

The failure of law enforcement to properly investigate rape is not limited to testing rape kits. Too often, investigations are closed before a kit is even taken. Investigating and solving a rape case takes actual police work. Detectives must find and interview witnesses, interview the victim, track down evidence, corroborate the account of events with both the victim and witnesses, and compare the case details to unsolved cases to try to detect patterns. Yet instead of doing this necessary legwork, police unfound, downgrade, and “disappear” rape cases. 

Take the Detroit police department, which, “under nine chiefs, both male and female, sustained a culture in which officers routinely neglected rape complaints or actively discouraged victims from seeking redress, all without fear of consequence,” according to Detroit Free Press columnist Nancy Kaffer. The department, like others, has a long history of underreporting rape. In 2001, the department admitted that the statistics it reported to the FBI for rape arrests — which were at least twice the national average throughout the 1990s — were seriously flawed.

That's a fair analysis, but it doesn't apply to every agency. The examples of HPD and the LA County Sheriff illuminate how widely that can vary from department to department. Not every agency suffers from a culture of neglected rape complaints, but when it happens, it compounds tragedy in bunches.

At one point, the National Institute of Justice had issued a grant to Houston PD (also Detroit) to develop protocols regarding when it was and wasn't appropriate to perform DNA testing on rape kits. There is a detailed website illuminating all sorts of interesting aspects about this project, but Grits has still never seen any final recommendation regarding exactly under what circumstances law enforcement should choose not to have a rape kit processed at the crime lab. (If I've just missed them, please, somebody point them out; once my job ended as the Innocence Project of Texas policy director, I stopped tracking these topics closely.)

Grits mentions this not to discourage testing of rape kits, nor to make excuses for those in the past who allowed rape cases to languish un-investigated. But because there are, in fact, viable circumstances under which the expense of rape-kit testing isn't justified - particularly in cases where the principle issue is consent, not identity - until such protocols are promulgated, law enforcement will continue to make decisions about whether to test rape kits on a case by case basis.

Better to just create a set of reasonable best practices then pressure everyone to follow them, than to insist that every rape kit be tested and backlogs always = zero. Then, police will know what's expected of them and the public will better understand why an un-tested rape kit isn't per se evidence of police negligence or incompetence.

Another question raised by such test-or-no-test protocols: Might there be rape victims who are unnecessarily subjected to invasive evidence-gathering procedures who needn't be? It's not fun to go through that (I'm told), and in cases where it's clear from the get-go it's not necessary, maybe a lot of these women needn't go through the experience in the first place. Stranger rape is pretty rare, after all. If police were trained up front when rape-kit evidence might be pertinent to an investigation and when it's unnecessary, maybe you could chip away at the number of un-tested kits from that direction.

Finally, having mused over this recent spate of reporting on the topic, Grits should point out a related, recent article from the Washington Post that raises the fraught and difficult question looming over all these rape-kit backlog debates: What happens when the results come back? Nothing simple, is the short answer.  Houston's approach seems as reasonable as any:

Houston tried a different model. A hotline was set up and publicized, so that any victim who wanted information about their old kit could ask for it. Then, police and prosecutors combed through the CODIS hits and decided which cases actually had a chance of moving forward in the criminal justice system. Victims were notified only if their cases seemed “actionable.” “What’s at stake is the well-being and mental health of sexual assault victims,” says Noël Busch-Armendariz, a researcher who was involved in Houston’s process. “You never know where people are in their lives and what support systems they have or don’t have ready for them.”

Louisville, by contrast, notified every woman on the list her kit would be tested, even if there turned out to be no one else's DNA in the sample. "In Virginia," according to the Post, "this dilemma would ultimately pit police, prosecutors, advocates and lawmakers against one another, making the situation far more complicated than they ever intended. Everyone wanted to do the right thing for victims; there was just no way to know what that was."

Ironically, for a situation in which everyone wants to "do the right thing for victims," often victims opinions in the process are either unsolicited or roundly ignored.

These are incredibly difficult questions which won't be resolved anytime soon. The USDOJ under the Trump Administration has issued a recommendation that every rape kit associated with a criminal complaint be tested, reported the Post, but law enforcement agencies on tight budgets likely will balk at truly unnecessary testing, especially if it becomes required in volume.

Your correspondent doesn't have solutions to these issues, but there's a growing urgency to confront them.

Pop quiz on forensics

A recent research paper from Jonathan Koehler, a Northwestern University law professor, titled "How trial judges should think about forensic science evidence," opened with a short quiz. Grits readers should be well educated on these topics, but let's see how you do:

"Here is a forensic science test for you. Please answer each of the three questions below True or False.

"1. Scientific tests conducted over the past 100 years have repeatedly demonstrated that everyone has a unique set of fingerprints.

"2. Recent scientific studies show that the chance that DNA samples from two different people will be identified as a “match” by a competent, well-trained DNA examiner is less than one in a million.

"3. Data from scientific tests conducted over the past few decades provide a reliable basis from which to estimate the accuracy of most forensic methods that have been admitted in U.S. courts.

Interview: Peter Neufeld, co-founder of the national Innocence Project, on prospects for state and national forensic-science reform

In the December episode of the Reasonably Suspicious podcast, we published an excerpt from an interview Grits conducted with national Innocence Project co-founder Peter Neufeld. We mainly discussed forensic-science topics including the abolition of the national forensic science commission, of which he was a member, and DNA mixture controversies. You can listen to the full interview here.


Find a transcript of our conversation below the jump.

Christmas and the Surveillance State: December Reasonably Suspicious podcast

Check out the December edition of Just Liberty's Reasonably Suspicious podcast, covering Texas criminal justice policy and politics. Two great interviews this month - one with reporter Brandi Grissom Swicegood about the alleged abuse and turmoil at the Gainesville State School, and another with Peter Neufeld, co-founder of the national Innocence Project, regarding forensic-science reform. You can listen to the latest episode here, or subscribe on iTunes, Google Play, YouTube, or SoundCloud.


If you haven't subscribed yet, take a moment to do so now to make sure you won't miss an episode. Topics this month include:

Top Stories
US v. Carpenter: SCOTUS appears likely to require a warrant for cell-phone location data.

Interview

Brandi Grissom, discussing the staff-on-youth sex scandal at the Gainesville State School.

Home Court Advantage

Evaluating a sharply split decision from the Texas Court of Criminal Appeals upholding a first-degree felony drug conviction in which a police officer stole the product and laced sheetrock with less than a gram of cocaine to frame the defendant. (See prior Grits coverage.)

Interview

Peter Neufeld of the national Innocence Project, discussing forensic science reform.

Errors and Updates

• Forensic hypnosis
• Austin City Council rejected police union contract
• DNA mixture evidence

The Last Hurrah

• TDCJ prison understaffing and staff safety
• Death penalty use declining: A first for Harris County in 40 years
• Dallas pilot program de-escalates mental-health calls by sending medical staff instead of cops

Find a full transcript of the podcast below the jump.

Police unions, the media, and me ... and other stories

Grits fell ill at the end of the holiday weekend, am only now really back on my feet, and find myself in a desperate need to clear my browser tabs. So, y'all get a roundup of all the stuff I don't have time to blog about right now.

Police officers indicted more often, but seldom convicted after shootings
More police officers in Texas are being charged after questionable shootings, but prosecutors who were able to convince grand juries to indict have been less successful at securing convictions at trial, reported Tasha Tsiaperas the Dallas Morning News (who has a really cool, bond-villain-type name!). In related news, Grits contributing writer Eva Ruth Moravec had a feature in the Houston Chronicle about a Freeport police officer acquitted by a Brazoria County jury for shooting his unarmed neighbor in the next apartment. It was as negligent a situation as one could imagine, so maybe civil court is still an option: The cop apparently slept with a loaded gun in his bed (and in this case, his finger on the trigger, safety off) and fired it through his headboard into the next door apartment. Attn: Texas Monthly, this is mandatory Bum Steer material.

Police unions, the media, and me
Most local media coverage of the Austin police contract has been dismissive of the push by the Austin Justice Coalition and their growing list of allies to get the city council to vote "no." This, despite opposition to the contract from hundreds of signators, more than a dozen groups, and even though, in a staunchly Democratic county, D precinct chairs unanimously voted for a resolution urging city council to kill the deal. Currently the vote is scheduled for December 14th. At the Texas Observer, Michael Barajas has a feature explaining more fully "How the expiration of Austin's police union contract could be a rare opportunity for reform." Former CLEAT mugwump and long-time police-union leader Ron DeLord, who was lead negotiator for the Austin Police Association on the contract, complained on Twitter that Barajas didn't talk to him. So I suggested DeLord do a podcast interview to air his views, and he agreed(!). I hope y'all are looking forward to that as much as I am. Mainly I want to talk to him about his books: See Grits' discussion of his latest one, and also the opening segment of our August Reasonably Suspicious podcast discussing his remarkably accurate prediction of Texas' police-pension crisis, which was a contentious legislative imbroglio this year resulting in outcomes with which no one is happy, but which brokered an uneasy, temporary truce among the parties. If the economy holds.

Evaluating police bodycams
Coupla items here: A new study found bodycams reduced use of force episodes at the Las Vegas PD while providing quality evidence that supported criminal convictions mostly of defendants, not cops. But many advocates, your correspondent included, believe the laws limiting transparency around footage reduce the accountability benefits. Supporting that claim, Nick Selby wrote on The Crime Report that, "In October of this year, the biggest-ever randomized study of body cameras showed no measurable reduction in complaints or use of force by officers in Washington, D.C." So the jury's still out on whether this will turn out to be an important accountability measure, as they were originally pitched in the hyped aftermath of the Ferguson protests.

With shortfall looming, a way to reduce DPS crime-lab volume
Plano PD is testing a device that can tell whether DNA exists on a piece of evidence before they send it to the lab instead of after. If this works as advertised, Governor Greg Abbott, his grants division, and DPS crime-lab folk should take heed. It might even be worth the Governor considering emergency grants to buy these for the biggest users of DPS DNA lab services to reduce the volume of cases. A lot of material sent has no DNA on it at all, and to screen that out up front would make a big difference on volume in a biennium when the Legislature told DPS to collect fees for part of their budget and the Governor has stopped them from doing it. That creates a shortfall unless they can find ways to reduce volume. This could be an important one.

Death penalty now mainly an LWOP plea-bargain chip
Texas will perform no more executions this year after the Court of Criminal Appeals issued a stay and halted Juan Castillo's planned trip to the death chamber. But expect life without parole sentences to keep stacking up as Christmas approaches. In 2016, according to the Texas Office of Court Administration, just three new death sentences were secured by Texas prosecutors, compared to 64 LWOP sentences. (A whopping 426 total capital cases were filed statewide last year, which was a ten percent increase from the year before, so many are called, but few are chosen.) These days, the death penalty is mostly a threat to get people to accept life without parole sentences in a plea bargain.

Harris judges sabotaging pretrial release order from feds
Harris County judges are sabotaging the pretrial release system mandated under a federal court order by refusing to release thousands of defendants who qualify, according to a report by the Texas Tribune (which doesn't use such strong language but supplies all the relevant details). Instead, the Sheriff has to release them outside of the purview of the Pretrial Services system, where they predictably have higher no-show rates. (One of the most important things pretrial services does to get them there are reminder calls and texts.) The key problem:

Defendants who are ordered for no-cost release by a judge or magistrate are entered into the county’s Pretrial Services department, which works to ensure those out on personal bonds know the date of their next court appearance and can provide additional conditions like drug testing, mental health services or GPS ankle bracelets. Those released by the sheriff aren’t monitored once they leave the jail.

On judges as gatekeepers, redux
Here's a law review article by Stephanie Damon-Moore on a question Grits has thought about a lot: "why trial judges, who have an independent obligation to screen expert testimony presented in their courts, would routinely admit evidence devoid of scientific integrity."

Beyond mass incarceration: Felony sentences rose quicker than imprisonment rates
We've established that the War on Drugs contributed more to mass incarceration than critics like John Pfaff have claimed. But now the statistician who best proved that has come out with a new analysis demonstrating even deeper, more insidious aspects to the drug war's role. Compared to new prison sentences, which themselves skyrocketed, the number of total felony sentences (including probation, deferred adjudication, etc.) went up even faster! See here, and check out the whole analysis:

Reasonably Suspicious: Police unions, collective bargaining, and accountability

Check out the latest episode of the Reasonably Suspicious podcast from Just Liberty. You can listen to it here or access it on all the usual channels: iTunes, Google Play, YouTube, or SoundCloud. 

If you haven't subscribed yet, take a moment to do so now. I'm enjoying the format and am hoping to do some interesting things in the coming months heading toward the 86th Texas Legislature. If the Wall Street Journal's right that the next billion internet users won't type, relying on voice and video, then it behooves an old dog to learn new tricks. And having cool, original music wrapped around the conversation - thanks to producer/guitar virtuoso Gabe Rhodes and some of the finest musicians in Texas (which is saying something) - makes it fun to put together.

This month's episode features three segments on police union politics, including one focused on Austin's "meet and confer" contract presently under negotiation (these highlights from the negotiating table recently made the rounds among city insiders), and a discussion of what Grits had dubbed the police union playbook on spinning to the press in the wake of police misconduct or high-profile "critical incidents." I'm perhaps most excited about the interview with Sam Sinyangwe, Campaign Zero's data specialist who has now twice visited Austin to support including accountability measures in the police union contract (or scuttling it if they're not included). I'll publish the full interview in a few days (in the meantime, you can also check out the speech he gave in Austin in September). But the segment in the podcast on why police unions too often get a political pass was worth the cost of admission.

Lots of other good stuff sprinkled throughout. As always, find a transcript of the podcast after the jump below.

Top Stories: 

• The Police Union Playbook on reacting to critical incidents

• If Harris County prosecutors are screening arrests, why are so many people arrested for Class C misdemeanors?

Interviews:

• Sukyi McMahon and Kathy Mitchell on the Austin police union contract

• Scott Henson interviews Campaign Zero's Sam Sinyangwe on why liberals and conservatives are both reluctant to criticize police union excesses

Game segment: Fill in the Blank

• Bexar and Dallas Counties cease arrests for misdemeanor marijuana possession

• Court of Criminal Appeals still denying DNA testing to capital defendants

• Real costs of incarceration top $1 trillion nationally

The Last Hurrah

• Unions now a minority at Dallas police pension board

• Time to make the Austin crime lab independent?

• Bipartisan push in Congress for asset forfeiture reform

The Case of the Sleeping Gatekeepers, or Bolstering Daubert

Over the past several years - really since the lead-up to the National Academy of Sciences 2009 study of forensics - Grits has been giving increasing thought to the failure of judicial gatekeepers to adequately vet dubious evidence from forensic analysts. The NAS clarified for the lay public for the first time that many common forensic methods have no basis in science, relying instead on subjective analyses. But the full implications of their findings are only beginning to become apparent.

The subject has been especially on my mind since we did a podcast segment in August discussing the lack of judicial gatekeeping when it comes to DNA mixture evidence. In the case we discussed, the trial court admitted two different analyses of the same DNA mixture that came to different conclusions, and told jurors to pick the one they preferred! The intermediate appellate court affirmed the conviction, but didn't evaluate the question of admissibility.

That seemed to me like such an abrogation of the gate keeping function, it's hard to understand what if anything labeled "forensic" Texas judges wouldn't admit into evidence? (I mean, there's dog-scent lineups, but we had to shame them into stopping those.)

Clearly I'm not the only one thinking about how to tighten up these lax gatekeeping mechanisms. In that vein, here are several new academic analyses approaching the question from different angles:

• Paul Giannelli, "Forensic Science: Daubert's Failure"
• Brandon Garrett, Gregory Mitchell, "The Proficiency of Experts"
• Edward Imwinkelried, "The Best Insurance Against Miscarriages of Justice Caused by Junk Science: An Admissibility Test That Is Scientifically and Legally Sound"

I already printed out Giannelli's piece; these get added to Grits' to-read stack.

RELATED: Why criminal court judges are poor forensic gatekeepers.

AND MORE: From Radley Balko.

Reasonably Suspicious: Listen to the podcast, join us for our Launch Party!

After several months of working out the kinks in a soft launch, Just Liberty's Reasonably Suspicious podcast now is up and running on numerous platforms - iTunes, Google Play, YouTube, SoundCloud, etc..  Please subscribe and give it a try! I'm proud of the results so far, and we're improving every time.

We've now got a fresh new logo created by the legendary Guy Juke, plus original music by producer/guitar virtuoso Gabe Rhodes and some of the best musicians in Texas. I couldn't ask for a smarter, more able co-host than Mandy Marzullo from the Texas Defender Service. The excuses for failure are dwindling! :)

Just Liberty will host a podcast launch party in Austin on Wednesday, September 20th to celebrate the new project. Please join us if you can! See our Facebook event page for details.

You can listen to the podcast here, or as usual find a transcript with links to underlying documents and news stories below the jump.



Here are the topics covered in the September 2017 episode:

Top Stories

• Police-union pension crisis predicted by Ron DeLord
• Prosecutors ill-advised to withhold witness statements
• Big implications for Harris County bail-reform litigation

Forensic Follies

• Junk Science Writs and the Goldilocks Problem
• First DNA-mixture "black box" broken open

Last Hurrah (quick takes)

Why criminal court judges are poor forensic gatekeepers

Yes. This.

Why do judges frequently fail to keep faulty forensics out in criminal cases despite the fact that they rigorously enforce Daubert’s gatekeeping requirements when presiding over civil cases? Daubert requires trial judges in both civil and criminal proceedings to determine “whether the reasoning or methodology underlying the testimony is scientifically valid.” As the relevant research reveals, however, judges are far more willing to fulfill their gatekeeping roles in civil cases than criminal ones. Challenges to forensic evidence pretrial, including Daubert hearings, are rare in the criminal context. As the NAS Report makes clear, “the vast majority of the reported opinions in criminal cases indicate that trial judges rarely exclude or restrict expert testimony offered by prosecutors.” The evidentiary standards that apply to expert forensic evidence should be identical in civil and criminal proceedings according to the Federal Rules of Evidence and relevant precedent, yet courts rigorously engage in gatekeeping of such evidence in civil proceedings while giving broad leeway to prosecutors in criminal proceedings. Therefore, the courts’ failure to exclude faulty forensics in criminal cases cannot be explained away simply by pointing to judges’ lack of scientific prowess.

In Just Liberty's next Reasonably Suspicious podcast, out tomorrow, Mandy Marzullo and I discuss the failure of Texas courts so far to adequately perform their gatekeeping function vis a vis DNA mixture evidence, updating a segment from last month's edition.

The passage above from an academic article gets to the heart of the matter: Though in theory, civil and criminal courts should be applying the same standards for scientific evidence, as a general rule they only perform a true "gatekeeping" function on the civil side.

In the past, Grits has been a tad more generous, attributing this outcome to lopsided disparities in resources on the criminal side, while civil disputes are often more equally matched in terms of both sides' ability to hire independent experts, etc.. But the reflexive posture of admitting anything the prosecution wants until their hand is absolutely forced - even disputed evidence like bite marks, hair microscopy, and black-box DNA mixture analyses - after a while becomes hard to deny.

The other segment on forensics in tomorrow's podcast discusses Texas' junk science writ, assessing a law professor's comparison between our version and California's. Left unsaid in that segment is that the reason we need habeas corpus remedies for bad science on the back end of the process primarily stems from judges' failure to perform a meaningful gatekeeping role on the front end. Well, I suppose now it hasn't been left unsaid.

Black-box DNA mixture analysis method used in NYC turned out to be flawed

Today the New York Times and ProPublica published a joint investigation by Lauren Kirchner into the use of DNA mixture evidence in New York titled "Thousands of Criminal Cases in New York Relied on Disputed DNA Testing Techniques."

The story covered some of the same ground as did a recent segment in Just Liberty's Reasonably Suspicious podcast discussing a DNA mixture case out of the Tyler 12th Court of Appeals here in Texas.

The big news out of the ProPublica story: After years of keeping its DNA-mixture analysis algorithms secret, the New York City lab finally had to reveal its source code (aka, the "FST code") to the defense and it didn't pass muster:

Again, the government refused to hand it over on the grounds that it was a “proprietary and copyrighted” statistical tool owned by the City of New York. 

The federal judge granted the defense access to the FST code in June 2016 under an order that bars wider disclosure. (The medical examiner’s office denied ProPublica’s public records request for the code, citing its “sensitive nature.”) 

Nathaniel Adams, a computer scientist and an engineer at a private forensics consulting firm in Ohio, reviewed the code for the defense. He found that the program dropped valuable data from its calculations, in ways that users wouldn’t necessarily be aware of, but that could unpredictably affect the likelihood assigned to the defendant’s DNA being in the mixture. 

“I did not leave with the impression that FST was developed by an experienced software development team,” Adams wrote in an affidavit. Pending more rigorous testing, “the correctness of the behavior of the FST software should be seriously questioned.” Characterizing Adams’ criticisms as merely stylistic rather than substantive, the lab told ProPublica that FST provided reliable calculations. 

Technology consultants wrote the software code for FST, according to a spokeswoman at the medical examiner’s office. Few, if anyone, at the lab or on the state’s DNA Subcommittee had the expertise to double-check the software, said a scientist in the lab who worked on the techniques who asked to remain anonymous for fear of career repercussions. “We don’t know what’s going on in that black box, and that is a legitimate question,” the scientist said, adding that evidence in older cases should “absolutely” be retested in light of growing questions about FST. “As a scientist, I can’t say no.”

The issue of proprietary "black box" software interpreting DNA mixture evidence mirrors the situation with private vendors used in Texas like STR-Mix and TrueAllele, as discussed on the Reasonably Suspicious podcast.

When New York's proprietary software was picked apart by the defense, they found previously unrevealed errors and flaws. Who's to say that won't also be the case with proprietary private software? The likelihood surely is greater than not.

Courts should allow defense experts access to the source code if they're going to admit DNA mixture evidence from these black box systems. There's just too much recent error and loads of evidence that the math and science involved in these black-box interpretations are well beyond the ken of most analysts seeking to interpret them.

Here are a few other tidbits from the ProPublica/Times story which add new information to the discussion here in Texas:

The FBI in May claimed to have validated STR-Mix for use in DNA mixtures involving up to five people. But having a law enforcement agency "validate" the method is far different from having validation from independent sources which would satisfy defense experts. In one famous case out of upstate New York, STR-Mix and their main competitor, TrueAllele, tested the same evidence and came back with different results.

"In the past three years, flaws in DNA methods have temporarily shut down testing in public crime labs in Austin, Texas, and Washington, D.C."

“Lab analysts 'make it seem like it’s a completely objective process,' said Bicka Barlow, a lawyer in California with a master’s degree in genetics and molecular biology. 'But I’m 100 percent convinced that there are many people who are incarcerated who were convicted with DNA evidence who are innocent.'”