Readers will recall, in 2015, we discovered that all Texas crime labs were miscalculating DNA-mixture probabilities - in one case, a sample jurors were told matched one person out of a billion really matched one out of 50. Well, the same thing was happening at the Houston crime lab. Check out this chart from the Bromwich report identifying cases where DNA analysts vastly over-estimated the probability ratios:
So in Houston, they discovered the crime lab was miscalculating probabilities in the aughts, updated the math, then in 2015 discovered that math was wrong and had to update it again. Wow! This '07 comment from Bromwich about the HPD lab would have applied equally to every Texas crime lab as recently as 2015:
It is clear that DNA analysts in the historical Crime Lab ... did not fully understand the scientific basis for calculating frequency estimates from DNA profiles obtained from evidence samples and that they were not trained in the methods of properly calculating statistics associated with DNA mixture profiles and partial DNA profiles.Notably, the method Texas labs were advised to shift to in 2015 - "probabilistic genotyping," most commonly using equipment/software from a company called STR-Mix - lately has itself come under fire. A federal judge out of Michigan recently excluded such evidence in a Daubert hearing. So it's possible DNA analysts will need to re-do their math again.
Returning to the Houston PD crime lab example, these are in many cases radical downward adjustments. From one in 6.3 million to one in 30? From a one in 2.9 million chance to one in 5? Would juries have convicted if they'd heard the lower numbers? I wonder if other crime labs were using the same method as HPD's during this period?
Of the cases in this chart, Franklin Alix was executed despite DNA testimony in his case having been declared unreliable. Only Josiah Sutton was ever exonerated; 11 remain incarcerated. By all accounts, the criminal-defense bar in Houston failed to step up in many instances, leaving most of these folks un-or-poorly represented.
It's highly likely there are more actually-innocent people on that list, but we'll never know. The DNA lab's other big problem was that it had a leaky roof and most of the historical DNA evidence was destroyed and couldn't be re-tested.
See prior, related Grits posts:
- Digesting new Daubert opinion on DNA mixtures
- Most crime labs accused innocent person in DNA mixture study
- A reluctant scoop: Changing interpretations of DNA mixtures vex legal system
- Labs must correct wrong DNA mixture analyses, learn when to recognize 'crap'
- Resources on DNA mixtures
- The challenge of notifying defendants in large-scale forensic error cases
- Courts punt on forensics surrounding DNA mixtures
- Black box DNA mixture analysis method used in NYC turned out to be flawed
- Changes to DNA mixture software raise questions about old versions
- News flash: 'Touch DNA' doesn't necessarily require touching
- DNA mixture SNAFU a mess but don't expect 'deluge' of innocence claims
Here's a shocker, most labs knew it was wrong. But since no one was calling their bullsh*t, they continued because they were getting paid.
ReplyDeleteThe Texas Forensic Science Commission was created because of the HPD debacle. How many scientists on the Commission knew of the problems?
All of them.
The FSC was stacked with forensic scientists that knew of the problems of crime labs. They knew that the information within the 2009 "Strengthening Forensic Science in the United States: A Path Forward" was dead on accurate. Yet progress in the last 10 years has moved at a snail's pace.
The members of the FSC have a dog in the hunt. They have no real ambitions to expose the problems in their own labs. There are numerous conflicts of interest, but none really exposed.
@11:17, I agree the FSC hasn't always been as aggressive as I would prefer. I particularly wish it hadn't been as dominated by medical examiners (whose work they don't oversee) over the years. And I agree that the conflicts of interest you allege may have in certain ways limited what they look at and when. However, it's also true:
ReplyDelete1) The HPD crime lab scandal happened before the publication of "Strengthening Forensic Science: A Path Forward." The Bromwich report was a precursor to it, and I'm believe I'm correct that the FSC never considered the Bromwich issues. They started out on arson, got derailed by John Bradley, and didn't get their feet under them for a few years after their creation.
2) Even the NAS in that 2009 report didn't adequately critique DNA mixture evidence and improperly gave it a pass. I was doing innocence work in a professional capacity during this period, was connected to the national innocence movement, etc., and DNA-mixture evidence was not being publicly discussed as junk science before 2015 in the same way as the comparative disciplines the NAS critiqued.
3) I don't believe the labs in Texas before 2015 *knew* they were using bad math. I was in the room when they were all told and talked to some of them afterward. My impression was they were some combination of embarrassed, defensive, and chagrined.
4) The FSC is not the most aggressive, independent, state-level oversight body on forensics one could imagine. But it *IS* the most aggressive one in existence now that Trump shut down its federal counterpart. Compared to other states, the FSC actually does move the ball forward, but slowly, and often, as you point out, in disciplines (e.g., blood spatter, bite marks) that don't impact the core work at most labs, as I agree eventually must happen. (People from NY have told me horror stories about their FSC counterpart.)
5) Progress has moved at a snail's pace, but from a national perspective, ~75% of the progress has come from Texas. Looking at what's happened since the NAS report, only Texas and CA have created habeas relief for forensic errors (they copied us), and no other state forensic commission has done as much as ours has.
6) To me, Texas' biggest problem by far is not the FSC but that the Government-Always-Wins faction has taken over the Court of Criminal Appeals. So we're starting to see situations like with Joe Bryan on blood spatter where they simply refuse to evaluate the science and reject the writ, even though the FSC had debunked it at length.
7) One final thought on why there hasn't been more progress: The junk-science writ isn't used very often. Most habeas writs are pro se and indigent prisoners can't afford to hire experts. So the only habeas writs that come with sufficient resources to challenge junk science are death-penalty cases and writs brought by nonprofits with resources, mostly the national Innocence Project. Because that's just a tiny fraction of writs, maybe 2% max, most viable challenges to forensic evidence are never explored by TX courts.
To me, that's why we haven't made more progress. IMO, the FSC does better than you portray but worse than they could.
"...My impression was they were some combination of embarrassed, defensive, and chagrined..."
ReplyDeleteDid you think they were going to admit they knew it was a real problem?
And...Is "snail's pace" an acceptable pace though?
The problem with the FSC "moving at a snail's pace" is that they are receiving complaints and allegations of negligence and misconduct within the crime labs at a much faster pace.
They aren't keeping pace. And this becomes a legal problem.
The information they receive is "Brady Information" as argued by FSC General Counsel Lynn Garcia herself. The FSC queried the Attorney General in 2015...
"As a state agency with possession of information that may be
covered by Article 39.142 of the Texas Code of Criminal
Procedure, what is the Commission's notification responsibility
when it receives such information?"
So when Lynn Garcia and the FSC completely ignores complaints or responds to complaints with "NFA" (No Further Action), they aren't actually doing their job. They are essentially covering up the problems of the crime labs or kicking the can down the road when the problems will eventually be uncovered "at a snail's pace". They are illegally hiding Brady Information.
The back of the FSC's 2017 Annual Report has a list of complaints not investigated. Undoubtedly most, if not all, of the complaints had Brady Information. The FSC just chooses to ignore them.
And, if you look in the back of the FSC's 2018 Annual Report...except you can't. Because the FSC has yet to publish the 2018 Annual Report.
But I'm sure the 2018 Annual Report has no information need by attorneys, probably. I'm sure there is no list of complaints that were dismissed or "NFA".
And I'm sure it'll get published eventually, when the snail catches up.
Agreed, they could do a better job. And certainly, I wish they published their annual reports more promptly. Our disagreement is that I can think those things and simultaneously acknowledge 1) they've occasionally done good, even groundbreaking work and 2) other states either do FAR worse or have no such entity at all. We also probably disagree that they are "illegally hiding Brady information." IANAL, but I think that misapprehends their role.
ReplyDeleteOh, and I actually do think the lab directors were genuinely surprised in 2015. I do not think they were all play acting - they had a huge problem dumped in their laps and their questions and responses in the moment reflected that. Maybe some knew and feigned surprise, but that wasn't my impression. And fwiw, I've dealt with my share of duplicitous government bureaucrats.
ReplyDeleteI've dealt with Lynn Garcia. And I know she/they are hiding Brady Information by not performing their job.
ReplyDeleteSo the FSC was given information to analyze that wasn't ever given to the defense? Are you sure? That seems unlikely to me, but maybe so.
ReplyDeleteI'll send you the information.
ReplyDeleteThe FSC is a "Catch and Kill" organization. When the info is given to them, they bury the info or produce cover-your-ass documents for the public...should the information ever reach the public.
You only know what they tell you, what they publicize. It's not transparent. There's too much Federal Grant money at risk.
Info sent to you, Scott. Use it wisely.
ReplyDelete