Monday, March 19, 2012

Shortcomings at crime lab accrediting body supply role for forensic commission

As Texas' Forensic Science Commission evolves in how it does its work in response to the actual, real-world complaints it receives, it's increasingly clear one of its primary roles will be to plug the gaps created by shortcomings in crime labs' accreditation process. Last week, Grits posted a detailed critique (pdf) of ASCLD/LAB, the crime-lab accrediting agency, submitted last year to the Forensic Science Commission in New York by a prominent attorney who criticized the group for lax oversight. He recommended NY state find a new accrediting body, as the United States Army Criminal Investigation Lab recently did following problems at an accredited lab.

At the Texas Forensic Science Commission, the Willis and Willingham cases involved arson, which is not an ASCLD/LAB accredited discipline. But more recent cases demonstrate why the TX FSC and ASCLD/LAB may end up butting heads in some instances, perhaps sooner than later, over the same issues identified in New York.

The best example may lie in the FSC's investigation of the controlled substances division at the El Paso crime lab. At their January meeting, serious unresolved concerns were raised about the lab and the commission initiated a detailed investigation with the help of the Department of Public Safety. I've uploaded a copy of the letter sent to the EPPD crime lab detailing the extensive checks they demanded be run to ensure the problems had been solved. These included hiring a qualified lab director, retesting all lab work performed since November 2011, and retesting ALL the work ever performed by an analyst found to be incompetent to perform the job, "giving priority to the 60 cases on the DPS list with the greatest possible impact."

The El Paso crime lab was accredited under "legacy" provisions by ASCLD/LAB, meaning they didn't have to meet every last requirement for their initial accreditation. However in April their "legacy" status runs out and the El Paso lab must meet much more rigorous ISO standards or risk losing their accreditation altogether. Ironically, in that light, it came out at the FSC's January meeting that ASCLD/LAB had lifted the probationary status of El Paso's controlled substances division, despite all the looming questions remaining about their competence, lack of leadership, and apparent ill-preparedness to meet the new standards required of them next month. As Grits reported in January:
while most of the media attention has focused on a single lab worker who couldn't pass basic competency tests, Commissioner Sarah Kerrigan, to the nodding affirmation of her peers, strongly urged that that lab worker not be used as a "scapegoat" to avoid bigger changes. When the El Paso lab began its certification process in 2006, the accrediting body found a list of shortcomings they asked them to fix, and which the lab claimed to have resolved. In 2011, though, when the incompetent lab worker came to light, a new assessment identified virtually all the same problems at the lab, still unresolved, that were cited in 2006.
A representative of ASCLD/LAB told the commission that lifting probation didn't mean all the lab's shortcomings have been rectified, which left everyone in the room wondering exactly what it did mean. Looking at the FSC letter detailing the oversight which still hadn't been performed when the lab's probation was lifted, one wonders what regulatory benefit, precisely, that ASCLD/LAB is providing and whether it really has any teeth.

Texas is one of only a few states that requires accreditation of its crime labs, and certainly having some standards to follow is better than flying blind. But standards unenforced are mere suggestions, and it's unclear what besides suggesting the accrediting body does when the labs it regulates don't follow its rules. If the FSC becomes the primary entity regularly performing meaningful crime-lab oversight, it will create a lasting, important role for the commission, particularly over the next decade or so as Texas and the rest of country struggle to correct deficiencies in what has historically passed for forensic science in the courtroom.

The Texas Forensic Science Commission - which at times during the Willlingham/Willis saga seemed to struggle to define its mission - appears to be hitting its stride now that the Culture War circus has left town, they have a chair who actually wants the agency to succeed, and they've found a role to play that the crime labs' accrediting body seems unable or unwilling to embrace.

11 comments:

  1. For another "genius" review from ASCLD/LAB from a Texas lab, see...

    http://tinyurl.com/7ygwqq6

    Of importance...
    They 14 months to perform this review.
    They never conducted an on-site visit.
    They never acquired documents to confirm/deny the lab manager's statements.
    They never asked a single question of the complainant for rebuttal.
    And the excuses provided demonstrate a questionable competent understanding of even basic scientific principles.

    Sadly, the Texas Forensic Science Commission used the ASCLD/LAB review to dismiss the complaints. (Yes, more than one complaint!)

    ReplyDelete
  2. Grits-

    The link to the ASCLD/LAB Review (anon 4:25 --above) refers to SWIFS crime lab.

    The THREE (3) complaints submitted to the FSC against the APD were:
    1. Submitted by APD analyst Cecily Hamiliton - ended in an FSC "Review" (not a full, independent investigation by the FSC. And, as discovered by the Texas Rangers, the ASCLD/LAB report which the FSC relied heavily on was found to have missed an incomplete corrective action document from the APD during the audit, as did the APD self-audit.)
    2. Submitted by APD analyst Debra Stephens - declared a "Human Resources issue" and dismissed by the FSC (That is, according to the FSC, a crime lab issuing preliminary reports before testing items (dry-labbing) is a "Personnel Issue", not a lab issue. And if you pull my right leg, it plays "Jingle Bells".)
    3. Submitted by Integrated Forensic Laboratories in Euless, Texas. (claiming that APD contaminated, incorrectly classified, and incorrectly weighed drug samples.)

    ReplyDelete
  3. Sorry, 6:53, I realized after I made that comment the link was to Dallas so I deleted my question, apparently just as you were typing a response. Thanks a lot for pointing that out.

    FWIW, I think they're taking the EP and the most recent two Austin complaints quite seriously. I blame John Bradley's chairmanship for them taking this long to get serious about crime lab investigations, but I will say my sense is they're serious now.

    I also think that perhaps commissioners didn't understand ASCLD/LAB's shortcomings as fully in 2010 as they do now. The memo linked in the post to the NY FSC was a pretty groundbreaking critique, as far as I can tell. (I'd not seen the case put that forcefully before.) IOW, perhaps there's reason for cautious optimism.

    ReplyDelete
  4. Grits-

    anon 4:25 here.

    While I appreciate your optimism for the future work of the TFSC, let me point out that:
    1. The ruling of the 2nd APD complaint (Oct. 2011, "personnel issues") came well after Bradley's departure.
    2. The ruling of the 2nd APD complaint (Oct. 2011) came well after the Attorney General's Opinion regarding the TFSC's authority/responsibility to investigate "incidents" of scientific misconduct (July 2011). IMO, "dry-labbing" represents an "incident".
    3. The 2nd APD complaint became an interest to the TFSC only AFTER the MSM discovered and reported on the whistleblower (Jan 2012). And by "interest", I mean "liability".
    4. The FSC has yet to request the opinion of ANY lab analyst during an Open FSC meeting (including those who have submitted complaints). Analysts are, inarguably, the best witnesses for identifying scientific misconduct as it occurs in the lab. In addition, lab analyst have the most at stake as they are the person handling the evidence, storing the evidence, signing lab reports, and providing expert witness testimony. I DARE any analyst, while providing expert witness testimony, to declare their lab protocols inadequate and unscientific, or declare their supervisors to be unqualified.
    5. A few of the TFSC Members use ASCLD/LAB for their own crime lab accreditation purposes and would rather not change the status quo lest they jeopardize their own accreditation status. Why this obvious conflict-of-interest has not been addressed by the TFSC, I'll never know (ahem...Peerwani).

    The TCCP 38.01, as written, is very clear regarding what the TFSC is supposed to do. IMO, there is very little wiggle room (and even less after the AG's Opinion). Accreditation was not to be trusted, hence the creation of the TFSC to investigate ACCREDITED labs.

    I would like to have the same positive outlook as you, Grits, but until those TFSC Members that run crime labs are replaced with academic scientists who have no dog in the hunt, there will always be the bias created by the "good ol' boys" network.

    That is...deny, deny, deny.

    ReplyDelete
  5. Dear GFB,
    My name is Ari Bernstein, and I am a social work student living in Chicago, IL. I am interested in corrections, particularly juvenile corrections, so I have stopped by your blog a couple times before. However today I have come by because I am doing a advocacy project accelerated release, earned credit, good time, merit credit, ect. I saw your 2009 post interviewing Lawrence and I was wondering if you have had any more recent reports on on the subject pass through. I can be reached at arielbrnstn@gmaill.com
    Thanks for you time if you dont have anything new, and keep up the good work.
    -Ari

    ReplyDelete
  6. 10:30, I understand your skepticism, which is why my optimism is "cautious." But I do think the FSC is taking the EP and the latest Austin complaints more seriously. Plus, IMO the commission is just now figuring out its role and function now that JB's foot is off their throat.

    The Austin complaint dismissed last fall wasn't nearly as specific as the latest ones. My recollection is the dismissed complaint WAS more about personnel issues and not specific charges about drylabbing, or even calling their scientific competence into question as in El Paso. And the FSC took up the "drylabbing" complaints at the next meeting after the complaints were submitted. So whether that was in response to MSM coverage or a function of only meeting quarterly amounts to debating the unknowable. As long as they're pursuing the cases and not saying "deny, deny, deny," I'll give them the benefit of the doubt on that one. Almost all these cases make it into the media before they get to the FSC.

    As for the makeup of the commission, it was a result of the Lege process, and the group is slanted toward institutional players - including those running ASCLD-accredited labs - because folks who didn't want the FSC to exist at all wanted to neuter it as much as possible on the front end. Would I prefer academic scientists who're used to more rigorous standards than forensic analysts? Sure, but that doesn't mean the current folks are all acting in bad faith. I don't think that's true, certainly not across the board.

    The Commission was in many ways set up to fail, especially after the AG opinion, and I'm not claiming they're the end all be all or will solve every problem. But they're what we've got for crime lab oversight and from my observation attending most of their meetings, they're doing a much more conscientious job now than they did under Bradley.

    BTW, can you give me a source for this assertion?

    "as discovered by the Texas Rangers, the ASCLD/LAB report which the FSC relied heavily on was found to have missed an incomplete corrective action document from the APD during the audit, as did the APD self-audit.)"

    I'd never heard that before.

    Ariel, it's not a subject I cover a lot, but here are a few things.

    ReplyDelete
  7. Grits-

    The statement from my comment "...as discovered by the Texas Rangers, etc., etc..."

    is related to the TFSC "Review" issued 09/08/2011 (aka. the 1st complaint against the APD) and available on the TFSC webpage.

    Middle of page 5, "...When auditors from the Texas Ranger team discovered that the Quality Assurance Manager had not completed a CAR for the contamination in the Sanders case, remedial action was taken immediately and the report was completed. Neither the ASCLD-LAB audit nor the APD Lab internal audit (See Exhibit H) conducted in August 2009
    (covering the previous year) picked up the fact that a CAR was not completed for the contamination. It was not until the Texas Rangers conducted their audit that the error was identified and a Class 2 CAR was completed (See Exhibit I)..."

    That is, for the TFSC's Review, the finding of an incomplete/missing CAR was downplayed of importance (A "Class 2" finding is the most severe). CARs are actually Brady Material describing the working conditions (or problems found) within the lab. If a CAR doesn't exist, then the lab operations are considered to be up to acceptable standards (or, no problems found). The fact that there was missing/incomplete CAR could be used by Defense Attorneys to demonstrate:
    1) a lab's ineptness to perform its duties by finishing CARs contemporaneously
    2) a deliberate attempt to hide problems in a lab. In which case, the ASCLD/LAB audit report and the APD self-audit report (both failing to identify the incomplete CAR) are fraudulent.

    What is most noticeable from this "Review" is the rubber-stamping from each report that the TFSC used...to rubber-stamp its own report. Largely, analyst Cecily Hamilton's assertions weren't discredited as false. The findings of the different investigations were inconclusive, but presented as if her claims were unfounded or false. IMO, each agency covered for the other.

    Hence the deny, deny, deny...

    ReplyDelete
  8. I'm having trouble seeing what 2:43 is seeing. It doesn't make sense to me that so many people in so many organizations (TFSC, ASCLD-LAB, DPS) would cover up for APD. What's APD to these people? Covering up doesn't benefit the reviewers or the organizations. Plus, in DPS's case, if they were really out to cover up for APD they would have covered up the incomplete CAR documentation, if it is as important as 2:43 would have us believe. All of that causes me to believe that these reviews are not coordinated efforts to cover up for APD, but reasonably independent assessments that are reaching a similar conclusion because it is the best conclusion. The last I checked, consensus is generally a better explanation than conspiracy.

    ReplyDelete
  9. whomever 5:15...

    To demonstrate the importance of the CAR document...a hypothetical.

    You are a criminal defense attorney.
    The evidence being used against your client has been determined to have blood on it, according to the crime lab report.
    Zealously defending your client, you believe that the blood on the evidence may have originated from inside the crime lab from an in-house contamination event, and not the crime scene. A defense strategy to demonstrate the incompetence of evidence handling inside the crime lab.
    You request from the crime lab for any CAR documents that report blood contamination events in the crime lab during the time the your evidence is being analyzed.
    The crime lab states that there are no CAR documents which describe blood contamination events.
    Because of the crime lab report, and the declaration of "We're Accredited!", and lack of documentation that describes in-house blood contamination events, your client accepts a plea bargain (yet, still claims innocence).
    One year later you discover, through an audit report by the Texas Rangers, that the crime lab actually had a contamination event at the same time the evidence used against your client was being analyzed. Except, the documentation for the blood contamination event hadn't been completed until a year after the fact (and after your request).
    Thus, in retrospect, the crime lab was not lying to you when they stated that they did not have a CAR documentation for the blood contamination event. They actually just hadn't completed it yet, or they had stuffed documentation of a bllod contamination event into an unrelated case file that they believed was the only case affected by the blood contamination event.

    This is, by definition, a Brady Violation (exculpatory info that should have been turned over to the defendant). Also, Discovery violations and/or Open Records violations.

    ReplyDelete
  10. To anon 5:15-

    As stated numerous times in previous comments, the TFSC DID NOT PERFORM AN INVESTIGATION. The TFSC only re-vomitted opinions of the audits of the ASCLD/LAB, the Texas Rangers, and the self-audit of the APD.

    "...reasonably independent assessments that are reaching a similar conclusion because it is the best conclusion..."
    If the BEST conclusion (the consensus) is that 2 out of 3 audits missed an incomplete CAR documentation (this includes the self-audit by APD that should have known that the CAR was incomplete), then the consensus could more aptly be called the "Best of the Poorest" assessments.

    Under normal situations (non-investigations), there is only a single audit that is reported. The People of Texas expect that single audit to be the BEST assessment, and shouldn't have to wait on three different assessments to come to a consensus.

    ReplyDelete
  11. 5:15 here.

    7:40 - Your hypotheticals don't apply in this instance. In this instance, as I understand it, CAR documentation was created, the problem was documented and acted upon, but the paperwork didn't go through the final sign-off. So the documentation related the CAR would have been available upon subpoena to any defense attorney who wanted it.

    But my comment was not about this specific CAR issue. My comment was about this notion that seems to be out there, that there is a conspiracy among various organizations to cover-up problems at APD. That is a pretty fringe notion to my mind. And as a reasonably rational person, I get rather concerned when the discussion starts sounding like a discussion of UFOs and little green men.

    ReplyDelete