The Austin PD crime lab may be getting more scrutiny from the Forensic Science Commission,
reports Patrick George at the Austin Statesman, this time based on a rare complaint from another lab which is challenging APD's drug testing results:
Another complaint has been filed against the Austin Police Department's crime lab, this time by an independent lab in North Texas that claims it received different results than the Austin lab when testing the same drug evidence.
Crime lab officials in Austin said they received the complaint filed by Integrated Forensic Laboratories in Euless, between Dallas and Fort Worth, late last week.
Austin police and lab officials have begun looking at the complaint this week and are preparing a response, said Bill Gibbens, the lab's forensic manager. Gibbens said he hopes to send a response to the Texas Forensic Science Commission by Friday.
It is the second complaint the Austin lab has faced this year. In January, a former crime lab scientist who was fired last year alleged that lab administrators do not have proper accreditation and that drug evidence was not analyzed before reports were submitted.
Gibbens said the latest complaint deals with two separate pending criminal cases from 2010. Because they are moving through the courts, he said, he could not discuss the details .
After the Austin police lab submitted its drug evidence tests to the courts, the suspects' attorneys asked for a second opinion from the independent lab in Euless, Gibbens said. That lab returned different test results than the Austin lab, he said.
"It's a difference of opinion in how we report substances," Gibbens said.
Gibbens said he's not sure how common it is for one lab to file a complaint on another lab. "It's the first time it's happened to us," he said.
Lynn Robitaille, the general counsel for the Texas Forensic Science Commission, said that it was the first time she had seen one lab file a complaint on another since she started there in December 2010. The commission was created by the state Legislature in 2005.
Robitaille said the commission's complaint screening committee heard the matter Friday. When they meet again in March, they are expected to recommend whether the commission should investigate the matter.
I wonder if the
allegations of drylabbing (submitting results without having done the testing) and the incongruous lab results are related? If you submit results without testing the evidence, that would certainly open the door for another lab to come out with different findings.
"It's a difference of opinion in how we report substances," Gibbens said.
ReplyDeleteHuh?
In light of all this, they are probably just as accurate whether they dry lab or not!!
ReplyDeleteThe second round of APD allegations were dismissed last October with the FSC stating
ReplyDelete"The Complaint was unanimously dismissed by the FSC, because it pertains to human resource decisions made by the APD lab. As such, it falls outside the Commission's jurisdiction."
Maybe these new allegations should also be shuffled off to Human Resources. HR does such a good job at solving problems. Just ask the analysts that were fired.
The FSC couldn't investigate their way out of a wet, blood-contaminated paper bag.
-SCP
Both APD and IFL, Inc. are ASCLD/LAB accredited. Isn't there enough evidence now to realize that this organization is not doing its job correctly? (Houston PD and El Paso DP were, too.)
ReplyDeleteIt's time for a National Organization to monitor accreditation, with National Standards for testing, reporting, etc. Everybody needs to be on the same page.
There is really no point in trying to evaluate the significance of this without additional information.
ReplyDeleteWhen two labs get different results, there is no way of correctly judging which was right and which was wrong without additional information - either additional testing, or additional review of case files to identify the root cause of the problem.
Without that additional information, any evaluation is really gross speculation based on gut instincts and/or biases. And Lord knows, we don't want that to happen around here.
With all the crime lab problems, the only way I can see around any of it is to hire somebody from the cast of NCIS or CSI to run it. They are the only ones who seem to get accurate results these days, but then they read the script.
ReplyDeleteFor all quantitative chemical tests (i.e., drug and toxicology testing, etc.), whenever a sample is tested by two different labs there will be a difference in the quantitative test results. This is inherent to the testing process, which is dependent upon a number of individual sampling and measuring steps, each one of which has its own measurement uncertainty. When those individual uncertainties are combined, the result is an overall uncertainty of measurement.
ReplyDeleteSo the exact same material, tested in two different labs, will almost always give two different numerical results. When there is a difference there needs to be an assessment of the significance and the source of the difference. That can sometimes be fairly involved.
I've given training to defense attorneys on inter-laboratory differences in testing. It's certainly something people should be more aware of than they are.
But the bottom line is that sometimes those differences have significance, and sometimes they don't. And generally, it takes someone who has technical expertise to make an adequate and reliable assessment.
The problem is that these same complaints against the medical examiners offices are all over Texas.......Houston, Bexar and Travis just to name a few. We have medical examiners appointed to the ethics commission even with complaints against them. Now imagine, by virtue of falsified rulings by MEs, how many innocent people are incarcerated and how many families were victimized by a system who rubberstamped homicides as suicides.
ReplyDelete12:28 -
ReplyDeleteI'm not sure what your point is. Autopsies are not reviewed by the TFSC. And eventually every lab will have complaints made against it. They are part adversarial process, and when people are unhappy with the results of the adversarial part of the process, it's a no cost thing to do to file a complaint. Same thing for incompetent lab trainees who have to be fired and are miffed about it. It doesn't cost anything to file an anonymous complaint.
Anon 12:50-
ReplyDeleteEmployee Grievance Procedures and Civil Lawsuits for Wrongful employment termination ALSO don't cost anything.
However, a forensic analyst shouldn't have to go to these extremes just to get some simple answers to their concerns of scientific practices in their labs.
Forensic analysts are the best witnesses of scientific negligence and misconduct because they see it first hand. As such, their complaints should be addressed immediately and with urgency.
In addition, if lab managers weren't miffed that the analyst reported them, then the lab managers wouldn't have fired the forensic analyst. COMPETENT lab managers would, in fact, encourage an investigation of their work product and could train their analysts properly, scientifically.
(Incompetent trainees are the result of incompetent trainers and incompetent lab managers.)
INCOMPETENT lab managers would just lie, knowing that the watchdog agencies are, themselves, composed of lab managers who also might be the target of future allegations. So, quid pro quo.
BTW, 12:50 --I work in your lab. Expect a complaint.
To 12:50-
ReplyDeleteIt costs plenty to NOT report a claim.
The complaints related to the data and analysis which lead to a ruling reflected on the autopsy report are in fact reviewed by the TFSC. These complaints should be taken very, very seriously. We have JPs in Texas acting as forensic patthologists and crime scene investigators for crying out loud.
ReplyDeleteAnonymous @ 0751,
ReplyDeleteIn the world of engineering, we rely on ASTM and ANSI standards which generally document minimum and maximum deviations allowed in testing protocols. It sounds from your post that these also exist in the Forensic sciences. Engineering standards also specify requirements for instrument calibration and laboratory conditions.
You are correct that all of these conditions need to be considered when evaluating laboratory results. The accuracy of the test and the precision of the results are driven by many external variables as well as the material being examined.
Scott in South Austin -
ReplyDeleteThat's right. ASTM and ANSI standards do apply in forensic sciences, as do ISO 17025 standards for testing and calibration laboratories. That certainly reduces inter-lab variation.
Many of the types of samples processed by labs are of such limiting quantity and poor quality that the test result is close to the limit of detection. With those sorts of samples, small differences in process between labs can be magnified. Also, because of the limitations in sample size, replicate testing may not be possible, or may be possible only to a limited extent.
7:51
Anon 12:50, here.
ReplyDeleteAnon 2:57: You say you work in a lab. If you work in a lab, then I am definitely not your employer. Don't get on the bad side of your boss thinking that my comments come from him.
My comments were meant to be general comments. They were too brief, so let me elaborate.
There is a tendency for this blog to pounce on complaints to the TFSC against a lab as being an indication that there is something wrong with the lab.
The state has set upon the TFSC so that it both evaluates complaints, and encourages that complaints be brought forward. So one measure of the success of the TFSC is that there be complaints to evaluate. Complaints are not brought forward by happy people - they are brought be unhappy people. In a standard business model, if you provide a service to 1,000 people you might strive to have 1% unhappy with your service. If 20% are unhappy and bring complaints forward, then you are in bad shape as a business.
But the criminal justice system is an entirely different beast. The way the criminal justice system works - being an adversarial system - there are lots and lots of people who will be unhappy with the end product. If there is a guilty verdict, then there will be lots of defendants and their families who are unhappy. If there is a not guilty verdict then there will be lots of victims and their families who are unhappy.
So the nature of the process means there will be lots of unhappy people and lots of complaints.
Then on top of that you have the problem that labs employ people, and sometimes some of those people will have to be let go. That is one of the things employers do. But what is clear from the recent history of the TFSC and APD is that lab employees who suffer adverse employment actions are unhappy and may complain to the TFSC.
The bottom line is that the mere occurrence of a TFSC complaint is of no meaning at all, because the process is set up so as to maximize the number of complaints.
What is of meaning is whether a complaint has substance. So in the case of APD, there was a recent complaint from a former DNA analyst, and there was much made of the fact that that complaint was made. The TFS reviewed that complaint, dismissed it, and issued a report of its findings (http://www.fsc.state.tx.us/documents/M_APD%20090811%20FINAL%20executed%20092811.pdf). The bottom line was, the complaint by the former employee was found to not have substance. The employee was not a whistle-blower - she was just an unhappy employee.
Don't get me wrong. I'm all for complaints being brought forward. But complaints have no meaning until they are evaluated. That's all I'm saying.
To anon 12:50/8:08-
ReplyDeleteThis is anon 2:57.
You used the terms "anonymous" and "trainee" in your comments -- neither of which applies to any of the complaints against the APD. As such, I assumed you were a jaded lab manager. My bad.
To continue with the conversation, though, the adversarial process takes place in the courtroom, not the lab. Lab results should be data-driven, not based on debated scientific opinions between lab managers and forensic analysts. Protocols must be scientific. Analysts must follow protocols.
Secondly, I have concerns regarding generalized conclusions and broad sweeping statements proffered as fact, and utilized extensively by prosecutors/lab managers for trials/informing news reporters as to the credibility of their scientific operations.
Upon closer inspection, the devil is in the details.
As an example (regarding the first complaint against APD) , you stated "...The TFS reviewed that complaint, dismissed it, and issued a report of its findings..."
The TFCS did not perform an investigation. It performed a "brief memorandum" review of those documents provided by the APD, ASCLD/LAB, Texas Rangers, etc. Arguably, this action is outside the legal jurisdiction as set forth in TCCP 38.01. They should either investigate fully and independently, or dismiss a complaint for lack of jurisdiction. They shouldn't be pursuing "reviews" that simply reiterate the findings of other agencies who may have biases or ulterior motives of their own. Should nefarious actions be discovered in other agencies, the TFSC becomes tainted by association.
Moreover, the TFSC didn't directly interview the analyst (or other analysts). So their "review" is entirely one-sided (The analyst did not submit the complaint, APD did). This is akin to, by analogy, the DA submitting evidence to a lab and stating (pre-testing), "we just want to confirm that the drugs found on the suspect was cocaine so we can convict." That is, "confirm what we already suspect." This is not in the TFSC's best interest for projecting unbiased and impartial investigations.
Example number two, you state "...The bottom line was, the complaint by the former employee was found to not have substance..."
In fact, their WAS substance to a part of the analyst's complaint. During the Texas Rangers inspection, they found an incomplete CAR document for the DNA contamination event reported -- missed by an ASCLD/LAB audit and an INTERNAL audit by the lab managers themselves. This raises red flags concerning the thoroughness of ASCLD/LAB audits and the honesty of APD for completing thorough audits of themselves. The APD should have been hammered on for this "oversight" since CAR documents should (must) be turned over by prosecutors to the defense attorneys as "Brady Material" (also are available to the public -- if they are completed, and known about). Even more so in this case since the results of the DNA analysis were inconclusive -- perhaps as a result of DNA contamination reported by the analyst. This violation was downplayed of importance in the TFSC review.
Example number three (regarding the second complaint against APD), from news reports (assuming they didn't misquote), APD Assistant Chief Mannix says "the TFSC investigated the complaint and found no merit."
In fact, the TFSC did not perform an investigation from this complaint (#11-07) stating that the claims were human resources-related and outside the TFSC's jurisdiction (October 2011). Based on the complaint-related documents that are now available on-line, this becomes (arguably) just another excuse from the TFSC to dismiss the complaint. Yet, it becomes spin for Chief Mannix.
"...If 20% are unhappy and bring complaints forward, then you are in bad shape as a business..."
ReplyDeleteIt would be interesting to know just how many analysts would report misconduct about their lab to the TFSC if they knew they weren't going to lose their jobs because they were "unhappy".
RE: Anon 7:51 and 11:25
ReplyDeleteDid anyone see the memo put out by DPS Deputy Assistant Director Pat Johnson regarding "uncertainty in the measurement" being added to lab reports?
http://www.tdcaa.com/sites/default/files/news/img-214151923-0001.pdf
I've never seen a DPS report, but is this to say that the "uncertainty in the measurement" was never reported before -- that this information was not required per accreditation standards? What about limits of detection? Is this listed on the lab report?
These are basic, fundamental, scientific information! Accreditation agencies have to tell the forensic labs to include this info on lab reports?!?
3:13 -
ReplyDeleteReporting of measurement uncertainty (e.g., 23 +/- .002 mg /100 ml) for all quantitative results is a new accreditation reporting requirement in 2012.
My point was, anon 9:07, that if you have to be told to record the uncertainty in measurement in your lab reports, you're not really a scientist.
ReplyDeleteThe standards for forensic science are far, FAR too low.
In the first allegations against the APD, incomplete documentation was found by the Texas Rangers related to the DNA contamination reported by the analyst. The lab managers didn't report this in their audit -- a sign of dishonesty or ineptness.
ReplyDeleteSo the first complaint did have substance in this respect. Yet, APD claims in the press that it was unsubstantiated. Go figure.
12:56 -
ReplyDeleteYour kidding, right?
What you are referring to was an internal administrative document that was not a part of the original complaint. It was an issue identified independently by the Rangers as part of their review.
What the TFSC report said about the Rangers investigation was:
"Texas Rangers Investigation Report (September 2010) (Exhibit D): The Texas
Rangers also conducted a review of the contamination and competency
allegations. The investigative team included two subject matter experts:
Blake Goertz, the Regional DPS Section Supervisor for DNA (Waco) and
Cathy McCord, the Regional DPS Section Supervisor for DNA (Lubbock).
The team concluded that: (1) the Lab has extensive procedures in place to
minimize contamination; (2) the complainant and investigators were unable to
identify a single case of bad science being used in criminal prosecutions; (3)
the APD’s DNA training program was sufficient and protocols are in line with
national standards; (4) the DNA analyst in question was well-trained and
competent; and (5) the Technical Leader’s qualifications were sufficient
during the Rangers’ audit and the previous five external audits. There was
one reporting issue found (the Quality Assurance Manager had failed to
complete a Corrective Action Report (“CAR”)). The issue was remedied
promptly."
As reflected in the DPS report of the Rangers investigation, the contamination event was fully evaluated and documented in the case file and was also addressed in the DNA report.
Really, if you are going to engage in misrepresentation, you should exercise more care in what you choose to misrepresent.
Even better, refrain from misrepresentation.
3:37-
ReplyDeleteAu Contrair.
According to the "review",
"...The most substantive allegation involving forensic science made by Ms. Hamilton was that contamination occurred in the DNA analysis in the Nathaniel Sanders case in May 2009..."
Sounds like it WAS part of her complaint.
"...The contamination was documented and addressed in the DNA report according to Lab policy..."
If lab policy is to complete a CAR, then the lab didn't follow its own policy. CARS are a huge deal because CARs are available to the public (and available to other analysts in the lab). Thus, if there is no completed CAR, then it is assumed that there is no contamination in the eyes of the public (or in the eyes of other analysts in the lab). Documents that are stuffed into a case file are not available to the public (and not necessarily known to other analysts in the lab). How many other CARS were not completed and available to the public (i.e. how far back did the investigation go)?
"...The DNA section also deemed the entire sample inconclusive..."
Was the inconclusive result of the DNA test because of the contamination? You won't find that kind of info in the lab reports.
"...Neither the ASCLD-LAB audit nor the APD Lab internal audit (See Exhibit H) conducted in August 2009 (covering the previous year) picked up the fact that a CAR was not completed for the contamination..."
Does this mean that only Texas Rangers will be able to discover violations of scientific misconduct in crime labs? Can the public trust the APD to report on complete and accurate audits?
When a Forensic analyst signs his/her name on a lab report, he/she is representing the the lab protocols. When the forensic analyst is providing expert witness testimony, he/she is testifying to the veracity of the lab and/or the competence of their fellow analyst. If a forensic analyst knows/believes that their fellow analyst is not following best practice, or knows/believes that the lab management is not following lab policies, a forensic analyst can not represent the "whole truth and nothing but the truth". Their work product is compromised by the actions of others.
So, 3:37, misrepresent much?
4:48 -
ReplyDeleteSo you you read the report. So you know that neither the Rangers or the TfSC substantiated any of the complaints of the analyst. Any yet you fail to acnowledge what they actually concluded. That about says it all, don't you think?
Kudos to the anonymous 7:36 who inadvertently proved the previous commentator's argument.
ReplyDeleteThe TFSC only reiterated the info it was provided. The did not substantiate the information.
And the APD audit and the ASCLD/LAB audit, provided as corroborating evidence that everything was "A-OK", were factually fraudulent since the CAR document was not disclosed for either audit. Thus, everything was not "A-OK". Because of investigational "blinders", the Tex Ranger's did not follow-up on this. The TFSC did not follow-up on this.
To people like anonymous 7:36, only the "happily ever after" conclusions are valid, not the actual findings or the snowball effect of the findings.