Sunday, September 18, 2011

Holy junk science, BAT van! Houston's portable breathalyzers accuracy questioned

From typical discussions of forensic errors in DNA exonerations, etc., you might think they occur mainly in the most serious crimes like rape, murder, etc., but law enforcement's use of flawed science may extend to more workaday crimes like DWI as well. Reported the Houston Chronicle recently ("Controversy continues to dog BAT vans," Sept. 8):
A driving while intoxicated case that sparked doubts on the accuracy of test results from the Houston Police Department's breath alcohol testing vans has been dismissed, while evidence from the vans in at least two other cases has also come into question.

A former HPD crime lab supervisor testified during a court hearing in July that she quit because she could not trust the accuracy and integrity of breath alcohol tests from the department's breath testing vehicles. Since then at least two other defense attorneys ...  say evidence in DWI cases they are handling could have been compromised because of the problems with the vehicles.

During the testimony in July, the former HPD lab supervisor, Amanda Culbertson, said the breath alcohol testing vehicles, also known as BAT vans, incurred such electrical problems as overheating. Those problems affect gauges, she said, and can alter the control sample used to calibrate the breath-test machine in the vans, possibly affecting the accuracy of test results.

HPD officials have acknowledged there have been problems, including air conditioning, in the BAT vans since they were purchased in 2008, but said that no cases should be compromised as a result of the temperature in the BAT vans.

But defense attorney Mark Thiessen said he does not believe that the breath testing machines were working properly in the BAT vans and that the tests were not run under proper protocols.
Paul Kennedy had an excellent post in August explaining how changes in temperature can affect such tests if the comparison sample isn't just right. Notably, the company that makes the breathalyzer won't actually reveal the source code behind its analysis, nor guarantee beyond one year that their products will be free from defects in material and workmanship. Even when working properly, the margin of error for some versions of the instrument is up to 25%.

Concerns about the accuracy of breathalyzer tests have been raised for several years now in Houston and elsewhere, but the practical implications of how many cases would be affected if they were deemed untrustworthy have scared away elected judges from closely interrogating the technology. Between the political clout of groups like MADD and the fact that so much government employment, fine income, and even trauma hospital funding ride on a steady stream of DWI revenue, judges are no more likely to question breath-test results than officials in Salem would have questioned that dunking in water might expose witches.

See also: Paycheck vs. Integrity: Houston PD crime lab supervisor resigns over faulty breathalyzers, feared retaliation


Anonymous said...

Once again, GFB demonstrates why journalists and attorneys have such difficulty with matters of science and measurement. Back to high school chemistry with you. Try to get a C this time.

Gritsforbreakfast said...

11:09, go ahead, dazzle me with your scientific acumen and explain the basis for your grade.

Anonymous said...

I guess the HPD lab supervisor who said the BAT tests were inaccurate also needs a high-school chemistry class, right 11:09?

Nobody in the government is going to slow down the gravy train by admitting the BATs aren't foolproof (which is why Culberson had to leave). Way too many jobs at stake, and MADD would go apeshit.

rodsmith said...

what scared me was this!

"Notably, the company that makes the breathalyzer won't actually reveal the source code behind its analysis, nor guarantee beyond one year that their products will be free from defects in material and workmanship. Even when working properly, the margin of error for some versions of the instrument is up to 25%."

So let's see first they refuse to show thier scoring system to anyone. So why is it even being used when there is no way for anone to test it! Second what kind of junk is it that they refuse to guarantee it's operation let alone it's accuracy after 1 year and they know some of them have a 1/4 FAILURE RATE WHEN BRAND FRIGGIN NEW!

which means 1 out of every 4 person ran though one is marked wrong. either a drunk is put back on the road. or an INNOCNET person is subjected to the american INJUSTICE system!

Anonymous said...

He called you a journalist? Them's fighting words!


Anonymous said...

Since I hold an earned master of science in physics, and my thesis (a photometric study of a binary star system) was based on differential photometry, I believe that I have an uncommon amount of insight into the inner machinations of ‘breathalyzers.

The problems of breathalyzers are manifold. This is not a myth, etc.. As one poster stated the fact that the source code is not open to public scientific scrutiny ought to be the first clue to the problems associated with these machines. The second clue ought to be something called the ‘partition ratio.’

Frankly, if I was placed on a DWI jury that was presented breathalyzer results as evidence I would in all good conscience have to give it very little if any credence. The science is simply not there.

The Fishing Physicist

Anonymous said...

Grits, you're referencing defense attorneys for scientific arguments? Really? The criteria for the Intoxilyzer as 0.020 is the acceptance criteria btwn the two results obtained, otherwise it is not considered a valid test. It is not the uncertainty of the measurement (the confidence that the true value of the unknown lies somewhere in that percent window). This is just one example that 11:09probably noted. And, it most certaintly doesn't mean that 1 out of 4 tests is completely wrong, rodsmith. That's a ridiculous misunderstanding of science.

Gritsforbreakfast said...

You know, you're right, 2:05. Henceforth, I'll only accept scientific information from anonymous blog commenters who provide no sources for their assertions, since of course they're renowned for their accuracy

rodsmith said...

well 2:05 I think the one has an understanding problem and is rediculous as you call it is in fact YOU!

NOT too many ways to take this quote from the original article!

"Even when working properly, the margin of error for some versions of the instrument is up to 25%."

the way any inteligent normal person would read that statement is

"when working propertly some versions of this instrument are WORNG 1 out of every FOUR times it's used!"

now if the reporter had some guts they would have discovered WHAT MODELS and IDENTIFIED them and found out what TEXAS organizations were USING THEM!

Anonymous said...

I say go for the blood. It typically will be higher than the breath and then you can test for what other intoxicant the suspect has ingested.

Anonymous said...

Pretty funny comment 11:09. Tell us Jethro why would someone take a chemistry class to calculate the error in a measurement system when you really just need to study up on your ciphering?
"If brains was lard, Jethro couldn't grease a pan."

The obvious problem with these machines is the degree of uncertainty relative to the measurement. It's like using a tape measure to determine the size of a gnats ass.