Friday, January 15, 2010

Confronting SCRAM

With the US Supreme Court this week revisiting at oral argument the question of requiring confrontation at trial for lab reports (i.e., giving the defense the right to cross-examine the lab worker), I found this anecdote from DA Confidential a particularly compelling example of why the right to vigorously question forensic evidence is important, regardless of complaints about "inefficiency" and cost. The case involved a defendant required to wear a SCRAM device, which is an anklet commonly used for DWI probationers that purportedly monitors blood alcohol levels. As DA Confidential tells it:
Where was I? Ah yes, the gentleman wearing a Scram. I'll call him Mr. Smith. So his bond was revoked and he was put back in jail when his Scram showed he'd been drinking just before Christmas. He told his lawyer (a young man I checked with today, and who was too modest to allow me to use his name, so I'll call him Mr. Jones*) that he'd not been drinking. Insisted, in fact, that he'd not been drinking. So, doing his duty, Mr. Jones went before the judge and tried to explain that there'd been a mistake. So the judge, wanting to be fair, had the people responsible for the Scram come into court.

They showed him their proof, a print out of a black-and-white graph.

On that graph were two lines, one showing whether the Scram had been tampered with, one showing whether Mr. Smith had been drinking. Both lines spiked. Thank you Scram people, Mr. Smith stays in jail.

But Mr. Jones wasn't satisfied. Presumably because his client was less than satisfied and, oddly in the face of technical and scientific evidence, still claimed to have been at work and not drinking.

So Mr. Jones went back to the Scram office and obtained another copy of the graph. But he made to sure to get a print out in color, not just a photocopy. Lo and behold, the color graph contained three lines (as opposed to two), each a different color:
  • one showing whether Mr. Smith used alcohol
  • one showing whether Mr. Smith tampered with the Scram
  • one showing Mr. Smith's body temperature
The spikes were in the bottom two lines, the ones showing body temp and tampering. The line showing his alcohol use was a flat-line at the bottom of the graph, and had been mistaken by everyone as the baseline, the line you'd draw at the bottom of every graph.

Ooops. Mr. Smith had not, it was now clear, used alcohol.

But that still left the tampering spike. Well, the ever-diligent Mr. Jones obtained the second page of the Scram report, which showed examples of spikes when the Scram had aluminum shoved under it, a wet cloth stuck under it, and when a sock got stuck under it.

Guess which it was? Right, the sock.

Finally, Mr. Jones obtained proof from Mr. Smith's employer that he'd been at work when he said he had. Even better, the hours he'd been at work were almost exactly the hours his sock had messed with the Scram.

Result: the judge commended the defense lawyer on his excellent work and politely requested the presence of the Scram people. I was not privy to all of what he said to them but I do know that they accepted responsibility for the mistake and it, hopefully, won't happen again.
As our correspondent notes, that was an especially fine piece of lawyering by a fellow later identified as Matt Dorsen, who works for lawyer Steve Lee. But the questions raised by the example have broader implications than just for this single defendant.

In how many previous cases, one wonders, have SCRAM employees misinterpreted a black and white graph to falsely accuse someone in court? How often has their word been taken over a defendant's, even in the face of adamant denials? Probably quite regularly. That's clearly what was happening here before the defense attorney's exceptional efforts. Will anyone now go back and look to see if something similar might have happened in past cases? No way. The judge just gave company officials a good talking to in private and the county will continue to use the device.

Cases like this show why, even when mistakes are discovered, we can't necessarily rely on judges or prosecutors to vet such forensic technologies properly - they don't have the knowledge or resources to do so and must rely on others to ensure they're not making decisions based on junk science. After all, as Williamson DA John Bradley pointed out to the House Criminal Jurisprudence Committee earlier this week, the reason they all went to law school is that they weren't any good at math or science.

That's why there must be an explicit right to confront forensic evidence and those who come up with it at the level of detail exhibited here, or else an affidavit purporting to interpret evidence becomes precisely as credible as the piece of paper which reads only "Trust us, we're the government." Misreading the graph because somebody was too cheap to use a color printer sounds like a segment out of Reno 911!, except this was a real defendant who a Travis County court tossed in jail over the error. I hope the guy didn't lose his job.

This SCRAM device, in particular, may not be ready for prime time, according to a recent story at The Crime Report:
many everyday products, such as hairspray and household cleaners, contain a type of alcohol that is detected by the bracelets. Exposure to large amounts of those products could trigger the monitor, leading some critics to charge that they produce false positives. Failing a test can mean being sent to jail.

“The problem is distinguishing the alcohol,” said Michael Hlastala, a professor of physiology at the University of Washington who has testified in several court hearings against the accuracy of SCRAM. “They don’t have a good way of separating out interference from drinking alcohol.”

“The device went to market before adequate scientific work was done. There are problems that have not yet been addressed experimentally,” he said.

Courts have yet to seriously address the reliability of SCRAM technology, according to The Crime Report:

Though the devices are widely used, it appears only a handful of cases have called their reliability into question. Kenneth Cooper claims in a lawsuit that he was jailed for 17 days in Henderson, Nev., based on several erroneous readings from his SCRAM monitor, which he was forced to wear after he pleaded no contest to driving under the influence. His attorney, James Dilbeck, said the time it takes for AMS to receive the data and contact law enforcement prevents wearers from effectively challenging what may be false reports.

“It sends a silent signal, and then a few days later someone comes and picks you up. By then you have no way to defend yourself other than your word,” he said. “This is an unregulated service, and judges are taking as gospel what these things report. And you cannot defend yourself. I think its garbage.”

Outside the courtroom, says Crime Report, "SCRAM has not been widely tested in independent, peer-reviewed studies. But in the testing that has been done so far, the devices have received mostly positive marks."

I don't know enough about the technology to assess its accuracy, but the story from Mr. DA Confidential doesn't inspire confidence. It does make me confident, though, that SCOTUS decided the Melendez-Diaz case properly last year when they found that defendants had the right to cross-examine lab workers whose reports are presented as evidence at trial. Without that right, in this instance Mr. Dorsen's client would still be sitting in the Travis County Jail based on a clumsy (and perhaps common) forensic error.


Anonymous said...

I deal with metrology every day in the automotive industry. I've been doing this for many years. From what I have read of forensic techniques we have much better controls. Gauge repeatability and reproducibility studies are mandatory for example. Still, I never accept any data at face value. It is wrong as often as it is right. I don't care if it comes from the best labs in the country. If it's important, you better verify.

Anonymous said...

Technology bashing is a very popular past-time these days, especially within the criminal-justice arena. And it's so easy to "cut and paste" content from other misinformed information sources to support your position about the so-call accuracy and reliability issues related to technology. Instead of bashing technology, maybe you should better understand the technology before providing commentary. I get it; that's what blogs are deliver opinion instead of fact-based information. Remember, there are five-million+ offenders being supervised within the community-corrections framework. Technology will always be needed!

Gritsforbreakfast said...

Sure technology will always be needed, 10:48. The question is will it always be reliable? Sufficiently answering that question will require more than snide assurances from anonymous blog commenters.

Bottom line: Please explain why you consider the Assistant District Attorney whose story was the central feature of this blog post a "misinformed information source"? Yes, the account was cut and pasted, with due credit, but he's describing real events that happened in court which wrongly sent a person to jail, not some theoretical legal or scientific abstraction.

Mike Howard said...

It never ceases to amaze me how people will ignore blatant facts about the failings of the criminal justice system. You're nit-picking or you don't understand how it really works seems to be a mantra from this group. But of course the bottom line is this, I don't have to understand the technology/science behind SCRAM to see that it doesn't work from time to time (or as pointed out by this post, more often than that). In fact, I'm pretty sure the ADA prosecuting the case doesn't understand the technology/science. Or the probation officers. Or the judge. You don't have to understand the technology/science to notice when it doesn't work. The bottom line is this - we the system shouldn't take SCRAM, fingerprints, eye witness IDs and the like just because those that offer them say they're reliable. Peer review, scientific studies AND cross examination are invaluable tools to make sure people are getting a fair shake. After all, we are talking about things that can either rightly or wrongly take away someone's freedom.

Gritsforbreakfast said...

Just for the record, Mike, this post doesn't claim SCRAM fails "more often than not." But it's certainly reasonable to conclude that SCRAM fails "more often than never."

Anonymous said...

This is not "technology bashing". In fact, the technology was not at fault. The data was good, there was a failure in interpretation. My god, if people had any idea how common this is. Seems the founders did well over two hundred years ago.

Anonymous said...

I worked with the SCRAM equipment since it's release in 2003 for use in the courts. It truly is a useful tool in combating DUI/DWI repeat offenders and as a sentencing stipulation. However, the developer of SCRAM (AMS) does not take into account false-positive possibilities and denies the defendant the ability to question the results. Typical scenario, SCRAM employees receive positive alert on client at 9:00PM previous evening. Readings are typically uploaded from the bracelet (attached to clients ankle) in evening hours. AMS command center is called to review readings. AMS will begin review of the alert AFTER all alcohol is burnt out of clients system. Alcohol is completely out of a persons system 12 to 18 hours after last drink, and that is if they drink a lot. I tested the bracelet getting up to a .19% BAC and all alcohol was out of my system 12 hours later. The problem with the system is that the accused has no way to prove they are innocent. By the time it is confirmed as a positive event by AMS, the client has no way to prove he is wrongly accused because the alcohol is out of the system and the SCRAM readings are the solo proof of the event. When drinking, alcohol consumed is absorbed by the body. The SCRAM detects and alerts are generated at .02% BAC and rises as more alcohol is introduced to the body, this rise continues until alcohol consumption is stopped, and then the burn-off of the alcohol begins. There were situations were I received readings that indicated a drinking event within its first few hours, and still rising. AMS SPECIFICALLY instructed me NOT to call the client in for a breathalyser test as confirmation, or back-up of the event. AMS wants to give the illusion that the SCRAM technology is full-proof and never wrong and there are consequences for the employee that questions the SCRAM results. I had a client that was called into the office in the middle of a drinking event (so the SCRAM said). Client was cooperative at all times. Readings in office showed alcohol in system (SCRAM) but client tested clean and showed no signs of intoxication. The people from SCRAM, with the client in the office, said there must be water in the bracelet and to change it and continue monitoring client for future events. Working with thousands of cases utilizing SCRAM, I believe probably 95% to 99% of positives were correct, but the thought that justice in its purist form, the chance to prove oneself innocent, is denied by the policies of AMS needs to be taken into account by courts and other law enforcement agencies.

Anonymous said...

please elaborate on what type of consequences an employee would receive from AMS?

I have been researching this for years due to a family member being sent to jail for what you describe. and I was with him during the entire time of the event and he was not drinking.

just curious.....

Unknown said...

The Scram bracelet is NOT 100% reliable. Even if it's right 99% of the time - That means that 1 out of every 100 people using the bracelet might end up in jail. The courts don't believe that you didn't drink. The judges act like the SCRAM bracelet "Doesn't make mistakes". My husband made a mistake by drinking and driving. Thank God nobody was hurt. He has to wear the Scram for 90 days. My husband accidently hit the bracelet while walking and a signal was sent that he "tampered" with it. He was told by the judge that if it happens again he could go to jail for a year. He is freaking out - He hasn't had a drink since the DUI and is doing everything right and yet he still might go to jail...