Cognitive bias and forensics: Expert psych testimony may change depending on which side calls them

Among randomly chosen forensic psychologists, found a new study from researchers in Texas and Virginia, "experts who believed they were working for prosecutors tended to rate sexually violent offenders as being at greater risk of re-offending than did experts who thought they were working for the defense." Reported the magazine Popular Science ("Researchers expose troubling bias in forensic psychology," 8/28):

It's already known, of course, that lawyers may choose to call only experts who support their cases. So there's probably a biased sample of psychologists going on the stand in the U.S., anyway. But four psychologists from the University of Virginia and Sam Houston State University wanted to see if a large sample of psychologists, chosen without a side in mind, might also be vulnerable to bias.

he researchers recruited 99 forensic psychologists and psychiatrists, pretending that they wanted help with real cases. (Well, the cases files were real, but the forensic psychologists' evaluations wouldn't have an effect on the real defendants' trials.) The researchers gave each of the forensic experts the same four case files, but told half of them that the defense had hired them, while telling the other half that the prosecution was paying the bills. The cases the experts examined were for violent sexual offenders, whose sentences depend heavily on their perceived likelihood of reoffending.

On average, in most measures, the defense-hired experts came to significantly different conclusions than the prosecution-hired ones. This was using surveys that previous studies have shown work well—that is, a bunch of psychologists assessing someone using the surveys will generally come to the same conclusions—when they're not used in court.

"Most expert witnesses believe they perform their job objectively. These findings suggest this may not be the case," one of the researchers, Daniel Murrie of the University of Virginia, said in a statement. He and his colleagues' work points to a need for scientists to develop ways to reduce this bias, they wrote in a paper they published last week in the journal Psychological Science.

It's unlikely that only forensic psychology suffers in this regard. The results remind me of studies of fingerprint examiners who said the same set of fingerprints were a match in one context and, given a different set of fact circumstances, said they were not. And of course, both sides gravitate toward their favorite experts who are more likely to support the state's or the defendant's point of view. The state, though, is more likely to have money to hire them.

If who hires forensic psychologists matters significantly, that helps explain findings in a study released earlier this year by the same research team, described here by Dr. Karen Franklin, showing that Montgomery County jurors in civil commitment cases paid little attention to risk assessment scores, which they didn't trust nor understand. "What did make a difference to jurors was whether the defense called at least one witness, and in particular an expert witness," wrote Franklin. "Overall, there was a huge imbalance in expert testimony, with almost all of the trials featuring two state experts, but only seven of 26 including even one expert called by the defense." Now that this followup study suggests who hires a forensic psychologist can substantively influence their testimony, the question becomes, to what extent has that imbalance has significantly affected real-world outcomes?