Thursday, July 14, 2016

Don't believe the headlines on the Harvard study purporting to show racial bias does not exist in police shootings

Racial bias in policing has emerged as one of the most contentious issues of our time. The issue has been difficult to study due to gaps in data, but this week a Harvard economist released a report that used statistical analyses to determine whether racial bias plays a role in police decisions on use of force. While news headlines covering the study declared no racial bias exists in officer-involved shootings, what the study really shows is the limitations of data and statistical analyses in getting to the bottom of the issue.

On Monday the New York Times covered the report in its Upshot column with an article, “Bias is Found in Police Use of Force but Not in Shootings.” The New York Times headline overstates the study’s findings, and Matt Drudge, Rush Limbaugh, and Rudy Giuliani, among others, picked up the story to conclude that the Black Lives Matter movement is premised on a bunch of hogwash. But the study is actually much more limited than the Times’s misleading headline, and it’s wrong for the author or media to conclude from its findings that bias does not exist in police shootings in the United States. Upshot yesterday published a Q&A with Dr. Fryer addressing the limitations of his report, and this follow-up included a much more measured description of the study’s conclusions. And today the New York Times Public Editor addressed reader complaints with the original Upshot article.

The main weaknesses in the study revolve around its limited data collection and overly broad interpretation, and its failure to take into account how racial biases impact policing and police reporting. While the author offers a series of disclaimers throughout the report recognizing these limitations, the sweeping conclusions he still draws – and certainly the conclusions media have drawn – seem to ignore those important limitations.

The piece of the study dealing with officer-involved shootings contains two parts. In the first part, the author used officer-involved shooting reports from ten different police departments to ask one question: whether race was a significant factor in “whether or not the officer shoots the suspect before being attacked.” To answer this question, the author studied incidents where an officer shot a suspect after being attacked or having a weapon pulled or attempted to be pulled on the officer, and compared those shootings to shootings where the officer was not attacked or threatened in these ways.

But the author’s data is flawed because it relied entirely on the officer’s own account of whether or not the officer was attacked or threatened, and no attempt was made to independently verify the level of threat as described by the officer. The author admits this weakness: “police departments … may contain police officers who present contextual factors at that time of an incident in a biased manner.” In other words, reports submitted by law enforcement may not be accurate reflections of the actual level of threat. This could occur through deliberate misreporting. It could be poor memory.  But it could also be the result of implicit racial bias, which (as other research has shown) causes individuals to perceive people of color as more threatening than identically positioned white people, which in turn may lead the officer to inaccurately report force-justifying behavior by black individuals. Since I began assembling data on all officer-involved shootings in Texas since 2015, I have seen numerous police accounts that describe the presence of a weapon or other threat, which is later disputed by witness or media accounts. Any quantitative study regarding police shootings needs to start with independently verified data, such as body camera footage.

In the second part of the study, the author used data provided by the Houston Police Department to come up with a dataset that purports to be comprised of incidents where an officer may have used lethal force, but did not. The author then compared that dataset, in which such force was not used, to the Houston dataset where the officer did shoot, to ask whether race was a significant factor in the decision to use potentially lethal force.

But the study does not acknowledge the barriers to extrapolating from this geographically limited data. Although headlines such as “Bias is Found in Police Use of Force but Not in Shootings” (New York Times) or “No Racial Bias in Police Shootings, Study by Harvard Professor shows” (Washington Times) would lead you to believe that its findings apply nationwide, this dataset dealt only with Houston, and indeed, only with the Houston Police Department.

And even just limiting the conclusions to Houston, I'm skeptical of isolating a dataset of incidents in which a shooting could have occurred but did not.  An economist at Columbia University was also skeptical and particularly troubled by the fact that the arrestee pool looked pretty different from the shooting victim pool.  For example, 18% of arrestees were female, but only 4% of shooting victims were female.

Neither part of the officer-involved shooting study takes into account a key stage where racial bias has been found pervasive: in decisions on whether to conduct traffic stops, searches or other discretionary policing. The author recognizes the role racial bias may play in the initial interaction, but this limitation in the study deserves more than the one-sentence disclaimer that appears. Police shootings do not occur in a vacuum, and some of the most shocking killings by police have been the result of this sort of discretionary policing that disparately impacts communities of color. Recall that Eric Garner was being arrested for selling cigarettes, Michael Brown was being ordered to get out of the road, and Philando Castile was pulled over because the officer thought he resembled a robbery suspect with a “wide-set nose.”

An underlying problem I see with the study is the basic assumption in economics that people take into account all available information, probabilities of events, and potential costs and benefits to act consistently in choosing the self-determined best course of action. This “rational actor” assumption does not leave room for implicit biases or other explanations for behavior that are not determined by conscious, rational decision-making. So, the author admits that “the penalties for wrongfully discharging a lethal weapon in any given situation can be life altering, thus, the incentive to misrepresent contextual factors on police reports may be large.” But yet the author attributes his finding that racial differences do not impact the decision to use lethal force to the fact that “officers face discretely higher costs for officer-involved shootings relative to non-lethal uses of force.” By assuming police behavior is the result of known incentives, the author fails to acknowledge how implicit biases affect perceptions in police/civilian interactions. And when an officer is considering whether a person is a threat, the way the officer perceives another person can mean everything.


Anonymous said...

Compliance with both law and when in contact with a law enforcement officer, judge, teacher, prison guard, (recommendations from a Doctor), construction worker, boat captain, TSA personnel, etc can be worthwhile and even save one's life.

I suspect more people die in a 24hr period in the US because of bad/faulty medical decisions than by bad LE shooting in an entire year. Doctors, attorneys, and law enforcement officers can be considered practitioners in their respective fields.

Bad law enforcement shootings do occur, but those are by far the exception and generally are prosecuted.

Gadfly said...

On the male/female issue, men are, on average, more violent. So, the difference between arrest and shooting pools, by sex, doesn't strike me as a big concern.

Gadfly said...

Another note: By no means do all modern economists work from a rational actor basis. suggest Dan Ariely for a good intro to behavioral economics.

Anonymous said...

And on a related note, in the NBA referees are statistically more likely to call fouls on black athletes. Hence all NBA referees are biased.

Gritsforbreakfast said...

@7:19, that final comment of yours is flat-out preposterous. "generally are prosecuted"? Just not true.

@Gadfly, that's correct re: behavioral statistics, but new, and a reaction/correction to the real flaws Amanda describes in the mainstream of the field.

@5:40, I realize you think that's clever, but the study under discussion performed regression across literally hundreds of variables searching for patterns. Just because you don't understand math beyond Algebra I doesn't mean those constraints should limit discussion among better-educated folk.

BTW, see other important critiques of this study here and here. The portion based on the Houston material IMO is most dubious. But it also included important findings related to lower-level use of force that remain undisturbed by either Amanda's critique or the others.

Gadfly said...

Grits, yes, on your last comment, indeed, Let's assume Fryer is at least halfway right on actual shootings, and that, rather than rational actors, police are Skinner's rats unconsciously internalizing fear of punishment. How do we get police more accountable, through a mix of fear of punishment, better psychological training in general, and learning from things like Project implicit, on use of sublethal force?

5:40, no, many bad shootings are still no-billled. Per the old joke, not only can prosecutors get grand juries to indict ham sandwiches, they can get them to NOT indict bad-actor cops. Fortunately, we have the second track of the civil system to partially remedy that.

John Gioffredi, Dallas Lawyer said...

Thank you again, Grits, for your typically insightful analysis in looking behind the headlines. Many of us (myself included) are either too busy or too lazy to take the time to do so, and may have otherwise drawn some incorrect conclusions from this study.

Anonymous said...

Nearly every shooting, good or bad, is presented to a local grand jury. Additionally, the Feds have the option of investigating the shooting. So if neither the Feds nor the local grand jury opt to not charge, case closed.

john said...

Race-baiting still gets lead “news” feed, yet it's total us-against-them bias, to protect their union jobs & brotherhood of COPS WAR ON AMERICA.
The only thing saving us a little is cameras, STILL the COPS ARE ALWAYS LET OFF.
They can kill you, go on paid administrative leave--while the union & courts cover it up; BUT IF YOU FIGHT FOR YOUR LIFE, THEY CAN KILL YOU EVEN FASTER, ESCALATING. For years, cop shows & movies have guaranteed you better fear the cops will come after you, bias is an understatement. Best guess, duress!
Those oft-blamed "few bad apples" have long been in charge, becoming legion. Legislators/politicians/judges/lawyers/BarCards have piled it on, as THEY fear for their crooked lives and need the cops as personal protection---thus, they offer cops more money, weapons, freedom to kill us, protection, etc. THE UNION BECAME A BROTHERHOOD AGAINST WE THE POOR PEOPLE.
As Queen Latifah just pointed out, on CBS (her family IS cops), UNTIL ACCOUNTABILITY COMES FROM THE TOP DOWN, things can't get better.
Yet, what group polices their own? Luckily for them, NRA and such groups are money-sucking cowards (wait—sounds like Congressjacks!), and no one will ever do anything, rote-conditioned to acquiesce & submit to tyranny & its willing henchmen.
(How about a column on the U.N. being okayed, against citizens?)
The only thing less indicted than COPS, is The WHillary of Babylon The Great. As president, she will make it all worse, and we bet our lives.

Anonymous said...

One man's "good" shoot is another man's "bad" shoot, depending on which side of the bullet or argument you are on. I have friends that believe any officer shooting "first" is a coward and needs to be prosecuted, no exceptions. Other friends believe people refusing to obey an officer's orders or committing a felony "get what they deserve", neither belief a rarity these days.

Anonymous said...

Ah, Harvard has way more credibility than you. That is a certainty.

Anonymous said...

I agree with everything John said. Wake up and see the injustice. One day one of your loved ones will be killed by a crooked afraid policeman, then you will know what people of color fear.

Anonymous said...

What to believe? A study from right wing extremist law enforcement biased institution like Harvard? Or the unbiased opinion a self-avowed cop hating radical like Scott Henson?

Ncrdbl1 said...

Main issue with the study is it does not support your position and so you look for ways to berate the study.

The main issue is blacks are more likely to be confrontational with [police than any other race which leads tot he shootings.

90% of all shooting have the person who was shot either being combatant or refusing to follow directions given by the officer.