Sunday, January 12, 2020

On the damage done by economists to criminal-justice policy: A personal polemic

In December, Grits was pleased to meet Texas A&M Associate Professor Jennifer Doleac, an economist who seeks to apply econometric methods to crime policy. Prof. Doleac runs a podcast called Probable Causation, on which she interviews economists about their criminal-justice-related work. She is an avid promoter of her profession. In her view, "if anyone is equipped to design solutions to existing problems and test whether they worked, it is economists."

Despite some fundamental disagreements, which will become evident shortly, she and I enjoyed a friendly and lively discussion. The following day, doctors put me under the knife. (Healing fine, thanks; radiation treatment begins soon.) But while recuperating over the holiday, our conversation got me thinking about what Grits has long considered the misbegotten role of economists in the making of criminal-justice policy. So I hope readers will forgive these longer-than-usual reflections on ideas Grits has been mulling over for several years.

To be fair, and fully transparent, I come to this conversation with baggage. Grits was an economics major at UT-Austin in the 1980s. But I left without a degree because, by the time I'd taken enough economics courses to understand the theories being propounded and the math behind them, I could tell they were filled with what seemed to me self serving lies, obviously false assumptions, and generally a great deal of intellectual guesswork being laundered through phony equations and masquerading as erudition. My interest in academic work flagged alongside confidence in my chosen field of study and pursuing another major would have added another year. So when your correspondent was offered full-time journalism work, I dove into it.

Not long ago, Richard Thaler received a Nobel Prize for his pioneering work in what's now called "Behavioral Economics," representing first steps toward a corrective for some of the flaws Grits identified as a student. "Behavioral economics" is shorthand for economists belatedly discovering the fields of psychology and sociology, as well as more honestly considering real-world marketing methods and the role of non-financial incentives. However, these are mostly micro-economic analyses (individual business or consumer decisions), not macro (how the economy works overall).

My college experience left Grits highly skeptical of economic theory, which at the macro level enjoys scarce more validity today than when I was a student. So it was admittedly with a shade of schadenfreude that Grits nodded approvingly at a recent Wall Street Journal retrospective titled, "Economists got the decade all  wrong. They're trying to figure out why."

Here's the thing: Economists also got the decade before that wrong! Following the onset of the 2008 economic crisis, then-Federal Reserve Board Chairman Alan Greenspan famously declared that the Fed had been operating under mistaken assumptions about the economy and corporate behavior.

And they were wrong before that, too! Balanced budgets combined with sustained economic growth and low inflation during the Clinton Administration flew in the face of all the models I was taught in the '80s.

Insofar as economists' job is to describe and predict the workings of the economy writ large, the profession has proven shockingly un-moored from reality, seeking to impose anachronistic theoretical models on situations that seldom conform to their assumptions in real life.

Dubbed the "dismal science" because of predictions of recessionary doom, economics more aptly deserves the "dismal" assessment because of its failure to produce useful results. As a field, economics has spent decades groping in the dark for relevance, blind to the reality that their byzantine modeling appears incapable of predicting real-world outcomes.

It's in this context that so many recent economists have latched onto criminal-justice topics. Unable to understand or explain the economy with their anachronistic theories, and having had their failures repeatedly exposed by the vicissitudes of history, they pretend the justice system is filled with "rational actors" and seek to apply the same ill-conceived assumptions to fresh terrain.

The late Gary Becker, winner of the Nobel Prize in 1992, was the first economist to attempt this trick. But his claims regarding what economic theory could explain vis a vis crime remained relatively restrained compared to what we see today. Becker limited his theorizing to property crimes, in which he tried to describe via mathematical modeling the cost-benefit analysis undertaken by a rational criminal. His assumptions work, more or less, for the "crime" of tax evasion - where you really do have rational actors making purely economic judgments - but immediately fall apart when applied to the sorts of crimes that generally fill jails and prisons.

Grits doesn't blame Becker for attempting to describe the world through math. His approaches have borne fruit in other areas like employment discrimination, which his disciples have much more usefully theorized. Rather, I blame policy makers for failing to recognize the shortcomings of Becker's (and his disciples') criminal-justice models, instead using them to establish a scientific veneer for simplistic-and-ultimately-false cost-benefit assumptions underlying mass incarceration.

Economists and their progeny have done a great deal of damage promoting criminal-justice policies based on Beckerite theorems. You don't see it as often these days, but in the 1980s and 1990s you couldn't swing a dead cat without hitting some John M. Olin professor of law and economics explaining to the public why tuff-on-crime policies were the way to go.

Too often, economic models simplistically assume criminals are rational actors and seek to influence their decisions through the "price" of criminality, i.e.,  punishment. This makes economists mostly a one-trick-pony in the policy arena: More punishment (increasing the price) applied more efficiently to more people becomes the solution to every unwanted behavior, from violent crime to scooters on the sidewalk.

When your only tool is a hammer, everything looks like a nail.

The fallacy of applying price theory to criminal punishment becomes fully evident as soon as one examines real-world situations. Take child molestation, one of the most terrible and despised crimes on the books. Applying an economists' frame, society should therefore apply extremely harsh punishments to maximize deterrence.

In reality, harsh punishments often deter reporting, leading to justice for fewer people. ("Yes, Uncle Ted fondled my privates, but I don't want him to go to prison, for my cousins to lose their father, for my Aunt to suffer financial insecurity," etc..)

Indeed, given harsh punishments on the books for the offense and the extent of public disapprobation associated with it, if child molesters were the rational actors Beckerite economists imagine, no one would ever engage in the behavior. But the nuances of human decision making stem from all manner of seemingly irrational impulses (some of which are intermixed with or disguised as rational ones).

The use of the justice system as a substitute for mental-health services and addiction treatment distances these rational-actor models even further from reality. Neither does the approach work within a coercive plea bargaining setting.

Alternatively, people may behave rationally along different vectors than simply demanding more-or-less punishment. E.g., victims may feel traumatized by the system, or even vulnerable to it out of their own potential criminal liability, to the point where they prefer crimes go unpunished than subject themselves and their families to its machinations.

All that said, Grits remains a consumer of academic economists' work on crime, some of which I find useful. That's mainly because most economists' criminal-justice work these days isn't really economics. It's applied mathematics.

Dr. Doleac's podcast is dubbed "Probable Causation." The name refers to Bayesian mathematical approaches using regression analysis to identify which variable may be associated with this or that observed change in the world. But probabilistic math is no more an "economic" approach when applied by economists than it is a "biological" approach when used for DNA-mixture analysis. Indeed, as Grits surely has no qualms about applying math to data, much of 21st century economists' work on justice topics may escape my criticisms of Becker, et. al., above.

The greater problem with applied math in the criminal-justice realm is the data to which said math is applied. The justice system typically doesn't gather data on the points upon which policy debates often hinge. Rather, it gathers data at the points where different bureaucratic entities interact when dealing with an individual. Cops hand off suspect to the county jail: a record is created. Charges filed by prosecutors on that person: another record is created. Then more, potentially, as prosecutors interact with judges and defense counsel, as those convicted enter prisons or probation, and so on.

For the most part, data generated from these interactions cannot answer the most pressing questions facing the justice system, such as what causes crime to rise or fall, what causes people to desist from crime, what incentives face various decision makers throughout the process, etc.. To quote sociologist William Bruce Cameron (not Einstein, despite the internet memes), “Not everything that counts can be counted, and not everything that can be counted counts.” Less well known, Cameron added, "if all of the data which sociologists require could be enumerated ... then we could run them through IBM machines and draw charts as the economists do."

The New Yorker recently published an essay demonstrating the difficulties of drawing firm conclusions from available criminal-justice data, and really that column only scratches the surface. We all rely on the same, limited information and certainly Grits readers see me crunch numbers all the time, where available, on this site. But in my experience, often available data can only give us a sense of what's going on, not provide definitive answers. Ever-more regression analyses on the same, incomplete datasets frequently yields little more insight than first-cut calculations.

There are, of course, happy occasions when some weird natural experiment affords unique opportunities for probative comparisons. Or sometimes, as with Texas racial profiling data, which was expanded in 2018 to include a wider array of datapoints, as mandated in Rep. Garnet Coleman's Sandra Bland Act, the government can gather more probative data if they're made to do so. (I'd love to see some heavyweight number crunchers tackle that new dataset, the next iteration of which comes out in March.)

But we don't have data to answer most of the biggest questions facing the justice system with any more nuance than an ape wields a sledgehammer. And society has suffered enough from economists recommending the sledgehammer in response to every criminal act.

MORE: On the limits of regression analysis by economists and justice researchers


Gadfly said...

Good post.

If you're looking for behavioral economics to read during your recovery time or whatever, I recommend Dan Ariely. He's done more in the way of field studies than any of the others.

Relevant to white collar crime, at least, he even wrote one book about behavioral economics's ideas on dishonesty and cheating.

Steven Michael Seys said...

On the issue of criminal justice, economics is as relevant as architecture, only peripherally. Sociology fails because they leave out the spiritual aspects and religion fails because they leave out the psychological aspects of the problem. We need a holistic approach that takes into account the entirety of the problem, from the thoughts of the criminal to the effects of the crime and the success of the punishment. But people are inherently lazy and uncooperative, so we want easy solutions using only the tools we have in our belt at the time.

Gunny Thompson said...

From Unfiltered Minds of Independent Thinkers of the 3rd Grade Dropout Section:

Much of what lies with basis of your theory, I can agree with. Much of what is alleged is in agreement with my belief but will require more consideration in developing the full impact of your position. My opinion of the criminal justice system use of predictive algorithms is a direct result of those in their academic towers need to justify their existence.

Gritsforbreakfast said...

Idk, Gunny, I think predictive algorithms are the result of applying new sets of tools to available problems and discovering that they're a better fit in some circumstances than others. The AI algorithms that find new strategies for the board game "Go," or guess whether alleles stacked or dropped out in DNA mixtures, or allow software engineers to predict precisely what goes on six miles underground in an oil well, may not as easily predict aberrant human behavior. But it was inevitable that people would apply those methods to find out.

There are points in the system - decisions about pretrial detention and parole, most prominently - where risk assessments occur by definition. Leaving humans in charge of those decisions in TX has left 70+% of jail inmates held pretrial and the prisons full of elderly men serving decades-long sentences. So I'm not per se an algorithm critic, even if, as always, the devil's in the details. As I said in the post, I'm criticizing the application of *economic theory* to criminal-justice problems, not mathematics.

jan said...

Don't drop by Grits as often as I did when my beloved son was incarcerated, so I did not know you had surgery with more treatment to very best wishes for a speedy and complete recovery. Dude, you were sure there for me when I needed a 'splainer! That said, pretty deep post for me. Reading it again. BTW, beloved son home since 2011, graduated TAMU working in petro and doing great. You are a blessing to folks like me, always presenting useful and helpful ideas/info thanks

Anonymous said...

A few months ago read The Undoing Project about Daniel Kahneman and Amos Tversky and their work that invented behavioral economics. Pretty amazing stuff.

Anonymous said...

You might appreciate a couple of blogs that regularly spell out the lack of "real world" in contemporary econ: and Hope your health goes well.--Mike Connelly