Of journalists, drunks, lamp posts, and Year-To-Date crime data

A note to my journalist friends: Can we please dispense with news stories featuring "year to date" crime statistics, murder rates, shootings of and by police officers, etc.? The Houston Chronicle had a feature this week citing a supposedly declining murder total from the year-to-date data, after two successive years of growth in that crime category.  I'm not picking on them especially; lots of people in the MSM do it. But the practice reflects a philistine view of math and crime trends that uses data as a drunk uses a lamp post: For support rather than illumination. It should stop.

Criminologist Jerry Ratcliffe, using data from Philadelphia, has shown why these trend predictions, especially when conducted mid-year or earlier, have little if any probative value.

To use calendar YTD comparisons with any confidence, we have to wait until the end of October before we can be more than 50% confident that the year-to-date is indicative of how we will enter the New Year. And even then we still have to be cautious. There was a chance at the end of November 2010 that we would end the year with fewer homicides, though the eventual count crept into increase territory.

The bottom line is that with crimes such as homicide, we need not necessarily worry about crime panics at the beginning of the year. This isn’t to say we should ever get complacent and of course every homicide is one too many; however the likely trend will only become clear by the autumn.

Frequently, real, long-term trends cannot be divulged from crime data until years have passed because of lags in reporting, differences between jurisdictional challenges and practices, and a wide array of variables which may drive different trends at different places and times (sometimes cyclically, as with summertime crime increases, and sometimes episodically, based on specific situations like the opiod and meth epidemics or drug cartel trafficking patterns). The less common the crime, the more time is needed before data may be meaningfully interpreted.

Murders, like shootings of police officers, are uncommon occurrences where small numerical changes can result in big spikes and troughs in annual totals, particularly when one is looking only at one city our county. And because datasets are small, error rates are high when predictions or assumptions are based on them. Your correspondent was taught that truism 30 years ago when I first began writing about these topics and was warned repeatedly by experts and editors not to overstate crime data. And that was in an era now known for promoting "tough on crime" memes and an if-it-bleeds-it-leads mentality in the press. But those lessons about data seem to have been lost. I see these YTD stories all the time, on a number of different topics involving small numbers of rare occurrences where they make little sense.

Perhaps expert analyses like Prof. Ratcliffe's can help re-familiarize reporters with the limits of and problems with using these year-to-date datasets to say murders (or police officer deaths, or other rare occurrences) are rising or falling. You can write at the end of the year about increases or decreases, but because of inherent limitations in the data, especially at the city level, these year-to-date comparisons probably misinform more than they illuminate. That's particularly true because the stories where they purport to go up are hyped much more than articles like the one in the Chron estimating that the murder total will be lower this year.

My fear is that these year-to-date stories are too easy for journalists to cease; the lamp post provides too much support for shoddy journalism and simplistic thinking, filling a vacuum where more probative data and expert knowledge suffer from gaps. So Grits isn't sanguine that most of the drunken journalists leaning against the street lamp in this metaphor will be able to stand on their own two feet, much less that they'll soon turn their faces toward the light.