How to clear up more murders than are actually committed

Crime statistics are notably slippery, as The Washington Post has recently discovered to its cost. It has appended an “Editor’s note” to a story originally published on 19 February that suggested the District of Columbia used a trick to calculate how many murders it solves.

The story originated at the end of last year on Homicide Watch DC, a website that covers violent deaths in Washington. The post described the methods used by the police department to calculate the closure rate for homicide cases, which was a remarkable 94 per cent for the year 2011.

At first glance, this claim suggests that 94 per cent of the homicides occurring in Washington DC during the year 2011 had been “closed” with at least one arrest. But that is not how the figures are calculated.

Instead, homicides committed in previous years and closed during 2011 are counted as well. In 2011, 108 homicides were reported, and 61 were closed – a closure rate of 56.5 per cent. To these were added another 37 cases committed in earlier years, to make a total of 98. That’s a closure rate of 90.7 per cent.

The department added another three closed cases whose timing was not made clear, making the numbers up to 101, or 93.5 per cent, rounded to 94 per cent.

It’s an odd way of doing the calculation. The denominator applies to homicides reported in a single year, and the numerator to closures made in the same year but of cases that originated over several years, some as long ago as 1989. Homicide Watch quoted Kristopher Baumann, the chairman of the District of Columbia’s police officers’ union as saying that it wasn’t an “accurate” way of measuring the closure rate. “We’re doing a good job closing cases” he said, “but closed cases from this year and closed cases from other years are different things.”

Theoretically this approach could lead to more homicides being closed in a particular year than had actually been reported.

This point was made by a former DC police captain (now a judge) interviewed by The Washington Post in its story of 19 February. W Louis Hennesy called the statistics “entirely unfair”. He told the reporter, Cheryl Thompson, that when he ran the homicide unit between 1993 and 1995, he had used annual reporting. “I felt like anytime you could have more than 100 per cent, it’s a little bit misleading and deceiving” he said.

However, DC’s police chief Cathy L. Lanier was “hopping mad” when she read the story, complaining that it had implied trickery when all that the DC police were doing was following federal guidelines. The Editor’s note clarifies that no manipulation of the actual data takes place – though that had never actually been claimed. All Ms Thompson had written was that the figures had been calculated by a “statistical mishmash that makes things seem much better than they are” and that 94 per cent was not “a true closure rate”.

That sounds like perfectly fair comment to me. But not to Ms Lanier or the Post’s editors, who have fallen over themselves to apologise for the story, and granted Ms Lanier space for an OpEd defending her view. Now the paper’s Ombudsman has weighed in. Crime statistics are clearly a sensitive subject in Washington.

Oonagh (not verified) wrote,

Tue, 06/03/2012 - 14:18

If you tried to get away with doing something like that in a Statistics GCSE paper you'd fail.

Martin Bonner (not verified) wrote,

Tue, 06/03/2012 - 17:17

It seems a reasonable way of calculating the closure rate to me. It's the ratio of "new cases" to "closed cases" - that's a useful thing to know I think. It gives an estimate of the chance of eventually finding a murderer for a given murder. Yes, it can go over 100% (eg, if there is a terrorist attack which kills many people, and the case is closed the next year), but

Alex Sutherland (not verified) wrote,

Tue, 10/04/2012 - 10:20

This is mashing together different questions:

1. How many homicides are closed in a given year?
2. How many homicides are committed in a given year?
3. How many homicides committed in a given year are also closed in the same year?

If we want to understand the clear-up rate for homicides in a given time period then we would allow data from a wider time-frame than annual, but to present 94% as the clear up rate is just misleading without qualifiers, Federal Guidelines or not.

bob (not verified) wrote,

Sun, 22/04/2012 - 21:09

How does sampling frame relate to the target population?
Does the survey question/measurement correspond to the conclusions drawn? Give an alternative
interpretation.