Validity issues

In our previous step, we encountered several examples of good identifying variations that enabled researchers to recover the causal effects of police on crime.

The examples included highly unusual events (terrorist attacks) and unexpected, major policy reforms which had little to do with local crime and crime-relevant factors but led to substantial increases in police presence and size. These variations allow researchers to give a causal interpretation to regression results.

There are plenty of other examples of good identifying variations used by researchers to identify the effect of police on crime. Johannes Andenaes documented that the number of robberies in Denmark sharply increased after the Danish police force was disbanded by the German occupation in 1944. Gregory DeAngelo and Benjamin Hansen showed that traffic fatalities in Oregon rose after a government budget cut, which led to a mass layoff of state troopers. Lan Shi argued that a riot in Cincinnati caused by a police shooting led to increased media attention and federal investigation on police oversight, and found the number of crimes in Cincinnati significantly increased while the number of arrests fell.

Internal Validity and External Validity

The use of a good identifying variation enables researchers to interpret their regression results as causal effects of police on crime. However, there is an important caveat here. The research findings based on these quasi-experimental identifying variations are valid only under the specifics of the given research setting, and may not be easily generalized to other population and research settings.

For example, consider the example of the terrorist attack in London in 2005, which immediately led to a substantial increase in police presence in Central London. During the six weeks following the attack, the number of police working hours increased by 43 percent and crime fell by 10 percent in Westminster, Camden, Islington, Tower Hamlets, and Kensington-Chelsea. (After six weeks, the number of police working hours returned to its normal level.) Importantly, this increase in police working hours had nothing to do with local crime environments at the time, and the 10 percent decline in crime can be plausibly attributed to increased police presence. (During the period, there was no significant change in the police hours and crime rates in other parts of London.) Thus, this study has high internal validity. We can be pretty confident that the observed crime drop in Central London was indeed caused by the increased police presence.

But it is not clear how the findings from this study can be applied to other empirical settings. If police departments in other countries were to increase their working hours by 40 percent, would they achieve a comparable crime drop? There are a number of potential problems in generalizing the findings from the London study to different population and settings. In other words, the external validity of this study may be low.

What are these potential problems? First, it is not clear whether a 40 percent increase in police working hours would lead to a 10 percent reduction in crime in places with different criminal justice environment. For example, the homicide rate in the U.S. is about four times higher than in the U.K, and Singapore has more police officers per population than the U.K. by more than a factor of 3. How can we be sure that similar increase in police would lead to comparable crime reductions in the U.S. or Singapore?

Second, the increased police presence in the London study was achieved by making the existing staffs of the police force to work a lot more hours than usual. However, this empirical setting is not very policy-relevant. In order to increase police presence by 40 percent, police departments will have to hire a lot more staffs and officers to increase their presence on the streets, instead of making existing staffs to work 40 percent longer. But hiring 40 percent more police officers all of sudden is just not very realistic. (Where is the money for the hires? Where can you find thousands of well-qualified officers instantaneously?)

Also, the London study is not very informative on whether a smaller increase in police presence will yield a proportional crime reduction. If the police department in Liverpool were to increase the number of officers by 10 percent (instead of 40 percent), will crime rates in Liverpool would go down by 2.5 percent (instead of 10 percent)?

All these limitations, however, do not make research findings based on quasi-experimental identifying variations useless. Rather, these limitations motivate researchers to obtain even more research evidence from different population and empirical settings.

The validity of a research finding from a single study is limited to the specifics of the empirical setting examined, but we can be more confident of its generalizability when many other studies report similar findings from various research settings.

So far, existing research evidence (mostly accumulated from recent data from the U.S. and other developed countries) strongly suggests that more police reduces crime. But more research from different empirical settings can help us to better understand the effect of police on crime in other countries and different times.