Tuesday, July 16, 2013

Some more evidence that Florida's 'Stand Your Ground' law increased firearm homicide rates

The acquittal of George Zimmerman in the shooting death of Trayvon Martin has rightly turned attention to the permissiveness of Florida's self-defense laws. Although the state's 2005 "Stand your Ground" law was not used by the defense, it nevertheless framed the Zimmerman case from the very beginning. References to "Stand your Ground" and self-defense were included in the judge's instructions to the jury, and in a post-verdict interview, one of the jurors admitted that the law factored into their decision.

Under the common law "Castle doctrine" principle, individuals facing an imminent threat of death or bodily harm do not have a duty to retreat and may respond with force when in one's home. Stand Your Ground laws (SYG) generally extend this principle to any location where a person has a legal right to be and allow the use of deadly force in self-defense when an individual is presumed to have a "reasonable fear" of death or severe bodily injury. Since the passage of Florida's law in 2005, over thirty states have followed suit and adopted similar expansions of the Castle doctrine.

By definition, SYG laws make homicide less costly by providing the attacker with an additional legal defense. Indeed, as expected, these laws are associated with greater numbers of homicides that are ruled "justifiable." More troubling is that determinations of "justifiability" exhibit a stark racial bias in both SYG and non-SYG states - white-on-black killings are the most likely to be ruled justifiable, with while black-on-white killings are the least likely.

Defenders of SYG laws argue that, although homicides are more likely to be ruled justified, SYG can be expected to reduce the overall rate of homicide and violent crime. By permitting persons being attacked to retaliate in full force rather than retreating, SYG laws theoretically increase the costs of committing a violent offense. Even if justifiable homicides increase, defenders would argue that these homicides substitute for otherwise non-justifiable homicides. The net homicide and violent crime rates, in the presence of SYG laws, should decrease.

Two recent studies find the opposite. Far from deterring homicide, SYG laws increase its incidence. Moreover, the laws have no appreciable deterrent effect on violent crime. Analyzing data from the FBI's Uniform Crime Reporting system, Cheng and Hoekstra find that SYG laws lead to roughly an 8% increase in reported murders and non-negligent manslaughters. McClellan and Terkin find a similar effect on firearm homicides and firearm accidents using monthly data from the CDC. These findings are consistent with a different understanding of the incentives generated by Stand Your Ground. Rather than increase the costs of violence, SYG laws decrease them by expanding the range of legal defenses available to an attacker. Because of the vagueness of the "presumption of reasonable fear," and the absence of many third-party witnesses, SYG laws stack the deck in favor of an assailant by raising the prosecution's evidentiary burden (as was made clear in the Zimmerman trial).

Whether SYG laws increase or reduce murder rates is an important policy question deserving of further study. The Cheng/Hoekstra and McClellan/Terkin papers provide convincing evidence, but it is always valuable to re-examine any scientific finding using different approaches and methods. Both of these studies use standard panel regression techniques to estimate the causal effect of Stand Your Ground laws while controlling for other potential confounders. While parametric regression is an ubiquitous and powerful tool for causal inference, it is a very model-dependent approach. This can sometimes lead to misleading conclusions when the model gets too far away from the data.

Any approach to figuring out whether some "treatment" T causes Y relies on comparing the factual (what actually happened) to the counterfactual (what would have happened had T been different). The fundamental problem of causal inference is that we can never observe the counterfactual - we only see what happened. Statistical approaches to determining causality rely on estimating an appropriate counterfactual from the data. The ideal counterfactual is a case that is identical to the "factual" one on all relevant characteristics except for T. However, such cases are often lacking. There is no exact copy of Florida somewhere in the U.S. that did not pass Stand Your Ground. Ideally we would like to pick the closest case possible, but even then such a case may be nonexistent, particularly when potential confounding variables for which we would like to control are highly correlated with our treatment. The counterfactual may be a case that has never been seen before. When regression techniques are used to estimate these "extreme" counterfactuals, they rely on extrapolation outside of the scope of the observed data. As Gary King and Langche Zeng show, such extrapolations are highly dependent on often indefensible modeling assumptions that become more and more tenuous as one gets further and further away from the data. Slight alterations to the model can yield drastically different results. Moreover, typical ways of presenting regression results (tables of coefficients) rarely make the counterfactual apparent. It is very difficult to get a sense of the extent to which the results in an empirical paper are based on extrapolation. While robustness checks help, basic regression papers often obscure the factual/counterfactual comparison on which a causal claim is based. This is not to say that regression is useless or that the Cheng/Hoekstra and McClellan/Terkin results are fundamentally flawed. However, it is worthwhile to see whether the finding holds when using a different approach to causal inference.

Instead of regression, I use the Synthetic Control method developed by Abadie, Diamond and Hainmueller to estimate the effect of Florida's 2005 Stand Your Ground law on firearm homicide rates. This method has been used to evaluate comparable state-level interventions. Abadie and Gardeazebal (2003) use it to measure the effect of terrorism on economic growth in the Basque Country while Abadie et.al. (2010) assess the impact of California's Proposition 99 on cigarette sales. Synthetic control methods compare the factual time series of the outcome variable in a unit exposed to the treatment (Florida) with a "synthetic" counterfactual constructed by weighting a set of "donor" units not exposed to the treatment (states without SYG) such that the synthetic control matches the factual unit as closely as possible on potential confounding variables and pre-treatment outcomes. By forcing the weights to be positive and sum to one, this method ensures that the estimated counterfactual stays within the bounds of the data, thereby guarding against extrapolation. The intuition is that a combination of control states can approximate the counterfactual of "Florida without Stand Your Ground" better than any one state. The "synthetic" Florida provides a baseline for comparing homicide rates after SYG was implemented in 2005. It would certainly be possible to use the synthetic control approach to evaluate the effect of SYG in other states. However, I focus here on Florida because it was the earliest to enact such a law and has the most years for which the effects of SYG can be observed.

I use state-level mortality data from the CDC's Wonder database to construct a measure of per-capita firearm homicides for each state in years 2000 to 2010. Following the lists in Cheng/Hoekstra and McClellan/Terkin I also obtain a set of state-level covariates from Census, BLS and DOJ data sources related to age and racial composition of the population, poverty, median income, urbanization, unemployment, incarceration, and federal police presence. All of the covariates are measured in 2000 - prior to the start of the time-series.

The rapid adoption of SYG laws after 2005 unfortunately limits the set of "donor" states available for constructing the synthetic control. Only 22 states do not have a "Stand Your Ground"-equivalent law in force during the 2000-2010 period: Arkansas, California, Colorado, Connecticut, Delaware, Hawaii, Iowa, Maine, Maryland, Massachusetts, Minnesota, Nebraska, Nevada, New Jersey, New Mexico, New York, North Carolina, Oregon, Pennsylvania, Rhode Island, Vermont, Wisconsin. Nevada, North Carolina and Pennsylvania passed SYG laws in 2011. Additionally, because of data privacy concerns, the CDC does not report data for regions where a sufficiently small number of events occurred, which further constrains the total set of viable donor states. Nevertheless, the pool of donors is able to provide a reasonable synthetic counterfactual for Florida.

Florida's SYG was passed in October, 2005 meaning that it really only affected years 2006 onward. Matching Florida to the pool of controls on the set of covariates and on firearm homicide rates from 2000 to 2005 yields a synthetic counterfactual that reasonably approximates Florida's pre-SYG homicide patterns. The figure below plots the actual trajectory of Florida's firearm homicide rate relative to the path followed by the synthetic Florida sans-SYG. Homicide rates in actual and synthetic Florida match up rather well in the 2000-2005 period. However, from 2006-2010, the factual and counterfactual diverge dramatically. Florida's firearm homicide rate sees a huge increase from 2006 to 2007, while the synthetic rate begins to decline. Although rates drop from 2007 to 2010, they remain significantly higher than they would have been had SYG not been in place. The results suggest that that Florida experienced about 1-1.5 more annual homicides from 2006-2010 than it would have had Stand Your Ground not been implemented.

Firearm homicide rates in Florida - Actual vs. Synthetic Control

As with all statistical techniques, it's important to evaluate how unlikely it is that the observed pattern was generated purely by randomness. That is, how significant is this result? Although there are no specific parameters and standard errors to estimate, one can get a sense of the "statistical significance" of the apparent effect of SYG using placebo tests on our donor pool. A placebo test applies the same synthetic control techniques to cases known to be unaffected by the treatment. The resulting distribution of "placebo effects" gives a sense of the types of patterns that we would see under the hypothesis of no effect - that is, pure randomness. If the pattern exhibited by Florida appears unusual relative to then one can be relatively confident that it is not due to chance.

Firearm homicide rates in Florida - Placebo tests
(Discards states with pre-2006 MSPE five times higher than Florida's)

The figure above plots the gap in firearm homicide rates between the actual time series and the estimated synthetic control for Florida and for each of the control states. Relative to the distribution of relevant placebos, the Florida effect stands out post-2006. Florida's is the most unusual line in the set and from 2007-2010 shows a positive deviation from the control greater than any of the placebo tests. Although the pool of control states is somewhat small, limiting the number of possible placebo tests, the trajectory of Florida's homicide rate is certainly unusual and difficult to attribute to pure chance.

Supporters of Florida's law point to reductions in the violent crime rate since 2005 as evidence that the law's deterrent effect is working. However, just looking at a trend as evidence of causation makes no sense - in order to assign causality, one needs to make a comparison with some counterfactual case. Violent crime rates in Florida have been overall declining since 2000, so it is unlikely that the downward trend would not have existed had SYG not been passed.

Unfortunately, it is difficult to evaluate whether SYG reduced violent crime rates using a synthetic control approach because no good counterfactual exists in the data. Florida generally has some of the highest violent crime rates in the country and they are consistently higher than those of any of the states in the donor pool (New Mexico is close, but still lower). As a consequence, it is impossible to find any combination of control states that consistently match Florida's pre-2005 trend. Any counterfactual for Florida's overall violent crime rate would rely heavily on extrapolation outside of the data.

While these results are certainly not definitive (the relative novelty of SYG laws limits the number of periods under observation), they corroborate existing findings. Florida's Stand Your Ground law did not have a deterrence effect on homicide, and may in fact have increased the state's murder rate. This and other evidence strongly suggests that state governments should re-think their approach to self-defense laws. While politically appealing from a "tough on crime" perspective, Stand Your Ground laws likely do much more harm than good.