www.elsblog.org - Bringing Data and Methods to Our Legal Madness

28 August 2012

The Society for Empirical Legal Studies Executive Director, Dawn Chutkow, passed along the following that might interest those planning to attend the CELS at Stanford in November.

Ted Eisenberg (Cornell) will
conduct an Empirical Training Workshop on November 8 - 9, in connection with CELS
2012 at Stanford Law School. Enrollment is limited. The workshop begins the day
before the conference. A brief description and a link for more
information follow.

The
Empirical Training Workshop is intended for professors and students who seek an
introduction to the statistical and programming skills needed to conduct
quantitative empirical legal research. Professor Eisenberg will guide
participants through an intensive 10-hour course on statistical analysis in the
legal context. Pre-registration and a small fee are required.

20 August 2012

One of the more important and interesting emerging trends involves the strong--and growing--interest in all things ELS from scholars based outside of the U.S. Evidence of this growing internationalization of ELS abounds and can be found in submissions to and publications in the Journal of Empirical Legal Studies, participation in the annual SELS conference, and the increased methodological sophistication demonstrated in papers by non-U.S. scholars circulating on SSRN. What I have consistently heard from an array of international scholars, however, are laments about the comparative paucity of ELS-related institutions outside of the U.S.

Consequently, developments on this final point warrant attention. In Brazil, under the leadership of Marcelo Guedes Nunes,The Brazilian Jurimetrics Association ("ABJ") is undertaking the critical task of developing a scholarly infrastructure to support ELS-related work in Brazil.

"The Brazilian Jurimetrics Association is a nonprofit organization gathering researchers in Law and Mathematics, which aims to investigate and promote the use of statistics and probability in the study of Law and its institutions. The ABJ has three main missions. The first is to gather and encourage researchers to investigate and describe the decision-making processes in which individual and specific norms are created. The second is to discipline Jurimetrics as a branch of legal knowledge, defining its premises, its fundamentals, concepts, and key relationships. The third is to collaborate with public and private entities’ strategic efforts to improve the mechanisms for judicial interpretation of laws through the development of laws and court management."

Having participated in the ABJ's Second Annual Brazilian Jurimetrics Conference I can attest to the first-rate empirical work undertaken by a growing number of outstanding scholars in Brazil. Anyone interested in the development of ELS work in a "BRIC" nation should learn about the ABJ (for those on Twitter: @abjurimetria).

13 August 2012

Bob Lawless (Ill.) and Paul Stancil (Ill.) were kind enough to pass along the following quote:

"There has been a tendency on the one hand to deify and on the other to decry the results of tabulations of court business. It is easy to go to either extreme. More soundly, however, students of law administration are learning the proper function of such mass statistics in providing trustworthy facts, so far as they go, of court activities which may be used to verify, support, disprove, or suggest general hypotheses. The facts must be of such general nature as will lend themselves to average or mass verification, such as the nature of the general run of business in the courts, the character of the parties to the suit, the general methods of termination of cases, whether by court, jury or some other form of trial or by agreement or withdrawal and so on; but within these limits the facts may be definitely ascertained. Moreover, facts in themselves do not prove what should be the policy of law administration or the direction of reform therein. Their function is but to cast light upon the factors which should shape the rules of policy.”

What makes the quote particularly notable was that it comes from The American Law Institute, A Study of the Business of the Federal Courts (1934).

In 2005, the National Research Council (NRC) evaluated the “More Guns, Less Crime” hypothesis using county-level crime data for the period 1977-2000. 17 of the 18 NRC panel members essentially concluded that the existing research was inconclusive on whether "right-to-carry" laws increased or decreased crime.

"We evaluate the NRC evidence, and improve and expand on the report’s county data analysis by analyzing an additional six years of county data as well as state panel data for the period 1977-2006. We also present evidence using both a more plausible version of the Lott and Mustard specification, as well as our own preferred specification (which, unlike the Lott and Mustard model used in the NRC report, does control for rates of incarceration and police). While we have considerable sympathy with the NRC’s majority view about the difficulty of drawing conclusions from simple panel data models, we disagree with the NRC report’s judgment that cluster adjustments to correct for serial correlation are not needed. Our randomization tests show that without such adjustments the Type 1 error soars to 44-75 percent. In addition, the conclusion of the dissenting panel member that RTC laws reduce murder has no statistical support.

Our paper highlights some important questions to consider when using panel data methods to resolve questions of law and policy effectiveness. Although we agree with the NRC’s cautious conclusion regarding the effects of RTC laws, we buttress this conclusion by showing how sensitive the estimated impact of RTC laws is to different data periods, the use of state versus county data, particular specifications, and the decision to control for state trends. Overall, the most consistent, albeit not uniform, finding to emerge from both the state and county panel data models conducted over the entire 1977-2006 period with and without state trends and using three different specifications is that aggravated assault rises when RTC laws are adopted."

01 August 2012

While I truly hope not a sign of the times, as the title implies, Just Post It: The Lesson from Two Cases of Fabricated Data Detected by Statistics Alone, illustrates how simple statistical tests can be used to detect falsified data. Interestingly, Uri Simonsohn's (Penn-Wharton) main point involves a plea for journals to make available raw data used to support findings in published papers. That such a policy may assist in rooting out fraudulent papers (and deterring such unethical conduct) is a positive spillover. The abstract follows.

"I argue that journals should require authors to post the raw data supporting their published results. I illustrate some of the benefits of doing so by describing two cases of fraud I identified exclusively through statistical analysis of reported means and standard deviations. Analyses of the raw data provided important confirmation of the initial suspicions, ruling out benign explanations (e.g., reporting errors; unusual distributions), identifying additional signs of fabrication, and also ruling out one of the suspected fraudster’s explanations for his anomalous results. If we want to reduce fraud, we need to require authors to post their raw data."