www.elsblog.org - Bringing Data and Methods to Our Legal Madness

17 May 2006

Congratulations to Lee Epstein (Wash U/Northwestern), Daniel Ho (Stanford), Gary King (Harvard), and Jeffrey Segal (SUNY-Stony Brook), whose article, The Supreme Court During Crisis (80 NYU L Rev 1 (2005)), won the APSA Law and Courts Section 2006 McGraw-Hill Award as "the best article published by political scientists on law and courts" during 2005. Interestingly, as Jeff Yates (Georgia) notes, Epstein et al.'s awarding-winning political science paper was published in a student-edited law review.

15 May 2006

At the recent American Law and Economics Association Annual Meeting held in Berkeley, John Donohue and Justin Wolfers presented a compelling analysis of the deterrence effect of the death penalty -- or rather, the lack of evidence thereof. Comparing execution rates with homicide rates, as well as using the natural experiment of the Furman abolition period, and contrasting U.S. trends to Canadian trends, Donohue and Wolfers cast considerable doubt on whether the death penalty has any deterrence effect. But Donohue and Wolfers look only at whether the death penalty deters crime, not whether the death penalty affects other criminal law matters, such as encouraging defendants in murder trials to accept plea bargains with harsher terms than they otherwise would, as Ilyana Kuziemko recently showed.

But more interesting than Donohue and Wolfers substantive case -- they ultimately conclude that it is entirely unclear whether the death penalty causes more or less murders -- is their methodology. They replicate the analyses of a handful of central studies that show there is a deterrence effect, and run a variety of robustness checks on each. As such, their paper provides a comprehensive re-examination of the primary data on the deterrence effect of the death penalty on crime.

A recent paper by Leandra Lederman (IU-Bloomington) and Warren Hrung (Fed. Reserve-NY) exploits unique aspects of the U.S. Tax Court to explore the possible effects of attorneys on case outcomes. In the U.S. Tax Court, evidently, a substantial number of private litigants proceed pro se. The Tax Court is also unique in that it retains records of settled cases (as well as, of course, tried cases). Thus, armed with "a unique data set consisting of a random sample of settled and tried cases" the authors set out to empirically assess the influence of legal counsel on various case outcomes, including financial and disposition time (through settlement or trial). Lederman and Hrung find, in part, that the presence of attorneys correlates with more favorable financial outcomes in tried but not settled cases. The presence of attorneys does not influence disposition time.

The Washington Post (via the WSJ Law Blog) is reporting that $200,000 is now the going rate for bonuses for Sup. Ct. clerks. Since most second-year BigLaw associates make $150K to $160K, a year at the Supreme Court is now a financial boon rather than a sacrifice: $63,335 (what a SCOTUS clerk makes) + $200,000 = $263,335. Is the value to the firm nonpecuniary or at least indirect--e.g., bragging rights on the number of former clerks? Would a firm bill out a former Sup Ct clerk at a higher rate right out of the box?

It is not outside the realm of possibility that law schools, who also compete for former Sup Ct clerks, will eventually offer bonuses as well. Granted, the external economies are different, and there is some evidence that a prestigious clerkship is not a valid predictor of the production and placement of future scholarship. See Paul L. Caron & Rafael Gely, What Law Schools Can Learn from Billy Beane and the Oakland Athletics, 82 Tex L. Rev. 1483 (2004). We'll see.

14 May 2006

It is with great pleasure that I introduce Tonja Jacobi as this week's guest blogger. Tonja joined the Northwestern Law faculty in 2004, and earned her PhD in political science from Stanford
University, where she wrote her dissertation on separation of powers
constraints on the judiciary. She also holds a Masters from the
University of California, Berkeley and a law degree from the Australian
National University. Her areas of interest include judicial politics,
game theory, American governmental institutions and constitutional law.
Her current projects include a positive political theory of doctrine,
an empirical study of the causes of litigiousness, a model of judicial
signaling behavior, and a critical analysis of the use of state
legislation as evidence of a national consensus in death penalty cases.

13 May 2006

Judge Luttig's departure from the Fourth Circuit has renewed debate over judicial compensation. Over at the W$J Law Blog, Peter Lattman cites Chief Justice Robert's 2005 statement that the low pay is a “direct threat to judicial independence" and asks whether federal judges need to make more money. According to Ann Althouse, the clear subtext of Luttig's resignation letter is that "Federal judges are underpaid compared to their alternatives."

What does the data show? I dug into my files to generate this chart, which appears to support the argument of lagging pay. Obviously, there is a growing disparity between judge pay and jobs of comparable prestige in the private sector. (click to enlarge.)

Sources: The American Lawyer, The Lawyer's Alamac

Yet, is corporate CLO and Am Law 50 partner the proper reference group? Granted, Luttig is leaving for Boeing, and Chief Justice Roberts was a partner at Hogan & Hartson (Am Law #21 in 2003) before joining the D.C. Circuit. But over the last twenty years, the earnings of the prototypical median law firm partner (25-29 years out of law school) have not grown nearly as fast. Consider this chart: (click to enlarge.)

"This data collection provides annual data on prisoners
under a sentence of death and prisoners whose offense sentences were
commuted or vacated during the period 1973-2004. Information is
supplied for basic sociodemographic characteristics such as age, sex, education, and state of incarceration. Criminal history data include
prior felony convictions for criminal homicide and legal status at the
time of the capital offense. Additional information is available for
inmates removed from death row by year-end 2004 and for inmates who
were executed."

In addition to the data and necessary documentation (e.g., codebook), ICPSR also makes available ready-to-use SAS, SPSS, and Stata files. Even those wholly uninterested in these data might wish to familiarize themselves with ICPSR's holdings. ICPSR is a veritable treasure-trove for empiricists, including, increasingly, legal empiricists.

11 May 2006

Over at Concurring Opinions, Dan Solove discusses his recent
satirical essay on the Multistate Bar Examination. Dan’s other blog posts make clear that he favors the abolition of the bar exam. “In lieu of the Bar,” Dan
writes, “states should permit all students who graduate from an accredited law
school to become members of the Bar after working a certain number of supervised
pro bono hours.”

I am somewhat skeptical of this proposal because of the
agency problems that plague legal education. What law professors find interesting (and thus worth teaching) may not
be a good preparation for a career as a practicing lawyer. The bar exam is at least a reference point
for core substantive knowledge. If we
think it is a poor gatekeeping device, then the solution is professional
engagement with state bar examiners … But most legal academics are rewarded for scholarship rather than
teaching or service to the profession. So we are back to incentives and agency
problems; we complain but nothing changes. Sustained complaining is what
is required, and the opportunity cost is quite high.

That said, the bar exam itself rests on some rather extravagant
theoretical and empirical claims. For
example, in a 1976 monograph entitled “The Relationships among Law School
Predictors, Law School Performance, and Bar Examination Results,” the authors
(two psychometricians from ETS) described the bar examination as “the ultimate
test of competency” (p. 1). The authors
also observed, “The bar examination measures, both essay and MBE, are in effect
achievement tests of some the skills
and knowledge developed, and tested, in law school” (p. 5, emphasis added).

Really? Few law
professors (or practicing attorneys, for that matter) would sign off on these statements. Perhaps this statement is more accurate: "The bar exam and the MBE are flawed, imprecise measures of competency and achievement; but they have persisted for decades with little or no change because of institutional incentives among legal educators." This situation can be fixed, but who is going to bear the cost?

10 May 2006

Should Eugene Volokh have received IRB approval before engaged in his law review experiment? Dan Filler asks this question here. As an experiment, the UCLA law professor participated in the UCLA Law Review write-on competition.

09 May 2006

This article provides an empirical study of the effect of precedent on judicial decisionmaking. Precedent is much analyzed as a controlling factor for judicial decisions but rarely tested for its effect. Some have argued that reliance on precedent creates a system of path dependence of the law, which may yield undesirable "herd" behavior. As a framework for the study of precedent, we use Dworkin's metaphor of a "chain novel," in which each deciding judge writes upon a background to which he or she must adhere, thereby influencing his decisions. This yields testable hypotheses about judicial decisionmaking. Given the considerable data on the ideological component of judicial decisionmaking, it provides a means to study the effect of precedent, as a constraint on judicial ideology. First, we examine "cases of first impression," for which there is no controlling precedent, like the very first chapter of a chain novel. The hypothesis is that the absence of precedent in these cases frees judges to use their ideology to a greater degree than in other cases with precedents, and this hypothesis is confirmed. Judges render more ideological opinions in cases of first impression. The second study considers the growth of precedent over time in the context of 42 U.S.C. Section 1983 litigation.

The hypothesis is that as precedents accumulate, they steadily narrow the range of judicial discretion, so the effect of ideology should dwindle over time, as might be expected from a chain novel. The data falsify this hypothesis, as ideology seems to increase somewhat as a determinant as more precedents build up. The results suggest that precedent acts as only a limited constraint on judicial decisionmaking and the path dependence of a system of stare decisis is exaggerated.

In his comments about the distinct shift toward empirical papers at ALEA (a point echoed by many, including Bill Henderson), PrawfsBlawg's Ethan Leib brushes up against an equally interesting point (here as well): whether legal scholars should recognize empirical methodology as a distinct subfield.

The point warrants careful consideration. Although the main thrust of Leib's comments is to suggest that empirical legal methodologists "get a free pass at needing to have a substantive field of expertise in the law," ironically, Leib's post fails to consider that methodology may constitute its own substantive subfield within law. Methodologists in fields outside of law argue that it should. (E.g., political scientists Gary King (Harvard) and Lee Epstein (Northwestern) wrote in 2002: "The law is important enough to have a subfield devoted to methodological concerns, as does almost every other discipline that conducts empirical research.") Finally, to the extent that legal scholarship continues in its trajectory toward the empirical, it is particularly important that folks with substantive legal backgrounds and expertise contribute to and help shape the development of a subfield devoted to empirical methodology and technique. As King & Epstein also noted: "Scholars toiling in the social, natural, and physical sciences can help, but a whole field [law] cannot count on others with differing goals and perspectives to solve all of the problems that law professors [legal scholars] may face."

08 May 2006

I first heard today about Edward Tufte’s
course, “Presenting Data and Information.” Tufte is Professor Emeritus of
Political Science, Statistics, and Computer Science at Yale and author of several
books about presenting information, including Visual Explanations, Envisioning
Information, and The Visual
Display of Quantitative Information. His one-day course travels around the
country, with upcoming sessions in Virginia, Oregon, Washington,
Colorado, Pennsylvania, New York, Connecticut,
and California.
More information can be found here.

Anyone
who has taken this course is certainly invited to share any comments about it.

07 May 2006

Over at Conglomerate, Fred Tung reports his impression of a strong empirical trend at this year's ALEA meeting. I agree. Every panel I attended had between one and three empirical papers. From my perspective, there was some very high-end applied econometrics on display. The phrase "robustness check" seemed to echo through the halls of the UC Clark Kerr Campus.

Here are some of the empirical papers that I particularly enjoyed (in the order they were presented):