Brian Leiter, who is an expert on the original legal realism (see, for example, this article), has been reading our NLR forum. He has some harsh things to say ... and they are worth reading.

I am not an expert on legal realism, nor do I intend to become one. But I have been reading quite a bit on the topic because I think it is highly relevant to understanding modern legal scholarship and the "industry" of legal education--i.e., it's crucial background.

Brian offers the most succinct rationale I have read for the pervasive influence of legal realism and why we shouldn't lament the failure of the original realism to develop an empirical component in the tradition of the social sciences:

The actual Legal Realists ... paid homage to the social
sciences, even adopting the rhetoric of the then-dominant behaviorism
(e.g., talk about the "stimulus" of the facts of the case), but their
actual scholarly practice was almost entirely insulated from the social
science of the day ... .

This isn't to say that the Realists weren't interested in what the courts do in fact,
it's just that their approach to the facts about what courts do almost
entirely eschewed social scientific inquiry, and for good reasons I
think. The paradigmatic Realist inquiries of the 1920s and
1930s [citing lots of examples] ... consisted in careful scrutiny of the underlying facts of
lines of cases, bringing out the gap between the official "doctrinal"
explanation for the decision and the actual sotto voce norms
that seemed to be at work in the judge's thinking. The goal was to
discover the non-legal norms that made best sense of the courts'
response to recurring "situation-types," i.e., patterns of fact that
seemed to elicit the same kind of results. ...

That the preceding was the heart of the Realists' "empirical" method
explains, of course, why the Realists were so influential in American
law: you didn't need social science training to do this kind of
analysis, you just needed to be a sensitive and skeptical reader of
court opinions, something good lawyers are, well, good at. ...

A "new legal realism" would continue the paradigm of scholarship
established by the old legal realists, namely, contrasting what courts
say they’re doing with what they actually do. "We are all realists
now" because this is what so many legal scholars do, including those
who know nothing of social science and don't even self-identify as
realists.

These are interesting and important points. From my own perspective, NLR is really about developing a theoretical framework (or a workable consensus) on how and when to utilize relatively new research tools and perspectives; and this requires a renewed engagement with the social sciences, which have changed a lot in theory and method since the 1930s. As Stewart noted in a comment, we should not overinterpret the name "new legal realism". Picking the name merely started a worthwhile conversation. If the conversation continues over a period of years, the NLR label will grow legs.

21 June 2006

In their opening comments and responses to other posts, Beth Mertz and Stewart Macaulay ably made the connection between the New Legal Realism (NLR) and established traditions of empirical inquiry among law and society scholars. Stewart reasserted a theme of his own classic work, that legal scholars should seek "a more accurate picture of law in action" before they offer proposals for changing law. Beth called attention to the need for socio-legal scholars to more self-consciously translate their research findings to have a broader effect within the legal academy, among policy audiences, and among the public. Both advocated a multiplicity of empirical methods, a self-critical humility for the nature of the claims that can be supported by different modes of empirical research, and at least an implicit recognition of the need to connect empirical inquiry to broader theoretical propositions about law and society. As empirical legal studies expands and matures in the legal academy, these principles are important for the intellectual significance of the enterprise.

In this essay I want to briefly argue that theoretically driven research that uses multiple methods can produce stronger validity claims, can better illuminate the social mechanisms through which law operates, and may lead to research findings that more readily translate to broader publics. In other words, research that advances the goals of NLR. I will try to carry the argument with the examples of three research projects, one by a colleague and two in which I have been personally involved.

Laura Beth Nielsen's License to Harass (Princeton University Press 2004) studied offensive public speech through observation and in-depth interviews with some 100 respondents, including white men, white women, and men and women of color. She found that the experience of these groups with sexist and racist speech on the street varied dramatically by social group, with women and people of color encountering harassment at shocking levels. While all social groups saw offensive speech as a serious social problem, a large majority of respondents in all social groups opposed legal intervention to deal with it. But the groups based their positions on different rationales. White men articulated a classic first amendment view, women invoked a discourse of autonomy (they would take care of it themselves), and people of color (especially men) spoke of a distrust of authority (they thought the law would be used against them). Without an in-depth investigation of the discourses employed by these different groups, we would know much less about their legal consciousness concerning offensive public speech. Nielsen is thus able to offer some profound insights into the experiences of different social groups on the street, their attitudes about the law, and about the role of law in shaping those attitudes and providing a license for offensive speech to continue to operate.

In Legalizing Gender Inequality (Cambridge University Press 1999), William Bridges and I examined four case studies of litigation concerning gender-based pay discrimination. Because the cases involved comparisons of wages across different predominantly male and predominantly female jobs, the cases were characterized as comparable worth cases. The defendant employers in these cases invoked a market defense, arguing that the wage differentials reflected market processes rather than gender discrimination. The courts in 3 of the 4 cases accepted that defense. Our book subjects those legal holdings to empirical scrutiny through a variety of labor market, employment payroll data, organization process data, and in-depth interviews. The different forms of data converge to reject the market defense. For example, in AFSCME v. State of Washington, we found that a state pay survey that was treated as a neutral scientific instrument for determining job-level pay was in fact riddled with arbitrariness and bureaucratic politics. As a state labor union representative commented, they were not "going to rob Peter to pay Pauline." The book concludes that the courts in these cases interpreted antidiscrimination law as potentially providing a remedy for between-job pay discrimination, while at the same time refusing to critically analyze actual wage-setting practices in these organizations. The result was that the courts legitimated between-job gender inequality in pay.

The final example concerns a project currently underway by Laura Beth Nielsen, John Donohue, Peter Siegelman, and myself. The project examines the changing dynamics of employment discrimination litigation from 1987 to 2002. We possess a full set of EEOC complaints from that period, in addition to a large random sample of federal court filings from 7 jurisdictions, which we have coded and begun to analyze quantitatively. The statistical data will allow us to comprehensively examine the system of discrimination disputing and to test whether it is operating differently than during the 1972-1987 period analyzed by Donohue and Siegelman in their earlier work. Employment discrimination litigation has become an intensely contested arena. Many commentators assert that the dramatic rise in discrimination lawsuits, which roughly tripled between 1991 and 1998, is a result of growing numbers of frivolous lawsuits.

We are also interviewing in depth parties and their lawyers in a selected subset of cases. After conducting some 50 in-depth interviews thus far, we already have gained important insights into how the system operates. The interviews have literally brought these cases to life. One African-American plaintiff described how he had been asked by his immediate supervisor whether he had ever had sex with his daughters, as he had heard that slaves used to do. When internal complaints did not remedy his work environment, he filed suit. In our dataset he is coded as an early settlement on the outcome variable. But what the coded file does not indicate is the gravity of the insult he received, the fact that as a result of the lawsuit he became divorced, went bankrupt, felt enormous physical stress. Although he won his job back as part of the settlement, he was laid off in a reduction in force a year later. When asked whether he would file the lawsuit again, he said no, that he "would have took it." In this case, as with almost all the plaintiffs we have interviewed, we find serious injuries, even though the proof of illegal discrimination sometimes is shaky. Indeed, the defense attorneys largely concede that most of the cases they encounter involve individuals who have been "wounded" at work.

While we have more data to collect and analyze, the combination of systematic statistics and in-depth interviews is likely to produce a better understanding of antidiscrimination law in action than presently exists. This combination should allow us to disseminate our results more effectively to legal, academic, policy, and public audiences than might be the case with only qualitative or quantitative data.

Larry Baum (OSU) has just published a new book with Princeton University Press that I'm sure will be required reading for all interested in this blog. Larry's earlier book, The Puzzle of Judicial Behavior (U of MI Press), is a standard in most graduate seminars on judicial behavior ( its bibliography alone makes it worthy of attention) and has deservedly garnered much attention in the field.

Judges and Their Audiences:A Perspective on Judicial Behavior

What motivates judges as decision makers? Political scientist Lawrence Baum offers a new perspective on this crucial question, a perspective based on judges' interest in the approval of audiences important to them.

The conventional scholarly wisdom holds that judges on higher courts seek only to make good law, good policy, or both. In these theories, judges are influenced by other people only in limited ways, in consequence of their legal and policy goals. In contrast, Baum argues that the influence of judges' audiences is pervasive. This influence derives from judges' interest in popularity and respect, a motivation central to most people. Judges care about the regard of audiences because they like that regard in itself, not just as a means to other ends. Judges and Their Audiences uses research in social psychology to make the case that audiences shape judges' choices in substantial ways. Drawing on a broad range of scholarship on judicial decision-making and an array of empirical evidence, the book then analyzes the potential and actual impact of several audiences, including the public, other branches of government, court colleagues, the legal profession, and judges' social peers.

Engagingly written, this book provides a deeper understanding of key issues concerning judicial behavior on which scholars disagree, identifies aspects of judicial behavior that diverge from the assumptions of existing models, and shows how those models can be strengthened.

Barry Friedman (NYU) recently published an extremely interesting piece (here) in the current Perspectives on Politics (4:2, June 2006, at 261-76) that will assuredly stimulate discussion, especially among those who study judicial behavior. As Steve Wasby (SUNY-Albany) suggests:

"One should run, not walk, to ... Barry Friedman's article.... It is, in my view (which will not make me friends), a very well-done article. He is an equal opportunity 'skewerer,' coming down on attitudinalists and 'strategists' alike."

Setting aside many and larger points of inter- (and, perhaps, intra-) disciplinary debate that Barry's piece provokes, among the important points for legal empiricists that warrant close attention include:

"Finally, empiricists in particular must take great care that the data upon which they rely presents an accurate picture of the legal world they are studying. Collecting data on the judiciary is extremely difficult and time consuming, and the temptation is great to allow the ready availability of data to define the questions that are asked and the way in which they are answered. That temptation should be avoided, because it runs the risk of presenting an incomplete and idiosyncratic view of the legal system."

Barry is correct on both fronts--Data collection is hard work and, nevertheless, the "temptation" should be avoided.

20 June 2006

Frank Cross (University of Texas at Austin) and Thomas Smith (San Diego) have a new paper on SSRN, “The Reagan Revolution in the Network of Law.” Here is the abstract:

This paper analyzes the effect of the Rehnquist Court on Supreme Court
precedent, using a network of all Court citations to other Supreme Court cases.
Network analysis enables a study of the Court’s use of precedent that may not
be readily visible. We find that the Rehnquist Court has made a dramatic alteration in the
network of precedent and, in the process, set the stage for a potentially
revolutionary change in the makeup of the law.

One obvious question about New Legal Realism is how it might differ from the old. Can we claim anything more than we have a website, while Karl Llewellyn didn't? (See http://www.newlegalrealism.org). As Bill noted in his Introduction to this blog forum, much of the old legal realism was aimed at freeing appellate judges from the confines of formal approaches so that they could carry out good policies or their "situation sense" as they decided cases. A new legal realism is not as focused on appellate judging as the old. Instead, new legal realism takes full advantage of about forty years of law and society research that suggests what the law in action looks like.

I've commented elsewhere that we would be making progress if mainstream legal scholarship would simply take account of this research. Although participants in this blog are familiar with such points as law is not free and law is delivered by actors with limited resources and interests of their own, too much writing in law reviews still proceeds with little concern with these well-documented realities. In a talk I gave at the contracts panel at the last AALS meeting, I played with an analogy to the fate of the nuclear submarine USS San Francisco. In January of 2005, it was cruising at top speed at a depth of more than 500 feet. It crashed into an undersea mountain, killing one sailor and injuring 97 others. The crash caused more than $88 million damage to the boat. The mountain was not on the boat's navigational charts, although potential hazards in the area were noted on other charts that the San Francisco's officers had never seen. Legal scholarship that proceeds without awareness of the existing empirical charts of the law in action risks a similar fate. I did conceded that contracts scholarship is unlikely to kill or injure anyone. Nonethelesss, it could be rendered ineffective, or even have unintended consequences, if it is not based on an accurate picture of the law in action. And a major part of that picture involves negotiation and discretion rather than a neat rule-bound system.

How do we get a more accurate picture of the law in action? We must turn to the methods of all the social sciences. Law is too complex to grasp fully if we favor one approach over all the others. My late wife Jackie Macaulay had a Ph.D. in social psychology from a department that prided itself as the "Dust Bowl of Empiricism." (She later became a lawyer). She suggested that the best working rule was to presume that all social science approaches are flawed, but the game is to understand the limits of each and use the best approach available for what you want to study. Sometimes we can get neat tables of numbers and use state of the art statistics. She did some of this kind of work herself, but she always suggested that I ask where the numbers in the tables came from. Human behavior doesn't turn itself into numbers by some magical or natural process. Someone has to code it, and the quality of the coding varies. Many small, unfounded assumptions can be hidden within apparently neutral quantitative results.

Our Wisconsin colleague Bert Kritzer compared what he learned from asking questions with what he learned from watching lawyers in contingent fee practice. He says, "the observational studies present more nuanced images, which tend to contradict the more straightforward results of the interview-based analyses." He quotes Bob Dingwall: "interviews construct data, observers find it; in an interview study, we can pick and choose the messages we hear and that we elicit, [while] in observation we have no choice but to listen to what the world is telling us." Of course, often it isn't easy to get permission to watch, and even when it can be done, it is a much more time-consuming approach. And, as with all other methods, there is a discipline to observation which, if ignored, can lead to unreliable results.

Sometimes we are limited by the available sources of information. We may not be able to interview a randomly selected sample. There may be a limited number of people who know what is going on and who are willing to talk to a researcher. Moreover, often such people are unwilling to talk if you do not promise confidentiality. There are, of course, a number of problems when we go down this road. For example, our informant may tell lies or spin his answers to make himself look good. She may seek to entertain us by telling us about the atypical or interesting events rather than the dull everyday practice. It is hard to know how representative our informant's knowledge is. If we were able to talk to more people with different experiences, would the story differ significantly? And other scholars cannot replicate our work because we cannot tell them who we talked to. I've seen some suggest that we should not interview business people or lawyers about contracts practices because we cannot get a good sample and our work cannot be replicated. I'm sure that my reaction is influenced by the alternative -- often if we don't rely on those informants we can get, we can do nothing. I'd prefer to recognize the limitations of such approaches and flag them in my articles. For example, in my article on Lawyers and Consumer Laws, I described the research that involved interviewing about 100 lawyers in five Wisconsin counties as well as its limitations. I concluded: "It [the article] should be read as a report from a preliminary study, offering suggestions that the author thinks are true enough to warrant reliance until someone is willing to invest enough to produce better data and lucky enough to find a way to get them." You can take some steps to minimize the risks, and we should where we can. At the minimum, we can challenge the story that we get and ask whether it is plausible. Then we can wait for other studies and see if they find much the same thing. One lesson of new legal realism, gleaned from good social science practice, is that we need a healthy dose of humility in drawing conclusions from our research. And this is true whatever flavor of social science we prefer.

Dean Elena Kagan represented the new Fellows of Class III, Section 4 of the American Academy of Arts & Sciences. She said: "The practice of law and the study of law are growing farther apart, to their mutual misfortune. Fewer law professors understand what it means to be a practicing lawyer or what issues lawyers are confronting. And fewer judges and lawyers look to scholars for insight or guidance. Less and less does practice inform scholarship, nor does scholarship improve practice." A new legal realism would encourage academics to study what lawyers do, and with what consequences to whom. This should be worth our attention. My guess is that what lawyers do, and don't do, matters. One way that we can translate more effectively between empirical research and legal practice is to gain a better understanding of the realities of law on the ground, and quite often (althought not always) it is lawyers who put the law into action. Of course, a thorough study of law on the ground means that we not only include law on the books and law in the attorney's office, but we need to add law in everyday life. This research into the living law has been ongoing for many years, and law professors and students should be far more aware of it.

What might we gain by the new legal realism project? Law professors advocate legal rules that they hope will gain some measure of, in Llewellyn's phrase, the good, the true and the beautiful. We write amicus briefs and law review articles and we draft statutes and regulations seeking reform. Recognizing all of the problems with offsetting power, we should do better if our proposals reflect the way law affects people from the ground up. If we care about justice, we need to understand much more than the text of legal rules. Here is where the new legal realism continues the spirit of the old.

19 June 2006

According to Amazon, the book was released on June 8th. Barnes & Noble’s website says it won’t be released until June 28th, which is probably correct. But I’m splitting the difference by posting it now.

Check out these endorsements from some of the ELS Blog’s favorite people:

“Not only is this a solid piece of research, it also does a
remarkable job of translating complex ideas-long floating around in the social
sciences-into prose accessible to a broad audience. This is an important
service. Plus, the book couldn't come at a better time. Senators (and
presidents), for the most part, understand the ideological component of judging
but the public seems to need a reminder. ARE JUDGES POLITICAL? provides just
that.” - Lee Epstein (Northwestern University)

“This is a significant book. The judiciary decides many
important policy questions in the United States, and in practice the
circuit court judiciary is much more important than the Supreme Court. This is
evident by the great and growing press and political attention to judicial
nominees. ARE JUDGES POLITICAL? greatly informs the debate, with its empirical
findings and its discussion of their pragmatic significance.” - Frank B. Cross (University of Texas-Austin)

“ARE JUDGES POLITICAL? is thoughtful and careful. It is a
terrific empirical introduction to the politics of judicial decisionmaking, an
area that is capturing the imagination of many in the legal academy. The
analysis of panel effects, which appear to be as large as partisan effects, is
especially interesting.” - Jeffrey A. Segal (Stony Brook University)

I went to Vancouver last week for the AALS workshop on intellectual property and fell behind on the
latest developments in the dispute between Lott and Levitt. Nothing has happened in court since my last update -- I just checked the docket report on Pacer -- but here is the news from outside of court:

2. The next day, James Lindgren (Northwestern) weighed in on the lawsuit on the Volokh Conspiracy. Unlike Lott’s complaint, Lindgren focuses on the context of the allegedly defamatory statement in Freakonomics, emphasizing that it is made right after raising questions about another one of Lott’s studies, a 1997 telephone survey. The question about the telephone survey is whether Lott actually conducted it. The statement about the failure to replicate Lott’s results involves a different study, and no one doubts that Lott conducted this one. Lindgren worries, however, that the juxtaposition of the comments about the two studies is misleading, perhaps suggesting that Lott did not conduct either one of them. But he still predicts the lawsuit will be unsuccessful.

“Jeffrey Rosen makes a powerful and accessible case for a restrained
judicial role, one that will challenge liberal proponents of Warren
Court activism and conservative proponents of Rehnquist Court activism.
His work combines sound historical scholarship with important
prescriptions for contemporary constitutional politics.”-- Mark A.
Graber, University of Maryland

What a great introduction, Bill -- thanks! We appreciate being invited to join this interesting conversation on the ELS Blog. Stewart Macaulay and Bob Nelson will be my co-hosts, and together we'll try to do justice to describing the ongoing efforts of the scholars who've been involved in the New Legal Realism Project (NLR).

NLR began with a group of social scientists and law professors who were concerned about the difficulties involved in translating between law and social science. Initial discussions were held at the American Bar Foundation and the University of Wisconsin's Institute for Legal Studies (ILS), both of which have had decades of experience in fostering the social science study of law. The First New Legal Realism Conference, funded and organized by ABF and ILS, was held in Madison in June 2004. Many of the papers from that conference are appearing in a unique publication collaboration between the Wisconsin Law Review, a student-edited law review, and Law and Social Inquiry, one of the leading peer-reviewed journals in the area of empirical research on law. (Working out that collaboration turned out to be a very practical exercise in interdisciplinary translation!) We've since followed up with a series of conferences and events, with more to come.

We started out with the observation that achieving a high standard in interdisciplinary translation requires some attention to the translation process itself. On the one hand, legal professionals may try to pick up on social science findings without understanding the frameworks from which those findings emerge. This can lead to misinterpretation of research results, or to reliance on faulty social science. On the other hand, social scientists may also assume that the translation process is transparent, failing to grasp the very different epistemology and goals of legal language and processes. I often quote Epstein and King's pithy summary: "An attorney who treats a client like a hypothesis would be disbarred; a Ph.D. who advocates a hypothesis like a client would be ignored." While law professors don't have to treat their favorite theories like clients, their training and professional culture can arguably predispose them to do so. And, to the extent that they are debating normative and interpretive principles rather than empirical evidence, law professors have defensible reasons for proceeding differently than social scientists do.

Despite the many divergences among the social sciences, I do believe that social scientists have a distinctive shared commitment: if our theories conflict with what we discover through well-established research, it is the theories and not the findings that need to be discarded or changed. This operates as a check on researcher hubris -- on our very human tendency to see what we want to see in the data we collect. (Dan Kahan provided a great example of this in his guest blog last week.) It is also a check on power, because at its best, empirical research can permit our research subjects to talk back to us -- to shake and jar the assumptions and pre-given frames with which we approach them. (And, because I was trained as an anthropologist, I have to say I think that well-done, rigorous participant observation is one of the best empirical methods for achieving this kind of check on theoretical assumptions. But I know that other methods have their advantages as well!)

On the other hand, social science disciplines themselves differ in terms of methods, goals, and epistemological assumptions. As Monahan and Walker explain in their classic text on social science in law, social scientists must often sacrifice precision in one part of the picture in order to obtain a fuller understanding of another (e.g., internal versus external validity). Obviously, the more that we can combine different kinds of methods, the less we have to sacrifice. Bill Henderson and Howard Gillman made a similar point in earlier posts on this blog. I liked Howard's comment that "empiricism takes a variety of different forms." As he correctly noted, excluding either qualitative or quantitative research from our definition of "empirical" is not only technically erroneous -- more importantly, it would cut off important sources of information.

To summarize, the NLR network of scholars is working to develop more informed and precise methods of translating between social science and law. This requires that we begin by examining the issue of translation itself, as opposed to assuming linguistic transparency among disciplines. If legal professionals are to obtain the fullest possible understanding of law in action, they will need to draw on the broad range of social science disciplines and methods currently available. To achieve this, we will need a theory and practice of translation that is itself empirically grounded. We will also need to synthesize research conducted using different methods (something Bob will be talking about on Wednesday).------------------------------A personal PS: Here I want to add a comment that represents just my personal take on this issue of interdisciplinary translation. (Like ELS researchers, NLR scholars come to the table with a variety of disciplinary and other positions -- this is mine.) As a linguistic anthropologist, let me put in a quick plug for my field. The past two decades of empirical research on language and communication in anthropology and sociolinguistics has shed some valuable light on processes of "translation" between different institutionally-based discourses (or, in the terminology of Michael Silverstein at the University of Chicago, "transduction"). One key finding is that our ongoing ability to communicate depends in structured ways on metalinguistic cues (and, specifically, on "metapragmatic" cues that signal us about how our speech connects with context). Thus we are likely to miscommunicate when we translate across disciplines if we do not first examine divergences in metalinguistic assumptions and structures. (This is not unlike Kahan's warning that we need to unpack underlying cultural worldviews in order to bridge communicative divides -- and indeed, language research has connected cultural worldviews with metalinguistic structure. In my own research, I've used transcripts of first-year law teaching to examine some aspects of legal metapragmatic structuring.)------------------------------There's lots more to say, and I'm looking forward to the discussion! ABF Director Bob Nelson, whose well-respected sociological research on law has combined multiple methods, will lead off on Wednesday. Tomorrow we'll get to hear from Stewart Macaulay, renowned for his research on relational contract and also co-editor of a classic text on social science of law.

A while back, Legal Realism became associated with an eccentric view of
judicial behavior, i.e., that it is better explained by judges’ breakfast selections than anything remotely identifiable as law. As Judge Kozinski described it, “If the judge has a good breakfast and a
good night’s sleep, he might feel lenient and jolly, and sympathize with the
downtrodden. If he had indigestion or a bad night’s sleep, he might be a grouch
and take it out on the litigants.” Alex Kozinski, What I Ate for Breakfast and Other Mysteries of Judicial Decision
Making, 26 Loy. L. A. L. Rev. 993, 993 (1992-1993); see also Ronald Dworkin, Unenumerated Rights: Whether and How Roe
Should Be Overruled, 59 U. Chi. L. Rev. 381, 391 n.16 (1992) (referring to
“the old realist thesis that the law is only what the judge had for
breakfast”); Ronald Dworkin, Law’s Empire 36 (Harvard 1986) (“Some realists ...
said there is no such thing as law, or that law is only a matter of what the
judge had for breakfast.”). As many have pointed out before, this is not an accurate characterization of Legal Realism. See, e.g., Frederick Schauer,The Limited Domain of the Law, 90 Va. L. Rev. 1909, 1923 (2004).

But now there’s a New Legal Realism, and we want to help start things off right, lest
it too become wrongly associated with one of the big three meals of the day. According to the New Legal Realism website,
the basic idea “is to develop rigorous, genuinely interdisciplinary approaches
to the empirical study of law.” This is right up our alley
here at the ELS Blog. We are therefore pleased to have with us this week Professors
Elizabeth Mertz
(American Bar Foundation & Wisconsin), Stewart Macaulay
(Wisconsin), and Robert
Nelson (American Bar Foundation & Northwestern). They will explain New
Legal Realism, what it is, what makes it new, and why it’s not about pancakes verses waffles.

16 June 2006

While independently replicating each others results (just to be safe and prudent), a colleague and I inadvertently uncovered an issue whose import I had previously (but no more) underestimated--stats software updates.

In our specific instance, Stata's move from version 9.0 to 9.1 (I now use 9.2) implicated, among other commands, those germane to clustered standard errors. One result of Stata's update was ever-so-slightly different output (again, only relating to our clustered standard errors). Because we were working off the identical data and do files (but, alas, as we subsequently realized, slightly different Stata versions) any discrepancy alarmed us. An incidental hallway chat with Ted Eisenberg reminded me of Stata's version control command which allows users to run prior Stata versions. Once I did, exact replication (and much relief) emerged. (And for purposes of this discussion, let's set aside disputes about what "replication" might mean in a legal sense.) Because I frequently flip between stats packages (e.g., SPSS and Stata), I've always been mindful of subtle differences between stats software packages. This episode taught me that I need to be similarly sensitive to subtle differences across versions within a software package.

To the extent that this is an issue that will likely persist over time, perhaps we need to develop a norm promoting the disclosure of the software version used to generate results so that those following our footsteps can save some time.

15 June 2006

Over the last two weeks, I am been cleaning and checking a large longitudinal dataset on the geography of large law firms and Fortune 500 companies. A large portion of this work has been done by research assistants. I am now left with coding and cleaning all the special cases and running various checks for reliability.

Although this work is time-consuming and tedious, the upside is that it is largely rote work that does not require intense concentration. To help pass the time, I have spent many hours listening to various audio files on NPR. I am a big fan of Fresh Air with Terry Gross. Here are some of the more interesting segments I recently heard:

Washington Post Congressional correspondent Juliet Eilperin
on her new book, "Fight Club Politics", which provides a detailed (and persuasive)
chronology of factors that have produced the intense partisanship in
the U.S. House of Representatives.

Actress CCH Pounder, who plays a detective on the FX drama, "The Shield." Pounder is black but is from Guyana and was schooled in Britain; in contrast, the characters she plays are usually African-American. This is a stunning interview on issues of race in the U.S.

Neil Young and Jonathan Demme on their new documentary, "Heart of Gold."

"The
2005 Annual Report presents an overview of major Commission activities
and accomplishments for fiscal year 2005. See the Commission's 2005
Sourcebook of Federal Sentencing Statistics for descriptive figures,
tables, and charts, and selected district, circuit and national
sentencing data."

For sentencing and crim law folks in general and empirical wonks in particular, these resources (particularly the Sourcebook) add to a critical (and growing) data base. As Doug notes, scholars pushing theories about or assessing the efficacy of the Booker decision (e.g.) might want to look at what is actually happening with federal sentencing.