July 08, 2009

The Posner/Landes machine rolled into the summer WIP Thursday, as
they presented their latest project (also co-authored by Northwestern
University's Lee Epstein,
who was not present but whom both effusively praised as having compiled
every judicial dataset conceivable to humanity): Inferring the Winning
Party in the Supreme Court from the Pattern of Questioning at Oral
Argument. Court observers such as Linda Greenhouse had long suspected
there was an important relationship between the number of questions
asked and the ultimate outcome of the case, but nobody had done the
empirical work to figure out exactly what it was. Posner, Landes &
Epstein looked at the effect the raw quantity of questions (and total
words in questions) had on the probability a given side would win their
case before the Court s using data tabulated from all arguments for all
cases decided in the 1979 to 2007 period . They found a consistent
correlation: Whether measured by total questions asked or total words
in questions, more is less. The more queries a given side received by
the justices, the less likely they were to emerge victorious. For
example, petitioners win about 62 percent of the cases before the court
but if the petitioner is asked fewer questions than the respondent,
that probability increases to 71 percent. On the other hand, if the
petitioner is asked more questions than the respondent, the probability
that the respondent will win increases from 38 to 50 percent.

There
are two models of judicial behavior, specifically, why judges bother to
ask questions at all, that the authors seek to fit this data into.
First is the "legalistic" paradigm, which posits that judges ask
questions in order to better understand the legal issues at stake and
figure out what the true legal right answer is. In this model, we
should expect more questions to be directed to the losing side, as
judges continue to probe unsatisfactory answers, creating a higher
total number of questions. Or alternatively, judges question the side
that they believe is weaker to begin with, in order to test their
intuition and give their attorney a chance to allay the judges
misgivings.

The second model is what the three call "realistic".
Here, judges are using questions to strategically influence their
colleagues (having already made up their mind themselves). Directing
questions to the party they want to see lose gives them the opportunity
to refute that side's best arguments and keep them on the defensive,
making them appear weaker than they might otherwise seem. Because
justices typically do not discuss cases before oral argument, and
because even conference discussions are highly constrained and
ritualized, questions at oral argument present a key opportunity to
sway other justices to one's position. One faculty member suggested a
novel way of testing this theory: breaking down the data into the eras
of the various Chief Justices, noting that Chief Justice Rehnquist was
notoriously more restrictive in what he allowed in conference than the
more free-wheeling Chief Justice Burger. Alternatively, Judge Posner
noted that there are substantial differences in the discussion norms
amongst the various appellate circuits; differences which could
manifest themselves in the questioning data if greater opportunity to
influence judges after the oral argument reduces the need to engage in
strategic questioning behavior.

The authors also looked at
how each Justice voted and the questions he asked over the 2004–2007
terms. Here they also found that the relative number of questions a
Justice asked the parties significantly predicted his vote. One
intriguing exception to the rule was perhaps the most important current
Justice on the court, the swing vote Justice Kennedy. He was the only
justice for whom the number of questions he asked the petitioner
relative to the respondent did not bear a statistically significant
correlation to the outcome of the case. Precisely because Justice
Kennedy often represents the crucial vote, the authors hypothesized, he
may be more conscientious about equally probing each side -- or he
simply might not want to tip his hand too far in advance.

Comments

You can follow this conversation by subscribing to the comment feed for this post.

For example, petitioners win about 62 percent of the cases before the court but if the petitioner is asked fewer questions than the respondent, that probability increases to 71 percent. On the other hand, if the petitioner is asked more questions than the respondent, the probability that the respondent will win increases from 38 to 50 percent.

If this correlation is meaningful (i.e. more questions from a PANEL, less likely to win), then specific JUDGES on a panel that ask a party more questions (relative to their usual number of questions and/or relative to other judges on the panel) should be more likely to vote against the party. Have these findings been validated by running this test?

I unfortunately don't have the standard deviations. I do know that with the exception of Justice Kennedy (for whom there is no correlation in any direction) and Justice Thomas (who, as Alvin indicates, doesn't ask enough questions for him to be measured), the findings do hold solid for the individual justices.