Wednesday, September 19, 2007

Leiter from the Other Side of the Fence

A further wrinkle (for those still paying attention) on the ongoing debate about Leiter rankings and whether someone like Erwin Chemerinsky can break the mold and create a great school from whole cloth:

One of the striking things about this conversation so far has been the discussion of rankings generally. There is, obviously, no consensus on how to rank schools. Various people have taken issue with Brian Leiter’s rankings, as well as the U.S. News and World Report rankings, and others. To a semi-outsider like me, however (a Canadian who’s spent years as a student, teacher, and practising lawyer in New York), what is striking is the near-unanimity about the notion of rankings. It’s worth pointing out that Americans live in a world of hierarchy to a degree that is exceptional, that they don’t always appreciate – and that is hard to square with the liberal democratic “American Dream” perspective that also characterizes the country. Perhaps, when everyone has a (theoretical) chance to be president and the country understands itself as a meritocracy, then things like rankings become all the more important as mechanisms for identifying the “meritorious.” And, if everyone in the country agrees that rankings actually signify something (even if it is only job prospects after graduation) then it makes sense that, in Mike’s words, the “ability curve” would be skewed to the right in the more highly ranked schools. Subject of course to individual exceptions and to disagreements about how to measure things, we assume the bulk of the “most talented” students, as measured by such things as high LSAT scores and strong GPAs at highly-ranked colleges, will take the “best” opportunities presented to them.

Now I’d like to point out how the same rankings make less sense in another jurisdiction. In general Brian Leiter’s ranking system makes sense to me for American schools, though I wouldn’t claim to be an expert on his methodology. This month, though, he turned his hand to Canadian schools and the results are entirely different. Full disclosure: I teach at a school that did not do well based on the Leiter metrics. However, I expect that most legal academics in Canada would find at least some of the Leiter results very surprising – if (and this is a large “if”) they were even prepared to accept that ranking law schools was a valid undertaking.

My law school attracts exceptionally strong students. Certainly, based on LSAT and GPA, our entering class this year puts the school in the top three in the country. Our faculty publishes extensively, and has particular strengths in interdisciplinary work and international work. We also live in what The Economist magazine (and again) and Mercer Human Resource Consulting have repeatedly called (more rankings!) one of the most liveable cities in the world.

However, Professor Leiter measures the quality of Canadian law students by where our students end up working. His methodology relies on factors such as “elite” law firm hiring, “national reach”, and clerkships with the Supreme Court of Canada to measure quality. Leiter’s survey is premised on the idea that hiring patterns reveal employer preferences based on the strength of students, rather than student preferences about careers and geography. However, law firms in Canada do not actually seem to prefer some schools to others for “elite” reasons, apart from perhaps at the margins. These measures are biased toward a view of what success means (e.g., working for certain firms, willingness to leave Vancouver) that does not work for our students. Their impressive entering credentials and national career opportunities notwithstanding, a substantial majority of our students choose to remain in Vancouver after law school. This is a choice that likely has more to to with lifestyle preferences, and less with prestige and the desire to work very hard on corporate deals on Bay Street (Canada’s equivalent of Wall Street) – or, indeed, on Wall Street. In fact, the competition is such that it reportedly takes a higher law school average to land a job at a large firm in Vancouver than in Toronto, which has a substantially larger legal market in every sense.

At the national level, Canadian law student career choices are also affected by the lower cost of education in Canada. Students attending the costliest American schools may graduate with debts running into six figures. Many of those have little choice but to accept well-paying associateships at elite law firms to pay off their student debts. By contrast, even the most expensive Canadian law schools still charge substantially lower fees, which means that graduating students’ career choices are less restricted.

Factors such as clerkships with the Supreme Court of Canada are also skewed by Canadian history and politics: by convention, three of our highest court’s nine judges should hail from Quebec. This means that students from Quebec, bilingual students, and/or students who have had exposure to both common and civil law systems have a strong edge in landing clerkships. Clerkships are also subject to geographic proportionality considerations that do not prevail in the United States. We can expect, for example, that some loosely-defined but fairly stable number of slots will go to western Canadian law students each year. The clerkships, if rotated through the various western Canadian law schools fairly equally (which, anecdotally, seems to happen), tend to give an edge to smaller law schools on a per capita student basis. My law school is the largest in western Canada.

(While I’m at it: on faculty quality – and leaving aside that the rankings divided our research productivity among 48 faculty members when in fact there are only 40 of us – Professor Leiter has measured faculty research productivity by focusing only on citations in certain indexed Canadian law journals. This measure of research productivity is biased against global law schools, like mine, where faculty members are actively involved in scholarship and research in Japan, Australia, the United States, and other countries. The measure also works against schools with faculty members who are committed to interdisciplinary research.)

I can’t say for certain that relative to any other Canadian school, my school has been disproportionately mistreated by the Leiter metrics. What I can say is that, more fundamentally, Leiter’s metrics may not translate well to Canada because Canadian schools, and Canadian students, are not in the thrall of hierarchy to the same degree as their American counterparts. The rankings would be more valid if we assumed that a substantial proportion of our law students behaved as though the factors measured by Professor Leiter matter, and took advantage of the most highly-ranked opportunities available to them. However, my experience suggests that this is not the case in Canada to the degree that it is in the United States. (The exception is probably the University of Toronto. U of T, along with being an excellent law school, is the one school in Canada that has made a conscious and energetic push to assess its performance on American-style standards and to place itself on par with the top American schools. Its students’ choices are likely to reflect the effect of that institutional culture.)

Maybe it’s because this is a smaller country – maybe it’s a cultural effect of the relatively greater historical impact of socialism in Canada – maybe geography continues to trump other factors in our fractured nation-state – or maybe it’s because the band of opportunities in Canada is narrower, there is very little “celebrity culture” for lawyers and legal academics, and the most successful among us are not so different from our “mediocre” colleagues. I could list myriad plausible suggestions for what amounts to a subtle but profound difference between the two countries. But the point is that Canadian law schools, like Canadian universities and Canadian law firms and Canadian society generally, seem to be substantially less focused on ranking and hierarchy. Why Americans take rankings so seriously is something worth thinking about.

15 comments:

Maybe Canadian law students don't differentiate much among the schools because they have not had good information about which schools are better? Certainly the behavior you describe isn't what is seen by Canadian grad students in philosophy who have had the benefit of Leiter's Philosophical Gourmet Report for many years. Perhaps once Canadian law students have access to good information on the relative quality of schools they will make use of it. (Or do you hold that there is no relative difference in the quality of the schools? That would be quite surprising.)

I think Americans are focused on rankings because of a need to establish what is the best. And we need to establish that because we believe that the better the school you attend (as defined by rankings), the better the education you will receive and the better the opportunities that will be available to you after you graduate. Or at least you will have more options.

The rankings allow prospective law students to make certain decisions about their career plans. Many law firms recruit only from certain law schools because they think they know what they are likely to get in terms of quality. If one's goal is a job at Skadden Arps it is probably best not to go to the law school at the University of Tulsa. However, if one's goal is to stay close to home and work for the DA's office in Norman, then Tulsa is probably a perfectly fine choice. This is assuming you can get into a "better" school to begin with.

It really just comes down to competition. If you're going to compete for a Supreme Court clerkship or a position at one of the major law firms, you are going to be at a disadvantage if your law school is considered second tier or is not well known in the community. Certain jobs require certain credentials.

Keeping in mind I don't believe the best lawyers come out of the best schools all the time, but a whole lot of other people do. And for those people, you have to have some way to judge what is best.

Oops! The deleted comment by Neil was actually a comment by me (Mike), which I'm posting directly below. It said it was from Neil because he had been using my computer before me and I forgot to log him out and log in as me. Okay, with proper attribution, my comment appears next.

I find myself in complete agreement with Sally, with the possible caveat that I think that the use by law firms, (Canadian and US) Supreme Court Justices and others of law school selectivity as a proxy for quality is statistically justified on average---although I agree with Sally that this nonetheless means that numerous graduates of less prestigious/selective schools end up as better lawyers than numerous graduates of more prestigious/selective schools. Firms and others are thus using school prestige/selectivity as a proxy for quality, which may or may not be warranted, depending on the search costs of finding the high end lawyers from the less prestigious schools. My own view is that these search costs are miniscule: You can just higher top grads of less prestigious schools and know they'll do excellent work. This was what the late Chief Justice Rehnquist did. Almost uniquely among his colleagues, he routinely hired from beyond top-10 institutions. But note that the Dorf/Rehnquist approach only works if you assume that high performance in (any) law school correlates strongly (though not perfectly) with good lawyering. From the various comments posted on my prior entries, it's not clear that all of my critics share that assumption.

You can just higher top grads of less prestigious schools and know they'll do excellent work.

Since law school grades are doled out on a curve, they are pretty meaningless without reference to the quality of the student body. Surely you have given A's on exams at your previous institution that would not have merited them at your current school.

[T]his nonetheless means that numerous graduates of less prestigious/selective schools end up as better lawyers than numerous graduates of more prestigious/selective schools. Firms and others are thus using school prestige/selectivity as a proxy for quality, which may or may not be warranted, depending on the search costs of finding the high end lawyers from the less prestigious schools.

But note that the Dorf/Rehnquist approach only works if you assume that high performance in (any) law school correlates strongly (though not perfectly) with good lawyering.

It depends what you mean by good lawyering. You can't look at grades alone. That would be ignoring the Moot Court champion with so-so grades who goes on to clerk for a state supreme court and then win lots of big cases in that state.