News and views about legal academia and the legal profession by Brian Leiter (University of Chicago) and Dan Filler (Drexel University)

July 31, 2013

Kyle McEntee, Derek Tokaz, and Law School Transparency: "Profits of Doom"

I was surprised to discover recently that Mr. Tokaz's primary extra-curricular activity--when not posing as a high-minded law school reformer--is running a website devoted to insulting, ridiculing, and defaming law professors, including as Prof. Diamond notes, one post calling for Diamond to be fired for having the temerity to disagree with Mr. Tokaz and other know-nothings about law schools. A class act. (Mr. Tokaz also uses the moniker BL1Y when posting his harassment and abuse of law faculty.)

UPDATE: In response to a reader query: you can reach Mr. Tokaz at derek.m.tokaz@gmail.com. He seems to respond to being called out on false statements of fact, which is one of his specialties.

July 30, 2013

Ten Million Dollar "unrestricted" gift to Northwestern Law

Reflections on "The Economic Value of a Law Degree" and the Response to It

It has been a curious two weeks in the legal academic blogosphere. Michael Simkovic, a law professor at Seton Hall (who, I learned, never went on the law teaching market, he was plucked out of practice at Davis Polk by a savvy hiring committee), and Frank McIntyre, a labor economist at the Rutgers Business School, produced a serious piece of empirical analysis of the economic outcomes for those earning a JD as opposed to those stopping with the BA. The result was hardly surprising: JDs do better, considerably better, at both the top and the bottom.

Most disappointing, however, were Brian Tamanaha's badly confused interventions. As someone who endorsed Tamanaha's book originally, I've been embarrassed by the arrival of serious research on the topic. Tamanaha's work, in the areas where I have expert knowledge, has always been a bit notorious for its confusions and theoretical overreaching, especially in its desire to make startling claims, the evidence and the arguments be damned. I had thought his book on law schools was different. (I suppose I should have been given pause by the way in which Tamanaha repeatedly tried to legitimize some of the most disgusting and deranged "scam" blogs out there, even posting encouraging comments on some of them.) In any case, it now turns out that Failing Law Schools is consistent with the rest of his work, and I was mistaken in commending it.

Well, maybe not wholly mistaken, since Tamanaha has scaled back his claims in the wake of the Simkovic & McIntyre analysis (also here--and see this reply). It's not law schools, per se, that are failing, but just Thomas Jefferson, Phoenix, Southwestern, and a few others he singles out for calumny. To be honest, I don't even know that they are failing, it really depends on whether the long-term economic outcomes for their graduates fall well below the 25th-percentile in the Simkovic & McIntyre study--Tamanaha has no evidence on that, nor do I. As Stephen Diamond (Santa Clara) aptly put it in responding to another tissue of confusions about the paper: "The challenge for an individual law student is to determine where they are likely to fall along the distribution. I don’t think the Simkovic/McIntyre paper was intended to be a calculator for prospective law students and so criticizing them for that issue is unfair. However, the paper does provide concrete evidence that such a distribution actually exists and that for most points on the distribution the present value of the earnings premium associated with a JD is positive."

Tamanaha's response to the Simkovic & McIntyre article has been, in short, a spectacular intellectual embarrassment and travesty: the conclusion now seems inescapable that he has no idea what he is talking about. (Naturally, the Cato Institute hacks thought his intellectual mess was brilliant!) A colleague at another top law school put it to me aptly in an e-mail last week:

Tamanaha is embarrassing himself terribly. His responses have the feel of someone desperate to defend a profitable franchise, and they are manifestly deficient on the merits. Simkovic and McIntyre have comported themselves like true scholars. The contrast is striking.

Tamanaha should apologize in public and simply admit that he was wholly out of his depth. I certainly apologize to anyone who took Failing Law Schools seriously based on my opinion. It has some good anecdotes, and some interesting history of law school regulation, but I should have been more cautious. (I still agree with Tamanaha that the ABA ought to lighten up on regulation, to allow more models of legal education--that point is quite independent of the Simkovic & McIntyre analysis.)

So here's the amazing fact: not a single meritorious criticism of the paper was voiced in the blogosphere despite nearly two weeks of intense discussion.* It has all been careless, in some cases idiotic, mispresentations; technical and conceptual confusions; or irrelevant ranting by ignoramuses. It all reminds me why, as I said long ago, blogs are bad for legal scholarship, since they provide a voice, sometimes--as with the ATL cesspool--a loud voice for people who literally have nothing to say because they have no relevant competence or skill.

Here's what I think we know and don't know after the first serious scholarly intervention in the debate about an economic value of a law degree:

1. The vast majority of those who got a JD over the last two decades are better off, financially, than similar individuals who stopped with the BA.

2. We don't know if other post-graduate degrees are more worthwhile, financially, than the JD; Simkovic & McIntyre did not attempt to address that systematically.

3. We don't know if the economic pattern for JD-holders will hold for the future, though Simkovic & McIntyre adduce some evidence that it will. [BL EDIT: rather good evidence, I will add, in light of a recent dishonest misreading]

And that, I think, is a fair summary of where we are. The critics of law schools, it turns out, simply do not understand the basic math behind valuation--net present value. They think law school is a bad value because they apparently don't know how to value anything that pays off over time.

We have had a massive downturn in applications to law schools the last three years, primarily as a result of ABA-induced better reporting of employment outcomes and New York Times coverage of the bad job market for new JDs. That's fine, some of these individuals may have made the correct decision by foregoing law school. But we've also had an hysterical cyber-reaction by miscreants and charlatans (like Campos), as well as by unemployed law school graduates desperate for someone to blame for the latest crisis of capitalism. Journalists have generally done a poor job sorting the wheat from the chaff, though there have been honorable exceptions (I think especially of Debra Weiss at the ABA journal).

*Perhaps I missed an intelligent response to the Simkovic & McIntyre paper buried somewhere in the bowels of cyberspace.

UPDATE: Predictably, Brian Tamanaha was not happy with the preceding, but I'll try to be succinct: (1) it is false that he and I have "skirmished on various subjects over the past fifteen or so years"; the one time I engaged his work was in the review of his book three years ago (anyone interested in the actual objections should read the review essay); (2) he did not "challenge" my "interpretation of the formalists and realists"--on the formalists, because I have no views on the historical figures denominated formalists, and on the realists, because he did not engage seriously with either mine or Fred Schauer's readings of the Realists in his book; (3) he wholly misrepresents the contents of David Rabban's recent book and also misrepresents, by implication, Rabban's opinion of Tamanaha's book, which was the same as mine (Rabban agrees with me that Tamanaha stated a prima facie plausible case against the attribution of "vulgar" formalism to 19th-century writers, but that does nothing to resolve the issue of what other kinds of formalism were common in the 19th-century--again see my review essay); (4) anyone who reviews Tamanaha's postings can see how utterly ludicrous it is for him now to pretend that he acknowledged the evidence that Simkovic & McIntyre adduced in support of the proposition that the current recession in the legal market is well within the parameters of past economic cycles in the legal profession; (5) I am agnostic on Tamanaha's motives, and do not think my correspondent meant "profitable enterprise" literally. I do think his work, in both legal theory and on legal education, exemplifies a pattern of careless scholarship, which is why I noted my one discussion of the former by way of evidence.

ANOTHER: Apt comments by Frank Pasquale, also a blogger at the Balkinization site.

Response: We would have to be off by
85 percent for our basic conclusion to be incorrect

In finance, valuation entails using historical data to establish a baseline scenario. This baseline is generally viewed as the center of a distribution of possible future outcomes. The baseline can be modified to construct upside and downside scenarios to get a sense of what could happen if the future is better or worse than the past. Scenario analysis can help understand how robust the findings are--that is, how much the future would need to deviate from the past to change the basic directional conclusion of the valuation analysis. For the extreme downside, this is sometimes called "break-even analysis."

We estimate the
present value of a law degree at the median as $610,000 as of the start of
law school. This figure is pre-tax and
pre-tuition, but includes opportunity costs and financing costs.

In other words,
some combination of the student and the federal government could pay up to
$610,000 for the law degree and break even. The government might contribute to the cost through debt forgiveness through Income Based Repayment, or through some other method.

As we note in the
paper, ABA data suggest that the typical tuition cost for law school, less
scholarships and grants, is roughly around $30,000 per year. Spread over 3 years, and assuming tuition
rises 6 percent per year nominal (i.e., at our discount rate), this comes to
$90,000 in present value terms as of the start of law school.

For law school to
cease to be a value-creating investment for the majority of law students, the present value of the lifetime earnings premium would have to fall to below $90,000—a drop of 85 percent.

At the “25th percentile” (more like the 15th because of regression to the median), toward the bottom of the distribution, the law school earnings premium is $350,000. Assuming tuition (less scholarships and grants) remains at $30,000, the 25th percentile premium would need to fall by 74 percent for a law degree to no longer be value-creating proposition toward the bottom of the distribution. At the mean, we’d have to be off by 91 percent.

July 28, 2013

Diamond wipes the floor with Tamanaha

Repetitive (and avoidable) mistakes

At the American Lawyer, Matt Leichter repeats many misrepresentations of our research that originally appeared in the tabloid Above the Law, even after Above the Law posted corrections and after we refuted many of these misrepresentations. He also refers to anonymous comments attacking our research from people who did not read it.

He erroneously claims that we "assume law school pays off equally for non-lawyers" when we in fact measure the earnings premium regardless of occupation. We do not assume anything.

Leicther's description of our take on BLS projections is lifted out of context, since we note that even BLS economists are skeptical of these sorts of projections.

He ignores the fact that we show not only current student loan default data, but also data that predates IBR. Law school student loan default rates were low even before IBR was available.

Steve Harper makes many of the same mistakes, and throws in a few disparaging remarks to boot.

Harper repeats Tamanaha's claim--which the Washington Post reported as false--that we only look at means and do not consider different points in the distribution. And he throws in a red herring about a bi-modal distribution.

Harper gets confused about present value and about the difference between medians and means, much like Campos and Tamanaha. Harper incorrectly reports that "a [law] degree returns at most a lifetime average of $687 a month" spread "over a 40-year career."

Harper gets confused about causal inference and controls for ability sorting and selection, and repeats erroneous claims from Paul Campos that the United States Census Bureau's Survey of Income and Program Participation does not constitute a representative sample. Harper throws in some new errors about the relationship between sample size and statistical significance.

Harper incorrectly claims that our findings of a premium depend on certain assumptions when--as we explicilty note in the paper--our findings are robust and do not depend on those assumptions. And he overlooks the data on which those assumptions are based.

Harper incorrectly claims that half of law graduates will remain "below the median income" even after they graduate. In fact, the median income for law graduates is 60 percent higher than the median income for similar bachelor's degree holders.

We've already responded to many of these same misrepresentations of our research from Above the Law, Brian Tamanaha, and Paul Campos. Simple fact-checking, either by reading the article or by checking our blog posts, could have prevented these errors.

Hopefully the editors at the American Lawyer will promptly post corrections and have a serious discussion with Mr. Leichter and Mr. Harper about the differences between critiquing research on the merits and misrepresenting the contents of that research--and impugning the integrity of its authors--in a nationally distributed publication.

July 26, 2013

Brian Tamanaha’s Straw Men (Part 3): We use better (and more) data than studies Tamanaha praised in his book

BT Claim 3:
16 years of data is not enough

“S&M’s bold
assertion that their 16-year study establishes valid ‘historical norms’ on law
degree earnings would be scoffed at by social scientists who take the notion of
‘historical norms’ seriously. That is more than enough time to confirm norms
governing the mating behavior of fruit flies, but 16 years is laughably
inadequate for predicting something as complex and subject to change as the
lifetime earnings of future law grads.”

Response Part 1: A fine idea for historical research

We will be
delighted to read the results of similar work on earnings premia carried back
into the distant past. We certainly are
not claiming to have uncovered hundreds of years of data on law school earnings
premia. But, ultimately, we are not
sure how valuable such a retrospective would be for today's graduate.

Response Part 2: Professor Tamanaha and other critics of law
school relied on—and praised—studies that use far less than 16 years of data

The literature has
numerous studies using smaller data sets than ours (citations available),
including several studies using only 3 years of data that were cited by
Professor Tamanaha in Chapter 11 (starting on page 137-38) of his recent
book, Failing
Law Schools. Professor Tamanaha cited these studieswithout
comment or criticism regarding the number of years of data used (although he
did criticize them on other grounds), so we find it odd that he views our study
as somehow deficient on this ground.

Professor Tamanaha
and other law school critics have cited and praised studies that were much less
rigorous and used much less data, including Herwig Schlunk’s “Mama’s Don’t Let
Your Babies Grow Up to Be Lawyers.”
Schlunk used a couple of years of data from Payscale.com, law.com, AbovetheLaw.com,
and other websites.

On page 217, note
18 of his book, Tamanaha called Schunk’s study “An excellent example of how [to
determine whether a law degree is a good investment] in economic terms.” Tamanaha also praised an article by Jim Chen
that used only starting salaries.

While we're happy
to admit that no study is perfect, if these studies have enough years of data for
Professor Tamanaha to cite in his book, then we struggle to understand his
objection to 16 years of data across age groups.

And we're now going
to take this opportunity to cite a personal favorite line from Professor
Tamanaha's post:

For a brief critique
of Professor Schlunk’s work, see the discount rate appendix of The
Economic Value of a Law Degree. A more thorough critique of Professors
Schlunk’s work—and Professor Tamanaha’s reliance on it—is contained in our book
review of Failing Law Schools,which will be posted on SSRN soon.

Reminder: Mike Simkovic is still guest-blogging...

...about his important paper with Frank McIntyre, and will continue blogging into next week. I may have a few items too, including some reflections on the remarkable display of cyber-stupidity their serious research has elicited.

UPDATE: As if on cue, we now have this from an adjunct law professor who has been riding the "don't go to law school" bandwagon for awhile. Amusingly, he refers to Campos as a "respected" academic. At least one adjunct thinks so.

Brian Tamanaha’s Straw Men (Part 2): Who's Cherry Picking?

Response:
Using more years of historical data is as likely to increase the
earnings premium as to reduce it

We have doubts
about the effect of more data, even if Professor Tamanaha does not.

Without seeing data
that would enable us to calculate earnings premiums, we can’t know for sure if
introducing more years of comparable data would increase our estimates of the
earnings premium or reduce it.

“I think you make far too much of the exclusion of the period from
1992-1995. Entry-level employment was similar to 1995-98 (as indicated by table
2 on page 9).

But this does not necessarily mean that the earnings premium was the
same or lower. One cannot form conclusions about all JD holders based solely on
entry-level employment numbers. As S&M's data suggests, the earnings
premium tends to be larger during recessions and their immediate aftermath and
the U.S. economy only began an economic recovery in late 1992.

Lastly, even if you are right about the earnings premium from
1992-1995, what about 1987-91 when the legal economy appeared to be quite
strong (as illustrated by the same chart referenced above)? Your suggestion to
look at a twenty year period excludes this time frame even though it might
offset the diminution in the earnings premium that would allegedly occur if
S&M considered 1992-95.”

There is nothing
magical about 1992. If good quality data
were available, why not go back to the 1980s or beyond? Stephen Diamond and
others make this point.

Our sample from
1996 to 2011 includes both good times and bad for law graduates and for the
overall economy, and in every part of the cycle, law graduates appear to earn
substantially more than similar individuals with only bachelor’s degrees.

This might be as
good a place as any to affirm that we certainly did not pick 1996 for any
nefarious purpose. Having worked with
the SIPP before and being aware of the change in design, we chose 1996 purely
because of the benefits we described here. Once again, should Professor Tamanaha or any other group
wish to use the publicly available SIPP data to extend the series farther back,
we'll be interested to see the results.