New York Law School's "Victory" Was a Defeat for Law Schools--And Why the ABA Transparency Rules Will Fail

Brian Tamanaha

When the fraud lawsuit against New York Law School was dismissed last week, the school proclaimed vindication. It was anything but that. What Judge Schweitzer ruled is that no reasonable consumer would have relied upon the obviously inflated employment percentages and salary numbers posted by the school.

New York Law School, for example, claimed that 92.6% of its 2008 graduates were employed within nine months of graduation, and the “midrange of full-time private sector salaries” was $71,250-$160,000 (representing that 75% of graduates in full time private sector jobs earned at least $71,250, and 25% earned at least $160,000). The judge observed that, for a low ranked law school like NYLS, numbers like these are improbably high.

It was the students’ fault, Judge Schweitzer held, for accepting these claims at face value, failing to conduct a more thorough investigation to uncover the true, undoubtedly much worse, job results for graduates. According to the judge, the law against fraud only protects “a reasonable consumer acting reasonably”—gullible fools are out of luck. The New York Daily Newsput Judge Schweitzer’s message bluntly: “YOU SHOULD have known your law degree was a dud.” (Is this holding really a victory for NYLS?)

Case dismissed!

No one should celebrate a judicial ruling that representations by a law school cannot be relied upon by reasonable people. Law schools, after all, are educational institutions charged with training competent and ethical lawyers. In the opinion of the judge, people should view law schools with the same skepticism they view used car dealers—beware—investigate—double check claims about mileage per gallon and blue book value.

While I have doubts about the soundness of the judge’s decision in this case, he is absolutely correct that prospective law students must not take at face value employment numbers advertised by law schools. The recent US News ranking confirms that, despite new ABA reporting rules, law schools nationwide continue to advertise unbelievably high employment numbers for graduates.

By all accounts, 2010 was the worst year so far in the most dismal market for legal employment in several decades (though 2011 might yet turn out to be worse). Only 64% of 2010 graduates obtained full time jobs as lawyers. Given the poor job market, and given the new reporting rules imposed by the ABA, one would have expected that the employment figures law schools reported for their 2010 graduates would be substantially lower than for 2009. That did not happen, however—numerous law schools continue to claim that over 90 percent of their 2010 graduates obtained employment.

Law schools did generally report lower employment numbers, but typically the reduction was small, only a few percentage points beneath their 2009 numbers. The 2009 numbers posted by law schools under the old rules, however, were themselves substantially inflated. Had the new ABA rules been effective in producing greater transparency, law schools across the board should have shown large drops.

A few striking anomalies will expose the depth and pervasiveness of the problem (additional anomalies are identified here and here).

Yale Law School, to its credit, reported a significant drop in employment, falling from 96.5% in 2009 to 91.8% in 2010. That sounds about right for a top school like Yale in a historically tough employment environment. What’s puzzling is that every other top 15 school reports a higher employment rate than Yale, despite Yale’s two major advantages: It is ranked #1, with the most outstanding student credentials, and the size of its graduating class is relatively small, making it easier to place most everyone.

Incredibly, a number of law schools ranked far beneath Yale reported a notably higher employment rate, including George Mason (96.4%), Loyola Marymount (94.1%), Kentucky (94.2%), and UNLV (93.2%). Even a few bottom schools reported employment on a par with Yale: Florida International (90.1%), Baltimore (91.2%), Akron (91.8%), Toledo (90.1%), and Atlanta’s John Marshall (91.6%). It is absurd to think that any of these schools exceeded or matched Yale’s employment rate.

Washington University (my own institution), even more so than Yale, reported a drastically reduced employment rate, falling from 95.5% in 2009 to 80.7% in 2010. (Ouch!) Our 80% employment rate is unusually low among top 25 law schools—not until the 76th ranked law school is a lower employment number reported. Indeed, a significant number of bottom 100 law schools (including New York Law School) report a higher employment percentage. Wash. U. paid a severe price in the ranking for reporting such a poor employment number, falling from 18th to 23rd, despite improving in other measures. Dean Syverud, the incoming Chair of the ABA Section on Legal Education, was determined to scrupulously comply with the new ABA reporting standards, knowing that the school would likely suffer as a result. Understandably, a number of students vocally questioned the wisdom of his decision, given the resulting fall in ranking.

A few ranking winners exhibited anomalies in the opposite direction. The two biggest climbers in the first tier were the University of Washington (up from 30 to 20) and Arizona State University (up from 40 to 27). Both schools somehow defied the legal recession and reported significant leaps in their employment rate. UW went from 89.5% in 2009 to 96% in 2010; ASU went from 89.8% in 2009 to 98.2% in 2010. (Both schools put Yale to shame--or perhaps the opposite.)

ASU’s feat is especially curious because its (usually) closely ranked peer school, Arizona (ranked 42 last year and 43 this), has similar scores on most measures except for employment. In a year when ASU reported a large jump in employment, claiming the highest rate in the entire country, Arizona’s employment rate dropped from 89.4% to 87.4% (a couple of percentage points down like most schools). This is an odd disparity given that they are even-handed competitors in the same legal market. These results are even more peculiar when one considers that Arizona’s bar pass rate (93.7%) was much higher than ASU’s (85.9%).

Looking at bar pass rates exposes another set of strange findings. At most law schools the bar pass rate is about the same as or higher than the employment rate—that makes sense because a graduate cannot hold a job as a lawyer without passing the bar. However, eighteen law schools in the top 100 report employment rates at least 10% above their bar pass rate. At the high-rising University of Washington, for example, 96% were reported as employed, but only 85% of the class passed the bar. ASU likewise had a much higher employment rate than bar pass rate (in contrast to Arizona).

Here is a list of the top 100 schools with the largest gap between their reported employment rates and their bar pass rates: San Diego (employment percentage is 23% higher than bar pass, 88.2% to 65%), UNLV (20%), West Virginia (20%), Washington and Lee (18%), LSU (15%), Hofstra (15%), Pacific (15%), George Mason (13%), Loyola Marymount (13%), Davis (12%), ASU (12%), University of Washington (11%), Seattle (11%), Syracuse (11%), UCLA (10%), Tulane (10%), Lewis & Clark (10%), and Kentucky (10%). The explanation for this presumably is that lots of their graduates are employed in non-lawyer jobs (it is also possible, but unlikely, that lots of their graduates did not take the bar).

Inexplicably, several schools list students in “JD required” jobs at a percentage higher than the school’s bar pass rate. At San Diego, for example, 76% of the class had “JD required” jobs (25% of these jobs were part time), but only 65% of the class passed the bar. At Loyola Marymount, 84% were in “JD required” jobs, but only 80.9% passed the bar. At ASU, 89.3% were in “JD required” jobs, although only 85.9% passed the bar. It’s not clear how the number of students in “JD required” jobs can exceed the number of students who passed the bar, since the latter is necessary for the former. (The employment figures are taken 9 months after graduation, subsequent to the bar results.)

When a law school lists an employment rate that is substantially higher than its bar pass rate, a significant percentage of graduates typically are counted by the school in questionable categories, mainly “academic” and “business.” The combined total of these two categories accounts for about 30% of the claimed employment for Syracuse and Hofstra; about 25% of employment at George Mason, San Diego, and West Virginia; about 20% at Loyola Marymount and Tulane. That’s a lot of academics and business folks coming out of law schools.

And we must not forget the mushy “JD preferred” category that many law schools stuff significant proportions of their graduating class into. Lots of law schools claim 10% to 20% of their graduates land in “JD preferred” jobs. I guess that’s a bunch of corporate compliance officers and FBI agents.

Imagine Judge Schweitzer learning about these numbers put out recently by law schools, and shaking his head, thinking that any prospective student who believes any of this is an unsalvageable fool.

It bears emphasizing that the ABA has already implemented a stricter set of reporting requirements—and the above results occurred under the new ABA regime.

The fundamental problem here is that law schools can technically tell the “truth” even when they report absurdly high employment percentages. The categories themselves—“academic,” “business,” “JD preferred,” “employed job unknown”—beg for abuse. They guarantee the situation will not improve.

The numbers this year bear a telling resemblance to US News employment numbers in the late 1990s and early 2000s, when certain law schools gamed more aggressively than others and the techniques used to boost scores were beginning to spread. By the late 2000s, when gaming had become pervasive, nearly all of the top 100 schools reported employment rates from the mid-nineties to 100%, reaching an equilibrium in which every school reported high employment. As a consequence, no school could gain a comparative advantage from employment numbers, which mostly washed out as a factor among peer schools.

To offer one illustration, for the class of 2008, ASU reported 99.7% employed and Arizona reported 97.3%—this is in stark contrast to the large gap that separates them this year, at 98.2% and 87.4%, respectively. This disparity gave ASU a huge ranking boost compared to its more modest neighboring school. When one digs into the underlying numbers, however, the disparity between them begins to dissipate (11.3% of ASU’s JD jobs are part time, compared to 1.7% at Arizona; ASU’s combined “academic” and “business” jobs constitute 17.7% of overall employment, while Arizona’s is 12.3%).

To regain its traditional parity with ASU, Arizona will be sorely tempted next year to more liberally categorize future graduates as employed in “academic” or “business” positions, or will use some other expedient to raise its employment rate to approximate ASU’s. ASU’s aggressive gaming thus puts pressure on Arizona to aggressively game.

Project this scenario nationwide and it becomes evident that advertised employment rates will once again creep up until they stabilize at a high equilibrium, with virtually all law schools misrepresenting their employment results to one degree or another.

The “weakest link” problem in game theory is a situation in which the worst actor(s) produce negative consequences that affect everyone. The law school ranking competition is a “weakest link” situation in the sense that, as long as aggressive massagers exist among law schools, there will be immense pressure on every law school to engage in aggressive massaging just to keep up. Good behavior under these circumstances will be punished in the ranking (and honorable deans will be fired)—and dubious behavior will be rewarded (and strategic deans will get raises). That is why the ABA reforms will inevitably fail.

There is only one possible solution: Only full-time “JD required” jobs should be counted and advertised by law schools. Any other category is susceptible to manipulation and will be exploited by law schools to conceal poor employment results. Current claims that the ABA reporting rules will improve transparency because they provide more “granular data” are wrong—they will produce more obfuscation by law schools.

If only full-time “JD required” jobs can be reported, law schools will be stripped bare and prospective students will finally get a clear look at their real job prospects. Law schools undoubtedly will vehemently oppose this proposal, calling it unfair, failing to credit them for all the great non-lawyer jobs their graduates are getting. If you find that argument persuasive, I have a used car I would like to sell you—it has terrific resale value and gets great gas mileage.

I just looked at two top fifteen schools, Harvard and Stanford. Harvard reports a drop in employment in 2010 over the previous years. Stanford seems to have an increase, but the school is smaller than Yale. The number would seem to be within a normal variation.

Brian Tamanaha wrote: "It’s not clear how the number of students in “JD required” jobs can exceed the number of students who passed the bar, since the latter is necessary for the former."

It used to be that law firms would allow recent graduates at least two tries to pass the bar before letting them go. For example, I worked in the US practice of a large London law firm. One of my colleagues failed the bar exam on her first try in July but passed in on her second in Feb.(and so kept her job). U.S. News only considers the passage rate for "first time" bar exam takers when doing their calculations. So someone like my former colleague would be counted as employed in a JD required job but also someone who failed the bar exam on her first try. If there are a lot of similarly situated graduates, then the bar passage rate could be lower than the number of graduates in jobs that require a JD.

jpk's comment reminds me of a joke- what's the difference between Phillip Morris and a law school? One produces a toxic product that is harmful to society and the other one . . . well, you get the drift.

"It’s not clear how the number of students in “JD required” jobs can exceed the number of students who passed the bar, since the latter is necessary for the former."

Since when is it required to pass the bar to gain a JD? I was under the impression that graduating from an accredited law school was sufficient to earn a JD. Taking the bar comes after graduation and after the JD is earned. A company could conceivably require an employee to have a JD regardless of whether he has passed the bar. The statistic may be unusual, but not inexplicable.

the problem for schools like Wash U is that they continue to hold themselves out as a "national" law school; one that can regularly place students at BigLaw firms. and that is where the recession has hit hardest. the mid level schools, whose employment rates brian finds so puzzling, may actually be placing grads in government or small firm jobs, just as they have always done. eventually the impact will trickle down to these schools as well, but i don't think its hard to see why wash u would suffer disproportionately as its grads initially fail to come to terms with the changing BigLaw reality.

and, just as side note, some of the schools brian whose numbers brian implicitly calls into question, have received an A+ transparency grade from the national jurist based on data collected by the law school transparency project. wash u didn't make the top 25...

Your explanation is not supported by the data, which shows that employment in "JD required" jobs falls off in sync with ranking. You can see the details here: http://balkin.blogspot.com/2011/09/sobering-numbers-law-graduates-who-do.html

In other words, contrary to your assertion, the impact of the recession has already "trickled down" to mid ranked law schools, and did so at least three years ago. More to the point, schools have been having trouble placing their graduates for a decade, at least, since about a third of law graduates have not found lawyer jobs during this period.

Nor does your account explain how schools like George Mason, UNLV, and those other mid and low ranked law schools placed better than or equal to Yale. Nothing you say about Wash. U. addresses that implausible result.

On the issue of "transparency"--in my view the transparency grading system used by LST is fine and schools should be criticized for not posting results (including my own). But you missed the point of my post. The numbers themselves are often dubious, so schools that post (in the name of transparency) them are merely advertising dubious numbers.

Of course, I do not know whether any particular school is fudging. My post merely points out anomalies which raise the very questions Judge Schweitzer said should put people on notice that something is fishy about the numbers that requires further investigation.

If your assertion is that your law school's numbers are reliable (and I assume you verified this), then good for your school. My point is that there are plenty of oddities in the employment numbers out there than cannot be easily explained away.

I am only pointing out that those who actually look at these numbers seriously--rather than just speculate about causation and correlation based on how Yale should place relative to other schools--seem to think that several lower schools are being very transparent about their placement numbers. If you are implying that these numbers are fabricated, I think you have to do more than just point to how Yale places and say everyone else should do worse--that is, if you're interested in more than innuendo.

This is especially true when there are numerous alternative explanations out there. Despite your "data" point (which I confess I fail to understand, may just be me), I did address how some mid level schools continue to get "Yale comparable" job numbers. They are filling local employment positions, some in cities where the legal job market is actually quite good (due to recession created legal jobs like foreclosures etc...) Higher ranked schools may still be graduating those with "greater" aspirations, or may be trying to serve legal markets with more law schools, or whose legal markets suffered more from the recession.

I don't have any "data" to support this, I am simply positing an alternative explanation for the numbers you seem to find so suspicious. In the interest of some charity you might stop to think about such alternatives before jumping straight to an implication of bad behavior/lying on the part of others. I simply fail to see how the numbers you have pointed to justify the kind of indictments you've made. But, again, maybe that's just me...

I am for the principle of charity, but at some point you must question what is going on. Let's focus on a number provided by NALP based upon data it gathered: 64% of 2010 graduates obtained full time lawyer jobs.

If you compare this with the posted employment rates, many in the 90% range, there is ample reason to wonder if something "funny" is going on.

Your suppositions about local markets as the explanation for anomalies cannot be the answer to the huge employment advantage claimed by ASU over Arizona--in the same market!--when they are equivalent and the latter had a much higher bar pass rate. And don't ignore the fact that, as I revealed (based upon information), 11% of ASU's jobs are part time compared to less than 2% for Arizona.

You argue that law schools deserve charity--I think (given what has been going on) prospective students deserve employment numbers that are not misleading.

For example, your defense of the integrity of law schools cannot ignore the well known fact that folks employed as "academics" are graduates temporarily put on the payroll (typically at low pay in part time positions) by law schools to help boost their employment rate. This is only one technique among others we have used.

If my questions offend law schools (or you), that is regrettable, and when we get this sorry episode behind us it won't be necessary.

Meanwhile, let me again remind you of the judge's decision: numbers that seem implausible cannot be relied upon and require further investigation. You appear to be asserting that there is no basis to doubt these numbers and we should take law schools at their word until I have "direct evidence" of fudging. How am I supposed to get this (short of an inside whistle-blower like what happened at Illinois)?

If that is your position, then law schools cannot be challenged at all about these numbers. Perhaps that is your position, but I cannot agree.

I am not trying to say that no school has ever fudged its numbers. I am simply saying that the data you point to is not terribly persuasive on that count. There are, as I have said, a number of alternative explanations, even in the case of Ariz and ASU (they can still have grads who seek different jobs/markets, class sizes can vary, etc). Although you are certainly right that we should be vigilant and suspicious moving forward, I'm not sure that pointing to other witches is a good way to go at this moment in time.

As to the "offense" I have taken, I hope you can understand that I don't like being called a liar (even by association)--particularly based on the kind of evidence you've given. If I come to believe that those whom I associate with HAVE lied, you can believe my indignation and anger with them will be far greater. At this point, I simply have no reason to believe that, and those external groups that have studied our numbers seem to agree.

And one last thing, I admittedly did not read Judge Schweitzer's opinion as carefully as you did, but I didn't remember it quite the same. I don't recall him saying that NYLS had blatantly lied about its numbers, and that only a gullible candidate wouldn't recognize the fraud. I thought the conclusion was (1) that NYLS did not present the information in a very transparent way--particularly regarding the percentages of salaries surveyed--and the plaintiffs should have been more active sifting the data; and (2) that the job market changed dramatically after the plaintiffs enrolled, so that the earlier numbers might have been accurate but should not have been taken as a promise of future performance. Again, maybe I skipped the parts I didn't like, but I didn't see the opinion as quite such a condemnation of the academy.

And I do apologize for being snotty about Wash U in the first comment, I think I just reacted to your seeming to hold out your institution out as an honest victim in all this...

"I don't like being called a liar (even by association)--particularly based on the kind of evidence you've given. If I come to believe that those whom I associate with HAVE lied, you can believe my indignation and anger with them will be far greater. At this point, I simply have no reason to believe that, and those external groups that have studied our numbers seem to agree."

Is essentially a complaint about Prof. Tamamha's pointing up of UNLV employment data. Bartrum's argument is that the surprising performance of schools such as his could be explained by some sort of local factors in hiring. However, it would be considerably more convincing if the local factors led on to think that a law school in Las Vegas is somehow in a "hot market." But, other than the actual climate, the economic facts about Las Vegas would lead on to expect a disastrous 4-5 years for UNLV. For the last few years Las Vegas has one of the worst performing metropolitan economies in any modern country - at the peak of the recession Las Vegas was ranked only ahead of Athen's Greece (not Georgia)

To be blunt, if anything the local factors in Las Vegas would lead to a strong inference that UNLV should have some of the worst employment data of any law school - but it's doing better than Yale. The mind boggles......

What is bad for a "metropolitan economy" may actually be good for legal jobs; someone has to clean up the mess. And if you can point to any credible evidence that UNLV grads are competing directly with Yale grads for jobs I'd be interested to see it. We have very few in our market, and very few of our grads head to NYC. I hope that helps unboggle your mind.

We are all being called "liars" (directly and by association). Read Above the Law (not to mention the New York Times). Remember Villanova and Illinois--and there are many other examples (which I detail in my forthcoming book). Our belated push for transparency, and the new ABA rules, is an effort to regain our lost credibility.

By raising the questions I do in this post, I am only making explicit what many are already saying about these numbers.

Like you, I do not enjoy being called a liar (or manipulator, or gamer, or fudger). And my proposal--eliminate everything but "JD required"--is designed to regain our collective credibility by eliminating gaming opportunities.

If we are to solve this problem, it is essential for individual faculty members to take responsibility for the numbers posted by law school administrations (in our name). If we don't like being snickered at by others for seemingly inflated numbers, we can check ourselves to verify what our own schools post. Up until now, most law professors (including me) have ignored or avoided this, asserting that it is not our responsibility.

We can't have it both ways--saying it's not our responsibility, then getting mad when people point out that our numbers seem dubious. (I'm not now talking about UNLV in particular, as I accept your assertion that your school's numbers are legitimate.)

If we don't want to be seen as liars (individually and collectively), we must make an effort to police ourselves. It's too bad that law schools are in this position, but we brought it on ourselves.

Law teaching was an honorable profession when I started 17 years ago and it was seen as such by the general public--I fervently hope we are able to regain that. That is why I am writing these posts, fully aware that they create discomfort and unhappiness (and outrage) among fellow law professors.

On the good side, I also receive many supportive emails from law professors. Many of us dislike being seen as liars and those that do want this situation to get cleaned.

"Here is a list of the top 100 schools with the largest gap between their reported employment rates and their bar pass rates:..."Following the colon are the names of 18 schools. Maybe the author should mast basic verbal expression before getting too critical of so many educational institutions.

My proposal is that only one statistic should be reported by law schools (and counted in the ranking): What percentage of the graduating class obtained full time "JD required" jobs. This basically means what percentage of the class landed jobs as lawyers (and judicial law clerks).

At some law schools this will only be 30% of the class--at other law schools 90%. This number will give prospective students a good sense of the strength of their credential in the job market (even if they might not want a lawyer job).

All other employment categories ("academic," "business,"...) should not be reported or counted.

These categories are where the gaming comes in. Some of these are real jobs, but others are not. And we cannot tell the difference from the outside.

Thanks for the answer. Is this a change in view? I seem to remember an earlier exchange in which I suggested that schools should not have to post the kind of information that they do now. If I remember correctly you disagreed, saying that the expectations for reporting were such that that was not a viable idea. I could be wrong in my memory of this.

In any event, your idea is intriguing. The potential for gaming is a problem, but it takes two to tango. It seems clear that this information is being taken, or people are encouraging people to take, this consumer information as a form of a contract. Students often see what they want to see. If 160k appears anywhere on the page, it will be taken as the salary the student wants to have and should have if they want it. There is no way to fight against that. Your idea may be a good one to combat this tendency. Good luck after the sustained effort by LST and others to demand information about everything, as if that were the answer.

My mind remains firmly boggled. What I see is a UNLV professor trying to come up with unsupported explanations of how UNLV could somehow be turning in astonishingly successful job placement statistics for first year lawyers, when it is not by any stretch of the imagination a highly regarded national law school. Moreover, Las Vegas is the epicenter of the recession and unless you are suggesting that the UNLV graduates were hired by foreclosure mills, the job claims have to be treated with considerable incredulity.

I am sure that you regularly suggest that your students "think like a lawyer." I am a lawyer, a litigator, who for 20+ has challenged economic evidence regularly, almost always with success. An economic "expert" making the argument that the could be some "fairy in the sky" reason why their law school is that special little place, even though every observable factor leads to an opposite conclusion would in every forum I have seen be a "greasy spot on the pavement" when it was over.

I do not think you see how ludicrous your arguments are. But I do think you need to consider that as a lawyer and as an educator you have ethical obligations - you should see yourself as having a fiduciary duty to your students - one that demands the highest level of honesty - and that includes not making or supporting unsubstantiable claims and making a reasonable enquiry as to their substance.

Frankly, what your posting shows is a sort of foetal crouch - a member of the UNLV faculty muttering "say it ain't so" "maybe there is pie in the sky" maybe there is some bizarre and unlikely plot twist. Ask yourself - when you went to law school, young and hopefully idealistic, did you ever see yourself postulating what you have done for several postings here. Professor Tamanaha has raised some inescapable issues about UNLV employment data - you have responded with "precious" cries of wounded innocence, absurd and unsupported suggestions that UNLV may be a "special snowflake", but no hard questions for the institution you are connected with.

I have not studied the numbers and have no bottom line conclusions, but I would like to note that one factual assumption Brian makes, in his comparison of the U of A and ASU, is probably incorrect. That is the two schools operate in the same employment market. ASU is in the Phoenix metropolitan area, with a population of over 4 million, and the U of A is in Tucson, with a population of about a 500,000. Phoenix is the state capital as well as the business and financial center; Tucson has the university and some military installations. Many students at ASU develop relationships with potential employers while in law school. It is not at all implausible to think their proximity to where the local jobs are gives them an advantage in securing those jobs.

Brian suggests that Arizona will be “sorely tempted” next year to resort to some “…expedient to raise its employment to approximate ASU’s. ASU’s aggressive gaming thus puts pressure on Arizona to aggressively game. ”

For the record, we have not succumbed to that pressure or changed in any fashion the way in which we report our employment data. Our employment statistics for the Class of 2011 can be found here. http://www.law.arizona.edu/Admissions/documents/2011piecharts2.pdf.

Lawrence Ponoroff, Dean, University of Arizona James E. Rogers College of Law