HOW TO TELL IF A METRIC IS ARBITRARY

08/16/2014

The U.S. Department of Education’s proposed gainful employment regulation comes at a time when stakeholders all across higher education debate what the best measure of a program’s or a student’s success is. Is it earnings? Student ratings? Employer satisfaction with a program’s graduates? Passing state licensure tests?

Law schools offer a unique and straight forward window into evaluating success: do students graduate? If they sit for the bar exam, do they pass?

During the gainful employment debate, we have repeatedly cited the George Washington University Law School (GWU) as an example of a program that would fail the debt-to-earnings metric. Not because the law school is a poor performer, but because of the Department’s faulty and arbitrary metrics. We estimate GWU’s debt-to-earnings ratio to be around 13.8 percent, which would not be low enough to pass gainful employment.

It is difficult to assess public and nonprofit programs’ performance relative to the regulation, because the Department of Education has not released gainful employment data for them. However, results have shown that when it comes to law schools, GWU is not the exception, it is the norm.

An analysis performed by InfiLaw System – a consortium of private sector law schools – found that 70 of the 110 private non-profit law schools in the United States would fail the debt-to-earnings metric. Luckily for the students attending those institutions the regulation doesn’t apply to them.

Law School Programs By Debt-to-Earnings

But for students attending a private sector law school like Western State University College of Law – they may not be as lucky. With a debt to earnings ratio of 13.58 percent, the program fails to meet the Department’s arbitrary debt-to-earnings metric, but it passes the programmatic cohort default metric with a rate below 1 percent (.84 percent to be exact).

The difference is that Western State University College of Law is a private sector institution and must pass two arbitrary metrics in order for students to get federal financial aid and George Washington University and 69 other private non-profit institutions do not have to pass anything.

This is why the Department previously argued that the gainful employment metrics were “designed to work together” and that they had “no magic mirror through which it can identify programs that are not preparing their students for gainful employment” and there was no “perfect single test” for gainful employment. Yet they have now created a regulation that fails a program if they don’t meet the arbitrarily set level of either metric.

The questionable nature of the Department’s regulation is amplified when examining graduate performance on the state bar. As you can see below, private sector law schools, private non-profits, and public law schools can perform comparable to one another with respect to outcomes and bar passage rates, but receive completely different regulatory treatment by the Department of Education. So the question becomes, why does the Department discriminate against the tax status of a law school?

*Note: Debt-to-earnings data is higher than reported above due to differing methodologies. In order to match private sector data and nonprofit data for the sake of comparison, different sources were needed.