Sunday, June 18, 2006

Baylor University School of Law's score in the most recent U.S. News & World Report rankings poses a puzzle. As I noted in an earlier post, Baylor did remarkably better in the USN&WR rankings than it did in my model of them—6.29% better. No other school's scores varied so widely. Inspired by an email from Paul Varriale, I looked into the matter. I now think I can explain the Baylor mystery. The explanation teaches us something not only about that particular law school, but also about how USN&WR ranks all law schools.

As I explained earlier, my model of the USN&WR rankings relies in large part on the data that schools report to the American Bar Association. The questionnaire that USN&WR sends to the law schools it ranks asks them to repeat what they told the ABA. In particular, USN&WR asks schools to tell it what they told the ABA about the median LSATs and GPAs of their first year class. Most schools evidently hew to that request. As I noted earlier,though, some few schools seem to report different numbers to the ABA and USN&WR. Baylor appears to fall in that latter group.

Specifically, it appears that USN&WR ranked Baylor based not the median LSAT and GPA of that school's entire first year class, but rather solely of the first year law students it admitted last fall. What's the difference? For most schools, there would be none. Baylor, however, offers an unusual opportunity for some students to enroll in the spring or summer preceding the first fall of their law school education. Judging from the data published in the ABA's official guide to law schools, Baylor lets students with relatively weak LSATs and GPAs start early. It admits students with relatively strong LSATs and GPAs in the fall.

Why do I think that USN&WR ranks Baylor based on the median LSAT and GPA of only its fall admits? First, because the 25th and 75th percentile LSATs and GPAs that USN&WR reports for Baylor correspond to those of its fall class—not to those of its first-year class as a whole. (USN&WR does not report the median LSATs and GPAs of the law schools it ranks, even though that is the number it in fact uses in calculating the rankings. Crazy, huh? Note that one in the burgeoning "suggested reforms" list.) Second, when I plug into my model of the USN&WR rankings the median LSAT and GPA of Baylor's fall admits (as opposed to the median LSAT and GPA of Baylor's first year class as a whole--the numbers it reported to the ABA), it ends up with a score only 1.47% above that assigned to it by USN&WR.

Let's suppose that we have thereby solved the mystery of Baylor's unusually low score in my model of USN&WR's rankings. So what? Here's what: USN&WR's reliance on the median LSAT and GPA of Baylor's fall admits would appear to contradict prior practice. As the New York Times reported on July 31, 2005:

Baylor University School of Law in Waco, Tex., places about 100 Students whose LSAT and G.P.A. scores are generally lower into summer and spring programs. In the questionnaire in which U.S. News asks for year-round admissions, Baylor has submitted only its fall figures. This is because, says the associate dean, Leah Jackson, the magazine's published methodology had told readers that the reported data was for the fall entering class. (U.S. News has since fixed the mistake.) "In an attached addendum, we give them the spring, summer and the aggregate numbers," Ms. Jackson says. "What they do with those numbers is up to them." This year, U.S. News changed Baylor's data to reflect the year-round figures.

Robert J. Morse, director of data research for U.S. News, says: "In my view, Baylor was coming up with a lawyerly version of why they want to give us what we don't want and hope that we don't catch them."

I assumed from that story that USN&WR would henceforth rank Baylor based not solely on the median LSAT and GPA of its fall admits, but on those of its entire first year class. That does not appear to have happened, however. We thus face a residual mystery: Why not?

One of two answers looks likely. First, USN&WR may have knowingly ranked Baylor based solely on the numbers for its fall admits. In that event, we will have learned something very interesting about the methodology that USN&WR uses for its rankings. If they know they can win the same favorable treatment, other law schools may well find Baylor's approach to admissions worth emulating. Second, USN&WR may have unknowingly ranked Baylor based on the numbers that school submitted. In that event, we will have learned something about Baylor's reporting practices (consistently aggressive), about the rank that USN&WR meant to assign Baylor (around 58 rather than 51), and, most importantly, about USN&WR's fact-checking practices (insufficient).

Which answer is it? I've written to USN&WR and hope to report back here with an answer, soon.

[Please note that I've edited this post to reflect edits that I made to my model of the USN&WR rankings.]

8 comments:

Anonymous
said...

I may be mistaken, but I was under the impression that a large number of schools have alternative acceptance programs and part-time programs, but only submit their full-time Fall admittance statistics to U.S. News. If this is true, why wouldn't these other schools deviate from your model as Baylor does?

For example:

According to the ABA guide, over one-third of the Brooklyn Law students this year are part-time. The full-time median LSAT is 163, but the part-time median is 159, which moves the overall median to 161.

You raise an interesting point, Tom. But Baylor isn't the only law school to admit students at times other than the fall semester. I suspect other schools with spring or summer incoming classes probably report their fall LSAT and GPA information to USNWR because (1) that's when almost all of the best applicants are looking to enter law school, and Baylor is trying to demonstrate its ability to compete for the most attractive students, and (2) that's what USNWR asks for (as I understand it). Additionally, as the prior comment notes, USNWR scores schools with part-time programs based only on their full-time students.

In any event, it strikes me as a bit of a tempest in a teapot. If using Baylor's full-year data drops them from 51 to 58, what does that mean? Does the "gap" between 51 and 58 affect quality of instruction, bar passage rates, or placement -- three key factors to most prospective law students (who, after all, are -- or at least were -- USNWR's target audience for its various rankings)? I doubt it. Having grown up, attended college (at Baylor) and law school (not at Baylor), and practiced law in Texas, I know quite a few Baylor law graduates. Baylor regularly moved between the bottom of the Top 50 and the next tier down before USNWR began ranking to 100. That notwithstanding, practically everyone I know who went to Baylor law school was impressed with the quality of instruction, Baylor typically has had the highest pass rate on the Texas bar exam for years, and Baylor graduates disproportionately (given the size of its student body compared to those at other Texas law schools) populate the most prestigious law firms in Texas and surrounding states, elected and appointed offices in Texas state government (during Democratic and Republican administrations alike), and all levels of the Texas judiciary.

Anon 1: The ABA treats part-time programs differently from programs with staggered starts. Ditto the USN&WR questionnaire. Maybe they shouldn't, but that's neither here nor there. The issue is one of answering accurately the questions asked.

Anon 2: If you can name the other schools that admit students in the spring or summer, I'll look into what they reported, too. As I noted above, the ABA and USN&WR do not treat those sorts of programs the same way they treat part time programs.

As to whether or not Baylor's actual rank matters, that's a matter of taste. I tend to not accord as much weight to the rankings as many people. But those many other people--including students choosing law schools right this moment--might well care about whether or not they have accurate data about Baylor. And, of course, we should about the *reasons* for errors in the rankings, if such there be. Deliberate misrepresentations deserve different treatment from innocent errors.

Like Baylor (and, apparently, Cardozo), Stetson enrolls new students in fall, spring, and summer. Cal Western enrolls new students in fall and spring, as do South Texas College of Law, Washburn, and Whittier. All part-time students at U. of Houston enroll in the summer. I haven't done exhaustive research, so I imagine there are other schools that do not enroll new students exclusively in the fall semester.

Thanks, Anon 4. A quick look at my notes indicates that, like Cardozo, those additional schools do not appear to have reported to the ABA different median LSATs or GPAs from the schools reported to USN&WR.

It bears noting that any school that puts its early admits into a part time program would not--or, at least, should not--fold their stats into those of its full time program. It's only the full time students' median LSATs and GPAs that count for purposes of the ABA and USN&WR.

That suggests another possible solution to the Baylor puzzle: Their early admits are in a part time program, and thus not properly grouped with Baylor's fall admits. But that would leave us scratching our heads over why Baylor grouped all the students together when it reported to the ABA, and why USN&WR complained last year about Baylor's reporting practices.