A reporter for the Web site writes of recently attending a presentation by Catherine Watt, a professor at Clemson, at the annual forum of the Association for Institutional Research. At that time, Ms. Wyatt “laid bare in a way that is usually left to the imagination the steps that Clemson has (rather brazenly) taken since 2001 to move from 38th to 22nd,” the reporter, Doug Lederman, writes.

“This was no accident,” Mr. Lederman adds. “When President James F. Barker took over the South Carolina institution in 2001, he vowed in his initial interview to move Clemson into the top 20.”

So how did Clemson do it? As Mr. Lederman tells it, “the easiest moves” centered on reducing class sizes — many of them to below 20, a U.S. News benchmark. Clemson, according to the article, has also “more or less” stopped admitting “full-time, first-time undergraduates who are not in the top third of their high school classes” and is “constantly reassessing its SAT average through the admissions cycle.”

The article also quotes Ms. Watt as talking about the “more questionable aspects of what we’ve done.” Mr. Lederman writes that “the university has ratcheted up the faculty salaries it reports to U.S. News by about $20,000, which it has achieved by actually increasing spending (paid for largely through increased tuition.)”

This account reminded The Choice of an article by our colleague Greg Winter in 2003 about a similar leap in the rankings by Washington University in St. Louis (fundraising was key), and another by Alan Finder that includes a reference to a $10,000 bonus clause in a college president’s contract pegged to an increase in its U.S. News ranking.

Once you’ve read the Inside Higher Ed piece, please return to this post and use the comment box below to tell our readers what you think.

I think most people are aware that rankings have serious issues. But if Clemson decides to reduce class sizes and pay teachers more, I’m all for those changes. If they increase tuition charges because of it, Clemson is running the risk of not filling their incoming class. I don’t see how they’re “gaming the system,” but rather making good changes for possibly shallow motives.

As I began the article, I expected to hear about pure PR moves, window-dressing that upped the rating without any real change. What I read was about class size (something that is more important in some classes than others, but a real issue), quality of students admitted (believe me, faculty notice the difference as students with higher qualifications fill our classes), and faculty compensation. Just paying a professor better doesn’t make that professor a better teacher (or better at anything else), but it does allow recruitment and retention of more highly qualified faculty, and if the institution really regards teaching as being an important part of faculty assessment for promotion and tenure, that has a real impact on the education of the students.
As a professor at Washington University, cited as another university that has risen in the rankings, I know all of this from my own experience over the past 25 years.

I thought that the most telling quote from the article was this one: “And as those last few results show, Watt said, many of the changes Clemson has made have helped students.” In other words, the fact that student retention and graduation rates have increased are a secondary result of the emphasis on moving up in the rankings, rather than the reason the changes were initiated. So, the question becomes is it better to do the right the for the wrong reason than not to do it at all? If more students are succeeding at Clemson as a result of the institution’s no-holds-barred quest for better rankings, then it seems that the rankings really are serving a useful purpose after all.

This also adds serious doubts about Clemson and its mission. They have, via Catherine Watts’ presentation and subsequent media coverage, now put it all on the line for the public to digest and consider.

Will their strategy backfire at least somewhat, as peer institutions may now decide to give Clemson a lower score in the U.S. News category related to reputation? This peer assessment vote accounts for 25 percent of the total ranking. The 2009 peer assessment votes are likely to have been compiled already by U.S. News for this round, but time will tell for Clemson.

Exactly: the US News and World Report ratings encourage colleges and universities to game the system by keeping a certain percentage of classes under 20 and “balancing” them with giant lecture courses–the entire system is a spurious way to measure academic excellence!

CU’s mascot is the Tiger. The moniker “Cheetah” was attached in the 80’s when the football program had repeated NCAA violations. While this does not fall in that category, the system has been gamed. How do you have a better student body? Only allow better students. How do you have a high paid faculty? Pay the faculty more. Deflation is here and Education, along with Health care and government, which have watched as the rest of us have struggled, are about to get a taste of life on the 00’s

What’s that saying? Don’t hate the player, hate the game? Some top schools make no secret of favoring early decision applicants because that will increase their admissions yield, which will make their US News rankings better. Clemson’s not doing anything that is much different than any other top school.

What this should tell us is how useless the US News rankings are for undergraduate institutions. Most people already know that they’re pretty useless for graduate school, business schools, etc., but maybe now the secret will be out! I’m not against rankings or metrics, but the US News ones seem particularly easily manipulated.

There are certain metrics that are important in juding Universities, like class sizes and competitiveness. By making investments in attacting and retaining top faculty and attracting higher achieving students, Universities should be building their reputations and rising in the rankings.

My biggest problem with the US News rankings is the “reputation” metric, which essentially maintains the “Old Boys” club of elite Uniersities. Some of the newer, less established, smaller Universities get knocked down by this one metric. Take Rice University, a small, elite, private University in Houston, TX as an example. If one were to look at all metrics in the US News rankings except “reputation,” Rice would be in the Top 10. However, because it is small, located in Texas and has achieved notoriety only in recent years, the East Coast, Ivy League bias works against Rice in the final rankings.

The US News rankings methodology is flawed, and it makes strategic sense for universities to exploit the flaws to their advantage. There is nothing untoward about universities acting strategically.

A better methodology, if significantly more difficult and costly to compile, would strive to measure the extent to which a university improves the prospects of its students in the four to six years the student spends at the university. This should be measured directly and not indirectly. Grades and test scores attempt to measure quality of input (setting aside whether these are useful measures of quality). Class size ostensibly measures quality of experience, but it is an imprecise measure. Employment prospects, average salary, and graduate school admission rates of alumni measure outputs, but the measure is imprecise. More importantly, the outputs do not account for the quality of inputs.

Prospective students should be interested instead in the likelihood that a university will provide the most positive and transformative educational experience. If students at University A have lower average SAT scores and high school GPAs but the same employment prospects and graduate school admission rates as University B, then University A would appear to have a more positive impact on its students, and prospective students should prefer University A.

US News’ argument would be that they are only evaluating colleges on what the presidents agree is important (small class sizes, faculty salary/retention, strong achievement in high school, etc.). If these are good benchmarks that are actually being achieved, is that gaming? Or improving?
And here is why the ratings change every year (besides to help the USN bottom line). For those who can’t believe that Clemson could possibly be 22nd–change the rubric to more closely mimic the ones currently “below” that “should be” ranked above Clemson.
Witness CalTech “dropping” from #1 to #8 in one year. Was CalTech any worse? Not a bit.
Good for Clemson, showing the way for others to follow.

One reaction, which highlights a fundamental truth beyond this story: even if it a reasonable but arbitrary standard, what gets measured and rewarded is what gets done. Something to ponder as we try to re-structure our national healthcare and insurance system.

I like that US News seems to have full disclosure on how they calculate school rankings. One reaction to schools managing to the survey would be to make the criteria obscure, but that just add another layer for confusion and abuse. If, in reality, there is minimal difference between and class size of 19 and 21, then USN can adjust their ranking model to be smoother (rather than a threshold test). If sub-20 class size actually is important to student learning then by all means USN should emphasize it and schools should strive to be under that number. It may be that there are not sufficient valid distinctions to continue an ordinal ranking system; USN could shift to a graded scale to recognize overall comparability. But what fun is that? Numeric rank is what generates headlines and attention for USN, especially since they change every year (even if the schools themselves do not).

For those upset with different salary calculations, wake up. Benefits, retirement contributions, stipends, allowances, etc., all are forms of compensation, and legitimately counted differently depending the purpose, e.g., IRS for one. Remember the recent debates about what an auto-worker “earns”? Similar differences for many government employees where salary + benefits + retirement pay + etc. can let those ‘underpaid’ folks earn much more than their private industry peers. (Why do you think government employees don’t get social security and instead have their own special plans?) “Salary” may be very different than what one “earns”.

Overall I thought that Clemson’s full disclosure regarding the steps they took to improve their ranking was good policy. Except, sadly, for the ‘win at all costs’ elements such as ranking all other schools as below average. Cheating is cheating. That isn’t right and hopefully they get slapped for that. (USN could make avoid that problem by asking schools to rank each other – so sure you can pick yourself as number 1, but you still need to pick a number 2.)

Colleges have been playing this game for years. Two of their more despicable tricks are:

1, Soliciting applications from large numbers of bottom-of-the-barrel students, so they can reject them and raise their rejection rate or “selectivity” ranking,

2. Rejecting applicants who are highly qualified and would probably end up enrolling at some other (better) college. This practice, known as “strategic admissions,” enables the college to maintain a high “yield.” Yield is the term for the percentage of accepted students who actually enroll.

It’s simply amazing that a magazine like U.S. News, which barely exists, has control of the entire higher education industry.

That ranking system has created level of competition between colleges that unfairly treats applicants like game pieces. Every school makes a point to say that there is no one score or GPA or whatever that they are looking for — only to give false hope to would-be applicants so they can appear more selective when they turn them down because they don’t have the GPA or SAT or whatever that will boost the school in the ratings.

School rankings are a competition, with clear rules and a transparent system. In any competition, it is to be expected that competitors will use such rules to their advantage, even if it’s seems incongruous with the original intent of said rules. One can liken it to losing teams fouling down the stretch in a basketball game. The team bending the rule by committing the foul shouldn’t gain an advantage from it, but they do – in that they get the ball back. Reducing classes from 21 to 20 doesn’t mean much, but it does under the rules US News has established.

Of course, Clemson has a long history of bending a little too much – the football team has long pushed the envelope of NCAA recruiting violations and has been placed on probation more than once…

Well, sure, you can be cynical about the rankings game (and I am, to a certain degree), but some of the things that they did (in particular, decreasing class sizes and admitting students of a higher academic caliber) are undeniably linked to an increase in over academic quality.

The Clemson leaders’ behavior sounds a little similar to some parts of the “teaching to the test” theory. Since U.S. News has one of the most popular college ranking systems, they analyzed it step by step and took definite and specific actions, such as reducing class size and being more selective in admissions that would help Clemson achieve a higher rating.

Regarding some of the more questionable tactics, rating every institution except Clemson as below average should have raised a red flag for U.S. News & World Report. How bizarre is it to rate schools like the University of Chicago, Yale, Amherst, UC-Berkeley and the University of Virginia below average?

If the editors of U.S. News want to be more objective, they can hire independent, experienced higher education consultants to reevaluate their methodology and to determine which questions should be self-reporting.

The thing about having quantitative benchmarks is this: there will always be those who modify their behavior to fit them. Thus, the insights revealed by Clemson should come as no surprise. But what is the purpose of this disclosure? Self-promotion? Perhaps, but by openly acknowledging their strategy to be questionable in nature, perhaps this is a ploy by Clemson to criticize the entire U.S. News process. I mean, who can take seriously a ranking that can be so obviously gamed? I am struck that they took a purposeful stance to say “no” more often, meaning they can move up the rankings, thus making them more appealing and attracting more applicants to whom they can say “no” to so as to move even higher up the rankings. If tuition cashflow were not so vital, they could just admit nobody and be the number one school in the nation.