Colleges Work to Maintain an Information Deficit About Their Effectiveness

To graduate from college students do not have to demonstrate anything whatsoever beyond passing grades. Students do not, for example, take a standardized test administered to students from multiple universities, so their performance cannot be measured against a common benchmark. As Philip Greenspun points out, “There is literally no way that a university can be embarrassed by its graduates’ poor overall performance.”

Without accountability, colleges don’t have an incentive to actually succeed at teaching students basic critical thinking skills. And so a majority of college graduates enter society without them. According to Kevin Carey, in the journal Democracy: “A 2006 study from the American Institutes for Research found that only 31 percent of adults with bachelor’s degrees are proficient in ‘prose literacy’ — being able to compare and contrast two newspaper editorials, for example. More than a quarter have math skills so feeble that they can’t calculate the cost of ordering supplies from a catalogue.” Remember, this is 31% of the 25% of Americans who even have bachelor’s degrees.

Unfortunately, it’s exceedingly difficult to figure out which colleges are helping students and which aren’t. Right now there is no “objective, publicly available information about how well colleges teach and how much college students learn. Nobody knows which colleges really do the best job of taking the students they enroll and helping them learn over the course of four years.”

It’s not for lack of data. Colleges are now collecting rich data sets about how students learn, how much time they spend studying, how engaged they are in class, and how well they know certain concepts. There are dozens of studies conducted by outside organizations and government regulators about student learning and professor teaching — and yes they probe for things like critical thinking and creativity.

The reason there’s no access to the data is because colleges do not want it to be public.

One of the most powerful special interests lobbies that nobody’s ever heard of…is the American Council on Education (ACE), the National Association of Independent Colleges and Universities (NAICU), and a host of other alphabet-soup organizations conspire to maintain higher education secrecy at all costs.

For example:

in 2006, Mark Schneider, the commissioner of the Department of Education’s National Center for Education Statistics, proposed adding some new questions to the annual survey all colleges are required to fill out in exchange for federal funds. Colleges would be asked if they participated in surveys and tests like NSSE and the CLA. [Tests which measure teacher effectiveness and overall student learning.] If the college answered “yes,” and had already chosen to make the data public, it would be asked to provide a link to the appropriate Web address. It would not be required to participate in any test or survey not of its choosing, or disclose any new information. It would just have to tell people where to find the information it had already, voluntarily, disclosed. One Dupont Circle rose up in anger and the proposal was summarily squashed.

Why are colleges so eager to keep private the data? The obvious reason is because the data are embarrassing and everyone would prefer to be held less accountable. The more interesting reason is that the older colleges with better reputations dominate the lobbying effort and they benefit disproportionally from the “existing, information-starved reputation market.”

See, the lack of data about which colleges are doing a good job — whose graduates are succeeding, whose are not — means that the customer (high school seniors) suffer from an information deficit. As Carey explains, this turns college into a “reputational good”: “You’re paying up-front for professors you’ve never met and degree programs you probably haven’t even chosen yet. Instead, you rely on what other people think of the college. Of course, some students simply have to go the college that’s nearest to them or least expensive. But if you have the luxury of choosing, in all likelihood, you choose based on reputation.”

If there were clear data about which colleges were doing a good job and which were not, colleges could distinguish themselves based on how well students actually learned. This would give newer entrants into the higher ed market a better shot at competing. At present, even if they’re doing a good job teaching students, newer colleges must wait for reputation to catch up to reality. This can take generations. Wouldn’t it be better if there was a detailed database online showing every possible metric?

Bottom Line: The sorry truth is that “colleges remain indifferent to how well they help students learn, graduate, and succeed in the workplace.” And “like the church…they see themselves as occupying an exalted place in human society, for which they are owed deference and gratitude.” We should demand that all data around the effectiveness of colleges at teaching students be made public and easily searchable so that consumers of higher education can make more informed choices.

I sent this to my father, who works in commercial (undergrad) education and is always marveling at all of the people he knows who mysteriously have graduate degrees. They use the word “ideal” instead of “idea,” for example, or can’t figure out a simple 20% tip. It’s a topic we discuss frequently, and is one of those I will bring up if I want to enjoy a hilarious diatribe from dear old dad. He’ll appreciate your post.

Reply link does not work in your browser because JavaScript is disabled.Reply

As a math professor, I view the problem as the lack of focus on basic principles. Teachers must understand how students think, and build FROM THERE stressing the principles, using logic and empirical verification. Then go back, summarize and review the principles. See “Teaching and Helping Students Think and Do Better” on amazon.

Reply link does not work in your browser because JavaScript is disabled.Reply

Ben – you bring up some interesting thoughts here. When reading this, I immediately thought of the recently elected president of Detroit’s school board, Otis Mathis. He is a college graduate (who majored in Enligh) and can barely write a coherent sentence. Article here: link to detnews.com

Even more interesting, however, is that Wayne State University, where he earned his degree, used to make all students pass a literacy test before they would receive a diploma. And they were sued by students like Mr. Mathis, who claimed the test was unfair. In 2007 they stopped the test, no doubt because of pressure from failing students. So in this case we have a university that actually tries to make sure its students reach a minimal level of literacy, and those very students are the people who force its removal.

I wouldn’t necessarily blame universities for this mess. I would argue that a student should have to pass a basic literacy test before even entering college. I know entrance exams (ACT and SAT) are supposed to ensure this, but clearly they don’t always hit the mark. The students who would benefit from the studies you’re proposing already have basic literacy and math skills. It’s the students who enter college without these skills who need to catch up. And they should be caught up before graduating high school, not the other way around.

I’m not going to argue that standardized measurement of the success of schools’ teaching methods would be a bad thing. I do, however, think that blame also lies elsewhere.

Reply link does not work in your browser because JavaScript is disabled.Reply

While I agree with many of your points, I would just like to point out that not all colleges are set up to give all students a well rounded education. If a student goes to a technical school (or majors in a technical science), then they may not have to deal much with taking classes where they learn more about the language.

Even at “great” schools, there are enough avenues for students to take different classes where they may never have to take a Math class or an English class the whole time they are there. This is not necessarily a bad thing, because they can then focus more on what interests them.

With that said, I completely agree that some schools live off their reputation. I just think that the issue is more complicated than you make it seem.

Reply link does not work in your browser because JavaScript is disabled.Reply

This post highlights a key problem with the educational system in our country. I agree with everything here. We pay, or if we’re lucky our parents pay, a ridiculous amount of money for a piece of paper and have no guarantee that anything will come of it. Luckily, I am in a specialized program where progress is tracked and expectations are set in order to make sure students graduate with the tools to succeed in our field. However, I can only think of a handful of other classmates that I believe can be successful. A majority just seem to go through the motions and have no drive towards success. I feel that a major problem lies in the fact that professors are settling for mediocracy. Sure, I had teachers that taught me the basics and were helpful during my first 2.5 years at college. Sadly though, it seems that as I near graduation the dedication to providing quality education has diminished (Ex: I have a prof this term who literaly has not taught a single class period & just sits in class for about 20 min. while we work on a mktg plan and then leaves). Thats not to say there aren’t a few who are excellent, because I have some profs who are great. To me, Universities need to start researching this type of data from succesful students input. After all, at a place where you pay for higher learning, don’t you expect the educators to help you learn?

Reply link does not work in your browser because JavaScript is disabled.Reply

Agreed. Almost zero concern is given in class or out of class time is focused on professional life after time at the university in many humanities departments.

It also seems odd that professors generally get less than 8 hours of professional development and training (from my experience). Certainly, most of this is informal, however that means their knowledge base tends to be discipline specific rather than integrated with best practice in teaching and the science of learning.

Often, students have no idea what they need to do along the way until they get out. This is partially their fault, but some responsibility is required on the part of professors due to their role as guides, mentors, and counselors. Hopefully, as the sage on the stage model of teaching loses more credibility–professors will take up the slack.

A travesty indeed…Its unfortunate that professors and even folks like Obama seem to lack an understanding of these systemic issues in the halls of higher education.

Reply link does not work in your browser because JavaScript is disabled.Reply

Did that quote from Kevin Carey (“like the church…they see themselves as occupying an exalted place in human society, for which they are owed deference and gratitude”) really come via Ezra Klein, our very own John the Baptist crying in the wilderness to make smooth the way for Jay-Z’s Christ triumphant(I thought Tupac was the anointed one)?

I like the rest of the quote, from That Old College Lie, by Kevin Carey: “They cherish their priests and mysteries, and they are disinclined to subject either to public scrutiny.”

The ‘priests’ of the intelligentsia, our dons of Harvard Square, so to speak, are reduced to being servants of their holy endowment managers, the virtual maintenance custodians for Wall Street slicksters.

I’ll bet that in the ghetto the exaltation of a guy like Otis Mathis is seen as a corrective for the fact that “more and more low-income students are getting priced out of higher education altogether.”

I’ll also bet that if he were white, the default descriptive would be ‘dyslexic’ instead of ‘dysfunctional’.

Reply link does not work in your browser because JavaScript is disabled.Reply

If I were running a university that was actually educating it’s students, don’t you think I’d be shouting this from the bell tower? I’d be sending out recruiting materials that point to the facts you’ve laid out here, and then demonstrate what students actually learn here that they won’t learn elsewhere. Hell, you can still use the picture with the black guy, asian guy, indian guy and white guy in wheelchair, you’d just bump the ridiculous diversity article to the back.

Why isn’t anyone doing that? I suspect it’s because none of them can actually say that they educate their students, so instead they tout buzz words like diversity, which they know they’ll never get called out on.

Reply link does not work in your browser because JavaScript is disabled.Reply

Sure, but I would bet a lot of people would be swayed by the absence of that comparative data. If I ran an ad campaign where I point to UCLA and say, “They won’t even release their data on this subject. What does that tell you?” I bet it would be fairly effective. It would also be a way of instantly creating some reputation, especially if any media got wind of it.

The fact that no one is doing this indicates to me that they can’t credibly do so.

Reply link does not work in your browser because JavaScript is disabled.Reply

Law school in Germany works like this. You study only law for 5 years in undergrad, then all students of the same state take the same state-administered closed-book bar exam. The grade of this bar exam is the decisive factor for everything, your university grades are irrelevant/never appear anywhere.
Obviously, you cant cram all the knowledge of 5 years in, so the bar exam tests alot of critical thinking skills. (They are written 5 hour examinations where you have to solve cases; no multiple choice or general explanation of theories)

The ministry of justice releases detailed data every year, about the exact distribution of grades, failrate etc. of all the universities.
Also all universities of the same state have the same low tuition fee of 1000€/year – so basically, everyone would be free to choose the statistically best university, without financial constraints etc…

I would guess 95% of all students dont know that these statistics exist, and 0,000001% have ever considered it in their choice of university.

It is obviously better if such data exists, but from my experiences, it doesnt really seem to matter for making choices.

Reply link does not work in your browser because JavaScript is disabled.Reply

The Foundation for Individual Rights in Education grades all colleges on the speech codes and amount of censorship inflicted on the students. This ought to give prospective students an idea of which colleges are going to waste their time pushing a political agenda, rather than assisting them in educating themselves. Here’s a good summary on the Huffington Post blog:link to huffingtonpost.com

Reply link does not work in your browser because JavaScript is disabled.Reply

How does this square with the notion that some schools are good in certain department?. Staying away from the obvious MIT engineering example, Colorado State University is known for turning out exceptional veterinarians. How is such a reputation gained?

Reply link does not work in your browser because JavaScript is disabled.Reply

I think these kind of niche departmental reputations are easier to build
because the total constituent base is smaller and so the communication
between the people living in the reality and the people relying on the
reputation to hire future vets is "faster".

Reply link does not work in your browser because JavaScript is disabled.Reply

Great post. You have touched on a subject which has bothered me for years; the lack of accountability of colleges when it comes to educating their students.
A few of points:
1)There are countless discussions demanding accountability in our public schools systems. However, when it comes to colleges which charge 5 to 10 times the cost of educating students in the public schools, there is essentially no discussion of accountability or standards.
2)We buy a $30k car and expect it to have guarantees like a 100k mile engine warantee yet we shell out up to $200k for a college education with no guarantees whatsoever regarding the level of student achievement.
3)How would the attitude of colleges change if instead of students paying them, the college invested their own money in the students to train them with the expectation that the college would be entitled to a share of future earnings. This is essentially what the military does. This model certainly has its problems such as would colleges promote only certain types of programs in order to get the best return on their investment. In any event, you can bet your ass that in this kind of system colleges would not be so laissez-faire with respect to student achievement.
4)It is outrageous that colleges charge as much as they do for what amounts to a maximum of 7 months of the year. If you subtract time set aside for finals, at best you are getting 6 months of education. What a scam!

Reply link does not work in your browser because JavaScript is disabled.Reply

Ben, you are almost four years out of high school now, right? I’d love to see a post comparing/contrasting what you’ve gained in the last four years vs. the experiences of your college-graduating peers.

Reply link does not work in your browser because JavaScript is disabled.Reply

While I agree with many of your points, I would just like to point out that not all colleges are set up to give all students a well rounded education. If a student goes to a technical school (or majors in a technical science), then they may not have to deal much with taking classes where they learn more about the language.

College will only teach you how to do algebra, algorithm or ledger , but will never teach you how to do your own life math. You need to learn that after you are long gone from college thing. Most of the points here is so true.

Reply link does not work in your browser because JavaScript is disabled.Reply