Just as the online learning revolution is taking off, hurtling into the infinite cloud, the world of higher education is struggling to better measure learning. That struggle is all the more urgent as college costs soar and ambitious new providers promise digital delivery that may upend traditional college education.

How should college credentialing take place in this new world of online learning? That was the question addressed at a recent forum sponsored by Education's Digital Future (EDF), an initiative of the Graduate School of Education (GSE).

Accreditation is the process by which colleges and universities are judged fit to confer diplomas and receive government subsidies. Many critics of the system assert that the accreditation process has relied on what Mitchell Stevens, co-convener of Education's Digital Future (EDF) and an associate professor of education, called a gentleman's agreement among government, schools and accreditation agencies. That worked just fine until very recently when, as Stevens noted at the forum, the political economy underlying the entire enterprise of U.S. higher education shifted. Tuition is going through the roof at many public institutions and at private universities that don't have large endowments. There is a great, unmet demand as poor, working-class and middle-class students who seek a college degree view it as financially out of reach. All of this is taking place as state financing for higher education is declining. The new kid on the block – online learning – is threatening to complicate things even further.

So, to use one of the accreditors' own favored expressions, what are the measurable outcomes? What is the role of the university, and how can we reliably know if it is fulfilling it?

Many experts have raised the alarm. Among them is Richard Arum, co-author (with Josipa Roksa) of the widely reviewed Academically Adrift: Limited Learning on College Campuses, which argues that most college students are not learning much of anything these days. The authors say there is an urgent need for institutional reform and verifiable accountability. Though many are skeptical of Arum's approach, there is widespread agreement that the American education system is in many ways broken. It is under-performing and is not going to get better without serious pressure from outside the academy.

Arum, a professor of sociology at New York University, was one of the participants at the forum, which also featured Emily Goligoski, of the Mozilla Foundation's Open Badges project; John Katzman, founder of the Noodle Education search engine and Princeton Review; and Therese Cannon, former executive vice president of the Western Association of Schools and Colleges (WASC). Cannon currently is helping the Minerva Project, one of the more prominent higher education upstarts.

Credits and badges

Perhaps reflecting their professional backgrounds, Arum and Cannon appeared most concerned about how to assess institutions of learning, while Goligoski and Katzman spoke largely about assessing individual learning. For example, achievement in U.S. higher education is measured through credit hours, but increasingly credits do not reflect the hours of work they are supposed to reflect. "If we truly applied the legal definition of credit hours [one hour of class time plus two hours of study], most seniors are legally sophomores," Arum said. Credits often are difficult to transfer from one institution to another, even though most people outside the elite universities end up accumulating credits from many institutions before they finally earn a degree. The consequence, in Cannon's words, is that "we don't have a shared idea anymore of what a college degree is."

One new form of certification is badges, which essentially are packages of information assembled by people online that publicly display their skills without recourse to formal credits and transcripts. Badges can include information about traditional courses taken at colleges, online courses and "informal learning" such as nonprofit experience, relevant hobbies or skills picked up on the side. Hundreds of organizations are issuing badges, according to Goligoski, with a few universities – for example Purdue (through its Passport platform) and the University of Illinois – also experimenting with what are essentially metadata-encoded ID cards. The idea is that badge-earners can display their competencies online in a flexible way that suits their needs as well as the needs of employers who may be reluctant to rely on often unreliable résumés.

"All learning doesn't merit college credit," Cannon said. "What Mozilla is doing is fantastic. But some of it may merit college credit, and some not. It's very difficult to accredit based on learning outcomes." The system, she noted, is not very rigorous. Essentially, it is based on trust, on the gentleman's agreement.

Missions and assessment

Katzman introduced himself as the ghost of unintended consequences, a reference to his having founded Princeton Review, the test-preparation and college rankings company that some consider to have contributed to much of the current rankings obsession. "I'm a believer in diplomas and the U.S. educational system," he said. "We do it really, really well. So as you reinvent it, don't break it."

That said, Katzman lamented that remedial education is "muddying the mission" of community colleges, sometimes squeezing out college-ready students, while for-profit institutions enroll unprepared students who end up racking up millions of dollars in debt. Colleges should teach college-level courses exclusively, he said, leaving remediation to other institutions. Meanwhile, he said, all institutions of learning seem to be heading toward asynchronous learning, in which classroom time is devoted to discussion while lectures and what's left of textbooks will be modularized online and made more interactive.

Arum's insistence on what he described as value-added assessment of conventional colleges, measuring what students learn once they get there, raised the hackles of a few of his co-panelists (and not a few of his reviewers). "I have no problem with institutions choosing their own goals," he said, "but that is not adequate to the task that a system-level organization requires. We need to be able to compare across units, identify which programs need improvement and fix them. You need comparable data."

Stevens, a sociologist and associate professor education at the GSE, and also a former colleague of Arum's at NYU, wasn't buying it: "Richard, you're sounding very industrial-era here. ... Why universal assessment? Why not a great fertility of certifications?" Arum's answer: Tax dollars are paying for much of the education system, and therefore the federal government must require universal comparability.

For Arum, what we need to know about online alternatives is the degree to which "their outcomes are in the ballpark of traditional delivery systems." The problem, he indicated, is that we used to know how to measure that ballpark. Today it's a whole new game.