To Raise Completion Rates, States Dig Deeper for Data

By Eric Kelderman
March 02, 2012

The Minnesota State Colleges and Universities system allows students to apply for enrollment until the Friday before classes began. But then administrators at one college looked at what happened to those students and discovered a pattern: Students who enrolled closer to the start of the semester didn't do as well as those who had signed up earlier.

As a result of that finding, the college required new students to apply at least 10 days before the start of the semester, and similar efforts are being studied at other institutions in the statewide system.

The change in policy is meant to save the students and the institutions from wasting time and money on courses that are more likely to lead to failure, says Leslie K. Mercer, associate vice chancellor for policy and planning at the statewide system, which collects extensive information on the roughly 280,000 students enrolled in credit-bearing courses.

While a relatively minor decision, that sort of change can make a big difference in helping colleges to ensure student success. And it's an example of the kinds of policies that are possible only when states and systems track and analyze the academic performance of individual students.

With the growing demand for improving college completion rates has come a need for more thorough information about just how well or poorly colleges and their students are performing on a variety of measures. In a growing number of states, that data is being used to improve the number of students who finish their degrees. Some states and higher-education systems are even connecting data about individual students to the jobs they get after college, to determine the average wage earned by graduates of particular programs.

Several nonprofit groups are involved in pushing states and institutions to dig deeper and connect the dots between data and policy, and the U.S. Department of Education provided $250-million through the federal stimulus bill to help states analyze what is working in education from preschool through college.

But the process of collecting, reporting, and using that information to guide policy and practice turns out to be complicated by limited and inconsistent definitions of whom to count as students, by bureaucratic hurdles, and even by institutional resistance to accountability.

Quantity and Quality

The problem with using data to inform and improve completion and graduation rates is not an inability to collect relevant information. All but a handful of states and higher-education systems can track individual students in some way, through financial-aid data, for example.

In Minnesota, the state college system began using a common set of data after the seven public four-year institutions were merged with the state's 24 two-year colleges in 1995, mostly to improve efficiency, says Craig V. Schoenecker, system director for research.

In addition, nearly all public colleges report reams of information to governing boards, coordinating agencies, state legislatures, and the federal government.

But one issue, according to the nonprofit group Complete College America, is that the academic performance and completion of most students in higher education isn't being counted. Much of information that public colleges are reporting on completion is only on full-time students who are enrolled for the first time—only a quarter of all college students, the group says. That leaves out the 40 percent of students who attend public institutions part time, the organization explains, as well as all the students who transfer to another college.

A related issue, says Tanya Garcia, a researcher for the State Higher Education Executive Officers Association, is that educators and states have struggled to find common definitions of who should be considered a degree-seeking student and even which students should be considered full time or part time. Different systems within a state may even have different definitions.

Students at private, nonprofit institutions also go uncounted by most states. Only 19 states follow the college outcomes of such students, according to a survey of postsecondary data systems by the executive-officers' association.

Making policy based on data is also difficult within and across states because of bureaucratic hurdles. Many states have more than one system that collects data on college students, such as a coordinating or governing board or a state agency. A 2010 survey of postsecondary data systems by the State Higher Education Executive Officers Association spelled out the problems bluntly: Those data systems "may contain data from only one institution of higher education, several institutions, institutions within a defined system, or all institutions in the state. Further, they may contain student data in the aggregate ... or at the unit record level."

In addition, states and higher-education systems make it difficult for researchers to study student performance and completion data because only 27 states will release that information, even when a request meets the federal requirements meant to protect sensitive information about individuals, the survey found.

Getting Better

Despite the challenges, the number of states and institutions collecting and using in-depth data is increasing. Two-thirds of the states, including Minnesota, have agreed to analyze and publicly report a common set of data points, such as enrollment in remedial courses, as well as degrees and certificates awarded, including for part-time and transfer students.

Those standards, called the Common College Completion Metrics, were developed by the National Governors Association.

The next step in Minnesota is mining the data on individual student performance to know when a person might need more intensive academic support—essentially using a student's early outcomes in the classroom to predict the need for more services.

And the data collected and reported by the Minnesota State system also improves its public perception. The system uses aggregate information on student completion to create an online "dashboard" for the system and each of its campuses. That public reporting not only helps parents and prospective students, but also creates a sense of transparency with the state's elected officials, says Ms. Mercer of the Minnesota State system.

"We're certainly in an era of people wanting a lot of accountability," she said.