As Institutional Researchers Meet, Studies Galore

Submitted by Doug Lederman on May 27, 2008 - 4:00am

Critics have fostered a stereotype of higher education as an industry devoid of experimentation, content to let mediocrity reign amid declining student outcomes and rising prices. But perusing the program of the annual forum[1] of the Association for Institutional Research, which began Saturday in Seattle, leaves a wholly different impression.

In session after session, college officials presented data they had collected or produced showing how their institutions had fared in attacking some perceived problem or challenge. Many related to issues related to the quality of the student experience -- underenrollment of first-generation students, for instance, or low graduation rates for community college transfers, or why students drop out. Others sought to examine institutions' performance on other fronts, examining whether one state's colleges and universities are on track to produce enough graduates in its state's high demand fields or the best ways of "benchmarking" one institution's success against a group of peers'.

Over the next several days, Inside Higher Ed will feature highlights of some of the more interesting or noteworthy papers presented at the institutional researchers' conference. The first batch -- looking at the efficiency and productivity of public four-year colleges, instances of racially insensitive behavior on campuses with differing degrees of diversity, and an alternative method of calculating student graduation rates -- appears below.

Cost Efficiency vs. Productivity at Four-Year Public Colleges

In their paper, “Degree Productivity and Cost Efficiency in U.S. Public Four-Year Colleges and Universities: Is There a Tradeoff?” Marvin A. Titus, an assistant professor of higher education at the University of Maryland, and Kevin Eagan, a doctoral student at the University of California at Los Angeles, examine how public universities are positioned to respond to the increasing calls from politicians and policy makers to be both more cost efficient and more productive in terms of turning out skilled graduates.

Using an analysis that compares different types of four-year public colleges to their related peers, the researchers find significantly greater variation in the cost efficiency of general baccalaureate public colleges than is true of public research and doctoral institutions. “The average doctoral university is operating 17 percent below the most cost efficient doctoral institution,” the researchers write. “The average research university is operating 23 percent below the most cost efficient research institution, while the average general baccalaureate and specialized four-year college is operating 29 percent below the most cost efficient general baccalaureate and specialized four-year institution.” Among four-year public colleges, Eagan said, general baccalaureate colleges appear to have the most ground to gain in terms of improving their cost efficiency.

Similarly, the researchers find, those institutions also show the greatest variation in their production of degrees, with the average general baccalaureate and specialized institution performing at a rate that is 51 percent less productive than the most "technically efficient" college of its type. The average "comprehensive" four-year public colleges produces bachelor's degree recipients at a rate that is 26 percent below the most efficient comprehensive college, the researchers find, while there is little or no variation in the productivity of public research and doctoral institutions.

But while the researchers' findings suggest that general baccalaureate and specialized four-year institutions appear to have significant ground to gain in terms of both cost efficiency and the production of more graduates, the scholars also conclude that it may be impossible for them to do both.

"If they try to become more cost efficient, there is some evidence that they may become less production efficient," Eagan said in an interview. "The calls for increased production and increased cost efficiency really are asking those institutions to do something that's contradictory, and it may be difficult for them to improve on both at the same time because they tend to be the last type of institutions that becomes funded by the state."

Racially Insensitive Behavior

A paper to be presented today by two administrators at San Jose State University aims to examine whether students are more or less likely to experience racially insensitive remarks at colleges and universities based on the diversity of their student bodies.

Berkeley Miller, San Jose State's associate director of assessment, and Sutee Sujitparapitaya, associate vice president for institutional research, surveyed students at nine institutions in the California State University System, aiming to gauge the extent to which they had been exposed to various forms of racially insensitive behavior. The campuses varied in their racial makeup: "Whites made up the large majority of the student bodies on two campuses, and somewhat over half in three campuses.... [M]inority students constituted a small majority on two campuses and a large majority on two others."

Over all, 17.8 percent of respondents "reported they had personally experienced or witnessed insensitive behavior or remarks over the past year," with differences by race: 13.1 percent of white students, 31.3 percent of international students, 32 percent of African American students, 24.6 of American Indian students, 22.6 percent of Asian students, and 12.6 percent for non-Mexican Latinos.

Consistent with what the researchers call "structural diversity" theory, which holds that when white students dominate a campus, "minorities are treated as symbols rather than individuals" and frequently experience discrimination, the study found that students were likelier than average (18.5 percent) to experience racially insensitive behavior on campuses where white students made up at least 70 percent of the student body. Many of the individual minority groups reported experiencing their greatest levels of racially insensitive behavior on white-dominated campuses.

As the proportion of white students on campus decreased, to 50 to 59 percent white and 40 to 49 percent white, so in general did the reported incidence of discrimination, the study found. But to the surprise of the researchers, who described the result as "clearly unanticipated," the proportion of all students reporting incidences of racially insensitive behavior peaked (21.1 percent) at "racially mixed" institutions where white students represented just 30 to 39 percent of the student body. Similar patterns existed for most racial groups, including white students.

The researchers call for more research to try to replicate the results, not simply because it is desirable to avoid racially insensitive behavior, but because "[s]tudents who personally experienced or witness insensitive behavior based on race or ethnicity expressed lower overall satisfaction" in college."

A Different Way to Measure Graduation Rates

Everyone wants more college students to finish their programs, but what's the right way to measure completion?

In a paper presented at the AIR forum, the six-year gradation rate (the federal measure, based on the idea of six years representing 150 percent of the standard time needed) as a way to measure success was questioned.

Ed Rugg, Donna Hutcheson and Erik Bowe from Kennesaw State University wrote the paper, which questions the assumptions behind the six-year rate. That model is based on characteristics of traditional students. However, many students in college today are nontraditional -- especially at colleges like Kennesaw State.

The paper proposes another model to analyze graduation rates, either in addition to or as a replacement for six-year rates, known as a "degree completion analysis." The suggested method takes a "back end" approach to measuring graduation rates rather than tracking the outcome of students from freshman year to graduation. Such an approach would look backwards at where graduates started, their path to graduation, differences between transfer students and those enrolled from the start, and so forth. Various students who are not counted in the federal rate (part-timers or transfer students, for example) suddenly would be examined.

Using the degree completion analysis approach, Kennesaw State awarded 2,200 bachelor's degrees in the fiscal year of 2006. Under the six-year model, of the 1,200 full-time freshmen who began of the fall of 1999 and 2000, only 400 graduated.