Connecting the Dots between Learning and Resources

King's College

For the better part of the last fifteen years, leaders
in American higher education have been working to improve
student success, focusing particularly on assessment
of learning outcomes and getting more students to degrees.
But progress in translating good intentions into improved
educational practices is at risk of being erased by
eroding fiscal support. In response to current financial
pressures, leaders are asking how to allocate scarce
resources for the greatest payoffs in student learning
and degree attainment. This is a particularly critical
issue for institutions serving at-risk students, as
they work to meet low-income students' educational and
financial needs with far fewer dollars than their wealthier
counterparts.

To better understand what is known and what remains to be discovered about cost effectiveness and the differences money either does or does not make in getting students to degrees with acceptable levels of learning, I created an occasional paper titled "Connecting the Dots Between Learning and Resources" for the National Institute for Learning Outcomes Assessment. In the paper, I present a conceptual approach for analyzing the relation of spending to student success, followed by an examination of existing research on the topic. While the full paper is available online at www.learningoutcomesassessment.org/documents/Wellman.pdf, I will summarize some key points for Diversity & Democracy readers here.

A Conceptual Framework for Thinking about Cost-Effectiveness

Ideally, to look at cost-effectiveness, one would look at the role of funds in producing educational value added, or the translation of inputs into outputs. Such a look would require better ways of evaluating learning than are currently available, as well as better ways to look at how funds are used within institutions. In the absence of ideal measures, a number of proxies can be created that measure different dimensions of spending in relation to types of outcomes:

Cost Variables

Spending per student or total revenues from all resources

Spending per student from unrestricted resources only

Spending per student by functional area (e.g., for instruction, student services, financial aid)

Spending per student for education and related expenses only (excluding sponsored research and public service and auxiliary enterprises)

Trends over time in spending and by functional area

Outputs

Credits earned

Completion of twelve units or more

First-year retention

Degrees completed

Certificates completed

Transfer from a two-year to a four-year institution

Job placement rates

Pass rates on examinations such as the GRE or licensure examinations

Lifetime earnings

Using this framework, one can imagine a number of ways to connect topics on the cost side of the matrix to the outputs side, to look at spending in relation to different measures of performance.

Spending on Academic and Student Services

A few studies have focused on spending and performance as measured by student
degree attainment and effective teaching practices.
For example, researchers Patrick Kelly and Dennis Jones
of the National Center for Higher Education Management
Systems (NCHEMS) looked at data for public institutions
to see whether spending makes a difference in access
to higher education, in state-level degree productivity,
or in research funding (2005). Recognizing that these
measures say nothing about quality, the researchers
found no consistent relationship between levels of
spending and measures of performance. Although their
major finding was that funding levels overall do not
explain differences in performance, Kelly and Jones
found that spending on student support services does
correlate with higher levels of degree attainment. Their
research suggests that the way resources are used may
matter as much as or more than the absolute level of
funds available.

In another study, NCHEMS researcher Peter Ewell (2003) compared spending per student for instruction and related expenses at two sets of institutions: one set identified by the National Survey of Student Engagement as outperforming its peers in student engagement and retention, and a second set of peer institutions. Ewell found that the more effective institutions did not spend more per student than their peers, but they did spend differently, putting proportionately more money into academic and student support. A follow-up study from Iowa State University found the same thing: total spending levels evidently mattered less to effective educational practices than did the distribution of the resources within the institution (Gansemer-Topf et al. 2004).

Another recent study provides additional support for the theory that student services expenses make a difference in low-income students' retention and degree completion. Using panel data developed by the Delta Cost Project, Cornell Higher Education Research Institute researchers Webber and Ehrenberg (2009) examined how spending in instruction, student services, and other areas influences the graduation and first-year persistence of undergraduate students. The strongest influence they found was from student service expenditures, with the highest marginal effects for students in institutions with low admissions selectivity and high proportions of Pell grant recipients. Simulations of the effect of reallocation from instruction to student services showed enhanced persistence and graduation rates.

Spending on Student Aid

Some consistent themes relevant to the role of student aid in increasing access and student success appear in the research. One finding is that achievement gaps between low-income students and other students are far wider in terms of college completion than in terms of various measures at the point of initial college entry (Kane 2004; McPherson, Schapiro, and Winston 1993; Mortenson 1998; Pell Institute 2004). Most student financial aid programs, however, make access and not degree attainment their primary goal. Work-study programs are an exception, and the research shows that these programs have had some success in increasing degree persistence among low-income students. In addition to providing financial help, these work opportunities increase student interaction with university staff and faculty and cultivate the student's identity as a member of the campus community.

A second consistent finding in the research literature relates to student financial aid. Institutions have rapidly increased their resources for student aid, through both "tuition discounts" and grants. Research shows that the majority of this has gone to "merit" aid, which is distributed primarily on the basis of academic merit rather than financial need (College Board 2008). Since merit aid goes to students who would go to college without it, institutions would better increase college access and persistence by spending their limited dollars in grant aid, in work-study programs, or on programs to enhance student success.

Spending on Faculty and Teaching

As institutions work to contain costs, the use of part-time and contingent faculty has increased precipitously. Some researchers have looked at the relationship between the use of part-time and contingent faculty and measures of student learning. Umbach (2007) surveyed faculty using data from the Faculty Survey of Student Engagement to evaluate the relationship between faculty appointment status and institutional engagement with effective teaching practices. He found that contingent faculty showed less use of active and collaborative teaching techniques, less likelihood of challenging students academically, less likelihood of spending time preparing for class, and less likelihood of interacting with students.

Examining first-to-second-year persistence and graduation rates and use of part-time or nontenured full-time instructors, Ehrenberg and Zhang (2005) found that increases in nontenured and part-time faculty reduced both rates. Using data from Ohio public institutions, Bettinger and Long (2004) also looked at how adjunct faculty affect student interest and course performance. This study found different effects depending on discipline, with an overall slight loss of student interest in subjects for courses taken from graduate students and adjuncts compared to full-time faculty, and with a slight positive effect from use of adjunct professors among disciplines with an occupational or vocational focus.

Learning Productivity

A promising area of work has been in what Bruce Johnstone calls "learning productivity"--ways
to increase learning with either less time or less costly
inputs (1995). A good example comes from Carol Twigg
(1999) and her colleagues at the National Center for
Academic Transformation, who assessed learning and costs
for courses delivered through a traditional lecture/discussion
session format and compared them to those of courses
delivered using distance-learning technology. The primary
cost difference came from less time spent in course
and materials preparation and from labor savings by
substituting low-cost "coaches" for faculty and teaching
assistants. The researchers found superior results at
reduced costs for the technology-enhanced courses.

The revenue meltdown of 2008 and 2009 has meant that
"learning productivity" efforts are expanding rapidly
through the efforts of several state systems and other
college leaders. In the University of Maryland system,
a comprehensive initiative to tackle inefficiency and
to increase effectiveness included limitations on reimbursements
for credits earned above the 120 units required for
the degree, and a requirement that all students earn
at least one semester's worth of credits through some
form of off-campus instruction: credit by examination,
study abroad, or distance-based courses.

Reconnecting the Threads

Although more research is sorely needed, threads of an emerging consensus begin to appear in what is known and what might reasonably be surmised from the research:

Intentionality matters as much as, or more than, money alone. Institutions whose leaders put resources behind instruction and student services show greater rates of persistence and graduation per dollar spent. If the national priority is to increase academic attainment, more can and should be done to focus on student learning and degree attainment as the first priority for resource use.

Focusing resources on instruction and student services helps to increase learning, retention, and degree attainment. Investments in faculty resources make a difference in student learning. Student services investments are especially important for increasing retention at institutions serving large proportions of at-risk students. Yet the pattern among public institutions across the country has been to disproportionately reduce funding in instruction and student services. Deliberate efforts must be made to protect funding for these resources.

Student financial aid programs need to be restructured to support the goal of student degree attainment as well as access. Student aid makes the biggest difference in access for low-income students, but less of a difference in student success. Grant aid allowing students to attend college full time and increased funding for on-campus work programs can help improve retention and graduation.

Excess credits and student attrition cost
money and do not help students get to the finish line.
Curriculum realignment, aggressive academic counseling,
and attention to course scheduling can all help increase
student success at reduced cost, both to the student
and to the institution. Redesigning curricula to ensure
coherence and to focus on learning results can be
cost-effective if done with an eye on spending as
well as on student success, and if accompanied by
attention to student and academic support services
aligned with the goal of increased learning success.

Our country is faced with the daunting challenge of increasing educational attainment despite negative funding patterns that threaten access and quality. We cannot meet these attainment goals unless we are successful in closing achievement gaps all across the educational pipeline. We need an investment strategy to do this, one that supports our educational and attainment goals. While the findings from this research are still quite tentative, they can help point the way to spending decisions that will make a difference in increasing success for all students.

This article has been adapted with permission from the National Institute for Learning Outcomes Assessment.

References

Bettinger, E., and B. T. Long. 2004. Do college instructors matter? The effect of adjuncts and graduate assistants on students' interest and success. NBER Working Paper No. 10370. Cambridge, MA: National Bureau of Economic Research.

College Board. 2008. Trends in student aid. New York: College Board. professionals.collegeboard.com/profdownload/trends-in-student-aid-2008.pdf.