Assessment of Outcomes: The Effect of Incentives on Student Participation Rates and Performance Levels

Article excerpt

INTRODUCTION

For some time assessment of outcomes of academic experiences has been center stage at schools at various levels of education in the United States. The need for assessment of outcomes has moved from the primary and secondary levels of education to higher education. The U.S. National Institute for Learning Outcomes Assessment (NILOA), established in 2008, is evidence of the current importance of assessment of outcomes at the higher education level in the U.S. The NILOA's charge is to assist organizations in finding and using methods of assessment of student learning outcomes at colleges and universities.

An important consideration, which has been studied little in the context of assessment of outcomes in higher education, is the effect of incentives on student participation in outcomes assessment measurement and performance on these measures. The intent of outcomes measurement is to assess the level of student learning/performance at a point in time. Incentive induced variability in this assessment could bring to question measurement validity. Here, in an exploratory study, the question, "Does varying the level and type of incentive have an impact on student participation in outcomes assessment measurement and performance on these measures?" is examined. Exploring the connection between incentives and the assessment of outcomes is the focus here.

THE ASSESSMENT ENVIRONMENT

The response to assessment of outcomes from the U.S. higher education academy has run the range from "rebels" to "believers", with most academicians in the middle of the distribution. Many among the "believers" reside in the sources of motivation for assessment. These sources include legislative bodies, accrediting bodies, and university administrations. Other "believers" are scholars in the assessment area who produce resources that can aid in the assessment process [Banta et al 1996, Palomba and Banta 1999]. The "believers" attitude may be typified by the following: "... if our accrediting body were to say "You no longer have to go to the trouble of assessing student learning" we would still do it anyway [Engelmann 2007]."

The "rebels," on the other end of the distribution, consider outcomes assessment "...scams run by bloodless bureaucrats..." where professors are marched to a "...Maoist countryside where they are made to dig onions ... and then compelled to spend the rest of their waking hours confessing how much they've learned by digging onions" [Fendrich 2007]. Others have compared outcomes based education (OBE) to effects based operations (EBO), Mr. Donald Rumsfeld's military campaign planning strategy where, 1) outcomes are identified, 2) measurable targets are identified, and 3) targets are targeted [Collender 2009], The analogy is obvious.

Given that assessment of outcomes is mandated by legislative bodies, accrediting bodies, and/or university administrations, embracing the requirement, or at least acquiescing to the requirement, appear to be the most common responses. Even Fendrich, quoted earlier, puts herself in the category of, "... resigned realists, who know that outcomes assessment is bureaucratic baloney but who recognize that resistance is futile [2007]". Callahan, Strandholm, and Dziekan [2010] note that whereas a segment of the faculty are less than enthusiastic in participation in assessment of outcomes, and view it as an interference and meaningless routine, the vast majority cooperate in the process.

DIRECT ASSESSMENT

Whatever the motivation for participation in assessing outcomes of learning, there is much written material available describing putting the mechanisms in place to accomplish the assessment of outcomes task. There has been a range of application of outcomes assessment from the institution wide level [Williford 1997], to the major discipline area [Moberg and Walton 2003], and on to the specific course level [Deeter-Schmelz and Ramsey 1998]. A range of methods for assessing outcomes has also been utilized. …