Despite an increasing focus on issues of accountability, assessment, and data-driven decision making (DDDM) within the postsecondary context, assumptions regarding their value remain largely untested. The current study uses empirical data from 114 senior administrators and 8,847 students at 57 institutions in five states to examine the extent to which institutional assessment and data-driven decision making shape the experiences of first-year students. Nearly all these schools regularly collect some form of assessment data, and more than half report using assessment data to inform decision making. However, the institutional adoption of policies related to the collection of assessment data or the application of data-driven decision making appears to have no relationship with student experiences or outcomes in the first year of college. Thus, findings from the current study are consistent with the small, but growing, body of literature questioning the effectiveness of accountability and assessment policies in higher education.
Roughly one in four students who begins college at a 4-year college or university does not return to that institution for a 2nd year (ACT Inc., 2016). This troubling statistic has not changed dramatically during the last 30 years, despite institutions of higher education implementing countless reforms in an effort to increase student success (e.g., learning, persistence, graduation). Among these efforts are hundreds of specific initiatives designed to facilitate student engagement, which has been found to predict student grades and persistence, particularly for underrepresented and underprepared students (Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008; Kuh, Kinzie, Buckley, Bridges, & Hayek, 2007)-with a strong focus on the critical 1st year of college (Barefoot et al., 2005; Upcraft & Gardner, 1989; Upcraft, Gardner, Barefoot, & Associates, 2005).
Although various piecemeal initiatives have certainly contributed to improved outcomes at many institutions, they have not appreciably increased national persistence rates. As a result, educational policymakers and administrators have come under growing pressure to address this critical issue. However, this mandate is complicated by economic circumstances that have heightened the need for greater operational efficiency within higher education (Altbach, Berdahl, & Gumport, 2005; Paulsen & Smart, 2001). Responsive to these external pressures and in an effort to demonstrate their effectiveness, many colleges and universities are now spending hundreds of thousands of dollars, annually, to cover the costs of assessment (Cooper & Terrell, 2013), most of which continues to focus on student experiences and outcomes (Bresciani, Gardner, & Hickmott, 2009; Ory, 1992; Schuh, & Associates, 2009; Schuh & Gansemer-Topf, 2010).
Underlying this approach is the assumption that educational quality is likely to be improved when decision makers develop policies and implement practices informed by relevant assessment data (i.e., “data-driven decision making” or DDDM). However, the assumption that assessment practices and DDDM by institutions of higher education yields improved student experiences and outcomes remains largely untested. Therefore, the purpose of this article was to articulate distinct perspectives on the use of assessment and DDDM in higher education, document the extent to which these practices have been implemented, and use empirical data from 114 senior administrators and 8,847 students at 57 diverse postsecondary institutions across five states to examine linkages between assessment/DDDM policies and student experiences in the 1st year of college. Specifically, this study addressed two research questions:
1. To what extent are institutions of higher education employing assessment and DDDM regarding students’ 1st year of college?
2. To what extent does institutional adoption of assessment and DDDM correspond to levels of 1st-year student engagement and perceived gains?