Research on Evaluation

CRDG faculty and staff do more than conduct evaluation studies; they also engage in research on evaluation. This research addresses the repeated call in the professional literature to establish a stronger theoretical and empirical foundation for evaluators’ understanding of evaluation practice, evaluation methods, evaluation theory, and the profession of evaluation.

Examples of our research on the practice of evaluation include the study of (a) the degree to which the participation of program personnel in evaluations affects evaluation methods and results and (b) the study of the degree to which programs are implemented as intended.

Our research on the methods of evaluation has examined the development, validation, and use of evaluation methods such as classroom observations, teacher logs, student self-report instruments, student assessments, and other data collection instruments. As a result of this kind of research, CRDG faculty and staff have published reports about the development of the instruments, as well as several evaluation instruments in the professional evaluation literature.

Our research on the theory of evaluation has addressed fundamental issues in evaluation—revisited regularly in new contexts over the years—that undergird how and why evaluations are conducted. Theory topics have included the extent to which evaluations should respect indigenous populations, the use of evaluation findings by program personnel, and others.

Our research on the profession of evaluation has included reviews of the degree to which research-on-evaluation studies have been reported in the professional evaluation literature and a study of evaluators’ perceptions of evaluation. Recent scholarship on this issue, as well as on other aspects of research on evaluation was published in Issue No. 148 (Winter 2015) of the New Directions for Evaluation an American Evaluation Association journal.

The results of CRDG faculty and staff’s work on the four broad evaluation topics have been presented at national evaluation conferences, published in several national refereed journals, and discussed in books. They have provided some of the background necessary to be awarded grants from the National Science Foundation and the U. S. Department of Education, and they have helped serve as the foundation for the selection of a CRDG faculty member as the 2013–2017 Editor-in-Chief of New Directions for Evaluation. CRDG faculty and staff intend to broaden their body of research on evaluation practice, methods, the profession, and theory and thereby continue to enhance CRDG’s stature and contributions nationally and internationally.

Brandon, P. R., & Fukunaga, L. (2014). The state of the empirical research literature on stakeholder involvement in program evaluation. American Journal of Evaluation, 35, 26–44. doi: 10.1177/1098214013503699

Young, D.B, Pottenger, F.M., Brennan, C.A., Brandon, P., & Nguyen, T. T. (2012). Science and engineering in early education: a conceptual framework for improving teaching and learning. Moscow, Russia: Proceedings of the International Scientific and Practical Conference Science Education in the School of the Information Age (translated from Russian).

Harrison, G. M. (2014, April). The effects of intrajudge consistency feedback in an Angoff standard-setting procedure. Paper presented at the annual meeting of the National Council on Measurement in Education, Philadelphia, PA.

Harrison, G. M., Vallin, L. M., Philippoff, J., Brandon, P. R., & Seraphin, K. (2014, April). Balancing development and measurement needs in an evaluation of a program under development. Paper presented at the annual meeting of the American Educational Research Association, Philadelphia, PA.

Vallin, L. M. & Philippoff, J. (2013, October). A closer look at research on evaluation: Common areas of research, design and methods. Paper presented at the annual meeting of the American Evaluation Association, Washington, D.C.

Brandon, P. R. (2012, April). Ruminations on research on evaluation . Paper presented at the meeting of the American Educational Research Association, Vancouver, B. C.