Evaluation & Research on the Effectiveness of TIEE

April 2006 Update

Introduction

Evaluation and research on the effectiveness of TIEE are essential so that we better understand aspects that are valuable, and what features need adjustment. Over the last several years Deborah Morris and Dan Udovic have headed up the evaluation of TIEE. We include here a summary of findings reported in D'Avanzo et al. 2006.

TIEE Users

For the TIEE study, the evaluators used surveys and interviews. The surveys provided descriptive data about a range of participants, while interviews allowed more in-depth study of some participants' views. The majority of survey respondents (61%; N=59) teach at liberal arts colleges, 26% are from research universities, and the rest are from community colleges and other institutions. The large number of liberal arts faculty was expected because teaching is emphasized at these schools, but we were pleased that a quarter of respondents taught at universities with large student populations. Over half of undergraduates are enrolled in community colleges and few faculty at these schools participated in the survey. We have submitted a proposal to NSF that includes plans for increasing community college faculty engagement with TIEE.

Role of TIEE in Teaching

About 50% of survey respondents rated themselves as "very familiar" with non-traditional pedagogy. Despite this, respondents allocate 25% or less of their course time for active learning, indicating that even faculty informed about active teaching primarily rely on traditional approaches in their classrooms. Ninety-three percent of those surveyed agreed or strongly agreed that TIEE had helped them improve their teaching of ecology. More specifically, respondents indicated that they used more active learning strategies and inquiry-oriented teaching approaches. However, while cooperative learning or inquiry approaches were used by many, relatively few faculty had employed alternatives to traditional tests or formative (ongoing) evaluation.

Adaptation and innovation of TIEE
One focus of our evaluation was the degree to which faculty modify activities and ideas to match their own teaching styles and the needs of their students. A key finding from both the survey and interviews was that faculty who use TIEE often adapt or modify the materials. Only 20% surveyed said that they used the TIEE materials "as is," while the rest used selected elements of TIEE activities or adapted them for a different use.

High quality ecological and pedagogical material
Users state that they value TIEE because of the high scientific and pedagogical quality of TIEE materials. Survey respondents indicated overwhelmingly that TIEE materials were scientifically accurate (100% "agreed," or "strongly agreed") and focused on core ecological principles (98%). In interviews, users often used the term "rigorous" to refer to both the scientific and pedagogical aspects of TIEE and that the quality far exceeds the type of labs generally published on the web or in lab manuals. Users frequently noted that TIEE fills a major need of helping students learn ecology through the use of real-world data and investigations.

Creating TIEE: sharing and scholarship
A major goal of TIEE is to provide a peer-reviewed venue for ecology educators to publish their work in a way that will be recognized in their institution's promotion and tenure process. Over 60% of individuals familiar with TIEE believe that publishing in TIEE would carry at least moderate weight in their institution's faculty evaluation processes.

Scientific Teaching with TIEE

The evaluation showed that few faculty were methodically studying the impact of their teaching on student learning. To address this, we are working closely (e.g., via workshops and conference calls) with a team of 15 selected faculty (see list below) from a range of institutions who are systematically studying possible impacts of TIEE on their teaching. These teams will present five posters at the 2006 ESA meeting in Memphis (http://www.esa.org/memphis/) describing their findings and effects of doing this research on their teaching.

These faculty have identified measurable outcomes — such as students' ability to make figures from raw data — and are using a variety of approaches, including pre/post tests and surveys, in their studies. Replicate measurements in the same course over a semester and several years plus across institutions can potentially result in publishable findings.

Our hypothesis is that these "practitioner researchers" will have a deeper understanding of why the student-active approaches featured in TIEE (such as groupwork) promote increased student learning and will be more committed to their use. This can be tested in a comparative study. During the last few years, Handelsman et al. (2004) and others have emphasized the need for "scientific teaching" — the application of scientific research methodology by faculty to their own teaching. Future plans for TIEE include expansion of this scientific research component.

Teaching Issues and Experiments in Ecology (TIEE) is a project of the Education and Human Resources Committee of the Ecological Society of America.
This page was originally published on 26 February 2005 and was last revised 30 March 2006.