Supporting and Assessing Liberal Arts Education

eAIR recently spoke with Charles Blaich and Kathy Wise from the Center of Inquiry at Wabash College about their work to help colleges and universities strengthen their assessment efforts.

Charles has served as the director of the Center of Inquiry since 2002 and has also led the Higher Education Data Sharing Consortium (HEDS) since 2010. Kathy is Associate Director of the Center of Inquiry and HEDS. They both also direct the Center’s Teagle Assessment Scholar Program. As part of their work with the Center, Charles and Kathy led the Wabash National Study of Liberal Arts Education (2006-2012) which grew to include a mix of 49 different institutions, including both public and private and those not specifically viewed as liberal arts institutions. Fundamentally, the Wabash National Study sought to uncover ways to support and assess liberal arts education.

Charles Blaich Kathy Wise

eAIR: The Wabash National Study of Liberal Arts Education focused on helping colleges and universities to improve student learning related to key liberal arts education outcomes, to which many institutions can relate. What are some success stories from that study?

Kathy: One of the things we found with the Wabash National Study is that it is important to prioritize and pick one thing to focus on and then make sure that everyone on campus is focusing on that same thing. One institution we worked with created a short, course-based survey centered on NSSE that focused on academic challenge. They invited faculty to a pilot study where they administered the survey in their classes, and IR compiled the data and gave it back to faculty. The survey was anonymous, but faculty could see themselves in relation to their peers and determine how positive changes in the classroom might be reflected in the data.

Charles: The institution that Kathy mentioned had a very good academic leader as well as an IR director, the director of the teaching and learning center, and some associate academic leaders who, over a period of about four or five years, decided to focus on academic challenge. They worked together until they could see a real change in their NSSE data. The stories are all very similar. There is usually a group of clever people working together, obtaining data, and choosing to focus on something specific. They have backing from their academic leaders, and over the course of several years they develop programming that works its way into the interstices of faculty and staff development and the work of the institution.

eAIR: The online description of the Teagle Scholars program points to a core reason institutions still struggle with closing the loop: "institutional governance structures, reward systems, and faculty and staff incentive structures—which began their evolution decades ago—are neither organized for nor experienced with using evidence for change." How have the Teagle Scholars gotten past such obstacles? Can you provide examples?

Kathy: They don’t necessarily get past the obstacles, but they work around the existing structures. The folks we’ve met who have been most successful are the ones who, when faced with these obstacles, don’t throw their hands in the air. They recognize that change is a long-term goal that takes time. The Teagle Scholars and assessment leaders who have been most successful develop a set of strategies to move things forward. There are a variety of things they do, but a key activity is reframing assessment from the incredibly persistent, mostly negative views that people have about it.

Charles: They try to reframe assessment as something to improve your teaching, not something you do in response to external requirements.

Kathy: Yes, they make it an internally-driven, inquiry-based way of improving the learning and teaching going on at the institution – improving faculty and staff’s work with their students. So, they think about it as faculty and staff development and not assessment, per se. Successful assessment leaders realize that there are negative views about assessment and they address them head-on. They also make sure that the processes they are engaged in focus on connecting with the work that faculty and staff are doing with their students, so they see some benefit from it in their day-to-day work. Some of the most successful Teagle Scholars and assessment leaders in general take a consultative approach. They don’t just sit in their offices waiting for people to come to them, they go out and get to know faculty and staff in the different departments and programs, learn about the work that they’re doing with students, and then find ways to work assessment in and connect it to their work.

eAIR: Would you generally describe some of the successes of the Teagle Assessment Scholars in helping institutions to become better at utilizing assessment evidence for change?

Charles: Certainly. Another thing that Teagle Scholars do well is experiments. They reframe assessment as being based upon trying out what works rather than being about proving value as a department. Instead of “Can you prove that your students have met your outcomes?” the question becomes “What experiment would you like to run next year to see if it has the desired effect?". So it’s not that you are trying to generate data to create proof, which is a very high standard for academics, but rather using evidence to see what you might try differently next time. Using the word “experiment” turns out to be helpful because it lowers the temperature from “We’ve done these analyses and shown that seminar X isn’t working” to “Let’s try this and see what happens.”

Kathy: A lot of examples come from the Wabash National Study schools. Like Charlie said, a big part of this work is being patient and persistent. Seeing significant change at your institution won’t be a one or two-year process. It might be four to six years before things change, so Teagle Scholars are a remarkably patient and persistent group. There was one institution that wanted to focus on faculty development. It was challenging because they had a large number of adjunct faculty. Like many institutions with a high proportion of adjuncts, these faculty members did not fit into the existing development work and departmental structures, and were not included in many activities. The institution decided to create a center for teaching and learning and focus on getting adjunct faculty involved, although all faculty could participate. They made a special effort to develop programs in which adjuncts could participate, created online as well as face-to-face programs, and planned activities at different times to accommodate adjuncts' schedules. It turned out to be amazingly successful and the adjuncts really appreciated it. They felt like they were part of the institution and that their work was valued. As a bonus, the program also helped them improve their skills.

Charles: This is where the Teagle Scholars are “crafty leaders.” The reason this institution wanted to focus on faculty development was that they received some data from the Wabash National Study, which included data on teaching and learning practices, and outcomes and conditions did not look good. So, they wanted to improve teaching and learning based on the data. A lot of the faculty blamed the adjuncts for this. The Teagle Scholar who developed the center for teaching and learning knew that adjuncts were not to blame, but it provided the impetus to get a teaching and learning center funded in which programs could be developed to benefit the full-time faculty and engage the part-time faculty. This was over the course of about five or six years – it takes that much time.

Kathy: Teagle Scholars find the path that will work at their institution. Everyone wants to know, “What’s the best practice? Can you give me some examples?” Examples are useful, but, unfortunately, you can’t just take what another institution did, drop it into your situation, and expect it to work the same way. Because institutions are all unique with different cultures, values, and governance structures, you need to adapt ideas to be able to work in your institution. There are often several ways to move something forward. It’s about picking the one thing you think will work best for the people there at that moment in time given everything else that is happening. Teagle Scholars are quite good at figuring out what that path is - and the path that works today might not work a year from now.

eAIR: In what ways has the collaborative work that the Center has engaged in with HEDS helped to strengthen liberal arts education?

Charles: When we started the Center in 2000, the premise was that if we wanted to strengthen liberal arts education in general, getting high-quality evidence about student learning and the conditions that support student learning into the hands of faculty and staff at institutions would drive improvement efforts, so, at the Center of Inquiry we designed the Wabash National Study and engaged in this massive longitudinal data collection with 49 institutions. We did collect a lot of high-quality data, and with our colleagues at research universities we published a lot of research articles. But the principal discovery was that data alone wasn’t sufficient to do much of anything. You have to be engaged in working with faculty, staff, and students to connect the evidence with their programs and engage in conversations about how to make sense of the data and act on it. We had to go much deeper at institutions to develop relationships with people on the ground and cultivate leaders on campus to make sense of and use the data. So, our collaborative work has expanded from a data collection effort to what I would call a “multi-front change effort.” Now when we work with data, we work with faculty, staff, and students to make sense of and act on the data in their particular and different institutional contexts. I appreciate using the word “collaborative” because that’s why we have different programs like the Teagle Scholars program and that’s why we got engaged with HEDS because it takes a lot of people working together to do this.

Kathy: Yes, from the beginning, it has been a journey about how we can help institutions improve. And, over time, we’ve realized that it needs to be a collaborative, community effort. All the different programs we started, as well as running workshops for people has been part of an effort to bring people from different parts of an institution together to talk about data, make sense of it, and, most importantly, to figure out what they are going to do with it and how they can gauge the effect of the changes they make. So, everything we’ve done starting with the Wabash National Study and moving forward to our work with HEDS and the Teagle Scholars Program has all been a journey to help people to work together.

eAIR: How do institutions involved in HEDS share the data to advance liberal arts education?

Charles: The institutions that join the HEDS consortium all agree to share both unit-record data and institutional data - unmasked - with one another. So, if we administer an alumni survey and Wabash College and Carleton College both participate, then they can both see each other’s data. There is an openness there that requires some trust. They agree to use the data for institutional improvement purposes, but not to share it outside the consortium. Because some comparisons are just more compelling than others, the HEDS data allows schools to really pick and choose comparison groups they form and share on campus. And, because schools have shared data with one another, they trust one another and they are very open about asking questions. We have a listserv that is very active and creates community. People from these schools start to work together on solving problems. They share lessons learned, for instance, on how they responded to a survey on campus climate and sexual assault. The sharing of data is a way of creating a community of practice based on the fact that we trust each other and we know a lot about each other based on what we share data-wise. We can develop friendships with colleagues based on the other interactions we have in the consortium.