Rigorously Studying Cultural Education

In my last post I mentioned a large-scale random assignment study of the effects of school tours of an art museum that I am conducting with my colleagues, Brian Kisida and and Dan Bowen. Some people have asked for more information about that project. So, here is a brief summary of what we are doing in that study as well as some related projects examining cultural education.

The random assignment study of field trips was made possible by the fact that the Crystal Bridges Museum of American Art opened in Northwest Arkansas, an area that had never before had a major art museum. Because there was intense interest from schools in the area in having school tours there were many more applicants for field trips than the museum could accommodate right away. We worked with the museum to randomly assign tours to applicants.

Specifically, the museum received more than 300 applications for tours during the first semester. We organized those applicants into matched pairs, which were often adjacent grades in the same school or the same grade in different schools with similar demographic characteristics. We then randomly assigned one school in each matched pair to be the treatment group and one to be the control group. We randomly ordered the matched pairs and the museum scheduled the first 55 treatment groups for school tours last spring. The 55 matched control groups were guaranteed a tour during the next semester for participating in the study.

We then administered surveys to the randomly assigned treatment and control group students and teachers a few weeks after the treatment group visited the museum. Those surveys were designed to measure five types of outcomes: 1) whether the school tour helped create cultural consumers (students who want to return to museums and engage in other cultural activities), 2) whether the school tour helped create cultural producers (students who want to make art), 3) whether the school tour increased student knowledge about art and history, 4) whether the school tour improved student critical thinking about works of art, and 5) whether the school tour altered student values, like empathy and tolerance.

We have already collected results from almost 6,000 K-12 students and teachers from 80 different schools during last spring’s research. This fall we are adding another 4,000 students and teachers to the study from another 60 or so schools. When it is all done and analyzed it will probably be the biggest, most comprehensive, and highly rigorous examination of the effects of school tours of an art museum.

As part of the study we are also asking students in grades 3-12 to write short essays in response to paintings that they have probably never seen before to assess how they critically analyze a new work of art after they’ve had a school tour of an art museum. Last semester we coded almost 4,000 essays in response to Bo Bartlett’s painting, The Box, which was pictured in my previous post. This semester we wanted to try something a little more abstract, so we we will be coding another 2,500 or so essays in response to Marsden Hartley’s painting, Eight Bells Folly, which is pictured above. Dan Bowen has taken the lead in the coding and analysis of these essays and will soon be on the job market in case anyone is looking for a great and innovative researcher to hire.

There are obvious limitations to our study. We can only measure short term effects since the control group receives the treatment the following semester. And we can only measure a limited set of outcomes from an art experience. But we will know a whole lot more and with higher confidence than we do now.

We are also conducting two studies with the Walton Arts Center, which is a performing arts theater in Fayetteville, Arkansas. In one study we are are working with our colleague in the music department, Lisa Margulis, to learn about the effects of information in program notes on students’ experiences during school field trips to see performances. We are randomly assigning students to receive program notes with information about the show they are seeing or “placebo” program notes that do not tell them about the show they are seeing. The question is whether information alters the experience.

And in the other study with the Walton Arts Center we are surveying more than 2,000 7th grade students in area schools to link the past performances they have seen on school field trips to their current behaviors as cultural consumers and producers as well as some empathy and tolerance outcomes. We are also going to use attendance zone boundaries as an exogenous source of variation to make stronger causal claims about how past school field trips may have contributed to current behaviors and attitudes.

We are also in talks with various folks about additional studies, all of which will use random assignment or similarly rigorous methods. This line of work is particularly exciting because there is a limited amount of rigorous research out there on how school cultural activities affect students.

Ha! Yes, we gave some students by lottery program notes that described the Walton Arts Center but had no information about the show they were seeing. This was the “placebo.” The treatment group had a program note that described the show. And there was a third condition of students who did not receive a program note at all.