Re(Focusing) Qualitative Methods for Writing Center Research

Rebecca Hallman, University of Houston
Ezekiel Choffele, Syracuse University
with Trixie Smith, Michigan State University

Over the past decade, writing center (WC) scholars have acknowledged a lack of and need for research-based scholarship that makes methodology explicit (Gillespie et al. 2002; Lerner 2002). In particular, Gillespie et al. call for a move toward more empirical research and argue for a mixed methods approach to move away from that which is deemed simply anecdotal. Some WC researchers have answered this call with a version of scientific empiricism known as replicable, agreeable, data supported (RAD) research (Babcock and Thonus 2012; Babcock 2012; Driscoll and Wynn Perdue 2012, 2014). Our own professional organization called specifically for a focus on RAD research at the 2015 IWCA Collaborative in Tampa, asking “ historically, who gains and who loses in RAD research? Who reaps the benefits? Who shoulders the burdens? And what role does or might our field play in reinforcing or disrupting this history?” We continue this line of inquiry by asking what types of research might be overlooked or devalued if we focus too heavily on a positivist lens?

Furthermore, we worry that some may misunderstand RAD to be synonymous with quantitative research and thus be less likely to pursue qualitative designs. Also adding to this complication is the pressure that many WC researcher-administrators face to balance multiple research demands and audiences, including: (1) the need for and immediacy of locally-based WC research that maintains applicability to specific WCs, thus helping us better understand and intervene in the day-to-day lives of our WCs; (2) the drive to conduct scholarly research that participates in larger conversations in the field of rhetoric/composition and writing center studies; and (3) the necessity of incorporating methods and approaches to research that appeal to college, university, and high school administrators. With these roles in mind, we ask: do quantitative-focused methodological project designs address the concerns of administrators, both in and outside the center? Are qualitative research projects rigorous enough to demonstrate transitions and accomplishments in and around WCs?

While these questions are valid and often necessary, we believe that this ongoing research paradigm creates a harmful binary in which the actual methods are being forgotten. Instead, we argue for a both/and approach that accepts attempts at creating mixed methods approaches that satisfy multiple WC researcher-administrator needs. In other words, we can see a need for for research projects that are primarily quantitative-based, RAD-focused, and/or qualitative in design.

In this essay, we focus our lens on developing pathways between assorted qualitative methods to build on and improve the options for qualitative WC researchers. We seek to make explicit qualitative approaches to writing center research and to demonstrate their value to the writing center community. We use qualitative methods to model our argument. In particular, we use oral history methodologies through interview-based approaches to learn about how three writing center scholars approach their own qualitative research project(s) and argue for a definition of qualitative research that extends beyond anecdote. In fact, we turn to cultural rhetoric’s approaches to methodology as we value stories and the communities they represent (see Powell et al for more this approach). Thus, we seek to constellate the qualitative methods of oral history, narrative inquiry, and storytelling to demonstrate how stories can be used in WC research. We also value the multi-modality of storytelling and show how using direct participants’ voices (through audio inserts) allows us to bring these qualitative researchers directly into the argument we present; they become co-authors of this essay as they lend their voices to the narrative. We are not arguing that this is a better way to do research or the only way to do research, but we do argue that qualitative methods are equally beneficial for and relevant to research generally and to the WC community specifically and that they deserve their place in our resources and professional development venues, including our conferences and journals.

Our Way Into This Project

Rebecca’s Story

On February 8, 2013 I was a second semester graduate student in the relatively new Rhetoric, Composition, and Pedagogy Program at the University of Houston in my first seminar with Jim Zebroski. That evening, I started a conversation with him about research methods in our field. In the email, I wrote the following:

Hi Dr. Zebroski,

During Tuesday night’s seminar, I formulated a question in the back of my mind that hasn’t gone away. As we were discussing positivism, scientism, and quantitative research, I was really troubled by the idea that eclectic research doesn’t seem all that possible for rhetoric/composition. I wasn’t sure exactly what you meant by this. Is there any possibility for conducting a kind of “true” eclectic research, in which the qualitative/quantitative paradigm is dropped so that these methods are integrated in a way?

In other words, isn’t there something valuable for us as a field in practicing quantitative research beyond assessment purposes and being able to communicate to others outside the discipline? I think my concern over this comes from my experience at UofL working with someone on my thesis who does a lot of quantitative research. I decided to work with her specifically to learn this methodology because I knew I was less comfortable with it and that it could be useful. She shared a story with me about her work/decision to do this kind of research, and it had to do with the ways in which you can move data around and look at different pieces in relation to one another via quantitative methods (and pivot tables in excel, specifically). She talked about a previous project in which she was looking at online communication practices. When she collected the data and entered it into the computer, she was able to play around with it a bit and in doing so she found all sorts of really fascinating data that suggested the differences between men/women in their online writing practices. This was not at all her project, but it quickly became a new one because it was something that she hadn’t even thought of focusing on before, and seeing it in the numbers made her think of the possibilities in new arguments. In this case at least, it seems as though she could argue something that would have been muted or invisible otherwise.

I’m not saying I particularly like or prefer quantitative research at all (in fact, I’m really excited to learn more about ethnographic methodology and teacher research and I think my work will primarily follow this kind of methodology). That being said, I feel like quantitative research can, if nothing else, encourage us to consider things that we wouldn’t have thought to look at before. Maybe quantitative research can be used to start new projects and show us where qualitative research can/needs to begin? Are there ways in which quantitative data can help us develop further research questions, interview questions, etc., in the way that the above example sparked an interest in the effect of gender on online vs face-to-face practices?

Thanks for reading,

Becky

With Jim’s permission, I share the following part of his response:

Hi,

Thanks for your excellent questions. my response is in your text. you have raised incredibly important questions Rebecca–i wish i could spend more of class on these. but given the seminar’s objectives i cannot. i am however VERY happy to talk more with you about this outside of class –in my office. anytime. i am copying my friend Nancy Mack because we have been battling this beast since 1982–40 years. it is a hydra–cut off a tentacle and it grows ten back in its place. youll see what that is like around 2052! remember me then!

Jim

As I read the rest of his response within my own text, I felt like I was joining the field in a new way and beginning to see the ways in which tensions, disagreements, and a variety of approaches manifest within our method(ologies). This email exchange was one of my first experiences as a participant in a scholarly dialogue—a back and forth exchange, a playing out of how one might look at things from other perspectives, a “thinking through” (Zebroski).

Since my first year in the program, I have participated in different kinds of research. I have conducted mixed-methods surveys that have both quantitative and qualitative components, but sometimes these projects seem to be more about the numbers than the people. In contrast, I have pursued qualitative projects that value context and people, thus resisting any urge to reduce people or their writing or their experiences to numbers. In all of these projects though, I have tried to be reflective enough to consider—who benefits from the kind of research that I’m doing? What is valued? And, at the front of my mind, I’m wondering still—is eclecticism possible? Can our projects mix methods in some egalitarian kind of fashion, or are we always privileging one method over another?

Ezekiel’s Story

As an undergraduate WC employee at The Writing Center @ MSU (Michigan State University), I had various experiences engaging with different aspects of WC research. One of the most useful experiences I had in developing an understanding of the context of our WC was working with the assessment committee.

While participating with this committee for 1 ½ years, I learned how to normalize data, interpret student responses, synthesis consultant narratives, and build both conference and administrative presentations. I worked for the WC @ MSU for five years and finished two degrees; during that time I developed fundamental relationships that helped shape who I am as a researcher. Working with Trixie Smith was a great opportunity to learn how to do WC research in a pragmatic and ethical way, but more importantly, it became more and more apparent to me that the goals of all of the research we did at the WC @ MSU was for the students we helped. Student focused research that appealed to the deans and improved the ways that we could reach our student and community populations.

It was late in my senior year and into my MA that I started to engage with cultural rhetorics and began trying to envision how to bring these theories to WC research. I began wrestling with questions like what a decolonial WC would look like or how it would function? Or the desire for WC spaces to “feel like home” and whose home that is and who is it favoring?

I began to consider in earnest the onus placed on international or multi-lingual speakers by the university and the ways economics felt more important than respectful treatment/understanding. Along with a few of my fellow graduate and undergraduate colleagues, I tried to understand the relationships race plays in our center and what the outcomes of this were.

As these questions developed, I began taking the writing center administration graduate seminar taught by our director Trixie Smith and our associate director Dianna Baldwin and found the space to push what it means to do WC research. I learned the necessity of being able to craft a mixed method research design that considered the stories of our population and of statistics that showed our year-to-year growth.

Our Approach

As we began to develop this brief qualitative study, one of our goals was to also provide a model of what qualitative research can look like. Beginning with our own stories and experiences with qualitative research, we developed a series of questions to ask self-described qualitative WC researchers to get at their stories and experiences, utilizing narrative inquiry (to develop the questions) in constellation with oral history methods (to hear and understand their stories).

After conducting our interviews and listening to the data, we knew that we needed to preserve as much of the stories as possible. To engage with our readers, we have provided a multi-modal approach for this project as a way to show (rather than just tell) the pro’s and con’s of an oral history style of qualitative analysis. Further, we draw directly from the words of our participants to make connections to the complex writing and research situations WC directors and researchers face. Both of these moves are intentional to demonstrate the necessity of qualitative research in a WC context. We utilize the narratives of our participants to develop a baseline for what interviewing (and qualitative research as a whole) can provide researchers in terms of methods of data collection, analysis, and synthesis.

The Story of Our Methods

Three self-proclaimed qualitative writing center researchers were chosen to be interviewed for this project: Andrea Scott (assistant professor of academic writing and faculty director of the writing center at Pitzer College); Jackie Grutsch McKinney (professor of rhetoric and composition and director of the writing center at Ball State University); and Michelle Miley (assistant professor of English and director of the writing center at Montana State University). These researchers were selected in part because of their commitment to qualitative method(ologies) made evident at their roundtable presentation “Studying Writing Center Stories: Methods for Qualitative Narrative Inquiry” at the 2015 IWCA Collaborative at CCCC, “The Coallboratory.” The theme of “The Collaboratory” was RAD research, and these scholars put together a panel which sought to “capture narratives as data” via ethnography, survey, and interview projects.

Using oral history methods, we generated a list of interview questions designed to get participants to tell us their stories. Specifically, we asked participants to discusst their earliest experiences with qualitative research, to talk about the research method(ologies) within qualitative research that inform their research, to discuss both the values and limitations of qualitative research, and to offer advice for new writing center researchers who are interested in qualitative methods. Questions were sent to participants in advance of the interview so they could prepare and think about the stories they wanted to tell; the interviewer then asked follow-up questions as the participants told their stories rather than closely following a set of previously established questions, which can cut off stories and shut down interviews.

Our interview questions included:

How long have you been conducting research in rhetoric/composition? In the writing center?

Tell me about your first qualitative research project. What were you hoping to learn? How did you do it/conduct it? (or what methods did you use?)

Describe how you situate yourself within the field (rhet/comp or WC) in terms of method(ologies)?

How did you come to be this kind of researcher?

What are the values/benefits of the qualitative research methods you practice?

What are the limitations, shortcomings, and struggles of the qualitative research methods you practice?

Describe the research project you learned the most from.

Why is qualitative research important to rhet/comp and WC studies?

What advice do you have for new researchers/scholars starting their first major qualitative research project?

Are there any new qualitative methods that you’d like to/plan to learn about?

Are there any other disciplines that you draw your method(ologies) from?

What scholars (both inside and outside of rhet/comp) have had the largest influence on you? Why?

Each interview lasted approximately 45-60 minutes and was recorded via phone using an iphone application called Call Recorder. The same interviewer conducted all interviews, and audio clips were then emailed to the interviewer who generated mp3 files for analysis.

After interviews were completed, the interviewer listened to each interview and created an interview log, which is “a detailed table of contents for the entire interview” (Lindahl 2013). The log follows the order of the interview and focuses on the interviewee’s words, rather than on the questions of the interviewer. Rather than creating a full transcription of the entire interview, the interview-log approach creates an entry for every one to four minutes of recording. Entries usually begin when the interviewer asks new questions but often occur more frequently than that, especially when interviewees go their own ways or digress. Each entry is marked with the time when the interviewee began speaking, and short sentences are used to describe what is said along with quoted parts when it is necessary to preserve language choice. As the interviewer created the log, she noted parts that seemed the most valuable, which are then often transcribed. Here, those sections appear as sound clips in the section below.

After completing logs for each interview and reviewing each interview’s content, the interviewer looked for patterns, commonalities, points of departure, and omissions and selected the sound clips below. The interviewer found that stories about first qualitative research projects, definitions of qualitative research, approaches to qualitative research, limitations of qualitative research, and advice for new qualitative researchers came up in each interview. Once logs were completed and these moments were noted, the interviewer imported each full interview into a separate Audacity file. Clips were then cut into shorter sections and sometimes combined when different moments of the interview addressed the categories above. For example, if the interviewee provided definitions of qualitative research at two different moments, the interviewer combined these moments into a single audio clip. Other than this, the interviewer made few edits to the interview clips themselves, other than eliminating long pauses and coughing.

Reflections on methodology

As the interviewer conducted the interviews for this article, she learned a few lessons. While she had prior experience conducting face-to-face interviews with people she didn’t know well prior to the interviews themselves, this project was different in that she knew two of the three interviewers before the study, and one of those two very well. Since the “Results” section of this article focus more on the interviewees and what they had to say, the interviewer wanted to include some of what she learned about her own presence within the interview. The clips below provide you with an idea of her voice/role in the interview itself and offer advice for those who may conduct phone-based distance interviews or interviews with people they already know in the future.

Reflective Lesson 1: When you interview someone you know about a story you know, remember to ask for details and clarification that they might forget to retell you.

Reflective Lesson 2: Find a way to let the interviewee know that you’re listening when interviewing on the phone. Since you no longer have body language and non-verbal communication as options for indicate active listening, interviewers need to find other ways to let the interviewees know they are engaged listeners.

Reflective Lesson 3: Even if you think you’ve tested your audio equipment, test it again, especially when working with new tools. When attempting to record audio from two separate devices (like two computers via Skype), be sure to test audio recording tools across two machines. In the interviewee’s first interview with Andrea, she recorded only her own audio and wasn’t able to pick up anything from Andrea’s end.

How Qualitative Research Happens in Writing Centers: What We Learned

In these interviews, we can see how Andrea, Jackie, and Michelle utilize qualitative research as a method for understanding the narratives of the people involved in writing research, ultimately leading them to better comprehend the why of research. We hear stories about early qualitative projects and how they shaped future approaches; moments where interviewees speak to the difficulties of interpreting qualitative data in a way that can be meaningful to administrators, WC directors, and academic publishers; and advice for new researchers that centers on maintaining s sense of patience and flexibility, both with the research projects and people involved, but also with ourselves as researchers.

Although we believe that these stories are meaningful compositions individually, we’ve decided to break them up into pieces based on common patterns and themes that we noticed as we listened to each interview alongside the others.

Stories about first qualitative research projects

Each interviewee discussed her first qualitative research project in the field of composition. Andrea describes how her background in comparative literature and research experiences in Germany led her to a current, ongoing project where she translated the survey questions about writing centers from Jackie’s book Peripheral Visions for Writing Centers for a German audience. In her story as a new researcher in an MA program, Jackie reflects on her very first qualitative research project in which she observed two first year composition classes and looked at how gender plays out in the teaching of composition. Finally, Michelle describes how her collaboration with a faculty member in the College of Technology at the University of Houston led her to move from a quantitative analysis to a more qualitative analysis of writing studios.

Andrea:

Jackie:

Michelle:

Definitions of qualitative research

Each interviewer shed light on what qualitative research can do that quantitative cannot in their own way. Michelle describes how qualitative methods strengthen the tie between theory and practice by understanding how theory is grounded in a WC context and how we can use qualitative methods to understand how we think about research. Andrea speaks to the theory and practice divide, stating that qualitative methods can provide knowledge for knowledge’s sake. Jackie relates that qualitative methods can allow researchers to consider macro concerns in regards to effective tutoring.

Andrea:

Jackie:

Michelle:

Approaches to qualitative research

Intrinsically tied to the process of using qualitative methods is the process of learning from failure, learning which questions to ask, figuring out how to code data, how to share the narratives, etc. Each interviewee describes this process in a different way. For Andrea, it is the ongoing process of effective translation, across cultures and events. For Michelle, it is the process of situation and when to utilize qualitative data (tutor trainings and conversations concerning why we do what we do in WC spaces) and how to adapt and use institutional ethnography as an approach to writing center research in new ways. Jackie describes how she uses narrative inquiry as a way into writing center research and the ways in which we can learn from storytelling.

Andrea:

Jackie:

Michelle:

Limitations of qualitative research

There are moments when each interviewer speaks to the difficulties of interpreting qualitative data in a way that can be meaningful to administrators and other WC directors. Andrea describes how any argument is an interpretation and explains that we can find ourselves in “epistemological despair” trying to figure out “how we know what we know.” Yet, Andrea explains that this kind of partiality is in a sense part of all research. Jackie describes the challenge of having a wealth of data and turning that into an article-length piece of scholarship, which often involves “grieving the loss of data.” She raises the question, “How do you decide what to include?” In addition, Jackie notes the amount of time it takes to conduct qualitative research and the ways in which it is difficult for researchers to study writing centers to which they don’t belong because of time constraints and, sometimes, because of a lack of trust. Similarly, Michelle notes the time factor as a limitation but also argues that taking time to do research carefully and reflectively is worthwhile. For her, this is a way in which qualitative research may be especially valuable over more quantitative methods that attempt to crunch numbers quickly.

Andrea:

Jackie:

Michelle:

Advice for new writing center researchers using qualitative methods

While these researchers all provide valuable evidence for new writing center researchers using qualitative methods, their stories seem to suggest that getting into the field and practicing is one of the best ways to start learning. Andrea argues that new researchers should take advantage of any opportunity to meet with scholars whose work they admire and to ask for feedback as projects are in process, especially in terms of methodologies. She also suggests that joining a writing group and getting feedback from other researchers is valuable. Michelle reminds new researchers to practice lots of forgiveness. She explains that there is no right or wrong way to do qualitative research and acknowledges that this kind of research is hard because in some ways it may seem less structured. Similarly, Jackie says that your first qualitative research project doesn’t have to be perfect or change the field; this kind of research is often still extremely valuable in terms of helping writing centers learn something and also helping us as researchers learn about our own approaches and tendencies. She argues that learning how to talk to people and become a people-based researcher can be tricky and take some time. She also emphasizes the importance of always testing recording equipment.

Andrea:

Jackie:

Michelle :

The results of qualitative research can be hard to condense into codified data, and there is often an overabundance of data for a given project, but as was often hinted to or explicitly stated, qualitative research affords WC directors and researchers a method of understanding why we do what we do. The importance of narrative cannot be stressed enough in relation to understanding how and why WCs function the way that they do because it is through these stories and interviews that the unquantifiable shapes meaning. Hearing how and why WC services are effective (or not) through narrative provides a view into our work that numbers cannot and strengthens the argument for why WC work is needed and what WCs can and cannot do. Additionally, while all of the stories and specific points of interest may not fit in a given article, those sound bytes or paragraphs are still available for other uses, such as annual reviews, internal WC assessment, and outreach campaigns.

Moving Forward with Qualitative Research in Writing Centers

The most consistent implication derived from these interviews is the necessity for mixed methods to gain a full perspective in future WC research. This means that there is a specific need for quantitative data (often times when working with administrators) and an equally important specific need for qualitative data (data gathered for the scope and effectiveness of programs and understanding the context in which student writers compose). In WC spaces, understanding context and the needs of student writers is one of the most important considerations when developing future programs, and through the utilization of qualitative research, WC directors can understand what student writers need and how to provide (or build) these services.

Additionally, by addressing the question of why we do what we do with qualitative research, we can resist programmatic paradigms of the fix-it shop or copy editing services by providing narratives that speak directly to the needs of students. These narratives in relation to numerical data build the space necessary for WC research to progress and continue to provide services that make a difference in client/student writing. These narratives also afford WC directors a method of understanding the consultant base and the context through which tutors and consultants do their work. While this may seem a small point, the process of reflection in WC consulting can often push our tutors and consultants to develop flexible ways of working with clients that meet the student writers where they need. These narratives can also provide the foundation for discussing shifts in programmatic goals, if necessary, to best understand the context of our client base.

Further implications include the development of systematic ways to incorporate qualitative data in administrative reports, building a robust knowledge of specific writing contexts through narrative, and the development of procedures to relate quantitative data with qualitative data.

One resounding reflective observation with which we would like to close this section is considering where our stories fit into these equations. As authors, we started with our experiences and the ways that they informed our approaches to research. After writing multiple drafts, these stories continued to burst out of the seams. We began with our narratives as a way to demonstrate the process of developing research questions as well as to embed ourselves into the research we do. When starting a WC orientated research project, we strongly encourage our readers to begin by engaging with reflection and introspection in an attempt to find what motivates their questions and research. The practice of trying to articulate your own positionality in relationship to your research is a valuable and necessary within the research process and one that we argue should be a part of the research text as well. A researcher’s positionality may find its way into an annual review or an essay that is later published, but at the least it will help guide the ethics of the research.

Closing Thoughts

The experiences of our interviewees show us that qualitative research can be a difficult and ongoing process. What works for one project may not work for another, but at the heart of qualitative research is the goal of understanding the unquantifiable emotions and experiences of the people who work and learn in writing centers and to find better ways to incorporate these phenomena into our research models. We are not saying that qualitative research is superior to quantitative research. Instead, we are pushing for the inclusion of both methods to gather data that better represents the complex environments of writing centers. We intend to help students with their writing, but what this means is different depending on the context. What we are hoping for in this essay is to start a conversation about methods for uncovering and developing the stories of our centers and methods for discovering the relevant contexts of specific writing centers as well as about the means to understand how and when contexts change and how such change can affect our work. This preliminary research indicates that WC research methods have to be adaptable for our work to remain relevant.

LaFrance, Michelle and Melissa Nicolas. “Institutional Ethnography as Materialist Framework for Writing Program Research and the Faculty-Staff Work Standpoints Project” College Composition and Communication 64.1 (2012): 130-150.