World History MOOC Report 9: In which I prefer peer evaluation to peer grading.

29102012

For the uninitiated, let me try to explain the process by which the assignments in my world history MOOC work. Students have a week to answer one of three questions with a 750-word essay. The guidelines for writing the essays stress all the important things: Evidence, argument, not plagiarizing. So far, I think, all the questions have been one sentence long.

You can type your essay right onto the Coursera site, but I’ve found it much easier to write mine in Word and cut-and-paste it into the box once it’s ready. Before you submit, you have to pledge that your work is your own. You can even go back and change your essay after you’ve submitted it as long as it’s still before the due date.

Once you’ve submitted your essay, it’s your responsibility to grade the essays of five other students over the course of the seven days following the deadline. You use the 0-3 scale that I’ve mentioned earlier. You award three grades: one for evidence, one for argument and one for exposition. There’s a space for anonymous comments, and also a place where you can leave your initials and your location if you so choose. Once you’re done, you see your essay again and you do the same thing to your own essay. You can even leave yourself comments! You get your grade back a week after you turn in the paper. I haven’t paid much attention to the math involved, but every single grade (your peers’ and your’s) is used to determine your final numerical grade.

On the one hand, there’s something incredibly appealing about students anonymously reading other students’ papers. I do something like this all the time in class, and am constantly stymied by the unwillingness of students to be critical of one another, especially when the author is in the room. This system could prevent that. It also saves the time during class that students spend reading each others’ work. This way they’d get more and probably better evaluations than they would ever get in a 50-minute class period.

On the other hand, when you move from peer evaluation to peer grading I still have many problems. I find the notion that students can grade their own work even for just a portion of their eventual final grade and get college credit for that to be too utopian to take seriously. And as I’ve already suggested before, there’s something insulting to professors about any system that suggests that anyone can grade. Let me explain how this played out in practice for me on the second assignment in order to illustrate my point.

The last one of my five peer papers was my first example of plagiarism. As a huge fan of Laura Gibbs’ Coursera Fantasy blog, I was fully prepared for such an occurrence. Even if I hadn’t expected this, just think about the audience. As I learned last summer while teaching in South Korea, there are huge differences in what constitutes plagiarism across cultures.

We students got very little guidance as to what plagiarism is. I spend at least ten minutes in class a semester on this subject, but all I remember reading was a few lines at the bottom of the writing guidelines and what’s in the pledge. Why spend more time on plagiarism? My example was hardly clear cut. Unlike the usual examples of plagiarism that I see when grading, a single word that I know the average college student would never use in any context did not set me off. This time the alarm bell sounded because the essay was almost completely off topic. When I Googled a uniquely-constructed sentence, I came to a year-old blog post. While the “works cited” section listed that blog post and noted that the author and the student were in fact the same person, none of the words duplicated in both essays had quotation marks in the assignment.

Can you plagiarize yourself? Leaving aside the actual answer to that question, where does the average MOOC student go for guidance if they’re faced with this kind of dilemma? The whole point of a MOOC is that the machine is supposed to run itself, but effective grading requires knowledge and experience that students and their peers don’t have. I decided to give the essay mostly “1”s for its off-topic character and to explain that some people might define this as plagiarism in by far the longest comment that I’ve written for this course.

To fully understand my decision here, you also need to understand what I think the problems are with the assignment itself. Jeremy tells us that a lot of thought went into the design of these assignments, and again I believe him. [It’s the parts that resemble traditional history classes that I don’t think have been changed enough.] However, this kind of assignment has the feel of being designed by committee because it doesn’t know what it is.

Is it the equivalent of a blue book essay? Sort of, but there’s a section for “works cited” at the bottom of the paper submission form. Not footnotes. Just works cited. Is it a research paper then? If so, why are we frequently reminded that we need to rely on the knowledge that we all hold in common from class in order to make the peer grading system work? But how can we cite the textbook then if the textbook is not required reading for the course? If forced to choose, I’d pick blue book essay over research paper if only because I wouldn’t want to teach proper footnoting techniques to 82,000 people. In my case, I pretty much have to treat it like a blue book exam essay since I don’t have the textbook and the forums aren’t really helpful for answering the essay questions. No matter what, it really should be one thing or the other.

So how’d I do this time around? Solid rock star all around (thank goodness). However, I can only find one comment from any of my peer evaluators. Does this mean that four of my peers couldn’t be troubled to leave comments? Does this mean that only one person evaluated me? I’m not sure, but either way it is pretty clear that peer grading isn’t quite working out the way its creators anticipated.

Available now!

"Rees has written a solid, comprehensive account of the technological creation of cold chains in the United States. I wish this book had been available for me to read when I was doing my own research."—Mansel G. Blackford, Ohio State University.