Friday, January 29, 2010

A final exam on grading

Teach Philosophy 101 has a thought-provoking "final exam" on grading, aimed at instructors. All of the questions raise important issues about grading, but here are a couple of the exam questions that struck me. Curious to know how ISW readers would answer them:

3. ESSAY QUESTION: Student X writes an excellent paper and gets an A. Student Y writes a poor paper but takes advantage of the offer made to all students in the class that they may rewrite their paper as many times as they wish. After several rewrites (responding to the teachers comments each time), student Y finally produces a paper that is as good as student X’s original paper. Is it appropriate for them both to receive the same grade? Does that send a misleading message to external audiences?

7. TRUE OR FALSE?: Although many teachers say that they want students to learn higher order mastery of the material (such as understanding and applying the material), they often grade their students on material that is less significant but easier to grade.

12 comments:

3. It is appropriate for both to receive the same grade. We are ultimately judging the quality of the work. Therefore, since the two papers are of equal quality, they should get the same grade.

I'm genuinely unsure what 'misleading message' might be sent by this. That hard work doesn't pay off? Surely not - student Y greatly improved his/her grade by hard work. That hard work is not required? Again, surely not - just because X did not go through several drafts does not mean s/he didn't work hard on the paper. Different people (not just students!) take very different lengths of time to write papers, but the results can be of equal quality.

7. Yes. From my (fairly limited, thus far in my career) exposure to the exams (in particular) given by other faculty, I've seen a lot of questions that asked about very trivial topics. (Such as, 'Which philosopher made the following quote'?)

3. I don't have a problem with giving the same grade. I would have serious worries about students knowingly turning in inferior work thinking they would turn in the 'real' paper sometime in the future, especially for lower level courses. This could easily lead to unnecessarily increasing my workload toward the end of the semester.

So I might consider something like this: you can revise as many times as you want, but to do so you need to meet with me after I return the paper to discuss the strengths and weaknesses and have the revisions done within a week.

Otherwise the "I can do this later" mindset could take over.

7. I test for (carefully selected) quotes in my intro level courses. I do this with multiple choice questions with carefully selected answer-choices. I don't think it is any more trivial than expecting intro level biology students to know definitions. It is simply the necessary groundwork to have decent discussions and understanding of the material. A student who doesn't know that Locke believes we should leave "enough and as good" or that Kant (as opposed to, say, Hobbes, Bentham, or Hume) talks about good will doesn't know the material and can't be said to understand it.

3. It is not alright for both of them to receive the same grade. The very idea, to me, seems preposterous. And the reasons are as follows:

(i) If students are allowed to rewrite their papers as many times as they want, then arguably one could and should conduct written tests/exams that don't impose any time restrictions. In an hour-long test, say, why penalize student Y if she couldn't finish the test on time but could have had if she'd been given two hours instead? Why not have day-long tests for that matter?

(ii) The notion that it is admissible to revise one's work as many times as possible (without due respect for any kind of time frame) until one gets an A grade is a harmful one to the student, for it doesn't prepare her well for the exigencies in the real world. Almost all tasks (with the possible exception of philosophizing) in the real world require that they be completed within a given time-frame, failing which the consequences are usually disastrous/unpleasant/unwanted/bad.

(iii) What incentive does Student X have in producing an excellent paper in one go if she can just do what student Y does and still get an A? Almost nil.

In order not to penalize students who may require more time than others in writing papers, it is up to the instructor to make sure that all students get reasonable and ample amount of time to finish the task. The time-frame for writing/submitting a paper can be set after consulting with the students. Also, in my opinion, it is a good idea to set markers. For instance, date X1 could be set as deadline for showing an introduction (and perhaps, an abstract), date X2 for showing two-three pages of the body, after which there could be some peer-reviewing amongst the students. And, so on. The idea is to pace the task of writing a paper. This will also help students who procrastinate indefinitely in finishing their work on time.

Interesting quiz—and I do think you've picked the two most interesting questions.

On (3), I'd argue that it is not appropriate for both to receive the same grade. I agree with the reasons given in earlier comments, but I also agree with the reason implied by the quiz. The "misleading message to external audiences" (i.e., those looking only at students' final grade for your course) is that these two students are equally capable. Perhaps Student Y has significantly improved his or her paper-writing ability in the revision process, perhaps not. If so, that will show up in grades on future papers (in your course or others). Someone deciding which student to hire or admit to graduate school would be interested to know that only one of them does things well the first time.

My solution to that kind of problem is to calculate the overall grade for the paper as a weighted average of the original and revised papers.

On (7), I plead ignorance. But I agree with Dr. Spence's point that testing "rote" learning is sometimes appropriate, just as it is in the sciences. Besides, it seems to me that the student who can tell you whether Kant or Mill is a consequentialist deserves more credit than the one who can't, even if he or she has a hard time, say, reasoning in a consequentialist way. The problem would be if an exam tested for nothing but rote learning.

Regarding the quotes: that was just one example, and hastily described. I had in mind questions where the quotes seem to be chosen with little regard for their representation of the author's general views.

If students are allowed to rewrite their papers as many times as they want, then arguably one could and should conduct written tests/exams that don't impose any time restrictions. In an hour-long test, say, why penalize student Y if she couldn't finish the test on time but could have had if she'd been given two hours instead?

This seems like a non-sequitur. Essays aren't like tests. The goal of a test is (I would say) to assess the grasp of some basic principles, ideas etc. The goal of an essay assignment is to see if the student can make a coherent argument for a thesis. (I assume we're not dealing with an essay exam. I've argued against those on this blog before...) In the latter, taking an extended amount of time and discussion can very appreciably improve the result. In the former, generally speaking, giving the student more and more time will not allow him or her to appreciably improve his or her score. If s/he doesn't know the answer after one hour, in most cases allowing another hour won't help. (I assume that the problem is not just that the student did not have time to give attention to all the questions.)

The notion that it is admissible to revise one's work as many times as possible (without due respect for any kind of time frame) until one gets an A grade is a harmful one to the student, for it doesn't prepare her well for the exigencies in the real world. Almost all tasks (with the possible exception of philosophizing) in the real world require that they be completed within a given time-frame...

Allowing students to revise their work as many times as they wish is not equivalent to abandoning time-limits. There will still be a due date (or, ultimately, the end of the semester).

I'm afraid I do not understand your third comment, Vishal.

The "misleading message to external audiences" (i.e., those looking only at students' final grade for your course) is that these two students are equally capable. Perhaps Student Y has significantly improved his or her paper-writing ability in the revision process, perhaps not. If so, that will show up in grades on future papers (in your course or others). Someone deciding which student to hire or admit to graduate school would be interested to know that only one of them does things well the first time.

I don't agree, David. There is no penalty in grad school for not being able to produce excellent work on the very first draft of a paper. (Unless one then simply submits the first draft without revising it!) Some can do this, some cannot. I myself take a long time to produce my papers. Surely the ultimate quality of my work should not be judged on how long it took me to produce it, and how many people I talked to about it. Even if it's not true that slow and steady wins the race, surely it's true that slow and steady writers can achieve just as good results as fast and furious ones?

Finally (sorry for the long-windedness of this comment), let me advance another consideration. The notion that one should, in grading a paper, take account of the process that produced the paper, seems dangerous to me - because we will often have very imperfect access (at best) to what that process was like. Even if student X produces an excellent paper on his/her first try, one will not know how many drafts that student went through that s/he did not turn in to you for comment. (I'll add, though, that I always require a draft, which in part serves as a way to discourage plagiarism - one 'process' that I am very concerned to be aware of!)

Essays aren't like tests. The goal of a test is (I would say) to assess the grasp of some basic principles, ideas etc. The goal of an essay assignment is to see if the student can make a coherent argument for a thesis.

Thank you, but I am well aware of the difference between writing a 1-hr (or a 2-hr) test and writing a paper. Indeed, the latter requires that a student present a coherent and cohesive argument to support a thesis, but that doesn't mean she should be allowed to rewrite (and that's the operative word, here) her paper as many times as she wants until she gets an A. That's a luxury only the academia can provide. In the real world, one doesn't get that many chances.

Having said the above, the student is always free to sit with her instructor and go over her draft(s) as many times as she wants (time/resource permitting) but when the final version is presented, she shouldn't get to decide that she requires more time after all to write a still better paper. David morrow's suggestion that "... the overall grade for the paper... [should be]... a weighted average of the original and revised papers" seems like a very reasonable and fair one.

Allowing students to revise their work as many times as they wish is not equivalent to abandoning time-limits. There will still be a due date (or, ultimately, the end of the semester).

Once again, I think revising drafts as many times as student Y wants (within some stipulated time period) is quite alright, but after submitting the final version, the student shouldn't get to choose to rewrite the paper because she didn't get an A. The student should have or develop the ability to judge the quality of her writing and not have her instructor tell her every time that she could do better to get an A. That would be akin to being spoon-fed.

There is no penalty in grad school for not being able to produce excellent work on the very first draft of a paper.

Grad school is an entirely different ballgame. I thought Dr Spence was talking about paper submissions made by undergrads.

There are several arguments against allowing a student to improve a paper grade with additional / as many as you want drafts.

Fairness to all students. This I think is a weak argument. An instructor can lay out pretty much whatever they want in a syllabus. So long as the material and means of evaluation are appropriate for the students who generally take that course, fairness isn't an issue. It isn't unfair that philosophy majors have an advantage in a philosophy course, and it isn't unfair if the instructor sets things up so that non majors (or those in need of writing help) aren't disadvantaged. If one student wants to put the work in on the front end, and another wants to work on it over the semester with feedback from the prof I don't see this as being unfair if the policies were known to all and announced in the syllabus.

Preparing them for the real world. I agree this is a real consideration, but I don't think it is decisive. It is only one objective, however you structure your deadlines will suffice, and there is no reason to think that each assignment should aim for the same teaching objective. I use both objective exams and take home essays. If I were to adopt this policy for the take home essays, my students would still get a very good dose of 'real world' with the objective portion of the exam.

I think fairness to students who may not have a solid understanding of how much work it takes to write a good paper is a more important consideration that these. I can easily imagine a student putting things off, to their detriment.

On the other hand, I can think of good arguments for it being an acceptable practice.

Fairness to students who may not have good writing skills. We are teaching writing as well as philosophy. I should say that I have been assuming that we are considering this policy because there are students who need to work on their writing and who may not understand how much work it is to write well. I would never consider it as a way to help students who can write well, but turn in a crap paper anyway because they can work on it later.

I don't have this policy, but I don't see anything wrong with it. Effectively it is no different from having a paper due at the end of the semester, allowing students as many drafts as they want, and also letting them turn the final draft in early if they want. At least, if you are willing to tell them what grade you would give the draft.

With respect to this question Which of these is the most important purpose for assigning grades? I would say Document student knowledge and abilities for use by others, including administrators, employers, and graduate or professional school admissions departments.

The point of the course though, is to help them acquire that knowledge and ability, and this policy might help them do that.

Aha. Vishal's second post has led me to see that we're talking at cross-purposes. We're interpreting the original question differently!

It says:Student Y writes a poor paper but takes advantage of the offer made to all students in the class that they may rewrite their paper as many times as they wish.

Vishal (and everyone else?) - you're taking this to mean that Y submits a 'final' paper, gets a grade on it, and then revises.

I was taking it to mean that Y submits a draft, is told (in instructor's comments etc) that it is poor, and then revises.

The question is at least mildly ambiguous between these two, but perhaps your interpretation is what was meant. In which case, I want to change my answer. This is not something I would do in my classes at all. It does indeed send a misleading message! (I do not even assign grades on drafts. I just make comments.)

(Vishal, in my final remark about grad school, I was addressing a comment by David Morrow, not Dr. Spence.)

Thanks for your clarification. It does seem now, after all, that the two of us are more or less in agreement on this matter. Though it is possible that my interpretation of the original question is incorrect and yours correct.

I just think that if student Y shows a (poorly written) draft of her essay to her instructor, then the latter may goad her into coming up with a better version through statements such as, "Here is premise A and here is conclusion C that is supposed to follow from A, but it seems it is not entirely clear why that should be so..." or "Perhaps, you could offer a more coherent argument in paragraph 6..." and so on, but not say, "This paper of yours will get you a C grade, so you might want to rewrite it..." On the surface, the two possible approaches (apropos paper evaluation) mentioned above seem alike, but, to me, there is a world of difference between the two.

I was interpreting the question differently than you. As you've surmised, I was imagining that student Y turned in a final draft and then rewrote it. I strongly agree with you that students should not be penalized for drafts they turn in prior to the "final" due date.

You make a very good point about the importance of speedy writing in grad school (and professional academia, for that matter). Since the ability to do quality work on short deadlines is still important in many other contexts, though, I still think a weighted average is appropriate.

I'm rather surprised that anyone has a problem with Student X and Student Y receiving the same grade, even on the assumption that they get a grade prior to revision, given that the scenario is explicitly that all the students have the same opportunity. Our task as teachers is to grade on whether students have learned to do what they are supposed to have learned to do; nothing else is relevant. The "real world" is not an environment designed specifically for learning, so it is not any more relevant to classroom practices that people often only get one shot at things than it is that people usually can't get ahead simply by doing well on tests; but in fact the behavior that is being reinforced is revising in order to improve what one has written, which even in "real-world" terms is behavior that should be encouraged, not discouraged. Even further, in academia there still is a deadline: the end of the course. And when that fact is taken into account, it becomes clear that Vishal has it exactly backwards: in most "real-world" situations, what matters is getting things right by the deadline, not getting things right on the first shot. There are exceptions, of course; but if we are going to try to hold academia to the measure of the "real world" rather than its own goals, the practice doesn't actually diverge all that sharply from the "real world" -- there are plenty of "real world" analogues.

On incentives, students like student X has exactly the same incentive to produce excellents papersthat led them to write the excellent paper they in fact wrote; the only way that would be modified is if they were more interested in outcompeting other students in the class than in getting the grade they want, which does not cohere with my experience of most students' attitudes.

And, quite frankly, if students don't get a specific incentive for revising their papers, it is students who don't do well right off who are not getting the right incentives -- their incentive to improve their writing skills is very weak. There are other ways to work this into the assignment than the one in this scenario, but any instructor who does major writing assignments, especially for lower-level classes, but does nothing specifically to encourage students to refine and revise their work, is falling down on the job of actually teaching.

So I see nothing wrong with the scenario, as such. Whether it was a prudent course structure, of course, will depend wholly on what is actually being taught and on the particular focus of the course (as determined by the objectives of the department and the instructor).

If you wish to use your name and don't have a blogger profile, please mark Name/URL in the list below. You can of course opt for Anonymous, but please keep in mind that multiple anonymous comments on a post are difficult to follow. Thanks!