Friday, August 21, 2009

Rubrics Were Great

Let me start by saying that this post reflects nothing more than my opinion as a working full-time classroom teacher. This is not a piece of educational theory, it isn't some policy report, and it certainly isn't some measure that I expect teachers to adopt immediately.

It's just the way I see things.

And the way I see things is like this: a rubric is an insult both to the intelligence and creativity of a student.

Last night, way too late perhaps, I got sucked into a Twitter discussion about rubrics. Arguments for and against were put forth and examples were given. This morning, the Twitter chatter continued as I heard about individual teachers' feelings for and against rubrics as well as situations where state education boards demand rubrics. And something that keeps coming up is the idea that rubrics are a) transparent and b) objective.

And I would argue that neither is true. (Add: 10:59AM EST -- Or rather, the former isn't true and the latter is of questionable relation to reality let alone the purposes of education).

First of all on the issue of transparency. Most rubrics come in one of two varieties. Either they are extremely didactic in a step-by-step hold-your-hand IKEA instruction manual sort of way or they are touchy-feely rubbish where you get a '1' for 'not demonstrating significant understanding' but a '5' for 'demonstrating unique depth and content mastery'. Rubrics of the latter variety are meant to satisfy the political needs of institutionalized learning, while rubrics of the former are theoretical expressions of teaching to the lowest common denominator.

What does any of this have to do with 'transparency'? Looks to me like it has everything to do with a dog-and-pony show. In these situations, the rubrics come off as more an insurance policy so the teacher scores well in an observation than anything else. And the argument may be made that this sort of rubric helps the student understand what it is that the teacher wants... which brings us to my next criticism.

Objectivity.

I don't want students to do 'what I want'. I don't want students to follow 'objective' rules. In fact, that's entirely the type of behavior I'm trying to break my students out of.

For twelve years, we condition students to follow rules. We teach them that if you do A, B, C, and D, then you will make the grade. We give them rubrics so that they can check off that they did A, B, C, and D and we assign grades and we call this education.

Who are we fooling?

I should step back a moment to give some context. I'm not some guy talking out the side of his mouth about this stuff. I understand exactly how rubrics work. I've twice worked on committees designing rubrics. I understand that on the surface, it appears -- and even seems to make some sense -- that rubrics would be the best option. After all, what's the alternative? Just telling the student you want a project done and not giving any guidance?

And I think that actually is the red herring.

The red herring is that rubrics are helping the student learn. I'd argue that rubrics -- if anything -- are teaching the students that education is just a matter of completing tasks on a checklist. I'd argue that rubrics are teaching students that if they complete the tasks as stated, they should expect success.

Except life doesn't work like that.

Life is more complicated. Could you imagine Socrates handing Euthyphro a rubric? I think it's far more likely that rubrics would have been the butt of Aristophanes' jokes: another example of how sophists con folks into thinking they understand things.

To the Greeks, the rubric would have been a device used by a teacher to demonstrate to others that the teacher's students 'got it'. Unfortunately, it wouldn't have had anything to do with whether or not the students actually 'got it'.

Poor Phidippides.

And so we raise a generation of kids who don't have the ability to deal with ambiguity. We raise a generation of kids who expect success for pleasing the teacher. We raise a generation of kids who don't want to take creative risks because those risks aren't going to improve their 'grade'.

I write this post at risk of sounding polemical. In fact, that's not my purpose, but I understand how my tone could trigger that response. What I'd really like to come out of this is a challenge to teachers to find more authentic ways to assess your students. Ways to connect, not via a mass-produced one-size-fits-all rubric, but by individualized 1 to 1 attention. Ways to share in the learning process in a communal and ongoing way, rather than by having students demonstrate 'understanding' by jumping through hoops and checking off items on a checklist. Ways to express to students that life is more complicated than a rubric and that success in life is not so easily defined.

Otherwise, I think we do our students a disservice. We set them up to engage with a world where more and more as this century progresses we are turning away from the old models of rubrics and other forms of so-called 'objectivity'.

Consider NASCAR.

There are numerous checklists that must be filled out before any given race. The cars themselves must meet dozens of requirements. On paper, everything has to be A+. Yet only one race car is going to cross that finish line first.

In other words, meeting the requirements of the rubric doesn't in any way ensure success. Yet, our students are conditioned to think otherwise.

So, what to do?

Well... let your students play in class. Give them open-ended assignments with no possible correct answer and no single conceivable way to get the assignment done. Don't explain things to your students, rather talk to them and allow what they say to teach them how they think. Teach your content through conversation whether f2f or online. Teach your content through trust. And don't give your students a list of things that suggests what you want, rather allow your students to figure out what it is that they want.

Because, in the end, this is about them learning. It's not about us proving why we gave a particular grade.

Well said. Rubrics are an attempt to make something "qualitative" into something that's "quantitative" so that we can assign a number or grade to it. Even though there may not be one "correct" answer, rubrics can give us some sense of a degree of "rightness" or "wrongness".

In my previous career as a band director, I would have my students record themselves playing excerpts of the music we were learning. I would then record my comments and suggestions on ways that they could improve, or what I thought they were doing particularly well. BUT, I also included a rubric in an attempt to quantify for them (and I suppose their parents) my expectations, and more importantly what I was hearing: rhythm, intonation, expression, tone, etc. etc. It also made me more objective so that I wasn't playing favorites (whether or not I was aware of actually doing that). And ultimately, it gave me and them a NUMBER or GRADE to put in the gradebook.

What about allowing students to design their own guidelines for assessment? In the end (unfortunately) most teachers have to assign grades. If we let the kids determine what kind of performance gets an A, could that be the solution for teachers who are often asked to justify grades? They would at least then be able to point to criteria that put the kids into certain grade categories, but the students would have been able to push the limits of possibilities for the assignments--perhaps well-beyond anything the teacher initially envisioned.

Hi Shelly... I think you are right on here... Assessment should be so much more conversation rather than assignment of numbers, which is seemingly more arbitrary than any non-rubric score. Great post!

@Cathy... I think that still creates the problems that Shelly is talking about. I like and utilize peer review of work but in the end, if I am truly the master of my content and classroom, I need to be the one evaluating and grading work with a criteria that I employ... students are usually either too harsh or not harsh enough...

Hmmm...Your position disappoints me in that you reduce the notion of rubrics into a black-white issue (reductio ad absurdum, really). Such fallacious reasoning doesn't reflect the open and courageous stance that you want to project.

You also cite little evidence beyond your own experience that suggest rubrics must be a certain way.

Aren't rubrics only limited by the imagination of their creator?

As you are clearly aware, there is no one way to teach anything wisely. Sometimes, not always, rubrics can assist students to think clearly about what they are asked to do. They can be extremely reflective, meaningful, and metacognitive if used well.

Doesn't it depend on how they are designed and what they are used for?

First, a disclaimer: I was one of the people engaging Shelley in the discussion late last night. Thanks to him for taking the time to explain his point of view in a forum where it's more easily possible (140 chars at a time isn't right for some things).

As I hope I made clear last night, I agree that rubrics aren't necessarily the best choice, that they're definitely not an across the board solution, and that there can certainly be horrible rubrics.

I also welcome your challenge to use better/individualized assessments wherever possible. But, given the realities of many teaching situations, those assessments are challenging if not impractical to implement. Rubrics can be a useful option, both for guiding students, which has its place, and for conveying information to other stakeholders in student learning (most notably parents).

Along those lines, I agree with one of the other commenters that when grades have to be given, a _good_ rubric can be useful. The example I offered of a good rubric was http://is.gd/2rgqQ This rubric is not perfect. It does things like assume the superiority of "the" scientific method (which is a whole different debate), but I contend that it does a good job of setting clear process goals.

And though following such process guides (or checklists) will by no means guarantee anyone a Nobel prize in physics or a Nascar win, not following those guides will almost certainly guarantee not winning either prize.

So, a question I would welcome exploration on... Does the utility of rubrics in general depend on a what specific (disciplines) rubrics are used for. My experience is teaching science, and though I emphasized writing skills and interdisciplinary learning, I am by no means a skilled/experienced humanities teacher. It would seem to me that trying to create a rubric for a piece of creative writing could be disastrous. On the other hand, a checklist/rubric for assessing the lab process skills needed to do more creative explorations in science does seem useful to me.

It would seem that a rubric (or set of rubrics) as the only assessment tool is not the answer. But, I'd also argue that there is no one-size-fit-all set of assessments. Can rubrics be one tool in a toolbox of assessments, that skilled teachers use when appropriate? I think they can be.

My perspective on rubrics has always been they were misused when solely a summative evaluation tool. They have much more power when used formatively, where students can reflect on their scores (however subjective they might be) and revise and improve their work. And that formative feedback ideally should come from a variety of sources, not always from the teacher.

Unfortunately I believe 99% of our use of rubrics in education is summatively, with no chance for students to improve after receiving the feedback.

Wow, this is an interesting conversation- and I would have to agree with osme points-those being it depends on the assignment and the task; AND students should help create and develop the rubric they will be using to assess whatever task or assignment the rubric will be used for. No 2 rubrics should ever be the same. And students should use the rubrics as a self-evaluation tool as well as a means of reflecting on their process. Of course this is only my opinion

As a teacher who uses rubrics a lot in her classroom, I found this post very thought-provoking. I agree that rubrics can be cookie-cutter ways for teachers to create a checklist of what needs to be done. However, that is not the fault of the rubric, but rather the person who creates it.

My rubrics are never the same from year to year. I am constantly revising them as I watch my students complete projects that can sometimes take over a month to complete and realizing what is most important for them to learn from the project. As in backwards design, I am constantly thinking of my objectives for the project and include only those objectives in the rubric. Rubrics also take the guesswork out of what a successful project will achieve so students can focus on creatively completing the project without wondering if they are on the right track.

If we want to move away from standardized tests, standardized assessments and isolated academic study into integrated project-based learning that allows for higher-order thinking, collaboration and application of academic skills and knowledge, what is the appropriate assessment to use? How will we guide our students through the process of completing such a project without using a test? Even a portfolio needs some kind of way to rate it.

Until a better solution comes along, rubrics provide that kind of structure and guidance while allowing for creativity and open-ended products--if they are well written.

I do agree that our students should not be mindlessly completing tasks to satisfy a rubric.

Thanks for starting such a lively discussion and having the gumption to 'come out and say it.' :-) I'm sorry that your experience with rubrics has been so negative!

@Christopher... I think you are right that it is probably inappropriate to damn rubrics in a broad stroke. That said, the vast majority of uses I have seen suggest do the same thing with student work: broad stroke praise or damnation of student work without a conversation about the actual substance of the work. If a teacher is assigning numbers without and justification or conversation why, it suggests to me that they aren't really thinking about or telling the student anything about their work...

Suppose we compare rubrics to John Searle's Chinese Room thought experiment (it could be a Latin Room instead). A bunch of students is in a room with a set of instructions, dictionaries, cribs, and tabs for converting snippets of Chinese symbols into snippets of English symbols. Chinese in, passable English out. Where does the 'understanding' happen?

If what we are after is 'understanding' in the mind of an individual student, then we're better off using The Pirate's Code: "more what you'd call 'guidelines' than actual rules."

A well-constructed rubric should include examples of how to subvert the rubric.

I agree with Mr. Sessums' comments: the rubric is limited only by the imagination of the creator. Rubrics and student creativity do not have to be mutually exclusive. While they are given a suggested task, students in my classes are free to propose any type of assignment they feel will meet the requirements of the rubric. As for the dread of objectivity in grading, I always make 25% of the rubric a subjective grade called "risk-taking" that simply asks whether or not the student challenged herself.

This essay is pompous and pseudo-intellectual nonsense covered by a thin veneer of classical references.

"Let your students play in class."They are already doing that. That's the problem.

"Give them open-ended assignments with no possible correct answer and no single conceivable way to get the assignment done."

We don't need to teach kids in this generation to be comfortable with ambiguity and challenge authority. This is not 1950. These kids are post-moderns. They have had "there is no such thing as absolute truth, there is no one right answer, follow your heart, believe in yourself, and everyone's point of view is equally valid" drilled into their heads since they were old enough to watch Disney Cartoons! Now they need to learn to respect authority and follow directions. They must learn that there are rules of spelling and grammar. (You cannot write a letter or a report just any way you want: you have to follow certain rules.) Make sure your sources have been peer reviewed. You cannot just slop together something: you have to meet specific requirements.

"Don't explain things to your students, rather talk to them and allow what they say to teach them how they think."

How is that going to work in math class? Or history? Or science? Lets see how they do on their standardized tests if we don't explain DNA, or electron orbits, or how to take a derivative. Lets just let them play and talk and listen to what they say. That's just bullshit. They will just sit there and swap ignorance. They will not spontaneously talk about content. They will spontaneously talk about their sex lives.

"Teach your content through conversation whether f2f or online. Teach your content through trust. And don't give your students a list of things that suggests what you want, rather allow your students to figure out what it is that they want."

What they want is to sit and talk about their social lives. All the evils of standardized testing were forced upon us as a reaction to this kind of non-sense.

Subscribe via RSS Reader

Teach Paperless: Now!

TeachPaperless began in February 2009 as a blog detailing the experiences of one teacher in a paperless classroom. It has grown to be something much more than that. In January 2011, TeachPaperless became a collaboratively written blog dedicated to conversation and commentary about the intertwined worlds of digital technology, new media, and education.

Buzz Paperless

TeachPaperless was noted as a Twitterer worth ReTweeting by Education Week's Digital Education blog. Also in Ed Week: "Shelly Blake-Plock has had some really intriguing posts already this year and I'm already behind. Considering he published 639 entries on his TeachPaperless blog in 2009 it's going to be hard to keep up, but well worth the try."

“When I originally contacted Shelley last week to inquire as to whether or not he would be willing to talk to my staff, he jumped right in, and he didn’t disappoint. What impressed me most about him as I listened to him describe his practice was his clear vision of what it meant for his students to function in a classroom that he designed: it was about them learning. He truly designed the environment with their learning–their unbridled learning–in mind. His decision was not a secretarial one, but rather came from a desire to push students to take control of information gathering, processing, and creating.” – Chalkdust 101

TeachPaperless was named one of the 'Top 25 Blogs for Educators' byWorld Wide Learn.

"I think you have some great ideas for teachers, and as we do professional development around the state of Maryland, we will point teachers to your blog." Debbie Vickers of Thinkport.org a partnership between Maryland Public Television and Johns Hopkins University's Center for Technology in Education

"The invention of the computer promised to lead us to a paperless society but has failed to deliver on that promise... until now, perhaps?" TeachPaperless was featured by Mid-continent Research for Education and Learning as an Everyday Innovation

Your friendly contributing bloggers...

License and Disclaimer

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 3.0 License.

You may re-use this content online for noncommercial purposes without needing to ask permission, as long as you credit the source in writing as Teach Paperless and on the web by adding a link back to our web site,www.teachpaperless.com

And of course, everything on this blog is the personal opinion of the individual bloggers and does not reflect the opinions of of anyone else, including employers, in any way. But that should be obvious by now.

Photo Credit: MJ Wojewodzki; a portion of a painted wall in the Villa of the Mysteries, Pompeii [2006]