Monthly Archives: September 2010

So I taught highschoolers professionally for five years, and now I’m running a startup software business. The biggest difference, even bigger than the reduction in human contact, is the stress of competition. As a teacher, I shared everything I learned, put it right on the internet, even, and expected others to do the same. As a programmer with a plan for a great grade-reporting program (bee tee dubs: it’s looking great), I can’t put my ideas out there very much, because two programmers might take it and get a jump on me. I have people sign a 3-page legal document before I talk to them about my ideas.

It’s disgusting.

Current teachers: relish the chances for collaboration you have. I don’t really know much about merit pay and other forms of rank that might be imposed on you, but the freedom to share your very best ideas with another person who can appreciate them is sacred. I didn’t realize it until it was too late.

It's hard to believe how good I am at estimating. REALLY hard to believe.

Each point represents a specific task. Before starting the task, I estimated how long it would take me to complete (“estimated hours”). Then, I timed how long it took me to complete each one (“actual hours”).

Questions I don’t know if my students could answer:

Which points represent my “worst” estimates?

What does a best-fit line mean on these axes?

Would a different line be more meaningful than this plain old best fit?

Am I a good estimator?

If I tell you I’ll be done with something in an hour and a half, how long would you expect to wait?

BTW, here’s a little update of my progress from last time. This graph shows the probability I’ll be finished with the project on any given date. It’s generated using guesses based on a) how long I said the project will take, b) what that actually means given that I’m clearly a freaking terrible estimator, and c) how many hours I’ve been working each day.

Think I’ll make the December 1 goal? Would your students have any idea?

I’d like to give my students as many chances to learn as possible. When they’re interested, I’d like to sit with them forever. Unfortunately, there are some pretty significant logistical obstacles here.

Almost everyone who disagreed with my automated debater for SBG+ (+remediation, +forgiveness of earlier scores, +timely & empowering reporting) disliked the idea of throwing out old assessment scores. The most convincing criticism I’ve read is at GSWP (alternate link, scroll to bottom):

SBG aficionados believe in instantaneous noise-free measures of achievement. If a student takes a long time before they “get it”, but then demonstrate mastery, that’s fine. This results in the practice of replacing grades for a standard with the most recent one. I think that is ok, as long as the standard keeps being assessed, but if you stop assessing a standard as soon as students have gotten a good enough score (which seems to be the usual way to handle it), then you have recorded their peak performance, not the best estimate of their current mastery. Think about the fluctuations in stock prices: the high for the year is rarely a good estimate of the current price, even if the prices have been generally going up.

The author at GSWP points out that averaging multiple assessment scores together works under the assumption that those different assessments were measuring the same thing, and that students’ skill levels are essentially unchanging. I think this is why some of us have such a strong reaction against averaging. We like to think that our students learn and improve so much during our class that the first assessments they take have almost no correlation to what they understand at the end of the class.

So. Given that we have to choose a final score eventually, how do we do it?

I’ve written up a google doc with some ideas about different grade calculation methods. You can edit it by clicking here, or just read it (and what others have added) below.

I think my favorite (right now) is the decaying average, but I’ve never tried it on actual data. Please leave your thoughts in the doc (you can create a new table to include a different method altogether) or in the comments!

This is a graph with unusual units. The horizontal axis measures the date. The vertical axis measures the estimated date of completion of the project I’m working on. The top line is a 95% estimate, the middle line is a 50% estimate, and the lower line is a 5% estimate.

So, for example, on Friday, September 10th, the estimate was that there was a 50% chance that I’d finish by September 24th. Today, September 14.5th, there’s a 95% chance I’ll be done by by October 29th. And, if everything goes REALLY WELL, there’s a 5% chance the project will be done by October 6th!

The completion dates keep moving up because I keep adding new tasks to complete. At the beginning (September 7th) I thought there were about 30 things to do, each taking a half-hour, so the estimates showed me finishing in a couple of days. Since then I’ve added a lot (A LOT) of things to do, many taking multiple hours, so the estimates show me finishing much later. If I complete some tasks faster than I estimated, the completion date will move closer. If I repeatedly take longer than I thought on multiple tasks, the program making these estimates starts to automatically inflate the times I enter.

What’s happening here? Will the project ever actually be completed? What should happen to the lines as the date marches on?

How would you lead your students to understand what these lines mean? They should be able to answer the following questions:

What would the lines look like if the project was running perfectly on schedule?

Which days seem to be the most productive?

What is the maximum estimate/date slope that a project could have if it was going to actually be finished?

Given the average estimate/date slope of a project, how could you figure out when the project will actually be finished?

What makes the lines move farther apart from each other or closer together??

When people respond to the automated sbg persuader, I get anonymous email with their disagreements. Here are a few messages along the same theme.

In response to the premise that “if a students’ level of understanding changes, so should his or her grade,” I received:

At age 46 I do not recall everything that I learned in my engineering classes 25 years ago. However, those grades reflected my ABILITY to grasp and use the information AT THAT POINT IN TIME. Given the time and putting in the effort, I could attain those grades again. For example, I just passed the Praxis II with high marks, but had to study almost a year to recall all that I had forgotten.

This writer is suggesting that his grades from college shouldn’t retroactively go down just because he hasn’t bothered to keep refreshing his memory every month for the last 25 years. The grades he earned represent what he knew THEN, and don’t need to constantly change to reflect what he knows NOW. Can’t the same argument be applied within a single year? Why should Sarah, who was fantastic at addition in October, be penalized because she can’t remember how to do it in May? And why should Chekol, who showed perfect memorization of Native American tribe names in September, have his grade lowered after forgetting half of them in April? Can’t these students just look this stuff up? We’ve already seen that they have the capacity to know it, after all.

I got many messages with this idea, that grades shouldn’t necessarily be lowered.

100, 100, 100, 100, 70, 50. Did this student forget addition? How do you forget addition after 4 successes? Or did their house burn down near the end of the year?

and in the opposite direction, with an extreme example, another commenter asked

But how can I account for the timeliness of learning?

He or she went on to wonder whether learning about fission after the enemy has the bomb should “count for as much.”

These questions strike deep into the heart of the philosophy of providing easy remediation and very flexible grades. I have my own ideas, but I’ll write them in the comments. Please write with your own responses!

There’s a problem with the term “Standards-Based Grading:” it’s too overloaded. The word “standard” means ten different things, and so newcomers to SBG don’t know what it is from the name (e.g. “I have standards too, you prick.”) Even within the community of SBG believers, there’s confusion: do you have to allow remediation under SBG? Is it still SBG if I have deadlines and late penalties?

But changing to standards-based grading can be very simple. Just group grades by knowledge. Don’t say, “you have 95% in projects, 80% on tests, and 85% in homework.” Instead, report that “you’ve earned 95% in graphing lines, 80% in graphing general functions, 85% in composing functions.” It doesn’t have to be philosophical – this is just more information for your students.

As people respond to the automated SBG persuader, I get anonymous emails with their disagreements. Here’s a big one. It is in response to the premise that “final grades would more accurately reflect current understanding if we could only use recent scores to calculate them.”

“How is that fair to the student that learned addition quicker? Shouldn’t the grade ultimately rank my students according to their ability?”

Sure, pitting kids against each other until they’re tearing at each other’s eyes is fun. But is it right? To me, the answer is “obviously not.” Is it effective as an educational tool? Obviously not. Is it the principle off of which we want kids to model their lives? Obviously not!

Is the point of education to figure out which kid is the best? Say it with me: obviously fucking not.

Sorry. I was being a little sarcastic up there, but let’s be reasonable. The argument for competition in a classroom is that a competitive atmosphere is motivating and can raise the overall level of achievement. What better way is there to get Sarah to raise her grade to a 95 than telling her that Rebecca has a 94?

The problem I have with this is that it focuses solely on the grade. Ranking students by grade, so that they compete for grades, makes them care about the grade. I want them to care about learning. As Alfie Kohn says in The Schools Our Children Deserve, “The difference between learning and achievement is hard enough to grasp; the difference between doing well and doing better than others is especially confusing in a society so obsessed with being Number One that the ideas of excellence and winning have been thoroughly conflated.”

Now I’m going to slam you with some research that Alfie Kohn compiled. Watch out. Just read the bold sections unless you’re going to check my references.

Susan Nolen’s study titled “Reasons for Studying: Motivational Orientations and Study Strategies” concluded that students who equate success with surpassing others are more likely to think in a “surface-level” way.

Carole Ames published “Children’s Achievement Attributions and Self-Reinforcement: Effects of Self-Concept and Competitive Reward Structure” in the Journal of Educational Psychology with a conclusion that students are more likely to feel empowered to affect their own achievements when those achievements are not linked to competitive results.

P.S. Fry and K.J. Coe concluded in “Interaction Among Dimensions of Academic Motivation and Classroom Social Climate: A study of the Perceptions of Junior High and High School Pupils” that competitive environments “cause students to dislike school and show less interest in a given subject.”

From David and Roger Johnson said in Cooperation and Competition: Theory and Research, Kohn concluded that when a group includes members of different ability levels, they “learn more effectively on a range of tasks when they’re able to cooperate with one another than when they’re trying to defeat one another.”

I’m glad that there’s research I can use to bolster my intuition. I don’t think that competition should play a significant role in education, and, since grades are often viewed as the “conclusion” or “you know, the point” of school, I really don’t think that grades should be based on competition.

If you use competition in your class, I’d love to hear how you justify it. Again, I’m open to the idea that I’m wrong, or missing a subtlety. I hope my intro wasn’t so caustic that you aren’t reading this. I guess that’s impossible, since you’re clearly reading this if you’re reading this. But you know what I mean.