Cranking MBT up to 11 with Specifications Grading (Part I)

I first implemented MBT in a Calculus I course in the spring of 2015 and realized, as Austin writes, that MBT is “self-evidently better” than traditional, points-and-partial-credit-based testing. I’ve used some form of MBT in every course since then.

Around the same time, a friend retweeted Robert Talbert of Grand Valley State University that made reference to “specifications grading”, a system proposed by Linda Nilson (I would strongly recommend you read her book if you are thinking of implementing specs grading). With my mind already opened to the possibility of alternative grading systems, I took the plunge and read several of his blog posts, then at the Chronicle of Higher Education, and now hosted on his own site. To be clear, most of what I have learned about implementing specs grading in a math course, I have learned from reading him. But in this post and the next, I’ll do my best to explain how the system works, and how naturally MBT fits into such a course.

Creating Learning Outcomes

As Katie wrote for MBT, the first step (which is absolutely essential to do before the term begins) is to comb through the entire course and create a list of learning outcomes. These outcomes should be specific, and can encompass more than merely content knowledge. Here is the list of outcomes I have created for my Fall 2016 modern algebra course, and here is the syllabus which outlines the system in more detail.

Creating Assessments

Your next job is to determine how to assess student progress on the learning outcomes, and then create a set of assessments to accomplish this. This doesn’t need to be done before the term starts, though I found that creating my assessments in advance helped me refine the wording on each outcome to be more flexibly assessable.

In my modern algebra course, the assessments will be primarily homework and take-home exams, because I feel that these are the best ways to assess the learning outcomes, but in a different course, with a different set of outcomes, you may find that the use of, say, a project (or series of projects) would work better.

Grading student work using specifications

Here is where specs grading and MBT have a major spiritual overlap (and where the title of this post comes from): the assessments are graded pass/fail based on whether or not the students meet the desired learning outcome(s) according to a list of specifications for how the work should be done. In other words, it’s not just that there are no points or partial credit on the exams; there are no points or partial credit in the entire course.

In a specs graded course, the specifications document (modern algebra example here) should describe exactly what a student needs to do in order to earn a passing designation, down to the details of formatting. If at any point the specifications are not met, the student does not receive a passing mark, and may revise the work (with some limitations, which I’ll get into next time). Thus, it’s important that the specs be written in such a way that they are easily checked by the student and easily enforced by the instructor.

In modern algebra this fall, I’ve created a series of 13 homework assignments (here’s an example), each of which assesses a handful of learning outcomes using 1-3 problems per outcome. Rather than grade the assignment, I’ll grade the outcome: if a student passes all problems associated with a given outcome, they pass the outcome. If they don’t, there are limited opportunities for revision, which I’ll describe next time. Students will be required to pass some subset of the outcomes twice: on the homework assignments (which I am encouraging students to work on in groups) and on the take-home exams (which must be done alone).

On Wednesday, I’ll talk about the opportunities for revision, how I compute a final grade, and how naturally MBT fits into a specs graded course.