This Blog is from the faculty Learning Assessment Council at Portland Community College, in Portland, Oregon. Faculty and others at PCC are responding to changes in Higher Education -- especially the "accountability movement" -- and seeking to improve teaching and learning based on evidence-backed best practices.

Wednesday, January 26, 2011

Grade Fog and..... Fog Lights

Assessment Blog Jan 25 2011

Extra! Extra! Hear All About it!!

There is fast-breaking news in the assessment world. The Lumina Foundation (which is funded in part with Gates money) has just released a report that has the potential to shake up Higher Ed in profound ways. It is called The Degree Qualifications Profile. The U.S. Higher Ed system has been playing catch-up with the European Union (and their Bologna Process.) And this counts as a significant step forward.

What's it all about? Diligent readers of this blog may be familiar with the phrase "grade fog." Since teachers are in charge of how they determine student grades, it is impossible to meaningfully compare one student's grade of B (from Instructor #1) to the next student's grade of B (From instructor #2.) #1 might try to factor in effort, and award students with good grades for trying hard. #2 doesn't care about effort -- or attendance or participation -- just the results. Those are very, very different grades of B.And that's in one institution, where both instructors belong to the same department or SAC and are teaching under the same CCOGs (or equivalent.) Comparisons get all the more obscure as we go from one institution to another, across divides of public and private, research centers and community colleges, for-profit and non-profit schools.

Into this foggy mess, the Lumina Foundation has just put out a preliminary report. They are setting outcomes associated with "quality" degrees. The purpose is to have clear and explicit outcomes that are generally accepted, across institutions, for college degrees (associates through masters -- doctorates are not yet covered.) And wherever there are outcomes, assessments are not far behind. At this point, the document is very general and abstract. The plan, it appears, it to use this as a framework for on-going conversations about how to fill in the details...

Is this a good thing for Higher Ed? Well, like most everything in life, the matter appears mixed.

From my point of view, outcomes coming from a non-profit foundation sound better than outcomes dictated by the Department of Education (with money tied.) And setting out outcomes can be better or worse, depending on assessments. The Biggest drawback of the No Child Left Behind approach (to my mind) is not that outcomes became required for each grade level, but that certain high stakes tests became the one and only way they were measured. So I will be waiting to see what happens when the assessment shoe drops (so to speak.)

Also from my point of view, though, one of the most vexing and intriguing pieces of the assessment whirl is what is often referred to as "measuring the unmeasurable." Here at PCC, we have language for some of what we are trying to do in our Core Outcomes. We promise that our students will become increasingly self-reflective and willing to turn toward our collective social and environmental problems (instead of running away.) We say our students will learn to understand who they are in this world through increased awareness of cultural variation and vast differences in the shared human venture of "meaning making." How has the Lumina Project benchmarked these vital aspects of education?

This is what the report says:

In addition, many colleges and universities emphasize their role in fostering personal growth and helping students examine their values and commitments. But such elements of institutional mission rarely are specified as criteria for awarding degrees. Therefore, they are not explicitly included in this Degree Profile, even though values reflection and personal growth are inherent in many of the competencies that the Profile does include.

I think I'll be staring at this paragraph a long, long time. If we measure what matters, and we are not measuring this...... well, you can see where that reasoning goes.