Let’s ask teachers to rethink assessments

Life is generally lived as an “open note” experience. Schools need to focus on preparing students for life, not just for academic tests conducted under artificial conditions that bear little resemblance to the real world outside academia. In line with this idea, rather than banning iPods, cell phones, laptops, and other types of technologies schools need to embrace them and find ways to adopt new assessments which can be taken by students with “open notes.”

The most challenging assessments and tests I took in graduate school classes were “open note.” The reason they were so challenging is they required thinking and analysis that went far beyond the knowledge and comprehension level. Many schools are fighting against digital culture in banning cell phones and iPods because they remain rooted in 19th century paradigms of education and assessment.

Mountain View [Idaho] recently enacted a ban on digital media players after school officials realized some students were downloading formulas and other material onto the players. “A teacher overheard a couple of kids talking about it,” said Maybon. Shana Kemp, spokeswoman for the National Association of Secondary School Principals, said she does not have hard statistics on the phenomenon but said it is not unusual for schools to ban digital media players. “I think it is becoming a national trend,” she said. “We hope that each district will have a policy in place for technology — it keeps a lot of the problems down.”

Rather than adopting policies about technologies that are banned, school districts would be better advised to have their teachers craft new assessments. Our goal should not be, “How can we maintain our instructional and assessment paradigms from the 19th century today in our 21st century digital culture?” but rather “How can we craft authentic assessments our students cannot fake and they can take with open notes?” Open notes should include “open devices” like cell phones and iPods.

The problem with this proposal is that it is very challenging to write and use authentic assessments. It is much easier to test at the knowledge and comprehension level, and that is why we see so many teachers doing it. When you want to have statistical reliability and validity with an assessment, it becomes much more difficult to assess higher order thinking skills with “messy assessments” that include rubrics and subjective analysis. NCLB also encourages this simplified look at assessment, encouraging school districts, administrators and teachers around the United States to focus almost exclusively on multiple-choice, black and white forms of assessment that can be graded via a scantron.

Why should students have to memorize formulas? Let’s take on an example from mathematics. How many times have you had to use the quadratic formula in “real life” outside of school? If you have, were you prohibited by someone from being able to look up the formula?

Do you think oral communication skills are important in life? If you do (and you should) then why do some schools hardly emphasize assessments which include oral communication? Again, the reason is that many schools are still focused on maintaining their 19th century pedagogic culture rather than preparing students for real life.

Let’s put an end to useless memorization and tradition in schools, and instead ask students to actually apply and use the knowledge and skills they are supposed to be learning in real contexts. That is what we do out here in the business world. It’s time schools stopped acting like the calendar still reads “1899.”

I fear that education is the last bastion of tradition rather than a leader in new ways of thinking. Alvin and Heidi Tofler discuss the idea of “obsoledge”, obsolete knowledge, which hinders the advancement of the information economy. Many teachers cannot let go of the way they “know” to teach.

There is fear of new technology. My district and county employ a filter company whose stated intention is to block all wiki, blog and personal websites. That includes almost all web 2.0 sites.

Apparently, this pleases parents. A school teaching the way the parent was taught and defending against the evils of the internet help parents feel their children are safe. Never mind that the children will not be prepared to live in the world they will live in.

http://www.oesd.wednet.edu/blog Jeff Allen

Do the kinds of tasks we are asking students to do on a test lend themselves to be easily “cheatable”? If the assessment is asking a student to do little more that recall information, then an iPod could easily be used to store and recover that information… a “cheatable moment”.

What if we were using assessments that asked students to access and use information to create a new idea, or solve a problem in an innovative way that demonstrates that they can THINK!?

We are going to continue to run into this wall as long as we have 2.0 students attending 1.0 schools.

http://blog.brettmoller.com Brett Moller

ah so true my friend… why is it that we continue to produced monkeys that become very good (or often not so good) at simply recalling facts. Furthermore, to your quadratic example…. What is the point of being able to recall it if you have no idea how it is applied. In fact it is much easier to look up the formula than try and work out how it is applied…. hmm some great thoughts.

http://tnturner.edublogs.org Tom Turner

Well said Wes! To take a quote from your article:
“Rather than adopting policies about technologies that are banned, school districts would be better advised to have their teachers craft new assessments. Our goal should not be, â€œHow can we maintain our instructional and assessment paradigms from the 19th century today in our 21st century digital culture?â€ but rather â€œHow can we craft authentic assessments our students cannot fake and they can take with open notes?”

This is one thing I never had to worry about when I was in the classroom. It’s also what I’m trying to teach to my teachers as a technology specialist. I can gain more insight into how well my students have learned a concept through the act of them creating/producing something, then to regurgitate facts that they will ultimately forget on a pen/paper test/quiz.

http://classroom20.ning.com/profile/kmurray352 Karla Murray

Wes,

Thank you for continuing to inspire us and cause us to reflect on past practices and re-think our present!

Wes, I agree with your comment on “open note” tests often being the hardest. I clearly remember these from my high school testing days, at the time I thought these “open book” (as we call them in NZ) were easy and I only did minimal study. I achieved ok for that test, but learnt the hard way!

I agree that we need to get students to actually apply and use the knowledge and skills they have been learning in real contexts. Thanks for reigniting this thought with me.

Cheers, Rachel

http://tabletpceducation.blogspot.com/ Bob

Do you see standardized tests as opposed to authentic assessments? Seems to me they are part of continuum of assessment. It seems to me that standardized tests results compare an individual against what others know. That information has value for some kinds of decisions about learning, planning, etc. Other assessments provide information for other kinds of decisions. Many variations and blends of these assessments exist, sometimes also with criterion referenced assessments. Let’s keep them. They each have uses for teachers.

Olivia Morris

Schools are so much into interpreting tests scores to make decisions about students abilities, that they are reluctant to try authentic forms of assessment using ipods and other social software as part of the assessment tool. If students are benefitting from collaborative learning then is logical for teachers to evaluate such learning using social softare. I agree that open book tests should include, the internet and iPod. Also, such tools enable teachers to make a better assessment of students’ creativity.

http://www.wesfryer.com Wesley Fryer

Bob, I think it is still fine for us to take standardized tests, but the menu of assessment options should go far beyond those things which can be tested in statistically valid and reliable ways. The problem today in many parts of the United States is we’ve developed myopic vision solely focused on those high stakes standardized assessments. The big reason for this is NCLB, which dictates that the only measuring stick that matters is the standardized measure. That is ridiculous, and is hurting our students as well as the future potentials of our workforce. I agree there are continuums of assessment. I am in favor of multiple forms of assessment, for reasons supported by multiple intelligences theories as well as the fact that we need to emphasize many things (like oral communication) which are not testable by more standardized means.

http://www.oesd.wednet.edu/blog Jeff Allen

OK…. Stop for a sec…. please think…..

Do the kinds of tasks we are asking students to do on a test lend themselves to be easily “cheatable”? If the assessment is asking a student to do little more than recall information, then an iPod could easily be used to store and recover that information… a “cheatable moment”.

What if we were using assessments that asked students to access and use information to create a new idea, or solve a problem in an innovative way that demonstrates that they can THINK!?

We are going to continue to run into this wall as long as we have 2.0 students attending 1.0 schools.

http://lostjohns.blogspot.com William Bishop

Great post Wes! I liked it so much that I used it at my blog and threw in my 2 cents worth. Hope you don’t mind. Anyway, I like how you’re thinking. We do have to move out of the past and toward the future. Thanks for the great post…

William Bishop (Bill)

http://ondas3.blogs.sapo.pt OctÃ¡vio Lima

Good post. It would be excellent if it had provided sites with hints or examples of this new 21st century assessment.

http://sjhs.stillwater.k12.mn.us Bruce Deger

This topic of “cheating” and assessment always reminds me of the many stories we have read over the years of college athletic scandals in which some athletes purportedly hired others to write their papers for them. In recent years, the “hiring” is more often done via a credit card online form a digital repository of papers. The typical reaction is to employ anti-plagiarism services and other technologies to catch the cheaters. If some of those resources could be redirected to creating authentic assessments instead, everybody would win.

What a great post, I’ve been forwarding it to many in my staff. My district has purchased hundreds of iPods and is working on how to have them used in ‘Open iPod” tests. If you tell students, “You’re going to have a test where you can use an iPod in any way you think will help you.” They will amaze you with their creativity and while they’re cramming that player full of content, will learn something too.
Why would you remove something that many students use everyday, and instead of showing them how it can be used educationally, not treat the tool as if it’s the problem. We have some students that spend 2+ hours a day on the bus. If they are provided audio books, lectures, etc. that is 2 more hours that education is available to them. Many students already own iPods, why not use the tool to engage them? For those tests that are easily copied from the iPod, the question is, what exactly is being assessed?

Adam

I am finishing up a very disappointing year in my computer science class. I desperately wanted to provide an environment where students could explore new technologies, create original content, contribute to the real world, but student apathy sucked the potential out of everything I wanted the class to be. There are two realities of opening up assesment and education as whole to the new technological world that I don’t feel are being adressed. First, kids don’t want to use technology to learn anymore than they want to use books to learn, sure if they’ve never used an ipod it’ll temporarily peek their interst, but they will get over it. Students resist learning, free thought, and do not inheritly feel proud of the things they create. Secondly, the advancements in social networking, and processing of portable devices are the best tools to combat the fact of student apathy. Problem is using these tools for education is left in the hands of the “imigrants” and have to be used contrary to desired application of “natives.” I wanted to believe that by empowering students with technology that my classroom would change overnight. But it didn’t, and now I’m left with the relization that changing student ideas about education is a huge obstacle, and the frontline, down in the dirt strategies, of fighting student apathy with the power to share and create through technology is not what everyone is talking about.

Adam: My own experiences teaching at the university level, supporting university faculty in their instructional uses of technologies and being a graduate student myself resonate with your comments. By the time many kids get to high school, and especially college, “the system” has conditioned them to believe they don’t have anything to contribute or offer in the classroom learning environment. At the extreme (which is sadly common in many contexts) university classes become little more than an exchange of money and time (by the student) for a grade from the professor. I agree this culture can be EXTREMELY difficult to change. Many students come into class wanting to know 1 thing: What do I have to do to get the grade I want in this class. In that situation, there can be little apparent motivation to explore new tools, extend learning beyond the required minutes of the class, share your voice, etc.

I don’t have a “silver bullet” answer for this (I don’t think one exists) but I think one key is cultivating a classroom culture where exploration and sharing with new tools is both expected and required. I think having students regularly write and blog about what they are reading, studying, and discovering and focusing on the personal relevance they find for those ideas can be useful. The summer 2005 course I taught on advanced topics on technology integration was one case where blogging seemed to really help cultivate this culture.

You’re getting at bigger ideas than just technology with your comments, however, and I sense part of what you’re asking is “how do I get my students to care about their learning?” You are right, student apathy is often a big obstacle, and very important to both acknowledge and address.

My main thought on how to effectively address student apathy is via passion and personal connections. This also ties into work that many are doing on the issue of “student engagement,” and how we get students REALLY engaged in learning. (Being engaged means many things, but I would assert NOT being apathetic would be a basic part of engagement.) I know Phil Schlechty and his group is doing a lot of work in this area.

You are so correct in pointing out how merely introducing social media and interactive web environments to students is insufficient, if our goal is engaging them in authentic as well as sustained learning experiences. The complexities that are present here are what cause Matthew Koehler and Punya Mishra to characterize technology infused teaching as a “wicked problem.” I think they are right, and your comment also reflects the “wicked” complexities of teaching with technology / social media.