Diversity and Peer Assessment in OldGlobe MOOC

I am finding OldGlobeMOOC a fascinating experience – quite unlike any other MOOC I have participated in, and my first xMOOC. For me one of the best things about OldGlobe is the diversity of the participant group. All the other MOOCs I have participated in have attracted groups in which similarities can easily be seen, i.e. mostly from academic backgrounds or interested in e-learning technologies.

But OldGlobeMOOC is truly diverse. It has attracted a huge age range from 11 to upper 80s, and people from all continents apart from Antarctica, although as you would expect the American participant presence is, I think, dominant – it would be interesting to see the analytics. But more importantly, it has attracted people from all walks of life and from very different education backgrounds. We have people very knowledgeable about the discipline of gerontology and related health and medical issues, but also very many people who have no subject related background other than we are all ageing and therefore all have a point of view.

This diversity is great. The stories being told in OldGlobe are richly diverse and a privilege to read.

But this diversity brings its own problems in relation to peer assessment of the OldGlobe assignments.

I mentioned in a previous post how the assignments are open to a range of approaches and creativity. Originally I hadn’t intended to complete the assignments, but I was drawn in by the energy and enthusiasm of the OldGlobe community and I’m glad I was. I have seen examples of participants posting videos of themselves speaking about the assignment question, writing about their own personal experiences, drawing on academic literature, posting links to videos, websites and photos, and drawing on literature and poetry to illustrate their response. A requirement of the assignment process is peer review of 5 assignments and I have reviewed some wonderful submissions. The first assignment I reviewed used this song in response to the question ‘What is ageing?”

Fantastic!

But the peer review process is where the wonderful diversity in OldGlobeMOOC creates problems. We have the whole continuum of people from those who have no experience of peer review to people who have worked in Higher Education for years and are very experienced in assessing student assignments and reviewing research articles.

This is a dilemma for OldGlobe, because some participants are getting a bit of a rough deal in terms of their feedback, despite the OldGlobe team urging participants to be generous with their feedback and scoring. For example, one participant has been accused of plagiarism for his/her original submission, another has been accused of plagiarism for using an essay site, even though this was cited as a source, another has received the feedback ‘I don’t get it’ and a mark of zero for an academic piece of work, others have received one line or less in their feedback. It all seems a bit of a lottery. So diversity brings both advantages and disadvantages. How might this dilemma be overcome?

I applaud the OldGlobe team for designing the MOOC to attract such a diversity of participants and for designing the assignment tasks such as they can be completed by anyone from any background. We are all getting older. We all live in a society where we can see people getting older. We all know old people. We all have some thoughts and perspectives about the ageing society. Even an 11 year old can say something about their grandparents and an 89 year old has a wealth of experience to share. We can all read other people’s discussion forum posts and assignments and have a response.

OldGlobe has asked us to answer the following questions when responding

Please type your 100-250 word peer assessment below.

What do you think about this participant’s portfolio item choice to answer this question of the week?
How does this participant’s perspective differ from your point of view?
How is your point of view similar?

I think the whole age range could do this, provided they could understand the submission.

The problem comes with the scoring. Here are the instructions:

Here is the rubric for the assignment. You’ll use this as a guide to complete your own work in the Submission Phase, and as a guide for grading your peers in the Evaluation Phase:

2 points

Assignment is completed with a clear commentary of 250 – 500 words that pertains to the question of the week

1 point

Assignment is completed with some commentary that may or may not pertain to the question of the week

0 points

Assignment is missing an item, a commentary, or both

It seems straightforward, but given the diversity of course participants is so open to misinterpretation or overly subjective interpretation, which some participants seem to have experienced.

For me, the peer review process on the first assignment has been positive. I have really enjoyed reading the submissions I was sent to review. They were not all academic pieces of work, but they had all been thought about and I appreciated the open sharing of experience however articulate or inarticulate that might be.

This MOOC is not for credit. Participants will simply get at Statement of Achievement.

All students wishing to obtain a Statement of Accomplishment must achieve 7 out of 12 points and submit 5 peer reviews each week. If a student fails to complete the 5 peer reviews, that week’s assignment will incur a 20% penalty.

This makes me wonder if we need points at all. I think the feedback is very valuable and I would prefer to call it feedback than peer review, which I think puts the emphasis in a different place.

But perhaps we don’t need the points. Perhaps it’s enough for participants to complete the assignments and 5 peer reviews to receive the Statement of Accomplishment. Of course, using this system, some people will receive the Statement of Accomplishment for exceptional work and some for simply submitting ‘any old thing’. But does that really matter, given that this course is not for Higher Education credit?

If it came to a choice between diversity and peer review – I would go for diversity, and trust that people are participating in the learning environment just as much as they want to and need to for their own purposes.

There is so much of interest in OldGlobeMOOC. As an educator myself I find this tension beween diversity and peer assessment very interesting, quite apart from the fascinating discussions about ageing.

Jenny, my own reaction to courses has been very similar. I had not articulated it quite that way, but you put the basic tension clearly and succinctly. Remember the truism about each class being a Gestalt with its own character and personality, almost like another student in the class (or bear in the living room)? That happens in the massive courses too. Sometimes the course mood can take a bad turn (exacerbated by that same tension) but not necessarily. Participants also take the initiative to form their own groups outside the course platform. These can continue after courses, sometime staying in place for the next iteration, other times morphing into something else ~ a book club, a cyber cafe.

HI Vanessa – thanks for your visit and comments. Yes – OldGlobe certainly has it’s own character. In terms of mood – there are naturally (as you would expect in a participant group of over 1600) some dissatisfied customers, but there have been comments in the forums from experienced Courser course-takers, that this MOOC has fewer aggressive posts in the forums than many of the others. This might be related to the nature of the subject and the recognised diversity in the course.