Learning analytics, MOOCs and the Examinations Factory

Gillian Fielding
Learning and Skills Development Manager
University of Salford

Reflections on the first day of the EUNIS conference

I have to say how delighted I am to be here, and thank you to UCISA for awarding me this bursary award. I confess I had not heard of EUNIS prior to the award and have rapidly come to realise how much we have to learn from this organisation. .

Expect the unexpected

And what a start to the Congress! Gerry Pennell from Manchester University gave a fascinating presentation about how he managed the IT for the London Olympics. IT Directors wouldn’t be surprised by his content but for me as a Digital Skills Manager there was so much in it. Having said that, the key difference from HE, and maybe of interest to IT Directors, was the deadline. It could NOT slip. Also of interest was everyone’s motivation in taking those roles considering they knew that the role would end immediately after the event. Understanding what drove people helped.

Other lessons for our sector: “expect the unexpected”. Cyberattacks were expected and ‘easy’ to deal with, wobbly poles caused by screaming crowds were not. Keeping calm and addressing issues was essential. It was essential to resolve the wobbly pole issue so potential photo finishes in the 100m finals were discernible. Gerry pointed out that projects can be delivered on time and should not drag on. Staying focused on the ‘must haves’, not getting side-tracked with bells and whistles, were key factors.

In year, immediate use of learning analytics

Use of learner analytics to go beyond retaining students was covered by Cheryl Reynolds from Huddersfield University in her really thought-provoking presentation. Cheryl and Cath Ellis used data from Turnitin’s GradeCentre to alter the curriculum during the module to suit immediately evident needs. Examples included students not using subject specific literature, incorrect use of possessive apostrophes and having poor, or non-existent, introductions. Data from GradeCentre was immediately analysed to form part of the next lesson on whatever was needed to find those that were closely associated with success. The data was presented to the students, carefully, so they could learn and improve. Cheryl pointed out this needs to be done discreetly so as not to embarrass anyone or negatively impact the low achievers.

In his presentation entitled ‘Blowing Backwards into the Future of Higher Education’, Anders Norberg of UMEA University pointed out the need and some of the issues we need to address to change our thinking about teaching and learning. He illustrated this by saying we often use ‘transfer learning’ in education but outside education people learn by ‘experimental learning’, suggesting this is an unsustainable mismatch. Anders raised many thought-provoking points, including that we think of learning in rooms, and that defines how we approach content and timescales. We need to move away from this, from ‘courses’ to ‘learning expeditions’.

MOOCs

Many aspects of MOOCs were covered, practical and otherwise, in the presentations from Yves Epelboin and Juan Antonio Martínez Carrascal. The number of MOOCs are rapidly, very rapidly, increasing. The main player is the Spanish ‘Miriada’ who are big in the South American market, ‘FutureLearn’ is next, then the French ‘Universite Numerique’ and the German ‘iVersity’. Interestingly, iVeristy has said there is no (MOOC) business in HE, they are focusing on working with the private sector.

Student retention – no one knows how many get to the end, some are lurkers and some are not interested in the quizzes and assessment. There is no research on what people want out of MOOCs. Flipped learning and SPOCs had benefits for HE in that they allowed academics to start dipping their toes in the waters of what’s needed for MOOCs. The advice from both presentations for HE was to start getting into MOOCs.

Prof Mark Stubbs’ presentation was on Manchester Metropolitan’s Eunis Elite Award-winning EQAL project to implement major system changes across the institution to easily enable students’ wishes for ‘engaging and well-organised courses’ and ‘inspirational tutors who know me’. The project saw the student information system, Moodle and timetabling all talk to each other and give students their timetables, amongst other things, on their mobile devices. The project included single course codes, five well-written learning outcomes with employability skills and provided academics with useful analytics, and took four years to complete. I’m not surprised it was award-winning.

Blackboard’s Dan Peters’ presentation on the opportunities that ‘selfies’ offer us, was another thought-provoking session. Dan gave a convincing case that selfies were evidence of students wanted to produce and that we should monopolise on that in education. Dan cited Stanford who had interestingly made lecture attendance optional after introducing the flipped classroom and problem-based learning approach. They found that attendance increased. They didn’t produce their own recording but used OERs.

Reasons for academics not to engage were discussed e.g. time, skill, however the arguments for were compelling: use of OERs; recordings don’t need to be BBC quality; students want short, engaging content.

Assessment and electronic assessments formed the final sessions of the day. Gill Ferrell pointed to the fabulous resources that JISC have produced to guide staff when considering or reviewing assessment. Possibly the most important point in this session was that peer reviews have a high impact on learning, though we need to explain to students why they are doing peer assessments, especially as it is not what they expect of HE. Gill also flagged the new JISC mailing list.

Copenhagen University and the Swiss Federal Institute addressed one of the sector’s key strategic challenges (see UCISA’s Strategic challenges for IT Services publication) by sharing their interesting practice in on-line assessments. Both are undertaking large numbers of on-line assessments in Copenhagen in the ‘Examinations Factory’ – a dedicated building. Practical issues covered cheating but little new here, looking over shoulders was still the same as when I was at school and did my 11+ (a long gone UK test for 10/11 year olds). Randomised questions helped mitigate the possibility of cheating. The other issues they’d had were also not new to exam cheating e.g. bringing data/devices (USBs and mobiles etc) into the testing room, and were easily combatted with locked systems and mobile detectors.

The benefits were improved quality, learning outcomes and efficiency, as well as easier to read exam scripts, so well worth it. Average cost €21 per exam. Many different types of exams were held, the obvious multiple choice, also ‘written’ exams, task based ones including conducting an Information Literacy search. The only exam types that were not possible were hand sketched submissions and ones which included written formula.

A great but exhausting first day. In the land of the ‘summer light’ I shall be getting an early night to rest my challenged brain.

You can follow Gillian’s Tweets @g_fielding, or follow the conference hashtag #eunis14.