In Daphne Koller’s TED talk about Massive Open Online Courses (MOOCs), she discusses what she believes are the pedagogical foundations of MOOCs. For her, and also restated on the Coursera website (the company she helped found), those foundations include the effectiveness of online learning, mastery learning, peer assessments and active learning.

Of course the evidence cited to support these foundations is scant and is quoted without giving any context to the research. MOOC critics also rarely produce any actual evidence when dismissing MOOCs as being pedagogically unsound or when stating that they offer a worse experience than face-to-face education on campus.

Given that my university (the University of Western Australia) was about to embark on hosting our own MOOCs and combine them with a flipped-classroom mode of teaching for our own students, a colleague (Martin Forsey) and I decided to examine the actual research evidence supporting (or not) the notion that MOOCs are based on sound pedagogical foundations. We published this in First Monday and what follows is a summary of our findings.

The MOOC fingerprint

The starting point for us was categorising the attributes that define a MOOC as they are presented on platforms such as edX, Coursera and Udacity. These were: online delivery of content, lectures formatted as short videos combined with formative quizzes; automated assessment and/or peer and self-assessment and an online forum for peer support and discussion. The pedagogical benefits of these characteristics of MOOCs translated into: the effectiveness of online learning, retrieval learning, mastery learning, enhanced learning through peer and self-assessment, enhanced attention and focus due to “chunking” content into small packages and finally peer assistance, or out-of-band learning.

Having drawn up this list of MOOC characteristics and their corresponding pedagogical effects, we reviewed the literature for empirical research to examine whether there was evidence to support any pedagogical benefits.

The Effectiveness of Online Learning

The most obvious characteristic of MOOCs is that they are delivered online. Reviewing the research, there is little evidence that online learning is any worse than face-to-face modes of teaching and there is plenty of evidence to suggest that it may in fact be better for some of the reasons that we are about to cover. Of course, this is not the general perception amongst US faculty staff at least. In a recent survey, 66% of faculty staff believed that online outcomes where inferior to traditional modes of teaching. The underlying subtext of why people believe that online teaching is less effective has not been extensively examined but certainly one would assume that it is related to a fear of technology generally which includes a fear of being replaced by that technology.

Retrieval Learning

MOOCs emphasise modes of learning that include retrieval learning and mastery learning. Retrieval learning acts to enhance long-term memory of facts by prompting recall of information from short-term memory. Each time this happens, it serves to improve overall learning as connections are made and strengthened each time there is a retrieval of information. In MOOCs, retrieval learning is enhanced through the practice of using frequent formative quizzes. Again there is evidence from multiple sources to support the notion that the use of tests and quizzes enhances learning over simply allowing students to listen or read content.

Mastery Learning

In regard to the benefits of mastery learning, this has a long history within education dating back to Bloom’s coining of the term in 1968. The basic premise is fundamental to the benefits of a site such as Salman Khan’s Khan Academy. Students are allowed to progress at their own pace until they achieve mastery of a particular topic and this may involve revisiting concepts that the topic is reliant on. Mastery learning is also enhanced by one-on-one tutorial style teaching and there have been arguments made that watching videos that can be controlled by the student gets closer to that mode of teaching than sitting in a lecture theatre.

Although there is evidence for the principle of mastery learning, there isn’t any evidence that MOOCs as they are currently implemented actually support this mode of learning. For the most part, MOOCs are being presented in the traditional timeline of different topics each week and are not especially structured to reinforce mastery learning compared with the Khan Academy for example.

Peer and Self-Assessment

When it comes to peer and self-assessment, there is general agreement that it is an effective means of marking. Assignments that are peer or self-assessed agree closely to those marked by instructors and tutors.

The evidence for this process having a learning benefit however is equivocal. Anecdotally, one would assume that giving students another task that engages them with the content and presents model answers would only serve to solidify concepts and knowledge. It is likely however that students do this anyway when instructors return marked work and hand out a marking template.

Short Videos

Stanford University lecturer Peter Norvig claimed that his decision to use short videos in MOOCs was inspired by the approach taken by the Khan Academy in keeping video length to 10 minutes or less. The argument was made that 10-15 minutes fits into the optimal time that students can maintain attention. However, on examination, the research that supports this assertion is particularly week. Also, Salman Khan has explained that the reason his videos are 10 minutes or less was because this was a limit imposed by YouTube when he first started uploading and not because of any pedagogical insight. There may be a benefit to short format videos, however more research will be necessary to provide evidence to support that notion.

Online Forums and Peer Support

Finally, there is the importance of the online forum to learning. There is little evidence to suggest that online forum discussions necessarily promotes learning in its own right. This is actually also the case for face-to-face discussions as well. What forums do achieve is that they facilitate the mechanics of a course by providing a means of unblocking students who are struggling either with content or some other aspect of the course. Online forums also can provide a knowledge base which accumulates over time. They are not currently being used in this fashion however and so this is something that MOOC providers could review in terms of the functionality offered by their discussion forums.

It is possible that when a course is constructed in a way to support group collaboration, online forums provide a mechanism for new knowledge to be created. This is the tenant of so-called “connectivist” style MOOCs, however again this awaits research to support this assertion.

So, are MOOCs an effective way to learn?

Overall, the evidence is that there is no reason to believe that MOOCs provide any less a valid learning experience than face-to-face courses. In many ways, they are simply a restatement of online learning environments which are optimised for large class sizes and modes of learning suited to todays digital milieu. When used for students enrolled in a university degree, they are usually combined with on-campus learning opportunities in a “flipped-classroom” style of presentation which brings the advantages of both environments.

What is exciting about the MOOC environment is that it will provide a rich opportunity to gather data that will tell us what does and doesn’t work and how students learn most effectively in as engaging an environment as can be provided. This will almost certainly mean that the current MOOC format will evolve rapidly over time as it is driven by this research supported by real data.

The review paper on which this article is based is available on First Monday