In 2012, and 2014 Libraries and Learning Services from the University of Auckland created two online courses to introduce students to the concept of academic integrity and its associated values and expectations. The challenge was to introduce the somewhat dry subject matter to a diverse group of students in an engaging way and to avoid large tracts of text that were difficult to comprehend. Initial research undertaken by the development team suggested that visually representing bodies of text was an effective way of communicating with the target audience. Dual coding theory, multimodal instructional design and cognitive load theories provided a framework for the learning design approach taken by the design team who ultimately selected Richard Mayer’s (2001) Cognitive Theory of Multimedia Learning as a basis for designing the two courses. This paper outlines how the two courses were designed in accordance principles outlined in Mayer’s theory and how the design team integrated seven of Mayer’s twelve principles to create dynamic, visual and aural elements such as animations, narrations and text to engage learners in a more meaningful learning process.

Multimedia objects, in a variety of formats are becoming more prevalent in the online learning environment (Sims et al. 2002). This is in part due to technology which makes locating and transmitting multimedia objects prevalent, and in part due to the increasing understanding of how multimedia objects contribute to learning and retention (Sims et al. 2002). This technology explosion has also increased the amount and complexity of knowledge and information to which we are exposed. Keller and Tergan (2005) see the use of visual objects such as images, videos and graphic novels as an effective strategy to assist reception, processing and organisation of this information explosion. Conversely if these types of multimedia objects are not used effectively in online learning settings they can result in cognitive overload and hinder deeper learning.

A number of researchers have developed frameworks to assist with effective application of multimedia elements into online learning resources. This paper examines one of these: Mayer’s (2001) Cognitive Theory of Multimedia learning and seven associated principles. The paper outlines how the theory and principles were applied to the design of two online courses on academic integrity developed by Libraries and Learning Services staff at the University of Auckland.

Multimedia is defined by Mayer as the presentation of information which contains two different: elements: textual elements and a pictorial elements (as cited in Keller and Grimm 2005:169). In turn these elements are committed to working memory through two different channels: a verbal channel and a visual channel. Of these two channels the visual channel is the most powerful with the brain processing visuals 60,000 times faster than text (Burmark 2008). Visuals also provide a depth of information in a split second which would require a large amount of text. They allow communication across cultures and convey emotion in a way which is more difficult in verbal or written communication (Burmark 2008). More importantly perhaps, visuals increase engagement, and students’ willingness to read accompanying text and can improve recall and retention by over 75 %, as visual imagery is more readily committed to long term memory (Burmark 2008).

Best practice for instructional design discourages too much text (Prensky 2001). In other words, within the context of online design the expectation would be that text plays no greater role than any other aspect, and should be consciously minimised in order to keep learners engaged. An online course consisting of pages of text would be considered to be of poor design.

Prensky (2001) further states that a variety of skills are developed by exposure to online activities such as games and other media. These skills include amongst others: mental mapping, visual literacy and constructing three dimensional mental models out of two dimensional images.

These skills (above) are the result of learners interacting with a mixture of elements on the screen: without this variety, interacting with text alone can result in deficits in understanding and information retained. It is possible that it is simply the result of the fact that learners are now reading off a screen more often, necessitating a different approach, rather than any inherent difference in the brain of a 21st century learner (Prensky 2001). As Jabr (2011:para 27) states: “When reading on screens, people seem less inclined to engage in what psychologists call metacognitive learning regulation—strategies such as setting specific goals, rereading difficult sections and checking how much one has understood along the way.”

Dual coding theory

According to dual coding theory, learning is maximised when both visual and verbal systems are activated. Dual coding presumes that multimodal instructional design is superior to unimodal design models as using both verbal and visual channels to present information leads to more processing of information than using only one channel. Learning is further maximised because processing with the two channels forges a connection between the channels which reinforces a cohesive mental model (Gellevij et al. 2002).

Redundancy and split attention effect

Sweller et al. (1990:217) extend this idea with reference to redundancy and what they call the split attention effect. When the same information is presented both verbally and visually and users can choose to read either the text or view the same information graphically, learning is slowed down because their attention is split. Either the text or graphics can be viewed as extraneous. Therefore according to Sweller et al. (1990) in order for learning to be optimised text must be dependent on and complementary to graphics and vice versa.

Cognitive load theory

Cognitive load theory (CLT) provides a framework to investigate the relationship between how information is structured and presented and the cognitive processes which enable people to receive and store that information (Paas et al. 2003). CLT theory specifically relates to working memory and its capacity for cognitive processing. Working memory possesses two subsytems, one for retaining visuospatial information and one for phonological information. Working memory performance specifically relates to cognitive ability and the capacity to learn (de Jong 2010).

CLT fosters better understanding of instructional design as presupposes that learners have a limited and selective attention span and that flexible instructional methods using a rich variety of media best supports effective learning (Zhang et al. 2006).

Oud (2009) indicates that there are three key types of cognitive load which instructors need to be mindful of: intrinsic (related to content), germane (related to activities undertaken by students), and extraneous (all other factors). Mayer and Moreno (2003) indicate that in order for meaningful learning (deep understanding) to occur a high degree of cognitive processing must occur. Without careful instructional design this can lead to cognitive overload as the cognitive load exerted by a particular learning activity may exceed the cognitive processing capacity.

Discrete multimedia learning objects such as comics, images and videos can be parcelled up in a way that reduces cognitive overload and a number of researchers suggest frameworks and structures for better understanding how multimedia elements should function and interact effectively in instructional settings. Early work by Sinatra (1986) equates creating visual messages with creating written messages. Visual messages combine a number of elements such as: objects, items, symbols, placement, light, angle, and tonal qualities to communicate a particular message or emotion in the same way that a writer uses words, sentences, and paragraphs to create a particular style of writing.

Kress and van Leeuwen (1996) similarly suggest that the study of visual communication should parallel the study of language-based communication, with a corresponding lexicon approach as well as a grammatical approach which includes: typeface style, framing, salience, colour and hue. Machin (2007) however questions this multimodal approach, as images and language work in different ways. He suggests that a grammatical approach is more relevant as it looks at how images interact with each other and text whereas a lexical approach emphasizes the image as an independent object free from interaction with other images and text.

The Cognitive Theory of Multimedia Learning developed by Mayer (2001) perhaps best addresses how verbal and visual messages align to ensure deep learning as well as principles for how multimedia items should be used in instructional design to ensure this and avoid cognitive overload. His theory is based on three assumptions: dual process (the brain uses separate channels for processing verbal and visual information), limited capacity (the brain can only process small amounts of information at a time) and active processing (in order for learning to be meaningful information must be assimilated, and organised into meaningful representations based on prior knowledge and experience) (Mayer 2001:67). Mayer argues that in active learning, five processes take place: selecting words, selecting images, organising words, organising images, then integrating the information. He suggests seven principles for use of multimedia elements in instructional materials: multimedia, coherence, spatial contiguity, temporal contiguity, modality, redundancy, and individual differences. Each of these elements will be discussed below to demonstrate how they were applied, their impact and any challenges in application.

Background to the online course design

In 2012 and 2014 respectively, the University of Auckland created two online courses about academic integrity. The first, simply called Academic Integrity was an online course created exclusively for University of Auckland students in order to improve students’ knowledge regarding appropriate academic integrity behaviour (University of Auckland 2012).

The second course was designed as a Massive Open Online Course (MOOC) in partnership with the company FutureLearn in the UK and was entitled Academic Integrity: Values, Skills, Action (University of Auckland 2014). The MOOC was created partly in response to enquiries from higher education institutions in New Zealand and internationally who wished to use the 2012 course in their own institutions. As the original course was designed with specific University of Auckland information in mind, content was not readily transferable to the MOOC environment. The MOOC was designed with a global audience in mind and dealt with the principles of academic integrity rather than specific institutional policies and procedures.

Academic integrity tends to be viewed by students as a rather abstract, dry subject area. However, traditionally, students at university level are expected to cope with large tracts of text about abstract concepts (Prensky 2001). Despite the different emphasis for the two courses, the approach the design team followed was similar. The step by step process of translating lengthy policy documents into a lively online course has been outlined in detail by Cook et al. (2015).

Five rounds of student usability testing across both courses were conducted to inform the online course design (Wang 2012). Usability testing provided clear evidence that students did not read large tracts of text, did not expect to encounter them and preferred not to deal with them. Additional findings from usability testing included students’ preference for ‘learning by doing’, multimedia, and interactivity, rather than large sections of text. Based on these findings, text-heavy content was replaced by interactive online activities, scenarios, and multimedia (Cook et al. 2013).

Students liked the use of images to translate difficult information into a “more easily digestible format” (Cook et al. 2013:147). These findings fit well with the educators’ growing awareness that the current generation of learners comes from a highly visual culture. While this does not exclude text as a teaching medium, it encourages the educators to balance it with visual elements (Blanch and Mulvihill 2013).

As a result of research and usability testing the design team were particularly drawn to Mayer’s (2001) theory and principles regarding the effective combination of words and images to encourage deeper learning by students. They mirrored the feedback received from students through usability testing and were intuitive and straightforward to apply to the design process.

Discussion of Mayer’s principles and their application

Mayer’s seven principles and how they were applied to the two courses are outlined below:

Multimedia principle

“A combination of text and visuals leads to better learning than just text” (Mayer 2001:63).

Rationale

When visuals and words (text) are presented simultaneously, learners have the opportunity to construct mental models of both and build connections between them, resulting in a richer learning experience than when words are presented alone. This is also a key tenet of social constructivism and connectivism, the application of which is discussed in more detail in the article by Cook et al. (2015).

Practical application

An examination of the original policy documents upon which both courses are based indicates the degree of material which has been transformed into visuals. This is at the heart of Mayer’s first principle of multimedia design and was integral to the design team’s approach. Student feedback solicited through usability testing in early iterations of the course resulted in the team actively discarding tracts of text and /or converting suitable material to images/comics (see Fig. 1). Usability testing conducted after a combination of images and text were introduced reinforced the positive impact of this multimedia approach and the increase in students’ understanding and engagement.

Fig. 1

Segment of the comic from the Academic Integrity course showing the procedures for handling academic misconduct by the University of Auckland 2012, Academic Integrity. Copyright 2012 by the University of Auckland

Comics were used where appropriate, as this medium allows for the integration of text, image and narrative. Reading comics involves both visual and verbal processing, aiding comprehension and reducing cognitive overload as opposed to traditional text-heavy textbooks (Weiner and Syma 2013). Graphic novels, in particular, have been shown to be highly effective for communication of key concepts, strategies and messages (Short et al. 2013). They increase student engagement and motivation, improve recognition of key ideas, and provide a forum where strategic messages can be aligned with context and real life or experience-based situations. They are beneficial for demonstrating interpersonal interactions which is not possible in a traditional textbook. While instructional materials such as textbooks engage the student at a cognitive level, graphic novels also engage students at an emotional and behavioural level (Hoover et al. 2010).

Spatial contiguity principle

Rationale

When words and pictures are placed in close proximity on the page (screen), fewer mental resources are needed to remember the elements, leaving more room to build connections between the elements. This allows learners to hold more information in their working memory at the same time.

Practical application

This principle could be said to be intuitive, although Mayer’s reasoning is that the meaning is created between the connections the learner makes between the two, meaning image and text inform and augment each other (Mayer 2001). An example of this in the original academic integrity course was to use personae or student characters to “speak” the abstract concepts. Rather than stating in text the concepts that underpinned academic integrity, it was presented as a speech bubble in the characters own words, demonstrating their understanding of the concept. The mental connection the learner forges between text and image is what Mayer believes gives rise to a deeper understanding. By encouraging learners to hold them in their working memory simultaneously, the interplay, and ultimately sense-making is more likely to happen. Additionally in this scenario, the learner can identify with the student character: it could easily be the learner speaking.

Another application of the Spatial Contiguity principle is the use of graphic advance organisers (Mayer 2001). These are cues about the content which can be incorporated into images to ensure that both the visual and the verbal mode are engaged. The process of ‘unpacking’ the meaning of the image, and content, is intuitive (Burmark 2008). The AI MOOC used both text and the facial expressions of student actors to signal the nature of the material to follow (see Fig. 2). This is also known in multimedia learning as a signal principle and stems from the idea that multimedia materials are more effective if there is a cue embedded to guide or prepare the learner or guide their attention to a relevant part of the material (Mayer 2001).

Fig. 2

A student worriedly considering referencing. An example of graphic advance organiser in the Academic Integrity MOOC by the University of Auckland 2014, Academic Integrity: Values, Skills, Action. Copyright 2014 by the University of Auckland

Temporal contiguity principle

Text and visuals - or narration and animation need to be presented together (Mayer 2001:p.96).

Rationale

This also related to reducing the load on the working memory. If words and images are presented simultaneously instead of separately, less effort is required by the learner to hold the information in their working memory, leaving more resources available to make connections, leading to a deeper learning experience.

Practical application

In both courses, videos and animation included verbal or visual narration. This assisted cognitive selection and organisation processes. Presenting information in two different ways allowed students to hold both pieces of information in their working memory simultaneously. This increased the likelihood of information being effectively assimilated and integrated. In this way simultaneous information was “designed to mesh with the human information processing system” (Mayer 2001:100). For example, in the academic integrity MOOC, the voice and image of the Lead Educator was combined with key terms that would be used throughout the course, with the on-screen text and the overlaying narration reinforcing the terms and the message (see Fig. 3).

Fig. 3

A still from a video used in the Academic Integrity MOOC by the University of Auckland 2014, Academic Integrity: Values, Skills, Action. Copyright 2014 by the University of Auckland

Coherence principle

Rationale

Extraneous information is problematic on a number of fronts: it crowds the working memory leaving fewer resources to process information, can interfere with the information organising process and can cause confusion about which theme is more important.

Practical application

The coherence principle has three key premises: student learning is compromised when interesting but unnecessary words and images are presented; student learning is compromised when interesting but unnecessary sounds and music are presented; and student learning is improved when these unnecessary elements are eliminated. In both online courses, screen space or “real estate” was as important as the need to streamline the content for learning optimisation. As well, as Mayer (2001:82) states, “space is an economic resource.” Acknowledging that a page or a screen does not have optimal space to present all the information required for learning to take place gives an additional incentive to remove extraneous material. As well as following Mayer’s coherence principle this streamlining also just makes good graphic and instructional design sense.

Comics provide a way to bundle information and present more of it without using large amounts of text. A balance of text and images is intrinsic to graphic comics (see Fig. 4). Designers must limit the amount of text used due to the parameters of the graphic comic medium.

Fig. 4

A definition of copyright presented in comic format from the Academic Integrity course showing the procedures for handling academic misconduct by the University of Auckland 2012, Academic Integrity. Copyright 2012 by the University of Auckland

The challenge is how much material to remove before the balance is tipped into oversimplification. However as long as the images are fully engaged in delivering the content as economically as possible this serves as a reasonable guide in deciding what is essential and what is extraneous. The most problematic area for these two courses was text. For both courses it was tempting to squeeze more and more information into articles and text boxes and the designers required discipline to avoid this. This is also where ongoing user feedback during the design process was useful. If there was any argument as to what to include and what to leave out, the learner had the final say through user feedback.

Modality principle

Rationale

Information delivered via spoken words and animation rather than written text leaves the visual/pictorial channel available to process images, meaning overload is avoided and deeper learning is more likely to occur.

Practical application

Both courses contain spoken words and animation. On screen text was also used in conjunction with these elements and the reasons for this will be discussed in the next section.

In the 2012 Academic Integrity Course combining both a student’s face and voice allowed abstract definitions to be made more visually memorable (see Fig. 5). It also allowed designers to present the new knowledge in a medium that students were confident in and comfortable with i.e. brief videos. This mode of delivery also encouraged learners to construct new knowledge by connecting video content with their existing beliefs, understanding and experience.

Fig. 5

Stills from a video explaining plagiarism to learners from the Academic Integrity course showing the procedures for handling academic misconduct by the University of Auckland 2012, Academic Integrity. Copyright 2012 by the University of Auckland

Similarly, in the academic integrity MOOC, the presence of a lead educator in the MOOC videos served several purposes. It allowed designers to present abstract concepts that formed the basis of the subject area of Academic Integrity. It fulfilled the requirements of the FutureLearn platform design and provided a cohesive approach and consistent presence across the entire course. It also provided additional information (skills and examples which would assist learners to avoid academic misconduct) without overwhelming learners.

The use of a “Lead Educator”, that is, an identifiable subject matter expert to provide expertise, guidance and bring a sense of coherence and uniformity across the course, was a requirement of the FutureLearn MOOC platform. The Lead Educator, Dr. Jason Stephens, a Senior Lecturer at the University of Auckland and Educational Psychologist, guided learners through the videos and exercises, commenting on content and prompting learners to reflect. The aim was to encourage learners to think about their own situations and apply the course content to their own ideas. User feedback indicated that the consistent format and the Lead Educator’s presence were appreciated in that they provided a predictable framework for learners to interact with the material. Scripts for the videos were written to emphasise specific key words, e.g. in the first week where the course and subject foundations were being laid, the words “fundamental”, “fairness”, “ethics” and “integrity” were emphasised. This reinforced other instances of these same words, whether spoken, written, or appearing in the video transcripts.

Compared with the Academic Integrity course platform, the FutureLearn MOOC platform provided a narrower range of design elements. The MOOC only allowed information to be presented in one of four ways (video, image, text, narration). Feedback from MOOC learners indicated that less narration would have been preferred and that videos over 2.5 minutes long would generally be avoided. This was a challenge however as the design team were unable to use some design elements utilised in the Academic integrity course including graphic comics, hovers and rollovers, images embedded in text and interactive exercises.

Rationale

Practical application

While the design team were aware of Mayer’s sixth principle, they deviated slightly from exact adherence to this with regards to the inclusion of the on-screen text alongside animation and narration in the MOOC. The theoretical rationale for application of Mayer’s Redundancy Principle is to avoid overloading of the visual channel, however in Week One of the MOOC a video was used to outline the six principles of the Fundamental Values Project. Inclusion of text in the video was deemed necessary to allow the course designers to convey threshold concepts at the right time. The six principles added as text were also the founding values of the whole course and purpose of the MOOC, so it was essential that they were emphasised. As they were also abstract concepts, and hard to grasp for novice learners, emphasis through both verbal and text channels was considered essential.

Both courses included material about academic writing practices such as paraphrasing and quoting. By their very nature, any examples of this material must be presented textually. This may be hard to reconcile with Mayer’s advice against combining animation, narration and on-screen text. Yet introducing narration and visual elements (animated highlighting of the key points in a block of text) allowed a text-heavy example to be broken into discrete visual parts, thereby hopefully reducing the risk of cognitive overload (see Fig. 6).

Fig. 6

Visual ‘highlighting’ effect combined with narrated explanation in a video from the Academic Integrity MOOC by the University of Auckland 2014, Academic Integrity: Values, Skills, Action. Copyright 2014 by the University of Auckland

Individual differences

Rationale

High knowledge learners (possessing high pre-existing knowledge) can use their existing knowledge to compensate for lack of visual cues/guidance, while low knowledge learners lack the prior knowledge that allows them to do this. Similarly high spatial learners, that is, learners who possess the ability to integrate verbal and visual messages from a presentation, have more processing capacity left over to deal with processing simultaneously presented images and verbal cues than low spatial learners. Low spatial learners must devote more of their processing capacity to “holding the presented images in memory” and therefore have less spare capacity for integrating verbal and visual information (Mayer 2001:161).

Practical application

Both courses were primarily geared towards low knowledge, low spatial learners, and as such it was important to provide alternative methods for presenting the same data to allow learners to utilise single cognitive channels (either visual or verbal) to assimilate information when necessary. This was done in a number of ways: both the academic Integrity course and the MOOC provided transcripts of all videos. This was essential anyway, as both courses had a high number of “English as second Language learners” but this approach also catered to learners who lacked the processing ability to pick up on visual cues and adequately assimilate verbal and visual content simultaneously. Additionally, as well as presenting information with a combination of visual techniques (video or images) and text simultaneously, longer text based explanations covering both the verbally and visually presented information were provided to allow single verbal channel processing (see Fig. 7).

Fig. 7

Example of a “Golden rule” providing a single channel verbal approach to aid assimilation and integration of information by low knowledge, low spatial learners, from the Academic Integrity MOOC by the University of Auckland 2014, Academic Integrity: Values, Skills, Action. Copyright 2014 by the University of Auckland

Application of Mayer’s Theory and Principles of Multimedia Learning allowed a systematic approach to the design of two academic integrity courses developed by the University of Auckland. It complemented findings obtained from extensive usability testing about what works for students when assimilating, integrating, retaining and creating deeper meaning from instructional materials. Additionally, it provided reassurance that the designers were catering appropriately to a wide range of learners through integration of visual and verbal elements in appropriate and effective ways. This approach will be used for development of additional online courses by the design team.

Future instructional design activities may also be enhanced by further investigating the alignment between Mayer’s theory and other cognitive instructional, social constructivist and connectivist learning theories, particularly as applied to e-learning initiatives (Virginia Tech, School of Education n.d.).

Authors’ contributions

Tricia Bingham and Stephanie Reid carried out the literature review and drafted the manuscript. Vanda Ivanovic contributed to the manuscript and selected the figures to illustrate it. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Blanch CL, Mulvihill TM (2013) The attitudes of some students in the use of comics in higher education. In: Syma CK, Weiner RG (eds) Graphic novels and comics in the classroom: essays on the educational power of sequential art. McFarland & Company, Jefferson, pp 35–47Google Scholar

Sims E, O’Leary R, Cook J, Butland G (2002) Visual literacy: what is it and do we need it to use learning technologies effectively? In: Winds of change in the sea of learning: Proceedings of the 19th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education. ASCILITE, Auckland, pp 885–888Google Scholar