Abstract

Multimedia environments consist of verbal and visual representations that, if appropriately processed, allow for the construction of an integrated mental model of the content. Whereas much is known on how students learn from verbal representations, there are fewer insights regarding the processing of visual information, alone or in conjunction with text. This chapter uses a semiotics approach to provide a defi- nition of visualizations as a specific form of external representation, and then discusses the differences between verbal and visual representations in how they represent information. Finally, it discusses how meaning is achieved when learning with them. The next section discusses basic perceptual and cognitive processes relevant to learning with visualizations. This background is used to specify the instructional functions that visualizations have either as self-contained instructional messages or as text adjuncts. Moreover, the role of individual differences in processing visualizations is highlighted. The chapter ends with methodological suggestions concerning the important role of interdisciplinary research and assessment methods in this area.

Introduction

Visualizations constitute a key component in multimedia-based instruction, which can be defined as learning from text and pictures (e.g., Mayer, 2005). Despite the fact that visualizations are used more and more frequently in informal and formal educational settings, not much is understood about their semiotic properties, how humans process them, and how they can be best designed to learn from. In educational research, visualizations are often treated in a uniform manner, despite the fact that the visualizations might serve vastly different functions depending on the audience and goals. Just as bad, visualizations are treated as functionally equivalent to text. As a consequence, reviews on learning with visualizations are equivocal, with studies showing widely varying effects (negative to positive) on learning. In the current chapter, we will try to provide a more differentiated view by first reviewing the literature from different disciplinary perspectives (education, semiotics, perception, and cognition) to characterize different types of visualizations, to distinguish them from verbal representations, and to describe how information is derived from them. This approach will attempt to provide a unique approach to addressing the question of when and why visualizations are effective for learning. After some summarizing remarks, directions for future research will be outlined in the final section of this chapter. It is important to note, however, that we will not review the more mainstream literature on the effectiveness of learning with visualizations, as comprehensive reviews can be found elsewhere (e.g., Anglin, Vaez, & Cunningham, 2004; Rieber, 1994).