Salience and Virtual Reality

Salience

Salience, or the degree to which something catches your attention, shows up frequently in everyday life. The salience properties of objects guide your attention and direct your behavior to certain actions. You reach for the cookie with more chocolate chips because it stands in distinction from the rest of the cookies; the extra number of chocolate chips captured your focus and drew your attention in. The degree to which something is salient correlates largely with the degree to which you pay attention to that something and ultimately learn from that something.

Features of Salience

There are many things that may make something salient, most of which are inherent to the properties of the object. Evolutionarily, we have learned that certain things in the environment are more important than others, therefore requiring more attention—thus, those “certain things” evolved to be more salient. Think of a growl by a predator in the woods or the presence of dark clouds gathering in the sky: these things, and many others, when observed, immediately catch our attention because of their implications for survival. Was that growl something that will kill you? Will these clouds bring a dreadful storm? In essence, this is how salience developed and became ingrained in our everyday lives; things that have implications for survival and procreation are more salient to us. As time went on and we became a more complex species, we developed likes and dislikes for certain things. These too bring salience with them. We are attracted to that which we like and averse to that which we don’t.

But what about the salience of objects that don’t have survival implications or a relationship to our tastes? Certainly, things that fall outside these categories still catch our attention. Most prominently, objects or occurrences that defy our expectations are also salient. When we expect a dog to come out of the dog house, but instead a ferret does, it’s a highly salient experience. In fact, this is the basis for learning! When something doesn’t go according to what is expected, that something is made more salient to us, so that it creates an impression and therefore we’re more likely to anticipate it in the future. Thus, in this context, we’d now expect a ferret to come out of the dog house, because of how salient the first encounter was. In effect, we were able to learn from our prediction error.

Furthermore, objects that are augmented with properties that they wouldn’t usually have are also highly salient. For example, a coffee mug sitting on a table which, with hidden built-in speakers, could play Beethoven’s 9th would be more salient than just a (normal) silent coffee mug sitting on a table. This is highly related to the prediction error salience mentioned above: when an object is augmented with a property that otherwise would not be natural, that object is more salient than its non-augmented counterpart.

Why it matters in VR

With virtual reality, we can create our own environments; we can manipulate what is in the environment and most importantly, how it is presented. While nothing in VR may signify chances of survival or procreation (except perhaps some dinosaurs), we can certainly create scenarios that violate our expectations. More specifically, we can create objects that are augmented with additional properties that will catch our attention. Let’s call these salience features. Many of these salience features can be implemented trivially and don’t have to interfere with the virtual object at hand; in fact, they serve to strengthen our perception of the virtual objects to which these features are attached. In addition to the expectancy-violation of say, a coffee mug that is paired with a sound/piece of music, the added salience features, usually multisensory, serve to strengthen cross-network activation in the brain. In other words, the more sensory modalities—and thus, salience features—associated with a specific object, the more robust our perception of that object is. Accordingly, there is a deeper level of processing involved in the perception of salient objects, such that they are encoded more strongly and learned more effectively. So, if one were to create a learning experience in VR, they could imbue various objects in, and properties of, the learning experience with salience features. This means that a virtual object, such as a model of a brain, has multisensory properties which make that model more salient, and therefore more a) exciting and b) memorable. By incorporating salience features in VR, you can augment the user’s perception, such that they pay more attention to something, encode it more deeply, and learn it more effectively for the future.