Interview

Often you are described as a scientist working in the field of integration of cognition and emotion. Can you explain what that means or give some real-world examples?

A lot of people are interested in understanding emotion as a single entity, as a specific domain of knowledge, or an object of study that you can study in itself. They might call themselves emotion researchers. I don’t think I’ve ever called or thought of myself as an emotion researcher, because I’m interested in how emotion affects other things. I’m also not interested in the other things in themselves, but in how they together with emotion lead to complex behaviours that we display in real life. This is obviously studied in a laboratory setting, so it’s well controlled and much simpler. What I’m interested in is how an emotional experience, for instance, anticipating a mild shock or the viewing of a very aversive image, like a mutilated body, how that affects perception, how we view and see the world, and how that affects cognition, how we go about planning, remembering, expressing things, processes that we call cognition.

One of the directions of your work is focused on attention. How do emotions influence attention?

The work that I’ve done on attention has two sides. One side deals with the claim that things that are emotional are so important that they should be processed without attention. Let’s say you’re walking in the forest, and you see something that could be a snake. You react immediately and are not aware why you reacted, you weren’t even paying attention to the snake, but before you knew it, you were running away from something and only then realised that it was a snake, and your heart is racing and so on. So perceiving and processing stimuli that have emotional significance such as a spider, a snake, an object that comes at you, a looming object as we call it, or something out there in the forest that is out to eat you, is supposed to be automatic, which means that it doesn’t depend on where you pay attention and you don’t even have to be aware of what’s going on. In a way this idea is intuitive. It makes sense that you would run away from the snake very quickly, but is it really true, that it does not depend on attention? This is one aspect of attention that I have studied. To answer that question more precisely we have to test it with laboratory methods, specific methodology that precisely controls how attention is allocated in one region or another region to one object or another object. After looking at this question with laboratory methods, we find that the processing of emotional stimuli like a snake or a spider does require attention. It is fast and it’s powerful, but it’s not automatic in the sense that it does not depend on attention or awareness.

But for that effect to take place, you need to burden the cognitive system?

I wouldn’t phrase it that way in terms of burdening the cognitive system because I don’t view the cognitive system as something separate from a separate emotional system. I think there are several pathways and several ways in which information is conveyed in the brain. Some are faster and some are a little bit slower but it’s not necessarily a matter of burdening the system. Some types of communication in the brain are very fast, for instance the visual system, which is part of the “cognitive system”, is very fast and very efficient at doing all sorts of very complex computations both with emotional stimuli but also with non-emotional stimuli. We get at an answer very quickly using “cognition” or the normal visual system apparatus because the system overall is very efficient.

But that is just one side of attention that I have studied and you can view it as two sides of a coin. If objects have emotional significance, for instance something was paired with shock in the past, or you had an encounter with a certain aversive element in real life, a snake or a spider or some kind of bug in your house, something that was aversive and unpleasant and negative, that element acquires affective significance that is actually going to affect how you pay attention to objects in the world. For instance, I’m from Brazil and we have lots of spiders and snakes. So let’s say you were in a village in Brazil with lots of snakes. Some of them are very poisonous and have a little red texture on their back. Let’s say that every time that little snake came to the village your family reacted with screaming and shouting, pulling you away from the snake and running away. The snake would acquire affective significance, and the next time you would see something with some of those features, it wouldn’t even have to be the snake, but something that is elongated and red, your system would be tuned to respond very fast to that kind of object. The processing of those kinds of stimuli is prioritised. We don’t have infinite processing capacity, we can’t process everything at the same time, so certain things are going to grab your attention more than others – they have more salience. One way to have salience is something red or something moving or something with specific features that are important for you…

…or being emotional.

Or being emotional by being previously associated with something that was aversive to you. Maybe you got bitten by that snake and even if the snake isn’t poisonous it would be an aversive experience or it could be that the people were screaming when that kind of snake was around. Because of that you direct your attention to those kinds of objects much more effectively and faster. The object is highlighted among all other objects in the visual world so that you take care of it before you pay attention to other things.

So these are the two sides of my attention research. One is that emotion makes you process these things faster and more efficiently, but at the same time it’s not magic, you still require some attentional resources. It doesn’t happen mysteriously; it involves the same circuitry we use to see and process other things in the world.

Cognition and emotion are often seen as two ways of processing information, so you can process something in a purely cognitive way or in a purely emotional way and these two ways compete with each other, but your findings suggest that there can’t be such a clear division.

I don’t think there is a clear division. Obviously there are some ways in which we can act in a very rational way, maybe when playing chess, following some rules or trying to solve a mathematical problem and in that strict sense we’d be doing something more cognitive. But I think that generally those are the exceptions and even when you play chess you are under an emotional state. Maybe that opponent recently beat you and you really want to win, so even those cases that can be thought of as more rational or more purely cognitive are not entirely pure.

In general there is no such separation and the anatomy shows just that. Regions in the brain are connected to each other through fibre tracts that you can imagine as basically highways that take information from one place in the brain to another place in the brain. The degree of connectivity in the brain is so intense that information is flowing from all parts to all other parts in the brain – it really doesn’t support the isolation of certain areas. There really aren’t any places in the brain that are little islands that don’t receive signals from other places. In many cases they receive signals from only one or two synapses away. In the case of regions that are typically thought of as being cognitive or emotional such as the amygdala for fear and lateral prefrontal cortex for higher cognition, the argument that there is no direct connection between those two areas is often used to oppose the notion of integration. But even if there is no direct connection it only takes one additional connection for the signal to get there which adds approximately 10 ms. 10 ms is a tiny little fraction that won’t affect anything in the human life. Even in the case of survival out in the jungle so to speak, that additional time might not be so devastating. It’s probably better not to have those 10 ms if a rat is trying to escape from a snake but even in that case the communication can still be really fast because of the high degree of interconnectivity. In 99% of the situations except maybe in really extreme cases of literal animal survival those differences, those tiny differences will not make a difference.

The bottom line is that there is no such isolation in the brain. What we find is actually a remarkable potential for communication and integration. The tools that we use to study how the internet is connected or how a virus spreads throughout the world as people take airplanes or how the stock market in one country influences all stock markets in the world rapidly, within the same day or even just within minutes, all these tools we can also use to study how brain regions are connected to each other. One of the things we see is a remarkable architecture in which every region in the brain is separated from every other region by only a few degrees of separation. The potential for integration is huge and this fact is disregarded by people who claim that there is a strict separation between emotion and cognition. This idea is a baggage that we have from centuries of philosophical thinking from Descartes and all the way back to Plato and other thinkers, but once we start looking at the brain it really doesn’t seem to be organised in a way that can support that separation.

Some ideas tend to take a strong hold not only in scientific circles but also in society. Lately it seems that almost everyone knows that we have a limbic system that is a part of the brain that is evolutionary primitive and is where emotions take place. How is this view wrong?

I think this view has captured the public’s imagination because we’ve always been fascinated by the apparent duality of our nature. It meshes well with a kind of British Victorian notion of higher centres controlling “the animal within”. This is a popular theme in literature as well, Frankenstein for example. The limbic system speaks to that notion, to the idea that there is some inner core in us that is very animal like. Well, first of all we are 100% animal like because we’re just one little ramification of a long series of evolutionary trajectories that lead to everything that is alive now. But also the idea of a separate limbic system was supposed to be based on a series of facts and ideas that simply have not held in research. It was popularised by Paul MacLean in the late 40s and all through the 60s and 70s when he wrote follow-up books on the idea. He wrote about the triune brain with a simple reptilian core, a more developed mammalian brain surrounding it and finally a neocortical brain at the top. These notions are fascinating for culture, for movies, but if you talk to specialists in brain evolution and read their papers you see that there’s very little basis for organising the brain in that way as completely separate inner cores building on top of one another until you get pure rationality which is the essence of an ultimate pure human being.

That’s an idea that somehow has a literary and psychological pull, but it doesn’t hold scientifically on many grounds. First of all, the evolutionary ideas that the limbic system notion was built upon have all been challenged. For many of the regions in and around the medial cortex like the hippocampus, hypothalamus and the anterior cingulate gyrus there’s little evidence that they are truly older than other parts of the brain. The idea that there’s a sequence, that we went from more primitive to more elaborate and that the primitive part is the emotional brain is wrong. For instance, parts of the brain that are really important for emotional experience like the anterior insula or even other parts of the insula including medial insula and posterior insula actually have little to do with what would be called the old brain, so there are many examples that show that what is old is not necessarily emotional and what is new is not necessarily cognitive. For instance our sensory systems are extremely old. Primates that diverged evolutionary from the line of humans 30 or 40 million years ago have a basic sensory structure of cortex that is very similar to ours, for example, they have primary and secondary visual areas, and other visual areas. The notion of old and new is much more complex than the simple view associated with the limbic system.

Another problem with the limbic system is that we can’t define it. Every time we come up with a definition there are always some shortcomings with that way of defining the limbic system. It’s a concept that has attracted a lot of attention and has been used over and over in neuroscience but it hasn’t been a concept that has helped us advance our understanding of brain organisation and how brain organisation supports behaviour. In that sense, not only is it an idea that has always been in flux, I don’t see it as a concept that has helped us understand behaviour better, which is the goal of neuroscience. We want to understand the brain, but the reason we want to understand it is because we want to understand behaviour, the way we behave.

Is there such a thing as an emotional brain, or is it simply that the whole brain is emotional?

I think this is where the question gets really difficult. If one starts like me to say that there are no regions that are purely emotional or purely involved in emotion, then one could argue that what I’m saying is that everything is equal, the same. There are no specialised regions for emotion but at the same time everything is not the same, so how can both be true? I think one way that we can think about it is that regions do have different roles. In the 1950s (and earlier) there was a notion that the brain or at least the cortex was an equipotential mesh that every part of the cortex was indistinguishable from each other. Karl Lashley for example stated that the amount of behavioural deficit that the animal would incur was proportionate to the amount of cortical tissue that was taken out. It didn’t matter where it came from, just how much cortex you took. We know now that that is far from the way the brain is organised. The brain has a lot of structure but at the same time a lot of communication as well, so the mapping between structure and function is extremely complex. It’s not the case that a given region is doing one function. How this region is collaborating with other regions at that moment is what leads to a function. In terms of your original question, what is in the brain that is emotional, I think that certain regions working with other regions in certain ways create things that we label emotional.

For example, certain regions like the amygdala are very important to assess the significance of visual items. If you suddenly encounter a snake, your systems are going to be put on alert very soon, your attention is devoted to the snake, which is detected as a significant event, a potential threat to your life, because you don’t know what kind of snake it is. A lot of systems including the amygdala and other cortical regions have descending projections to a series of structures along the brainstem that regulate the body and prepare it to act in the face of an emergency. There are changes in blood pressure, neuroendocrine changes, in heart rate, sweating, palpitations, the stomach feels strange and different, there might be some shaking etc. All these physiological changes are associated with viewing the snake. The combination of all those events together can be labelled an emotion and it is fine to label it that way, because it does feel different, it feels like what we call an emotion. But there is no one special place that is responsible for all that and does just that. It’s the integration of all these processes that put your body in this case on alert that we can legitimately call an emotion. But it’s a reaction that involves a constellation of regions, not only subcortical and not only cortical, it also involves the body, how the body is reacting and how you’re perceiving your body’s reaction.

In research emotions are often used as distracters to impair some sort of cognitive task, but are there also ways in which emotions can improve on something cognitive?

Especially in the case of vision it does improve performance, because we pay attention more quickly to emotional information. But that’s just the case of vision; you pay attention more quickly to the snake, so maybe that’s not so surprising. In terms of higher cognition we started doing research with the idea that emotion was always going to hurt cognition. But what we’re finding in several studies is that emotion is actually helping cognition. That was surprising to us. One possibility is that the manipulations we use in the laboratory are not intense enough. Obviously we’re trying to be as ethical as possible, so for example when we shock individuals we apply mild electrical stimulation. It’s really a mild stimulus and the subjects themselves choose the level that is unpleasant but not painful for them. To many people who volunteer to our studies, because participation in our studies is purely voluntary, those kinds of levels of unpleasant stimulation are not so intense. It’s more of a jolt, an arousal, an energising effect when for example they see a cue maybe a little red square that tells them they might get a shock in the next 10s. They might experience a little bit of fear but in addition to that they get energised and alert and that alertness is actually helping them perform the cognitive task better. What we think is that emotion in moderation is actually something that is energising and helps cognitive performance. It’s like taking a really strong cup of coffee. But for certain individuals, people that are typically more anxious in daily life, these electrical stimulations are more aversive and we start seeing that the expectation of shock starts hurting performance to a larger extent, even though the shock is mild.

What about for example in decision-making? Do emotions benefit when you have to make a decision, is it beneficial to include emotional information into a decision?

Some of the work that we’ve done deals with some kinds of decision-making but very basic ones. We were using simple problems similar to what you would find in assessments of IQ and some subjects were solving these better under threat of shock. But we also saw a fair amount of people for whom the threat of shock impaired performance.

More broadly, for example Antonio Damasio and others have written extensively on the need to incorporate emotional information in decision-making to become a better decision maker. There is this myth, for example in Star Trek with Spock as a purely rational being who makes decisions that are perfect because he’s not influenced by processes of emotion that only confuse you in the end. In effect, I think that’s a very ineffective way of doing decision-making because you’re not integrating all sorts of other information that are critical for you. Situations that would require pure rational thinking are mostly artificial, mathematical problems for example. But it’s real life problems such as are you going to keep this job, marry this person, take this trip now or save money, that require the integration of emotional information. These problems are not like artificial mathematical puzzles, because they involve a lot of factors that have to do with your social life, family, ability to keep a job, be promoted etc., and if you don’t use information, which you could call emotional, you’re not going to make a good decision. The decision is much poorer because you’re not integrating and using as a basis for the decision all sorts of information that are really critical. That’s one of the reasons we talk about social intelligence and other kinds of intelligence but in essence what those are is just the ability to integrate information in a bigger decision making process instead of just relying on a subtype of information.

That’s not to say that emotion is always perfect for that. If someone came at me and punched me just as I was about to make a decision, the distress and anger at that moment would make me a poorer decision maker, because my resources would be mobilised for something quite different, trying to figure out what to do, run away, fight back, call the police… Our resources are not infinite, they have limited capacity and in a moment of acute stress we are focusing on something so strongly that we’re not going to be good at anything else, not even other kinds of emotions. So I’m not saying that it’s always advantageous but it often is. In situations that are not extreme it is certainly advantageous to use as broad a range of information as possible including what some people would call emotional information.

If we look at the integration of cognition and emotion is this something that the brain always does or is this something that happens just sometimes?

The brain is always doing that. Information is flowing through the brain at all times but this flow is modulated by certain factors. If I’m trying to do what I’m going to call here a cognitive task, there’s a flow of information that is helping me do that. For example I’m planning a trip somewhere and have to decide how I’ll travel, I have to schedule two trains and maybe a bus and a plane. I’m maintaining information about the number of trips I have to do, how much money I have, can I afford the plane, do I have to take a second bus… What emotion does is affect the flow of information. If something happens suddenly and someone starts screaming that there is a fire, my resources are going to be diverted from other computations that I was doing to decide what I am going to do in this emergency. The information is flowing at all times and is just being diverted depending on what the context determines, what it calls for at that moment. It might seem at any given moment that you’re not using emotion; but there is the potential of doing that all the time. If I’m deciding whether to use the bus, my previous bad experiences will influence the decision. For instance maybe the last time I travelled there was a problem and we were waiting for the replacement bus for five hours. All these things are flowing simultaneously and you’re solving the problem by integrating all this information, they’re not encapsulated from one another, they are always communicating. Sometimes they don’t seem to be doing that because I have never had a bad bus trip, so it doesn’t really matter, it doesn’t have a special significance to me, but this potential is always there and really depends on what at that given moment are the factors and stimuli that you have to be considering.

One other topic that I would like to talk about is functional specialisation. This is an idea that has been dominant in neuroscience for a long time and a lot of research has gone into pinpointing which regions of the brain are doing certain functions. But lately the one area – one function approach seems not to be accurate or as true as it used to be thought. What kind of challenges does this pose for neuroscience?

I think neuroscience, any science is determined to a large extent by the tools that are used. For a long time we relied on tools like the lesion method and the idea of a double dissociation. If you lesion region A, behaviour A is impaired but behaviour B is intact. And at the same time if you lesion region B, behaviour A is now intact, but behaviour B is impaired. The tools and research done in this manner, the culture in neuropsychology of looking for double dissociations and looking at the brain in this way suggested that there is specialisation, that any given region is doing one thing and one thing alone. I think that’s a very misleading way of looking at the brain, only one or two regions and functions at a time. There are similar problems with neuroimaging, which has been used a lot in the last 20 years. I have mixed feelings about this technique, because it has generated a lot of work that is very bad not only in quality but also in the type of science that is done, searching for specific regions, the region of religion, the region of love, of fear, of empathy, you name it. It has done a tremendous disservice to our understanding of the brain.

However, at the same time, and especially in the last five years, there’s more reason for optimism, because there has been a trend towards understanding how regions talk to each other and are parts of networks of regions involved in behaviours in a combined way. The notion of specialisation in the history of neuroscience has always been a pendulum, say starting by viewing the brain as highly specialised and then less and less specialised going to the extreme of there being no specialisation at all with Karl Lashley and the equipotential brain and swinging back to a brain more and more specialised again with neuropsychology and parts of neuroimaging. But now the tools of connectivity analysis and also the tools of network science, developed in trying to understand the structure of the internet, the spread of information, the spread, as I mentioned before, of a virus, gene networks and how they communicate with each other, and so forth, all these tools in other fields are now understandably being applied in brain research too. The pendulum is swinging back again to the notion that there is less specialisation, because there’s more communication and more integration.

I think there is no correct answer in terms of how specialised or how distributed the brain is, because it depends on the question and context. There might be cases where certain behaviours still rely on a smaller set of regions doing something more circumscribed, but even this is highly dynamic. In some cases it’s going to be a different network or a broader network involving a different set of regions that is going to take up that function. The bottom line is not that everything is the same, because the brain does have specialisation. Especially the sensory parts of the brain are relatively specialised. Lesioning the primary visual cortex will result in blindness and lesioning the primary auditory cortex will result in problems with hearing. But the other extreme of the amount of specialisation that is sometimes emphasised is a very impoverished view of how the brain really works. The nucleus accumbens for example is sometimes talked about in the popular media as the reward region or even the chocolate region or the racing car region in men because if men look at pictures of Ferraris the nucleus accumbens lights up. That’s a very impoverished view of how the brain really works, because these regions are always in communication with each other. A specific task of a person looking at a Ferrari might engage the nucleus accumbens and it might also be engaged when that person is looking at chocolate, but that doesn’t mean it’s a reward region. It’s involved in a whole lot of other functions some positive, and some actually negative, for example functions involved in dread, a type of fear.

I think the situation is much more complex. The problem we have in general is a problem of communication. We like to communicate things in a simple way, so if I’m writing a book for the general public I might be tempted to simplify things so much that it seems the brain is organised in a very simple way. If I give an interview to someone and explain how things work they might edit out the complexity. So the amygdala becomes the fear centre, even if the person in that interview explained that the amygdala was very important for fear conditioning but was also important for a dozen other things, including reward and value processing, attention, vigilance, novelty, so many things. In many cases the media is partly responsible for choosing these appealing little sound bites claiming that scientists have found the fear centre in the brain or the pleasure centre in the brain. It’s fascinating to finally “understand” how the brain works and why we have fear and where pleasure comes from and it’s simple, you can grasp it, it’s not complex. No one interviews a person to ask them about complex aspects of quantum physics. The physical world that allows us to be here is equally important and fascinating, but quantum physicists can’t give simple answers about how quantum mechanics works. I think there has been an attraction about the brain because it seems that we can give simple answers to really complex behaviour but I think that is just completely wrong. We fell into a pattern of simple explanations, but one day we might have to be giving answers that are as involved and complex as some in quantum physics…. The problem is that at that point people might lose their interest if the explanation is too complex for them to understand. If you haven’t studied physics for years, you can’t follow the ideas about entanglement and all sorts of very bizarre states that only exist in the microscopic world where quantum physics applies. Maybe if we can’t go all the way in giving these complex explanations that make people turn away from the subject, maybe one way is that we can gradually increase the complexity of our answers to be more in line with what we see when we do these kinds of experiments.

The metaphors we use to describe the brain often imply functional specialisation. If the brain is a computer that implies that there is a specialised processor, a specialised memory and so on. Or even pictures from functional imaging seem to imply that the parts of the brain that are black don’t really do anything special and that when we have to do something, an area lights up, does its thing and shuts down again. Is there a more appropriate metaphor or simplification that is more in line with current findings or do you think that these metaphors are always too simple and not helpful?

I think we come up with metaphors to help but in many cases they actually hurt. For instance, we have been responsible for conveying this misleading view that the brain works by publishing our results in terms of brain activation maps where one little blob is activated and everywhere else is dark. For scientists that’s OK, because we understand the background, that there is activation nearly everywhere, what’s being shown is just the specific contrast of two conditions, the difference between for example engaging in a certain cognitive task versus engaging in a different cognitive task. A very specific task can sometimes lead to a few regions of the brain that appear to be more tuned to one task than another. It’s always more than one region even though sometimes people claim that it’s only one region. This also depends on the method that we use for displaying information, because we only show results that are statistically reliable, that we can be certain about, which leads to the side effect of only showing a few regions. So when a person looks at the newspaper and sees these few regions activated, it seems like the brain is only doing that. That’s completely misleading, because if you showed activation in the whole brain without thresholding, which is a process by which you show only a little island of activation and the rest is dark, then you’d see activation in lots of regions. But what we’re interested in is these hotspots, where more stuff is going on, more blood flow is supplying that region, or there is more blood oxygenation reflecting how the neurons are operating locally. That’s what we’re interested in. So instead of showing a map that is coloured everywhere with a little bit of warmer colour in that region we decided to put black elsewhere and just colour that region. So I think if we’d made the decision earlier that we would show everything along with hotspots of activation in three, four regions, it would have been a much better way of communicating our findings.

It’s basically like a colour map. Imagine for example a temperature map of Europe that shows the temperature of all regions. Cooler regions would be coloured blue and warmer regions yellow and some regions in Southern Europe that are really hot will be shown in red. There’s temperature everywhere and we use a colour scale showing it everywhere. But if we wanted to know where the hottest places are, because for example we don’t want to travel to places that are too hot, we could by the same process of thresholding show only regions that are hotter than 30 degrees Celsius. In that case there will be a few islands of colour maybe in the south of Italy, parts of Spain and Greece and they would be the only temperature regions shown. That’s equally misleading but it’s pretty clear because everyone knows what temperature is, that the picture doesn’t mean there’s only temperature in three places in Europe. But this might not be so clear in pictures of brain imagining. For example when people are seeing a fearful face and activation is only shown in the amygdala, someone looking at that picture might come to the conclusion that the amygdala lights up when there’s fear and nothing else in the brain activates. So, therefore, the amygdala is the fear centre. I think we need to do a better job at that. I have actually had this conversation with other colleagues of trying to change how we show our results. Maybe it is something we should go back to and maybe try to convince people that we should convey information in a way that is less misleading to people who might not be aware of how these procedures really take place and their implications.

As a last question, something for the future, what you’re most excited about what do you expect will happen in this line of research?

What I’m really excited about are some studies that we’re doing now using tools from network science, network theory and mathematical graph theory and applying these techniques to look at network interactions. What we’re seeing is that under threat, when a person sees a little red square that says that they might receive a shock, not only do certain brain regions get activated, something that we knew already, but the regions also group together in very different ways depending on the conditions, suggesting that the transfer of information is much more dramatic and stronger when a person is under threat. That’s one of the things we’re really excited about.

Thank you very much.

dr. Luiz Pessoa
Department of Psychology
University of Maryland
College Park, USA