Is there an interaction between facial expression and facial familiarity? An investigation using performance data and event-related potentials.Nele Wild-Wall
Zusammenfassung89
Summary10111.121.1.1314151.2.1.2.1.16384#482171819704#337201.2.2.21221.2.3.23242526271.2.4.2829303132331.3.1.3.1.34658#2891.3.2.352.36372.1.382.1.1.2.1.2.39719#31240725#2082.1.3.41487#4562.1.4.42432.2.2.2.1.442.2.2.45344#34346472.2.3.48335#44249529#317525#32450527#431504#44351684#2442.2.4.5253542.3.2.3.1.552.3.2.5657460#2872.3.3.58402#47059417#27260415#281419#44661428#425687#2422.3.4.6263642.4.2.4.1.65662.4.2.2.4.3.67682.4.4.69469#42070712.5.2.5.1.2.5.2.722.5.3.73440#403742.6.2.6.1.75762.6.2.77782.6.3.79802.6.4.500#45481500#362490#35782448#49583712#4022.6.5.84852.7.868788899091929394953.9697591#239983.1.3.1.1.993.1.2.100386#4523.1.3.101102332#530103607#294595#287104719#4263.1.4.1051061073.2.3.2.1.3.2.2.1083.2.3.109489#5013.3.1101111121131144.1154.1.1164.2.117118119469#4171201214.3.1221234.4.1241255.1261271281291301311321331341351361371381391406.1416.1.6.2.6.3.1426.4.143Danksagung
List of AbbreviationsenTable of contentsHelpPart II: Does facial expression affect the discrimination of facial familiarity?

The data presented in the previous chapter suggested a facilitative interaction for personally familiar faces and the discrimination of facial expressions. This does not contradict the assumption of different functional processing stages or neural brain regions which subserve both processes. However, it does not necessarily imply, that the opposite holds true – the interaction between facial expressions and the discrimination of facial familiarity. From everyday life it is conceivable, that facial expressions are very important cues for social interactions between familiar people. Expecially the happy expression is a special cue for familiar people when meeting with each other. In addition, emotions are more often expressed to familiar instead of unfamiliar people. Therefore, facial expressions might be a cue for familiarity.

As outlined already, the functional model of face recognition by Bruce and Young (1986) predicts an independency of the two processes in the direction just described. By using the Garner paradigm (see above), Schweinberger and Soukup (1998) only found evidence for an asymmetric interaction between facial familiarity and the discrimination of facial expressions but not in the opposite direction. On the other hand, the proposed model by Haxby et al. (2000), which is more related to the human neural system, allows for the possibility of an interaction between circumscribed brain structures and their cognitive functions. There is also data suggesting an opposite interaction between facial expressions and the discrimination of facial familiarity. People personally familiar were recognized faster as familiar displaying a neutral expression when compared to a happy one (Endo et al., 1992). On the other hand, the recognition of famous faces was also faster (Endo et al., 1992) and more accurate (Baudouin et al., 2000a) when showing a happy expression. In addition, a happy expression increases the rating of familiarity when compared to a condition in which the same face is presented with a neutral expression (Baudouin et al., 2000). There is also evidence, that changing of the expressive context of a face can modulate face specific processes in the medio-temporal cortex during short-term memory retrieval. Results of Guillaume and Tiberghien (2001) show an increased positive parietal slow wave during short-term recognition of faces, when the expression is changed from study to test. This indicates a more difficult classification of the face as ‘old’, even if the facial expression concerning the task is irrelevant.

The present chapter attempts to elucidate upon the question of whether an interaction is found between facial expressions and facial familiarity within a familiarity discrimination task. Considering the results from the previous chapter, the opposite interaction was only evident for personally familiar faces but not for famous or experimentally familiarized, and unfamiliar ones. If, as argued, increased affective arousal evoked by personally familiar faces may influence an interaction between facial familiarity and facial expressions, an interaction should be present in both directions. In addition it should only emerge for personally familiar faces. The temporal properties of both tasks should decide upon the functional locus of interaction. Thus the functional locus for both processes might be different from the proposed locus in the previous Part. In the following Experiment 7 the stimulus set with personally familiar and unfamiliar faces is used. In addition, Experiment 8 will use the set with experimentally familiarized and unfamiliar faces. Again, event-related components as temporal markers of the functional processes in question can help to pinpoint the process which is facilitated. A modified working model compared to the previous part is used. Facial expression processing is assumed to influence the discrimination of facial familiarity (Figure 29).

Within a familiarity discrimination task stored representations of faces certainly have to be accessed. If it is assumed that the stored representations of familiar faces are dependent on often encountered facial expressions, then there might be an advantage for such expressions (e.g. neutral faces or happy facial expressions). Thus it might be possible that an interdependency will also emerge for learned familiarized faces as well as for personally familiar ones. One correlat of familiar face perception within a priming paradigm is the N250r component mentioned earlier. No priming is conducted in the present paradigm, but the averaged waveforms include first and repeated presentations of the same person and also the same picture. Although not exactly comparable to the N250r (Schweinberger et al., 2002) the amplitude distribution for personally familiar and unfamiliar faces in Experiments 2, and 3 of the previous chapter clearly show effects in the time intervals that are typical for the N250r component. It was observed (e.g. in Experiment 2) that independently of each other, parieto-occipital negativity is enhanced for personally familiar and unfamiliar faces and for displayed disgust and happiness in the time intervals from 200 to 300 ms poststimulus. This amplitude effect in the present data is possibly a correlate of the same process that mediates the N250r component in a priming paradigm. In this case similar characteristics can be expected. This enhanced parieto-occipital negativity for personally familiar faces in Experiment 2 may reflect the accessability of the stored representation. The N250r component also emerges, although with reduced amplitude, for unfamiliar faces within a priming paradigm. Reduced parieto-occipital negativity was observed for unfamiliar faces in the same time intervals in Experiment 2. Recent results concerning the N250r suggest that perceptual factors may also play a role in the accessing of stored representations of familiar faces (Schweinberger et al., 2002). When different pictures of the same famous person were repeated a reduction was found. Hence it may not be surprising that facial expressions also affect the process that mediates the N250r.

Happiness may have a special role for the discrimination of familiarity, since it is the most universal social cue when aquainted people meet each other. According to this logic it can be predicted that a happy expression facilitates the decision for a face to be familiar. For the same reason it might be harder and more error prone to decide for unfamiliarity when an unfamiliar happy face is presented.

According to the findings mentioned above for the following experiments it is hypothesized, that facial expression affects the discrimination of facial familiarity. The highly familiar happy and neutral expressions might facilitate the decision for a face as being familiar. In addition, the decision for unfamiliarity might be harder when unfamiliar faces display these two expressions. Again, error rates should behave in the same way, being lower in case of a facilitated response.

Experiment 7
Rationale

In Experiment 7 the question is raised whether the observed interaction of personal familiarity onto the discrimination of facial expressions also holds true for an interaction in the opposite direction, that is, an effect of facial expression on the discrimination of facial familiarity for personally familiar and unfamiliar faces. Because the perception of facial expressions is a fast process and does not need to be slowed down by a hard condition in order to interfere in the functional processing streem, portraits with weak expressive intensity were omitted. Instead portraits with a neutral expression were added. Participants had to perform a speeded familiarity discrimination task in which all presented people displayed happiness, disgust, or a neutral expression in randomized order. It is expected that the frequently seen happy expression will facilitate the decision for familiarity (also see Endo et al. 1992). The neutral expression may also shed some light on the special role of the happy expression that was evident in all previous experiments. A neutral expression is probably the most frequently seen expression on personally familiar faces, but also on unfamiliar faces in general. If the pronounced facilitative effect of happiness is based on the frequency with which this expression is encountered on the face of a personally familiar person, comparable effects are expected for personally familiar portraits with happy and neutral expressions when compared to disgust. If the expressiveness resp. emotional arousal of a face is the crucial factor, the facilitation should only be evident for personally familiar faces displaying happiness, but not for a neutral expression. However, an expression of disgust may disturb the perception of familiarity, either because it is rarely encountered on a familiar face or because it may disturb the configuration of the internal features as a happy and certainly as a neutral expression more. In this case a facilitative effect is only expected for familiar happy faces.

Method

Participants. Twenty participants (all women; mean age = 25,0 years; aged between 20 and 34) took part in Experiment 7. They were personally familiar with half of the presented persons displayed in the experiment. Participants received either course credit or a payment of 12 €. The mean handedness score was 75 (ranging from –83 to +100; Oldfiled, 1971).

Design and Procedure. In the present experiment the same stimulus set was used as in Experiments 1 to 3 with the exception that portraits with weak expressive intensity were omitted and portraits with a neutral expression were added. Hence 2 x 3 different conditions arose: familiar and unfamiliar happy faces displaying a neutral expression, happiness, and disgust. In a two-choice reaction time task participants had to discriminate between whether the presented face was personally familiar or not. All 28 people of the stimulus set were presented with happiness, disgust, and a neutral expression in three different perspectives. Trials were presented in randomized order with the same trial scheme as in the previous experiments. The stimulus set was repeated twice, whereas, after a single repetition, the hand-to-key assignment was changed in order to calculate an LRP for all conditions. The order of the key assignment was counterbalanced across participants. Before the experimental blocks participants viewed all 28 persons, who were included in the stimulus set, with a neutral expression and made a verbal response about personal familiarity. This was done to avoid errors, because people in the stimulus set were displayed repeatedly. In a pilot study one participant happened to classify an unfamiliar person as familiar, and error rates increased extremely due to the repetitions of the people.

Electrophysiological recordings. In the present experiment the same electrode setup was used as in all previous experiments, which recorded event-related potentials. Furthermore, electrophysiological data were treated the same way as in the previous experiment and averaged to the above mentioned six conditions.

Data analysis. For statistical analysis the same tests and procedures were used as in the previous experiments, with the exception that the within subject factor expressive intensity was omitted. The factor expression contained three levels (happy, disgust, and neutral). Analysis of the S-LRP, and LRP-R onsets, the LhEOG as well as of the peak amplitudes for the N170, and P300 components were also the same as in the previous experiments. Peak latency measures of the N170 and P300 components were analyzed with a repeated measure ANOVA including both within subject factors mentioned above. The latency values of the P300 component were based on jackknifing averages. The F-values were corrected according to the equation Fc = F/(n-1)2 (Ulrich, & Miller, 2001).

Figure 30. Reaction time and error rates for the familiarity discrimination task of Experiment 7.

Results

Reaction time and error percentage. As can be seen in Figure 30 the classification for a portrait as familiar was faster than for unfamiliar portraits (559 ms vs. 609 ms; F(1,19) = 72.2, p < .01). In addition, the expression of a portrait also affected RT (F(2,38) = 4.8, p <.05). Post hoc comparisons revealed a trend for RTs on portraits with neutral expressions being slightly faster when compared to the expression of disgust (580 ms vs. 590 ms; t(19) = 2.4, p < .07). Most importantly, the classification of familiarity was affected by facial expression (F(2,38) = 5.1, p = 0.011). The classification of a portrait as familiar was facilitated when the face displayed either a happy (t(19) = -4.5, p < .01) or a neutral expression (t(19) = -3.1, p < .05) when compared to disgust. No effect of expression was found for unfamiliar faces.

The mean error percentage of 3.9 was fairly low. There were no effects in error percentage concerning the factors familiarity nor expression.

Event related potentials. The N170 component is most pronounced at the electrode site P10 (Figure 31). As is obvious in the figure, there is no difference in peak latency between conditions. Although based on small differences, there was an effect of familiarity (F(1,19) = 9.2, p < .01), and of expression (F(1,19) = 12.3, p< .01) on the peak amplitude of the N170 component. The mean amplitude for familiar faces was 0.4 µV larger when compared to unfamiliar faces. Furthermore, a smaller amplitude arose from faces with a neutral expression (-5.8 µV) when compared to faces expressing happiness (-6.2 µV; t(19) = -3.5, p < .01) or disgust (-6.3 µV; t(19) = -4.3, p = .01).

Figure 31. The N170 component for the familiarity discrimination task of Experiment 7 at the electrode site P10 separated for expression and for familiarity.

The P300 component was most prominent at the electrode site Pz (Figure 32). To all appearances there is quite a difference in peak latency between familiar (519 ms) and unfamiliar (575 ms) faces with the first ones peaking earlier than the latter ones. This was confirmed by a repeated measures ANOVA based on jackknifing averages (Fc(1,19) = 35.9, p < .01). There was no effect of expression on peak latency (Fc = 1.4) nor an interaction (Fc < 1). Familiarity also had a strong effect on peak amplitude (F(1,19) = 50.6, p < .01) yielding higher amplitudes for familiar (M = 11.6 µV) when compared to unfamiliar faces (M = 8.7 µV).

Figure 32. The P300 component for the familiarity discrimination task of Experiment 7 at the electrode site Pz separated for expression and separate graphs for familiarity.

Figure 33 displays the S-LRP for personally familiar and unfamiliar faces averaged for the different expressions. The expected difference within personally familiar faces for an earlier onset of happiness (M = 329 ms) when compared to disgust (M = 362 ms) was confirmed by a one-tailed t-test with jackknifing averages (tJ(19) = 1.75, p < .05). No differences were found between expressions for unfamiliar faces (tJ < 1).

Figure 33. The stimulus-locked LRP for the familiarity discrimination task of Experiment 7 separated for expression and separate graphs for familiarity.

Figure 34. The response-locked LRP for the familiarity discrimination task of Experiment 7 separated for expression and for familiarity.

There was no effect of facial expression in the interval between LRP-onset and response for personally familiar nor for unfamiliar faces (Figure 34). In addition, experimental conditions did not affect the LhEOG, which was calculated for stimulus-, and for response-locked epochs (ps > .10). Hence, an influence of horizontal eye movements on the S-LRP, and the LRP-R could be denied.

The statistical analysis of the mean amplitude distribution (Figure 35, Appendix 6.4.) of consecutive time intervals starting from 200 ms until 600 ms after stimulus onset revealed differences within facial familiarity (Fs(27,513) > 8.5, ps < .001), and for facial expression(Fs(54,1026) > 5.5, ps < .001) for all intervals tested (for all results see Appendix 6.4.). In addition, over all intervals starting from 200 ms poststimulus the calculation of vector scaled data revealed topographical differences for facial familiarity (Fs(27,513) > 3.5, ps < .007), as well as for facial expression (Fs(27,513) > 5.9, ps < .001). Because the factor expression included 3 levels, additional post-hoc analysis were calculated for any two of the expressions. Topographical differences between happiness and disgust started at 300 ms poststimulus (Fs(27,513) > 3.7, ps < .006). When comparing happiness or disgust to the neutral expression, topographical differences started early from 200 ms post-stimulus and remained significant (for exceptions see Appendix 6.4.) to the end of the tested intervals.

Figure 35. Differences of the mean amplitude distribution between pairs of happiness (H), disgust (D), and the neutral expression (N) as well as between unfamiliar (UF) and familiar faces (F; bottom row) for the familiarity discrimination task of Experiment 7 in all tested time intervalls; a grey shading equals a negative difference.

Discussion

As expected, the categorization for a face as being personally familiar yielded faster RTs than the categorization as unfamiliar. This is a result often found for tasks involving a familiarity discrimination (Endo et al., 1992; Phillips et al., 1998). Most important, facial expression affected the recognition of personal familiarity in the respect that the frequently seen happy and neutral expressions facilitated the decision for a face as being personally familiar when compared to disgust. No such effect was found for unfamiliar faces. Complementing the results of Part I, which pointed to an interaction between personal familiarity and the discrimination of facial expressions, the present experiment revealed an interaction in the opposite direction or personally familiar faces. Happy or neutral facial expressions facilitated the discrimination of a face as being personally familiar. A note of caution should be added. As mentioned before, all persons were presented with a neutral expression in advance of the experimental blocks, in order to exclude systematic errors based on classifying a person as familiar or unfamiliar by mistake. Hence neutral pictures of all people might have been primed, and interpretations which rely on the results of these pictures have to be carried out with caution.

In line with the RT results an erlier S-LRP onset was found exclusively for happy familiar faces when compared to familiar faces displaying disgust. No differences were found between personally familiar faces displaying happines and disgust when considering the LRP-R onset as well as the N170 or P300 peak latency. Together, the results clearly point to the response selection stage as indexed by the S-LRP which is facilitated for happy personally familiar faces within a familiarity discrimination task.

The results are in line with other findings of e.g. Endo et al. (1992), who found faster RT in a familiarity discrimination task for familiar faces when they expressed happiness or a neutral expression. Admittedly, for personally familiar faces they found decreased RT only for the neutral expression. Due to the mouth being closed in all pictures, it is likely that the participants in the present experiment were also familiar with this rather weak happy expression of all personally familiar faces. Therefore, the findings of Endo et al. (1992) concerning personally familiar faces might not stand up against the present results.

The expressions might have disturbed the spatial arrangement of the internal features to a different degree. It is conceivable that this is not true for the neutral expression, and only partly true for happiness, but mainly true for disgust. Hence, familiar faces should be most easily recognized with a neutral or a happy expression, whereas disgust should disturb the recognition the most. The lack of a difference between the neutral and happy expression is also explainable within the present stimulus set since happiness was expressed with the mouth being closed. This makes it more comparable to a neutral expression.

The response selection stage as the locus of interaction stands in contrast to the late perceptual processing stage, which was considered as the possible functional locus of interaction for the expression discrimination task in Experiment 2. Other processes may subserve the interaction between facial expressions and facial familiarity within a familiarity discrimination task. Since a happy face is the most common gesture between aquainted people, it is probable that the facilitation is not due to personal familiarity. It may depend on a strong stimulus-response learning between a happy facial cue and the feeling of familiarity. Hence the response selection stage as the facilitated process is the most likely to be expected. However, the facilitation for personally familiar faces with a neutral expression did not fit into this reasoning. Possibly, the initial pre-exposure of the neutral faces had a much stronger effect than expected. Thus, to proof this reasoning, the same facilitative effect of happiness displayed on familiar faces should also emerge for experimentally familiarized faces. If, on the other hand, an unfamiliar face displays a happy expression, the decision for unfamiliarity should be more error prone.

An additional conclusion can be drawn from the data concerning the locus of facilitation for the familiarity decision. It is a result often found, that famous or experimentally familiarized faces are categorized faster in an identity discrimination task than unfamiliar ones (Phillips et al., 1998). To my knowledge it has been never addressed on which functional processing stage this advantage for familiar faces is located. As cited above, there is numerous evidence that the peak latency of the N170 component is not influenced by familiarity. Therefore, early visual perception may not be faster depending on facial familiarity. Studies which compared amplitudes and topographies from event-related potentials found differences between familiar and unfamiliar faces starting around 300 ms after stimulus onset (Paller et al., 2000; Endl, Walla, Lindinger et al., 1998). In the present experiment a clear effect in P300 peak latencies was found with an earlier peak for personally familiar faces. This difference is still evident in the later response selection stage (S-LRP), indicating a propagation of the effect which is already present in the P300 peak latency. No other processing stage was influenced by familiarity. Hence the results clearly point to late perceptual processing stages that are faster for familiar faces within a familiarity decision.

Taken together, the results of Experiment 7 point towards an interaction between facial expressions and the recognition of familiarity for personally familiar faces only. In addition, the effect of facial familiarity on S-LRP onset points to the response selection stage as the functional locus of the facilitation found for happy and neutral expressions. However, the results do not clearly elucidate on whether the found interaction between facial expressions and the discrimination of facial familiarity is due to personal importance of the familiar faces, or because of the mere perceptual familiarity which may be stronger for the often seen neutral and happy expression. A third explanation might be the special status a happy expression has as a social cue when aquainted people meet each other. It is necessary to have better control over the learning experience concerning facial expressions, when a face becomes familiarized. Therefore, a last experiment is carried out using unfamiliar faces, where half of the faces were familiarized in a learning block with different facial expressions.

Experiment 8
Rationale

In the present experiment the question was raised, whether the found interaction between facial expressions and the discrimination of personally familiar faces also holds true for experimentally familiarized faces. Assuming that the representations of familiar faces also relies on perceptual factors like facial expression when the face is familiarized, appearance of the same expression should facilitate the recognition of a familiar face. Whereas, if personal significance and arousal for familiar faces is crucial for an interaction of both processes, an interaction should not emerge between both processes.

The stimulus material in the present experiment consisted of initially unfamiliar faces. Half of the faces were familiarized in a learning block always with a neutral expression, and either a happy, or angry facial expression. For the following experimental block an equal amount of unfamiliar faces was added. Participants had to discriminate familiarized from unfamiliar faces. Faces were presented in randomized order either with a happy or angry facial expression. Importantly, for half of the familiarized faces the angry expression was never encountered, for the other half of faces the happy expression was not previously shown.

It is the main hypothesis that the facial expression which was encountered during familiarization in the learning block would facilitate the decision for a face as being familiar. This should be reflected in reduced RT and error rates for these conditions but not for the conditions with the facial expressions which were not encountered in the learning block.

Method

Participants. Twelve participants (9 women and 3 men, mean age = 26,6 years, aged between 20 and 39) took part in the present experiment. All participants had normal or corrected to normal vision. They received either course credits or 10,00 € for participation. Handedness was not determined, because electrophysiological data were not recorded.

Stimulus and apparatus. For the present experiment the same stimulus set was used as in Experiment 4 and 5. The same size of the stimuli was used for the initial learning phase and for the following experimental blocks respectively. Again, all stimuli were presented on a 17-inch screen. The viewing distance was kept constant at 1m. ERTS® served as experimental software for stimulus presentation and response recording.

Learning-procedure. Participants had to undergo a 1 hour training session in order to become familiar with two sets of 20 people (half of them being male). The learning procedure was exactly the same as in Experiments 4. The presented facial expressions were the same. That is, one set of 20 people was familiarized with a neutral and happy expression, whereas the other set of 20 people was presented with neutral and angry expressions. The familiarized sets were counterbalanced across participants as described above.

Design and data analysis. Participants continued with the experimental blocks after a short break of about 10 minutes. They were then required to discriminate between whether the displayed portrait belonged to one of the 40 familiarized people or not by pressing a corresponding key on the right or left. All persons presented were displayed with the two facial expressions happiness, and anger. The whole stimulus set was presented in randomized order and repeated twice. The trial schema was the same as in all previous experiments. Participants had to perform 6 blocks of trials. In case of too slow (> 2000 ms) or incorrect responses feedback was provided after the trial by showing the words “Zu langsam!” (Too slow!) or “Falsch!” (Wrong!) for 500 ms on the screen. A summarized feedback for the block was provided during the breaks. At the beginning and in the middle of the experimental blocks the hand to key assignment was changed. In order to learn the key assignment, participants performed 40 practice trials by pressing the corresponding key according to the words “gelernt” (learned) or “ungelernt” (unlearned) presented on the screen.

Statistical analyses of RT and error rates was the same as in Experiments 4 and 5 with the exception that now the factor expression contained 3 levels (neutral, happy, and angry). In addition, the within-factor familiarity type was used. It contained the levels ‘familiar person with learned expression’, ‘familiar person with unlearned expression’, and ‘unfamiliar person’.

Results and Discussion

Familiarization. Error rates for the test-blocks of the learning procedure decreased from 11.0% over 4.7% (t(11) = 5.1, p < .001) to 3.5% for the third block.

Reaction time and error percent. Figure 36 displays the RT and error rate of Experiment 8. No effect at all emerged for the error rates in the present experiment. The RT strongly depends on the familiarity type (F(2,22)=15.3, p < .01). Post-hoc tests revealed faster RTs for faces that were familiarized in the previous learning block with (M = 608 ms) or without the displayed expression (M = 639 ms) when compared to unfamiliarized faces (M = 669 ms; ts(11) > -5.2, ps < .01). Although visible in Figure 36 and numerically evident, no statistical difference was found between the two types of familiarized faces (t(11) = -2.2, p = .15). Hence the main hypothesis of the present experiment was not confirmed. The discrimination of a face as being familiar was independent of the expression that was encountered during familiarization. However, the numerical difference between familiarized faces with learned expressions and familiarized faces with unlearned expressions lead towards a view that the perceptual experience with which a person is familiarized may play at least a minor role concerning the representations that are built up.

Figure 36. Reaction time and error rates for the familiarity discrimination task of Experiment 8 separated for expression and familiarity.

Summary and Discussion of Part II

In Part II the question was raised, whether emotional facial expressions affect the recognition of facial familiarity. In Part I it has been shown that the opposite also holds true, but appears to depend strongly on the degree of familiarity. Familiarity facilitated the discrimination of facial expressions only for personally familiar faces but not for famous or experimentally familiarized faces. In addition, this effect was specific for facial expressions, since it was only evident for personally familiar faces displaying happiness. Accordingly, facial expression affected the discrimination of facial familiarity. It was supposed that the expression of happiness again has a special role for the interaction between the processes in question. Possible reasons might be the increased frequency of happiness encountered on familiar faces, the special role a happy expression has as a social cue for familiarity, or the emotional arousal a happy face might evoke.

Two experiments were conducted using different stimulus sets with personally familiar, or experimentally familiarized, and unfamiliar faces. It was the main hypothesis, that the frequently encountered happy expression facilitates the discrimination of a face as personally familiar. In addition, a similar effect was expected for the experimentally familiarized faces with the objection that this time the effect could be modulated by the perceptual experience during the learning block.

For personally familiar faces it was shown that facial expression influenced the discrimination of facial familiarity. Personally familiar faces were recognized faster as familiar with the often seen neutral and happy expressions than when displaying disgust. No such effect was evident for unfamiliar faces. Event-related potentials clearly point to the response selection stage, which is facilitated for personally familiar faces displaying happiness when compared to disgust. It again has to be noted that in Experiment 7 all familiar and unfamiliar persons were presented with a neutral expression once before the experimental blocks in order to prevent participants from making systematic errors (see 3.2.2.). Therefore, the neutral condition was omitted from statistical analysis of ERPs. In contrast to the results of Experiment 7, no interaction between facial expressions and the discrimination of experimentally familiarized faces was found in Experiment 8. The decision for a face as being familiar was independent of the facial expression which had been encountered during the learning block.

According to accepted common sense it is not surprising to find the so called ‘smiling bias’ (Endo et al., 1992). That is, shorter response latencies for happy familiar faces. In everyday life displaying happiness is the most common gesture when someone familiar approaches us. The present results are in agreement with previous findings showing that familiar face recognition is facilitated by a happy expression (Baudouin et al., 2000; Endo et al., 1992). Baudouin et al. (2000) only found an interaction between facial expressions (happy vs. neutral) and a familiarity decision for their accuracy data and familiarity ratings but not for RT. Compared to a neutral expression more errors were made for happy unfamiliar faces. Familiarity ratings were increased for happy faces in general. The results of the present Experiment 7 extend these findings to response latencies but they show no effect on error percentage and the discrimination of unfamiliar faces. This might be due to lower error rates within the present data (under 5% when compared to the over 20% that Baudouin et al., 2000 found). In their article Baudouin et al. (2000) argue, that the effects found for both, familiar and unfamiliar, faces imply the decision stage as a common level of interaction. Based on the model of Bruce and Young (1986), the cognitive system gets two concurrent inputs from the face recognition unit and the facial expression analysis. The “smiling bias” is observable “when a strong smiling input is concurrent with a weak familiar input” (Baudouin et al, 2000a, pp 291). On the one hand, the results of Experiment 7 support this interpretation with the locus of confluence found in the response selection stage as indexed by the S-LRP. On the other hand it can be supposed that the ‘familiar input’ for personally familiar faces was strong in the familiarity discrimination task. The interpretation cited above does not apply to the present data, since an effect was still found for personally familiar faces. In addition, results of Experiment 7 revealed a strong difference in RTs between happy and neutral familiar faces and faces displaying disgust. Hence, the alternative explanation is excluded, that an emotional expression induces a bias per se, because people may feel more inclined to express their emotional state to familiar people (Baudouin et al., 2000). In this case, a facilitation of familiar face discrimination should have emerged for faces displaying happiness and disgust, but not for the neutral expression. This was not the case. Unfortunately, the interpretation based on the trials with neutral expressions has to be taken with caution, since all persons were displayed with a neutral expression before the experimental blocks in Experiment 7. The advantage for personally familiar faces with a neutral expression might be explained by the pre-exposure which may be comparable to a priming effect (Begleiter et al., 1995). This argument is reduced slightly by the fact, that also unfamiliar faces were pre-exposed and hence, a facilitation for the neutral expression should have emerged for unfamiliar faces also. However, decreased priming effects for unfamiliar faces when compared to familiar ones are often found (Jemel et al., in press; Schweinberger et al., 1995). Therefore, the pre-exposure of neutral faces might have primed the discrimination of unfamiliar neutral faces to a lesser extend.

In the previous Part I it was argued that the interaction between facial familiarity and the discrimination of facial expressions could be subserved by increased arousal elicited by personally familiar faces. The same interpretation might also account for the familiarity discrimination task of Experiment 7. The observed facilitation for the familiarity discrimination of happy and neutral personally familiar faces might be due to increased arousal for these faces when compared to unfamiliar ones. No interaction between facial expressions and the discrimination of facial familiarity was present in Experiment 8 using experimentally familiarized faces. No increased arousal is expected here for the stimulus set of unfamiliar faces. This argument is not negated by ERP results which pointed to a different locus of interaction for the two tasks of Part I and II. Late perceptual processing stages were facilitated for the expression discrimination task in Part I. The present data of Part II clearly point to the response selection stage as being facilitated for the familiarity discrimination task. If arousal processes are the basis for an interaction between facial expressions and the discrimination of facial familiarity the locus may to a large extend depend on the temporal properties of the processes involved and the task in general. The familiarity discrimination task of Experiment 7 yielded faster RT when compared to the expression discrimination of Experiment 2 (583 ms vs. 693 ms; t(34) = 4.2, p < .01). For this particular stimulus set of personally familiar and unfamiliar faces it can be assumed, that the perception of facial expressions takes longer when compared to the discrimination of facial familiarity. Hence, the facilitative effect of facial expressions on the discrimination of personally familiar faces emerged in a later processing stage (the response selection stage) because expression information is available late. In contrast, the facilitative effect of facial familiarity on the discrimination of facial expressions was observed at an earlier stage (late perceptual processing) because the information about facial familiarity might be available earlier.

Another explanation suggests, that the frequency with which a facial expression is seen on familiar faces could explain the observed facilitative effect. The representations of familiar faces might not be image-independent as it is supposed by the functional model of face recognition by Bruce and Young (1986). The facilitative effect of happy (and neutral) faces in Experiment 7 might just emerge because familiar faces are most frequently seen with a happy (and also a neutral) expression. In return, stored representations might depend on these expressions. However, a proof of this interpretation would have been a comparable effect for the expression with which faces were familiarized in Experiment 8. This was not the case, although a insignificant numerical difference between familiarized faces with and without a learned expression pointed into this direction. It can, however, be assumed that the perceptual familiarity for the familiarized faces in Experiment 8 was high because error rates in the learning session were fairly low and decreased from 11 % to 3.5 % from the first to the third matching-to-sample block. For this reason, expression dependent representations of familiar faces can be ruled out as an explanation.

The facilitated familiarity discrimination for personally familiar faces displaying a happy or a neutral expression may be based on the increasing importance of internal facial features for familiar faces (Ellis et al., 1979; Young et al., 1985). Although it has been shown that perceptual information which is used for the recognition of identity and expression differs to a large degree, there is also an overlap (Calder et al., 2001) which could explain an interaction between both processes. Accordingly, only personal familiarity leads to increased reliance on internal facial information which is important for both the discrimination of familiarity and of expression. In addition, the discrimination of expression largely relies on internal facial features. Hence, an interaction between facial expressions and the discrimination of facial familiarity would only be expected for highly familiar faces but not for faces with a low degree of familiarity or unfamiliar faces. For the type of faces first mentioned it might be, that internal facial features attract more attention when compared to unfamiliar faces. In contrast, the discrimination of unfamiliar faces might have relied more on external features which makes an interaction between facial expressions and the discrimination of facial familiarity less possible. This may go some way to explaining why Schweinberger and Soukup (1998) failed to show an interaction between facial expressions and the discrimination of identity (see 1.2.3.), since they only used unfamiliar faces within the Garner paradigm. The increased variation in the irrelevant expression dimension in the orthogonal condition did not affect the discrimination of identity at all, because participants may have relied on external facial features. The strategy of the participants might have been to ignore the internal facial features including the facial expression. Thus the lack of effect is not surprising.

Another important point might be the distortion of internal facial features through particular facial expressions. Distortion might be higher for expressions of disgust than for happiness, and might be completely absent for the neutral expression. In fact, happiness is the most easily recognized expression. This is supported by lower error rates for its recognition (Calder et al., 2001; Ekman, Friesen, & Ellsworth, 1972) and often found decreased RT (Leppänen et al., 2003; Kiria & Endo, 1995; Hugdahl et al., 1993). If happiness or a neutral expression causes less distortion of internal features when compared to disgust faster RT should be expected for the discrimination of highly familiar faces with a happy or neutral expression. This is not expected for unfamiliar faces, because the decision for unfamiliarity may not rely on internal features, but more on external features like hairstyle or face shape. The same might hold for experimentally familiarized faces. The results of Experiments 7 and 8 are in line with this interpretation.

To summarize, an interaction between facial expressions and the discrimination of facial familiarity was observed in Part II. This complements the results of Part I, which found an interaction in the opposite direction. Again, this interaction only emerged for personally familiar faces, but not for experimentally familiarized faces. ERP results point strongly to the response selection stage which is facilitated for personally familiar faces displaying happiness when compared to disgust. It was argued that a possible influence of facial expression on the stored representations of a familiar face is unlikely to explain the facilitation of the familiarity discrimination for happy or neutral personally familiar faces. Otherwise, such an effect should also have emerged for experimentally familiarized faces. It has been discussed that increased arousal for personally familiar faces might subserve an interaction between facial expressions and the discrimination of facial familiarity. Finally, the exclusiveness of the interaction for personally familiar faces may also be explained by the higher importance of internal facial features for highly familiar faces.