Rapid influence of emotional scenes on encoding of facial expressions: an ERP study

doi:10.1093/scan/nsn021 SCAN (2008) 3, 270–278 Rapid influence of emotional scenes on encoding of facial expressions: an ERP study Ruthger Righart,1 and Beatrice de Gelder1,2 1 Cognitive and Affective Neurosciences Laboratory, Tilburg University, The Netherlands and 2Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Bldg. 36, Main St., Charlestown, MA 02129, USA In daily life, we perceive a person’s facial reaction as part of the natural environment surrounding it. Because most studies have investigated how facial expressions are recognized by using isolated faces, it is unclear what role the context plays. Although it has been observed that the N170 for facial expressions is modulated by the emotional context, it was not clear whether individuals use context information on this stage of processing to discriminate between facial expressions. The aim of the present study was to investigate how the early stages of face processing are affected by emotional scenes when explicit categorizations of fearful and happy facial expressions are made. Emotion effects were found for the N170, with larger amplitudes for faces in fearful scenes as compared to faces in happy and neutral scenes. Critically, N170 amplitudes were significantly increased for fearful faces in fearful scenes as compared to fearful faces in happy scenes and expressed in left-occipito-temporal scalp topography differences. Our results show that the information provided by the facial expression is combined with the scene context during the early stages of face processing. Keywords: context; face perception; emotion; event-related potentials; P1; N170 INTRODUCTION The recognition of facial expressions has traditionally been studied by using isolated faces (Ekman, 1992; Adolphs, 2002). However, facial expressions can be rather ambiguous when viewed in isolation. This ambiguity may be resolved if the accompanying context is known (de Gelder et al., 2006; Barrett et al., 2007). In comparison to the study of object perception (Palmer, 1975; Bar, 2004; Davenport and Potter, 2004), there have only been a few behavioral studies that investigated the question how contexts may influence face processing. Facial expressions of fear tend to be perceived more frequently as expressing anger when subjects had heard a story about an anger provoking situation in advance (e.g. about people who were rejected in a restaurant) (Carroll and Russell, 1996). Facial expressions that were viewed in the context of emotional scenes were categorized faster in congruent visual scenes (e.g. faster recognition of a face conveying disgust in front of a garbage area) than in incongruent scenes (Righart and de Gelder, 2008). fMRI studies have also shown that facial expressions are interpreted differently given the context information that is available (Kim et al., 2004; Mobbs et al., 2006). Behavioral and fMRI studies are not able to show at what stage of processsing emotional contexts affect face recognition. This may relate to an early stage of encoding, but it may also relate to a later stage of semantic associations that are Received 15 April 2008; Accepted 8 July 2008 We are grateful to Thessa Caus for assistance with the materials and EEG recordings. Correspondence should be addressed to Beatrice de Gelder, Cognitive and Affective Neurosciences Laboratory, Department of Psychology, Warandelaan 2, PO Box 90153, 5000 LE Tilburg, The Netherlands. E-mail: . made between face and context. Because of the timesensitivity of event-related potentials (ERPs), it is possible to investigate how contexts affect different stages of face processing. The N170 is an ERP component that has been related to face encoding (Bentin et al., 1996; George et al., 1996; Itier and Taylor, 2004). The N170 occurs at around 170 ms after stimulus onset and has a maximal negative peak on occipito-temporal sites. Although some studies have observed that the N170 is insensitive to facial expressions (Eimer and Holmes, 2002; Holmes et al., 2003), other studies have shown that the N170 amplitude is modified by facial expressions of emotion (Batty and Taylor, 2003), especially for fearful expressions (Batty and Taylor, 2003; Stekelenburg and de Gelder, 2004; Caharel et al., 2005; Righart and de Gelder, 2006; Williams et al., 2006). This suggests that emotional information may affect early stages of face encoding. Another ERP component, the P1, with a positive deflection occurring at occipital sites at around 100 ms after stimulus onset, has also been related to face processing. Most studies have centred on this component because of its relation to spatial attention and physical features (Hillyard and Anllo-Vento, 1998). However, recent studies have found that the P1 amplitude is larger for faces than for nonface objects (Itier and Taylor, 2004; Herrmann et al., 2005), and that facial expressions affect the P1 amplitude as well (Batty and Taylor, 2003; Eger et al., 2003). The P1 may reflect a stage of face detection, and precedes the N170 that may reflect a stage of configural processing (Itier and Taylor, 2004). This time-course is consistent with earlier suggestions that global processing of faces occurs at around 117 ms, ß The Author (2008). Published by Oxford University Press. For Permissions, please email: ERPs for faces in context SCAN (2008) while fine processing of facial identity and expressions may occur at around 165 ms (Sugase et al., 1999). An important question is how emotional contexts in which faces are perceived affect the aforementioned P1 and N170 components. In our previous study it was found that the N170 amplitude was larger when the face appeared in a fearful scene, especially when the face expressed fear (Righart and de Gelder, 2006). This effect was not found for the P1, though effects have been observed on this component for scenes only (Smith et al., 2003; Carretie et al., 2004; Righart and de Gelder, 2006), and for facial expressions in interaction with bodily expressions (Meeren et al., 2005). While previous studies have reported that the N170 elicited by facial expressions is modulated by the emotional context, it is not yet clear whether individuals use context information in this stage when the task requires them to attend to the face and to discriminate explicitly between facial expressions. We hypothesized that the N170 component will still be affected by emotion regardless of the changed task conditions (Caharel et al., 2005). In addition, it was not clear from our previous study how facial expressions of fear are processed compared with other facial expressions (e.g. happiness) as a function of the emotional context. In the present study, we investigated whether explicit categorization of facial expressions (fear, happiness) affects the P1 and the N170 for faces. As the discrimination of fine expressions may be associated with later processing stages than the P1 (compare with Sugase (...truncated)