Olfactory Discrimination: When Vision Matters?

Chemical Senses, Feb 2009

Many previous studies have attempted to investigate the effect of visual cues on olfactory perception in humans. The majority of this research has only looked at the modulatory effect of color, which has typically been explained in terms of multisensory perceptual interactions. However, such crossmodal effects may equally well relate to interactions taking place at a higher level of information processing as well. In fact, it is well-known that semantic knowledge can have a substantial effect on people's olfactory perception. In the present study, we therefore investigated the influence of visual cues, consisting of color patches and/or shapes, on people's olfactory discrimination performance. Participants had to make speeded odor discrimination responses (lemon vs. strawberry) while viewing a red or yellow color patch, an outline drawing of a strawberry or lemon, or a combination of these color and shape cues. Even though participants were instructed to ignore the visual stimuli, our results demonstrate that the accuracy of their odor discrimination responses was influenced by visual distractors. This result shows that both color and shape information are taken into account during speeded olfactory discrimination, even when such information is completely task irrelevant, hinting at the automaticity of such higher level visual–olfactory crossmodal interactions.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://chemse.oxfordjournals.org/content/34/2/103.full.pdf

Olfactory Discrimination: When Vision Matters?

M. Luisa Dematt e` 0 1 2 Daniel Sanabria 2 3 Charles Spence 2 0 Center for Mind/Brain Sciences, University of Trento , Corso Bettini 31, Rovereto ( TN ) 38068, Italy 1 Department of Cognitive Sciences and Education, University of Trento , Rovereto ( TN ), Italy 2 Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford , Oxford, UK 3 Departamento de Psicologa Experimental y Fisiologa del Comportamiento, Universidad de Granada , Granada, Spain Many previous studies have attempted to investigate the effect of visual cues on olfactory perception in humans. The majority of this research has only looked at the modulatory effect of color, which has typically been explained in terms of multisensory perceptual interactions. However, such crossmodal effects may equally well relate to interactions taking place at a higher level of information processing as well. In fact, it is well-known that semantic knowledge can have a substantial effect on people's olfactory perception. In the present study, we therefore investigated the influence of visual cues, consisting of color patches and/or shapes, on people's olfactory discrimination performance. Participants had to make speeded odor discrimination responses (lemon vs. strawberry) while viewing a red or yellow color patch, an outline drawing of a strawberry or lemon, or a combination of these color and shape cues. Even though participants were instructed to ignore the visual stimuli, our results demonstrate that the accuracy of their odor discrimination responses was influenced by visual distractors. This result shows that both color and shape information are taken into account during speeded olfactory discrimination, even when such information is completely task irrelevant, hinting at the automaticity of such higher level visual-olfactory crossmodal interactions. Introduction A growing body of chemosensory research now shows that peoples odor perception can be modulated by the presence of various different nonolfactory cues concerning the source/ identity of the odor in question. Color cues, in particular, appear to have a particularly strong effect on peoples olfactory judgments. For example, Zellner et al. (1991) (see also Engen 1972; Davis 1981; Stevenson and Oaten 2008) have demonstrated that people are better able to correctly identify odors when they are colored appropriately (e.g., a red solution smelling of cherry) than when they were colored inappropriately (e.g., a green solution smelling of cherry). Similarly, Blackwell (1995) has also shown that the ability of participants to identify the odor of a colored solution can be modulated by the appropriateness of the colorodor combination that happens to be presented. However, despite the existence of such empirical evidence, the exact role played by the appropriateness of the color is still a matter of some debate, especially given that a number of investigators have failed to observe any consistent effects of color appropriateness on peoples evaluation of odors (e.g., for evidence on the effect of color appropriateness on odor intensity judgments, see Zellner and Kautz 1990; Zellner and Whitten 1999). Meanwhile, Symons (1963) has documented interactions taking place in the opposite direction, such that the presentation of an odor can also enhance peoples sensitivity to visual stimuli (see also Hartmann 1933; though see Knasko 1995). Another issue that has been investigated extensively by those researchers interested in the nature of any multisensory interactions taking place between olfaction and vision concerns the specific nature of the crossmodal associations that are shared between odors and colors. For instance, Dematte` et al. (2006) recently highlighted the existence of systematic correspondences between specific odors and colors using both direct and indirect measures of human perception. In particular, Dematte` and her colleagues demonstrated that crossmodal odorcolor associations could be highlighted both by explicitly asking participants to pick appropriate combinations of odors and colors and also by looking at the variations in peoples performance induced by the manipulation of the stimulusresponse assignments during a speeded classification task. Such associations would appear to be stable across individuals (see also Gilbert et al. 1996; Kemp and Gilbert 1997; Schifferstein and Tanudjaja 2004). On the other hand, O sterbauer et al. (2005) used neuroimaging (in particular, functional magnetic resonance imaging) to show that the pattern of neural activity in brain regions such as the orbitofrontal cortex and the insular cortex varied as a function of the perceived goodness of fit (or congruence) between specific pairings of odors and colors. Taken together, results such as these would appear to suggest the existence of a privileged multisensory association between specific olfactory stimuli and particular colors. The existence of strong and stable crossmodal associations between particular odors and colors may indeed reflect interactions taking place at a perceptual level. For example, Kemp and Gilbert (1997) presented the participants in their study with a bottle containing a liquid to smell. The participants then had to choose from a set of colored chips the one whose color best matched that of the odor. The results showed that the weaker the perceived intensity of the odor the lighter was the color chosen by the participants (see also Schifferstein and Tanudjaja 2004). According to the authors, the positive correlation between the visual dimension of lightness and the olfactory dimension of perceived odor intensity represented an interaction taking place at a perceptual (i.e., rather than at a more decisional) level (see also Gilbert et al. 1996). It is important to note, however, that the influence exerted by higher levels of information processing also has a substantial effect on peoples olfactory perception. One example here comes from the olfactory illusion described by Herz and von Clef (2001) (see also Herz 2003; de Araujo et al. 2005): The hedonic valence of an ambiguous odor (based primarily on the smell of isovaleric acid), one that participants could easily associate with either parmesan cheese or human vomit, was shown to change as a function of the verbal label that was provided to the participants before the odor was presented (e.g., parmesan cheese = pleasant; vomit = unpleasant). Results such as these highlight the importance of access to semantic information related to the source of the stimulation for olfactory identification. Up until now, the primary interest of researchers has been devoted toward investigating odorcolor correspondences, whereas very little systematic research has been conducted on the effects that may be exerted by specific images on olfactory perception/performance. Sakai et al. (2005) conducted one of the very few studies to have demonstrated that people evaluate the intensity of an odor as being higher when viewing an appropriate (i.e., matching) picture (e.g., the picture of an apple while smelling apple juice) than when viewing an inappropriate picture (e.g., the picture of a pear while smelling apple juice). Sakai and his colleagues asked the participants in their study to smell an odor and to indicate from amongst a group of pictures the best and worst matching picture for the odor. After a number of trials, they then observed that participants rated an odor as smelling stronger when it was presented together with the subjectively best matching picture than when it was presented together with the subjectively worst matching picture. Gottfried and Dolan (2003) have shown that visual cues can facilitate olfactory detection performance. In particular, they found that people were able to detect the presence (vs. absence) of pleasant and unpleasant odors more rapidly and accurately while viewing a semantically congruent picture (e.g., a picture of a double-decker bus when presented with the odor of diesel, a stimulus with which the Londoners tested in their study was presumably very familiar!) than when looking at a semantically incongruent picture (e.g., a picture of a cake while presented with a fishy odor). In the present study, we attempted to further elucidate the role of specific visual stimuli on olfactory discrimination and, in particular, the specific effect (if any) exerted by seen shapes on olfactory discrimination. We were also interested in investigating any possible differences between the effects (i.e., facilitation vs. interference) exerted by compatible versus incompatible colors and/or shapes on peoples speeded odor discrimination responses. On the basis of a study by Naor-Raz et al. (2003), we hypothesized that different visual information (color vs. shape) might differentially interact with odor information. In particular, Naor-Raz et al. (2003) reported a series of experiments demonstrating that whereas the shape of an object can automatically elicit the conceptual representation of that object (thus also the information related to its color), the color of an object cannot. That is, color information interacts with shape information at an unimodal visual level rather than at a higher (or amodal) level of information processing (e.g., at a semantic level), suggesting that the representation of a object that may be accessed via color or shape information may not be the same. Given the findings of Naor-Raz et al. (2003), we thought it is possible that color cues and picture information might interact with the representation of an object elicited by its odor in different ways. If colors and shapes do indeed have differential access to object representations in the human brain as they suggested, then we would expect to find significant differences in how such information interferes with (or facilitates) odor discrimination performance according to the specific visualolfactory compatibility of each kind of stimulus. Alternatively, however, one might expect to see the same pattern of facilitation or interference mediated by visualolfactory compatibility regardless of the specific visual distractor that was presented (i.e., color or shape). Such a result would also be of theoretical interest because it would raise the possibility that many of the examples of colorodor interaction reported in previous research may actually have reflected the influence of semantic factors determined by the semantic associations that a particular group of participants has with that color, rather than necessarily by any more bottom-up perceptual effect of the color on olfaction. Materials and methods Participants Forty-one university students (24 females and 17 males) with a mean age of 24 years (ranging from 18 to 34 years) took part in this experiment. All the participants completed a confidential questionnaire at the start of their experimental session in order to confirm that they had a normal sense of smell with no history of olfactory dysfunction and normal or corrected-to-normal vision (i.e., they had no noticeable deficits of color vision). All the participants were naive to the purpose of the study. The experiment lasted for approximately 50 min, and the participants were given a 5 UK Sterling gift voucher in return for their participation. The experimental procedure was approved by the Ethical Committee of the Department of Experimental Psychology, University of Oxford. Apparatus and materials Two odorants were used as target stimuli: Strawberry and lemon (A118750 and 406803, respectively, provided by Quest International, Ashford, UK). The odorants were diluted at a concentration of 10% in diethyl phthalate (529633, Quest International) as recommended by Quest International. They were presented to participants by means of a custom-built computer-controlled olfactometer. The custombuilt olfactometer controlled the delivery of odorized and odorless air at a flow rate of 8 l/min. There were 9 possible visual stimuli (7 7 in size) representing the different combinations of the color factor (black and white, red, or yellow) and the shape factor (no drawing, a stylized drawing of a strawberry 3.76 3.76 in size, or a stylized drawing of a lemon 4.66 3.84 in size; see Figure 1). Each square was displayed centered on the middle of a 38.1-cm computer monitor. The RGB values corresponding to each color were 231, 0, 0 (red) and 248, 248, 0 (yellow). The E-Prime software (Schneider et al. 2002a, 2002b) was used to control the presentation of both the olfactory and the visual stimuli and to collect participants responses. Design and procedure The participants sat in a comfortable chair, 70 cm from a computer monitor. They were instructed to rest their head on a chinrest to which a nosepiece was attached. The top of the 2 output tubes of the nosepiece was positioned approximately 3 cm below the participants nostrils. The participants were instructed to fixate on the fixation cross in the center of the screen and told to identify each and every one of the odors that were presented by pressing the Z Figure 1 Visual stimuli used in the experiment, reflecting the combination of the color (black and white, yellow, or red) and shape (no shape, lemon, or strawberry) factors. or M keys on the keyboard, as instructed at the start of each block of trials. Simultaneous with the onset of the target stimulus, a pure tone was presented (at 55 dB[A] as measured from the participants head level, 22 kHz) for 300 ms from 2 loudspeaker cones, one positioned on either side of the monitor. The participants were instructed to inhale whenever they heard the auditory cue. This procedure was designed to ensure that the participants perceived the odors as soon as they were presented. The odor target was presented continuously through the olfactometer until a response was made, or until 5300 ms had elapsed, at which time the trial was terminated. At the same time as the odor was presented, the fixation cross disappeared and a visual stimulus was presented at the center of the monitor (see Figure 2). The shape was shown for as long as the olfactory stimulus was delivered. The participants were instructed to inhale slowly through their nose whenever they heard the tone and to try to identify each target odor as rapidly as possible while avoiding making errors. They were also informed that a visual stimulus would appear on the screen and that they had to keep fixating the screen while trying to ignore the distractor. After the participants response had been detected, odorless air was delivered for a period of 7 s, in order to ensure that any residual odors were extracted between successive trials. The experiment consisted of 4 experimental blocks of trials in which each of the 9 visual stimuli was randomly presented 3 times in combination with each of the 2 odors, giving rise to a total of 216 trials for each participant. Each block of trials was followed by a 5-min break in order to avoid participants habituating to the olfactory stimuli. The assignment of the odor targets to the response keys was counterbalanced across participants. Figure 2 Time line describing the experimental procedure used in the experiment. Results The mean reaction time (RT) data for each participant on those trials where he/she correctly discriminated the identity of the odor were filtered, and responses falling 2.5 standard deviations or more from the participants mean for each condition were considered as outliers and discarded from any further analysis. This filtering process resulted in the removal of less than 2% of trials overall. The remaining data were then analyzed by performing an analysis of variance (ANOVA) on the accuracy and RT data. The accuracy data were analyzed using a repeated-measures ANOVA that took into account the compatibility among odor, color, and shape. Thus, the within-participants variables in this analysis were odor (lemon vs. strawberry), color compatibility (compatible, incompatible, or neutral), and shape compatibility (compatible, incompatible, or neutral). The results revealed no significant main effect of odor F(1,40) = 2.11, not significant (n.s.). However, the analysis revealed significant main effects of both color compatibility, F(2,80) = 6.21, P < 0.01, and shape compatibility, F(2,80) = 3.55, P < 0.05 (see Figure 3). In fact, pairwise Bonferronicorrected comparisons revealed that the participants responded significantly less accurately when an incompatibly colored distractor was presented on the screen (M = 92%) than when a black and white distractor was presented (M = 94%, P < 0.01; Cohens d = 0.60). The difference in response accuracy between the compatibly and incompatibly colored distractor conditions was also borderline significant (P = 0.06, Cohens d = 0.30). Responses to odors tended to be less accurate when an incompatible shape (M = 93%) was presented than when a neutral shape (M = 94%, P = 0.06; Cohens d = 0.30) appeared on the screen even though this result was only marginally significant. None of the other Figure 3 Mean discrimination accuracy for the odor target as a function of its compatibility (compatible, white bars; incompatible, black bars; and neutral, gray bars) with the type of visual distractor presented simultaneously on the screen (color information or pictorial information). Error bars represent the standard errors of the means. factors were significant, including the interaction between color and shape (F < 1, n.s.). The analysis of the RT data from those trials in which the participants responded correctly revealed a significant main effect of odor F(1,40) = 6.59, P < 0.05, with the participants responding more rapidly to the strawberry odor (M = 1204 ms) than to the lemon odor (M = 1284 ms) overall (see Table 1 for a summary of participants performance in the various conditions). None of the other terms reached significance. Discussion The main aim of the study reported here was to investigate the nature of any multisensory interactions between olfaction and vision. More specifically, we wanted to determine whether olfactory information would interact with color cues and with pictorial information in different ways, given the finding of Naor-Raz et al. (2003) that color and pictorial information may elicit object representations at different levels of information processing (e.g., at a perceptual vs. at a conceptual level). The participants in our study had to perform a speeded odor discrimination (lemon vs. strawberry) task while trying to ignore the visual distractors that were presented at the same time. Both features (i.e., color and shape) of the visual distractors appeared capable of influencing peoples performance during the task as reflected in the modulation of the accuracy of our participants responding. This effect would seem to be more clear-cut for odorcolor interactions, as it was possible to define the difference in performance as an interference effect exerted by the incongruity between the odors and the colors. On the other hand, the effect of shape is somewhat harder to interpret: Shape compatibility appeared to play a role in olfactory discrimination (resulting in a main effect of shape) although the differences in the accuracy of participants discrimination responses do not allow us to determine whether performance was facilitated on the congruent trials or impaired on those trials where the odor and shape information were incongruent. Table 1 Participants performance in the olfactory discrimination task as a function of the compatibility between the odor target and either the color or the shape presented on the screen Incompatible Incompatible The upper half of the table shows the mean accuracy of the responses, whereas the lower half of the table shows the mean RTs. The standard errors of the means are reported in parentheses. It is also interesting to note that there was no significant interaction between color and shape, thus showing that these 2 features had independent effects on the performance of our participants. Additionally, the asymmetry in the results reported here (i.e., modulation of response accuracy but no variation of the speed of responses) would appear to be consistent with the large amount of interindividual variability often described by olfactory studies related, for example, to the long latency of responses to odor stimuli, to fluctuations in olfactory sensitivity (e.g., see Stevens and Dadarwala 1993) or to the diversity in crossmodal associations with odor stimuli (e.g., Dalton et al. 2000). The interference effect reported in the present study is, at first glance, reminiscent of an extensively investigated effect concerning the interactions between the dimensions of a stimulus, known as Garner interference (see Garner 1974; see also Melara and OBrien 1987; Melara and Marks 1990a, 1990b). Garner interference is usually observed during speeded discrimination tasks in which the participants have to discriminate between 2 variations along a dimension of interest (e.g., low vs. high tone) while at the same time being presented with variations in an irrelevant dimension (e.g., dim vs. bright light). When 2 dimensions interact, then Garner interference is observed, that is, the manipulation of the irrelevant dimension interferes with the discrimination of the relevant dimension as compared with the discrimination performance when the irrelevant dimension does not vary. On the other hand, when 2 dimensions are separable, the manipulation of one dimension does not result in any significant change in performance (see Melara and Marks 1990a). A number of researchers, including Melara and Marks (1990a), have interpreted Garner interference in terms of an interaction between the dimensions under investigation that hinders a participants ability to selectively attend to just the relevant dimension. Different dimensions can interact with each other to elicit Garner interference, such as unimodal integral dimensions (e.g., saturation and brightness), unimodal/crossmodal corresponding dimensions (e.g., color and tone), and spatial configural dimensions (e.g., pairs of brackets). Integral dimensions (the synesthetes experiences would be of this kind; see Melara and OBrien 1987) would appear to interact at a perceptual level of information processing, whereas corresponding dimensions are supposed to be more linguisticsemantic in nature (for a review, see Marks 2004). It is important to note, however, that even if there appear to be some superficial similarities between the present study and the results of research using the Garner interference paradigm (e.g., the variation of the irrelevant visual dimension led to a variation of the accuracy of the discrimination of the relevant odor dimension), there are also some important differences that caution against interpreting our results in terms of Garner interference. First, we did not find any interference effect on the speed of participants discrimination responses, which is what is normally observed in studies of Garner interference (at least in prior studies that have used visual and auditory stimuli; cf., Melara and OBrien 1987; Melara and Marks 1990a, 1990b). Additionally, in the present study, there were 2 irrelevant dimensions rather than just one, and this raises the question of whether having an intrinsic compatibility between the dimensions within the same visual distractor (e.g., compatible when the shape of a lemon was colored yellow and incompatible when the shape of a lemon was colored red) would have had an effect on participants discrimination performance. In order to exclude this possibility, we conducted a final analysis of our data in which only the within-dimension compatibility of the visual distractors was considered (e.g., the strawberry picture colored in red = compatible, the lemon picture colored in red = incompatible): No modulation of participants discrimination performance was observed (in terms of either the accuracy or the RT data) as a function of the compatibility of the colorpicture combination, suggesting that the intrinsic compatibility of the visual distractor was not relevant to the task at hand. Even though our results would not appear to be easily comparable to those described in the previous literature on Garner interference, an interpretation of the nature of the interaction between odors and visual features might still be possible. In particular, the inbalance between the effects exerted by colors and shapes could be understood by taking into account what Naor-Raz et al. (2003) have proposed, that is, that colors and pictures can lead to the access to an objects representation at different levels of information processing (i.e., colors at a perceptual level and pictures at a semantic level). In the present study (involving olfactory discrimination), it is possible that participants made a comparison between the just-detected odor and the olfactory representation of the target odors that she/he had stored in memory (e.g., see Larsson and Backman 1993; Herz and Engen 1996). In that case, in order to efficiently perform the task, it would be sufficient to keep a perceptual representation of the olfactory stimulus active without the need to process the olfactory information any further (e.g., without eliciting a conceptual representation of the object). Therefore, one may wonder whether in the present study, color and odor information might have interacted because the representation elicited by the olfactory and visual dimensions were at the same level of information processing (i.e., perceptual; e.g., see Blackwell 1995; Morrot et al. 2001; see also Dematte` et al. 2006), whereas the weaker interaction between olfaction and shape information could have resulted from the distance between the 2 representations: Perceptual for odors and conceptual for shapes (cf., Naor-Raz et al. 2003). In conclusion, the present study constitutes the first study to have taken account of the possibility that different types of visual information (i.e., color vs. shape) might differently interact with the processing of olfactory information. In the future, it will be interesting to investigate whether the depth of olfactory information processing modulates the nature and/or magnitude of the olfactoryvisual interactions that are observed. One could think, for example, of using odor mixtures to investigate whether colors and pictorial information can guide odor discrimination in such a context through a cognitive bias (de Araujo et al. 2005) or via perceptual enhancement of odor intensity (Zellner and Kautz 1990). University of Trento, Italy to M.L.D. Acknowledgements We would like to thank Quest International (UK) for generously providing the olfactants used in this study.


This is a preview of a remote PDF: https://chemse.oxfordjournals.org/content/34/2/103.full.pdf

M. Luisa Demattè, Daniel Sanabria, Charles Spence. Olfactory Discrimination: When Vision Matters?, Chemical Senses, 2009, 103-109, DOI: 10.1093/chemse/bjn055