A Fruity Note: Crossmodal associations between odors and musical notes

Chemical Senses, Feb 2012

Odors are notoriously difficult to describe, but they seem prone to a variety of crossmodal associations. In the present study, we generalize the previously-shown association between odors (from perfumery) and pitch (Belkin et al. 1997) to odors related to food and drink (in this case those associated with wine). We also demonstrate that, to a lesser extent (25% of the odor tested), participants preferentially match specific odors to certain types of instruments. The ratings of the odors along a number of dimensions are used in principal components analysis (PCA) to explore the psychological dimensions underlying the odor-pitch associations. The results demonstrate that both pleasantness and complexity, but not intensity, appear to play a role when choosing a pitch to match an odor. Our results suggest that these features of odor stimuli constitute psychological dimensions that can be consistently matched to auditory features.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://chemse.oxfordjournals.org/content/37/2/151.full.pdf

A Fruity Note: Crossmodal associations between odors and musical notes

Anne-Sylvie Crisinel 0 Charles Spence 0 0 Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford , UK Odors are notoriously difficult to describe, but they seem prone to a variety of crossmodal associations. In the present study, we generalize the previously-shown association between odors (from perfumery) and pitch (Belkin et al. 1997) to odors related to food and drink (in this case those associated with wine). We also demonstrate that, to a lesser extent (25% of the odor tested), participants preferentially match specific odors to certain types of instruments. The ratings of the odors along a number of dimensions are used in principal components analysis (PCA) to explore the psychological dimensions underlying the odor-pitch associations. The results demonstrate that both pleasantness and complexity, but not intensity, appear to play a role when choosing a pitch to match an odor. Our results suggest that these features of odor stimuli constitute psychological dimensions that can be consistently matched to auditory features. Introduction We constantly have to deal with multiple complex sensory inputs from our environment. However, we have only limited attentional resources with which to process them. We thus need effective strategies to deal quickly and accurately with the available information while avoiding central overload. With experience, we become very efficient at categorizing stimuli (for example, as living versus non-living objects, see Logothetis and Sheinberg 1996, for a review), and at identifying dimensions within sensory modalities (for example, the brightness of visual stimuli, or the pitch of auditory stimuli). We also learn to associate such dimensions across sensory modalities, enabling us, for example, to estimate the size of an object as a function of its impact sound, using auditory dimensions such as intensity and duration (Grassi 2005; see Spence and Zampini, 2006, for a review). However, despite many attempts, there is no generally accepted classification or set of psychological dimensions for odors. In an effort to better understand the processing of odor stimuli, several crossmodal associations have been uncovered by researchers. Such associations have been reported between odors and colors (both hue and lightness; Gilbert et al. 1996, Kemp et al. 1997, Schifferstein and Tanudjaja 2004), odors and abstract symbols (Seo et al. 2010), and odors and the pitch of sounds (Belkin et al. 1997). The level at which such crossmodal associations occur is still unclear (see Spence 2011a, for a recent review). What is clear, though, is that recent findings indicate that multisensory integration can occur at a lower level of information processing than was previously thought (see Schroeder and Foxe 2005, for a review). Indeed, the latest research has demonstrated that the olfactory tubercle of mice responds to auditory stimuli, as well as having its activation modulated crossmodally when simultaneously presented with both auditory and olfactory stimuli (Wesson and Wilson 2010). Presenting crossmodally congruent pairs of color and odor stimuli has been shown to give rise to increased activity in the orbitofrontal cortex as well as the insular cortex in humans, two areas previously identified as encoding the hedonic value of smells (O sterbauer et al. 2005). There are thus several candidates for neuronal substrates where olfactory information interacts with information from the other senses. Belkin et al. (1997) demonstrated that certain odors (in particular, those that are commonly used in perfumery) are consistently matched to the pitch of a tone. In the present study, by contrast, we investigated the nature of any crossmodal associations between the odors that are present in wine (mostly food odors) and pitch, in order to compare them with the associations demonstrated recently between tastes/flavors and musical notes (Crisinel and Spence, 2010b). We expected to find similar crossmodal associations between odors smelled orthonasally and musical notes. Moreover, as in Crisinel and Spences previous study, participants in the present study had to choose not only a pitch but also a musical instrument to match the odor stimuli. Materials and methods Participants 30 participants took part in the experiment (22 females, aged 18-55 years). The experiment was approved by the Central University Research Ethics Committee of Oxford University. Participants gave their informed consent, reported no cold or other impairment of their sense of smell, and no hearing impairment. The experiment lasted for approximately 40 minutes and the participants were compensated for their time with a 5 (UK Sterling) gift voucher. Samples from the Nez du Vin aroma kit (Brizard & Co, Dorchester, UK) were used as olfactory stimuli in this study. The kit is designed to help wine amateurs learn the odors commonly found in wine. Odors are represented either by a single typical molecule or a mix. 20 out of the 54 samples from the kit were selected (almond, apple, apricot, blackberry, caramel, cedar, dark chocolate, cut hay, green pepper, honey, lemon, liquorice, mushroom, musk, pepper, pineapple, raspberry, smoked, vanilla, and violet). The selection aimed to cover a wide range of odors. The samples were presented in small glass bottles identified by a number written on the side of the bottle. The odors were used in the concentration provided in the kit. The auditory stimuli came from an online musical instrument samples database from the University of Iowa Electronic Music Studios (http://theremin.music.uiowa.edu/ MIS.html, downloaded on 31/10/09). They consisted of notes played by 4 types of instruments (piano, strings, woodwind, and brass). The pitch of the notes ranged from C2 (64.4Hz) to C6 (1046.5Hz) in intervals of two tones. Thus, the participants had a choice of 52 different sounds (13 notes 4 instruments) to choose from when selecting a sound to match an odor. The sounds were edited to last for 1500 ms, and were presented over closed-ear headphones (Beyerdynamic DT 531) at a loudness of 70 dB ( 1 dB). The experiment was programmed in E-Prime (version 1.2). The participants were first given the number of the sample that they were to smell. After opening the glass bottle and smelling its content orthonasally, they had to choose a sound to match the smell. The sounds were presented on four scales corresponding to the four types of instruments. Pitch increased along the scales (horizontally), the direction was randomly chosen for each trial. The sounds could be heard by clicking on the scales. The participants were free to click on as many of the sounds as they wished before making their choice. After having made their response, they rated the appropriateness of a range of adjectives to describe the smell on 9-point scales. The adjectives rated included three categories: amodal descriptors (complex, familiar, intense, and pleasant), odor descriptors (acrid, earthy, floral, fruity, nutty, spicy, and woody), and taste descriptors (bitter, salty, sour, and sweet). This last category was added in order to compare the results of the present study with previously reported associations between tastes/flavors and sounds (Crisinel & Spence, 2010b). Odors are known to be commonly described by taste adjectives (e.g., Stevenson et al., 1995). All the scales were presented one at a time in a random order, and were anchored by the words not at all on the left side, and extremely so on the right side. Finally, the participants had to try and identify the sample and note down their response on a sheet listing all sample numbers. The 20 olfactory stimuli were presented once in a random order. The participants were free to smell the sample as often as they liked during a trial. Although the participants in the present study were instructed to focus on the odor of the stimuli, their color (which ranged from dark brown to transparent) might affect the chosen pitch, as lower pitched sounds tend to be associated with darker colors (Melara 1989). Thus, as a control for the influence of the color of the samples, the last 8 participants were blindfolded while smelling the samples, which were given to them by the experimenter. Data analysis Missing answers were replaced by the mean of the variable, as there was no more than one missing data point by variable, and no participant fail to respond on more than two occasions. Mixed analyses of variance (ANOVA) were used for interval data (choice of pitch and ratings), with odors as a within-subjects variable and being blindfolded as a between-subjects variable. The data were further subjected to a principal components analysis, using SPSS version 16, and following the approach suggested by Palland (2005). Chi-square tests were used for nominal data (choice of instrument). In order to evaluate the correlations between ratings and the choice of instrument, ratings were binned in 3 groups (ratings from 1 to 3, 4 to 6, and 7 to 9) and chisquare tests for independence were used. Results Ratings of odors Mixed ANOVAs (Greenhouse-Geisser corrected) were conducted to check whether the participants rated the odors differently on the descriptive scales (complex, familiar, intense, pleasant, acrid, earthy, floral, fruity, nutty, spicy, woody, bitter, salty, sour, and sweet) and if the visual appearance of the sample had an effect on the ratings. The results demonstrated a significant main effect of the odor on the ratings of all adjectives (p <.05), but not of being blindfolded. The interaction between odor and being blindfolded was not significant, except for the ratings of woodiness (F(9.13, 255.50) = 2.18, p = .02) and sweetness (F(9.79, 274.09) = 2.01, p = .03). Participants responses for identifying the odors were classified into 3 categories: exact identification, identification of the sample category (for example, naming another citrus fruit for lemon, or another berry for raspberry), and incorrect identification. Participants were able to correctly (and exactly) identify the olfactory stimulus in 17.7% of the cases. In a further 17.3% of cases, they identified the category of the stimulus correctly. These figures varied greatly depending on the odor under consideration: No participant was able to identify the blackberry or the violet (10% identified that it was a kind of berry, respectively of flower), while 53.3% identified the lemon odor correctly, with a further 30% recognizing it as a citrus fruit. One-way ANOVAs revealed that the identification of the stimulus had an influence on the ratings of familiarity (F(2, 597) = 45.04, p < .01), intensity (F(2, 597) = 7.91, p < .01), pleasantness (F(2, 597) = 28.07, p < .01), acridity (F(2, 597) = 8.29, p < .01), fruitiness (F(2, 597) = 5.98, p < .01), nuttiness (F(2, 597) = 7.58, p < .01), spiciness (F(2, 597) = 4.20, p = .02), saltiness (F(2, 597) = 8.10, p < .01), and sweetness (F(2, 597) = 6.66, p = .01). Correctly identified stimuli were rated as less acrid, nutty, and salty, and more familiar, intense, pleasant, fruity, spicy, and sweet. Types of instruments A chi-square test for independence was conducted to assess whether different types of instruments were chosen for different odors. The results indicated that the odors influenced the choice of instruments, v2(57, N = 599) = 117.82, p <.001. The strength of this effect, measured by computing Cramers V, can be classified as medium (V = .25), according to Cohens (1988) guidelines. Further chi-square tests for goodness of fit were conducted to determine which odors induced a distribution of instrument choice that was different from that expected by chance. Out of the 20 odors used, 5 gave rise to significant preferences in the choice of instrument: apricot (v2(3, N = 30) = 14.53, p = .002), blackberry (v2(3, N = 30) = 18.53, p < .001), musk (v2(3, N = 30) = 8.13, p = .04), raspberry (v2(3, N = 30) = 13.47, p =.004), and vanilla (v2(3, N = 30) = 11.07, p = .01) (see Figure 1). Chi-square tests for independence were conducted to assess whether different types of instruments were chosen for different ratings (ratings were binned into 3 groups). The choice Figure 1 Choice of instrument as a function of the odor presented. Only odors that led to significant preferences for instruments are shown. The total count per category is 30. of instrument was not independent of the ratings for complex, intense, pleasant, acrid, floral, fruity, spicy, bitter, salty, sour, and sweet (see Table 1, Figures 2, 3, and 4). Pitch A mixed ANOVA, with Greenhouse-Geisser correction, was conducted in order to assess whether the odors or visual information affected the choice of pitch. The results indicated that the odors affected the choice of pitch, F(9.94, 278.19) = 11.33, p < .001 (see Figure 5). Visual information had no main effect on the choice of pitch, F(1, 28) = .259, p = .615, and the interaction between visual information and odor was not significant, F(9.94, 278.19) = .94, p = .50. The range of chosen pitch (54.6-72.3, in MIDI note numbers) was very similar to the range found previously for tastes/ flavors (50.8-71.5, Crisinel and Spence, 2010b). Principal components analysis The pitch, as well as the 15 other ratings (complex, familiar, intense, pleasant, acrid, earthy, floral, fruity, nutty, spicy, woody, bitter, salty, sour, sweet), were subjected to principal components analysis (PCA). The suitability of this approach was assessed first. The correlation matrix revealed the presence of several coefficients above .3. The KaiserMeyer-Oklin value was .82, thus attaining the recommended value of .6 (Kaiser 1970, 1974), and the Bartletts test of sphericity (Bartlett 1954) reached statistical significance, supporting the factorability of the correlation matrix. PCA revealed the presence of four components with eigenvalues over 1, explaining 29.6%, 15.2%, 10.5%, and 6.7% of the variance, respectively. We decided to keep only three components, based on the inspection of the scree plot and results of parallel analysis (using the program Monte Carlo PCA for Parallel Analysis, developed by Marley Watkins, 2000), which demonstrated three components with eigenvalues exceeding the corresponding criterion values for Table 1 Dependence of descriptive ratings (amodal, olfactory, and gustatory descriptors) and choice of instruments assessed by chi-square tests (df = 6, N = 599) and Cramers V Familiar Intense Pleasant Acrid Floral Fruity Spicy Bitter Sour Salty Sweet Significant results (p < .05) are in bold. a randomly generated data matrix of the same size (16 variables by 600 trials). Varimax rotation was performed. The first component contributed 20.8% of the total variance explained (55.3%), while the second and third components contributed 20.3% and 14.2%, respectively (see Figure 6). The first component has strong positive loadings of familiar (.513), pleasant (.754), floral (.732), fruity (.832), and sweet (.806), while acrid (.391) and bitter (.345) have strong negative loadings on it. This suggests that the first component represents the hedonic evaluation of the olfactory stimuli. The second component has strong positive loadings of complex (.545), intense (.316), earthy (.747), nutty (.734), spicy (.619), woody (.780), and salty (.524), which suggests a connection with complexity (and maybe with the light-heavy dimension, see Zarzo and Stanton 2009). Finally, the third component has strong loadings of complex (.413), intense (.621), salty (.303), and sour (.750), which seems to link it to intensity. Pitch had a positive loading on the first component, and a negative loading on the second component. It thus seems that pleasantness and complexity are the essential factors in the choice of pitch. Discussion Our results confirm the existence of consistent crossmodal associations between odors and pitch. Moreover, they also demonstrate that some odors are preferentially matched to a specific type of musical instrument. The use of the term Figure 2 Choice of instrument as a function of the ratings of amodal descriptors (only adjectives that had a significant effect on the choice of instrument are shown): complex (A), intense (B), and pleasant (C), binned in three categories. The total count across categories is 600 (30 participants 20 stimuli). The piano was avoided for odors rated as more complex. Higher intensity ratings led to a higher proportion of participants choosing brass instruments. Brass instruments were also preferred for unpleasant stimuli, while the piano was associated to pleasant odors. note to describe components of a perfume might thus be more than merely a metaphor. Fruit odors seem to be consistently associated with highpitched notes. This result accords well with previous results demonstrating that sour and sweet tastes, two qualities present in fruits, are associated with high pitch (Crisinel and Spence 2010b). Given that taste qualities are easily associated with odors (see, for example, Stevenson et al. 1995), the extension of taste-sound associations to odors was to be expected. The similarity of the associations described in the present study with those of taste/flavors-notes associations previously reported (Crisinel and Spence 2010b) seems to suggest that smelling the odors orthonasally as compared to retronasally does not necessarily affect the associations that people make. However, it should be noted that only 3 odors were present in both studies (almond, lemon, and vanilla), and they werent represented by the same chemical. Moreover, the flavors of the previous study were presented in solutions, adding taste and somatosensory sensations to the retronasal odors, thus preventing a rigorous comparison. Figure 5 Mean pitch matched to each odor. MIDI (musical instrument digital interface) note numbers were used to code the pitch of the chosen notes. Western musical scale notation is shown on the right-hand y-axis. High-pitched notes were preferred for fruits. Unsurprisingly, participants were better able to name the odors that they rated as more familiar. Correctly identified stimuli were rated as more intense, possibly because more intense stimuli were easier to identify. The effect of identification on nuttiness is probably due to the fact that the almond odor, which was the only nut odor, was correctly (or nearly-correctly) identified in 54.4% of all cases. Correctly identified stimuli were also rated as more pleasant. This result accords well with previous reports concerning the existence of a correlation between pleasantness and familiarity ratings of odors (see, for example, Distel 1999). However, this correlation tends to vary with the odors used and might have been reversed with more unpleasant odors (see, for example, Seo et al. 2008). PCA suggests three components (see Figure 6). The first component is strongly linked to the hedonic evaluation of the olfactory stimuli. Many studies have shown that the hedonic value is a salient (or even the only, see Yeshurun and Sobel 2010) psychological dimension of odors (Berglund et al. 1973; Schiffman et al. 1977; Zarzo et al. 2008), thus confirming the validity of our approach on this data set. According to the PCA, the choice of pitch is linked to the first two components, i.e. hedonic value and complexity. Subjective intensity was not linked to the choice of pitch. However, it would probably have been matched between the olfactory and auditory stimuli if participants had been free to choose the intensity of the sound on top of the pitch and musical instrument. Given that the perceived familiarity and pleasantness of olfactory stimuli are not independent (Distel 1999), it would be interesting to repeat the present study in wine specialists, for whom the familiarity of the odors would most probably be much higher (given that all of the stimuli were taken from an educational kit designed to learn the aromas found Figure 6 Two-dimensional projections of the loadings of the pitch and the various ratings on the rotated components extracted through principal components analysis (PCA). First and second components (A), first and third components (B), and second and third components (C). The first component is linked to hedonic value, the second component to complexity, and the third component to intensity. in wine). A higher familiarity with the odors would likely affect their pleasantness ratings. Answering the question of whether these changes would affect the choice of pitch and/or instrument would help to better define the role played by familiarity and pleasantness in these associations. Another extension of the experiment reported here would involve the use of different wines as stimuli, in order to have more complex and naturalistic stimuli. The differences between the stimuli would be smaller, and might only be easily detected by wine specialists. They might thus not be large enough to induce crossmodal associations with sounds. Additionally, though, there might be an added complication in that the nose of the wine (orthonasal smell) and its palate (involving the combination of taste and retronasal smell) might lead to different results. That said, it is clear that many wine writers already suggest that certain wines match (or are in some sense similar to) certain musical notes or pieces of music (see Spence 2011b, for a review). Given that the experience of food involves complex mixtures of tastes and smells (as well as the inputs from other modalities) in a specific temporal pattern, one can only wonder whether the results reported here could be generalized. Could more complex food stimuli be matched to more complex combinations of musical notes such as chords, or even to pieces of music? Music has been shown to activate brain mechanisms related to semantic processing and musical excerpts can prime related words (Koelsch et al. 2004). Both concrete (for example, river, staircase) and abstract words (for example, illusion, devotion) could be primed by short musical excerpts. It might thus also be possible to prime taste words. Indeed, in a recent study, Mesz et al. (2011) asked a number of musicians to improvise short pieces of music in accordance to taste words (bitter, salty, sour, and sweet). The words elicited consistent and reliable musical patterns. Moreover, non-musical experts were found to be able to recognize the target word when listening to the improvisations. Now that consistent crossmodal associations between auditory and both gustatory (Crisinel and Spence 2010b) and olfactory stimuli (Belkin et al. 1997; see also the results reported here) have been demonstrated, the next step will be to investigate to what extent the perception of simultaneously-presented stimuli in two sensory modalities can be affected by the congruency of their matching. Congruent sounds (of eating potato chips or drinking coffee) have been shown to increase the pleasantness of chip and coffee odors (Seo and Hummel 2011). It would be interesting to investigate whether this effect would also be found for the musical notes used in our study, which are not in any way related to the sound of consuming food items. Both shapes and lighting conditions have been shown to affect the evaluation of taste (Gal et al. 2007; Oberfeld et al. 2009). Moreover, odors have been shown to modulate the rating of tactile stimuli (Dematte` et al. 2006), suggesting that crossmodal influences can occur even when the stimuli presented in the two modalities do not constitute features of the same object. Visual stimuli, which have been shown to dramatically modify the perception of olfactory stimuli in some contexts (see, for example, Gottfried and Dolan 2003; Morrot et al. 2001), may well constitute a somewhat different case, more similar to Seo and Hummels study. There, the crossmodal associations result from the learning of the color (or other visual features) of objects, which lead to specific expectations when smelling (or tasting) colored stimuli (see Shankar et al. 2010, for a review). As the crossmodal associations described in the present study occurred with stimuli that do not themselves produce sounds, they cannot work through the same mechanism. The independence of odor-sounds associations from learned associations of features that often co-occur in objects lends support to the existence of a weak version of synesthesia, much more common than the strong variety (Martino and Marks 2001). Whether the two share common mechanisms remains, however, a question for future research (see Spence, 2011a, for a review).


This is a preview of a remote PDF: https://chemse.oxfordjournals.org/content/37/2/151.full.pdf

Anne-Sylvie Crisinel, Charles Spence. A Fruity Note: Crossmodal associations between odors and musical notes, Chemical Senses, 2012, 151-158, DOI: 10.1093/chemse/bjr085