A Fruity Note: Crossmodal associations between odors and musical notes
Crossmodal Research Laboratory, Department of Experimental Psychology, University of Oxford
Odors are notoriously difficult to describe, but they seem prone to a variety of crossmodal associations. In the present study, we generalize the previously-shown association between odors (from perfumery) and pitch (Belkin et al. 1997) to odors related to food and drink (in this case those associated with wine). We also demonstrate that, to a lesser extent (25% of the odor tested), participants preferentially match specific odors to certain types of instruments. The ratings of the odors along a number of dimensions are used in principal components analysis (PCA) to explore the psychological dimensions underlying the odor-pitch associations. The results demonstrate that both pleasantness and complexity, but not intensity, appear to play a role when choosing a pitch to match an odor. Our results suggest that these features of odor stimuli constitute psychological dimensions that can be consistently matched to auditory features.
We constantly have to deal with multiple complex sensory
inputs from our environment. However, we have only
limited attentional resources with which to process them. We
thus need effective strategies to deal quickly and accurately
with the available information while avoiding central
overload. With experience, we become very efficient at
categorizing stimuli (for example, as living versus non-living objects,
see Logothetis and Sheinberg 1996, for a review), and at
identifying dimensions within sensory modalities (for
example, the brightness of visual stimuli, or the pitch of auditory
stimuli). We also learn to associate such dimensions across
sensory modalities, enabling us, for example, to estimate the
size of an object as a function of its impact sound, using
auditory dimensions such as intensity and duration (Grassi
2005; see Spence and Zampini, 2006, for a review).
However, despite many attempts, there is no generally
accepted classification or set of psychological dimensions for
odors. In an effort to better understand the processing of
odor stimuli, several crossmodal associations have been
uncovered by researchers. Such associations have been
reported between odors and colors (both hue and lightness;
Gilbert et al. 1996, Kemp et al. 1997, Schifferstein and
Tanudjaja 2004), odors and abstract symbols (Seo et al.
2010), and odors and the pitch of sounds (Belkin et al. 1997).
The level at which such crossmodal associations occur is
still unclear (see Spence 2011a, for a recent review). What
is clear, though, is that recent findings indicate that
multisensory integration can occur at a lower level of information
processing than was previously thought (see Schroeder and
Foxe 2005, for a review). Indeed, the latest research has
demonstrated that the olfactory tubercle of mice responds
to auditory stimuli, as well as having its activation
modulated crossmodally when simultaneously presented with
both auditory and olfactory stimuli (Wesson and Wilson
2010). Presenting crossmodally congruent pairs of color
and odor stimuli has been shown to give rise to increased
activity in the orbitofrontal cortex as well as the insular
cortex in humans, two areas previously identified as encoding
the hedonic value of smells (O sterbauer et al. 2005). There
are thus several candidates for neuronal substrates where
olfactory information interacts with information from the
Belkin et al. (1997) demonstrated that certain odors (in
particular, those that are commonly used in perfumery)
are consistently matched to the pitch of a tone. In the present
study, by contrast, we investigated the nature of any
crossmodal associations between the odors that are present in
wine (mostly food odors) and pitch, in order to compare
them with the associations demonstrated recently between
tastes/flavors and musical notes (Crisinel and Spence,
2010b). We expected to find similar crossmodal associations
between odors smelled orthonasally and musical notes.
Moreover, as in Crisinel and Spences previous study,
participants in the present study had to choose not only a pitch
but also a musical instrument to match the odor stimuli.
Materials and methods
30 participants took part in the experiment (22 females, aged
18-55 years). The experiment was approved by the Central
University Research Ethics Committee of Oxford
University. Participants gave their informed consent, reported no
cold or other impairment of their sense of smell, and no
hearing impairment. The experiment lasted for approximately 40
minutes and the participants were compensated for their
time with a 5 (UK Sterling) gift voucher.
Samples from the Nez du Vin aroma kit (Brizard & Co,
Dorchester, UK) were used as olfactory stimuli in this study.
The kit is designed to help wine amateurs learn the odors
commonly found in wine. Odors are represented either by a single
typical molecule or a mix. 20 out of the 54 samples from the kit
were selected (almond, apple, apricot, blackberry, caramel,
cedar, dark chocolate, cut hay, green pepper, honey, lemon,
liquorice, mushroom, musk, pepper, pineapple, raspberry,
smoked, vanilla, and violet). The selection aimed to cover
a wide range of odors. The samples were presented in small
glass bottles identified by a number written on the side of
the bottle. The odors were used in the concentration provided
in the kit.
The auditory stimuli came from an online musical
instrument samples database from the University of Iowa
Electronic Music Studios (http://theremin.music.uiowa.edu/
MIS.html, downloaded on 31/10/09). They consisted of notes
played by 4 types of instruments (piano, strings, woodwind,
and brass). The pitch of the notes ranged from C2 (64.4Hz)
to C6 (1046.5Hz) in intervals of two tones. Thus, the
participants had a choice of 52 different sounds (13 notes 4
instruments) to choose from when selecting a sound to match
an odor. The sounds were edited to last for 1500 ms, and
were presented over closed-ear headphones (Beyerdynamic
DT 531) at a loudness of 70 dB ( 1 dB).
The experiment was programmed in E-Prime (version 1.2).
The participants were first given the number of the sample
that they were to smell. After opening the glass bottle and
smelling its content orthonasally, they had to choose a sound
to match the smell. The sounds were presented on four scales
corresponding to the four types of instruments. Pitch
increased along the scales (horizontally), the direction was
randomly chosen for each trial. The sounds could be heard by
clicking on the scales. The participants were free to click on
as many of the sounds as they wished before making their
choice. After having made their response, they rated the
appropriateness of a range of adjectives to describe the smell
on 9-point scales. The adjectives rated included three
categories: amodal descriptors (complex, familiar, intense, and
pleasant), odor descriptors (acrid, earthy, floral, fruity,
nutty, spicy, and woody), and taste descriptors (bitter, salty,
sour, and sweet). This last category was added in order to
compare the results of the present study with previously
reported associations between tastes/flavors and sounds
(Crisinel & Spence, 2010b). Odors are known to be commonly
described by taste adjectives (e.g., Stevenson et al., 1995). All
the scales were presented one at a time in a random order,
and were anchored by the words not at all on the left side,
and extremely so on the right side. Finally, the participants
had to try and identify the sample and note down their response
on a sheet listing all sample numbers. The 20 olfactory stimuli
were presented once in a random order. The participants were
free to smell the sample as often as they liked during a trial.
Although the participants in the present study were
instructed to focus on the odor of the stimuli, their color
(which ranged from dark brown to transparent) might affect
the chosen pitch, as lower pitched sounds tend to be
associated with darker colors (Melara 1989). Thus, as a control for
the influence of the color of the samples, the last 8
participants were blindfolded while smelling the samples, which
were given to them by the experimenter.
Missing answers were replaced by the mean of the variable,
as there was no more than one missing data point by
variable, and no participant fail to respond on more than two
occasions. Mixed analyses of variance (ANOVA) were used
for interval data (choice of pitch and ratings), with odors as
a within-subjects variable and being blindfolded as a
between-subjects variable. The data were further subjected
to a principal components analysis, using SPSS version
16, and following the approach suggested by Palland (2005).
Chi-square tests were used for nominal data (choice of
instrument). In order to evaluate the correlations between
ratings and the choice of instrument, ratings were binned in
3 groups (ratings from 1 to 3, 4 to 6, and 7 to 9) and
chisquare tests for independence were used.
Ratings of odors
Mixed ANOVAs (Greenhouse-Geisser corrected) were
conducted to check whether the participants rated the odors
differently on the descriptive scales (complex, familiar,
intense, pleasant, acrid, earthy, floral, fruity, nutty, spicy,
woody, bitter, salty, sour, and sweet) and if the visual
appearance of the sample had an effect on the ratings. The
results demonstrated a significant main effect of the odor
on the ratings of all adjectives (p <.05), but not of being
blindfolded. The interaction between odor and being blindfolded
was not significant, except for the ratings of woodiness
(F(9.13, 255.50) = 2.18, p = .02) and sweetness (F(9.79,
274.09) = 2.01, p = .03).
Participants responses for identifying the odors were
classified into 3 categories: exact identification, identification of
the sample category (for example, naming another citrus
fruit for lemon, or another berry for raspberry), and
incorrect identification. Participants were able to correctly (and
exactly) identify the olfactory stimulus in 17.7% of the cases.
In a further 17.3% of cases, they identified the category of the
stimulus correctly. These figures varied greatly depending on
the odor under consideration: No participant was able to
identify the blackberry or the violet (10% identified that it
was a kind of berry, respectively of flower), while 53.3%
identified the lemon odor correctly, with a further 30%
recognizing it as a citrus fruit.
One-way ANOVAs revealed that the identification of the
stimulus had an influence on the ratings of familiarity (F(2,
597) = 45.04, p < .01), intensity (F(2, 597) = 7.91, p < .01),
pleasantness (F(2, 597) = 28.07, p < .01), acridity (F(2,
597) = 8.29, p < .01), fruitiness (F(2, 597) = 5.98, p < .01),
nuttiness (F(2, 597) = 7.58, p < .01), spiciness (F(2, 597) = 4.20,
p = .02), saltiness (F(2, 597) = 8.10, p < .01), and sweetness
(F(2, 597) = 6.66, p = .01). Correctly identified stimuli were
rated as less acrid, nutty, and salty, and more familiar, intense,
pleasant, fruity, spicy, and sweet.
Types of instruments
A chi-square test for independence was conducted to assess
whether different types of instruments were chosen for
different odors. The results indicated that the odors influenced
the choice of instruments, v2(57, N = 599) = 117.82, p <.001.
The strength of this effect, measured by computing Cramers
V, can be classified as medium (V = .25), according to
Cohens (1988) guidelines. Further chi-square tests for goodness
of fit were conducted to determine which odors induced a
distribution of instrument choice that was different from that
expected by chance. Out of the 20 odors used, 5 gave rise to
significant preferences in the choice of instrument: apricot
(v2(3, N = 30) = 14.53, p = .002), blackberry (v2(3, N = 30) =
18.53, p < .001), musk (v2(3, N = 30) = 8.13, p = .04), raspberry
(v2(3, N = 30) = 13.47, p =.004), and vanilla (v2(3, N = 30) =
11.07, p = .01) (see Figure 1).
Chi-square tests for independence were conducted to assess
whether different types of instruments were chosen for
different ratings (ratings were binned into 3 groups). The choice
Figure 1 Choice of instrument as a function of the odor presented. Only
odors that led to significant preferences for instruments are shown. The total
count per category is 30.
of instrument was not independent of the ratings for
complex, intense, pleasant, acrid, floral, fruity, spicy, bitter, salty,
sour, and sweet (see Table 1, Figures 2, 3, and 4).
A mixed ANOVA, with Greenhouse-Geisser correction, was
conducted in order to assess whether the odors or visual
information affected the choice of pitch. The results indicated
that the odors affected the choice of pitch, F(9.94, 278.19) =
11.33, p < .001 (see Figure 5). Visual information had no
main effect on the choice of pitch, F(1, 28) = .259, p =
.615, and the interaction between visual information and
odor was not significant, F(9.94, 278.19) = .94, p = .50.
The range of chosen pitch (54.6-72.3, in MIDI note numbers)
was very similar to the range found previously for tastes/
flavors (50.8-71.5, Crisinel and Spence, 2010b).
Principal components analysis
The pitch, as well as the 15 other ratings (complex, familiar,
intense, pleasant, acrid, earthy, floral, fruity, nutty, spicy,
woody, bitter, salty, sour, sweet), were subjected to principal
components analysis (PCA). The suitability of this
approach was assessed first. The correlation matrix revealed
the presence of several coefficients above .3. The
KaiserMeyer-Oklin value was .82, thus attaining the
recommended value of .6 (Kaiser 1970, 1974), and the Bartletts test
of sphericity (Bartlett 1954) reached statistical significance,
supporting the factorability of the correlation matrix. PCA
revealed the presence of four components with eigenvalues
over 1, explaining 29.6%, 15.2%, 10.5%, and 6.7% of the
variance, respectively. We decided to keep only three
components, based on the inspection of the scree plot and
results of parallel analysis (using the program Monte Carlo
PCA for Parallel Analysis, developed by Marley Watkins,
2000), which demonstrated three components with
eigenvalues exceeding the corresponding criterion values for
Table 1 Dependence of descriptive ratings (amodal, olfactory, and
gustatory descriptors) and choice of instruments assessed by chi-square
tests (df = 6, N = 599) and Cramers V
Significant results (p < .05) are in bold.
a randomly generated data matrix of the same size (16
variables by 600 trials). Varimax rotation was performed. The
first component contributed 20.8% of the total variance
explained (55.3%), while the second and third components
contributed 20.3% and 14.2%, respectively (see Figure 6).
The first component has strong positive loadings of
familiar (.513), pleasant (.754), floral (.732), fruity (.832), and
sweet (.806), while acrid (.391) and bitter (.345) have
strong negative loadings on it. This suggests that the first
component represents the hedonic evaluation of the
olfactory stimuli. The second component has strong positive
loadings of complex (.545), intense (.316), earthy (.747), nutty
(.734), spicy (.619), woody (.780), and salty (.524), which
suggests a connection with complexity (and maybe with the
light-heavy dimension, see Zarzo and Stanton 2009). Finally,
the third component has strong loadings of complex (.413),
intense (.621), salty (.303), and sour (.750), which seems to
link it to intensity. Pitch had a positive loading on the first
component, and a negative loading on the second
component. It thus seems that pleasantness and complexity are
the essential factors in the choice of pitch.
Our results confirm the existence of consistent crossmodal
associations between odors and pitch. Moreover, they also
demonstrate that some odors are preferentially matched
to a specific type of musical instrument. The use of the term
Figure 2 Choice of instrument as a function of the ratings of amodal
descriptors (only adjectives that had a significant effect on the choice of
instrument are shown): complex (A), intense (B), and pleasant (C), binned in
three categories. The total count across categories is 600 (30 participants
20 stimuli). The piano was avoided for odors rated as more complex. Higher
intensity ratings led to a higher proportion of participants choosing brass
instruments. Brass instruments were also preferred for unpleasant stimuli,
while the piano was associated to pleasant odors.
note to describe components of a perfume might thus be
more than merely a metaphor.
Fruit odors seem to be consistently associated with
highpitched notes. This result accords well with previous results
demonstrating that sour and sweet tastes, two qualities
present in fruits, are associated with high pitch (Crisinel and
Spence 2010b). Given that taste qualities are easily
associated with odors (see, for example, Stevenson et al. 1995),
the extension of taste-sound associations to odors was to
be expected. The similarity of the associations described in
the present study with those of taste/flavors-notes
associations previously reported (Crisinel and Spence 2010b) seems
to suggest that smelling the odors orthonasally as compared
to retronasally does not necessarily affect the associations
that people make. However, it should be noted that only
3 odors were present in both studies (almond, lemon, and
vanilla), and they werent represented by the same chemical.
Moreover, the flavors of the previous study were presented in
solutions, adding taste and somatosensory sensations to the
retronasal odors, thus preventing a rigorous comparison.
Figure 5 Mean pitch matched to each odor. MIDI (musical instrument
digital interface) note numbers were used to code the pitch of the chosen
notes. Western musical scale notation is shown on the right-hand y-axis.
High-pitched notes were preferred for fruits.
Unsurprisingly, participants were better able to name the
odors that they rated as more familiar. Correctly identified
stimuli were rated as more intense, possibly because more
intense stimuli were easier to identify. The effect of
identification on nuttiness is probably due to the fact that the
almond odor, which was the only nut odor, was correctly
(or nearly-correctly) identified in 54.4% of all cases.
Correctly identified stimuli were also rated as more pleasant.
This result accords well with previous reports concerning
the existence of a correlation between pleasantness and
familiarity ratings of odors (see, for example, Distel 1999).
However, this correlation tends to vary with the odors used
and might have been reversed with more unpleasant odors
(see, for example, Seo et al. 2008).
PCA suggests three components (see Figure 6). The first
component is strongly linked to the hedonic evaluation of
the olfactory stimuli. Many studies have shown that the
hedonic value is a salient (or even the only, see Yeshurun and
Sobel 2010) psychological dimension of odors (Berglund
et al. 1973; Schiffman et al. 1977; Zarzo et al. 2008), thus
confirming the validity of our approach on this data set.
According to the PCA, the choice of pitch is linked to the first
two components, i.e. hedonic value and complexity.
Subjective intensity was not linked to the choice of pitch. However,
it would probably have been matched between the olfactory
and auditory stimuli if participants had been free to choose
the intensity of the sound on top of the pitch and musical
Given that the perceived familiarity and pleasantness of
olfactory stimuli are not independent (Distel 1999), it would
be interesting to repeat the present study in wine specialists,
for whom the familiarity of the odors would most probably
be much higher (given that all of the stimuli were taken
from an educational kit designed to learn the aromas found
Figure 6 Two-dimensional projections of the loadings of the pitch and the
various ratings on the rotated components extracted through principal
components analysis (PCA). First and second components (A), first and third
components (B), and second and third components (C). The first component
is linked to hedonic value, the second component to complexity, and the
third component to intensity.
in wine). A higher familiarity with the odors would likely
affect their pleasantness ratings. Answering the question
of whether these changes would affect the choice of pitch
and/or instrument would help to better define the role
played by familiarity and pleasantness in these associations.
Another extension of the experiment reported here would
involve the use of different wines as stimuli, in order to have
more complex and naturalistic stimuli. The differences
between the stimuli would be smaller, and might only be easily
detected by wine specialists. They might thus not be large
enough to induce crossmodal associations with sounds.
Additionally, though, there might be an added complication in
that the nose of the wine (orthonasal smell) and its palate
(involving the combination of taste and retronasal smell)
might lead to different results. That said, it is clear that many
wine writers already suggest that certain wines match (or are
in some sense similar to) certain musical notes or pieces of
music (see Spence 2011b, for a review).
Given that the experience of food involves complex
mixtures of tastes and smells (as well as the inputs from other
modalities) in a specific temporal pattern, one can only
wonder whether the results reported here could be generalized.
Could more complex food stimuli be matched to more
complex combinations of musical notes such as chords, or even
to pieces of music? Music has been shown to activate brain
mechanisms related to semantic processing and musical
excerpts can prime related words (Koelsch et al. 2004). Both
concrete (for example, river, staircase) and abstract words
(for example, illusion, devotion) could be primed by short
musical excerpts. It might thus also be possible to prime taste
words. Indeed, in a recent study, Mesz et al. (2011) asked
a number of musicians to improvise short pieces of music
in accordance to taste words (bitter, salty, sour, and sweet).
The words elicited consistent and reliable musical patterns.
Moreover, non-musical experts were found to be able to
recognize the target word when listening to the improvisations.
Now that consistent crossmodal associations between
auditory and both gustatory (Crisinel and Spence 2010b) and
olfactory stimuli (Belkin et al. 1997; see also the results
reported here) have been demonstrated, the next step will
be to investigate to what extent the perception of
simultaneously-presented stimuli in two sensory modalities can be
affected by the congruency of their matching. Congruent
sounds (of eating potato chips or drinking coffee) have been
shown to increase the pleasantness of chip and coffee odors
(Seo and Hummel 2011). It would be interesting to
investigate whether this effect would also be found for the musical
notes used in our study, which are not in any way related to
the sound of consuming food items. Both shapes and lighting
conditions have been shown to affect the evaluation of taste
(Gal et al. 2007; Oberfeld et al. 2009). Moreover, odors have
been shown to modulate the rating of tactile stimuli
(Dematte` et al. 2006), suggesting that crossmodal influences
can occur even when the stimuli presented in the two
modalities do not constitute features of the same object. Visual
stimuli, which have been shown to dramatically modify the
perception of olfactory stimuli in some contexts (see, for
example, Gottfried and Dolan 2003; Morrot et al. 2001), may
well constitute a somewhat different case, more similar to
Seo and Hummels study. There, the crossmodal associations
result from the learning of the color (or other visual
features) of objects, which lead to specific expectations when
smelling (or tasting) colored stimuli (see Shankar et al.
2010, for a review). As the crossmodal associations
described in the present study occurred with stimuli that do
not themselves produce sounds, they cannot work through
the same mechanism. The independence of odor-sounds
associations from learned associations of features that often
co-occur in objects lends support to the existence of a weak
version of synesthesia, much more common than the
strong variety (Martino and Marks 2001). Whether the
two share common mechanisms remains, however, a
question for future research (see Spence, 2011a, for a review).